Feed aggregator

Stanford Web Services Blog: Module of the Day: JS Injector

Planet Drupal - Mon, 12/05/2014 - 18:09

Today's "Module of the Day" is a handy little module called JS Injector that allows you to insert snippets of Javascript into your site.

In this post, I'll walk through how to enable and configure the module, use it to shout, "Hello, World!", and then go into some more advanced configuration.

Categories: Elsewhere

Russell Coker: BTRFS vs LVM

Planet Debian - Mon, 12/05/2014 - 09:37

For some years LVM (the Linux Logical Volume Manager) has been used in most Linux systems. LVM allows one or more storage devices (either disks, partitions, or RAID sets) to be assigned to a Volume Group (VG) some of which can then allocated to a Logical Volume (LVs) which are equivalent to any other block device, a VG can have many LVs.

One of the significant features of LVM is that you can create snapshots of a LV. One common use is to have multiple snapshots of a LV for online backups and another is to make a snapshot of a filesystem before making a backup to external storage, the snapshot is unchanging so there’s no problem of inconsistencies due to backing up a changing data set. When you create a snapshot it will have the same filesystem label and UUID so you should always mount a LVM device by it’s name (which will be /dev/$VGNAME/$LVNAME).

One of the problems with the ReiserFS filesystem was that there was no way to know whether a block of storage was a data block, a metadata block, or unused. A reiserfsck --rebuild-tree would find any blocks that appeared to be metadata and treat them as such, deleted files would reappear and file contents which matched metadata (such as a file containing an image of a ReiserFS filesystem) would be treated as metadata. One of the impacts of this was that a hostile user could create a file which would create a SUID root program if the sysadmin ran a --rebuild-tree operation.

BTRFS solves the problem of filesystem images by using a filesystem specific UUID in every metadata block. One impact of this is that if you want to duplicate a BTRFS filesystem image and use both copies on the same system you need to regenerate all the checksums of metadata blocks with the new UUID. The way BTRFS works is that filesystems are identified by UUID so having multiple block devices with the same UUID causes the kernel to get confused. Making an LVM snapshot really isn’t a good idea in this situation. It’s possible to change BTRFS kernel code to avoid some of the problems of duplicate block devices and it’s most likely that something will be done about it in future. But it still seems like a bad idea to use LVM with BTRFS.

The most common use of LVM is to divide the storage of a single disk or RAID array for the use of multiple filesystems. Each filesystem can be enlarged (through extending the LV and making the filesystem use the space) and snapshots can be taken. With BTRFS you can use subvolumes for the snapshots and the best use of BTRFS (IMHO) is to give it all the storage that’s available so there is no need to enlarge a filesystem in typical use. BTRFS supports quotas on subvolumes which aren’t really usable yet but in the future will remove the need to create multiple filesystems to control disk space use. An important but less common use of LVM is to migrate a live filesystem to a new disk or RAID array, but this can be done by BTRFS too by adding a new partition or disk to a filesystem and then removing the old one.

It doesn’t seem that LVM offers any benefits when you use BTRFS. When I first experimented with BTRFS I used LVM but I didn’t find any benefit in using LVM and it was only a matter of luck that I didn’t use a snapshot and break things.

Snapshots of BTRFS Filesystems

One reason for creating a snapshot of a filesystem (as opposed to a snapshot of a subvolume) is for making backups of virtual machines without support from inside the virtual machine (EG running an old RHEL5 virtual machine that doesn’t have the BTRFS utilities). Another is for running training on virtual servers where you want to create one copy of the filesystem for each student. To solve both these problems I am currently using files in a BTRFS subvolume. The BTRFS kernel code won’t touch those files unless I create a loop device so I can only create a loop device for one file at a time.

One tip for doing this, don’t use names such as /xenstore/vm1 for the files containing filesystem images, use names such as /xenstore/vm1-root. If you try to create a virtual machine named “vm1″ then Xen will look for a file named “vm1″ in the current directory before looking in /etc/xen and tries to use a filesystem image as a Xen configuration file. It would be nice if there was a path for Xen configuration files that either didn’t include the current directory or included it at the end of the list. Including the current directory in the path is a DOS mistake that should have gone away a long time ago.

Psychology and Block Devices

ZFS has a similar design to BTRFS in many ways and has some similar issues. But one benefit for ZFS is that it manages block devices in a “zpool”, first you create a zpool with the block devices and after that you can create ZFS filesystems or “ZVOL” block devices. I think that most sysadmins would regard a zpool as something similar to LVM (which may or may not be correct depending on how you look at it) and immediately rule out the possibility of running a zpool on LVM.

BTRFS looks like a regular Unix filesystem in many ways, you can have a single block device that you mount with the usual mount command. The fact that BTRFS can support multiple block devices in a RAID configuration isn’t so obvious and the fact that it implements equivalents to most LVM functionality probably isn’t known to most people when they start using it. The most obvious way to start using BTRFS is to use it just like an Ext3/4 filesystem on an LV, and to use LVM snapshots to backup data, this is made even more likely by the fact that there is a program to convert a ext2/3/4 filesystem to BTRFS. This seems likely to cause data loss.

Related posts:

  1. Starting with BTRFS Based on my investigation of RAID reliability [1] I have...
  2. BTRFS Status March 2014 I’m currently using BTRFS on most systems that I can...
  3. BTRFS and ZFS as Layering Violations LWN has an interesting article comparing recent developments in the...
Categories: Elsewhere

Web Omelette: Sending HTML Emails with Webform in Drupal 7

Planet Drupal - Mon, 12/05/2014 - 09:12

Have you ever wondered how you can include HTML markup in the emails you send with Webform? Out of the box, you cannot. But I am going to show you a simple way to achieve this using the Mime Mail module and some simple theming. Additionally, I will show you how to control which webforms should send HTML emails and which should not.

First though, make sure you install and enable the Mime Mail and Mail System modules (the latter is a dependency of the former). With Drush, all you have to do is use this command:

drush en mimemail -y

It will take care of all you need to do. If you commit the module to your repo, don't forget that the Mail System module also gets downloaded, so make sure you include it as well.

Next, edit your theme's template.php file and paste this block of code (explained after):

function your_theme_webform_mail_headers($variables) { $headers = array(); $headers = array( 'Content-Type' => 'text/html; charset=UTF-8; format=flowed; delsp=yes', 'X-Mailer' => 'Drupal Webform (PHP/'. phpversion() .')', ); return $headers; }

Make sure you change your_theme with the name of your theme. So what happens here? We override the theme_webform_mail_headers() declared by the Webform module. We do this in order to add a content type to the mail headers, and set it to HTML. And that's pretty much it.

If you now clear your caches and test a webform, you'll see that you can add anchor tags and other basic HTML tags.

One problem you might run into though is that all your webforms are now sending emails in HTML format - a result only partially desired. You'll notice that the default email that you send no longer provides any spacing and all the text gets put on one line - as HTML in fact.

So what you can do is make a selection of webforms for which you'll want HTML emails. A handy way of doing this is by adding a field to your webform content type that will be used to swith HTML emails on/off for a given node. So to illustrate this, let's say we added a new field to the relevant content type called HTML Emails (with the machine name: field_html_email). This field is a boolean type (a single checkbox basically) with the values of 1 for on and 0 for off.

It follows to adapt the theme override above and replace it with something like this:

function your_theme_webform_mail_headers($variables) { $headers = array( 'X-Mailer' => 'Drupal Webform (PHP/' . phpversion() . ')', ); // Get the HTML Email field $html_email_field = field_get_items('node', $variables['node'], 'field_html_email'); // Check if this webform node needs to send HTML emails if (!empty($html_email_field)) { $html = $html_email_field[0]['value'] == 1 ? TRUE : FALSE; } if ($html === TRUE) { $headers['Content-Type'] = 'text/html; charset=UTF-8; format=flowed; delsp=yes'; } return $headers; }

If you consult the documentation for this theme function, you'll know that the $variables parameter contains also the node object which uses Webform to send the email. So we basically check for the value of our field and if it is 1, we add the HTML information to the mail headers. Otherwise, we return the $headers array containing the value it does by default (essentially making no changes).

You can now clear the caches and test it out. Edit a node of the respective content type and check the box. You'll see that it now sends HTML emails. However, if you uncheck the box, it will fallback to the default format that comes with the Webform module.

Hope this helps.

In Theming | Drupal var switchTo5x = true;stLight.options({"publisher":"dr-8de6c3c4-3462-9715-caaf-ce2c161a50c"});
Categories: Elsewhere

Ryan Szrama: Beyond Wombats

Planet Drupal - Mon, 12/05/2014 - 04:35

I accidentally started publishing open source software in 2006, the first integration of the QuickBooks Web Connector with anything. This was pre-Ubercart when I was just cutting my teeth on PHP / MySQL development at Prima Supply, and I thought it would be fun to claim the code was written by wombats while I just published it online. I decided to own the silliness and start blogging on bywombats.com using Drupal 4.7 at the time - and immediately picked up a freelance contract doing QuickBooks integration work.

Topics: DrupalWork
Categories: Elsewhere

Andrew Pollock: [debian] Day 103: Today's Debian efforts

Planet Debian - Mon, 12/05/2014 - 04:15

I had a really productive day today actually working on Debian, as planned for my Mondays.

I'm still working through my list of packages, trying to get them all vaguely up to date for jessie. It's mostly just addressing Lintian issues that mostly revolve around old standards versions, with the occasional new upstream release. I've also been doing bug triage where the bugs aren't overwhelming.

Today I made uploads for pssh (a new upstream release), pymetrics (mostly just a rebuild), rcs-blame (mostly just a rebuild) and simpleproxy (mostly just a rebuild).

I need to revisit simpleproxy, because I'm having problems convincing the resulting binary to be linked correctly for relro. It's weird, because I'm doing everything I'm supposed to be doing and I can see all the other hardening flags being passed in except for this one.

I really like simplifying down debian/rules using dh, that really makes things readable. You can see the useful stuff without losing it in all the boilerplate. For some reason I was never a fan of CDBS, but I'm quite liking dh. I think it's because it's less opaque than CDBS.

Categories: Elsewhere

Benjamin Mako Hill: Google Has Most of My Email Because It Has All of Yours

Planet Debian - Mon, 12/05/2014 - 04:11

For almost 15 years, I have run my own email server which I use for all of my non-work correspondence. I do so to keep autonomy, control, and privacy over my email and so that no big company has copies of all of my personal email.

A few years ago, I was surprised to find out that my friend Peter Eckersley — a very privacy conscious friend who is Technology Projects Director at the EFF — used GMail. I asked him why he would willingly give Google copies of all his email. Peter pointed out that if all of your friends use GMail, Google has your email anyway. Any time I email somebody who uses GMail — and anytime they email me — Google has that email.

Since our conversation, I have often wondered just how much of my email Google really has. This weekend, I wrote a small program to go through all the email I have kept in my personal inbox since April 2004 (when GMail was started) to find out.

One challenge with answering the question is that many people, like Peter, use GMail to read, compose, and send email but they configure GMail to send email from a non-gmail.com “From” address. To catch these, my program looks through each message’s headers that record which computers handled the message on its way to my server and to pick out messages that have traveled through google.com, gmail.com, or googlemail.com. Although I usually filter them, my personal mailbox contains emails sent through a number of mailing lists. Since these mailing lists often “hide” the true provenance of a message, I exclude all messages that are marked as coming from lists using the (usually invisible) “Precedence” header.

The following graph shows the numbers of emails in my personal inbox each week in red and the subset from Google in blue. Because the number of emails I receive week-to-week tends to vary quite a bit, I’ve included a LOESS “smoother” which shows a moving average over several weeks.

From eyeballing the graph, the answer to seems to be that, although it varies, about a third of the email in my inbox comes from Google!

Keep in mind that this is all of my personal email and includes automatic and computer generated mail from banks and retailers, etc. Although it is true that Google doesn’t have these messages, it suggests that the proportion of my truly “personal” email that comes via Google is probably much higher.

I would also like to know how much of the email I send goes to Google. I can do this by looking at emails in my inbox that I have replied to. This works if I am willing to assume that if I reply to an email sent from Google, it ends up back at Google. In some ways, doing this addresses the problem with the emails from retailers and banks since I am very unlikely to reply to those emails. In this sense, it also reflects a measure of more truly personal email.

I’ve broken down the proportions of emails I received that come from Google in the graph below for all email (top) and for emails I have replied to (bottom). In the graphs, the size of the dots represents the total number of emails counted to make that proportion. Once again, I’ve included the LOESS moving average.

The answer is surprisingly large. Despite the fact that I spend hundreds of dollars a year and hours of work to host my own email server, Google has about half of my personal email! Last year, Google delivered 57% of the emails in my inbox that I replied to. They have delivered more than a third of all the email I’ve replied to ever year since 2006 and more than half since 2010. On the upside, there is some indication that the proportion is going down. So far this year, only 51% of the emails I’ve replied to arrived from Google.

The numbers are higher than I imagined and reflect somewhat depressing news. They show how it’s complicated to think about privacy and autonomy for communication between parties. I’m not sure what to do except encourage others to consider, in the wake of the Snowden revelations and everything else, whether you really want Google to have all your email. And half of mine.

If you want to run the analysis on your own, you’re welcome to the Python and R code I used to produce the numbers and graphs.

Categories: Elsewhere

Matthew Garrett: Oracle continue to circumvent EXPORT_SYMBOL_GPL()

Planet Debian - Sun, 11/05/2014 - 17:14
Oracle won their appeal regarding whether APIs are copyrightable. There'll be ongoing argument about whether Google's use of those APIs is fair use or not, and perhaps an appeal to the Supreme Court, but that's the new status quo. This will doubtless result in arguments over whether Oracle's implementation of Linux APIs in Solaris 10 was a violation of copyright or not (and presumably Google are currently checking whether they own any code that Oracle reimplemented), but that's not what I'm going to talk about today.

Oracle own some code called DTrace (Wikipedia has a good overview here - what it actually does isn't especially relevant) that was originally written as part of Solaris. When Solaris was released under the CDDL, so was DTrace. The CDDL is a file-level copyleft license with some restrictions not present in the GPL - as a result, combining GPLed code with CDDLed code will (in the absence of additional permission grants) result in a work that is under an inconsistent license and cannot legally be distributed.

Oracle wanted to make DTrace available for Linux as part of their Unbreakable Linux product. Integrating it directly into the kernel would obviously cause legal issues, so instead they implemented it as a kernel module. The copyright status of kernel modules is somewhat unclear. The GPL covers derivative works, but the definition of derivative works is a function of copyright law and judges. Making use of explicitly exported API may not be sufficient to constitute a derivative work - on the other hand, it might. This is largely untested in court. Oracle appear to believe that they're legitimate, and so have added just enough in-kernel code (and GPLed) to support DTrace, while keeping the CDDLed core of DTrace separate.

The kernel actually has two levels of exposed (non-userspace) API - those exported via EXPORT_SYMBOL() and those exported via EXPORT_SYMBOL_GPL(). Symbols exported via EXPORT_SYMBOL_GPL() may only be used by modules that claim to be GPLed, with the kernel refusing to load them otherwise. There is no technical limitation on the use of symbols exported via EXPORT_SYMBOL().

(Aside: this should not be interpreted as meaning that modules that only use symbols exported via EXPORT_SYMBOL() will not be considered derivative works. Anything exported via EXPORT_SYMBOL_GPL() is considered by the author to be so fundamental to the kernel that using it would be impossible without creating a derivative work. Using something exported via EXPORT_SYMBOL() may result in the creation of a derivative work. Consult lawyers before attempting to release a non-GPLed Linux kernel module)

DTrace integrates very tightly with the host kernel, and one of the things it needs access to is a high-resolution timer that is guaranteed to monotonically increase. Linux provides one in the form of ktime_get(). Unfortunately for Oracle, ktime_get() is only exported via EXPORT_SYMBOL_GPL(). Attempting to call it directly from the DTrace module would fail.

Oracle work around this in their (GPLed) kernel abstraction code. A function called dtrace_gethrtimer() simply returns the value of ktime_get(). dtrace_gethrtimer() is exported via EXPORT_SYMBOL() and therefore can be called from the DTrace module.

So, in the face of a technical mechanism designed to enforce the author's beliefs about the copyright status of callers of this function, Oracle deliberately circumvent that technical mechanism by simply re-exporting the same function under a new name. It should be emphasised that calling an EXPORT_SYMBOL_GPL() function does not inherently cause the caller to become a derivative work of the kernel - it only represents the original author's opinion of whether it would. You'd still need a court case to find out for sure. But if it turns out that the use of ktime_get() does cause a work to become derivative, Oracle would find it fairly difficult to argue that their infringement was accidental.

Of course, as copyright holders of DTrace, Oracle could solve the problem by dual-licensing DTrace under the GPL as well as the CDDL. The fact that they haven't implies that they think there's enough value in keeping it under an incompatible license to risk losing a copyright infringement suit. This might be just the kind of recklessness that Oracle accused Google of back in their last case.

comments
Categories: Elsewhere

Daniel Pocock: Is Uber on your side?

Planet Debian - Sun, 11/05/2014 - 09:40

Crowdsourcing ventures with disruptive business models are a regular point of contention these days.

In London, taxi drivers are threatening to create gridlock as part of an anti-Uber protest. In Melbourne, Uber drivers have been issued with $1,700 fines for operating without a taxi license. San Francisco city officials, despite being the birthplace of many of these ventures, are debating whether AirBNB should be regulated.

An orderly society or an old-school protection racket?

Just what exactly is it that established players in these industries are trying to achieve through their protests and lobbying efforts?

In the case of apartment rentals, many people have sympathy for respecting the wishes of neighbourhoods over those of individual landlords. In the case of car pooling schemes, the arguments tend to come not from motorists at large but from those who are afraid of competition.

Without competition, could things be any worse?

Melbourne actually provides the perfect backdrop for this debate. Only a couple of years before Uber came on the scene, the government had made a detailed study into the taxi industry. One of Australia's most prominent economic experts, a former chairman of the Australian Competition and Consumer Commission spent 18 months studying the industry.

One of the highlights of the incumbent system (and the reason I suggest Melbourne is the perfect backdrop for this debate) is the way licenses are issued to taxi drivers. There are a fixed number of licenses issued by the government. The licenses are traded on the open market so prices can go up and down just like real-estate. Under the rules of Australia's pension scheme, people have even been able to use money from their pension fund to purchase a taxi license as an investment. It goes without saying that this has helped rampant speculation and the price of a license is now comparable to the price of a house.

The end result is that no real taxi driver can afford a license: most of them have to rent their license from one of the speculators who bought the license. These fixed rental fees have to be paid every month whether the driver uses their car or not. Consequently, taxi drivers have cut back on other expenses, they are often criticised for failing to keep their cars clean and the industry as a whole is criticised due to the poor quality of drivers who don't even know their way around the city. The reason, of course, is simple: by the time some newly arrived immigrant has learnt his way around Melbourne he has also figured out that the economics of driving a taxi are not in his favor. Realizing there is no way to break even, they take other jobs instead.

It was originally speculated that the government review would dramatically reduce or abolish these speculative practices but ultimately lower license charges have only been used for the issue of 60 new licenses, barely 1% of the taxi fleet in the city today. Furthermore, the new licenses were only available to existing players in the industry.

Uber to the rescue?

Uber drove into the perfect storm as they launched their service in Melbourne in 2013.

Uber drivers get a significant benefit over their competitors in traditional taxis. In particular, as they don't have the fixed monthly payment to rent a taxi license, they don't have to work every day and can even take holidays or take time to clean the cars. These things may simultaneously benefit road safety and passenger comfort.

Meanwhile, those people who speculated on the old taxi licenses have tried hunger strikes and all kinds of other desperate tactics to defer the inevitable loss of their "investment".

The reality is that crowdsourcing is here to stay. Even if Uber is stopped by bullying and intimidation, the inefficiency of Melbourne's taxi system is plain for all to see and both customers and drivers will continue looking for alternatives. Other car-pooling apps based on barter or cost sharing will continue to find ways to operate even if the Uber model is prohibited.

It is interesting to note that the last great reform of Melbourne taxis, under Premier Jeff Kennett in the 1990s, simply resulted in a change of paint with the aim of making them look like those in New York City. Disruptive services like Uber (with their numerous technology-powered innovations to save time and money) appear to be doing far more to improve the lives of passengers and drivers.

The hidden cost

That said, large scale schemes like Uber do also have a down side for customer privacy. Hailing cabs in the street leaves no records of your movements. This new model, however, is leaving a very detailed trail of breadcrumbs that can be used for both marketing purposes or extracted (lawfully or otherwise) by some third party who wishes to monitor a particular customer's past or future movements. This is the trade-off that arises when we benefit from the efficiencies of any cloud-based service.

Categories: Elsewhere

Pages

Subscribe to jfhovinne aggregator