Elsewhere

I wrote that I would double-check how secure the module selection and downloading is in Puppet.

Well, puppet module resolves, fetches and downloads unsigned tarballs from a HTTP source and unpacks them without any verification whatsoever.

Related: I've been looking at rpm/yum GPG behaviour. rpm supports checking the signature of RPMs as a separate operation from installing them. You can't ask it to not install a package if the signature is absent or not correct.

yum is better when dealing with repositories. It can be told to check the GPG signature on all RPMs both globally (the [main] section of yum.conf) and on a per-repository basis. GPG signature checking can be disabled on the command line with --nogpgcheck. It cannot be selectively enabled on the command line.

However, yum install can install local RPMs and RPMs on web servers as well as from repositories. In both of these cases, it will not check the GPG signature at all, no matter what you've put in your yum.conf.

Finally, even if all the above worked properly, the GPG keys published by Fedora have almost no public signatures (none at all for EPEL), so neither you nor I could establish a trust path to them. Luckily I can establish a trust path to the RH security key.

At Four Kitchens, we have done quite a bit of work within the education industry. As we began looking into expanding our footprint within the education web technology space, we discovered that there was a corresponding need in the marketplace waiting to be filled; especially within higher ed. Universities and higher ed institutions continue to look for ways to cut costs, deliver content more effectively and easily, ease administration, and facilitate online learning/training. While several (open source and proprietary) solutions exist, there seems to be little clarity into what the options are, and perhaps more importantly, what the possibilities may be.

So we set out on a quest to understand the “lay of the land” a little better. Broadly speaking, there are three categories of solutions:

Learning Management Systems (LMS)

Succinctly, an LMS is a software application for the administration, documentation, tracking, reporting, and delivery of education courses or training programs. There are an incredible number of LMSs available to choose from; some proprietary, some open source, some completely custom built, some entirely off the shelf. Traditionally, LMSs have focused more on content delivery than content management, which is part of the reason we found that almost no academic institution is entirely happy with the LMS they chose to implement. Said another way, a complete solution for an academic institution involves the best aspects of CMS and LMS, and traditional LMSs have been very poor at CMS functionality.

Part of the problem, I venture to guess, is that there is surprisingly little available documentation around what the business and functional requirements of a good LMS are! The argument could be made that every academic institution has different needs and it is thus difficult to compile a unified set of requirements. But despite those differences, is there core functionality that we can distill as universally applicable? It would seem that at a minimum, the requirements of an LMS are:

1. Classroom management (attendance, etc)
2. Delivering content to students
3. Collecting completed assignments
4. Reporting student performance
5. Streamlining administration
6. Improve customer service
7. Developing and queueing standard content, and providing opportunities to tailor content
8. Provide self-service learning for students and employees
9. Deploying learning resources and programs quickly
10. Extending, maintaining, and enhancing communities

Some of the LMSs that we feel best meet these objectives are:

Canvas: Arguably the most robust and full featured LMS offering currently available, Canvas is on a meteoric rise and has been gaining use in both the higher ed and K-12 segments. It is an open source and open API product, and is one of the few (only?) LMSs that has a free mobile app to go along with it.

Sakai: Sakai is a full featured LMS built by Stanford, which has been adopted by MIT, Berkeley, and others. It is not open sourced, but operated on a community backed license.

Moodle: Moodle is a fully featured and powerful LMS, but the once dynamic open source community behind it has seen some attrition. The LMS isn’t on the leading edge anymore, but with continued support from developers, it continues to remain very relevant.

ELMS: Headed by Bryan Ollendyke of Penn State University, ELMS is the most prominent Drupal based LMS. A lot of progress has been made, but there is still some work to be done, and Bryan is a well of knowledge on the subject and is happy to talk to anyone that wants to help further its development.

Blackboard: The original dominant LMS, Blackboard has since fallen out of favor with the emergence of the other more modern, elegant, and open source solutions. It is regarded as a very rigid LMS, and awful as a CMS. Blackboard has attempted to stay relevant by acquiring Moodle and Angel.

While perhaps biased, Instructure (the makers of Canvas) have a feature comparison chart on their website.

Site building solutions and tools

Site building tools are especially important to higher ed institutions, as they frequently need to provide academic units and departments an easy way to brand themselves under the institution’s overall brand. ImageX Media, Chapter 3, Funny Monkey, and several others have done excellent work to provide open source solutions that provide the ability to manage content, rapidly deploy sites, control branding, etc. Our four favorite Drupal based solutions are:

• OpenEDU, ImageX Media
• Open Academy, Chapter Three
• Julio, Funny Monkey
• Open Scholar, Harvard

Social based “LMS” platforms

This class of solutions is centered around giving teachers and students a way to interact easily online. It shifts away from the “heft” of traditional LMSs to try to facilitates easy communication, discussions, sophisticated learning opportunities (traditional and e-learning), content distribution/sharing, etc. Taking the view that allowing educators and students to connect more easily provides the greatest value, all the other functionality and bells and whistles are built around that core premise. While none of these services use Drupal, the big players are:

Edmodo: With 7.4 million users at over 80,000 schools, Edmodo is considered one of the best solutions for Elementary and Middle Schools. It boasts very good security safeguards and granular permissions controls, a high degree of personalization, and customers speak highly of the responsive support team. Edmodo was the first of its kind, but is now seeing increased competition from the next two in this list.

Lore (Coursekit): Lore is at the newcomer to this arena, but is moving to the forefront rather quickly thanks to strong financial backers. It is “Course-centric”, in that each course has it’s own private “social network”. Users love its clean and elegant UI.

Schoology: Schoology combines some of the best elements of Lore and Edmodo. One of the big selling points is that unlike Edmodo’s “wall” which can get cluttered. Schoology has threaded discussion boards to make content easier to find. It also has better security and permissions granularity controls than Lore. Despite the lite/social nature of the service, Schoology bills itself as being able to perform full-blown LMS functions.

What to choose??

With all these options out there (and these are truly just a sliver of the pantheon of offerings), it is easy for education institutions to feel completely overwhelmed by the choices available. Perhaps this is natural given our open source leanings at Four Kitchens, but we believe that the best option for education institutions is the one that makes data exchange, interoperability, and connected systems most easy to achieve. The process efficiencies, cost savings, ease of administration, reporting that higher ed and K-12 institutions crave can only be realized when the systems work in concert. A comment made by one educator that I talked to should be of particular note to technical service providers; he said, “All these educational services and websites try to be software and lock you in. What I would REALLY like is for a way to make all these systems talk to each other. If you can do that, you’ve got a real winner!”

Date: Wednesday, February 27, 2013 - 17:00 to 18:00

Several organizers of the Downtown Los Angeles Drupal group are meeting for the year's first Downtown Drupal Governance Meetup on Wednesday, February 27, 2013 between 5-6pm.

In the spirit and tradition of our user group and events, this meetup is open to all Downtown Drupal members and is an excellent opportunity for anyone who's interested in getting involved or is just interested in observing our open governance model.

Join us in person at Droplabs or remotely via Google+ Hangout!

Agenda

Our agenda for this meeting is to review (and possibly publish) the proposed Downtown LA Drupal Governance policy:

   http://groups.drupal.org/downtown-los-angeles/governance  

If you plan to attend, please be prepared and familiarize yourself with both our group's mission at http://groups.drupal.org/downtown-los-angeles and our current governance proposal at http://groups.drupal.org/downtown-los-angeles/governance prior to this meetup.

Join Us on Google+ and Twitter!

To join the meeting, use the following URL:

   Google+ hangout: https://plus.google.com/hangouts/_/7378d9b61d65107cb08a67d67ef7e75fd8a2f67d
   Short URL: http://ex.tl/ZMW  

During the meetup, the event organizers will be monitoring Twitter for feedback and questions that mention @DowntownDrupal.

Location and Directions

This event is hosted by Droplabs (@Droplabs), a collaborative coworking space, classroom and hackerspace in Downtown Los Angeles.

   Droplabs
   651 Clover St.
   Los Angeles, CA 90031

We're quickly gaining a reputation for being the only coworking space around with a with a low-cost business model: come work here and use our tables, chairs, WiFi and conference room with a low, daily or monthly membership fee and only pay more for the extras, such as 24/7 access, equipment rental and a locker for your belongings.

Droplabs is in the Mission Junction neighborhood of Los Angeles at Big Art Labs, just 1 mile down Main St. from Philippes (the first-ever venue for Drupal meetups in Southern California!) and Union Station. We're one block west of The Brewery, the largest live-and-work artists' colony in the world.

Free parking in our large parking lot is first-come, first-served. After parking in the lot, follow the yellow signs that point to Droplabs. (If our lot is full, you can park for free on Clover St.)

Droplabs is a brief walk from the Main St. / Lamar St. stop on the the Metro Local 76 bus line. This is also the Lincoln Heights / Chinatown DASH stop.

To carpool or catch the Droplabs shuttle from Union Station, post below in the comments.

What to Bring

Just bring your laptop, your business cards or whatever else you need. You're also welcome to bring some light food, sodas or beers to share with others at the meetup.

Please note that our guest wireless network is limited to 1Mb per client, so bring your MiFi router or a phone you can tether with if for some reason you need a lot of bandwidth. Access to our high-speed network is included with a Droplabs membership.

About Downtown Los Angeles Drupal

The Downtown Los Angeles Drupal group regularly meets in and around Downtown Los Angeles, California. We organize a large number of weekly and monthly events on various Drupal topics, including the general Downtown Los Angeles Drupal meetup that has been meeting regularly each month since early 2010.

The Downtown area is the most active area for Drupal in the Greater Los Angeles Area and has seen hundreds of Drupal events, including job fairs, meetups and workshops, study group meetings, conferences about design and theming, paid trainings, FREE tutoring sessions from Drupal professionals and barn raisings to benefit non-profits and members of our community here in and around Downtown Los Angeles.

Attending Drupal events in and around Downtown Los Angeles is one of the best ways to meet and talk with other Drupaleros and we encourage you to attend as many meetings and special events as you'd like. Whether it's to find solutions to problems you've been having, sharing something you've learned or just meeting interesting like-minded people, the Downtown Los Angeles Drupal events are an essential resource for Drupal professionals and hobbyists alike.

If you aren't already a member of Downtown Los Angeles Drupal, it's easy to join our community. Our community calendar is on our "Events" tab on our home page at http://groups.drupal.org/dtla  

The Downtown Los Angeles Drupal group proudly participates in the California Drupal Travelers Program and can host businesses and community members who are visiting the area. Inquire within by contacting any of the Downtown Los Angeles Drupal organizers.

Tags:

• Meetup
• Planet Drupal

The best type of blog post is the type that leads to action.

It seems that my recent post Drupal Modules: Usability Starts With A Name seems to have done exactly that.

As a result of that post and the comments that followed (on this website and in the issue queue), I have renamed the Storm module to Project Management.

Project Management is now the name and the description of the module. A new user now knows the context of the module without any additional explanation.

I have wondered since whether it was the right decision. But I am reassured that there was unanimous support for this change in comments, and the number of people participating in the issue queue has increased significantly since the change.

You can try Project Management (still a work in progress for Drupal 7) on this demo site. A version is also available for Storm (the stable modules for Drupal 6).

This practical example is, I hope, a motivation for more module maintainers to reflect that "Usability Starts With A Name".

Category: WebsitesTags: Drupal PlanetStormProject ManagementusabilityDrupal

Writing engaging, reusable microcontent is tricky business. Whether you need titles, tweets, or summaries, consider the destination channels and the workflow.

Writing short bits of user-facing text -- microcontent -- is no picnic. Coming up with a punchy, attention-grabbing tweet is tough enough; writing a memorable 50 character title for a breaking news story can stress out even a creative wordsmith. It's like the writer's equivalent of Fitts's Law: the smaller the target, the narrower the margin for error.

In heavy-duty, reuse-oriented publishing systems, it's common practice to save several variations of an article's title and summary text. That gives writers some breathing room in more forgiving display contexts, but ensures they don't blow past hard limits for the short stuff.

We're currently working with a client on the nitty-gritty details of their new content model, and we're trying to iron out the best mix of fields to provide flexibility without overloading content authors. How many variations are enough? Karen McGrane's advice is simple and to the point: "As many as the writers will fill out, but no more." We plan to do some experiments with simple prototype interfaces to see what they're comfortable with, but before proceeding I did a quick review of the microcontent landscape to better understand the constraints of popular formats and channels.

From longest to shortest, here's the rundown:

• App.net post: 256 chars
• Twitter card summary text: 200 chars
• Facebook og:description text: 160 chars
• Google page description: 155 chars

One of the better features of the views module in Drupal is the ability to cache your view's output. This can come in handy when your view is doing a lot of computation. Caching your view will save your server a lot of unneeded work. One of the big current drawbacks of this feature is if you enable caching for your view and you have an exposed filter, you'll run into the following scenario:

User A enters a value for the filter

• Read more

So you have a virtualhost set up for Apache and your local site is running at http://yoursite.local and everything is working great on your computer. Now you want to test that awesome responsive design you've been working on with some other device(s), so you try to go to yoursite.local which won't work unless you want to modify the host file on the device(s), which is something you can\'t even do to an iPhone/iPad without jailbreaking it. Gah.

So, then you try to access the site with your device by just using your computer's IP address + the path to your docroot (for instance 10.0.1.10/mysite/trunk/htdocs/) - great idea and now the site at least sort of loads, but all the links to the CSS/JS and image files are broken...

In my own search to avoid jailbreaking I finally figured out how to enable normal-looking-pages access to devices following a bit of trial and error. The whole setup to enable this is done on the computer your site exists on - no need to fiddle with any of your other device(s) settings:

1. Edit the /etc/host file on your computer and add you computer's IP along with the relative path to your site (relative to Apache's docroot). For example:

10.0.1.10 path/to/site/

2. Add a virtualhost to apache like so, using your computer's IP address as the server name and this time use the absolute path for the site:

<VirtualHost *:80>
DocumentRoot "/Users/myuser/Sites/path/to/site/"
ServerName 10.0.1.10
</VirtualHost>

3. Restart Apache and now you should be able to access your site with any other device on your network by simply using your computer's IP address as the url.

Categories: Drupalmobiledevelopmentresponsive

So as previously mentioned I gave a talk at FOSDEM this year on the future of paravirtualisation under Xen. I thought the talk was reasonably well attended despite being at six o'clock in the evening. In the end I underran my half hour slot but fortunately people had plenty of interesting questions! I've posted my slides on my xenbits space.

I spent a fair chunk of the rest of the time on the Xen.org booth, which is hard work but it is always nice to speak to users and other developers. I also managed to catch up with various friends from other projects and managed to meet a few folks who I've only ever met in email before.

I didn't get to see as many talks as I wanted (for example I missed pretty much all of the ARM track :-() due to clashes with my stints on the booth, my own talk, and talks clashing with each other. Still it was all in all a worthwhile, hopefully I'll be able to make it again next year!

Next up: LinaroConnect in Hong Kong!

Knowing what your customers are doing before spending money in your Drupal Commerce store can be valuable information for your business. It could help you create the shopping experience that meets your customers needs. With Google Analytics you can collect valuable visitor behavior information. Drupal Commerce is where the actual sale happens. In this tutorial I will learn you how to put those two together. It enables you to integrate Drupal Commerce order information with Google Analytics.

Configure the Google Analytics profile

First enable ecommerce tracking in your site's Google Analytics Profile. Learn how to do that in this Google Analytics tutorial on enabling ecommerce tracking. And while you're at it, make sure it reflects the right currency amount in that profile. 

Configure Google Analytics Module

Download and enable Google Analytics module. To allow Drupal Commerce order tracking you should at least enter the Google Analytics UA id at example.com/admin/config/system/googleanalytics. If you want to do more advanced things like for example 'user segmentation' or 'internal site search' tracking, have a look at this tutorial about configuring Google Analytics module.

Configure Commerce Google Analytics Module

Download and enable Commerce Google Analytics module. No configuration required. Yes, module maintainers rock. They are making this even cooler: Rules integration is currently in development. 

Configure Drupal Commerce product Titles and SKU's

Make sure your Drupal Commerce product titles and Stock Keeping Units make sense as they will be used in reporting. Note that Commerce product titles are a different thing than the titles of your product display nodes. If you're using Commerce Kickstart, have a look at this tutorial on how to change the product titles.

Test ordering Drupal Commerce products

Always be testing: place a few orders in your store, and check if they show up in your Google Analytics profile. Make sure you set the Google Analytics date range to include today - by default it doesn't include today. And make sure the order amounts, product prices, tax amounts and shipping cost are correctly tracked.

Testing orders without paying

When testing your ecommerce order analytics you don't want to actually pay for the order with money. Have a look at the Commerce No Payment module to be able to pay without really paying for it.

Tracking Email Metrics with Drupal Commerce

If you're interested in tracking eCommerce metrics regarding email campaigns, have a look at [my upcoming tutorial about Tracking MailChimp campaign Metrics with Drupal Commerce and Google Analytics].

Category: Drupal Planet Drupal Commerce Google Analytics

Episode Number:  108 Post Topics:  Drupal Contrib Drupal 7 Drupal Planet Site Administration

The Drupal 7 Coffee Module is a handy module that streamlines the administration of a Drupal 7 site by allow a quick search like mechanism for navigating to various administrative screens or for adding new content to your Drupal website.

In this episode you will learn:

• How the Drupal Coffee module can speed up your Drupal administration tasks
• How to use the Drupal Coffee module to navigate around Drupal administration pages

DDoD Video: 

read more

Subunit is seven and a half years old now – Conrad Parker and I first sketched it up at a CodeCon – camping and coding, a brilliant combination – in mid 2005.

revno: 1 committer: Robert Collins <robertc@robertcollins.net> timestamp: Sat 2005-08-27 15:01:20 +1000 message:  design up a protocol with kfish

It has proved remarkably resilient as a protocol – the basic nature hasn’t changed at all, even though we’ve added tags, timestamps, support for attachments of arbitrary size.

However a growing number of irritations have been building up with it. I think it is time to design another iteration of the protocol, one that will retain the positive qualities of the current protocol, while helping it become suitable for the next 7 years. Ideally we can keep compatibility and make it possible for a single stream to be represented in any format.

Existing design

The existing design is a mostly human readable line orientated protocol that can be sniffed out from the regular output of ‘make’ or other build systems. Binary attachments are done using HTTP chunking, and the parser has to maintain state about the current test, tags, timing data and test progression [a simple stack of progress counters]. How to arrange subunit output is undefined, how to select tests to run is undefined.

This makes writing a parser quite easy, and the tagging and timestamp facility allow multiplexing streams from two or more concurrent test runs into one with good fidelity – but also requires that state be buffered until the end of a test, as two tests cannot be executing at once.

Dealing with debuggers

The initial protocol was intended to support dropping into a debugger – just pass each line read through to stdout, and connect stdin to the test process, and voila, you have a working debugger connection. This works, but the current line based parsers make using it tedious – the line buffered nature of it makes feedback on what has been typed fiddly, and stdout tends to be buffered, leading to an inability to see print statements and the like.  All in-principle fixable, right ?

When running two or more test processes, which test process should stdin be connected to? What if two or more drop into a debugger at once? What is being typed to which process is more luck than anything else.

We’ve added some idioms in testrepository that control test execution by a similar but different format – one test per line to list tests, and have runners permit listing and selecting by a list. This works well, but the inconsistency with subunit itself is a little annoying – you need two parsers, and two output formats.

Good points

The current protocol is extremely easy to implement for emitters, and the arbitrary attachments and tagging features have worked extremely well. There is a comprehensive Python parser which maps everything into Python unittest API calls (an extended version of the standard, with good backwards compatibility).

Pain points

The debugging support was a total failure, and the way the parser depraminates it’s toys when a test process corrupts an outcome line is extremely frustrating. (other tests execute but the parser sees them as non-subunit chatter and passes the lines on through stdout).

Dealing with concurrency

The original design didn’t cater for concurrency. There are three concurrency issues – the corruption issue (see below for more detail) and multiplexing. Consider two levels of nested concurrency: A supervisor process such as testrepository starts 2 (or more but 2 is sufficient to reason about the issue) subsidiary worker processes (I1 and I2), each of which starts 2 subsidiary processes of their own (W1, W2, W3, W4). Each of the 4 leaf processes is outputting subunit which gets multiplexed in the 2 intermediary processes, and then again in the supervisor. Why would there be two layers? A concrete example is using testrepository to coordinate test runs on multiple machines at once, with each machine running a local testrepository to broker tests amongst the local CPUs. This could be done with 4 separate ssh sessions and no intermediaries, but that only removes a fraction of the issues. What issues?

Well, consider some stdout chatter that W1 outputs. That will get passed to I1 and from there to the supervisor and captured. But there is nothing marking the chatter as belonging to W1: there is no way to tell where it came from. If W1 happened to fail, and there was a diagnostic message printed, we’ve lost information. Or at best muddled it all up.

Secondly, buffering – imagine that a test on W1 hangs. I1 will know that W1 is running a test, but has no way to tell the supervisor (and thus the user) that this is the case, without writing to stdout [and causing a *lot* of noise if that happens a lot]. We could have I1 write to stdout only if W1′s test is taking more than 5 seconds or something – but this is a workaround for a limitation of the protocol. Adding to the confusion, the clock on W1 and W3 may be very skewed, so timestamps for everything have to be carefully synchronised by the multiplexer.

Thirdly, scheduling – if W1/W2 are on a faster machine than W3/W4 then a partition of equal-timed tests onto each machine will lead one idle before the other finishes. It would be nice to be able to pass tests to run to the faster machine when it goes idle, rather than having to start a new runner each time.

Lastly, what to do when W1 and W2 both wait for user input on stdin (e.g. passphrases, debugger input, $other). Naively connecting stdin to all processes doesn’t work well. A GUI supervisor could connect a separate fd to each of I1 and I2, but that doesn’t help when it is W1 and W2 reading from stdin.

So additional requirement over baseline subunit:

1. make it possible for stdout and stderr output to be captured from W1 and routed through I1 to the supervisor without losing its origin. It might be chatter from a noisy test, or it might be build output. Either way, the user probably will benefit if we can capture it and show it to them later when they review the test run. The supervisor should probably show it immediately as well – the protocol doesn’t need to care about that, just make it possible.
2. make it possible to pass information about tests that have not completed through one subunit stream while they are still incomplete.
3. make it possible (but optional) to pass tests to run to a running process that accepts subunit.
4. make it possible to route stdin to a specific currently process like W1. This and point 3 suggest that we need a bidirectional protocol rather than the solely unidirectional protocol we have today. I don’t know of a reliable portable way to tell when some process is seeking such input, so that will be up to the user I think. (e.g. printing (pdb) to stdout might be a sufficiently good indicator.)

Dealing with corruption

Consider the following subunit fragment:

test: foo starting serversuccess:foo

This is a classic example of corruption: the test ‘foo’ started a server and helpfully wrote to stdout explaining that it did that, but missed the newline. As a result the success message for the test wasn’t printed on a line of its own, and the subunit parser will believe that foo never completed. Every subsequent test is then ignored. This is usually easy to identify and fix, but its a head-scratcher when it happens. Another way it can happen is when a build tool like ‘make’ runs tests in parallel, and they output subunit onto the same stdout file handle. A third way is when a build tool like make runs two separate test scripts serially, and the first one starts a test but errors hard and doesn’t finish it. That looks like:

test: foo test: bar success: bar

One way that this sort of corruption can be mitigated is to put subunit on it’s own file descriptor, but this has several caveats: it is harder to tunnel through things like ssh and it doesn’t solve the failing test script case.

I think it is unreasonable to require a protocol where arbitrary interleaving of bytes between different test runner streams will work – so the ‘make -j2′ case can be ignored at the wire level – though we should create a simple way to safely mux the output from such tests when the execute.

The root of the issue is that a dropped update leaves bad state in the parser and it never recovers. So some way to recover, or less state to carry in the parser, would neatly solve things. I favour reducing parser state as that should shift stateful complexity onto end nodes / complex processors, rather than being carried by every node in the transmission path.

Dependencies

Various suggestions have been made – JSON, Protobufs, etc…

A key design goal of the first subunit was a low barrier to entry. We keep that by being backward compatible, but being easy to work with for the new revision is also a worthy goal.

High level proposal

A packetised length prefixed binary protocol, with each packet containing a small signature, length, routing code, a binary timestamp in UTC, a set of UTF8 tags (active only, no negative tags), a content tag – one of (estimate + number, stdin, stdout, stderr, test- + test id), test status (one of exists/inprogress/xfail/xsuccess/success/fail/skip), an attachment name, mime type, a last-block marker and a block of bytes.

The content tags:

• estimate – the stream is reporting how many tests are expected to run. It affects everything with the same routing code only, and replaces (doesn’t adjust) any current estimate for that routing code. A estimate packet of 0 can be used to say that a routing target has shutdown and cannot run more tests. Routing codes can be used by a subunit aware runner to separate out separate threads in a single process, or even just separate ‘TestSuite’ objects within a single test run (though doing so means that they will need to process subunit and strip packets on stdin. This supercedes the stack of progress indicators that current subunit has. estimates cannot have test status or attachments.
• stdin/stdout/stderr: a packet of data for one of these streams. The routing code identifies the test process that the data came from/should go to in the tree of test workers. These packets cannot have test status but should have a non-empty attachment block.
• test- + testid: a packet of data for a single test. test status may be included, as may attachment name, mime type, last-block and binary data.

Test status values are pretty ordinary. Exists is used to indicate a test that can be run when listing tests, and inprogress is used to report a test that has started but not necessarily completed.

Attachment names must be unique per routing code + testid.

So how does this line up?

Interleaving and recovery

We could dispense with interleaving and say the streams are wholly binary, or we can say that packets can start either after a \n or directly after another packet. If we say that binary-only is the approach to take, it would be possible to write a filter that would apply the newline heuristic (or even look for packet headers at every byte offset. I think mandating adjacent to a packet or after \n is a small concession to make and will avoid tools like testrepository forcing users to always configure a heuristic filter. Non-subunit content can be wrapped in subunit for forwarding (the I1 in W1->I1->Supervisor chain would do the wrapping). This won’t eliminate corruption but it will localise it and permit the stream to recover: the test that was corrupted will show up as incomplete, or with incomplete attachment data.

listing

Test listing would emit many small non-timestamped packets. It may be useful to have a wrapper packet for bulk amounts of fine grained data like listing is, or for multiplexers with many input streams that will often have multiple data packets available to write at once.

Selecting tests to run

Same as for listing – while passing regexes down to the test runner to select groups of tests is a valid use case, thats not something subunit needs to worry about : if the selection is not the result of the supervisor selecting by test id, then it is known at the start of the test run and can just be a command line parameter to the backend : subunit is relevant for passing instructions to a runner mid-execution. Because the supervisor cannot just hand out some tests and wait for the thing it ran to report that it can accept incremental tests on stdin, supervisor processes will need to be informed about that out of band.

Debugging

Debugging is straight forward . The parser can read the first 4 or so bytes of a packet one at a time to determine if it is a packet or a line of stdout/stderr, and then either read to end of line, or the binary length of the packet. So, we combine a few things; non-subunit output should be wrapped and presented to the user. Subunit that is being multiplexed and forwarded should prepend a routing code to the packet (e.g. I1 would append ’1′ or ’2′ to select which of W1/W2 the content came from, and then forward the packet. S would append ’1′ or ’2′ to indicate I1/I2 – the routing code is a path through the tree of forwarding processes). The UI the user is using needs to supply some means to switch where stdin is attached. And stdin input should be routed via stdin packets. When there is no routing code left, the packet should be entirely unwrapped and presented as raw bytes to the process in question.

Multiplexing

Very straight forward – unwrap the outer layer of the packet, add or adjust the routing code, serialise a header + adjusted length + rest of packet as-is. No buffering is needed, so the supervisor can show in-progress tests (and how long they have been running for).

Parsing / representation in Python or other languages

The parser should be very simple to write. Once parsed, this will be fundamentally different to the existing Python TestCase->TestResult API that is in used today. However it should be easy to write two adapters: old-style <-> this new-style. old-style -> new-style is useful for running existing tests suites and generating subunit, because thats way the subunit generation is transparent. new-style->old-style is useful for using existing test reporting facilities (such as junitxml or html TestResult objects) with subunit streams.

Importantly though, a new TestResult style that supports the features of this protocol would enable some useful features for regular Python test suites:

• Concurrent tests (e.g. in multiprocessing) wouldn’t need multiplexers and special adapters – a regular single testresult with a simple mutex around it would be able to handle concurrent execution of tests, and show hung tests etc.
• The routing of input to a particular debugger instance also applies to a simple python process running tests via multiprocessing, so the routing feature would help there.
• The listing facility and incrementally running tests would be useful too I think – we could go to running tests concurrently with test collection happening, but this would apply to other parts of unittest than just the TestResult

The API might be something like:

class StreamingResult(object): def startTestRun(self): pass def stopTestRun(self): pass def estimate(self, count, route_code=None): pass def stdin(self, bytes, route_code=None): pass def stdout(self, bytes, route_code=None): pass def test(self, test_id, status, attachment_name=None, attachment_mime=None, attachment_eof=None, attachment_bytes=None): pass

This would support just-in-time debugging  by wiring up pdb to the stdin/stdout handlers of the result object, rather than actual stdin/stdout of the process – a simple matter once written. Alternatively, the test runner could replace sys.stdin/stdout etc with thunk file-like objects, which might be a good idea anyway to capture spurious output happening during a test run. That would permit pdb to Just Work (even if the test process is internally running concurrent tests.. until it has two pdb objects running concurrently

Generation new streams

Should be very easy in anything except shell. For shell, we can have a command line tool that when invoked outputs a subunit stream for one instruction. E.g. ‘test foo completed + some attachments’ or ‘test foo starting’.

Over the last few days I've done quite a lot of work to try and get our puppet configuration up to modern best practises. The Puppet Labs folks strongly encourage you to make as much use of puppet modules as possible. A puppet module gathers together puppet manifests, facter facts and other bits and pieces into a reusable component that you could potentially share with others. Many modules (of very mixed quality) are available on the web, in particular at github and via Puppet Lab's own Forge.

Since version 2.7.14, the puppet command-line tool has built-in support for managing modules:

# puppet module list /etc/puppet/modules ├── auth (???) ├── interfaces (???) … # puppet module install puppetlabs-apt …

However, they have not provided support for managing modules in puppet manifests themselves. This strikes me as a bit odd: the whole point of using puppet to manage your machines is to capture the configuration in one place. If your configuration depends on a collection of modules and module versions, you'd ideally record that in the puppet configuration itself. Otherwise building a new puppet master is a mixture of manually installing modules and setting it up as a client of your existing master.

Other people to the rescue: Ryan Coleman (and Pieter van de Bruggen) have written puppet_module_provider which does exactly this. (Note that they both work for Puppet Labs). Now I can define which modules and what versions are necessary:

class puppetmaster { … module { 'rcoleman/puppet_module': ensure => '0.0.3', } module { 'puppetlabs/firewall': ensure => '0.0.4', } module { 'puppetlabs/lvm': ensure => '0.1.1', } module { 'puppetlabs/ntp': ensure => '0.2.0', } …

I've decided to pin specific versions given the highly variable nature of module quality. There's no guarantee of backwards compatibility except by reputation of the publisher. The puppetlabs modules are likely to be of higher quality than average, so over time I'll probably gain confidence to change the specific versioning to 'latest' or similar. I also want to double-check how secure the module selection and downloading is (whether it's over HTTPS or cryptographic signatures are checked etc.)

14 FebA distribution for Drupal User Groups II

I had an earlier blogpost about our work on a design and distribution package that Drupal User Groups all over the world could use. After having the concept refined and the wireframes ready, we went on with creating the design and organizing a code sprint to get things going.

Read more

Image Style Basics

Drupal offers the possibility to easily process images via image styles (formerly known as imagecache presets). In our current case of a photo heavy website we had to integrate a watermark image on all images supplied by the site. For this purpose we used the image style action Overlay (watermark (canvasactions_file2canvas_image).

The configuration of the style is really simple - but could be enriched with scaling and all other available actions.Read more

Next round of phpMyAdmin 4.0 translation status report is coming.

So let's look at which translations are at 100% right now (some changes this week due to new strings to translate):

• Czech
• Dutch
• English (United Kingdom)
• French
• Greek
• Russian
• Spanish
• Turkish

There are also several languages which need just few strings to be complete:

• Danish
• Estonian
• Galician
• German
• Japanese
• Polish
• Portuguese (Brazil)
• Slovenian
• Swedish

As you can see, there is still lot of languages missing, this might be your opportunity to contribute to phpMyAdmin. Also you are welcome to translate phpMyAdmin 4.0 using translation server.

If your language is already fully translated and you want to help as well, you can translate our documentation as well.

Filed under: English Phpmyadmin | 0 comments | Flattr this!

The TableField module implements a custom field that allows users to attach tabular data to any content page/entity type. Data can be entered into the table manually or imported via a CSV upload. The amount of rows and columns can be configured globally on an entity type or on a per node basis. The field can also be configured to allow users to export the tabular data as a CSV file.

In this tutorial, we'll create a Company content type and use this module to display opening times.

One of my main goals here at the DA is to communicate - A LOT - about what's going on. It's my intention to put a blog post up after every board meeting summarizing our discussion so that you get more than just the meeting minutes to go by. Well, today was my very first board meeting as Executive Director, and, well, the open meeting was REALLY short. Like under five minutes short.

Body: 

Magda

aGov is a Drupal distribution built to address the accessibility, security and design guidelines for Australian Government sites. In a previous blog, I talked about some of the benefits aGov has for public sector agencies trying to do more with less in the online environment. In this post, I'm going to show you are practical example of building a micro-site using aGov, to show you how easy, and quick, it is to get up and running.

We're going to be building a site for a government initiative, but you could easily use the same methodology for a smaller public sector agency, a special event, an informational site, or a minister.

With 2013 well underway, we thought it was time to circle up in the office and bust out an open source sprint. Our focus this time was on RedHen, our native Drupal CRM, which is fully open sourced and lives on Drupal.org. In one afternoon, with our team of 8, we were able to give the issue queue some lovin' and also roll the release of RedHen 1.0!

So kudos and applause to Lev, Sean, Tauno, Brandon, Brett, Gabe and David, ThinkShout's newest team member who was in Portland for the first time, for coding up a storm. Here's a quick update on the most recent 1.0 releases we've been able to get out the door:

• RedHen core, 'nough said.
• RedHen membership, a separate module to manage association memberships.
• Entity registrations, ThinkShout's Drupal 7 answer to Signup which is becoming the defacto Drupal event registration solution.
• Poultry, the responsive theme that can be used to power RedHen's interface.
• RedHen Demo, an installation profile that packages all of RedHen's features into a fictional pet shelter.

As I said, RedHen is fully open sourced and its success is possible due to the support we receive from the community. So I also want to give a shout out to all of the contributors who submitted patches.

Sometimes the message is at least as important as the code behind it. As such, our wordsmith Gabe Carleton-Barnes made the RedHen project page decipherable and, dare I say, fun?

Tags: RedHenCode SprintEntity registrationsPoultryCRMDrupalDrupal Planet