Feed aggregator

Aten Design Group: Drupal Migrate for Development Content

Planet Drupal - Tue, 14/10/2014 - 19:43

Drupal and many of the people who work with it are moving toward a configuration in code model of site development. One of the advantages of a config in code approach is that the code you add, share, and modify works for all the members of your team and across environments. Instead of everyone syncing databases (or passing around notes on how to update their environment because something changed), everything stays up-to-date with the latest code in your version control system. This essentially provides a known, common state for everyone to work against.

Configuration only gets you part of the picture, though. Modules like Features and configuration initiatives for Drupal 8 separate configuration from content. Configuration is sharable settings; content is the information that site stores/uses. This separation makes sense in organizing your code or site, but leaves a big gap in your ability to build and test a project. If I'm working on a project locally, I can't share a link to the article node I'm having trouble with because it’s a combination of configuration and content that exists only on my local machine.

You need standardized content to test against and to provide a common ground to review variations with your team. But what do you do when you're starting on a project and you don't have content from a client yet? You still need to develop code that uses content and you still need to style the site.

Luckily, we can use a common approach for bringing in content from another source, the Migrate module, to help create content we can share and test against. Additionally, the content can be updated, version controlled, and contributed back as the project rolls forward. And – this is very important for development content – when we're finished with dev content we can remove it with Migrate's rollback functionality. Content created with some modules like Devel Generate and even manually created development content aren’t easily removed when you're finished. At Aten, we commonly had many nodes with "DELETE" in the title to make it "easy" for us to find and remove it later, which is less than ideal.

How does this all work? This is our internal workflow:

  • Create a resources folder in your project. Typically we now have a "root" level that has resources, a public_html folder (which has the Drupal files), and other project files.
  • Inside of the resources, we create a content directory and add content files like YAML files, CSVs, etc. (more on that in a minute)
  • We have started to use gulp and we have a task that will convert the YAML files to JSON.
  • We create a custom module in the project for migrate and add migrate classes for each of the content files we need to import. Typically this will be something like "project_content". For dev specific content, we name the migrations with "Dev" on the end. When we have production content (which is awesome to have early in a project), we leave that suffix off the class name since that content isn't something we need to rollback later.
  • We've created a script that is shared in the project that enables/disables modules, enables and reverts features, runs the migrations, updates various other things related to the project setup. If I add a new migration, I update that script's configuration to include it for others working on the project. I hope to share more about this script soon.
Creating Test Content

Now we need to create the content. Typically this requires some insight provided by our Drupal architecture document, but I have also created a couple of tools to help out with this process which help me stay in code:

The typeinfo commands allow you to inspect content types/entities on the site. For example, if you are going to create content for an Article content type, you would run:

drush typeinfo article

That will output the content type's fields, field types, and some other information. Often this provides a good overview of what pieces you will need to create in your content file. If you have a taxonomy or entity reference field in that content type, you can also get more information about that via another typeinfo command:

drush typeinfo-field field_article_type

This will return a few specifics that may show you the taxonomy that field uses. And now we can use the taxonomyinfo command to list terms in that vocabulary:

drush taxonomyinfo-term-list article_type

We can also extend this functionality to automatically stub some of the content (à la devel-generate) by creating another drush command. This command lets us get the YAML with some data populated for us:

drush stub-content article --include-id --include-title --count=5 > resources/content/article.yaml

An example of this command is here: https://gist.github.com/robballou/a7aa247aa7bdfb3a1b2c

The stub content functionality makes some really rudimentary content and you can expand that with content from your favorite ipsum replacement or other sources.

We can migrate from a variety of sources: from CSV files, JSON files, or even other databases. CSV files are a popular choice because you can collaborate on a spreadsheet (especially via Google sheets) and export that data. JSON is another nice solution because the data can match the destination closely. In some of our projects we have even used YAML and converted that to JSON since the readability of YAML is slightly better than JSON — which means we can have people write content who don't know the ins-and-outs of JSON!

Some systems may have access to a PHP YAML library and it could be used to create a Migrate YAML source class. This would eliminate the need to convert files but may rely on that YAML library to be available on local, staging, and production servers. We've used the node.js/gulp approach because it can be shared between environments and projects that may not have this PHP support built in. Migrating and Removing Test Content

This article won't get into the details of creating your Migrate code, but the next step in the process is creating and testing the code to get this content into Drupal. When this is done, commit this to your version control system to share with others working on the project or with other systems.

As an example, we'll say we created a migration called ArticlesDev which has a handful of articles in it. The content uses a variety of the fields in the content so we can make sure all the functionality works and includes several nodes so we can test lists of various sizes. We can import the content into any system with:

drush migrate-import ArticlesDev

If the article content type changes down the line, you can update the content files and re-run the migration, updating the existing content (or adding new content):

drush migrate-import --update ArticlesDev

Development-specific content may never get imported on shared systems, but if you do want to use that content for client acceptance testing or for any other case, you can easily remove this content with:

drush migrate-rollback ArticlesDev Content in Code

If you're working in a team or if you need a client to review functionality, development content can be very handy. Building on this workflow, you can get a set of content in place early in the process, update it as things change, and get rid of it if you don't need it anymore. Your team and your clients have a common ground when discussing the project. As a bonus, your development migration code can be used as a basis for creating or importing live content as you get it from the client.

Categories: Elsewhere

Gunnar Wolf: When Open Access meets the Napster anniversary

Planet Debian - Tue, 14/10/2014 - 18:58

Two causally unrelated events which fit in together in the greater scheme of things ;-)

In some areas, the world is better aligning to what we have been seeking for many years. In some, of course, it is not.

In this case, today I found our article on the Network of Digital Repositories for our University, in the Revista Digital Universitaria [en línea] was published. We were invited to prepare an article on this topic because this month's magazine would be devoted to Open Access in Mexico and Latin America — This, because a law was recently passed that makes conditions much more interesting for the nonrestricted publication of academic research. Of course, there is still a long way to go, but this clearly is a step in the right direction.

On the other hand, after a long time of not looking in that direction (even though it's a lovely magazine), I found that this edition of FirstMonday takes as its main topic Napster, 15 years on: Rethinking digital music distribution.

I know that nonrestricted academic publishing via open access and nonauthorized music sharing via Napster are two very different topics. However, there is a continuous push and trend towards considering and accepting open licensing terms, and they are both points in the same struggle. An interesting data point to add is that, although many different free licenses have existed over time, Creative Commons (which gave a lot of visibility and made the discussion within the reach of many content creators) was created in 2001 — 13 years ago today, two years after Napster. And, yes, there are no absolute coincidences.

Categories: Elsewhere

SitePoint PHP Drupal: Quick Tip: Up and Running with Drupal 8 in Under Five Minutes

Planet Drupal - Tue, 14/10/2014 - 18:51

In this quick tip, we’ll be installing a local instance of Drupal 8, beta 1. By the end, you’ll have a copy of Drupal that’s not only ready to be extended with Symfony bundles and other packages, but also ready to accept content and display it to end users.

Step 1: Prepare Environment

Continue reading %Quick Tip: Up and Running with Drupal 8 in Under Five Minutes%

Categories: Elsewhere

Open Source Training: The Payment Module: A Simpler Alternative to Drupal Commerce

Planet Drupal - Tue, 14/10/2014 - 18:45

Here at OSTraining we've given significant coverage to Drupal Commerce. You can watch a video class with nearly 30 lessons and download a book, "Building E-commerce Sites with Drupal Commerce".

However, Drupal Commerce is an enterprise-quality solution and a good number of OSTraining members have asked for simpler solutions.

For those members, we often recommend the Payment module which makes it easy to add e-commerce fields to your content.

Payment supports about half-a-dozen gateways (PayPal, Stripe, iDEAL, Authorize.net, Ogone, Rabo OmniKassa) and we'll use PayPal in this tutorial.

Categories: Elsewhere

Marco d'Itri: The Italian peering ecosystem

Planet Debian - Tue, 14/10/2014 - 18:34

I published the slides of my talk "An introduction to peering in Italy - Interconnections among the Italian networks" that I presented today at the MIX-IT (the Milano internet exchange) technical meeting.

Categories: Elsewhere

DrupalCon Amsterdam: Doei Doei, DrupalCon Amsterdam

Planet Drupal - Tue, 14/10/2014 - 18:04

This post was originally shared on the Drupal Association blog.

DrupalCon Amsterdam has wrapped up, and now that we're over the jet lag, it's time to look back on one of the most successful DrupalCons to date. DrupalCon Amsterdam was the largest European DrupalCon yet, by far. Just to knock your socks off, here are some numbers:

  • More than 2,300 attendees showed up to 120 sessions and nearly 100 BoFs
  • 115 attendees showed up to the community summit and the business summit
  • 146 training attendees, 400 trivia night attendees, and 400 Friday sprinters made the week a success
  • …and through it all, we ate 1,200 stroopwafels.

The fun extended to more than just the conference -- with 211 transit passes and 56 bike rentals, attendees from over 64 countries were able to enjoy all the city of Amsterdam had to offer. What a success!
As with any DrupalCon, DrupalCon Amsterdam wouldn't have been a success without lots and lots of help from our passionate volunteers. We'd like to take a moment to send out a big THANK YOU to all of our track chairs and summit organizers:

  • Pedro Cambra - Coding and Development
  • Théodore Biadala - Core Conversations
  • Steve Parks - Drupal Business
  • Lewis Nyman and Ruben Teijeiro - Frontend
  • Michael Schmid - Site Building
  • Bastian Widmer - DevOps
  • Cathy Theys, Ruben Teijeiro, Gábor Hojtsy and the Core Mentors - Sprints
  • Adam Hill and Ieva Uzule - Onsite Volunteer Coordinators
  • Baris Wanschers - Social Media
  • Emma Jane Westby - Business Summit
  • Morten Birch and Addison Berry - Community Summit

We also appreciate everything our core mentors did to make DrupalCon Amsterdam a hit — and it’s thanks to lots of hard work from our passionate community members that Drupal 8 is in Beta!
We hope you had as fun and exiting a time in Amsterdam as we did. For those of us who weren’t able to make it, and even for those who were, you can relive the fun on the flickr stream, or catch any number of great sessions on the Drupal Association YouTube channel.  And remember to mark your calendar for DrupalCon Barcelona in September 2015. See you there! Below is Holly Ross' set of slides from the Amsterdam closing session.

DrupalCon Amsterdam Closing Session from DrupalAssociation
Image credit to Pedro Lozano on Flickr.

Categories: Elsewhere

ComputerMinds.co.uk: Language lessons: 10 Useful tips

Planet Drupal - Tue, 14/10/2014 - 14:00

To complete my series on multilingual Drupal, here are some mini-lessons I've learnt. Some of them are are to improve the experience for administrators & translators, others cover obscure corners of code. The first few don't require any knowledge of code, but the later ones will be much more technical.

Categories: Elsewhere

Amazee Labs: Drupalcon Amsterdam: some (user) experience

Planet Drupal - Tue, 14/10/2014 - 13:46
Drupalcon Amsterdam: some (user) experience

DrupalCon Europe 2014 was all about a new era that has now began on Wednesday, October 1st with the official beta 1 release - the era of Drupal 8. Compared with the DrupalCon Europe 2013, this conference was more about business and development and less about UX related matters. However, UX topics and raised questions are definitely worth mentioning.

UX and Dev: We need to have a date!

One of the most frequent UX topics of last year was better user experience crafted for the end-users of a website together with the importance of user research as a business objective. Knowledge of how and why people use a product conveys better understanding where the areas of improvements that need immediate mitigation are. 

Last week, voices were raised to provide a clear statement that the new and shiny Drupal 8 and the further development of Drupal on both core and contrib levels, needs to get UX research and design involved. Some actions were taken during the last year. However, there is definitely more potential. In her talk "Engaging UX and Design Contributors", Dani Nordin makes it clear. All people who contribute to Drupal have a common goal to create a better, highly scalable and stable version of our content management system. And to do so, UX researches, designers and developers are bound to work closely together, listen to each other and actually apply results of user research. To make this happen, we have to take action within the Drupal community. For example, UX researches and designers should be more involved in the development processes and procedures. Sometimes, designers and UX professionals are treated, as if their contribution is not important, which leads to a decrease of their motivation to spend their spare time for the community. Raising this topic lead to a lot of discussions. This resulted in the UX sprint producing a list of goals and directions that the community should go for in 2014 & 2015.

It’s about interaction.

More and more discussions within the community were devoted to interaction design. The word “design” has a rather broad definition and can be understood differently. For example, “design” for the majority of people only means visual design or visual representation of the object.  UX researches and designers differentiate the visual and interaction design. The latter is about the workflows or concrete steps that users should perform to achieve the goal. No doubt, visual design is crucial for making people want to start an interaction with an object in the first place. However, the quality of interactions is the key for the product’s success. The best way to determine a user-friendly interaction is to actually attempt to accomplish a task, find the obstacles, redefine the workflow and test again before the implementation even starts. The best way to do so is prototyping. No matter how you create your prototypes, you can use HTML, as Roy Scholten suggests in his talk “Because it's about the interactions. (Better UX through prototyping)”, or a broad palette of other tools that do not require advanced programming skills. The most crucial point is to prove that interactions really work and they do so for real users.  

Don’t prototype in Drupal!

The topic of the importance of prototyping gained further attention in the presentation of Dani Nordin. Don’t prototype in Drupal is a strong statement, and it has a point. One of the most popular prototyping tools Axure can help with the design at different levels: high and low fidelity. Creating Axure-based prototypes is faster and cheaper than spending days or weeks on Drupal site building and coding. Prototypes allow, however, to ensure the directions of a project, make it possible to talk with stakeholders about tangible and interactive product before applying the development forces and, again, it’s the way to receive the user feedback. This insighs can be applied not only to the products built in Drupal, but to Drupal itself. This will improve not only the life of content creators, but can help developing more elegant solutions using the better design features of the Drupal core together with contrib modules. 

 

Categories: Elsewhere

Paul Johnson: Welcome to Amsterdam, the "free ride" stops here

Planet Drupal - Tue, 14/10/2014 - 11:18

For a long time I’ve harboured the belief that too few in Drupal receive the recognition they deserve. I often wonder how many bright (young) stars fall through the cracks of our vast community, fade away and perhaps leave altogether. When the Drupal project was small if you did some heavy lifting, people noticed. Now our numbers have swelled beyond 1 million it is no wonder worthy effort goes relatively unnoticed.

Few of us are seeking the limelight but being noticed, that’s another matter. It’s not just recognition either. We should be mentoring emerging talent, supporting promising businesses and championing contributions from end users. But with such a vast and growing community, as Dries so eloquently put in his keynote, the status quo is unsustainable.

“Social capital and altruism is what we already do and I don’t think it is very scalable”

DriesNote

We should remember the material for the Dries keynote was crowd sourced and as he said we should “keep an open mind, don’t draw to conclusions”. Now is the time to debate openly, engage in constructive conversation and help invigorate Drupal.

What struck me at Amsterdam was that when the audience members having contributed to Drupal Core and contrib were invited to stand, surprisingly few remained seated. In many respects talking to DrupalCon attendees was preaching to the converted. The job is for us to reach the rest.

Understanding how our ecosystem works is a very important first step

I believe the path to a healthier community hinges on how we record contributions, not how we recognise them. Currently we track what individuals contribute and there is a fixation on code based contributions. How about documentation, organising camps, mentoring, community affairs, design and so many more valuable activitites. Introducing a way to differentiate between efforts made by individuals, agencies and end users and a move towards “Track all types of contributions” for me is a giant leap forwards. It couldn't happen sooner.

How we might use this new data is less important than the ability to "track how our community really works”. Most significantly it will bring to the surface things are not working. Who is funding what? Are we depending too heavily on small groups or certain individuals? Could specific areas do with more help? Achieving the tracking Dries would like, could happen within weeks. The Rankomatic? I like to think of it as Drupal Analytics, which is open for all to query. Expose the data like data.gov.uk and the community will surely hack insightful mashups and we will have something very empowering.

Is D.O ready for a reboot?

What word is conspicuously missing from the D.O homepage? Contribute, yup, seriously. How can we expect more people to do so if we don’t clearly signpost how? When Dries talked about “Free riders” I reflected on how long (years) it took me to find a niche in Drupal. I existed on the fringes under employed. It was only when I was noticed by Isabel Schultz during DrupalCon London that things really flew for me. We need more "talent scouts". Why is Get Involved buried in the footer and not somewhere more obvious? Are many “free riders” just looking for a way to engage?

Rewarding contributions through enhanced visibility, leading by example will certainly help. Encouraging companies who don’t contribute to start doing so via subtle design changes to user and marketplace profile pages has merit. Will invigorate the contributors themselves to even greater heights? That's certainly how I roll.

What Dries suggests is sweeping change. In this respect the cynic in me worries about the glacially slow speed of change on Drupal.org (D.O). 5 months ago I created an issue proposing the community spotlight feature on the homepage of D.O. It had unanimous support but nothing ever happened. Maybe I did something wrong? Perhaps I don't understand the process?

The good news is the Drupal Association has hired staff whose role is to deliver these kinds of enhancements. It also feels like a more measured and strategic approach is being taken. I hear talk of user personas. The concept designs by During his TED Talk Harish stated companies that will thrive in the future are those who "play their role in terms of serving the communities who actually sustain them”.

Drupal is a platform having a community that sustains many thousands of businesses. Drupal has strong evidence that Harish’s hypothesis is already true. Drupal shops, even end users who contribute to the project tend to be the ones enjoying the best outcomes. The problem is we operate in a bubble, where everyone is hooked on open source. Out in the real world people’s thinking is years behind and the principles of open source sound quite peculiar even alien. Those coming to use Drupal need introducing to the benefits of taking a benevolent role.

We know who Lullabot, Palantir, Phase2, Wunderkraut are. The various Drupal shops are contributing, often in bucket loads. What we actually need is more companies like that. Indeed we need more end users contributing back too. Like how Oxfam, The Whitehouse, Sony have been instrumental in developing significant contributions to the Contrib space. Again Dries’s advertising model could help to champion these organisations, communicating why they chose this path and the benefits they have reaped over time. It’s not about penalising “free riders” it is more about persuading them to engage with Drupal, to start contributing.

“If everybody that used the commons contributed a little bit things would be perfectly fine”

Dries talked about The Tragedy of the commons - overgrazing in Drupal translates not to the software rather the "Free riders" are depending too much on people’s generosity, personal time or indeed time which they should be running their business. I’ve seen situations where people have neglected their business for the benefit of Drupal. This goes way beyond the well known plight of Alex Pott, I assure you. I'm sure the few examples I have seen are a small fraction. Here everyone loses.

Time for change

As Dries said “An imperfect solution is better than no solution”. We’ve tried for some time to encourage those who sit on the sidelines to start contributing. Now is the time for a small revolution. Recording, recognising, rewarding code and non code contributors on D.O is simply a way of scaling how it has always been. And if that sounds too radical, how about we try it out on beta.drupal.org? We need to take a leap. After all, aren’t we the round pegs in the square holes? We are the ones who think different. It’s way overdue we took some of our own medicine.

Further information: Outlandish Josh's Thoughts on the DriesNote: Towards Drupal "Contribution Credits"Watch the Dries KeynotePhoto credit - Steffen Rühlmann
Categories: Elsewhere

PreviousNext: Architecting DrupalCI at DrupalCon Amsterdam

Planet Drupal - Tue, 14/10/2014 - 01:44

At the recent DrupalCon Amsterdam sprints something amazing happened, people from all corners of the globe assembled to sprint on DrupalCI. DrupalCI is an initiative born out of the requirement for new testbot infrastructure. Our goal is to implement a brand new Continuous Integration (CI) workflow that can not only be used for Drupal but anyone wishing to run a CI infrastructure / Automated tasks. Until this point we had only corresponded via a weekly hangout and IRC.

While this was keeping us on track with building out some of the components, the conference gave us an opportunity to sit down in the same room and perform an end-to-end architectural review to ensure we didn't have any gaps. A modular design approach has been used to ensure that many of the following components could be used as a standalone entity in any infrastructure.

Categories: Elsewhere

CiviCRM Blog: The Road to Drupal 8

Planet Drupal - Tue, 14/10/2014 - 00:42

As part of the Google Summer of Code, I began work on getting CiviCRM and the upcoming Drupal 8 working together nicely. I made an update about midway through and it's time for another update.

I had separated the project into a number of milestones. Phases 1, 2 and 3 dealt with varying aspects of the core CiviCRM module functionality. This work has largely been completed and there are pull requests pending into CiviCRM core, though the front end user experience is still a bit rough (for example, the CiviCRM menu bar doesn’t sit well alongside the Drupal menu).

The installation process is quite different with Drupal 8: civicrm now installs as if it were any other Drupal module — simply by clicking enable. It's no longer necessary to use the CiviCRM installer before enabling the module. This handles the most common use case where CiviCRM is installed in the same database as Drupal itself. Custom options can be configured by adding configuration settings to Drupal's settings.php. For example, if Drupal is configured with a second database named 'civicrm', it will install civicrm there instead.

The next big milestone was Views integration. This went ahead quite smoothly, and I managed to reduce the code count from approximately 15,000 lines of code to under 2,000 by automating a lot of the discovery of the CiviCRM database. In general, when new fields appear in CiviCRM it should only be necessary to explicitly declare how the field is related to other data types (eg. as joins or relationships).

One quick nice-to-have was integrating Drupal's entity reference field with CiviCRM Contacts — mostly by creating a bare-bones Drupal entity. A similar technique may prove to work with Drupal's Rules module, but unfortunately time did not allow this yet.

Finally, for the first time, the module comes with a few simple integration tests (using Drupal's SimpleTest module) which will hopefully become more fully fleshed out with time. With a judicious set of tests, this should allow a more stable experience as future versions of CiviCRM are released.

Drupal 8 has finally released its first beta just over a week ago. Now that it has stabilised somewhat, we need to update the CiviCRM module to resolve breaking changes and we can then release an alpha version that the community can test, experiment with and help get into a stable state.

Drupal 8 repo: https://github.com/torrance/civicrm-drupal

Categories: Elsewhere

Philipp Kern: pbuilder and pam_tmpdir

Planet Debian - Tue, 14/10/2014 - 00:28
It turns out that my recent woes with pbuilder were all due to libpam-tmpdir being installed (at least two old bug reports exist about this issue: #576425 and #725434). I rather like my private temporary directory that cannot be accessed by other (potential) users on the same system. Previously I used a hook to fix this up by ensuring that the directory actually exists in the chroot, but somehow that recently broke.

A rather crude but working solution seems to be "session required pam_env.so user_readenv=1" in /etc/pam.d/sudo and "TMPDIR=/tmp" in /root/.pam_environment. One could probably skip pam_tmpdir.so for root, but I did not want to start fighting with pam-auth-update as this is in /etc/pam.d/common-session*.
Categories: Elsewhere

Konstantinos Margaritis: SIMD optimizations, cont.

Planet Debian - Mon, 13/10/2014 - 21:19

A friend of mine told me that I should advertise my passion and know-how about SIMD more, and I decided to follow his advice. Though I am terrible at marketing and even more at personal marketing, I've made an attempt to do just that, advertise the fact that I'm offering SIMD Optimization Services (with emphasis on PowerPC AltiVec/VMX/VSX, and ARM NEON, but I'm ok with SSE as well, the logic is pretty much the same, though the difference(s) are in the details). For this reason I'm offering a free evaluation of your performance critical code (open/closed, able to sign NDAs if needed) to let you know if it's worth optimizing it, what kind of a performance gain you would get and how much it would cost you to get that result.
You can read more here.

Categories: Elsewhere

Drupal Easy: Professionally-Trained, Community-Oriented, Drupal Career Online Students Ready for Work

Planet Drupal - Mon, 13/10/2014 - 20:10

The fifth class of our Drupal career training program is just about at the halfway mark, and our students are eager to put their new skills to work. The six Drupal Career Online students will be ready for junior-developer-level work in mid-November, and we're looking for forward-thinking organizations willing to help our graduates on the next leg of their Drupal career journey.

As we've done for the past five classes, (we've had more than 60 graduates so far) we're looking to make introductions between our upcoming graduates and organizations looking for people with Drupal site-building and development skills. Our Work Experience Drupal (WE Drupal) program is designed to provide students with valuble experience in internship-type settings. WE Drupal host companies are asked to make a 6-10 week commitment to one or more of our students, provide them with guidance, mentoring, and the professional experience that is so difficult to come by for new Drupal site builders and developers. In return, you get the efforts of a well-prepared, super-eager Drupal novice to help you lighten the task-load for your staff.

-->

read more

Categories: Elsewhere

John Goerzen: Update on the systemd issue

Planet Debian - Mon, 13/10/2014 - 19:46

The other day, I wrote about my poor first impressions of systemd in jessie. Here’s an update.

I’d like to start with the things that are good. I found the systemd community to be one of the most helpful in Debian, and #debian-systemd IRC channel to be especially helpful. I was in there for quite some time yesterday, and appreciated the help from many people, especially Michael. This is a nontechnical factor, but is extremely important; this has significantly allayed my concerns about systemd right there.

There are things about the systemd design that impress. The dependency system and configuration system is a lot more flexible than sysvinit. It is also a lot more complicated, and difficult to figure out what’s happening. I am unconvinced of the utility of parallelization of boot to begin with; I rarely reboot any of my Linux systems, desktops or servers, and it seems to introduce needless complexity.

Anyhow, on to the filesystem problem, and a bit of a background. My laptop runs ZFS, which is somewhat similar to btrfs in that it’s a volume manager (like LVM), RAID manager (like md), and filesystem in one. My system runs LVM, and inside LVM, I have two ZFS “pools” (volume groups): one, called rpool, that is unencrypted and holds mainly the operating system; and the other, called crypt, that is stacked atop LUKS. ZFS on Linux doesn’t yet have built-in crypto, which is why LVM is even in the picture here (to separate out the SSD at a level above ZFS to permit parts of it to be encrypted). This is a bit of an antiquated setup for me; as more systems have AES-NI, I’m going to everything except /boot being encrypted.

Anyhow, inside rpool is the / filesystem, /var, and /usr. Inside /crypt is /tmp and /home.

Initially, I tried to just boot it, knowing that systemd is supposed to work with LSB init scripts, and ZFS has init scripts with carefully-planned dependencies. This was evidently not working, perhaps because /lib/systemd/systemd/ It turns out that systemd has a few assumptions that turn out to be less true with ZFS than otherwise. ZFS filesystems are normally not mounted via /etc/fstab; a ZFS pool has internal properties about which dataset gets mounted where (similar to LVM’s actions after a vgscan and vgchange -ay). Even though there are ordering constraints in the units, systemd is writing files to /var before /var gets mounted, resulting in the mount failing (unlike ext4, ZFS by default will reject an attempt to mount over a non-empty directory). Partly this due to the debian-fixup.service, and partly it is due to systemd reacting to udev items like backlight.

This problem was eventually worked around by doing zfs set mountpoint=legacy rpool/var, and then adding a line to fstab (“rpool/var /var zfs defaults 0 2″) for /var and its descendent filesystems.

This left the problem of /tmp; again, it wasn’t getting mounted soon enough. In this case, it required crypttab to be processed first, and there seem to be a lot of bugs in the crypttab processing in systemd (more on that below). I eventually worked around that by adding After=cryptsetup.target to the zfs-import-cache.service file. For /tmp, it did NOT work to put it in /etc/fstab, because then it tried to mount it before starting cryptsetup for some reason. It probably didn’t help that the system’s cryptdisks.service is a symlink to /dev/null, a fact I didn’t realize until after a lot of needless reboots.

Anyhow, one thing I stumbled across was poor console control with systemd. On numerous occasions, I had things like two cryptsetup processes trying to read a password, plus an emergency mode console trying to do so. I had this memorable line of text at one point:

(or type Control-D to continue): Please enter passphrase for disk athena-crypttank (crypt)! [ OK ] Stopped Emergency Shell.

And here we venture into unsatisfying territory with systemd. One answer to this in IRC was to install plymouth, which apparently serializes console I/O. However, plymouth is “an attractive boot animation in place of the text messages that normally get shown.” I don’t want an “attractive boot animation”. Nevertheless, neither systemd-sysv nor cryptsetup depends on plymouth, so by default, the prompt for a password at boot is obscured by various other text.

Worse, plymouth doesn’t support serial consoles, so at the moment booting a system that uses LUKS with systemd over a serial console is a matter of blind luck of typing the right password at the right time.

In the end, though, the system booted and after a few more tweaks, the backlight buttons do their thing again. Whew!

Categories: Elsewhere

Steve McIntyre: Successful Summer of Code in Linaro

Planet Debian - Mon, 13/10/2014 - 19:06

It's past time I wrote about how Linaro's students fared in this year's Google Summer of Code. You might remember me posting earlier in the year when we welcomed our students. We started with 3 student projects at the beginning of the summer. One of the students unfortunately didn't work out, but the other two were hugely successful.

Gaurav Minocha was a graduate student at the University of British Columbia, Vancouver, Canada. He worked on Linux Flattened Device Tree Self-checking, mentored by Grant Likely from Linaro's Office of the CTO. Gaurav achieved all of his project's goals, and he was invited to Linaro's recent Linaro Connect USAConnect conference in California to meet people and and talk about his project. He and Grant presented a session on their work; it was filmed, and video is online. Grant said he was very happy with Gaurav's "strong, solid performance" during the project.

Varad Gautam was a student at Birla Institute of Technology and Science, Pilani, India. He succeeded in porting UEFI to the BeagleBone Black. Leif Lindholm from the Linaro Enterprise Group was his mentor for the summer. At the end of the summer, Varad delivered a UEFI port ready for booting Linux and his code was included in Linaro's September UEFI release. Leif said that he was "very pleased with Varad's self sufficiency and ability to pick up an entirely new software project very quickly". We were hoping to invite Gaurad to Connect in California also, but travel document delays got in the way. With luck we'll see him at the next Connect in Hong Kong in February 2015.

Well done, guys! It was great to work with these young developers for the summer, and we wish them lots more success in their future endeavours.

Google have also just confirmed that they will be running the Summer of Code program again in 2015. I'm hoping that Linaro will be accepted again next year as a mentoring organisation. I'll post more about that early next year.

Categories: Elsewhere

Stanford Web Services Blog: Module of the Day: Stanford MetaTag NoBots - Hide your site from search engines!

Planet Drupal - Mon, 13/10/2014 - 18:00

When we launch a site at Stanford Web Services, we open the doors and roll out the red carpet for the search engines to index the site. However, before launch we like to keep the content under wraps and ask the search engines not to index the site. To do this, we use a module called Stanford MetaTag NoBots.

Categories: Elsewhere

Acquia: How to setup auto-translation of nodes using Rules and Tmgmt

Planet Drupal - Mon, 13/10/2014 - 15:17

This how-to guide describes how to set up automatic machine-based translations for content on a Drupal site. Whenever a node is created, translation jobs will be created for every language specified, and depending on how you set up your translator, you should be able to completely automate the translation process. For this project we are using SDL, but you should be able to use other translators.

1. Download and enable the translation management modules. This can be done through Drush or through the modules interface in drupal.

Categories: Elsewhere

Pages

Subscribe to jfhovinne aggregator