Drupal Association News: Drupal.org User Research Results: User Personas

Planet Drupal - Sun, 28/09/2014 - 19:16

Back in May the Drupal Assocation with the help of the Drupal.org Content Working Group kicked off user research project for Drupal.org. Our goal was to understand who are our users, how do they use the website, what is their ideal user experience. We wanted to understand what are the main segments of our audience and define their priority so we could focus our development efforts. This user research is a first step on the path to a complete Drupal.org redesign. For the redesign to be successful, we wanted to have a complete picture of everyone who uses Drupal.org.

This user research was conducted with the help of Whitney Hess, user experience coach. We are thankful to Whitney for her time and experience she shared with us. It was a great pleasure working with her.

Now to the results we want to share.

The Process

We kicked of the project with a workshop during DrupalCon Amsterdam. Participants included members of the Drupal.org Working Groups (Content, Software and Infrastructure), members of the DSWG Leadership teams (Developer and Community Tools), and the Drupal Association staff members.

Findings from this first workshop you can find in the blog post.

Our next step was user interviews. During DrupalCon Austin and in the weeks following the event we’ve conducted 30 user interviews with various people: new to Drupal, long-term community members, ex-Drupalistas; developers, site builders, designers, content strategists, PMs, etc.; located in North and South America and Europe. Interviews were conducted by Whitney Hess, Joshua Mitchell, Roy Scholten and Tatiana Ugriumova (myself).

Once we had completed several interviews, Whitney and I started synthesizing them and developing personas.

In the middle of July, during our annual all-staff onsite week at the Drupal Association, we had the second workshop of this project. Drupal Association staff and Whitney Hess took a look at the work-in-progress personas and our original objectives, to make sure they still made sense and to prepare them for review by the working groups and board.

The Results

Objectives for a new Drupal.org:

  1. Be the home of the Drupal community. Central source of relevant info/answers, collaboration, education and talent.
  2. Provide learnable, efficient tools to help coordinate the advancement of Drupal ecosystem.
  3. Encourage people to develop themselves, their Drupal proficiency, their careers & build human connections over time.

Based on the insights we learned in our research we decided to center our personas around competency in Drupal and the Drupal ecosystem.

We modified Dreyfus's model of skill acquisition as a basis for our persona structure. This model suggests 5 levels of competency, which for our purposes we call:

  • Newcomer
  • Learner
  • Skilled
  • Expert
  • Master

For further clarity and focus, we determined primary and secondary personas. Primary personas are who we design for first. Each primary persona requires their own set of features and content, and the needs of a primary persona will not be met if we do not design for them explicitly. The needs of secondary personas can mostly be met by focusing on the primary personas. However, there are a few needs specific to them that we will need to accommodate for.

Based on the goal of growing the community and the usage of Drupal, we determined the primary personas to be: Learner and Skilled. Our secondary persona is Expert. Newcomer and Master are our tertiary personas.

This is a very short summary of the research results. More information and an actual personas can be found in the full report below.

Get the full Drupal.org User Personas report: Google Doc | Pdf

We'd like again to say thanks to Whitney for helping us during this process.

As I mentioned above, user research is the first step of a greater project. Our next step is content strategy work for Drupal.org. We'll be posting RFP for it shortly, watch this blog.

Categories: Elsewhere

Netstudio.gr Blog: You own an e-shop? Automate your shipments

Planet Drupal - Sun, 28/09/2014 - 15:45

The problem

If you own an e-shop, you know the process:: Orders come, you write the shipping vouchers by hand or through the software that your courier service has given to you, you print them, and you stick them on the packages. This has to be done quickly and with no mistakes.

The solution

Recently, we developed for a client the automation of this process. When an order is received, it automatically gets assigned a voucher ID. The client receives an email, SMS or both with a link to the tracking page. This page resides inside the e-shop, so the prestige of the site is increased. Alternatively, before the voucher creation, the shop owner checks the address for possible extra charges and informs the client. This automation reduces the time it needs to prepare the shipments to one third and keeps the client happy as he may check his shipment progress through the tracking page without having to call the e-shop. Finally, mistakes are gone, as the voucher gets printed with the data that the client has entered himself. For more info call us at +30 2108004447 or fill in the contact form.

Categories: Elsewhere

Verbosity: Migrating Drupal 8 in Europe

Planet Drupal - Sun, 28/09/2014 - 13:40

This week we're in Europe for DrupalCon Amsterdam! This is starting to feel suspiciously close to a beta so it is time to dive into Migrate again so you can start working on your new sites with real-world data. Let's begin!

What's up with migrate?

Migrate in the Drupal context means running a migration from the new Drupal 8 site and pulling data from an existing site. It replaces the old upgrade-in-place system that was used in prior versoins of Drupal. So do a fresh install of Drupal 8 and have an old Drupal 6 site on the same host. After you've logged into Drupal 8 you can connect to your Drupal 6 site.

How is Drupal 8 migrate different from Drupal 7 migrate (and migrate_d2d)?

In the older versions of Migrate the process invovled defining your field mappings and manually creating the new content types in the D7 system. This is no longer necessary.

Drupal 8 migrations will automatically create the needed content types and establish the mappings between the old and new fields by default. Therefore much less configuration is needed.

For times when it is necessary to customizae your migration you can use the included hooks or you can use the configuration schema to use the included plugins with your custom data set. The hooks go further than in D7, allowing you to alter data in the prepareRow stage without having to create a custom migration to deal with the data.

Migrate from Drupal 6? What about from Drupal 7?

Migrate frees us from the need to update each sequential version. You can leafrog versions! With Drupal 6 being close to end-of-life it is important to have a pathway for these users. For this reason, the D6 to D8 path was built first.

For Drupal 7: soon. This is now in-progress now that we are finalizing the Drupal 6 code.

  • Drush! The UI is still in a contrib sandbox, so for now you must use Drush. The latest version - from Github.
  • Composer. It is needed to install Drush. Go into the drush folder and run "composer install". Already installed? "composer update"
  • D6 Database. Have it on the same host as the new Drupal 8 install.
  • D6 Files. Probably a good idea to be on the same host. Can be file path or http(s) address.
  • D8 Database. A new, empty database. Use the old Creating the database howto if new to this.
  • D8 Files. Check out the git repo... unless of course a beta becomes available. Then use that.
Using Drupal Tools

If you do not currently have a Drupal 8 install one route to get there is to use the Drupal Tools for your platform. It includes all the software you need and the correct versions. It is available for Linux, Mac, and Windows.

If you install the Drupal Tools package it is not necessary to install Git, Drush, or Composer.

The installation should be current as it includes a version of Drush which must be up-to-date. So if you had installed Tools before, double check the version if you have any trouble.

Install Drupal8

Using the database credentials you created, install your new Drupal 8 site.

If you need to rebuild your site remember to delete your ENTIRE files folder and old settings.php. Reinstalling without doing this step will cause problems.

Install Composer / Drush

Make sure you have run the Composer installer. You should be able to type which composer and get a result in your terminal.

Check out the latest Drush. Go into the Drush folder and run composer install.

Next time you git pull (or git fetch) in the Drush folder, make sure to run composer update.

Find an isuse to test
  • Go to the Drupal project issue queue and filter down by Component: migration system, Version: 8.x.
  • Pick an issue that is not already fixed.
  • If you are sprinting with lots of people, pick something further down the list so you are not working on the same thing as someone else.
  • Read the posts. If it is easy enough you think you can handle it, post a comment to say you are doing some work on it.
  • Post the results of your tests.
Time to Migrate

Put the manifest.yml file with the migrations you wish to run in your D8 site root. Then go there on the command line and run the following command, using your D6 database credentials.

You can install Drupal 8 at this stage. If you do, be sure to enable all of the necessary modules. For example, if you use the Book module, it is not enabled by default, so you should enable it now or your book nodes will simply become a regular content type.

When Drupal 8 is no longer in beta the manifest.yml file will not be necessary unless you are doing some custom work. In most cases all that will be necessary is to put in the database credentials and the system will run all the migrations that the system knows about by default.

You will find a manifest.yml file attached to many Migrate issues that will enable you to begin the migration. Here is a sample of what I am using... I have added together many different issues and I run them all at the same time:

# user
- d6_user
- d6_user_profile_field
- d6_user_profile_field_instance
- d6_user_profile_entity_display
- d6_user_profile_entity_form_display
- d6_profile_values:user
- d6_filter_format
- d6_user_role
- d6_user_picture_entity_display
- d6_user_picture_entity_form_display
- d6_user_picture_file
- d6_user_picture_field
- d6_user_picture_field_instance
# taxonomy
- d6_taxonomy_vocabulary
- d6_taxonomy_settings
- d6_taxonomy_term
# nodes
- d6_node
- d6_node_revision
- d6_node_type
- d6_view_modes
- d6_filter_format
- d6_field_instance_per_form_display
- d6_field_instance_widget_settings
- d6_field_formatter_settings
- d6_field_instance
- d6_field
- d6_field_settings
- d6_node_settings
- d6_cck_field_values:*
- d6_cck_field_revision:*
# taxonomy fields
- d6_term_node_revision
- d6_term_node
- d6_vocabulary_entity_display
- d6_vocabulary_entity_form_display
- d6_vocabulary_field_instance
- d6_vocabulary_field
# blocks
- d6_block
- d6_menu
# custom blocks
- d6_custom_block
- d6_filter_format
# book
- d6_book
- d6_book_settings

Now that you have created a manifest.yml file in the Drupal 8 root directory you can run the migration.

drush migrate-manifest --legacy-db-url=mysql://d6user:d6pass@localhost/d6 manifest.yml

If you are using Drupal Tools (Acquia Dev Desktop), and if you have also put your D6 site into Dev Desktop, you will need to specify the port number. You can find your database settings by creating an AcquiaDevDesktop terminal and typing drush status to get the exact settings for your D6 site. The result should look something like this:

drush migrate-manifest --legacy-db-url=mysql://drupaluser:@ manifest.yml

Note that the database user for D6 is called "drupaluser" and it uses the local IP address and port number rather than the host name. Again, run drush status if you are having trouble connecting to verify these values.


After you have run the migration check your work. Did things do what you expected? Post the results of your findings to the issue queue you of the item you were working on.

Was the result successful? If so, post the result.

Did something fail? Post the result.

Post your results! Don't be afraid to comment on Drupal.org. If you provide examples of your tests you will help the migration path improve.

Rolling back (starting over)
  • It is possible to re-run the migration. This can be helpful if you forgot to run a component, or if you have new items in the source site that you would like to add to the Drpual8 site.
  • To completely "roll-back" you really need to reinstall Drupal 8. To do this you must do three things: (1) empty your database, (2) delete settings.php, (3) remove the files folder completely.
Going further Category: DrupalDrupal 8Drupal Planet
Categories: Elsewhere

Deeson: Data visualisation of who's making Drupal 8

Planet Drupal - Sun, 28/09/2014 - 10:15

I've been having fun for the last few weeks building an interactive data visualisation tool, The Drupal8r, to show how the Drupal community is coming together to develop the next release, Drupal 8.

What is Drupal?

Drupal is a free open-source web development platform which is designed to support online content, community and commerce. We've been using it since 2007.

Because it is open-source and built, developed and maintained by a huge global community who constantly update it as technology evolves.

Drupal 8, the latest release, is coming soon.

There are more than half a million people in the Drupal community, all working to make it the best platform around and usually in their spare time. 

We wanted to celebrate the community's collaboration and show who is making some of the biggest contributions and commitments in getting Drupal 8 ready for launch. And so the idea for The Drupal8r was born.

The Drupal community

There are hundreds of people in the Drupal community who write the platform's core code as their community contribution (called the contributors). This code is tested and reviewed by a handful of people who then commit it to the core (known as the committers).

D3 data visualisation

I used the d3.js JavaScript library for manipulating documents based on data and Drupal git-log data to build a chart which displays the names of code contributors and committers with their corresponding Drupal 8 modules. At the moment you can view data from the latest 5000 commits to Drupal 8.

Have a go!

Have a play with The Drupal8r, our fun and interactive data visualisation tool to see who has committed and contributed to modules for the next release of Drupal 8. 

Come and play with The Drupal8r now.

For mobile users

The text is small for mobile users and our human eyes haven't quite caught up with technology yet. So, have a look at this short video to see how it works instead.

The Drupal8r data visualisation tool Keep in mind

When using it for just committers or contributors, the width of the connection represents the number of commits. When you are viewing both,  the width of the connection has no meaning. 

The Drupal8r can only show a certain amount of items on the page, by default The Drupal8r shows 100 items. It shows all the modules and committers, but the number of contributors are capped and grouped together under 'other'.

Four things The Drupal8r shows
  1. A lot of people contribute - getting Drupal 8 ready is a massive team effort. It isn't just one person contributing to each module, but many people. Thanks everyone!
  2. In terms of pure numbers, Alex Pott, Webchick, and Nathaniel Catchpole have been the most prolific contributors to the Drupal 8 project. Impressive stuff.
  3. But the number of commits isn't necessarily a reasonable measure of an individual's contribution. Dries Buytaert and Jennifer Hodgdon may have been focusing on the bigger issues. The core team have reviewed and tested more than 5000 bug patches and bring us close to Drupal 8's launch - what a feat.
  4. For those in the Drupal community, see if you can spot the person who is in there twice under different names...
Coming soon...

Next month I'm going to write another blog post over here on Labs about how I made The Drupal8r and the technologies used.

I'll be enhancing The Drupal8r over the next few months. I'm going to add an option to download more data to include in the chart and a contributor filter, so you'll be able to link to a particular contributor's commits.

Don't miss out on our development of the chart, so sign up to our newsletter below to keep in the loop.

Categories: Elsewhere

Unimity Solutions Drupal Blog: DrupalCon Amsterdam - Diary - Saturday 27th Sep

Planet Drupal - Sun, 28/09/2014 - 08:53

I am proud to receive a Grant from the Drupal Association to attend DrupalCon Amsterdam. As a note of thanking the community, I started Day 1 here, Saturday, 27th September Volunteering to help Tote Bag and Badge Stuffing activity at Onyx Lounge at Amsterdam Rai. 

Categories: Elsewhere

Benjamin Mako Hill: Community Data Science Workshops Post-Mortem

Planet Debian - Sun, 28/09/2014 - 07:02

Earlier this year, I helped plan and run the Community Data Science Workshops: a series of three (and a half) day-long workshops designed to help people learn basic programming and tools for data science tools in order to ask and answer questions about online communities like Wikipedia and Twitter. You can read our initial announcement for more about the vision.

The workshops were organized by myself, Jonathan Morgan from the Wikimedia Foundation, long-time Software Carpentry teacher Tommy Guy, and a group of 15 volunteer “mentors” who taught project-based afternoon sessions and worked one-on-one with more than 50 participants. With overwhelming interest, we were ultimately constrained by the number of mentors who volunteered. Unfortunately, this meant that we had to turn away most of the people who applied. Although it was not emphasized in recruiting or used as a selection criteria, a majority of the participants were women.

The workshops were all free of charge and sponsored by the UW Department of Communication, who provided space, and the eScience Institute, who provided food.

The curriculum for all four session session is online:

The workshops were designed for people with no previous programming experience. Although most our participants were from the University of Washington, we had non-UW participants from as far away as Vancouver, BC.

Feedback we collected suggests that the sessions were a huge success, that participants learned enormously, and that the workshops filled a real need in the Seattle community. Between workshops, participants organized meet-ups to practice their programming skills.

Most excitingly, just as we based our curriculum for the first session on the Boston Python Workshop’s, others have been building off our curriculum. Elana Hashman, who was a mentor at the CDSW, is coordinating a set of Python Workshops for Beginners with a group at the University of Waterloo and with sponsorship from the Python Software Foundation using curriculum based on ours. I also know of two university classes that are tentatively being planned around the curriculum.

Because a growing number of groups have been contacting us about running their own events based on the CDSW — and because we are currently making plans to run another round of workshops in Seattle late this fall — I coordinated with a number of other mentors to go over participant feedback and to put together a long write-up of our reflections in the form of a post-mortem. Although our emphasis is on things we might do differently, we provide a broad range of information that might be useful to people running a CDSW (e.g., our budget). Please let me know if you are planning to run an event so we can coordinate going forward.

Categories: Elsewhere

DebConf team: Wrapping up DebConf14 (Posted by Paul Wise, Donald Norwood)

Planet Debian - Sat, 27/09/2014 - 21:40

The annual Debian developer meeting took place in Portland, Oregon, 23 to 31 August 2014. DebConf14 attendees participated in talks, discussions, workshops and programming sessions. Video teams captured a lot of the main talks and discussions for streaming for interactive attendees and for the Debian video archive.

Between the video, presentations, and handouts the coverage came from the attendees in blogs, posts, and project updates. We’ve gathered a few articles for your reading pleasure:

Gregor Herrmann and a few members of the Debian Perl group had an informal unofficial pkg-perl micro-sprint and were very productive.

Vincent Sanders shared an inspired gift in the form of a plaque given to Russ Allbery in thanks for his tireless work of keeping sanity in the Debian mailing lists. Pictures of the plaque and design scheme are linked in the post. Vincent also shared his experiences of the conference and hopes the organisers have recovered.

Noah Meyerhans’ adventuring to Debian by train, (Inter)netted some interesting IPv6 data for future road and railwarriors.

Hideki Yamane sent a gentle reminder for English speakers to speak more slowly.

Daniel Pocock posted of GSoC talks at DebConf14, highlights include the Java Project Dependency Builder and the WebRTC JSCommunicator.

Thomas Goirand gives us some insight into a working task list of accomplishments and projects he was able to complete at DebConf14, from the OpenStack discussion to tasksel talks, and completion of some things started last year at DebConf13.

Antonio Terceiro blogged about debci and the Debian Continuous Integration project, Ruby, Redmine, and Noosfero. His post also shares the atmosphere of being able to interact directly with peers once a year.

Stefano Zacchiroli blogged about a talk he did on debsources which now has its own HACKING file.

Juliana Louback penned: DebConf 2014 and How I Became a Debian Contributor.

Elizabeth Krumbach Joseph’s in-depth summary of DebConf14 is a great read. She discussed Debian Validation & CI, debci and the Continuous Integration project, Automated Validation in Debian using LAVA, and Outsourcing webapp maintenance.

Lucas Nussbaum by way of a blog post releases the very first version of Debian Trivia modelled after the TCP/IP Drinking Game.

François Marier’s shares additional information and further discussion on Outsourcing your webapp maintenance to Debian.

Joachim Breitner gave a talk on Haskell and Debian, created a new tool for binNMUs for Haskell packages which runs via cron job. The output is available for Haskell and for OCaml, and he still had a small amount of time to go dancing.

Jaldhar Harshad Vyas was not able to attend DebConf this year, but he did tune in to the videos made available by the video team and gives an insightful viewpoint to what was being seen.

Jérémy Bobbio posted about Reproducible builds in Debian in his recap of DebConf14. One of the topics at hand involved defining a canonical path where packages must be built and a BOF discussion on reproducible builds from where the conversation moved to discussions in both Octave and Groff. New helpers dh_fixmtimes and dh_genbuildinfo were added to BTS. The .buildinfo format has been specified on the wiki and reviewed. Lots of work is being done in the project, interested parties can help with the TODO list or join the new IRC channel #debian-reproducible on irc.debian.org.

Steve McIntyre posted a Summary from the d-i / debian-cd BoF at DC14, with some of the session video available online. Current jessie D-I needs some help with the testing on less common architectures and languages, and release scheduling could be improved. Future plans: Switching to a GUI by default for jessie, a default desktop and desktop choice, artwork, bug fixes and new architecture support. debian-cd: Things are working well. Improvement discussions are on selecting which images to make I.E. netinst, DVD, et al., debian-cd in progress with http download support, Regular live test builds, Other discussions and questions revolve around which ARM platforms to support, specially-designed images, multi-arch CDs, and cloud-init based images. There is also a call for help as the team needs help with testing, bug-handling, and translations.

Holger Levsen reports on feedback about the feedback from his LTS talk at DebConf14. LTS has been perceived well, fits a demand, and people are expecting it to continue; however, this is not without a few issues as Holger explains in greater detail the lacking gatekeeper mechanisms, and how contributions are needed from finance to uploads. In other news the security-tracker is now fixed to know about old stable. Time is short for that fix as once jessie is released the tracker will need to support stable, oldstable which will be wheezy, and oldoldstable.

Jonathan McDowell’s summary of DebConf14 includes a fair perspective of the host city and the benefits of planning of a good DebConf14 location. He also talks about the need for facetime in the Debian project as it correlates with and improves everyone’s ability to work together. DebConf14 also provided the chance to set up a hard time frame for removing older 1024 bit keys from Debian keyrings.

Steve McIntyre posted a Summary from the “State of the ARM” BoF at DebConf14 with updates on the 3 current ports armel, armhf and arm64. armel which targets the ARM EABI soft-float ARMv4t processor may eventually be going away, while armhf which targets the ARM EABI hard-float ARMv7 is doing well as the cross-distro standard. Debian is has moved to a single armmp kernel flavour using Device Tree Blobs and should be able to run on a large range of ARMv7 hardware. The arm64 port recently entered the main archive and it is hoped to release with jessie with 2 official builds hosted at ARM. There is talk of laptop development with an arm64 CPU. Buildds and hardware are mentioned with acknowledgements for donated new machines, Banana Pi boards, and software by way of ARM’s DS-5 Development Studio - free for all Debian Developers. Help is needed! Join #debian-arm on irc.debian.org and/or the debian-arm mailing list. There is an upcoming Mini-DebConf in November 2014 hosted by ARM in Cambridge, UK.

Tianon Gravi posted about the atmosphere and contrast between an average conference and a DebConf.

Joseph Bisch posted about meeting his GSOC mentors, attending and contributing to a keysigning event and did some work on debmetrics which is powering metrics.debian.net. Debmetrics provides a uniform interface for adding, updating, and viewing various metrics concerning Debian.

Harlan Lieberman-Berg’s DebConf Retrospective shared the feel of DebConf, and detailed some of the work on debugging a build failure, work with the pkg-perl team on a few uploads, and work on a javascript slowdown issue on codeeditor.

Ana Guerrero López reflected on Ten years contributing to Debian.

Categories: Elsewhere

Ritesh Raj Sarraf: Laptop Mode Tools 1.66

Planet Debian - Sat, 27/09/2014 - 11:09

I am pleased to announce the release of Laptop Mode Tools at version 1.66.

This release fixes an important bug in the way Laptop Mode Tools is invoked. Users, now when disable it in the config file, the tool will be disabled. Thanks to bendlas@github for narrowing it down. The GUI configuration tool has been improved, thanks to Juan. And there is a new power saving module for users with ATI Radeon cards. Thanks to M. Ziebell for submitting the patch.

Laptop Mode Tools development can be tracked @ GitHub

AddThis:  Categories: Keywords: 
Categories: Elsewhere

Niels Thykier: Lintian – Upcoming API making it easier to write correct and safe code

Planet Debian - Sat, 27/09/2014 - 09:08

The upcoming version of Lintian will feature a new set of API that attempts to promote safer code. It is hardly a “ground-breaking discovery”, just a much needed feature.

The primary reason for this API is that writing safe and correct code is simply too complicated that people get it wrong (including yours truly on occasion).   The second reason is that I feel it is a waste having to repeat myself when reviewing patches for Lintian.

Fortunately, the kind of issues this kind of mistake creates are usually minor information leaks, often with no chance of exploiting it remotely without the owner reviewing the output first[0].

Part of the complexity of writing correct code originates from the fact that Lintian must assume Debian packages to be hostile until otherwise proven[1]. Consider a simplified case where we want to read a file (e.g. the copyright file):

package Lintian::cpy_check; use strict; use warnings; use autodie; sub run { my ($pkg, undef, $info) = @_; my $filename = "usr/share/doc/$pkg/copyright"; # BAD: This is an example of doing it wrong open(my $fd, '<', $info->unpacked($filename)); ...; close($fd); return; }

This has two trivial vulnerabilities[2].

  1. Any part of the path (usr,usr/share, …) can be asymlink to “somewhere else” like /
    1. Problem: Access to potentially any file on the system with the credentials of the user running Lintian.  But even then, Lintian generally never write to those files and the user has to (usually manually) disclose the report before any information leak can be completed.
  2. The target path can point to a non-file.
    1. Problem: Minor inconvenience by DoS of Lintian.  Examples include a named pipe, where Lintian will get stuck until a signal kills it.

Of course, we can do this right[3]:

package Lintian::cpy_check; use strict; use warnings; use autodie; use Lintian::Util qw(is_ancestor_of); sub run { my ($pkg, undef, $info) = @_; my $filename = "usr/share/doc/$pkg/copyright"; my $root = $info->unpacked my $path = $info->unpacked($filename); if ( -f $path and is_ancestor_of($root, $path)) { open(my $fd, '<', $path); ...; close($fd); } return; }

Where “is_ancestor_of” is the only available utility to assist you currently.  It hides away some 10-12 lines of code to resolve the two paths and correctly asserting that $path is (an ancestor of) $root.  Prior to Lintian 2.5.12, you would have to do that ancestor check by hand in each and every check[4].

In the new version, the correct code would look something like this:

package Lintian::cpy_check; use strict; use warnings; use autodie; sub run { my ($pkg, undef, $info) = @_; my $filename = "usr/share/doc/$pkg/copyright"; my $path = $info->index_resolved_path($filename); if ($path and $path->is_open_ok) { my $fd = $path->open; ...; close($fd); } return; }

Now, you may wonder how that promotes safer code.  At first glance, the checking code is not a lot simpler than the previous “correct” example.  However, the new code has the advantage of being safer even if you forget the checks.  The reasons are:

  1. The return value is entirely based on the “file index” of the package (think: tar vtf data.tar.gz).  At no point does it use the file system to resolve the path.  Whether your malicious package trigger an undef warning based on the return value of index_resolved_index leaks nothing about the host machine.
    1. However, it does take safe symlinks into account and resolves them for you.  If you ask for ‘foo/bar’ and ‘foo’ is a symlink to ‘baz’ and ‘baz/bar’ exists in the package, you will get ‘baz/bar’.  If ‘baz/bar’ happens to be a symlink, then it is resolved as well.
    2. Bonus: You are much more likely to trigger the undef warning during regular testing, since it also happens if the file is simply missing.
  2. If you attempt to call “$path->open” without calling “$path->is_open_ok” first, Lintian can now validate the call for you and stop it on unsafe actions.

It also has the advantage of centralising the code for asserting safe access, so bugs in it only needs to be fixed in one place.  Of course, it is still possible to write unsafe code.  But at least, the new API is safer by default and (hopefully) more convenient to use.


[0] Lintian.debian.org being the primary exception here.

[1] This is in contrast to e.g. piuparts, which very much trusts its input packages by handing the package root access (albeit chroot’ed, but still).

[2] And also a bug.  Not all binary packages have a copyright – instead some while have a symlink to another package.

[3] The code is hand-typed into the blog without prior testing (not even compile testing it).  The code may be subject to typos, brown-paper-bag bugs etc. which are all disclaimed (of course).

[4] Fun fact, our documented example for doing it “correctly” prior to implementing is_ancestor_of was in fact not correct.  It used the root path in a regex (without quoting the root path) – fortunately, it just broke lintian when your TMPDIR / LINTIAN_LAB contained certain regex meta-characters (which is pretty rare).

Categories: Elsewhere

Richard Hartmann: Release Critical Bug report for Week 39

Planet Debian - Fri, 26/09/2014 - 21:21

The UDD bugs interface currently knows about the following release critical bugs:

  • In Total: 1393
    • Affecting Jessie: 408 That's the number we need to get down to zero before the release. They can be split in two big categories:
      • Affecting Jessie and unstable: 360 Those need someone to find a fix, or to finish the work to upload a fix to unstable:
        • 50 bugs are tagged 'patch'. Please help by reviewing the patches, and (if you are a DD) by uploading them.
        • 20 bugs are marked as done, but still affect unstable. This can happen due to missing builds on some architectures, for example. Help investigate!
        • 290 bugs are neither tagged patch, nor marked done. Help make a first step towards resolution!
      • Affecting Jessie only: 48 Those are already fixed in unstable, but the fix still needs to migrate to Jessie. You can help by submitting unblock requests for fixed packages, by investigating why packages do not migrate, or by reviewing submitted unblock requests.
        • 0 bugs are in packages that are unblocked by the release team.
        • 48 bugs are in packages that are not unblocked.

How do we compare to the Squeeze release cycle?

Week Squeeze Wheezy Diff 43 284 (213+71) 468 (332+136) +184 (+119/+65) 44 261 (201+60) 408 (265+143) +147 (+64/+83) 45 261 (205+56) 425 (291+134) +164 (+86/+78) 46 271 (200+71) 401 (258+143) +130 (+58/+72) 47 283 (209+74) 366 (221+145) +83 (+12/+71) 48 256 (177+79) 378 (230+148) +122 (+53/+69) 49 256 (180+76) 360 (216+155) +104 (+36/+79) 50 204 (148+56) 339 (195+144) +135 (+47/+90) 51 178 (124+54) 323 (190+133) +145 (+66/+79) 52 115 (78+37) 289 (190+99) +174 (+112/+62) 1 93 (60+33) 287 (171+116) +194 (+111/+83) 2 82 (46+36) 271 (162+109) +189 (+116/+73) 3 25 (15+10) 249 (165+84) +224 (+150/+74) 4 14 (8+6) 244 (176+68) +230 (+168/+62) 5 2 (0+2) 224 (132+92) +222 (+132/+90) 6 release! 212 (129+83) +212 (+129/+83) 7 release+1 194 (128+66) +194 (+128/+66) 8 release+2 206 (144+62) +206 (+144/+62) 9 release+3 174 (105+69) +174 (+105/+69) 10 release+4 120 (72+48) +120 (+72/+48) 11 release+5 115 (74+41) +115 (+74/+41) 12 release+6 93 (47+46) +93 (+47/+46) 13 release+7 50 (24+26) +50 (+24/+26) 14 release+8 51 (32+19) +51 (+32/+19) 15 release+9 39 (32+7) +39 (+32/+7) 16 release+10 20 (12+8) +20 (+12/+8) 17 release+11 24 (19+5) +24 (+19/+5) 18 release+12 2 (2+0) +2 (+2/+0)

Graphical overview of bug stats thanks to azhag:

Categories: Elsewhere

The Cherry Hill Company: Why Drupal, and some digression.

Planet Drupal - Fri, 26/09/2014 - 20:01

Recently, there was a thread stated by a frustrated Drupal user on the Code4Lib (Code for Libraries) mailing list. It drew many thoughtful and occasionally passionate responses. This was mine:

I think that it is widely conceded that it is a good idea to use the most suitable tool for a given task. But what does that mean? There is a long list of conditions and factors that go into selecting tools, some reflecting immediate needs, some reflecting long term needs and strategy, and others reflecting the availability of resources, and these interact in many ways, many of them problematic.

I have given the genesis of Cherry Hill’s tech evolution at the end of this missive. The short version is that we started focused on minimizing size and complexity while maximizing performance, and over time have moved to an approach that balances those agains building and maintenance cost along with human and infrastructure resource usage.

Among the lessons we have learned in the...

Read more »
Categories: Elsewhere

Steve Kemp: Next week I shall be mostly in Kraków

Planet Debian - Fri, 26/09/2014 - 19:20

Next week my wife and I shall be mostly visiting Poland, and spending a week in Kraków.

It has been a while since I've had a non-Helsinki-based holiday, so I'm looking forward to the trip.

In other news I've been rationalising DNS entries and domain names recently, all being well this zone should be served by Amazon shortly, subject to the usual combination of TTLs and resolution-puns.

Categories: Elsewhere

Appnovation Technologies: DrupalCamp Montreal 2014

Planet Drupal - Fri, 26/09/2014 - 19:16

The seventh annual DrupalCamp Montreal took place at McGill University on September 12nd and 13th.

var switchTo5x = false;stLight.options({"publisher":"dr-75626d0b-d9b4-2fdb-6d29-1a20f61d683"});
Categories: Elsewhere

Deeson: Take a look at our swag for DrupalCon Amsterdam

Planet Drupal - Fri, 26/09/2014 - 16:38
DrupalCon Amsterdam

Following the success of our DrupalCon t-shirt last year, Team Deeson is back and ready for the 'Dam armed with stickers, posters and a new tee to give away (because we're nice like that).

Deft with the Delft

This year we've taken The Netherlands' traditional and iconic Delftware as our inspiration for our DrupalCon Amsterdam tee - well, when in Amsterdam....

Drupal blossoms

Our swag has been produced by our insanely talented designer, Rachael Case.

Rachael reversed the colour scheme, and added elements of other ‘classic’ Dutch imagery, including a windmill (of course), tulips and a bike. But if you look closer, the tulips ‘blossoms’ out of the Drupal drop. We also have the Drupal logo perched jauntily on the bike. 

To finish it all off, we also added a rather tongue in cheek strapline, celebrating the ‘social’ side of Amsterdam...

Roll up, roll up! Tweet to get a tee

So if you want to grab your swag - tweet us @DeesonAgency and we’ll sort you out at DrupalCon Amsterdam. Zie je daar!

For posters and stickers, come and see us at stand 203 at DrupalCon for a chat. Don't leave it too late though, they have a habit of going fast!

See you there

So if you want swag, come and find us in Amsterdam between 29th September and 3rd October. John, Tim and I will be around to chat about all things Drupal, open-source and open data.

Categories: Elsewhere

Code Karate: Drupal 7 jReject Module

Planet Drupal - Fri, 26/09/2014 - 15:12
Episode Number: 170

In this DDoD we look at the jReject module. This module allows you to display a modal popup notifying the user / visitor that their browser is outdated and wont work well with the site. In the video, you will see that jReject comes with a wide variety of customizations to fit your brand and preferences.

Tags: DrupalDrupal 7Drupal PlanetTips and TricksJavascriptModal Forms
Categories: Elsewhere

Jakub Wilk: Pet peeves: debhelper build-dependencies (redux)

Planet Debian - Fri, 26/09/2014 - 14:05
$ zcat Sources.gz | grep -o -E 'debhelper [(]>= 9[.][0-9]{,7}([^0-9)][^)]*)?[)]' | sort | uniq -c | sort -rn 338 debhelper (>= 9.0.0) 70 debhelper (>= 9.0) 18 debhelper (>= 9.0.0~) 10 debhelper (>= 9.0~) 2 debhelper (>= 9.2) 1 debhelper (>= 9.2~) 1 debhelper (>= 9.0.50~)

Is it a way to protest against the current debhelper's version scheme?

Categories: Elsewhere

Kristian Polso: My road to DrupalCon Amsterdam

Planet Drupal - Fri, 26/09/2014 - 12:50
My first trip to a DrupalCon is just about to start! The yearly DrupalCon Europe is held in Amsterdam from 29th of September to 3rd of October 2014. There are a lot of sessions to be seen, wide array of sprints, social events and much more.
Categories: Elsewhere

KYbest: Using PHP_CodeSniffer in PhpStorm

Planet Drupal - Fri, 26/09/2014 - 12:46

PHP_CodeSniffer is a PHP5 script, which can validate PHP, JavaScript and CSS type source codes according to the different coding standards. In other words, you can easily check your source code’s standardization with a script, instead of knowing every detail about the coding standards by heart. You can use PHP_CodeSniffer in different ways, for example you can run it simply from terminal but thanks for the PhpStorm’s built-in support it becomes a much more effective tool.

Categories: Elsewhere

Holger Levsen: 20140925-reproducible-builds

Planet Debian - Fri, 26/09/2014 - 12:34
Reproducible builds? I never did any - manually

I've never done a reproducible build attempt of any package, manually, ever. But what I've done now is setting up reproducible builds on jenkins.debian.net which will build hundreds or thousands of packages, hopefully reproducibly, regularily in the future. Thanks to Lunar's and many other peoples work, this was actually rather easy. If you want to do this manually, it should take you just a few minutes to setup a suitable build environment.

So three days ago when I wasn't exactly bored I decided that it was a good moment to implement some reproducible build jobs on jenkins.d.n, and so I gave it a try and two hours later the basic implementation was working, and then it was an evening and morning of fine tuning until I was mostly satisfied. Since then there has been some polishing, but the basic setup is done and has been working since.

What's the result? One job, reproducible_setup will just create a suitable environment for pbuilding reproducible packages as documented so well on the Debian wiki. And as that job runs 3.5 minutes only (to debootstrap from scratch), it's run daily.

And then there are currently 16 other jobs, which test reproducible builds in different areas: d-i, core, some six major desktops and some selected desktop applications, some security + privacy related packages, some build chains we have in Debian, libreoffice and X.org. Most of these jobs run several hours, but luckily not days. And they discover packages which still fail to build reproducibly, which already has caused some bugs to be filed, eg. #762732 "libdebian-installer: please do not write timestamps in Doxygen generated documentation".

So this is the output from testing the reproducibilty of all debian-installer packages: 72 packages were successfully built reproducibly, while 6 packages failed to do so. I was quite impressed by these numbers as AFAIK noone tried to build d-i reproducibly before.

72 packages successfully built reproducibly: userdevfs user-setup usb-discover udpkg tzsetup rootskel rootskel-gtk rescue preseed pkgsel partman-xfs partman-target partman-partitioning partman-nbd partman-multipath partman-md partman-lvm partman-jfs partman-iscsi partman-ext3 partman-efi partman-crypto partman-btrfs partman-basicmethods partman-basicfilesystems partman-base partman-auto partman-auto-raid partman-auto-lvm partman-auto-crypto partconf os-prober oldsys-preseed nobootloader network-console netcfg net-retriever mountmedia mklibs media-retriever mdcfg main-menu lvmcfg lowmem localechooser live-installer lilo-installer kickseed kernel-wedge kbd-chooser iso-scan installation-report installation-locale hw-detect grub-installer finish-install efi-reader dh-di debian-installer-utils debian-installer-netboot-images debian-installer-launcher clock-setup choose-mirror cdrom-retriever cdrom-detect cdrom-checker cdebconf-terminal cdebconf-entropy bterm-unifont base-installer apt-setup anna 6 packages failed to built reproducibly: win32-loader libdebian-installer debootstrap console-setup cdebconf busybox

What's also impressive: all packages for the newly introduced Cinnamon Desktop build reproducibly from the start!

The jenkins setup is configured via just three small files:

That's it and that's enough to keep several cores busy for days. But as each job only takes a few hours each is scheduled twice a month and more jobs and packages shall be added in future (with some heuristics to schedule known good packages less often...)

I guess it's an appropriate opportunity to say "many thanks to Profitbricks", who have been donating the powerful virtual machine jenkins.debian.net is running on since October 2012. I also want to say "many many thanks to Helmut" (Grohne) who has recently joined me in maintaining this jenkins setup. And then I'd like to thank "the KGB trio" (Gregor, Tincho and Dam!) for providing those KGB bots on IRC, which are very helpful for providing notifications on IRC channels and last but not least thanks to everybody who contributed so that reproducible builds got this far! Keep up the jolly good work!

And If you happen to know failing packages not included in job-cfg/reproducible.yaml I'd like to hear about those, so they'll get regularily tested and appear on the radar, until finally bugs are filed, fixed and migrated to stable. So one day all binary packages in Debian stable will be build reproducibly. An important step on this road is probably to have this defined as an release goal for Jessie+1. And then for jessie+1 hopefully the first 10k packages will build reproducibly? Or whooping 23k maybe? And maybe release jessie+2 with 100%?!? We will see! Even Jessie already has quite some packages (someone needs to count them...) which build reproducibly with just modified dpkg(-dev) and debhelper packages alone...

So let's fix all the bugs! That said, an easier start for most of you is probably the list of useful things you (yes, you!) can do!

Oh, and last but surely not least in my book: many thanks too to the nice people hosting me so friendly in the last days! Keep on rockin'!

Categories: Elsewhere

Petter Reinholdtsen: How to test Debian Edu Jessie despite some fatal problems with the installer

Planet Debian - Fri, 26/09/2014 - 12:20

The Debian Edu / Skolelinux project provide a Linux solution for schools, including a powerful desktop with education software, a central server providing web pages, user database, user home directories, central login and PXE boot of both clients without disk and the installation to install Debian Edu on machines with disk (and a few other services perhaps to small to mention here). We in the Debian Edu team are currently working on the Jessie based version, trying to get everything in shape before the freeze, to avoid having to maintain our own package repository in the future. The current status can be seen on the Debian wiki, and there is still heaps of work left. Some fatal problems block testing, breaking the installer, but it is possible to work around these to get anyway. Here is a recipe on how to get the installation limping along.

First, download the test ISO via ftp, http or rsync (use ftp.skolelinux.org::cd-edu-testing-nolocal-netinst/debian-edu-amd64-i386-NETINST-1.iso). The ISO build was broken on Tuesday, so we do not get a new ISO every 12 hours or so, but thankfully the ISO we already got we are able to install with some tweaking.

When you get to the Debian Edu profile question, go to tty2 (use Alt-Ctrl-F2), run

nano /usr/bin/edu-eatmydata-install

and add 'exit 0' as the second line, disabling the eatmydata optimization. Return to the installation, select the profile you want and continue. Without this change, exim4-config will fail to install due to a known bug in eatmydata.

When you get the grub question at the end, answer /dev/sda (or if this do not work, figure out what your correct value would be. All my test machines need /dev/sda, so I have no advice if it do not fit your need.

If you installed a profile including a graphical desktop, log in as root after the initial boot from hard drive, and install the education-desktop-XXX metapackage. XXX can be kde, gnome, lxde, xfce or mate. If you want several desktop options, install more than one metapackage. Once this is done, reboot and you should have a working graphical login screen. This workaround should no longer be needed once the education-tasks package version 1.801 enter testing in two days.

I believe the ISO build will start working on two days when the new tasksel package enter testing and Steve McIntyre get a chance to update the debian-cd git repository. The eatmydata, grub and desktop issues are already fixed in unstable and testing, and should show up on the ISO as soon as the ISO build start working again. Well the eatmydata optimization is really just disabled. The proper fix require an upload by the eatmydata maintainer applying the patch provided in bug #702711. The rest have proper fixes in unstable.

I hope this get you going with the installation testing, as we are quickly running out of time trying to get our Jessie based installation ready before the distribution freeze in a month.

Categories: Elsewhere


Subscribe to jfhovinne aggregator - Elsewhere