Mediacurrent: Evaluating the security of Drupal contrib modules

Planet Drupal - mer, 15/06/2016 - 17:01

At Mediacurrent, we take website security very seriously. Drupal has some security best practices baked into its APIs that let developers create their own modules, hopefully with secure code, to contribute back to the community. You contribute back, don’t you?

Catégories: Elsewhere

InternetDevels: Live and learn: why Drupal is the best choice for university websites

Planet Drupal - mer, 15/06/2016 - 15:06

Drupal web development has many benefits.
Discover why Drupal is cool for university websites,
and see a nice website example by InternetDevels.

Read more
Catégories: Elsewhere

Andrew Shadura: Migrate to systemd without a reboot

Planet Debian - mer, 15/06/2016 - 13:51

Yesterday I was fixing an issue with one of the servers behind kallithea-scm.org: the hook intended to propagage pushes from Our Own Kallithea to Bitbucket stopped working. Until yesterday, that server was using Debian’s flavour of System V init and djb’s dæmontools to keep things running. To make the hook asynchronous, I wrote a service to be managed to dæmontools, so that concurrency issued would be solved by it. However, I didn’t implement any timeouts, so when last week wget froze while pulling Weblate’s hook, there was nothing to interrupt it, so the hook stopped working since dæmontools thought it’s already running and wouldn’t re-trigger it. Killing wget helped, but I decided I need to do something with it to prevent the situation from happening in the future.

I’ve been using systemd at work for the last year, so I am now confident I’m happier with systemd than with dæmontools, so I decided to switch the server to systemd. Not surprisingly, I prepared unit files in about 5 minutes without having to look into the manuals again, while with dæmontools I had to check things every time I needed to change something. The tricky thing was the switch itself. It is a virtual server, presumably running in Xen, and I don’t have access to the console, so if I bork break something, I need to summon Bradley Kuhn or someone from Conservancy, who’s kindly donated the server to the project. In any case, I decided to attempt to upgrade without a reboot, so that I have more options to roll back my changes in the case things go wrong.

After studying the manpages of both systemd’s init and sysvinit’s init, I realised I can install systemd as /sbin/init and ask already running System V init to re-exec. However, systemd’s init can’t talk to System V init, so before installing systemd I made a backup on it. It’s also important to stop all running services (except probably ssh) to make sure systemd doesn’t start second instances of each. And then: /tmp/init u — and we’re running systemd! A couple of additional checks, and it’s safe to reboot.

Only when I did all that I realised that in the case of systemd not working I’d probably not be able to undo my changes if my connection interrupted. So, even though at the end it worked, probably it’s not a good idea to perform such manipulations when you don’t have an alternative way to connect to the server :)

Catégories: Elsewhere

Enrico Zini: On discomfort and new groups

Planet Debian - mer, 15/06/2016 - 10:14

I recenyly wrote:

When you get involved in a new community, such as Debian, find out early where, if that happens, you can find support, understanding, and help to make it stop.

Last night I asked a group of friends what do they do if they start feeling uncomfortable when they are alone in a group of people they don't know. Here are the replies:

  • Wait outside the group until I figure the group out.
  • Find someone to talk for a while until you get comfortable.
  • If a person is making things uncomfortable for you, let them know, and leave if nobody cares.
  • Sit there in silence.
  • Work around unwelcome people by bearing them for a bit while trying to integrate with others.
  • Some people are easy to bribe into friendship, just bring cake.
  • While you don't know what is going on, you try to replicate what others are doing.
  • Spend time trying to get a feeling of what are the consequences of taking actions.
  • Purposefully disagree with people in a new environment to figure out if having a different opinion is accepted.
  • Once I was new and I was asked to be the person that invites everyone for lunch, that forced me to talk to everyone, and integrate.
  • When you are the first one to point something out, you'll probably soon find out you're not alone.
  • The reaction on the first time something is exposed, influences how often similar cases will be reported.

I think a lot of these point are good food for thought about barriers of entry, and about safety nets that a group has or might want to have.

Catégories: Elsewhere

PreviousNext: Ensuring Drupal 8 Block Cache Tags bubble up to the Page

Planet Drupal - mer, 15/06/2016 - 09:33

Whilst working on a Drupal 8 project, we found that cache tags for a Block Content entity embedded in the footer weren't bubbling up to the page cache.

Read on to find out how we debugged this and how you can ensure this doesn't happen to you to.

Catégories: Elsewhere

Russ Allbery: Review: Matter

Planet Debian - mer, 15/06/2016 - 05:37

Review: Matter, by Iain M. Banks

Publisher: Orbit Copyright: February 2008 ISBN: 0-316-00536-3 Format: Hardcover Pages: 593

Sursamen is an Arithmetic, Mottled, Disputed, Multiply Inhabited, Multi-million Year Safe, and Godded Shellworld. It's a constructed world with multiple inhabitable levels, each lit by thermonuclear "suns" on tracks, each level supported above the last by giant pillars. Before the recorded history of the current Involved Species, a culture called the Veil created the shellworlds with still-obscure technology for some unknown purpose, and then disappeared. Now, they're inhabited by various transplants and watched over by a hierarchy of mentor and client species. In the case of Sursamen, both the Aultridia and the Oct claim jurisdiction (hence "Disputed"), and are forced into an uneasy truce by the Nariscene, a more sophisticated species that oversees them both.

On Sursamen, on level eight to be precise, are the Sarl, a culture with an early industrial level of technology in the middle of a war of conquest to unite their level (and, they hope, the next level down). Their mentors are the Oct, who claim descendance from the mysterious Veil. The Deldeyn, the next level down, are mentored by the Aultridia, a species that evolved from a parasite on Xinthian Tensile Aranothaurs. Since a Xinthian, treated by the Sarl as a god, lives in the heart of Sursamen (hence "Godded"), tensions between the Sarl and the Aultridians run understandably high.

The ruler of the Sarl had three sons and a daughter. The oldest was killed by the people he is conquering as Matter starts. The middle son is a womanizer and a fop who, as the book opens, watches a betrayal that he's entirely unprepared to deal with. The youngest is a thoughtful, bookish youth pressed into a position that he also is not well-prepared for.

His daughter left the Sarl, and Sursamen itself, fifteen years previously. Now, she's a Special Circumstances agent for the Culture.

Matter is the eighth Culture novel, although (like most of the series) there's little need to read the books in any particular order. The introduction to the Culture here is a bit scanty, so you'll have more background and understanding if you've read the previous novels, but it doesn't matter a great deal for the story.

Sharp differences in technology levels have turned up in previous Culture novels (although the most notable example is a minor spoiler), but this is the first Culture novel I recall where those technological differences were given a structure. Usually, Culture novels have Special Circumstances meddling in, from their perspective, "inferior" cultures. But Sursamen is not in Culture space or directly the Culture's business. The Involved Species that governs Sursamen space is the Morthanveld: an aquatic species roughly on a technology level with the Culture themselves. The Nariscene are their client species; the Oct and Aultridia are, in turn, client species (well, mostly) of the Nariscene, while meddling with the Sarl and Deldeyn.

That part of this book reminded me of Brin's Uplift universe. Banks's Involved Species aren't the obnoxious tyrants of Brin's universe, and mentoring doesn't involve the slavery of the Uplift universe. But some of the politics are a bit similar. And, as with Uplift, all the characters are aware, at least vaguely, of the larger shape of galactic politics. Even the Sarl, who themselves have no more than early industrial technology. When Ferbin flees the betrayal to try to get help, he ascends out of the shellworld to try to get assistance from an Involved species, or perhaps his sister (which turns out to be the same thing). Banks spends some time here, mostly through Ferbin and his servant (who is one of the better characters in this book), trying to imagine what it would be like to live in a society that just invented railroads while being aware of interstellar powers that can do practically anything.

The plot, like the world on which it's set, proceeds on multiple levels. There is court intrigue within the Sarl, war on their level and the level below, and Ferbin's search for support and then justice. But the Sarl live in an artifact with some very mysterious places, including the best set piece in the book: an enormous waterfall that's gradually uncovering a lost city on the level below the Sarl, and an archaeological dig that proceeds under the Deldeyn and Sarl alike. Djan Seriy decides to return home when she learns of events in Sarl, originally for reasons of family loyalty and obligation, but she's a bit more in touch with the broader affairs of the galaxy, including the fact that the Oct are acting very strangely. There's something much greater at stake on Sursamen than tedious infighting between non-Involved cultures.

As always with Banks, the set pieces and world building are amazing, the scenery is jaw-dropping, and I have some trouble warming to the characters. Dramatic flights across tower-studded landscapes seeking access to forbidden world-spanning towers largely, but don't entirely, make up for not caring about most of the characters for most of the book. This did change, though: although I never particularly warmed to Ferbin, I started to like his younger brother, and I really liked his sister and his servant by the end of the book.

Unfortunately, the end of Matter is, if not awful, at least exceedingly abrupt. As is typical of Banks, we get a lot of sense of wonder but not much actual explanation, and the denouement is essentially nonexistent. (There is a coy epilogue hiding after the appendices, but it mostly annoyed me and provides only material for extrapolation about the characters.) Another SF author would have written a book about the Xinthian, the Veil, the purpose of the shellworlds, and the deep history of the galaxy. I should have known going in that Banks isn't that sort of SF author, but it was still frustrating.

Still, Banks is an excellent writer and this is a meaty, complex, enjoyable story with some amazing moments of wonder and awe. If you like Culture novels in general, you will like this. If you like set-piece-heavy SF on a grand scale, such as Alastair Reynolds or Kim Stanley Robinson, you probably like this. Recommended.

Rating: 8 out of 10

Catégories: Elsewhere

ActiveLAMP: 10 Things Every Jr. Drupal Web Developer Needs to Know

Planet Drupal - mer, 15/06/2016 - 04:00

Jonathan provides a list of 10 things that he feels every Jr. Drupal Web Developer needs to know

Catégories: Elsewhere

Joey Hess: second system

Planet Debian - mar, 14/06/2016 - 23:15

Five years ago I built this, and it's worked well, but is old and falling down now.

mark I outdoor shower

The replacement is more minimalist and like any second system tries to improve on the design of the first. No wood to rot away, fully adjustable height. It's basically a shower swing, suspended from a tree branch.

mark II outdoor shower

Probably will turn out to have its own new problems, as second systems do.

Catégories: Elsewhere

DrupalCon News: The artist formerly known as Development and Operations

Planet Drupal - mar, 14/06/2016 - 23:13

So, we've all heard of the word DevOps, we've all heard of the word FullStack developer. But what are you really? You might question yourselves every day whether or not you are working in one of those roles. Or do you prefer to call yourselves just an engineer?

Catégories: Elsewhere

Miloš Bovan: Mail sender authentication

Planet Drupal - mar, 14/06/2016 - 22:00
Mail sender authentication

This article is part of the Google Summer of Code blog series.

As mentioned in the previous blog post, the Mailhandler prototype for Drupal 8 has been finished. The prototype has already one of the main features implemented - processing mail messages and creating nodes. However, processed messages could be created by anyone which is not really nice for a module aiming to be used in production. To solve this problem, I've been working on user authentication implementation for almost a week.

The goal was to authenticate a user (sender of an email) before doing any mail processing. The meta issue with its child issues formalizes the requirements.

An obvious way to identify a mail sender is by checking the From field from the mail header. As this field is required to be present in mail headers (RFC 2822) we can assume we will always be able to identify the sender's mail address.
While writing a blog post about building a prototype, I acquainted you with Inmail. This is the module we are going to rely on to process mail messages. Inmail has a nice concept of plugins separated into deliverers, analyzers, and handlers. Deliverers are used to specify where a mail is coming from, analyzers to identify different types of messages and handlers to do some actions with mail messages and analyzer results. Last week, I already added a handler plugin and here I am going to talk about analyzers.

I started implementing an Inmail analyzer plugin and named it MailhandlerAnalyzer. It inherits AnalyzerBase from Inmail and currently has only one method - analyze() which accepts $message (represents a mail message entity) and processor_result (collects outcomes of a single mail processing pass) parameters. To preserve analyzer results, MailhandlerAnalyzerResult is created with $sender, $user and $body properties. The $sender will represent a sender's mail address (obtained from From mail header field), $user is a user entity in the current system and the $body is analyzed message body. After populating those fields in the analyzer, the job of MailhandlerNode is to authenticate and authorize the sender. If the conditions are met (user is validated) the handler will process the message and do the actions (create a node). This gives us a basic level of security.

However, From mail handler field can be faked by a malicious user. To bridge this obstacle, we introduced support for digitally signed emails. By its definition: digital signature is a mathematical scheme for demonstrating the authenticity of digital documents and it provides:

  • Authentication - since the secret (private) key is bound to a specific user, successful signature verification will show the identity of the sender
  • Integrity - it guarantees the message was not changed during transmission. Any change of the message content after signing it invalidates the signature
  • Non-repudiation - a user can not deny sending and signing the message

Handling digital signature is not supported in PHP by default and we had to find the tools to do it. We decided to use GnuPG implementation of the OpenPGP (Pretty Good Privacy) standard defined by RFC 4880 and gnupg PHP extension. If you are not familiar with the concept of digital signature I would recommend you looking into this ~2 minutes read.

As a starting point, we have to add mailhandler_gpg_key field to store GPG Key for each user. I wanted to preserve public_key and fingerprint properties for each key and decided to go with custom field type. Drupal.org documentation provides a nice explanation how to create a custom field type, corresponding widgets and formatters in Drupal 8.

Mailhandler GPG Key field

The next thing in our PGP journey is to extend analyzer and handlers to deal with signed messages. Usually, when handling emails, you will have to deal with regular expressions a lot. This case is not an exception. A new method called isSigned() is added to MailhandlerAnalyzer which analyzes the message and returns a boolean value. While we are dealing with messages at the low level, isSigned() populates the context data in case of signed messages. Context consists of:

  • PGP Type - the signed message could be inline-signed (clear-text signature) or as nowadays standard PGP/MIME.
  • Signed text - extracted the actual body without PGP signature
  • Signature - PGP signature

Populating the general result properties of the analyzer result instance (In case of signed messages MailhandlerAnalyzerResultSigned), the analyzer finishes its job. At this point, we can analyze the signed message and be sure if we are dealing with signed or unsigned mail messages.

Next, we have to adapt our handler to support signed messages too. That looks easy, as all the hard work is completed by the analyzer. The only thing we have to do is to verify the signature. We do that by getting the signed text and signature from the analyzer result and pass them to gnupg_verify() function. If the signature is valid and the corresponding user's GPG key is not disabled, revoked or expired we will continue the handling process. One last step to check before creating a node is to assert the user is authorized for that action.

To summarize, implementing support for digitally signed emails certainly provides an acceptable level of security for Mailhandler module. Keep in mind, that you will need to have GnuPG PHP extension enabled on your server to get GPG support. In either way, you will benefit from Mailhandler's default authentication system. The interesting thing about last week is that authentication support produced more than 1,2k new lines of code in the Github pull request

Next week, I am going to work on footer detection in a mail message. This will allow us to remove any unnecessary and unintended content from a node body. To support all those features, most of the work will be done in the analyzer. The mid-term evaluation period starts in less than a week which means the next week will be used to do preparation for it as well.

Over and out.



Milos Tue, 06/14/2016 - 22:00 Tags Google Summer of Code Drupal Open source Drupal Planet Add new comment
Catégories: Elsewhere

The Jibe: Applying BEM inspired classes to your Drupal 8 theme

Planet Drupal - mar, 14/06/2016 - 20:18
Applying BEM inspired classes to your Drupal 8 theme Leah Wagner June 14, 2016 11:18am
Catégories: Elsewhere

Olivier Grégoire: Fourth week at GSoC: let's integrate this new window!

Planet Debian - mar, 14/06/2016 - 18:57

I began by trying to integrate my window like a new view. That was not a good idea because my UI is actually integrated in the call view. So, I tried to integrate it like the buttons present on the view. Not a really good idea too, I have some issue with the clutter who do strange stuff. Finally, I work in a new window. I think that's a good idea because when you want to debug ring, it's more easier with a separate window.

I have meet some conflicts between my clean installation from the official website and my installation test. In response to this issue, I began to install my program test in another folder.

Catégories: Elsewhere

NEWMEDIA: PCI Compliance & Drupal Commerce: Which Payment Gateway Should I Choose?

Planet Drupal - mar, 14/06/2016 - 15:18
PCI Compliance & Drupal Commerce: Which Payment Gateway Should I Choose? When version 3.0 of the Payment Card Industry Data Security Standard (PCI-DSS) became mandatory in 2015, many eCommerce merchants were ill prepared to meet the more stringent security requirements with the existing modules available for Ubercart and Drupal Commerce. Thankfully, several of the more popular payment processors on the market rose to the challenge and have provided solutions that significantly reduce this complexity. Better yet, there are several contributed modules with stable releases that leverage these new technologies.
Rick Manelius Tue, 06/14/2016 - 13:18

One of the main driving forces for writing the community supported Drupal PCI Compliance White Paper was to help developers and merchants to make smart choices that balance functionality with security. At the time, there were dozens of payment gateway modules available and knowing what risks were involved with each was no easy task. Ultimately, many of them posed a significant risk to the merchant because they were requiring Drupal to temporarily store and transmit card data.

For those wishing to better understand these nuances, I highly recommend reading the white paper because you need to be able to defend your choices, especially if you’re trying to pass an audit or have to deal with the fallout of a data breach. That said, if I do nothing else but help you pick a sane default, I’ll rest better at night.

My (Current) Recommendations

Historically, Hosted Payment Pages (HPPs) were the preferred solution because they fully offload the entry of the card to a service like Paypal, Authorize.net SIM, or Recurly. This greatly reduces the number of potential attack vectors (thereby increasing security) and is the easiest way to get to a PCI SAQ A status (thereby reducing your costs to obtain and maintain compliance). From a security perspective, HPP are always a good choice. The challenge has always been to convince clients to opt for this because they lose control of the customer experience when customers are redirected to an external site.

Fortunately, for those who want the best of both worlds, there is the inline iframe option. The beauty here is that the customer is kept on the website the entire time and (usually) unaware that the credit card entry is being submitted through the form within the iframe loaded directly from the payment gateway itself. Unfortunately, throughout much of 2014 and 2015, only one module provided this option (Hosted PCI). While I originally recommended this option, the setup and ongoing fees were prohibitive and it generally took several days to get an account provisioned.

Since that time, two major players have created inline options: Commerce Braintree and Commerce Stripe. I highly recommend these two modules because they have extended the checkout inference in a straightforward way and allow users to get either one installed and configured in less than an hour. The only caveat for the Stripe module is site-builders MUST remember to enable the correct rule (the module provides two). If you enable the direct post rule, it still will fall within SAQ A-EP because direct post solutions can be attacked by on-page keyloggers. It’s a subtle but important detail.

Finally, remember that these recommendations are current as of the publishing of this article on June 14, 2016. The PCI-DSS standard will continue to evolve as will the payment gateways that are trying to provide the best solutions for their customers. Still, if you’re looking for two solid solutions to start using immediately, you’ll get a lot of mileage out of the Stripe and Braintree modules.

Catégories: Elsewhere

The Sego Blog: Enhanced Drupal 8 Contact Forms with Contact Storage

Planet Drupal - mar, 14/06/2016 - 14:51
06/14/2016Enhanced Drupal 8 Contact Forms with Contact Storage

Drupal 8’s improvements to contact form management and creation is one of my favorite site building upgrades that has been added to Drupal 8’s core.

We now have the ability to field contact forms in the same familiar way as many other entities and can have multiple form variations living at different urls on the same site.

Catégories: Elsewhere

Arpit Jalan: Third week of Google Summer of Coding

Planet Drupal - mar, 14/06/2016 - 14:38
TL;DR Safe Search detection of the Google Cloud Vision API allows the end users to avoid any explicit content or violence in images to be uploaded on the site. I worked on integrating this feature to the module as a constraint to those image fields which have the “Safe Search” feature enabled.
Let me first give a brief summary about the current status of the module Google Vision. In the earlier weeks, I had implemented the services and wrote tests for the module, which are now committed to the module.Now, coming to the progress of this week.Initially, I implemented the Safe Search detection as a function which worked in the same way as the Label detection.However, my mentors suggested that this feature should rather be a Constraint on the image fields, which would be validated if the feature is enabled for the field. It depends on the user whether to keep safe search on their site or not, and they can implement it any time they want by just enabling/disabling the configuration of the image field. Hence, now it is a user choice, rather than the developer’s choice whether to implement this feature or not.Presently, the code is under review by my mentors whether it needs changes or is ready for commit.Constraints and Validators are wonderful features of Drupal 8. Constraints, as the name goes, are certain restrictions which we pose on the various fields. Validators are used to implement the logic to create the situation under which the constraints are to be applied.This week had a lot of new things stored for me. I had no idea about the constraints and validators when I was asked to implement them at the first place. I spent hours on them, learning about them and seeking guidance from my mentors on the issues I faced. I developed gradually on it, and by the end of the week, I was able to implement them for the safe search detection feature.
Catégories: Elsewhere

ComputerMinds.co.uk: Drupal 8 Event Subscribers - the successor to alter hooks

Planet Drupal - mar, 14/06/2016 - 14:00

In Drupal 7 if you wanted to tweak the functionality of an existing function then an alter hook was your best friend, but in Drupal 8 it's "all change!"

With the introduction of Symphony, your new BFF is an Event Subscriber. Alter hooks still exist, but it is recommended that we all move towards Events to fall in line with Symphony.

If you are interested in the comparison between alter hooks and events then I recommend this article from PreviousNext.

Catégories: Elsewhere

Code Enigma: Doing more with Drush sql-sanitize

Planet Drupal - mar, 14/06/2016 - 13:38
Doing more with Drush sql-sanitize Language English Doing more with Drush sql-sanitize

Support team lead Dan Smith shows us how to anonymise personal data in Drupal using Drush.

Tue, 2016-06-14 12:38By galooph


The sites that we build very often end up containing large amounts of user data, much of which allows individual people to be identified. Not just names, but social media accounts, telephone numbers, addresses, occupation details etc. In the UK, user data comes under the Data Protection Act, particularly personal data, which the Act defines as "data which relate to a living individual who can be identified".

So, your site contains lots of personal data, but you want to give a database dump to a Drupal developer so that she can code some new feature for you. How can you handle this without getting on the wrong side of the Information Commissioner's Office (ICO)? You could manually delete all the personal data before you created the database dump, but then your developer won't have any realistic data to work with. Fortunately, there are ways of sanitising the personal data, a process that the ICO refers to as anonymisation. The ICO has a useful code of practice that is worth reading – Anonymisation: managing data protection risk code of practice.

The Drupal command line tool, Drush, comes with a command to sanitise the users table, drush sql-sanitize. While this is better than nothing, it only sanitises user passwords and email addresses. Obviously, it doesn't know about all of the additional fields storing personal data that you added when you built your site.

Note: Drush sql-sanitize will sanitise data in the current database – you do not want to play with this on a live or production server!

With that caveat out of the way, assume I have a site with profile fields set up for a user to enter a LinkedIn url (field_profile_linkedin_page), a telephone number (field_profile_telephone_number), and a twitter username (field_profile_twitter_handle). How can I tell Drush to sanitise these fields too?

Thankfully, Drush is kind enough to provide a hook, hook_drush_sql_sync_sanitize(), which makes this straightforward. Assuming you’ve got a custom module that you’re using already, my_module, you would add a drush include file, /my_module/my_module.drush.inc, containing:

/** * Implements hook_drush_sql_sync_sanitize. */ function my_module_drush_sql_sync_sanitize($source) { // The query to sanitise field_data_field_profile_linkedin_page $linkedin_page_query = "UPDATE field_data_field_profile_linkedin_page SET field_profile_linkedin_page_value = CONCAT('http://uk.linkedin.com/user-name/', FLOOR(100000 + (RAND() * 900000)));";   // The query to sanitise field_revision_field_profile_linkedin_page $linkedin_page_query .= "UPDATE field_revision_field_profile_linkedin_page SET field_profile_linkedin_page_value = CONCAT('http://uk.linkedin.com/user-name/', FLOOR(100000 + (RAND() * 900000)));";   drush_sql_register_post_sync_op('sanitise_linkedin_page', dt('Sanitise field_profile_linkedin_page'), $linkedin_page_query);   // The query to sanitise field_data_field_profile_telephone_number $telephone_number_query = "UPDATE field_data_field_profile_telephone_number SET field_profile_telephone_number_value = CONCAT('+44', FLOOR(1000000000 + (RAND() * 100000000)));";   // The query to sanitise field_data_field_profile_telephone_number $telephone_number_query .= "UPDATE field_revision_field_profile_telephone_number SET field_profile_telephone_number_value = CONCAT('+44', FLOOR(1000000000 + (RAND() * 100000000)));";   drush_sql_register_post_sync_op('sanitise_telephone_number', dt('Sanitise field_profile_telephone_number'), $telephone_number_query);   // The query to sanitise field_data_field_profile_twitter_handle $twitter_query = "UPDATE field_data_field_profile_twitter_handle SET field_profile_twitter_handle_value = SUBSTRING(MD5(RAND()) FROM 1 FOR 8);";   // The query to sanitise field_revision_field_profile_twitter_handle $twitter_query .= "UPDATE field_revision_field_profile_twitter_handle SET field_profile_twitter_handle_value = SUBSTRING(MD5(RAND()) FROM 1 FOR 8);";   drush_sql_register_post_sync_op('sanitise_twitter_handle', dt('Sanitise field_profile_twitter_handle'), $twitter_query); }

The hook works by allowing you to specify database queries that will be run as part of the Drush sql-sanitize process.

The first query added is used to sanitise the LinkedIn url:

$linkedin_page_query = "UPDATE field_data_field_profile_linkedin_page SET field_profile_linkedin_page_value = CONCAT('http://uk.linkedin.com/user-name/', FLOOR(100000 + (RAND() * 900000)));";

The database table storing the LinkedIn field data is field_data_field_profile_linkedin_page, and the column containing the actual url string is field_profile_linkedin_page_value. The query will step through each entry and replace the url with the output of CONCAT('http://uk.linkedin.com/user-name/', FLOOR(100000 + (RAND() * 900000))).

CONCAT is the SQL command to concatenate strings together, RAND returns a random floating point number between 0 and 1, and FLOOR returns the largest integer value not greater than the value it’s passed. Putting that together, the real LinkedIn urls are replaced with ‘http://uk.linkedin.com/user-name/’.

LinkedIn seem to use a few different url formats, and this is just the one that I picked as an example.

Don’t forget that Drupal stores revisions of data, so as well as sanitising the field_data_field_profile_linkedin_page table, we need to repeat the query on the field_revision_field_profile_linkedin_page table too. This is easy to overlook!

Now that we have a query built to sanitise both of the LinkedIn tables, we let Drush know about it by calling drush_sql_register_post_sync_op().

drush_sql_register_post_sync_op('sanitise_linkedin_page', dt('Sanitise field_profile_linkedin_page'), $linkedin_page_query);

The first parameter isn’t actually used for anything at the moment. The second parameter is the text that Drush will display when the the sql-sanitize command is run, and the third parameter is the SQL query that you want to run.

Moving on to the telephone number field, we need a new sanitisation query:

$telephone_number_query = "UPDATE field_data_field_profile_telephone_number SET field_profile_telephone_number_value = CONCAT('+44', FLOOR(1000000000 + (RAND() * 100000000)));";

This is very similar to the LinkedIn query above, replacing all user telephone numbers with ‘+44’. Again, don’t forget to do the same on the corresponding revision table before calling drush_sql_register_post_sync_op().

The final example is the twitter username field. This one is a little different as instead of a random number, I’ll generate a random 8 character string. One way to do this is to use the SUBSTRING, MD5 and RAND SQL functions.

$twitter_query = "UPDATE field_data_field_profile_twitter_handle SET field_profile_twitter_handle_value = SUBSTRING(MD5(RAND()) FROM 1 FOR 8);";

This generates an MD5 hash of a random value, which we then take the first 8 characters from. Yet again, don’t forget to do the same thing on the corresponding revision table!

With our hook_drush_sql_sync_sanitize() implementation complete, we can try running it with Drush sql-sanitize. Don’t forget – you do not want to be running this on a production server!

$ drush sql-sanitize   The following post-sync operations will be done on the destination: * Reset passwords and email addresses in users table * Truncate Drupal's sessions table * Sanitize field_profile_linkedin_page * Sanitize field_profile_telephone_number * Sanitize field_profile_twitter_handle   Do you really want to sanitize the current database? (y/n):

All being well, you should see a new item in the bullet list for each call to drush_sql_register_post_sync_op() that you made in your hook.

After answering ‘y’, I checked my database. For the LinkedIn tables, I had entries like:

For the telephone values, I had entries like:

  • +441007009394
  • +441058298812
  • +441070465880

And for the twitter values, I had entries like:

  • 8845fb7e
  • 9af12ccd
  • 43df577c

Not the best twitter usernames in the world, but adequate for testing.

Now that you’ve got a sanitised database, you can export it using Drush sql-dump. Before you do that, though, you should be aware that Drupal’s cache tables in the database can also potentially leak information. To protect against this, you can instruct Drush sql-dump to export just the structure of certain tables, omitting the actual data.

You can specify these structure-only tables directly on the command line, using the --structure-tables-list option (see the sql-dump documentation for more information). Alternatively, you can add a list of tables to the $options['structure-tables'][‘your_ref’] array in your drushrc.php file. That way, you don’t have to type the list each time, just use the --structure-tables-key=your_ref option with sql-dump.

So, with the addition of a small chunk of code, we can generate a completely anonymised database dump very easily. If you keep this in mind during development and add a sanitisation query every time you add a new field which stores personal data, it’s even easier.


Main image by Mike Mozart, released on Flickr.com under the Creative Commons Attribution 2.0 Generic license.

BlogDenial of Service, what you need to know BlogDesign and build a website in a day FAQMy site doesn't load for me, is it down? BlogDrupal Migrate Tricks: Disabling rules before running a migration.
Catégories: Elsewhere

Into my Galaxy: GSoC’ 16: Port Search Configuration module; coding weeek #3

Planet Drupal - mar, 14/06/2016 - 09:56

Google Summer of Code (GSoC’ 16) is entering into the mid-Term evaluation stage. I have been working on the porting search configuration module for Drupal for the past three weeks.

Search configuration module helps to configure the search functionality in Drupal. This is really an important feature when it comes to a content management system like Drupal. I am almost mid-way in the port process as indicated in the timeline of Google Summer of Code.

It is really a great feeling to learn the Drupal concepts this summer. I would like to take this opportunity to share with you some key aspects I had to deal with in the past week.

Once a module is installed and later on if some changes are amended, we need not rewrite the Drupal code. Instead, Drupal gives up the option to make use of a hook, function hook_update_N which helps us to write the update needed and the database schema gets updated accordingly. Currently, since my module is just going into its first release after this port process, I need not write an update function. I just have to make this update in connection with the earlier code. The same hook works for both Drupal 7 and 8.

Another feature is the hook_node_insert, this hook is invoked to insert a new node into the database. So, here we are writing into the database a drupal record. In Drupal 7, this insertion was done by the drupal_write_record(). But,in D8, it has been replaced by the merge query and the entity API. The merge queries support the insert and update options to the database.

In connection with the nodes, another hook function is the hook_node_update. This functionality updates the node contents which has been inserted into the Drupal record (database). This takes in an argument, obviously,  the node has to be passed, for which the updating is intended.

The hook_uninstall gets invoked as the name suggests, in the uninstall process of the modules. The hook removes the variable used by the module so as to free the memory. It also modifies the existing tables if required. The D7 version of Drupal used the  variable_del for removing the variables.

For instance,


// Drupal 7 code

This has been replaced by the delete() of the configuration API.



search_config.settings is the default configuration file.

I will post the updates on this port process regularly. Please wait for the future posts.

Catégories: Elsewhere

OpenLucius: Don’t ask what the blockchain can do for you…. Ask what you can do for the blockchain!

Planet Drupal - mar, 14/06/2016 - 09:50

The title is the abstract. The article is aimed at current reputable CMS web-based frameworks like Drupal, Wordpress, Joomla, Hippo, Rails, Django, etc.

Why this blog post?

Dries Buytaert creator of Drupal raised a simple, yet extensive question on twitter (‘@Dries’):

“What could the Blockchain mean for a CMS like Drupal? #brainstorm”

Let us analyse this question step by step. And then come to a surprising conclusion: it is much more interesting for Drupal to change the question.

Catégories: Elsewhere

Simon McVittie: GTK versioning and distributions

Planet Debian - mar, 14/06/2016 - 03:56

Allison Lortie has provoked a lot of comment with her blog post on a new proposal for how GTK is versioned. Here's some more context from the discussion at the GTK hackfest that prompted that proposal: there's actually quite a close analogy in how new Debian versions are developed.

The problem we're trying to address here is the two sides of a trade-off:

  • Without new development, a library (or an OS) can't go anywhere new
  • New development sometimes breaks existing applications

Historically, GTK has aimed to keep compatible within a major version, where major versions are rather far apart (GTK 1 in 1998, GTK 2 in 2002, GTK 3 in 2011, GTK 4 somewhere in the future). Meanwhile, fixing bugs, improving performance and introducing new features sometimes results in major changes behind the scenes. In an ideal world, these behind-the-scenes changes would never break applications; however, the world isn't ideal. (The Debian analogy here is that as much as we aspire to having the upgrade from one stable release to the next not break anything at all, I don't think we've ever achieved that in practice - we still ask users to read the release notes, even though ideally that wouldn't be necessary.)

In particular, the perceived cost of doing a proper ABI break (a fully parallel-installable GTK 4) means there's a strong temptation to make changes that don't actually remove or change C symbols, but are clearly an ABI break, in the sense that an application that previously worked and was considered correct no longer works. A prominent recent example is the theming changes in GTK 3.20: the ABI in terms of functions available didn't change, but what happens when you call those functions changed in an incompatible way. This makes GTK hard to rely on for applications outside the GNOME release cycle, which is a problem that needs to be fixed (without stopping development from continuing).

The goal of the plan we discussed today is to decouple the latest branch of development, which moves fast and sometimes breaks API, from the API-stable branches, which only get bug fixes. This model should look quite familiar to Debian contributors, because it's a lot like the way we release Debian and Ubuntu.

In Debian, at any given time we have a development branch (testing/unstable) - currently "stretch", the future Debian 9. We also have some stable branches, of which the most recent are Debian 8 "jessie" and Debian 7 "wheezy". Different users of Debian have different trade-offs that lead them to choose one or the other of these. Users who value stability and want to avoid unexpected changes, even at a cost in terms of features and fixes for non-critical bugs, choose to use a stable release, preferably the most recent; they only need to change what they run on top of Debian for OS API changes (for instance webapps, local scripts, or the way they interact with the GUI) approximately every 2 years, or perhaps less often than that with the Debian-LTS project supporting non-current stable releases. Meanwhile, users who value the latest versions and are willing to work with a "moving target" as a result choose to use testing/unstable.

The GTK analogy here is really quite close. In the new versioning model, library users who value stability over new things would prefer to use a stable-branch, ideally the latest; library users who want the latest features, the latest bug-fixes and the latest new bugs would use the branch that's the current focus of development. In practice we expect that the latter would be mostly GNOME projects. There's been some discussion at the hackfest about how often we'd have a new stable-branch: the fastest rate that's been considered is a stable-branch every 2 years, similar to Ubuntu LTS and Debian, but there's no consensus yet on whether they will be that frequent in practice.

How many stable versions of GTK would end up shipped in Debian depends on how rapidly projects move from "old-stable" to "new-stable" upstream, how much those projects' Debian maintainers are willing to patch them to move between branches, and how many versions the release team will tolerate. Once we reach a steady state, I'd hope that we might have 1 or 2 stable-branched versions active at a time, packaged as separate parallel-installable source packages (a lot like how we handle Qt). GTK 2 might well stay around as an additional active version just from historical inertia. The stable versions are intended to be fully parallel-installable, just like the situation with GTK 1.2, GTK 2 and GTK 3 or with the major versions of Qt.

For the "current development" version, I'd anticipate that we'd probably only ship one source package, and do ABI transitions for one version active at a time, a lot like how we deal with libgnome-desktop and the evolution-data-server family of libraries. Those versions would have parallel-installable runtime libraries but non-parallel-installable development files, again similar to libgnome-desktop.

At the risk of stretching the Debian/Ubuntu analogy too far, the intermediate "current development" GTK releases that would accompany a GNOME release are like Ubuntu's non-LTS suites: they're more up to date than the fully stable releases (Ubuntu LTS, which has a release schedule similar to Debian stable), but less stable and not supported for as long.

Hopefully this plan can meet both of its goals: minimize breakage for applications, while not holding back the development of new APIs.

Catégories: Elsewhere


Subscribe to jfhovinne agrégateur - Elsewhere