Feed aggregator

Triquanta Web Solutions: What the fq? A short summary of Solr query fields.

Planet Drupal - Tue, 24/03/2015 - 08:24
#And how to use them in Drupal

A popular search engine for Drupal is Apache Solr. Although installation and configuration of Solr can be done almost completely via the Drupal Admin UI, in some cases it can be very instructive to see and understand what data is sent to and from Solr when a search is done from Drupal.

First place to look when using Solr inside Tomcat is the log file of Tomcat, usually /var/log/tomcat6/catalina.out. If this file is not present in this directory on your system use

locate catalina.out

or a similar command to find it.

If Solr is used in its own Jetty-container and is run as a seperate service (which is the only option for Solr 5.x), log4j is used to implement logging and configuration is done in the log4j.properties-file.

By default the logs are written to 'logs/solr' under the Solr root, this can be set with in log4j.properties with the 'solr.log'-option, for example:


For more information about log4j, see Apache Log4j 2.

In the log, each line like the following represents one search query:

INFO: [solrdev] webapp=/solr path=/select params={spellcheck=true& spellcheck.q=diam& hl=true& facet=true& f.bundle.facet.mincount=1& f.im_field_tag.facet.limit=50& f.im_field_tag.facet.mincount=1& fl=id,entity_id, entity_type, bundle, bundle_name, label, ss_language, is_comment_count, ds_created, ds_changed, score, path, url, is_uid, tos_name& f.bundle.facet.limit=50& f.content.hl.alternateField=teaser& hl.mergeContigious=true& facet.field=im_field_tag& facet.field=bundle& fq=(access_node_tfslk0_all:0+OR+access__all:0)& mm=1& facet.mincount=1& qf=content^40& qf=label^5.0& qf=tags_h2_h3^3.0& qf=taxonomy_names^2.0& qf=tos_content_extra^0.1& qf=tos_name^3.0& hl.fl=content& f.content.hl.maxAlternateFieldLength=256& json.nl=map& wt=json& rows=10& pf=content^2.0& hl.snippets=3& start=0& facet.sort=count& q=diam& ps=15} hits=10 status=0 QTime=12

NB: one way to get a clearer look at the log-lines, is by copying one of them into a text editor and replace '&' with '&\n' and ',' with ',\n' to get a more readable text.

Here '[solrdev]' indicates the core the query was submitted to and 'path=/select' the path.

Everything between the {}-brackets is what is added to the query as parameter. If your Solr host is localhost, Solr is running on port 8080 and the name of your core is solrdev then you can make this same query in any browser by starting with:


followed by all the text between the {}-brackets.

This looks like no simple query and in fact a lot is going on here: not only is the Solr/Lucene index searched for a specific term, we also tell Solr which fields to return, to give us spellcheck suggestions, to higlight the search term in the return snippet, to return facets etcetera.

For better understanding of the Solr query we will break it down and discuss each (well, maybe not all) of the query parameters from the above log line.

Query breakdown q: Search term

The most basic Solr query would only contain a q-field, e.q.


This would return all fields present in Solr for all matching documents. This will either be fields directly defined in the schema.xml (in this examples we use the schema's based on the schema included in the Search API Solr module) like bundle_name:

<field name="bundle_name" type="string" indexed="true" stored="true"/>

or dynamic fields, which are created according to the field definition in the schema.xml, eg:

<dynamicField name="ss_*" type="string" indexed="true" stored="true" multiValued="false"/>

The above query run on my local development environment would return a number of documents like this one:

<doc> <bool name="bs_promote">true</bool> <bool name="bs_status">true</bool> <bool name="bs_sticky">false</bool> <bool name="bs_translate">false</bool> <str name="bundle">point_of_interest</str> <str name="bundle_name">Point of interest</str> <str name="content">Decet Secundum Wisi Cogo commodo elit eros meus nisl turpis.(...) </str> <date name="ds_changed">2015-03-04T08:45:18Z</date> <date name="ds_created">2015-02-19T18:55:44Z</date> <date name="ds_last_comment_or_change">2015-03-04T08:45:18Z</date> <long name="entity_id">10</long> <str name="entity_type">node</str> <str name="hash">tfslk0</str> <str name="id">tfslk0/node/10</str> <long name="is_tnid">0</long> <long name="is_uid">1</long> <str name="label">Decet Secundum Wisi</str> <str name="path">node/10</str> <str name="path_alias">content/decet-secundum-wisi</str> <str name="site">http://louis.atlantik.dev/nl</str> <arr name="sm_field_subtitle"> <str>Subtitle Decet Secundum Wisi</str> </arr> <arr name="spell"> <str>Decet Secundum Wisi</str> <str>Decet Secundum Wisi Cogo commodo elit eros meus nisl turpis. (...) </str> </arr> <str name="ss_language">nl</str> <str name="ss_name">admin</str> <str name="ss_name_formatted">admin</str> <str name="teaser"> Decet Secundum Wisi Cogo commodo elit eros meus nisl turpis. Abluo appellatio exerci exputo feugiat jumentum luptatum paulatim quibus quidem. Decet nutus pecus roto valde. Adipiscing camur erat haero immitto nimis obruo pneum valetudo volutpat. Accumsan brevitas consectetuer fere illum interdico </str> <date name="timestamp">2015-03-05T08:30:11.7Z</date> <str name="tos_name">admin</str> <str name="tos_name_formatted">admin</str> <str name="url"> http://louis.atlantik.dev/nl/content/decet-secundum-wisi </str> </doc>

(In this document I have left out most of the content of the fields content and spell).

Obviously, when searching we are not interested in all fields, for example the afore mentioned field content contains a complete view (with HTML tags stripped) of the node in Drupal, which most of the times is not relevant when showing search results.

fl: Fields

So the first thing we want to do is limit the number of fields Solr is returning by adding the fl parameter, in which we name the fields we want returned from Solr:


This would return documents like:

<doc> <float name="score">0.1895202</float> <str name="bundle">point_of_interest</str> <str name="bundle_name">Point of interest</str> <long name="entity_id">10</long> <str name="entity_type">node</str> <str name="label">Decet Secundum Wisi</str> <str name="path">node/10</str> <str name="ss_language">nl</str> <str name="url"> http://localhost/nl/content/decet-secundum-wisi </str> </doc>

Here we not only use fields which are direclty present in the index (like bundle) but also a generated field score which indicates the relevance of the found item. This field can be used to sort the results by relevance.

By the way, from Solr 4.0 on, the fl can be added multiple times to the query with in each parameter one field. However the "old" way of a comma-seperated field list is still supported (also in Solr 5).

So in Solr 4 the query could (or should I say: should?) be written as:


NB: in Solr 3 this would give a wrong result, with only the first fl field returned.

Please note that you can not query all dynamic fields at once with a fl-parameter like 'fl=ss_*': you must specify the actual field which are created while indexing: fl=ss_language,ss_name,ss_name_formatted... etc.

fq: Filter queries

One thing we do not want is users finding unpublished content which they are not allowed to see. When using the Drupal scheme, this can be accomplished by filtering on the dynamic fields created from

<dynamicField name="access_*" type="integer" indexed="true" stored="false" multiValued="true"/>

To filter, we add a fq-field like this:


The queries in fq are cached independent from the other Solr queries and so can speed up complex queries. The query can also contain range-queries, e.g. to limit the returned documents by a popularity-field present in the Drupal-node (and of course, indexed) between 50 and 100, one could use

fq=+popularity:[50 TO 100]

For more info see the Solr wiki, CommonQueryParameters

To add filter queries programmatically in Drupal when using the Apache Solr Search-module, implement


and use


to add filters to the query.

For example, to filter the query on the current language, use:

function mymodule_apachesolr_query_alter(DrupalSolrQueryInterface $query) { global $language; $query->addFilter("ss_language", $language->language); }

If using the Solr environment with Search API Solr Search, implement

hook_search_api_solr_query_alter(array &$call_args, SearchApiQueryInterface $query)

In this hook you can alter or add items to the


to filter the query.


In both cases the name of the field to use can be found via the Solr admin. In this case it is a string fieks created from the dynamic ss_ *-field in 'schema.xml'.

rows and start: The rows returnd

The 'row'-parameter determines how many documents Solr will return, while 'start' detemines how many documents to skip. This basically implements pager-functionality.

wt: Type of the returned data

The 'wt'-parameter determines how the data from Solr is returned. Most used are:

  • xml (default)
  • json

See https://cwiki.apache.org/confluence/display/solr/Response+Writers for a complete list of available formats and their options.

qf: Boosting

The DisMax and EdisMax plugins have the abilty to boost the score of documents. In the default Drupal requestHandler ("pinkpony") the Edismax-parser is set as query plugin:

<requestHandler name="pinkPony" class="solr.SearchHandler" default="true"> <lst name="defaults"> <str name="defType">edismax</str>

Boosting can be done by adding one or more qf-parameters which define the fields ("query field") and their boost value in the following syntax:



qf=content^40& qf=label^5.0& qf=tags_h2_h3^3.0& qf=taxonomy_names^2.0&

In Drupal this is done by implementing the relevant (Search API or Apache Solr) hook and adding the boost.

For example, let's say you want to boost the result with a boost value of "$boost" if a given taxonomy term with id "$tid" is present int the field "$solr_field"

For Apache Solr module we should use:

function mymodule_apachesolr_query_alter(DrupalSolrQueryInterface $query) { $boost_q = array(); $boost_q[] = $solr_field . ':' . $tid . '^' . $boost; $query->addParam('bq', $boost_q); }

For Search API Apache Solr:

function mymodule_search_api_solr_query_alter(array &$call_args, SearchApiQueryInterface $query) { $call_args['params']['bq'][] = $solr_field . ':' . $tid . '^' . $boost; }

NB: both examples we ignore any boost-values set by other modules for the same field. In real life you should merge the exisitng boost-array with our new one.

Negative boosting

Negative boosting is not possible in Solr. It can however be simulated by boosting all documents who do not have a specific value. In the above example of boosting on a specific taxonomy term, we can use:

$boost = abs($boost); $boost_q[] = '-' . $solr_field . ':' . $tid . '^' . $boost;

or, for Search API

$boost = abs($boost); $call_args['params']['bq'][] = '-' . $solr_field . ':' . $tid . '^' . $boost;

where the '-' for the field name is used to indicated that we want to boost the items that do not have this specific value.

mm: minimum should match

The Edismax parser also supports querying phrases (as opposed to single words). With the 'mm' parameters the minimum amount of words that must be present in the Solr-document is given. For example: if the query is "little red corvet" and 'mm' is set to '2', only documents which contain at least:

  • "little" and "red"
  • "little" and "corvette"
  • "red" and "corvette"

are returned, and documents which contain only of the words are not.

q.alt: for empty queries

If specified, this query will be used when the main query string is not specified or blank. This parameter is also specific for the EdisMax query handler.

And even more

Of course the above mentioned fields are not all fields used in a Solr query. Much more can be done with them and there are a lot of other parameters to influence the output of Solr.

To mention just a few:


Facets are one of the strong assets of Solr. A complete description of all possibilities and settings for facets would take to far in this scope, but a number of usefull parameters are discussed her.

The Drupal Facet API module and its range of plugins have a plethora of settings and possibilites, like the Facet Pretty Paths-module or, for an example of the numerous possibilities of the Facet API, the Facet API Taxonomy Sort module.

The most important Solr parameters in relation to facets are:


Turn on the facets by using "facet=true" in the query.


The field to return facets for, can be added more than once.


This indicates the minimum amount of results for which to show a facet.

If this is set to '1', all facets items which would return documents are returned, if set to '0' a facet item for each value in the field will be returned, even if clicking on the item would return zero documents.


How to sort the facet items. Possible values are:


Sort by number of returned documents


Amounts to sorting by alphabet, or, in the Solr-wiki style: return the constraints sorted in their index order (lexicographic by indexed term). For terms in the ascii range, this will be alphabetically sorted.


The maximum amount of facet items returned, defaults to 100.


The number of facet items skipped. Defaults to '0'.

'facet.limit' and 'facet.offset' can be combined to implement paging of facet items.

Highlighting hl

Highlighting is turned on with 'hl=true' and enables highlighting the keywords in the search snippet Solr returns. Like facetting and spellchecking, highlighting is an extensive subject, see http://wiki.apache.org/solr/HighlightingParameters for more info.


According to the Solr wiki: currently the only legal value is "simple". So just use 'simple'.


Set the tags used to surround the highlight, defaults to <em> / </em> To wrap the highlight in for example bold tags, use:

hl.simple.pre=<b>&hl.simple.post=<b> Spellchecking spellcheck

Spellchecking is enabled with 'spellcheck=true'.

Because spellchecking is a complicated and language dependend process, it is not discussed here in full, see http://wiki.apache.org/solr/SpellCheckComponent for more information about the spellcheck component.

If the query for the spellchecker is given in a seperate 'spellcheck.q'-parameter like this:


this word is used for spell checking. If the 'spellcheck.q'-parameter is not set, the default 'q'-parameters is used as input for the spellchecker. Of course the word in the 'spellcheck.q' should bare a strong relation to the word in the 'q'-parameter, otherwise ununderstandable spelling suggestions wpould be given.

One can also make seperate requests to the spellchecker:


Where <word> in the 'q-parameter is the word to use as input for the spellchecker.

One important subject releated to spellchecking is the way content is analyzed before it is written into the index or read from it. See SpellCheckingAnalysis for the default settings for the spellcheck-field.

In Drupal there is a spellcheck module for Search API Search API Spellcheck which can be used for multiple search backends.


Although most of the parameters mentioned above are more or less hidden by the Drupal admin interface, a basic understanding of them, can help to understand why your Solr search does (or more usually: does not) returns the results you expected.

As said in the introduction: looking at the queries in the Tocmat log can help a lot when debugging Solr.

Categories: Elsewhere

Drupal.org frontpage posts for the Drupal planet: Announcing The Aaron Winborn Award to honor amazing community members

Planet Drupal - Tue, 24/03/2015 - 05:17

In honor of long-time Drupal contributor Aaron Winborn (see his recent Community Spotlight), whose battle with Amyotrophic lateral sclerosis (ALS) (also referred to as Lou Gehrig's Disease) is coming to an end later today, the Community Working Group, with the support of the Drupal Association, would like to announce the establishment of the Aaron Winborn Award.

This will be an annual award recognizing an individual who demonstrates personal integrity, kindness, and above-and-beyond commitment to the Drupal community. It will include a scholarship and stipend to attend DrupalCon and recognition in a plenary session at the event. Part of the award will also be donated to the special needs trust to support Aaron's family on an annual basis.

Thanks to Hans Riemenschneider for the suggestion, and the Drupal Association executive board for approving this idea and budget so quickly. We feel this award is a fitting honor to someone who gave so much to Drupal both on a technical and personal level.

Thank you so much to Aaron for sharing your personal journey with all of us. It’s been a long journey, and a difficult one. You and your family are all in our thoughts.

Front page news: Planet Drupal
Categories: Elsewhere

Russ Allbery: Review: Fukushima

Planet Debian - Tue, 24/03/2015 - 04:59

Review: Fukushima, by David Lochbaum, et al.

Author: David Lochbaum Author: Edwin Lyman Author: Susan Q. Stranahan Author: Union of Concerned Scientists Publisher: The New Press Copyright: 2014 ISBN: 1-59558-927-9 Format: Kindle Pages: 320

This is a very interesting book, and I can recommend it, but there are two things you should be aware of up-front. The packaging does not necessarily make clear what expectations you should have of it going in.

First, the subtitle (The Story of a Nuclear Disaster) should have appended to it And Its Implications for US Nuclear Power Policy. This book is very concerned with the impact of the Fukushima disaster on US policy and nuclear regulation, to the point where I think more than half of the book is about US agencies, nuclear regulatory history, and US reaction. There's nothing wrong with that, of course: the US should take a hard look at its own nuclear energy policy given the events at Fukushima, and it's a worthy topic for a book. But if you go into this book expecting a broader perspective, you will be disappointed. For example, I think the fact that France has a lot of nuclear power was mentioned maybe twice in the whole book, and French reaction was never discussed at all. There is a very detailed examination of exactly what happened at Fukushima (more on that in a moment), but most of the policy implications are examined purely from a US perspective. Even Japanese nuclear policy gets somewhat short shrift.

Second, note that the fourth listed co-author is the Union of Concerned Scientists. For those not familiar with US environmental groups, the UCS has a reputation as an anti-nuclear advocacy organization. I don't think that's entirely fair; I think the UCS's position on nuclear power is better summarized as holding that it is theoretically possible to run a nuclear power plant safely, but the actual US nuclear power industry is not very close to that standard, and it would require much tighter regulation and more investment in safety systems to reach that standard. But be aware that the authors of this book have a clear position on the adequacy of current nuclear power safety standards, namely that they aren't. And they don't try to conceal that position in this book. Personally, I prefer authors to be open about their perspective in books like this, but your mileage may vary.

There, disclaimers out of the way. I bought this book for a specific reason: I had followed some of the news coverage at the time of the earthquake and tsunami, and then (like many people, I suspect) lost track of the final outcome as the story fell out of the news and I started ignoring people who didn't understand how large the Pacific Ocean is. Now that we've had the benefit of several years of analysis and thoughtful reconstruction of events, I wanted to know what had actually happened. I'm happy to say that this book delivers quite well on that front. Roughly the first half of the book is a detailed blow-by-blow description of exactly what happened at Fukushima, at least as well as we've been able to reconstruct, told as an engrossing and dramatic narrative. There may be a little too much interleaving of reactions within the US government, which I suspect will particularly annoy non-US readers, but the level of factual detail is excellent, clear, and well-explained.

What I wasn't expecting, but was pleasantly surprised by, is that it's also a great story. There's tension, conflict, heroism, hard choices, and moral quandries, and the authors do a great job conveying factual information while still giving the reader the sense of being in the middle of the unfolding drama. They resist the urge to disclose all the results of later analysis in the middle of the story, which may provide a slightly less clear view of the disaster, but which makes the telling far more compelling. I usually read non-fiction more slowly than fiction, but Fukushima dragged me in. I found myself grabbing moments to read just another few pages.

Unfortunately, this is only about half the book. The other half is a mix of other things that won't have as broad of appeal: an analysis of the challenges of US nuclear regulation, a history of the US nuclear power industry, and a presentation of the authors' opinions about the best path forward for regulation of nuclear power in the US. Since I'm a US citizen and resident with an interest in both nuclear power and regulation of nuclear power in my country, I found this interesting, if not as engrossing as the rest of the book. But it felt a bit oddly tacked on, and I think it's a stretch to say that it's part of the story of Fukushima.

The authors try to draw that link by presenting the Japanese nuclear power industry as heavily influenced by their US counterparts, and their regulatory problems as similar to the problems in the US, but there is nowhere near enough detail about Japanese regulatory practices here to support that conclusion. I think the largest weakness, and the most obvious gap, in this book is the lack of detailed analysis of the history and players in the Japanese nuclear regulatory environment. This is an odd miss. If one is concerned about regulatory inadequacy, Japanese government policy is far more obviously part of the story of Fukushima than US policy. I can only speculate that the authors had inside sources for the US policy discussions but not for the Japanese policy discussions (and, sadly, fall back on painting with a rather broad brush and making unsupported generalizations about Japanese regulatory approaches in a few spots). The result feels like two partly-unrelated books stacked and partly shuffled together.

So, there are parts of Fukushima that are rather disappointing, particularly for non-US readers. But I still recommend it as a great detailed history of the actual incident and a summary of what we now think happened. That summary is unfortunately sketchy and still very unclear, but I don't think that's the fault of the authors. The inside of a nuclear power plant during a meltdown is a very difficult environment to measure or analyze, and there's a lot of data that we will probably never have. Some details may never be known. But what we do know, and how that knowledge unfolded, is told very well.

This is the only book-length treatment on Fukushima I've read, so I can't compare it against other books on the same topic. But it satisfied my curiousity nicely. If you have a similar curiosity, I recommend this book to your attention, although be aware of its approach and its US-centric analysis going in so that you're not surprised by a mismatch of expectations.

Rating: 8 out of 10

Categories: Elsewhere

lakshminp.com: The Drupal 8 plugin system - part 4

Planet Drupal - Tue, 24/03/2015 - 02:53

We defined what a plugin is, discussed some plugins used in core and wrote our own custom plugin previously. We shall tune it up a bit in this post.

Real world plugins have a lot more properties than the label property mentioned in our breakfast plugin. To make our plugin more "real world", we introduce 2 properties, image and ingredients. It makes more sense now to have a custom annotation for breakfast instead of using the default Plugin annotation.

How different are custom annotations from the usual Plugin annotation?

1) They convey more information about a plugin than what an @Plugin does. Here's a custom annotation for a views display plugin from search_api, taken from here.

/** * Defines a display for viewing a search's facets. * * @ingroup views_display_plugins * * @ViewsDisplay( * id = "search_api_facets_block", * title = @Translation("Facets block"), * help = @Translation("Display the search result's facets as a block."), * theme = "views_view", * register_theme = FALSE, * uses_hook_block = TRUE, * contextual_links_locations = {"block"}, * admin = @Translation("Facets block") * ) */

2) Custom annotations are a provision to document the metadata/properties used for a custom plugin.
Check out this snippet from FieldFormatter annotation code for instance:

/** * A short description of the formatter type. * * @ingroup plugin_translatable * * @var \Drupal\Core\Annotation\Translation */ public $description; /** * The name of the field formatter class. * * This is not provided manually, it will be added by the discovery mechanism. * * @var string */ public $class; /** * An array of field types the formatter supports. * * @var array */ public $field_types = array(); /** * An integer to determine the weight of this formatter relative to other * formatter in the Field UI when selecting a formatter for a given field * instance. * * @var int optional */ public $weight = NULL;

That gave us a lot of information about the plugin and its properties.
Now that you are convinced of the merits of custom annotations, let's create one.

Checkout the module code if you haven't already.

$ git clone git@github.com:badri/breakfast.git

Switch to the custom-annotation-with-properties tag.

$ git checkout -f custom-annotation-with-properties

and enable the module. The directory structure should look like this:

The new Annotation directory is of interest here. Custom annotations are defined here.

/** * A glimspe of how your breakfast looks. * * This is not provided manually, it will be added by the discovery mechanism. * * @var string */ public $image; /** * An array of igredients used to make it. * * @var array */ public $ingredients = array();

Now, the plugin definition is changed accordingly. Here's the new annotation of the idli plugin.

/** * Idly! can't imagine a south Indian breakfast without it. * * * @Breakfast( * id = "idly", * label = @Translation("Idly"), * image = "https://upload.wikimedia.org/wikipedia/commons/1/11/Idli_Sambar.JPG", * ingredients = { * "Rice Batter", * "Black lentils" * } * ) */

The other change is to inform the Plugin Manager of the new annotation we are using.
This is done in BreakfastPluginManager.php.

// The name of the annotation class that contains the plugin definition. $plugin_definition_annotation_name = 'Drupal\breakfast\Annotation\Breakfast'; parent::__construct($subdir, $namespaces, $module_handler, 'Drupal\breakfast\BreakfastInterface', $plugin_definition_annotation_name);

Let's tidy the plugin by wrapping it around an interface. Though this is purely optional, the interface tells the users of the plugin what properties it exposes.
This also allows us to define a custom function called servedWith() whose implementation is plugin specific.

With the plugin class hierarchy looking like this now:

The servedWith() is implemented differently for different plugin instances.

// Idly.php public function servedWith() { return array("Sambar", "Coconut Chutney", "Onion Chutney", "Idli podi"); } // MasalaDosa.php public function servedWith() { return array("Sambar", "Coriander Chutney"); }

We make use of the interface functions we wrote in the formatter while displaying details about the user's favorite breakfast item.

// BreakfastFormatter.php foreach ($items as $delta => $item) { $breakfast_item = \Drupal::service('plugin.manager.breakfast')->createInstance($item->value); $markup = '<h1>'. $breakfast_item->getName() . '</h1>'; $markup .= '<img src="'. $breakfast_item->getImage() .'"/>'; $markup .= '<h2>Goes well with:</h2>'. implode(", ", $breakfast_item->servedWith()); $elements[$delta] = array( '#markup' => $markup, ); }

And the user profile page now looks like this.

Derivative plugins

We have Idly plugin instance mapped to the Idly class and Uppuma instance mapped to the Uppuma class. Had all the plugin instances been mapped to a single class, we would have got derivative plugins. Derivative plugins are plugin instances derived from the same class.
They are employed under these scenarios:
1. If one plugin class per instance is an overkill.
There are times where you don't want to define a class for each plugin instance. You just say that it is an instance of a particular class that is already defined.
2. You need to dynamically define plugin instances.
The Flag module defines a different Flag Type plugin instance for different entities. The entities in a Drupal site are not known beforehand and hence we cannot define one instance each for every entity. This calls for a plugin derivative implementation.

Lets add derivative plugins to our breakfast metaphor.

$ git checkout -f plugin-derivatives

Here's a new user story for the breakfast requirement. We are going to add desserts to our menu now. All desserts are of type Sweet. So, we define a derivative plugin called Sweet which is based on breakfast.

This calls for 3 changes as shown in the new directory structure outlined below:

1) Define the Sweet plugin instance class on which all our desserts are going to be based on.

/** * A dessert or two whould be great! * * * @Breakfast( * id = "sweet", * deriver = "Drupal\breakfast\Plugin\Derivative\Sweets" * ) */ class Sweet extends BreakfastBase { public function servedWith() { return array(); } }

Note the "deriver" property in the annotation.

2) Next, we define the deriver in the Derivative directory.

/** * Sweets are dynamic plugin definitions. */ class Sweets extends DeriverBase { /** * {@inheritdoc} */ public function getDerivativeDefinitions($base_plugin_definition) { $sweets_list = drupal_get_path('module', 'breakfast') . '/sweets.yml'; $sweets = Yaml::decode(file_get_contents($sweets_list)); foreach ($sweets as $key => $sweet) { $this->derivatives[$key] = $base_plugin_definition; $this->derivatives[$key] += array( 'label' => $sweet['label'], 'image' => $sweet['image'], 'ingredients' => $sweet['ingredients'], ); } return $this->derivatives; } }

3) The derivative gets sweets info from the sweets.yml present in the module root directory. This could even be an XML/JSON or any file format which holds metadata. I've used a YAML file for consistency's sake.

mysore_pak: label: 'Mysore Pak' image: 'https://upload.wikimedia.org/wikipedia/commons/f/ff/Mysore_pak.jpg' ingredients: - Ghee - Sugar - Gram Flour jhangri: label: 'Jhangri' image: 'https://upload.wikimedia.org/wikipedia/commons/f/f8/Jaangiri.JPG' ingredients: - Urad Flour - Saffron - Ghee - Sugar

The class hierarchy for derivative plugins looks a little big bigger now.

Clear your cache and you must be able to see the sweets defined in the yaml file in the breakfast choice dropdown of the user profile.

Enjoy your dessert!

That concludes our tour of Drupal 8 plugins. Hope you liked it and learnt something. Stay tuned for the next series about services.

Categories: Elsewhere

Colan Schwartz: Drupal-specific hosting: Choose a provider from those offering comprehensive platforms

Planet Drupal - Tue, 24/03/2015 - 02:12

There are many Web hosting companies claiming their ability to host Drupal sites in the enterprise space. However, most of the time, these providers simply provide the hardware or a virtual machine (VM/VPS) with an operating system (OS) capable of hosting Drupal if you build the application stack, configure it and manage it all yourself. They may even claim that they can set all of this up for you, but they'll charge extra for the labour. They don't have a comprehensive platform where instances can be automatically deployed as needed.

What do I mean by a "platform"?

I'm considering a somewhat complex requirement: a solution that includes a fully-managed Drupal application stack with several environments where it is trivial to move code, databases and files between them.

While it seems like it would be difficult to find such a service, there are now several available. While these options may seem expensive relative to generic Web-site hosting, they'll save you immensely on labour costs, as you won't have to pay to do any of the following yourself:

  • Installation of the application stack on each environment (Dev, Staging and Prod, see below.)
  • Managing all OS software updates and security patches.
  • Managing the configuration between environments (a.k.a. Configuration Management)
  • Self-hosting and managing a deployment tool such as Aegir or creating and managing your own deployment scripts.
  • Developing, maintaining and documenting processes for your development workflow.

The environments in question are Dev (or Development, the integration server where code from all developers is combined and tested), Staging (or QA, where quality assurance testing is done with the content and configuration from Production/Live and the code from Dev) and Prod (Production/Live, your user-facing site). Typically, these three (3) server environments are set up for you, and the Drupal hosting provider provides a simple-to-use command-line (CLI) and/or Web interface to deploy code to Dev, to Staging and finally to Prod while allowing you to easily move site data (content and configuration) from Prod to Staging to Dev. Your developers then work in their own sandboxes on their laptops pushing code up through the environments, and pulling site data from them.

This allows you to significantly reduce or eliminate the resources you'll need for system administration and DevOps. You can focus your project resources on application development, which is to get your content management system up and running as quickly as possible, while leaving infrastructure issues to the experts. Even if these items are outsourced, they add additional costs to your project.

However, you may have specific requirements which preclude you from going with such a managed solution. You may have a need to keep the environments behind your corporate firewall (say to communicate securely with other internal back-end systems such as CRMs, authentication servers or other data stores) or your country/jurisdiction has strict privacy legislation preventing you from physically locating user data in regions where electronic communication is not guaranteed to be kept private. For example, in Canada, personal information can only be used for purposes that users have consented to (see PIPEDA). As a good number of the described hosting services rely on data centres in the United States, the PATRIOT Act there (which gives the US government full access to any electronic communication) could make it difficult to enforce such privacy protections.

At the time of this writing, the following Drupal hosting companies are available to provide services as described above. They're listed in no particular order.

I haven't had a chance to try all of these yet as I have clients on only some of them. It would be great if someone who's evaluated all of them were to publish a comparison.

This article, Drupal-specific hosting: Choose a provider from those offering comprehensive platforms, appeared first on the Colan Schwartz Consulting Services blog.

Categories: Elsewhere

Carl Chenet: Unverified backups are useless. Automatize the controls!

Planet Debian - Tue, 24/03/2015 - 00:00

Follow me on Identi.ca  or Twitter  or Diaspora*

Unverified backups are useless, every sysadmins know that. But manually verifying a backup means wasting time and resources. Moreover it’s boring. You should automatize it!

Charlie Chaplin Modern Times

Backup Checker is a command line software developed in Python 3.4 on GitHub (stars appreciated :) ) allowing users to verify the integrity of archives (tar, gz, bz2, lzma, zip, tree of files) and the state of the files inside an archive in order to find corruptions or intentional of accidental changes of states or removal of files inside an archive.

Backup Checker on github

The new feature of the latest version 1.4 is the control of outdated archives with the new outdated parameter. Lots of data are outdated quite fast, because they are dependent of other data, or because they are only useful in a specific context.

Hey, this database dump is 6 months old, it’s useless today!

Backup Checker now controls the expiration duration and triggers a warning if the given duration starting from the last modification of the archive (mtime) is expired. Short examples of the warning:

WARNING:root:/backups/backups-12022015.tar.gz is outdated. Was good until 01/03/15 00:00:00 – now 22/03/15 21:38:20

You won’t be surprized any more by outdated useless data in your backups.

Backup Checker also offers lots of other controls. Check the features list!

Installing Backup Checker

Backup Checker is available from PyPI using the following command:

# pip3.4 install backupchecker

It’s also available for your Debian Squeeze or Debian Wheezy. Check how to get it for your specific distributions.

What about you? How and what for do you use Backup Checker? We would be happy to get your feedbacks. The project cares about our users and the outdated feature was a awesome idea in a feature request by one of the Backup Checker user, thanks Laurent!


Categories: Elsewhere

Drupal Association News: Viva DrupalCon Latin America

Planet Drupal - Mon, 23/03/2015 - 22:51

2015 started out strong with our first DrupalCon of the year, which took place from 10-12 February in Bogota, Columbia. Nothing feels better than to bring the power of DrupalCon to a new region where attendees can revel in their love for Drupal, the community, and enjoy time together. As people listened to the Driesnote, attended sessions and sprints, and celebrated with some Tejo, we heard a lot of “this is a real Con” and “it feels so good to experience this in my own backyard”.

Sharing the gift of DrupalCon with the Latin American community was a joy for Drupal Association staff and community organizers. It wouldn’t have happened without help from Aldibier Morales, Carlos Ospina, Ivan Chaquea, Nick Vidal, and Jairo Pinzon, who helped organize the event. Conversely, it better connected the Drupal Association with this region, helping us better understand the high level of contribution as well as new ways to support this region.

263 people attended DrupalCon Latin America from 23 countries including 12 Latin American countries. 63% of attendees said that this was their first DrupalCon, which underscores why it’s so important to bring DrupalCon to new parts of the world. Attendees were primarily developers from Drupal Shops, but there was more diversity than expected. The event also attracted a higher level of beginners than expected and 14% of attendees were women, which falls between DrupalCon Europe (10% women) and DrupalCon North America (22%). Below are some demographic tables that compare DrupalCon Latin America with DrupalCon Austin.

As you can imagine, the most attended sessions were focused on Drupal 8. DrupalCon Latin America was the first event to offer translated sessions and all sessions were recorded and posted to the DrupalCon YouTube Channel. Thanks to Lingotek, 20 additional session recordings were translated, too, and can be found on Youtube.

One of the big takeaways for Drupal Association staff was finding out how many attendees contribute to Drupal. When Megan Sanicki, COO, asked in her keynote introduction presentation how many people contributed, a large number of hands went up. It explains why DrupalCon Latin America had the largest percentage of attendees attend the sprint compared to any other DrupalCon --  38.4% of attendees showed up to make a difference. Thanks to the sprint leads, YesCT, alimac, DevelCuy, jackbravo and the other 19 sprint mentors, 101 people were able to participate in the sprints.

We’re also happy that financially the event achieved its budget goals. When planning DrupalCon Latin America, we knew that hosting the event in a new region would create budget challenges. We accepted that and were willing to operate this DrupalCon at a loss. We see this as an investment in a region that will grow because DrupalCon was hosted here. Below is the high level budget and further below is a more detailed view into the expenses.

DrupalCon Latin America Budget Budget Actual Income $150,150 $104,513.74 Expenses $250,750 $188,034.40 Net Profit -$99,920 -$83,520.66

Overall, DrupalCon Latin America was a success! Session evaluations came back strong and the event received a high Net Promoter Score of 80. Also, attendees felt that they received more value than expected (see chart below).

While we hoped for larger numbers, it’s important to point out that DrupalCon Amsterdam in 2005 had about 100 attendees. When the event returned in 2014, it hosted 2,300 people. All regions have to start somewhere and DrupalCons have the power to infuse community members with a burst of energy and passion that helps the community grow. We saw this immediately after DrupalCon Latin America with the growth of Global Training Days. Last year, the region hosted 7 trainings total, but right after DrupalCon Latin America, the region hosted 10 - not even ¼ of the way into the year. Additionally, three Latin American community members nominated themselves in the Drupal Association At-Large Board Elections.

We are thrilled that we were able to bring DrupalCon to new regions of the world. Be sure and attend the closing session of DrupalCon Los Angeles to find out where we are bringing DrupalCon next.


Business (sales, marketing) Front end (design, themer) C-Level Site Builder Other (PM, Trainer, etc) Site Administrator

Job Function Austin Latin America Developer 40% 48% Business (sales, marketing) 11% 12% Front end (design, themer) 13% 10% C-Level 9% 9% Site Builder 11% 8% Other (PM, Trainer, etc) 9% 12% Site Administrator 7% 3%


How I use Drupal Austin Latin America Drupal Shop 47% 61% Site Owner 30% 12% Freelance 5% 9% Evaluating 6% 4% Hobbyist 2% 2%


Skill Level Austin Latin America Advanced 37% 40% Intermediate 39% 38% Beginner 23% 22%


SESSION STATISTICS DrupalCon Latin America: Highest Attended Sessions Count #d8rules - Web-automation with Rules in Drupal 8 87 An Overview of the Drupal 8 Plugin System 70 Drupal 8 CMI on Managed Workflow 67 Getting Content to a Phone in less than 1000ms 58 AngularJS + Drupal = Un Dúo Superheróico! 52 DevOps, por donde comenzar? 49 The Future of Commerce on Drupal 8 (and beyond) 43 I Want it All and I Want it Now: Configuration Management and CI 38 SEO for Drupal 37


DrupalCon Latin America: Youtube views (as of 3/11/2015) # of views DrupalCon Latin America 2015: Keynote by Dries Buytaert 1053 DrupalCon Latin America 2015: Keynote by Larry Garfield 546 DrupalCon Latin America 2015: The Future of Commerce on Drupal 8 (and beyond) 407 DrupalCon Latin America 2015: Drupal 8 CMI on Managed Workflow 241 DrupalCon Latin America 2015: AngularJS + Drupal = Un Dúo Superheróico! 238


  Staff Wages, Benefits, Overhead $106,621.54 Catering $11,784.76 Staff Travel & Accommodations $11,552.25 Event Planning $9,244.45 Registration Materials, Conference Supplies, Tees $8,180.90 Taxes, Fees, VAT $7,009.54 Speaker Fees, Travel Awards, Etc $6,973.16 Translation $6,772.00 IT, Wifi, Electrical $6,705.89 Archiving $5,500.00 Design $4,500.00 Conference Facility $3,013.78 Shipping $176.13 Total Expenses $188,034.4


Categories: Elsewhere

Martin-&#201;ric Racine: This and That

Planet Debian - Mon, 23/03/2015 - 20:27

I haven't blogged anything in months and figured that now might be a good time to get around that. Here it goes:

Free Software

While I occasionally upgrade the packaging of the software I maintain at Debian to keep up with best practices, my activity downsizing goes on. Simply put: I never had any ambition to become a Debian Developer. My involvement has always remained pragmatic and mostly from the perspective of packaging software that I found useful. Even then, my motivation for doing that keeps on dwindling into nothingness, because key pieces of software keep on breaking, whenever someone upstream decides to reinvent the wheel.

For instance, GNOME no longer works at all on Geode chipsets and it barely works on Nouveau chipsets. This happened as soon as GNOME 3.14 was uploaded into unstable, right before the freeze started. Then again, I wouldn't jump to a conclusion that GNOME itself might be at fault, since Plymouth also stopped working on the same two video platforms at the same time. For all we know, this could be caused by some changes in the X.Org server code. Bugs were filed, additional information was provided, but no fix has taken place.

Given how Geode and Nouveau represent 80% of my hardware investment (my Intel laptop being the sole exception), it essentially means that the upcoming Debian "stable" is useless for me. Now try and remain motivated, even just as a mere Free Software end-user. At this point, I'm done.


Finland is holding national elections this April. I still have no idea who I'll vote for this time. The guy I voted for last time has become a career politician with an inflated ego and zero connection to the average Finn's aspirations and worries. Meanwhile, two friends are standing as candidates: one who is a razor-sharp fact finder and who is a proven pragmatic decision-maker, but whose values are slightly off with mine, and one whose actions come straight from the heart but whose concept of today's Finnish reality leaves a lot to be desired.

National Defence

There's been a lot of recent articles about how former hardware and locations of the Finnish defence forces and border guards have been sold, often for peanuts, to Russian interests. In some cases, we're only talking about buildings formerly used for on-site staff accommodations. In other cases, former patrol boats and navy harbours changed hands. Now, to top it all, it appears that our north-western neighbour, Norway, has sold a former submarine base to German investors who, in turn, leased it to – you guessed it – Russian interests.

Looking at Russian actions in Ukraine, I cannot help but feel great concern that strategic locations are falling into potentially dangerous hands. Just seeing the picture of a former navy harbour with a handful of patrol boats on standby, right on the Finnish coastline, half-way between Helsinki and Turku, was a sobering experience. While the whole idea of shooting at people – even invading armies – gives me the creeps, at this point, I cannot help but start pondering whether defending this country might in fact be an occupation worth training for.


It has now been 6 years since I held my last dayjob. Since then, the only thing I've found is an unpaid training in the national bureaucracy. I've also freelanced as an actor and model, but that barely brought me pocket change, if even that. Seeing my face on posters advertising a movie I participated in last year was indeed nice, getting some media attention in connection to that too, but it hasn't lead to additional gigs. As far as I can tell, this was just my Warholian 15 minutes of fame.

However, there's a larger issue at stake. Newspapers recently published an employment statistics map for Nordic countries and the truth couldn't be more bleak: while Norway and Sweden's employment figures are nearly spotless for almost every province, those of Finland are – save for a couple of mildly successful provinces – outright catastrophic. Given this and despite feeling relatively happy living in Finland and having developed a will to defend this country from an eventual Russian assault, I've come to the conclusion that I would be better off going West, with a strong preference for Norway.

Now, the main question is, doing what? 6 years later, I have strong doubts that I would be remotely considered for any high-tech job. Besides, come to think of it, I wouldn't want any new office job. Off the top of my head, my idea of a cool job that would allow me to stay physically fit would be working as a tourist guide in Lapland. However, if Norway is anything like Finland, someone probably needs a dozen of permits of all sorts (first aid certification, C or even D class driving license, college degree in tourism, etc.) that I cannot afford. What then?

Categories: Elsewhere

DrupalCon News: Sessions and Training Opportunities Have Been Announced

Planet Drupal - Mon, 23/03/2015 - 19:40
One of the most exciting aspects of preparing for a DrupalCon is selecting the sessions that will be presented. It’s always incredibly cool and humbling to see all the great ideas that our community comes up with— and they’re all so great that making the official selections is definitely not an easy process!   
Categories: Elsewhere

Commercial Progression: Drupal 8 Release Date from Dreisy-Wan Kenobi plus Backdrop & Lift (EP7)

Planet Drupal - Mon, 23/03/2015 - 19:23

Commercial Progression presents Hooked on Drupal, “Episode 7: Dreisy-Wan Kenobi You're our only hope!".  In this latest installment, lead developers Brad Czerniak and Chris Keller are given a mysterious droid with a message from "Dreisy-Wan Kenobi" concerning the future release date of Drupal 8.  After a brief moment of reminiscing about the highlights of DrupalCamp Michigan, Brad and Chris muse about the cryptic messages that seem to predict when Drupal 8 will launch, what the fate of the Backdrop CMS will be, and what Acquia Lift actually is.  Tune in for cosmic revelations from beyond time and space.

Hooked on Drupal is available for RSS syndication here at the Commercial Progression site. Additionally, each episode is available to watch online via our YouTube channel, within the iTunes store, on SoundCloud, and now via Stitcher.

If you would like to participate as a guest or contributor, please email us at



Content Links and Related Information


Hooked on Drupal Content Team

BRAD CZERNIAK - Developer Talent

CHRIS KELLER - Developer Talent


 Podcast Subscription

Tags:  Hooked on Drupal, podcast, Drupal, Drupal 8, Backdrop, Lift, Planet Drupal
Categories: Elsewhere

Drupal governance announcements: Evolving and documenting Drupal core's structure, responsibilities, and decision-making

Planet Drupal - Mon, 23/03/2015 - 17:41

The Drupal project just turned 14 years old. There are now over 1 million known installations of Drupal, and Drupal.org has over 1 million users. Drupal 8 has over 2,700 contributors (almost three times that of Drupal 7) and over 13,000 commits so far (50% more commits per day on average than Drupal 7). I wanted to take an opportunity to reflect on our current governance structure and try to evolve the Drupal leadership team and decision-making, enable better scaling, and document both the formal and informal processes we have currently in place.

Over in the Drupal core issue queue, I've proposed an evolution to Drupal core's structure and decision-making process which documents how things are currently done, and also proposes some incremental improvements:

  1. Defines roles and responsibilities that are currently carried out by individuals within the core committer team: product managers, framework managers, and release managers. This is done to provide transparency, to help expedite decision-making, and to ensure that these roles are easier to fill in the future, as we can eliminate the requirement for core committers to be “superhuman” contributors capable of doing anything and everything, at all times.
    • This document also adds the concept of “provisional” product, framework, and release managers, without actual commit access, who work alongside the core committers until they gain the necessary experience to play a full committer role.
    • In so doing, the document also appoints two additional core committers—Alex Bronstein (effulgentsia) and Jess (xjm)—who have been playing this "provisional" role for some time now, informally.
  2. Lays out an explicit decision-making framework to make it clear who needs to be involved in what types of changes, and to what degree. This documents the process we already use, but also introduces some changes. The added transparency should make it easier for contributors who are proposing changes to direct their questions to the right people.
  3. Clearly outlines the role of subsystem maintainer (formerly component maintainer) as an active “maintenance” role: performing or organizing regular maintenance tasks: triaging the subsystem's queue(s), reviewing patches in need of review, etc. These responsibilities also come with a more formal opportunity to sign off on proposed changes that significantly affect the subsystem. The advantages to this are additional transparency, delegating and scaling responsibilities, and reducing the workload that currently falls to core committers. Going forward, subsystem maintainers who are not currently active will no longer be listed in MAINTAINERS.txt.

This document builds on ideas that have been blogged about or presented at Drupal events by many people, including Randy Fay (rfay), Larry Garfield (Crell), Cathy Theys (YesCT), Gábor Hojsty, Greg Dunlap (heyrocker), Jess (xjm), Alex Pott (alexpott), Nat Catchpole (catch), Jennifer Hodgdon (jhodgdon) and others. It has been reviewed by numerous people, including the existing core committer team. Special thanks to Angie Byron who has spent weeks helping me co-author this proposal.

Categories: Elsewhere

Drupalpress, Drupal in the Health Sciences Library at UVA: equipment booking system — content types

Planet Drupal - Mon, 23/03/2015 - 16:16

For our equipment booking system we needed two kinds of content types: reservation and equipment. Patrons would create nodes of the reservation content type. Staff would create nodes of the equipment content type(s). Because our library circulates a lot of different equipment we need a bunch of content types — one for each: Apple TV, Laptop, iPad Air, Digital Video Camera, Projector, etc. The equiment content types all need the same field structure — title, description, image, accessories and equipment category. It’s tedious creating these individually, but once you get one fully fleshed out (and the skeletons of the rest in place) then the Field Sync module will finish the job with the push of a button.

The reservation content type would be canonical for each kind of equipment. In other words, we don’t have to create a Reservation_iPad and a Reservation_Laptop and a Reservation_Projector, etc. There’s just one: Reservation. The way we accomplish this is by using an entity reference field that looks for any nodes of equipment content types.

When a patron creates a node of the reservation content type, he/she will select the equipment to be reserved in the entity reference field. This entity reference structure allows us to offer a pretty helpful feature for patrons navigating from a specific equipment page to the reservation form. An Entity Reference Prepopulate and Display Suite combination gives us a “Reserve This Item” link on equipment pages (say Apple TV – Copy 1) that sends the patron to a node/add reservation page that has that piece of equipment (in this case Apple TV – Copy 1) selected in the equipment field.

There’s good documentation out there for Entity Reference Prepopulate — check out this video. But it might be worth explaining how we built the URL that prepoluates the entity reference field. With Display Suite you can create a custom code field — we hardcode a URL to the node/add reservation form, with a token for the id of the currently viewed entity appended onto it. When the user clicks the link, the Entity Reference Prepopulate module kicks off, sees the id and builds the form with that entity referenced.

Categories: Elsewhere

Chromatic: Through the Looking-Glass: MidCamp 2015

Planet Drupal - Mon, 23/03/2015 - 14:56

photograph taken for Aaron Winborn (aaron), a long-time member of the Drupal community.

This last weekend I was fortunate to attend and speak at the second MidCamp in Chicago. It was hosted within the University of Illinois at Chicago campus and was a very well organized Drupal Camp. MidCamp drew a very diverse crowd (around 350 attendees, I believe) and had a schedule packed with solid sessions.

MidCamp becomes MADCamp

Prior to the event, plenty of Alice in Wonderland references were thrown around and there was even a “MADCamp” track, but it wasn’t entirely clear why. It was then revealed that MidCamp – tired of being the middle child – was rebranding itself as MADCamp (Midwest Area Drupal Camp). This was a creative move that added an entertaining twist to the atmosphere.


The keynote speakers were Tiffany Farriss, the CEO of Palantir.net and Drupal Association Treasurer, and Jen Lampton, a Founding Forker of Backdrop CMS and a notable member of the Drupal community.

Tiffany’s keynote, The Economics of Drupal Contribution, discussed the changes needed to make core contribution more attractive for organizations and individuals. Depending heavily on a small number individuals and their spare time leads to burnout and is not a sustainable model, she says. She also referenced a number of moves made by the Linux Foundation as ideas for helping solve those problems; including significantly shorter release cycles and a higher percentage of development being funded. Drupal 8 Accelerate is an example of an existing attempt to rethink the economics of core contribution.

Jen Lampton (photo by Marty Vernon)

Jen’s keynote was titled PHP for Everyone and gave a great overview of what PHP is and why it doesn’t deserve the bad rap it’s been given in the past. She outlined the arguments against it and rebutted them with its strengths. As she pointed out, PHP is currently involved with 80% of the web and is used by a number of large organizations like Facebook. These organizations are working hard to improve the language and each version sees significant improvements, continuing to modernize an otherwise older language.

I also enjoyed having a discussion with Jen about their experiences with Backdrop CMS thus far. After receiving some initial friction, it seems that the Drupal community has accepted and is embracing Backdrop. It will be very interesting to see the role that Backdrop plays and the audience it attracts as Drupal 8 is released and adopted.


As mentioned, the schedule was packed with great sessions. I’m thankful to have had the opportunity to present a session about organizing Features. As for those I attended, there were two themes that stuck out to me the most: automation tools and headless Drupal.

Automation Tools

Fredric Mitchell, VP of Engineering at Better Weekdays, presented on using Grunt with Drupal. Among the Drupal tasks that can be automated with Grunt, using it with Drush Make provides a very interesting build workflow. Phase2, Fredric’s previous employer, has shared their grunt-drupal-tasks tools, along with a blog post about using them.

Allan Chappell of Promet Source presented another interesting Drupal 7 build workflow using Composer, the dependency manager of choice in the PHP community. He also pointed to a number of related tools Promet has shared, such as Drupal Tangler, and demoed how Composer can be used to manage Drupal 7 core, contributed modules, their dependencies, libraries, and patches.

Jeff Geerling, a Technical Architect of Acquia, introduced everyone to Ansible, a tool we began using recently at CHROMATIC. He demonstrated Ansible using Dramble, his cluster of 6 Raspberry Pi 2 computers; a combination of web servers, a load balancer, a Redis cache server, and a MySQL server. He installed Drupal 8 on the web servers and used LEDs to help demonstrate just how easy it is to manage servers with Ansible. Impressive!

Headless Drupal Steve Persch (photo by Marty Vernon)

Off with Drupal’s Head was a panel discussion led by Steve Persch, a senior engineer at Palantir.net. The consensus of the panel was… well… that there is no consensus. While there is plenty of talk about Headless Drupal, many things remain unclear. Where to cut off the head (what Drupal should and shouldn’t be responsible for) and when to use the headless approach is still up for debate and the right answer may depend on the project. Some consider it only useful for web applications while others recommend it only when fighting Drupal to build beautiful pages or forms isn’t worth the trouble. Regardless, a key takeaway was that a headless project is no less work. While it does avoid wrestling with the Drupal theme system, introducing a JavaScript framework is no simple task.

Steve also later presented Rendering HTML with Drupal: Past, Present and Future. He anticipates that many Drupal 7 projects will undergo redesigns without a corresponding Drupal upgrade, two tasks that often currently happen at the same time. Headless Drupal makes that easier and he sees clear decoupling as the future of rendering HTML with Drupal. He doesn’t, however, anticipate that a particular JavaScript MVC framework will be permanently paired with Drupal because front-end technologies are currently evolving at a much faster rate than back-end technologies and Drupal’s release cycle.

Wrapping Up

After Steve’s session I headed to Saturday night’s social event at the Moxee Restaurant and Brewery. The night was filled with great company, food, drinks, and entertaining bouts of table shuffleboard. Getting away from our code and bright screens never fails to bring the community closer together.

MidCamp/MADCamp was another excellent opportunity to meet and catch up with the myriad of personalities and companies that comprise the Drupal community. It was also a humbling reminder of how fast things move and how much more there is to learn. I recommend attending camps to anyone that has the opportunity. Being smaller than DrupalCon (North America, at least) allows them to become a more personal experience. That being said, I hope to see everyone again at DrupalCon Los Angeles in May!

Categories: Elsewhere

undpaul: A book about Configuration Management in Drupal 8

Planet Drupal - Mon, 23/03/2015 - 13:54

In December 2013, Packt Publishing asked us to write a book about the upcoming Drupal 8. They had seen the Drupal Association survey that showed the new feature Configuration Management was the most popular topic they wanted to learn about in Drupal 8. Since we are long-time evangelists of tracking configuration changes in code, we were excited about having the opportunity to write this book, which is expected to be published in March 2015 (like this week!). It's even more exciting because Packt actually donates a portion of sales of Drupal-related books to the Drupal Association. For this to work, you need to order or pre-order directly at Packt.

Since we did so much testing with and writing about Drupal 8, we also wanted to build something with it. So we built a microsite for the book and we were surprised about how smooth this worked. It took 9 hours on a Sunday to find an appropriate HTML template, install and configure Drupal and make it look the way it does now. After the book is published, the site will get some more functionality so we can publish questions and answers as well as errata. Since it will be quite a while until Drupal 8 is actually released, we are expecting there will be a few changes and some of our code might become outdated.

You can read a sample chapter in an article on Packt's website.

What will be in the book?

Chapter 1, Understanding Configuration Management, will give you a quick overview of Configuration Management. You will learn what types of configuration exist, why managing configuration is a good idea, and how to get started with it. It will provide a look at the several ways in which configuration was managed in Drupal 7 and then show how Drupal 8 approaches the problem.

Chapter 2, Configuration Management for Administrators, provides an introduction on how to use Configuration Management for users who are not developers, but administrators of a Drupal website who want to make use of the advantages of this new feature. We will show you how to use the Configuration Management interface and how to create a copy of your website, and you will learn how to move a configuration made on one site to another site.

Chapter 3, Drupal 8's Take on Configuration Management, will show you the inner workings of the Configuration Management system in Drupal 8. You will learn about config and schema files, and read about the difference between simple configuration and configuration entities.

Chapter 4, Configuration Management API, will teach you how to get your hands dirty and learn about the Configuration Management API of Drupal 8. Here, you will dive into the Simple Configuration API and learn how configuration can be overridden. Later, you will take a closer look at how to create custom configuration entity types, and we'll also teach you about the configuration's context system.

Chapter 5, The Anatomy of Schema Files, covers schema files and explains how Drupal uses them for Configuration Management. You will learn about the structure of schema files used by Drupal and write your own schema for custom configuration.

Chapter 6, Adding Configuration Management to Your Module, will teach you how to access configuration objects and how schema files are structured in the previous chapters. (You will surely want to know how to get all this fancy stuff into your shiny new module for Drupal 8). You will learn how to include the default configuration in custom modules, how to define and use your own configuration, and how to create configuration forms.

Chapter 7, Upgrading Your Drupal 7 Variables to Drupal 8 Configuration, will show you ways to convert your Drupal 7 variables into Drupal 8 Configuration objects and how to provide an upgrade path in your modules.

Chapter 8, Managing Configuration for Multilingual Websites, allows you to build comprehensive multilingual websites in which you can display a site's content in different languages and translate the user interface. While many features were built into Drupal's core in previous versions, building multilingual sites remained a very painful task. In this chapter, we will take a look at how Drupal 7 deals with different languages on a site and how Drupal 8 is trying to fix weaknesses from previous versions.

Chapter 9, Useful Tools and Getting Help, provides a list of links and tools provided by the Drupal community; these will be useful if you reach a point where you need help when dealing with Configuration Management.

Like what you see?

Go pre-order the book directly on Packt's website, or follow us on Twitter.

drupal planet english

Categories: Elsewhere

Iztok Smolic: Fastest way to build a landing page on your Drupal site

Planet Drupal - Mon, 23/03/2015 - 13:53

Landing pages are a must-have for any web business. Every marketer will tell you that pointing ads to a home page is a waste of money. Actually, any campaign should have a dedicated landing page to maximise the the conversion. Here is the problem: setting-up landing pages in Drupal is not easy. Modules like Panels and Display Suite sure […]

The post Fastest way to build a landing page on your Drupal site appeared first on Iztok.

Categories: Elsewhere

Mario Lang: Why is Qt5 not displaying Braille?

Planet Debian - Mon, 23/03/2015 - 11:59

While evaluating the cross-platform accessibility of Qt5, I stumbled across this deficiency:

#include <QApplication> #include <QTextEdit> int main(int argv, char **args) { QApplication app(argv, args); QTextEdit textEdit; textEdit.setText(u8"\u28FF"); textEdit.show(); return app.exec(); }

(compile with -std=c++11).

On my system, this "application" does not show the correct glyph always. Sometimes, it renders a a white square with black border, i.e., the symbol for unknown glyph. However, if I invoke the same executable several times, sometimes, it renders the glyph correctly.

In other words: The glyph choosing mechansim is apparently non-deterministic!!!

UPDATE: Sune Vuorela figured out that I need to set QT_HARFBUZZ=old in the environment for this bug to go away. Apparently, harfbuzz-ng from Qt 5.3 is buggy.

Categories: Elsewhere

Jonathan Dowland: Linux music players, 2015 edition

Planet Debian - Mon, 23/03/2015 - 11:10

Now I'm back to Linux on the Desktop for my dayjob, I was slightly nervous about checking out the state of the art for Linux music players; an area I've never felt the Linux desktop was very strong on.

However for the time being I've largely side-stepped the issue by listening to BBC 6 Music for most of the day. For better or worse, I scrobble, and somebody has written a neat web app for scrobbling along to radio stations. When I want to listen to something different for a change, I've been trying out a trial of Google Play Music, for which somebody has written a Chrome extension to scrobble. On the rare occasions I listen to local music, I'm using VLC.

Google Play Music seems pretty good, but I'm not getting a lot from my trial because 6 Music is generally fantastic.

Scrobbling 6 Music has revealed a bit of a disconnect for how I use last.fm, and how website thinks you should use it. Within a day or two, my "music compability" with 6 Music was (predictably) "SUPER". Looking at my "Top artists", right near the top are 6 Music's current playlist favourites Courtney Barnett and Nadine Shah, who I can (at least) recall the songs that have been played; just below them are Young Fathers, who I cannot. A little lower are Hot Chip and Slaves: both artists who have current singles out which I enjoyed for a while, but the relentless BBC playlist policy is overdoing them and I'm inclined to switch over when they come on now. If I listen to a whole album in a given week, then the artist will likely (and rightly) be sat at the top of "last 7 days"; if I don't, then it could be something I can't even remember listening to.

Categories: Elsewhere

Web Omelette: Custom access control for Drupal 7 entities

Planet Drupal - Mon, 23/03/2015 - 09:05

Have you ever used hook_node_access() to control access to various node operations on your site? Have you found this hook unbelievably awesome? Thought to yourself, boy, I'm unstoppable now? Well, I sure did.

The problem is that Drupal 7 entities do not start and end with nodes and you may want to control access to other entities in the same way. However, there is no hook_taxonomy_term_access() or hook_fieldable_panels_pane_access() that you can use to apply this tactic. So what can you do in these cases?

The first thing is to understand that each entity implementation is different. Some have a centralised access callback for all the operations while others simply define permissions or access callbacks to be used directly inside the hook_menu() definition. And I'm sure there are also other ways of handling this but these I think are the most common.

In this article we are going to look at two entity examples: taxonomy terms (from core) and fieldable panels panes (from contrib). We will trace the access pipeline from request to response and see what we can do to intervene in this process and add our own logic. So let's begin.

Taxonomy terms

The first entity type we are going to look at is the regular taxonomy term introduced by core. It's actually pretty easy to understand how this entity type is built and what we can do in order to hook into the access pipeline.

If we take a look at taxonomy_menu(), we can quickly figure out which paths it creates to add and edit terms and what access callback and arguments are defined:

Adding terms to a vocabulary

$items['admin/structure/taxonomy/%taxonomy_vocabulary_machine_name/add'] = array( 'title' => 'Add term', 'page callback' => 'drupal_get_form', 'page arguments' => array('taxonomy_form_term', array(), 3), 'access arguments' => array('administer taxonomy'), 'type' => MENU_LOCAL_ACTION, 'file' => 'taxonomy.admin.inc', );

Editing a particular term

$items['taxonomy/term/%taxonomy_term/edit'] = array( 'title' => 'Edit', 'page callback' => 'drupal_get_form', // Pass a NULL argument to ensure that additional path components are not // passed to taxonomy_form_term() as the vocabulary machine name argument. 'page arguments' => array('taxonomy_form_term', 2, NULL), 'access callback' => 'taxonomy_term_edit_access', 'access arguments' => array(2), 'type' => MENU_LOCAL_TASK, 'weight' => 10, 'file' => 'taxonomy.admin.inc', );

By nature, viewing taxonomy term pages has to do with node access since they show a list of nodes so we will skip this aspect here and stick with just the add/edit operations.

So let's change the access callbacks and potential passed arguments inside a hook_menu_alter() implementation.

/** * Implements hook_menu_alter(). */ function demo_menu_alter(&$items) { $items['admin/structure/taxonomy/%taxonomy_vocabulary_machine_name/add']['access callback'] = 'demo_taxonomy_term_access'; $items['admin/structure/taxonomy/%taxonomy_vocabulary_machine_name/add']['access arguments'] = array(3, 'add'); $items['taxonomy/term/%taxonomy_term/edit']['access callback'] = 'demo_taxonomy_term_access'; $items['taxonomy/term/%taxonomy_term/edit']['access arguments'][] = 'edit'; }

So what happens here? We are simply replacing the access callback definition for these two menu items with our own custom function demo_taxonomy_term_access(). Additionally, we standardise a bit the arguments this function gets: $entity (either term or vocabulary object) and $op (add or edit). This covers both cases.

Now let's write our callback function:

/** * Access callback for taxonomy term add/edit operations * * @param null $entity * @param null $op * @return bool */ function demo_taxonomy_term_access($entity = null, $op = null) { if ($op === 'add') { return user_access('administer taxonomy'); } if ($op === 'edit') { return taxonomy_term_edit_access($entity); } }

Inside the function we run a check on $op and perform the access logic we want. In doing so, we make use of the $entity object that can be either the term being edited or the vocabulary to which a new term is added. In this example we are replicating exactly the intended access checks of the Taxonomy module. To add a term, a user needs to have the administer taxonomy permission while to edit one it needs either the same permission or a vocabulary specific permission (as seen inside taxonomy_term_edit_access()). Now it's up to you to include inside this logic whatever else you need. And you end up with something not so dissimilar to hook_node_access() but for taxonomy terms.

Fieldable Panels Panes (FPP)

FPP is a contributed module that creates an entity type that is primarily used inside a Panels context. Regardless of any of this though, it too exposes CRUD operations on the entities of this type. And consequently, there are access implications. So let's see how we can hook into this pipeline by starting where we did with the Taxonomy module: at fieldable_panels_panes_menu().

By looking inside this hook implementation we can find many defined paths which relate to these CRUD operations. And we also see that many of them have fieldable_panels_panes_access() as the access callback with some specific arguments passed to it.

But what does this function actually do? Nothing but taking the parameters and deferring to the controller class responsible for this entity type and its access() method. And by checking fieldable_panels_panes_entity_info(), the principle function responsible for defining the entity type, we find that this is the PanelsPaneController class. In there, we find all the logic for determining access rights for various operations.

Now that we know all this, what can we do to hook into this pipeline? We could do like before and override the hook_menu() implementation. But since there are so many menu items and the FPP class controller is already doing such a nice job, that may be counter productive. So let's instead override the entity definition and replace the class controller with one of ours that extends PanelsPaneController. In there, we then do what we want.

First, we implement hook_entity_info_alter():

/** * Implements hook_entity_info_alter(). */ function demo_entity_info_alter(&$entity_info) { $entity_info['fieldable_panels_pane']['controller class'] = 'DemoPanelsPaneController'; }

Right after this, we create a file inside our module called DemoPanelsPaneController.inc which contains the following to start with:

<?php /** * Overrides fieldable panels panes controller functionality */ class DemoPanelsPaneController extends PanelsPaneController { }

Finally, we edit the module's .info file and make sure it loads this file:

files[] = DemoPanelsPaneController.inc

Then we clear the cache. If all went well, nothing really has changed on the site in terms of functionality. However, the DemoPanelsPaneController class is being used for controlling the fieldable_panels_pane entity type. And since this one extends PanelsPaneController, all previous functionality remains. It follows to now override the access() method and include our own logic to it:

public function access($op, $entity = NULL, $account = NULL) { // $account not always full (defaults to current user) return parent::access($op, $entity, $account); }

In the example above, nothing is really changed because the logic is deferred back to the parent class. But you could add some logic in addition or instead of that depending on various contextual factors.

However, I strongly recommend/warn you to stick to the minimum amount of deviation from default is needed, and for all rest, defer back to the original logic. This is to prevent the opening of any security holes. For example, if your access() method looks like this:

public function access($op, $entity = NULL, $account = NULL) { if ($op === 'update') { return false; } }

You are indeed denying access to the edit form but now anybody can create and delete entities because there is no more check on those operations. So make sure you understand this and do not leave any loopholes. The fix in this case would be:

public function access($op, $entity = NULL, $account = NULL) { if ($op === 'update') { return false; } return parent::access($op, $entity, $account); }

If the operation is edit, the access is denied for everybody (probably not a good idea but suitable for this demo purpose). However, if the operation is delete, create or view, we defer to the logic of the parent class to handle those cases. In which case the default FPP permissions will be used.


In this article we've seen two ways we can hook into the access checking pipeline of entities in Drupal 7. We've learned that there is more than just one way of going about it depending on how the entity type in question has been defined. The purpose was to illustrate how you can approach the matter and where you need to look in order to find a solution. Hope this helps.

In Drupal var switchTo5x = true;stLight.options({"publisher":"dr-8de6c3c4-3462-9715-caaf-ce2c161a50c"});
Categories: Elsewhere

Jan Wagner: Wordpress dictionary attack

Planet Debian - Mon, 23/03/2015 - 08:23

Today early in the morning my monitoring system notified me about unusual high outgoing traffic on my hosting plattform. I traced the problem down the webserver which is also hosting this abondened website.

Looking into this with iptraf revealed that this traffic is coming only from one IP. At first I thought anybody might grabbing my Debian packages from ftp.cyconet.org. But no, it was targeting my highly sophisticated blogging plattform.

$ grep /var/log/nginx/vhosts/access_logs/blog.waja.info-access.log | tail -2 - - [23/Mar/2015:08:20:12 +0100] "POST /wp-login.php HTTP/1.0" 404 22106 "-" "-" - - [23/Mar/2015:08:20:12 +0100] "POST /wp-login.php HTTP/1.0" 404 22106 "-" "-" $ grep /var/log/nginx/vhosts/access_logs/blog.waja.info-access.log | wc -l 83676 $ grep /var/log/nginx/vhosts/access_logs/blog.waja.info-access.log | wc -l 83782 $ grep /var/log/nginx/vhosts/access_logs/blog.waja.info-access.log | grep -v wp-login.php | wc -l 0

It makes me really sad to see, that dictionary attacks are smashing with such a high power these days, even without evaluating the 404 response.

Categories: Elsewhere

Drupal core announcements: Plan to finalize the Migration system

Planet Drupal - Mon, 23/03/2015 - 05:56

If you move to Drupal 8 from Drupal 6/7, you'll be using the new core migration system.

Migrate team lead benjy has just posted a plan to finalize the migration system, a "meta" issue which outlines what work's already been completed, what's left to be done, what's blocking what, and how to get involved.

Migrations are an extremely high-impact place to throw time and energy, so if fixing Drupal 8 release blockers isn't your thing, but being able to actually move to Drupal 8 when it's ready is, please jump in and lend a hand! :D (Especially if you've worked with Migrate module in D7 contrib.)

Categories: Elsewhere


Subscribe to jfhovinne aggregator