Radium on Drupal: Retrieving Entities from MongoDB in Drupal 8

Planet Drupal - Tue, 17/05/2016 - 18:49
Normally in Drupal, all the entities are stored in the same database, which is usually MySQL or other Relational Databases. What if we have data elsewhere, and want to access them in Drupal? I often use MongoDB, and I think it is a good idea if I could directly retrieve or edit from Drupal, without having to import data into Drupal. What we could utilize is a module called External Entities. It does what its name says: allowing Drupal to treat external resources as a special type of entities. Similar modules include Remote Entity API (which doesn't have a Drupal 8 version yet). Currently, External Entities comes with two build-in drivers: REST and Wiki. We will be creating a new driver for MongoDB. Today, we will just deal with retrieving data, not modifying.
Categories: Elsewhere

Acquia Developer Center Blog: Drupal in China: Impressions after a Hackaton and a DrupalCamp

Planet Drupal - Tue, 17/05/2016 - 16:49

In the Open Source community, the best way to get involved, and learn, is to dive in, head-first, at the deep end.

This is exactly the reason why we run hackathons: to introduce new concepts to developers in an environment where they can be nurtured. By combining a hackathon with a DrupalCamp, developers get enveloped in a blanket of Drupal and Open Source for a few days, essential for rapid development of skills in new technology.

Tags: acquia drupal planet
Categories: Elsewhere

LevelTen Interactive: Drupal Commerce - How to set up Search API to search SKUs with and without dashes

Planet Drupal - Tue, 17/05/2016 - 07:00

So, you've built your Drupal Commerce site and are now setting up Search. You probably aren't going to use Drupal core's search since it won't give you enough control over what and how search results are returned. You are going to set up your own search server and index using Search API.

In this example, I am using Solr Search with Acquia Search for Search API. It doesn't matter how you have your index and server set up - as long as you can index your Commerce Products and Product displays, the custom module that I will describe shortly should work. For my setup, I have downloaded...Read more

Categories: Elsewhere

FFW Agency: Community, Connections, Console & Personalization. FFW Rocks DrupalCon.

Planet Drupal - Tue, 17/05/2016 - 05:08
Community, Connections, Console & Personalization. FFW Rocks DrupalCon. brent.bice Tue, 05/17/2016 - 03:08

A week of beignets, beer and learning Drupal! FFW let the good times roll at DrupalCon 2016 New Orleans. In case you missed out, here are four must watch presentations to jazz up your week.

Is size just a number?: Reflecting on community growth, mentoring, and where we spend our efforts

FFW’s Dave Hernandez discusses the challenges our Drupal community faces as one of the largest open source communities in the world. From capacity to growing pains, Dave tackles potential problems such as the limited number of high-end contributors and burnout.

He also discusses ways we might shift our focus, like supporting smaller, more productive events, one-on-one mentoring programs to help nurture existing contributors, and other ways to make sure we get the most out of our limited volunteer hours and efforts.

GE Energy Connections & FFW: Delivering Business Results Beyond Clicks and Conversions 

GE Energy Connections Digital Strategy Leader, Holly Bounds and FFW’s Brent Bice share stories about how Drupal not only generates traffic, conversion and increased revenue, but provides significant intrinsic value to organizations through:

  • Reduced support and maintenance
  • Improved internal collaboration
  • Increased productivity
  • And significant cost savings.

​Looking for ways to reduce support costs by 22% and save millions of dollars per year? Watch this presentation!

Writing Command Line Tools for Drupal 8 Modules

FFW’s Jesus Olivas, maintainer of Drupal Console, and other panelists discuss a new collaborative effort to unify the way that command line tools should be written for Drupal 8 modules. Jesus provides a great walk through of writing a scripting interface for your Drupal 8 modules code using an object-oriented API built on top of Symfony Console components.

Going beyond the implementation of the CLI tool, they also provide guidance on best practices for decoupled module development and the latest progress and future plans for collaborative efforts between the teams to use common implementations for some of the more complex common functions, such as site installation, configuration management, and bootstrapping, and what you can do to help make the future of command line tools easier for everyone to manage.

Web Personalization for Drupal: Your Roadmap to Get Started

Dave Sawyer, FFW’s leading personalization expert, and Acquia’s John Money deliver a fantastic presentation for getting started with personalizing user experiences in Drupal. In this presentation, Dave covers use cases, prerequisites and much more as he explains why Drupal is the best CMS to execute a personalization strategy.

Tagged with Comments
Categories: Elsewhere

ActiveLAMP: Going back to Drupal, it's fun again!

Planet Drupal - Tue, 17/05/2016 - 04:00

Actually, we never left. We didn’t stop building Drupal sites, even through the long release cycle. However, we did move our company website, activelamp.com, off of Drupal about 18 months ago. Our company site had been built on Drupal since the Drupal 4.7 days. That was back when it started to become uncool to write and maintain your own home-grown CMS. I eventually found Drupal, ditched my custom CMS, and never looked back.

Our site started on Drupal 4.7, upgraded onto Drupal 5, then Drupal 6, and also Drupal 7 all at the beginning of the release cycles of Drupal. About 18 months ago, when our site was in dire need of an update, we evaluated Drupal 8 but realized with no release date in sight, and the fact that we did not want to chase HEAD and develop on unstable API’s, we decided to go a different route and build our updated site on Jekyll, a popular static generator. It’s more fun to tinker with new technology when working on non-billable stuff, which is what we did. We brushed up on our Ruby skills and built out a Jekyll site (which is this site you’re looking at if you’re reading this blog post before Q3 of 2016).

We’re getting ready for another update to our company website and moving back to Drupal to do it. Jekyll was great, but it came with its disadvantages over something like Drupal. This post will highlight some of the advantages and disadvantages of working with Jekyll the past 18 months, as well as highlight why we’re excited to put activelamp.com on Drupal 8 in Q3 of this year.

Categories: Elsewhere

Virtuoso Performance: DrupalCon NOLA migration sprint summary

Planet Drupal - Tue, 17/05/2016 - 01:00
DrupalCon NOLA migration sprint summary

DrupalCon NOLA is over - perhaps too soon for those of us who love this great city, or perhaps just soon enough for those who have given Drupal and the city of New Orleans every last ounce of energy they had this past week.

We had a very productive week sprinting for migration - it may not be reflected in the number of commits made by the end of the extended sprints, but we'll see the fruits over the coming weeks. Our focus was on triaging the issue queue, and on diving deeply into some of the more complex outstanding issues, rather than picking the low-hanging fruit.


I previously wrote about the upfront triage the first Sunday - I prepped a Google sheet with various breakdowns of outstanding core migration system issues, and with xjm and alexpott leading the way we got about halfway through the "Migrate critical" issues. We picked up with the other half on Friday, with a few more people attending - Ryan Weal, ultimike, penyaskito, vasi, and a few more gathered around the table. Because we were working through from the oldest created issues to newest, the first round caught most of the hardest issues and the Friday session went quicker. As of Sunday morning, we were down to 18 open migrate-criticals, of which 14 were touched in the past week (11 in the previous 48 hours), so we've got momentum going.

Specific issues addressed at NOLA

I had meant to cover everything worked on this week - but I had to cut it off, for space and time. This represents fewer than half the migration issues that saw work during DrupalCon.

Migrate D6 i18n nodes

This is the biggest outstanding issue at the moment. The problem here is quite complex - the representation of translated content has completely changed in Drupal 8. In particular, translations of a node are no longer separate nodes - all translations of a specific piece of content are represented by a single node, so merging the revision histories of multiple nodes into one is... challenging. vasi has come up with a clever solution here - my current project needs this, so I had real-world data to test against. This week:

  1. vasi got the tests green
  2. I rerolled the patch against 8.1.x (the issue is targeting 8.2.x, but my client is on 8.1.x at the moment).
  3. bwinett performed a manual test - migrating from D6 to 8.1.1 (using my reroll) using the core upgrade UI - and successfully validated that the right translations and revision history were in the right place.
  4. I performed a manual test with my client's data, with less successful results. Our scenario is a custom migration into a site containing manually-entered data, so we cannot preserve node and revision IDs. It turns out that the patch is assuming source revision IDs match the destination, when it attempts to consolidate multiple translations into a single revision.
  5. For the patch to do that, it needs to perform a lookup in the migration map table given just a revision ID. The revision map table has a two-column key - revision ID and language - and the API (MigrateIdMapInterface) doesn't currently support partial lookups.
  6. So, vasi opened an issue to extend the API and implemented a solution there, which is has now been committed.

At this writing, vasi is working on the next version of the node i18n patch using the idmap extension, and I'll be testing it against my client's data.

Migrate content type language settings from Drupal 6 & 7

Speaking of i18n, penyaskito, quietone (remotely), and vasi have moved this issue forward, rerolling the previous 8.0.x patch for 8.2.x and continuing to refine it. Next step (per penyaskito): Add an assertion that language_alterable is TRUE when there's no i18n_lock_node_TYPE.

Invalid passwords after D7 to D8 migration

alexpott took this one on and produced a patch which quietone has now manually verified - next step is complete code review.

Random failure in \Drupal\migrate_drupal_ui\Tests\d7\MigrateUpgrade7Test

A major annoyance with testing on drupal.org, this test has started randomly failing. This tests the upgrade UI, which is batched, and what happens per batch is non-deterministic (affected by timing/load), so it's difficult to track down. We had some discussion about this at the closing weekend Saturday extended sprint, and alexpott has been actively pursuing it.

Refactor EntityFile and use process plugins instead

I created a patch for this, which vasi and quietone have helped refine, and benjifisher applied CodeSniffer. Next step is for me to respond to benjy's review.

Destination bundle set in destination plugin, not in process

I've created a first draft of this patch, which the testbot tells me still needs work.

Reduce/remove tight coupling of migration destination plugins

quietone provided a suggested draft for the change record, and I tweaked the patch to deprecate rather than entirely remove md_entity:field_storage_config.

Cckfield Plugins must distinguish core versions

jmuzz has continued refining this patch.

Mentored sprint

The mentored core sprint (with mentors including heddn, vegantriathlete, and willwh) on Friday also contributed. 

Fixes to docs and code style for migrate/src/*.php files

A few sprinters participated in rerolling the existing patch for 8.1.x (with some guidance from hussainweb): jeffrey.vargas, kurthill4, nJim, and The Sean.

Rename MigrationCreationTrait as it no longer creates migrations - it configures them

Participants in this patch include barbarae, hussainweb, douggreen, and hanoii.

Value should be null when is produced skip process

rakesh.gectcr submitted a reroll of this patch, cmanalansan also worked on it. This one is committed!

D6->D8: Migrating links without leading slash leads to fatal error

ancapaaron and bsnodgrass did manual testing on D6 and D7 respectively - looks like this is something we only need to worry about on D6.

Non-NOLA progress

Not everyone gets to go to every DrupalCon - while we fortunate ones were sprinting on-site, work progressed around the world...

Sql::getRowBySource doesn't adhere to MigrateIdMapInterface::getRowBySource

On the BC-breakers list but the NOLA sprinters didn't get to this - quietone working remotely rerolled the 7-month-old patch that was there, it's now ready for review.

Migrated custom block body field is hidden on form and display

rocketerrbkw identified this problem and has provided a patch - ready for review.

Last imported timestamps not set in map tables

milesw found a nasty "oops!" in a recent patch - looks good, just needs tests.

Variable to config: statistics.settings [d7]

vprocessor rerolled the existing patch and quietone reviewed.

Next steps

A lot of the work done at DrupalCon (not to mention a lot of work done previous to DrupalCon as well) needs review - let's get in there and see what we can move forward to RTBC.

I feel like we have a lot of momentum going now - but in my experience that DrupalCon momentum can dissipate pretty quickly. Let's keep it going - I'd like to propose ongoing migration sprints on a regular basis. Specifically:

  1. Each sprint lasts a calendar day - during that day (according to your local calendar), those interested in pushing D8 migration forward try to find at least one chunk of time to work on migration issues, coordinating on IRC in #drupal-migrate.
  2. For now, the theme will be Migrate-critical issues. Once we trim that list down to just a couple, we can pick different themes.
  3. Since not every day of the week is equally open for everyone, let's rotate it - I suggest doing it every 8 days. So, Monday May 23 (8 days after the last DrupalCon extended sprint), Tuesday May 31, Wednesday June 8, etc...
  4. I'll try to put out a blog post the day before each sprint day going over the status of key migrate-criticals.
Obligatory travelogue

I just couldn't keep up the daily blogging pace from mid-week on... Most of the meals have faded from my memory (and frankly I was finding better food earlier in the week than I was later), but I will leave you with my musical memories.

Wednesday night, sleep-deprived, I was heading back to my hotel on Esplanade when I ran into a couple of former Acquia colleagues on Royal. With very little arm-twisting, they convinced me to lead them to Frenchmen Street, where we picked a club pretty much at random (30°/-90°). It was shortly after 9pm and the band started up right after we got there, so I figured I'd stay for one set and still get a good night's sleep. It turned out they only played one set - and it lasted almost to 1am. And that band (Soul Company, with a somewhat different lineup than you see there) was simply awesome - high-energy jazz-funk (or funk-jazz?), very tight as they ran through covers of Herbie Hancock, Stevie Wonder, CCR/Ike & Tina, and many more. They closed with a medley of Kiss and Purple Rain, with a guest singer whose falsetto would have made Prince (R.I.P.) himself jealous. The late night pretty much wiped out my Thursday, but it was well worth it.

Saturday night I closed out the week with an actual plan - local legend John Boutté, (yes, I only knew him from Treme) was at d.b.a., so I made sure to get there good and early. This show was, unfortunately, a mixed bag - people gabbing loudly in the back of the room made it difficult to enjoy the performance, especially on the quieter tunes.

<rant>Who are these idiots who pay a cover for a show and then pay absolutely no attention to the artist!? WHY ARE YOU EVEN HERE!!!?</rant>

Sigh... Anyway, Boutté and his band (3+ piece horn section, guitar, and bass) did a fine job despite the distractions. Highlights were gentle takes on Time After Time and Hallelujah, with the emotional center being a combination of a nearly-a-cappella You've Got to Be Carefully Taught followed by a funky Southern Man.

And so, as my train departed Sunday, I left satisfied - a great week in and out of the convention center, can't wait until DrupalCon comes back here (hint, hint...)!

mikeryan Mon, 05/16/2016 - 18:00 Tags
Categories: Elsewhere

Gizra.com: 12 Things I learned at my first DrupalCon

Planet Drupal - Mon, 16/05/2016 - 23:00

For years I have been hearing about DrupalCon from Brice and Amitai. Every six months they would send me a massive group photo and challenged me to locate them among the crazy mustaches, viking helmets, and identical t-shirts. Needless to say, I failed every time and the number of people in those pictures grew every year. I’m also happy to say that that last group photo - from a week ago - included me as well (Bonus points if you can spot me).

2016 DrupalCon Group Photo.

My first Con was an overwhelmingly great experience and I learned a ton of new things. Here are the top 12:

1) Count down from 100 if you can’t fall asleep at night

DrupalCon’s sessions and keynotes are diverse and engaging. For instance, the Community Keynote by @schnitzel (Michael Schmid), was full of tips to keep your brain ready and aware, such as: Start your day doing things you dislike, drink plenty of water that will force you to take a lot of pee breaks, and play with kids to clear your mind.

The enormous amount of people and ideas exchanged in DrupalCon are so invigorating that you might find it hard to sleep at night. Try counting backwards slowly from hundred to zero. I have already put it to the test and it works - that tip alone was worth the trip.

Michael Schmid (@schnitzel) delivers the Community Keynote 2) Gator omelette for breakfast

New Orleans is a seafood and meat town. Crab, crawfish, sausages, bbq, and alligator - the Queen City is not known for its consumptions of vegetables.

Breakfast portions are huge and everything is golden-brown. But in New Orleans there is a special name for that little strip of green ground in the middle of a boulevard - “neutral ground” (thanks Trivia Night!). Perhaps they can grow fresh vegetables there!

A typical three-person breakfast. We’ve never finished it!

Continue reading…

Categories: Elsewhere

OSTraining: How to Use Pathauto in Drupal 8

Planet Drupal - Mon, 16/05/2016 - 20:10

Many modules have been in flux during the early stages of Drupal 8's development.

Few modules have changed as much as Pathauto, which the vast majority of Drupal sites use to control their URLs.

In this tutorial, I'll show you the current way to use Pathauto with your Drupal 8 site.

Categories: Elsewhere

Dries Buytaert: Cross-channel user experiences with Drupal

Planet Drupal - Mon, 16/05/2016 - 16:55

Last year around this time, I wrote that The Big Reverse of Web would force a major re-architecture of the web to bring the right information, to the right person, at the right time, in the right context. I believe that conversational interfaces like Amazon Echo are further proof that the big reverse is happening.

New user experience and distribution platforms only come along every 5-10 years, and when they do, they cause massive shifts in the web's underlying technology. The last big one was mobile, and the web industry adapted. Conversational interfaces could be the next user experience and distribution platform – just look at Amazon Echo (aka Alexa), Facebook's messenger or Microsoft's Conversation-as-a-Platform.

Today, hardly anyone questions whether to build a mobile-optimized website. A decade from now, we might be saying the same thing about optimizing digital experiences for voice or chat commands. The convenience of a customer experience will be a critical key differentiator. As a result, no one will think twice about optimizing their websites for multiple interaction patterns, including conversational interfaces like voice and chat. Anyone will be able to deliver a continuous user experience across multiple multiple channels, devices and interaction patterns. In some of these cross-channel experiences, users will never even look at a website. Conversational interfaces let users disintermediate the website by asking anything and getting instant, often personalized, results.

To prototype this future, my team at Acquia built a fully functional demo based on Drupal 8 and recorded a video of it. In the demo video below, we show a sample supermarket chain called Gourmet Market. Gourmet Market wants their customers to not only shop online using their website, but also use Echo or push notifications to do business with them.

We built an Alexa integration module to connect Alexa to the Gourmet Market site and to answer questions about sale items. For example, you can speak the command: "Alexa, ask Gourmet Market what fruits are on sale today". From there, Alexa would make a call to the Gourmet Market website, finding what is on sale in the specified category and pull only the needed information related to your ask.

On the website's side, a store manager can tag certain items as "on sale", and Alexa's voice responses will automatically and instantly reflect those changes. The marketing manager needs no expertise in programming -- Alexa composes its response by talking to Drupal 8 using web service APIs.

The demo video also shows how a site could deliver smart notifications. If you ask for an item that is not on sale, the Gourmet Market site can automatically notify you via text once the store manager tags it as "On Sale".

From a technical point of view, we've had to teach Drupal how to respond to a voice command, otherwise known as a "Skill", coming into Alexa. Alexa Skills are fairly straightforward to create. First, you specify a list of "Intents", which are basically the commands you want users to run in a way very similar to Drupal's routes. From there, you specify a list of "Utterances", or sentences you want Echo to react to that map to the Intents. In the example of Gourmet Market above, the Intents would have a command called GetSaleItems. Once the command is executed, your Drupal site will receive a webhook callback on /alexa/callback with a payload of the command and any arguments. The Alexa module for Drupal 8 will validate that the request really came from Alexa, and fire a Drupal Event that allows any Drupal module to respond.

It's exciting to think about how new user experiences and distribution platforms will change the way we build the web in the future. As I referenced in Drupalcon New Orleans keynote, the Drupal community needs to put some thought into how to design and build multichannel customer experiences. Voice assistance, chatbots or notifications are just one part of the greater equation. If you have any further thoughts on this topic, please share them in the comments.

Categories: Elsewhere

Red Route: Jumping through hoops with the golden flexbox hammer

Planet Drupal - Mon, 16/05/2016 - 16:11

I've been a big proponent of using Flexbox for a while, especially since hearing Zoe Mickley Gillenwater speaking about it at Smashing Conference Oxford 2014.

In particular, I use justify-content: space-between a lot. But one issue with it is what happens in the last row. If the number of child items doesn't divide nicely into the number of items per row, there will be a big gap between them, as you can see from this Codepen example:

See the Pen space-between by malcomio (@malcomio) on CodePen.

It can look pretty ugly, especially if the parent element is wide. One possible solution is to have the items in the last row fill the available space. But for the tiles layout on the Gallery Guide, that wouldn't work - it would make the last row items much too big. Ideally, the last row would be given a different behaviour - perhaps using a different justify-content value, perhaps using floats, but as far as I'm aware, there isn't a nice CSS way to achieve this.

The suggestion I found on StackOverflow is to add extra elements. Given that the rows are being generated by a Drupal view, we can achieve this using a preprocess function, adding dummy rows, which don't affect small screens because their height is set to zero.

Here's a Codepen example showing the idea:

See the Pen space-between with dummy rows by malcomio (@malcomio) on CodePen.

The relevant views all use the unformatted list format, so in the implementation of template_preprocess_views_view_unformatted we add a variable to say how many extra rows are needed to make it fit nicely:

define('GALL_VIEWS_ITEMS_PER_ROW', 4); /** * Implements template_preprocess_views_view_unformatted(). */ function gall_preprocess_views_view_unformatted(&$variables) { // Add dummy rows so that flexbox looks nice. $view_id = $variables['view']->id(); $tiles_views = _gall_tiles_views(); if (in_array($view_id, $tiles_views)) { $remainder = count($variables['view']->result) % GALL_VIEWS_ITEMS_PER_ROW; $rows_to_add = GALL_VIEWS_ITEMS_PER_ROW - $remainder; if ($remainder && $rows_to_add) { $variables['extra_rows'] = $rows_to_add; } } }

Once we've added this counter, we can use it to create a loop in our views-view-unformatted.html.twig template:

{% if extra_rows %} {% for i in 1..extra_rows %} {% endfor %} {% endif %}

And, as if by magic, the view rows are aligned left. Problem solved.

But maybe the problem was one of my own making. Even before I'd finished building this, I was realising that maybe it would have been easier to just use floats. To paraphrase Abraham Maslow, or perhaps Abraham Kaplan, someone who has just discovered a hammer will always be looking for nails. As always, there's another way I could have solved this, and the new way isn't always better than the old way. Having said that, I do like the way that flexbox helps to keep my margins tidy...

Tags:  Drupal Drupal 8 The Gallery Guide CSS flexbox All tags
Categories: Elsewhere

Tim Millwood: Workflow Initiative - DrupalCon New Orleans 2016

Planet Drupal - Mon, 16/05/2016 - 10:36
Last week I presented the plan for the Drupal Workflow Initiative at DrupalCon New Orleans. Please...
Categories: Elsewhere

Darryl Norris's Blog: Get Your Libraries And Breakpoint Information From The UI

Planet Drupal - Sun, 15/05/2016 - 19:31

Have you ever try to get data from your libraries and/or breakpoints in Drupal 8 ? Drupal 8 core does not provide a UI for this information.  And sometimes is nice to have the ability to know your data from the UI. Instead of trying to hunt down all that information by searching many files. For this reason, I decide to write few modules that will allow you to get some of the libraries and breakpoint information from the UI. 

Libraries UI

  • Project Page: https://www.drupal.org/project/libraries_ui
  • Module Description: This module will provide a UI to display all libraries provide by modules and themes. Once libraries_ui is been installed visit /admin/config/media/libraries_ui to get all breakpoints information.

Breakpoints UI

  • Project Page: https://www.drupal.org/project/breakpoints_ui
  • Module Description: This module will provide a UI to display all breakpoints provide by modules and themes. Once breakpoints_ui is been installed visit /admin/config/media/breakpoints_ui to get all
Categories: Elsewhere

Attiks: Dream fields for Drupal 8

Planet Drupal - Sun, 15/05/2016 - 17:13

I went to Drupalcon NOLA and was looking for a new way to contribute, since there've been a lot of discussion about the front-end part, and after reading @dries blog post Turning Drupal outside-in I started looking at the field UI. I stumbled upon the core issue titled The options under the Add field drop-down describe the data you want to store, but the user was imagining the widget it would produce and decided that the outside-in approach might be a good approach.

By Peter Droogmans

Categories: Elsewhere

Joachim's blog: What goes on in Drupal Code Builder?

Planet Drupal - Sun, 15/05/2016 - 15:13

Drupal Code Builder library is the new library which powers Module Builder. I recently split Module Builder up, so Drupal Code Builder (DCB) is the engine for generating Drupal code, while what remains in the Module Builder module is just the UI.

DCB is an extensible framework, so if you wanted to have DCB create scaffold code for a particular Drupal component or system, you can.

DCB's API is documented in the README. It's based on the idea of tasks: for example, list the hooks and plugin types that DCB has parsed from the site code, analyze the site code to update that list, or generate code for a module. There are Task classes, and you call public methods on these to do something.

The generators

Broadly, there are three things you want to do with DCB: collect and analyze data about a Drupal codebase to learn about hooks and plugin types, report on that data, and actually generate some code.

The Generate task class is where the work of creating code begins. The other task classes are all pretty simple, or at least self-contained, but the Generate task is accompanied by a large number of classes in the DrupalCodeBuilder\Generate namespace. You can see from the file names that these represent all the different components that make up generated code.

Furthermore, as well as all inheriting from BaseGenerator, there are hierarchies which can probably be deduced from the names alone, where more specialized generators inherit from generic ones. For example, we have:

  • File
    • PHPFile
    • ModuleCodeFile
    • PHPClassFile
      • Plugin
      • Service
    • API (this one's for your mymodule.api.php file)
    • YMLFile
    • Readme

and also:

  • PHPFunction
    • HookImplementation
    • HookMenu
    • HookPermission

However, these hierarchies are only about code re-use. In terms of PHP code, HookImplementation is only related to ModuleCodeFile by the common BaseGenerator base class. As the process of code generation takes place, there will be a tree of components that represents components containing each other, but it's important to remember that class inheritance doesn't come into it.

Also, while the generators in the hierarchies above clearly represent some tangible part of the code we're going to generate, some are more abstract, such as Module and Hooks. These aren't abstract in the OO sense, as they will get instantiated, but I think of them as abstract in the sense that they're not concrete and are responsible for code across different files. (Suggestions for a better word to describe them please!)

The process of generating code starts with a call to the Generate task's generateComponent() method. The host UI application (such as Module Builder module, or the Drush command) passes it an array of data that looks something like this:

[ 'base' => 'module', 'root_name' => 'mymodule, 'readable_name' => 'My module', 'hooks' => [ 'form_alter' => TRUE, 'install' => TRUE, ], 'plugins => [ 0 => [ 'plugin_type' => 'block', 'plugin_name' => 'my_plugin', 'injected_services' => [ 'current_user', ], ], ], 'settings_form' => TRUE, 'readme' => TRUE, ]

(How you get the specification for this array as a list of properties and their expected format is a detailed topic of its own, which will be covered later. For now, we're jumping in at the point where code is generated.)

Assembling components

The first job for the Generate task class is to turn this array of data into a list of generator classes for the necessary components.

This list is built up in a cascade, where each component gets to request further components, and those get to request components too, and so on, until we reach components that don't request anything. We start with the root component that was initially requested, Module, let that request components, and then repeat the process.

This is best illustrated with the AdminSettingsForm generator. This implements the requiredComponents() method to request:

  • a permission
  • a router item (on Drupal 7 that's a menu item, but in DCB we refer to these a router item whatever the core Drupal version)
  • a form

In turn, the Permission generator requests a permissions YAML file. You'll see that there are two Permission generators, each with a version suffix. The Permission7 generator requests a hook_permission() hook, which in turn requests a .module file. The Permission8 generator is somewhat simpler, and just requests a YMLFile component.

Meanwhile, the router item requests a routing.yml file on D8, and a hook_menu() on D7.

These two parts of the cascade end when we reach the various file generators: ModuleCodeFile and YMLFile don't request anything. The process that gathers all these generators works iteratively: every iteration it calls requiredComponents() on all the components the previous iteration gave it, and it only stops once an iteration produces no new components. It's safe to request the same component multiple times; in the D7 version of our example, both our hook_menu() and hook_permission() will request a ModuleCodeFile component that represents the .module file. The cascade system knows to either combine these two requests into one component, or ignore the second if it's identical to what's already been requested.

We now have a list of about a dozen or so components, each of which is an instantiated Generator object. Some represent files, some represent functions, and some like Hooks represent a more vague concept of the module 'having some hooks'. There's also the Module generator which started the whole process, whose requiredComponents() did most of the work of interpreting the given array of data.

Assembling a tree of components

The second part of the process is to assemble this flat list of components into a tree. This is where the notion of which component contains others does come into play. This is a different concept from requested components: a component can request something that it won't end up containing, as we saw with the AdminSettingsForm, which requests a permission.

The Generate task calls the containingComponent() method on each component, and this is used to assemble an array of parentage data. There's nothing fancy or recursive going on here; the tree is just an array whose keys are the identifiers of components, and whose values are arrays of the child component identifiers.

This tree now represents a structure of components where child items will produce code to be included in their parents. One part of this structure could be represented like this:

  • module
    • routing.yml
    • router item
    • permission.yml
    • permission
    • .install
    • hook_install()

Some components, such as the Hooks component, are no longer around now: their job was to be a sort of broker for other components in the requesting phase, and they're no longer involved. The root component, Module, is the root of the tree. All the files we'll be outputting are its immediate children. (This is not a file hierarchy, folders are not represented here.)

Assembling file contents

We now have everything we need to start actually generating some code. This is done in a way that's very similar to Drupal's Render API: we recurse into the tree, asking each component to return some content both from itself and its children.

So for example, the router items contribute some lines to the routing.yml file, which then turns them into YAML. The .install component, which is an instance of ModuleCodeFile, produces a @file docblock, and then gets the docblock, function declaration, and function body from the hook_install component, and glues them all together.

Finally, each file component (the immediate children of the module component in the tree) gets to say what its file name and path should be.

So the Generate task has an array of data about files, where each item has a file name, file path, and file contents. This is returned to the caller to be output to the user, or written to the filesystem. Module Builder presents the files in a form, and allows the files to be written. The Drush command outputs them to terminal and optionally writes them too.

Extending it with new components

The best way to add new things for DCB to generate is to inherit from existing basic classes. If these don’t provide the flexibility, there’s always a case to be made to give them more configurable options: for example, the AdminSettingsForm class inherits from Form, but neither of those do very little for the actual generated form class, as the work for that is mostly done by the PHPClass class.

The roadmap for DCB at the moment consists of the following:

  • Generalize the injected services functionality that’s already in Plugins, so generated Form classes and Services can have them too.
  • Add Forms as a basic component that you can request to generate. (It’s currently there only as a base for the AdminSettingsForm generator.)

And as ever, keep adding tests, keep refactoring and improving the code. But I'm always interested in hearing new ideas (or you know, better yet, patches) in the issue queue.

Categories: Elsewhere

Jonathan Dowland: Announcement

Planet Debian - Sun, 15/05/2016 - 04:11

It has become a bit traditional within Debian to announce these things in a geeky manner, so for now

# ed -p: /etc/exim4/virtual/dow.land :a holly: :fail: reserved for future use . :wq 99

More soon!

Categories: Elsewhere

Dirk Eddelbuettel: Rcpp 0.12.5: Yet another one

Planet Debian - Sun, 15/05/2016 - 03:54

The fifth update in the 0.12.* series of Rcpp has arrived on the CRAN network for GNU R a few hours ago, and was just pushed to Debian. This 0.12.5 release follows the 0.12.0 release from late July, the 0.12.1 release in September, the 0.12.2 release in November, the 0.12.3 release in January, and the 0.12.4 release in March --- making it the ninth release at the steady bi-montly release frequency. This release is one again more of a maintenance release addressing a number of small bugs, nuisances or documentation issues without adding any major new features.

Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 662 packages on CRAN depend on Rcpp for making analytical code go faster and further. That is up by almost fifty packages from the last release in late March!

And as during the last few releases, we have first-time committers. we have new first-time contributors. Sergio Marques helped to enable compilation on Alpine Linux (with its smaller libc variant). Qin Wenfeng helped adapt for Windows builds under R 3.3.0 and the long-awaited new toolchain. Ben Goodrich fixed a (possibly ancient) Rcpp Modules bug he encountered when working with rstan. Other (recurrent) contributor Dan Dillon cleaned up an issue with Nullable and strings. Rcpp Core team members Kevin and JJ took care of small build nuisance on Windows, and I added in a new helper function, updated the skeleton generator and (finally) formally deprecated loadRcppModule() for which loadModule() has been preferred since around R 2.15 or so. More details and links are below.

Changes in Rcpp version 0.12.5 (2016-05-14)
  • Changes in Rcpp API:

    • The checks for different C library implementations now also check for Musl used by Alpine Linux (Sergio Marques in PR #449).

    • Rcpp::Nullable works better with Rcpp::String (Dan Dillon in PR #453).

  • Changes in Rcpp Attributes:

    • R 3.3.0 Windows with Rtools 3.3 is now supported (Qin Wenfeng in PR #451).

    • Correct handling of dependent file paths on Windows (use winslash = "/").

  • Changes in Rcpp Modules:

    • An apparent race condition in Module loading seen with R 3.3.0 was fixed (Ben Goodrich in #461 fixing #458).

    • The (older) loadRcppModules() is now deprecated in favour of loadModule() introduced around R 2.15.1 and Rcpp 0.9.11 (PR #470).

  • Changes in Rcpp support functions:

    • The Rcpp.package.skeleton() function was again updated in order to create a DESCRIPTION file which passes R CMD check without notes. warnings, or error under R-release and R-devel (PR #471).

    • A new function compilerCheck can test for minimal g++ versions (PR #474).

Thanks to CRANberries, you can also look at a diff to the previous release. As always, even fuller details are on the Rcpp Changelog page and the Rcpp page which also leads to the downloads page, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: Elsewhere

Antoine Beaupré: Long delays posting Debian Planet Venus

Planet Debian - Sat, 14/05/2016 - 21:47

For the last few months, it seems that my posts haven't been reaching the Planet Debian aggregator correctly. I timed the last two posts and they both arrived roughly 10 days late in the feed.

SNI issues

At first, I suspected I was a victim of the SNI bug in Planet Venus: since it is still running in Python 2.7 and uses httplib2 (as opposed to, say, Requests), it has trouble with sites running under SNI. In January, there were 9 blogs with that problem on Planet. When this was discussed elsewhere in February, there were now 18, and then 21 reported in March. With everyone enabling (like me) Let's Encrypt on their website, this number is bound to grow.

I was able to reproduce the Debian Planet setup locally to do further tests and ended up sending two (unrelated) patches to the Debian bug tracker against Planet Venus, the software running Debian planet. In my local tests, I found 22 hosts with SNI problems. I also posted some pointers on how the code could be ported over to the more modern Requests and Cachecontrol modules.

Expiry issues

However, some of those feeds were working fine on philp, the host I found was running as the Planet Master. Even more strange, my own website was working fine!

INFO:planet.runner:Feed https://anarc.at/tag/debian-planet/index.rss unchanged

Now that was strange: why was my feed fetched, but noted as unchanged? For that, I found that there was a FAQ question buried down in the PlanetDebian wikipage which explicitly said that Planet obeys Expires headers diligently and will not get new content again if the headers say they did. Skeptical, I looked my own headers and, ta-da! they were way off:

$ curl -v https://anarc.at/tag/debian-planet/index.rss 2>&1 | egrep '< (Expires|Date)' < Date: Sat, 14 May 2016 19:59:28 GMT < Expires: Sat, 28 May 2016 19:59:28 GMT

So I lowered the expires timeout on my RSS feeds to 3 hours:

root@marcos:/etc/apache2# git diff diff --git a/apache2/conf-available/expires.conf b/apache2/conf-available/expires.conf index 214f3dd..a983738 100644 --- a/apache2/conf-available/expires.conf +++ b/apache2/conf-available/expires.conf @@ -3,8 +3,18 @@ # Enable expirations. ExpiresActive On - # Cache all files for 2 weeks after access (A). - ExpiresDefault A1209600 + # Cache all files 12 hours after access + ExpiresDefault "access plus 12 hours" + + # RSS feeds should refresh more often + <FilesMatch \.(rss)$> + ExpiresDefault "modification plus 4 hours" + </FilesMatch> + + # images are *less* likely to change + <FilesMatch "\.(gif|jpg|png|js|css)$"> + ExpiresDefault "access plus 1 month" + </FilesMatch> <FilesMatch \.(php|cgi)$> # Do not allow scripts to be cached unless they explicitly send cache

I also lowered the general cache expiry, except for images, Javascript and CSS.

Planet Venus maintenance

A small last word about all this: I'm surprised to see that Planet Debian is running a 6 year old software that hasn't seen a single official release yet, with local patches on top. It seems that Venus is well designed, I must give them that, but it's a little worrisome to see great software just rotting around like this.

A good "planet" site seems like a resource a lot of FLOSS communities would need: is there another "Planet-like" aggregator out there that is well maintained and more reliable? In Python, preferably.

PlanetPlanet, which Venus was forked from, is out of the question: it is even less maintained than the new fork, which itself seems to have died in 2011.

There is a discussion about the state of Venus on Github which reflects some of the concerns expressed here, as well as on the mailing list. The general consensus seems to be that everyone should switch over to Planet Pluto, which is written in Ruby.

I am not sure which planet Debian sits on - Pluto? Venus? Besides, Pluto is not even a planet anymore...

Mike check!

So this is also a test to see if my posts reach Debian Planet correctly. I suspect no one will ever see this on the top of their feeds, since the posts do get there, but with a 10 days delay and with the original date, so they are "sunk" down. The above expiration fixes won't take effect until the 10 days delay is over... But if you did see this as noise, retroactive apologies in advance for the trouble.

If you are reading this from somewhere else and wish to say hi, don't hesitate, it's always nice to hear from my readers.

Categories: Elsewhere

Blue Drop Shop: Drupal Camp Session Recordings: A Year in Review

Planet Drupal - Sat, 14/05/2016 - 17:46

It has been nearly a year since I’ve updated the status of my camp recording kits. Since DCSTL15, two other camps took me up on my proposal to sponsor my travel and hotel in exchange for me recording and posting their sessions: TCDrupal and BADCamp. And, of course, as a MidCamp organizer, that counts too. And with each those camps, I’ve iterated and learned from invaluable successes and failures.

First off, here is a link to the current kit.

With everything, each kit is still under $450. In addition, zip ties to hold the VGA to HDMI dongle tight and some gaffers tape to secure everything to the podium are needed.


At Twin Cities, I learned that, while I try, I cannot reasonably start and stop every recording in every room, especially at camps with five concurrent sessions spread over multiple floors and buildings. The amount of volunteer participation at TCDrupal is incredibly impressive. I had loads of help at my disposal, but only a few moments to outline how the kits work, so I spent a lot of time troubleshooting from room to room.

BADCamp is another camp that sprawls over a campus and is a bit looser on the room monitor support. So this time, I came armed with printed instructions at each podium for hooking up to the kit (link). I added some basic troubleshooting and my phone number. I missed about half the session starts, but speakers were mostly able to follow the instructions and run things without me. That was a huge win. Unfortunately, remembering to also start/stop the audio record was hit or miss.

By the time MidCamp rolled around, I simplified the instructions further and also set the backup audio record to just run all day, removing the failure point of missed audio. The big red button is easy and enticing. The little button on the audio recorder remote...not so much. MidCamp, with two days of four concurrent sessions was my first 100% captured camp since St. Louis.

Pain Points

There are four recurring issues with this setup:

  • VGA-only laptops
  • Recurring audio problems
  • File segmenting
  • Random projector problems

Hopefully, the time of laptops that only have VGA out is coming to an end. I've tried several different VGA-to-HDMI converters with basically no luck. And to spend hundreds of dollars or more for a fool-proof converter when modern laptops have better video output is a hard pill to swallow. I don't foresee this being a long-term problem.

The audio issues are baffling. In some cases, no audio at all is recorded with the screen capture, while other times it is sped up and choppy, hence the importance of the backup audio files from the voice recorder. But this means post-processing time which delays uploads. I intend to contact Hauppauge support, but honestly don't expect to get very far as I am using their device as it was not intended. Lastly, the capture device has a touch panel for adjusting gain and muting the audio. It is a little to easy to accidentally mute the audio.

Minor annoyance: occasionally, the recordings will split into two or more files, meaning I have to stitch them together in post.

At MidCamp for the past two years (both held at different locations on UIC campus), some of the projectors would intermittently go dark during presentations. While this has no impact on the recording, it is extremely unsettling for the presenter and annoying for the attendees. I recall this happening in some cases at Twin Cities, but not at BADCamp. So this one currently has me stumped with no good plan of resolution at this time.

Next Steps

For obvious reasons, I can't record all the sessions at all the camps. And already I have firm plans to record Twin Cities in June, St. Louis in September, and BADCamp in October. Talking to folks at Drupalcon, I also now have soft commitments with Drupal GovCon in July and Drupal Camp New Jersey in January. And other camps have reached out, but I have conflicts.

I managed to pack up a complete kit into a 10" Pelican case. This means that if I can start training some proxies and write up some detailed instructions and troubleshooting, then this solution can scale. Maybe folks won’t have experience with the post-production, but I can help with that remotely, if needed. The beauty of these kits is that with timely starts and stops and good audio, the MP4 file on the thumb drive can be uploaded as soon as it is collected.

The good news is that the more camps I can record, the more data I can collect and the more I can refine the process to make it scalable.

Stay tuned!

Categories: Elsewhere

Thadeu Lima de Souza Cascardo: Chromebook Trackpad

Planet Debian - Sat, 14/05/2016 - 17:26

Three years ago, I wanted to get a new laptop. I wanted something that could run free software, preferably without blobs, with some good amount of RAM, good battery and very light, something I could carry along with a work laptop. And I didn't want to spend too much. I don't want to make this too long, so in the end, I asked in the store anything that didn't come with Windows installed, and before I was dragged into the Macbook section, I shouted "and no Apple!". That's how I got into the Chromebook section with two options before me.

There was the Chromebook Pixel, too expensive for me, and the Samsung Chromebook, using ARM. Getting a laptop with an ARM processor was interesting for me, because I like playing with different stuff. I looked up if it would be possible to run something other than ChromeOS on it, got the sense that is, it would, and make a call. It does not have too much RAM, but it was cheap. I got an external HD to compensate for the lack of storage (only 16GB eMMC), and that was it.

Wifi does require non-free firmware to be loaded, but booting was a nice surprise. It is not perfect, but I will see if I can get to that another day.

I managed to get Fedora installed, downloading chunks of an image that I could write into the storage. After a while, I backed up home, and installed Debian using debootstrap.

Recently, after an upgrade from wheezy to jessie, things stopped working. systemd would not mount the most basic partitions and would simply stop very early in the boot process. That's a story on my backlog as well, that I plan to tell soon, since I believe this connects with supporting Debian on mobile devices.

After fixing some things, I decided to try libinput instead of synaptics for the Trackpad. The Chromebook uses a Cypress APA Trackpad. The driver was upstreamed in Linux 3.9. The Chrome OS ships with Linux 3.4, but had the driver in its branch.

After changing to libinput, I realized clicking did not work. Neither did tapping. I moved back to synaptics, and was reminded things didn't work too well with that either. I always had to enable tapping.

I have some experience with input devices. I wrote drivers, small applications reacting to some events, and some uinput userspace drivers as well. I like playing with that subsystem a lot. But I don't have too much experience with multitouch and libinput is kind of new for me too.

I got my hands on the code and found out there is libinput-debug-events. It will show you how libinput translates evdev events. I clicked on the Trackpad and got nothing but some pointer movements. I tried evtest and there were some multitouch events I didn't understand too well, but it looked like there were important events there that I thought libinput should have recognized.

I tried reading some of libinput code, but didn't get too far before I tried something else. But then, I had to let this exercise for another day. Today, I decided to do it again. Now, with some fresh eyes, I looked at the driver code. It showed support for left, right and middle buttons. But maybe my device doesn't support it, because I don't remember seeing it on evtest when clicking the Trackpad. I also understood better the other multitouch events, they were just saying how many fingers there were and what was the position of which one of them. In the case of a single finger, you still get an identifier. For better understanding of all this, reading Documentation/input/event-codes.txt and Documentation/input/multi-touch-protocol.txt is recommended.

So, in trying to answer if libinput needs to handle my devices events properly, or handle my device specially, or if the driver requires changes, or what else I can do to have a better experience with this Trackpad, things were tending to the driver and device. Then, after running evtest, I noticed a BTN_LEFT event. OK, so the device and driver support it, what is libinput doing with that? Running evtest and libinput-debug-events at the same time, I found out the problem. libinput was handling BTN_LEFT correctly, but the driver was not reporting it all the time.

By going through the driver, it looks like this is either a firmware or a hardware problem. When you get the click response, sound and everything, the drivers will not always report it. It could be pressure, eletrical contact, I can't tell for sure. But the driver does not check for anything but what the firmware has reported, so it's not the driver.

A very interesting I found out is that you can read and write the firmware. I dumped it to a file, but still could not analyze what it is. There are some commands to put the driver into some bootloader state, so maybe it's possible to play with the firmware without bricking the device, though I am not sure yet. Even then, the problem might not be fixable by just changing the firmware.

So, I left with the possibility of using tapping, which was not working with libinput. Grepping at the code, I found out by libinput documentation that tapping needs to be enabled. The libinput xorg driver supports that. Just set the Tapping option to true and that's it.

So, now I am a happy libinput user, with some of the same issues I had before with synaptics, but something you get used to. And I have a new firmware in front of me that maybe we could tackle by some reverse engineering.

Categories: Elsewhere

Drupal Association News: Hello, World! (Goodbye, Drupal Association)

Planet Drupal - Sat, 14/05/2016 - 14:06

My first day on the job, I got on an airplane and flew to Australia to attend DrupalCon Sydney. As first days on the job go, that’s gotta be up there as one of the best. It definitely set the tone for life in the Drupal community - it’s been an exciting adventure every single day. I’ve traveled around the world, worked with incredibly smart people, and learned four or five Git commands (thanks Cathy!).

So it’s not without some sadness that I share that my last day on this job will be June 3. Why am I leaving? Simply put, because I can. Drupal 8 is out and thriving. The Association is doing more and doing it better than it ever has. Now is the time for me to take a step back, eat some cake, and then find something new to jump into (after a nap, and probably some more cake).

Luckily, the Drupal community has an amazing individual ready to step in to lead the Association. I’m proud beyond words to see Megan Sanicki take on these challenges and work with you all as the next Executive Director of the Association. I know she will continue to build an Association that operates from its values for and with the Drupal community. We’ve been working together on this transition for a little while now, and I can’t wait to see what she does.

I just want to share a couple of thanks before I go. First, I’m deeply proud of the team that we have built at the Drupal Association. The Drupal Association staff are the rainbow unicorns of teams. They are honest about their opinions, but kind in their delivery. They are fierce in their loyalty to the community, and even more so in their loyalty to each other. They genuinely care about every interaction, and even when things go sideways, you can trust that their intentions were nothing but good. I learned from them. Every. Single. Day. I owe them a heck of a lot more than this thank you, but I wanted to get it out in the world. They are the best. Treat them well.

Secondly, I want to thank the dozens of community members who have gone out of their way to support me in this role. I’ll be following up personally with as many of you as I can, but I wanted to call out a few of you in particular. Angie taught me that introverts can learn to like hugs. George and Tiffany taught me to take my time and find the exact right words. Paul taught me that you can’t have too many passion projects. Donna taught me that it’s not summer everywhere. Cathy taught me Git (well, four or five commands that I can remember). There is so much generosity in Drupal.

The Association board and Megan will be working hard over the next few weeks on this transition to make sure that we continue to grow our support of the community, keep producing amazing DrupalCons, and ensure that Drupal remains the best darn CMS out there. I’ll be over here rooting for all of you. You’ll find me next to the cake.

Categories: Elsewhere


Subscribe to jfhovinne aggregator - Elsewhere