Drupal has some powerful tools for creating and managing complex content relationships. Views relationships using Entity References across more than one content type can be used to establish multi-tiered content relationships. The results can be great, but setting up Entity References across content types with Views can be... well, complicated.
We recently added new functionality to a client’s site that involved Entity References to group content and display it dynamically. The project included a sidebar of links to all the related content that spanned three content types. This wasn’t really possible as a menu block because the links needed to contain some of the actual content. I also considered doing all of this in code but that could mean running a number of extra database queries on every page load. With Views, all of that can be easily cached.
When it comes to relationships in Views, creating a relationship from one content type to another, the number of permutations is usually small enough that it only takes a few minutes to try all of them until you get the desired output. When you try to relate content to content to content, the permutations of configuration options in Views gets a little unwieldy. And, I create and/or modify Views with complicated relationships so infrequently that I never remember how to do it. Trying to debug the query built by Views looks even scarier. Clayton Dewey was able to assist me, and we got relationships working even when needing to chain five different nodes together through their reference fields.
Here’s one portion of the sidebar:
Rather than go through the specific case with our client, I’ve setup a very general one.
Content Type A has an entity reference field to Content Type B. Content Type B has an entity reference field to Content Type C:
Here’s how that content relates.
When trying to set this up in views, relationship descriptions for entity reference fields don’t help. Is it A bridge to the Content entity that is referencing Content via field_other_content or is it A bridge to the Content entity that is referenced via field_content?
A bridge to the Content entity that is referenced via field_content means your entity is referenced by another via an entity reference field. A bridge to the Content entity that is referencing Content via field_other_content means your entity has the reference field. This will become more clear as we go through our example.
Let’s say we’re on the page for node 4, of Content Type C, and on this page we want to list all the related nodes that are of Content Type C. To do that, you need the B node referencing your C node. Then the A node referencing the B node. Next all the B nodes referenced by the A node. Finally, all C nodes referenced by the B nodes. Phew!
To do this with Views, we start with a contextual filter of the current node.
Next we setup the layers of relationships. First from that C node to the B node. Thus we’re using an entity reference relationship called A bridge to the Content entity that is referencing Content via field_c.
Each B node has an entity reference field to its C nodes. To make things easier to read we’re setting the Identifier and the Administrative title.
To get the A node we need another relationship for the entity reference field in A nodes I’ve named field_b. A bridge to the Content entity that is referencing Content via field_b. Again, the content this relationship gives you is doing the referencing. And we’re connecting it to the previous relationship.
Now we want to go back down the chain and get all B nodes. So, now we’ve got content that is referenced. So, we use A bridge to the Content entity that is referenced via field_b. Again, include the previous relationship.
To get all C nodes it’s done the same way.
Finally, to output the titles of all those C nodes, we use the correct relationship for that field. For this Views preview we use the nid of C 4, and get the following output of node titles:
You’ll want to turn on caching for the database queries and if possible, for the rendered HTML in query settings.
See, that wasn’t so hard, was it?
There is a huge amount of exciting things happening around Drupal multilingual at DrupalCon Amsterdam. This time we'll have a meetup of all the people who love localize.drupal.org. The site seriously needs people who care about it enough to devote time to maintaining and fixing bugs. I set up this BoF to gather people interesting in the well-being of the site titled We love localize.drupal.org. We need to upgrade to Drupal 7, support the whole range of new Drupal 8 APIs, drastically improve performance and then get new features going. See you there!
Sometime last year, a local Drupalist named Kelly Albrecht reached out to me. He had this idea that came out of his own personal experiences, and was further inspired by NYC Camp and Forest Mars (Forest, of course, coined the acronym NERDS). The first-ever NERD (New England Regional Developers) Summit will be held at the University of Massachusetts Amherst, September 12-14, 2014. Its seed was, in part, the Western Mass Drupal Camp. But in the interests of making Drupal better, NERDS has a bigger mission and a bigger tent.
In starting NERDS, Kelly’s “a-ha!” moment came when he was having trouble finding good developers to hire at Last Call Media. He found an amazing new addition to his development team, but she had to be urged to apply for the job because she assumed she wasn’t good enough. Now she’s one of his best developers. All of this caused him to wonder: How many more closeted tech geeks, who are usually marginalized in our industry for various reasons, could be encouraged to learn the skills they need to be great web professionals?
Kelly was also thinking about all the other things going on right now in Drupal, and in other open source communities. We all know the incredibly important post by Larry Garfield, Getting off the island. Drupal is working hard to cross-pollinate with other open source technologies. I like to call that “tech diversity”. We also know that the Drupal community is working hard to increase human diversity in its community as well. And there is increasing awareness that diversity in open source, and in tech in general, will help make our technologies, our companies, and our working lives better.
As the NERDS steering committee came together to start talking, a clear vision started to form. What if we put a huge effort into making a richer, more diverse talent pool in New England? What if we put on a free, welcoming, friendly, annual conference, where people could come together to learn with and from peers? Would that start to shift things for the better? What if we also brought Drupal, Wordpress, general web development, UX strategists, Rubyists, Python coders, hackers, and everyone together for this event, so that we could get off our islands and out of our silos and grow our communities? Would these connections then help spur momentum for ongoing collaborative, informal learning and business networking throughout the rest of the year?
So those are the main ideas behind NERDS. Kelly, myself, and a great bunch of volunteers are working to help make it a reality. Even though we’re new and a little disorganized, we’ve got momentum, passion, and the willingness to experiment and iterate. If being part of this idea appeals to you, please consider submitting a session by 8/15, volunteering to do planning tasks now or event tasks on the day(s) of the conference, or joining Acquia, Pantheon, and others in sponsoring the conference (there are many levels and options, and you can get mad karma points for sponsoring childcare). Tomorrow’s richer, better, more diverse open source communities will thank you.NERDS in a Nutshell
- NERDS is a FREE collaborative learning conference. Yes, free. It you want a t-shirt with our awesome logo and meal tickets, you can pay more. But you don’t have to. Either way, if you want to attend, you must register to let us know you’re coming.
- Friday is a learning day for beginners and for experienced devs who want to level up and try new things (Susan Buck of thewc.co, keynote); Saturday is the main sessions day (Ashe Dreyden, keynote); Sunday is sprinting day and a hackathon (Damien McKenna, keynote).
- Have kids? We will have childcare for ages 0-10, and a youth Scratch and maker track. Really.
- NERDS is a brand new, volunteer-run, sponsor-funded experiment. There’s already much momentum and over 50 excellent proposed sessions. It’s happening, and we need sponsors, volunteers to help before and during the conference, and we’re still accepting sessions until 8/15 if you’d like to submit something.
Our next board meeting is scheduled for 20 August, 2014 and in addition to our regular updates we will have a couple of great topics to discuss: DrupalCon Austin Recap and Licensing issues on Drupal.org.
Our DrupalCon Austin Recap will cover all the results of the Con, from the logistics to the sessions, to the marketing, to the finances. There is a lot of great news to report, along with some really great questions that we are going to have to answer to continue to improve the conference.
Licensing on Drupal.org has been an issue for a very long time now. With so much contributed content, so few volunteers in this area, and with little guidance, the policies we do have in place have been applied inconsistently and some issues that have been reported have never really been addressed. I pulled together a group of folks who have been working on licensing over the years and they developed a proposal for the board's consideration about how to best move forward to provide clear, consistent, and timely action for licensing issues.
Flickr photo: Kristen Pol
Last March, Facebook announced HACK, a new open source programming language for its HipHop Virtual Machine (HHVM) touted to interoperate seamlessly with PHP. Following the announcement, I was fascinated watching everything they attempted with the new release in trying to wring as much performance as possible from PHP. This gave me an idea: Although we don’t face the exact challenges that Facebook does on a daily basis, it would be interesting to see what would happen if I tried running Drupal under HHVM, the latest iteration of Facebook’s execution engine.
Official HHVM packages are only distributed for Ubuntu and Debian, but thankfully some enterprising people have packaged them up for CentOS. So with a quick stand up of a Virtual Machine and some additional packages, we were good to go. To test, we chose a fairly complex site that we developed last year, built on the usual Panels and Display Suite with significant relationships between the content. This meant that on any given page, Drupal was likely to load and render several entities beyond just the one on the page.
I took the current PHP and database for the site and stood it up on a local VM with 4 cores and 6.5 GB of memory. We did some minor optimization of FPM to set the number of servers, along with some increases to the caches in MySQL. Other than that, it’s pretty much a vanilla installation of PHP 5.3 with XCache, Percona 5.5, and nginx. The goal was to provide as much of an “apples to apples” comparison of the two interpreters without as much regard for making everything as fast possible. We then spidered the site and sampled out 1000 URLs at random and ran JMeter to generate 30 concurrent requests against the site running with HHVM and PHP-FPM. We recorded time and load on the server.PHP-FPM
Starting off, we ran the test against a cold start of the site. We cleared the Drupal cache and restarted nginx, PHP-FPM and MySQL. We then hit the home page with a single request to build the persistently cached items. As the requests started ramping up, the time to complete each one went up as expected.
What was happening on the server mirrors that.
We checked the actual processes that were generating the CPU and memory usage, in this case only pulling out mysqld since tracking each process from PHP-FHM was challenging.
MySQL was using anywhere from 40 – 60%, with a few spikes to just over 100% of one core. All the other cores were entirely used by PHP-FPM. Similarly, MySQL was using, on average, about 300 MB of memory. I’m not entirely sure what was causing the areas where processing pauses and response times spike. I saw them on all scenarios, and my guess is that they were due to some sort of I/O blocking, maybe from MySQL or nginx.
As a second test, we ran the exact same URLs against the system, clearing the Drupal cache to see if a burn in iteration for XCache and MySQL would help. Overall, it reduced the median response time by a whole 400 milliseconds and increased the throughput by a whopping 2.081 pages per minute. The response times in general were a little more choppy with fewer of the peaks and valleys from the previous run, but the results were pretty consistent.HHVM
After that we restarted the machine, switching out PHP-FPM for HHVM. Thankfully, it supports fastCGI, so it was a simple matter of altering the nginx configuration slightly. The setup was the exact same as for PHP-FPM; we made sure that the Drupal cache was cleared and restarted MySQL and nginx. We then hit the homepage with a single request and started up JMeter.
The first thing we noticed was that it completed in just under half the time, 6:28 as opposed to 13:21. Every part of the response graph was better, and both the peaks and valleys were significantly lower. The little hiccup at the very start before it dropped, was the JIT compiler running.
Looking at the server stats, those were improved as well.
HHVM runs as a single process, so we were able to capture CPU and memory usage separately for them.
Just looking at these graphs, it’s pretty easy to tell a couple of things. Namely, both CPU and memory usage for HHVM are improved over PHP-FPM. Peak memory usage for HHVM was just about 320 MB for a total system usage of approximately 20% compared to the 25 – 26% under PHP-FPM. Likewise, total CPU usage was lower with only a couple of spikes to over 90% and the median closer to 60%, compared to consistent spike to 95% CPU and a median closer to 70 – 75%.
Similar to the PHP-FPM test, we also ran the scenario against a warm start of HHVM. Like PHP-FPM, there was very little difference, only about a 200 millisecond difference in median response time.Analysis
If we are to look at the high level, HHVM compares very well to PHP-FPM.
We saw more than double the throughput and less than half the average response time. Combined with the decrease in system resources needed, it’s a pretty compelling argument to switch to HHVM.
There are some important considerations, however. The biggest is that while the HHVM team is attempting to get as close to Zend PHP as possible, they aren’t there yet. As of the latest reports, HHVM passes 99.83% of the unit tests for Drupal, but there’s no idea how whatever idiosyncrasies exist in various contributed modules will affect it. For instance, we couldn’t get GD to work at all, despite all indications that it should. Thankfully, it’s easily replaceable with ImageMagick – for most manipulations. In fact, we didn’t run into any pages during our test that failed with HHVM, but you never know what might not work until you actually run into it. In addition, the testing was done entirely on the front end. While we went through a couple of common scenarios on the administrative side, we didn’t test that thoroughly. Some PHP modules have been ported, such as APC and memcache, and there is work by third parties to add others, such as MongoDB, Ice and Redis, but many modules haven’t and probably will never be.
It’s also a moving target. Right now the HHVM team is looking at around an 8-week release cycle. Presumably there won’t be significant regressions as they move forward, but you never know. Similarly, they are targeting Ubuntu and Debian for official packages, so if you’re running Fedora or CentOS you have to either build from source or depend on a third party repository that may not be up to date.Disclaimers
The performance results shown above are from running a production site on a very much ‘non-production’ virtual machine running on MacBook Pro with a 2.3 GHz Core i7 processor. There was very little tuning on any portion of the stack to ensure best performance. Load testing was performed from a separate machine over a wireless network, albeit one that was not being used for any other purpose. The load testing did not include any wait time or requests for non-PHP assets and was not intended to simulate real usage, merely to benchmark the performance of the PHP interpreters.
DrupalCon Amsterdam is coming up in just a few weeks and it is full of opportunities to learn about and get all your questions answered when it comes to multilingual Drupal. What's better, you can get involved making things happen and learn from those implementing the features firsthand. Here are my picks:Multilingual Drupal 8 site building and programming
- There is no excuse to not attend some of the sprints at and around DrupalCon. Sprints start two days ahead of the start of the conference on Saturday the week before. And there are still sprints going on the Sunday after the conference. It is not just the last day of DrupalCon itself where you can get involved and make a difference. In fact the leads are actually focusing more on the sprint on the weekend days. Also the weekend sprints are in a really cool venue. The best way to learn is to do!
- You are looking for more of a directed guide of Drupal 8 still with the possibility to do it all hands-on? Look no further than the Drupal 8 multilingual hands-on lab presented by Aimee Degnan of Hook42 and myself from Acquia. The schedule info is a bit misleading, this session spans two timeslots and lasts two hours. Bring your laptop with Drupal 8 freshly installed!
- Dive deeper into the APIs of Drupal 8! Francesco Placella from Tag1 presents Multilingual Content in D8: a Highly Evolved Permutated API showing how to code with the new system. While not strictly multilingual, in Field API is dead. Long live Entity Field API! swentel, yched and amateescu show how the most essential content element storage system changed and this is full of multilingual support of course.
- Multilingual Site Setup and Management is a half hour session from Robert Vandenberg and Rob Bailey of Lingotek back to back with Building a multilingual & multi-country e-commerce site for luxury brands another half hour talk from Arthur Murauskas of Adyax with real life examples.
- Dagmar Muth and Urs Bucher from Amazee Labs are presenting Building a Multilingual, Multidomain Drupal Site synthesizing their experience from multiple customers.
- I proposed the Multilingual therapy (questions and answers) BoF again! Waiting experienced folks as well as everyone with questions. You'll see even if you arrive with several questions, you can answer others! There is no set agenda, just to answer all the questions!
The localize.drupal.org site seriously needs people who care about it enough to devote time to maintaining and fixing bugs. I set up one more BoF to gather people interesting in the well-being of this site titled We love localize.drupal.org. We need to upgrade to Drupal 7, support the whole range of new Drupal 8 APIs, drastically improve performance and then get new features going.
These are all the multilingual pieces that I collected. There may still be more, BoF scheduling just started and I may have missed a session or two. Let us know in the comments what other great events happen around multilingual Drupal. See you in Amsterdam!
I'm incredibly lazy, motivated, but lazy; and I hope you are too. This drives all of us to try and automate everything in life and makes Drupal developers look like rock stars of productivity while lounging in sleep pants with their morning coffee. What am I talking about? Drush, and specifically a new form of chain automation with drush that I'm going to be show-casing today. This is something I've been raving about the sandbox / dev build of on twitter for awhile now.
A few weeks ago I was ready to turn off the comments on my blog. Despite having Mollom running, I was left with a non trivial amount of spam comments to manually deal with each day. It felt like a waste of my time. I love the great comments I get. But there are always people who want to ruin the party, and for the web, it is spammers.
On its own, Mollom is not effective enough.Tags: Drupal Site buildingPlanet Drupal
On previous Drupal projects I've had the requirement to provide some sort of confirmation email to email addresses entered into an Email (module) field. These were typically fields like "Work Email" or "Secondary Email". I had written a few small custom modules to handle these cases but found myself repeating the same thing. I knew that this could be useful as a contrib but never got around to it.
I recently had a requirement to confirm email changes to the user account email (e.g. $user->mail). I went to my goto module for this situation, the Email Confirm module. But this time I decided to dive deeper into what Email Confirm was actually doing... and it looked fairly straight forward. I was hoping that I could possibly extend this module to be used with an Email field, but that ended up not being the case.
So I decided to take the plunge and create the Email Field Confirm module. Boy was I in for a ride...
The Email Confirm module only works with the User entity which happens to have the $user->data property / db table. The module makes use of this to avoid any schema changes and retains the relationship of the new email address to the user account. I had started out down a similar path but came to realize this wasn't going to work for entities other than the User entity. Node entities do not have the data property and I couldn't rely on other entity types to have it. This is the point that I realized this was not going to be a simple module.
Time to really sit down and figure out what this module needed to do.
My goal was to allow for any new email address added to an Email Field to be (optionally) confirmed. A field can be reused on multiple entity types and bundles so I need to allow for configuration at the field instance along with storing any pending email address data down to the specific entity instance (e.g. entity_id). I also noticed that the Email Confirm module would stash the new email address away until it was confirmed so I added that to my list of desired features for Email Field Confirm.
Just tell me what it does already!Features
At a high level, it met the goals I was after. A confirmation email will be sent to any new email addresses that have not already been confirmed by the same user elsewhere (e.g. another Email field) on the site. A field instance can optionally be configured to save the new email address with the entity or keep the original email address until the new one is confirmed.
This works on both single-value and multi-value Email fields, however there are some limitations with the multi-value field.
With multi-value fields it has proven more difficult to accurately identify what the original email value may be have been. I wasn't able to easily identify if the end user was changing an email address vs. just removing and adding another. It is also easy to re-order the values of a multi-value field so relying on the $delta wasn't helpful.
So with single-value fields we have the capability to retain the original email address until the new email address is confirmed. We also have the option to notify the original email address that a change has been made.
Some other notable features include:
- Ability to resend a pending non-expired confirmation email.
- Configure if the acting user (e.g. the user adding the email address) or the entity author/owner is responsible for confirming the email address.
- Hooks for email confirmation and expiration. This module actually makes use of these to handle updating / revering single value email fields to the new or original value.
- Rules integration -- with events similar to the aforementioned hooks.
- Permission to bypass email confirmation. (typically for trusted roles.)
- Permission to manually confirm any email address. (typically for administrative roles.)
- Configurable confirmation and notification emails with token replacement.
There is currently a beta release available for download on the Email Field Confirm project page. It has been pretty stable so far. Besides having more sites use the module and report back and defects or feature requests, I hope to get some automated testing (most likely Behat) in place.
My last post on field collections involved revisioning with Workbench Moderation and the issues faced. Since then the module has been developed further, but I've also come across a potential replacement: Inline Entity Form. This is a short comparison of the two modules.
As part of our day-to-day maintenance of Drupal Commons, we often assist with Drupal contributed modules that are included as part of Commons but not specific to the application, whether that means fixing bugs by writing or reviewing patches, or coordinating with other module maintainers and the Drupal Security team to help reduce the time between reported issues and security advisories.
In the fall of 2012 while doing a talk at a local conference in Los Angeles I was approached by Oscar Menjivar, founder and CEO of Teens eXploring Technology (TxT), a non-profit organization teaching inner city teenagers from South Los Angeles about technology and leadership. Oscar was looking into Drupal as a potential technology to include in the summer coding academy his organization holds every year.
With DrupalCon Amsterdam and The Prenote right around the corner, it seemed like a good time to revisit this recording from when I had the tables turned on me at DrupalCon Portland and got interviewed by Ray Saltini from Blink Reaction. He asked me some great questions about Drupal, and especially why you should come to Drupal community events like DrupalCon. See you in Amsterdam!
I'm happy to share news that Amazon has joined the Acquia family as our newest investor. This investment builds on the recent $50 million financing round that Acquia completed in May, which was led by New Enterprise Associates (NEA).
Acquia is the largest provider of Drupal infrastructure in the world. We run on more than 8,000 AWS instances and serve more than 27 billion hits a month or 333 TB of bandwidth a month. Working with AWS has been an invaluable part of our success story, and today's investment will further solidify our collaboration.
We did not disclose the amount of the investment in today's news announcement.
What do you get when you bring together thousands of diverse open source developers in Portland, Oregon? Great parties with delicious craft beer! But you also get Linux kernel hackers mingling with Docker devops engineers who are talking with PHP and Perl developers. Great minds from across the world learn from each other to make open source even better at OSCON, the annual open source conference.
This year, 16 amazing volunteers helped represent Drupal to over 4,200 open source developers at the annual OSCON trade show. At the Drupal Association booth, volunteers handed out Drupal stickers, shared how Drupal can be the solution to a broad range of web needs, and talked about the extensive contrib project ecosystem and active Drupal community. Our helpful volunteers also answered a lot of questions ranging from "What is Drupal and how much does it cost?" to "When is Drupal 8 going to be released?"
OSCON will return next year, July 20-24, and we'd love to have you join us to spread the joy of Drupal. To get involved with spreading the word, get in touch the Drupal Association via their contact form or through Twitter.
July 23, 2014
Our web development army continues its reconnaissance operations on the best DrupalCamp events! August 7-10 we have taken one more strategic point — Hungarian Drupalaton!
A world-wide famous Balaton lake has become a location for one of the biggest Drupal venues in Hungary. InternetDevels company has supplied this event by becoming its silver sponsor!
The participants had no chance for boredom — workshops and code sprints were supported by wonderful launch-breaks and exciting leisure time activities!Read more