InternetDevels: DrupalTour siteseeing

Planet Drupal - Mon, 22/06/2015 - 15:38

If you want pizza you can either go to cafe or order delivery, right? So why should Drupal be different? :) We made Drupal delivery possible to any Ukrainian city with DrupalTour!

Read more
Categories: Elsewhere

Niels Thykier: Introducing dak auto-decruft

Planet Debian - Mon, 22/06/2015 - 15:11

Debian now have over 22 000 source packages and 45 500 binary packages.  To counter that, the FTP masters and I have created a dak tool to automatically remove packages from unstable!  This is also much more efficient than only removing them from testing! :)


The primary goal of the auto-decrufter is to remove a regular manual work flow from the FTP masters.  Namely, the removal of the common cases of cruft, such as “Not Built from Source” (NBS) and “Newer Version In Unstable” (NVIU).  With the auto-decrufter in place, such cruft will be automatically removed when there are no reverse dependencies left on any architecture and nothing Build-Depends on it any more.

Despite the implication in the “opening” of this post, this will in fact not substantially reduce the numbers of packages in unstable. :) Nevertheless, it is still very useful for the FTP masters, the release team and packaging Debian contributors.

The reason why the release team benefits greatly from this tool, is that almost every transition generates one piece of “NBS”-cruft.  Said piece of cruft currently must be  removed from unstable before the transition can progress into its final phase.  Until recently that removal has been 100% manual and done by the FTP masters.

The restrictions on auto-decrufter means that we will still need manual decrufts. Notably, the release team will often complete transitions even when some reverse dependencies remain on non-release architectures.  Nevertheless, it is definitely an improvement.


Omelettes and eggs: As an old saying goes “You cannot make an omelette without breaking eggs”.  Less so when the only “test suite” is production.  So here are some of the “broken eggs” caused by implementation of the auto-decrufter:

  • About 30 minutes of “dak rm” (without –no-action) would unconditionally crash.
  • A broken dinstall when “dak auto-decruft” was run without “–dry-run” for the first time.
  • A boolean condition inversion causing removals to remove the “override” for partial removals (and retain it for “full” removals).
    • Side-effect, this broke Britney a couple of times because dak now produced some “unexpected” Packages files for unstable.
  • Not to mention the “single digit bug closure” bug.

Of the 3, the boolean inversion was no doubt the worst.  By the time we had it fixed, at least 50 (unique) binary packages had lost their “override”.  Fortunately, it was possible to locate these issues using a database query and they have now been fixed.

Before I write any more non-trivial patches for dak, I will probably invest some time setting up a basic test framework for dak first.


Filed under: Debian, Release-Team
Categories: Elsewhere

Lunar: Reproducible builds: week 8 in Stretch cycle

Planet Debian - Mon, 22/06/2015 - 14:48

What happened about the reproducible builds effort this week:

Toolchain fixes

Andreas Henriksson has improved Johannes Schauer initial patch for pbuilder adding support for build profiles.

Packages fixed

The following 12 packages became reproducible due to changes in their build dependencies: collabtive, eric, file-rc, form-history-control, freehep-chartableconverter-plugin , jenkins-winstone, junit, librelaxng-datatype-java, libwildmagic, lightbeam, puppet-lint, tabble.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues but not all of them:

Patches submitted which have not made their way to the archive yet:

  • #788747 on 0xffff by Dhole: allow embedded timestamp to be set externally and set it to the time of the debian/changelog.
  • #788752 on analog by Dhole: allow embedded timestamp to be set externally and set it to the time of the debian/changelog.
  • #788757 on jacktrip by akira: remove $datetime from the documentation footer.
  • #788868 on apophenia by akira: remove $date from the documentation footer.
  • #788920 on orthanc by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #788955 on rivet by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #789040 on liblo by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #789049 on mpqc by akira: remove $datetime from the documentation footer.
  • #789071 on libxkbcommon by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #789073 on libxr by akira: remove $datetime from the documentation footer.
  • #789076 on lvtk by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #789087 on lmdb by akira: pass HTML_TIMESTAMP=NO to Doxygen.
  • #789184 on openigtlink by akira: remove $datetime from the documentation footer.
  • #789264 on openscenegraph by akira: pass HTML_TIMESTAMP=NO to Doxygen.
  • #789308 on trigger-rally-data by Mattia Rizzolo: call dh_fixperms even when overriding dh_fixperms.
  • #789396 on libsidplayfp by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #789399 on psocksxx by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #789405 on qdjango by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #789406 on qof by akira: set HTML_TIMESTAMP=NO in Doxygen configuration.
  • #789428 on qsapecng by akira: pass HTML_TIMESTAMP=NO to Doxygen.

Bugs with the ftbfs usertag are now visible on the bug graphs. This explain the recent spike. (h01ger)

Andreas Beckmann suggested a way to test building packages using the “funny paths” that one can get when they contain the full Debian package version string.

debbindiff development

Lunar started an important refactoring introducing abstactions for containers and files in order to make file type identification more flexible, enabling fuzzy matching, and allowing parallel processing.

Documentation update

Ximin Luo detailed the proposal to standardize environment variables to pass a reference source date to tools that needs one (e.g. documentation generator).

Package reviews

41 obsolete reviews have been removed, 168 added and 36 updated this week.

Some more issues affecting packages failing to build from source have been identified.


Minutes have been posted for Tuesday June 16th meeting.

The next meeting is scheduled Tuesday June 23rd at 17:00 UTC.


Lunar presented the project in French during Pas Sage en Seine in Paris. Video and slides are available.

Categories: Elsewhere

Annertech: Tough at the top (of Google) - Content Strategy, SEO, and Low Bounce Rates

Planet Drupal - Mon, 22/06/2015 - 12:44
Tough at the top (of Google) - Content Strategy, SEO, and Low Bounce Rates

Annertech is #1 on Google for a number of key search phrases and when we're not, we're usually only beaten by the Drupal Ireland page from g.d.o. (groups.drupal.org/ireland). How did we get to the top of Google? How do we stay there? Two words: hard work - but it really revolves around two other words: content strategy. Let's get down to the details.

Categories: Elsewhere

LevelTen Interactive: DrupalCon LA 2015 Video: Mediacurrent Interview

Planet Drupal - Mon, 22/06/2015 - 07:00

In last weeks' DrupalCon interview, we featured Percona.... Read more

Categories: Elsewhere

Midwestern Mac, LLC: DrupalCamp St. Louis 2015 finished, session videos available online!

Planet Drupal - Sun, 21/06/2015 - 20:28

DrupalCamp St. Louis 2015 was held this past weekend, June 20-21, 2015, at SLU LAW in downtown St. Louis. We had nine sessions and a great keynote on Saturday, and a full sprint day on Sunday.

The view coming off the elevators at SLU LAW.

Every session was recorded (slides + audio), and you can view all the sessions online:

The Camp went very well, with almost sixty participants this year! We had a great time, learned a lot together, and enjoyed some great views of downtown St. Louis (check out the picture below!), and we can't wait until next year's DrupalCamp St. Louis (to be announced)!

Categories: Elsewhere

Wim Leers: Eaton & Urbina: structured, intelligent and adaptive content

Planet Drupal - Sun, 21/06/2015 - 20:08

While walking, I started listening to Jeff Eaton’s Insert Content Here podcast episode 25: Noz Urbina Explains Adaptive Content. People must’ve looked strangely at me because I was smiling and nodding — while walking :) Thanks Jeff & Noz!

Jeff Eaton explained how the web world looks at and defines the term WYSIWYG. Turns out that in the semi-structured, non-web world that Noz comes from, WYSIWYG has a totally different interpretation. And they ended renaming it to what it really was: WYSIWOO.

Jeff also asked Noz what “adaptive content” is exactly. Adaptive content is a more specialized/advanced form of structured content, and in fact “structured content”, “intelligent content” and “adaptive content” form a hierarchy:

  • structured content
    • intelligent content
      • adaptive content

In other words, adaptive content is also intelligent and structured; intelligent content is also structured, but not all structured content is also intelligent or adaptive, nor is all intelligent content also adaptive.

Basically, intelligent content better captures the precise semantics (e.g. not a section, but a product description). Adaptive content is about using those semantics, plus additional metadata (“hints”) that content editors specify, to adapt the content to the context it is being viewed in. E.g. different messaging for authenticated versus anonymous users, or different nuances depending on how the visitor ended up on the current page (in other words: personalization).

Noz gave an excellent example of how adaptive content can be put to good use: he described how we he had arrived in Utrecht in the Netherlands after a long flight, “checked in” to Utrecht on Facebook, and then Facebook suggested to him 3 open restaurants, including cuisine type and walking distance relative to his current position. He felt like thanking Facebook for these ads — which obviously is a rare thing, to be grateful for ads!

Finally, a wonderful quote from Noz Urbina that captures the essence of content modeling:

How descriptive do we make it without making it restrictive?

If it isn’t clear by now — go listen to that podcast! It’s well worth the 38 minutes of listening. I only captured a few of the interesting points, to get more people interested and excited.1

What about adaptive & intelligent content in Drupal 8?

First, see my closely related article Drupal 8: best authoring experience for structured content?.

Second, while listening, I thought of many ways that Drupal 8 is well-prepared for intelligent & adaptive content. (Drupal already does structured content by means of Field API and the HTML tag restrictions in the body field.) Implementing intelligent & adaptive will surely require experimentation, and different sites/use cases will prefer different solutions, but:

  • An intelligent_content module for Drupal 8: allow site builders/content strategists to define custom HTML tags (e.g. <product_description>) to capture site-specific semantics. A CKEditor Widget could hugely simplify the authoring experience for creating intelligent content, by showing a specific HTML representation while editing (WYSIWOO!), thanks to HTML (Twig) templates associated with those custom HTML tags.
  • An adaptive_content module for Drupal 8: a text filter that allows any tag to be wrapped in a <adaptive_content> tag, which specifies the context in which the wrapped content should be shown/hidden.
  • The latter leads to cacheability problems, because the same content may be rendered in a multitude of different ways, but thanks to cache contexts in Drupal 8 and the fact that text filters can specify cache contexts means adaptive content that is still cacheable is perfectly possible. (This is in fact exactly what it was intended for!) cache contexts

I think that those two modules would be very interesting, useful additions to the Drupal ecosystem. If you are working on this, please let me know — I would love to help!

  1. That’s right, this is basically voluntary marketing for Jeff Eaton — you’re welcome, Jeff! 

  • Drupal
  • structured content
Categories: Elsewhere

Enrico Zini: debtags-rewrite-python3

Planet Debian - Sun, 21/06/2015 - 18:04
debtags rewritten in python3

In my long quest towards closing #540218, I have uploaded a new libept to experimental. Then I tried to build debtags on a sid+experimental chroot and the result runs but has libc's free() print existential warnings about whatevers.

At a quick glance, there are now things around like a new libapt, gcc 5 with ABI changes, and who knows what else. I figured how much time it'd take me to debug something like that, and I've used that time to rewrite debtags in python3. It took 8 hours, 5 of pleasant programming and the usual tax of another 3 of utter frustration packaging the results. I guess I gained over the risk of spending an unspecified amount of hours of just pure frustration.

So from now on debtags is going to be a pure python3 package, with dependencies on only python3-apt and python3-debian. 700 lines of python instead of several C++ files built on 4 layers of libraries. Hopefully, this is the last of the big headaches I get from hacking on this package. Also, one less package using libept.

Categories: Elsewhere

Blue Drop Shop: Camp Record Beta Test Four: DrupalCamp STL 2015

Planet Drupal - Sun, 21/06/2015 - 17:58

Following a successful MidCamp and with some new ideas how to improve the kit, I was eager to hit the road for more testing. Problem is, I'm a freelancer with a limited budget, and getting to camps comes out of my own pocket. On a lark, I tweeted the following:

Planning a #drupalcamp and need your sessions recorded? Sponsor me & I will record your sessions. Ping me! #drupal /cc @drupalstl @tcdrupal

— Kevin Thull (@kevinjthull) April 8, 2015

To my delight, both Twin Cities and St. Louis camps took me up on my offer. Of course, the stakes are even higher now, because it's no longer my own money on the line.

But I'm also feeling more confident about this solution and improve on the process with each camp. Connecting to non-HDMI-capable laptops remains the biggest challenge overall. I've added in a couple (full) DisplayPort to HDMI converters and even successfully tested a new VGA to HDMI converter that got my ancient Sony VAIO to display on my home flatscreen:

The new VGA to HDMI converter shows promise. My ancient Sony Vaio WinXP laptop just connected! #drupalcamp pic.twitter.com/PXb0kBvsCl

— Kevin Thull (@kevinjthull) June 16, 2015

And at DrupalCamp STL I finally got the 100% success rate that I've been shooting for! Three sessions needed fixing in post, but overall, this camp went very smoothly. A huge bonus was the fact that the two rooms were next to each other, minimizing the distance to cover when trying to coordinate laptop hookups and verify timely starts and stops of the records.

Twin Cities is next week, with a much more challenging schedule: five concurrent sessions across two buildings and multiple floors. My Fitbit will likely hit a new high. That, and I need to finally get down to some documentation and podium signage. It's time to share the knowledge I've gained and get more hands and minds involved.

And now for the learnings from DCSTL:

  • swapping thumb drives throughout the day means recordings can be posted during camp
  • well-timed presenter starts/stops means no trimming, which means more recordings can be posted during camp
  • one room had screen flicker and setting the PVR resolution to 1080 helped (typically, the resolution needs to come down to 720 for this, as well as fixing color shifts)
  • having extra SD cards means bad audio can be fixed during down times, which means more recordings can be posted during camp
  • power strips at the podium shouldn't be assumed, and the powered USB hub and voice recorder both have short plugs
  • never plug the powered usb into the laptop, because that can kill your record if resolution changes or the laptop goes to sleep
  • taping down individual components means less cord chaos throughout the day
  • access to ethernet port with a reasonably large pipe going up will get videos posted faster
Categories: Elsewhere

Steve Kemp: We're all about storing objects

Planet Debian - Sun, 21/06/2015 - 02:00

Recently I've been experimenting with camlistore, which is yet another object storage system.

Camlistore gains immediate points because it is written in Go, and is a project initiated by Brad Fitzpatrick, the creator of Perlbal, memcached, and Livejournal of course.

Camlistore is designed exactly how I'd like to see an object storage-system - each server allows you to:

  • Upload a chunk of data, getting an ID in return.
  • Download a chunk of data, by ID.
  • Iterate over all available IDs.

It should be noted more is possible, there's a pretty web UI for example, but I'm simplifying. Do your own homework :)

With those primitives you can allow a client-library to upload a file once, then in the background a bunch of dumb servers can decide amongst themselves "Hey I have data with ID:33333 - Do you?". If nobody else does they can upload a second copy.

In short this kind of system allows the replication to be decoupled from the storage. The obvious risk is obvious though: if you upload a file the chunks might live on a host that dies 20 minutes later, just before the content was replicated. That risk is minimal, but valid.

There is also the risk that sudden rashes of uploads leave the system consuming all the internal-bandwith constantly comparing chunk-IDs, trying to see if data is replaced that has been copied numerous times in the past, or trying to play "catch-up" if the new-content is larger than the replica-bandwidth. I guess it should possible to detect those conditions, but they're things to be concerned about.

Anyway the biggest downside with camlistore is documentation about rebalancing, replication, or anything other than simple single-server setups. Some people have blogged about it, and I got it working between two nodes, but I didn't feel confident it was as robust as I wanted it to be.

I have a strong belief that Camlistore will become a project of joy and wonder, but it isn't quite there yet. I certainly don't want to stop watching it :)

On to the more personal .. I'm all about the object storage these days. Right now most of my objects are packed in a collection of boxes. On the 6th of next month a shipping container will come pick them up and take them to Finland.

For pretty much 20 days in a row we've been taking things to the skip, or the local charity-shops. I expect that by the time we've relocated the amount of possesions we'll maintain will be at least a fifth of our current levels.

We're working on the general rule of thumb: "If it is possible to replace an item we will not take it". That means chess-sets, mirrors, etc, will not be carried. DVDs, for example, have been slashed brutally such that we're only transferring 40 out of a starting collection of 500+.

Only personal, one-off, unique, or "significant" items will be transported. This includes things like personal photographs, family items, and similar. Clothes? Well I need to take one jacket, but more can be bought. The only place I put my foot down was books. Yes I'm a kindle-user these days, but I spent many years tracking down some rare volumes, and though it would be possible to repeat that effort I just don't want to.

I've also decided that I'm carrying my complete toolbox. Some of the tools I took with me when I left home at 18 have stayed with me for the past 20+ years. I don't need this specific crowbar, or axe, but I'm damned if I'm going to lose them now. So they stay. Object storage - some objects are more important than they should be!

Categories: Elsewhere

Joachim Breitner: Running circle-packing in the Browser, now using GHCJS

Planet Debian - Sat, 20/06/2015 - 22:50

Quite a while ago, I wrote a small Haskell library called circle-packing to pack circles in a tight arrangement. Back then, I used the Haskell to JavaScript compiler fay to create a pretty online demo of that library, and shortly after, I create the identical demo using haste (another Haskell to JavaScript compiler).

The main competitor of these two compilers, and the most promising one, is GHCJS. Back then, it was too annoying to install. But after two years, things have changed, and it only takes a few simple commands to get GHCJS running, so I finally created the circle packing demo in a GHCJS variant.

Quick summary: Cabal integration is very good (like haste, but unline fay), interfacing JavaScript is nice and easy (like fay, but unlike haste), and a quick check seems to indicate that it is faster than either of these two. I should note that I did not update the other two demos, so they represent the state of fay and haste back then, respectively.

With GHCJS now available at my fingertips, maybe I will produce some more Haskell to be run in your browser. For example, I could port FrakView, a GUI program to render, expore and explain iterated function systems, from GTK to HTML.

Categories: Elsewhere

Russell Coker: BTRFS Status June 2015

Planet Debian - Sat, 20/06/2015 - 06:47

The version of btrfs-tools in Debian/Jessie is incapable of creating a filesystem that can be mounted by the kernel in Debian/Wheezy. If you want to use a BTRFS filesystem on Jessie and Wheezy (which isn’t uncommon with removable devices) the only options are to use the Wheezy version of mkfs.btrfs or to use a Jessie kernel on Wheezy. I recently got bitten by this issue when I created a BTRFS filesystem on a removable device with a lot of important data (which is why I wanted metadata duplication and checksums) and had to read it on a server running Wheezy. Fortunately KVM in Wheezy works really well so I created a virtual machine to read the disk. Setting up a new KVM isn’t that difficult, but it’s not something I want to do while a client is anxiously waiting for their data.

BTRFS has been working well for me apart from the Jessie/Wheezy compatability issue (which was an annoyance but didn’t stop me doing what I wanted). I haven’t written a BTRFS status report for a while because everything has been OK and there has been nothing exciting to report.

I regularly get errors from the cron jobs that run a balance supposedly running out of free space. I have the cron jobs due to past problems with BTRFS running out of metadata space. In spite of the jobs often failing the systems keep working so I’m not too worried at the moment. I think this is a bug, but there are many more important bugs.

Linux kernel version 3.19 was the first version to have working support for RAID-5 recovery. This means version 3.19 was the first version to have usable RAID-5 (I think there is no point even having RAID-5 without recovery). It wouldn’t be prudent to trust your important data to a new feature in a filesystem. So at this stage if I needed a very large scratch space then BTRFS RAID-5 might be a viable option but for anything else I wouldn’t use it. BTRFS still has had little performance optimisation, while this doesn’t matter much for SSD and for single-disk filesystems for a RAID-5 of hard drives that would probably hurt a lot. Maybe BTRFS RAID-5 would be good for a scratch array of SSDs. The reports of problems with RAID-5 don’t surprise me at all.

I have a BTRFS RAID-1 filesystem on 2*4TB disks which is giving poor performance on metadata, simple operations like “ls -l” on a directory with ~200 subdirectories takes many seconds to run. I suspect that part of the problem is due to the filesystem being written by cron jobs with files accumulating over more than a year. The “btrfs filesystem” command (see btrfs-filesystem(8)) allows defragmenting files and directory trees, but unfortunately it doesn’t support recursively defragmenting directories but not files. I really wish there was a way to get BTRFS to put all metadata on SSD and all data on hard drives. Sander suggested the following command to defragment directories on the BTRFS mailing list:

find / -xdev -type d -execdir btrfs filesystem defrag -c {} +

Below is the output of “zfs list -t snapshot” on a server I run, it’s often handy to know how much space is used by snapshots, but unfortunately BTRFS has no support for this.

NAME USED AVAIL REFER MOUNTPOINT hetz0/be0-mail@2015-03-10 2.88G – 387G – hetz0/be0-mail@2015-03-11 1.12G – 388G – hetz0/be0-mail@2015-03-12 1.11G – 388G – hetz0/be0-mail@2015-03-13 1.19G – 388G –

Hugo pointed out on the BTRFS mailing list that the following command will give the amount of space used for snapshots. $SNAPSHOT is the name of a snapshot and $LASTGEN is the generation number of the previous snapshot you want to compare with.

btrfs subvolume find-new $SNAPSHOT $LASTGEN | awk '{total = total + $7}END{print total}'

One upside of the BTRFS implementation in this regard is that the above btrfs command without being piped through awk shows you the names of files that are being written and the amounts of data written to them. Through casually examining this output I discovered that the most written files in my home directory were under the “.cache” directory (which wasn’t exactly a surprise).

Now I am configuring workstations with a separate subvolume for ~/.cache for the main user. This means that ~/.cache changes don’t get stored in the hourly snapshots and less disk space is used for snapshots.


My observation is that things are going quite well with BTRFS. It’s more than 6 months since I had a noteworthy problem which is pretty good for a filesystem that’s still under active development. But there are still many systems I run which could benefit from the data integrity features of ZFS and BTRFS that don’t have the resources to run ZFS and need more reliability than I can expect from an unattended BTRFS system.

At this time the only servers I run with BTRFS are located within a reasonable drive from my home (not the servers in Germany and the US) and are easily accessible (not the embedded systems). ZFS is working well for some of the servers in Germany. Eventually I’ll probably run ZFS on all the hosted servers in Germany and the US, I expect that will happen before I’m comfortable running BTRFS on such systems. For the embedded systems I will just take the risk of data loss/corruption for the next few years.

Related posts:

  1. BTRFS Status Dec 2014 My last problem with BTRFS was in August [1]. BTRFS...
  2. BTRFS Status March 2014 I’m currently using BTRFS on most systems that I can...
  3. BTRFS Status July 2014 My last BTRFS status report was in April [1], it...
Categories: Elsewhere

Drupal core announcements: Requiring hook_update_N() for Drupal 8 core patches beginning June 24

Planet Drupal - Sat, 20/06/2015 - 02:58

In [policy, no patch] Require hook_update_N() for Drupal 8 core patches beginning June 24, the Drupal 8 release managers outline a policy to begin requiring hook_update_N() implementations for core patches that introduce data model changes starting after the next beta release. The goal of this policy change is to start identifying common update use-cases, to uncover any limitations we have for providing update functions in core, and to prepare core developers for considering upgrade path issues as we create the last few betas and first release candidates of Drupal 8. We need your help reviewing and communicating about this proposed policy, as well as identifying core issues that will be affected. Read the issue for more details.

Categories: Elsewhere

Norbert Preining: Localizing a WordPress Blog

Planet Debian - Sat, 20/06/2015 - 02:09

There are many translation plugins available for WordPress, and most of them deal with translations of articles. This might be of interest for others, but not for me. If you have a blog with visitors from various language background, because you are living abroad, or writing in several languages, you might feel tempted to provide visitors with a localized “environment”, meaning that as much as possible is translated into the native language of the visitor, without actually translating content – but allowing to.

In my case I am writing mostly in English and Japanese, but sometimes (in former times) in Italian and now and then in my mother tongue, German. Visitors from my site are from all over the world, but at least for Japanese visitors I wanted to provide a localized environment. This blog describes how to get as much as possible translated of your blog, and here I mean not the actual articles, because this is the easy part and most translation plugins handle that fine, but the things around the articles (categories, tags, headers, …).

Starting point and aims

My starting point was a blog where I already had language added as extra taxonomy, and have tagged all articles with a language. But I didn’t have any other translation plugin installed or used. Furthermore, I am using a child theme of the main theme in use (that is always a good idea anyway!). And of course, the theme you are using should be prepared for translation, that is that most literal strings in the theme source code are wrapped in __( ... ) or _e( ... ) calls. And by the way, if you don’t have the language taxonomy, don’t worry, that will come in automatically.

One more thing: The following descriptions are not for the very beginner. I expect certain fluency with WordPress, where for example themese and plugins keep their files, as well as PHP programming experience is needed for some of the steps.

With this starting point my aims were quite clear:

  • allow for translation of articles
  • translate as much as possible of the surroundings
  • auto-selection of language either depending on article or on browser language of visitor
  • by default show all articles independent of selected language
  • if possible, keep database clean as far as possible
Translation plugins

There is a huge bunch of translation plugins, localization plugins, or internationalization plugins out there, and it is hard to select one. I don’t say that what I propose here is the optimal solution, just one that I was pointed at by a colleague, namely utilizing the xili-language plugin.

Installation and initial setup

Not much to say here, just follow the usual procedure (search, install, activate), followed by the initial setup of xili-language. If you haven’t had a language taxonomy by now, you can add languages from the preference page of xili-language, first tab. After having added some languages you should have something similar to the above screen shot. Having defined your languages, you can assign a language to your articles, but for now nothing has actually changed on the blog pages.

As I already mentioned, I assume that you are using a child theme. In this case you should consult the fourth tab of the xili-language settings page, called Managing language files, where on the right you should see / set up things in a way that translations in the child theme override the ones in the main theme, see screen shot on the right. I just mention here that there is another xili plugin, xili-dictionary, that can do a lot of things for you when it comes to translation – but I couldn’t figure out its operation mode, so I switched back (i.e., uninstalled that plugin) and used normal .po/.mo files as described in the next section.

Adding translations – po and mo files

Translations are handled in normal (at least for the Unix world) gettext format. Matthias wrote about this in this blog. In principle you have to:

  • create a directory languages in your child theme folder
  • create there .po file named local-LL.po or local-LL_DD.po, where LL and LL_DD are the same as the values in the field ISO Names in the list of defined languages (see above)
  • convert the .po files to .mo files using msgfmt local-LL.po -o local-LL.mo

The contents of the po files are described in Matthias’ blog, and in the following when I say add a translation, then I mean: adding a stanza

msgid "some string" msgstr "translation of some string"

to the po file, and not forgetting to recompile it to mo file.

So let us go through a list of changes I made to translate various pieces of the blog appearance:

Translation of categories

This is the easiest part, simply throw in the names of your categories into the respective local-DD_LL.po file, and be ready. In my case I used local-ja.po which besides other categories contains stanzas like:

msgid "Travel" msgstr "旅行" Translation of widget titles

In most cases the widget titles are already automatically translated, if the plugin/widget author cared for it, meaning that he called the widget_title filter on the title. If this does not happen, please report this to the widget/plugin author. I have done this for example for the simple links plugin, which I use for various elements of the side-bar. The author was extremely responsive and the fix will be in the next release is already in the latest release – big thanks!

Translation of tags

This is a bit a problem, as the tags appear in various places on my blog: next to the title line and the footer of each blog, as well as in the tag cloud in the side bar.

Furthermore, I want to translate tags instead of having related tag groups as provided by xili tidy tags plugin, so we have to deal with the various appearances of tags one by one:

Tags on the main page – shown by the theme

This is the easier part – in my case I had already a customized content.php and content-single.php in my child theme folder. If not, you need to copy the one from the parent theme and change the appearance of it to translate tags. Since this is something that depends on the specific theme, I cannot give detailed advice, but if you see something like:

$tags_list = get_the_tag_list( '', __( ', ', 'mistylake' ) );

(here the get_the_tag_list is the important part), then you can replace this by the following code:

$posttags = get_the_tags(); $first = 1; $tag_list = ''; if ($posttags) { foreach($posttags as $tag) { if ($first == 1) { $first = 0; } else { $tag_list = $tag_list . __( ', ', 'mistylake' ); } $tag_list = $tag_list . '<a href="' . esc_url( home_url( '/tag/' . $tag->slug ) ) . '">' . __($tag->name, 'mistylake') . '</a>'; } }

(there are for sure simpler ways …) This code loops over the tags and translates them using the __ function. Note that the second parameter must be the text domain of the parent theme.

If you have done this right and the web site is still running (I recommend testing it on a test installation – I had white pages many times due to php programming errors), and of course you have actual translations available and are looking at a localized version of the web site, then the list of tags as shown by your theme should be translated.

Tag cloud widget

This one is a tricky one: The tag cloud widget comes by default with WordPress, but doesn’t translate the tags. I tried a few variants (e.g. creating a new widget as extension of the original tag cloud widget, and only changing the respective functions), but that didn’t work out at all. I finally resorted to a trick: Reading the code of the original widget, I saw that it applies the tag-sort-filter filter on the array of tags. That allows us to hook into the tag cloud creating and translate the tags.

You have to add the following code to your child theme’s functions.php:

function translate_instead_of_sort($tags) { foreach ( (array) $tags as $tag ) { $tag->name = __( $tag->name , 'mistylake' ); } return $tags; } add_action('tag_cloud_sort', 'translate_instead_of_sort');

(again, don’t forget to change the text domain in the __(.., ..) call!) There might be some more things one could do, like changing the priority to be used after the sorting, or sort directly, but I haven’t played around with that. Using the above code and translating several of the tags, the tag cloud now looks like the screenshot on the right – I know, it could use some tweaking. Also, now the untranslated tags are sorted all before the translated, things one probably can address with the priority of the filter.

Having done the above things, my blog page when Japanese is selected is now mostly in Japanese, with of course the exception the actual articles, which are in a variety of languages.

Open problems

There are a few things I haven’t managed till now to translate, and they are mostly related to the Jetpack plugin, but not only:

  • translation of the calendar – it is strange that although this is a standard widget of WordPress, the translation somehow does not work out there
  • transalation of the meta text entries (Log in, RSS feed, …) – interestingly, even adding the translation of these strings did not help get them translated
  • translation of simple links text fields – here I haven’t invested by now
  • translation of (Jetpack) subscribe to this blog widget

I have a few ideas how to tackle this problem: With Jetpack the biggest problem seems that all the strings are translated in a different text domain. So one should be able to add some code to the functions.php to override/add translations to the jetpack text domain. But somehow it didn’t work out in my case. The same goes for things that are in the WordPress core and use the translation functions without a text domain – so I guess the translation function will use the main WordPress translation files/text domain.


The good thing of the xili-language plugin is that it does not change the actual posts (some plugins save the translations in the the post text), and is otherwise not too intrusive IMHO. Still, it falls short of allowing to translate various parts of the blog, including the widget areas.

I am not sure whether there are better plugins for this usage scenario, I would be surprised if not, but all the plugins I have seen were doing a bit too much on the article translation side and not enough on the translation of the surroundings side.

In any case, I would like to see more separation between the functionality of localization (translating the interface) and translation (translating the content). But at the moment I don’t have enough impetus to write my own plugin for this.

If you have any suggestions for improvement, please let me know!


Categories: Elsewhere


Subscribe to jfhovinne aggregator - Elsewhere