Olivier Berger: Using RDFAlchemy together with RDFLib’s SPARQLStore to query DBPedia and process resources in OO way

Planet Debian - Wed, 14/05/2014 - 17:41

I’ve been searching for interesting ways to manipulate RDF graphs in Python, to create an application that would handle Linked Data Resources in an OO-way, i.e. using Python classes and not tables/sets/lists of triples. The data will be persisted in graphs in a triple store, accessed through a SPARQL enpoint.

In this post, I’ll illustrate how I managed to tie RDFLib’s SPARQLStore plugin and RDFAlchemy to reach a rather nice looking result.

RDFLib provides tools to manipulate graphs, but most of the examples I found didn’t load Graph instances from SPARQL, and generally used SPARQLWrapper results (tables) manually. Here comes a first tool to the rescue : SPARQLStore, which allows to dynamically query a remote SPARQL endpoint when navigating an RDFLib graph. I couldn’t find a lot of documentation about it, but I found some hints in slide 51 of the excellent presentation by Tatiana Al-Chueyr Linking the world with Python and Semantics.

Now, I know how I can manipulate the RDFLib Graphs queried from a remote SPARQL endpoint with the usual methods, I’m still not satisfied, because I want OO stuff, not lists of (predicate, object) tuples.

Here comes the second tool, RDFAlchemy, which allows to create “descriptor” classes mapping RDF Resources to Python classes. Here again, there isn’t much docs available, but I found the excellent tutorial by Ted Lawless Reading and writing RDF for VIVO with RDFAlchemy.

Now we can put the 2 pieces together, modulo a few precautions : the SPARQLStore plugin of RDFLib will generate SPARQL queries everytime a graph is navigated, whereas RDFAlchemy expects the graph to be in memory (at least AFAIU). So we’ll have to manually pre-load the contents of the graphs that we need for all the attributes of the descriptor classes.

I’ve written an example code that illustrates this by trying to query french films in Wikipedia. Here’s a copy of the gist. Attention, it will query DBPedia a lot, so pay attention to the bandwidth and memory if you change parameters.

It isn’t perfect, and I still need to investigate benefits and limitations of the approach. On clear limitation is the number of SPARQL queries made on the endpoint, instead of a smarter pre-loading.

Also, on a side note, during tests I could spot a few issues with SPARQLStore, which seems lagging behind… probably not used by so many people. The RDFAlchemy “project” doesn’t seem to be in great health, mostly unmaintained from what I can see (and notice that I linked to the GitHub clone/fork that seemed to be the latest maintained while the original author’s seems dead), but nevertheless, the code works with a more recent RDFLib, so that’s not so bad.

Stay tuned for more adventures in Linked Data land in Python.

Categories: Elsewhere

Joey Hess: how I wrote init by accident

Planet Debian - Wed, 14/05/2014 - 17:05

I wrote my own init. I didn't mean to, and in the end, it took 2 lines of code. Here's how.

Propellor has the nice feature of supporting provisioning of Docker containers. Since Docker normally runs just one command inside the container, I made the command that docker runs be propellor, which runs inside the container and takes care of provisioning it according to its configuration.

For example, here's a real live configuration of a container:

-- Exhibit: kite's 90's website. , standardContainer "ancient-kitenet" Stable "amd64" & Docker.publish "1994:80" & Apt.serviceInstalledRunning "apache2" & Git.cloned "root" "git://kitenet-net.branchable.com/" "/var/www" (Just "remotes/origin/old-kitenet.net")

When propellor is run inside this container, it takes care of installing apache, and since the property states apache should be running, it also starts the daemon if necessary.

At boot, docker remembers the command that was used to start the container last time, and runs it again. This time, apache is already installed, so propellor simply starts the daemon.

This was surprising, but it was just what I wanted too! The only missing bit to make this otherwise entirely free implementation of init work properly was two lines of code:

void $ async $ job reapzombies where reapzombies = void $ getAnyProcessStatus True False

Propellor-as-init also starts up a simple equalivilant of rsh on a named pipe (for communication between the propellor inside and outside the container), and also runs a root login shell (so the user can attach to the container and administer it). Also, running a compiled program from the host system inside a container, which might use a different distribution or architecture was an interesting challenge (solved using the method described in completely linux distribution-independent packaging). So it wasn't entirely trivial, but as far as init goes, it's probably one of the simpler implementations out there.

I know that there are various other solutions on the space of an init for Docker -- personally I'd rather the host's systemd integrated with it so I could see the status of the container's daemons in systemctl status. If that does happen, perhaps I'll eventually be able to remove 2 lines of code from propellor.

Categories: Elsewhere

DrupalCon Austin News: So you're going to DrupalCon Austin. Now What?

Planet Drupal - Wed, 14/05/2014 - 16:30

You've got your tickets to DrupalCon Austin. What happens next? This fun and helpful infographic maps out the next steps for you. From reserving a space to sleep to selecting your schedule and keeping up with all the latest news, this infographic comes complete with links to help you plan out your DrupalCon Austin adventure.

Categories: Elsewhere

Steve Kemp: An email client and a new desk.

Planet Debian - Wed, 14/05/2014 - 16:16

Today I released version 0.25 of my console mail client, which is a release focussed upon portability (DragonFly BSD, and MacOSX specifically).

Over the past couple of weeks I've written a fair bit of code, wondering if I want to make the jump to a graphical email client, but the conclusion for the moment has to be no.

With the scripting support built into my client, and even before then using the hooks/hacks that mutt supported, I just process mail so much more quickly than via a GUI system.

I also benefit from reading the mail on the host to which it is delivered - mail gets filtered by something like procmail, and I read it in-situa. IMAP is available if I travel, but I rarely do so.

Having a GUI client might be fun, but it would mean I'd read mail on my desktop - pretty much the only system I don't backup (except for images, videos, and local media). It would also involve running imapsync, or similar, to pull the mail in, and relaying through the remote server to avoid my ISPs poor IP-reputation.

In short I believe if I use a GUI client I'll get slower, and I'll still need the remote host regardless.

It was this time last year when I thought it was functional, but now it is functional, battle-tested, and reliable.

So I guess I'm done with email for the next few years. Maybe in that time somebody will write something better - console based for preference, GUI as a last resort, and certainly not another webmail client.

In other news ..

I had a fun interview on Monday, it went well until they admitted they couldn't afford me - so their goal is to pay a junior member a small salary and hope to get somebody senior to work part-time for a similarly minimal salary. Might work for somebody else, but it wouldn't for me right now, so on that basis I declined.

The most annoying thing about interviewing is the waiting, between the early flirting about duties and expectations, to scheduling meetings, and then awaiting decisions.

On that note I'm half-way through building a new desk which is a nice physical job I can really concentrate upon. I'm currently waiting for the stain to dry on the legs, and then I'll get the damn thing finished. It probably looks more "rustic" than "modern", but it smells nice, so that's the main thing ;)

Expect pictures when it is finished.

Categories: Elsewhere

Mediacurrent: Energize Your Web Project at Drupalcon

Planet Drupal - Wed, 14/05/2014 - 14:22

Here at Mediacurrent, we’re counting down the days until Drupalcon Austin. This year, we’re proud to be a Platinum sponsor, and we’re bringing our A-game with over a dozen teammates, The Weather Channel Case Study, Power Sessions, Office Hours, an After-party (with LingoTek), and a ton of other activities. If you’re attending Drupalcon Austin this year, here are 5 reasons to pay Mediacurrent a visit:

Categories: Elsewhere

Wouter Verhelst: Maven...

Planet Debian - Wed, 14/05/2014 - 14:04

This is wrong in so many ways...

mvn clean


Downloading <foo>...
Categories: Elsewhere

Acquia: Ultimate Guide to Drupal 8: Episode 2 - Mobile Improvements

Planet Drupal - Wed, 14/05/2014 - 13:58

Welcome to the second instalment of an 8-part blog series we're calling "The Ultimate Guide to Drupal 8." Whether you're a site builder, module or theme developer, or simply an end-user of a Drupal website, Drupal 8 has tons in store for you! This blog series will attempt to enumerate the major changes in Drupal 8. Successive posts will gradually get more technical, so feel free to skip to later parts (once they're published) if you're more on the geeky side.

Categories: Elsewhere

Francesca Ciceri: A month with Mozilla

Planet Debian - Wed, 14/05/2014 - 13:53
Random thoughts of a new contributor

I've started to contribute to Mozilla - and particularly to the Firefox Desktop QA team - at the end of March, in order to apply for the OPW with Mozilla as Bug Wrangler.
While being already part of the Free Software world as contributor clearly helps, it's always difficult to find your way at first in such a big project, no matter how helpful the people you're working with are.

It's about trying to understand how the project is organized and who works on what, what is expected from you as contributor and the specific style of interaction and contribution accepted in the project.

So, here's a couple of thought about it: the necessary disclaimer is that I'm a long-time Debian contributor meaning that, inevitably, Debian is my reference model. Also, everything I'm writing here regards only the specific part of the project I've been working in: Desktop QA and Bugmastering.


The first thing that I noticed is the huge amount of documentation. Everything is very well documented and while this is definitely a plus in my book, it's sometimes difficult to keep track of all this resources.

AFAICT, there are basically three main sources of documentation:

  1. the wiki
  2. the Mozilla Developer Network
  3. the site of the QA team

Some of the documents on these three different places are sometimes similar, making them redundant and it's difficult to remember where you read what. At least for me. My workaround has been to bookmark pages as a mad woman, and also to make a list on my wikipage of the pages I found more useful.
My top five of useful pages? Here we go!

  1. a bug's life: lifecycle of bugs in BMO
  2. Tyler's BMO survival guide (parts 1, 2 and 3)
  3. Tyler's Triage Guidelines
  4. on reducing testcases
  5. bug verification walkthrough

And speaking of triage, special shout-out for the most useful addon ever for Mozilla triagers/testers: Nightly Tester Tools.

Community interaction

For someone coming from Debian, where everything is done via public mailing lists and/or IRC chans, Mozilla was a bit of a cultural shock. The mailing lists are not very much used, at least in the QA community (or I'm not subscribed to the right ones ;)). As for IRC, while there are many channels, they are not very active: meetings and important discussions are held using a proprietary videoconference software (Vidyo), and are mostly restricted to employees (but other people can join if they have a guest URL).

On the other hand, every time I asked for help - even the most stupid question - on IRC, I got a useful reply. And thanks to the folks on #testdays and #qa (especially petruta and Aleksej) I was able to speed up the learning process and understand how things work in the project. So, while I'd like to see more public meetings, it's also true that the community is working well on interaction among contributors.

Community outreach

This is where I think Debian could borrow a couple of ideas. Mozilla basically got me hooked thanks to their Bugdays: every Monday and Wednesday are - respectively - dedicated to Triage and Verification of bugs. The events are open to everyone, the wikipages for the events provide a good tutorial explaining the workflow in detail, and if you have any doubts there are always people willing to help you on the dedicated IRC chan. Every bug worked on during these events is also tagged: so that later it's possible to check how successful the day was in terms of participation.

Bugzilla and the triaging/verification workflow

About Bugzilla I can only say that I like it. I like how flexible is for search queries, I like the possibility to have custom tags on the whiteboard (as the [bugday-xxxxxx] one) and the smart way to interact among people in the bug without making the bug go stale (ie, mostly the needinfo feature). But beside the software used as BTS, I must admit that I really like the workflow and the whole concept of triaging in Mozilla. The granularity of rights (everyone, canconfirm, editbugs) and the clear procedure for a permission upgrade; the presence of a team constantly working on cleaning the incoming bugs, regardless of the product/component, to make easier for the developers the filtering and the resolution as well; the attention to details of the verification process: each bug marked as Resolved → Fixed need to be Verified, to be sure that it's been actually fixed... All these things are really impressive.

This, obviously, is something that can be done inside an upstream project where there are people doing paid work, and where the products are not too many. I cannot see how this kind of QA process can be applied to bugs against an entire OS.
But there are some things we (as in Debian) could take inspiration from.

Organizing regular Bugdays, for instance. Creating a team of triagers to help both with incoming and with very old bugs. And this, again, is a kind of work that can be done by everyone, not only coding contributors: meaning that an additional effect would be to increase the diversity in the contributors pool.

As a side note: I'll start my OPW internship next Monday, and will spend three months playing with bugs and learning how to be a Bug Wrangler! \o/\o/

Categories: Elsewhere

Code Karate: Drupal 7 Commerce Stripe Module

Planet Drupal - Wed, 14/05/2014 - 13:37

The Commerce Stripe module integrates Stripe with the Drupal Commerce checkout and payment system.

Categories: Elsewhere

Modules Unraveled: 107 The Community Summit at DrupalCon Austin with Addison Berry and Mortendk - Modules Unraveled Podcast

Planet Drupal - Wed, 14/05/2014 - 10:41
Published: Wed, 05/14/14Download this episodeTrack
  • What exactly is the Community Summit?
  • When is it? Monday, June 2 - the day before the conference itself starts
  • How did it go in Prague?
  • Is there anything that will be new or different in Austin?
Episode Links: Community Summit in AustinSubmit Your Desired ProjectDrupalCon Social EventsMorten on TwitterMorten on Drupal.orgMorten’s BlogHire Morten for Theming StuffAddison on TwitterDrupalize.meAddison’s BlogDareConf 2014GruntTags: 
Categories: Elsewhere

Modules Unraveled: 107 The Community Summit at DrupalCon Austin with Addison Berry and Mortendk - Modules Unraveled Podcast

Planet Drupal - Wed, 14/05/2014 - 10:41
Published: Wed, 05/14/14Track
  • What exactly is the Community Summit?
  • When is it? Monday, June 2 - the day before the conference itself starts
  • How did it go in Prague?
  • Is there anything that will be new or different in Austin?
Episode Links: Community Summit in AustinSubmit Your Desired ProjectDrupalCon Social EventsMorten on TwitterMorten on Drupal.orgMorten’s BlogHire Morten for Theming StuffAddison on TwitterDrupalize.meAddison’s BlogDareConf 2014GruntTags: 
Categories: Elsewhere

Jose Luis Rivas: transloadit-api v1.0.0-rc1

Planet Debian - Wed, 14/05/2014 - 07:10

A release candidate for v1.0.0 of transloadit-api is out.

You can install it via npm and give it a try.

npm install transloadit-api@1.0.0-rc1

Now it supports the full transloadit API, together with signature's creation, assemblies, notifications and templates management.

The source is on github, the docs are in this website as well as in the comments in the code (which is the source for the website) and of course, any issue just report it on the github's tracker. It has a lot of tests but there are some tests missing, specially for operations that require internet.

Hopefully I will have time to write them this week and then release a proper v1.0.0.

Categories: Elsewhere

Mart&iacute;n Ferrari: Introducing Yakker: an open, secure and distributed alternative to WhatsApp

Planet Debian - Wed, 14/05/2014 - 04:45

Did I get your attention with the title? Good. In this post I will outline something that I have been thinking about for some months:

I believe it is possible to build a system with the simplicity and functionality of WhatsApp or Viber, which provides end-to-end encryption, is built on free software and open protocols, that supports federation and is almost decentralised, and that would allow interested companies to turn a profit without compromising any of these principles.


This is the first post of a series that I will be publishing in the next few days. Many parts of these posts will be technical, but I expect that the main concepts can be understood by a wider audience.

What I am proposing seems like a bold statement, I know. Maybe there is some fatal flaw somewhere in my thinking, and that is why I am publishing this: I hope to get constructive feedback, and maybe get enough traction to start implementing it Real Soon Now™.

I have been thinking about this problem since February, when I discussed this extensively with friends at FOSDEM. I have already published a critique of Telegram, which had way more impact than I ever imagined, showing that there is people out there interested in this kind of stuff. The last posts about DNSSEC and DANE were part of my musings about this, too.

There are many components that need to be built for this to happen. But more importantly, this can only be useful if it gains a critical mass. And that's why I think making this a viable business tool is very important. At the same time, that means I need to think extra carefully to make it impossible for any for-profit company to mutate this effort into Just Another Walled Garden.

My goals for this architecture are:

  • First and foremost, target the same people that nowadays is using a plethora of walled gardens for their instant communication needs. That is WhatsApp, Viber, Skype, Facebook messenger, etc.
  • It must focus on mobile, that is what people care about, without forgetting about other use cases.
  • Creating an account and placing the first call/text message should be as easy as it is currently with the competition.
  • All communication must be encrypted end-to-end with public-key cryptography; nobody but the user has access to the private keys.
  • Most components must be decentralised, and allow for competition.
  • There should be as little trust as possible placed on any part of the system.
  • Anybody can set up a compatible service provider and offer it to its users, while having full interoperability with other providers.
  • Compatible services which are not part of the network must be able to interoperate.
  • Contacting a person should happen even if they are not subscribed to the service. The client application must fall-back seamlessly to using interoperability gateways, PSTN termination, or the mobile network.
  • Interoperability with the competition is desirable, but possibly is better left to be implemented by the client applications.

I have identified a few components needed for this to work, I will expand on each one later.

  1. A flagship mobile application for Android and iOS, based on Lumicall or CSIPSimple, but with several important modifications.
  2. One or many directory and authentication services, based on ENUM and DNSSEC. These are the most critical piece of this idea, and possibly must only be operated by community-governed non-profits.
  3. One or many service providers, that offer simple account creation, registration, and optionally PSTN termination (which can be the main way of generating profit). An API needs to be defined for operations that are not part of the communications protocol, like account creation, credit purchasing, and balance querying.
  4. A network governing charter, and a trusted non-profit organisation that oversees that any participating parties are following the charter. This organisation defines which directory services are to be trusted (and possibly operates one of them), and which service providers the client application can use to create accounts.
Key points

Some of the issues that need to be solved are:

  • How to handle and distribute public keys securely without the user understanding anything about security.
  • How to make registration painless and password-free, while offering an acceptable level of security.
  • How to fund development of the client application, and maintenance of the directory services.
  • How to get companies interested in this, so them would bring users to the network.
  • How to allow the user to migrate from one service provider to another, to improve competition.
  • How to prevent any party from subverting the spirit of the network.
  • How to make the client application work everywhere and have reasonably quality.
To be continued

I think I have answers to most of these problems. I will elaborate in the next few days, stay tuned!


The name is something I've chosen a name in less than 2 minutes, while starting this post, so probably is awful.

The distributed part is only half true, as the directory services need to be centralised, but I think it's good enough.

I am aware that Lumicall seems to be trying to build something similar. I only found about that recently, when I was thinking about this design. Sadly, I think it has several shortcomings, but it is definitely one of the building blocks of this project.

Categories: Elsewhere


Subscribe to jfhovinne aggregator - Elsewhere