Agrégateur de flux

Laura Arjona: Six months selfhosting: my userop experiences

Planet Debian - dim, 19/04/2015 - 02:06

Note: In this post I mention some problems and ask questions (to myself, like “thinking aloud”). The goal is not to get answers to those questions (I suppose that I will find them soon or later in the internet, manuals and so), but to show the kind of problems and questions that arise in my selfhosting adventures, which I suppose are common to other people trying to administer a home server with some web services.

Am I an userop? Well I’m something in the middle of (GNU/Linux) user and sysadmin: I have studied computer technical engineering but most of my experience has been in helpdesk, providing support for Windows users. I’m running Debian in some LAMP boxes at work (without GUI) since 2008 or so, and in my desktops (with GUI) since 2010. I don’t code nor package, but I don’t mind trying to read code and understand it (or not). I know a bit of C, a bit of Python, of PHP, and enough Perl to open a Perl file and close it after two minutes,  understanding that it’s great, but too much for me :) I translate software, so I’m not scared to clone a repository, edit files, commit or submit a patch. I’m not scared of compiling a program (except if it’s an Android app: I try to avoid setting up the development environment just to try some translation that I made… but I built my Puma before it was the binary available for download or in F-Droid).

In conclusion, I feel more like a “GNU/Linux power user” than a “sysadmin”. Sometimes just a “user” or even a “newbie” (for example, I don’t know very well the Unix/Linux folder tree… where are the wallpapers stored? Does it depend on the desktop that I use?).

Anyway. I won’t stop my free software + free networks digital life because I don’t know many things. I bought a small server for home last September, and I wanted to try to selfhost some services, for me and for my family. I want to be a “home sysadmin” or something like that, so I joined the “userops” mailing list :)

Here you have my experiences on selfhosting/being an userop until now.

Mail

I even didn’t try to setup my mail server, because many people say it’s a pain (although nice articles were published about how to do it, for example this series in ArsTechnica) and I need a static IP which is 14€/month more to my ISP, and Gandi, the place where I rented my domain name, provides mail, and they use Debian and Roundcube, and sponsor Debian too, so I decided to trust on them.

So this is my strategy now, to try to keep mail under my control:

  • Trust my domain provider.
  • Backup my mail and keep local copies, removing sensible stuff from the server.
  • Use and spread the word about GPG encryption.
  • Try not to send photos or videos by mail, just send the link to my MediaGoblin instance (see below).
MediaGoblin

I’ve setup two MediaGoblin instances (yes, two!). I managed to do it in Debian 7 stable (I think NodeJS’ npm was not needed then), but soon later I upgraded to Jessie so now it’s even better.

I installed Nginx and PostgreSQL via apt, to use them for both instances (and probably some more software later).

One instance is public, and I use a Debian user, a PostgreSQL database, and it’s running in http://media.larjona.net
I have requested an SSL cert to Gandi but I still didn’t deployed it (lazy LArjona!!).

The other instance is private, for family photos. I didn’t know very well how much of my existing setup could reuse and how to keep both instances in case of downtimes or attack… I know more or less the concept or “chroot” but I don’t know how to deploy it in my machine. So I decided to use another Debian user, another PostgreSQL database, deploy MediaGoblin in a different folder, and create another virtual server in my Nginx to serve it. I managed to setup that virtual server to http-authenticate and to serve content via a different port, and use a self-signed SSL certificate (it’s only for family, so it does not matter). I created another (unprivileged) Debian user with a password for the nginx authentication, and gave my family the URL in the form https://mediaprivate.larjona.net:PortNumber and the user and password (mediaprivate is a string, and PortNumber is a number). I think they don’t use the instance too much, but at least I upload photos there from time to time and email the link instead of emailing the photos themselves (they don’t use GPG either…).

Upgrades

I upgraded MediaGoblin from 0.7.1 to 0.8.0 successfully, I sent a report about how I did it to the mailing list. First I upgraded the public instance, when I figure out the process, I upgraded the second instance to test my instructions, and then, I sent the report with the instructions to the mailing list.

Static site and LimeSurvey: the power of free software (with instructions)

I wanted to act as a mirror of floss2013.libresoft.es and surveys.libresoft.es since they suffer a downtime and I participated in that project (not in the sysadmin part, but in the research and content creation).

The static site floss2013.libresoft.es offered a zip with the whole website tree (since the website was licensed as AGPL), and I had access to the git repo holding the development copy of the website. So I just cloned the repository and setup another nginx virtual server in my machine, and tuned my DNS zone in Gandi website to serve floss2013.larjona.net from home. 10 minutes setup YAY! #inGitWeTrust #FreeSoftwareFTW :)

For surveys.larjona.net I had to install a LimeSurvey instance. I knew how to do it because we use LimeSurvey at work, but at home I had Nginx instead of Apache, and PostgreSQL instead of MySQL. And no PHP… I searched about how to install PHP in Nginx (I can use apt-get, nice!) and how to install LimeSurvey with Nginx and PostgreSQL (I had documentation about that, so I followed, and it worked).

For making available the data (one survey and its results, so people can login as visitor to query and get statistics), I downloaded the LimeSurvey export dataset that we were providing in the static website, followed the replication instructions (hey, I wrote them!), and they worked #oleole! (And here, dear researchers, gets demonstrated that free software and free culture really empower your research and help spreading your results).

Etherpad: not so easy, it seems!

I’m trying to install Etherpad-Lite, but I’m suffering a bit. I think I did everything ok according to some guides but I get “Bad Gateway” and these kind of errors when trying to browse with Lynx in the host:

[error] 3615#0: *24 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 127.0.0.1, server: pad.larjona.net, request: "GET / HTTP/1.0", upstream: "http://127.0.0.1:9001/", host: "pad.larjona.net" 2015/04/17 20:52:56 [error] 3615#0: *24 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: pad.larjona.net, request: "GET / HTTP/1.0", upstream: "http://[::1]:9001/", host: "pad.larjona.net"

I’m not sure if I need to open some port in iptables, my router, or change my nginx configuration because the guides assume you’re only serving one website in the port 80 (and I have several of them, now…), or what… I’ve spent three chunks of time (maybe ~2h each?) on this, in different days, and couldn’t figure it out, so I decided to round-robin in my TODO list.

Userops thoughts Debian brings peace of mind (for me)

On one side, maintaining a Debian box it’s quite easy, and the more software that it’s packaged, the less time that I spend installing or upgrading. I like being in stable, I’m in Jessie now (I migrated when it was frozen), but I’ll stay in stable as much as I can.

I like that I can use the software that I installed via apt-get for several services (nginx, PostgreSQL…). About the software that is not packaged (MediaGoblin, LimeSurvey, EtherPad, maybe others later), I wonder how dependencies and updates are handled. And maybe (probably) I have installed some components several times, one for each service (this sounds like a Windows box #grr).

For example MediaGoblin uses PyPump. PyPump 0.5 is packaged in Debian Jessie. MediaGoblin uses PyPump 0.7+. What if PyPump 0.7+ gets, let’s say, into Jessie-backports? Can I benefit from that?

I know that MediaGoblin upgrade instructions includes upgrading the dependencies, but what about some security patch in one dependency? Should I upgrade the pip modules periodically? How to know if some upgrade is recommended because patches a vulnerability, or it’s just new features (and maybe breaking my setup)?

This kind of things are the “peace of mind” that Debian packaging brings to me: when some piece of software is packaged, I know maybe I need to care about proper setup and configuration, but later, it’s kind-of-easy to maintain (since the Debian maintainers care about the rest). I don’t mind about cloning a repo and compiling, I mind about later, or coexistance with other program/services. I trust in the MediaGoblin community and I’m an active member (I’m not developer, but hang on IRC, follow the mailing list, etc) but for example I don’t know anything about the EtherPad project. And I don’t feel like joining the community (I’m already an active member in Debian, MediaGoblin, F-Droid, Pump.io, translator of LimeSurvey and many other small apps that I use, and in the future will use more services, like OwnCloud, XMPP…), joining the community of each software that I use is becoming not sustainable :s

Free software is more than software

I follow the userop mailing list, and it’s becoming very technical. I mostly understand the problems (which are similar to the problems that I face: how to isolate different services, how to easily-configure them, how to make them installable by average user…) But I don’t understand most of the solutions proposed, and I think that probably we need technical solutions, but in the meanwhile, some issues can be addressed not with software, but with other means: good documentation, community support, translations, beta-testers…

This is my conclusion until now. When a project is well documented, I think I can find my way to selfhost no matter if the software is packaged (or “contained”) or not. MediaGoblin, and LimeSurvey, are well documented, and the user support channels are very responsive.

I find lots of instructions that assume that you will use a whole machine for their service (and not for other things). And lots of documentation for the LAMP stack, but not for Nginx + PostgreSQL and Node instead of PHP… So, for each “particularity” of my setup, I search the internet and try to pick good sources to help me do what I wanted to do.

I’m kind of privileged

Some elements, not software related, to take into account as “pre-requisites for succeed” selfhosting services:

  • I knew what to search.
  • I knew which sites to visit from the results (arch wiki, debian wiki, stack overflow, etc: some of them were not the Top1 in the results).
  • I had time to read several sources and make my mind about what to do and how.
  • I can read, understand, and write in English.
  • I have no fear about my broken English.
  • I have no impostor syndrome.
  • I felt welcome in the FLOSS communities where I hanged out.

These aspects are not present in a lot of people. If I look around to the “computer users” that I know (mostly Windows+Android, some GNU/Linux users, some Mac OSX users, some iOS users), I find that they search things like “X does not work” or they cannot write a proper search query in English. Or they trust some random person writing a recipe in their blog, without trying first to understand what the recipe does. Other people just say “I’m not a professional sysadmin, I’ll just do what «everybody» does (aka use Google services or whatever). What if I try and I don’t succeed?”. Things like that.

We may need some technical solutions (and hackers are thinking about that, and working on that). But I feel that we need (more) a huge group of beta-testers, dogfooding people, aventurers that try the half-cooked solutions and provide successful and unsuccessful experiences, to guide the research and make software technologies advance. I’m not sure if I am an userop, but I feel part of that “vanguard force”, I want to be part of the future of free software and free networks.


Filed under: My experiences and opinion Tagged: Communities, Contributing to libre software, Debian, Developer motivations, English, free networks, Free Software, Freedom, innovation, MediaGoblin, Moving into free software, Project Management, selfhosting, sysadmin
Catégories: Elsewhere

Gregor Herrmann: RC bugs 2015/11-16

Planet Debian - dim, 19/04/2015 - 00:52

only one week left until the jessie release. yay!

in the last weeks I didn't find many RC bugs that I could fix; still, here's the short list; nice feature: I mostly help others or could build an work done by others.

  • #669735 – dpkg-www: "dpkg-www: transition towards Apache 2.4"
    sponsor QA upload from Jean-Michel Nirgal Vourgère
  • #669777 – yocto-reader: "yocto-reader: transition towards Apache 2.4"
    sponsor NMU from Jean-Michel Nirgal Vourgère, upload to DELAYED/2
  • #669796 – w3c-linkchecker: "w3c-linkchecker: transition towards Apache 2.4"
    cherry-pick periapt's commit from 2012 (pkg-perl)
  • #764284 – testdisk: "[testdisk] after ntfs-3g upgrade, testdisk cannot be installed (Depends: error)"
    tag + sid to get it out of jessie RC bugs
  • #780629 – libibverbs1: "libibverbs1: please add Breaks: libopenmpi1.3"
    upload NMU prepared by Andreas Beckmann, adding a Breaks, upload to DELAYED/2
  • #780729 – pbuilder: "pbuilder must define PATH as in debian-policy (and as used on buildds)"
    downgrade
  • #782160 – src:chrony: "chrony: Multiple issues: CVE-2015-1821 CVE-2015-1822 CVE-2015-1853"
    sponsor maintainer upload by Joachim Wiedorn
Catégories: Elsewhere

Neil McGovern: Taking office

Planet Debian - sam, 18/04/2015 - 15:36

Yesterday, my first term started as the Debian Project Leader. There’s been a large number of emails congratulating me, and thanks to everyone who sent those. I’d also like to thank Mehdi Dogguy and Gergely Nagy for running, and of course Lucas Nussbaum for his service over the past two years.

Lucas also did a great handover, and so (I hope!) I’m aware of most of the issues that are ongoing. As started previously, I’ll keep my daily log of activities in /srv/leader/news/  on master.debian.org.

Catégories: Elsewhere

Russ Allbery: Review: The Long Way to a Small, Angry Planet

Planet Debian - sam, 18/04/2015 - 08:24

Review: The Long Way to a Small, Angry Planet, by Becky Chambers

Publisher: CreateSpace Copyright: 2014 ISBN: 1-5004-5330-7 Format: Kindle Pages: 503

The Wayfarer is a tunneling ship: one of the small, unremarked construction ships that help build the wormhole network used for interstellar transport. It's a working ship with a crew of eight (although most people would count seven and not count the AI). They don't all like each other — particularly not the algaeist, who is remarkably unlikeable — but they're used to each other. It's not a bad life, although a more professional attention to paperwork and procedure might help them land higher-paying jobs.

That's where Rosemary Harper comes in. At the start of the book, she's joining the ship as their clerk: nervous, hopeful, uncertain, and not very experienced. But this is a way to get entirely away from her old life and, unbeknownst to the ship she's joining, her real name, identity, and anyone who would know her.

Given that introduction, I was expecting this book to be primarily about Rosemary. What is she fleeing? Why did she change her identity? How will that past come to haunt her and the crew that she joined? But that's just the first place that Chambers surprised me. This isn't that book at all. It's something much quieter, more human, more expansive, and more joyful.

For one, Chambers doesn't stick with Rosemary as a viewpoint character, either narratively or with the focus of the plot. The book may open with Rosemary and the captain, Ashby, as focal points, but that focus expands to include every member of the crew of the Wayfarer. We see each through others' eyes first, and then usually through their own, either in dialogue or directly. This is a true ensemble cast. Normally, for me, that's a drawback: large viewpoint casts tend to be either jarring or too sprawling, mixing people I want to read about with people I don't particularly care about. But Chambers avoids that almost entirely. I was occasionally a touch disappointed when the narrative focus shifted, but then I found myself engrossed in the backstory, hopes, and dreams of the next crew member, and the complex ways they interweave. Rosemary isn't the center of this story, but only because there's no single center.

It's very hard to capture in a review what makes this book so special. The closest that I can come is that I like these people. They're individual, quirky, human (even the aliens; this is from more the Star Trek tradition of alien worldbuilding), complicated, and interesting, and it's very easy to care about them. Even characters I never expected to like.

The Long Way to a Small, Angry Planet does have a plot, but it's not a fast-moving or completely coherent one. The ship tends to wander, even when the mission that gives rise to the title turns up. And there are a lot of coincidences here, which may bother you if you're reading for plot. At multiple points, the ship ends up in exactly the right place to trigger some revelation about the backstory of one of the crew members, even if the coincidence strains credulity. Similar to the algae-driven fuel system, some things one just has to shrug about and move past.

On other fronts, though, I found The Long Way to be refreshingly willing to take a hard look at SF assumptions. This is not the typical space opera: humans are a relatively minor species in this galaxy, one that made rather a mess of their planet and are now refugees. They are treated with sympathy or pity; they're not somehow more flexible, adaptable, or interesting than the rest of the galaxy. More fascinatingly to me, humans are mostly pacifists, a cultural reaction to the dire path through history that brought them to their current exile. This is set against a backdrop of a vibrant variety of alien species, several of whom are present onboard the Wayfinder. The history and background of the other species are not, sadly, as well fleshed out as the humans, but each with at least a few twists that add interest to the story.

But the true magic of this book, the thing that it has in overwhelming abundance, is heart. Not everyone in this book is a good person, but most of them are trying. I've rarely read a book full of so much empathy and willingness to reach out to others with open hands. And, even better, they're all nice in different ways. They bring their own unique personalities and approaches to their relationships, particularly the complex web of relationships that connects the crew. When bad things happen, and, despite the overall light tone, a few very bad things happen, the crew rallies like friends, or like chosen family. I have to say it again: I like these people. Usually, that's not a good sign for a book, since wholly likeable people don't generate enough drama. But this is one of the better-executed "protagonist versus nature" plots I've read. It successfully casts the difficulties of making a living at a hard and lonely and political job as the "nature" that provides the conflict.

This is a rather unusual book. It's probably best classified as space opera, but it doesn't fit the normal pattern of space opera and it doesn't have enough drama. It's not a book about changing the universe; at the end of the book, the universe is in pretty much the same shape as we found it. It's not even about the character introduced in the first pages, or really that much about her dilemma. And it's certainly not a book about winning a cunning victory against your enemies.

What it is, rather, is a book about friendships, about chosen families and how they form, about being on someone else's side, about banding together while still being yourself. It's about people making a living in a hard universe, together. It's full of heart, and I loved it.

I'm unsurprised that The Long Way to a Small, Angry Planet had to be self-published via a Kickstarter campaign to find its audience. I'm also unsurprised that, once it got out there, it proved very popular and has now been picked up by a regular publisher. It's that sort of book. I believe it's currently out of print, at least in the US, as its new publisher spins up that process, but it should be back in print by late 2015. When that happens, I recommend it to your attention. It was the most emotionally satisfying book I've read so far this year.

Rating: 9 out of 10

Catégories: Elsewhere

Steve Kemp: skx-www upgraded to jessie

Planet Debian - sam, 18/04/2015 - 02:00

Today I upgraded my main web-host to the Jessie release of Debian GNU/Linux.

I performed the upgraded by changing wheezy to jessie in the sources.list file, then ran:

apt-get update apt-get dist-upgrade

For some reason this didn't upgrade my kernel, which remained the 3.2.x version. That failed to boot, due to some udev/systemd issues (lots of "waiting for job: udev /dev/vda", etc, etc). To fix this I logged into my KVM-host, chrooted into the disk image (which I mounted via the use of kpartx), and installed the 3.16.x kernel, before rebooting into that.

All my websites seemed to be OK, but I made some changes regardless. (This was mostly for "neatness", using Debian packages instead of gems, and installing the attic package rather than keeping the source-install I'd made to /opt/attic.)

The only surprise was the significant upgrade of the Net::DNS perl-module. Nothing that a few minutes work didn't fix.

Now that I've upgraded the SSL-issue I had with redirections is no longer present. So it was a worthwhile thing to do.

Catégories: Elsewhere

Code Karate: Drupal Views Module: Creating lists of content on your Drupal site

Planet Drupal - ven, 17/04/2015 - 21:14
Episode Number: 203

In this episode we cover an overview of the Drupal 7 Views module. The Drupal Views module is probably the most popular Drupal module and is installed in almost every Drupal 7 website I build. It’s so popular in fact that it’s included in Drupal 8 by default.

Tags: DrupalViewsDrupal 7Site BuildingDrupal Planet
Catégories: Elsewhere

Drupal.org Featured Case Studies: National Baseball Hall of Fame and Museum

Planet Drupal - ven, 17/04/2015 - 20:30
Completed Drupal site or project URL: http://www.baseballhall.org/

The National Baseball Hall of Fame and Museum (BHoF) is an American institution. For 75 years they have housed the archive of America's favorite game, welcoming new inductees each year and connecting generations with their huge love and knowledge of the sport.

BHoF has a large and dedicated audience, but their location in Central New York limits the number of physical visits to the museum. To reach a wider audience, they needed to unlock the full potential of their online presence.

Cogapp helps organizations use digital media, specializing in large-scale, mission-critical projects for prominent institutions.

BHoF appointed Cogapp to perform a discovery phase to research user engagement, the kinds of content that are of interest to users, and key value propositions of the website to its visitors. This work then fed into developing the site, with the central objective being to showcase the vast number of artifacts in the Hall's collection, creating connections that bring these objects to life for site visitors.

Key modules/theme/distribution used: Islandora Imagecache ExternalParagraphsEntity APIMetatagFeaturesStrongarmMasterVarnish HTTP Accelerator IntegrationOrganizations involved: CogappTeam members: alxbridgechapabutassos
Catégories: Elsewhere

Drupal Watchdog: VIDEO: DrupalCon Amsterdam Interview: Angie Byron

Planet Drupal - ven, 17/04/2015 - 18:59

Angie Byron is Director of Community Development at Acquia. For this interview, during the final day of DrupalCon Amsterdam, we were able to find an empty auditorium. Alas, filming with my brand-new GoPro camera, we got off to a topsy-turvy start...

RONNIE RAY: I’ve had you upside down.

ANGIE BYRON: Oh hahaha!

I go by Angie Byron or webchick, and more people know me as webchick than Angie Byron.

Today, what I love to do at DrupalCons, on the last day of the sprint days, is just walk around all the tables and see what everyone is working on, cause there’s hundreds of people here and they’re all sort of scratching their own itches on everything from Drupal-dot-org to, like, what is the newest coolest content staging thing gonna be?, to how are we going to get Drupal 8 done?

And everybody working together and collaborating with people they don’t get to see all the time, it’s a lot of fun for me.

I feel like we made a lot of really great decisions about the Drupal 8 release management stuff here that we’ll be able to put into practice, and help try and focus efforts on getting the critical issues resolved, trying to clean up the loose ends that we still have, and getting the release out the door faster.

And the other thing I’m going to work on for the next month is something called Drupal Module Upgrader, which is the script that can help contrib modules port their modules to Drupal 8. It automates a lot of that task.

Now that Beta is here it’s a great time for people to update their modules, so I want to work on tools to help facilitate that.

RR: What are you reading, besides books on Drupal?

AB: Not much. Although I love reading kids books, because I have a daughter who’s 16 months now and she loves to be read to. So my latest books I’ve been reading are Where is the Green Sheep? and Go, Dog, Go! and a bunch of Richard Scarry stuff and things like that because she loves to know what everything’s called. She loves books.

There’s a Dr. Seuss book called Oh, The Places You’ll Go! That book is dark, man, that is like a dark book. It’s entertaining. I remember it from when I was a kid but I don’t remember it like that!

RR: Music?

AB: I listen to a lot of old music cause I’m one of those curmudgeonly people who thinks the best music was already made. So, like I’ve been having like a ‘70s rock, ‘80s pop, ‘90s punk rock, like – that’s sort of what’s in my chain all the time. Hair metal, junk like that. How to relive my kid-age stuff.

I think the community has grown to such an enormous size now that I guess one thing I wonder about, – not really worry about– but am curious about, is if can we still maintain that small-knit community feel that we had back when I started, when we were 70 people at a DrupalCon – not the 2,500 people we have now.

It’s cool to kind of walk around DrupalCon, especially on a sprint day, especially because I feel we have retained that – and people are finding people to connect with and cool things to work on and stuff like that.

I think it’s something we all need to collectively be intentional about is, you know, it’s not just enough that Drupal is just a great software project, it’s also about the people and trying to maintain that welcome feeling – that got us all in the door – for generations to come.

So that’s something I would leave as a parting note.

Tags:  DrupalCon DrupalCon Amsterdam Video Video: 
Catégories: Elsewhere

Chapter Three: Presentation: Drupal 8 Module Development

Planet Drupal - ven, 17/04/2015 - 18:15

This session was presented at Bay Area Drupal Camp, San Diego Drupal Camp, Phoenix Drupal Camp, and Stanford Drupal Camp.



Have you written a few simple modules for Drupal 7, and are a little bit nervous to find out the changes you'll be facing in Drupal 8?

Catégories: Elsewhere

Aten Design Group: Speeding up Complex Drupal Data Loads with Custom Caches

Planet Drupal - ven, 17/04/2015 - 17:27

Recently we had the task of loading data from a content type with 350 fields. Each node is a University’s enrollment data for one year by major, gender, minority, and a number of other categories. CSV exports of this data obviously became problematic. Even before we got to 350 fields, with the overhead of the Views module we would hit PHP timeouts when exporting all the nodes. If you’re not familiar with Drupal's database structure, each field’s data is stored in a table named ‘field_data_FIELDNAME’. Loading an entire node means JOINing the node table by entity_id with each related field table. When a node only has a handful of fields, those JOINs work fine, but at 350 fields the query runs slow.

On this site we’re also plotting some of the data using highcharts.js. We really hit a wall when trying to generate aggregate data to plot alongside a single university's. This meant loading every node of this content type to calculate the averages, which turned our slow query into a very slow query. We even hit a limit on the number of database JOINs that can be done at one time.

In retrospect this is a perfect case for a custom entity, but we already had thousands of nodes in the existing content type. Migrating them and implementing a custom entity was no longer a good use of time. Instead, we added a custom table that keeps all the single value fields in a serialized string.

The table gets defined with a hook_schema in our module's .install file:

function ncwit_charts_schema() {   $schema['ncwit_charts_inst_data'] = array( 'description' => 'Table for serialized institution data.', 'fields' => array( 'nid' => array( 'type' => 'int', 'default' => 0, 'not null' => TRUE, 'description' => 'node id for this row', ), 'tid' => array( 'type' => 'int', 'default' => 0, 'not null' => TRUE, 'description' => 'intitution term id that this data belongs to', ), 'year' => array( 'type' => 'int', 'default' => 0, 'not null' => TRUE, 'description' => 'school year for this node', ), 'data' => array( 'type' => 'blob', 'not null' => FALSE, 'size' => 'big', 'serialize' => TRUE, 'description' => 'A serialized array of name value pairs that store the field data for a survey data node.', ), ), 'primary key' => array('nid'), );   return $schema; }

The most important part of the array is 'data' with type 'blob', which can be up to 65kB. Not shown is another array to create a table for our aggregate data.

When a new node is saved hook_node_insert() is invoked. hook_node_update() fires both when a new node is saved and when it's updated.

/** * Implements hook_node_insert(). * save serialized field data to inst_data table for a new node * For a new node, have to use this */ function ncwit_charts_node_insert($node) { ncwit_charts_serialize_save($node); }     /** * Implements hook_node_update(). * save serialized field data to inst_data table */ function ncwit_charts_node_update($node) { if (isset($node->nid)) { // we're also calling this function from hook_node_insert // because hook_node_update doesn't have the nid if is a new node ncwit_charts_serialize_save($node); } else { return; } }

Now we actually process the fields to be serialized and store. This section will vary greatly depending on your fields.

function ncwit_charts_serialize_save($node) { // save each value as a simple key => value item foreach ($node as $key => $value) { $data[$key] = $value[LANGUAGE_NONE][0]['value']; }   $fields = array(); $fields['nid'] = $node->nid; $fields['tid'] = $node->field_institution_term[LANGUAGE_NONE][0]['tid']; $fields['year'] = $node->field_school_year[LANGUAGE_NONE][0]['value']; $fields['data'] = serialize($data);   db_merge('ncwit_charts_inst_data') ->key(array( 'nid' => $node->nid, )) ->fields($fields) ->execute();

When a node is deleted we have some clean-up to do.

/** * Implements hook_node_delete(). * Also remove node's data from inst_data */ function ncwit_charts_node_delete($node) { if ($node->type !== 'data_survey') { //only care about data_survey nodes return; }   $query = db_select('ncwit_charts_inst_data', 'i'); $query->fields('i')->condition('i.nid', $node->nid); $result = $query->execute(); $data = $result->fetchAssoc(); if ($data > 0) { db_delete('ncwit_charts_inst_data')->condition('nid', $node->nid)->execute(); } }

When first installed or when fields get changed, we added a batch process that re-saves the serialized strings. Aggregate data is calculated during cron and saved in another table. Rather than loading every node with JOINs, the data comes from a simple query of this custom table.

Pulling the data out of the database and calling unserialize() gives us a simple associative array of the data. To pass this data to highcharts.js we have a callback defined that returns the arrays encoded as JSON. Obviously this gets more complicated when dealing with multiple languages or multi-value fields. But in our case almost everything is a simple integer.

This process of caching our nodes as serialized data changed our loading speed from painfully slow to almost instant. If you run into similar challenges, hopefully this approach will help you too.

Catégories: Elsewhere

EvolvisForge blog: Tricks for using Googlemail at work

Planet Debian - ven, 17/04/2015 - 15:54

For these who similarily suffer from having to use Googlemail at work. If anyone else has more of these, please do share.

Deactivate the spamfilter

The site admins can do that. Otherwise, you will have work-relevant eMails, for example from your own OTRS system, end up in Spam (where you don’t see it, as their IMAP sucks) and deleted without asking 30 days later. (AIUI, the only way to get eMails actually deleted from Google…)

Do not use their SMTP service

Use your own outgoing MTA. This brings back the, well, not feature but should-have-been-granted-but-Google-doesn’t-do-it-anyway that, when you write to a mailing list, you also get your own messages into your own INBOX.

Calendars…

I have no solutions for this. I stopped using the Googlemail calendars because they didn’t think it a problem that, when I accept an invitation in Kontact (KDEPIM as packaged in Debian sid), the organiser of the calendar item in the sender’s calendar (for which I do not have write permissions) changes to me (so the actual meeting organiser cannot change anything afterwards) and/or calendar items get doubled. I now run a local uw-imapd (forward-ported to sid by means of a binNMU) for sent-mail folders etc. and a local iCalendar directory for calendars.

Catégories: Elsewhere

Midwestern Mac, LLC: Thoughts on the Acquia Certified Developer - Front End Specialist Exam

Planet Drupal - jeu, 16/04/2015 - 22:34

Previously, I posted my thoughts on the Acquia Certified Developer - Back End Specialist exam as well as my thoughts on the Certified Developer exam. To round out the trifecta of developer-oriented exams, I took the Front End Specialist exam this morning, and am posting some observations for those interested in taking the exam.

My Theming Background

I started my Drupal journey working on design/theme-related work, and the first few Drupal themes I built were in the Drupal 5 days (I inherited some 4.7 sites, but I only really started learning how Drupal's front end worked in Drupal 5+). Luckily for me, a lot of the basics have remained the same (or at least similar) from 5-7.

For the past couple years, though, I have shied away from front end work, only doing as much as I need to keep building out features on sites like Hosted Apache Solr and Server Check.in, and making all my older Drupal sites responsive (and sometimes, mobile-first) to avoid penalization in Google's search rankings... and to build a more usable web :)

Catégories: Elsewhere

Achieve Internet Blog: Is Your Site Ready For Google’s New Search Ranking Algorithm?

Planet Drupal - jeu, 16/04/2015 - 20:35
How does mobile-friendliness affect Google search rankings? Google reports:
Catégories: Elsewhere

Daniel Pocock: Debian Jessie release, 100 year ANZAC anniversary

Planet Debian - jeu, 16/04/2015 - 19:48

The date scheduled for the jessie release, 25 April 2015, is also ANZAC day and the 100th anniversary of the Gallipoli landings. ANZAC day is a public holiday in Australia, New Zealand and a few other places, with ceremonies remembering the sacrifices made by the armed services in all the wars.

Gallipoli itself was a great tragedy. Australian forces were not victorious. Nonetheless, it is probably the most well remembered battle from all the wars. There is even a movie, Gallipoli, starring Mel Gibson.

It is also the 97th anniversary of the liberation of Villers-Bretonneux in France. The previous day had seen the world's first tank vs tank battle between three British tanks and three German tanks. The Germans won and captured the town. At that stage, Britain didn't have the advantage of nuclear weapons, so they sent in Australians, and the town was recovered for the French. The town has a rue de Melbourne and rue Victoria and is also the site of the Australian National Memorial for the Western Front.

Its great to see that projects like Debian are able to span political and geographic boundaries and allow everybody to collaborate for the greater good. ANZAC day might be an interesting opportunity to reflect on the fact that the world hasn't always enjoyed such community.

Catégories: Elsewhere

Acquia: Drupal is fun to use - meet Karen Grey

Planet Drupal - jeu, 16/04/2015 - 18:49
Language Undefined Drupal is more fun - meet Karen Grey

I sat down with Karen Grey at Drupal Camp Brighton 2015 to find out more about who she is and what she does with Drupal. I apologize for taking her out of the code sprints for that time! Since we spoke, Karen has taken on a position as Senior Drupal Developer at i-KOS in their Brighton office.

Catégories: Elsewhere

Drupal Watchdog: RESTful Web Services Module Basics

Planet Drupal - jeu, 16/04/2015 - 17:30
Article

Drupal 7 does not have built-in support for representational state transfer (REST) functionality. However, the RESTful Web Services module is arguably the most efficient way to provide resource representations for all the entity types, by leveraging Drupal's powerful Entity API. Unmodified, the module makes it possible to output the instances of the core entity types – node, file, and user – in JSON or XML format. Further entity type resources and formats are possible utilizing hooks in added code.

As with any REST solution, the RESTful Web Services module supports all four of the fundamental operations of data manipulation: create, read, update, and delete (CRUD). The corresponding RESTful API HTTP methods are POST, GET, PUT, and DELETE, respectively.

Anyone hoping to learn and make use of this module – especially for the first time – will likely be frustrated by the current project documentation, which is incomplete, uneven, and lacking clear examples. This article – a brief overview – is intended to introduce what is possible with this module, and help anyone getting started with it.

We begin with a clean Drupal 7 installation (using the Standard profile) running on a virtual host with the domain name "drupal_7_test". After installing and enabling the module, we find that it does not have the configuration user interface one might expect. In the demonstration code below, we focus on the node entity type.

Nabbing a Node

The simplest operation – reading an entity instance – is performed using a simple GET request containing the machine name of the entity type and the entity's ID.

Catégories: Elsewhere

Drupal Association News: We Love Our Volunteers!

Planet Drupal - jeu, 16/04/2015 - 16:15

This week is National Volunteer Week, a week to recognize that volunteerism is a building block to a strong and thriving community.  The Drupal Community is no different: as an open-source project our volunteers are vital to the health and growth of our project.  There are so many roles and levels of contribution within our Drupal ecosystem that we at the Drupal Association wanted to highlight how much your contribution means to us and our work.  I took some time and asked around, here’s some of the glowing praise our staff has to say about our phenomenal volunteers. 

“I am continually impressed with the volunteers that I get to work with.  Not only do they rock at their jobs, but they are so dedicated to the work that they do for Drupal and the Cons specifically!  Anyone who has volunteered for a Con knows that it is a large undertaking, and a responsibility that isn't taken lightly. These volunteers come back each week with positive attitudes, valuable ideas and great results.  Although I have only been at the Association for a little over six months, I can truly say that these volunteers are what gives our Cons the 'special sauce' and I am lucky to get to work with volunteers from around the globe on a daily basis.” 

- Amanda Gosner, DrupalCon Coordinator

“Most of my day is spent with Drupal Association staff, who have the luxury of getting paid to think about Drupal for 8 hours a day. A good chunk of my job is working with volunteers though-- the Board of Directors, Drupal.org Working Groups, Community Organizers, DrupalCon session speakers. So many of you give so much of your time and your smarts back to the project and the community, and it's my privilege and duty to learn from you all.”

- Holly Ross, Executive Director

"I look forward to working working with community volunteers to help build and improve Drupal.org. The site would not be where it is today without everyone's work."

- Neil Drumm, Drupal.org Lead Architect

 

“I want to thank Cathy and Jared for being my sprint mentor at DrupalCon Latin America. I made my first comment on the issue queue. It felt so good to cross into that world finally, even if it is was just a baby toe crossing over.”

- Megan Sanicki, COO

 

“It feels like I’m hearing news every day about the amazing programs our community members put together all over the world — from Los Angeles to Uganda and beyond. Without help from amazing community volunteers who donate time working on social media, in the issue queues, or even volunteers who take a brief moment to drop a note in my inbox (“have you seen this?”), these stories would never be shared with our wider community.” 

- Leigh Carver, Content Writer

Today, we invite you to take a few minutes to recognize your fellow Drupal contributors by tweeting or sending a message via IRC to appreciate each other.  After all, without our volunteers, our Drupal Community would not be as lively, bright, and welcoming.  Want to lend a hand?  Our get involved page has plenty of ways to volunteer with the project.

Catégories: Elsewhere

KnackForge: Drupal 7 - Hooking Ajax events and views refresh

Planet Drupal - jeu, 16/04/2015 - 15:40
Drupal has a solid Ajax interface, we can hook into the Ajax events at various places. I will explain some 5 important methods,   1) beforeSerialize - called before data is packed and runs before the beforeSend & beforeSubmit 2) beforeSubmit - called before the ajax request 3) beforeSend - called just before the ajax request 4) success - called after ajax event returns data 5) complete - called after the request ends   Lets say you want to capture some ajax event (in built or made by other module) to do some event like Views refresh. We can use a very simple logic to do that.  
Catégories: Elsewhere

Pronovix: Prototyping LinkForward with Drupal, a startup case study

Planet Drupal - jeu, 16/04/2015 - 15:10

This is the story of how I built a first prototype for LinkForward, a web application for power networkers, and how I built it in record time using Drupal (12 hours), without a single line of custom code.

Catégories: Elsewhere

Pages

Subscribe to jfhovinne agrégateur