Feed aggregator

Greater Los Angeles Drupal (GLAD): Drupalgeddon followup and Security Presentation on November 20th in Pasadena, CA

Planet Drupal - Tue, 18/11/2014 - 21:40

As you may have heard, Drupal has recently had a significant security announcement nicknamed "Drupalgeddon". You may not have heard, though, that here in Los Angeles, California, we're dedicating a portion of several upcoming meetups to web security and helping provide our group members with any resources they need.

If you can make it, please join us this week for a special presentation, Better Sleep Through Web Security. It's this Thursday, November 20th, at 6pm in Pasadena, California.

If you haven't heard of Drupalgeddon or don't know if your Drupal sites have been updated since the announcement, please stop reading and see the SA-CORE-2014-005 FAQ immediately. You need to take action and contact a service provider about protecting your Drupal site and hosting environment.

   Date and time: November 20, 2014 at 6pm Pacific Time
   Location: Fuller Theological Seminary, at 135 N Oakland Ave
Pasadena, CA 91101 (Building "Glasser 110")
   Video conference: https://glad.zoom.us/j/129319220
   Phone: +1 415-762-9988 or +1 646-568-7788
   Meeting ID: 129 319 220

This meetup will have a video conference for those who can attend only by video conference or phone. The video conference is with Zoom, one of our many great sponsors, and you can join with iPhone, Android, PC and Mac at https://glad.zoom.us/j/129319220

Better Sleep Through Web Security

Christefano Reyes presents Better Sleep Through Web Security, an in-depth overview of web security, what to do do if your website is hacked, and how to sleep better by following basic web security best practices.

The "Drupalgeddon" vulnerability has been covered in mainstream news including Forbes, the BBC and The Register, and has brought web security, frequently an overlooked part of web development, back to the center stage.

This particular vulnerability, officially known as SA-CORE-2014-005, allows attackers with specialized knowledge to send requests to any unprotected Drupal website that result in arbitrary SQL execution, which in turn may lead to privilege escalation, arbitrary PHP execution and total server control.

Topics that will be covered in this presentation include:

  • Security vs. Privacy
  • Common Attack Vectors
  • Drupal's security record and the Drupal Security Team
  • SA-CORE-2014-005 (also known as "Drupalgeddon")
  • I've Been Hacked! Now What?
  • Best Practices for Helping Others and Yourself
  • Resources
  • Questions / Answers

Christefano is one of the founders of Exaltation of Larks, a Drupal design and engineering firm with a worldwide team of Drupal experts; and Droplabs, an open source-friendly coworking space and business incubator near Downtown Los Angeles. As an advocate of open source software and self-declared meetup junky, he helps organize meetups and conferences all over the Greater Los Angeles Area, including the Los Angeles Chess meetup and LA Geek Dinners.

We hope to see you at Thursday's meetup!

Tags: DrupalgeddonDrupageddonSan Gabriel Valley DrupalPlanet Drupal
Categories: Elsewhere

Christian Perrier: Bug #770000

Planet Debian - Tue, 18/11/2014 - 19:13
Martin Pitt reported Debian bug #770000 on Tuesday November 18th, against the release.debian.org pseudo-package.

Bug #760000 was reported as of August 30th: so there have been 10,000 bugs reported in 3 months minus 12 days. The bug rate increased quite significantly during the last weeks. We can suspect this is related to the release and the freeze (that triggers many unblock requests)

I find it interesting that this bug is directly related to the release, directly related to systemd and originated from one of the systemd packages maintainers, if I'm right.

So, I'll take this opportunity to publicly thank all people who have brought the systemd packages to what they are now, whether or not they're still maintaining the package. We've all witnessed that Debian if facing a strong social issue nowadays and I'm very deeply sad about this. I hope we'll be able to go through this without losing too many brilliant contributors, as it happened recently.

Please prove me right and do The Right Thing for me to be able to continue this silly "round bug number" contest and still believe that, some day, bug #1000000 will really happen and I'm still there to witness it.

Ah, and by the way, systemd bloody works on my system. I can't even remember when I switched to it. It Just Worked.

Categories: Elsewhere

Stanford Web Services Blog: BADCamp 2014: Caryl’s Highlights

Planet Drupal - Tue, 18/11/2014 - 18:00

BADCamp is one of the highlights in my Drupal world. It’s like a Drupalcon (Drupal Convention) with lots of people and sessions, but more low key. It’s a great way to meet people and to learn more about Drupal. Here are some of my highlights:

Higher Ed Summit

This was a great chance to learn about the issues facing many of our colleagues at other universities.

Categories: Elsewhere

Stanford Web Services Blog: BADCamp 2014: Megan's take-aways from the Front End Summit

Planet Drupal - Tue, 18/11/2014 - 18:00

During BADCamp this year, I participated in the Front End Summit. We learned about Drupal 8, had a series of lightning talks, and a lively panel featuring a diverse range of voices from the Drupal front end community. Here are some of my take-aways.

Drupal 8 Core Rocks

Learning more about Drupal 8, it’s exciting to see that so many of the modules we work with in our “toolkit” on Stanford Sites have been moved into core in Drupal 8, like Menu Block, for example. It validates the direction we are going with Sites and with our stack.

Categories: Elsewhere

Stanford Web Services Blog: BADCamp 2014: Joe's take-aways from the Higher Ed Summit

Planet Drupal - Tue, 18/11/2014 - 18:00

We’ve got a first timer here! I attended the Higher Ed Summit at my first ever BADCamp this year. I learned about Drupal’s presence on numerous college campuses, software as a service, building tools, project management, and Drupal communities. Here are just a few of my take-aways.

Categories: Elsewhere

Stanford Web Services Blog: BADCamp 2014: Linnea's thoughts from the Higher Ed Summit

Planet Drupal - Tue, 18/11/2014 - 18:00

During BADCamp this year, I participated in the Higher Ed Summit. We learned about how other universities are rolling out Drupal and central web policies through a panel discussion, had a series of lightning talks, a number of birds-of-a-feather discussions, and two featured talks. Here are some of my take-aways.

Categories: Elsewhere

Stanford Web Services Blog: BADCamp 2014: John's highlights

Planet Drupal - Tue, 18/11/2014 - 18:00

This year marked the 8th Bay Area Drupal Camp (BADCamp), and likewise it was my 8th year attending. BADCamp is one of the largest Drupal events other than DrupalCon itself, and I am always amazed at the sheer size of the event, the generosity of the sponsors, the hard work of the volunteers, and the quality of all the participants.

Categories: Elsewhere

Michal Čihař: Mercurial support in Weblate

Planet Debian - Tue, 18/11/2014 - 18:00

Weblate has started as a translation system tightly bound to Git version control system. This was in no means design decision, but rather it was the version control I've used. But this has shown not to be sufficient and other systems were requested as well. And Mercurial is first of them to be supported.

Weblate 2.0 already had separated VCS layer and adding another system to that is quite easy if you know the VCS you're adding. Unfortunately this wasn't the case for me with Mercurial as I've never used it for anything more serious than cloning a repository, committing fixes and pushing it back. Weblate needs a bit more than that, especially in regard to remote branches. But nevertheless I've figured out all operations and the implementation is ready in our Git.

In case somebody is interested in adding support for another version control, patches are always welcome!

Filed under: English phpMyAdmin SUSE Weblate | 0 comments | Flattr this!

Categories: Elsewhere

Phase2: Transitioning to Drupal 8 templates with Twig

Planet Drupal - Tue, 18/11/2014 - 16:08

As many of us know, Drupal 8 beta was released at the beginning of October which has given us a great preview of the goodies to come. One of those goodies is a new theme engine, Twig, which replaces PHPTemplate. Twig gives us a lot of extra power over our templates and allows us to clean up a lot of the extra cruft that PHPTemplate forced us to add.

So you may be asking, how will our brand new Drupal 8 templates look? Well they are going to be a lot leaner. The extra cruft of php tags, functions, and other miscellaneous bits are no longer needed – in fact, they don’t even work in Twig templates. Arguably, it will be easier to read at a glance what is happening in the Twig versions of Drupal templates. For a great example of how the new templates will look, we can crack open the Views module and compare the templates in the Views module 7.x version to the ones in the Views 8.x version. So let’s compare the views-view.tpl.php file from Views 7.x to views-view.html.twig file in Views 8.x.

Twig Comments

Starting from the top, lets work our way down to see what has changed. In the documentation section we can see that for the large block of comments, as well as single line comments, the {# #} syntax is used.

In the list of variables variables that are available to use, you may notice a few other changes. For example, Twig is not raw PHP, so the ‘$’ is not used at the beginning of variable names. Another change is that is that the $classes_array has been replaced by the attributes variable.

Rendering Variables

 

Moving down to line #40 we can see the first instance of a variables rendered using Twig. The double curly bracket syntax, {{ my_variable }}, tells Twig that this is a variable and it needs to be rendered out. A variation of this syntax uses dot-notation to render variables contained in an array/object. This syntax looks like {{ my_variable.property_or_element }}, which makes it extremely easy to use. Dot notation is not used in this particular Twig template.

Twig Filters

Another powerful feature of Twig is template filters. Filters allow us to do simple manipulations of variables. The syntax for using a filter is similar to the syntax for rendering a variable. The difference is that after the variable name, you insert a | (pipe), followed by the name of the filter. For an example to make a string from a variable all lowercase you would do {{ variable_name|lower }} which transform all of the characters to lowercase. If you have used templating systems from other frameworks, such as Angular.js, this syntax may look familiar to you. This particular Views template does not use filters, but you can see examples of different filters on the Twig documentation site. If none of the predefined filters satisfy your requirements, you can extend Twig to create your own filter. The Twig documentation site provides details about creating your own custom filters..

Twig Logic

Jumping to line #42, we can see the {% %} (curly-bracket and percentage symbol) syntax, which is used for template logic, such as if statements. This syntax tells Twig that we are not rendering something, but rather that we need to process some logic, such as a control structure or loop, in our template.

The Takeaway

This blog post is a high level overview of how Twig templates in Drupal 8 will look.  With Twig, we can choose to use the out-of-the box tools it provides, or we can dive in and extend it with additional features such as new filters. For more information I would highly recommend reading through the Twig documentation for designers and for developers.

Categories: Elsewhere

Drupalize.Me: Twig Filters: Modifying Variables in Drupal 8 Template Files

Planet Drupal - Tue, 18/11/2014 - 16:00
Something that's super fun about my job is that occasionally I get tasked with things like, "Learn how Twig works so you can tell us how it fits into our curriculum plans.". And I get to spend some time exploring various new features in Drupal 8, with an eye towards being able help explain them.
Categories: Elsewhere

Drupal.org Featured Case Studies: KSS Architects

Planet Drupal - Tue, 18/11/2014 - 15:42
Completed Drupal site or project URL: http://kssarchitects.com/

KSS Architects is a nationally recognized architecture firm with offices in Princeton, New Jersey and Philadelphia, Pennsylvania. KSS works with clients in the fields of education, culture, land development, urban development, and corporate environments.

After completing brand strategy for KSS, TOKY Branding + Design created a website that would better showcase the architecture firm’s personality, process, and projects. TOKY specializes in Web and print work for clients in architecture, building, and design, as well as the arts, education, and premium consumer products.

Key modules/theme/distribution used: BreakpointsCampaign MonitorDate Popup AuthoredEntity referenceField collectionGoogle Site SearchjQuery UpdateMaxlengthMediaMediaElementMenu blockMetatagPictureTaxonomy displayTypogrifyview_unpublishedVocabulary Permissions Per RoleOrganizations involved: TOKY Branding + DesignTeam members: Daniel Korte
Categories: Elsewhere

Joachim's blog: Building Fast and Flexible Application UIs with Entity Operations

Planet Drupal - Tue, 18/11/2014 - 14:38

Now I've finished the Big Monster Project of Doom that I've been on the last two years, I can talk more about some of the code that I wrote for it. I can also say what it was: it was a web application for activists to canvass the public for a certain recent national referendum (I'll let you guess which one).

One of the major modules I wrote was Entity Operations module. What began as a means to avoid repeating the same code each time I needed a new entity type soon became the workhorse for the whole application UI.

The initial idea was this: if you want a custom entity type, and you want a UI for adding, editing, and deleting entities (much like with nodes), then you have to build this all yourself: hook_menu() items, various entity callbacks, form builders (and validation and submit handlers) for the entity form and the delete confirmation form. (The Model module demonstrates this well.)

That's a lot of boilerplate code, where the only difference is the entity type's name, the base path where the entity UI sits, and the entity form builder itself (but even that can be generalized, as will be seen).

Faced with this and a project on which I knew from the start I was going to need a good handful of custom entities (for use with Microsoft Dynamics CRM, accessed with another custom module of mine, Remote Entity API), I undertook to build a framework that would take away all the repetition.

An Entity UI is thus built by declaring:

  • A base path (for nodes, this would be 'node'; we'll ignore the fact that in core, this path itself is a listing of content).
  • A list of subpaths to form the tabs, and the operation handler class for each one

With this in hand, why stop at just the entity view and edit tabs? The operation handlers can output static content or forms: they can output anything. One of the most powerful enhancements I made to this early on was to write an operations handler that outputs a view. It's the same idea as the EVA module.

So for the referendum canvassing application, I had a custom Campaign entity, that functioned as an Organic Group, and had as UI tabs several different views of members, views of contacts in the Campaign's geographic area, views of Campaign group content (such as tasks and contact lists), and so on.

This approach proved very flexible and quick. The group content entities were themselves also built with this, so that, for example, Contact List entities had operations for a user to book the entity, input data, and release it when done working on it. These were built with custom operation handlers specific to the Contact List entity, subclassing the generic form operation handler.

An unexpected bonus to all this was how easy it was to expose form-based operations to Views Bulk Operations and Services (as 'targeted actions' on the entity). This allowed the booking and release operations to work in bulk on views, and also to be done via a mobile app over Services.

A final piece of icing on the cake was the addition of alternative operation handlers for entity forms that provide just a generic bare bones form that invokes Field API to attach field widgets. With these, the amount of code needed to build a custom entity type is reduced to just three functions:

  • hook_entity_info(), to declare the entity type to Drupal core
  • hook_entity_operations_info(), to declare the operations that make up the UI
  • callback_entity_access(), which controls the access to the operations

The module has a few further tricks up its sleeve. If you're using user permissions for your entities, there's a helper function to use in your hook_permission(), which creates permissions out of all the operations (so: 'edit foobar entities', 'book foobar entities', 'discombobulate foobar entities' and so on). The entity URI callback that Drupal core requires you to have can be taken care of by a helper callback which uses the entity's base path definition. There's a form builder that lets you easily embed form-based operations into the entity build, so that you can put the sort of operations that are single buttons ('publish', 'book', etc) on the entity rather than in a tab. And finally, the links to operation tabs can be added to a view as fields, allowing a list of entities with links to view, edit, book, discombobulate, and so on.

So what started as a way to simplify and remove repetitive code became a system for building a whole entity-based UI, which ended up powering the whole of the application.

Categories: Elsewhere

Dirk Eddelbuettel: RcppAnnoy 0.0.3

Planet Debian - Tue, 18/11/2014 - 12:48

Hours after the initial blog post announcing the first release of the new package RcppAnnoy, Qiang Kou sent us a very nice pull request adding mmap support in Windows.

So a new release with Windows support is on now CRAN, and Windows binaries should be available by this evening as usual.

To recap, RcppAnnoy wraps the small, fast, and lightweight C++ template header library Annoy written by Erik Bernhardsson for use at Spotify. RcppAnnoy uses Rcpp Modules to offer the exact same functionality as the Python module wrapped around Annoy.

Courtesy of CRANberries, there is also a diffstat report for this release. More detailed information is on the RcppAnnoy page page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Categories: Elsewhere

Jonathan Wiltshire: Getting things into Jessie (#3)

Planet Debian - Tue, 18/11/2014 - 12:16
Make sure everything you’ve changed is in the changelog

We do read the diffs in detail, and if there’s no explanation for something that’s changed we’ll ask. We also expect it to be in the changelog.

Do save some round-trips by making sure your changelog is in order. One round-trip about your package is an inconvenience; when it’s scaled up to the number of requests we receive, it’s a serious time-sink for us.

Getting things into Jessie (#3) is a post from: jwiltshire.org.uk | Flattr

Categories: Elsewhere

Josselin Mouette: Introspection (not the GObject one)

Planet Debian - Tue, 18/11/2014 - 11:00
Disclaimer: I’m not used to writing personal stuff on Debian channels. However, there is nothing new here for those who know me from other public channels.


Yesterday, I received the weirdest email from well-known troll MikeeUSA. He thought I shared his views of a horrible world full of bloodthirsty feminists using systemd in their quest for domination over poor white male heterosexuals. The most nauseating paragraph was probably the one where he showed signs of the mentality of a pedocriminal.

At first, I shrugged it off and sent him an email explaining I didn’t want anything with his stinky white male supremacist theories, assorted with a bit of taunting. But after discovering all that stuff was actually sent to public mailing lists, I took the time for a second look and started a bit of introspection.

MikeeUSA thought I was a white male supremacist because of the so-called SmellyWerewolf incident, 6 years ago.
Oh boy, people change in six years. Upon re-reading that, I had trouble admitting I was the one to write it. Memory is selective, and with time, you tend not to remember some gruesome details, especially the ones that conflict most with your moral values.

I can assure every reader that the only people I intended to mock then were those who mistook Debian mailing lists for advertising channels; but I understand now that my message must have caused pain to a lot more people than that. So, it may come late, but let me take this opportunity to offer my sincerest apologies to anyone I may have hurt at that time.


It may seem strange for someone with deeply-rooted values of equality to have written that. To have considered that it was okay to stereotype people. And I think I found this okay because to me, those people were given equal rights, and were therefore equal. But the fight for equality is not over when everyone is given the same rights. Not until they are given the same opportunities to exert those rights. Which does not happen when they live in a society that likes to fit them in little archetypal peg holes, never giving you the chance to question where those stereotypes come from.

For me, that chance came from an unusual direction: the fight against prostitution. This goes way back for me. Since when I was a teenager, I have always been ticked off at the idea of nonconsensual sex that somehow evades criminal responsibility because of money compensation. I never understood why it wasn’t considered as rape. Yet it sounded weird that a male heterosexual would hold such opinions; after all, male heterosexuals should go to prostitutes as a kind of social ritual, right?

It was only three years ago that an organization of men against prostitution was founded in France. Not only did I find out that I was not alone with my progressive ideas, I was given the opportunity to exchange with many men and women who had studied prostitution: its effects on victims, its relationship to rape culture and more generally to the place men and women hold in society. Because eventually, it all boils down to little peg holes in which we expect people to fit: the virile man or the faggot, the whore or the mother. For me, it was liberating. I could finally get rid of the discomfort of being a white male heterosexual that didn’t enter the little peg holes that were made for me.

And now, after Sweden 15 years ago, a new group of countries are finally adopting laws to criminalize the act of paying for sex. Including France. That’s too bad for MikeeUSA, but this country is no longer the eldorado for white male supremacists. And I’m proud that our lobbying made a contribution, however small, to that change.
Categories: Elsewhere

Oliver Davies: Include CSS Fonts by Using a SASS each Loop

Planet Drupal - Tue, 18/11/2014 - 10:39

Using a file structure similar to this, organise your font files into directories, using the the font name for both the directory name and for the file names.

Tags:
Categories: Elsewhere

Promo Kids: Reinstall Drupal 8 without pain

Planet Drupal - Tue, 18/11/2014 - 10:09

While developing Drupal Promo Kit we reinstall Drupal very frequently.
First of all, you don't have to delete everything and repeat installation process from scratch. Only three things should be done:

Categories: Elsewhere

Erich Schubert: Generate iptables rules via pyroman

Planet Debian - Tue, 18/11/2014 - 09:46
Vincent Bernat blogged on using Netfilter rulesets, pointing out that inserting the rules one-by-one using iptables calls may leave your firewall temporarily incomplete, eventually half-working, and that this approach can be slow. He's right with that, but there are tools that do this properly. ;-) Some years ago, for a multi-homed firewall, I wrote a tool called Pyroman. Using rules specified either in Python or XML syntax, it generates a firewall ruleset for you. But it also adresses the points Vincent raised:
  • It uses iptables-restore to load the firewall more efficiently than by calling iptables a hundred times
  • It will backup the previous firewall, and roll-back on errors (or lack of confirmation, if you are remote and use --safe)
It also has a nice feature for the use in staging: it can generate firewall rule sets offline, to allow you reviewing them before use, or transfer them to a different host. Not all functionality is supported though (e.g. the Firewall.hostname constant usable in python conditionals will still be the name of the host you generate the rules on - you may want to add a --hostname parameter to pyroman) pyroman --print-verbose will generate a script readable by iptables-restore except for one problem: it contains both the rules for IPv4 and for IPv6, separated by #### IPv6 rules. It will also annotate the origin of the rule, for example: # /etc/pyroman/02_icmpv6.py:82 -A rfc4890f -p icmpv6 --icmpv6-type 255 -j DROP indicates that this particular line was produced due to line 82 in file /etc/pyroman/02_icmpv6.py. This makes debugging easier. In particular it allows pyroman to produce a meaningful error message if the rules are rejected by the kernel: it will tell you which line caused the rule that was rejected. For the next version, I will probably add --output-ipv4 and --output-ipv6 options to make this more convenient to use. So far, pyroman is meant to be used on the firewall itself. Note: if you have configured a firewall that you are happy with, you can always use iptables-save to dump the current firewall. But it will not preserve comments, obviously.
Categories: Elsewhere

Jaldhar Vyas: And The Papers Want To Know Whose Shirts You Wear

Planet Debian - Tue, 18/11/2014 - 07:52

Today I was walking past the Courant Institute at NYU when I saw a man wearing a t-shirt with a picture of a cow diagramming all the various cuts of beef.

Now I've lost all interest in science. Thanks a lot jerks.

Categories: Elsewhere

Antoine Beaupré: bup vs attic silly benchmark

Planet Debian - Tue, 18/11/2014 - 06:39

after see attic introduced in a discussion about bup, i figured out i could give it a try. it was answering two of my biggest concerns with bup:

  • backup removal
  • encryption

and seemed to magically out of nowhere and basically do everything i need, with an inline manual on top of it.

disclaimer

Note: this is not a real benchmark! i would probably need to port bup and attic to liw's seivot software to report on this properly (and that would amazing and really interesting, but it's late now). even worse, this was done on a production server with other stuff going on so take results with a grain of salt.

procedure and results

Here's what I did. I setup backups of my ridiculously huge ~/src directory on the external hard drive where I usually make my backups. I ran a clean backup with attic, than redid it, then I ran a similar backup with bup, then redid it. Here are the results:

anarcat@marcos:~$ sudo apt-get install attic # this installed 0.13 on debian jessie amd64 [...] anarcat@marcos:~$ attic init /mnt/attic-test: Initializing repository at "/media/anarcat/calyx/attic-test" Encryption NOT enabled. Use the "--encryption=passphrase|keyfile" to enable encryption. anarcat@marcos:~$ time attic create --stats /mnt/attic-test::src ~/src/ Initializing cache... ------------------------------------------------------------------------------ Archive name: src Archive fingerprint: 7bdcea8a101dc233d7c122e3f69e67e5b03dbb62596d0b70f5b0759d446d9ed0 Start time: Tue Nov 18 00:42:52 2014 End time: Tue Nov 18 00:54:00 2014 Duration: 11 minutes 8.26 seconds Number of files: 283910 Original size Compressed size Deduplicated size This archive: 6.74 GB 4.27 GB 2.99 GB All archives: 6.74 GB 4.27 GB 2.99 GB ------------------------------------------------------------------------------ 311.60user 68.28system 11:08.49elapsed 56%CPU (0avgtext+0avgdata 122824maxresident)k 15279400inputs+6788816outputs (0major+3258848minor)pagefaults 0swaps anarcat@marcos:~$ time attic create --stats /mnt/attic-test::src-2014-11-18 ~/src/ ------------------------------------------------------------------------------ Archive name: src-2014-11-18 Archive fingerprint: be840f1a49b1deb76aea1cb667d812511943cfb7fee67f0dddc57368bd61c4bf Start time: Tue Nov 18 00:05:57 2014 End time: Tue Nov 18 00:06:35 2014 Duration: 38.15 seconds Number of files: 283910 Original size Compressed size Deduplicated size This archive: 6.74 GB 4.27 GB 116.63 kB All archives: 13.47 GB 8.54 GB 3.00 GB ------------------------------------------------------------------------------ 30.60user 4.66system 0:38.38elapsed 91%CPU (0avgtext+0avgdata 104688maxresident)k 18264inputs+258696outputs (0major+36892minor)pagefaults 0swaps anarcat@marcos:~$ sudo apt-get install bup # this installed bup 0.25 anarcat@marcos:~$ free && sync && echo 3 | sudo tee /proc/sys/vm/drop_caches && free # flush caches anarcat@marcos:~$ export BUP_DIR=/mnt/bup-test anarcat@marcos:~$ bup init Dépôt Git vide initialisé dans /mnt/bup-test/ anarcat@marcos:~$ time bup index ~/src Indexing: 345249, done. 56.57user 14.37system 1:45.29elapsed 67%CPU (0avgtext+0avgdata 85236maxresident)k 699920inputs+104624outputs (4major+25970minor)pagefaults 0swaps anarcat@marcos:~$ time bup save -n src ~/src Reading index: 345249, done. bloom: creating from 1 file (200000 objects). bloom: adding 1 file (200000 objects). bloom: creating from 3 files (600000 objects). Saving: 100.00% (6749592/6749592k, 345249/345249 files), done. bloom: adding 1 file (126005 objects). 383.08user 61.37system 10:52.68elapsed 68%CPU (0avgtext+0avgdata 194256maxresident)k 14638104inputs+5944384outputs (50major+299868minor)pagefaults 0swaps anarcat@marcos:attic$ time bup index ~/src Indexing: 345249, done. 56.13user 13.08system 1:38.65elapsed 70%CPU (0avgtext+0avgdata 133848maxresident)k 806144inputs+104824outputs (137major+38463minor)pagefaults 0swaps anarcat@marcos:attic$ time bup save -n src2 ~/src Reading index: 1, done. Saving: 100.00% (0/0k, 1/1 files), done. bloom: adding 1 file (1 object). 0.22user 0.05system 0:00.66elapsed 42%CPU (0avgtext+0avgdata 17088maxresident)k 10088inputs+88outputs (39major+15194minor)pagefaults 0swaps

Disk usage is comparable:

anarcat@marcos:attic$ du -sc /mnt/*attic* 2943532K /mnt/attic-test 2969544K /mnt/bup-test

People are encouraged to try and reproduce those results, which should be fairly trivial.

Observations

Here are interesting things I noted while working with both tools:

  • attic is Python3: i could compile it, with dependencies, by doing apt-get build-dep attic and running setup.py - i could also install it with pip if i needed to (but i didn't)
  • bup is Python 2, and has a scary makefile
  • both have an init command that basically does almost nothing and takes little enough time that i'm ignoring it in the benchmarks
  • attic backups are a single command, bup requires me to know that i first want to index and then save, which is a little confusing
  • bup has nice progress information, especially during save (because when it loaded the index, it knew how much was remaining) - just because of that, bup "feels" faster
  • bup, however, lets me know about its deep internals (like now i know it uses a bloom filter) which is probably barely understandable by most people
  • on the contrary, attic gives me useful information about the size of my backups, including the size of the current increment
  • it is not possible to get that information from bup, even after the fact - you need to du before and after the backup
  • attic modifies the files access times when backing up, while bup is more careful (there's a pull request to fix this in attic, which is how i found out about this)
  • both backup systems seem to produce roughly the same data size from the same input
Summary

attic and bup are about equally fast. bup took 30 seconds less than attic to save the files, but that's not counting the 1m45s it took indexing them, so on the total run time, bup was actually slower. attic is also (almost) two times faster on the second run as well. but this could be within the margin of error of this very quick experiment, so my provisional verdict for now would be that they are about as fast.

bup may be more robust (for example it doesn't modify the atimes), but this has not been extensively tested and is more based with my familiarity with the "conservatism" of the bup team rather than actual tests.

considering all the features promised by attic, it makes for a really serious contender to the already amazing bup.

Next steps

The properly do this, we would need to:

  • include other software (thinking of Zbackup, Burp, ddar, obnam, rdiff-backup and duplicity)
  • bench attic with the noatime patch
  • bench dev attic vs dev bup
  • bench data removal
  • bench encryption
  • test data recovery
  • run multiple backup runs, on different datasets, on a cleaner environment
  • ideally, extend seivot to do all of that
Categories: Elsewhere

Pages

Subscribe to jfhovinne aggregator