Feed aggregator

John Goerzen: The Simple Joys of the Plains

Planet Debian - Sun, 08/05/2016 - 18:24

We love to go exploring as a family. Last year, we gave Jacob and Oliver a theme: “find places older than Grandpa.” They got creative really quick, realizing that any state park counts (“dirt is older than grandpa!”) as did pretty much any museum. Probably our hit from last year was the visit to the tunnels under Ellinwood, KS.

Beatrice, NE

This year, our theme is “places we can fly to”. A couple of weeks ago, Laura had a conference in the beautiful small town of Beatrice, NE. So all four of us flew up, and Jacob, Oliver, and I found fun activities while Laura was at her conference.

We walked around Beatrice a bit, and I noticed this rails-to-trails area. Jacob and Oliver were immediately interested (since it was railroad-related). They quickly turned it into a game of kick-the-dandelion, trying to kick dandelions off their stems and see how high in the air they could get them. The answer: pretty high.

Of course, you can’t go wrong with swimming. Here’s Oliver getting ready for some swimming.

Right near Beatrice is the Homestead National Monument. Of course, the bales decorated like a minion got their attention.

Like the other national parks, this one has a junior ranger program. You complete a few things in an activity book and take a pledge to protect the park, and then you get a badge and some stickers. Here’s Oliver proudly taking his pledge, holding the new raccoon he bought in their gift shop.

Canyon, TX

Laura and I have been to Canyon, TX, twice — the first was for our honeymoon. Yes, we did get some strange looks when we told people we were going to Amarillo for our honeymoon. But it was absolutely perfect for us. We both enjoy the simple gifts of nature.

We kept thinking “we’ve got to take the boys here”. So this weekend, we did. We flew a Cessna out there.

Almost every little general aviation airport seems to have a bowl of candy, a plate of cookies, or some such thing for people that are flying through. I often let Jacob and Oliver choose ONE item.

They hit the jackpot when we stopped at West Woodward, Oklahoma for fuel and a break. Two whole fridges stocked with stuff: cans of pop in one, and all sorts of snacks in the other. In typical GA fashion, there was a jar in the fridge asking for $1 if you took something. And it clearly hadn’t been emptied in awhile.

They also had a nice lounge and a patio. Perfect for munching while watching the activities on the ramp.

After landing at the beautiful little Tradewind Airport in Amarillo, we ate dinner at Feldman’s Wrong-Way Diner in Canyon, TX. Oh my, was that ever popular with the boys.

The eagerly looked around to find anything that was “wrong” — a plane hanging upside down from the ceiling, a direction sign saying “Tattoine – 30 parsecs”, movie posters hung upside down, whatever it might be. The fact that model trains were whirring past overhead certainly didn’t hurt either.

They had a giant bin of crayons by the entrance. Jacob and Oliver each grabbed a fistful, and decided it would be fun to do some math problems while we wait. Oliver particularly got into that, and was quite accurate on his large addition problems. Impressive for a first-grader!

Of course, the big highlight of the area is Palo Duro Canyon. Jacob and Oliver were so eager to explore the canyon that they were just about bubbling over with excitement the night before. They decided that we should explore one of the most difficult trails in the canyon – one that would take us from the bottom of the canyon all the way to the top and back, about 2.5 miles each way.

And they LOVED it. We’d stop every few minutes to climb on some rocks, smash up some pieces of sandstone, munch on a snack, or even watch a lizard scurry past.

At the “trading post” in the canyon, both boys explored the gift shop. Jacob happily purchased a Texas magnet and Palo Duro Canyon keychain, which he carried around the rest of the weekend. Oliver loves stuffed animals, and he bought a cuddly little (but long) snake. When we got back to the hotel, he tied a couple of knots in it, and it became “snake airlines”. Here is the snake airline taking off.

He named it “Rattletail the friendly snake”, which I thought was a pretty nifty name.

The hotel’s waffle maker made Texas-shaped waffles, clearly a hit!

Saturday, we explored the absolutely massive Panhandle-Plains Historical Museum. (How does something that huge wind up in Canyon, TX?) Both boys enjoyed spending hours there. Here’s Oliver in Pioneer Town (an indoor recreation of a 1900s town) sending a telegraph message.

Oliver wanted to help with the plane. He helped me tie it down in Amarillo, helped check it over during preflight, basically got involved in every part of it. Jacob studied aviation maps (sectionals) with me, planning our flight, figuring out how fast we’d go. I loaded Avare (an Android app) on an old tablet for him, so he had aviation maps in the cockpit just like me. He would be telling us how fast we were going every so often, pointing out landmarks, etc.

When it was time to head back home, both boys wanted to stay longer — a sure sign of a good trip. They wanted to hike another trail in the canyon, go back to the museum, and “eat at Feldman’s 18 more times.” (We got there twice, which was plenty for Laura and me!)

On our drive home, Oliver said, “Dad-o, will you teach me to be a pilot? You should be my flight instructor. Then I could fly everywhere with you.”

Now that just makes a dad’s day.

Categories: Elsewhere

Danny Englander: Drupal 8 Development: 5 Things to Add to Your Theming Toolbox

Planet Drupal - Sun, 08/05/2016 - 17:03

My uncle used to say, "You have to use the right tool for the job." This is no different when it comes to Drupal 8 theming and development. I have been having an absolute ball learning and theming with Drupal 8 the past several months. You can read more about how I got inspired here. In this article, I will outline some of the tools and methods I have been using while I've been building a new Drupal 8 theme.

1. Twig Debugging

One of my primary tools for Drupal 8 theming is Twig debugging so you'll definitely want to enable this while building out a theme and site. Chapter Three wrote a great post on enabling Twig debugging so I won't rehash that here how to get it up and running.

Twig's debug output will really come in handy for printing out:

  • Theme hook names
  • All possible suggested template names
  • Existing templates being used
  • The full path of the current template in use

After you write your own theme hook, those new template names will also show up in the debug output.

<div class="postscript"> <!-- THEME DEBUG --> <!-- THEME HOOK: 'region' --> <!-- FILE NAME SUGGESTIONS: * region--postscript.html.twig * region--postscript-second.html.twig x region.html.twig --> <!-- BEGIN OUTPUT from 'themes/custom/hibiscus/templates/layout/region.html.twig' -->

Sample output from Twig debugging

2. Devel Kint

Kint, part of the Devel module for Drupal 8 is awesome for inspecting arrays that contain your entity info, field names, view modes, and more. Kint is similar to Devel Krumo but for Drupal 8. If you really want to dig in with theming, Kint will be invaluable, especially for writing custom preprocess functions, creating variables, and theme hooks.

3. Search Kint

Search Kint for Drupal 8 is similar to Search Krumo for Drupal 7. It extends Kint by giving you two additional functions that are huge time savers. The first is a search box and select list to search any arrays on the page. The second function is a "get path" box where you can easily copy an array path. This takes the guess work out of things, especially since Kint provides much more info than what we were used to in Krumo.

4. Drupal Console

Drupal Console is fast becoming one of my new favorites. This is an amazing tool which can perform dozens of Drupal 8 tasks efficiently in Terminal. I'm just getting started with Console but I really like it already. Console can import / export config, clear cache, generate module / theme scaffolding, generate entities, create nodes, and much more. There's even has a "learning" mode as well as some auto-complete functionality. So it does a lot of what Drush does but it goes beyond with much needed and added functionality. At this point, I am pretty sure I will be switching over to using Drupal Console instead of Drush.

module module:debug Display current modules available for application module:download Download module or modules in application module:install Install module or modules in the application module:uninstall Uninstall module or modules in the application

Just a few sample commands available out of dozens in Drupal Console

5. Drupal Template Helper

There is also now a Google Chrome plugin called Drupal Template Helper which moves all the your Twig debug output to a Chrome web inspector tab. This makes your main HTML inspection area a lot cleaner and easier to scroll through. I even got this extension running in Opera which has been my browser of choice lately for web development. Note, I've been using Opera for theming for a few reasons. It's built upon Webkit so it's the same familiar Web Inspector UI as Chrome that you're used to. Opera also seems faster, less buggy, and less bloated than Chrome.


This is just a basic rundown of my Drupal 8 Toolbox. I am sure I will pick up more tools as time goes by. Do you have any favorite tools I have not mentioned here?

  • Drupal
  • Twig
  • Theming
  • Drupal Planet
  • Drupal 8
Categories: Elsewhere

Satyam Zode: Google Summer of Code 2016 With Debian Reproducible Builds : Introduction

Planet Debian - Sun, 08/05/2016 - 14:15

This is the first blog post among series of posts which I will be writing throughout the summer about Google Summer of Code 2016 under Debian Reproducible Builds Experience.


I am Satyam Zode I am a final year Computer Science student (Satyam_z on IRC). I live in Pune, India (GMT +5:30). I am pursuing my undergraduate degree in Computer Engineering from Pune Institute of Computer Technology, Pune. I have been programming for the past 4 years. I am well versed in C/C++, Python3, and Golang. My Alioth and Github handles are satyamz-guest and satyamz respectively. I have been using GNU/Linux and free software from last four years. I am an open source enthusiast and I have been following Hacker culture since past three years.

Accepted into Google Summer of Code 2016 under Debian Project:

I am glad that I have got an opportunity to contribute to the Debian Project via Google Summer of Code 2016. I am accepted for project Improving diffoscope tool and reproducibility of Debian packages. This Summer and beyond I will be working with Debian Reproducible Builds team to improve the debbugging tool called Diffoscope (previously known as debbindiff). Thanks a bunch to Debian community, Lunar, Holger Levsen, Reiner Herrmann, Mattia Rizzolo and reproducible-builds folks for giving me this opportunity. Here is my GSoC'16 Proposal. And Yay! It really feels great :smile:

Project details:

I will be working on Diffoscope tool which is debbugging tool developed under reproducible-builds effort. Basically, Diffoscope compares two files and shows the difference in text and html format. Diffoscope is mainly developed to compare two Debian packages which may consist of binary files, tar files, text files etc. Diffoscope helps to identify difference between two Debian packages with respect to timestamps, file ordering etc. Diffoscope will try to get to the bottom of what makes files or directories different. It will recursively unpack archives of many kinds and transform various binary formats into more human readable form to compare them. It can compare two tarballs, ISO images, Debian packages or PDF just as easily. Diffoscope helps to identify the reproduciblity of Debian packages. During this summer I will be improving Diffoscope. I will be mainly working on:

  • Finish Parallel processing part.
  • Add command line option - -ignore-profile to allow users to ignore arbitrary differences.
  • perform fuzzy-matching across archives.
  • Add new file comparators to extend support of diffoscope for new file types.

My next blog post will be regarding community bonding. Thanks for reading :)

Categories: Elsewhere

Ingo Juergensmann: Avoiding Gated Communities with Diaspora, Friendica and others

Planet Debian - Sun, 08/05/2016 - 14:13

At the Chaos Communication Congress 32c3 in Hamburg last year, there was a talk by Katharina Nocun named "A New Kid on the Block - Conditions for a Successful Market Entry of Decentralized Social Networks". The short abstract is this: 

The leading social networks are the powerful new gatekeepers of the digital age. Proprietary de facto standards of the dominant companies have lead to the emergence of virtual “information silos” that can barely communicate with one another. Has Diaspora really lost the war? Or is there still a chance to succeed?

Maybe some of you attended that talk or have already seen the recording. For those who haven't, here it is for your convenience: 

It's all about Social Networks and Gated Communities vs. open communities. It's like Facebook on the Gated Community side and Diaspora as an example on the other, the open side.

At timecode 17:20 Katharina mentions that the Top10 of Diaspora pods have more than half a million users. But when you look more closely at the statistics from the-federation.info you can spot a different result that is most likely true for marketing statistic of Facebook as well: there is a difference between total users and current active users. Whereas indeed the total users are easily surpassing the half million users mark, it's a total different issue for the active users count of the last month: 15488 active users in total versus 546783 total users of the Top10 Diaspora sites. That's only 2.83% of active users. A quite awful turnaround rate. 

Many users are just quick lurkers, that came passing by, looking at Diaspora (and other alternative networks), get a quick login and a first try-out and never come back after a few days. I can confirm this from my own Friendica node at Nerdica.net where I currently have a total of 13 users: 7 users never posted any content, 1 user is already automatically set to expired because of this, and 8 users never came back after first day of registration. 

Therefor I cannot confirm with Katharinas conclusion that Diaspora "is not dead, it's pretty alive". All these alternative Social Networks are pretty much dead or - to put it in more friendly words - are alive in a rather small niche or small communities like data/privacy aware peoples.

Am I happy about this?

No, definitely not, because I am one of these data/privacy aware activits. I'm no big fan of such monolithic and centralized networks like Facebook. I'm a enthusiastic advocate of self-hosting and decentralized platforms and communication protocols, such as XMPP.

So, what can be done about these kind of Gated Communities like Facebook? Are you still on Facebook, because most of your family and friends are over there and not on Diaspora/Friendica? Are you still using Skype instead of XMPP? Why are you doing this? I'm really interested in this, because I don't understand it.

PS: please watch the video in full length! Katharina has some other good points as well! :)

Kategorie: DebianTags: DebianSoftwareFriendicaDiaspora 
Categories: Elsewhere

Guido Günther: Debian Fun in April 2016

Planet Debian - Sun, 08/05/2016 - 12:27
Debian LTS

April marked the twelfth month I contributed to Debian LTS under the Freexian umbrella. I only spent 2 hours (instead of expected 11,25) working on LTS things:

  • Uploaded gtk+3.0 to wheezy-proposed-updates to fix CVE-2013-7447 (#818090).
  • Further work on figuring out how to support Xen and QEMU in Wheezy LTS including writing up a summary and a interview with one prospective company
  • Prepared a patch for Jessie's nss to fix CVE-2016-1950, CVE-2016-1979, CVE-2016-1978, CVE-2016-1938 based on Antiones work for Wheezy. This is currently pending review by the security team.
  • Started to look into CVE-2014-7210 in pdns

The missing hours will be caught up during May - hopefully also by working on a QEMU/libvirt update in Wheezy should there be any interest - so I've you're relying on QEMU/KVM in wheezy now would be a good time to comment on it.

Other Debian things
  • Attended the 9th Debian Groupware Meeting in the LinuxHotel in Essen resulting in a new upload of calypso to unstable.
  • Uploded libvirt and python-libvirt 1.3.3, virt-sandbox 0.5.1+git20151113-3 and virt-manager 1.3.2-3 to unstable and libvirt 1.3.4~rc1 to experimental
Categories: Elsewhere

Petter Reinholdtsen: What is the best multimedia player in Debian?

Planet Debian - Sun, 08/05/2016 - 09:40

Where I set out to figure out which multimedia player in Debian claim support for most file formats.

A few years ago, I had a look at the media support for Browser plugins in Debian, to get an idea which plugins to include in Debian Edu. I created a script to extract the set of supported MIME types for each plugin, and used this to find out which multimedia browser plugin supported most file formats / media types. The result can still be seen on the Debian wiki, even though it have not been updated for a while. But browser plugins are less relevant these days, so I thought it was time to look at standalone players.

A few days ago I was tired of VLC not being listed as a viable player when I wanted to play videos from the Norwegian National Broadcasting Company, and decided to investigate why. The cause is a missing MIME type in the VLC desktop file. In the process I wrote a script to compare the set of MIME types announced in the desktop file and the browser plugin, only to discover that there is quite a large difference between the two for VLC. This discovery made me dig up the script I used to compare browser plugins, and adjust it to compare desktop files instead, to try to figure out which multimedia player in Debian support most file formats.

The result can be seen on the Debian Wiki, as a table listing all MIME types supported by one of the packages included in the table, with the package supporting most MIME types being listed first in the table.

The best multimedia player in Debian? It is totem, followed by parole, kplayer, mpv, vlc, smplayer mplayer-gui gnome-mpv and kmplayer. Time for the other players to update their announced MIME support?

Categories: Elsewhere

Jay L.ee: Drupal BackgroundField Module Configuration Manual

Planet Drupal - Sun, 08/05/2016 - 05:08

Yesterday I wrote a blog post on how to configure a Drupal module called Background Images Formatter, which is part 2 of the Drupal Background Images Module Configuration Manual. Today I'll continue with part 3 with a module called BackgroundField. Then tomorrow I'll finally reveal how to create clickable background takeover ads.

Tags: Drupal 7Drupal Planet
Categories: Elsewhere

Jeff Geerling's Blog: How to build your own Raspberry Pi Cluster

Planet Drupal - Sun, 08/05/2016 - 04:53

The banana is for scale.

When I originally built the Raspberry Pi Dramble 6-node Pi cluster in 2014 (for testing Ansible with bare metal hardware on the cheap), I compiled all the code, notes, etc. into a GitHub repository. In 2015, I decided to take it a step further, and I started hosting www.pidramble.com on the cluster, in my basement office!

Categories: Elsewhere

ARREA-Systems: Our business solution runs on Drupal 8.1

Planet Drupal - Sun, 08/05/2016 - 04:27
Our business solution runs on Drupal 8.1 Sun, 05/08/2016 - 10:27



Our back-office management solution is now running on version Drupal 8.1. The live demo is updated with the latest version.

It has been a long run since the project was initiated while Drupal 8 was still under alpha stage. And there is still plenty of work to do.

One objective is to make a full distribution package including most of the current functionalities available in the demo version. Our main issue with this target is the lack of resources and time. Thus if any of Drupalists are enthusiastic about business process solutions and would like to contribute, they are welcome.

Categories: Elsewhere

Russ Allbery: remctl 3.11

Planet Debian - Sun, 08/05/2016 - 01:53

The primary change in this release is porting the remctl extension to PHP 7. PHP 7.0 included a major change to the PHP API for binary extensions, which required quite a lot of porting (contributed by Nish Aravamudan). Due to the depth of the changes, the extension has been forked and the PHP 5 version of the extension should be considered frozen. Currently, they both provide the same functionality, but expect new functionality to be released only for PHP 7 or later.

This release also includes numerous portability fixes for older versions of Heimdal and numerous fixes to the RPM spec, both thanks to Jeffrey Hutzelman.

You can get the latest release from the remctl distribution page.

Categories: Elsewhere

Russ Allbery: rra-c-util 6.0

Planet Debian - Sun, 08/05/2016 - 01:44

Quite some time ago, I added strlcat and strlcpy functions to my portability C library. I've subsequently become convinced that those functions are a bad idea, and have been moving all of my code to asprintf and other functions. This release completes that change for all the code provided by rra-c-util and removes strlcpy and strlcat from rra-c-util.

network_set_freebind, network_set_reuseaddr, and network_set_v6only, to set various socket options, are now public functions, since INN wanted to use them directly rather than only as part of other interfaces.

This release also has new Autoconf probes for Perl that assist with linking with embedded Perl, checking the Perl version, and checking for Perl modules, and a fix to the OpenSSL Autoconf macros for 1.1.0. It also adds a replacement for gss_oid_equal for older versions of Heimdal that lack it.

Finally, rra-c-util 6.0 implements the transition in C TAP Harness 4.0 from SOURCE and BUILD to C_TAP_SOURCE and C_TAP_BUILD for all the test suite helper code provided by this package.

You can get the latest version from the rra-c-util distribution page.

Categories: Elsewhere

Russ Allbery: C TAP Harness 4.0

Planet Debian - Sun, 08/05/2016 - 01:07

When I originally wrote my test framework for C, I used SOURCE and BUILD as the preprocessor symbols and environment variables that pointed to the source and build directories of the software being tested. Subsequent discussion and thought convinced me that I should have used some sort of prefix on those variables to distinguish from other uses.

This release starts the process of changing to C_TAP_SOURCE and C_TAP_BUILD instead. You now have to use the new names when setting preprocessor directives when building the test harness. For now, the test harness will set all four environment variables, but test code should switch to the new environment variables, since I'll drop the old SOURCE and BUILD variables in a later release.

I also fixed a missing va_end() call in is_double(), thanks to a report from Julien ÉLIE.

You can get the latest release from the C TAP Harness distribution page.

Categories: Elsewhere

DrupalEasy: Summertime, and the hiring is (Drupal)Easy

Planet Drupal - Sun, 08/05/2016 - 00:18

It’s almost summer, and at DrupalEasy, that means it is almost Intern Season! Our Spring Drupal Career Online class is three-fourths of the way to graduation, and we have just three budding Drupalists who are looking for work experience through internships (the others are already spoken for!)  If you’ve got too much to do, and not enough capacity to do it, an intern might be just the ticket through our (Work Experience) WE Drupal Program.

We love sowing the Drupal Community with well-trained new talent, all of whom have already devoted hundreds of hours, thousands of dollars, and more than three months of their lives to learning, practicing, engaging and developing their passions for Drupal in their quest to become professionals.  We’ve found that they have a lot to offer organizations who can use their eager new Drupal passion and help them build really great first Drupal Experience entries on their resumes.  If you need some extra bandwidth, or have some tasks or projects suited to a new site-builder type, why not engage an intern?

Hosting an intern is also a great way to test out talent and take some of the lower-level workload off of senior developers (like taking care of your own site, or those simpler tasks you need to get done for your clients.)  Here’s the deal: you bring on a graduate of our Drupal Career training program, either paid or unpaid in mid-June.  They devote their new Drupal enthusiasm and best-practice foundational skills to your projects for 2 to 3 months while you give them some guidance and experience.  You and the intern then decide if they move on, or continue on as an (already indoctrinated) contractor or employee.

If you’d like to learn more, you can check out how we approach WE Drupal, fill out a Host Application (no commitment, just a way for us to learn what you are looking for)  or email me.  

Summer is just around the corner, so WE hope you don’t delay.

Categories: Elsewhere

DrupalCon News: We are sprinting - come join!

Planet Drupal - Sat, 07/05/2016 - 16:17

Extended sprints have officially kicked off at Launch Pad.  We will be here all day, so stop by and join.

The Extended Sprints are located at 643 Magazine Street.  The front door is set back a little bit.  When you arrive, please send @joelpittet a text so that he can come let you in the front door.

We have breakfast, lots of sunlight and a rooftop deck, so come join!

Thank you for sprinting.


Categories: Elsewhere

Elena 'valhalla' Grandi: Pyra preorders

Planet Debian - Sat, 07/05/2016 - 14:30
Pyra preorders

If you've met me at a conference you may have noticed that instead of a laptop I was using a handeld which looks like a laptop scaled down to nintendo DS size, the https://en.wikipedia.org/wiki/Openpandora.

I've used it as my main computing device while travelling for a few years, even for work (as a programmer)so happily that when EvilDragon announced at FOSDEM (link points to youtube video) https://www.youtube.com/watch?v=4T-w1KqrVsM that he was working on a successor device I started saving money for it even before I knew many details about the specs, other that they would have been way better than the Pandora ones (which is getting painful to use a browser on, because of its 256MB RAM).


Now this successor device is almost ready, they have opened the preorders https://www.dragonbox.de/en/45-pyra, and they have already reached the absolute minimum number of orders for mass production and are almost there for a more reasonable number of 1000 devices, so if you want a chance to get one of the first batch devices now it's time to visit their store.

A few highlights, from my point of view, include:

* It will run Debian with just a custom kernel/bootloader (and a few configuration only packages): most of the kernel mods are being submitted upstream, so maybe one day there won't even be a need for this kernel (but e.g. with Pandora upstream didn't accept the custom way they managed the keyboard; on the Pyra the keyboard is managed in a more standard way, but there may be other similar issues).

* It has been designed with modularity in mind: the CPU board is socketed on the main board and in the future upgrades may require just replacing the CPU board. I haven't read the details on the actual licensing, but it seems that the hardware design will be open enough that 3rd party boards may also be a possibility.

* just like on Pandora: real keyboard. hardware analog volume wheel. Huge user-replaceable battery (I don't think that there are any independent reviews of the pyra battery yet, but the one on the Pandora is still able to go through a day of FOSDEM — i.e. alternating often between on with wifi and suspendend — and only go down to 50% or so charge). Stylus (and 3d-printed quill) friendly touchscreen. Long term support from the producer.

* The 4G version has been designed in such a way that the GSM modem can be actually turned off (just like on the https://neo900.org/)

There are of course a few bad parts:

* PowerVR. The good news is that there is a risk that no 3d drivers will be available at all, and this means that the Pyra has been tested and considered good enough with just (FOSS) software acceleration.

* The price: yes, it is expensive. I'm happy I've saved money in advance for it, otherwise I wouldn't have been able to afford it. Some of it is a problem of small production, some is actual product quality. If you consider that it can take the place of both a laptop (and small ones are getting quite expensive, now that netbooks have disappeared) and a smartphone (I don't do lots of voice calls) it will start going down from "oh so **** high" to "high, but not unreasonably so"

Disclaimer: I have preordered one, so I am interested in the success of the project because it will mean better software and better support for the device.

Edit: forgot the link to the press kit the images comes from http://cloud.openpandora.org/index.php/s/a0Q0TXV8gh5NLAC?path=%2F, which also includes more infos on specs etc.
Categories: Elsewhere

Craig Small: Displaying Linux Memory

Planet Debian - Sat, 07/05/2016 - 14:00

Memory management is hard, but RAM management may be even harder.

Most people know the vague overall concept of how memory usage is displayed within Linux. You have your total memory which is everything inside the box; then there is used and free which is what the system is or is not using respectively. Some people might know that not all used is used and some of it actually is free.  It can be very confusing to understand, even for a someone who maintains procps (the package that contains top and free, two programs that display memory usage).

So, how does the memory display work?

What free shows

The free program is part of the procps package. It’s central goal is to give a quick overview of how much memory is used where. A typical output (e.g. what I saw when I typed “free -h”) could look like this:

total used free shared buff/cache available Mem: 15G 3.7G 641M 222M 11G 11G Swap: 15G 194M 15G

I’ve used the -h option for human-readable output here for the sake of brevity and because I hate typing long lists of long numbers.

People who have good memories (or old computers) may notice there is a missing “-/+ buffers/cache” line. This was intentionally removed in mid-2014 because as the memory management of Linux got more and more complicated, these lines became less relevant. These used to help with the “not used used memory” problem mentioned in the introduction but progress caught up with it.

To explain what free is showing, you need to understand some of the underlying statistics that it works with. This isn’t a lesson on how Linux its memory (the honest short answer is, I don’t fully know) but just enough hopefully to understand what free is doing. Let’s start with the two simple columns first; total and free.

Total Memory

This is what memory you have available to Linux. It is almost, but not quite, the amount of memory you put into a physical host or the amount of memory you allocate for a virtual one. Some memory you just can’t have; either due to early reservations or devices shadowing the memory area. Unless you start mucking around with those settings or the virtual host, this number stays the same.

Free Memory

Memory that nobody at all is using. They haven’t reserved it, haven’t stashed it away for future use or even just, you know, actually using it.  People often obsess about this statistic but its probably the most useless one to use for anything directly. I have even considered removing this column, or replacing it with available (see later what that is) because of the confusion this statistic causes.

The reason for its uselessness is that Linux has memory management where it allocates memory it doesn’t use. This decrements the free counter but it is not truly “used”. If you application needs that memory, it can be given back.

A very important statistic to know for running a system is how much memory have I got left before I either run out or I start to serious swap stuff to swap drives. Despite its name, this statistic will not tell you that and will probably mislead you.

My advice is unless you really understand the Linux memory statistics, ignore this one.

Who’s Using What

Now we come to the components that are using (if that is the right word) the memory within a system.

Shared Memory

Shared memory is often thought of only in the context of processes (and makes working out how much memory a process uses tricky – but that’s another story) but the kernel has this as well. The shared column lists this, which is a direct report from the Shmem field in the meminfo file.


For things used a lot within the kernel, it is inefficient to keep going to get small bits of memory here and there all the time. The kernel has this concept of slabs where it creates small caches for objects or in-kernel data strucutures that slabinfo(5) states  “[such as] buffer heads, inodes and dentries”. So basically kernel stuff for the kernel to do kernelly things with.

Slab memory comes in two flavours. There is reclaimable and unreclaimable. This is important because unreclaimable cannot be “handed back” if your system starts to run out of memory. Funny enough, not all reclaimable is, well, reclaimable. A good estimate is you’ll only get 50% back, top and free ignore this inconvenient truth and assume it can be 100%. All of the reclaimable slab memory is considered part of the Cached statistic. Unreclaimable is memory that is part of Used.

Page Cache and Cached

Page caches are used to read and write to storage, such as a disk drive. These are the things that get written out when you use sync and make the second read of the same file much faster. An interesting quirk is that tmpfs  is part of the page cache. So the Cached column may increase if you have a few of these.

The Cached column may seem like it should only have Page Cache, but the Reclaimable part of the Slab is added to this value. For some older versions of some programs, they will have no or all Slab counted in Cached. Both of these versions are incorrect.

Cached makes up part of the buff/cache column with the standard options for free or has a column to itself for the wide option.


The second component to the buff/cache column (or separate with the wide option) is kernel buffers. These are the low-level I/O buffers inside the kernel. Generally they are small compared to the other components and can basically ignored or just considered part of the Cached, which is the default for free.


Unlike most of the previous statistics that are either directly pulled out of the meminfo file or have some simple addition, the Used column is calculated and completely dependent on the other values. As such it is not telling the whole story here but it is reasonably OK estimate of used memory.

Used component is what you have left of your Total memory once you have removed:

  • Free memory – because free is not used!
  • The Cached value – recall this is made up of the Page Cache plus the Reclaimable part of Slab
  • The buffers

Notice that the unreclaimable part of slab is not in this calculation, which means it is part of the used memory. Also note this seems a bit of a hack because as the memory management gets more complicated, the estimates used become less and less real.


In early 2014, the kernel developers took pity on us toolset developers and gave us a much cleaner, simpler way to work out some of these values (or at least I’d like to think that’s why they did it). The available statistic is the right way to work out how much memory you have left. The commit message explains the gory details about it, but the great thing is that if they change their mind or add some new memory feature the available value should be changed as well. We don’t have to worry about should all of slab be in Cached and are they part of Used or not, we have just a number directly out of meminfo.

What does this mean for free?

Poor old free is now at least 24 years old and it is based upon BSD and SunOS predecessors that go back way before then. People expect that their system tools don’t change by default and show the same thing over and over. On the other side, Linux memory management has changed dramatically over those years. Maybe we’re all just sheep (see I had to mention sheep or RAMs somewhere in this) and like things to remain the same always.

Probably if free was written now; it would only need the total, available and used columns with used merely being total minus available. Possibly with some other columns for the wide option.

The code itself (found in libprocps) is not very hard to maintain so its not like this change will same some time but for me I’m unsure if free is giving the right and useful result for people that use it.


Categories: Elsewhere


Subscribe to jfhovinne aggregator