Planet Debian

At the recent Ubuntu Developer Summit, I managed to convince a few people (after assurances that there would be no permanent damage) to plug a USB stick into their machines so we could watch Xorg crash and wedge their console. What was this evil thing, you ask? It was an AVR microprocessor connected to USB, acting as a USB HID Keyboard, with the product name set to “%n”.

Recently a Chrome OS developer discovered that renaming his Bluetooth Keyboard to “%n” would crash Xorg. The flaw was in the logging stack, triggering glibc to abort the process due to format string protections. At first glance, it looks like this isn’t a big deal since one would have to have already done a Bluetooth pairing with the keyboard, but it would be a problem for any input device, not just Bluetooth. I wanted to see this in action for a “normal” (USB) keyboard.

I borrowed a “Maximus” USB AVR from a friend, and then ultimately bought a Minimus. It will let you put anything you want on the USB bus.

I added a rule for it to udev:

SUBSYSTEM=="usb", ACTION=="add", ATTR{idVendor}=="03eb", ATTR{idProduct}=="*", GROUP="plugdev"

installed the AVR tools:

sudo apt-get install dfu-programmer gcc-avr avr-libc

and pulled down the excellent LUFA USB tree:

git clone git://github.com/abcminiuser/lufa-lib.git

After applying a patch to the LUFA USB keyboard demo, I had my handy USB-AVR-as-Keyboard stick ready to crash Xorg:

- .VendorID = 0x03EB, - .ProductID = 0x2042, + .VendorID = 0x045e, + .ProductID = 0x000b, ... - .UnicodeString = L"LUFA Keyboard Demo" + .UnicodeString = L"Keyboard (%n%n%n%n)"

In fact, it was so successfully that after I got the code right and programmed it, Xorg immediately crashed on my development machine. :)

make dfu

After a reboot, I switched it back to programming mode by pressing and holding the “H” button, press/releasing the “R” button, and releasing “H”.

The fix to Xorg is winding its way through upstream, and should land in your distros soon. In the meantime, you can disable your external USB ports, as Marc Deslauriers demonstrated for me:

echo "0" > /sys/bus/usb/devices/usb1/authorized echo "0" > /sys/bus/usb/devices/usb1/authorized_default

Be careful of shared internal/external ports, and having two buses on one port, etc.

© 2012, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

A number of years back, I read yet another complaint about someone having trouble finding a computer with Linux preinstalled.

So I did something about it: I created LinuxSi.com, where it is possible to register computer stores in Italy (this was an Italian Linux mailing list) that are helpful towards people wishing to buy a Linux machine.

Fast forward past getting married, having kids and buying a house, and LinuxSi.com is not something I have much time to run any more.  I still think it's a useful service, even if the site itself is a bit creaky.

In any event, I've put it up for auction with Flippa.com, and there's one week left on the auction.  Right now, it's going for just $10, which even with the low amounts of adsense income it brings in, you'd make back pretty quickly.

I hope that it goes to someone who cares about promoting Linux in Italy - if nothing else, the domain name is a good one that could be employed for many things.

<iframe frameborder="0" height="315" src="http://www.youtube.com/embed/IFe9wiDfb0E" width="560"></iframe>

A new release 0.2.4 of RProtoBuf is now on CRAN. RProtoBuf provides GNU R bindings for the Google Protobuf data encoding library used and released by Google.

This release once again contains a number of patches kindly contributed by Murray Stokely, as well as an added header file needed to build with the g++ 4.7 version which has become the build standard on CRAN.

The NEWS file entry follows below:

2012-05-15 Dirk Eddelbuettel <edd> * DESCRIPTION: Release 0.2.4 * R/zzz.R (.onLoad): pass lib.loc=libname to readProtoFiles * R/internals.R (readProtoFiles): use lib.loc argument * man/readProtoFiles.Rd: Document new lib.loc argument 2012-05-14 Dirk Eddelbuettel <edd> * R/lookup.R: comment-out unlockBinding() which R CMD check does not like as it turns out that we do not need it here anyway * inst/unitTests/runit.addressbook.R (test.ascii): Small correction by converting to character before comparing * DESCRIPTION: Remove Depends: on int64 which is no longer used, lower Depends: on Rcpp to 0.9.6 which should be sufficient 2012-05-13 Dirk Eddelbuettel <edd> * src/rprotobuf.h: Added '#include <unistd.h>' to make g++-4.7 happy * src/*: Reversed out SVN revisions 426 to 434 related to int64 2012-04-07 Dirk Eddelbuettel <edd> * R/zzz.R: Add 'lib.loc=libname' to packageDescription() call as requested by Brian Ripley 2012-03-20 Dirk Eddelbuettel <edd> * Applied another patch by Murray: - src/mutators.cpp: Deal with case of repeated strings - inst/unitTests/runit.golden.message.R: test for this case 2011-07-17 Dirk Eddelbuettel <edd> * Applied another patch by Murray: - src/wrapper_Descriptor.cpp: Add missing readASCIIFromConnection C++ function that was called in read.R but not defined. - R/read.R: Remove an _ in the readASCII C++ functions to match the naming convention of the other nearby functions. - man/readASCII.Rd: Add examples of ascii export/import of messages. - inst/unitTests/runit.addressbook.R: Add tests of ascii export/import of messages. 2011-07-07 Dirk Eddelbuettel <edd> * src/wrapper_FileDescriptor.cpp (rprotobuf): Applied patch by Murray to correct a cut & paste error in variable name CRANberries also provides a diff to the previous release 0.2.3. More information is at the RProtoBuf page which has a draft package vignette, a 'quick' overview vignette and a unit test summary vignette. Questions, comments etc should go to the rprotobuf mailing list off the RProtoBuf page at R-Forge.

Way before Debian, Launchpad bug tracking system just reached 1 million bugs reported with one "bug" reported against Edubuntu basically mentioning it should invade schools.

What to say about this?

Hard without being harsh towards my friends working in the Ubuntu "world", indeed. Still, I really think that here, too much noise kills signal and the LP BTS is often hardly usable.

I counted up to 217 bugs reported against samba4 (which is, after all, not so widely used yet) just because it apparently has upgrading issues between pre 12.04 versions of Ubuntu and Oneiric. It indeed seems that some automated bug reporting is now active and whenever a user encouters an upgrade issue with a package, a bug is being reported. I guess this is somehow an opt-in system (I hope so..:-)) but the default is very clearly using it.

This feature is apparently what caused the recent bump in number of bugs reported in LP, making them even less useful, particularly to Debian package maintainers. I'm sure there are tools to help dealing with that and I was already answered that work is in progress to change this (and use a dedicated website for such reports or something like this). But, still, that seems to be the scary side of popularity...the very same popularity that is slowly but constantly hiding the work we're doing in Debian to indirectly make Ubuntu popular.

(moving to more general concerns)

I know that things are not all black or all white, but it always saddens me to feel that slowly....but, again, constantly, more and more people tend to forget that Debian is behind Ubuntu, is the ground on which it is built and Ubuntu wouldn't exist without it. When doing work, a human need is to get reward for it...and we are getting less of it...slowly, but constantly.

Don't take me wrong. I have many friends working directly for Ubuntu. Some paid by Canonical for this. Some really involved up to "top level" (yes, including the very very top level even if I killed him once). I don't want to throw offense on them. I don't even know if they can do something about what I'm expressing below. I would just have them (and others) know.

Let's take an example. I recently activated a few languages in D-I (Burmese, Tibetan, Uyghur). I'm happy with that, this is something I'm doing for 8 years now. But all these new translators were indeed only interested in one thing : "have Ubuntu translated in their language". No offense intended, but they didn't really care about *Debian* being translated in their language. I think that some didn't even know what Debian is.

In the same field, I am more and more "fighting" to keep the level of translation completeness in Debian (see my regular spa^W reports). In some way, I still succeed, but the price to pay is more and more and more personal investment and work. That's still working for the strong set of languages we support. That works much less for most others. When someone "disappears" (or just switches to some other priorities), it's more and more difficult to find someone else popping up.

And, for the "strong set", something else is happening : work duplication. There are "strong" French, German, Italian, whatever, l10n teams in Debian.....and there are similar teams for Ubuntu. And, mostly, those do not really work together.

And sometimes, this is kinda discouraging. So, seeing the explosion happening on what is, whatever we think or write, the "other side", is not somethnig that can make one entirely happy. And this is why I won't celebrate Launchpad's millionth bug report.

Particularly when I see that millionth bug report not even ack'ing that this Edubuntu marvel is based on the grounds set by some pionneers many years ago in a few schools in Norway (hello, Petter and others).

Yeah, sometimes sad. To balance this, let's release wheezy and have millions of people benefit from it without even knowing.

Hey, thanks, Peter for the hint!

Never heard of that option until you blogged about it. So, now I'm also ready to "tar taf", "tar xaf", "tar caf"..:-). Harder for me than you because I was used to "tar tfz" or "tar tfj"..:-)

And I suspect that "tar taf" is prone to typos...we'll see.

I’ve been noticing a number of odd things happening surrounding my blog lately, and I thought it’s about time to figure out what’s going on and how to stop it.

The first problem is that people are illegally copying my posts, probably using RSS scraping, and putting them up on their own ad-infested sites. It is trivial to find them using Google for any somewhat unique word or phrase in one of my posts. Lately one of them, linux-support.com, actually sends me pingbacks announcing the fact that they’ve scraped me! Most of these sites seem to be nothing but content farms for selling ad impressions, and almost none of them have any identifiable names for the owners.

(There is an exception: I have specifically set up sites like Planet Debian and Goodreads to copy my blog posts.)

I’m obviously an advocate of open content, but I do not feel it right that others should be profiting by putting photos and stories about Free Software, or photos of my family, on their ad farms. While I release a great deal of content under GPL or Creative Commons licenses, I have never done so with my blog – an intentional decision.

What should I do about this? Is it worth fighting a battle over, or is it about as useless as trying to block every spam follower on my twitter account?

So that’s the first weird thing. The second weird thing just started within the last few weeks. I have been getting a surprising amount (a few a week) of email addressed to me. It does not bear the appearance of being 100% automated spam, though it is possible that it is. It’s taken a few forms:

• Someone wanting to buy an ad on my blog
• Someone wanting to send me a story hyping their product (and intending me to pretend that I wrote the story)
• Someone wanting me to write a story about their website and link to it

The profit motive in all of these is high, and in at least the second and third, so is the sleaze factor.

I’ve gotten two emails lately of this form:

Hi John,

I am curious if you are the administrator for this site: changelog.complete.org/archives/174-house-outlaws-fast-forwarding-senate-pres-next

I am a researcher / writer involved with a new project whose mission it is to provide accurate and useful information for those interested in the practice of law, whether as a lawyer or paralegal. I recently produced an article detailing the complex relationship between law and technology and the legal implications on personal privacy and free speech. I would love to share this resource with those who might find it useful and am curious of you are the correct person to contact about such a request?

Thank you!

All my best,

The details vary – the URLs appear to be random (the one cited above was little more than a link to an article), the topics the website claims to discuss range from law to schizophrenia (that one actually came with a link to the site, which again seemed to be a content farm). I am slightly tempted to reply to one of these and ask where the heck people are getting my name. It seems as if somebody has put me into a mailing list they sell containing sleazebag bloggers.

Frankly, I am puzzled at this attention. I guess I haven’t checked, but I can’t imagine that my blog has anything even remotely resembling a high PageRank or anything else. It’s not high-traffic, not Slashdot, etc. Either people are desperate, naive, failing to be selective, or maybe working some scam on me that I don’t know yet.

In any case, I’m interested if others have seen this, or any advice you might have.

It's not too obvious how to test email receive handlers in Appengine.  The important observation is that the handlers take HTTP POSTs with multipart/form-data encoded data.  In python you can build an email to be handled with the following code:
from email.message import Message

def test_email(self):
body = Message()
body.add_header('to', 'test-unknown@other-app.com')
body.add_header('from', 'test@app.com')
body.add_header('Content-Type', 'multipart/alternative', boundary=self.boundary)
text = Message()
text['content-type'] = 'text/plain'
text.set_payload('I am I! Don Quixote!  The man of La Mancha!')
body.attach(text)

post(payload=body.as_string())

This year, the annual Debian conference will be held in Managua, Nicaragua. And I'll be lucky enough to spend two weeks visiting the country after Debconf, along with Elisabeth.

Yes, I'll arrive in Nicarague on July 2nd, spend nearly the entire Debcamp, then Debconf, then we'll spend 16 days around the western part of Nicaragua, trying to discover the magic of this country.

So, this post is about sharing our plans with my readers. Of course, I do not know the country so we may have made mistakes and bad choices. We'll see.

Immediately after Debconf, Elisabeth will join in Managua. She'll be landing on July 15th. We'll then spend a night in an hotel near the airport and immediately leave the day after for Matagalpa, in North Nicaragua. We rented a car for the entire trip indeed, and will be on our own on wild Nicaragua roads..:-)

We'll spend two nights in Matagalpa. We plan to visit some coffee or cigar plants, probably have a trip to Lake Apanas and Jinotega.

Then, we'll have a short road trip to Esteli where we spend again two nights. We'll be visiting a coffee growing place (beneficio seco de café). A full day visit is planned at Miraflor natural reserve to enjoy te beauties of hundreds orchids and some local natural marvels.

The next move will be to Leon, where we'll spend 4 nights, visiting a cigar factory (tabacaleras de puros?) on the way, as well as San Jacinto, a place with hydrothermal sources and "Hervideros" (geysers).

Four nights in Leon leaves plenty of time for several activities *and* enjoying the colonial city. We'll have a full day at Juan Venado Island reserve with boat trip from Las Penitas (on a fisherman's boat from what the travel agency mentioned), then another full day climbing on the Cerro Negro volcano. Indeed, I was originally considering climbing the Momotombo, but our travel agency warned about the high difficulty. I would have loved that myself but maybe not the two of us...and this is a trip for both of us! So, we played the safe option..:-)

After these 4 nights in Leon, we'll move to Granada for 3 nights, through Leon Viejo (the former site of Leon).

From Granada, one day will there be used for a visit in the Masaya National Park and see the beauties of Masaya volcano (this is indeed something that could be done for Debconf day trip, IMHO, as it doesn't seem that far from Managua). Another day will be spent to Las Isletas on lake Nicaragua and others visiting the colonial city of Granada. Or, of course, whatever things we don't even known about now..:-)

Then we'll move to what I personnally consider the peak of the trip: 3 nights on Ometepe island on lake Nicaragua. Just check Wikipedia to see why Ometepe is, in my opinion, THE place to go in Nicaragua. Here, I'll have my volcano..:-). Indeed, Elizabeth "authorized" me to book a local guide and then climb Concepcion Volcan, if the weather allows for it. 1600m height, that doesn' seem to be a big issue....except when starting from a little bit above sea level and are climbing a volcano that looks like s postcard volcano : nearly a perfect cone shape.

So, let's cross fingers for having good weather that day. I promise myself I'll record the GPS track of that one and, even if I'll probably be walking most of the climbing (except if I have a very trained guide...), I'll add it to my run tracks!

We might also be going to climb Ometepe's other volcano (Maderas) the day after so that Elizabeth also enjoys these beauties. There also seems to be great places around Maderas such as San Ramon Cascade, Finca el Porvenir, etc.

Then, at the end of all this, it will be time to come back to Managua in the final day and fly back to Paris in the early morning of July 31st.

All over, I'll be in Nicaragua from July 2nd until July 31st! Full month away, yay! Hurrah for the crazy number of holidays those lazy French people have..:-)

During this trip, we might find it interesting some local geeks (not too many as Elizabeth is not that deeply interested in beersigning!) and share a few nice things in local places which are only known by locals.

In case you're interested, out (very clever) travel agency is named Nicaragua Adventures and they're definitely worth contacting if you want to travel around .ni, particularly if we prefer booking things in advance as we do. They speak Spanish (of course!), English and French. They're very responsive to e-mail as well.

CRAN now tests packages against g++-4.7 (as this version has become the default on Debian's testing variant. This compiler switch once again triggered a set of build failures, mostly from include files now deemed missing. For RcppSMC, it came down to a five-character patch of explicitly stating one max() call as std::max()

No other changes were made at this point. The NEWS entry is below:

0.1.1 2012-05-14 o Version 0.1.1 o Minor g++-4.7 build fix of using std::max() explicitly

Courtesy of CRANberries, there is also a diffstat report for 0.1.1 relative to 0.1.0 As always, more detailed information is on the RcppSMC page,

The first Beta of the upcoming PostgreSQL 9.2 was released yesterday (see announcement). Your humble maintainer has now created packages for you to test. Please give them a whirl, and report any problems/regressions that you may see to the PostgreSQL developers, so that we can have a rock solid 9.2 release.

Remember, with the postgresql-common infrastructure you can use pg_upgradecluster to create a 9.2 cluster from your existing 8.4/9.1 cluster and run them both in parallel without endangering your data.

For Debian the package is currently waiting in the NEW queue, I expect them to go into experimental in a day or two. For Ubuntu 12.04 LTS you can get packages from my usual PostgreSQL backports PPA. Note that you need at least postgresql-common version 0.130, which is available in Debian unstable and the PPA now.

I (or rather, the postgresql-common test suite) found one regression: Upgrades do not keep the current value of sequences, but reset them to their default value. I reported this upstream and will provide updated packages as soon as this is fixed.

You can get inspired from the most unlikely of places, and this time it was a twitter background image. I came across the twitter profile of @Nitish today, a fellow GSoC student from India. One thing that held my attention, apart from the fact that he tweets crazy, was the background picture in his profile. Yes I had not seen something like that before and I absolutely loved the concept.

The image was large grid and each tile carried the profile picture of one of his twitter followers. It was beautiful. The background image was generated by twilk, a very popular application that I had never heard of. I wanted to try it, but I figured my follower count might be a little too low for the application to work. Twilk works only with twitter and I wanted it for Facebook. Instead of looking for something similar, I decided to write it myself. I had worked with the Python Imaging Library some time back and knew it would not be very difficult task.

The first thing I needed was the profile pic thumbnails of my friends. Facebook API supports a call which returns a small 50x50 pixel profile pic of a user, if we send a valid user ID. I wrote a quick script that returned the user ID’s of my friends on Facebook as JSON data. I grepped the user ID’s from the JSON and wrote a small wget script to download the profile pics of all my friends to a directory.

Once the download was in progress, I started my work with the imaging library. I knew the image count beforehand and after some quick math, I fixed the dimensions of the output image that I wanted. Once the dimensions were fixed, the work was fairly easy. I knew before hand how many images will be there in each row and how many rows will be there in total. All I had to do was to randomly pick one of the images from the directory, and arrange it on the output image canvas. Few lines of code later I get the result that I wanted and it is on my twitter profile. Yay! :)

I have uploaded the python code in here. If you are running it, make sure you change the values according to your requirements. The image below is a cropped out from the output image to fit this div. Click on it to see the actual output.

When I showed this to @Jaseeemabid, he asked me to put this up on Google App Engine and start a service so that even non geeks could use this. I liked the idea but I am slightly busy(lazy) now. I’ll do it when I am free or when I see that there is some demand for this! :)

Update

As a Redditor pointed out, this may not be the best way to do it. This was just a quick hack to get something done. The JSON could be processed in Python itself and files can be downloaded as well. I modified an old PHP script to get the JSON data and since the structure was relatively simple, I grepped the relevant information and fed it to wget as I prefer wget over any other tool for mass downloads.

And if you like this post, you should follow me on twitter :)

For years, no decades, I've typed tar tzf something, tar xzf something. Except when someone annoying sent an uncompressed tar file and I had to then go and take out the z in the middle.

Then came bzip2, and we learned tar tjf, tar xjf. OK, I could live with that. One emerging problem was that the tab completion now worked the wrong way around conceptually, because you had to pick and type the right letter first in order to see the appropriate set of files to unpack offered for completion.

Then came lzma, which was (quick, guess?), tar tJf, tar xJf. And then there was lzop, which was too boring to get its own letter, so you had to type out tar -x --lzop -f.

But lzma was short-lived, because then came xz, which was also J, because lzma was now too boring as well to get its own letter.

Oh, and there is also the old compress, which is Z, and lzip, which I'd never heard of.

But stop that. Now there is

-a, --auto-compress
use archive suffix to determine the compression program

This handles all the above compression programs, and no compression. So from now on, I always use tar taf and tar xaf. Awesome.

The finger movements will be almost the same on QWERTY and AZERTY, and easier than before on QWERTZ.

Actually, this option is already four years old in GNU tar. Funny I'd never heard of it until recently.

I read a translated poem about Russia being "the Motherland" and its vast bosom years ago. Having driven through a significant part of it, I can agree on the "vast" part...

Also, as I am on a train and without access to the Internet, I will refrain from linking to a lot of pages; sorry. (Turns out I am posting this a week later, but I will still not link to stuff now; no time).

Russia in general

• All receipts you receive are torn before you get them; this is most likely due to the old Soviet voucher system, more on that below.
• Russia was hot with temperatures ranging from 27 to 32 degrees Celsius between Moscow and Ulan Ude.
• There aren't a lot of pedestrians bridges, but a lot of pedestrian tunnels. The sides of those tunnels are packed with tiny shops, often only two meters wide and 50-70 cm deep. Everything from stockings to candy over glasses to flowers and watches is being sold through a tiny window by some poor woman who somehow managed to get in there.
• Toilet brushes stand in water. In Germany, that's a sure sign of a really dirty toilet; in Russia, it's the thing to do. If you are lucky, there's blue cleaning stuff added to the water. If not, it will still have color. You are free to guess which.
• Queuing is war.

Moscow Sights Kreml

Our remaining time in Moscow was spent with touring the usual suspects; the Kreml is a lot less impressive in real life, the Red Square is tiny when compared to the stories I heard about it and the Chapel ofi St. Basil is even more colorful and impressive in real life. Lenin's body was inaccessible because workers built seats for the May 9th parade to the left and the right of it and they apparently thought it would be a good idea to block access to one of the main tourist attractions while doing so. A river tour of Moscow was a nice cool-off and we got to see quite a few things.

We managed to see the weekly military parade within the Kreml grounds, but it was mostly pomp and little substance. The National Treasure which you can access with an extra ticket within the Kreml grounds is nice, but less impressive than the tourist guides would make you believe. That being said... There's another museum within the museum and.... Whoah... Tourists pay extra, visitors go through the only non-security-theater check I encountered in Russia, guards are armed, people can only enter and leave in batches, and the stuff which is presented is mind-boggling. Disregarding the fist-to-calf-sized chunks of gold and platinum which are still in their original form directly from the mine, there is real, actual treasure galore. Little heaps of uncut and cut diamonds, an outline of Russia filled with cut diamonds and other random "we have this stuff" displays can be found as well. Then, you have various tiaras and other jewellery made from various gems. Not incorporating, but largely made of. All that pales in comparison to the crown, royal apple, scepter, etc. It's hard to put the amount of tiny multi-colored light points that shine at you into words. I was just standing there, swaying back and forth to catch the moving pattern of pinpoints. It's said that this collection is equalled only by the ones in the Tower of London and the one Shaw of Iran had and boy do I believe it.

TV Tower

Getting up there was funny.

The old-style Soviet queuing system was used:

• Go to a counter to tell an attendant what you want; receive stub
• Go to another counter, hand over stub to another attendant, pay for what you want; receive voucher
• Go to third counter, hand over voucher; receive ticket for tower The whole thing was made even more absurd by the fact that counter one was in the middle, counter two to the right and counter three to the left. As Russians do not believe in queuing and everybody just tries to get in first, this made for a nice little exercise.

"Security" for approaching the tower was multi-level, the guards see you approach along a long walkway way in advance and the main guard shed had several small cabins separated by thick glass. So good so menacing. But in a twist that would make Bizarro and Garry Larson proud, I was required, by means of metal detector gate, metal detector wand and even an x-ray machine to remove every shred of metal and other hard objects from myself and the camera bag and put them onto a table. Once I was without anything except my clothes and the bag was completely empty, I could pass. Everything I had had to remove was just laying there, not inspected in the least, for me to stuff back into pockets and bag and to take with me. This "everything" included a Spot Messenger 2 with lots of green and red blinky lights. The guard did not even glance and it. Security theater? Security theater.

The view from 364 meters down on Moscow was nice, but there was a lot of Smog so I couldn't see very far. Jumping on the glass floor while looking down was a lot of fun, though.

Subway to Thiefing

I bet Christopher Nolan rode the subway in Moscow at least once. That unnerving sound you hear during several key scenes in "The Dark Knight"? Two thirds of all subways make the same sound while moving.

Also, I had an encounter with a pickpocket down there; very classical, too. Guy approaches quickly, talks loudly and sounds as if it's really important (in Russian... duh... that's sure to keep me interested). His approach made me turn and protect my left leg pocket automatically, most likely marking the target for the tiny woman standing behind me. Now, I have to tell you something about my usual travel layout. As my normal pockets are very deep, it looks as if their content was in the leg pocket. Plus, there's an extra, hidden leg pocket where I keep the passports and train tickets. The outermost leg pocket is protected by a velcro flap, but it contains nothing of value; usually the appropriate phrasebook, local map, maybe a tissue or chewing gum. Due to this layering, the outermost pocket looks as if it's full to the brim with stuff. Also, I took pains to make it a habit to protect said leg pocket with my hand, nothing else. This looks as if that's the target, but what I am actually doing is protect my normal pocket with my forearm. The right side is different, but the most easily accessibly pocket always holds some small change. I pay from that stash but my actual wallet is well out of reach. Anyway, once the guy ran off, talking to several others, most likely marking all them for the actual pickpockets, I wanted to enter the subway. While the Russian-style queuing took place, I felt an unusual tug at the velcro flap. I looked down and saw a tiny woman to the left of me with a jacket held over her right side with the left arm; I look up to check no one is trying to steal from my permanently assigned female, feel another tug, look the woman into the eyes, look up again and around me, look down again and she is gone. All that took maybe three seconds and I had boarded the subway after an additional two.

In hindsight, it makes sense to choose the time of entry for attack. It's crowded, you are being pushed around, and once you are in the subway, it will start moving more or less immediately while the thief remains in the station.

In this case, she would only have gotten a grubby map of Moscow's subway and an English-Russian phrasebook, but she got nothing at all.

Moscow-Novosibirsk

Where to begin...

If you think a few hours on a train are a long time, try over fifty hours. Things get so bad, you start getting land-sick while not in a moving train. You even start missing the familiar tunk-cachunk, tunk-cachunk, tunk-cachunk... of driving over rails with gaps in them when you are not moving.

The defining element of the Trans-Siberian Railway are birch trees. And birch trees. And then more birch trees. You would not believe how many birch trees there are. This is made "worse" by the way the Russian Railway protects their rails. Left and right of the track, there's a cleared area of maybe ten to twenty meters, sometimes as little as three. Outside of that, they plant ten to twenty meters of birch trees, presumably to catch snow during winter. Beyond that protective perimeter, there's the normal landscape.As a result, on top of the near endless stretches of birch woods, you see most if not all scenery through a layer of birch trees. You get sick sick of birch trees after a few hours and you see them for days on end.

Bullet points to save myself some typing and you some reading...

• More than a thousand kilometers without a single hill. Flat as a pan.
• The whole route is powered by electricity. No diesel engines in sight.
• Many stations are little more than a heap of smoothed gravel, bordered by some wooden planks. Some stations have obviously been built by locals and are even less well-defined.
• You can see people in the middle of nowhere, walking along the railway tracks. At first this seemed counter-intuitive, but most if not all roads out there are dirt tracks. As there seems to be standing water across a third of Russia, this dirt is turned into mud. After walking maybe twenty meters across a parking lot, I had to scrape a heavy, thick cake of black earth from my soles. The railway is the only functioning footpath those people have. Many people even build shoddy bridges towards the tracks from their homes, obviously preferring to walk along the tracks over walking through the village.
• Railway crossings along the Trans-Siberian route, no matter how tiny, have a small cabin beside them. While the train passes, there's one guy or gal standing in said cabin, holding a yellow stick vertically out towards the train. Sometimes, you have not seen any living thing, other than birch trees, for twenty minutes and there, in the middle of nowhere right beside a dirt track, there's someone holding a stick out towards the train. Weird.
• Railway crossings of paved roads will always have two steel plates coming out of the ground, angled towards oncoming traffic on each side. This may not stop a heavy truck at full speed, but a car will disintegrate on these barriers without touching a passing train.
• The railways is important for Russia. Two parallel tracks cut across the whole country, transporting everything back and forth. Where "everything" means mostly coal and birch wood, I guess.
• All freight trains are usually 70 tanker waggons or 100 box waggons long, but you see the odd 100 tanker waggons, as well. You have more than enough time and opportunity to count them and then some.
• There are supposedly women at every station, selling what they cook at home. Unfortunately, this was only true for two stations. The things we did manage to get were very nice; I do wonder why anyone would offer (or buy) cooked and peeled potatoes, though.
• Every waggon has its own hot-water stove. They are powered by coal. Yep, you have a coal fire burning in every single waggon on the Trans-Siberian.

Novosibirsk The non-existent hostel

We arrived at ~0200 local and made our way to the hostel we had booked a room with. Walking to the correct address, we saw several signs but they all turned out to be for a police station and some other state agency. We walked back, forth, double-checked, triple-checked: no hostel. We then walked around the building through some not-quite-nice back alleys, but other than a few entries to private flats, there was nothing. Thankfully, the booking slip included a number which we called and after at least twenty rings (no kidding), when I had given up and wanted to hang up, it stopped ringing. Dead silence. After maybe ten seconds, someone started talking in Russian. I asked him if he spoke English and told him that we could not find the hostel. He mumbled something about being sorry and that we should wait, he would come down. Fast forward a minute or two and someone walked towards us.

Again, he mumbled about being sorry, that the hostel "did not work" at the moment and that we would need to sleep in his private apartment. He ushered us into some back alley entrance, into his flat, and proceeded to remove the sheets from the couch on which he had slept; after putting on new sheets, we had our "hostel" bed, ready to sleep on. We briefly considered if he would murder us in our sleep, but him and me even got to talking a bit. Over cheese, sausage and rum (at 0300), he admitted that the hostel did not exist and he merely planned to turn his flat into a hostel for the summer while he and his family moved into their summer house (the Russian term of which escapes me, at the moment) in the countryside. He had accepted our reservation as he thought he would be finished by that time. He did not even get started, though. While he sent us an overbooking notice through booking.com two days before, we were on the train at that time, so... booking.com even called him to check what happenend to us as we did not book another place through them. Good customer service/protection, that.

Next morning, he didn't even want to take our money (we paid anyway) and, as a means of compensation, drove us into the city in the morning and to a train museum well outside the city limits, one of the fabled scientist cities, and a large lake which everyone in Novosibirsk claims is an ocean, in the afternoon.

Foreigners, foreigners!

All in all, Novosibirsk was relatively uneventful, safe for one bizarre episode. We took our lunch in a local fast food joint (why do all the good stories happen there, and not at the various truly local places?) and threw the cashier our well-rehearsed "Niet Russkie; anglisky?" with phrasebook in hand and he actually understood a few words of English (beef, chicken, fries). We told him, in our worst Russian, that we are from Germany wished him a nice day and went to sit down. A few minutes later, a girl approached us, literally hopping from one foot to the other and wringing her hands. She told us that the cashier had told her that we spoke English and if it would be OK if she talked to us. We suspected some sort of elaborate ruse, but went with it. Turns out, she had English at school and really wanted someone to practice English on. Two young men passed our table and exchanged a few words with her, sitting down out of sight. When she told us that she had to leave now but if it would be OK if the two boys joined us we suspected a ruse yet again. But those two were law students, one with a minor in English and one with a minor in German; both of them also extremely nervous, asking us if we would talk to them. When they had to leave, they told us that the three of them worked at the burger joint and that their shift was just about to start when the news that foreigners were here spread amongst staff like wildfire. The girl stopped by several times in between cleaning tables, getting in a sentence or two before being cussed at by her supervisor. All in all, this took about twenty minutes and seeing three people so nervous and grateful to talk with us felt beyond absurd.

On the other hand, not a single traveller we met even considered stopping in Novosibirsk during their transit so there really does seem to be a shortage of non-Russians there.

Weird, and memorable.

Novosibirsk-Irkutsk

• Birch trees.
• Lots of burnt underwood, presumably to prevent larger fires.
• Birch trees.
• Sticky, stuffy, 30+ degree waggon with windows that could be opened but which were locked (this is why I always carry a Swisstool with me).
• Birch trees.

Irkutsk / Listvianka / Lake Baikal Listvianka

Aah, lake Baikal... the oldest and deepest lake on Earth which holds a fifth of the global non-salt water reserves; a must-see in my book.

Quad tours at break-neck speeds, dry-suit diving with Russian regulators, walking barefoot in between and across drift ice that made its way onto the shorei, and extended hiking around the lake's coast...

All of which I could not do because I was ill and had to spend two solid days in bed.

The draft from the open window in between Novosibirsk and Irkutsk was enough to give me a rather bad cold which peaked at Lake Baikal.

Still, the area was lovely and we were glad to be out of a train and able to unpack our stuff without having to repack immediately for once.

I am not sure where my current losing streak with regards to diving is coming from (Grimsey, diving north of the Arctic circle with birds that plummet into the water and hunt fish: Only guy who does this is on the Icelandic mainland that day; Svalbard, diving north of the Arctic circle in permanent darkness: The few people who do this privately did not reply while I was there; Baikal, oldest, deepest, largest lake on Earth: ill), but I will most likely return to Russia for a week of ice diving in Lake Baikal next winter or the one after that.

As an aside, I saw several people walking to Lake Baikal with buckets to get their water. Other people got it from a well which was still half frozen. If you have running water consider yourself lucky...

Irkutsk

Nice city, largely uneventful. The farther east you get within Russia, the more normal women look. In Moscow, just as in Paris, they are way over-dressed and even service personnel will walk with high heels. Thankfully, I don't have to wear heels, but for the other males out there: Walking and standing in these things hurts and thus most if not all people who stand and walk for a living have flat shoes.

We happened upon preparations for a military parade, complete with cordon, viewing podests, at least half a dozen TV cameras etc, but were not sure if it would start soon enough for us to catch our train.We asked someone who told us it would start at 2100 local, at 1945 local it seemed about to start, and sure enough at 1955 sharp, the whole thing went under way. About a dozen groups of 50-100 people each, all in their own, respective uniforms stood against one side of a cordoned-off street and several higher-ups on the other side. Two highest-ups shouted into microphones and the throng of people on the other side shouted back answers. Then, the two highest-ups stood in the back of a jeep each and drove past said throng, stopping in front of each group, shouting into microphones mounted in the back of the jeeps and the groups shouted back once again. After that, all groups marched around the make-shift plaza once, saluting the higher ups. Once they were done, and they took ages, two trucks drove by with soldiers jumping out of the moving trucks and moving into crouching positions. They ran around in a circle a few times and engaged in pretend hand-to-hand combat. I am sure they are skilled at whatever style they wanted to show, but they were overdoing things so badly, they were funny, not imposing. When they jumped over some barriers, the barriers fell to pieces and everyone scrambled to make it look as if that was part of the show. While carrying off the gear, it fell into further pieces which was even more funny. An armoured personnel carrier ended the show; several tougher looking guys jumped off of that one and their mock combat involved fully automatic fire (of blanks), several flashbangs, smoke grenades and, to top things off, the machine gun mounted on the APC moving down the opposing team with blanks.

I never witnessed a "real" military parade in person but this one was somewhat disappointing. On the one hand, there was a distinct lack of ballistic missile carriers and tanks like you see in movies, documentaries and games, on the other hand, the whole thing had a make-do feeling to it. The cordoning police had designated spots to stand on, yet walked around. They were standing to attention, yet checking their cell phones. Several people in one uniformed group were wearing track suits and jeans. Another uniformed guy had a grocery bag with him; yet another one was carrying a huge water bottle. Bikers zig-zagged through the cordon and when the whole show was just about to wrap up the police finally started putting up barriers around the unmoving pedestrians, not blocking the bikers. One little girl was standing well within the cordoned area, watching with big eyes and after she did not react to the police talking to her, they just built the barriers in a curve around her.

And to top it all off, some guy with a cane walked all through the parade with his personal camcorder, trying to direct the whole show while being ignored by everyone. Still, I am sure he managed to mess up some otherwise perfectly good TV scenes.

Irkutsk-Russian border

• Diesel-powered trains.
• Single track most of the time with frequent stops to let other trains pass.
• Distinctively less developed cities, stations, streets, and other infrastructure along the road.
• 32+ degrees in our waggon.
• The train attendant was extremely unfriendly and just generally miserable even by Russian standards.
  • No toilet paper or towels at all on toilets.
  • While the other attendants made a point of presenting themselves well, he shuffled around in slacks all the time (not bad per se, but Russia is big on uniforms, so...)
  • He took all our tickets and stubs (including the ones not from this part of the journey) and kept them without comment. After we asked for them several times, he barked at us that we would get them back before Ulan Bator. Why? No idea...
  • He refused to let us exit the train during the very few stops. We were unable to exit through other waggons as the connecting door was locked. Being stuck in a train sucks.
• Border and customs took NINE HOURS!!! Stuck in blistering heat without a breeze, without access to a toilet, just waiting for bureaucracy to go its way. I checked all doors, we were locked into said waggon and there were no 'break glass to leave in emergency' windows. Especially nice as there's a coal fire burning in the hot-water stove and the whole train is plastered with warning signs about fire and what to do. In our case, presumably, burn to death; preferably without disturbing the attendant.
• The Russian stamp for entering Russia (by plane) has a plane on it, the departure one a train.
• The Russian side of the border is built like a fortress. There are several towers and bridges over the rails so trains can be checked from above, and reinforced holes dug into the ground in which soldiers stand and check the train from below.

In this age of 3d accelerated desktops and all that fancy stuff, one does not expect practical innovation happening in the remote terminal emulation area. But it has just happened. It is called Mosh, a shorthand for "Mobile Shell".

What does it do better than ssh we have learned to love?

• Less lag! Being UDP based, it is not prone to TCP congestion effects. Considering that voip, games and everything else latency critical has been UDP based, it is (almost) surprising that it wasn't done for interactive terminals before...
• Even less lag! Mosh provides local echo and line editing when the other side is not being responsive. To do this, mosh actually becomes a terminal emulator of it's own. This stuff is sweet on unstable 3G and conference wifi networks.
• Survives suspending. Resume your laptop and *bam* all your remote mutt and vim editors are still there instead of the "connection reset" you get from ssh.
• Roaming. Got another IP? Moved from wifi to ethernet to 3G? your sessions are still open! Another thing a TCP based protocol couldn't do easily...

It doesn't replace ssh, as it still borrows authentication from ssh. But that's cool, as you can keep your ssh authorized keys.

Available in Debian unstable,testing and Backports today, and many other systems as well. Hopefully an Android client comes available soon, as the above mentioned advantages seem really tailored for android like mobile systems.

Caveat: This is new stuff, and thus hasn't quite been proven to be secure.

When I set an alarm, my clock, now running on the computer in my pocket, is smart enough to tell me how much time will pass until the alarm is scheduled to sound. This has eliminated the old problem of sleeping past meetings before being surprised by an alarm precisely half a day after I had originally planned to wake.

The price has been having to know exactly how little I will sleep: a usually depressing fact that had previously been obscured by my difficulty doing time arithmetic in my most somnolent moments.

I3→I2→I1→SI2→SI1→VS2→VS1→VVS2→VVS1→IF→FL

The GIA diamond clarity scale, shown above, is rather opaque.

Sparkleshare is a file sharing utility that keeps folders in sync on multiple computers in a similar fashion to the commercial and proprietary Dropbox. Sparkleshare recently released a Windows client, finally allowing me to start switching my co-workers at the Progressive Technology Project away from Dropbox.

Overall, I'm very impressed. In particular, I appreciate the Sparkleshare authors' decision to build on top of existing tools (git for storage and revision control, ssh for transport, and ssh public key infrastructure for authorization and authentication). That means I don't have to learn new tools and protocols to debug and it means Sparkleshare can focus on the file sharing pieces.

Despite my overall enthusiasm, I do have some serious concerns.

Protecting your credentials

A researcher found a startling security flaw in Dropbox - if you can copy a particular file from a user's computer to your own computer, you can impersonate them flawlessly, accessing all their Dropbox files without having to know the user's password. Unfortunately, Sparkleshare suffers from this same vulnerability if you use the default configuration.

When you first install Sparkleshare, it creates a password-less ssh private/public key pair and then makes the public part easily accessible to you so you can add it to your server (or gitorious account, etc). Very convenient. But it also means that all an attacker needs to do is copy your private key (and your Sparkleshare configuration file) and they get complete control over your files.

This problem is easily avoided. If you are running Linux or Mac OS X and you have your own key loaded in your ssh agent, Sparkleshare will happily use that key. So - simply by using your existing (presumably password-protected key) on your remote Sparkleshare servers, you can mitigate this problem. Sparkleshare will still load your Sparkleshare generated key, but if you don't provide that key with access to anything, no harm is done.

Confirming each use of your key

However... that leads to a new problem. If you are like me, your ssh agent is configured to ask for a confirmation every time your key is used. And, Sparkleshare regularly polls the remote git repository for changes. At best repeatedly clicking to confirm is tedious. At worst, it prevents you from intelligently rejecting malicious requests, thus defeating the whole purpose of the check.

It is possible to launch Sparkleshare via ssh-agent in an environment in which you are not requiring the confirmation when Sparkleshare uses your key, but still requiring it for all other uses. However, given the trade offs, I've decided to add a password to my Sparkleshare provided ssh key rather than using my existing key:

ssh-keygen -p -f ~/.config/sparkleshare/sparkleshare.jamie@progressivetech.org.key

Now, I am prompted to enter my passphrase when I start Sparkleshare and don't have to confirm every use of the key. And, I continue to confirm each use of my main key.

git was designed to store code, not documents

Just because something is designed for one purpose doesn't mean it can't be used for another. However, there are a few limitations.

Large files are one. git can handle files over 100MB, but may run into memory problems. I had to alter the git windowMemory setting, raising it higher than the size of the largest file.

Another problem is disk space. Since git keeps full revision history on every machine, you have to download more data than just the files that are checked out. The more edits you make to a repository, the more disk space beyond what is checked out is needed (and deleting files won't help).

Lastly, you can't use file modification times. With git, the file modification time will depend on when you checkout the files. With some fancy hook writing, you could tweak things so that the file modification date is the same as the commit date, but that still won't help you if you add an existing directory to Sparkleshare because all the files will have the same commit time.

This "bug" has been reported to git and it has been rejected because mucking with the modification time of files can have bad results when you are using make to compile code. As Linus colorfully put it:

I'm sorry. If you don't see how it's WRONG to set a datestamp back to something that will make a simple "make" miscompile your source tree, I don't know what definition of "wrong" you are talking about. It's WRONG. It's STUPID. And it's totally INFEASIBLE to implement.

Well, did I mention that git was designed for source code?

No server validation

Of all the problems I encountered, this one is by far the most serious.

Perusing ~/.config/sparkleshare/debug.log is very informative. On a default installation, after you have added a project, you'll see:

19:32:27 [Fetcher][/home/jamie/SparkleShare/.tmp/bar] Fetching folder: ssh://foo@bar.org/foo/bar 19:32:27 [Fetcher] Disabled host key checking for bar.org

Wah.

Once the project is added, whatever key fingerprint is given is stuffed into your ~/.ssh/known_hosts file. Host key checking is only disabled when you initially create the project, so if you connect to the proper server the first time, you are protected from subsequent man-in-the-middle attacks. However... if the initial fingerprint is wrong, your personal ssh configuration is now poisoned. This seems like a very bad idea. I've opened a debian bug to address it.

I recently setup an Alix board from PC Engines. I installed Debian onto a Compact Flash card using grml for the first time.

After using parted to create a single partition, I ran:

grml-debootstrap --arch i386 --filesystem ext4 --grub /dev/sdb --hostname yaqeen --mirror http://http.us.debian.org/debian \ --nopassword --release squeeze --target /dev/sdb1

I had to manually mount the partition and add console redirection for grub (and /etc/inittab).

And then, when booting, I had to manually edit the grub configuration because it was trying to find the root filesystem on /dev/sdb instead of /dev/sda. I'm sure there is some tweaking I can do via /etc/deboostrap/scripts to fix those needs.

Overall, I was very happy with the results. Thanks grml developers!

This always flashes by too quickly for me to read...

I'm preserving it here for next time.

Press the spacebar to pause... KEY MAPPING FOR CONSOLE REDIRECTION: Use the <ESC><0> key sequence for <F10> Use the <ESC><!> key sequence for <F11> Use the <ESC><@> key sequence for <F12> Use the <ESC><Ctrl><M> key sequence for <Ctrl><M> Use the <ESC><Ctrl><H> key sequence for <Ctrl><H> Use the <ESC><Ctrl><I> key sequence for <Ctrl><I> Use the <ESC><Ctrl><J> key sequence for <Ctrl><J> Use the <ESC><X><X> key sequence for <Alt><x>, where x is any letter key, and X is the upper case of that key Use the <ESC><R><ESC><r><ESC><R> key sequence for <Ctrl><Alt><Del>