Elsewhere

Hideki Yamane: Intel 910 SSD 400GB - $420

Planet Debian - Tue, 16/09/2014 - 10:10

Intel SSD 910 (400GB, SSDPEDOX400G301) is cheaper than ever in Japan - only $420 (and its spec sheet says "Recommended Customer Price BULK: $1929.00", wow).
Categories: Elsewhere

Károly Négyesi: What JS makes this monkey dance?

Planet Drupal - Tue, 16/09/2014 - 06:27

Today I found myself in a complex codebase and my ticket was: this JS on this old page makes the monkey dance, can you make it dance on the new page? Well, my JS knowledge is... limited but I have a really mean right click. So I right clicked and looked around the source Chrome showed me. There was a div with a class slideshow-node-embed-processed. Now that's important: -processed is added by the jQuery once plugin that Drupal 7 happens to ships with. It's much easier to recognize the handiwork of the plugin than actually use it -- this is true for many similar reverse engineering scenarios. Next step is ag slideshow-node-embed (you have ag installed, don't you?) which comes back with a single JS file called sites/all/themes/foo/js/node-embed.js. Next ag -A2 -B2 node-embed.js -- just searching is pointless but by adding some context to it hopefully we can see some settings or CSS necessary.

Categories: Elsewhere

Drupal core announcements: Drupal core security release window on Wednesday, September 17

Planet Drupal - Tue, 16/09/2014 - 04:20
Start:  2014-09-17 (All day) America/New_York Sprint Organizers:  David_Rothstein

The monthly security release window for Drupal 6 and Drupal 7 core will take place on Wednesday, September 17.

This does not mean that a Drupal core security release will necessarily take place on that date for either the Drupal 6 or Drupal 7 branches, only that you should prepare to look out for one (and be ready to update your Drupal sites in the event that the Drupal security team decides to make a release).

There will be no bug fix release on this date; the next window for a Drupal core bug fix release is Wednesday, October 1.

For more information on Drupal core release windows, see the documentation on release timing and security releases, and the discussion that led to this policy being implemented.

Categories: Elsewhere

Drupal Toolbox: Using VBO (and Rules) to remove spam users

Planet Drupal - Mon, 15/09/2014 - 21:10
Using VBO (and Rules) to remove spam users
Categories: Elsewhere

Ritesh Raj Sarraf: apt-offline 1.5

Planet Debian - Mon, 15/09/2014 - 20:17

I am very pleased to announce the release of apt-offline, version 1.5.

In version 1.4, the offline bug report functionality had to be dropped. In version 1.5, it is back again. apt-offline now uses the new Debian native BTS library. Thanks to its developers, this library is much more slim and neat. The only catch is that it depends on the SOAPpy library which currently is not stock in Python. If you run apt-offline of Debian, you may not have to worry as I will add a Recommends on that package. For users using it on Microsoft Windows, please ensure that you have the SOAPpy library installed. It is available on pypi.

The old bundled magic library has been replaced with the version of python magic library that Debian ships. This library is derived from the file package and is portable on almost all Unixes. For Debian users, there will be a Recommends on it too.

There were also a bunch of old, outstanding, and annoying bugs that have been fixed in this release. For a full list of changes, please refer to the git logs.

With this release, apt-offline should be in good shape for the Jessie release.

apt-offline is available on Alioth @ https://alioth.debian.org/projects/apt-offline/

AddThis:  Categories: Keywords: 
Categories: Elsewhere

Keith Packard: xserver-pending-fixes

Planet Debian - Mon, 15/09/2014 - 19:14
A Forest of X Server Changes

We’ve got about another month left in the X server merge window for 1.17 and I’ve written a small set of fixes which haven’t been reviewed yet for merging. I thought I’d advertise them a bit and see if I couldn’t encourage a few of you to take a look and see if they’re useful, correct and complete.

All of these are in my personal X server repository:

git://people.freedesktop.org/~keithp/xserver.git Cleaning up the X Registry

Branch: registry-fixes

I’ll bet most of you don’t even know about this code. It serves as a database mapping various X enumerations to strings to aid in diagnostics. For the security extensions, SECURITY and XSELinux, it holds names for all of the request, event and errors in the core protocol and all registered extensions. For X-Resource, it has the names of the registered resource types.

The X registry gets the request, event and error data from a file, “protocol.txt”, which is installed in /usr/lib/xorg/protocol.txt on my machine. It gets the resource names as a part of resource type allocation.

So, what’s wrong with this? Three basic things:

  1. A simple bug — protocol.txt is left open while the server runs. This consumes a file descriptor for no good reason.

  2. protocol.txt is read and parsed even if the security extensions aren’t available. This wastes time and memory.

  3. The resource names are kept even if X-Resource isn’t in use.

The fixes remove the configure options for including the registry code; these functions are only used by the above extensions, so we can tell whether to include the code based solely on whether the extensions are being built.

Getting rid of the TCP listener by default

Branch: listen-fixes

We’ve had the ‘-nolisten’ option for a while now to disable inbound TCP connections. It’s useful for security reasons, but we’ve never enabled this by default. This patch sequence provides configure options for each of the listen sockets (tcp, unix and local), leaves unix and local enabled by default and disables tcp by default.

A new option, ‘-listen’, is added which allows the user to override the -nolisten defaults in case they actually want to use TCP connections to X.

Glamor bug fixes

branch: glamor-fixes

This branch fixes two bugs:

  1. Scale a large pixmap down to a small pixmap. This happens when you display enormous images in a web page. Iceweasel sends the whole huge image to X and uses Render to scale it to the screen. If the image is larger than a single texture, the X server splits it up into tiles, but the code which tries to perform the merged scale is just broken. Five patches fix this.

  2. Shader-based trapezoids. This code uses area coverage to compute trapezoids. That violates the Render spec, which requires point sampling. Further, the performance of these trapezoids is lower than software (by a lot). This one patch removes the code.

Present bug fixes

branch: present-fixes

A selection of small bug fixes:

  1. Clear pending flips at CloseScreen. This removes a reference to any pending flip pixmap, allowing it to be freed. Otherwise, we’ll leak memory across server reset.

  2. Add support for PresentOptionCopy. This has been in the protocol spec for a while, and was completely trivial to implement. However, it never got done. One tiny little patch.

  3. Expose the Present API to drivers via sdksyms.sh. Until now, the present extension APIs have only been available inside the X server. This exposes them to drivers. This took a few cleanup patches first.

Use Present for Glamor XV

branch: glamor-present-xv

Painting XV to the screen should be done at vblank time to avoid tearing. Present offers vblank synchronized operations. Hooking those two together required a few new present APIs to expose the vblank functionality outside of the present code, then a bit of glamor code to hook up that new API to the XV bits.

Switching Glamor to a GL core profile context

branch: glamor-core-profile

This patch set is still in progress, but demonstrates how close we are. We’ll be requiring OpenGL 3.3 for this so that we get texture swizzling, which is required for our single channel objects.

The changes present on the branch are:

  1. Switch single channel surfaces from GLALPHA to GLRED.

  2. Use vertex array objects.

  3. Switch ephyr over to using a core 3.3 profile.

Still left to do is

  1. Switch Render code to VBOs

The core code uses VBOs everywhere, but the Render code doesn’t. This means that all Render drawing fails, which makes the resulting server not very useful.

My main objective for getting this done is to reduce memory usage by about 16MB, which is the space allocated for software rendering in Mesa in case someone does something which the hardware doesn’t handle, and that can only with some legacy OpenGL APIs.

Please help out!

All of these friendly little patches are looking for a bit of review so that they can get merged before the 1.17 window closes.

Categories: Elsewhere

Thomas Goirand: Backporting libjs-angularjs and libjs-d3 to Wheezy

Planet Debian - Mon, 15/09/2014 - 19:13

If you didn’t notice, Javascript isn’t as simple as it used to be… Want to backport the 2 simple javascript libs? No problem. You then “just” need to backport a bunch of other packages which are build-dependencies… (and file #761670, #761672, and #761674 on the way when rebuilding…). Here’s the short list:

gyp node-abbrev node-ansi node-ansi-color-table node-archy node-async node-block-stream node-combined-stream node-contextify node-cookie-jar node-cssom node-delayed-stream node-diff node-eyes node-forever-agent node-form-data node-fstream node-fstream-ignore node-github-url-from-git node-glob node-graceful-fs node-gyp node-htmlparser node-inherits node-ini node-jake node-jsdom node-json-stringify-safe node-lockfile node-lru-cache node-marked node-mime node-minimatch node-mkdirp node-mute-stream node-node-uuid node-nopt node-normalize-package-data node-npmlog node-once node-optimist node-osenv node-qs node-queue-async node-read node-read-package-json node-request node-retry node-rimraf node-semver node-sha node-sigmund node-slide node-smash node-tar node-tunnel-agent node-uglify node-underscore node-utilities node-vows node-whic node-which node-wordwrap nodejs npm ruby-ronn

Yes, that’s 66 packages above… And of course, backporting some ruby stuff makes sense… :)

Categories: Elsewhere

Mediacurrent: Top Drupal Marketing Automation Modules

Planet Drupal - Mon, 15/09/2014 - 18:19

A strong lead management process requires B2B marketing professionals to respond to each prospect within the buying process. However, as your business grows, understanding and responding relevantly to a buyer’s interest is almost impossible to do manually. To ensure that your marketing efforts are targeting customers and prospects with the right messages at the right time, integrating a marketing automation platform into your Drupal website is key. 

Categories: Elsewhere

Vincent Sanders: NetSurf 3.2

Planet Debian - Mon, 15/09/2014 - 17:08
We recently released a new version of NetSurf this was largely to address numerous small bugs but did also include the persistent caching implementation I have written about previously. A release used to require the release manager (usually me) to perform a lot of manual processes and while we had a checklist it was far too easy to miss things.

The Continuous Integration (CI) system combined with signed release tags in git has resulted in a greatly simplified process indeed it has become almost completely automated. The majority of the manual work is now confined to doing the tasks that require actual decision making and checking we are releasing what was intended.

By having the CI system build release binaries the project now has a much clearer and importantly traceable process, I can recommend such a system to any project that produces releases especially if they release binaries for any of their targets.

I have also managed to package and upload this version of NetSurf ready for the Debian Jessie release. I would like to thank Jonathan Wiltshire for his assistance in ensuring this was a good quality package.

The release incorporates the successfully merged work of Rupinder Singh who was our our GSoc 2014 student. Rupinder mainly made improvements to our core DOM implementation and was very responsive and enthusiastic throughout his time despite the mentor team sometimes not being available.

This work goes towards improving NetSurf in the future by ensuring the underlying features are present in our core libraries. The GSoc mentors and all project developers are all pleased with the results of this years GSoc participation and would like to thank everyone involved in making our participation possible.

Along with the good news comes a little bad:
PowerPC Mac OS X
Despite repeated calls for assistance with new hardware and Java builds none has been forthcoming meaning that from this release we ware no longer able to ship PowerPC builds for MAC OS X.

The main issue is the last version of MAC OS X that runs on PPC is Leopard and there is no viable Java 1.6 port necessary for our CI system to run. Additionally the fully loaded PPC Mac mini (kindly donated to us by Mythic Beasts) had become far too slow to keep up with our builds and was causing long delays.
Bugs
We have a lot of bugs, in fact just during this release cycle we have 30 more bugs reported than we closed.So while the new bug reporting system has been a success and our users are reporting issues when they find them the development team is not keeping up..

The failure to keep up stems from the underlying issue of lack of manpower. We have relatively few active developers which is especially problematic when there are many users for a platform, such as RISCOS, but the maintainer is unable to commit enough time to fixing issues.

If you would like to help making NetSurf a better browser we are always happy to work with new contributors.

Categories: Elsewhere

Dries Buytaert: Reflections on Drupal in China

Planet Drupal - Mon, 15/09/2014 - 16:14
Topic: DrupalLocation: China

I just spent the past week in China, and I thought I'd share a few reflections on the state of Drupal in China.

First, let me set the stage. There are 1.35 billion people living in China; that is almost 20 percent of the world's population. Based on current trends, China's economy will overtake the US within the next few years. At that point, the US economy will no longer be the largest economy in the world. China's rapid urbanization is what has led to the country's impressive economic growth over the past couple of decades and it doesn't look like it is going to stop anytime soon. To put that in perspective: China currently produces and uses 60 percent of the world's cement.

In terms of Drupal, the first thing I learned is that "Drupal" sounds like "the pig is running" ("Zhu Pao") in Chinese. Contrary to a pig's rather negative reputation in the West, many Chinese developers find that cute. A pig is a more honorable sign in Chinese astrology and culture. Phew!

In terms of adoption, it feels like the Drupal community in China is about 8 to 10 years behind compared to North America or Europe. That isn't a surprise, as Open Source software is a more recent phenomenon in China than it is in North America or Europe.

Specifically, there are about 5 Drupal companies in Shanghai (population of 21 million people), 3 Drupal companies in Beijing (population of 23 million people) and 5 Drupal companies in Hong Kong (population of 7 million people). The largest Drupal companies in China have about 5 Drupal developers on staff. Four of the 5 Shanghai companies are subsidiaries from European Drupal companies. The exception is Ci&T, which has 40 Drupal developers in China. Ci&T is a global systems integrator with several thousand employees worldwide, so unlike the other companies I met, they are not a pure Drupal play. Another point of reference is that the largest Drupal event in China attracted 200 to 300 attendees.

Given that China has 4 times the population of the US, or 2 times the population of Europe, what are we missing? In talking to different people, it appears the biggest barrier to adoption is language. The problem is that Chinese Drupal documentation is limited; translation efforts exist but are slow. The little documentation that is translated is often outdated and spread out over different websites. Less than 20 percent of the Chinese Drupal developers have an account on Drupal.org, simply because they are not fluent enough in the English language. Most Drupal developers hang out on QQ, an instant messaging tool comparable to Skype or IRC. I saw QQ channels dedicated to Drupal with a couple thousand of Drupal developers.

There is no prominent Chinese content management system; most people appear to be building their websites from scratch. This gap could provide a big opportunity for Drupal. China's urbanization equals growth -- and lots of it. Like the rest of the economy, Drupal and Open Source could be catching up fast, and it might not take long before some of the world's biggest Drupal projects are delivered from China.

Supporting Drupal's global growth is important so I'd love to improve Drupal's translation efforts and make Drupal more inclusive and more diverse. Drupal 8's improved multilingual capabilities should help a lot, but we also have to improve the tools and processes on Drupal.org to help the community maintain multi-lingual documentation. Discussing this with both the Drupal Association and different members of our community, it's clear that we have a lot of good ideas on what we could do but lack both the funding and resources to make it happen faster.

Special thanks to Fan Liu (Delivery Manager @ Ci&T), Jingsheng Wang (CEO @ INsReady Inc.) and Keith Yau. All the Drupal people I met were welcoming, fun and are working hard.

Categories: Elsewhere

Acquia: Custom Distributions on Acquia Cloud: Part 1 -- Drush Make

Planet Drupal - Mon, 15/09/2014 - 15:54

Every developer has a slightly different approach to building their Drupal sites. I’ve tried just about every approach, and they all have their merits, but my favorite is Drush Make. Before joining Acquia, I didn’t realize Acquia Cloud supported Drush Make, but I was delighted to discover that I was wrong. Assuming I’m not the only person who had missed this fact, I wanted to spend a little time highlighting where this exists and how I’m using it.

Categories: Elsewhere

Jonathan Brown: Drupal & Bitcoin

Planet Drupal - Mon, 15/09/2014 - 14:46

Almost everything we do on the web will work better with autonomous blockchain technologies such as Bitcoin & ethereum because they allow systems to be built with unbreakable rules. Unlike Facebook, Twitter, Airbnb, Uber, PayPal or eBay, no executive authority can step in and say the rules don't apply to you.

Of all the blockchain technologies, Bitcoin is currently the most high profile. It is a massive area of growth in the startup eco-system.

Drupal 8 is going to be a fantastic platform for building startups, but we need to make sure that it is also a fantastic platform for blockchain startups.

I've started by creating a Drupal 7 & 8 project called Coin Tools. It provides various components that would be useful for building a Bitcoin web product. Many of the altcoins are very similar to Bitcoin so the project could easily be extended to accommodate them.

I've also created a Blockchain BoF at DrupalCon Amsterdam on the Tuesday at 10:45.

Coin Tools base module

Contains the following field types:

  • Address
  • Amount
  • Transaction
Widgets

Formatters

Coin Tools Daemon
  • facilitates configuration and access to bitcoind service
  • triggers a hook when a Bitcoin transaction is detected
  • provides a full UI for browsing transactions and sending and receiving bitcoin

I am currently working on a full implementation of BIP 70 which provides a much improved payment experience for the customer. Then I will add integration with Payment / Commerce. This means it will be possible to receive payments in Commerce without using a third party payment processor.

Coins Tools Fiat
  • obtains bitcoin exchange rates from BitcoinAverage (which I consider to be the gold standard), falling back to BitPay BBB
  • facilitates rendering of fiat amounts
  • user can select preferred fiat currency
  • current bitcoin value block

If you would like help developing your Bitcoin startup on Drupal, please get in touch.

Categories: Elsewhere

Makina Corpus: A Drupal front-end theme with Bootstrap, LESS and Gulp

Planet Drupal - Mon, 15/09/2014 - 14:45
More and more articles with these words are appearing right now: here's our approach for a front-end theme complying with Web good practices.
Categories: Elsewhere

Junichi Uekawa: ARM assembly.

Planet Debian - Mon, 15/09/2014 - 13:51
ARM assembly. I was reading up some docs on Unified Assembly Language (UAL). and confusions. I don't seem to be able to find a comprehensive doc about what works and what doesn't. Heh.

Categories: Elsewhere

Julien Danjou: Python bad practice, a concrete case

Planet Debian - Mon, 15/09/2014 - 13:09

A lot of people read up on good Python practice, and there's plenty of information about that on the Internet. Many tips are included in the book I wrote this year, The Hacker's Guide to Python. Today I'd like to show a concrete case of code that I don't consider being the state of the art.

In my last article where I talked about my new project Gnocchi, I wrote about how I tested, hacked and then ditched whisper out. Here I'm going to explain part of my thought process and a few things that raised my eyebrows when hacking this code.

Before I start, please don't get the spirit of this article wrong. It's in no way a personal attack to the authors and contributors (who I don't know). Furthermore, whisper is a piece of code that is in production in thousands of installation, storing metrics for years. While I can argue that I consider the code not to be following best practice, it definitely works well enough and is worthy to a lot of people.

Tests

The first thing that I noticed when trying to hack on whisper, is the lack of test. There's only one file containing tests, named test_whisper.py, and the coverage it provides is pretty low. One can check that using the coverage tool.

$ coverage run test_whisper.py
...........
----------------------------------------------------------------------
Ran 11 tests in 0.014s
 
OK
$ coverage report
Name Stmts Miss Cover
----------------------------------
test_whisper 134 4 97%
whisper 584 227 61%
----------------------------------
TOTAL 718 231 67%


While one would think that 61% is "not so bad", taking a quick peak at the actual test code shows that the tests are incomplete. Why I mean by incomplete is that they for example use the library to store values into a database, but they never check if the results can be fetched and if the fetched results are accurate. Here's a good reason one should never blindly trust the test cover percentage as a quality metric.

When I tried to modify whisper, as the tests do not check the entire cycle of the values fed into the database, I ended up doing wrong changes but had the tests still pass.

No PEP 8, no Python 3

The code doesn't respect PEP 8 . A run of flake8 + hacking shows 732 errors… While it does not impact the code itself, it's more painful to hack on it than it is on most Python projects.

The hacking tool also shows that the code is not Python 3 ready as there is usage of Python 2 only syntax.

A good way to fix that would be to set up tox and adds a few targets for PEP 8 checks and Python 3 tests. Even if the test suite is not complete, starting by having flake8 run without errors and the few unit tests working with Python 3 should put the project in a better light.

Not using idiomatic Python

A lot of the code could be simplified by using idiomatic Python. Let's take a simple example:

def fetch(path,fromTime,untilTime=None,now=None):
fh = None
try:
fh = open(path,'rb')
return file_fetch(fh, fromTime, untilTime, now)
finally:
if fh:
fh.close()


That piece of code could be easily rewritten as:

def fetch(path,fromTime,untilTime=None,now=None):
with open(path, 'rb') as fh:
return file_fetch(fh, fromTime, untilTime, now)


This way, the function looks actually so simple that one can even wonder why it should exists – but why not.

Usage of loops could also be made more Pythonic:

for i,archive in enumerate(archiveList):
if i == len(archiveList) - 1:
break


could be actually:

for i, archive in enumerate(itertools.islice(archiveList, len(archiveList) - 1):


That reduce the code size and makes it easier to read through the code.

Wrong abstraction level

Also, one thing that I noticed in whisper, is that it abstracts its features at the wrong level.

Take the create() function, it's pretty obvious:

def create(path,archiveList,xFilesFactor=None,aggregationMethod=None,sparse=False,useFallocate=False):
# Set default params
if xFilesFactor is None:
xFilesFactor = 0.5
if aggregationMethod is None:
aggregationMethod = 'average'
 
#Validate archive configurations...
validateArchiveList(archiveList)
 
#Looks good, now we create the file and write the header
if os.path.exists(path):
raise InvalidConfiguration("File %s already exists!" % path)
fh = None
try:
fh = open(path,'wb')
if LOCK:
fcntl.flock( fh.fileno(), fcntl.LOCK_EX )
 
aggregationType = struct.pack( longFormat, aggregationMethodToType.get(aggregationMethod, 1) )
oldest = max([secondsPerPoint * points for secondsPerPoint,points in archiveList])
maxRetention = struct.pack( longFormat, oldest )
xFilesFactor = struct.pack( floatFormat, float(xFilesFactor) )
archiveCount = struct.pack(longFormat, len(archiveList))
packedMetadata = aggregationType + maxRetention + xFilesFactor + archiveCount
fh.write(packedMetadata)
headerSize = metadataSize + (archiveInfoSize * len(archiveList))
archiveOffsetPointer = headerSize
 
for secondsPerPoint,points in archiveList:
archiveInfo = struct.pack(archiveInfoFormat, archiveOffsetPointer, secondsPerPoint, points)
fh.write(archiveInfo)
archiveOffsetPointer += (points * pointSize)
 
#If configured to use fallocate and capable of fallocate use that, else
#attempt sparse if configure or zero pre-allocate if sparse isn't configured.
if CAN_FALLOCATE and useFallocate:
remaining = archiveOffsetPointer - headerSize
fallocate(fh, headerSize, remaining)
elif sparse:
fh.seek(archiveOffsetPointer - 1)
fh.write('\x00')
else:
remaining = archiveOffsetPointer - headerSize
chunksize = 16384
zeroes = '\x00' * chunksize
while remaining > chunksize:
fh.write(zeroes)
remaining -= chunksize
fh.write(zeroes[:remaining])
 
if AUTOFLUSH:
fh.flush()
os.fsync(fh.fileno())
finally:
if fh:
fh.close()


The function is doing everything: checking if the file doesn't exist already, opening it, building the structured data, writing this, building more structure, then writing that, etc.

That means that the caller has to give a file path, even if it just wants a whipser data structure to store itself elsewhere. StringIO() could be used to fake a file handler, but it will fail if the call to fcntl.flock() is not disabled – and it is inefficient anyway.

There's a lot of other functions in the code, such as for example setAggregationMethod(), that mixes the handling of the files – even doing things like os.fsync() – while manipulating structured data. This is definitely not a good design, especially for a library, as it turns out reusing the function in different context is near impossible.

Race conditions

There are race conditions, for example in create() (see added comment):

if os.path.exists(path):
raise InvalidConfiguration("File %s already exists!" % path)
fh = None
try:
# TOO LATE I ALREADY CREATED THE FILE IN ANOTHER PROCESS YOU ARE GOING TO
# FAIL WITHOUT GIVING ANY USEFUL INFORMATION TO THE CALLER :-(
fh = open(path,'wb')


That code should be:

try:
fh = os.fdopen(os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL), 'wb')
except OSError as e:
if e.errno = errno.EEXIST:
raise InvalidConfiguration("File %s already exists!" % path)


to avoid any race condition.

Unwanted optimization

We saw earlier the fetch() function that is barely useful, so let's take a look at the file_fetch() function that it's calling.

def file_fetch(fh, fromTime, untilTime, now = None):
header = __readHeader(fh)
[...]


The first thing the function does is to read the header from the file handler. Let's take a look at that function:

def __readHeader(fh):
info = __headerCache.get(fh.name)
if info:
return info
 
originalOffset = fh.tell()
fh.seek(0)
packedMetadata = fh.read(metadataSize)
 
try:
(aggregationType,maxRetention,xff,archiveCount) = struct.unpack(metadataFormat,packedMetadata)
except:
raise CorruptWhisperFile("Unable to read header", fh.name)
[...]


The first thing the function does is to look into a cache. Why is there a cache?

It actually caches the header based with an index based on the file path (fh.name). Except that if one for example decide not to use file and cheat using StringIO, then it does not have any name attribute. So this code path will raise an AttributeError.

One has to set a fake name manually on the StringIO instance, and it must be unique so nobody messes with the cache

import StringIO
 
packedMetadata = <some source>
fh = StringIO.StringIO(packedMetadata)
fh.name = "myfakename"
header = __readHeader(fh)


The cache may actually be useful when accessing files, but it's definitely useless when not using files. But it's not necessarily true that the complexity (even if small) that the cache adds is worth it. I doubt most of whisper based tools are long run processes, so the cache that is really used when accessing the files is the one handled by the operating system kernel, and this one is going to be much more efficient anyway, and shared between processed. There's also no expiry of that cache, which could end up of tons of memory used and wasted.

Docstrings

None of the docstrings are written in a a parsable syntax like Sphinx. This means you cannot generate any documentation in a nice format that a developer using the library could read easily.

The documentation is also not up to date:

def fetch(path,fromTime,untilTime=None,now=None):
"""fetch(path,fromTime,untilTime=None)
[...]
"""
 
def create(path,archiveList,xFilesFactor=None,aggregationMethod=None,sparse=False,useFallocate=False):
"""create(path,archiveList,xFilesFactor=0.5,aggregationMethod='average')
[...]
"""


This is something that could be avoided if a proper format was picked to write the docstring. A tool cool be used to be noticed when there's a diversion between the actual function signature and the documented one, like missing an argument.

Duplicated code

Last but not least, there's a lot of code that is duplicated around in the scripts provided by whisper in its bin directory. Theses scripts should be very lightweight and be using the console_scripts facility of setuptools, but they actually contains a lot of (untested) code. Furthermore, some of that code is partially duplicated from the whisper.py library which is against DRY.

Conclusion

There are a few more things that made me stop considering whisper, but these are part of the whisper features, not necessarily code quality. One can also point out that the code is very condensed and hard to read, and that's a more general problem about how it is organized and abstracted.

A lot of these defects are actually points that made me start writing The Hacker's Guide to Python a year ago. Running into this kind of code makes me think it was a really good idea to write a book on advice to write better Python code!

The Hacker's Guide to Python

A book I wrote talking about designing Python applications, state of the art, advice to apply when building your application, various Python tips, etc. Interested? Check it out.

Categories: Elsewhere

DrupalCon Amsterdam: DrupalCon for Designers

Planet Drupal - Mon, 15/09/2014 - 11:54

Unlike DrupalCon Austin, there is no separate UX track at DrupalCon Amsterdam. Ruben and I had to balance both design and development into one track. It was challenging, but it forced us to be really careful about every decision remade.

We really wanted there to be an overlap between sessions, frontend development and design are closer than ever in the workplace, so we wanted to reflect that in our session line up.

Here are a few sessions we think complement each other really well.

The State of the Frontend

Not sure what to learn or where to start in frontend development? Let David and Brian guide you through the landscape, introducing new tools and techniques.

We've also planned this session as an introduction to the entire track, pointing signposts towards other sessions in that track that can fill in more in-depth knowledge about particular subjects. I would encourage everyone with an interest in frontend development to attend the session if only to better understand which sessions in the frontend track are right for them.

We also have a few session that compliment and feed in to each other. Here are a few sessions we think work really well together.

Because it's about the interactions. (Better UX through prototyping) & Axure Prototyping for Drupal

How do we shift from an old design process to a new one while keeping our clients and team members happy? Roy Scholten, UX co-maintainer for Drupal Core, walks us through why prototyping is a better way to design and how to introduce this into your work life.

Then, in Axure Prototyping for Drupal, Dani Nordin takes us on a deep dive of Axure, a prototyping tool you can use without coding. I've been using it recently on a client project and I've been impressed with the way it introduces some concepts of frontend development without asking you to write code.

The Future of HTML and CSS

As a designer, knowing the limits and capabilities of your medium is important, also it means you can call your developers out when they hit you with “That's not possible” which is fun. If you're interested in seeing where web browsers are heading and finding new tricks to add to your repertoire, Preston's talk on The Future of HTML and CSS should be a treat.

Getting a CLUE at the Command Line

As a designer/front-end dev, I've never had a formal education on the command line, it's always been something I've picked up as I've needed it. It used to scare me. This is me using the command line a few years ago:

I'm so happy Emma Jane is talking on the command line in Getting a CLUE at the Command Line in the same vein as her introduction to Git at DrupalCon Prague last year. The command line is an immensely powerful and productive tool and I'm looking forward to picking up a few tips.

Open Source Design

Something I've struggled with, both in my day job and as the maintainer of Drupal's admin theme is how do we bring successful design together with Open Source. I'm really happy that Jan-C. Borchardt, agreed to speak at DrupalCon and share his knowledge and experience with us. Jan is a big supporter of usability and design in free and open source software, with projects including A guide to Usability in Free Software, OwnCloud, Libre Projects, and the brilliant Terms of Service; Didn't Read.

It's really important for Drupal for us to gain insight from people outside the community and learn from other projects. Thanks again Jan for agreeing to speak! I can't wait!

--
Lewis Nyman (LewisNyman)
DrupalCon Amsterdam Frontend Track Chair

Categories: Elsewhere

Cyril Brulebois: Freelance Debian consultant

Planet Debian - Mon, 15/09/2014 - 11:20

I’m not used to talking about my day job but here’s an exception.

Over the past few years I worked in two startups (3 years each). It was nice to spend time in different areas: one job was mostly about research and development in a Linux cluster environment; the other one was about maintaining a highly-customized, Linux-based operating system, managing a small support team, and performing technological surveillance in IT security.

In the meanwhile I’ve reached a milestone: 10 years with Debian. I had been wondering for a few months whether I could try my luck going freelance, becoming a Debian consultant. I finally decided to go ahead and started in August!

The idea is to lend a hand for various Debian-related things like systems administration, development/debugging, packaging/repository maintenance, or Debian Installer support, be it one-shot or on a regular basis. I didn’t think about trainings/workshops at first but sharing knowledge is something I’ve always liked, even if I didn’t become a teacher.

For those interested, details can be found on my website: https://mraw.org/.

Of course this doesn’t mean I’m going to put an end to my benevolent activities within Debian, especially as a Debian Installer release manager. Quite the contrary in fact! See the August and September debian-boot@ archives, which have been busy months. :)

Categories: Elsewhere

Martin Pitt: autopkgtest 3.5: Reboot support, Perl/Ruby implicit tests

Planet Debian - Mon, 15/09/2014 - 10:23

Last week’s autopkgtest 3.5 release (in Debian sid and Ubuntu Utopic) brings several new features which I’d like to announce.

Tests that reboot

For testing low-level packages like init or the kernel it is sometimes desirable to reboot the testbed in the middle of a test. For example, I added a new boot_and_services systemd autopkgtest which configures grub to boot with systemd as pid 1, reboots, and then checks that the most important services like lightdm, D-BUS, NetworkManager, and cron come up as expected. (This test will be expanded a lot in the future to cover other areas like the journal, logind, etc.)

In a testbed which supports rebooting (currently only QEMU) your test will now find an “autopkgtest-reboot” command which the test calls with an arbitrary “marker” string. autopkgtest will then reboot the testbed, save/restore any files it needs to (like the tests file tree or previously created artifacts), and then re-run the test with ADT_REBOOT_MARK=mymarker.

The new “Reboot during a test” section in README.package-tests explains this in detail with an example.

Implicit test metadata for similar packages

The Debian pkg-perl team recently discussed how to add package tests to the ~ 3.000 Perl packages. For most of these the test metadata looks pretty much the same, so they created a new pkg-perl-autopkgtest package which centralizes the logic. autopkgtest 3.5 now supports an implicit debian/tests/control control file to avoid having to modify several thousand packages with exactly the same file.

An initial run already looked quite promising, 65% of the packages pass their tests. There will be a few iterations to identify common failures and fix those in pkg-perl-autopkgtest and autopkgtestitself now.

There is still some discussion about how implicit test control files go together with the DEP-8 specification, as other runners like sadt do not support them yet. Most probably we’ll declare those packages XS-Testsuite: autopkgtest-pkg-perl instead of the usual autopkgtest.

In the same vein, Debian’s Ruby maintainer (Antonio Terceiro) added implicit test control support for Ruby packages. We haven’t done a mass test run with those yet, but their structure will probably look very similar.

Categories: Elsewhere

MariqueCalcus: Drush

Planet Drupal - Mon, 15/09/2014 - 09:30

Dru(pal)sh(ell) is an essential tool if you are working with Drupal intensively. If you don't know it yet, Drush is command line shell and scripting interface for Drupal. It helps you to quickly perform various administation and maintenance tasks using only a terminal. If you are not convinced, you should probably install it and give it a try. You won't regret it. It's awesome and will save you headaches, time and again more time. If you are using Drush, you must have been using the command "drush cc all" like a million time. I have. But Drush can do much more.

Read More...
Categories: Elsewhere

Gergely Nagy: Looking ahead

Planet Debian - Mon, 15/09/2014 - 08:34

A little more than a year ago, I started working as a syslog-ng OSE developer full-time. That was a tremendously important milestone in my career, as one of the goals I wanted to achieve in life - to work on free software for a living - became a reality. Rocket boots were fired up, and we accomplised quite a lot in the past year, and I'm very, very proud of the work we did - we, the whole community. I enjoyed every bit of it, but as it turns out, some of the other desires I wish to pursue, and new challenges I am looking for, will lead me in a different direction. At the end of August, I handed in my resignation, and the past friday was my last work day at BalaBit.

There are a few questions that will likely be asked, and I'll try my best to answer them beforehand. Questions such as: What will happen to syslog-ng?, Who will be the new maintainer?, How does this affect your Debian work?, and so on, and so forth.

syslog-ngWhat will happen to syslog-ng?

After careful consideration, with the syslog-ng team at BalaBit, we decided that they will take over OSE-related responsibilities as well. They will do releases, they will engage the people on GitHub and the mailing list, they will take care of the @sngOSE twitter account.

During the past few months, we've been working on pushing the team closer to the community: we moved issues to GitHub, the team submitted pull requests, and in general, we moved closer to each other in every possible way.

While they may not have the open source maintainer experience I had, they are all capable folk, and will quickly learn on the job. The team taking over also has the advantage of having more manpower, and better collaboration between the premium edition and OSE.

Who will be the syslog-ng maintainer?

There will be no single bottle neck. This has both good and bad implications, but I believe the good ones outweigh the disadvantages.

What about the roadmap? When will 3.6.1 happen?

The roadmap is already laid out for 3.6, but it is the team's judgement when it will be released. The latest 3.6.0beta2 was released together with the team, too. I expect there will be delays (I planned 3.6.1 to be released on September 27th), but not much; a few weeks, perhaps.

What will your involvement in syslog-ng development be?

With changing jobs, my involvement will drastically decrease. I will remain a syslog-ng user, and I will continue packaging it for Debian and Ubuntu, and will keep my unofficial repository running. I do not see myself contributing much, perhaps bug reports, opinions and an occasional idea.

Nevertheless, my expertise is still considerable, and I will help the team and the community in whatever way I can - but I will be severely time constrained.

Debian

While BalaBit were very free software friendly, and allowed me to work on Debian tasks from time to time, my other activities took up an increasingly big chunk of my paid time, and I had to cut back a little. At the new job, things will be a bit different: while I won't have as much time to do Debian work on the job, I will have a lot more free time, in which I hope to do more for Debian.

I will - as promised above - continue maintaining syslog-ng, and all other packages I currently maintain. Apart from those, I wish to make myself more useful within the Clojure and Hy teams.

MiscellaneousWhere to are you moving?

This is something I will answer in due time. For now, I will have two weeks off, which I wish to spend my way, picking up projects I neglected for far too long.

Categories: Elsewhere

Pages

Subscribe to jfhovinne aggregator - Elsewhere