Forbidden Projects

Project X

By Tom on October 4, 2011 7:31 PM | No Comments

I'm still here, and programming. Work has been taking up most of my "programming brain" recently, but the results will hopefully be released as free and open source software very soon.

For the last few months I've been working on a new non-work project as well, which I'm not going to say too much about just yet - let's call it Project X for now. It's a piece of major "itch scratching" for me, but I have a feeling that many other scientists who use Linux will love it. Perhaps many other people besides. Watch this space...

Command line weight loss

By Tom on July 6, 2011 9:25 PM | No Comments

$ sudo setweight --bmi=21

It'd be nice if it were that simple. But it almost is!

Scales of Truth is a command line implementation of The Hacker's Diet. The basic principle is to lose (or gain, I guess, theoretically at least) weight by taking an engineering approach to your body's energy requirements. There are quite a few implementations of this or similar things (e.g. The Hacker's Diet Online and Physics Diet), and an important feature is that the required day-to-day administration, i.e. typing in your weight, is not very time-consuming. Less than five minutes a day? Well opening a website, logging in, typing a value in and so on seems like a lot of work to me. With Scales of Truth, you simply click over to one of the many terminals you no doubt have open, and type:

$ sot XXX

Where "XXX" is your current scale reading (in your choice of unit). Scales of Truth does all the necessary calculations, backs up your readings using Git (because losing months of figures would really suck), and displays some interesting statistics:

Current mass estimate: XXX kg (instantaneous BMI XX.X kg/m^2)
7-day change: -.42 kg (-1757.2 kJ or -419.9 kcal per day)
30-day change: -2.77 kg (-2704.2 kJ or -646.3 kcal per day)

If you happen to miss a few readings, it interpolates the missing values automatically so your running average stays up to date. For the average reader of this site, the whole procedure probably takes slightly less than two seconds. That is, assuming you can resist taking a look at your progress using "sot --graph":

That's my own graph for the last few months, so I've taken the liberty of removing the actual numbers. Suffice to say that each tick on the mass axis corresponds to two kilograms, and I'm rather pleased with progress so far...

You can download Scales of Truth right here. Simply download the file, open it in a text editor, satisfy yourself that it's not going to do anything evil to your computer, then follow the instructions.

The T Word

By Tom on October 31, 2010 1:11 PM | No Comments

Refuse to be terrorised.

A criminal does not become a terrorist because of which country they came from, because they choose to attack aeroplanes, because they are religiously or politically motivated or because they use bombs. They become a terrorist because we respond with fear and define them as such. Well, we can choose not to do that.

Everyone should remember that the perpetrators of the recent cargo plane bombing, the Times Square bombing, the underpants bombing, the liquid bomb plots, the 2007 London and Glasgow attacks and countless others (at least in the "Western world") did not succeed.

Trackball Configuration

By Tom on September 13, 2010 11:01 PM | No Comments

I have a slightly strange configuration for my trackball. I have it set up left-handed, so the main button on the right is "left click", and vice-versa. There are two smaller buttons, one on each side, and I have the left-hand one as "middle click" and the right-hand one turning the ball into a "scroll wheel" when it's held ("wheel emulation" in X speak). To complicate matters, I have the trackpad on my laptop right-handed. In previous times, I'd configure this with some lines like these in the relevant section of xorg.conf to set up the buttons properly just for the trackball:

Option "Buttons" "9"
Option "EmulateWheel" "on"
Option "EmulateWheelButton" "9"
Option "ButtonMapping" "3 8 1 4 5 6 7 2 9"

That worked until the business of HAL for X input configuration came along. Then I put something like this in /etc/hal/fdi/policy/trackball.fdi:

<deviceinfo version="0.2">
<device>
<match key="info.product" string="Logitech USB Trackball">
   <merge key="input.x11_driver" type="string">mouse</merge>
   <merge key="input.x11_options.Buttons" type="string">5</merge>
   <merge key="input.x11_options.EmulateWheel" type="string">true</merge>
   <merge key="input.x11_options.EmulateWheelButton" type="string">9</merge>
   <merge key="input.x11_options.ZaxisMapping" type="string">4 5</merge>
   <merge key="input.x11_options.ButtonMapping" type="string">3 2 1 4 5 6 7 8 9</merge>
   <merge key="input.x11_options.MinSpeed" type="string">0.40</merge>
   <merge key="input.x11_options.MaxSpeed" type="string">0.60</merge>
   <merge key="input.x11_options.AccelFactor" type="string">0.010</merge>
</match>
</device>
</deviceinfo>

That worked, but never quite satisfactorily. In some versions, the trackball would only work properly if plugged in after I'd booted up and logged in. In later versions (notably Fedora 12 and 13) it would never work correctly at all. In Fedora 13, the HAL method was deprecated and replaced with a method via files under /etc/X11/xorg.conf.d/. I added a file called "01-trackball.conf" containing:

Section "InputClass"
Identifier     "trackball"
MatchProduct   "Logitech USB Trackball"
Option         "Buttons" "9"
Option         "EmulateWheel" "on"
Option         "EmulateWheelButton" "9"
Option         "ButtonMapping" "3 8 1 4 5 6 7 2 9"
EndSection

Section "InputClass"
Identifier     "trackpad"
MatchProduct   "SynPS/2 Synaptics TouchPad"
Option         "ButtonMapping" "1 2 3"
EndSection

These lines define the exact button configuration for both the trackball and the trackpad, and the combination of InputClass and MatchProduct means that the rule works even if the trackball isn't present when X starts.

But it still didn't quite work. It seemed that the settings always got overridden by Gnome's central mouse options, left or right handed but not my strange mixture. The main buttons (left and right click) on the trackball had to be the same as the trackpad because Gnome couldn't understand the idea of having the two configured differently.

With a little clear thought, it's probably already obvious what was going wrong, though it was infuriating for a long time. Gnome (specifically gnome-settings-daemon) does indeed override your nicely thought out X config. Fortunately, besides "left handed" and "right handed" modes, it also has an extra mode called "don't screw with my damn setup you piece of ****". It just takes a little more surgery to enable:

$ gconftool-2 -s /apps/gnome_settings_daemon/plugins/mouse/active --type bool false

With Gnome's mouse config stuff safely (and permanently) anaesthetised, you're clear to configure things your way again.

Thinkpad DVD Noise

By Tom on August 22, 2010 9:40 PM | No Comments

My handy household hint for the weekend:

One problem I've been experiencing with my new(ish) Thinkpad T500 is the large amount of noise made by the DVD drive. It's loud enough to be distracting when watching a film - not quite enough to drown it out, but enough to be clearly heard over even the louder parts until your mind starts filtering it out. The tray rattles against the casing, which makes it much worse. I found the noise could be reduced significantly by squashing one or two bits of folded tissue between then tray and the casing above, but that's a nasty solution.

Of course, there's a proper fix for this. Just go into the BIOS config (press the blue ThinkVantage button when the BIOS boot screen shows up, hold it down until the message goes away, then follow the instructions), and find the options for the CD drive, and set it to "Silent". Problem solved - the drive becomes almost inaudible, and it doesn't seem to have any adverse effect on the playback of DVDs.

iAudio X5 Tagging

By Tom on June 30, 2010 9:17 AM | 2 Comments

The Cowon iAudio X5 is, in many ways, the ideal music player for the discerning geek. It's Linux friendly, behaving as a normal USB mass storage device requiring no special software nor drivers; it plays Ogg/Vorbis and FLAC files; it has a line in connector; it has fantastic sound quality; it organises your files by folder, so you can have your files organised how you really want them. It also has a battery which lasts ages, particularly the X5L version. When my X5L was younger, a full charge would last for about two weeks of fairly constant use at work. Now it's several years old - I've actually forgotten how old exactly - and it still lasts for many days.

It does, however, have a particular weirdness when handling Ogg/Vorbis files. Its tag parser is extremely basic, and apparently just skips one character (the equals sign) after finding the letters "ARTIST" in the tags. But most recent ripping software adds an extra field, called ARTISTSORT. The X5 often finds that instead of ARTIST, and thinks the name of the artist is something like "ORT=Haza, Ofra" instead of "Ofra Haza".

So, I wrote a little script to remove the ARTISTSORT tags altogether. It's a cheap solution - it'd probably be better just to move that tag to the end (possibly the start, I didn't really test) of the tags field, but this field isn't used for sorting on the X5 anyway.

Here's the script ("ogg-retag"). It's really simple:

#!/bin/sh

if [ -e ogg.tags ]; then
        echo "ogg.tags file exists.  Please remove it first."
        exit 1
fi

for FILENAME in "$@"; do

        echo "Processing "$FILENAME

        # Dump tags to file, removing ARTISTSORT tag
        vorbiscomment -l "$FILENAME" | grep -v "ARTISTSORT=" \

> ogg.tags

        # Write tags back
        vorbiscomment -w -c ogg.tags "$FILENAME"

done

rm -f ogg.tags

To run, simply "cd" into the folder containing the problematic Ogg/Vorbis files, then run "ogg-retag *.ogg" or similar.

Easier Partial "DCommitting"

By Tom on May 24, 2010 2:14 PM | No Comments

I've been making extensive use of Git-SVN recently, because I vastly prefer the user interface of Git to that of SVN, yet am collaborating on a project which uses SVN. My workflow consists of keeping a set of commits on top of the commits from SVN, which do things like add .gitignore files (hint: use git svn create-ignore to make these automatically), or contain code that I'm not ready to show other people just yet, or don't ever want to release.

This isn't the most advanced use of git-svn possible, but it's quite neat, simple and difficult to make mistakes with. When I described the method on IRC to someone who was asking about exactly this situation, it sounded like my method was of interest. I promised to do a short write-up, and here it is.

My workflow consists of three simple(ish) aliases. The first one is the simplest, for updating to the latest SVN trunk:

git stash && git svn rebase && git stash pop

This just stashes any local changes to the working tree, then rebases the unpublished, "floating" commits on top of the latest commits from SVN.

The second is used when I have one or more commits to add to the SVN repository. Having previously created a branch "for-commit" (it doesn't matter where it's based initially), I open "gitk" and make sure it's up to date. It probably looks something like this:

Then I invoke the second alias, which is:

git stash && git checkout for-commit && git reset --hard remotes/git-svn

This stashes local changes, checks out the "for-commit" branch and resets it to the most recent known head of the Subversion trunk. Then I refresh my gitk window, which then looks like this:

Nothing too exciting has happened, but we're now on the "for-commit" branch which points in a sensible place. I right-click* each commit that's to be committed to Subversion, and cherry-pick them onto the "for-commit" branch:

So, now the "for-commit" branch has diverged from "master" because the commits have been re-written by the cherry-pick operation. Now comes the clever(ish) part, which is the third alias:

git svn dcommit && git checkout master && git svn rebase && git stash pop

This commits the contents of the for-commit branch to SVN, then switches back to master and rebases it on top of the latest SVN branch, which now includes the commits just added. Git-SVN is clever enough to realise that some of the commits, but not all, are now in SVN. Gitk looks like this after a final update:

There are a couple of "sidings" in the history. The lower of the two is the old location of the "floating" commits on master. The upper one is the previous location of the for-commit branch. You could tidy them up by restarting gitk, but I find they're useful for keeping track of what I did. If you look closely, you'll notice that someone else added a commit to the Subversion branch between the last time I updated and the final commit, and git-svn dealt with that too.

So there you go. Maybe that's useful to someone.

* I don't actually "right-click", since I'm left-handed. I also use a trackball instead of a mouse. You should get one - they're awesome.

Drilling for Oil

By Tom on May 22, 2010 1:08 PM | No Comments

I find it somehow disturbing that we are able to drill a hole into our planet and make it "bleed", and in such a way that it's so hard to do anything about it.

In fact, come to think of it, the whole idea of drilling a hole that deep, under so much water, is pretty incredible. I've never really thought about it before.

Expat Financial Scams

By Tom on April 4, 2010 8:32 PM | No Comments

A somewhat eye-opening experience last week.

I had a phone call from a company offering "independent" financial advice regarding offshore savings and pensions - to "make the most" of my residence abroad. The company in question will not be mentioned on here, but there is plenty of information about their dodgy business methods and even their recruiting strategies available on the web. To cut a long story short: It's nothing more than a scam: their representatives are usually not qualified financial advisors, but simply salespeople who will subject you to high-pressure sales tactics in the hope of getting their (huge, up-front) commission which is taken from your contributions, and can be as much as your whole first year's worth of payments.

While the underlying products they may advertise may be basically sound, they tend to be very expensive and unsuitable for people in my kind of situation. The worst thing, however, is that they often use restrictive contract terms which lock you in to paying huge fees which negate any potential tax benefits.

It seems like there's quite a lot of this kind of thing going around. Needless to say, I know where I'll tell any other financial "advisors" who call me to go...

Most readers probably think they're far too clever to fall for such a thing, but I've now experienced first-hand how powerful the illusion is and how easy it can be to fall under the spell, and it's very scary. Beware.

As ever, if something sounds too good to be true, it probably is..!

OpenCL Calculation and Reduction

By Tom on February 25, 2010 9:35 PM | No Comments

Otherwise known as "calculating a long list of numbers then adding them all up". For a GPGPU (OpenCL) simulation program at work, I needed to calculate around 160 numbers which would be averaged to produce one result for storage in a 1024x1024 element array. That's 160 numbers for each of 1024x1024 pixels, which would be a lot to store as an intermediate result for a later step of averaging on the GPU, or (heaven forbid) to be copied back to system memory.

The magic word to search for in tackling this is reduction, and there's plenty of hardcore compsci knowledge about how to make it go as fast as possible in a parallel environment. But, basically the trick is to have 160x1024x1024 threads operate in groups of 160 (one group for each of the 1024x1024 overall elements). Threads cooperating like this can share memory, and each thread writes its individual value to an array in that local memory. Then, one of the 160 threads adds up all the values and does a single write of the final average value to the global array. For the kernel to test if it's running the "chosen thread" is as simple as something like this:

if ( get_local_id(0) == 0 )

The only bit of "funny business" is that each of the 160 threads has to have finished calculating before the results can be added. That's done with this statement, which guarantees that all previous local memory writes have completed for all threads:

barrier(CLK_LOCAL_MEM_FENCE);

This is a really simple example: for one thread to do all of the averaging is a waste of resources when the reduction itself could be parallelised. In that case, one thread would (say) add up values 0-79 while another added up 80-159, then one of those threads would (after another barrier) add up the remaining two values. It's easy to see how it can be broken down more and more, and there are variations which make better use of the GPU resources, avoid memory conflicts, and so on.

So, if you'd ever heard of the thread groups and local memory used in OpenCL (also CUDA) and wondered what they were good for, now you know..

NVidia's OpenCL Programming Guide has a lot of discussion of this topic, and there's loads more to be found around the web.

Forbidden Projects

Free and Open Source Software in Reciprocal Space