Forbidden Projects

Flame Challenge

2013-03-03T22:17:25Z

I've entered the Flame Challenge. The challenge, specifically, is to answer the following question, scientifically (i.e. not poetically), for an 11 year old, in 300 words or less or as a graphic or video entry. The entries will be judged by 4th-6th graders in schools in the USA. It was too much of a challenge for me to resist, and now that the deadline has passed I can reveal my answer:

What is time?

Imagine looking at every slice of a loaf of bread, one by one, starting with the crust at one end and progressing towards the other. First you'd see the crust, then you'd see all the slices from the middle of the bread. If the loaf had an air bubble inside, then you'd see a hole in some of the slices. The slice from right at the edge of the bubble would have a small hole, and the ones which were cut right through the middle of the bubble would have bigger holes. On your way through the slices, you'd see a small hole first, then a bigger one, then another smaller one, then no hole at all.

The passing of time from moment to moment is like moving from one slice to another. Many things stay the same, just like every slice of a loaf of bread looks roughly the same, but some things change --- like the sizes of the holes in the bread slices. Things that change quickly, like an explosion, look very different from one "slice" to the next. Things that aren't changing at all look exactly the same in every slice. Something that's moving, like a car, changes its position from one slice to another.

When you think of time this way, you see that the passage of time is quite similar to moving along in distance. This is actually how scientists often think about it, too. Have you heard of Einstein's Theory of Relativity? That's based on treating space and time as very similar things. The main difference is that we have no choice but to move forwards in time. You can choose to stop moving through the slices of bread and spend longer looking at a particular slice, but you can't stand still in time!

My approach was to make the link between the dimension of time and the dimensions of space by using a familiar household object. There's so much more that I could've added with a longer word limit, like how things can be measured in time and space by creating an object that changes its appearance regularly along the dimension of interest. A ruler is just a stick which looks slightly different as you move along its length (the numbers go up), and a clock is just an object which looks slightly different as you move through time (the hands go round). You can even extend the concept of a crystal to dimensions of time as well, making a space-time crystal. What I find most interesting of all is how our perception of time is linked to its physical reality: exactly why are we compelled to advance through time, while having apparent free will concerning our movements in other dimensions? Is it all in our heads? Perhaps somehow related to the second law of thermodynamics applied to the physical and chemical processes in our brains?

The winners will be announced at the World Science Festival in June this year. In the unlikely event that I win a trip to NYC at that time, I'll be flying in a highly sleep-deprived state from an experiment at the Linac Coherent Light Source.

Ten years of Sylpheed

2012-11-27T22:40:58Z

This came up on one of my mailing lists recently, and I realised that I've also been using Sylpheed as my email client for ten years. Since shortly after I started using Linux, in fact. It was the very first program I compiled and installed from source, and I remember being taken through the process of learning the difference between "libgtk+" and "libgtk+-dev" by a kind helper on IRC.

A whole decade later, I'm still using Sylpheed almost every single day. In all that time, text editors, web browsers, version control systems, desktop environments, IRC clients, IM clients, window managers and even types of input device have all come and gone from my computing environment. Sylpheed simply never gave me any reason to replace it. In ten years, I think I can remember it crashing a single digit number of times. It's always one of the first things I install on a new machine - and my current laptop is my third such new machine since then. That's not including two work machines, where it was also the first thing I installed.

Discounting things like the kernel, SSH, the X server and so on, the only other program that's achieved this is GKrellM, and that's only because the Sylpheed email prompted me to think about and remember it. In fact, not even the X server has survived these ten years: in 2002, it was XFree86.

An impressive feat, and a mark of true software quality. If anything I produce lasts ten years on even one person's screen, I'll be very happy indeed!

Which programs, if any, survived the last ten years on your desktop?

Reproducible random ordering from "sort -R"

2012-08-09T18:12:53Z

Random discovery for the day (actually yesterday, but never mind): "sort -R", which randomises the order of lines in its input, uses a crytographic random number generator seeded using numbers which include its PID and the time. It's the same random number generator as used by "shred": ISAAC.

So, if you need to get the same random order of lines each time, as I did for a test script, you need to provide your own source of "random" numbers. Any old binary file will do, for example:

$ sort -R --random-source=/bin/ls

Remote control repair

2012-03-02T20:40:49Z

I repaired my DVD player by replacing the IR receiver for the remote control. It hadn't worked since about 2005, but I'd been hesitating about actually sorting it out. Admittedly, for a lot of that time I was convinced it was the remote control itself that was defective, and didn't want to give my credit card details to one of the many dodgy-looking replacement remote control websites.

Anyway, in case anyone's interested, the model was a Panasonic SC-HT520 and the replacement part I used was a Vishay TSOP 1838 (example , but the photo shows the wrong thing) which uses a 950 nm wavelength and a 38 kHz carrier. The pin assignments are exactly the same as the original one from the player, so you just have do undo some screws (hint: take the grey panel off the front of the DVD tray before disconnecting the power), desolder the old one and solder the replacement. The tracks on the PCB are nice and large, so it's dead easy. Total cost: €1.55, and about half an hour (plus many hours of research and figuring out what would be suitable, but that doesn't count).

I remember reading somewhere that the receivers on certain Panasonic units often seem to go bad, so if anyone is interested and needs more guidance than the above, post a comment here and I'll put some photos up.

Project X

2011-10-04T18:31:02Z

I'm still here, and programming. Work has been taking up most of my "programming brain" recently, but the results will hopefully be released as free and open source software very soon.

For the last few months I've been working on a new non-work project as well, which I'm not going to say too much about just yet - let's call it Project X for now. It's a piece of major "itch scratching" for me, but I have a feeling that many other scientists who use Linux will love it. Perhaps many other people besides. Watch this space...

Command line weight loss

2011-07-06T20:25:06Z

$ sudo setweight --bmi=21

It'd be nice if it were that simple. But it almost is!

Scales of Truth is a command line implementation of The Hacker's Diet. The basic principle is to lose (or gain, I guess, theoretically at least) weight by taking an engineering approach to your body's energy requirements. There are quite a few implementations of this or similar things (e.g. The Hacker's Diet Online and Physics Diet), and an important feature is that the required day-to-day administration, i.e. typing in your weight, is not very time-consuming. Less than five minutes a day? Well opening a website, logging in, typing a value in and so on seems like a lot of work to me. With Scales of Truth, you simply click over to one of the many terminals you no doubt have open, and type:

$ sot XXX

Where "XXX" is your current scale reading (in your choice of unit). Scales of Truth does all the necessary calculations, backs up your readings using Git (because losing months of figures would really suck), and displays some interesting statistics:

Current mass estimate: XXX kg (instantaneous BMI XX.X kg/m^2)
7-day change: -.42 kg (-1757.2 kJ or -419.9 kcal per day)
30-day change: -2.77 kg (-2704.2 kJ or -646.3 kcal per day)

If you happen to miss a few readings, it interpolates the missing values automatically so your running average stays up to date. For the average reader of this site, the whole procedure probably takes slightly less than two seconds. That is, assuming you can resist taking a look at your progress using "sot --graph":

That's my own graph for the last few months, so I've taken the liberty of removing the actual numbers. Suffice to say that each tick on the mass axis corresponds to two kilograms, and I'm rather pleased with progress so far...

You can download Scales of Truth right here. Simply download the file, open it in a text editor, satisfy yourself that it's not going to do anything evil to your computer, then follow the instructions.

The T Word

2010-10-31T13:11:50Z

Refuse to be terrorised.

A criminal does not become a terrorist because of which country they came from, because they choose to attack aeroplanes, because they are religiously or politically motivated or because they use bombs. They become a terrorist because we respond with fear and define them as such. Well, we can choose not to do that.

Everyone should remember that the perpetrators of the recent cargo plane bombing, the Times Square bombing, the underpants bombing, the liquid bomb plots, the 2007 London and Glasgow attacks and countless others (at least in the "Western world") did not succeed.

Trackball Configuration

2010-09-13T22:01:05Z

I have a slightly strange configuration for my trackball. I have it set up left-handed, so the main button on the right is "left click", and vice-versa. There are two smaller buttons, one on each side, and I have the left-hand one as "middle click" and the right-hand one turning the ball into a "scroll wheel" when it's held ("wheel emulation" in X speak). To complicate matters, I have the trackpad on my laptop right-handed. In previous times, I'd configure this with some lines like these in the relevant section of xorg.conf to set up the buttons properly just for the trackball:

Option "Buttons" "9"
Option "EmulateWheel" "on"
Option "EmulateWheelButton" "9"
Option "ButtonMapping" "3 8 1 4 5 6 7 2 9"

That worked until the business of HAL for X input configuration came along. Then I put something like this in /etc/hal/fdi/policy/trackball.fdi:

   mouse
   5
   true
   9
   4 5
   3 2 1 4 5 6 7 8 9
   0.40
   0.60
   0.010

That worked, but never quite satisfactorily. In some versions, the trackball would only work properly if plugged in after I'd booted up and logged in. In later versions (notably Fedora 12 and 13) it would never work correctly at all. In Fedora 13, the HAL method was deprecated and replaced with a method via files under /etc/X11/xorg.conf.d/. I added a file called "01-trackball.conf" containing:

Section "InputClass"
Identifier     "trackball"
MatchProduct   "Logitech USB Trackball"
Option         "Buttons" "9"
Option         "EmulateWheel" "on"
Option         "EmulateWheelButton" "9"
Option         "ButtonMapping" "3 8 1 4 5 6 7 2 9"
EndSection

Section "InputClass"
Identifier     "trackpad"
MatchProduct   "SynPS/2 Synaptics TouchPad"
Option         "ButtonMapping" "1 2 3"
EndSection

These lines define the exact button configuration for both the trackball and the trackpad, and the combination of InputClass and MatchProduct means that the rule works even if the trackball isn't present when X starts.

But it still didn't quite work. It seemed that the settings always got overridden by Gnome's central mouse options, left or right handed but not my strange mixture. The main buttons (left and right click) on the trackball had to be the same as the trackpad because Gnome couldn't understand the idea of having the two configured differently.

With a little clear thought, it's probably already obvious what was going wrong, though it was infuriating for a long time. Gnome (specifically gnome-settings-daemon) does indeed override your nicely thought out X config. Fortunately, besides "left handed" and "right handed" modes, it also has an extra mode called "don't screw with my damn setup you piece of ****". It just takes a little more surgery to enable:

$ gconftool-2 -s /apps/gnome_settings_daemon/plugins/mouse/active --type bool false

With Gnome's mouse config stuff safely (and permanently) anaesthetised, you're clear to configure things your way again.

Thinkpad DVD Noise

2010-08-22T20:40:20Z

My handy household hint for the weekend:

One problem I've been experiencing with my new(ish) Thinkpad T500 is the large amount of noise made by the DVD drive. It's loud enough to be distracting when watching a film - not quite enough to drown it out, but enough to be clearly heard over even the louder parts until your mind starts filtering it out. The tray rattles against the casing, which makes it much worse. I found the noise could be reduced significantly by squashing one or two bits of folded tissue between then tray and the casing above, but that's a nasty solution.

Of course, there's a proper fix for this. Just go into the BIOS config (press the blue ThinkVantage button when the BIOS boot screen shows up, hold it down until the message goes away, then follow the instructions), and find the options for the CD drive, and set it to "Silent". Problem solved - the drive becomes almost inaudible, and it doesn't seem to have any adverse effect on the playback of DVDs.

iAudio X5 Tagging

2010-06-30T08:17:42Z

The Cowon iAudio X5 is, in many ways, the ideal music player for the discerning geek. It's Linux friendly, behaving as a normal USB mass storage device requiring no special software nor drivers; it plays Ogg/Vorbis and FLAC files; it has a line in connector; it has fantastic sound quality; it organises your files by folder, so you can have your files organised how you really want them. It also has a battery which lasts ages, particularly the X5L version. When my X5L was younger, a full charge would last for about two weeks of fairly constant use at work. Now it's several years old - I've actually forgotten how old exactly - and it still lasts for many days.

It does, however, have a particular weirdness when handling Ogg/Vorbis files. Its tag parser is extremely basic, and apparently just skips one character (the equals sign) after finding the letters "ARTIST" in the tags. But most recent ripping software adds an extra field, called ARTISTSORT. The X5 often finds that instead of ARTIST, and thinks the name of the artist is something like "ORT=Haza, Ofra" instead of "Ofra Haza".

So, I wrote a little script to remove the ARTISTSORT tags altogether. It's a cheap solution - it'd probably be better just to move that tag to the end (possibly the start, I didn't really test) of the tags field, but this field isn't used for sorting on the X5 anyway.

Here's the script ("ogg-retag"). It's really simple:

#!/bin/sh

if [ -e ogg.tags ]; then
        echo "ogg.tags file exists.  Please remove it first."
        exit 1
fi

for FILENAME in "$@"; do

        echo "Processing "$FILENAME

        # Dump tags to file, removing ARTISTSORT tag
        vorbiscomment -l "$FILENAME" | grep -v "ARTISTSORT=" \

> ogg.tags

        # Write tags back
        vorbiscomment -w -c ogg.tags "$FILENAME"

done

rm -f ogg.tags

To run, simply "cd" into the folder containing the problematic Ogg/Vorbis files, then run "ogg-retag *.ogg" or similar.

Easier Partial "DCommitting"

2010-05-24T13:14:39Z

I've been making extensive use of Git-SVN recently, because I vastly prefer the user interface of Git to that of SVN, yet am collaborating on a project which uses SVN. My workflow consists of keeping a set of commits on top of the commits from SVN, which do things like add .gitignore files (hint: use git svn create-ignore to make these automatically), or contain code that I'm not ready to show other people just yet, or don't ever want to release.

This isn't the most advanced use of git-svn possible, but it's quite neat, simple and difficult to make mistakes with. When I described the method on IRC to someone who was asking about exactly this situation, it sounded like my method was of interest. I promised to do a short write-up, and here it is.

My workflow consists of three simple(ish) aliases. The first one is the simplest, for updating to the latest SVN trunk:

git stash && git svn rebase && git stash pop

This just stashes any local changes to the working tree, then rebases the unpublished, "floating" commits on top of the latest commits from SVN.

The second is used when I have one or more commits to add to the SVN repository. Having previously created a branch "for-commit" (it doesn't matter where it's based initially), I open "gitk" and make sure it's up to date. It probably looks something like this:

Then I invoke the second alias, which is:

git stash && git checkout for-commit && git reset --hard remotes/git-svn

This stashes local changes, checks out the "for-commit" branch and resets it to the most recent known head of the Subversion trunk. Then I refresh my gitk window, which then looks like this:

Nothing too exciting has happened, but we're now on the "for-commit" branch which points in a sensible place. I right-click* each commit that's to be committed to Subversion, and cherry-pick them onto the "for-commit" branch:

So, now the "for-commit" branch has diverged from "master" because the commits have been re-written by the cherry-pick operation. Now comes the clever(ish) part, which is the third alias:

git svn dcommit && git checkout master && git svn rebase && git stash pop

This commits the contents of the for-commit branch to SVN, then switches back to master and rebases it on top of the latest SVN branch, which now includes the commits just added. Git-SVN is clever enough to realise that some of the commits, but not all, are now in SVN. Gitk looks like this after a final update:

There are a couple of "sidings" in the history. The lower of the two is the old location of the "floating" commits on master. The upper one is the previous location of the for-commit branch. You could tidy them up by restarting gitk, but I find they're useful for keeping track of what I did. If you look closely, you'll notice that someone else added a commit to the Subversion branch between the last time I updated and the final commit, and git-svn dealt with that too.

So there you go. Maybe that's useful to someone.

* I don't actually "right-click", since I'm left-handed. I also use a trackball instead of a mouse. You should get one - they're awesome.

Drilling for Oil

2010-05-22T12:08:19Z

I find it somehow disturbing that we are able to drill a hole into our planet and make it "bleed", and in such a way that it's so hard to do anything about it.

In fact, come to think of it, the whole idea of drilling a hole that deep, under so much water, is pretty incredible. I've never really thought about it before.

Expat Financial Scams

2010-04-04T19:32:01Z

A somewhat eye-opening experience last week.

I had a phone call from a company offering "independent" financial advice regarding offshore savings and pensions - to "make the most" of my residence abroad. The company in question will not be mentioned on here, but there is plenty of information about their dodgy business methods and even their recruiting strategies available on the web. To cut a long story short: It's nothing more than a scam: their representatives are usually not qualified financial advisors, but simply salespeople who will subject you to high-pressure sales tactics in the hope of getting their (huge, up-front) commission which is taken from your contributions, and can be as much as your whole first year's worth of payments.

While the underlying products they may advertise may be basically sound, they tend to be very expensive and unsuitable for people in my kind of situation. The worst thing, however, is that they often use restrictive contract terms which lock you in to paying huge fees which negate any potential tax benefits.

It seems like there's quite a lot of this kind of thing going around. Needless to say, I know where I'll tell any other financial "advisors" who call me to go...

Most readers probably think they're far too clever to fall for such a thing, but I've now experienced first-hand how powerful the illusion is and how easy it can be to fall under the spell, and it's very scary. Beware.

As ever, if something sounds too good to be true, it probably is..!

OpenCL Calculation and Reduction

2010-02-25T21:35:08Z

Otherwise known as "calculating a long list of numbers then adding them all up". For a GPGPU (OpenCL) simulation program at work, I needed to calculate around 160 numbers which would be averaged to produce one result for storage in a 1024x1024 element array. That's 160 numbers for each of 1024x1024 pixels, which would be a lot to store as an intermediate result for a later step of averaging on the GPU, or (heaven forbid) to be copied back to system memory.

The magic word to search for in tackling this is reduction, and there's plenty of hardcore compsci knowledge about how to make it go as fast as possible in a parallel environment. But, basically the trick is to have 160x1024x1024 threads operate in groups of 160 (one group for each of the 1024x1024 overall elements). Threads cooperating like this can share memory, and each thread writes its individual value to an array in that local memory. Then, one of the 160 threads adds up all the values and does a single write of the final average value to the global array. For the kernel to test if it's running the "chosen thread" is as simple as something like this:

if ( get_local_id(0) == 0 )

The only bit of "funny business" is that each of the 160 threads has to have finished calculating before the results can be added. That's done with this statement, which guarantees that all previous local memory writes have completed for all threads:

barrier(CLK_LOCAL_MEM_FENCE);

This is a really simple example: for one thread to do all of the averaging is a waste of resources when the reduction itself could be parallelised. In that case, one thread would (say) add up values 0-79 while another added up 80-159, then one of those threads would (after another barrier) add up the remaining two values. It's easy to see how it can be broken down more and more, and there are variations which make better use of the GPU resources, avoid memory conflicts, and so on.

So, if you'd ever heard of the thread groups and local memory used in OpenCL (also CUDA) and wondered what they were good for, now you know..

NVidia's OpenCL Programming Guide has a lot of discussion of this topic, and there's loads more to be found around the web.

OpenStreetMap's Role in Haiti

2010-02-25T20:12:49Z

This is very cool. Volunteer contributors used satellite imagery, which was made available especially, to create detailed maps of Haiti which the aid workers used to help find their way.