WizBlog

WizBlog tag:www.bitwiz.org.uk,2009-01-26:/s//1 2010-06-30T09:15:40Z Forbidden Projects Movable Type 4.23-en iAudio X5 Tagging tag:www.bitwiz.org.uk,2010:/s//1.77 2010-06-30T08:17:42Z 2010-06-30T09:15:40Z

Tom Cowon iAudio X5 is, in many ways, the ideal music player for the discerning geek. It's Linux friendly, behaving as a normal USB mass storage device requiring no special software nor drivers; it plays Ogg/Vorbis and FLAC files; it has a line in connector; it has fantastic sound quality; it organises your files by folder, so you can have your files organised how you really want them. It also has a battery which lasts ages, particularly the X5L version. When my X5L was younger, a full charge would last for about two weeks of fairly constant use at work. Now it's several years old - I've actually forgotten how old exactly - and it still lasts for many days.

It does, however, have a particular weirdness when handling Ogg/Vorbis files. Its tag parser is extremely basic, and apparently just skips one character (the equals sign) after finding the letters "ARTIST" in the tags. But most recent ripping software adds an extra field, called ARTISTSORT. The X5 often finds that instead of ARTIST, and thinks the name of the artist is something like "ORT=Haza, Ofra" instead of "Ofra Haza".

So, I wrote a little script to remove the ARTISTSORT tags altogether. It's a cheap solution - it'd probably be better just to move that tag to the end (possibly the start, I didn't really test) of the tags field, but this field isn't used for sorting on the X5 anyway.

Here's the script ("ogg-retag"). It's really simple:

#!/bin/sh

if [ -e ogg.tags ]; then
        echo "ogg.tags file exists.  Please remove it first."
        exit 1
fi

for FILENAME in "$@"; do

        echo "Processing "$FILENAME

        # Dump tags to file, removing ARTISTSORT tag
        vorbiscomment -l "$FILENAME" | grep -v "ARTISTSORT=" \

> ogg.tags

        # Write tags back
        vorbiscomment -w -c ogg.tags "$FILENAME"

done

rm -f ogg.tags

To run, simply "cd" into the folder containing the problematic Ogg/Vorbis files, then run "ogg-retag *.ogg" or similar.
]]> Easier Partial "DCommitting" tag:www.bitwiz.org.uk,2010:/s//1.76 2010-05-24T13:14:39Z 2010-05-30T10:42:08Z

Tom

This isn't the most advanced use of git-svn possible, but it's quite neat, simple and difficult to make mistakes with. When I described the method on IRC to someone who was asking about exactly this situation, it sounded like my method was of interest. I promised to do a short write-up, and here it is.

My workflow consists of three simple(ish) aliases. The first one is the simplest, for updating to the latest SVN trunk:

git stash && git svn rebase && git stash pop

This just stashes any local changes to the working tree, then rebases the unpublished, "floating" commits on top of the latest commits from SVN.

The second is used when I have one or more commits to add to the SVN repository. Having previously created a branch "for-commit" (it doesn't matter where it's based initially), I open "gitk" and make sure it's up to date. It probably looks something like this:

Then I invoke the second alias, which is:

git stash && git checkout for-commit && git reset --hard remotes/git-svn

This stashes local changes, checks out the "for-commit" branch and resets it to the most recent known head of the Subversion trunk. Then I refresh my gitk window, which then looks like this:

Nothing too exciting has happened, but we're now on the "for-commit" branch which points in a sensible place. I right-click* each commit that's to be committed to Subversion, and cherry-pick them onto the "for-commit" branch:

So, now the "for-commit" branch has diverged from "master" because the commits have been re-written by the cherry-pick operation. Now comes the clever(ish) part, which is the third alias:

git svn dcommit && git checkout master && git svn rebase && git stash pop

This commits the contents of the for-commit branch to SVN, then switches back to master and rebases it on top of the latest SVN branch, which now includes the commits just added. Git-SVN is clever enough to realise that some of the commits, but not all, are now in SVN. Gitk looks like this after a final update:

There are a couple of "sidings" in the history. The lower of the two is the old location of the "floating" commits on master. The upper one is the previous location of the for-commit branch. You could tidy them up by restarting gitk, but I find they're useful for keeping track of what I did. If you look closely, you'll notice that someone else added a commit to the Subversion branch between the last time I updated and the final commit, and git-svn dealt with that too.

So there you go. Maybe that's useful to someone.

* I don't actually "right-click", since I'm left-handed. I also use a trackball instead of a mouse. You should get one - they're awesome.

]]> Drilling for Oil tag:www.bitwiz.org.uk,2010:/s//1.75 2010-05-22T12:08:19Z 2010-05-22T13:29:44Z

Tom we are able to drill a hole into our planet and make it "bleed", and in such a way that it's so hard to do anything about it.

In fact, come to think of it, the whole idea of drilling a hole that deep, under so much water, is pretty incredible. I've never really thought about it before.

]]> Expat Financial Scams tag:www.bitwiz.org.uk,2010:/s//1.74 2010-04-04T19:32:01Z 2010-04-04T20:48:18Z

Tom

I had a phone call from a company offering "independent" financial advice regarding offshore savings and pensions - to "make the most" of my residence abroad. The company in question will not be mentioned on here, but there is plenty of information about their dodgy business methods and even their recruiting strategies available on the web. To cut a long story short: It's nothing more than a scam: their representatives are usually not qualified financial advisors, but simply salespeople who will subject you to high-pressure sales tactics in the hope of getting their (huge, up-front) commission which is taken from your contributions, and can be as much as your whole first year's worth of payments.

While the underlying products they may advertise may be basically sound, they tend to be very expensive and unsuitable for people in my kind of situation. The worst thing, however, is that they often use restrictive contract terms which lock you in to paying huge fees which negate any potential tax benefits.

It seems like there's quite a lot of this kind of thing going around. Needless to say, I know where I'll tell any other financial "advisors" who call me to go...

Most readers probably think they're far too clever to fall for such a thing, but I've now experienced first-hand how powerful the illusion is and how easy it can be to fall under the spell, and it's very scary. Beware.

As ever, if something sounds too good to be true, it probably is..!

]]> OpenCL Calculation and Reduction tag:www.bitwiz.org.uk,2010:/s//1.73 2010-02-25T21:35:08Z 2010-02-25T22:13:25Z

Tom

The magic word to search for in tackling this is reduction, and there's plenty of hardcore compsci knowledge about how to make it go as fast as possible in a parallel environment. But, basically the trick is to have 160x1024x1024 threads operate in groups of 160 (one group for each of the 1024x1024 overall elements). Threads cooperating like this can share memory, and each thread writes its individual value to an array in that local memory. Then, one of the 160 threads adds up all the values and does a single write of the final average value to the global array. For the kernel to test if it's running the "chosen thread" is as simple as something like this:

if ( get_local_id(0) == 0 )

The only bit of "funny business" is that each of the 160 threads has to have finished calculating before the results can be added. That's done with this statement, which guarantees that all previous local memory writes have completed for all threads:

barrier(CLK_LOCAL_MEM_FENCE);

This is a really simple example: for one thread to do all of the averaging is a waste of resources when the reduction itself could be parallelised. In that case, one thread would (say) add up values 0-79 while another added up 80-159, then one of those threads would (after another barrier) add up the remaining two values. It's easy to see how it can be broken down more and more, and there are variations which make better use of the GPU resources, avoid memory conflicts, and so on.

So, if you'd ever heard of the thread groups and local memory used in OpenCL (also CUDA) and wondered what they were good for, now you know..

NVidia's OpenCL Programming Guide has a lot of discussion of this topic, and there's loads more to be found around the web.

]]> OpenStreetMap's Role in Haiti tag:www.bitwiz.org.uk,2010:/s//1.72 2010-02-25T20:12:49Z 2010-02-25T20:25:20Z

Tom This is very cool. Volunteer contributors used satellite imagery, which was made available especially, to create detailed maps of Haiti which the aid workers used to help find their way.]]> The Mysterious Missing Milliamps tag:www.bitwiz.org.uk,2010:/s//1.71 2010-02-10T19:28:06Z 2010-02-10T21:36:57Z

Tom

My new A7+ Freerunner arrived a few days ago. Before long I'd flashed it with the latest version of SHR-testing, made sure stuff worked, and moved my main SIM over to it. Then I set about installing all the most cutting-edge unstable userspace and 2.6.32 kernel stuff on the old A6 Freerunner ready to have some fun. But I quickly noticed that something wasn't right - it wouldn't charge properly. During my first Sunday hacking session, it didn't seem to be able to charge from a USB connection to my computer, and seemed to be getting quite warm to the touch as well.

The current_now sysfs node indicated that the device was using many hundreds of milliamps more current than it should have been. So much, in fact, that the current provided to it over USB by my laptop wasn't enough to charge the battery - with barely enough power to keep the thing running the battery slowly discharged. With the higher current provided by the mains charger, the battery would charge, but more slowly that normal. But this only seemed to happen with the 2.6.32 kernel.

Lots of discussion on IRC (mostly with DocScrutinizer) followed, and it emerged that one possibility was a short circuit in the uSD card slot. Sure enough: I'd moved my uSD card from the A6 to the A7+ Freerunner (since the new one didn't come with a card), leaving the A6's SD slot empty. On my device, some of the pins protruded up, perhaps enough to short out against the metal SD slot lid:

I cut a roughly 1x1.5cm rectangular piece of plastic from the packaging of some small halogen bulbs I'd bought the previous day, and put it into the slot as a "dummy". Testing again: no current drain. Dummy removed, the drain came back, and so on. The Doc had hit the nail on the head - the culprit was found!

But there was one more piece to the puzzle. Why didn't the drain seem to happen with the older 2.6.29 kernel? Turns out that a small detail had been left out from the kernel during an earlier merge. Because of this, the Glamo SD engine's power supply was left on all the time, when normally it should get switched off when no SD card was present. The resulting waste of a small amount of power became a huge power drain when the short circuit was added. Hopefully the SD electronics aren't too fried..

]]> Gitolite tag:www.bitwiz.org.uk,2010:/s//1.70 2010-01-26T23:13:18Z 2010-01-27T07:18:49Z

Tom Git repositories to use Gitolite, something I've been meaning to do for a long time. This gives me much better control over access permissions, potentially letting me give other people access to my repository without handing over shells and/or real user accounts.

I've done some testing, but things might have broken. If they have, address your complaints to the usual address..

]]> New Toys tag:www.bitwiz.org.uk,2010:/s//1.69 2010-01-25T18:18:06Z 2010-01-25T18:47:47Z

Tom

I've also ordered myself a second Freerunner. This was a bit less of a budgeted expense, but I have no real shortage of money at the moment (thanks to not drinking regularly). I realised that the reason for my recent lack of productivity wasn't time as such, rather the faff involved with switching into a very unstable environment for development then having to go back to a usable setup at the end of a "session". Developing in "sessions" like this seems to be a nice way to avoid getting anything done at all - I had the same problem at the start of my DRM work when we changed to Linux 2.6.29 from 2.6.24. It's much better to be able to work in a semi-continuous stream as time allows.

There's another reason for this purchase though. I'm affected strongly by the infamous Freerunner buzz problem in Germany, whereas I didn't notice it back in the UK. I was going to send my FR in to get both the buzz and #1024 (standby time) fixes done, but I've decided instead just to buy a new one with both fixes already. Then I'll use the new one day-to-day while my current one becomes a development platform, installed with all the latest and most unstable software I can find, so that I can stomp on the nastiest bugs with some degree of comfort.

And there's one more new toy: A 32TB RAID6 array with 4 optical fibre channel connections for storing and analysing our data on at work. All my analyses just went from being I/O limited to being firmly CPU bound..

]]> Code Offsetting tag:www.bitwiz.org.uk,2010:/s//1.68 2010-01-23T09:56:09Z 2010-01-23T10:15:10Z

Tom I just came across The Alliance for Code Excellence, where you can offset bad code you've written in the past with donations which support open-source projects which "decrease the propagation of bad code".]]> New Year tag:www.bitwiz.org.uk,2010:/s//1.66 2010-01-08T21:05:16Z 2010-01-08T21:24:33Z

Tom and computers!), the completion of my PhD thesis and passing of the subsequent examination before being whisked off my feet by the beginning of an international adventure. I suspect the adventure will be ongoing for the rest of my life. When I moved out of my last long-term accommodation in Cambridge back in May, I knew that things would never be the same again.

I won't go into more detail than that, because his journal has become more about technical things and "productive" aspects of my life rather than personal wibbling. Along which lines, I hope to soon be able (/allowed) to post details of the exciting science I've been doing...

]]> New Laptop Time tag:www.bitwiz.org.uk,2009:/s//1.65 2009-12-22T22:53:29Z 2009-12-22T23:19:20Z

Tom
Of course I'm backed up to the hilt with distributed version control, so I'm not in immediate danger of losing anything particularly important. However, it's apparent that I'll need to buy a new laptop in the near future. At the moment I'm looking at a Lenovo Thinkpad T500 with WSXGA+ (1680x1050) screen and Radeon graphics, but does anyone have any other suggestions? My non-negotiable requirements are:

Linux-friendly wifi and graphics.
Dual core, or at least HT. This really does make a huge difference.
Widescreen. 1680x1050 with a 15.4" screen gives a resolution I like.
UK keyboard layout (i.e. UK market, ideally with delivery to Germany possible).
DVD drive.

Ideally it would also have:

Decent battery life, or the possibility to buy spare or larger batteries during the next few years once the original one becomes a plastic box of jelly.
VGA output.
Fast-ish hard drive (7200rpm or higher. I'm not sure how much of a difference this makes, but I do a lot of compiling and so on. No need to go overboard with solid-state disks for hundreds of extra pounds.

I'm not too bothered about:

Bluetooth (I don't use it at the moment).
Huge hard drive - I get on fine with only about 80Gb at the moment.

Any suggestions on a comment or email to this address..
]]> Trip to Stanford tag:www.bitwiz.org.uk,2009:/s//1.64 2009-12-20T11:11:21Z 2009-12-20T11:43:28Z

Tom Stanford Linear Accelerator Cente. We used the LCLS to zap various kinds of object and measure the resulting signals. Without going into too many gory details, I'm happy to be able to say that the experiment was a great success and we got several terabytes of useful data, which we'll be analysing for many months to come.

I hardly had any time to escape the campus or to explore, but I did manage to find some sites of geeky interest. Here's a photo of the research stations at the end of the two mile long linear accelerator, which is hidden by the trees at the back of the picture. The large concrete building on the right is End Station A, where the first experimental evidence for the existence of quarks was recorded around 1966. Their experiment was like a much larger version of Rutherford's scattering experiment with alpha particles and gold foil. Today, End Station A contains test experiments to prepare for the International Linear Collider. I wasn't able to go inside - it's probably possible, subject to the particle accelerator's beam being directed elsewhere and talking to the right people, but there wasn't time and I wasn't keen to push the limits of my security pass (I also couldn't find the door..).

The similar concrete monstrosity on the left is End Station B, which probably contains similarly cool things (remember what I wrote about concrete?). The long building which comes out between and runs off the picture to the left is the LCLS beam transport hall. Standing where I took this picture, below me and to the left would be the undulator hall, which contains magnets which cause the electron beam to emit X-ray pulses of unimaginable brightness and brevity. The tunnel emerges on the other side of the hill I'm standing on, where the electrons and X-rays move into the building where we worked. The electron beam is dumped (basically by directing it into the ground) and the X-rays make their way into our experiment.
]]> Comments Enabled tag:www.bitwiz.org.uk,2009:/s//1.63 2009-11-20T21:51:29Z 2009-11-20T21:53:01Z

Tom
]]> Internal Memory Bottlenecks and Their Removal tag:www.bitwiz.org.uk,2009:/s//1.62 2009-11-15T16:53:12Z 2009-11-15T16:59:23Z

Tom
At "8 stages", the price of this was that the display became jittery and unstable. However, the same speed improvement is seen at the "4 stages" setting. I've also seen some occasional artifacts with this setting, so I'm using 2 stages at the moment, where the speed is still right up there. I'll be testing some more and seeing if things can be tuned even more.

Because we don't make the maximum use possible of the 2D engine, this doesn't immediately translate into a huge increase in the UI speed. But the differences are very obvious with x11perf and some of my test programs. The program I showed in the screenshot recently jumped from 45-48fps right up to 95-98fps!
]]>