Saturday, May 13, 2006

UNIX versus iTunes and the iPod, via MacOSX

Our household has been, um, blessed with an iPod for some months now. Currently it is being managed by iTunes from a Mac. However, our music collection lives on our main machine that runs Linux, and that has caused some problems.

The standard procedure is: rsync the music directory from the server to the Mac (over the wireless LAN, ouch!) and then import the directory into iTunes, and then maintain the iPod from there. This is tedious but has until recently worked: the rsync is slow over the wi-fi, importing into iTunes seems to involved having a complete copy of all the imported music, and then you have to plug in the iPod and sync.

Then we stepped out of our cosy English niche and ripped an African CD, with French song titles.

It ripped just fine, and we have these nicely named files in the main music tree. So we blithely went to rsync the files to the Mac. No go. Rsync's attempts to create the files on the Mac failed. It turns out that the Mac's HFS Plus filesystem uses Unicode UTF-8 Normal Form D encoding for its filenames and prevents creation of files whose names' byte sequences are not in this form.

UNIX filenames are simply byte sequences. There is no specified encoding. Since we were creating filenames from the FreeDB index, I expect these names are in Latin-1 encoding. The FreeDB Database Format Specification states that ``Database entries must be in the US-ASCII, ISO-8859-1 (the 8-bit ASCII extension also known as "Latin alphabet #1" or ISO-Latin-1) or UTF-8 (Unicode) character set'' but unhelpfully provides no indication as to how one should identify the encoding in use.

Our current tree contains Latin-1 encoded filenames, so we will be sticking with that for now. What is then needed is a way to make a tree we can rsync to the Mac. So the plan is to keep a hard-linked directory tree beside the "master" music directory which contains UTF-8 NFD encoded names. To this end I have made a small tool called macify which will maintain a link tree.

No comments: