October 1st, 2006

Spoons for all

Yes, I am still around. In the spirit of my previous post, I've decided to write a little (?) about another of my projects: Spoon. This will hopefully allow me to get my thoughts straight on what it is and where it's going. Also, dear reader, if you would like to comment, please do.


Or: What is that thing? I'm scared

First, what is Spoon? Spoon is primarily an experiment at this stage. It isn't ready for use. It probably won't be ready for use for a while yet.

Spoon is a replacement for scrollkeeper. When you install a package in GNOME (assuming the maintainer's been a good boy / girl), you get a manual with it. This manual is / should be read though Yelp. In order to do that, Yelp has to know where to find the manual. This is where scrollkeeper comes in. At install time, it takes in some information about the manual, adds it to its "database" and when Yelp asks, it passes a list of documents it knows about. Yelp uses this to build its table 'o content and find any files it needs.

Scrollkeeper is a good idea. In an ideal world, KDE, GNOME, XFCE and everyone else would register their documents with scrollkeeper and each help browser would be able to access any document on the system. Unfortunately, scrollkeeper has some, erm, interesting features:

It's unmaintained

JHBuild currently maintains 3 patches against scrollkeeper to stop it crashing whenever its run (!). Not esoteric "only happens when I throw 200Gb at it in an argument", but "I dared to invoke it as its supposed to be invoked" style crashes.

It's main access is though a temporary file

No library. Yelp creates and executes a command through a shell and captures the output. This output is a temporary filename. Yelp then opens that and reads the contents to get access to the metadata

The metadata file is a pain to use for developers, doc writers, translators and everyone else

It's an XML file. It has some strange fields in it. It's strange. So strange, gnome-doc-utils has facilities for generating these files, so people don't need to see them.

It can't handle unknown translations

I want to read the Beanstalk manual in Klingon. I know the Beanstalk package correctly installed the Klingon documentation. I fire up Yelp in the Klingon language, open the Beanstalk manual. It's in English. WTF? Scrollkeeper can't handle Klingon as its not been translated into Klingon itself. But the best part: Scrollkeeper won't complain. Not during installation. Not during running. Never. It'll just silently return the English language version.

Registration is a royal pain

Installation should be "copy file to this location. Done.". With scrollkeeper, it isn't. The metadata has to be registered with scrollkeeper. That's a pain, for everyone.

There are many other problems with scrollkeeper unfortunately, but these are the main ones that Spoon is looking to address. The other problems will (hopefully) be solved in addition to these, but for now...

The Present

Or: How I learned to love the future

A few months ago, I made a proposal to create a scrollkeeper replacement. At the time, I chose the stupid name "Spoon", which has stuck. The idea of Spoon is to provide a new scrollkeeper while maintaining an easy upgrade path. This proves quite difficult. A few discussions with the Fearless Leader (shaunm) revealed that this route had been attempted before, but no-one could agree on anything. From this, I decided the best way forward would be to create some example code. If there's code available, there is something concrete to argue over, instead of just ideas. Plus, I have a scar deep within me where I last dealt with scrollkeeper's eccentricities. So, Spoon was created.

Currently, it is celebrating it's 0.3'rd birthday. It's different from scrollkeeper in a number of ways:

  • It has library access through libspoon

  • It's metadata system is (IMO) nicer to deal with

  • It doesn't require translation. If a document is available in Klingon, it'll be used. No translating of Spoon required

  • No registration process. Drop the file in one of the searched directories and it'll work

Well, that's the major user-visible changes. It also provides a nice, easy upgrade method from scrollkeeper. At install time, it reads and parses all the scrollkeeper files on the system and converts them to the new Spoon scrolls. Currently, it works either in uninstalled mode (default) or can be installed (with a configure switch and big warning). The Spoon scroll system is very similar to .desktop files. An example would look like (Really hope this works now):

# An example spoon scroll file

[Spoon Scroll]
name=The Beanstalk Manual
name[de]=Das Beanstalk Manual
name[es]=Beanstalks R us
# A comment just to break the flow
description=An example scroll. Beanstalk is the arch enemy of Jack. Don't buy
magic beans
description[de]=Eine Beispielrolle. Bohnenstange ist der Bogenfeind von Jack.
Magische Bohnen nicht kaufen
description[es]=Una voluta del ejemplo. El Beanstalk es el enemigo del arco de
Gato. No comprar habas mágicas

The actual format is described on the wiki page, if you're interested (along with most of this discussion). The idea being that translations can be done in the same style as .desktop files. No messing around with XML omf files.

The Future

Or: D'ya hear that sonny? That's the sound of progress

Well, that's the history of Spoon up until now. Here's where I start talking about what should happen and what will happen.

Currently, the Spoon code is pretty volatile. Look at it the wrong way and it may well mutilate any cattle you own. The code needs cleaned up lots. There are a lot of "Temporary bodge to make it work. I'll fix it later" type code in it.

Currently, it'll only check the first language in the environment before falling back to C locale. If you have "LANG=de;fr;es;en_GB", it'll only look for the German manual before handing out the C version. That really needs fixed. In addition, the abomination that is it's config system needs destroyed. Instead, the XDG data dirs should be used. That should remove quite a lot of the ugly code mentioned above. (Thanks vuntz for pointing this out).

Then, there is another util that needs created - scrollkeeper-update. This'll allow people to install older packages and still allow them to work with Spoon. When installing a scrollkeeper omf, you first copy the file into the directory (/usr/share/omf), then call 'scrollkeeper-update'. The brilliantly named 'scrollkeeper-install' utility can't be used like you expect. No, I don't know why not. So, Spoon needs a replacement for either scrollkeeper-update or scrollkeeper-install (which is used internally by scrollkeeper-update to add the new file to its database). This is the tough part.

The scroll file needs at least one other field: Heritage. This would hold the scrollkeeper seriesid when it was replaced with a swanky new reverse DNS style identifier in the "series" entry, allowing newer versions to be correctly identified.

Once these things have been done, Spoon would be (relatively) ready to think about getting used. The library currently only provides 3 functions - init, shutdown and for_each. The for_each calls a (user-supplied) function for each scroll it finds (duh). This would be enough to attempt to remove scrollkeeper from Yelp and replacing it with Spoon (in the process, tearing out a heap of code). For real use though, the library would need lots more functions added to it.

Omega Point

Or: What's the point of this post?

First, the point is to let people know about Spoon. If you're interested. If not, probably better to have skipped this post. I tell you this now. Second, to get my head straight on what needs done in Spoon. I like to write things down to keep track of them. Third, to ask for feedback. Please? Pretty please? Mainly, on the Spoon scroll format. Something I've missed? Something you don't like? Change the field names? Features that'd be useful to include as a util / within the library (I have a few of these already, but more is always good)? Fourth, it's Sunday. I've been up since 6am (midday now. Stupid Grand Prix's) and feel the need to do something semi-constructive with the day.
  • Current Music
    Guns N' Roses - November Rain