Site: Home | LnBlog | The Joy of ROX | LinLog
The Joy of ROX: Main | News | Software | Wrappers and Resources | Contact
Copyright (c) 2003 Peter A. Geer
Making and distributing verbatim copies of this paper is permited, provided that the copyright notice and this notice are maintained.
Revisions:
16-Jan-2004: Corrected various typographical and gramatical
errors.
This document is a short overview of my experience building application directories for non-ROX applications. So far, I've repackaged about fifty or sixty programs. They are mostly smaller applications, i.e. nothing as large as the GIMP or GNOME. Most of them use automake and autoconf, are writtin in C/C++ and/or assembly language, and often include man pages as primary documentation. Some of them, however, are binary-only or have some unusual quirks.
I will also cover some aspects of making application wrappers for ROX, as some of the same principles will apply. Of course, there is no magic formula for making AppDirs; everything depends on the program you're repackaging. However, I'll try to cover general points rather than just telling you what I did with every package. Any feedback or suggestions are welcome.
Please note that the information presented here is based on my experience with the ROX 1.3.x development series and current 2.0.x stable series for GTK+ 2.x. There have been significant changes since the 1.2.x stable series for GTK+ 1.2, so some of what I describe may not apply to it. The basics, though, should hold for both versions.
Before we get into the gorey details, I think it's important to stop for a minute and consider what our goals are. In any pursuit, if you forget where you're going, you probably won't get there. Our main goal is, obviously, to repackage conventional applications as application directories, but that's not really all there is to it. What we really want is for ROX users to download our AppDir packages rather than installing from conventional packages like RPMs or source tarballs.
As a consequence of this, we need to keep up with the current release - nobody wants to download out of date packages. We also need to
Another goal is to leverage the power of ROX to make our AppDirs smarter and easier to use than conventional packages. This includes the obvious things, like on-demand automatic compiling and making the AppDirs relocatable, but is not limited to them. We can provide extras like custom AppMenus and intelligent handling of different types of commandline arguments to make things easier on the user. Always remember that it's the little things that make us happy on a day to day basis.
By way of summary, here is a short list of what should be, in my opinion, the priorities of an AppDir packager.
If you're reading this, you're probably aware of what an AppDir requires, but let's cover it again, just for completeness.
An AppDir is simply a directory with three files in it: an executable called AppRun, an XML file called AppInfo.xml, and an image file called .DirIcon (older versions use an XPM file named AppIcon.xpm, which is deprecated, but still supported). If you want to run your AppDir as a panel applet, then you also need an AppletRun executable, which can simply be a symlink to AppRun. Everything else is optional.
Building an AppDir is not hard. The .DirIcon file, which is the icon displayed for the AppDir, is the easiest. It can be basically any image file in any format, although a PNG file in the range of 32x32 to 64x64 pixels in size is preferable. (Actually, it can't really be in any format. The format has to be supported by GDK-Pixbuf, but I can't think of any common formats that aren't.) You can just use ROX's "Set Icon" menu item to copy it into the AppDir or do it by hand if you prefer.
The AppInfo.xml file is also not a problem, as you can just copy the one used by ROX-Session or ROX-Filer, change the appropriate fields, and delete the rest. This file holds information about the AppDir, such as a description, version number, the author, the project homepage, etc. It also contains an <AppMenu> section, where you can define custom menu items for the AppDir's right-click context menu in ROX-Filer, which we'll cover later.
Last, and most inportant, you must have an AppRun, which is the program that's run when the user clicks on the AppDir in a filer window. When it comes to repackaging existing programs, which are probably written in C, the AppRun file is almost always a shell script that sets up the environment, compiles the executable if it is not present, and then launches the program. However, the only requirement is that there is a file called AppRun and it is executable. If you really wanted to, you could simply rename your main executable "AppRun" and be all set. While this technique does work well for Perl or Python scripts, it isn't really a good idea for compiled packages.
By convention, there are a few other things that go into an AppDir. First, there is the Help directory. This is just a normal directory where you put all the program documentation. There are also one or more directories to hold the compiled binaries for the program. These are named things like Linux-ix86 or Linux-alpha and hold the binaries for the associated platform.
Lastly, there is a convention for the functionality of the AppRun script. You can see this in the AppRun for ROX-Session or ROX-Filer. This functionality includes checking for the presence of a binary for the current platform, compiling it if one is not found (compilation takes place in an xterm when in a graphical environment), and then running the program. In some packages, we will add a lot to this, but it is important to keep this basic functionality intact.
I have made an AppDir template that you can use to make your own AppDirs. It contains an AppInfo.xml file that's ready to be filled in and an AppRun script with a few more advanced features. Among other things, it has support for saving settings to be used by the AppRun, some command-line friendly switches, and a few other goodies. It might be a bit complex if you're not much of a shell scripter, but I find it very useful.
In my opinion, writting an AppRun script is perhaps the hardest part of building an AppDir, if for no other reason than that shell scripts are so difficult to debug. (Example gotcha: Did you know that BASH, and probably some other shells, will choke on any shell script that is saved as an MS-DOS format text file?) The good news is that you can steal the bulk of the script directly for ROX-Session, as the code to compile the application will most likely only require tweaking, not a complete rewrite.
The main thing that you'll have to do is pass parameters to make so that everything will be put in the right place when you do "make install." You may need parameters for configure, or you may not (more on this later). Either way, you should be passing parameters to make rather than relying on configure. You can get the names of the make parameters you need by looking at the Makefiles or simply by watching the output of the build and using trial and error. The usual ones are prefix, bindir, etc., but you'll probably have to check anyway, as not every Makefile is the same. Once you know what "make install" is doing, you can go back and try it again with different parmaeters.
There are a couple of things to remember when passing parameters to make. First, make is case-sensitive. Therefore, "make prefix=$APP_DIR install" is not the same thing as "make PREFIX=$APP_DIR install." One will work as expected, and one will use the default prefix (probably /usr/local).
Second, you should know that it's not always possible to make make do what you want just by passing parameters. Sometimes you'll need to edit the Makefile. This is fine if the program uses just a Makefile, but if the Makefiles are generated by autoconf, then you don't want to touch them. You can always try to monkey around with configure.in and such if you know how, but for those who don't (like me) it's much easier to just add a few commands after the "make install." For example, you may want to move the man pages into the Help directory or rename them from prog.1, prog.5, etc. to prog.man. I do this with most AppDir because ROX only picks up the MIME type of a man page as x-man, or whatever, if the file name ends in .man. The .1, .5, etc. files are recognized as simple TROFF formatted text files, which, while accurate, isn't quite good enough.
If you're repackaging a program that is normally built and installed in the conventional "./configure ; make && make install" way, then you're going to want to put code in your AppRun script to compile it. That's where -D comes in
If you're not a programmer, or not acquainted with GCC, you probably don't know about -D. It's an option passed to GCC that tells it to pass a defininition to the C preprocessor. In case you're not familiar with the intellectual abortion that is the C preprocessor (I'm not a big fan of C, in case you couldn't guess), it is the program that handles things like #define and #include statements in the source code. It "processes" the C source code and performs textual substitution of #defined constants and macros, dumps #included files into the source, and does other unpleasant sounding things.
The -D option tells GCC to include additional #defines at compile
time. So, for example, passing GCC the
-DSRC_DIR=\"/usr/local/src\"
option has the same effect as putting
#define SRC_DIR \"/usr/local/src\"
in the source code. The main use of this option is for hard-coding
configuration data, such as file paths, into an executable without
requiring the user to edit the source files. Of course, it would be
much nicer to just hard-code a default value and let the use over-ride
it in a configuration file, environment variable, or command-line
option, but not every programmer wants to be bothered with that.
I first came up against this problem when trying to repackage LBreakout2. In fact, this turned out to be a problem with all the LGames. This is a problem because the entire point of an application directory is that all the program data is contained within it and can be copied, relocated, etc. simply by moving around that single directory. Obviously, if you hard-code data paths into the binary, you can't just move the AppDir around. However, the configure script sometimes will not allow you to give it a relative path. So what's an AppDir packager to do?
We are left with two options. The first one I tried, which is the more difficult and less effective, is to patch the source code. We can simply find the preprocessor variable in the source and change the code to use getenv() instead. This is a lot of work and it needs to be repeated with every upgrade. In other words, this is exactly the kind of thing we want to avoid.
The other method is to pass a parameter to make. We often do this when running "make install", but you can do it when running any make rule. In the case of LBreakout2, the variable we want to make into a relative path is called DATADIR. While configure might not like relative paths, make itself doesn't know the difference. Therefore, we can just run make DATADIR=\".\" and all of a sudden the lbreakout2 executable will be looking for its data relative to the current directory. We must simply remember to change to the right directory in the AppRun script before we execute the binary. This method is pretty easy to use and requires little or nothing in the way of maintenance, so it's definitely the prefered method when we can use it.
There's one last thing to consider with regard to conpiling software. As I mentioned, some configure scripts don't like relative paths, while others do. Normally, it doesn't matter, as you can just pass the relative paths you need straight to make, which doesn't even check the path. However, there is one case in which you cannot bypass the configure script, and that is the case of config.h.
You see, autoconf has this nifty feature where it lets you write a file called config.h.in. When the configure script runs, it will examine this file and and turn it into a header file called config.h, which has all sorts of useful platform-specific information in it. The problem with this is that sometimes data paths are defined in this file. This is not something we can override with make, because the string is hard-coded right in the header file. We also can't (or shouldn't) change the header file because it is generated by configure. Thus, our only option is to pass options to configure to use a relative path for whatever the relevant variable is. In most cases, using --prefix=. should do what we want, but it's difficult to be certain.
The thing that really kills me about this is that you have no idea that there's a problem with the configuration untill it's too late. For example, I discovered this when repackaging GQView 1.3.2, and I didn't even know there were any absolute paths left in the executable until it failed to load the help file from /usr/local/share/gqview. I ran make with prefix=. and assumed that everything was alright. None of the output of the build process led me to believe that this would not work. Live and learn.
When you build an AppDir, that has lots of documentation or data directories, there are two approaches you can take. The first is to manually copy the documentation, data, and whatever else to the location you want it. The second is to let "make install" install everything and them put code in AppRun to move things around to the appropriate place and set the appropriate environment variables.
Personally, I think the second approach is better. The main reason is that it reduces the amount of work you'll have to do, and thus the potential for error, when you try to package the next version as an AppDir. Ideally, you will be able to just drop the new source directory in place and the AppRun script will take care of the rest.
The one place where the do-it-yourself approach is better than putting code in AppRun is when the documentation, data, etc. is packaged as a seperate archive. I ran into this when packaging LGeneral, which has a seperate data archive, and RealVNC, which has a documentation archive. If the standard install process consists of nothing more than extracting the archive to the proper place, which was the case with both these applications, then there's no point in complicating the AppRun script. Just extract the archives to the right place in the AppDir and be done with it.
There are just a few more things to think about when building an AppDir. The first is the AppMenu. This is the menu that shows up above the main directory operations menu when you right-click on an AppDir. All this does is pass an option to the AppRun script, so it is very simple to use, but is still very powerful.
One of my favorite uses for the AppMenu is to set up quick configuration toggles. All this involves is adding code to AppRun to detect an option and then writing the appropriate entry to a configuration file. This file is read later in the script and the environment or run options are set accordingly.
I've experimented with two ways of setting this up. The first is to use simple output redirection through the shell. I simply echo values into files and then read the files to get the values back. The code looks something like this:
if [[ "$1" == '--use-sound' ]] ; then echo '1' > $CONFIG_DIR/use_sound fi USE_SOUND=`cat $CONFIG_DIR/use_sound
Yes, this is a rather primitive way of doing this, but it's also simple and effective. The other way, which is much more elegant, stores configuration preferences in an XML file. However, I had to write a Perl script to extract data from and write data to XML files. This adds dependencies in the form of the Perl binding to Expat and it adds complexity to the configuration process. I'm still in the early stages of working with this method, but it seems promising, especially for programs with a large number of options, some of which the user may want to hand edit.
It's also worth noting that AppMenus can be used to send non-configuration options to AppRun. I use this technique a lot when writing AppWrappers. For example, I have an Xnest wrapper that has a nested AppMenu that can be used to set the display on which the nested X server should run. I did the same thing with an SSH wrapper that allows you to select a system to connect to from the AppMenu. Remember, it's the little things that make using AppDirs really nice. Take advantage of them.
Another thing to consider is argument handling. Some programs can take lots of arguments. Some can only take one at a time. Some might need their arguments massaged a little before using them. You should consider adding code to your AppRun to account for things like this. For example, if a program only works on one file at a time, you could add a loop to operate on each input file in sequence.
If you're packaging a commandline program, you might also want to consider a simple GUI front-end. I have used PyMessage with a few AppDirs in order to prompt the user for an action or to get an input string. For example, I added code to the AppDir for GToaster that lets the user drag an ISO image onto the AppDir and then prompts to mount or burn it. I also did a wrapper for SSH that prompts the user for a user@host name and to choose an SSH or SFTP connection. This is a pretty simple interface, but, to me, it actaully seems to work pretty well.
There is one last thing that I want to touch on, and that's the command line. AppDirs are designed to work with ROX-Filer, and ROX-Filer is a GUI file manager and X11 desktop environment. You can't run it unless you're running X. However, AppDirs are more than just a ROX feature; they're also a way of packaging and distributing programs, and some of those programs need to run from the command line.
So how to we deal with this? Well, there are a couple of ways. If we only have one executable program in the AppDir, then it's easy: just run the AppRun script from the command line. If you want, you can even make a symlink to it somewhere in your path. (The AppDir template I mentioned earlier has some code to support this. I've written a little more about it on this page. Please read this before you try it, as there are some unexpected complications.)
But what if you have a package with more than one executable, such as ImageMagic? Well, you have a couple of options. I've outlined them both on my AppDir ideas page. My favorite solution is to create a "--exec" option to the AppRun script. The code for it looks something like this:
case $1 in # Catch calls to other executables. # This is useful for general command-line use. --exec) EXEC_FILE_PATH="$APP_DIR/$PLATFORM/$2" if [ -x "$EXEC_FILE_PATH" ] ; then shift ; shift exec "$EXEC_FILE_PATH" "$@" else echo "Unable to find $EXEC_NAME in binary directory." echo "Use $0 --lsexec to list the existing binaries." fi ;; # List the possible executables. --lsexec) ls "$APP_DIR/$PLATFORM" ;; ...
This option allows you to run any of the binaries in the package as an
argument to AppRun. So, in the ImageMagic example, we could now take a
screenshot by running
/usr/apps/ImageMagic/AppRun --exec import -window root screen.jpg
This is better than using symlinks to the
binaries themselves, or some similar method, because it maintains the
features of the AppDir like automatic platform detection and a custom
environment. (I think it's also a good idea to add a "--lsexec" option
to simply list the available binaries. This saves the user having to
worry about their platform.) This system makes things fairly easy for
the user without compromising the power or flexibility of the
AppDir.
The strangest package I've ever done was the Sawfish pager. The newest version of it supports being used as a GTK+ plug, which means it can be run in the ROX panel. The build process is simple enough, using only a GNUmakefile to build the executable and install some Lisp scripts in $HOME/.sawfish. I was able to edit the GNUmakefile without incident, but I was faced with the problem of starting the pager. This can only be done through the Lisp interface to Sawfish. Fortunately, there is a sawfish-client program which can be used to interact with the Lisp environment from the command line. I used this to configure and start the pager in AppRun and it worked perfectly... until I dragged the AppDir to the panel. The pager started up correctly, but ROX inexplicably popped up a message saying that the program quit with out ever creating a widget. I knew this couldn't be right because I was looking at the pager in the panel when this message popped up. When I dismissed the message box, the pager was terminated. It turns out that the AppRun script terminated too fast, so apparently ROX thought that the program had crashed. I was able to fix this simply by adding a "sleep 5" command at the end of the AppRun script. Ugly, I know, but it worked.
As a general rule, repackaging proprietary (i.e. not open source) applications is very easy. I've done this with Opera, Realplayer, Adobe Acrobat, and three Loki games. Proprietary applications are often completely self contained and only put a symlink or a wrapper script in /usr/bin. Thus, repackaging such an application is usually no more difficult than pointing the installer at the AppDir. Editing the wrapper script may be necessary, but it is usually trivial.
I had an interesting time with the GTK+ 2.2 developer release of GQView. There was nothing particularly unusual about the package itself, but it did introduce me to one nuance of repackaging that I'd never seen before.
Like many programs, GQView uses a header file called config.h. This is a file that's generated automatically by the configure script and contains various information specific to your system. However, in this case, it also contained a #defined path to the program help file directory. I don't know why they put this path in config.h as opposed to setting it with flags to GCC, like they did with the path to the i18n files. I'm sure it seemed like a good idea at the time.
Now, as I've mentioned above, I often tend to ignore flags to the configure script. You usually use then to set things like prefix and docdir, but I always override those when I run make anyway, and some configure scripts won't take the relative paths we need, so why even bother messing with configure? Well, in this case, I didn't have a choice. Since config.h is generated by configure, make doesn't even come into play.
By itself, this isn't particularly earth shattering. The only problem was that I had no clue there was a hard coded path in the program until I ran it. Paths that are defined with the -D flag to GCC are easy to find, but this one is a different story. Perhaps the configure script did spit this out sompleace in it's outout, but I sure didn't see it. I'm lucky if I can spot anything in all that output. I guess the only solution is to make 'configure --prefix=.' the default policy and adjust it for scripts that don't like relative paths.
MPlayer is, in my opinion, the best media player currently available
for Linux. Sure, the developers are a bit loopy, and the more I read
their home page, the more I question their - well, let's not go there.
The point is that they put out a pretty good product. However, it was
one of the more difficult repackagings I've done.
(Note: You
may be objecting that Xine is better. I've heard good things about Xine
as well, but I've always had better luck with MPlayer. Plus, last time
I checked, the Xine user interface was bad to the point of being almost
unusable. Some of the blame can probably be placed on the people who
write the pretty-but-impossible-to-read-or-understand skins - the
default skin in particular was pretty horrifying - but that's a
different story.)
Anyway, MPlayer has a reputation for being very difficult to install and configure. This reputation is entirely deserved, as MPlayer is a pain in the neck to configure. Part of this is due to what I refer to as OSS programmer's disease. This simply refers to the aversion many OSS programmers seem to have to packaging everything that a program needs in order to run together. In the case of MPlayer, this includes skins and subtitle fonts. (Third party codecs are also seperate, but, being third party, that's at least understandable.) Why the MPlayer team doesn't package a default skin with the source distribution I'll never know. It's just unforgivable! The GUI can't run without a skin, so this ought to be a no-brainer. Maybe it's because the GUI is not built by default, which doesn't make much sense either, but that's another story.
Like the LGames, MPlayer hard-codes paths to data and configuration files in a few places. After playing with it for a while, I was able to get it to build with all relative paths. Once this was accomplished, it was a simple matter to set up a default skin, subtitle font, and codec configuration file in the appropriate directories. This at least gives the user everything she needs to run MPlayer out of the box. On the down side, it makes things a little more complicated when upgrading to newer versions, but it's still the best way to go.
There are still two problems left, though. First is the issue of per CPU optimizations. In older versions of MPlayer, the program was optimized for a particular CPU at compile-time. This made binary distribution a real problem, as two i586-based systems with identical software would not necessarily be able to run the same binary. In the new version, there is a compile-time option to use run-time CPU detection, allowing us to use the same binary on multiple systems. The down side is that this comes with a small performance penalty. On older systems, like mine, where video playback isn't all that fast to begin with, this might be a problem.
The solution was actually pretty simple. I simply changed added a variable for run-time CPU detection. If it is set, then run-time CPU detection is used and all is well. If it is unset, then the AppDir makes $PLATFORM directories with $CPU directories inside them. This allows for multiple binaries to be built for the same architecture but still allows for a single binary that faster systems can use out of the box.
The other problem is that MPlayer uses external codecs to play some Windows file formats, such as Windows Media format. This is not a problem in and of itself, except that MPlayer requires that these codecs be installed in /usr/lib/win32 (yes, this is hard coded) before the program is compiled. Unfortunately, there's not really anything that can be done about this short of patching the code, which I don't want to do. I just notify the user in the documentation and on the download page and let them figure it out.