The freeze script is checked in to the BitKeeper repository bk://jonslug.bkbits.net/oe-scripts. To get a copy of freeze just execute the normal BitKeeper commands to clone the repository then check out the script - it is in the scripts subdirectory:

The freeze script is not yet checked in.

Other approaches

There are at least three other approaches to the problem. One is to rewrite bitbake in something faster. This offers limited chances success because the OpenEmbedded source tree and the .bb file format force every file to be fully parsed in the absence of external information. (This is because each file has to be parsed using the full distro/machine environment to be sure of finding out what it PROVIDES.)

The other approaches are described on http://www.openembedded.org/cgi-bin/moin.cgi/BuildsFaster. One is BBCOW, which avoids the memory problem without working round the need to parse the files. The other is http://www.openembedded.org/cgi-bin/moin.cgi/HolgerSchurigBuildSystem which uses a more formal method to build a list of .bb files to read.

Putting a script on a wiki page is difficult. I can't see a way of doing this without either making this page unreadable or creating the opportunity for a Trojan horse or worse.

The solution to this problem (using the existing bitbake implementation) is to explicitly list each .bb file required for the build. This can be done simply by setting BBFILES in conf/local.conf to the list of files. Nevertheless the list changes, so automatically generating it is highly desireable.

The freeze script

The freeze script is at the end of this wiki page (in a form designed to allow it to be copy'n'pasted) - this is temporary, it should be in the OE source tree somewhere!

freeze is a bash shell script which examines every sub-directory in the OE build work directory and works out which .bb file was used to generate it. It then writes these files to a special .conf file called frozen-bbfiles.conf and writes a second file, frozen-packages.conf, which contains the corresponding package directories.

The second file is equivalent to the nslu2-package-symlinks directory tree, however it is specific to the build - openslug, unslung or your chosen variant.

Because of the way bitbake works it is possible to include both these files (in the order frozen-packages then frozen-bbfiles) in conf/local.conf after the original BBFILES assignment. If the files do not exist bitbake skips them, if they do exist they change the value of BBFILES to a reduced subset.

Using freeze

freeze contains built-in help! This is intended to be at least as up-to-date as this wiki page, so the page is just a guide. Use freeze itself to get the exact instructions.

Use freeze --help to get the basic startup information. This suggests that you define an alias frz in the same file as you use to setup your OE build environment. This is actually quite important - freeze has built-in heuristics to work out where to find files based on the NSLU2 setup instructions, but this means passing it the shell variables (OEROOT and OEBUILD at least) which are used by the build. The frz alias gets this right! You can copy and paste from the freeze --help output into your startup file (assuming you use a bash-alike shell, csh users are on their own...)

freeze is designed just to produce the two frozen files. It doesn't hack your environment, it doesn't edit any files, it doesn't change anything. It's also a shell script, so you can check it for bugs.

To return to the full build (either nslu2-package-symlinks or the complete packages/*/*.bb setting) just delete the frozen .conf files. To stop using freeze completely just delete the two include lines from conf/local.conf.

You can use freeze with the output of nslu2-package-symlinks, but if you are adding new packages to OE you probably want to use the full build (if your machine can stand it.) This is because the full build can show up a few problems (like duplicate package names) which the symlink build hides.

Problems with freeze

The most likely problem is that a new package name will fail the heuristics used within freeze to work out the corresponding .bb file name. The heuristics are contained in the sequence of check commands within the script.

At present the only problem package is portmap. It is the only package in the whole tree which uses a version number containing a hyphen character! If portmap did not have a version number of this form the check commands would be non-heuristic in that there would be no chance of a mismatch.

A fail means that the wrong .bb file is chosen. The failure should be obvious, but if a build fails immediately after a freeze in a non-obvious way check to see if a different version of some signifcant package got built.

A simple check on the correctness of a freeze is to repeat the build - it should complete almost instantly and build nothing (assuming your build tree was complete and stable before the freeze!)

The freeze script

Putting a script on a wiki page is difficult. You should copy the whole of the following script and paste it into an editor. There should be no characters in this script which might cause an editor to damage it (i.e. there are no control characters, other than newline).

(:table border=0 width=100% bgcolor=#eeffee:) (:cell:) TBD this is difficult TBD (:tableend:)

When bitbake runs an OpenEmbedded build it consumes about 250KBytes for each .bb file it reads - regardless of whether it uses the file. This means that a full build parsing all the .bb files in the OpenEmbedded BitKeeper repository consumes over 500MByte - there are over 2200 .bb files!

This has two major consequences. Firstly the build is impossible on a machine with less than this amount of virtual memory (DRAM and swap space). Secondly the time to build a single package is almost always dominated by the time to load the cache of .bb files.

The NSLU2PackageSymlinks BitKeeper repository addresses this problem by identifying the individual package sub-directories required for an NSLU2 build. The technique uses symbolic links and it fails if you have a non-standard build directory - if the cloned BitKeeper repository is not called openembedded. The technique also still loads about three times as many .bb files as are actually required.