bitbake runs an OpenEmbedded build it consumes about 250KBytes for each
.bb file it reads - regardless of whether it uses the file. This means that a full build parsing all the
.bb files in the OpenEmbedded BitKeeper repository consumes over 500MByte - there are over 2200
This has two major consequences. Firstly the build is impossible on a machine with less than this amount of virtual memory (DRAM and swap space). Secondly the time to build a single package is almost always dominated by the time to load the cache of
The NSLU2PackageSymlinks BitKeeper repository addresses this problem by identifying the individual
package sub-directories required for an NSLU2 build. The technique uses symbolic links and it fails if you have a non-standard build directory - if the cloned BitKeeper repository is not called
openembedded. The technique also still loads about three times as many
.bb files as are actually required.
The solution to this problem (using the existing
bitbake implementation) is to explicitly list each
.bb file required for the build. This can be done simply by setting
conf/local.conf to the list of files. Nevertheless the list changes, so automatically generating it is highly desireable.
freeze script is checked in to the BitKeeper repository bk://jonslug.bkbits.net/oe-scripts. To get a copy of
freeze just execute the normal BitKeeper commands to clone the repository then check out the script - it is in the scripts subdirectory:
bk clone bk://jonslug.bkbits.net/oe-scripts oe-scripts
bk get freeze
freeze is a bash shell script which examines every sub-directory in the OE build
work directory and works out which
.bb file was used to generate it. It then writes these files to a special
.conf file called
frozen-bbfiles.conf and writes a second file,
frozen-packages.conf, which contains the corresponding
The second file is equivalent to the
nslu2-package-symlinks directory tree, however it is specific to the build -
unslung or your chosen variant.
Because of the way
bitbake works it is possible to include both these files (in the order
conf/local.conf after the original
BBFILES assignment. If the files do not exist
bitbake skips them, if they do exist they change the value of
BBFILES to a reduced subset.
freeze contains built-in help! This is intended to be at least as up-to-date as this wiki page, so the page is just a guide. Use
freeze itself to get the exact instructions.
freeze --help to get the basic startup information. This suggests that you define an alias
frz in the same file as you use to setup your OE build environment. This is actually quite important -
freeze has built-in heuristics to work out where to find files based on the NSLU2 setup instructions, but this means passing it the shell variables (
OEBUILD at least) which are used by the build. The
frz alias gets this right! You can copy and paste from the
freeze --help output into your startup file (assuming you use a bash-alike shell,
csh users are on their own...)
freeze is designed just to produce the two frozen files. It doesn't hack your environment, it doesn't edit any files, it doesn't change anything. It's also a shell script, so you can check it for bugs.
To return to the full build (either nslu2-package-symlinks or the complete packages/*/*.bb setting) just delete the frozen
.conf files. To stop using
freeze completely just delete the two
include lines from
You can use
freeze with the output of nslu2-package-symlinks, but if you are adding new packages to
OE you probably want to use the full build (if your machine can stand it.) This is because the full build can show up a few problems (like duplicate package names) which the symlink build hides.
The most likely problem is that a new package name will fail the heuristics used within
freeze to work out the corresponding
.bb file name. The heuristics are contained in the sequence of
check commands within the script.
At present the only problem package is portmap. It is the only package in the whole tree which uses a version number containing a hyphen character! If
portmap did not have a version number of this form the
check commands would be non-heuristic in that there would be no chance of a mismatch.
A fail means that the wrong
.bb file is chosen. The failure should be obvious, but if a build fails immediately after a freeze in a non-obvious way check to see if a different version of some signifcant package got built.
A simple check on the correctness of a freeze is to repeat the build - it should complete almost instantly and build nothing (assuming your build tree was complete and stable before the freeze!)
There are at least three other approaches to the problem. One is to rewrite
bitbake in something faster. This offers limited chances success because the OpenEmbedded source tree and the
.bb file format force every file to be fully parsed in the absence of external information. (This is because each file has to be parsed using the full distro/machine environment to be sure of finding out what it
The other approaches are described on http://www.openembedded.org/cgi-bin/moin.cgi/BuildsFaster. One is BBCOW, which avoids the memory problem without working round the need to parse the files. The other is http://www.openembedded.org/cgi-bin/moin.cgi/HolgerSchurigBuildSystem which uses a more formal method to build a list of
.bb files to read.