Create a package (tarball) for distribution

From NAS-Central Buffalo - The Linkstation Wiki
Revision as of 20:54, 23 February 2007 by Ls2newbee (Talk | contribs)

Jump to: navigation, search

This article assumes you want to download and build some software from the software's source code. It further assumes the source code is available as some tar file (a tar ball) and that the software source follows a common structure and comes with a common mechanism to build it.

That common structure needs to be a GNU build system, starting with

  • a configure script, made with
  • GNU autoconf and using
  • GNU automake and
  • GNU libtool.

Further the source code author should have used autoconf, automake correctly and in compliance with the GNU coding standards by keeping directory output variables in the Makefiles unexpanded. This allows to use make install and overriding install locations. This is a feature which is used later in this description in order to prepare for packaging. In case the author did not confine to these rules you may try to fix this by using pack_sync script as described below.

Since there are many, many different software packages out there, there is no guarantee that the particular package can be build exactly as described here. There is no guarantee that it is packed as a tar file, and there is no guarantee that you have everything to build it. That's why many packages come with documentation! It might sound shocking, but it is a clever idea to read that one first. Often the documentation also explains what additional tools and libraries are needed to build the software, and how to fine-tune the build.

The remainder of this article further assumes you have all the common development tools installed.

1) Download the source to a preferred place:

#I suggest to create a folder on /dev/hda3.../mnt/ on ppc-LS, /mnt/hda/ on the LS2
mkdir -p <compiling-folder> 
cd <compiling-folder>    
wget http://<download-location>/<app-source>

2) Untar the package - use xzvf if it has a tar.gz-extension (=.tgz) and xjvf if it has tar.bz2

tar xzvf <app-source>.tar.gz

3) change into the new directory

cd <app-source>

4) check which options are available for ./configure by executing

./configure --help

5) configure the application with the right prefix for the final destination (e.g. /usr/local), and not for your local temporary installation directory PACKAGE/usr/local. If you cross-compile you also need to point to the cross compiler, linker, assembler.

./configure --prefix=/usr/local CC=... ...

6) Compile, but don't install the application

make

7) Become root

su

8) Clean and prepare the temporary installation directory structure to which the package will be temporarily installed before it is packed. This cleanup ensures that no left-overs from previous build attempts or accidentally moved files will be packaged, too. Warning: If you do this on the wrong directory, you might accidentally delete your whole Linux system ...

rm -rf PACKAGE # DANGER Don't do this with the the wrong directory!
# Create temporary version of the prefix directory
mkdir -p PACKAGE/usr/local

9) Install the compiled application afterwards to your temporary packaging directory, temporarily overriding the prefix setting for the installation only. This must not trigger a recompilation, and it won't if the software author has created the configure system correctly. It should keep path information already compiled into the application and related files. That is, the application should contain the final destination pathes, as set with --prefix=... with configure, while only the installation is redirected to our temporary packaging directory.

make install prefix=PACKAGE/usr/local

10) Optional in case if you did a proper installation to your system in advance: In order to check if the package files are identical with those of the living system, you may download pack_sync script to the current directory and execute it:

wget http://downloads.linkstationwiki.net/uploads/mktarball/pack_sync 
./pack_sync [PACKAGE]

Only in case that you're not using PACKAGE but another name you have to specify this. The script takes a walk thru the directory tree calling itself recursively. It compares all files with those of the living system (same directory path but w/o prefix <current-directory>/PACKAGE). If there are differences you are prompted to confirm a replacement. If you are not sure: Deny the replacement. Get informed about the differences. You may not want to replace configuration files if those of the living system contain personal data. But if you want the replacement, just run pack_sync again. Note that pack_sync may be called as often as you like but it doesn't have an undo.

This doesn't make sense. The mentioned tool does not change any hard-coded pathes compiled into binaries. These are, however, the ones who make trouble. Applications use hard coded pathes to find other component of themselves. This is not the cleverest idea, but many do. E.g. gcc does this. That is the reason why an applications must be compiled for the place where it will finally be installed. You can't compile this stuff for another location and fix it afterwards [well, in theory you can, but that is a tricky procedure, reshuffling the contents of object files)]. You either have a build system which supports this (e.g. a GNU build system), or you have to hack the build system until it complies. Running a diff on the binaries later will not fix the problem.

==> ls2newbee: Sorry if I wasn't clear enough. pack_sync does not patch any files. The idea was to install the package first to the running system. These files wil reside there where they shouldbe at any other installation. In a second step the whole installation is done again from scratch now targeting to a PACKAGE directory for creation of a tarball of compiled executables. This second make may compile wrong pathnames because the files actually do not reside where they should be on the target system (as on any system). So pack_sync is able to replace the files in the PACKAGE directory (having wrong hard coded path names) by the files of the first installation (having correct hard coded path names). For this purpose it compares files of the 2nd 'tarball' installation with the corresponding files of the 1st 'real' installation. The result is - in theory - a tarball with correct path names because the files are actually taken from the living system (i.e. from the first installaion).

In other words: If pack_sync replaced *all* files by those of the running system (correct paths!), you'd get a correct tarball. However there is no need to replace files if there is no difference.

If you have any questions, please feel free to send me a private message.

    • Please delete these comments/replace with final statement after the discussion has lead to a final result (ls2newbee)

10a) In case the software author didn't create the configuring system as properly as assumed by the previous step you may fix this by pack_sync. For configuring use

./configure --prefix=$PWD/PACKAGE/usr/local --sysconfdir=$PWD/PACKAGE/etc ...

and for installation to the packaging directory just

make install

Since you have pointed the --prefix to some local path you probably have a package which can't be installed anywhere else than on <some strange path>/PACKAGE/usr/local while you really want it to be working in /usr/local. In order to fix this you must run pack_sync as described above.

11) Continue as root, package the data up.
11a) For example, tar the file-structure up into a nice package

tar cvzf <appname>_<architecture>.tar.gz -C PACKAGE .

where architecture is ppc for LS1/HG/HS, mips for the LS2 and arm9 for the LS Pro

11b) Or build an ipkg package. The following is only a rough sketch, check the ipkg documentation for details

ipkg-proto PACKAGE  # creates an initial prototype file
# edit prototype file
# create CONTROL files
# add entries for CONTROL files to prototype file
ipkg-mk -c  # build the package

Scripting some of the above steps might make sense, instead of typing the same commands again and again.