Getting packages

Slackbuilds

ref. http://slackbuilds.org/

Slackware Alien builds

http://www.slackware.com/~alien/slackbuilds/

Automation

Each time you get a package from the links above, you perform a number of steps which are in most cases the same:

For each application or package, this needs to be done. It would be helpful to have a script which performs these actions by itself just indicating which package is needed. The script getpkgs streamlines this process. Download it in a proper location. There may also be a HINTS.TXT file which may indicate manual actions to be done (see below).

You should never download a script and run it -especially as root or even sudo- unless you have carefully inspected it and understand what it does.

The script is written with the smallest set of requirements. In essence, it only needs a standard Slackware installation and even in this case the script requires only (part of) following package sets

There is no requirement for X and others to run it, but most packages themselves depend on graphical stuff, so you will probably need a default installation in most cases anyway. The script itself can be run from a command line. It is written as a bash script using mostly standard tools sed, awk, ... Nevertheless, to successfully run this script, following requirements need to be taken into account:

Note that, although the script itself needs nothing else, several packages require other sets, so, in general, do install all standard package sets.

The script

The script getpkgs creates two subfolders,  one for the sources, the other for the built packages:

VERSION is specified by /etc/slackware-version
MACHINE is specified by uname -m, modified as is done in the slackbuild scripts


The script runs in several stages:
  1. download the SLACKBUILDS.TXT which contains the description of all the packages. It is conveniently renamed to SLACKBUIKDS-$VERSION.TXT
  2. download sources: In this stage, the build/$VERSION/[$PACKAGE] subdirectory is created and all the files are downloaded from the internet and extracted into this folder
  3. build sources: In this stage, the files retrieved from previous steps are built using the $PACKAGE.Slackbuild script. The results end up where Slackbuild decides, but it is expected to be found in /tmp.
  4. install binaries: In this stage, the $PACKAGE....tgz is searched for in the /tmp, copied to the pkgs folder and installed.
When the script is run it checks what has been done so far, and does not redo completed tasks. Stages 2. 3. and 4. can be run separately by adding the options -s, -b or -i as additional parameter. For instance, only getting all the required sources can be done by:

localuser@bellatrix:~/Development$ ./getpkgs -s dosbox

As an example, running without the option -s gives

localuser@bellatrix:~/Development$ ./getpkgs dosbox

[...]
--2013-04-23 20:35:13--  http://surfnet.dl.sourceforge.net/project/dosbox/dosbox/0.74/dosbox-0.74.tar.gz
Connecting to 192.168.1.24:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 1265711 (1.2M) [application/x-gzip]
Saving to: 'dosbox-0.74.tar.gz'

100%[======================================================================================================>] 1,265,711    257KB/s   in 4.8s  

2013-04-23 20:35:20 (257 KB/s) - 'dosbox-0.74.tar.gz' saved [1265711/1265711]

dosbox-0.74/
dosbox-0.74/acinclude.m4
dosbox-0.74/config.sub
dosbox-0.74/AUTHORS
dosbox-0.74/configure
dosbox-0.74/missing
dosbox-0.74/ChangeLog
dosbox-0.74/visualc_net/
dosbox-0.74/visualc_net/dosbox.sln
...

After a time it ends with:

Slackware package /tmp/dosbox-0.74-i486-2_SBo.tgz created.

Verifying package dosbox-0.74-i486-2_SBo.tgz.
Installing package dosbox-0.74-i486-2_SBo.tgz:
PACKAGE DESCRIPTION:
# DOSbox (DOS emulator/virtual machine for X11 and Unix)
#
# It allows you to play many of the old games you grew up loving, as
# well as many apps designed to run on DOS.
#
Executing install script for dosbox-0.74-i486-2_SBo.tgz.
Package dosbox-0.74-i486-2_SBo.tgz installed.

localuser@bellatrix:~/Development$

You can now test the package with the relevant command:

localuser@bellatrix:~/Development$ which dosbox
/usr/bin/dosbox
localuser@
bellatrix:~/Development$ dosbox
DOSBox version 0.74
Copyright 2002-2010 DOSBox Team, published under GNU GPL.
---
CONFIG:Loading primary settings from config file /home/local/localuser/.dosbox/dosbox-0.74.conf
MIXER:Got different values from SDL: freq 44100, blocksize 940
ALSA:Can't subscribe to MIDI port (65:0) nor (17:0)
MIDI:Opened device:oss

And a dosbox appears:



Solving dependencies and other options

Often packages rely on others to be available before they can be successfully run or even built and installed. As indicated, this requires digging trough the info and slackbuilds site to get the needed material. The getpkgs script can perform these operations by itself. As an example, aMule requires cryptopp and wxPython. To get everything downloaded, built and installed at once, add the option r (recursive):

localuser@bellatrix:~/Development$ ./getpkgs -r aMule

After a while, you have aMule ready

localuser@bellatrix:~/Development$ amule

giving the aMule configuration popup:




Note: The script relies on the information in the SLACKBUILDS.TXT and slackbuild scripts. But even then it might fail due to other reasons, for instance sources which are hidden behind all kind of forms etc. In this case, actions to correct the problems can be done manually after which the script can be restarted.

Note: The script does not set "interesting" properties as they are made available in several places. All is kept configured default.

Note: several packages requires additional actions as creating users and group
s. This must always be done manually.

Following a list of options:

localuser@bellatrix:~/Development$ ./getpkgs -h

usage: getpkgs [options] package [rootname]
Getting slackbuild packages (http://slackbuilds.org)

  a:  abort on first error found, otherwise keep continuing
  b:  only build downloaded sources
  c:  do not show dialogs or do not wait for user interaction
  d:  do not install or build final package, only dependencies (requires r)
  D:  list all root packages (which do not serve as dependent ones)
  f file:  slackbuilds description file, default SLACKBUILDS.TXT
  g:  group installation dependencies per package
  G:  group source dependencies per package
  h:  show this help
  H:  Dryrun, parse dependencies but do nothing
  i:  only install built packages
  L location:  Slackware location, default = http://slackbuilds.org/slackbuilds/(version)
  o directory: Use directory as output directory
  q:  query for possible matching packages (case insensitive)
  r:  recursive, process dependencies specified in REQUIRES field as well
  s:  only download sources
  S:  update SLACKBUILD descriptions
  t user:  Specify user for extracting Slackbuild tar files
  u:  update package sources (and its dependencies if -r specified)
  U:  update all package sources (progress monitor if Verbose)
  v:  Show additional information
  V:  Show settings
  x:  remove installed package(s, when recursive)
  X:  remove all versions of installed package(s when recursive)
  Y:  remove builds of installed packages (requires x or X)
  Z:  remove sources of installed packages (requires x or X)

Example: getpkgs -gr vlc

Sometimes, it is useful to collect all dependencies for a package. This can be achieved by specifying the -g and -G options. -g Places all installable packages into a 'rootname' subfolder, -G all sources. If rootname is not specified, the first one before the dependencies are handled is taken as rootname.

The HINTS.TXT file

When a file HINTS.TXT is available, getpkgs uses this to message manual actions. These actions are sometimes needed as, otherwise a successful package can not be built. Hints may be specific download instructions, but also specific changes to be made to the system. When a HINT is detected, the script will stop and allow you to perform the manual actions before continuing. In case of the DOWNLOAD hint, the script will not stop if the needed files are already present.

Each line in this file is composed of a comma separated entry of which
  1. the name of the package
  2. the instruction
    #,REQUIRES,This is a message you may want to display always even before trying to download something
    #,DOWNLOAD,Additional download instructions for instance howto get a URL
    #,BUILD,Before building you may correct the options if needed
    #,INSTALL,Installation may require specific settings like creating a group with groupadd -g 215 GROUPNAME
    #,NOSUDO,Indicates that you have to start getpkgs as root on a local filesystem if not it won't build
    #,VERSION,If $PACKAGE.info version has been updated without updating SLACBUILDS.TXT
    #,POSTINSTALL,When this package is used as dependency for others you may need to perform this action
    #,OPTIONALPACKAGES,You can add a space separated list of packages which should be treated as needed dependencies
  3. informative text shown in the dialog, or additional info for the getpkgs script

An empty line or a line starting with a # is ignored.

As an example get jdk when the HINTS.TXT file contains the line,

jdk,DOWNLOAD,\nhttp://www.oracle.com/technetwork/java/javase/downloads/jdk7u-downloads.html\n
running with this hint:

localuser@aldebaran:/home/shared/installation/packman/test$ ./getpkgs jdk
Getting jdk from http://slackbuilds.org/slackbuilds/14.0/development/jdk.tar.gz
wget -v -nd http://slackbuilds.org/slackbuilds/14.0/development/jdk.tar.gz --proxy-user= --proxy-password=
--2013-11-06 23:55:47--  http://slackbuilds.org/slackbuilds/14.0/development/jdk.tar.gz
Connecting to 192.168.1.24:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 3516 (3.4K) [application/x-gzip]
Saving to: 'jdk.tar.gz'

100%[=====================================================================================>] 3,516       --.-K/s   in 0s     

2013-11-06 23:55:48 (588 MB/s) - 'jdk.tar.gz' saved [3516/3516]

jdk/
jdk/slack-desc
jdk/README
jdk/jdk.SlackBuild
jdk/profile.d/
jdk/profile.d/jdk.sh
jdk/profile.d/jdk.csh
jdk/jdk.info

Now following dialog appears:



Other scripts

Sometimes it is not obvious how a package is named. Manually browsing through the SBo site or SLACKBUILDS.TXT file requires some effort. For instance, to find a possible match for the 7zip compressor:

localuser@aldebaran:/home/shared/installation/packman/ins$ ./getpkgs -q 7z
j7z: j7z version 1.1.0 in /misc/j7z
p7zip: p7zip version 9.20.1 in /system/p7zip
localuser@aldebaran:/home/shared/installation/packman/ins$

Several other related scripts can be found in my collection

getpkgs options overview

NAME
getpkgs - a script for collecting Slackbuilds.org packages
SYNOPSIS
getpkgs [OPTION] ... PACKAGENAME [ROOTNAME]
DESCRIPTION
The script performs an installation of a slackbuild script. It downloads the SLACKBUILDS.TXT description file from http://slackbuilds.org, downloads the slackbuild script, downloads the sources, build the sources and installs the package. If the slackbuild info file specifies REQUIRES, it can handle these additional packages as well.
OPTIONS
-a:
Abort on the first error found. Normally, the script continues in most cases even when errors are encountered. This way, it tries to do as much as possible in advance. For instance, if downloading of one package fails, it may still be possible to continue with others anyway. However, in some cases, it may be useful to abort immediately. For instance when debugging an installation. With the -a option specified, the script will check at critical places and stop there.
-b:
Build downloaded sources. If they are not available in the ./srcs subdirectory, they are not downloaded and the script aborts when -a is specified or continues with its dependencies when -r is specified.
-c:
Do not show dialogs or do not wait for user interaction. When manual interventions are needed, they can be specified in the HINTS.TXT file. The script will check the HINTS.TXT for specific actions and normally stop and allow you to execute them. When option -c is specified, the script will ignore this information and continue. This can be used for instance when this script is used in another and let it continuing without user intervention. For instance if you have a list of packages and you want to download them in background.
-d:
Do not install or build the final package, only the dependencies, which requires -r to be specified as well.
-D:
List all the root packages from the srcs folder.  A root package is one which is not used as a dependency for others. Rebuilding all the root packages will automatically rebuild all the packages. This option only checks the already downloaded srcs, not the ones in the SLACKBUILDS.txt as it would require downloading all the slackbuild scripts available. This script will also take a long time to run, especially on larger sets.
-f file:
The name of the slackbuilds description file, the default is SLACKBUILDS.TXT as it is used on http://slackbuilds.org
-g:
Group the installation dependencies per package. This results in all the build packages which are needed as dependencies to be added and collected to a dedicated subfolder in ./pkgs/$VERSION/$MACHINE/$PACKAGE. For instance, eclipse requires jdk as well. If eclipse is built with the option -g, then there will be a dedicated subfolder eclipse containing both eclipse-*.tgz and jdk-*.tgz. This makes it easier to install an application as all the required dependencies are immediately found in the same directory. On the other hand, it results in some libraries found in several locations. When also option -r is specified, the script will run all dependencies with an additional ROOTNAME command line appended. This will have as result that the scripts can find the original location where all the dependencies should be placed. The ROOTNAME can also be used manually if you want to add a package yourself in this location.
-G:
Group source dependencies per package, this has the same effect as -g only for collecting the sources. Note that the sources do not use a machine specific subfolder.
-h:
Show help and a list of options
-H:
Dryrun, parse dependencies but do nothing else. The option -H will automatically set -r as well. This requires that the Slackbuild script is available, so this one will be downloaded if needed, but not the sources. If dependencies are found in the info, they will be checked in turn as well. So a whole chain of slackbuild scripts downloads can be triggered. As only the slackbuilds are downloaded and nothing else is done, it gives a fast, tree like overview of all the packages that are implied (dependencies). It will also check if manual interventions are required and note them as well.
-i:
Only install the built packages. They are looked for in the ./pkgs subfolder.
-L location:
The location where the slackbuilds are fetched from. The default is http://slackbuilds.org/slackbuilds/$VERSION. From this place, the description SLACKBUILDS.TXT and all slackbuild scripts are downloaded.
-o directory:
Use directory as output directory. All files are saved in this location. If not specified, the current directory is used.
-q:
Query for possible matching packages in SLACKBUILDS.TXT (case insensitive). If SLACKBUILDS.TXT is not available, it is downloaded first. The search is performed with a grep -i, so all partial case insensitive matches are validated. The output contains an exact name to be used and the version as found in the descriptions.
-r:
Recursive operation. This processes the dependencies specified in REQUIRES field as well. If a package indicates another one in this field a new ./getpkgs script is started with the dependent one as parameter. The dependent package is handled before the parent. This guarantees that the sources are downloaded built and installed before the parent package is handled. The same actions are performed as for the parent. Thus if only -s (download sources) is specified, then the dependent package will only get its sources. If -g or -G is specified, then all the dependent packages are grouped together in the originating root package, even if they are detected several layers deeper. Thus a grand (grand...) -child ends up there as well. An exception is done when option -d is specified. In this case all dependencies are fully downloaded built and installed, but the original root package has only its sources downloaded. Note that some care has to be taken as circular dependencies will end in an endless loop until system shell resources are exhausted. This is also valid for the OPTIONALPACKAGES specification in the HINTS.TXT.
-s:
Only download sources as specified in the info file. Note that specifying this option will download both the slackbuild and the actual sources.
-S:
Update the SLACKBUILD descriptions. The existing one is moved to an archive subfolder with a timestamp appended. A new description is downloaded from http://slackbuilds.org or the location if specified with -L.
-t user:
Specify user for extracting Slackbuild tar files. In general, when a tar file is extracted, it honors the user as specified during the creation of the tar file. This could result in a problem if this user is not allowed. For instance on a shared NFS file system with all_squash where it is not possible to change ownership of the tar file, this would result in an error (the file will be created though). To avoid this error, the proper user can be specified.
-u:
Update the package sources. The update is detected by comparing the version in the descriptions with the version in the slackbuild.info file. Note that if the version is overwritten in the HINTS.TXT file, this will cause still the original version to be installed. A message will be shown however. The option -u has the same effect as -s except that the existing sources are moved to the archive subdirectory with a timestamp appended. The original built binary in the ./pkgs subdirectory will be moved to this archive as well under an $ARCH subdirectory. If the option -g is also specified, all the built binaries (thus also the other dependencies if they exist) are moved as well. If -r is specified, all its dependent packages will have its sources updated as well. They will also get the same timestamp in the archive.
-U:
Update all package sources. For every package found in the ./srcs, a check is performed to see if the version is different from the SLACKBUILDS.TXT. If so, the existing one is moved to the archive, and a new one is downloaded. Note the all packages in the archive get the same timestamp, even if they are not related. Also, note that if option -G is used, this has no effect.
-v:
Show additional information
-V:
Show settings
-x:
Uninstall package. When option -r (recursive) is specified, all dependencies are removed as well. Use option -r with caution as there is no checking if the dependencies are still used by other packages. Removing these might break other applications.
-X:
The same as option -x but this removes also all other versions. For instance, if a package was installed and then later updated with a new version, both will be present in the system. Using X in stead of x, both will be removed, not just the current. This applies to -r as well.
-Y:
Remove builds of installed packages (requires x or X)
-Z:
Remove sources of installed packages (requires x or X)
The script also uses some internal options. They can be specified, but it is not supposed to be used manually. They are used by the script itself:
-T:
Set the timestamp. This option is used to collect all items in the archive with the same timestamp. The timestamp is created using date +%Y%j%H%M%S and it is handled over from parent to child if needed. On a sidenote, the script has only second time resolution. This implies that only one script might be started at the same time if it is using the timestamp (-u option). The reason is that, if more then one script is running with the same timestamp, even if the specified package is different, it may trigger the same change, or move of the same directories when recursive (option -r) is also active. This could result in unexpected changes.
-R:
Set original shell level, this is used for 'pretty printing' of verbose messages and the dependency tree.
The default selected options are 'sbi' which will just download build and install without checking for any requirements/dependencies. All files are saved in the current directory.

Some more examples:
 getpkgs               downloads the descriptions, SLACKBUILDS.TXT file
 getpkgs dosbox        downloads builds and install dosbox in ./srcs and ./pkgs in current directory
 getpkgs -D            shows all packages in ./srcs which are root
 getpkgs -S            update Slackbuilds descriptions
 getpkgs -U            update all packages in ./srcs
 getpkgs -s vlc        get sources of vlc
 getpkgs -grsbi vlc    get sources of vlc, build and install, collect dependent packages
 getpkgs -u vlc        update sources of vlc, if the Slackbuilds version is different
 getpkgs -ur vlc       update sources of vlc and all its dependencies
 getpkgs -grubi vlc    update sources of vlc and all its dependencies, build and install updates as well
 getpkgs -grXYZ vlc    uninstall and cleanup everything related to vlc.


K. E. Venken - Hoboken (BE) - 2013