Introduction
Yum (Yello Dog Updater Modified) is a popular high level package management tool for RPM (Red Hat Package Manager) packages. RPM packages are used by several popular distributions based on Red Hat Linux. Some of these distributions are Red Hat Enterpriose Linux (RHEL), CentOS, Fedora Core, SuSe, OpenSuSe, Mandrivia etc. CentOS is built from Red Hat Enterprise sources and hence can share the packages with RHEL and Fedora Core. SuSe and Mandrake RPMs have a slightly different format, as a result they are somewhat incompatible with the rest of the distributions.
Irrespective of the differences in the RPM file format, each of these distributions is capable of maintaining a yum repository for package management. In my experience, infrastructures based on Red Hat will have a mix of RHEL, Fedora Core and CentOS servers and the RPMs can be shared among these versions with slight care of keeping the dependencies in mind.
With the release of Red Hat Enterprise 5, Red Hat has formally dumped the ancient package manager up2date and have started using yum as the official package manager. Fedora and CentOS were using yum anyway from quite long. Even with yum there is a difference in the repository structure between older and newer versions. The newer tool to maintain the repository metadata is called as ‘createrepo‘ and the repository it creates is repomod. For creating and maintaining old style yum repository a tool called yum-arch is used. yum-arch is now depricated. Following table briefly presents the tool and the type of repository supported by it:
| Yum Tools | ||
|---|---|---|
| Tool | Package | Provides |
| genbase | apt | apt support |
| yum-arch | yum | yum support |
| createrepo | createrepo | repomod support (new yum and new apt) |
To the best of my knowledge, the up2date program which comes with the RHEL 4 Update 5 does not support the newer yum repositories created using createrepo. So if you have such servers in your setup then you are better off also creating the old style repo using the yum-arch command and also the new style repo using the createrepo command. Both the headers can co-exists simultaneously. The new headers are stored by createrepo in a directory called repodata whereas the old headers are stored by yum-arch in a directory called headers.
I have had opportunity to set up a local yum repositories at my work place and also for several clients.
This document describes a repository server on CentOS 5 for CentOS 5. In between I have also mentioned some points worth noting if you have RHEL 4 or older servers in your setup or you wanted to create an RHEL 4 server as a repository server.
Its not necessary to have a CentOS, Fedora Core or RHEL server as a repository server. I have very recently setup a Debian Etch server to do the same task. Starting from Debian Etch and Ubuntu Fiesty “createrepo” command is available as a Deb package.
Repository Server
In order to create a Yum repository server for your organisation following software is required:
- A webserver, or FTP server or NFS server to serve the repository to clients.
createrepopackage which is available with CentOS, Fedora Core and RHEL 5. For RHEL 4 this needs to be installed from external sources. One good source is Dag Wieers RPM repository.
On CentOS do the following to install the dependencies:
# yum install httpd createrepo
On RHEL 4 (or RHEL 3) server we need to download createrepo, python-urlgrabber, sqlite, python-sqlite, python-elementtree and yum packages from the Dag Wieers repository as they do not have these two packages in the official repository. To install these packages for RHEL4 following commands can be used.
$ sudo up2date -i httpd $ wget http://dag.wieers.com/rpm/packages/python-urlgrabber/python-urlgrabber-2.9.7-1.2.el4.rf.noarch.rpm $ wget http://dag.wieers.com/rpm/packages/createrepo/createrepo-0.4.6-1.el4.rf.noarch.rpm $ wget http://dag.wieers.com/rpm/packages/yum/yum-2.4.2-0.4.el4.rf.noarch.rpm $ wget http://dag.wieers.com/rpm/packages/python-elementtree/python-elementtree-1.2.6-7.el4.rf.x86_64.rpm $ wget http://dag.wieers.com/rpm/packages/python-sqlite/python-sqlite-1.0.1-1.2.el4.rf.x86_64.rpm $ wget http://dag.wieers.com/rpm/packages/sqlite/sqlite-2.8.17-1.el4.rf.x86_64.rpm $ sudo rpm -Uvh *.rpm $ rm -f *.rpm
Client
On the client machines which will be configured to use this repository we would need the following installed:
- Yum (depends on
python-elementtree,python-sqliteandsqlite)
On CentOS, Fedora Core and RHEL 5 these should already be installed. On RHEL 4 and older they can be installed from Dag Wieers RPM repository. Although up2date from RHEL 4 update 2 onwards supports yum and apt repositories but still I have seen problems quite often and anyways, its abandones by Red Hat from RHEL 5 onwards.
Following is how we can installed yum on a RHEL 4 (x86_64) machine:
$ wget http://dag.wieers.com/rpm/packages/sqlite/sqlite-2.8.17-1.el4.rf.x86_64.rpm $ wget http://dag.wieers.com/rpm/packages/python-sqlite/python-sqlite-1.0.1-1.2.el4.rf.x86_64.rpm $ wget http://dag.wieers.com/rpm/packages/python-elementtree/python-elementtree-1.2.6-7.el4.rf.x86_64.rpm $ wget http://dag.wieers.com/rpm/packages/yum/yum-2.4.2-0.4.el4.rf.noarch.rpm $ sudo rpm -Uvh sqlite-2.8.17-1.el4.rf.x86_64.rpm python-sqlite-1.0.1-1.2.el4.rf.x86_64.rpm python-elementtree-1.2.6-7.el4.rf.x86_64.rpm yum-2.4.2-0.4.el4.rf.noarch.rpm
Creating a Directory Tree
I normally create local repositories over http protocol, so I have created the directory tree in the web root.
$ mkdir /var/www/html/yum/CentOS/5/i386/os $ mkdir /var/www/html/yum/CentOS/5/i386/extras $ mkdir /var/www/html/yum/CentOS/5/i386/updates
Next copy all the RPMs from the distribution CDs as follows:
$ cd /var/www/html/yum/CentOS/5/i386/os $ for num_cd in 1 2 3 4 5 6;do read "Enter CD number $num_cd. Press any key when ready.." key_press;cp -ar /media/cdrom/* /var/www/html/yum/CentOS/5/i386/os/ && eject;done
In case you have the ISO files, you can mount them as follows and copy the files:
$ mkdir temp_mnt $ for num in 1 2 3 4 5 6; do mount -o loop CentOS5-i386-disc$num.iso temp_mnt;cp -ra temp_mnt/* /var/www/html/yum/CentOS/5/i386/os/;umount temp_mnt;done
Since we will be generating our own headers we need to remove the repodata directory from the copied tree. In CentOS 5 the repodata directory was in the root of the distribution CD:
$ rm -rf /var/www/html/yum/CentOS/5/os/repodata
Please note that in RHEL 5 CDs there are multiple repodata directories one per channel, they can be removed as follows:
$ rm -rf /var/www/html/yum/RHEL/5/i386/os/Cluster/repodata $ rm -rf /var/www/html/yum/RHEL/5/i386/os/ClusterStorage/repodata $ rm -rf /var/www/html/yum/RHEL/5/i386/os/Server/repodata $ rm -rf /var/www/html/yum/RHEL/5/i386/os/VT/repodata
The reason why we need to remove this repodata directory and generate our own is because we have copied this tree from CD and hence the base URL for all the RPMs in the headers is stored as media:.
Creating The Base OS Repository
createrepois a bourne shell script which in turn just calls the underlying python program called /usr/share/createrepo/genpkgmetadata.py. To create a yum repository of all the packages from base operating systems we need to execute the createrepo command in the directory which we want to be turned into a repository as follows:
$ createrepo /var/www/html/yum/CentOS/5/i386/os/
The createrepo command will create all the required headers in the repodata directory. Normally we won't add any new RPM file in the base distribution, so we won't need to generate these headers again. But if for some reason we copy a new RPM file in the base repository or replace an existing RPM file for various reasons, then we need to regenerate the headers with createrepo command.
Configuring a client to use this base repository
On the client machine, remove the information for existing standard distribution repositories. The various configuration for repositories will be present either in etc/yum.conf file or /etc/yum.repos.d directory. If its the later, remove all the files from the /etc/yum.repos.d directory and if its the former, delete all lines defining the default distribution repositories.
The IP address for my local yum server is 192.168.23.43.
The following is an example of CentOS yum client, change it appropriately for a Fedora Core or RHEL yum client. Usually the repositories can be mixed also, but that may in some cases lead to dependency problems.
$ sudo rm -f /etc/yum.repos.d/* $ sudo vi /etc/yum.repos.d/centos-base.repo [base] name=CentOS-$releasever - Base baseurl=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/os/ gpgcheck=1 gpgkey=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/os/RPM-GPG-KEY-CentOS-5 #released updates #[updates] #name=CentOS-$releasever - Updates #baseurl=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/updates/ #gpgcheck=1 #gpgkey=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/os/RPM-GPG-KEY-CentOS-5
Don't bother about the second section containing the updates repository, we have not configured it yet and will be doing it shortly. This is the reason why the second repository has been commented out.
Now to start using this new base repository, do the following:
$ sudo yum clean all $ sudo yum update
Now any package in the repository can be installed using ''yum install'' and so on.
Setting up Updates and Extras Repository
After the base repository has been setup, it can only be used for installing new packages, but what about the security fixes/patches released by the distribution providers for existing packages? We need an updates repository for applying such packages. There is no hard and fast rule that we need to have an updates repository. So, if you prefer to have only one repository containing an updated/patched version of RPM packages then you can do that as well. All you need to do is replace the concerned RPM file in the base repository with an patched/updated RPM file and then re-run the createrepocommand to re-generate the headers. After that a yum update on the client side will update the concerned RPM.
However, its always advisable to keep the updated packages in a seperate repository for better management and not to touch the base repository after setup. This is the standard approach and this is what we are going to do next.
To start with we need a source from where we can get the updated packages. Choosing one of the mirrors of the distribution in question and continuously syncing the updates from it is a good idea.
I synchronise with my chosen mirrors for CentOS and Fedora Core as follows:
$ rsync rsync://mirrors.kernel.org/centos/5/updates/i386/RPMS/*.rpm /var/www/html/yum/CentOS/5/i386/updates/ $ rsync rsync://mirrors.kernel.org/centos/5/addons/i386/RPMS/*.rpm /var/www/html/yum/CentOS/5/i386/extras/ $ rsync rsync://mirrors.kernel.org/centos/5/centosplus/i386/RPMS/*.rpm /var/www/html/yum/CentOS/5/i386/extras/ $ rsync rsync://mirrors.kernel.org/centos/5/extras/i386/RPMS/*.rpm /var/www/html/yum/CentOS/5/i386/extras/ $ rsync rsync://mirrors.kernel.org/centos/5/fasttrack/i386/RPMS/*.rpm /var/www/html/yum/CentOS/5/i386/extras/
This rsync job can be run as a periodic cron job. The frequency depends on how critical updates are for you. Usually once a day is good enough. After initial sync we can convert this collection of RPMs into a repository by using createrepo utility.
$ createrepo /var/www/html/yum/CentOS/5/i386/updates/ $ createrepo /var/www/html/yum/CentOS/5/i386/extras/
Now we have the updates repository ready, so we can uncomment the commented out section in /etc/yum.repos.d/centos-base.repo file so that it now looks as follows:
$ sudo vi /etc/yum.repos.d/centos-base.repo [base] name=CentOS-$releasever - Base baseurl=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/os/ gpgcheck=1 gpgkey=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/os/RPM-GPG-KEY-CentOS-5 # released updates [updates] name=CentOS-$releasever - Updates baseurl=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/updates/ gpgcheck=1 gpgkey=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/os/RPM-GPG-KEY-CentOS-5 # Other CentOS repositories [extras] name=CentOS-$releasever - Extras baseurl=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/extras/ gpgcheck=1 gpgkey=http://192.168.23.43/yum/CentOS/$releasever/$ARCH/os/RPM-GPG-KEY-CentOS-5
In some cases we have found that the company firewall is blocking all outgoing connections and the only available connections are http and that too using a proxy server. In such cases, we have found that a program called lftp is extremely helpful. RPM packages of lftp are available from here Dag Wieers repository. After installing this we can create a cron job from the following quick script:
$ vi ~/bin/lftp_centos_repo_sync.sh #!/bin/sh export http_proxy=http://localproxy.local:8080 export HTTP_PROXY=$http_proxy cd /var/www/html/yum/CentOS/5/i386/updates lftp -c mget "http://mirrors.kernel.org/centos/5/updates/i386/RPMS/*.rpm" createrepo /var/www/html/yum/CentOS/5/i386/updates cd /var/www/html/yum/CentOS/5/i386/extras lftp -c mget "http://mirrors.kernel.org/centos/5/fasttrack/i386/RPMS/*.rpm" lftp -c mget "http://mirrors.kernel.org/centos/5/addons/i386/RPMS/*.rpm" lftp -c mget "http://mirrors.kernel.org/centos/5/centosplus/i386/RPMS/*.rpm" lftp -c mget "http://mirrors.kernel.org/centos/5/extras/i386/RPMS/*.rpm" createrepo /var/www/html/yum/CentOS/5/i386/extras $ chmod +x ~/bin/lftp_centos_repo_sync.sh
Restricting a repository for a particular set of packages
Sometimes it is desirable to add a repository in local yum configuration, but restrict it to a certain number of packages only. This can be done by adding includepkgs line in the repository config as follows:
$ cat /etc/yum.repos.d/dag-wieers.repo [dag] name=Dag-CentOS-$releasever baseurl=http://192.168.23.43/yum/RHEL/$releasever/$ARCH/dag/ gpgcheck=1 gpgkey=http://192.168.23.43/yum/RHEL/$releasever/$ARCH/dag/RPM-GPG-KEY.dag.txt includepkgs=clamav clamav-devel clamav-db unrar
This will make sure that only listed packages are downloaded/updated from the local dag repository.