How to build a mirror of deb.freexian.com

Using a private mirror is recommended as soon as you have more than a couple of Debian systems relying on Freexian's ELTS repositories.

Prerequisites

This page describes how to install a Debian mirror using deb.freexian.com as source of the packages. We provide instructions for configuring debmirror here, because it’s the simplest to use. But if you have more advanced needs there are other tools that you may want to investigate, such as reprepro and aptly.

Feature debmirror aptly reprepro
Mirror a Debian archive
Mirror a partial Debian archive (filter by package)
Include dependencies in a partial mirror
Publish your own packages to a Debian archive
(Manually) test updates and publish them to internal repository after testing
Publish a mirror to Amazon S3 / Swift

User

The mirror can be run as a regular, non-root user. Transferring files from one computer to another does not need special rights.

If you are using aptly/reprepro this user can also own your repository’s GPG Keyring. The keyring is used to keep both the public signing key of your package source and the key to sign your own package files (this is not needed for debmirror, as it just copies the source and does not allow any modification like own package upload).

Simply create the new user with the following command:

sudo adduser --disabled-password mirror_freexian

From now on, everything, except creating the directory structure for your mirror data and installing new packages, should be done as this user.

Directory Structure

As we will put the data in /srv/FreexianMirror, we need to prepare a directory structure:

/srv/FreexianMirror/
|-keyring
|-data
sudo mkdir -p /srv/FreexianMirror/data
sudo mkdir -p /srv/FreexianMirror/keyring
sudo chown -R mirror_freexian:mirror_freexian /srv/FreexianMirror

gpg keys

In order to check the validity of the contents of the mirror, we need to obtain the public part of the repository signing key of Freexian. As we need the keyring in our Freexian mirror data directories you can use:

wget https://deb.freexian.com/extended-lts/archive-key.gpg -O /srv/FreexianMirror/keyring/trustedkeys.kbx

If you also want to mirror Debian, include the Debian archive keyring in the trusted keys:

cat /usr/share/keyrings/debian-archive-stretch-stable.gpg >> /srv/FreexianMirror/keyring/trustedkeys.kbx

Repositories

Freexian provides 2 relevant repositories for ELTS users: jessie & jessie-lts. (These examples are for jessie, but apply to other ELTS releases too.) Clients only need to use one of these, depending on whether they have a separate APT source entry for Debian’s jessie archive or not.

Server Suite Size All packages in Debian release Stable & LTS updates ELTS updates Signed By
deb.debian.org jessie Large Debian
security.debian.org jessie/updates Smaller Debian
deb.freexian.com jessie Large Freexian
deb.freexian.com jessie-lts Smaller Freexian

Note that during the lifetime of an ELTS release, the Debian archive keys will expire and Debian will archive the release from deb.debian.org to archive.debian.org. So, either you should maintain an internal mirror for the full archive and configure APT to ignore expired signatures, or you should mirror Freexian’s full release suite (jessie).

Configuring debmirror

debmirror is already available as a Debian package for a long time.

Apart from the packages that should be already available due to steps above, debmirror can be installed with the following commands:

sudo apt install debmirror

Now create /srv/FreexianMirror/mirrorbuilder.sh with your preferred editor (and maybe chmod it to 700) and the following contents:

#!/bin/bash

# we are using the keyring created in our $HOME directory
# if you want to use a different keyring, set GNUPGHOME here
export GNUPGHOME=/srv/FreexianMirror/keyring

# Architecture, can be:
#   amd64, armel, armhf, i386    for Jessie LTS
#   amd64, arm64, armhf, i386    for Stretch LTS
#   multiple options can be provided, comma-separated
# -> debmirror parameter -a
arch="amd64"

# section: most of the package will be in main, but contrib and non-free are available as well
# -> debmirror parameter -s
section="main,contrib,non-free"

# Release can be either
#   jessie or jessie-lts    for Jessie ELTS (jessie contains all packages including updates, jessie-lts only contains only LTS & ELTS updates)
#   stretch or stretch-lts  for Stretch LTS (stretch contains all packages including updates, stretch-lts only contains only LTS & ELTS updates)
#   multiple options can be provided, comma-separated
# -> debmirror parameter -d
release="stretch"

# Mirror sources? (deb-src)
sources="--nosource"
#sources="--source"

# Server is the name of the server you want to mirror
# -> debmirror parameter -h
server="deb.freexian.com"

# path under which the mirror is available on the mirrored server
# /extended-lts -> http://deb.freexiam.com/extended-lts
# -> debmirror parameter -r
inPath="/extended-lts"

# protocol to access the server, may be http or rsync
# -> debmirror parameter --method
proto="http"

# path in your filesystem where the mirror is available
outPath="/srv/FreexianMirror/data/$inPath"

# debugoptions
debugoptions=""
#debugoptions="-v --debug"

# More options:
# --progress      Shows files as they are downloaded
# --nocleanup     Do not clean up the local mirror after mirroring is complete. Use this option to keep older repository
# --i18n          Mirror non-English translations of package descriptions
# --exclude=REGEX Exclude some packages from the mirror
moreoptions="--progress"

#
# Mirror Freexian ELTS
/usr/bin/debmirror $debugoptions $moreoptions $sources --rsync-extra=none -a "$arch" -s "$section" -h "$server" -d "$release" -r "$inPath" --method="$proto" "$outPath"

# If we also wanted a local Debian mirror, we could have a second run:
#release="stretch"
#server="deb.debian.org"  # or server="archive.debian.org"
#inPath="debian"
#outPath="/srv/FreexianMirror/data/$inPath"
#/usr/bin/debmirror $debugoptions $moreoptions --rsync-extra=none -a "$arch" --source -s "$section" -h "$server" -d "$release" -r "$inPath" --method="$proto" "$outPath"

We would suggest running the mirror daily, e.g. with the following in /etc/cron.daily/freexian-mirror.sh:

#!/bin/sh
su mirror_freexian -c /srv/FreexianMirror/mirrorbuilder.sh

Webserver configuration

In order to make your files available to your client, you need to configure your webserver.

apache

Create, for example a config file for the Stretch release, /etc/apache2/sites-available/freexianmirror.conf:

<VirtualHost *:80>
        ServerName freexianmirror.example.com
        ServerAdmin webmaster@localhost

        DocumentRoot /srv/FreexianMirror/data
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /srv/FreexianMirror/data>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Require all granted
        </Directory>

        ErrorLog /var/log/apache2/freexianmirror.example.com-error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog /var/log/apache2/freexianmirror.example.com-access.log combined
</VirtualHost>

Enable it with the command:

a2ensite freexianmirror
systemctl restart apache2

nginx configuration

In case you are using nginx as webserver, you can use the following configuration in /etc/nginx/sites-available/freexianmirror.conf:

server {
        listen 80;
        listen [::]:80;

        root /srv/FreexianMirror/data;
        autoindex on;

        server_name freexianmirror.example.com;

        location / {
                allow all;
        }

}

Enable it with the command:

ln -s ../sites-available/freexianmirror.conf /etc/nginx/sites-enabled/freexianmirror.conf
systemctl restart nginx

Client configuration

On your client side you need to configure apt to look at packages on your own mirror. For that to happen you need to add the file freexianmirror.conf to /etc/apt/sources.list.d with the following content:

deb http://freexianmirror.example.com/extended-lts stretch main contrib non-free

Of course you can also add similar lines for stretch-lts, jessie or jessie-lts.