User:Wizzup/Jenkins

From Maemo Leste Wiki
Jump to navigationJump to search

Setting up a Jenkins build slave requires the following steps to be executed on a machine that matches the Maemo devuan version (Not sure if this is strictly required, we just did it that way before). 

apt install default-jre-headless
apt install jenkins-debian-glue jenkins-debian-glue-buildenv-devuan
apt purge exim4-base exim4-config exim4-daemon-light
apt --purge autoremove
apt install ntp

useradd -m -s /bin/bash jenkins
passwd jenkins
su - jenkins # add ssh pub key from jenkins master to .ssh/authorized_keys
apt install sudo
sudo vi /etc/sudoers # allow passwordless sudo for sudo group
gpasswd -a jenkins sudo

Also, this might not be necessary, but one could add this: 

# cat /etc/sudoers.d/jenkins
jenkins ALL=NOPASSWD: /usr/sbin/cowbuilder, /usr/sbin/chroot, /bin/chmod, /bin/rm, /bin/mv, /bin/cat
Defaults env_keep+="DEB_* DIST ARCH ADT QEMU_LOG_FILENAME"

Also copy over /etc/jenkins from one of the existing nodes, specifically for the debian glue config, see below.

Then add it to Jenkins as a node (I just copied an existing node and changed the host/ip).

Then run a build, it will likely fail, because our keys are not yet added to the pbuilder environment that just got created during the first build.

Add it like so: 

cd /var/cache/pbuilder/base-beowulf-arm*.cow
wget https://maedevu.maemo.org/extras-key.asc
wget https://maedevu.maemo.org/testing-key.asc

chroot /var/cache/pbuilder/base-beowulf-arm*.cow
apt install gnupg1
apt-key add < extras-key.asc
apt-key add < testing-key.asc
rm *.asc
exit
cd -


/etc/jenkins/pbuilderrc: 

MIRRORSITE="http://pkgmaster.devuan.org/merged/"
COMPONENTS="main contrib non-free"
DEBOOTSTRAPOPTS=("${DEBOOTSTRAPOPTS[@]}" "--keyring=/usr/share/keyrings/devuan-keyring.gpg" "--no-merged-usr")
PBUILDERSATISFYDEPENDSCMD=/usr/lib/pbuilder/pbuilder-satisfydepends-apt
EXTRAPACKAGES="$EXTRAPACKAGES devuan-keyring"
AUTOCLEANAPTCACHE=yes

if [[ $ARCH == arm* ]] ; then
   EXTRAPACKAGES="$EXTRAPACKAGES pinthread"
   export LD_PRELOAD="$LD_PRELOAD /usr/lib/pinthread.so"
   export PINTHREAD_CORE=0
fi

/etc/jenkins/debian_glue: 

# Example configuration file for the jenkins-debian-glue software.
# Install this file as /etc/jenkins/debian_glue to enable it.

# Set GnuPG ID that should be used for signing the reprepro repository.
# If you build packages for Squeeze your reprepro repositories *MUST* be
# signed and the keyring that holds the public key (REPOSITORY_KEYRING)
# must be set.
# Expected format: DEADBEEF

if [ "$release" = "leste" ]; then
	KEY_ID=4AA81E3E026EFE82E47D6901545FEC4E0927F6FD
else
	# extras
	KEY_ID=89F632F52BFE13EBBB2EBD0D2700BD8E6604EC2E
fi

# If TRUNK_RELEASE is set then the package(s) of the repository
# receiving the packages that are built will be copied to the
# repository specified in TRUNK_RELEASE. This provides the option
# to have all recent versions of packages in a central repository.
# TRUNK_RELEASE='release-trunk'

# If packages are build for more than one architecture you can
# decide which architecture should build the "Architecture: all"
# packages. On all other architectures only the arch specific
# packages will be build then.
# If unset it will default to the architecture of the host system.
MAIN_ARCHITECTURE="amd64"

# Hook scripts for pbuilder/cowbuilder, used e.g. for autopkgtest.
# Default:
# PBUILDER_HOOKDIR=/usr/share/jenkins-debian-glue/pbuilder-hookdir/

# To enable a pbuilder configuration file for usage as documented
# in man pbuilder(8) under --configfile set this option.
# By default no user specific pbuilder configuration file is used.
# PBUILDER_CONFIG=/etc/jenkins/pbuilderrc

# Base directory for reprepro repositories, can be overwritten
# via REPOSITORY on demand
# Default:
DEFAULT_REPOSITORY='/srv/repository'

# In order to keep the build versions unique and to ensure that there
# are no duplicates, the version number from the changelog is padded
# with the date and the build number.
# To override this behaviour and preserve the version number from the
# changelog, set USE_ORIG_VERSION to true.
USE_ORIG_VERSION=true
SKIP_DCH=true

DCH_OPTS="--multimaint-merge --ignore-branch"
#DCH_EXTRA_OPTS="--distribution=maemo7 --force-distribution -R"
#DCH_EXTRA_OPTS="

#maybe? line 103 in /usr/bin/generate-git-snapshot
UNRELEASEAD_APPEND_COMMIT="false"

# By default reprepro repositories are not verified but assumed to be
# trustworthy.
# Please note that if you build packages for Squeeze, the reprepro
# repositories *MUST* be signed and verifiable. I.e. you need to set
# KEY_ID and the corresponding keyring in REPOSITORY_KEYRING that
# holds the public key portion for that KEY_ID.
# REPOSITORY_KEYRING=/etc/apt/trusted.gpg.d/my-custom-keyring.gpg
REPOSITORY_KEYRING=/var/lib/jenkins/.gnupg/keyring.gpg

#release=${release:-kawai}
#REPOSITORY=/srv/repository
# If $release is set then "${REPOSITORY}/release/${release}"
# is used as release repository. If you want to use a different
# directory set RELEASE_REPOSITORY accordingly.
# Default:
# RELEASE_REPOSITORY="${REPOSITORY}/release/${release}"

RELEASE_REPOSITORY="${DEFAULT_REPOSITORY}/${release}"

# Remove packages from a $release before processing incoming
# This allows to rebuild and provide versions already existent
# in the release repository they are built for.
# Default:
# REMOVE_FROM_RELEASE=false

# By default the underlying build system (cowbuilder/pbuilder)
# automatically destroys the build environment, even if building
# fails for whatever reason. If DEB_KEEP_BUILD_ENV is set then
# a copy of the build environment is created under
# /var/cache/pbuilder/build/debug.$JOBNAME
# Please be aware that those build environments usually require
# quite some disk space, so do not forget to remove them.
# If unset the build environment won't be kept if building fails.
# DEB_KEEP_BUILD_ENV=true

# Comma separated list of extra repositories for resolving dependencies.
# Usage example:
# REPOSITORY_EXTRA='deb http://security.debian.org/ wheezy/updates main,deb-src http://security.debian.org/ wheezy/updates main'
# Default:
# REPOSITORY_EXTRA=

PARENT_DISTRO=
DEVUAN_BUILD=

case "$distribution" in
    stretch*)
        PARENT_DISTRO=ascii
        ;;
    buster*)
        PARENT_DISTRO=beowulf
        ;;
    bullseye*)
        PARENT_DISTRO=
        ;;
    bookworm*)
        PARENT_DISTRO=
        ;;
    *)
        DEVUAN_BUILD=1
esac

if [ -z "$DEVUAN_BUILD" ]; then
        # Debian
        MIRROR="http://deb.debian.org/debian"
        SECURITY_MIRROR="http://security.debian.org/debian-security"
        SECURITY_FOLDER="${distribution}/updates"

        # This option is needed for pbuilder to work nice in Devuan environment
        PBUILDER_CONFIG=/etc/jenkins/debian_mirror
        # Initialize it with command:
        # echo 'MIRRORSITE=http://deb.debian.org/debian' > /etc/jenkins/debian_mirror
else
        # Devuan
        MIRROR="http://pkgmaster.devuan.org/merged"
        SECURITY_MIRROR="${MIRROR}"
        SECURITY_FOLDER="${distribution}-security"
fi

REPOSITORY_EXTRA="deb ${MIRROR} ${distribution}-updates main contrib non-free"
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb ${SECURITY_MIRROR} ${SECURITY_FOLDER} main contrib non-free"
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution} main contrib non-free"

# Pull in deps from -devel when building -devel
if [ "$ENABLE_MAEMO_DEVEL" = "yes" ]; then
	REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution}-devel main contrib non-free"
fi

if [ "$ENABLE_MAEMO_EXPERIMENTAL" = "yes" ]; then
	REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution}-experimental main contrib non-free"
fi

# Pull in deps from the main repo when building extras
if [ "$release" = "extras" ]; then
	REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/leste ${distribution} main contrib non-free"
fi

# Pull in deps from main repo when building for Debian
if [ -n "$PARENT_DISTRO" ]; then
	REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/leste ${PARENT_DISTRO} main contrib non-free"
fi

# Device specifics
backports_jobs="mesa"
#n900_jobs="libmatchbox2-n900 hildon-desktop-n900"

echo "job_name: $JOB_NAME"
case "$JOB_NAME" in
*-source)
	_curpkgname="$(echo $JOB_NAME | sed 's,-source$,,')"
	;;
*-binaries/*)
	_curpkgname="$(echo $JOB_NAME | cut -d '/' -f1 | sed 's,-binaries$,,')"
	;;
*-repos/*)
	_curpkgname="$(echo $JOB_NAME | cut -d '/' -f1 | sed 's,-repos$,,')"
	;;
*)
	_curpkgname="irrelevant"
	;;
esac

if echo "$backports_jobs" | grep -qw "$_curpkgname"; then
	case "$distribution" in
	ascii*|stretch*|beowulf*|buster*)
	        REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb ${MIRROR} ${distribution}-backports main contrib non-free"
	        ;;
	esac
fi

#if echo "$n900_jobs" | grep -qw "$_curpkgname"; then
#	REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution} n900"
#fi

droid4_jobs="pvr-omap4"
if echo "$droid4_jobs" | grep -qw "$_curpkgname"; then
	REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution} droid4"
fi

network_jobs="pine64-uboot zkgroup signald anbox-image"
if echo "$network_jobs" | grep -qw "$_curpkgname"; then
	PBUILDER_USENETWORK=yes
fi

# Comma separated list of URLs with keys for extra repositories.
# Usage example:
# REPOSITORY_EXTRA_KEYS='http://foorepo.domain.com/key1.gpg,http://barrepo.domain.com/key2.gpg'
# Default:
# REPOSITORY_EXTRA_KEYS=
REPOSITORY_EXTRA_KEYS='http://maedevu.maemo.org/testing-key.asc,http://maedevu.maemo.org/extras-key.asc'

# autopkgtests are executed automatically inside the cowbuilder environment by
# default, corresponding to the ADT='internal' setting or if ADT is unset.
# To execute autopkgtests also externally via adt-run you need to set ADT to
# either 'external' (no internal runs) or 'all' (internal + external runs).
# The external autopkgtests depend on a configured ADT_RUNNER setting.
# To skip internal as well as external autopkgtests set ADT='skip'.
# Usage example (see adt-virt-qemu(1) for details regarding image setup):
# ADT='external'
# ADT_RUNNER="adt-virt-qemu /srv/adt-${distribution:-}-${architecture:-}.img"
# Default:
# ADT=

# Control usage of eatmydata to speed up builds. If eatmydata is present on the
# host system and when building for a recent distribution (Debian/jessie +
# Ubuntu/vivid or newer) eatmydata is enabled automatically.
# To force its usage (skipping any host + distribution checks) set it to 'true'.
# To disable its usage set it to 'false'.
# USE_EATMYDATA=true

# Enable ccache (compiler cache for fast recompilation of C/C++ code)
# to speed up builds.
USE_CCACHE=true

# Specify the format (as accepted by the 'date' command) for the generated
# build timestamp.
# Default: %Y%m%d%H%M%S
# TIMESTAMP_FORMAT="%Y%m%d%H%M%S"

# In case you are explicitly marking your new package version as "UNRELEASED"
# inside the debian/changelog file, you may not get a package build version
# number containing the SCM commit. Set this to 'true' in order to work around
# this.
# Default: false
# UNRELEASED_APPEND_COMMIT=false
Retrieved from "https://leste.maemo.org/index.php?title=User:Wizzup/Jenkins&oldid=1282"