Sync

2022-10-10T10:29:32Z

Wizzup:

This page explains and suggests various ways of synchronizing contacts, calendars and files.

= Calendar, Notes, Tasks and Contacts =

Synchronizing calendar, notes and tasks (using CalDav) and contacts (using CardDav) can be done using syncevolution (other [https://syncevolution.org/documentation/backends/ backends are available], but not currently documented here). A frontend to running synchronization tasks is available in the repository (Syncevolution Frontend).

== GUI ==

The "Syncevolution Frontend" application is a graphical interface to syncevolution. While the interface can technically help you set up the synchronisation configuration through the graphical interface, we haven't tried it and do not have instructions for it at the moment. We only use it for scheduling and running synchronisations - the configuration still has to be done using the console, currently.

(Contributions welcome)

<gallery widths=320px heights=173px>
File:syncevolution.png|main syncevolution screen
File:syncevolution-2.png|syncevolution for a specific sync target
</gallery>

== Synchronizing with syncevolution CalDav + CardDav ==

If you get stuck in an odd state setting this up, you could consider running the following to get to a clean state, at least with regards to syncevolution configuration:

<pre>
rm -rf /home/user/.config/syncevolution
</pre>

=== NextCloud synchronizing ===

We will be synchronizing the calendar and such using local '''maemo-events''', '''maemo-notes''' and '''maemo-tasks''' backends using the '''caldav''' backend.
For contacts, we will synchronize '''evolution-contacts''' using '''carddav'''.

A few URLs are required, first, the '''main Nextcloud DAV url''', this will appear like:

<pre>https://my.nextcloud.org/remote.php/dav</pre>

then there is the '''url for a specific calendar''', this is called the "Private Link" and you can get it by clicking "Copy Private Link" for a given calendar in Nextcloud, this might look like so:

<pre>
https://my.nextcloud.org/remote.php/dav/calendars/userhere/calendarnamehere/
</pre>

Finally, for your address book (contacts) synchronization, the '''contacts URL''' is needed (in Nextcloud, go to contacts, settings, "Copy Link" for your contacts):

<pre>
https://my.nextcloud.org/remote.php/dav/addressbooks/users/userhere/contacts/
</pre>

Given the above, a setup to synchronize calendar, tasks, notes and contacts could look like this:

<pre>
syncevolution --configure \
--template webdav \
username=NEXTCLOUD-USER-HERE \
password=NEXTCLOUD-PASS-HERE \
syncURL=MAIN-DAV-URL-HERE \
target-config@nextcloud

syncevolution --configure \
--template webdav \
database=CALENDAR-URL-HERE \
backend=caldavtodo \
target-config@nextcloud nextcloudtodo

syncevolution --configure \
--template webdav \
database=CALENDAR-URL-HERE \
backend=caldavjournal \
target-config@nextcloud nextcloudjournal

syncevolution --configure \
--template webdav \
database=CALENDAR-URL-HERE \
backend=caldav \
target-config@nextcloud nextcloudcal

syncevolution --configure \
--template webdav \
database=CONTACTS-URL-HERE \
backend=carddav \
target-config@nextcloud contacts

syncevolution --configure \
--template SyncEvolution_Client \
sync=none \
syncURL=local://@nextcloud \
username= \
password= \
nextcloud

syncevolution --configure \
sync=two-way \
backend=maemo-tasks \
database= \
nextcloud nextcloudtodo

syncevolution --configure \
sync=two-way \
backend=maemo-notes\
database= \
nextcloud nextcloudjournal

syncevolution --configure \
sync=two-way \
backend=maemo-events \
database= \
nextcloud nextcloudcal

syncevolution --configure \
sync=two-way \
backend=evolution-contacts \
nextcloud contacts
</pre>

All of the above is configuration that only needs to be done once.

Depending on your exact situation, your initial synchronisation could only use only values from your phone, from your nextcloud, or both. If you want to merge both, my understanding is that the '''slow''' method achieves this:

<pre>
syncevolution --sync slow nextcloud nextcloudjournal
syncevolution --sync slow nextcloud nextcloudtodo
syncevolution --sync slow nextcloud nextcloudcal
syncevolution --sync slow nextcloud contacts
</pre>

Use the '''two-way''' synchronization method for '''normal synchronization''' (daily, hourly, etc synchronizations), like so:

<pre>
syncevolution --sync two-way nextcloud nextcloudjournal
syncevolution --sync two-way nextcloud nextcloudtodo
syncevolution --sync two-way nextcloud nextcloudcal
syncevolution --sync two-way nextcloud contacts
</pre>

==== Warning: TLS Chain validation issues ====

If you run into problems with TLS Chain validation, first try to figure out if the problem is on your end. Syncevolution did have a problem for me validation the chain of my next cloud instance, so for testing purposes I set '''SSLVerifyServer = 0''' in '''/home/user/.config/syncevolution/nextcloud/peers/target-config/config.ini'''

== Radicale synchronising ==

Radicale is a simple CardDAV and CalDAV server, and is set up similarly to nextcloud. Just browse to the Radicale web URL, and use the following URLs:

# The main DAV url is simple the URL to your radicale server (without the .web part)
# The calendar URL is simply the http(s) URL to the calendar that you want to synchronize
# The contacts URL is simple the http(s) URL of the addressbook that you want to synchronize

The username (and optional password) are the radicale credentials.

=== Radicale Testing Server ===

For testing purposes, here is how to quickly run a radicale server, this assumed it's installed.

<pre>
mkdir -p ~/radicale
cd ~/radicale
python3 -m radicale --config "" --storage-filesystem-folder `pwd` -H localhost:5223 --logging-level info
</pre>

Now browse to http://localhost:5223, create an account (password is optional), and you should be able to synchronize, assuming the devices can reach your radicale server (don't have it listen on localhost only, perhaps use 0.0.0.0:5223 for LAN testing)

= Files synchronization =

TODO

= Known Issues =

* Synchronizing calendar to radicale gives errors if there are unnamed calendar events.
* Synchronizing contacts to radicale gives errors if there are contacts without Firstname or Lastname.

Sync

2022-10-10T10:26:56Z

Wizzup:

This page explains and suggests various ways of synchronizing contacts, calendars and files.

= Calendar, Notes, Tasks and Contacts =

Synchronizing calendar, notes and tasks (using CalDav) and contacts (using CardDav) can be done using syncevolution (other [https://syncevolution.org/documentation/backends/ backends are available], but not currently documented here). A frontend to running synchronization tasks is available in the repository (Syncevolution Frontend).

== GUI ==

<gallery widths=320px heights=173px>
File:syncevolution.png|main syncevolution screen
File:syncevolution-2.png|syncevolution for a specific sync target
</gallery>

== Synchronizing with syncevolution CalDav + CardDav ==

If you get stuck in an odd state setting this up, you could consider running the following to get to a clean state, at least with regards to syncevolution configuration:

<pre>
rm -rf /home/user/.config/syncevolution
</pre>

=== NextCloud synchronizing ===

We will be synchronizing the calendar and such using local '''maemo-events''', '''maemo-notes''' and '''maemo-tasks''' backends using the '''caldav''' backend.
For contacts, we will synchronize '''evolution-contacts''' using '''carddav'''.

A few URLs are required, first, the '''main Nextcloud DAV url''', this will appear like:

<pre>https://my.nextcloud.org/remote.php/dav</pre>

then there is the '''url for a specific calendar''', this is called the "Private Link" and you can get it by clicking "Copy Private Link" for a given calendar in Nextcloud, this might look like so:

<pre>
https://my.nextcloud.org/remote.php/dav/calendars/userhere/calendarnamehere/
</pre>

Finally, for your address book (contacts) synchronization, the '''contacts URL''' is needed (in Nextcloud, go to contacts, settings, "Copy Link" for your contacts):

<pre>
https://my.nextcloud.org/remote.php/dav/addressbooks/users/userhere/contacts/
</pre>

Given the above, a setup to synchronize calendar, tasks, notes and contacts could look like this:

<pre>
syncevolution --configure \
--template webdav \
username=NEXTCLOUD-USER-HERE \
password=NEXTCLOUD-PASS-HERE \
syncURL=MAIN-DAV-URL-HERE \
target-config@nextcloud

syncevolution --configure \
--template webdav \
database=CALENDAR-URL-HERE \
backend=caldavtodo \
target-config@nextcloud nextcloudtodo

syncevolution --configure \
--template webdav \
database=CALENDAR-URL-HERE \
backend=caldavjournal \
target-config@nextcloud nextcloudjournal

syncevolution --configure \
--template webdav \
database=CALENDAR-URL-HERE \
backend=caldav \
target-config@nextcloud nextcloudcal

syncevolution --configure \
--template webdav \
database=CONTACTS-URL-HERE \
backend=carddav \
target-config@nextcloud contacts

syncevolution --configure \
--template SyncEvolution_Client \
sync=none \
syncURL=local://@nextcloud \
username= \
password= \
nextcloud

syncevolution --configure \
sync=two-way \
backend=maemo-tasks \
database= \
nextcloud nextcloudtodo

syncevolution --configure \
sync=two-way \
backend=maemo-notes\
database= \
nextcloud nextcloudjournal

syncevolution --configure \
sync=two-way \
backend=maemo-events \
database= \
nextcloud nextcloudcal

syncevolution --configure \
sync=two-way \
backend=evolution-contacts \
nextcloud contacts
</pre>

All of the above is configuration that only needs to be done once.

Depending on your exact situation, your initial synchronisation could only use only values from your phone, from your nextcloud, or both. If you want to merge both, my understanding is that the '''slow''' method achieves this:

<pre>
syncevolution --sync slow nextcloud nextcloudjournal
syncevolution --sync slow nextcloud nextcloudtodo
syncevolution --sync slow nextcloud nextcloudcal
syncevolution --sync slow nextcloud contacts
</pre>

Use the '''two-way''' synchronization method for '''normal synchronization''' (daily, hourly, etc synchronizations), like so:

<pre>
syncevolution --sync two-way nextcloud nextcloudjournal
syncevolution --sync two-way nextcloud nextcloudtodo
syncevolution --sync two-way nextcloud nextcloudcal
syncevolution --sync two-way nextcloud contacts
</pre>

==== Warning: TLS Chain validation issues ====

If you run into problems with TLS Chain validation, first try to figure out if the problem is on your end. Syncevolution did have a problem for me validation the chain of my next cloud instance, so for testing purposes I set '''SSLVerifyServer = 0''' in '''/home/user/.config/syncevolution/nextcloud/peers/target-config/config.ini'''

== Radicale synchronising ==

Radicale is a simple CardDAV and CalDAV server, and is set up similarly to nextcloud. Just browse to the Radicale web URL, and use the following URLs:

# The main DAV url is simple the URL to your radicale server (without the .web part)
# The calendar URL is simply the http(s) URL to the calendar that you want to synchronize
# The contacts URL is simple the http(s) URL of the addressbook that you want to synchronize

The username (and optional password) are the radicale credentials.

=== Radicale Testing Server ===

For testing purposes, here is how to quickly run a radicale server, this assumed it's installed.

<pre>
mkdir -p ~/radicale
cd ~/radicale
python3 -m radicale --config "" --storage-filesystem-folder `pwd` -H localhost:5223 --logging-level info
</pre>

Now browse to http://localhost:5223, create an account (password is optional), and you should be able to synchronize, assuming the devices can reach your radicale server (don't have it listen on localhost only, perhaps use 0.0.0.0:5223 for LAN testing)

= Files synchronization =

TODO

= Known Issues =

* Synchronizing calendar to radicale gives errors if there are unnamed calendar events.
* Synchronizing contacts to radicale gives errors if there are contacts without Firstname or Lastname.

File:Syncevolution-2.png

2022-10-10T10:26:48Z

Wizzup:

2022-04-28T08:36:38Z

Wizzup:

Setting up a Jenkins build slave requires the following steps to be executed on a machine that matches the Maemo devuan version (Not sure if this is strictly required, we just did it that way before).

<pre>
apt install default-jre-headless
apt install jenkins-debian-glue jenkins-debian-glue-buildenv-devuan
apt purge exim4-base exim4-config exim4-daemon-light
apt --purge autoremove
apt install ntp

useradd -m -s /bin/bash jenkins
passwd jenkins
su - jenkins # add key
apt install sudo
sudo vi /etc/sudoers # allow passwordless sudo for sudo group
gpasswd -a jenkins sudo
</pre>

Also copy over /etc/jenkins from one of the existing nodes, specifically for the debian glue config, see below.

Then add it to Jenkins as a node (I just copied an existing node and changed the host/ip).

Then run a build, it will likely fail, because our keys are not yet added to the pbuilder environment that just got created during the first build.

Add it like so:

<pre>
cd /var/cache/pbuilder/base-beowulf-arm*.cow
wget https://maedevu.maemo.org/extras-key.asc
wget https://maedevu.maemo.org/testing-key.asc

chroot /var/cache/pbuilder/base-beowulf-arm*.cow
apt install gnupg1
apt-key add < extras-key.asc
apt-key add < testing-key.asc
rm *.asc
exit
cd -
</pre>

/etc/jenkins/pbuilderrc:

<pre>
MIRRORSITE="http://pkgmaster.devuan.org/merged/"
COMPONENTS="main contrib non-free"
DEBOOTSTRAPOPTS=("${DEBOOTSTRAPOPTS[@]}" "--keyring=/usr/share/keyrings/devuan-keyring.gpg" "--no-merged-usr")
PBUILDERSATISFYDEPENDSCMD=/usr/lib/pbuilder/pbuilder-satisfydepends-apt
EXTRAPACKAGES="$EXTRAPACKAGES devuan-keyring"
AUTOCLEANAPTCACHE=yes

if [[ $ARCH == arm* ]] ; then
EXTRAPACKAGES="$EXTRAPACKAGES pinthread"
export LD_PRELOAD="$LD_PRELOAD /usr/lib/pinthread.so"
export PINTHREAD_CORE=0
fi
</pre>

/etc/jenkins/debian_glue:

<pre>
# Example configuration file for the jenkins-debian-glue software.
# Install this file as /etc/jenkins/debian_glue to enable it.

# Set GnuPG ID that should be used for signing the reprepro repository.
# If you build packages for Squeeze your reprepro repositories *MUST* be
# signed and the keyring that holds the public key (REPOSITORY_KEYRING)
# must be set.
# Expected format: DEADBEEF

if [ "$release" = "leste" ]; then
KEY_ID=4AA81E3E026EFE82E47D6901545FEC4E0927F6FD
else
# extras
KEY_ID=89F632F52BFE13EBBB2EBD0D2700BD8E6604EC2E
fi

# If TRUNK_RELEASE is set then the package(s) of the repository
# receiving the packages that are built will be copied to the
# repository specified in TRUNK_RELEASE. This provides the option
# to have all recent versions of packages in a central repository.
# TRUNK_RELEASE='release-trunk'

# If packages are build for more than one architecture you can
# decide which architecture should build the "Architecture: all"
# packages. On all other architectures only the arch specific
# packages will be build then.
# If unset it will default to the architecture of the host system.
MAIN_ARCHITECTURE="amd64"

# Hook scripts for pbuilder/cowbuilder, used e.g. for autopkgtest.
# Default:
# PBUILDER_HOOKDIR=/usr/share/jenkins-debian-glue/pbuilder-hookdir/

# To enable a pbuilder configuration file for usage as documented
# in man pbuilder(8) under --configfile set this option.
# By default no user specific pbuilder configuration file is used.
# PBUILDER_CONFIG=/etc/jenkins/pbuilderrc

# Base directory for reprepro repositories, can be overwritten
# via REPOSITORY on demand
# Default:
DEFAULT_REPOSITORY='/srv/repository'

# In order to keep the build versions unique and to ensure that there
# are no duplicates, the version number from the changelog is padded
# with the date and the build number.
# To override this behaviour and preserve the version number from the
# changelog, set USE_ORIG_VERSION to true.
USE_ORIG_VERSION=true
SKIP_DCH=true

DCH_OPTS="--multimaint-merge --ignore-branch"
#DCH_EXTRA_OPTS="--distribution=maemo7 --force-distribution -R"
#DCH_EXTRA_OPTS="

#maybe? line 103 in /usr/bin/generate-git-snapshot
UNRELEASEAD_APPEND_COMMIT="false"

# By default reprepro repositories are not verified but assumed to be
# trustworthy.
# Please note that if you build packages for Squeeze, the reprepro
# repositories *MUST* be signed and verifiable. I.e. you need to set
# KEY_ID and the corresponding keyring in REPOSITORY_KEYRING that
# holds the public key portion for that KEY_ID.
# REPOSITORY_KEYRING=/etc/apt/trusted.gpg.d/my-custom-keyring.gpg
REPOSITORY_KEYRING=/var/lib/jenkins/.gnupg/keyring.gpg

#release=${release:-kawai}
#REPOSITORY=/srv/repository
# If $release is set then "${REPOSITORY}/release/${release}"
# is used as release repository. If you want to use a different
# directory set RELEASE_REPOSITORY accordingly.
# Default:
# RELEASE_REPOSITORY="${REPOSITORY}/release/${release}"

RELEASE_REPOSITORY="${DEFAULT_REPOSITORY}/${release}"

# Remove packages from a $release before processing incoming
# This allows to rebuild and provide versions already existent
# in the release repository they are built for.
# Default:
# REMOVE_FROM_RELEASE=false

# By default the underlying build system (cowbuilder/pbuilder)
# automatically destroys the build environment, even if building
# fails for whatever reason. If DEB_KEEP_BUILD_ENV is set then
# a copy of the build environment is created under
# /var/cache/pbuilder/build/debug.$JOBNAME
# Please be aware that those build environments usually require
# quite some disk space, so do not forget to remove them.
# If unset the build environment won't be kept if building fails.
# DEB_KEEP_BUILD_ENV=true

# Comma separated list of extra repositories for resolving dependencies.
# Usage example:
# REPOSITORY_EXTRA='deb http://security.debian.org/ wheezy/updates main,deb-src http://security.debian.org/ wheezy/updates main'
# Default:
# REPOSITORY_EXTRA=

PARENT_DISTRO=
DEVUAN_BUILD=

case "$distribution" in
stretch*)
PARENT_DISTRO=ascii
;;
buster*)
PARENT_DISTRO=beowulf
;;
bullseye*)
PARENT_DISTRO=
;;
bookworm*)
PARENT_DISTRO=
;;
*)
DEVUAN_BUILD=1
esac

if [ -z "$DEVUAN_BUILD" ]; then
# Debian
MIRROR="http://deb.debian.org/debian"
SECURITY_MIRROR="http://security.debian.org/debian-security"
SECURITY_FOLDER="${distribution}/updates"

# This option is needed for pbuilder to work nice in Devuan environment
PBUILDER_CONFIG=/etc/jenkins/debian_mirror
# Initialize it with command:
# echo 'MIRRORSITE=http://deb.debian.org/debian' > /etc/jenkins/debian_mirror
else
# Devuan
MIRROR="http://pkgmaster.devuan.org/merged"
SECURITY_MIRROR="${MIRROR}"
SECURITY_FOLDER="${distribution}-security"
fi

REPOSITORY_EXTRA="deb ${MIRROR} ${distribution}-updates main contrib non-free"
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb ${SECURITY_MIRROR} ${SECURITY_FOLDER} main contrib non-free"
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution} main contrib non-free"

# Pull in deps from -devel when building -devel
if [ "$ENABLE_MAEMO_DEVEL" = "yes" ]; then
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution}-devel main contrib non-free"
fi

if [ "$ENABLE_MAEMO_EXPERIMENTAL" = "yes" ]; then
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution}-experimental main contrib non-free"
fi

# Pull in deps from the main repo when building extras
if [ "$release" = "extras" ]; then
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/leste ${distribution} main contrib non-free"
fi

# Pull in deps from main repo when building for Debian
if [ -n "$PARENT_DISTRO" ]; then
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/leste ${PARENT_DISTRO} main contrib non-free"
fi

# Device specifics
backports_jobs="mesa"
#n900_jobs="libmatchbox2-n900 hildon-desktop-n900"

echo "job_name: $JOB_NAME"
case "$JOB_NAME" in
*-source)
_curpkgname="$(echo $JOB_NAME | sed 's,-source$,,')"
;;
*-binaries/*)
_curpkgname="$(echo $JOB_NAME | cut -d '/' -f1 | sed 's,-binaries$,,')"
;;
*-repos/*)
_curpkgname="$(echo $JOB_NAME | cut -d '/' -f1 | sed 's,-repos$,,')"
;;
*)
_curpkgname="irrelevant"
;;
esac

if echo "$backports_jobs" | grep -qw "$_curpkgname"; then
case "$distribution" in
ascii*|stretch*|beowulf*|buster*)
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb ${MIRROR} ${distribution}-backports main contrib non-free"
;;
esac
fi

#if echo "$n900_jobs" | grep -qw "$_curpkgname"; then
# REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution} n900"
#fi

droid4_jobs="pvr-omap4"
if echo "$droid4_jobs" | grep -qw "$_curpkgname"; then
REPOSITORY_EXTRA="${REPOSITORY_EXTRA},deb http://maedevu.maemo.org/${release} ${distribution} droid4"
fi

network_jobs="pine64-uboot zkgroup signald anbox-image"
if echo "$network_jobs" | grep -qw "$_curpkgname"; then
PBUILDER_USENETWORK=yes
fi

# Comma separated list of URLs with keys for extra repositories.
# Usage example:
# REPOSITORY_EXTRA_KEYS='http://foorepo.domain.com/key1.gpg,http://barrepo.domain.com/key2.gpg'
# Default:
# REPOSITORY_EXTRA_KEYS=
REPOSITORY_EXTRA_KEYS='http://maedevu.maemo.org/testing-key.asc,http://maedevu.maemo.org/extras-key.asc'

# autopkgtests are executed automatically inside the cowbuilder environment by
# default, corresponding to the ADT='internal' setting or if ADT is unset.
# To execute autopkgtests also externally via adt-run you need to set ADT to
# either 'external' (no internal runs) or 'all' (internal + external runs).
# The external autopkgtests depend on a configured ADT_RUNNER setting.
# To skip internal as well as external autopkgtests set ADT='skip'.
# Usage example (see adt-virt-qemu(1) for details regarding image setup):
# ADT='external'
# ADT_RUNNER="adt-virt-qemu /srv/adt-${distribution:-}-${architecture:-}.img"
# Default:
# ADT=

# Control usage of eatmydata to speed up builds. If eatmydata is present on the
# host system and when building for a recent distribution (Debian/jessie +
# Ubuntu/vivid or newer) eatmydata is enabled automatically.
# To force its usage (skipping any host + distribution checks) set it to 'true'.
# To disable its usage set it to 'false'.
# USE_EATMYDATA=true

# Enable ccache (compiler cache for fast recompilation of C/C++ code)
# to speed up builds.
USE_CCACHE=true

# Specify the format (as accepted by the 'date' command) for the generated
# build timestamp.
# Default: %Y%m%d%H%M%S
# TIMESTAMP_FORMAT="%Y%m%d%H%M%S"

# In case you are explicitly marking your new package version as "UNRELEASED"
# inside the debian/changelog file, you may not get a package build version
# number containing the SCM commit. Set this to 'true' in order to work around
# this.
# Default: false
# UNRELEASED_APPEND_COMMIT=false
</pre>