rs-aggregator

Aggregate ResourceSync Sources

• The component in this repository is intended for system administrators and developers.
• Source location: https://github.com/EHRI/rs-aggregator.
• Documentation: https://ehri.github.io/rs-aggregator/.
• In case of questions contact the EHRI team.

The Destination in a
ResourceSync Framework configuration keeps zero or
more sets of resources from zero or more Sources synchronized.
See also: Definitions.

rs-aggregator can be used, out-of-the-box, as such a Destination that aggregates sets of resources
from a list of Sources.

Docker quick start

1. Clone or download this repository to your local drive.

2. Start a Docker daemon (if it is not already running) switch to the
   docker directory and run the start-script.

   cd rs-aggregator/docker
   ./start.sh

   If you see the rs-aggregator logo…

then you have just built a Java8 capable docker container, imported required libraries,
compiled, tested and packaged the source code and started the rs-aggregator application.
To gracefully stop the aggregator (without interrupting a synchronisation run)
you can run the stop-script.

./stop.sh

The aggregated resources are in docker/destination, grouped by host name of the Sources.
Sources are synchronized per set of resources described by a capabilityList.
Each set of resources will be in a subdirectory contingent to the path that led to the
corresponding capabilityList. In this base directory you will find:

• __MOR__ a directory containing the metadata: the capabilityList and its child-sitemaps.
• __SOR__ a directory containing the set of resources.
• __SYNC_PROPS__ a directory containing a report for each synchronisation run in the
  form of an xml-properties file.

Quick configuration

The configuration files are in cfg. When running the Docker container rs-aggregator will
look in docker/cfg.

CapabilityList URI’s

cfg/uri-list.txt | docker/cfg/uri-list.txt

Each set of resources is denoted by a distinguished capabilityList. The file
cfg/uri-list.txt should contain a list of URI’s pointing to capabilityLists you
want to follow. This list is read each time before a synchronisation run, so changes
you make to the list are hot-deployed: you do not need to restart the aggregator.

Application context

cfg/syncapp-context.xml | docker/cfg/syncapp-context.xml

The Spring configuration file. At this moment notable configuration details are:

• job-scheduler
  • See for available schedulers the package nl.knaw.dans.rs.aggregator.schedule.
  • See for available timer options the description on properties of the bean.
• resource-manager
  During synchronisation the nl.knaw.dans.rs.aggregator.sync.SyncWorker and companions
  are doing all the heavy lifting, while The nl.knaw.dans.rs.aggregator.syncore.ResourceManager
  gets relatively easy to accomplish and simple tasks. An implementation of
  resource-manager is dedicated to a specific storage system for the aggregated
  resources. At this moment only the nl.knaw.dans.rs.aggregator.sync.FsResourceManager
  is available, aimed at storing resources on the file system.