jenkins-gitlab-integrator

Server for integration GitLab CE with Jenkins (in last version GitLab (9.x) integration with jenkins enable only in EE version).

Server works only with multibranch pipeline jobs (maybe i will implement non-multibranch pipeline)

How it work:

• every push starts build in jenkins (if the branch does not already exist, then the server will try to start the job until it reaches the limit of attempts or build starts.)
• for each merge request server checks the build and write comment about status

Menu:

• Usage
  • Install requirements libs
  • Configure database schema
  • Configure server
  • Exec server
  • Admin UI
  • App stats
• Development
  • Prepare dev machine
  • Prepare DB
  • Libs
  • GitLab webhook processing
• Thanks

Usage" class="reference-link"> Usage

Install requirements libs" class="reference-link"> Install requirements libs

pip install -r requirements.txt

Configure database schema" class="reference-link"> Configure database schema

Edit alembic.ini , set sqlalchemy.url for connect database

Run migrations alembic upgrade head

vim alembic.ini
alembic upgrade head

Configure server" class="reference-link"> Configure server

config:

host: 0.0.0.0 # listen address
port: 8000 # listen port
log_level: INFO # log level
server_url: http://jenkins-gitlab-integrator.example.local:8000 #server url for gitlab webhooks (need for auto create webhooks)
gitlab_webhook_token: adEleRYsiViB1_34 #token for auth gitlab webhooks (Secret Token, Private-Token header)
#generate secret
#from cryptography.fernet import Fernet
#Fernet.generate_key()
session_secret: N5I6xGINvJ6RigIwd_SX7nHM4J7zYc6ONx6MGH3F__o= # salt for cookies
#user list
users:
  - username: root
    password: root
    permission: [ADMIN_UI]
#mysql connection, same as alembic.ini
mysql:
  db: jenkins_integrator
  host: 127.0.0.1
  user: root
  password: test
  port: 3306
  minsize: 5
  maxsize: 5
#gitlab api config
gitlab:
  url: https://gitlab.example.local #gitlab url
  access_token: adEleRYsiViB #gitlab user api token (look to gitlab docs)
#background workers config
workers:
  enable: yes #enable or disable run workers
  max_attempts: 1400 # how many try for do task
  scan_timeout: 60 # period between task run
jenkins: #settings for jenkins
  user_id: sergei.kravchuk #jenkins user
  api_token: 2342b01c03caaa0465d144e310893ba9 # jenkins api token

Exec server" class="reference-link"> Exec server

Copy config to anower dirs and modify

cp config/alembic.yml /opt/jenkins-gitlab-integrator/config/
cp config/server.yml /opt/jenkins-gitlab-integrator/config/

Run migrations

docker run --rm -v /opt/jenkins-gitlab-integrator/config:/opt/app/config -it alfss/jenkins-gitlab-integrator:latest migrate

Init example data

docker run --rm -v /opt/jenkins-gitlab-integrator/config:/opt/app/config -it alfss/jenkins-gitlab-integrator:latest init_example_data

Start server

docker run -d -v /opt/jenkins-gitlab-integrator/config:/opt/app/config alfss/jenkins-gitlab-integrator:latest

Or you can make docker image with your config in /opt/app/config

Admin UI" class="reference-link"> Admin UI

Admin UI provide:

• management for delayed tasks
• show current config & version
• manage jenkins groups (webhooks)

Go to http://server:port/

App stats" class="reference-link"> App stats

http://server:port/api/v1/stats return json with stats.

{
    "coroutines_run": 2, # current execute coroutines
    "task_in_queue": 0, # count task with status new
    "app_version": "1.0.0",
    "server_url": "http://jenkins-gitlab-integrator.example.local:8000"
}

Development" class="reference-link"> Development

Prepared dev machine" class="reference-link"> Prepared dev machine

apt-get install -y python3-dev python3 python3-venv
#init sandbox
python3 -m venv ~/py-sandbox/py3/server-jenkins-notify-py3
#use sandbox
source ~/py-sandbox/py3/server-jenkins-notify-py3/bin/activate
#install requirements
pip install -r requirements.txt

Prepare DB" class="reference-link"> Prepare DB

#create database
create database jenkins_integrator DEFAULT CHARACTER SET utf8;
vim alembic.ini
alembic -c config/alembic.ini upgrade head

Build UI

cd angular-admin-ui
ng build --watch -d /static/

Run server

python -m server.main -c /path/to/config/server.yml

Run test

make test

Libs" class="reference-link"> Libs

Python Libs

• [https://github.com/aio-libs/aiohttp]
• [https://github.com/aio-libs/aiomysql]
• [http://alembic.zzzcomputing.com/en/latest/]
• [https://pypi.python.org/pypi/aiodns]
• [https://www.sqlalchemy.org/]
• [https://pypi.python.org/pypi/trafaret]
• [http://jinja.pocoo.org/docs/2.9/]
• [https://github.com/aio-libs/aiohttp-security]
• [https://github.com/aio-libs/aiohttp-session]
• [https://pypi.python.org/pypi/cryptography]

UI Libs

• [https://angular.io/]

Gitlab webhook processing" class="reference-link"> Gitlab webhook processing

Push processing

Mege processing

Thanks" class="reference-link"> Thanks

• Alex Zolotyh aazolotyh@gmail.com