This is a container environment containing services used by the Linux Foundation Release Engineering team. It is loosely influenced by the work done by openfrontier (author of the Gerrit container).
The primary services are:
- Gerrit
- Jenkins
- Nexus
And the secondary services that support these:
- OpenLDAP
- NGINX
- MariaDB [TODO]
- HAProxy [TODO]
Most of the documentation for individual services and how they are configured can be found under the config/ directory, and $SERVICE.env files contain environment variables used by the docker containers.
Add the following to /etc/hosts:
127.0.1.1 jenkins.localhost 127.0.1.2 gerrit.localhost 127.0.1.3 nexus.localhost
Jenkins/Gerrit login: workshop/workshop Nexus login: admin/admin123
Add the following to /etc/hosts:
127.0.1.1 jenkins.localhost 127.0.1.2 gerrit.localhost 127.0.1.3 nexus.localhost
docker-compose up -d
Will bring up an environment containing all the services with authentication backed by LDAP, a simple ci-management repo in Gerrit, and a basic job in Jenkins that verifies commits to the ci-management repo.
Once the environment is up and running, copy your ssh public-key and add it to the workshop user in Gerrit. This can be either be done through the web interface or from the commandline:
./gerrit-auth.sh ~/.ssh/id_rsa.pub
Then you can clone the ci-management repo and modify it to your hearts content:
git clone ssh://[email protected]:29418/ci-management.git
Move into the cloned directory:
cd ci-management
Set the gitreview username:
git config --add gitreview.username "workshop"
And ensure the Change-Id hook exists:
git review -s
Putting up a patchset for review that modifies "*.yaml" files should trigger the ci-management-jjb-verify job and add a -1/+1 Verified vote.
To bring up a single service in the foreground you can use:
docker-compose up $SERVICE
Note: dependent services will still be launched but in the background. Hitting '^C' will stop this service, but not the others.
If a service is backed by a Dockerfile, then changes to the Dockerfile or files under '$SERVICE/' will require rebuilding the container:
docker-compose build
To tear down the environment removing all the volumes, and start from scratch, run:
docker-compose down -v
To run a specific version of one of the services, edit the .env file, and rebuild the containers. For example, to run Jenkins 2.80 set the value in the .env file:
JENKINS_CONTAINER_VERSION=2.80
and run:
docker-compose up -d --build
to rebuild the Jenkins image before launching it.
For other useful docker-compose commands such as logs, see:
docker-compose -h
In order to fully configure both Jenkins and Gerrit, another container 'init' is added as part of the startup to generate ssh keys, create the ci-management repo, configure users, and push the ci-management jobs to Jenkins.
This is done in a weakly idempotent fashion by creating files after the command execute successfuly, so that if the environment is restarted the container doesn't die or modify existing data.
The goal of this project is to have an easily created workshop where releng work can be tested or proof-of-concepts created.
Some examples:
- Jenkins Plugin upgrades
- Gerrit upgrades
- Jenkins and Gerrit-Trigger testing
- Nexus configuration
- Gerrit/LDAP group integration
The following is a list of automation tasks still needed before the environment can be considered stable:
General: - [x] Replace 'sandbox' names with 'workshop' since sandbox was just a
placeholder
- [ ] Setup OpenLDAP over SSL by default
- [ ] Make things more configurable. There are a lot of hardcoded names
- in Groovy scripts which could be pulled from environment variables
- [x] Collapse environment config into single file and add lots of
- comments, so users don't need to track down the correct file
Nexus: - [ ] Configure Nexus to use LDAP (admin/admin123, or LDAP) - [x] Setup and configure Nexus
- [x] Create 'logs' Nexus site repo.
Gerrit: - [ ] Remove postgres container configuration and replace with MariaDB
(or make optional)
Jenkins: - [ ] Fix (on Jenkins restart):
WARNING: Caught exception evaluating: instance.hasExplicitPermission(attrs.sid,p) in /configureSecurity/. Reason: java.lang.NullPointerException
- [ ] Make Groovy scripts Idempotent
- [x] Set Markup Formatter to HTML Output
- [x] Add LOGS_SERVER, SILO, NEXUS_URL, JENKINS_HOSTNAME
- [x] Create XML config file 'jenkins-log-archives-settings' (depends on credentials)
- [x] Install environment injector plugin
- https://wiki.jenkins.io/display/JENKINS/EnvInject+Plugin
- [x] Install plugin for build description
- https://plugins.jenkins.io/description-setter
- [x] Manually install postbuildscript.hpi
- http://mirrors.jenkins-ci.org/plugins/postbuildscript/0.17/postbuildscript.hpi
Init: - [ ] Make steps strongly idempotent (verify the state they modify)