Making A Multi-Module Project Build And Test In CircleCI
Finally :
The problem is that the unit tests for sa-store have never worked under circleci because it requires os-details. And sa-explorer would not build and test under circle because it requires sa-store to be built.
I eventually solved this problem by moving to circleci 2 and creating a multi build workflow.
To help with debugging a script can be created locally that submits the job. Like :
#!/usr/bin/env bash
curl --user ${CIRCLE_TOKEN}: \
--request POST \
--form revision=6a3537d0c503cfe7409442c25f02f752a07c838b\
--form config=@config.yml \
--form notify=false \
https://circleci.com/api/v1.1/project/bitbucket/sbnlocean/<project>/tree/master
Make sure ti create a token and save it locally, export to CIRCLE_TOKEN.
The revision is the commit to build.
This then allows the job to be run without having to commit on each config.yml change.
Eventually I settled on a three stage build for sa-explorer. The final stage uses doo to test in a headless chrome environment. This requires npm to be available which in turn requires nodejs. Also, since I'm testing with chrome, the image needs chrome, hence the circleci/clojure:lein-2.8.1-node-browsers image in step 3.
version: 2
jobs:
os-details:
working_directory: ~/os-details
docker: # run the steps with Docker
- image: circleci/clojure:lein-2.7.1
steps:
- run:
name: checkout-os-details
command: git clone https://sbnlocean@bitbucket.org/sbnlocean/os-details.git .
- restore_cache:
key: os-details
- run: lein do install, deps
- save_cache: # generate and store cache in the .m2 directory using a key template
key: os-details
paths: ~/.m2
sa-store:
working_directory: ~/sa-store
docker: # run the steps with Docker
- image: circleci/clojure:lein-2.7.1
steps:
- restore_cache:
key: os-details
- run:
name: checkout-sa-store
command: git clone https://sbnlocean@bitbucket.org/sbnlocean/sa-store.git .
- run: lein do install, deps
- save_cache: # generate and store cache in the .m2 directory using a key template
key: sa-store
paths: ~/.m2
build:
working_directory: ~/sa-explorer
docker: # run the steps with Docker
- image: circleci/clojure:lein-2.8.1-node-browsers
steps:
- checkout
- restore_cache:
key: sa-store
- run:
name: install-karma
command: |
npm install karma --save-dev
npm install karma-jasmine karma-chrome-launcher jasmine-core --save-dev
sudo npm install -g karma-cli
- run: lein deps
- save_cache: # generate and store cache in the .m2 directory using a key template
key: sa-explorer
paths: ~/.m2
- run: lein doo chrome doo once
workflows:
version: 2
build-and-test:
jobs:
- os-details
- sa-store:
requires:
- os-details
- build:
requires:
- sa-store
It seems that there must be a job called build present otherwise an error is reported.
I also used the yaml parser at https://yaml-online-parser.appspot.com/ to help get the formatting right. This is the second project where yaml layout has become an enormous pain.
Note that git clone uses https not ssh because ssh will require manual confirmation of the bitbucket address and this can't be automated.
Note that the config must be .yml not .yaml and must be called config not circle. Also, if circle can't find a config it will make one up. If you dont include a workflow into the config it will inject one giving some strange errors.