Overview

Design of CI configuration

  • Script
    • Bourne shell script
    • PowerShell script

  • Script should be source-able using the same script for every CI service (because generating multiple scripts means changing those scripts). Then the script in turn should do something different based on which CI is being used (detected using the environment variables).

  • Steps for CI should have information on how to retrieve modules/distributions. For example, inside a monorepo this is a directory.

Caching

  • Caching inside of a single CI system: this can be used for artifacts that are created internally to the build that can be rebuilt as long as inputs do not change.
    • e.g., for a given repo, the dependencies installed from CPAN

  • Caching outside of the CI system: some build artifacts can be stored outside of the CI caching mechanism so that they can be shared across repos.

    1. Given
      1. Projects { P_A, P_B }
      2. Tasks { Task₀, Task₁ }
      3. OS os and architecture arch (determined by what the workflow is running on); (ignoring APIs/ABIs for now)
    2. VersionString(P_A) = V_A, VersionString(P_B) = V_B
      1. e.g., using git describe --always
    3. Task₀(P_A) has output A_O₀{V_A, os,arch}.
    4. Where Task₁(P_B) running on {os,arch} depends on A_O₀{V_A,os,arch}
      1. V_A for P_B is determined by turning a project dependency such as P_B has-runtime-dependency-on (P_A where P_A source is git:A @ master) into a concrete dependency.

    Specifically on GitHub:

    • Workflow running on every push, pull request:
      1. We can store A_O₀{V_A,os,arch} as a GitHub release under tag continuous-build when finished running Task₀(P_A) under the CI workflow for P_A for the default/release branch (e.g., master).
      2. When running Task₁(P_B) under the CI workflow for P_B, retrieve A_O₀{V_A,os,arch} from GitHub releases if it exists. Otherwise build A_O₀{V_A,os,arch} inside of the CI workflow.
    • Workflow running on every tag creation:
      1. When running Task₀(P_A) under the CI workflow for P_A, delete all A_O₀{Version,Os,Arch} where Version < VersionString(P_A).

Multiple version testing

For example, say you want to test a binding library B that binds to library A (e.g., FFI-based binding).

  • You can build and install all different versions of A in parallel into different folders.
  • You only need to build the dependencies of library B once since they do not depend on library A.
  • You build library B with a different path to test each version.

Examples

  • Travis CI: 
     
install: skip
script: eval "$(curl .../bootstrap.sh)"
  • AppVeyor: 
     
build: off
test_script:
  - ps: . { iwr -useb .../bootstrap.ps1 } | iex
  • GitHub Actions: 
     
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run Orbital Transfer
        run: |
          eval "$(curl .../bootstrap.sh)"

Services

GitHub Actions

Actions:

Examples:

Travis CI

AppVeyor

GitLab CI

Cirrus CI

Common features