CircleCI Ideas

Provide the capability to source / run a file before every RUN step

In CircleCI 1.0 you could do this by adding commands to the .circlerc file. This was great - because it was driven by CirclCI itself and didn't rely on shell specific tricks (e.g. like $BASH_ENV). In CircleCI 2.0 this capability is not present and there is currently no way to do this for the official Alpine docker images. You have to include your step(s) in every single run command.

We have large modular build and deploy scripts with many run commands and having this capability keep things simple and follows DRY principals.

It's also a very flexible feature

- People can easily carry environment variables across step (or even set them from within a step and have them available to other steps)

- You can setup you shell the way you need to before each run step

More detail here:

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • Sep 12 2018
  • Taking votes
  • Attach files
  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    15 Sep, 2018 04:54pm

    [Additional note for item 3] You also can't force the shell command to bash at the job level, because it's not yet present before the first run step in which you actually install bash.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    15 Sep, 2018 04:50pm

    Nathan, this is great functionality! I think it will be useful in many ways. I do think this makes things a little easier. However, I still hold to the idea that there should be a known file sourced with every container - there are just so many uses for it - and frankly, it's the most simple way to solve this (and several other) problems.

    In the research and testing I've done - I've identified several possible solutions to this:

    1. Bring back the concept of a file that is sourced every time a container is launched (e.g. the .circelcirc approach).
    2. Set $ENV=$BASH_ENV in the parent environment, this fixes the issue for Alpine images (because ash sources the file pointed to by $ENV). However, due to the lack of variable interpolation in circle.yml, there is no way to set this.
    3. Update this functionality here:
      • "The default value of shell option is /bin/bash -eo pipefail if /bin/bash is present in the build container. Otherwise it is /bin/sh -eo pipefail. The default shell is not a login shell (--login or -l are not specified by default). Hence, the default shell will not source your ~/.bash_profile, ~/.bash_login, ~/.profile files. Descriptions of the -eo pipefail options are provided below."
      • This appears to only check for the presence of bash one time (at the very beginning). For example if you use an Alpine image (which doesn't have bash by default), and immediately install bash as your first step, it will still use /bin/sh -eo pipefail on subsequent containers. If it checked for the presence of bash every time it spun up a container, that would also fix this issue.
  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    12 Sep, 2018 05:40pm

    You can read more about the new 2.1 features here:

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    12 Sep, 2018 05:38pm

    Curious if you'd be OK using the new 2.1 config (currently in preview) that has a new `commands` key where you can define reusable steps that take parameters. One of the parameter types is "steps" that lets you pass a sequence of steps to the command and then reference them in a flow of other steps. In that way you could create a command that does whatever think you want to do before a series of steps, then pass the steps you want to run. Using that technique it would end up looking something like this:


    type: steps
    - run: echo "whatever you want to do here"
    - steps: << parameters.steps >>

    image: alpine:latest
    - prefixed:
    - run: "some step"
    - prefixed:
    - run: "some other step"