CircleCI Ideas

Support multiple files in .circleci folder

We are migrating to a mono-repo setup for a number of previously separate codebases, and our CircleCI config file has grown to many hundreds of lines. Now that config is within a `.circleci/` directory, is there an opportunity to split up the config file into multiple pieces to make it more comprehensible?

 

There was a similar feature request files here previously: https://discuss.circleci.com/t/support-multiple-files-in-circleci-folder/18705

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • Feb 23 2018
  • Taking votes
  • Attach files
  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    14 Apr 02:58am

    I recently wrote about an easy way to split your yml file here: https://blog.roopakv.com/breaking-up-circleci-yml/

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    7 Apr 09:49am

    I'm currently doing this with the following structure and script (doesn't allow aliases atm):
    .circleci/
    ci/
    commands/
    jobs/
    main.yml


    #!/usr/bin/env bash
    echo "Packing all YAML files..." && \
    circleci config pack ./.circleci/ci > ./.circleci/config.yml && \
    echo "Done packing." && echo "Validating compiled config.yml" && circleci config validate

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    28 Feb 04:25pm

    Just noticed this related idea: https://ideas.circleci.com/ideas/CCI-I-208

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    28 Feb 04:21pm

    We often will have application code and library code in the same repo. It would be nice to be able to treat these as different workflows in CircleCI. I we commit a change in the library code it would trigger the library workflow and if we commit a change in the application code it would trigger the application workflow. As it is now, we have to separate them into different repositories.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    10 Feb 04:59pm

     I would like this feature as well

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    12 Dec, 2019 01:17pm

    > How about orbs instead?

     

    Being able to simply #include another file (or similar) is miles simpler than using Orbs.  I want to empower everyone on our engineering team to make changes to the build as necessary, but I definitely don't want everyone to have to learn how to interact with Orbs.  Most people just need to make a relatively small change to a shell command in our config.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    3 Dec, 2019 04:26pm

    How about orbs instead?

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    4 Sep, 2019 01:30am

    We recently hit 1000 line config and there is little to no duplication. I would love to see this too.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    26 Jun, 2019 01:27pm

    Yeah, much like Ansible or other "programming  languages" (ugh, hate describing it like this,) could you add in `import` support?

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    17 Jun, 2019 11:27pm

    If anyone wants to do this right now, this repo I just published makes it easy to generate a single config.yml from many, and just uses the circleci-cli tool under the hood to pack and process files. It even lets you use circleci 2.1 syntax, including orbs, on CircleCI Enterprise, by compiling them at dev time down to fully expanded 2.0 syntax: https://github.com/samsalisbury/circleci-multi-file-config 

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    1 May, 2019 09:52pm

    It should be a great enhancement because you may have many complex process happening on ci, so is important to keep those process in order.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    30 Jan, 2019 09:11am

    Just the ability to include other files would be nice. But something like Orbs but just with local & private files would be great. We need to be able to treat pipeline config as proper code.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    22 Oct, 2018 03:44pm

    Same thing happened to us. We’re moving to one repo, but we have five projects running in it. A config file per project would be great, plus a master one. Maybe a file for each job in a `jobs` folder, and then a `workflows.yml` to tie them together.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    23 Aug, 2018 03:33pm

    yeah I would love to see that feature too