Local orbs in seperate yml files.
T
Toby Pinder
Orbs are great as reusable components but also considerably tidy up individual components as part of single, complicated build setups.
Currently there is a way to do local, "private" orbs "locally" by including the orb configuration under a key in
.circleci/config,yml
. However this still means we must maintain a giant singular yml file which can grow to thousands of lines potentially with all the workflows etc.A "simple" solution for local orbs would be if circleci read additional .yml files in
.circleci/orbs/*
for a given project and merged the contents under the orbs
key.This effectively means that the yml contents of
.circleci/orbs/hello.yml
would be equivalent to defining the same content under orbs: hello:
This allows circleci build components to be worked on in modular fashion and provides a step between a local orb that is "inline" and one that must be maintained completely externally from the project.
CCI-I-704
F
Francesco Bianco
My solution is to use a linker eg. https://github.com/javanile/yamlinc that create a one large YAML file by inclusion. $include is the tag.
J
Jason Hoos
Another option to consider: allow the "orbs" section to specify a reference to a URL that the orbs can be read from (so that private orbs could be hosted in S3, for example), or to a Docker image (and a path to a file within that image) that the orbs can be read from. This would make it easy for organizations to host private orb repositories, without running into the API rate limit you mentioned. It would also allow for easy sharing of orbs across private organization repositories.
We'd really like to be able to maintain a set of common orbs across all of our projects, but for various compliance reasons we can't just put them where they are publicly visible, so hosting them in the public repository isn't an option for us.
N
Nathan Dintenfass
We agree this would be useful, and we are working through approaches here. One of the reasons we didn't do this originally was that pulling lots of files before your jobs run risks hitting API rate limiting problems with GitHub (et al.) because we'd be pulling an arbitrary set of files via the API. That said, our current thinking is to create a mechanism where you can execute a job that can dynamically decide what the configuration should be -- with that capability doing
circleci config pack
to conjure your configuration from a decomposed orb (or set of orbs) will become fairly trivial to do yourself.J
Juan Carlos Medina Medina
To take it a step further, I think the
.circle/config.yml
file should always be processed with circleci config pack
which would allow us to even split large workflows and jobs into their own files. /2cents