CircleCI Ideas

Start multi-container builds as soon as individual containers become available

Some of our build steps use all our containers, other steps only use 1. If the next queued step uses all containers, then all previous steps that only use 1 have to finish before the big step can start. From what I observed, it seems that containers that finished in multi-container builds are immediately released so that 1-container steps can run immediately before all containers finished up. However, this does not happen at the start of a step yet, i.e. if there’s only 1 container available for the all-containers step, it’s still waiting instead of using what is already available.

Pretty much the same question as Start build even if not all parallel containers are available, but for 2.0 instead of 1.0. It makes sense to me that the builds would wait for the required amount of containers before starting in 1.0, since you could e.g. SSH between containers, which requires all other containers to have gotten to the same step in some cases. But in 2.0, since all containers are completely independent, I don’t see why they would wait anymore.

This feature would give us drastically better container utilization since it would completely eliminate idle waiting.

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • Feb 7 2018
  • Taking votes
  • Attach files