CircleCI Ideas

'rspec --bisect' support: Extend no_output_timeout max


rspec --bisect is a great tool for combatting test pollution (having one test pollute the test environment via database records, environment changes, etc) by having a systematic way of tracking down order-based failures. Instead of making this a reactive process that you run when your team is seeing these test pollution problems, we like to get ahead of the problem by actively running rspec --bisect on our test suite (via CirleCI) regularly, that way we have a quick path to finding a fix when it's introduced.



'rspec --bisect' takes a while! CircleCI has a configurable no_output_timeout that you can extend to support this, but unfortunately it has a hardcoded max, somewhere in the range of 5 hours IIRC



Extend the no_output_timeout to something larger, please!

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • May 31 2018
  • Not planned
  • Attach files
  • Admin
    Alexey Klochay commented
    2 Jul, 2018 09:22am

    Thanks for your feedback. Currently we are not planning to allow extending the output timeout—we cap it at the current value for security reasons. We would suggest:

    • Modifying the bisect functionality to output something during the run time, possibly even at regular intervals.
    • Possibly running bisect for parts of your test suite—I’m not sure this would make sense as failures might be produced by tests in unrelated parts of the test suite.