CircleCI Ideas

Mask secrets in logs

Secrets may be accidentally printed in the build logs. Travis hide them in the logs masking them as [secret]  every time that it finds a match with the provided secrets. See https://discuss.circleci.com/t/masking-secrets-in-output-logs/22231

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • May 8 2018
  • Shipped
  • Sep 9, 2019

    Admin Response

    Hi All,

    We're currently beta testing thris feature. If you'd like to sign up to get early access, you can do so here https://forms.gle/e1ucYVaLZJ83Dvw76 . Otherwise, you should see this be automatically enabled over the next few weeks.

    Best,

    George

  • Attach files
  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    May 21, 2018 18:06

    ran into this today and exposed an aws secret key, expecting behavior of other ci systems. bitbucket pipelines for example protects against this.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    July 31, 2018 01:40

    I do think the develop team at Circle CI should promote this feature as highest priority for enterprise (pay) users. 

    I tested with other pipelines, such as bamboo, travis ci, bitbucket pipeline, no secrets can be echo in build logs. 

    This feature becomes a block when we plan to move our application to production environment, because the developers can easily get the api keys (such as aws, database, apigee, etc) after change the pipeline from the feature branches. 

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    August 08, 2018 18:13

    I've voted for this, but just to add more colour to my vote. Because the reason for this proposal is more than just "write sensible scripts", because it's entirely possible code you are testing also accidentally leaks secrets through its test output. And you could keep going enumerating the different paths through which your secret environment variables get accidentally exposed.

     

    Having the CI system as a backstop, replacing secret values in _all_ output, just seems like the right thing to do therefore. 

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    August 12, 2018 04:04

    This would absolutely be essential for us.

     

    Really hoping to try out Circle as a Travis replacement however I'm contemplating all the extra adjustments I'll need to make. Currently lean pretty heavily on their secret scrubbing, and I like a good set -x in my builds..

    After the adjustments there would be the extra safe guard considerations as an accidental leak would probably go unnoticed and be very bad.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    August 18, 2018 05:38

    Bitbucket mask the "secret" variables with the variable name. This was actually confusing at first, until I realised what was occurring.  I like a combination of both travis and bitbucket. So, after defining an AWS_SECRET_KEY in the environment variables and checking a box that it's a secret, it would be good to mask with something like [PROJECT-SECRET:::AWS_SECRET_KEY].  If I had several secrets and something was going wrong, I'd want to know which secret was which if I was debugging. Also know where the secret came from would be good, project or org/context etc.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    November 14, 2018 11:36

    From CircleCI docs about setting environment variables:

    The value of the variables are neither readable nor editable in the app after they are set.

    Well... they are readable. How hard is it to fix this bug?

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    December 12, 2018 14:39

    Accidentally used `env` while debugging, can't find any plausible way to clear build log. Definitely should be masked.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    December 18, 2018 13:37

    Many companies  can't use this service because of absence of this feature. Organizations with lots of new engineers can't protect themselves against possible threats that printing of environment variables may cause. 

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    February 20, 2019 02:09

    important secrets are been displayed on logs, like on api call to trigger new circle jobs

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    March 05, 2019 03:06

    This is a must have feature.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    March 05, 2019 16:34

    This feature should absolutely be prioritized imho.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    April 26, 2019 16:23

    I'm a bit flabbergasted that this isn't already a thing for CircleCI.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    May 09, 2019 14:00

    IMO it should depend on the configuration because sometimes I want to check whether I passed a correct env.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    May 10, 2019 18:22

    We agree that masking the variables in the log output would be a very valuable feature for CircleCI to have. However, the work is currently behind other high-urgency items that are necessary to maintain the stability of our platform. We will consider this feature again in the future.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    May 10, 2019 18:39

    And how should we protect in the meanwhile? this secrets are suppose to be secrets for a reason. IMHO this is a highest security priority

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    June 18, 2019 10:33
                   
                   

    https://circleci.circleci.com/attachments?attachment_only=true&selected_id=6703816136338172082&single_viewer=false

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    August 13, 2019 05:31

    This is a really important issue. See this article for example: https://www.zdnet.com/google-amp/article/ci-build-logs-continue-to-expose-company-secrets

     

    I cannot opensource a project of mine because of this issue. In there, I'm using now.sh to deploy my application and the token is exposed (https://github.com/zeit/now-cli/issues/1937).

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    September 13, 2019 11:05

    Are some users randomly picked to be tested on this beta feature? Cause I think we are being affected by it, and it's incorrectly masking information that we need to see in the logs!!!!! We submitted an issue ticket yesterday and no one has responded so far :(

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    November 25, 2019 12:23

    It seems that we have an ENV var set in our `config.yml` wich is `RAILS_ENV=test`. And the Circle CI masks very occurrences of `test` in our CI output. I assume this, because the documentation is not clear about what exactly is masked and how to add/remove a masked term.

    So, copy tasting a test file path from Circle CI is now broken and has to be fixed manually.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    November 25, 2019 13:13

    Circle CI currently masks every occurence of ENV variables value. In our project, we have `RAILS_ENV: test` set in config.yml resulting in all occurrences of `test` being masked from our build output.

    Our test files path looks like `test/models/xxxxxx_test.rb`, so these kind of paths now then become `****/models/xxxxxx_****.rb` making copy pasting the path more difficult and confusing, especially for beginners in our team.

    Could you add some sort of whitelist, or even better mask only env vars set in Circle CI Admin UI and not what is in `config.yml`, since it is a file in a repo, and there is no reason to write secrets there as good practices says that repos should not contain any secret.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    28 Jan 18:55

    A whitelist should be implemented!