CircleCI Ideas

Mask secrets in logs

Secrets may be accidentally printed in the build logs. Travis hide them in the logs masking them as [secret]  every time that it finds a match with the provided secrets. See https://discuss.circleci.com/t/masking-secrets-in-output-logs/22231

  • Avatar32.5fb70cce7410889e661286fd7f1897de Guest
  • May 8 2018
  • Shipped
  • Sep 9, 2019

    Admin Response

    Hi All,

    We're currently beta testing thris feature. If you'd like to sign up to get early access, you can do so here https://forms.gle/e1ucYVaLZJ83Dvw76 . Otherwise, you should see this be automatically enabled over the next few weeks.

    Best,

    George

  • Attach files
  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    28 Jan 06:55pm

    A whitelist should be implemented!

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    25 Nov, 2019 01:13pm

    Circle CI currently masks every occurence of ENV variables value. In our project, we have `RAILS_ENV: test` set in config.yml resulting in all occurrences of `test` being masked from our build output.

    Our test files path looks like `test/models/xxxxxx_test.rb`, so these kind of paths now then become `****/models/xxxxxx_****.rb` making copy pasting the path more difficult and confusing, especially for beginners in our team.

    Could you add some sort of whitelist, or even better mask only env vars set in Circle CI Admin UI and not what is in `config.yml`, since it is a file in a repo, and there is no reason to write secrets there as good practices says that repos should not contain any secret.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    25 Nov, 2019 12:23pm

    It seems that we have an ENV var set in our `config.yml` wich is `RAILS_ENV=test`. And the Circle CI masks very occurrences of `test` in our CI output. I assume this, because the documentation is not clear about what exactly is masked and how to add/remove a masked term.

    So, copy tasting a test file path from Circle CI is now broken and has to be fixed manually.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    13 Sep, 2019 11:05am

    Are some users randomly picked to be tested on this beta feature? Cause I think we are being affected by it, and it's incorrectly masking information that we need to see in the logs!!!!! We submitted an issue ticket yesterday and no one has responded so far :(

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    13 Aug, 2019 05:31am

    This is a really important issue. See this article for example: https://www.zdnet.com/google-amp/article/ci-build-logs-continue-to-expose-company-secrets

     

    I cannot opensource a project of mine because of this issue. In there, I'm using now.sh to deploy my application and the token is exposed (https://github.com/zeit/now-cli/issues/1937).

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    18 Jun, 2019 10:33am
                   
                   

    https://circleci.circleci.com/attachments?attachment_only=true&selected_id=6703816136338172082&single_viewer=false

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    10 May, 2019 06:39pm

    And how should we protect in the meanwhile? this secrets are suppose to be secrets for a reason. IMHO this is a highest security priority

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    10 May, 2019 06:22pm

    We agree that masking the variables in the log output would be a very valuable feature for CircleCI to have. However, the work is currently behind other high-urgency items that are necessary to maintain the stability of our platform. We will consider this feature again in the future.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    9 May, 2019 02:00pm

    IMO it should depend on the configuration because sometimes I want to check whether I passed a correct env.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    26 Apr, 2019 04:23pm

    I'm a bit flabbergasted that this isn't already a thing for CircleCI.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    5 Mar, 2019 04:34pm

    This feature should absolutely be prioritized imho.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    5 Mar, 2019 03:06am

    This is a must have feature.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    20 Feb, 2019 02:09am

    important secrets are been displayed on logs, like on api call to trigger new circle jobs

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    18 Dec, 2018 01:37pm

    Many companies  can't use this service because of absence of this feature. Organizations with lots of new engineers can't protect themselves against possible threats that printing of environment variables may cause. 

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    12 Dec, 2018 02:39pm

    Accidentally used `env` while debugging, can't find any plausible way to clear build log. Definitely should be masked.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    14 Nov, 2018 11:36am

    From CircleCI docs about setting environment variables:

    The value of the variables are neither readable nor editable in the app after they are set.

    Well... they are readable. How hard is it to fix this bug?

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    18 Aug, 2018 05:38am

    Bitbucket mask the "secret" variables with the variable name. This was actually confusing at first, until I realised what was occurring.  I like a combination of both travis and bitbucket. So, after defining an AWS_SECRET_KEY in the environment variables and checking a box that it's a secret, it would be good to mask with something like [PROJECT-SECRET:::AWS_SECRET_KEY].  If I had several secrets and something was going wrong, I'd want to know which secret was which if I was debugging. Also know where the secret came from would be good, project or org/context etc.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    12 Aug, 2018 04:04am

    This would absolutely be essential for us.

     

    Really hoping to try out Circle as a Travis replacement however I'm contemplating all the extra adjustments I'll need to make. Currently lean pretty heavily on their secret scrubbing, and I like a good set -x in my builds..

    After the adjustments there would be the extra safe guard considerations as an accidental leak would probably go unnoticed and be very bad.

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    8 Aug, 2018 06:13pm

    I've voted for this, but just to add more colour to my vote. Because the reason for this proposal is more than just "write sensible scripts", because it's entirely possible code you are testing also accidentally leaks secrets through its test output. And you could keep going enumerating the different paths through which your secret environment variables get accidentally exposed.

     

    Having the CI system as a backstop, replacing secret values in _all_ output, just seems like the right thing to do therefore. 

  • Avatar40.8f183f721a2c86cd98fddbbe6dc46ec9
    Guest commented
    31 Jul, 2018 01:40am

    I do think the develop team at Circle CI should promote this feature as highest priority for enterprise (pay) users. 

    I tested with other pipelines, such as bamboo, travis ci, bitbucket pipeline, no secrets can be echo in build logs. 

    This feature becomes a block when we plan to move our application to production environment, because the developers can easily get the api keys (such as aws, database, apigee, etc) after change the pipeline from the feature branches. 

  • Load older comments