Show test results for prospective merge of a Github PR
complete
J
Joseph
Github PRs show if the PR branch has conflicts with the base branch.
CircleCI shows test results on the PR branch.
What is missing is the ability to see whether test results fail when the PR branch is merged into the base branch.
Admittedly, it’s not everyday that PRs break tests of other PRs, but it does happen - for example when there’s a change in UI text:
A feature developer issues PR A testing for some specific text in the UI: “The status is FOO”. PR A contains a view test checking for the existence of “The status is FOO”.
A another developer issues PR B that implements a product decision to change all uses of “FOO” to “BAR”. Github says branch is clear to merge
PR B is merged (Github says there are no conflicts; CircleCI says tests pass)
PR A is merged, without problem (Github says there are no conflicts; CircleCI says tests pass)
The base branch now has a test which fails because the application produces a view with the text “The status is BAR”, but the test in PR A expects “The status is FOO”.
Fail.
All of the ways of dealing with this currently seem bad:
Constantly merge into the PR branch, making a mess of the commit history
Rebase the PR branch onto the base and force push onto Github (a no no for obvious reasons)
Catch the test failures later in the base branch (inconsistent with gitflow and other branching strategies)
A feature would be nice, but a work around would be great too.
CCI-I-926
Samuel Parkinson
The https://github.com/CircleCI-Archived/build-tools-orb repository was archived in November 2022.
I would like to checkout the
origin/pull/:pull_request_id/merge
ref in CircleCI, what's the best way to do so?Ideally I'd like to pass the
checkout
step a ref
argument to achieve this.B
Bailey Michael
Liya Ai
merge-with-parent
doesn't exactly implement this request... - Merge with parent, always merges with the parent branch, which may be different than the destination branch of the PR. Desired behaviour is to call the github api to find the destination branch of the PR being built.
- it only merges AFTER the config.yml is parsed... if the un-merged config.yml references files that are changed, moved or deleted as a result of the merge, things may fail. Desired behaviour is to merge before parsing the config.yml
merge-with-parent
-> Drew Ayling
Bailey Michael:
In regards to point 1, I have created an issue on the build-tools-orb repo to request the destination branch from github as an option to the step. For now, the destination branch is still a parameter that you can set beyond the parent branch i.e. you can set it to your default branch of ‘main’ or ‘master’ if all of your PRs are going to default (not ideal if some PRs don’t intend to do this).
Regarding point 2, this would be a significant feature to add. Since CircleCI only processes the configuration file and you choose whether or not to check out the code as part of the CI job, there’s no “merge-ability” before we process the configuration file. I think the most straightforward solution to this problem is to ensure that the PR is in-sync and up-to-date with the destination branch.
Federico Zivolo
Drew Ayling: Can this request get reopened? What you implemented is not what was asked.
Liya Ai
Federico Zivolo / Bailey Michael - Drew is no longer at CircleCI. I'll be bringing the idea to our team and will share updates as I have them, thank you!
Hector Zhao
Liya Ai: Most other CI tools we looked at have this feature, please bump this idea up again :)
Hector Zhao
Liya Ai: Also
merge-with-parent
is very broken and doesn't do what exactly what we want.https://discuss.circleci.com/t/how-to-retrieve-a-pull-requests-base-branch-name-github/36911 This is a previous post of how to implement this feature. I reckon it's pretty easy to do. It would be much preferable for CircleCI to implement this than we using it directly as we would have to expose our own github access token to CircleCI.
Liya Ai
Hector Zhao: Hi Hector - Thanks for the feedback. I've re-bumped this to our Product Team & will post here if there are any updates.
L
Liya Ai
complete
Hi Everyone! In case you missed Gabriel's note, this feature has been merged into the build-tools orb. https://github.com/CircleCI-Public/build-tools-orb You can learn a little more about orbs here: https://circleci.com/orbs
Michael Prentice
@liya-ai: I guess that this is
merge-with-parent
as listed on https://circleci.com/developer/orbs/orb/circleci/build-tools?Michael Prentice
And for anyone else looking, there are examples of this here:
Liya Ai
Michael Prentice: Yes!
Merged in a post:
PRs should be built against the temporary merge with target branch
D
Daniel Lahyani
When building a pull-request there are scenarios where the build and tests passed successfully, everything may seem to be perfect for merging and yet merging the pull-request can break the build.
This can happen if the PR's branch does not stem from the HEAD of the target branch, in such case it is possible that the PR's branch will pass tests while merging it will break the build. Or if we have 2 PR's that introduce conflicting changes, in such case they both will pass the build but once one is merged to the target branch merging the other one will break the build.
To minimize / prevent the undesired state of broken builds the PRs build should not run on the PR's branch HEAD commit but on the a "fictive" commit that simulates the merging of the PR's branch into the target branch. Luckily this can be easily achieved as this "fictive" commit is provided by GitHub (and other SCM providers), on checkout simply pulling the branch +refs/pull/${CIRCLE_PR_NUMBER}/
merge
instead of +refs/pull/${CIRCLE_PR_NUMBER}/head.There are few feature request posts (listed below) talking exactly about that for more than 2 years already, and this is a feature is really necessary to ensure stable builds. It is supported by other CI services (eg. Travis).
References:https://discuss.circleci.com/t/prs-should-also-build-the-temporary-merge-with-target-branch/10706
CCI-I-431
G
Gabriel Simmer
This has been merged into the build-tools orb! https://github.com/CircleCI-Public/build-tools-orb You can learn a little more about orbs here: https://circleci.com/orbs
N
Ned Twigg
Example. This PR is straightforward and passed CI, so I merged it: https://github.com/diffplug/spotless/pull/642But now main is broken. That means that some of the commits which are important to the PR are missing from the PR, and that is bad. I would not have switched to CircleCI if I realized this was missing.
A
A J
You can achieve this with bors bot: https://bors.tech/ , just give it access to your repo and do bors r+ and it should create a new branch, by default named trying and staging and merge multiple open pr (by commenting bors r+ ) and it will be picked up by circleci and the branch's overall pr status would be flagged (and if all passes, would be merged into master)
S
Some User
Bumping this after 361 days. All other CI systems have this.
C
Celestine Kao
If you're using GitHub as your VCS, the ghpr orb has a command that builds a branch of the prospective merge.
B
Breno Ribeiro
https://ideas.circleci.com/ideas/CCI-I-894 would solve this for me.Voted on both ideas. :)
Load More
→