A follow up to my previous blog post about CI
Integration of code
When a pull request is created in Azure DevOps (similar things happen in GitHub) the integrated code is available in a special branch in the refs/pull/[pull_request_number] ref spec. If the origin or target branch are updated with new commits, the code in this branch is updated to reflect the new changes. Subsequent CI workflows will also be re-triggered.
You will also note that if CI is triggered by a pull request, the variable $(Build.SourceBranch) in Azure DevOps will look something like: refs/pull/24/merge. If you aren't running a headless job, the first thing ADO does is check out this branch:
When working locally, you can add this ref spec to your .git/config file and make these branches available locally. However for brevity of fetching changes, it's best practice to keep what you fetch to the bare minimum you need to do your work. GitHub says this about how you should do it.
Back on the CI VM, the next step in your YAML file would be to run a command like npm install then run your test suite via jasmine or something: npm test. These would all run locally on the VM and return results to each step. Your Jasmine tests would be running on the merged code between the origin and target branch. We should assume that a good developer would have run tests on the origin branch locally first, before creating the pull request.
But this is not a node project
And ServiceNow has no idea what refs/pull/24/merge is. ServiceNow only tracks branches in refs/heads/ ref spec. We really only have one option here, if we want to checkout the integrated code into a test instance to run ATF, we need to materialise this integration branch into refs/heads/. ADO actually tells us what to do, take another look at the above screen shot and you will see the message:
So that's just what I did:
- script: |
git status
git switch -c $(temp_branch)
git push origin $(temp_branch)
displayName: 'Create temporary real integration branch'temp_branch is set as a variable like so:
temp_branch: $[replace(variables['System.PullRequest.SourceBranch'], 'refs/heads', 'temp')]Note when creating the branch we don't need to write out the full path name, we just want a new branch called temp/[origin_branch_name].
Once we have created the branch in the remote repo, we need the ServiceNow test instance to refresh its local copy. The easiest way to do that is to tell it to checkout the primary branch.
- task: ServiceNow-CICD-SC-Apply@1
inputs:
connectedServiceName: $(test)
appSysId: $(app)
branchName: $(primary_branch)
- task: ServiceNow-CICD-SC-Apply@1
inputs:
connectedServiceName: $(test)
appSysId: $(app)
branchName: $(temp_branch)
- task: ServiceNow-CICD-TestSuite-Run@1
inputs:
connectedServiceName: $(test)
testSuiteSysId: '$(test_suite)'
browserName: 'any'Then we can checkout the integration branch and run ATF. COOL!
So what's the catch? There's always a catch right?
Well kinda. The main reason for testing the integrated code, not just the origin branch of a pull request, is that the target branch may include breaking changes. Imagine that our new code relied on an API from a script include. Since creating the branch, another developer has committed code to the target branch that refactors that API. We haven't made any changes to that API in our commits, so there is no conflict flagged in the pull request. However if we just merged the code as is it may not work as expected because the API has changed. If we only test the origin branch we are not testing these scenarios.
Except there's a catch. All commits that come from ServiceNow, update a file called checksum.txt. If a change is made to the target branch a conflict will be thrown in the pull request asking you to choose which version of the checksum.txt file you want to keep.
I'm not yet sure what the best thing to do in this case is. I suspect that just deleting the contents of the file all together will be ok and that ServiceNow will just re-validate the files to ensure that they are valid XMLs. Still it means that the CI won't be kicked off automatically and it will wait until the issues with the pull request are resolved, requiring manual intervention.
This limits the usefulness of having this part automated. There is probably a way to ignore the checksum altogether and just let ServiceNow take care of when it needs to be updated.
Where to next?
You probably noticed this was only part1.5. I'm still yet to finalise my experiments with the deployment side of CI/CD. AES pipelines presents an opportunity to better integrate into other parts of the SDLC lifecycle including release and change management apps. Stay tuned for part2 when I have had the time to muck around with these pipelines a bit more.
