🔥Let’s Do DevOps: GitHub Action — Check if Each Commit Contains Valid Jira Ticket ID
🔥Let’s Do DevOps: GitHub Action — Check if Each Commit Contains Valid Jira Ticket ID
This blog series focuses on presenting complex DevOps projects as simple and approachable via plain language and lots of pictures. You can do it!
Hey all!
I’ve been running a project in my enterprise to migrate all our code from an internal BitBucket server to GitHub. This BitBucket server is older, and uses a great deal of modules from the community to support all sorts of cool bespoke features. I’ve been doing my best to replicate those features in GitHub using a mixture of ingenuity and Actions.
One of those features is called the Yet Another Commit Checker. It integrated with BitBucket to validate any commit pushed to the server contains a reference to a valid Jira ticket, and works well.
The YACC (an amazing short name for this module) did all this validation pre-commit push to the server. So if a commit wasn’t valid, the git push was rejected and a message (with an adorable teddy bear) was printed to the terminal.
GitHub Cloud (different from GitHub server, which lives within your org) doesn’t support pre-commit checks. They fear they’d be DoS’d by folks sending them hundreds of thousands of commits, and the asymetric validation required on their side would down their services.
So we’re stuck with validating post-push. That’s okay, folks will need to rebase their PRs, rather than amending their commits locally. No big deal.
Let’s talk about how the Action works and I’ll share what we built. Scroll to the end of this article for a link to the Action itself if you want to borrow it for your own Org.
NOTE: This is an actual GitHub Action in the Marketplace, my first one! I hope it’s useful to you :) ❤
Temp Measure: Regex!
As a temporary measure, I created an Action that can live in each repo and validate post-commit-push that each commit matches a regex for how our tickets look. Link:
Let’s Do DevOps: Commit Regex Validation with GitHub Actions
This blog series focuses on presenting complex DevOps projects as simple and approachable via plain language and lots…faun.pub
However, that quickly had scaling issues. The Action was deployed to every single repo, rather than published as an Action in our Org, and it matched a finite set of Jira ticket project names. As we built more projects, each Action needed updating in every repo.
Let’s Make an Org-Wide Action — Action Code
Now, we can’t actually go to our Org settings and say “use this action everywhere”. I desperately wish we could, but we can’t.
However, we can put all the code in a single place and publish it as an Action within our Org. That way all the repos can just say “run that Action” and as we discover bug-fixes and develop this Action, we only need to make changes in one place. Let’s do that first.
First we need to define the variables. Jira uses a username and API token to call information from it, so I created those in our Jira instance. The caller would need to send that information to this Action.
We could potentially have this Action access those secrets, but I wanted to call the Action from other Orgs in our enterprise also, and was concerned that inter-Org calling would prevent the Action from accessing Org secrets. So I require we send them to this Action.
We also need a “base branch” — we only validate the commits since the PR branch was cut from the target branch. I don’t want to check all the commits forever, that would take forever and do us no good. Note how each input can be set as required or not. In this case, they all are — I can’t derive any defaults for this Action.
| name: 'Check Jira for ticket exist' | |
| description: 'I validate that a list of tickets exist in PF Jira' | |
| inputs: | |
| jira-username: | |
| description: 'Username used to poll Jira for ticket information' | |
| required: true | |
| jira-api-token: | |
| description: 'API Token used to poll Jira for ticket information' | |
| required: true | |
| BASE_BRANCH: | |
| description: 'Base branch' | |
| required: true |
Then we start the Action core code — it’s written in bash. On line 7–10, we assign “input” values (those sent to the Action) and map them to local bash variables.
Then on line 13, we create a regex static commit list. Some commits don’t match a Jira ticket — those that are generated by GitHub to merge pull requests, and branches, and some tools that use a commit message we want but doesn’t match our standard.
| runs: | |
| using: "composite" | |
| steps: | |
| - id: trigger_jenkins_job_using_api | |
| run: | | |
| #Parameters | |
| jira_username="${{ inputs.jira-username }}" | |
| jira_api_token="${{ inputs.jira-api-token }}" | |
| BASE_BRANCH="${{ inputs.BASE_BRANCH }}" | |
| # Some commits don't have valid Jira ticket leads, like those built by whitesource or github | |
| # This list permits some commits to show as valid and not poll against Jira | |
| commit_static_permit_list="(^Merge pull request \#)|(^Merge branch)|(^Revert \")" | |
| # Initialize invalidTicket as false, will be set to true by any non-exist tickets | |
| invalidTicket=false |
Then we set the current branch using git metadata (line 2) and have git tell us the has of the commit where our PR branch diverged from the target (base) branch (line 5).
Then (line 8) we get a list of all commits that have happened since we PR branch diverged from the target/base branch. Those are the commit we want to check!
| # Find current branch name | |
| CURRENT_BRANCH=$(git branch | grep ^\* | cut -d "*" -f 2 | cut -d " " -f 2) | |
| # Find hash of commit most common ancestor, e.g. where branch began | |
| BRANCH_MERGE_BASE=$(git merge-base ${BASE_BRANCH} ${CURRENT_BRANCH}) | |
| # Find all commits since common ancestor | |
| BRANCH_COMMITS=$(git rev-list ${BRANCH_MERGE_BASE}..HEAD) |
Then we start iterating over the commit in a while loop. On line 3, we have git tell us the commit message of each commit and check the commit message ( --format=%B) and use grep in regex (-iqE) mode to check if the commit matches our static bypass list. If yes, we print the commit message as debug and validation (line 5) and continue to the next commit hash (line 6).
| while IFS= read -r commit; do | |
| # Check if ticket in static permit-list for non-Jira commit_sha | |
| if $(git log --max-count=1 --format=%B $commit | grep -iqE "$commit_static_permit_list"); then | |
| echo "************" | |
| echo "Commit message \"$(git log --max-count=1 --format=%B $commit)\" matches static permit-list and is valid, continuing" | |
| continue | |
| fi |
Then we attempt to extract a Jira ticket number from the commit message (line 3). There’s a lot going on in that line — first we tell git the commit hash and print the commit message, then we convert it to upper-case tr '[a-z]' '[A-Z]' and use sed to match any ticket-like symbol at the beginning of the commit message sed -nE 's/^([a-zA-Z]{2,}-[0-9]{2,}).*$/\1/p' , and then a final check to make sure we only are checking the FIRST commit message head -n 1.
Then on line 6, we check the line count of the response. If it’s 0, there was no commit message found, and we set the canary var invalidTicket=true.
If a valid ticket is found, we print the ticket we’re going to check Jira for.
| # Filter commit message to just ticket at beginning | |
| unset TICKET_TO_CHECK | |
| TICKET_TO_CHECK=$(git log --max-count=1 --format=%B $commit | tr '[a-z]' '[A-Z]' | sed -nE 's/^([a-zA-Z]{2,}-[0-9]{2,}).*$/\1/p' | head -n 1) | |
| # If line count is zero, couldn't find valid ticket number to check | |
| if [[ $(echo "$TICKET_TO_CHECK" | awk 'NF' | wc -l) -eq 0 ]]; then | |
| echo "************" | |
| echo "❌ Couldn't identify valid ticket number to check in commit" | |
| echo "❌ Invalid commit message: \"$(git log --max-count=1 --format=%B $commit)\"" | |
| invalidTicket=true | |
| else | |
| # If valid ticket number found, check it | |
| echo "************" | |
| echo "Checking if this ticket exists: $TICKET_TO_CHECK" |
Now that we have a ticket-like symbol to validate, we send an authenticated REST call to our Jira instance. If the CURL response says “Issue doesn’t exist”, we print out a bunch of useful debug info and set our canary var invalidTicket=true. If that error message isn’t seen, we decide the commit is valid and continue to the next commit.
| # Check if ticket exists | |
| unset CURL | |
| CURL=$(curl -s --url "https://$JIRA_URL/rest/api/3/search?jql=key=${TICKET_TO_CHECK}" --header 'Accept:application/json' --user ${jira_username}:${jira_api_token} 2>&1) | |
| if [[ "$CURL" == *"Issue does not exist"* ]]; then | |
| echo "❌ Ticket referenced in commit doesn't exist in Jira. You must use real Jira tickets at the start of commit messages" | |
| echo "❌ Recognized ticket in commit: $TICKET_TO_CHECK" | |
| echo "❌ Commit message: \"$(git log --max-count=1 --format=%B $commit)\"" | |
| echo "❌ Hash: $commit" | |
| echo "************" | |
| # Set this variable to trigger rejection if any commit fails regex | |
| invalidTicket=true | |
| else | |
| echo "Ticket $TICKET_TO_CHECK is valid" | |
| fi | |
| fi | |
| done <<< $BRANCH_COMMITS |
At the very end, outside of our while loop over commit messages, we check if the canary var for failed commits is set to true. If yes, at least one (could be all of them, we don’t care) of the commits is bad and we exit 1 to tell the Action to fail. If not, we print a happy message and exit 0.
| # If any commit are invalid, print reject message | |
| if [[ "$invalidTicket" == true ]]; then | |
| echo "❌ Your push was rejected because at least one commit message on this branch is invalid" | |
| echo "❌ Please fix the commit message(s) and push again." | |
| echo "❌ https://help.github.com/en/articles/changing-a-commit-message" | |
| echo "************" | |
| exit 1 | |
| else | |
| echo "************" | |
| echo "All commits are valid" | |
| echo "************" | |
| exit 0 | |
| fi |
Let’s Make an Org-Wide Action — GitHub Settings
Okay, now we have an Action all ready to go. We have created a repo in GitHub, uploaded that code, and we’re ready to use it!
But not really. There is a setting in this repo that must be swapped over before anyone is allowed to call your action. Let’s go look at it.
Settings → Actions → General → Access. This defaults to “Not accessible”, which does what it says — no one is allowed to call this repo as an Action. You either need to switch this to Org-wide access (2nd bullet) or Enterprise-wide access (3rd bullet).
Hit save, and your Action is ready to be called.
Calling the Action From Another Repo
Other repos can now call this Action. However, those repos (calling repos) still require an Action that references this Action. So let’s make one.
First we set a name (line 1) and an on, which means what triggers this Action. Since this is a validation on a pull request only, we set it to on: [pull_request] .
We establish a job Commit_Checker and set where it should run (line 8). This could easily be public runners if your Jira instance is accessible publicly.
And on line 11 we start establishing tasks. First, we need all our code, and we check out the repo at the base ref (the target branch). Our next step (in the next section) is to check out the repo at the head, the latest commit on the PR branch, which is something we require to get all the git metadata for both branches.
| name: Git Commit Checker | |
| # Run on push to any branch | |
| on: [pull_request] | |
| jobs: | |
| Commit_Checker: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v2 | |
| with: | |
| fetch-depth: 0 | |
| ref: '${{ github.event.pull_request.base.ref }}' |
First we checkout the head.ref, the latest commit on the PR branch. Then we set the BASE_BRANCH . An (untested) theory of mine is that that PR metadata isn’t available to the child Action as it runs, so I set that here.
On line 9 we do something unique to GitHub Actions — we write the mapping that variable BASE_BRANCH is set to the value of the var $BASE_BRANCH and use the tee tool to append (-a) to the $GITHUB_ENV variable. That variable persists for downstream actions in the same job, otherwise variables are destroyed after each task.
| - name: Prep | |
| run: | | |
| # Checkout branch | |
| git checkout -q ${{ github.event.pull_request.head.ref }} | |
| # Set variables | |
| BASE_BRANCH=${{ github.event.pull_request.base.ref }} | |
| # Write BASE_BRANCH to GITHUB_ENV so commit checker can use | |
| echo "BASE_BRANCH=$BASE_BRANCH" | tee -a $GITHUB_ENV |
Next we call the shared action from the KyMidd (hey, that’s me!) location and send the required information, including the address of your Jira instance. This instance must be reachable from wherever the Action runs from. If you followed this walk-through, that’s a public runner, so your Jira would need to be on the internet.
| - name: Jira Commit Checker | |
| id: jira_commit_checker | |
| uses: KyMidd/ActionJiraCommitValidate@v1 | |
| with: | |
| jira-username: ${{ secrets.JIRA_COMMIT_CHECKER_USERNAME }} | |
| jira-api-token: ${{ secrets.JIRA_COMMIT_CHECKER_API_TOKEN }} | |
| JIRA_URL: 'https://foo.atlassian.net' | |
| BASE_BRANCH: ${{ env.BASE_BRANCH }} |
Require This Action to Pass for PR Merge
Right now this action would run and (hopefully) work properly, but it would be purely informational. It wouldn’t block a PR from merging, which is probably what it SHOULD be doing, since we’re assuming our organization requires valid ticket numbers on each commit.
In each repo that’s calling this action, you would need to create branch protection rules that “protect” code from being merged into those branches using a bunch of rules. One of the rules you can set is to require Status Checks (like the one outputted by this Action) to show a successful run to permit merging. It’ll look like this:
Summary
We looked at a novel way to isolate ticket numbers in the beginning of a commit, as well as learned some about git to find all the commits from where our PR branch was… well, branched from the base branch. And then we wrote a GitHub Action that can check into our ticketing system to make sure those tickets exist, and we even required it to pass in order for PRs to merge.
That’s a huge amount of work, great job! You can find all the code for the stuff we did here:
GitHub - KyMidd/ActionJiraCommitValidate at v1
This Action reads all commits on a PR branch that deviate from a base branch, and checks each commit for a leading…github.com
And you can find a link to the GitHub Actions Marketplace entry for this Action here:
Check Commits for Jira - GitHub Marketplace
This Action reads all commits on a PR branch that deviate from a base branch, and checks each commit for a leading…github.com
If you use it and like it, please leave a star on the repo so I know!
Good luck out there!
kyler