Our build pipeline in Azure DevOps has not been kicking off when a Pull Request is submitted for the relevant branch.
The troubleshooting documentation goes over common causes of a trigger failing. None of which seems like the explicit cause.
Known Facts
- Issue first appeared around the time frame team member left. An existing Service Connection is associated with this Azure Active Directory account.
- The Classic UI version of the Pipeline is used (not YAML).
- The trigger for CI/CD (push) seems to be functioning since a merge of the PR would kick off a build.
- Service Connection used is with a Service Account.
- We have multiple Build Pipelines some of them not in use.
Classic vs YAML
The issue started to occur after a team member had left the company. Checking the Service Connection used for that Build Pipeline, I was able to verify that it wasn't tied to that member who left, but one to a Service Account. My first thought was perhaps the Classic Build Pipeline is obsolete and created a YAML version of it. This new pipeline also didn't kick off when PRs were created/updated.
Webhooks
The next step was to take a closer look at the Webhooks for the git repository. Under Settings -> Hooks:
There are three hooks automatically created by Azure DevOps once a Build Pipeline is created. Looking at the history, the responses were all successful (200s) for each of the three (push, pull_request, issue comment):
I noted down the details and recreated them with a new commit pushed. No luck.
Build Definitions
I checked build pipeline definition for Triggers -> Pull request validation to ensure that we didn't have conflicting logic. Each one made sense.
Service Account
Now time to focus on the service account. Logging into Github Enterprise with the service account associated with the Service Connection that is used to clone/sync the repository, I wanted to verify that the associated GHE personal access token (PAT) has the proper scope. Settings -> Developer Settings -> Personal Access Tokens:
The selected scope for repo seems appropriate.
Next was logging into Azure DevOps as the service account. Once logged I checked the Triggers -> Pull request validation for the pipeline in question and see
The remote repository's webhooks are missing or incorrect.
This looks promising. However, odd since I recreated the webhooks. Logging in using my AAD account (not service account) doesn't show this warning/error. I click the Restore button and commit then pushed.
BAM. The PR trigger kicks off for both pipelines (classic and YAML).
I suspect those hooks were created with the same Classic build definition, but was originally associated to a Service Connected created by the team member who is no longer with the company. We must have hit an issue with the build pipeline in the past and created a new Service Connection with a service account and then updated the build pipeline step of pull from the repository with it. Why the push trigger still worked while the PR trigger did not is still a mystery.
Summary
Pull request build pipeline trigger does not kick off. Problem with with the existing webhooks. Resolved by deleting the webhooks and having Azure DevOps detect that there is an issue to recreate them based off the Build Pipeline.