HOW-TO · DEV

How to set up AI code review automations in GitHub Actions using Claude or GPT models

advanced30 minBy Eruo Fredoline
Target environment
Ubuntu 24.04 · Ollama 0.4.x
PREREQUISITES

GitHub repository with Actions enabled, valid API key for the chosen AI provider, repository admin access to configure secrets

What this does

This guide configures a GitHub Actions workflow that automatically triggers an AI model to review pull request diffs and post comments directly on the PR. The workflow runs on every opened or updated pull request, analyzing added and changed lines for potential bugs, style violations, security issues, and logic errors. The result is a structured AI comment thread attached to the PR that developers can read and act upon before merging.

Steps

  1. Create a .github/workflows/ directory in the repository root if it does not already exist.

  2. Create a file named ai-code-review.yml inside that directory.

  3. Add the following workflow content, replacing the API endpoint and model name to match the chosen provider:

name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout PR
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install anthropic requests

      - name: Run AI Code Review
        env:
          AI_API_KEY: ${{ secrets.AI_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: python .github/workflows/ai_review_script.py

  1. Create .github/workflows/ai_review_script.py with the diff extraction, AI prompt construction, and PR comment posting logic.

  2. Navigate to Settings → Secrets and variables → Actions in the repository and add AI_API_KEY with the secret value.

  3. Push the changes to a feature branch and open a pull request to trigger the workflow.

  4. Verify the workflow runs under the Actions tab.

  5. Confirm AI review comments appear in the Conversation tab of the pull request.

  6. Adjust the prompt inside ai_review_script.py to focus on the issues most relevant to the codebase.

  7. Set branch protection rules to require review approval before merging if desired.

Verification

Navigate to the pull request conversation tab and confirm at least one AI-generated comment appears with analysis of the code changes. The workflow run under the Actions tab should show a green checkmark with status "success".

Common failures

  • Workflow never triggers: Confirm the workflow file exists at .github/workflows/ai-code-review.yml and the on: pull_request trigger is not blocked by a branch protection rule that predates the workflow.
  • API key not found: Ensure AI_API_KEY is stored as a repository secret and referenced exactly as secrets.AI_API_KEY. Organization-level secrets require explicit secrets.ACTIONS_DEPLOY_KEY configuration.
  • Rate limit exceeded: AI provider APIs impose per-minute or per-day limits. Add a retry-backoff loop to the Python script and consider caching responses for repeated identical diffs.
  • No diff detected in the script: The fetch-depth: 0 checkout action is required so the script can compute the diff between the base and head commits. Without it, git diff produces empty output.
  • Comments not posted due to permissions: The default GITHUB_TOKEN has read-only permissions in most contexts. Add a permissions block with contents: read and pull-requests: write to the workflow, or use a GitHub App token with appropriate scope.

Related guides

← All how-to guidesCourses →