How to tune an AI model for code review by providing it your team style guide
AI code review pipeline configured, team style guide document in Markdown or text format, access to update the AI review prompt or configuration
What this does
Out of the box, an AI code review tool applies general-purpose conventions when evaluating pull requests. By injecting a team-specific style guide into the review prompt, the model learns to enforce project conventions such as naming patterns, module organization rules, testing requirements, and documentation standards. This produces review feedback that aligns with the team's agreed-upon practices rather than generic recommendations.
Steps
Locate the AI review script used by the CI pipeline, typically
.github/workflows/ai_review_script.pyor a similar file referenced in the workflow YAML.Open the script and find the section where the prompt is constructed before sending to the AI API. The prompt typically includes a system-level instruction string.
Read the entire prompt template and identify where static instructions are inserted. This is where the style guide should be injected.
Add a step early in the script to load the style guide content from the repository. For example:
import pathlib
style_guide_path = pathlib.Path("./docs/style-guide.md")
style_guide_content = style_guide_path.read_text() if style_guide_path.exists() else ""
- Append the style guide content to the system prompt using a clear delimiter:
system_prompt = f"""
You are reviewing code for a team that follows the conventions below.
Adhere strictly to these rules when providing feedback.
--- TEAM STYLE GUIDE ---
{style_guide_content}
--- END STYLE GUIDE ---
Now review the following diff and flag violations of the above conventions.
"""
Ensure the style guide file is included in the workflow checkout step by verifying
fetch-depthis set appropriately. The style guide must be present in the checkout directory at runtime.Commit and push the updated script and style guide to a test branch.
Open or update a pull request that exercises typical code patterns to verify the review comments reflect the style guide rules.
Iterate on the style guide content if review comments still do not reflect expected conventions. Add explicit examples in the style guide for ambiguous rules.
Update the prompt in the script to specify the output format for violations, for example requesting a table with columns for file, line, rule violated, and suggested fix.
Verification
Review a recent AI code review comment on a pull request. Confirm that at least one comment references a rule that appears verbatim in the team style guide. The rule should be quoted or paraphrased rather than a generic convention.
Common failures
- Style guide file not found at runtime: The CI runner uses a minimal checkout. If the style guide is in a subdirectory or a branch not currently checked out, the file load fails silently. Verify the file path and ensure it exists in the default branch or include it in the checkout step explicitly.
- Prompt exceeds model context limit: Long style guides combined with large diffs can exceed the model's maximum context window. Truncate or summarize the style guide to the rules most relevant to code review, and prefer concise bullet points over lengthy prose.
- AI ignores style guide rules: Some models have difficulty following injected instructions if the style guide contradicts general training. Add explicit enforcement phrases such as "You MUST reject any code that violates the following rule:" rather than softer phrasing like "please consider."
- Style guide rules not specific enough: Generic rules like "write clean code" produce generic feedback. Each rule must be concrete and actionable, for example "Variable names must use snake_case and must not exceed 30 characters."
- Changes to style guide not reflected in next review run: The CI script reads the file at runtime. If the style guide is updated but not yet pushed, or if a cached version is used by a custom AI endpoint, the next review run may still use the old content. Confirm the latest commit is present in the CI run.