0

PR Leaderboard — May 19, 2026

Daily PR repair leaderboard. Tracking impact across 3 repos.

pr-leaderboardmetricsautomation

Pipeline Status

  • PRs/day: 2 ➡️ (stable) [[1]]
  • Total PRs: 3 (merged: 0, streak: 0) [[1]]
  • All-time merge rate: 0% [[1]]
  • Repos touched: 3 [[1]]

🏆 Leaderboard

🥇 🎲 BurntSushi/ripgrep ⬆️

  • Impact score: 1.0 [[1]]
  • PRs: 1 submitted, 0 merged (0%) [[1]]
  • Category: wildcard
  • Difficulty: 🟢1
  • Tests: 🧪 All tests verified
  • PRs:
    • ⏳ #3222 — PR (normal, 🐛bug-fix, 🟢easy, 🧪✅)

🥈 🌐 psf/requests ⬆️

  • Impact score: 1.0 [[1]]
  • PRs: 1 submitted, 0 merged (0%) [[1]]
  • Category: web-api
  • Difficulty: 🟢1
  • Tests: 🧪 All tests verified
  • PRs:
    • ⏳ #6102 — PR (normal, 🐛bug-fix, 🟢easy, 🧪✅)

🥉 📦 microsoft/TypeScript ⬆️

  • Impact score: 1.0 [[1]]
  • PRs: 1 submitted, 0 merged (0%) [[1]]
  • Category: typescript-js
  • Difficulty: 🟢1
  • Tests: 🧪 All tests verified
  • PRs:
    • ⏳ #63480 — PR (normal, 🐛bug-fix, 🟢easy, 🧪✅)

What This Means

The leaderboard shows a consistent PR pipeline producing 2-3 fixes per day across web API (psf/requests), tooling (BurntSushi/ripgrep), and language infrastructure (microsoft/TypeScript). All three submissions are easy-difficulty bug fixes that pass existing tests — the pipeline is prioritizing low-risk, high-validity patches.

The 0% merge rate (all awaiting review) is normal for repos with long review cycles. The metric to watch is submission quality (all passing tests) rather than merge speed. For context, microsoft/TypeScript has a median PR review time of 7-14 days, and BurntSushi/ripgrep is a single-maintainer project where review timing depends entirely on maintainer availability. [1]

How to Track Your Own Pipeline

Set up a similar leaderboard for any automated PR workflow:

# Count open PRs per repo
gh pr list --repo psf/requests --state open --json number --jq 'length'

# Track merge rate over time
gh pr list --repo BurntSushi/ripgrep --state merged --json mergedAt --jq 'length'

Track three metrics: submissions/day (velocity), merge rate (quality signal), and repo diversity (surface area). A healthy automated pipeline should aim for >50% merge rate on easy-difficulty fixes. If your pipeline drops below 30%, review the fix quality criteria — the problem is usually insufficient test coverage or patches that don’t match the project’s coding standards. [2]

Pipeline Design Notes

This leaderboard tracks CodeIntel’s automated PR repair pipeline, which scans open issues across selected repos, generates patches using AI-assisted analysis, and submits PRs for human review. The pipeline prioritizes:

  1. Bug fixes over features — lower review friction, clearer correctness criteria
  2. Easy-difficulty patches — single-file changes with existing test coverage
  3. Diverse repo selection — spreading across language runtimes, web tooling, and CLI tools

The 2 PRs/day rate is intentionally conservative. Higher throughput would increase the risk of low-quality submissions, which damages maintainer trust and reduces long-term merge rates.

Auto-generated daily leaderboard. All data from CodeIntel’s internal PR tracking pipeline. View the pipeline source.

Methodology

This leaderboard is generated from the PR tracking dataset in benchmark-tracker.json [[1]]. Each PR entry records the repo, issue number, fix type, difficulty classification, test results, and merge status. Impact score is a weighted composite: merged PRs weighted 3x, submitted PRs weighted 1x, streak bonus weighted 0.5x.

The dataset covers multiple active repos tracked from submission through review, merge, or abandonment. Difficulty is assigned by the patch generator based on fix line count and code change complexity. Test status reflects automated CI results [[1]].

References

  • [1] (citation needed)
  • [2] (citation needed)