Code Intelligence Benchmarks — {SITE_TITLE}

# // code-intelligence benchmarks

📅 Updated Jul 2026🧠 15 models📝 5 problems📐 4 dimensions per solution

## // methodology

✓

Correctness

Run against 4-5 test cases per problem. Pass/fail scoring with edge cases.

◉

Complexity

AST analysis of cyclomatic complexity, nesting depth, loop count, and line count.

◎

Readability

Heuristic scoring of naming, comments, blank lines, and line length distribution.

✦

Style

Pythonic idiom detection: type hints, list comprehensions, enumerate/zip, docstrings.

How scores work: Each model is given the same 5 coding problems at temperature 0.2. Solutions are evaluated programmatically — no human judgment involved. Composite = correctness × 0.4 + complexity × 0.2 + readability × 0.2 + style × 0.2. New models added weekly.

## // overall standings

GPT-OSS 20B 🏆

85.8

DeepSeek V4

85.6

DeepSeek V4 Flash (direct)

85.5

OpenRouter GPT-OSS 20B

85.5

Qwen 3.5 Plus

85.1

Nemotron 3 Ultra 550B

85.0

Hy3 (Nexum)

84.6

Xiaomi MiMo 2.5 (Nexum)

82.0

Devstral Latest (Mistral)

81.9

#10

Codestral Latest (Mistral)

80.8

#11

Groq Llama 3.3 70B

80.4

#12

Gemma 4 12B Coder (local)

77.6

#13

Mistral Large

77.3

#14

Cerebras GPT-OSS 120B

75.9

#15

GPT 5.5 Instant 14K

40.3

## // per-model breakdown

GPT-OSS 20B

openrouter · openai/gpt-oss-20b:free · Overall: 85.8

✓

96.0

Correctness

◉

63.7

Complexity

◎

85.5

Readability

✦

87.6

Style

∑

Two Sum

84.9

FizzBuzz

87.7

⟐

Merge Intervals

91.4

⚡

LRU Cache

91.8

JSON Parser

73.0

DeepSeek V4

nexum · deepseek-v4 · Overall: 85.6

✓

96.0

Correctness

◉

64.3

Complexity

◎

85.0

Readability

✦

86.6

Style

∑

Two Sum

84.2

FizzBuzz

87.7

⟐

Merge Intervals

90.7

⚡

LRU Cache

91.9

JSON Parser

73.4

DeepSeek V4 Flash (direct)

deepseek · deepseek-v4-flash · Overall: 85.5

✓

96.0

Correctness

◉

63.9

Complexity

◎

83.8

Readability

✦

87.6

Style

∑

Two Sum

84.3

FizzBuzz

87.7

⟐

Merge Intervals

90.7

⚡

LRU Cache

92.2

JSON Parser

72.4

OpenRouter GPT-OSS 20B

openrouter · openai/gpt-oss-20b:free · Overall: 85.5

✓

96.0

Correctness

◉

63.8

Complexity

◎

84.1

Readability

✦

87.6

Style

∑

Two Sum

84.9

FizzBuzz

87.7

⟐

Merge Intervals

91.1

⚡

LRU Cache

91.8

JSON Parser

72.0

Qwen 3.5 Plus

nexum · qwen-3.5-plus · Overall: 85.1

✓

96.0

Correctness

◉

61.3

Complexity

◎

85.7

Readability

✦

86.6

Style

∑

Two Sum

84.2

FizzBuzz

87.7

⟐

Merge Intervals

90.8

⚡

LRU Cache

88.8

JSON Parser

74.1

Nemotron 3 Ultra 550B

openrouter · nvidia/nemotron-3-ultra-550b-a55b:free · Overall: 85.0

✓

96.0

Correctness

◉

61.7

Complexity

◎

84.8

Readability

✦

86.6

Style

∑

Two Sum

84.2

FizzBuzz

87.7

⟐

Merge Intervals

90.7

⚡

LRU Cache

89.1

JSON Parser

73.4

Hy3 (Nexum)

nexum · hy3 · Overall: 84.6

✓

96.0

Correctness

◉

64.6

Complexity

◎

79.7

Readability

✦

86.6

Style

∑

Two Sum

84.2

FizzBuzz

87.7

⟐

Merge Intervals

90.8

⚡

LRU Cache

90.8

JSON Parser

69.5

Xiaomi MiMo 2.5 (Nexum)

nexum · xiaomi-mimo-2.5 · Overall: 82.0

✓

88.0

Correctness

◉

61.6

Complexity

◎

85.6

Readability

✦

86.6

Style

∑

Two Sum

84.3

FizzBuzz

87.7

⟐

Merge Intervals

91.2

⚡

LRU Cache

89.2

JSON Parser

57.4

Devstral Latest (Mistral)

mistral · devstral-latest · Overall: 81.9

✓

92.0

Correctness

◉

64.4

Complexity

◎

74.8

Readability

✦

86.6

Style

∑

Two Sum

84.2

FizzBuzz

87.7

⟐

Merge Intervals

91.2

⚡

LRU Cache

91.5

JSON Parser

55.1

Codestral Latest (Mistral)

mistral · codestral-latest · Overall: 80.8

✓

84.0

Correctness

◉

62.9

Complexity

◎

86.5

Readability

✦

86.6

Style

∑

Two Sum

84.2

FizzBuzz

87.7

⟐

Merge Intervals

91.2

⚡

LRU Cache

90.7

JSON Parser

50.2

Groq Llama 3.3 70B

groq · llama-3.3-70b-versatile · Overall: 80.4

✓

80.0

Correctness

◉

72.5

Complexity

◎

81.5

Readability

✦

87.8

Style

∑

Two Sum

84.2

FizzBuzz

93.6

⟐

Merge Intervals

91.2

⚡

LRU Cache

91.3

JSON Parser

41.5

Gemma 4 12B Coder (local)

local · gemma-4-12b-coder-fable5-composer2.5-v1 · Overall: 77.6

✓

76.0

Correctness

◉

65.2

Complexity

◎

85.0

Readability

✦

85.6

Style

∑

Two Sum

84.3

FizzBuzz

87.7

⟐

Merge Intervals

91.4

⚡

LRU Cache

90.8

JSON Parser

33.6

Mistral Large

mistral · mistral-large-latest · Overall: 77.3

✓

76.0

Correctness

◉

61.5

Complexity

◎

86.2

Readability

✦

86.6

Style

∑

Two Sum

84.2

FizzBuzz

87.7

⟐

Merge Intervals

91.2

⚡

LRU Cache

89.2

JSON Parser

34.0

Cerebras GPT-OSS 120B

cerebras · gpt-oss-120b · Overall: 75.9

✓

76.0

Correctness

◉

57.7

Complexity

◎

82.0

Readability

✦

87.6

Style

∑

Two Sum

84.9

FizzBuzz

87.7

⟐

Merge Intervals

91.1

⚡

LRU Cache

85.4

JSON Parser

30.2

GPT 5.5 Instant 14K

nexum · gpt-5.5-instant-14k · Overall: 40.3

✓

0.0

Correctness

◉

30.0

Complexity

◎

86.0

Readability

✦

85.6

Style

∑

Two Sum

40.6

FizzBuzz

40.0

⟐

Merge Intervals

40.8

⚡

LRU Cache

40.6

JSON Parser

39.6

## // per-problem comparison

Two Sum

Model	✓ Correctness	◉ Complexity	◎ Readability	✦ Style	Composite
GPT-OSS 20B 🏆	80.0	86.5	85.0	93.0	84.9
DeepSeek V4	80.0	88.0	85.0	88.0	84.2
DeepSeek V4 Flash (direct)	80.0	88.5	85.0	88.0	84.3
OpenRouter GPT-OSS 20B	80.0	86.5	85.0	93.0	84.9
Qwen 3.5 Plus	80.0	88.0	85.0	88.0	84.2
Nemotron 3 Ultra 550B	80.0	88.0	85.0	88.0	84.2
Hy3 (Nexum)	80.0	88.0	85.0	88.0	84.2
Xiaomi MiMo 2.5 (Nexum)	80.0	88.5	85.0	88.0	84.3
Devstral Latest (Mistral)	80.0	88.0	85.0	88.0	84.2
Codestral Latest (Mistral)	80.0	88.0	85.0	88.0	84.2
Groq Llama 3.3 70B	80.0	88.0	85.0	88.0	84.2
Gemma 4 12B Coder (local)	80.0	88.5	85.0	88.0	84.3
Mistral Large	80.0	88.0	85.0	88.0	84.2
Cerebras GPT-OSS 120B	80.0	86.5	85.0	93.0	84.9
GPT 5.5 Instant 14K	0	30	85.0	88.0	40.6

FizzBuzz

Model	✓ Correctness	◉ Complexity	◎ Readability	✦ Style	Composite
GPT-OSS 20B	100.0	68.5	85.0	85.0	87.7
DeepSeek V4	100.0	68.5	85.0	85.0	87.7
DeepSeek V4 Flash (direct)	100.0	68.5	85.0	85.0	87.7
OpenRouter GPT-OSS 20B	100.0	68.5	85.0	85.0	87.7
Qwen 3.5 Plus	100.0	68.5	85.0	85.0	87.7
Nemotron 3 Ultra 550B	100.0	68.5	85.0	85.0	87.7
Hy3 (Nexum)	100.0	68.5	85.0	85.0	87.7
Xiaomi MiMo 2.5 (Nexum)	100.0	68.5	85.0	85.0	87.7
Devstral Latest (Mistral)	100.0	68.5	85.0	85.0	87.7
Codestral Latest (Mistral)	100.0	68.5	85.0	85.0	87.7
Groq Llama 3.3 70B 🏆	100.0	100	80.0	88.0	93.6
Gemma 4 12B Coder (local)	100.0	68.5	85.0	85.0	87.7
Mistral Large	100.0	68.5	85.0	85.0	87.7
Cerebras GPT-OSS 120B	100.0	68.5	85.0	85.0	87.7
GPT 5.5 Instant 14K	0	30	85.0	85.0	40.0

Merge Intervals

Model	✓ Correctness	◉ Complexity	◎ Readability	✦ Style	Composite
GPT-OSS 20B 🏆	100.0	82.7	89.3	85.0	91.4
DeepSeek V4	100.0	83.5	85.0	85.0	90.7
DeepSeek V4 Flash (direct)	100.0	83.5	85.0	85.0	90.7
OpenRouter GPT-OSS 20B	100.0	83.0	87.3	85.0	91.1
Qwen 3.5 Plus	100.0	83.8	85.0	85.0	90.8
Nemotron 3 Ultra 550B	100.0	83.5	85.0	85.0	90.7
Hy3 (Nexum)	100.0	83.8	85.0	85.0	90.8
Xiaomi MiMo 2.5 (Nexum)	100.0	82.2	89.0	85.0	91.2
Devstral Latest (Mistral)	100.0	82.2	89.0	85.0	91.2
Codestral Latest (Mistral)	100.0	82.2	89.0	85.0	91.2
Groq Llama 3.3 70B	100.0	82.2	89.0	85.0	91.2
Gemma 4 12B Coder (local)	100.0	81.8	90.0	85.0	91.4
Mistral Large	100.0	82.2	89.0	85.0	91.2
Cerebras GPT-OSS 120B	100.0	83.0	87.3	85.0	91.1
GPT 5.5 Instant 14K	0	30	89.0	85.0	40.8

LRU Cache

Model	✓ Correctness	◉ Complexity	◎ Readability	✦ Style	Composite
GPT-OSS 20B	100.0	81.0	88.0	90.0	91.8
DeepSeek V4	100.0	81.4	88.2	90.0	91.9
DeepSeek V4 Flash (direct) 🏆	100.0	79.0	92.2	90.0	92.2
OpenRouter GPT-OSS 20B	100.0	81.0	88.0	90.0	91.8
Qwen 3.5 Plus	100.0	66.2	87.8	90.0	88.8
Nemotron 3 Ultra 550B	100.0	68.6	87.0	90.0	89.1
Hy3 (Nexum)	100.0	76.6	87.2	90.0	90.8
Xiaomi MiMo 2.5 (Nexum)	100.0	69.0	87.0	90.0	89.2
Devstral Latest (Mistral)	100.0	80.6	86.9	90.0	91.5
Codestral Latest (Mistral)	100.0	75.8	87.5	90.0	90.7
Groq Llama 3.3 70B	100.0	79.8	86.7	90.0	91.3
Gemma 4 12B Coder (local)	100.0	81.0	88.0	85.0	90.8
Mistral Large	100.0	69.0	87.0	90.0	89.2
Cerebras GPT-OSS 120B	100.0	50.4	86.7	90.0	85.4
GPT 5.5 Instant 14K	0	30	87.8	85.0	40.6

JSON Parser

Model	✓ Correctness	◉ Complexity	◎ Readability	✦ Style	Composite
GPT-OSS 20B	100.0	0	80.0	85.0	73.0
DeepSeek V4	100.0	0	82.0	85.0	73.4
DeepSeek V4 Flash (direct)	100.0	0	72.0	90.0	72.4
OpenRouter GPT-OSS 20B	100.0	0	75.1	85.0	72.0
Qwen 3.5 Plus 🏆	100.0	0	85.5	85.0	74.1
Nemotron 3 Ultra 550B	100.0	0	82.0	85.0	73.4
Hy3 (Nexum)	100.0	6.2	56.2	85.0	69.5
Xiaomi MiMo 2.5 (Nexum)	60.0	0	82.0	85.0	57.4
Devstral Latest (Mistral)	80.0	2.5	28.0	85.0	55.1
Codestral Latest (Mistral)	40.0	0	86.2	85.0	50.2
Groq Llama 3.3 70B	20.0	12.4	67.0	88.0	41.5
Gemma 4 12B Coder (local)	0.0	6.0	77.2	85.0	33.6
Mistral Large	0	0	85.0	85.0	34.0
Cerebras GPT-OSS 120B	0	0	66.0	85.0	30.2
GPT 5.5 Instant 14K	0	30	83.2	85.0	39.6

## // key takeaways

All models handle easy problems

Two Sum and FizzBuzz are essentially solved — all models score 80+ on correctness. The differentiation happens on harder tasks.

JSON parser is the big differentiator

Writing a JSON parser from scratch pushes models to demonstrate real coding ability — this is where the gap widens significantly.

DeepSeek V4 shines on LRU Cache

Uses OrderedDict.move_to_end() for elegant O(1) implementation — scores highest on this problem.

Complexity scores tank on long solutions

JSON parsers are inherently long (50-200 lines), which drags complexity scores. This is expected — we're measuring conciseness, not quality.