Ada Wynn

By the numbers

[Placeholder]

This page gathers the site's headline numbers and a selection of its charts.

Headline numbers

Decisions ruled
4,324
Ruled by me
3,403
Overrule rate
33.6%of my 2,782 rulings on an agent's recommendation, I chose something else
Work items done
3,672
Pull requests in the shiplog
7,253
Repositories
79

Updated .

[Placeholder] Charts

My evals caught a model that wasn't ready to shipStatic dataLink

How often each model matched my own ruling, with 95% confidence intervals.

Matched my rulingRandom pick27.6%Fine-tuned adapter (checkpoint 125)36.6%Plain Qwen52.0%Opus 5.555.5%
Show as table
My evals caught a model that wasn't ready to ship
CategoryRandom pickFine-tuned adapter (checkpoint 125)Plain QwenOpus 5.5Random pick, confidence intervalFine-tuned adapter (checkpoint 125), confidence intervalPlain Qwen, confidence intervalOpus 5.5, confidence interval
Matched my ruling134 of 486 (27.6%)182 of 497 (36.6%)259 of 498 (52.0%)276 of 497 (55.5%)21.9% to 33.3%31.3% to 41.9%46.1% to 57.9%49.7% to 61.4%
[Placeholder]How often Elizabeth overrules the recommendationUpdated 6 November 2026Link

Of the decisions Elizabeth ruled each week, the share where she chose something other than the recommended option.

0%25%50%75%this week, partialJulAugSepOctNovshort ticks on the axis: too few samples that week to show a rate

Only rulings marked as either following or departing from the recommendation are counted.

Show as table
How often Elizabeth overrules the recommendation, as of 2026-11-06
WeekSamplesCountedRateConfidence interval
Week of Jul 2710no datano interval
Week of Aug 321no datano interval
Week of Aug 1021911050.2%43.7% to 56.8%
Week of Aug 17813745.7%35.3% to 56.5%
Week of Aug 24884045.5%35.5% to 55.8%
Week of Aug 31126no datano interval
Week of Sep 7874450.6%40.3% to 60.8%
Week of Sep 14712129.6%20.2% to 41.0%
Week of Sep 211103531.8%23.9% to 41.0%
Week of Sep 2861417829.0%25.5% to 32.7%
Week of Oct 533911433.6%28.8% to 38.8%
Week of Oct 1230610534.3%29.2% to 39.8%
Week of Oct 192497329.3%24.0% to 35.3%
Week of Oct 2645013530.0%25.9% to 34.4%
Week of Nov 2 (partial)1533724.2%18.1% to 31.5%
[Placeholder]Accuracy against cost, by modelStatic dataLink

How often each model found the planted problem, against what it cost per function, on 39 held-out functions.

Detection rate: real errors flagged higher is better

0%25%50%75%100%$0.0001$0.0010$0.010$0.100

False-positive rate: clean functions flagged lower is better

0%25%50%75%100%$0.0001$0.0010$0.010$0.100
  • DeepSeek V4 Flash, ten calls
  • DeepSeek V4 Flash, one call
  • Claude Sonnet 5, ten calls
  • Claude Sonnet 5, one call
  • GPT-5.6 Terra, medium effort, ten calls
  • GPT-5.6 Terra, medium effort, one call

The cost axis is logarithmic: each gridline is ten times the last.

Answers are graded against a fixed keyword answer key, not by another model.

The differences between models are not statistically significant.

Show as table
Accuracy against cost, by model
SetupCost (US dollars)Detection rate: real errors flagged (higher is better)False-positive rate: clean functions flagged (lower is better)Confidence interval, Detection rate: real errors flaggedConfidence interval, False-positive rate: clean functions flagged
DeepSeek V4 Flash, ten calls$0.001920 of 23 (87.0%)3 of 16 (18.8%)67.9% to 95.5%6.6% to 43.0%
DeepSeek V4 Flash, one call$0.000813 of 23 (56.5%)1 of 16 (6.3%)36.8% to 74.4%1.1% to 28.3%
Claude Sonnet 5, ten calls$0.03815 of 23 (65.2%)2 of 16 (12.5%)44.9% to 81.2%3.5% to 36.0%
Claude Sonnet 5, one call$0.02316 of 23 (69.6%)2 of 16 (12.5%)49.1% to 84.4%3.5% to 36.0%
GPT-5.6 Terra, medium effort, ten calls$0.02620 of 23 (87.0%)1 of 16 (6.3%)67.9% to 95.5%1.1% to 28.3%
GPT-5.6 Terra, medium effort, one call$0.004716 of 23 (69.6%)1 of 16 (6.3%)49.1% to 84.4%1.1% to 28.3%
[Placeholder]Pull requests merged each weekUpdated 6 November 2026Link

Pull requests merged each week, with the part linked to a tracked work item shown inside each bar.

0200400600800AugSepOctNov
  • Total
  • Merged pull requests

The linked part stands in for work done by agents; it is an estimate, not a label on each pull request.

Show as table
Pull requests merged each week, as of 2026-11-06
WeekTotalMerged pull requests
Week of Aug 10329194
Week of Aug 1715872
Week of Aug 24318239
Week of Aug 3111995
Week of Sep 7229130
Week of Sep 14372289
Week of Sep 21324260
Week of Sep 28784417
Week of Oct 5466231
Week of Oct 12417260
Week of Oct 19452300
Week of Oct 26638472
Week of Nov 2 (partial)292205
[Placeholder]Old problems cleaned up so farUpdated 6 November 2026Link

The running total of older problems flagged by the architecture checks that have been removed, week by week.

01.5K3K4.5K6KJulAugSepOct
  • Old problems removed so far

Shows totals only.

Show as table
Old problems cleaned up so far, as of 2026-11-06
WeekOld problems removed so far
Week of Jul 6698
Week of Jul 13786
Week of Jul 201,465
Week of Jul 272,437
Week of Aug 32,784
Week of Aug 103,024
Week of Aug 173,026
Week of Aug 243,052
Week of Aug 313,065
Week of Sep 73,120
Week of Sep 143,172
Week of Sep 213,457
Week of Sep 283,457
Week of Oct 53,479
Week of Oct 124,502
Week of Oct 194,562
Week of Oct 265,595
[Placeholder]Processor cores busy in working hoursUpdated 6 November 2026Link

Processor cores busy across the cluster each day during working hours: the average, the 90th percentile and the peak, against the cluster's capacity.

075150225300Average in working hours (3pm–4am Denver)90th percentile in working hoursPeak in working hoursCapacityAugSepOctNov

Data starts 2026-08-12.

Sampled every 15 minutes, so the peak is the highest sample that day.

Show as table
Processor cores busy in working hours, as of 2026-11-06
DayLineValue
Aug 11Average in working hours (3pm–4am Denver)4.1 cores
Aug 12Average in working hours (3pm–4am Denver)8.3 cores
Aug 13Average in working hours (3pm–4am Denver)8.4 cores
Aug 14Average in working hours (3pm–4am Denver)6.3 cores
Aug 15Average in working hours (3pm–4am Denver)6.7 cores
Aug 16Average in working hours (3pm–4am Denver)8.8 cores
Aug 17Average in working hours (3pm–4am Denver)8.3 cores
Aug 18Average in working hours (3pm–4am Denver)6.6 cores
Aug 19Average in working hours (3pm–4am Denver)12.6 cores
Aug 20Average in working hours (3pm–4am Denver)10.5 cores
Aug 21Average in working hours (3pm–4am Denver)8.9 cores
Aug 22Average in working hours (3pm–4am Denver)9.6 cores
Aug 23Average in working hours (3pm–4am Denver)10 cores
Aug 24Average in working hours (3pm–4am Denver)10.3 cores
Aug 25Average in working hours (3pm–4am Denver)8.4 cores
Aug 26Average in working hours (3pm–4am Denver)8.3 cores
Aug 27Average in working hours (3pm–4am Denver)11.1 cores
Aug 28Average in working hours (3pm–4am Denver)5.7 cores
Aug 29Average in working hours (3pm–4am Denver)6.2 cores
Aug 30Average in working hours (3pm–4am Denver)4.7 cores
Aug 31Average in working hours (3pm–4am Denver)5.6 cores
Sep 1Average in working hours (3pm–4am Denver)5.6 cores
Sep 2Average in working hours (3pm–4am Denver)8.2 cores
Sep 4Average in working hours (3pm–4am Denver)6.2 cores
Sep 5Average in working hours (3pm–4am Denver)6.8 cores
Sep 6Average in working hours (3pm–4am Denver)4.2 cores
Sep 7Average in working hours (3pm–4am Denver)4.7 cores
Sep 8Average in working hours (3pm–4am Denver)8.9 cores
Sep 9Average in working hours (3pm–4am Denver)9 cores
Sep 10Average in working hours (3pm–4am Denver)12.9 cores
Sep 11Average in working hours (3pm–4am Denver)11.5 cores
Sep 12Average in working hours (3pm–4am Denver)17.7 cores
Sep 13Average in working hours (3pm–4am Denver)12.1 cores
Sep 14Average in working hours (3pm–4am Denver)4.9 cores
Sep 15Average in working hours (3pm–4am Denver)15.3 cores
Sep 16Average in working hours (3pm–4am Denver)20.8 cores
Sep 17Average in working hours (3pm–4am Denver)28 cores
Sep 18Average in working hours (3pm–4am Denver)19 cores
Sep 19Average in working hours (3pm–4am Denver)43.5 cores
Sep 20Average in working hours (3pm–4am Denver)25 cores
Sep 21Average in working hours (3pm–4am Denver)27.9 cores
Sep 22Average in working hours (3pm–4am Denver)23.2 cores
Sep 23Average in working hours (3pm–4am Denver)22.8 cores
Sep 24Average in working hours (3pm–4am Denver)8 cores
Sep 25Average in working hours (3pm–4am Denver)33.2 cores
Sep 26Average in working hours (3pm–4am Denver)30.6 cores
Sep 27Average in working hours (3pm–4am Denver)35.7 cores
Sep 28Average in working hours (3pm–4am Denver)22.8 cores
Sep 29Average in working hours (3pm–4am Denver)26.8 cores
Sep 30Average in working hours (3pm–4am Denver)26.7 cores
Oct 10Average in working hours (3pm–4am Denver)28 cores
Oct 11Average in working hours (3pm–4am Denver)34.7 cores
Oct 12Average in working hours (3pm–4am Denver)21.2 cores
Oct 13Average in working hours (3pm–4am Denver)30.3 cores
Oct 14Average in working hours (3pm–4am Denver)26.6 cores
Oct 15Average in working hours (3pm–4am Denver)12.4 cores
Oct 16Average in working hours (3pm–4am Denver)30.4 cores
Oct 17Average in working hours (3pm–4am Denver)46.7 cores
Oct 18Average in working hours (3pm–4am Denver)67.6 cores
Oct 19Average in working hours (3pm–4am Denver)53.9 cores
Oct 20Average in working hours (3pm–4am Denver)37.9 cores
Oct 21Average in working hours (3pm–4am Denver)32.1 cores
Oct 22Average in working hours (3pm–4am Denver)56.7 cores
Oct 23Average in working hours (3pm–4am Denver)42.3 cores
Oct 24Average in working hours (3pm–4am Denver)75.3 cores
Oct 25Average in working hours (3pm–4am Denver)54.7 cores
Oct 26Average in working hours (3pm–4am Denver)51.4 cores
Oct 27Average in working hours (3pm–4am Denver)56.6 cores
Oct 28Average in working hours (3pm–4am Denver)88.9 cores
Oct 29Average in working hours (3pm–4am Denver)86.5 cores
Oct 30Average in working hours (3pm–4am Denver)73.4 cores
Oct 31Average in working hours (3pm–4am Denver)85.4 cores
Nov 1Average in working hours (3pm–4am Denver)91.1 cores
Nov 2Average in working hours (3pm–4am Denver)61.3 cores
Nov 3Average in working hours (3pm–4am Denver)63.1 cores
Nov 4Average in working hours (3pm–4am Denver)67.3 cores
Nov 5Average in working hours (3pm–4am Denver)158.5 cores
Aug 1190th percentile in working hours6.9 cores
Aug 1290th percentile in working hours16.4 cores
Aug 1390th percentile in working hours14.5 cores
Aug 1490th percentile in working hours10.9 cores
Aug 1590th percentile in working hours16 cores
Aug 1690th percentile in working hours14.9 cores
Aug 1790th percentile in working hours17.9 cores
Aug 1890th percentile in working hours8.3 cores
Aug 1990th percentile in working hours21.3 cores
Aug 2090th percentile in working hours18.3 cores
Aug 2190th percentile in working hours13.2 cores
Aug 2290th percentile in working hours14.1 cores
Aug 2390th percentile in working hours17.5 cores
Aug 2490th percentile in working hours18.3 cores
Aug 2590th percentile in working hours15.4 cores
Aug 2690th percentile in working hours21.2 cores
Aug 2790th percentile in working hours21.3 cores
Aug 2890th percentile in working hours10.8 cores
Aug 2990th percentile in working hours15 cores
Aug 3090th percentile in working hours7.3 cores
Aug 3190th percentile in working hours14.8 cores
Sep 190th percentile in working hours7.4 cores
Sep 290th percentile in working hours19.4 cores
Sep 490th percentile in working hours10.6 cores
Sep 590th percentile in working hours18 cores
Sep 690th percentile in working hours8.2 cores
Sep 790th percentile in working hours10.9 cores
Sep 890th percentile in working hours22.7 cores
Sep 990th percentile in working hours23.8 cores
Sep 1090th percentile in working hours28.8 cores
Sep 1190th percentile in working hours23.1 cores
Sep 1290th percentile in working hours29.6 cores
Sep 1390th percentile in working hours26.8 cores
Sep 1490th percentile in working hours5.3 cores
Sep 1590th percentile in working hours30.7 cores
Sep 1690th percentile in working hours33.3 cores
Sep 1790th percentile in working hours42.8 cores
Sep 1890th percentile in working hours45.7 cores
Sep 1990th percentile in working hours86.4 cores
Sep 2090th percentile in working hours49.4 cores
Sep 2190th percentile in working hours55.2 cores
Sep 2290th percentile in working hours53.5 cores
Sep 2390th percentile in working hours57.9 cores
Sep 2490th percentile in working hours9.3 cores
Sep 2590th percentile in working hours58.1 cores
Sep 2690th percentile in working hours60.1 cores
Sep 2790th percentile in working hours55.8 cores
Sep 2890th percentile in working hours44.5 cores
Sep 2990th percentile in working hours48.6 cores
Sep 3090th percentile in working hours49.6 cores
Oct 1090th percentile in working hours46.6 cores
Oct 1190th percentile in working hours54.5 cores
Oct 1290th percentile in working hours46.2 cores
Oct 1390th percentile in working hours48.1 cores
Oct 1490th percentile in working hours45.6 cores
Oct 1590th percentile in working hours19.6 cores
Oct 1690th percentile in working hours52.3 cores
Oct 1790th percentile in working hours65.1 cores
Oct 1890th percentile in working hours93.1 cores
Oct 1990th percentile in working hours89.4 cores
Oct 2090th percentile in working hours79 cores
Oct 2190th percentile in working hours73.4 cores
Oct 2290th percentile in working hours88.7 cores
Oct 2390th percentile in working hours76.6 cores
Oct 2490th percentile in working hours98.6 cores
Oct 2590th percentile in working hours97.7 cores
Oct 2690th percentile in working hours106.5 cores
Oct 2790th percentile in working hours90 cores
Oct 2890th percentile in working hours136.4 cores
Oct 2990th percentile in working hours121.6 cores
Oct 3090th percentile in working hours106.3 cores
Oct 3190th percentile in working hours157.6 cores
Nov 190th percentile in working hours135.7 cores
Nov 290th percentile in working hours117.8 cores
Nov 390th percentile in working hours103.7 cores
Nov 490th percentile in working hours118.5 cores
Nov 590th percentile in working hours194.8 cores
Aug 11Peak in working hours13.7 cores
Aug 12Peak in working hours26.4 cores
Aug 13Peak in working hours17.9 cores
Aug 14Peak in working hours13.9 cores
Aug 15Peak in working hours25.4 cores
Aug 16Peak in working hours23.3 cores
Aug 17Peak in working hours23.6 cores
Aug 18Peak in working hours21.4 cores
Aug 19Peak in working hours68.6 cores
Aug 20Peak in working hours28.3 cores
Aug 21Peak in working hours25.4 cores
Aug 22Peak in working hours19.7 cores
Aug 23Peak in working hours30.4 cores
Aug 24Peak in working hours30.1 cores
Aug 25Peak in working hours26.6 cores
Aug 26Peak in working hours25.6 cores
Aug 27Peak in working hours44 cores
Aug 28Peak in working hours29.4 cores
Aug 29Peak in working hours23.4 cores
Aug 30Peak in working hours22.6 cores
Aug 31Peak in working hours21 cores
Sep 1Peak in working hours26.4 cores
Sep 2Peak in working hours31.8 cores
Sep 4Peak in working hours14.6 cores
Sep 5Peak in working hours25.2 cores
Sep 6Peak in working hours18.9 cores
Sep 7Peak in working hours19.2 cores
Sep 8Peak in working hours29.5 cores
Sep 9Peak in working hours37.3 cores
Sep 10Peak in working hours39.7 cores
Sep 11Peak in working hours27 cores
Sep 12Peak in working hours44 cores
Sep 13Peak in working hours37.7 cores
Sep 14Peak in working hours11.8 cores
Sep 15Peak in working hours41.1 cores
Sep 16Peak in working hours38.1 cores
Sep 17Peak in working hours56.6 cores
Sep 18Peak in working hours56 cores
Sep 19Peak in working hours99.6 cores
Sep 20Peak in working hours106.5 cores
Sep 21Peak in working hours66.4 cores
Sep 22Peak in working hours77.5 cores
Sep 23Peak in working hours70.9 cores
Sep 24Peak in working hours48.1 cores
Sep 25Peak in working hours64.3 cores
Sep 26Peak in working hours74.2 cores
Sep 27Peak in working hours69.9 cores
Sep 28Peak in working hours68.1 cores
Sep 29Peak in working hours102.3 cores
Sep 30Peak in working hours53.4 cores
Oct 10Peak in working hours63.6 cores
Oct 11Peak in working hours73.3 cores
Oct 12Peak in working hours95.7 cores
Oct 13Peak in working hours70 cores
Oct 14Peak in working hours67.3 cores
Oct 15Peak in working hours36 cores
Oct 16Peak in working hours59 cores
Oct 17Peak in working hours92.2 cores
Oct 18Peak in working hours137.1 cores
Oct 19Peak in working hours126.2 cores
Oct 20Peak in working hours98.4 cores
Oct 21Peak in working hours83.2 cores
Oct 22Peak in working hours93.5 cores
Oct 23Peak in working hours125.2 cores
Oct 24Peak in working hours125.1 cores
Oct 25Peak in working hours151.1 cores
Oct 26Peak in working hours148.4 cores
Oct 27Peak in working hours108.6 cores
Oct 28Peak in working hours155.1 cores
Oct 29Peak in working hours153.9 cores
Oct 30Peak in working hours185.7 cores
Oct 31Peak in working hours198 cores
Nov 1Peak in working hours158.6 cores
Nov 2Peak in working hours147.1 cores
Nov 3Peak in working hours122.8 cores
Nov 4Peak in working hours152 cores
Nov 5Peak in working hours212.9 cores
Aug 11Capacity240 cores
Aug 12Capacity240 cores
Aug 13Capacity240 cores
Aug 14Capacity240 cores
Aug 15Capacity240 cores
Aug 16Capacity240 cores
Aug 17Capacity240 cores
Aug 18Capacity240 cores
Aug 19Capacity240 cores
Aug 20Capacity240 cores
Aug 21Capacity240 cores
Aug 22Capacity240 cores
Aug 23Capacity240 cores
Aug 24Capacity240 cores
Aug 25Capacity240 cores
Aug 26Capacity240 cores
Aug 27Capacity240 cores
Aug 28Capacity240 cores
Aug 29Capacity240 cores
Aug 30Capacity240 cores
Aug 31Capacity240 cores
Sep 1Capacity240 cores
Sep 2Capacity240 cores
Sep 4Capacity240 cores
Sep 5Capacity240 cores
Sep 6Capacity240 cores
Sep 7Capacity240 cores
Sep 8Capacity240 cores
Sep 9Capacity240 cores
Sep 10Capacity240 cores
Sep 11Capacity240 cores
Sep 12Capacity240 cores
Sep 13Capacity240 cores
Sep 14Capacity240 cores
Sep 15Capacity240 cores
Sep 16Capacity240 cores
Sep 17Capacity240 cores
Sep 18Capacity240 cores
Sep 19Capacity240 cores
Sep 20Capacity240 cores
Sep 21Capacity240 cores
Sep 22Capacity240 cores
Sep 23Capacity240 cores
Sep 24Capacity240 cores
Sep 25Capacity240 cores
Sep 26Capacity240 cores
Sep 27Capacity240 cores
Sep 28Capacity240 cores
Sep 29Capacity240 cores
Sep 30Capacity240 cores
Oct 10Capacity240 cores
Oct 11Capacity242.7 cores
Oct 12Capacity256 cores
Oct 13Capacity256 cores
Oct 14Capacity256 cores
Oct 15Capacity256 cores
Oct 16Capacity256 cores
Oct 17Capacity256 cores
Oct 18Capacity256 cores
Oct 19Capacity256 cores
Oct 20Capacity256 cores
Oct 21Capacity256 cores
Oct 22Capacity256 cores
Oct 23Capacity256 cores
Oct 24Capacity256 cores
Oct 25Capacity256 cores
Oct 26Capacity256 cores
Oct 27Capacity255 cores
Oct 28Capacity256 cores
Oct 29Capacity256 cores
Oct 30Capacity256 cores
Oct 31Capacity256 cores
Nov 1Capacity256 cores
Nov 2Capacity256 cores
Nov 3Capacity256 cores
Nov 4Capacity256 cores
Nov 5Capacity256 cores
[Placeholder]Tokens per day, local model serverUpdated 6 November 2026Link

Tokens read and written each day by the model server running on avalon.

025M50M75M100MOctNov
  • avalon model server

Shows the last 90 days.

Show as table
Tokens per day, local model server, as of 2026-11-06
Dayavalon model server
Oct 280
Oct 2954.6M
Oct 3078.6M
Oct 3154.3M
Nov 132.3M
Nov 245.1M
Nov 346.8M
Nov 494.2M
Nov 5670.9K
Nov 62.3M
[Placeholder]Agent activityUpdated 6 November 2026Link

Subagents launched each finished week, and the most running at once that week.

01K2K3K4KSubagents launchedMost running at once015304560JulAugSepOct
  • Subagents launched
  • Most running at once (right axis)
Show as table
Agent activity, as of 2026-11-06
WeekSubagents launchedMost running at once
Week of Jul 1330523
Week of Jul 201,77123
Week of Jul 273,14423
Week of Aug 32,94642
Week of Aug 102,84036
Week of Aug 1781226
Week of Aug 241,29222
Week of Aug 3156423
Week of Sep 768425
Week of Sep 1451229
Week of Sep 2190228
Week of Sep 281,61241
Week of Oct 51,14128
Week of Oct 121,33640
Week of Oct 191,36940
Week of Oct 261,77937