← Subnet 114 · test bench

s1_edge10 · edge10

stage-1 fail

compressor variants/exp_edge10.py · md5 6152fa6bb754 · 23/150 runs graded · tasks: django__django-12419, django__django-13821, django__django-14373, sympy__sympy-15875, sympy__sympy-19637 · other arms: edge10_footer

Score
-0.094
90% CI -0.27 … +0.09
Weighted saving
-18.7%
pooled per run vs baseline
Resolved
8/8
baseline 8/8
Stage gates
S1 fail · S2 fail
tasks 5/5 (need 3)
Cost
$0.32
8 graded runs

Tokens per run vs baseline

bar = arm · tick = baseline · mean per graded run
Weighted209K of 176K↑ 18.7%
Fresh input25K of 24K↑ 3.2%
Cached input1.22M of 987K↑ 23.2%
Output21K of 18K↑ 17.3%
Steps29.5 of 27.0↑ 9.3%

Tasks

x = baseline resolves · y = arm resolves · r = log2(W base / W arm)
TaskBase x/nArm y/nW baseW armrZoneSteps b / aOut/step b / a
django__django-124192/22/2123K243K-0.986maintain24 / 34480 / 750
django__django-138212/22/2206K195K+0.075maintain28 / 32668 / 551
django__django-143732/22/247K44K+0.076maintain15 / 12198 / 318
sympy__sympy-158751/11/1604K666K-0.143maintain62 / 671159 / 1032
sympy__sympy-196371/11/156K41K+0.450maintain18 / 12204 / 228

Runs

20 runs incl. baseline
TaskArmRepResultSteps FreshCachedOutputWeightedTimeCost
django__django-12419baseliner1resolved3327K932K13K160K390s$0.030
django__django-12419baseliner2resolved1619K396K9K86K431s$0.019
django__django-12419edge10r1resolved4033K1.60M33K291K1283s$0.059
django__django-12419edge10r2resolved2929K1.07M20K195K934s$0.039
django__django-13821baseliner1resolved2843K1.05M14K189K448s$0.036
django__django-13821baseliner2resolved2933K1.16M24K222K505s$0.046
django__django-13821edge10r1resolved4339K1.62M23K271K463s$0.049
django__django-13821edge10r2resolved2125K587K12K120K462s$0.025
django__django-14373baseliner1resolved1212K174K2K34K391s$0.006
django__django-14373baseliner2resolved1814K314K5K59K414s$0.012
django__django-14373edge10r1resolved1212K198K4K43K337s$0.009
django__django-14373edge10r2resolved1213K203K4K45K392s$0.010
sympy__sympy-15875baseliner1resolved6232K3.56M72K604K1352s$0.117
sympy__sympy-15875baseliner2no result–––––––
sympy__sympy-15875edge10r1resolved6734K4.25M69K666K1322s$0.119
sympy__sympy-15875edge10r2no result–––––––
sympy__sympy-19637baseliner1resolved1814K306K4K56K346s$0.011
sympy__sympy-19637baseliner2no result–––––––
sympy__sympy-19637edge10r1resolved1213K197K3K41K330s$0.008
sympy__sympy-19637edge10r2no result–––––––