ablA · listing

stage-1 fail

compressor variants/ablation/ablate_listing.py · md5 afdcc12fddfb · 18/18 runs graded · tasks: dask__dask-12430, vega__altair-4019 · other arms: bashother

Score

+0.000

90% CI +0.00 … +0.00

Weighted saving

-27.8%

pooled per run vs baseline

Resolved

0/6

baseline 0/6

Stage gates

S1 fail · S2 fail

tasks 0/2 (need 1)

Cost

$0.47

6 graded runs

Tokens per run vs baseline

bar = arm · tick = baseline · mean per graded run

Weighted419K of 328K↑ 27.8%

Fresh input39K of 30K↑ 26.6%

Cached input2.54M of 1.95M↑ 30.3%

Output42K of 34K↑ 23.5%

Steps66.7 of 54.2↑ 23.1%

Tasks

x = baseline resolves · y = arm resolves · r = log2(W base / W arm)

Task	Base x/n	Arm y/n	W base	W arm	r	Zone	Steps b / a	Out/step b / a
dask__dask-12430	0/3	0/3	324K	410K	-0.341	none	56 / 63	638 / 679
vega__altair-4019	0/3	0/3	332K	428K	-0.367	none	53 / 70	614 / 603

Runs

12 runs incl. baseline

Task	Arm	Rep	Result	Steps	Fresh	Cached	Output	Weighted	Time	Cost
dask__dask-12430	baseline	r1	failed	63	32K	2.32M	42K	388K	1355s	$0.073
dask__dask-12430	baseline	r2	failed	38	22K	1.01M	27K	203K	354s	$0.045
dask__dask-12430	baseline	r3	failed	66	32K	2.39M	36K	381K	1065s	$0.068
dask__dask-12430	listing	r1	failed	85	39K	4.15M	58K	626K	1248s	$0.106
dask__dask-12430	listing	r2	failed	47	26K	1.46M	33K	271K	346s	$0.056
dask__dask-12430	listing	r3	failed	57	31K	1.91M	37K	333K	367s	$0.065
vega__altair-4019	baseline	r1	failed	52	37K	1.84M	36K	327K	1153s	$0.065
vega__altair-4019	baseline	r2	failed	28	21K	634K	14K	127K	233s	$0.027
vega__altair-4019	baseline	r3	failed	78	39K	3.49M	51K	541K	1323s	$0.094
vega__altair-4019	listing	r1	failed	53	49K	1.61M	36K	318K	1010s	$0.067
vega__altair-4019	listing	r2	failed	82	42K	3.57M	52K	555K	541s	$0.096
vega__altair-4019	listing	r3	failed	76	45K	2.52M	38K	410K	1430s	$0.074