compressor variants/ablation/ablate_comments.py · md5 6344f621186e · 27/27 runs graded · tasks: conan-io__conan-20032, django__django-13417, getmoto__moto-10057 · other arms: grephits
| Task | Base x/n | Arm y/n | W base | W arm | r | Zone | Steps b / a | Out/step b / a |
|---|---|---|---|---|---|---|---|---|
| conan-io__conan-20032 | 0/3 | 0/3 | 177K | 216K | -0.290 | none | 31 / 40 | 482 / 401 |
| django__django-13417 | 3/3 | 3/3 | 87K | 101K | -0.207 | maintain | 19 / 25 | 366 / 318 |
| getmoto__moto-10057 | 3/3 | 3/3 | 242K | 305K | -0.334 | maintain | 51 / 61 | 283 / 290 |
| Task | Arm | Rep | Result | Steps | Fresh | Cached | Output | Weighted | Time | Cost |
|---|---|---|---|---|---|---|---|---|---|---|
| conan-io__conan-20032 | baseline | r1 | failed | 28 | 31K | 823K | 10K | 143K | 125s | $0.026 |
| conan-io__conan-20032 | baseline | r2 | failed | 29 | 33K | 1.01M | 21K | 196K | 191s | $0.041 |
| conan-io__conan-20032 | baseline | r3 | failed | 35 | 32K | 1.20M | 13K | 192K | 192s | $0.033 |
| conan-io__conan-20032 | comments | r1 | failed | 33 | 33K | 990K | 10K | 163K | 162s | $0.028 |
| conan-io__conan-20032 | comments | r2 | failed | 33 | 29K | 992K | 14K | 169K | 175s | $0.031 |
| conan-io__conan-20032 | comments | r3 | failed | 55 | 34K | 2.05M | 26K | 317K | 330s | $0.054 |
| django__django-13417 | baseline | r1 | resolved | 15 | 20K | 331K | 6K | 71K | 286s | $0.015 |
| django__django-13417 | baseline | r2 | resolved | 30 | 25K | 787K | 10K | 134K | 255s | $0.024 |
| django__django-13417 | baseline | r3 | resolved | 13 | 17K | 257K | 5K | 57K | 220s | $0.012 |
| django__django-13417 | comments | r1 | resolved | 25 | 19K | 559K | 8K | 101K | 306s | $0.019 |
| django__django-13417 | comments | r2 | resolved | 15 | 15K | 286K | 5K | 58K | 206s | $0.012 |
| django__django-13417 | comments | r3 | resolved | 35 | 23K | 888K | 11K | 144K | 272s | $0.025 |
| getmoto__moto-10057 | baseline | r1 | resolved | 52 | 32K | 1.71M | 14K | 246K | 261s | $0.037 |
| getmoto__moto-10057 | baseline | r2 | resolved | 46 | 35K | 1.40M | 13K | 213K | 240s | $0.034 |
| getmoto__moto-10057 | baseline | r3 | resolved | 55 | 35K | 1.85M | 16K | 268K | 682s | $0.041 |
| getmoto__moto-10057 | comments | r1 | resolved | 86 | 40K | 3.37M | 24K | 450K | 831s | $0.061 |
| getmoto__moto-10057 | comments | r2 | resolved | 39 | 28K | 1.06M | 11K | 167K | 212s | $0.028 |
| getmoto__moto-10057 | comments | r3 | resolved | 58 | 37K | 2.09M | 18K | 300K | 1049s | $0.045 |