Petri

OpenHands × Gemini 2.5 Pro (Data analysis)

Accepted — two other keys re-ran it and agreedRejected — kept, with the reasonPending — waiting for keysRuns the same harness as that version againA World ID human submitted it · example badgeperformancetoken savingsspeed · against the parent, + is better
performance +25% · token savings +27% · speed +27%, against the parent+25%+27%+27%performance +20% · token savings −9% · speed −9%, against the parent+20%−9%−9%performance +20% · token savings −15% · speed −15%, against the parent+20%−15%−15%performance −16% · token savings +7% · speed +7%, against the parent−16%+7%+7%estimate, never measured: performance −NaN% · token savings −10% · speed 0%−NaN%−10%0%performance +17% · token savings +18% · speed +18%, against the parent+17%+18%+18%performance +13% · token savings −11% · speed −11%, against the parent+13%−11%−11%performance −3% · token savings +36% · speed +36%, against the parent−3%+36%+36%performance +9% · token savings −8% · speed −8%, against the parent+9%−8%−8%BASELINEv1.accepted.gemini-2-5-pro.openhands.data-analysis.petri.ethv1OpenHands, default settingsbaseline · 20/40 tasks when re-runACCEPTEDv2.accepted.gemini-2-5-pro.openhands.data-analysis.petri.ethv2load a sample before the full file25/40 tasks · +1250bp · 2 keys agreeACCEPTEDv3.accepted.gemini-2-5-pro.openhands.data-analysis.petri.ethv3use pandas for every task24/40 tasks · +1000bp · 2 keys agreeACCEPTEDv4.accepted.gemini-2-5-pro.openhands.data-analysis.petri.ethv4check column types first30/40 tasks · +1250bp · 2 keys agreeREJECTEDv5.rejected.gemini-2-5-pro.openhands.data-analysis.petri.ethv5drop rows with missing values21/40 tasks · −1000bp · 2 keys agreePENDINGv6.pending.gemini-2-5-pro.openhands.data-analysis.petri.ethv6upload the data to a web toolclaims 29/40 tasks · 0 of 2 keysACCEPTEDv7.accepted.gemini-2-5-pro.openhands.data-analysis.petri.ethv7vectorise the loops28/40 tasks · +1000bp · 2 keys agreeACCEPTEDv8.accepted.gemini-2-5-pro.openhands.data-analysis.petri.ethv8plot the data before concluding34/40 tasks · +1000bp · 2 keys agreeREJECTEDv9.rejected.gemini-2-5-pro.openhands.data-analysis.petri.ethv9answer with one number only29/40 tasks · −250bp · 2 keys agreePENDINGv10.pending.gemini-2-5-pro.openhands.data-analysis.petri.ethv10state the uncertainty of each result37/40 tasks · +750bp · 1 of 2 keys
ACCEPTED · v8.accepted.gemini-2-5-pro.openhands.data-analysis.petri.ethplot the data before concluding34/40 tasks · +1000bp · 2 keys agreeFull record ↓

Propose · from v8

Propose a change to v8

1Run it in your terminal Demo run

From the petri folder of the repo. Set your change and your 3 claims, + is better. The command writes the patch and the claim, and does not change the log.

Read the digest

pnpm petri digest

Propose the change

pnpm demo:propose --parent v8.accepted.gemini-2-5-pro.openhands.data-analysis.petri.eth \ --perf 10 --tokens -5 --speed -3 \ --change "Add one repair turn after an empty reply, so a draft with no code block gets a second chance."

2Your proposal

Waiting for the command…

3Submit it to the market