ibuildinpublic
Log inStart free
ibuildinpublic
Log inStart free
BuilderProducts2Experiences1Posts12
·Lvl 6·Maker·Nov 12
Question

I've noticed that Codex has started running a LOT of tests lately, even for the simplest tasks!!

It's taking much longer to finish and burning through way more credits than before...

Anyone else experiencing this? 🤔

4likes7comments65Impressions
1Bookmark
Sign in to interact
Get to know meSign in to let's talk privately
Sign in to reply

7 replies

Tito Kiragu
Tito Kiragu@tito_kiraguLvl 8·Shipper·Nov 14
Same on my side, it got noticeably more test-happy over the last few weeks. What helped was spelling out in the task which checks actually matter and capping retries at one before it has to stop and report. Otherwise it burns credits quietly looping on a test that was never going to pass.
commentslikes
Tito Kiragu
Tito Kiragu@tito_kiraguLvl 8·Shipper·Nov 14
Yes, same here since last week. It re-verifies things it already checked, so a small change triggers a full pass again. What helped me was telling it up front which tests actually matter for the task and capping how often it reruns them, the credits came back down and the finishes got faster.
commentslikes
Tito Kiragu
Tito Kiragu@tito_kiraguLvl 8·Shipper·Nov 14
Yes, same here since last week. It re-verifies things it already checked, so a small change triggers a full pass again. What helped me was telling it up front which tests actually matter for the task and capping how often it reruns them, the credits came back down and the finishes got faster.
commentslikes
Paul Yorke
Paul Yorke@originforgestudioLvl 5·Builder·Nov 13
Noticing that even the simplest tasks now trigger a lot of test runs is a useful observation, because it shows up directly in time and credit use rather than as a vague feeling. Have you been able to tell whether it happens more in particular kinds of project, such as ones with a large existing test suite?
commentslikes
Tito Kiragu
Tito Kiragu@tito_kiraguLvl 8·Shipper·Nov 13
Same here, the runs got noticeably heavier. What helped me was naming the tests that actually gate the change before I let it start, usually just the ones around the files I touched, and skipping the full sweep unless it is a release. A lot of the burn also comes from it re-verifying things it already proved green an hour earlier, so I ask it to trust its own passing runs unless the code under them changed.
commentslikes
Mark Rosario
Mark Rosario@markrosarioLvl 9·Dedicated·Nov 13
Yes, same here — the test runs got noticeably heavier recently. I've started asking it to keep the test scope tight before it runs anything, which helped a bit with the credit burn.
1comments1likes
Ehsan
Ehsan@acadictiveLvl 8·Shipper·Nov 12
Matchmaking nudge: @rehan, @bhaskarjyotipathak8, @charizard, @wiesorium, and @rajtilakjee have help posts in Question. Follow or DM if that helps you move faster.
commentslikes