I'm currently on a team that uses post-checkin validation in our developers' workflow:
- Developer checks in code (and hopefully runs tests first)
- CI machine syncs, builds, run tests
- If tests fail, code is already checked in but dev (and related people) are made aware (e.g. mail)
I've seen or read of several teams who instead use pre-checkin validation (aka "gated checkins"):
- Dev submits changes (and hopefully runs tests first)
- Checkin system builds changes and runs test
- If tests fail, changes are not checked in, and dev is informed of test failures.
- If tests pass, changes are checked in, and dev is informed.
It seems like there's a couple of tradeoffs between these options, such as the turnaround for developers to know if their code is submitted (post-checkin is immediate, pre-checkin waits for the validation to finish), and the known state of code that is checked in (pre-checkin flushes out all issues found by the validation tests, but post-checkin could allow for easily buggy code to get checked in on the Friday afternoon before the dev goes on vacation).
Is there any reference material on the tradeoffs/benefits between these two approaches? Or any measured research on whether developers are affected by either one? (E.g. developer productivity is interruptions from pre-checkin validation distracting, since their code is not checked in; or, developers with pre-checkin validation tend to find bugs quicker and can turn around high quality features faster).