fail-closed-setup
A runner setup that lets a broken test pass.
19 / 19 true, 95% Wilson interval 0.83 to 1.00, on 10 open-source projects.
Shown by default at info. Gates only if you raise its severity.
What it reports
The setup of a Jest or Vitest config lets a test pass over a condition that should fail it:
- mock reset: a mock or spy set in one test survives into the next;
- console guard: a
console.errororconsole.warnlogged during a test does not fail it; - unhandled requests: a request no MSW handler matches passes with a warning;
- global server: the MSW server is not started in a
beforeAllof the setup.
The two MSW guards are judged only when the suite has a mock server.
The finding is about the config and the setup files it loads, not about one test. It is reported once per missing guard, at the setup file (or the config when it has none). A config that cannot be read statically gives one could not decide row instead.
Example
jest.config.ts (2)
:0 setup does not guard the console (no throwing console.error/warn spy or fail-on-console package)
:0 setup does not restore mocks between tests (no restoreMocks or afterEach restoreAllMocks)What to do
Add the missing guard once, in the config or a setup file, so every test under that config fails on it:
// vitest.config.ts
export default defineConfig({
test: { restoreMocks: true, setupFiles: ['./test/setup.ts'] },
});// test/setup.ts
beforeEach(() => {
vi.spyOn(console, 'error').mockImplementation((...args) => {
throw new Error(args.join(' '));
});
});
beforeAll(() => server.listen({ onUnhandledRequest: 'error' }));A fail-on-console package counts as a console guard. An afterEach(() => vi.restoreAllMocks()) counts as a mock reset; an afterAll one does not.
Make it gate
It reports at info until you raise it:
// .spirolorc.jsonc
{ "rules": { "fail-closed-setup": "warn" } }spirolo explain fail-closed-setup prints the same claim and status from your installed binary.