Tautological Tests

Somewhere in your codebase is a test that has never failed and never will. It asserts that a mock returns what you told it to return. It always passes. It’s useless. And a philosophical argument from 1936 explains exactly why.

The argument is verificationism. In Language, Truth, and Logic, A.J. Ayer split meaningful statements in two: analytic statements (“all bachelors are unmarried”) are true purely by definition and say nothing about the world; synthetic statements (“the server responds in under 200ms”) make a claim the world can actually confirm or refute. Anything else, he argued, is meaningless.

Swap “the world” for “the system under test” and you get a principle worth stealing:

A test is meaningful only if it could fail for a reason that tells you something about the system.

An analytic test is true by construction — it cannot lose, so it proves nothing. A synthetic test is one the running code could actually refute, and that possibility of failure is where its value lives. This is the software cousin of Popper’s falsifiability: a claim nothing could ever refute isn’t a strong claim, it’s an empty one.

Before writing an assertion, ask: could this ever be false if the system were broken? If no, delete it.

1. The obvious tautology

Nobody ships this on purpose, but it’s the pure form of the problem:

it('adds numbers', () => {
  expect(2 + 2).toBe(4);
});

It can never fail for a reason that tells you anything about your system, because there is no system in it. It’s analytic, tautologically true, and belongs to the language’s test suite, not yours.

2. The disguised tautology

This is the same test above wearing a costume. It mocks the exact thing it claims to verify, so the assertion checks the mock, not the code. It’s still analytic: the test oracle — whatever decides pass or fail — is the test’s own setup, not an observation about the system.

❌ Disguised tautology ✅ Synthetic fix 
// pricing.ts
export class PricingService {
  applyDiscount(price: number, percent: number): number {
    return price - (price * percent) / 100;
  }
}

// pricing.test.ts
import { PricingService } from './pricing';

it('applies a discount', () => {
  const service = new PricingService();
  jest.spyOn(service, 'applyDiscount').mockReturnValue(90);

  const result = service.applyDiscount(100, 10);

  expect(result).toBe(90); // asserting the mock we just configured
});

// pricing.ts
export class PricingService {
  applyDiscount(price: number, percent: number): number {
    return price - (price * percent) / 100;
  }
}

// pricing.test.ts
import { PricingService } from './pricing';

it('applies a discount', () => {
  const service = new PricingService();

  const result = service.applyDiscount(100, 10);

  expect(result).toBe(90);
});

The only difference is one line — jest.spyOn(...).mockReturnValue(90). In the bad version, applyDiscount never runs; the test just confirms that a stub returns what it was told to return. In the good version, the real subtraction and division execute. Swap the - for a +, or forget the / 100, and only the second test notices.

3. The synthetic fix

Same test as above, viewed from the other direction: it’s synthetic in Ayer’s sense, a genuine claim about the system that the running code could actually violate. A bug in the formula fails it. But it’s weak verification: it only checks one point in a much larger space of inputs — one price, one percentage.

it('applies a discount', () => {
  const service = new PricingService();
  expect(service.applyDiscount(100, 10)).toBe(90);
});

4. Weak verification in practice

Ayer knew conclusively verifying an empirical claim under all conditions is almost never possible, so he distinguished strong verification (conclusive proof) from weak verification (evidence that makes a claim probable). Testing lives entirely in the weak column — you never prove a system correct with examples, you only gather evidence. Property-based tests take that seriously: instead of one hand-picked expectation, they check a general rule against hundreds of generated inputs, including edge cases a human wouldn’t think to write.

❌ One hand-picked case ✅ A property, checked broadly 
it('applies a discount', () => {
  const service = new PricingService();
  expect(service.applyDiscount(100, 10)).toBe(90);
});

import fc from 'fast-check';
import { PricingService } from './pricing';

it('never returns a result outside [0, price]', () => {
  const service = new PricingService();

  fc.assert(
    fc.property(
      fc.float({ min: 0, max: 10_000 }),
      fc.float({ min: 0, max: 100 }),
      (price, percent) => {
        const result = service.applyDiscount(price, percent);
        expect(result).toBeLessThanOrEqual(price);
        expect(result).toBeGreaterThanOrEqual(0);
      }
    )
  );
});

This doesn’t prove applyDiscount is correct for every float — it can’t. It’s still evidence, not proof. But it’s broader evidence: a negative percent, a zero price, or a rounding error at the boundary will surface here, where a single toBe(90) would stay silent forever.

5. Verifiable in principle isn’t good enough

Ayer drew one more line: between statements verifiable in practice and those verifiable only in principle — decidable in theory, but beyond anyone’s reach to actually check. A test behind .skip, a feature flag that never flips in CI, or a flaky test quarantined and forgotten is verifiable only in principle. It could fail — but nothing ever gives it the chance. It protects you exactly as much as no test at all. Run it, or delete it.

The checklist

That’s the principle, but principles are hard to apply mid pull-request when you have thirty seconds and a green checkmark to review. Here’s the checklist form. Run any new or changed test through these questions before you commit it.

Before you write the assertion

  • Could this test fail? Name the defect it would catch. If you can’t name one, delete the test — it’s decoration, not verification.
  • Where does the expected value come from? It has to come from somewhere other than the code under test: a hand-computed value, a spec, a reference implementation, a golden file, a known invariant. If the “oracle” is the implementation copied into the assertion, you’ve proven x == x and nothing else.

Before you trust a double

  • Am I asserting my own setup? If a mock is configured to return X and the test checks that the collaborator returned X, the test verifies the mock, not the system. Apply the delete-the-system check: delete the production code and re-run the test. If it still passes, it was never testing anything.
  • Is this a real boundary, or a shortcut? Reserve doubles for genuine edges — network, clock, filesystem, third-party services, anything non-deterministic. Code you own and could exercise directly should run for real. Doubling everything turns the test into a mirror of the implementation: it breaks on every harmless refactor and misses every real bug.

Before you call it done

  • What would a failure actually tell me? A good assertion points at the defect. A pile of unlabelled expect calls only tells you something broke, somewhere.
  • Am I testing behaviour or structure? Assert on outputs, state, events, persisted effects — things a caller could observe. Assertions on private internals or call order break on refactors that change nothing observable, which teaches everyone to ignore red tests.
  • Is this empirical or analytic? If a test only exercises the language or a library — arithmetic, an ORM saving a row — it belongs to someone else’s test suite, not yours.
  • Am I still chasing proof? You can’t get it. Where a general law exists — round-trips, idempotence, a bound like 0 ≤ discount(price, rate) ≤ price — check it against hundreds of generated inputs instead of one hand-picked pair.
  • Did I cover the edges, not just the happy path? Empty, zero, one, maximum, invalid input, the error path. Failure handling is behaviour too, and it needs the same independent oracle as the happy path.
  • Is it deterministic? Time, randomness, ordering, concurrency, locale — if any of these can flip a test from pass to fail without a code change, it isn’t a test, it’s a coin flip that occasionally coincides with the truth.
  • Does it clean up after itself? No leaked state, no dependency on some earlier test having run first, unless that ordering is a deliberate, documented strategy.
  • Would a small mutation break it? Flip an operator, change a constant, delete a line. If nothing in the suite goes red, coverage was measuring execution, not verification.
  • Does it actually run? A test behind .only, a .skip with no reason, a flag that never flips in CI, or a quarantine nobody revisits protects nothing.

If you only keep one question, keep this one: if I deleted the code this test exercises, would the test still pass? If yes, you don’t have a test. You have a test-shaped object.

Ayer’s test for a meaningful statement turns out to be a rather good test for a meaningful test: if nothing in your system could ever prove it wrong, it was never really saying anything at all.