Here's a dirty secret in computer science research: when a paper says "our new system is 12% faster than the baseline," that number often isn't backed by any actual statistical test. It's just two averages compared with the naked eye. And when researchers do run tests, they often run lots of them — across different benchmarks, workloads, and configurations — which introduces a well-known problem called the multiple comparisons problem.

The multiple comparisons problem is straightforward once you see it. If you flip a coin 20 times looking for "something surprising," you'll probably find something surprising just by chance. Same with benchmarks: run enough comparisons and some will look statistically significant even if nothing real is happening. Statisticians have tools to correct for this (Bonferroni, Benjamini-Hochberg, and friends), but those tools need to know how many comparisons you actually did and how they were structured. A list of p-values alone isn't enough — you need to know what the analyst was looking at, including comparisons they considered but didn't report.

This is where Tacet comes in. It's a programming language (well, more of a DSL and type system) that forces you to declare your analysis structure upfront. When you write your evaluation in Tacet, the language tracks:

• What quantities you generated
• What comparisons you made — and could have made
• How those comparisons are grouped and related

The type system then automatically accounts for statistical validity, applying the right corrections based on what you actually did. You can't accidentally hide a comparison from the bookkeeping, because the language sees everything you did before you got to the "reported" number.

The key insight is treating statistical validity as a type system problem rather than an ethics or discipline problem. Instead of trusting researchers to remember every comparison they ran, the compiler does the accounting. This is analogous to how memory-safe languages don't ask you to be careful with pointers — they make certain classes of bugs unrepresentable.

It's a small but pointed intervention in the reproducibility crisis. Most fixes for bad empirical practice ask people to be more virtuous. Tacet instead makes it easier to be correct than to be sloppy, by pushing the burden onto tooling.