Glossary

Placebo test: definition and a real example

By Justin JaegerPublished Updated

Running random signals through the same test on the same markets. A real signal has to do clearly better than this range.

Definition

Running random signals through the same test on the same markets. A real signal has to do clearly better than this range.

Example from the register

Public macro, flow and positioning signals kept their direction after 2015 in about 49% of cases where they looked significant. Random placebo signals on the same 135 markets did so in 53%. Details in the placebo study.

Why it matters

Most strategy research reports the best result and leaves out how it was found and what failed. Base rates, placebo tests, trial counts and out-of-sample checks are the four tools that separate an idea with evidence behind it from a lucky backtest.

See it applied in the base rates, the dead ends and the methodology.

Questions

What is a placebo test?

Running random signals through the same test on the same markets. A real signal has to do clearly better than this range.