Data Mining
Searching data for patterns until something appears significant, without a prior hypothesis.
Formula
Expected False Positives = Number of Independent Tests x Significance Level
Unit
ratio (x, times)
In depth
Testing enough hypotheses guarantees that some will appear significant by chance alone, which is why a strategy found by searching is far weaker evidence than one predicted in advance and then tested. The arithmetic is unforgiving: at a 5% significance level, a hundred tests produce five apparent discoveries from pure noise. In finance this is compounded because the same historical data has been searched by thousands of people, so any pattern found in it has already survived an enormous unrecorded search. The defences are a prior economic reason for the pattern, out-of-sample testing, and honest reporting of how many things were tried.
Worked example
Testing 100 indicator combinations at a 5% threshold produces about 5 that look statistically significant purely by chance. Reporting only the best of them, without saying 100 were tried, describes noise as a discovery.
Illustrative figures, chosen so the arithmetic is easy to follow. Not a live price and not a valuation of any company.
Educational reference only
This entry explains what “Data Mining” means. It is not investment advice and not a recommendation to buy or sell any security. Any numbers above are illustrative, not live prices, and nothing here predicts price direction or rates a stock. Consider your own circumstances and consult a SEBI-registered investment adviser before acting.