InterviewSkill
Probability and statistics basics for data, ML, experimentation, and model evaluation interviews.
It is the probability of observing results at least as extreme as the sample, assuming the null hypothesis is true.
A small p-value suggests the observed data is unlikely under the null, but it does not prove the alternative hypothesis or measure practical importance.
It is a range of plausible values for a population parameter.
A 95% confidence interval means that if the sampling process repeated many times, about 95% of intervals would contain the true parameter.
Sample means tend toward a normal distribution as sample size grows.
This is useful because many inference methods rely on approximate normality of sample means even when raw data is not normal.
Correlation shows association; causation means one variable produces a change in another.
Correlation can be caused by confounders, reverse causality, or coincidence. Causal claims need experiments or careful causal analysis.
Type I is a false positive; Type II is a false negative.
Type I errors reject a true null hypothesis, while Type II errors fail to reject a false null. The tradeoff depends on alpha, power, and sample size.
Power is the probability of detecting a real effect when one exists.
Power increases with larger sample size, lower noise, larger effect size, and a more permissive significance threshold. Low power makes experiments inconclusive.
Define the hypothesis, primary metric, sample size, randomization unit, guardrail metrics, and decision rule before launch.
Good A/B tests avoid peeking, metric switching, sample-ratio mismatch, and interference between groups. The design should match the product decision.
A confounder is related to both the treatment and the outcome, making the observed relationship misleading.
Confounding is why correlation can look causal. Randomization, stratification, matching, or regression controls can reduce confounding risk.