The Mann-Whitney U test is your go-to method for comparing two independent groups when your data does not follow a normal distribution or when sample sizes are small. It determines if there is a real, consistent difference in the central values of your groups rather than just looking at averages.
Understanding Group Differences
In many real-world scenarios, data doesn't fit perfectly into a standard distribution. When you need to compare two groups—such as conversion rates between two different website designs or performance metrics for two distinct teams—you need a way to detect if observed differences are genuine or simply due to chance.
This method evaluates whether the distribution of values in one group is shifted relative to the other. By focusing on the rankings of your data points, it effectively highlights whether one group consistently ranks higher or lower than the other.
Why Rank-Based Analysis Matters
Standard statistical tests often struggle when faced with skewed data or extreme values, as these can pull averages away from the true center. Because this test ranks the data, it ignores the exact distance between values and focuses solely on their relative order. This makes it a neutral and reliable way to identify differences in groups that contain outliers or irregular data points.
Interpreting Results on Lattice
Lattice provides a clear breakdown of your test results. You will see the p-value, which helps you decide if the difference is statistically significant, alongside the effect size. The effect size provides the 'so what' factor, telling you if the difference you see is small, medium, or large based on standardized thresholds.
Instead of complex statistical jargon, Lattice translates these outputs into plain language, allowing you to confidently conclude whether the performance of your groups is truly different and what the implications are for your next steps.
1 · Intent → method
An LLM picks svt_run_mannwhitney from a fixed catalog.
2 · Method → numbers
Deterministic Python engine runs the math. Same input → same output.
3 · Numbers → plain language
A second LLM translates the result into your domain’s vocabulary.
Why use the Mann-Whitney U test instead of a t-test?
A t-test assumes your data follows a specific bell-curve shape (normal distribution). If your data is skewed or has outliers, the Mann-Whitney U test is more reliable because it relies on the relative ranks of the data rather than the exact values.
What does the 'effect size' tell me in this test?
The effect size (rank-biserial r) tells you the magnitude of the difference between your groups. It classifies the difference as small, medium, or large, helping you distinguish between a statistically significant result and one that actually matters for your business or research.
Tool input schema
Schema for svt_run_mannwhitney not exported yet (run pnpm export:registry).