The Bayesian t-test is an alternative to standard group comparison methods. Use it when you need to compare two groups and want to know the probability of a true difference rather than just a p-value. It is particularly effective when your data has outliers or unequal group sizes.
Understanding Probabilistic Group Comparisons
Traditional tests often rely on rigid assumptions about data distribution. The Bayesian t-test shifts the focus by calculating the full range of likely values for the difference between two groups. This allows you to speak in terms of actual probabilities—for example, how likely it is that group A truly outperforms group B.
By using the Bayesian t-test, you get a clearer picture of your data. Instead of a binary result, you receive a range of credible values, known as the 95% Highest Density Interval (HDI), which shows the most plausible values for the difference between your groups.
Handling Outliers with Student-t Likelihood
Data is rarely perfectly bell-shaped. Standard methods can be heavily influenced by one or two extreme values. The Bayesian t-test uses a 'Student-t' distribution, which is naturally more resistant to outliers. This keeps your results stable even if your data has unexpected extremes.
This approach essentially 'ignores' the extreme noise that might otherwise pull the mean away from the true center of your data, leading to a more accurate representation of the groups you are studying.
Making Practical Decisions
Beyond simple significance, the Bayesian t-test helps you decide if a difference actually matters. By defining a 'Region of Practical Equivalence' (ROPE), you can set a threshold for what constitutes a meaningful difference. The tool then calculates how much of the probability distribution falls within this threshold.
This allows for three clear decision states: the groups are practically equivalent, they are significantly different, or the evidence is currently inconclusive. This structure helps you make data-backed choices without needing to guess if a tiny statistical difference actually changes anything in reality.
1 · Intent → method
An LLM picks bayesian_ttest from a fixed catalog.
2 · Method → numbers
Deterministic Python engine runs the math. Same input → same output.
3 · Numbers → plain language
A second LLM translates the result into your domain’s vocabulary.
How is the Bayesian t-test different from a regular Welch t-test?
The Bayesian t-test uses a Student-t distribution to account for outliers and provides a full probability distribution of the differences, whereas a standard Welch t-test focuses on a single p-value.
What does the 'ROPE' result mean in this test?
ROPE stands for Region of Practical Equivalence. It defines a range of differences so small they are essentially zero. If the majority of your result falls within this range, you can conclude the two groups are practically equivalent.
Tool input schema
Schema for bayesian_ttest not exported yet (run pnpm export:registry).