When a dashboard shows you a standard deviation, a p-value or an NPS, it is fair to ask whether the number is right. Our answer is not "trust us". Every statistic PaperSurvey reports is checked against SciPy, the reference scientific computing library used across academia and industry, by an automated test suite that runs on every code change. This article explains how that works and how you can reproduce any figure yourself. If you want the exact formula behind a specific number, the Statistical Methodology page lists them all.
One golden fixture, generated from SciPy
The core of the system is a single golden fixture: a file of known inputs paired with the answers SciPy produces for them. For each statistic we compute, from the mean and standard deviation through correlations, t-tests, chi-square, Cronbach's alpha and NPS, the fixture records SciPy's result to full precision. SciPy is the arbiter because it is independently maintained, widely peer-reviewed and the de facto standard for scientific computing in Python.
Because the expected answers come from an outside reference rather than from our own code, the fixture cannot quietly agree with a bug of ours. If our formula drifted, the fixture would still hold SciPy's correct value, and the test would fail.
Both engines are tested against the same fixture
PaperSurvey computes statistics in two places. One engine powers the dashboards, the automatic reports and the exports. A second engine powers the public calculators and the live figures you see as you edit an analysis view. These are two separate implementations of the same formulas.
On every code change, both engines are run against the same golden fixture. Each engine's results must match SciPy to the last meaningful decimal. This is what lets a scanned paper survey, a web survey, an export and a public calculator all report the same number for the same data. If a change nudged any figure off its SciPy reference, the test suite blocks it before it can ship.
The conventions are documented, not hidden
Many statistics have more than one defensible definition, and the honest thing is to state which one we use rather than leave you guessing why our figure differs from another tool by a hair. Our choices match the most common conventions in Excel and R:
- Standard deviation and variance use the sample formula that divides by n minus 1, treating your responses as a sample of a wider audience. This matches Excel's STDEV.S and R's
sd(). - Percentiles and quartiles use linear interpolation, the type 7 definition, which is what Excel's PERCENTILE.INC and R's default
quantile()return. - P-values are two-tailed unless a test is inherently one-sided, so they answer "is there a difference in either direction".
- NPS is rounded once at the end, as the percentage of promoters (9 to 10) minus the percentage of detractors (0 to 6).
Every one of these conventions, and the exact formula for each statistic, is spelled out on the Statistical Methodology page alongside the academic references behind them.
Reproduce any number yourself
The fixture proves our code matches SciPy, but you do not have to take that on faith either. You can recompute any figure from the same raw data the app uses:
- Open the Responses tab and export your data. The Excel, CSV, SPSS and R exports all contain the raw per-respondent answers, so you can load them into your own tool and run the statistic yourself.
- For a quick check without leaving the browser, paste the same numbers into the matching free calculator. Because the calculators are tested against the same SciPy reference as the dashboard, they agree with it for the same data.
- Where a widget offers a workbook download, the Excel file carries the calculation as live formulas rather than pasted values, so you can trace each cell back to the inputs and watch the result recompute.
If you ever find a figure that does not reconcile, it is almost always a difference in convention (for example a population standard deviation instead of the sample one, or rounding applied at a different step). The Statistical Methodology page is the place to check which convention we use.
Why this matters
Survey data often feeds decisions that matter: a pass or fail threshold on an exam, a reported NPS in a board deck, a significance claim in a research paper. Testing every formula against SciPy on every change means the numbers you present are the numbers a statistician would get from the same data, and that you can show your working if anyone asks.
Next steps
- Statistical Methodology lists every formula and convention with references.
- Descriptive Statistics for Survey Data walks through the spread and shape statistics on real data.
- Exporting survey data covers the Excel, CSV, SPSS and R exports you can recompute from.
- The free calculators let you check any figure against the same tested engine.