# Chi-Square, Cramér's V and P-Values: Testing Significance in Survey Cross-Tabs Source: PaperSurvey.io Help Center URL: https://www.papersurvey.io/help/statistical-analysis/article/chi-square-tests-and-cross-tab-significance --- A cross-tab answers the question "do these two answers move together?". In this walkthrough we use the Employee Wellbeing Study, a 300-respondent survey, to test whether shift workers think about quitting more often than their colleagues. PaperSurvey runs the chi-square test of independence for you and reports the p-value and effect size on every comparison card, so you rarely need to compute anything by hand. When you do have counts from outside the app, the free [Chi-Square Calculator](/pages/chi-square-calculator) applies the identical math to a pasted table. ## Run the comparison 1. Open the survey and go to **Analysis**, then the **Compare Responses** tab. 2. Under **How do responses differ based on:** pick the grouping question. Here we choose "Which shift do you usually work?". The basis question must be close-ended or a rating/NPS scale; Number and open text questions cannot be compared here. 3. PaperSurvey automatically cross-tabulates every other eligible question against your choice and runs a chi-square test on each pair. Cards are sorted so significant, strong associations come first. In the default Simple mode the page translates the statistics into plain language. For our shift question the insight banner reads: "1 of 5 questions show a notable relationship with Which shift do you usually work?. The strongest association is with Are you considering leaving your job in the next 12 months?. 4 other questions showed no notable relationship." [Image: Compare Responses in Simple mode with the plain-language insight banner] Each card offers **Side by side**, **Stacked** and **Proportions** chart styles, and a **Chart**, **Table** or **Both** view. The page also reminds you that these comparisons are exploratory. ## Read the chi-square result in Advanced mode Switch the toggle in the top right from **Simple** to **Advanced** to see the full test statistics. The card for "Are you considering leaving your job in the next 12 months?" shows the side-by-side bars and the cross-tab table, with the stats line underneath: [Image: Advanced cross-tab card with chi-square, p-value, Cramér's V and margin of error] The table counts tell the story before any statistic does. Only 21 of 165 morning-shift workers say Yes to leaving, against 18 of 85 on the evening shift and 24 of 50 on the night shift. Almost half the night shift is considering leaving. The stats line confirms this is not noise: - **χ² = 30.5612, df = 4**. The chi-square statistic measures how far the observed counts sit from what independence would predict. Degrees of freedom = (rows - 1) × (columns - 1), here (3 - 1) × (3 - 1) = 4. - **p = < 0.0001, Sig.** The probability of seeing a table this uneven if shift and leaving intention were unrelated. PaperSurvey flags significance at p < 0.05 and displays very small values as "< 0.0001". - **V = 0.2257 (Small)**. Cramér's V is the effect size on a 0 to 1 scale. The strength labels are the same bands the free [Cramér's V Calculator](/pages/cramers-v-calculator) uses: 0.1 is small, 0.3 medium, 0.5 large. So this is a real but modest association, worth acting on without overselling it. - **MoE ±5.7%**. The margin of error for the percentages at this sample size, at 95% confidence. In Advanced mode with the **Table** or **Both** view you can also tick the **z-scores** checkbox to overlay adjusted standardized residuals on each cell, highlighting exactly which cells drive the result (values beyond ±1.96 are colored). ## What the p-value means, and multiple comparisons A p-value converts your test statistic into a probability. If you ever need to do that conversion yourself, for example from a χ² value and df quoted in a report, the [P-Value Calculator](/pages/p-value-calculator) accepts a z, t, chi-square or F statistic and returns the same p the app computes. One test at p < 0.05 has a 5% false-positive risk, but Compare Responses runs one test per question. To protect you, when five or more comparisons are shown in Advanced mode a correction selector appears with **No correction**, **Bonferroni (p < X)** and **FDR (Benjamini-Hochberg)** options, and Bonferroni is pre-selected by default. That means the significance threshold on this page is already tightened to 0.05 divided by the number of comparisons. Our leaving-intention result clears even that stricter bar comfortably. ## Small tables get Fisher's exact test Chi-square is an approximation that weakens on small tables. Whenever both questions have exactly two answer options, PaperSurvey additionally runs Fisher's exact test (two-tailed) and treats it as the primary verdict. Here is "Have you taken a sick day in the last month?" grouped by "Do you exercise at least twice a week?": [Image: 2x2 cross-tab card with Fisher's exact test verdict] Of the 66 people who reported a sick day, only 21 (31.8%) are regular exercisers while 45 (68.2%) are not, even though exercisers make up 168 of the 300 respondents. The stats line reads "χ² = 20.0811 | df = 1 | p = < 0.0001 | Sig. (Fisher) | Fisher p = < 0.0001 | V = 0.2587 (Small) | MoE ±5.7%". The "Sig. (Fisher)" label tells you the verdict comes from the exact test, not the chi-square approximation. To run the same exact test on a 2x2 table you typed up from somewhere else, use the free [Fisher's Exact Test Calculator](/pages/fisher-exact-test-calculator); the chi-square calculator on its own does not fall back to Fisher. A note on question types: multiple-choice answers are exploded per selected option, so the cross-tab grand total counts option combinations rather than respondents. For clean 2x2 Fisher tables, prefer single-choice Yes/No questions like the ones here. ## Show your work with the Workbook Every Advanced-mode card has a **Workbook** link that downloads an Excel file for that comparison. It contains the raw and numerically coded data, an Observed Frequencies sheet, an Expected Frequencies sheet, a Standardized Residuals sheet, and a step-by-step Chi-Square Calculation sheet that builds the (Observed - Expected)² / Expected contributions and computes the p-value with a live =CHISQ.DIST.RT formula, plus the Cramér's V and margin of error figures. It is exactly the working you would show a reviewer or a statistics teacher. In Simple mode the **Download workbook** link produces a lighter file with a Summary sheet and observed counts, without the calculation sheets, so download from Advanced mode when you need the full derivation. ## Built-in safeguards Two protections keep you from quoting a test that does not hold up: - **Unreliable verdicts.** When any expected cell count falls below 1, the card is marked "Unreliable" instead of Sig. or Not Sig., the χ² and p figures are dimmed and struck through, and a warning explains the problem. A softer warning appears when more than 20% of cells have expected counts below 5. This is the app telling you the chi-square approximation cannot be trusted for that pairing, not a bug. - **Exploratory framing.** The page reminds you these are exploratory comparisons. Treat a surprising significant card as a hypothesis to confirm, not a final conclusion. ## Dashboards and external data The same test is available as the **Cross-Tab** block in Analysis views, so you can pin a key comparison to a dashboard, and it works on public shared views too. Note the widget always uses uncorrected p-values. PaperSurvey has no standalone contingency-table input; the app always builds tables from your collected responses. For counts from other sources, or to double-check a result by hand, paste the table into the [Chi-Square Calculator](/pages/chi-square-calculator), [Fisher's Exact Test Calculator](/pages/fisher-exact-test-calculator) or [Cramér's V Calculator](/pages/cramers-v-calculator). For deeper modelling, export your responses from the Responses tab to Excel, SPSS or R and continue in your stats software. ## Next steps - [Comparing group scores with t-tests and ANOVA](/help/statistical-analysis/article/comparing-groups-with-t-tests-and-anova.md) - [Margin of error and confidence intervals for survey results](/help/statistical-analysis/article/margin-of-error-and-confidence-intervals-for-survey-results.md) - [Filtering data in analysis](/help/analysis-reports/article/filtering-data-in-analysis.md) - [Exporting survey data](/help/exporting-data/article/exporting-survey-data.md) --- Need more? The complete PaperSurvey.io help center and blog is available as a single document at https://www.papersurvey.io/llms-full.txt.