Reviewing Recognized Answers

The Verify tab shows you the answers PaperSurvey was unsure about. The Review tab shows you everything else: one row per recognized answer, across every response in the survey, so you can spot patterns the flagging rules would never catch on their own.

It is the right place to answer questions like "did the recogniser read this question consistently", "which answers did it accept at low confidence", and "what did this particular respondent actually write".

What each row shows

Column What it tells you
Entry № Which response the answer belongs to
Question The question that was answered
Image The exact part of the scanned page the answer was read from
Answer What PaperSurvey recorded
Recognition Accuracy How confident the recogniser was, from 0% to 100%
Reason Why the answer was flagged, if it was
Verified Whether a person has confirmed the answer
Verified by Who confirmed it, or whether it was accepted automatically
Updated When the answer last changed

Sort by Recognition Accuracy ascending to put the least certain answers first. That single sort is usually the fastest way to find the answers worth a second look.

Correcting a single answer

Most of the time you only need to fix a handful of answers, not all of them.

  1. Hover over the Answer cell of the row you want to change
  2. Click the pencil that appears beside the value
  3. The row opens underneath, showing the scanned snippet, the answer options, and the answer's history
  4. Choose the correct answer and click Confirm

The row closes and updates in place, so you keep your position in the list. Only one answer is open for editing at a time.

While an answer is open you can use keyboard shortcuts to work faster: Ctrl-Enter (Command-Enter on a Mac) confirms, Ctrl-Backspace clears the answer, Ctrl-I marks it invalid, and Ctrl-1 through Ctrl-9 pick an option by position.

Marking an answer as invalid

If the marks are contradictory and you genuinely cannot tell what the respondent meant, use Invalid Answer rather than guessing. The answer is recorded as invalid, exports show the invalid code (-97 by default), and the answer is left out of charts and statistics. This is different from Mark as no response, which means the respondent left the question blank.

Seeing who changed an answer

Every change to an answer is recorded. When you open an answer for editing, its Edit history appears below the options, showing each previous value, who changed it, and when. Changes to written answers are shown as a word-by-word difference, so you can see exactly what was altered.

Answers that have never been changed since recognition show no history.

Opening the whole response

A single answer sometimes only makes sense next to the rest of the sheet. Click the Entry № in any row to open that respondent's whole response.

The panel lists every answer they gave, in the order the questions appear on the survey, each with its scanned snippet, its recognition accuracy and its verification state. The scanned pages sit alongside, and clicking an answer brings up the page it was read from. You can edit any answer here the same way you would in the table.

Verify remaining at the bottom confirms the answers of that response that nobody has looked at yet. By default it leaves answers PaperSurvey flagged for a human, because those are exactly the ones that deserve individual attention. Clear the checkbox if you want to confirm those too.

Working through many answers at once

Quick Filters offers the searches most teams need, with a live count beside each, for example low accuracy choice questions that were accepted automatically.

Filter by Question narrows the list to one question, or to a whole family of them: number and date questions, choice questions, text questions, answers where "other" was selected, or answers with no response at all.

Each column header also filters on its own, so you can combine, for example, a recognition accuracy range with a particular flag reason.

Once the list shows what you want, tick the rows you need and use Bulk Actions:

  • Mark as verified confirms the selected answers as they stand
  • Mark as no response records the selected answers as blank
  • Mark as unverified flags the selected answers so somebody looks at them again

When every row on the page is ticked and more answers match your filters, PaperSurvey offers to extend the selection to the whole filtered set rather than just the current page.

Bulk actions are available to owners, administrators, members and operators. Viewers cannot open the Review tab.

Get Started with PaperSurvey.io

Start your 14-day free trial now, no credit card required.

Get Started