How BlockSurvey Score uses AI decision-making to tell you if a survey is ready to publish

A score you can trust has to be earned

Before a creator publishes, BlockSurvey Score reads the whole survey and gives it a score out of 100, a grade, a verdict (ready, needs work or not ready) and a list of specific problems to fix. We call the problems "roasts" because they are blunt on purpose: "your title tells respondents nothing about what they're walking into" is more useful than "improve the title".

Behind that score sit a lot of small questions. Is the title specific? Does the survey open with something easy? Are two questions asking the same thing? Does this question ask about two things at once? Each one is a judgment about meaning. A fixed rule cannot tell a good opening question from a bad one, and a language model asked to "review this survey" gives a different essay every time, with no number we can stand behind.

So we split the work three ways: code counts, Jev judges, and code turns the judgments into a score.

Step 1: Code does the counting

Anything that is a fact, not an opinion, is settled in plain code. A raw survey carries around 60 settings per question, almost all defaults, so first we cut it down to a compact digest of what a survey methodologist would actually read: the title, the welcome and thank-you text, and each question with its options.

Then code checks the things it can check exactly: how many questions there are, how long the survey will take, whether a required question has no answer options, whether a number range overlaps. These are the only checks that can block a survey outright, because they do not depend on anyone's opinion. Jev does not count or do arithmetic reliably, so we never ask it to.

Step 2: Jev makes the judgments

Jev is a decision model. We give it some text and a typed question about it, and it answers with a choice, a score on a scale, or a yes/no probability. It does not write text. The rubric is a long list of such questions, and each one uses the type that fits it:

• Score for "how well": how clearly does the title say what the survey is about? How logical is the order? The answer sits on an ordered scale, from "generic" to "specific".
• Noul for "is there a problem": does this question ask two things at once? Do two options overlap? Is any question presuming something about the respondent? The answer is a probability.
• Choice for "which kind": is this survey research, feedback or a form? Is this element a real question, or a statement or acknowledgement?

Why Jev. There are dozens of these judgments, and every one is short and independent of the others. Jev evaluates all the questions in a request in parallel, so asking many of them costs about the same time as asking one. Its answers also come with probabilities, so we know how sure each judgment is, and that matters for how we use it.

The work runs in rounds. First, one small request settles what kind of survey this is and what role each element plays. That matters: a form is not scored on research coverage it was never meant to have, and a "Got it" step is not counted as a duplicate question. Then the whole-survey questions go in one request, and the per-question judgments go in small batches that run at the same time. The questions in a batch are written to be read literally, and to describe concrete situations at each level, not vague degrees, because Jev answers exactly what it is asked.

Code counts. Jev judges. Code does the maths.

What a judgment turns into

Here is how a few Jev judgments become what the creator sees.

The words of every roast are written by us, in code, ahead of time, one per rubric item and level. Jev supplies the judgment and we supply the wording. That is deliberate. Jev does not generate text, and it means the same problem is always described the same way.

When a judgment is about the whole survey, such as "there are redundant questions", we also ask Jev a second, per-question version of it. Its answers point the roast at the specific questions to fix, so the creator gets a checklist, not a vague complaint.

Step 3: Code turns judgments into a score

Every number in the report is policy, not inference. Each rubric item has a weight and sits in one of six categories: purpose and framing, structure and flow, coverage and gaps, question quality, respondent experience, and trust and ethics. Code combines the judgments in each category into a category score, combines those into the overall 0 to 100, and maps it to a grade and a verdict.

Because the maths is in code, changing how much something matters is a one-line change to a weight. Nothing needs to be re-asked of Jev, and no prompt needs rewriting. The same Jev answers can be re-scored under a new policy.

When Jev is not sure

A judgment is not the same as a fact, so a score should not treat them the same. Jev's probabilities let us weigh each answer by how sure it was:

• A clear answer produces a finding at its full severity.
• A leaning answer produces a finding one step lower, marked "probably", so the creator knows Jev is not certain.
• A very unsure answer becomes a note, not a finding, and counts for little in the score. A judgment Jev cannot make should not pull a category towards "half bad".

The same care shows up in small ways. When Jev says something is missing but cannot name the missing topic, the roast says so ("treat it as a hunch") and is stepped down. Jev's honesty about what it does not know is part of what makes the score worth reading.