How to craft survey questions?

What You’ll Learn

By the end of this lesson, you’ll be able to:

• Understand the importance of crafting the right questions in surveys.
• Retain different questioning techniques & question types.
• Craft right questions for your survey and receive thoughtful and accurate responses.

1. Why is Crafting the Right Questions Important in Surveys?

Crafting the right questions in a survey is crucial because it directly impacts the accuracy and reliability of the data collected. Poorly crafted questions can lead to confusion, misinterpretation, and biased responses, resulting in skewed or incomplete data. On the other hand, well-designed questions that are clear, concise, and relevant to the topic can elicit thoughtful and accurate responses. By crafting the right questions, survey creators can ensure that they gather the information they need to make informed decisions, identify trends and patterns, and gain a deeper understanding of their target audience.

The art of asking questions is the source of all knowledge. - Thomas Berger

If you are a business owner, a marketing professional, an academic student, or anyone wanting to craft the right questions for your surveys, then this lesson is for you.

2. How to Phrase Your Questions?

Question phrasing influences the quality of the survey response. Incorrectly phrased questions lead to skewed responses.

Four types of question phrasing to avoid are discussed in this section. They are Leading, Double-barreled, Absolute, and Assumptive.

2.1 Leading Question

“Don’t you think our sandwich is great?”

This question leads and influences the respondent into saying that their sandwich is great. A leading question suggests a particular answer or guides the respondent towards a specific response. Leading questions can skew survey results, making them unreliable.

Instead, frame the question as, **“How would you rate our sandwich?”.

2.2 Double-barreled Question

“How satisfied are you with our sandwich’s price and quality?”

This is an example of a Double-barreled question. This question asks 2 things at once. They ask both about the sandwich’s price and the sandwich’s quality. This question can be confusing to the respondent. The respondent might feel differently about the price and quality. The respondent can find it hard to explain in a single response.

So, split the question into two, one for price and one for quality.

2.3. Absolute Question

“Is this the best sandwich you've ever had?”

Absolute questions are those that consist of an absolute term, such as “ever,” “always,” “only,” “everyone,” etc. Such terms make the question extreme or biased. These questions leave no room for nuance. This can lead to skewed responses because of absolute terms.

Use scaled responses or open-ended questions that leave room for nuance.

2.4 Assumptive Question

“Why did you like our sandwich?”

This question is created based on assumptions. This example assumes the respondent already likes the sandwich, which may not be the case. Assumptive question paves the way for bias. The respondent may abandon the survey due to the confusion of the Assumptive question.

The question can be instead framed as, “How did you feel about having our sandwich?”.

3. Ordering Your Questions

The ordering or sequencing of survey questions profoundly affects how well they are received.

3.1 Where to place the Demographic Question?

Few experts recommend placing the demographic questions at the beginning of the survey, and few recommend placing the demographic questions at the end. In reality, there is no fixed right way of doing this. It depends on the purpose of the survey.

If the purpose demands placing the demographic question at the beginning of the survey, then place it at the beginning. On the contrary, if the purpose makes placing at the beginning makes the respondent uncomfortable, then placing at the end will help.

3.2 What are Primacy and Recency effects?

Respondents usually tend to remember questions placed at the beginning of the survey. This is called the Primary effect.

Also, respondents well remember the questions placed at the end of the survey. This is called the Recency effect.

Usually, respondents tend to forget the questions placed in the middle of the survey.

Primacy and Recency effects play a vital role in the placement of questions.

3.3 How to avoid Respondent Fatigue?

Placing very complex questions at the beginning of a survey can cause respondents to experience survey fatigue. It is a best practice to place easy questions at the beginning of the survey. This inspires the respondents to start the survey.

Open-ended questions require complex thinking before answering. So, it is not a good practice to place open-ended questions at the beginning of the survey. Instead, they should be placed at the last.

Begin with simpler questions first and gradually include more complex questions.

3.4 Logical Flow of Questions

The questions should follow a logical sequence, guiding the respondents throughout the survey. Think of framing & ordering questions as telling a story. Each question should naturally lead to the next question.

The respondents should feel the story unfolding while taking the survey.

4. Emotions behind Survey Questions

The emotion or connotation of a survey question influences the way it is received by the respondent. The question may express a positive, a negative, or a neutral emotion.

4.1 Positive Emotions

Positive connotations are words that evoke favorable emotions or reactions. For example, ‘beneficial’, ‘successful’, and ‘advantageous’ are a few words that evoke positive emotions.

4.2 Negative Emotions

Negative connotations are words that evoke adverse emotions or reactions. For example, ‘harmful’, ‘failure’, and ‘detrimental’ are a few words that elicit negative emotions.

Positive or Negative connotations may result in respondents abandoning the survey altogether.

4.3 Neutral Emotions

Neutral connotations are impartial and do not tilt towards positive or negative emotions.

For example, ‘outcome’, ‘result’, and ‘factor’ are a few words that evoke neutral emotions.

Neutral language serves as a cornerstone for fair and unbiased surveys. Neutral language safeguards the integrity of the data collected and upholds its purpose.

Neutral wording helps to gain genuine insights.

5. Using Scales In Questions

Scaled responses provide a structured way for respondents to indicate their feelings or opinions about a particular subject.

They offer shades of opinion, rather than a binary "yes" or "no.”

5.1 Likert Scale

Developed by Rensis Likert, Likert is a multi-point rating scale capturing & representing people's attitudes to a topic.

For example, respondents are asked to indicate their level of agreement or disagreement on a symmetric scale, which ranges from "strongly agree" to "strongly disagree.”

The Likert scale is ideal for measuring attitudes or opinions about a particular topic.

Ensure there's a balanced number of positive and negative response options.

5.2 Differential Scale

This Differential scale, for example, asks respondents to rate a product based upon a seven-point rating scale between two bipolar adjectives, such as "Efficient-Inefficient" or "Clean-Dirty.”

Also called the bipolar scale, it is suitable for brand or product positioning studies.

Always ensure that the bipolar adjectives used are genuinely opposites, which prevents confusion.

5.3 Numeric Scale

As the name implies, this scale uses numbers as options.

For example, respondents are asked to rate their satisfaction on a scale of 1 to 10. Perfect when looking for a quantifiable rating for evaluation. It's essential to define what each end of the scale represents, e.g., 1 being "Not at all satisfied" and 10 being "Extremely satisfied.”

Regardless of the scale chosen, please ensure that respondents fully understand the meaning and context of the options provided.

6. Types of Questions

Two common types of questions used are open-ended and closed-ended questions.

Each has its merits and limitations, and understanding when and how to use them is essential and necessary for effective data collection.

6.1 Open-Ended Questions

Open-ended questions allow respondents to express their thoughts, feelings, and opinions in their own words. This can provide rich & more detailed insights. These questions can yield unexpected information or perspectives the survey creator didn't anticipate.

By not providing predetermined answers, open-ended questions can minimize the risk of leading or influencing the respondent.

6.2 Close-Ended Questions

Responses can be quickly quantified and are more straightforward to analyze.

They provide uniform data since all respondents choose from the same options.

Close-ended questions are quick and easy for respondents to answer, leading to higher completion rates.

7. Question Complexity

While complex survey questions can yield detailed insights, they come with downsides.

Introducing complex queries may overwhelm respondents, leading to fatigue and decreased survey participation.

Complex questions sometimes introduce bias, leading to the unreliability of the collected data. Some respondents may lack the knowledge to answer complex questions and, therefore, provide irrelevant answers or abandon the survey altogether.

For instance, consider a question inquiring about multifaceted aspects of a restaurant experience, asking respondents to assess its food quality, dining ambiance, and overall satisfaction with a single question. The complexity of this question may overwhelm participants, leading to vague responses.

Individuals might struggle to express their opinions cohesively.

8. Question Length

Lengthy survey questions can lead to reduced engagement. Lengthy questions tend to collect low-quality data.

Respondents may feel overwhelmed by the sheer volume of information requested, leading to rushed or incomplete responses.

Lengthy questions not only compromise the accuracy of the data but also result in a decline in survey participation. Lengthy questions ultimately diminish the survey's effectiveness.

Test Your Knowledge

Key Takeaways

• Rightly framing questions plays an important role in the success of your survey.
• Focus on Question phrasing & ordering to avoid a skewed response.
• Complex questions overwhelm respondents and lead to survey fatigue.

Congratulations! You have completed this lesson.