How To Craft Survey Questions?

The way questions are crafted impacts the overall reception of a survey. The knowledge of the science behind questions is a must for survey creators. Questions should be asked in a way that supports survey engagement and survey completion.

The art and science of asking questions is the source of all knowledge - Thomas Berger

Are you a business owner, a marketing professional, an academic student, or anyone wanting to run surveys then this article is for you. When you finish reading this article, you will receive comprehensive understanding of how to create survey questions.

This article consists of many sections each discussing a particular aspect of question creation. It includes Question Phrasing, Question Ordering, Question Connotation, Question Scales, Question Types, Question Complexity, and Question Length.

First, I will start with Question Phrasing.

1. Question Phrasing

How the question is phrased influences the quality of survey response. Incorrectly phrased questions lead to skewed responses.

It is not the answer that enlightens, but the question - Eugene Jonesco

Now, let us discuss four different types of question phrasing. They are Leading Question, Double-barreled Question, Absolute Question and Assumptive Question.

1.1 Leading Question

“Don’t you think our sandwich is great?”

This question leads and influences the respondent into saying that their sandwich is great. A leading question suggests a particular answer or guides the respondent towards a specific response. Leading questions can skew survey results making them unreliable. Instead frame the question as , “How would you rate our sandwich?”.

1.2 Double-barreled Question

“How satisfied are you with our sandwich’s price and quality?”

This is an example of Double-barreled question. This question asks 2 things at once. They ask both about the sandwich’s price and sandwich’s quality. This question can be confusing to the respondent. The respondent might feel differently about the price and quality. The respondent cannot express it here. So, split the question into two, one for price and one for quality.

1.3. Absolute Question

“Is this the best sandwich you ever had?”

Absolute questions are those that consists of an absolute term, such as “ever”, “always”, “only”, “everyone” etc. Such terms makes the question extreme or biased. These questions leave no room for nuance. This can lead to skewed responses because of absolute terms. Use scaled responses or open-ended questions that leaves room for nuance.

1.4 Assumptive Question

“Why did you like our sandwich?”

This question is created based on assumptions. This example assumes the respondent already likes the sandwich, which may not be the case. Assumptive question paves way for bias. The respondent may abandon the survey due to the confusion by Assumptive question. The question can be instead framed as, “How did you feel about having our sandwich”.

Let’s move on to discuss Question Ordering.

2. Question Ordering

The Ordering or Sequencing of survey question has a profound effect on how well it is received.

Ask the right questions if you are going to find the right answers - Vanessa Redgrave

2.1 Demographic Question

Few suggests placing the demographic questions at the beginning of the survey, and few suggest placing the demographic questions at the end. Actually there is no right answer to this question. It depends on the context of the survey.

If the context demands placing the demographic question in the beginning of the survey imparts trust in respondent, then place it at the beginning. Contrary if the context demands placing at the beginning makes respondent vulnerable, then placing at the end will help.

2.2 Primacy and Recency

Primacy and Recency effects plays a vital role in the placement of questions. Respondents usually tend to remember questions placed at the beginning of survey, which is called Primary effect. On the contrary respondents also well remember the questions placed at the end of survey. This is called Recency effect. Usually respondents tend to forget the questions placed at the middle of survey.

2.3 Avoid Fatigue

Placing very complex questions at the beginning of a survey can cause respondents survey fatigue. It is usually a best practice to place easy questions at first. This gives inspiration and drive to the respondents to start the survey.

Open-ended questions sometimes requires complex thinking before answering. So it is not a good practice to place open-ended question at the beginning of the survey, instead it should be placed at the last. Begin with simpler questions first and gradually include more complex questions.

2.4 Logical Flow

The questions should follow a logical sequence guiding the respondents throughout the survey. Think of it as telling a story. Each question should naturally lead to the next question. The respondents should get the feel of a story unfolding while taking the survey.

3. Question Connotation

Understanding the connotations of the words are crucial. The question may have positive connotation, negative connotation or neutral.

The important thing is not to stop questioning - Albert Einstein

Usually it is best to go for neutral wording while creating survey questions.

3.1 Positive

Positive connotations are words that evoke favorable emotions or reactions. For example, beneficial, success, advantageous are few words that evoke positive emotions.

3.2 Negative

Negative connotations are words that elicit adverse emotions or reactions. For example, harmful, failure, detrimental are few words that elicit negative emotions.

3.3 Neutral

Neutral connotations are impartial and does not tilt towards positive or negative emotions. For example outcome, result, and factor are few words that evoke neutral connotations. Positive or Negative connotations may result in respondent abandoning the survey altogether.

Neutral language serves as a cornerstone for fair and unbiased surveys. The neutral language safeguards the integrity of the data collected and upholds the purpose of the data. Neutral wording helps to gain genuine insights.

4. Question Scales

Asking questions is one of the best way to grow as a human being - Michael Hyatt

Scaled responses provide a structured way for respondents to indicate their feelings or opinions about a particular subject. They offer shades of opinion, rather than a binary "yes" or "no.”

4.1 Likert Scale

Developed by Rensis Likert, this is a multi-point rating scale used to represent people's attitudes to a topic. Typically, respondents are asked to indicate their level of agreement or disagreement on a symmetric scale, which ranges from "strongly agree" to "strongly disagree.” Ideal for measuring attitudes or opinions about a particular topic. Ensure there's a balanced number of positive and negative response options.

4.2 Differential Scale

This bipolar scale asks respondents to rate a product, brand, or company based upon a seven-point rating scale between two bipolar adjectives, such as "Efficient-Inefficient" or "Clean-Dirty.” Suitable for brand or product positioning studies. Always ensure that the bipolar adjectives used are genuinely opposites to prevent confusion.

4.3 Numeric Scale

As the name implies, this scale uses numbers as response options. For example, asking respondents to rate their satisfaction on a scale of 1 to 10. Perfect when looking for a quantifiable rating or evaluation. It's essential to define what each end of the scale represents, e.g., 1 being "Not at all satisfied" and 10 being "Extremely satisfied.”

Remember, clarity is crucial. Regardless of the scale chosen, always ensure that respondents fully understand the meaning and context of the options provided.

5. Question Types

Two common types of questions utilized are open-ended and closed-ended questions. Each has its merits and limitations, and understanding when and how to use them is crucial for effective data collection.

Judge a man by questions rather than his answers - Voltaire

5.1 Open Ended

Open-ended questions allow respondents to express their thoughts, feelings, and opinions in their own words. This can provide richer, more detailed insights. These questions can yield unexpected information or perspectives that weren't anticipated by the survey creator.

By not providing predetermined answers, open-ended questions can minimize the risk of leading or influencing the respondent.

5.2 Closed Ended

Responses can be quickly quantified and are more straightforward to analyze using statistical methods. They provide uniform data since all respondents choose from the same set of options. Closed-ended questions are Quick and easy for respondents to answer, leading to higher completion rates.

6. Question Complexity

While complex questions in a survey can potentially yield detailed insights, they come with downsides. Introducing complex queries may overwhelm respondents, leading to confusion and a decline in survey participation.

Half of science is asking the right questions - Roger Bacon

Complex questions may introduce bias, leading to the unreliability of the collected data. Some respondents may lack the knowledge to answer complex questions and, therefore, provide irrelevant answers or altogether abandon the survey.

For instance, consider a question inquiring about multifaceted aspects of a restaurant experience, asking respondents to assess its food quality, dining ambiance, and overall satisfaction within a single query. The complexity of this question may overwhelm participants, leading to vague or inconsistent responses, as individuals might struggle to express their opinions cohesively.

7. Question Length

Lengthy survey questions can lead to respondent fatigue and reduced engagement. Lengthy questions tend to collect low-quality data.

Asking the right questions takes as much skill as giving the right answers - Robert Half

For instance, consider a question that combines multiple inquiries about a restaurant, including food satisfaction, customer service, and likelihood to recommend, all within one extensive paragraph. Respondents may feel overwhelmed by the sheer volume of information requested, leading to rushed or incomplete responses.

This not only compromises the accuracy of the data but also results in a decline in survey participation. Lengthy questions can hinder respondents’ ability to provide thoughtful and accurate feedback, ultimately diminishing the survey's effectiveness.

Test your knowledge on Crafting Survey Questions

Now let’s move onto final thoughts.

Final Thoughts

In conclusion, crafting effective survey questions is an art backed by science. From understanding the nuances in question phrasing, ordering, and connotation, to choosing the right question scales and types, each aspect plays a crucial role in how well a survey is received and the quality of data collected.

Successful people ask better questions - Anthony Robbins

Remember to avoid complexity and lengthiness in your questions to prevent respondent fatigue and ensure high-quality responses.

By applying these principles, you can create surveys that provide valuable insights and drive informed decision-making.