Types of Questions and Survey Piloting

Types of Questions in a Survey

The choice of question type depends on the nature of the information the researcher wants to obtain. Each type has its advantages and limitations.

Open-ended Questions

The respondent answers in their own words, without preset answer options. Example: "In your opinion, what is the main problem in the department's work?"

Advantages: allow for unexpected answers, rich data, deep understanding of opinions. Disadvantages: difficulty of coding and analysis, require more time from the respondent, possible irrelevant answers.

Closed-ended Questions

The respondent chooses from predetermined answer options. This is the main question type in quantitative research.

Dichotomous Questions

Offer two answer options: "Yes / No", "Agree / Disagree", "Male / Female". Example: "Have you used this service in the last month? Yes / No." Simple to analyze, but do not capture nuances of opinion.

Multiple Choice Questions

Offer several options from which the respondent chooses one or several. Example: "Which sources of information do you use? (Check all that apply): □ Internet □ Television □ Newspapers □ Radio □ Social Networks □ Other: ___"

Important: include the "Other" option for completeness and do not create excessively long lists (optimal is 5–8 options).

Ranking Questions

The respondent arranges the options in order of preference. Example: "Rank the following factors by importance to you (1 — most important): __ Salary; __ Career Growth; __ Work-life Balance; __ Team; __ Job Content."

Limitations: difficult for respondents if the number of options is large (recommended no more than 5–6); assumes the respondent has a clear opinion about each option.

Rating Scales

The respondent rates an object on a given scale. The most common is the Likert Scale:

Other scales: semantic differential (pairs of opposite adjectives: "Innovative — Traditional"), numerical scale (rating from 1 to 10), visual analog scale (a line on which the respondent marks a position).

Question Formulation

The quality of wording directly affects the validity of the data. The following errors should be avoided:

Leading Questions

Prompt the "right" answer to the respondent. Bad: "Don't you think the company should increase salaries?" Good: "How do you assess the current level of salary in the company?"

Double-barreled Questions

Ask about two things at once. Bad: "How satisfied are you with your salary and working conditions?" Good: Split into two separate questions — one about salary, another about working conditions.

Ambiguous Questions

Allow for ambiguous interpretation. Bad: "Do you often arrive late?" (what does "often" mean?). Good: "How many times in the last month did you arrive late to work?"

Jargon and Complex Vocabulary

Questions must be understandable to the target audience. Bad: "What is your assessment of the synergistic effect of cross-functional integration?" Good: "How effectively, in your opinion, do different company departments interact?"

Negative Wording

Negations hinder understanding. Bad: "Don't you think management doesn't pay enough attention to feedback?" Good: "How do you assess management's attention to staff feedback?"

Response Scale Design

Number of Scale Points

• 5-point scales — most common, offer sufficient variability with simplicity of completion
• 7-point scales — provide more fine-grained differentiation of answers
• 4- or 6-point (even) — remove the "neutral" middle option, forcing the respondent to take a side

Scale Labeling

It is recommended to label all scale points (not just the endpoints) to ensure uniform interpretation. Labels must be meaningfully equidistant.

Scale Direction

A single direction should be used (from negative to positive or vice versa) throughout the survey to minimize completion errors.

Survey Structure and Flow

Funnel Approach

The survey starts with general, easy questions and gradually moves to more specific and sensitive ones. This helps "warm up" the respondent and reduces the likelihood of refusal.

Logical Order

Questions are grouped by thematic blocks (sections). Each section has a heading and a brief introduction. Transitions between sections should be smooth and logical.

Survey Length

Optimal completion time is 15–20 minutes. Longer surveys lead to lower quality answers (respondent fatigue) and higher rates of incomplete responses.

Instructions

Each section should contain clear instructions for completion. For example: "Rate each statement on a scale from 1 to 5, where 1 means completely disagree, 5 means completely agree."

Survey Piloting

What is Piloting?

Piloting (Pilot Testing, Pre-testing) is preliminary testing of the survey on a small group of respondents similar to the target audience, before the main data collection. This is a mandatory stage in the development of any research instrument.

Why Conduct Piloting?

• Identify unclear or ambiguous wordings
• Test skip logic and transitions
• Estimate survey completion time
• Detect missing answer options
• Test technical functionality (for online surveys)
• Assess respondents' overall impression of the survey

How to Conduct Piloting?

Pilot sample size: 10–30 people representing the target audience of the study. There is no need to use probability sampling for the pilot.

Methods of collecting feedback:

• "Think-aloud protocol" — respondent voices their thoughts while completing
• Debriefing interview — after completion, difficulties are discussed
• Analyzing answer patterns — identifying questions everyone skips or answers the same way

What to Check During Piloting?

Iterations After Piloting

Based on the pilot results, the survey is revised. If significant changes are made, it is recommended to conduct repeat piloting. Only after successful piloting does main data collection begin.

Coding Survey Data for SPSS

Principles of Coding

Each survey question must be transformed into one or several numeric variables:

• Dichotomous questions: 0 = No, 1 = Yes
• Likert Scale: 1 = Completely disagree, ..., 5 = Completely agree
• Multiple choice (several answers): Each option is a separate dichotomous variable (0/1)
• Open-ended questions: Coded after content analysis into thematic categories

Codebook

Create a document recording for each variable: the SPSS variable name, question wording, variable type, value codes, handling of missing values. The codebook ensures reproducibility and transparency of the study.

Entering Data in SPSS

1. In Variable View, create all variables with correct names, types, labels, and levels of measurement
2. In Data View, enter data (each row is one respondent)
3. Conduct data checks: Analyze → Descriptive Statistics → Frequencies to identify entry errors (values outside allowable range)

Practical Tasks

Task 1

Question: Identify problems in the following question wordings and propose corrected versions:

a) "Don't you disagree that our company doesn't provide good enough working conditions?" b) "How satisfied are you with salary, schedule, and team?" c) "How often do you do sports?"

Solution: a) Problem: double negative, leading wording. Correction: "How do you assess the working conditions in the company?" (1 — Very poor, 5 — Excellent). b) Problem: double question (three topics in one). Correction: Split into three separate questions: "How satisfied are you with your salary?", "How satisfied are you with your work schedule?", "How satisfied are you with your team?" c) Problem: ambiguity (what does "often" mean). Correction: "How many times per week do you engage in physical exercise?" with options: 0 / 1–2 / 3–4 / 5 or more.

Task 2

Question: You have developed a 40-question survey to study bank client satisfaction. Devise a piloting plan: specify pilot sample size, feedback collection method, and key evaluation criteria.

Solution: Sample size: 15–20 people from among bank clients, representing various age groups and service types (retail, corporate).

Feedback collection method:

• Distribute the survey in paper and online format (to test both channels)
• Conduct debriefing interviews with 5–7 pilot participants
• Record each respondent's completion time

Evaluation criteria:

• Completion time should not exceed 15 minutes
• All questions must be understood correctly (according to interview data)
• Percentage of missed responses per question must not exceed 5%
• No technical problems with online completion
• Sufficient variability in answers (no question should receive identical answers from all)

Task 3

Question: Code the following question for SPSS data entry: "Which communication channels do you use to contact the company? (Check all that apply): □ Phone □ Email □ Website chat □ Social networks □ Personal visit"

Solution: Create 5 separate dichotomous variables in SPSS:

• comm_phone (Numeric, 0 = Not checked, 1 = Checked, label: "Channel: Phone")
• comm_email (Numeric, 0 = Not checked, 1 = Checked, label: "Channel: Email")
• comm_chat (Numeric, 0 = Not checked, 1 = Checked, label: "Channel: Website chat")
• comm_social (Numeric, 0 = Not checked, 1 = Checked, label: "Channel: Social networks")
• comm_visit (Numeric, 0 = Not checked, 1 = Checked, label: "Channel: Personal visit")

Level of measurement: nominal. Missing values: if the respondent did not answer the entire question, all 5 variables receive System Missing.