DEPARTMENT OF MANAGEMENT STUDIES,
VIVEKANANDHA INSTITUTE OF ENGINEERING AND TECHNOLOGY FOR WOMEN,
He never does a proper thing without giving an improper reason for it. - George Bernard Shaw Questionnaire Design: Questionnaire Design Objectives: Objectives To understand:
Objectives of questionnaires
Advantages and disadvantages
Design of questionnaires
Type of questions used
Common problems and pitfalls What is a questionnaire?: What is a questionnaire? An instrument (form) to
collect answers to questions
collect factual data
gathers information or measures
A series of written questions/items in a fixed, rational order.
A questionnaire is a research instrument consisting of a series of questions and other prompts for the purpose of gathering information from respondents
Why using a questionnaire?: Why using a questionnaire? A well designed questionnaire:
Gives accurate and relevant information to your research question
Minimises potential sources of bias
Will more likely be completed
?As simple and focused as possible Advantages of questionnaires: Advantages of questionnaires Can reach a large number of people relatively easily and economically
Provide quantifiable answers
Relatively easy to analyse Disadvantages of questionnaires: Disadvantages of questionnaires Provides only limited insight into problem
Limited response allowed by questions
Maybe not the right questions are asked
Need to get it right first time
Hard to chase after missing data Types of Questionnaire: Types of Questionnaire Face to face
Interviewer -administrated Self-administered questionnaire: Self-administered questionnaire Advantages:
Cheap and easy to administer
Completed at respondent's convenience
No influence by interviewer Self-administered questionnaire: Self-administered questionnaire Disadvantages:
Low response rate
Questions can be misunderstood
No control by interviewer
Time and resouces loss Interview-administered questionnaire: Interview-administered questionnaire Advantages:
Participation by illiterate people
Clarification of ambiguity
Interview-administered questionnaire: Interview-administered questionnaire Disadvantages:
Needs more resources
Only short questionnaires possible
Especially on telephone
Difficult for sensitive issues
Stages in designing a questionnaire (1): Stages in designing a questionnaire (1) Planning the study:
Decide on goals
Identify risk factors for getting bitten by dogs
Know the subject
Literature, experts on dog bites
Formulate a hypothesis
Postmen more likely to get bitten by dogs than the normal population
Define information needed to test hypothesis
Occupation, owning dog, outdoor activities, attitude towards dogs Stages in designing a questionnaire (2): Stages in designing a questionnaire (2) Determine study population:
Know the respondents
?Questionnaire needs to be adapted to your population, not the opposite!
Stages in designing a questionnaire (3): Stages in designing a questionnaire (3) Design questions:
Content of the questions
Format of the questions
Presentation and layout
Coding schedule (if appropriate)
Pilot and refine questionnaire What makes a well designed questionnaire?: What makes a well designed questionnaire? Good appearance
easy on the eye
Short and simple
Relevant and logical
? High response rate
? Easy data summarisation and analysis
Basic Rules: Basic Rules On first page
Study title in bold
On all pages
Identifying mark/ unique identifier
Directions in bold
Self-addressed envelope!! Question order: Question order Decide on order of items/questions
Easy ? difficult
General ? particular
Factual ? abstract
Where to place sensitive questions?
Be aware of ordering effects! Question order (2): Question order (2) Group questions by topic/ response options
With closed format
Relevant to main subject
Neither demographic nor personal questions
Don’t put most important item last
Questionnaire introduction: Questionnaire introduction Covering letter/ interview introduction
Who you are/ you work for
Why you are investigating
Where you obtained the respondent’s name
How and where you can be contacted
Guarantee of confidentiality
Length of interview (be honest)
? Usefulness of study should be clear to all respondents Content of Questions: Content of Questions Clear focus on research question
Avoid unnecessary information
Contact information (if non-anonymised)
Format of Questions: Format of Questions Adjust to responding audience
Professionals vs. public
Middle class vs. prisoners
Keep sentences simple and short
Define key words (“fully vaccinated”)
Remember option “don’t know”
Format of Questions: Format of Questions Ask for one information at a time
Do you own a dog or have frequent contacts with dogs?
Use mutually exclusive and exhaustive answer options
Vertical order of answer options Be accurate: Be accurate Do you often touch dogs?
How often did you touch a dog during the past 3 months?
Three times or more
Not at all
Don´t know Bias: Bias Bias = systematic differences in the measurement of a response
Information Bias: Information Bias Recall bias
Cases more likely to remember than controls
Different interviewer – different interpretations
Different interpretation of similar questions
Reduce by structured questionnaire Non-response bias: Non-response bias Those who respond are different from those who do not
Telephone interviews: more females, elderly
Ensure high response rate
Random choice of interview partners
Correct during analysis (eg age, sex)
Format of questions : Format of questions Two main question formats
Closed format ? forced choice
Yes ? Always ?
No ? Sometimes ?
Don’t know ? Never ?
Open format ? free text
What is your most distressing symptom? Please describe: ________________________________________________________________________________________________ Open or Closed?: Open or Closed? Closed
Simple and quick
Reduces discrimination against less literate
Easy to code, record, analyse
Easy to compare
Easy to report results Closed questions: Closed questions Disadvantages:
Restricted number of possible answers
Loss of information
Insert field „others“
Open questions: Open questions Advantages:
Allows exploration of issues to generate hypothesis
qualitative research, focus groups, trawling questionnaires
Used even if no comprehensive range of alternative choices
Good for exploring knowledge and attitudes
Detailed and unexpected answers possible Open questions: Open questions Disadvantages:
Difficult to analyse!
Difficult to compare groups
Closed Questions: Closed Questions Straightforward response
What is your age in years? ___ years
How long have you owned a dog? ___ years
What is your sex (gender)?
Did you stay in Hotel X?
Don’t know ?
Closed Questions: 2. Checklist
Which of the following outdoor activities did you do last week?
Closed Questions Closed Questions: Closed Questions 3. Rating scale
Did you do use sunscreen during the following outdoor activities during the past six months?
Always Sometimes Seldomly Never
Running ? ? ? ?
Walking ? ? ? ?
Cycling ? ? ? ?
Closed Questions: Closed Questions 4. Rating scale
How useful would you think that information on the risk of biting from stray dogs would be? (please circle)
1 2 3 4 5 6 7
Not at all useful Very useful
Analogue How much is your pain severe (put the tick on the line)
0 10 Closed Questions: Closed Questions 5. Scales for measuring attitude (Lickert)
Stray dogs carry a higher risk of rabies
No, I strongly disagree ?
No, I disagree quite a lot ?
No, I disagree just a little ?
I’m not sure about this ?
Yes, I agree just a little ?
Yes, I agree quite a lot ?
Yes, I strongly agree ? Problems and Pitfalls: Problems and Pitfalls Avoid questions that ask two things at once - you won’t know which ‘bit’ people are answering:
Have you ever had stomach ache and diarrhoea?
Do you go to the woods a lot?
Problems and Pitfalls: Problems and Pitfalls Avoid jargon/abbreviations/slang
How often do you get up at night to DW (Drink Water)
Avoid not mutually exclusive options
What age are you?
35-40 ? Problems and Pitfalls: Problems and Pitfalls Avoid leading questions
Do you think that the food in the hotel made you sick?
Did the hotel staff seem unhygenic to you?
Do you agree that the hospital staff were close to exhaustion?
Avoid making questionnaire too long
Typographical / spelling errors
Questionnaire Validation: Questionnaire Validation Use or adapt existing questionnaires
Validated (and possibly harmonised)
Needs to be tested (pilot) Piloting and Evaluation: Piloting and Evaluation Pilot with a similar group of people to your intended subjects
Highlights problems before starting
Effects of alternative wording
Overall impression on respondents and interviewers
Final polishing after several amendments Presentation and layout: Presentation and layout Clear consistent layout
Adequate space to answer
Large font size
Appropriate page breaks
printed on recycled paper/is an equal opportunity employer etc Presentation and layout: Presentation and layout
Using colour or printing questionnaire on coloured paper may help
Use filter questions, if necessary
Give clear instructions about how to answer the questions
Coding Schedule: Coding Schedule Questionnaire can be pre-coded
Quicker and easier data entry
Male ? 1 Ill ? 1
Female ? 2 Not ill ? 0
Don’t know ? 3 Don’t know ? 9
Single ? 1 Separated ? 3
Married ? 2 Divorced ? 4 Widowed ? 5 Don’t know ? 9 Summary: Summary A well designed questionnaire:
Will give appropriate data which allow to answer your research question
Will minimise potential sources of bias, thus increasing the validity of the questionnaire
Will much more likely be completed