A-team Spring Session #2Questionnaire Design: A-team Spring Session #2 Questionnaire Design February 28, 2007


Questionnaire or Survey?: Questionnaire or Survey? Questionnaire is an actual instrument: I.e. web questionnaire (Perseus) Survey is actually a verb/method: “to study or examine comprehensively” Your questionnaire is actually “surveying” Often used interchangeably


Yields Different Types of Data:: Yields Different Types of Data: Descriptive Socioeconomic parameters to better understand the larger population represented by the sample. (e.g., income, age, college/school, major, class standing) Behavioral Patterns of use, recreation, entertainment, personal behavior. (e.g., # UGA bus rides per day/week) Preferential Opinions & preferences about socio-political issues. (e.g., opinion about new parking plan)


Types of Responses/Measurement Scales: Types of Responses/ Measurement Scales Nominal Scales Used to categorize objects; “name” them Object is in a category or it is not No order implied along any dimension Response sets that are nominal: Yes/No (dichotomous) Can be “Choose one,” “Choose all that apply,” “Choose one and specify,” or “Choose all and specify” from listing of characteristics


Examples of Nominal Scales: Examples of Nominal Scales Immediately after aerobic exercising I generally feel: ___Exhausted ___Invigorated ___Thirsty ___Sweaty ___Overheated ___Nauseated [Note: make sure (+) and (-) options offered in listing] Indicate your sex: ___Male ___Female Have you ever resided in Brumby Hall? ___Yes ___No


Measurement Scales (Cont’d): Measurement Scales (Cont’d) Ordinal Scales (a.k.a. rank, order, rank-order) Used to rank objects according to amount of characteristic the object possesses Order reflects varying amounts or levels Rank reflects range from high to low amounts Ranking has no absolute zero Intervals from one rank to next not the same Likert scales are ordinal but sometimes treated as interval scales (judgment call)


Examples of Ordinal Scales: Examples of Ordinal Scales Order of finish in a horse race Rank in class (e.g., achievement) Highest degree earned Order of preference A higher number indicates a higher rank, e.g., “more” of characteristic possessed Watch for (reverse) coding


Examples of Ordinal Scales: Examples of Ordinal Scales Rank-order your on-campus living preferences for the next academic year, with 1 = first priority, 2 = second priority, and so on: ___ Single Room ___One-bedroom Apartment ___ Double Room ___ Multiple-bedroom Apartment ___ Suite (2 double rooms separated by a bathroom) Rank-order your reasons for attending this workshop, with 1=most influential reason, 2=second greatest influence, and so on: ___ Surveys are my life ___My boss sent me ___ To get the handouts ___ My thirst for knowledge ___ To earn CEUs ___ To get the free gift


Measurement Scales (Cont’d): Measurement Scales (Cont’d) Interval Scales A ranking/rating using interval score values The difference between intervals is equal The difference between 1 & 2 is the same as the difference between 4 & 5 Still a focus on the amount of a characteristic an object possesses Likert-type (pronounced Lick-ürt) scales often treated like interval scales (although considered ordinal): 5=strongly agree, 4=agree, 3=no opinion, 2=disagree, 1=strongly disagree


Examples of Interval Scales: Examples of Interval Scales Likert Scale Example: Parking on campus should be free. __Strongly Agree __Agree __Neither Agree nor Disagree __Disagree __Strongly Disagree Non-Likert Scale Example When driving a UGA van, the safest following distance under ideal conditions (in seconds) is __1.5 __3 __4 __8 __10 __25


Examples of Interval Scales: Examples of Interval Scales Non-Likert Scale Example (cont’d): Please rate your satisfaction with the following student activities on a scale of 1 to 5, with 1=very dissatisfied, 2=dissatisfied, 3=neither dissatisfied nor satisfied, 4=satisfied and 5=very satisfied. [could add not applicable or did not attend option]. __Welcome Week __Dawgs After Dark __Movie-O-Rama __Concert on the Quad __ Halloween “I Vant to Drink Your BlooooooDrive”


Measurement Scales (Cont’d): Measurement Scales (Cont’d) Ratio Scales Empirically meaningful zero/absolute zero; true absence of characteristic; (e.g., height, weight; more common in physical/biological sciences) Have all characteristics of nominal, ordinal, and interval scales Can be converted to ordinal scales Can be converted to categories Education examples: income, age, # years of education, # meetings with academic advisor


Examples of Ratio Scales: Examples of Ratio Scales Indicate the number of times you accessed the University Health Center in the last 30 days: ___ Indicate your age: ___


Likert Scales: Likert Scales Present question/item stem in a declarative sentence (one statement under consideration). Response options represent varying degrees of agreement or endorsement of one statement. Response options should be worded to represent approximately equal intervals; use equal # positive and negative possibilities. The question stem doesn’t have to span the range of the construct (as in Thurstone or Guttman); response options infer levels of phenomena.


Likert Scales: Likert Scales Often 5, 7, or 9 response-options sets A 6 response-options set is also common Strongly disagree Moderately disagree Mildly disagree Mildly agree Moderately agree Strongly agree


Likert Scales: Likert Scales Midpoint often used but optional What does midpoint wording imply? Neither agree nor disagree: Apathy? Agree and disagree equally: Strong paradox? Common midpoint wording Neither agree nor disagree Agree and disagree equally Neutral


Likert Scales: Likert Scales Most used in surveys of opinions, beliefs, attitudes Useful if statements are fairly strong (but not extremely) Everyone can agree, have no opinion, or have little opinion about a mild statement Write clear statements that reflect true differences of opinion


Likert Scale Examples: Likert Scale Examples Exercise is an essential component of a healthy life-style. 1=Strongly Disagree, 2=Moderately Disagree, 3= Mildly Disagree, 4=Mildly Agree, 5=Moderately Agree, and 6=Strongly Agree Combating drug abuse should be a top national priority. 1=Completely True, 2=Mostly True, 3=Equally True and Untrue, 4=Mostly Untrue, and 5=Completely Untrue


Semantic Differential Scales: Semantic Differential Scales Response options consist of one but usually several adjective pairs One adjective is negative, the other positive; each serves as the (-) or (+) end of a continuum that characterizes the stimulus


Semantic Differential Scales: Semantic Differential Scales Individual lines/points are placed between the two extremes (adjectives) 7 or 9 lines/points are common Respondents check/select lines or points closest to the adjectives if they hold extreme views Respondents check/select lines or points toward the middle of the continuum if they hold more moderate views


Semantic Differential Examples: Semantic Differential Examples Automobile Salesmen Honest __ __ __ __ __ __ __ Dishonest Quiet __ __ __ __ __ __ __ Noisy Friendly __ __ __ __ __ __ __ Not Friendly Fair __ __ __ __ __ __ __ Unfair Trustworthy __ __ __ __ __ __ __ Untrustworthy


Length of a Survey: Length of a Survey Sufficient to capture needed data Short enough to hold participants’ attention Type of survey affects length Types of questions affect length Quantitative/Qualitative/Mixed approach affects length Participant characteristics affect length


Measurement Scales: More Tips: Measurement Scales: More Tips Avoid providing categories/options that overlap; difficult or impossible to analyze Frequently happens with age, income, class hours, years of service, hours worked, etc. Example: Select the category that best describes your annual, gross income: __$0-$10,000 __$10,000 – $30,000 __$30,000 - $60,000


Measurement Scales (Cont’d): Measurement Scales (Cont’d) Be thoughtful with Use of “other” or “does not apply” or “not applicable” in listing of characteristics/options Positive: Obtain option you may not have considered Positive: Prevents forced responses Negative: Can give response already listed or spurious data Potential Negative: Adds to analysis time


Analysis: Analysis


Analysis (Cont’d): Analysis (Cont’d)


References: References DeVellis, R. F. (1991). Scale development: Theory and applications. Newbury Park, CA: Sage. Miller, T. K., (1999). CAS: The book of professional standards for higher education. Washington, DC: Council for the Advancement of Standards in Higher Education. Payne, D. (1992). Measuring and evaluating educational outcomes. New York: Macmillan. Rea, L. M. & Parker, R. A. (1997) Designing and conducting survey research (2nd Ed.). San Francisco: Jossey-Bass. Schuh, J. H. & Upcraft, M. L. (2001). Assessment practice in student affairs: An applications manual. San Francisco: Jossey-Bass. Upcraft, M. L. & Schuh, J. H. (1996). Assessment in student affairs: A guide for practitioners. San Francisco: Jossey-Bass.