class11 2004

Building Scales: 

Building Scales What do we mean by scale? Why do we build scale? How to build scale?

Scale: What it is: 

Scale: What it is A scale is a composite measure of a concept, a measure composed of information derived from several questions or indicators. de Vaus (2002)

Scale: Why to use Multiple Indicators: 

Scale: Why to use Multiple Indicators Cover all dimensions of a concept Helps in developing more valid measure Increased reliability Greater precision Simplified analysis

Summated Scaling: Likert: 

Summated Scaling: Likert Source: Steel et al., 1994

Summated Scaling: Likert: 

Summated Scaling: Likert

Summated Scaling: Likert: 

Summated Scaling: Likert

Summated Scaling: Likert: 

Summated Scaling: Likert

Building Likert Scale: 

Building Likert Scale Constructing a rough scale Selecting the best items Creating the final scale variable

Building Likert Scale: 

Building Likert Scale Construct a rough scale Identify the concept the scale is designed to measure Develop a set of questions which have face validity Collect responses from a group of people Score each person’s response to each question Reverse code items as required If selecting items from secondary sources: Conceptual approach Empirical approach: correlation matrix

Building a Likert Scale: 

Building a Likert Scale Select the best items: Item analysis Test for unidimensionality Tests whether each item on the scale measures same underlying concept Item-to-scale coefficient Drop an item from the scale if its item-to-scale coefficient is less than 0.3

Building a Likert Scale: 

Building a Likert Scale Reliability of items Is tested by looking at the consistency of a respondent in answering each item in the scale Item-item correlation or Cronbach’s alpha coefficient For an item to be considered reliable its alpha value should be at least 0.7 Alpha value is influenced by reliability of individual item and the number of items in the scale To improve scale’s reliability drop unreliable items Factor Analysis

Building a Likert Scale: 

Building a Likert Scale Alpha for Scale = 0.65 Source: de Vaus, 2002

Building a Likert Scale: 

Building a Likert Scale Create the final Scale Compute a new variable that represents scale score of each respondent.

Factor Analysis: 

Factor Analysis A mathematical technique of reducing a large set of variables to a smaller set of underlying variables referred to as factors. This technique can be used to validate a scale or index by demonstrating that its constituent items load on the same factor

Factor Analysis: 

Factor Analysis Factor 1 Factor 2 Variable 1 Variable 2 Variable 3 Variable 4 Variable 5 Variable 6 Variable 7

Factor Analysis: 

Factor Analysis Source: Tarrant et al., 2003

Factor Analysis: 

Factor Analysis Steps: Selection of variables Extraction of initial factors Rotation of final factors Construction of scales

Factor Analysis: 

Factor Analysis Selection of Variables Extraction of factors is based on correlation between variables. Therefore, factors are extracted if the variables are sufficiently correlated irrespective of whether there is any underlying dimension or not. Correlation between variables should not be ‘causal’, rather it is assumed to be produced by some third, common factor. On the face of it, variables should seem to be related to some common factors. Sufficient correlation: correlation matrix

Factor Analysis: 

Factor Analysis Correlation Matrix

Factor Analysis: 

Factor Analysis Extraction of Initial Variables Methods of extraction How many factors to extract Eigenvalue: indicates the amount of variance in the pool of original variables that the factor explains. Higher the value, the greater is the variance explained. To be retained, in general, eigenvalue of a factor should be greater than 1. Communality: indicates the extent of variance in a variable that is explained by the factors extracted. The higher the figure the better the set of selected factors explain the variance for that variable. In general, the cut off point is 0.3

Factor Analysis: 

Factor Analysis

Factor Analysis: 

Factor Analysis Rotation of final factors: Factor rotation indicates which variables most belong to each factor Number of methods of rotating variables Rotated factor matrix Strength of ‘loading’ determines the factor to which particular variable belongs Cut off value for loading, in general, is 0.3 Infer conceptual commonality from the empirical commonality of the variables that load on a given factor

Factor Analysis: 

Factor Analysis OUTPUT PROTECTION

Factor Analysis: 

Factor Analysis Factor scores and scales Unweighted factor-based scales Weighted factor-based scales Factor scales

Interpretation of Scales: 

Interpretation of Scales Loss of detail Cannot say how a person has responded to any particular question If two people have the same scale scores it does not mean that their answers to particular questions are identical Scale scores should be interpreted in relative rather than absolute terms

Slide26: 

Conceptual Content Cognitive Mapping (3CM)

Conceptual Content Cognitive Mapping (3CM): 

Conceptual Content Cognitive Mapping (3CM) A technique for measuring people’s perspective on, or cognitive maps of complex domains (Kearney and Kaplan, 1997) Individual’s understanding of the topic Important aspects from an individual’s perspective Perceived relationships among aspects

Cognitive Maps or Mental Models: 

Cognitive Maps or Mental Models Hypothesized knowledge structures embodying people’s assumptions, beliefs, facts, and misconceptions about the world (Kearney and Kaplan, 1997). An interpretive framework of the world which exists in the human (animals?) mind and affects actions and decisions Influences how new information is understood and whether or not that information will impact behavior Information provided does not equal information received Understanding of existing maps is essential to appropriately frame new information in way that it gets integrated

Why 3CM: 

Why 3CM Environmental issues are complex, involve many stakeholders, and have no easy solutions or right answers Differences in value orientations and attitudes account for conflict over forest management issues? It is possible that people differ not only in their forest values but also in their understanding, or knowledge of different forest values Cognitive maps of the stakeholders differ- background, training, experience As important as whether or not people know the facts is how they understand the issue Knowledge among stakeholders that they perceive things differently from others

3CM Methodology: 

3CM Methodology Introduction of topic to the participants Listing of the components or aspects that they perceive to be important, and writing these on separate cards Grouping or arranging the cards

Stakeholders’ Preferences forMultiple Forest Valuesin NW Ontario : 

Stakeholders’ Preferences for Multiple Forest Values in NW Ontario

Research Questions (1): 

Research Questions (1) Do people tend to rank various forest values in the same way, or is there no particular pattern in rankings? Is there a significant difference between peoples’ preferences for different forest values? Do members belonging to different groups have similar or dissimilar preferences for different forest values?

Research Questions (2): 

Research Questions (2) Even if the ranking of a particular value is similar across groups, does intensity of preference differ? What socio-demographic characteristics of the people affect their ranking of forest values?

Data Collection - Sample: 

Data Collection - Sample Ontario Ministry of Natural resources (OMNR) - 31 Forest Industry - 36 Environmental Non-governmental Organizations (ENGOs) - 33 Aboriginal People - 20

Data Collection: 

Data Collection Demographic and socio-economic background of the participants Introduction of the topic Identification and writing of individual’s forest values on separate cards Organization of forest values into groups Explanation and labeling of clusters Ranking of groups and individual card within a group

Data Analysis: 

Data Analysis Three stage hierarchical clustering Dominant value themes Spiritual Value Environmental Value Recreation Value Economic Service Value Economic Product Value Exploded Logit Model (Allison & Christakis, 1994)

Results (1): 

Results (1) Value Differences

Results (2): 

Results (2) Value Differences

Results (3): 

Results (3) Groups

Results (4): 

Results (4) Groups

Results (5): 

Results (5) Groups

Results (6): 

Results (6) Education

Results (7): 

Results (7) Income

Results (8): 

Results (8) Gender

Advantages of 3CM: 

Advantages of 3CM Focuses on those concepts or things that an individual considers important in relation to a particular domain or issue Allows the expression of misconceptions and ensures that the researcher’s own ideas are not imposed on participants Captures perceived relationships among the relevant concepts Allows for exploration and discovery of one’s knowledge structure Data collected from 3CN reveals the levels of expertise among participants Is applicable in a variety of situations Generates a feeling of fulfillment among the study participants since it increases one’s sense of clarity about the domain being measured