ISE 298 Methods: Human Performance & Variability Human Factors and ErgonomicsUsability Methods San Jose State UniversityKevin Corker 9/24/07kevin.corker@sjsu.edu408 924 3988: ISE 298 Methods: Human Performance & Variability Human Factors and Ergonomics Usability Methods San Jose State University Kevin Corker 9/24/07 kevin.corker@sjsu.edu 408 924 3988


Usability Characteristics: Usability Characteristics Ease of learning - How fast can a user who has never seen the user interface before learn it sufficiently well to accomplish basic tasks? Efficiency of use - Once an experienced user has learned to use the system, how fast can he or she accomplish tasks? Memorability - If a user has used the system before, can he or she remember enough to use it effectively the next time or does the user have to start over again learning everything? Error frequency and severity - How often do users make errors while using the system, how serious are these errors, and how do users recover from these errors? Subjective satisfaction - How much does the user like using the system?


ISO Standards : ISO Standards "[Usability refers to] the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of user." - ISO 9241-112. "Human-centered design is characterized by: the active involvement of users and a clear understanding of user and task requirements; an appropriate allocation of function between users and technology; the iteration of design solutions; multi-disciplinary design." - ISO 13407


Measures: Measures Effectiveness•Efficiency•Satisfaction•Ease of learning•Memorability•Error: frequency and severity ------------------------------------------------------------------- •Effectiveness: A user's ability to successfully use a Web site to find information and accomplish tasks. •Efficiency: A user's ability to quickly accomplish tasks with ease and without frustration. •Satisfaction: How much a user enjoys using the Web site.


Metrics : Metrics There are two types of usability metrics that can be captured during a usability test. Performance data (what actually happened) Must be sure that there is a causal link between the required performance and the dimensions of usability Preference data (what participants thought) Must be sure that preference expressions and user-thoughts are based on and represent full range system “uses” and “users”


Must be sure that there is a causal link between the required performance and the dimensions of usability: Must be sure that there is a causal link between the required performance and the dimensions of usability Behavior measured must represent full range of system operations Time of performance [limit methods: e.g. no-talk aloud protocols if you are using “time” as a primary dependent variable] Erroneous behavior and recovery Complexity measures [relates to “learnability and memorability”] Causal models must relate dependent variables to “performance” prediction


Must be sure that preference expressions and user-thoughts are based on and represent full range system “uses” and “users” : Must be sure that preference expressions and user-thoughts are based on and represent full range system “uses” and “users” Participant pool: Selection: demographics [age, experience, availability, physical characteristics], number [experimental power] Reward/payment/solicitation-recruitment Scenario and Task development Range of tasks Which tasks System failure modes IRB


Variability: Variability Sources: Mental Differences Physical Differences Experiential Differences Use of the system or not Types: Within subject: refers to a situation in which an individual takes different actions at different times in effectively the same situation. Between subject refers to a situation in which two different people do different things in essentially the same circumstance.


Experimental Design : Experimental Design Hold individual within subject variability constant except for: Use of the system Hold individual between subject variability constant except for: Use of the system What hypotheses are formed based on these two approaches to variability? What experimental analyses are used based on these two approaches to variability?