Psyslos_ivc_brief

Views:
 
     
 

Presentation Description

No description available.

Comments

Presentation Transcript

Identifying and Assessing Learning Outcomes in Psychology : 

Identifying and Assessing Learning Outcomes in Psychology By Jerry Rudmann Irvine Valley College March 9, 2005

Motivation for this work : 

2 Motivation for this work Accountability to our students New accreditation standards Program review

Slide 3: 

The Ten Student Learning Outcomes (Psychology) Each of the 10 major SLOs include a subset of specific SLOs for the psychology major http://www.apa.org/ed/pcue/taskforcereport2.pdf

One - Theory and Content of PsychologyStudents will demonstrate familiarity with the major concepts, theoretical perspectives, empirical findings, and historical trends in psychology.Two - Research Methods in PsychologyStudents will understand and apply basic research methods in psychology, including research design, data analysis, and interpretation.Three - Critical Thinking Skills in Psychology Students will respect and use critical thinking, skeptical inquiry, and, when possible, the scientific approach to solve problems related to behavior and mental processes.Four - Application of PsychologyStudents will understand and apply psychological principles to personal, social, and organizational issues.Five - Values in PsychologyStudents will be able to weigh evidence, tolerate ambiguity, act ethically, and reflect other values that are the underpinnings of psychology as a discipline. : 

One - Theory and Content of PsychologyStudents will demonstrate familiarity with the major concepts, theoretical perspectives, empirical findings, and historical trends in psychology.Two - Research Methods in PsychologyStudents will understand and apply basic research methods in psychology, including research design, data analysis, and interpretation.Three - Critical Thinking Skills in Psychology Students will respect and use critical thinking, skeptical inquiry, and, when possible, the scientific approach to solve problems related to behavior and mental processes.Four - Application of PsychologyStudents will understand and apply psychological principles to personal, social, and organizational issues.Five - Values in PsychologyStudents will be able to weigh evidence, tolerate ambiguity, act ethically, and reflect other values that are the underpinnings of psychology as a discipline. Category 1 of the 10 SLO Areas in Psychology: Knowledge, Skills and Values Consistent with the Science and Application of Psychology

Six - Information and Technological LiteracyStudents should demonstrate information competence and the ability to use computers and other technology for many purposes.Seven - Communication SkillsStudents will be able to communicate effectively in a variety of formats.Eight - Sociocultural and International AwarenessStudents will recognize, understand, and respect the complexity of socio-cultural and international diversity.Nine - Personal Development Students will develop insight into their own and others’ behavior and mental processes and apply effective strategies for self-management and self-improvement.Ten - Career Planning and DevelopmentStudents will emerge from the major with realistic ideas about how to implement their psychological knowledge, skills, and values in occupational pursuits in a variety of settings. : 

Six - Information and Technological LiteracyStudents should demonstrate information competence and the ability to use computers and other technology for many purposes.Seven - Communication SkillsStudents will be able to communicate effectively in a variety of formats.Eight - Sociocultural and International AwarenessStudents will recognize, understand, and respect the complexity of socio-cultural and international diversity.Nine - Personal Development Students will develop insight into their own and others’ behavior and mental processes and apply effective strategies for self-management and self-improvement.Ten - Career Planning and DevelopmentStudents will emerge from the major with realistic ideas about how to implement their psychological knowledge, skills, and values in occupational pursuits in a variety of settings. Category 2 of the 10 Outcome Areas: Knowledge, Skills and Values Consistent with a Liberal Arts Education that are Further Developed in Psychology

Establish expected student outcomes : 

6 Establish expected student outcomes Psychology faculty met to review… Departmental mission statement APA learning outcomes for a BA in psychology All psychology course outlines

Learning Objectives vs. Student Learning Outcomes : 

7 Learning Objectives vs. Student Learning Outcomes Learning objectives Appear on course outline Valuable skills and content covered in the course Often include goals instructor has for the course Learning Outcomes Overarching products of the course Often higher level thinking and/or skills applicable outside of class Are an end product displayed by the student

Process now following : 

8 Process now following Reviewed APA Outcomes for the BA in psychology Reviewed all psychology course outlines Developed list of “robust” program outcomes Reviewed course outlines again to identify where are taught Developed an assessment plan to measure progress on the robust outcomes Some assessment in the classroom (embedded) Some assessment through an annual survey Some assessment through co curricular activities Maybe develop a comprehensive exit assessment

Learning Outcomes : 

9 Learning Outcomes Identified 10 SLOs for the introductory course and major Chose to assess the top 3 in the spring term of 2003

Our SLOsThe student should be able to… : 

10 Our SLOsThe student should be able to… Explain why psychology is a science and not a pseudo-science. Describe the major theoretical perspectives in psychology. Describe the basic structure and functioning of the brain and nervous system. Critically evaluate explanations of human behavior: Evaluate qualifications of the information’s source. Evaluate representativeness of the data. Correctly differentiate between correlational and causative evidence. Identify and explain various strategies for coping with stress. Define the major types of maladaptive behavior.

Evidence of Learning : 

11 Evidence of Learning Assessment methods will provide evidence of learning What WASC / AACJC says about evidence Quantitative and Qualitative Direct or Indirect

Methods That Provide Direct Evidence : 

12 Methods That Provide Direct Evidence Student work samples Collections of student work (e.g. Portfolios) Embedded assessment Observations of student behavior (e.g., group) Review of student projects Performance on a case study or problem Performance on problem and analysis (Student explains how he or she solved a problem) Locally developed tests Standardized tests Pre-and post-tests Essay tests blind scored

Methods That Provide Indirect Evidence : 

13 Methods That Provide Indirect Evidence Alumni, Employer, Student Surveys Focus groups Exit Interviews with Graduates Graduate Follow-up Studies Course Retention Transfers Transfer course success Percentage who go on to graduate school

Assessment Plan for the First SLO : 

14 Assessment Plan for the First SLO Intended Outcome - Explain why psychology is a science and not a pseudo-science. Assessment Measures & Techniques - Three short answer essays. Standard - At least 85% of students completing the final exam will respond correctly to these questions.

Short-answer essay prompts for SLO #1 : 

15 Short-answer essay prompts for SLO #1 Explain why psychology is a science and not a pseudo-science by explaining the methodology psychologists use to understand human behavior. Explain why psychology is a science by explaining the self-correcting nature of the scientific method as used by psychologists. Both scientific psychology and pseudo-psychology provide explanations of human behavior. Please provide several examples of each.

Scoring Rubric for SLO #1 Intended Outcome: Explain why psychology is a science and not a pseudo-science. : 

16 Scoring Rubric for SLO #1 Intended Outcome: Explain why psychology is a science and not a pseudo-science.

Assessment Plan for the Second and Third SLOs : 

17 Assessment Plan for the Second and Third SLOs Intended Outcomes – SLO #2 Describe the major theoretical perspectives in psychology.SLO #3 Describe the basic structure and functioning of the brain and nervous system. Related Courses – Intro Psychology covers this. Assessment Measures & Techniques – Thirty objective format questions embedded in final exam. Assessment Criteria - At least 85% of students completing the final exam will respond correctly to 80% of the embedded questions.

Annual Survey of Psychology Classes : 

18 Annual Survey of Psychology Classes Are you a psychology major? What psychology courses have you successfully completed? Where might you transfer? 10 self-assessment items (experimental)

Summary – Spring 2003 : 

19 Summary – Spring 2003 Embedded assessment for 3 content SLOs 3 essay prompts 30 multiple choice items Annual survey of all psychology classes Self perceptions of abilities

Selected Results : 

20 Selected Results 232 students in seven psychology courses (four Introductory) completed the survey 85 students in three Introductory classes completed the essays and 30 multiple-choice items

Short-answer essay results for SLO #1 : 

21 Short-answer essay results for SLO #1 Explain why psychology is a science and not a pseudo-science by explaining the methodology psychologists use to understand human behavior. 27.8% met criterion of success Explain why psychology is a science by explaining the self-correcting nature of the scientific method as used by psychologists. 11.1% met criterion of success Both scientific psychology and pseudopsychology provide explanations of human behavior. Please provide several examples of each. 11.1% met criterion of success Summary: Overall, 11.1% of the sample met our criterion of success.

Multiple-Choice Item Results : 

22 Multiple-Choice Item Results

Revisions to short-answer essay prompts for SLO #1 : 

23 Revisions to short-answer essay prompts for SLO #1 Explain why psychology is a science and not a pseudo-science by describing the research methods psychologists use to study and understand human behavior. Explain why psychology is a science by explaining the self-correcting nature of the scientific method as used by psychologists. Both scientific psychology and pseudopsychology provide explanations of human behavior. Please provide several examples of pseudopsychology. Explain why your examples portray pseudopsychology.

More efficient assessment procedure was use in Spring 2004 : 

24 More efficient assessment procedure was use in Spring 2004 Attempted on-line assessment. Tried pre and post testing.

Added this assessment:The Paranormal Belief Test(adapted from…Walker, Hoekstra, & Vogl, (2002) Science education is no guarantee of skepticism, Skeptic, vol 9, no 3. : 

25 Added this assessment:The Paranormal Belief Test(adapted from…Walker, Hoekstra, & Vogl, (2002) Science education is no guarantee of skepticism, Skeptic, vol 9, no 3. Please rate how much you believe the following statements. Use the 7-point scale provided. 1 – Do not believe in this at all. 2 – I doubt very much that this is real. 3 – I doubt that this is real. 4 – I am unsure if this is real or not. 5 – I believe that this may be real. 6 – I believe that this is real. 7 – I strongly believe this is real. __ 1. A person’s personality can be easily predicted by their handwriting. __ 2. A person can use their mind to see the future or read other people’s thoughts. __ 3. A person’s astrological sign can predict a person’s personality and their future. __ 4. An ape-like mammal, sometimes called Bigfoot, roams the forests of America. __ 5. The body can be healed by placing magnets on to the skin near injured areas. __ 6. Healing can be promoted by placing a wax candle in your ear and lighting it. __ 7. A dinosaur, sometimes called the Lock Ness Monster, lives in a Scottish lake. __ 8. Sending chain letters can bring you good luck; ignoring them can bring you bad luck. __ 9. The government is hiding evidence of alien visitation at places such as Area 51. __ 10. Voodoo curses are real and have been known to kill people. __ 11. A broken mirror can bring you bad luck for many years. __ 12. Houses can be haunted by the spirits of people who have died in tragic ways. __ 13. Water can be accurately detected by people using “Y” shaped tree branches. __ 14. Animals, such as cats and dogs, are sensitive to the presence of ghosts.

Spring 2004 Results : 

26 Spring 2004 Results

Spring 2004 Results : 

27 Spring 2004 Results

Spring 2004 Results : 

28 Spring 2004 Results

Selected results of Paranormal Belief Pretest : 

29 Selected results of Paranormal Belief Pretest

Selected results of Paranormal Belief Pretest : 

30 Selected results of Paranormal Belief Pretest

Selected results of Paranormal Belief Pretest : 

31 Selected results of Paranormal Belief Pretest

Consequences of 2003-04 SLO work : 

32 Consequences of 2003-04 SLO work Now offer a “Career’s-in-Psychology” course (Psychology 160) Developed a critical thinking course Networked with other psychology departments (share outcomes statements, assessment tools, etc.)

Ideas we may pursue : 

33 Ideas we may pursue Provide exiting psychology majors with an incentive: recognition awards (certificate, honors cords at graduation) for those who achieve the highest level of competency on the program outcomes. Conduct applied studies of different teaching strategies. Network with psychology faculty at other colleges. Further involve the adjuncts. Publish learning outcomes in program brochures, course syllabi, etc.

SLO Writing Resources : 

34 SLO Writing Resources The California Assessment Institute (CAI) has a number of training resources for helping faculty write SLOs. http://cai.cc.ca.us/SLOworkshops/Strand2/index.htm

Rubrics : 

35 Rubrics Scoring rubrics are especially helpful to faculty. They help clarify and simplify the assessment of essays, products, and performances. The next few slides present creation of a scoring rubric for an APA-style report.

36 Rubric for grading papers written in APA style

Rubrics are Great! : 

37 Rubrics are Great! Clearly communicates expectations. More efficient scoring. Facilitates agreement among faculty. If used for grading, can be shared with students. Helps develop metacognitive skills.

Mary Allen’s Rubric Archive : 

38 Mary Allen’s Rubric Archive http://www.calstate.edu/acadaff/sloa/

New Resource Webpage : 

39 New Resource Webpage http://research.ccc.cccd.edu/SLO_Menu.htm/

authorStream Live Help