Toward a Pedagogical Framework for Undergraduate Research: Toward a Pedagogical Framework for Undergraduate Research Pam Mabrouk Northeastern University p.mabrouk@neu.edu


Goals for Today’s Session: Goals for Today’s Session Stimulate you to think about UR in a new way Introduce cognitive apprenticeship as a model for UR experience [and experiential education] Suggest some strategies to deepen student learning in UR experiences based on educational theory/practice Encourage Deliberate application of educational theory to UR Experimentation in UR Formal assessment of UR Reporting of findings in peer reviewed literature


Outline – Backward Design: Outline – Backward Design (Whom are we attempting to educate?) What do we want them to know/be able to do? (outcomes) What characteristics do we want them to exhibit? (evidence) What activities are therefore appropriate for learning/assessment?


UR – What Is It?: UR – What Is It? Council on Undergraduate Research: “An inquiry or investigation conducted by an undergraduate that makes an original intellectual or creative contribution to the discipline.” Chris Bosso and Pam Mabrouk: Undergraduate research is a transformative collaboration between undergraduates and supervising faculty on research that extends knowledge or creative works that offer novel insights. These activities utilize the standard research or creative practices of the relevant discipline with the intent of contributing the resulting original work to the scrutiny of the greater university scholarly and artistic communities. Wenzel, Thomas J. CUR Quarterly 1997, 17, 163.


What is Research?: What is Research? Not everyone uses the same definition Majority (62% of 135 respondents): discovery of new knowledge 11%: data collection 38.5% physical chemists 5%: research doesn’t have to be new, it just has to be new to the student 66.7% physical chemists 20.2% faculty respondents explicitly included publication or presentation in their definition Majority included disclaimer – publication not normal outcome of UR Pelligrini, J.; Mabrouk, P.A., unpublished results.


Who Are Our Students?: Who Are Our Students? Children? Adults? Andragogy (Knowles) Adult learning theory (Patricia Cross, Stephen Brookfield) M.S. Knowles. The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development (5th edition). Houston: Gulf Publishing, 1998. K. Patricia Cross. Adults as Learners: Increasing Participation and Facilitating Learning. San Francisco: Jossey-Bass, 1982. Stephen D. Brookfield. Understanding and Facilitating Adult Learning: A Comprehensive Analysis of Principles and Effective Practices. San Francisco: Jossey-Bass, 1991.


Andragogy (Malcolm S. Knowles): Andragogy (Malcolm S. Knowles) Our students Are autonomous and self-directed Bring invaluable self-knowledge and life experience Come primed to learn Possess a problem-centered rather than subject-centered focus Are internally motivated M.S. Knowles. The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development (5th edition). Houston: Gulf Publishing, 1998.


Is There an Educational Model Appropriate to UR?: Is There an Educational Model Appropriate to UR? Benefits? Framework for understanding phenomenologically; Insight into how students learn; Methods for promoting student learning; and Assessing student learning and UR This is not one size-fits-all


UR is Inherently Constructivist: UR is Inherently Constructivist Research is active effort by organisms to make sense of their environment (situated) Knowledge is constructed based on the researcher’s knowledge and experience Spiraling deepens with repeated exposure and application


An Educational Framework: Cognitive Apprenticeship?: An Educational Framework: Cognitive Apprenticeship? Kardash, CarolAnne M. J. Ed. Psych. 2000, 92, 191-201. “Evaluation of an Undergraduate Research Experience: Perceptions of Undergraduate Interns and Their Faculty Mentors.”


Graduate Education as a Cognitive Apprenticeship: Graduate Education as a Cognitive Apprenticeship Gail Richmond. Journal of Research in Science Teaching. 1998, 35, 583-587. “Scientific Apprenticeship and the Role of Public Schools: General Education of a Better Kind.” K.K. Stewart and J.J. Lagowski. Journal of Chemical Education. 2003 80, 1362-6. “Cognitive Apprenticeship Theory and Graduate Education.”


Cognitive Apprenticeship Theory: Cognitive Apprenticeship Theory Modeling – watching experts Articulating – think aloud modeling by instructors before, during, and afterward (reflection) transparency vulnerability Approximating – students approximate real task Scaffolding - providing physical aids and assistance Coaching – providing student with feedback Fading - removing coaching and scaffolding supports Generalizing – the big picture


Cognitive Apprenticeship as a Model for STEM Education: Cognitive Apprenticeship as a Model for STEM Education Modeling Approximating Fading Self-Directed Learning Generalizing College classroom Active learning PBL Undergraduate Research Internships CO-OP Graduate School Post-doc Type of Learning


Bloom’s Taxonomy – What Do We Want Students to Learn: Bloom’s Taxonomy – What Do We Want Students to Learn Cognitive – knowledge Affective – attitudes Psychomotor - skills


Bloom’s Taxonomy: Bloom’s Taxonomy Cognitive Knowledge Comprehension Application Analysis Synthesis Evaluation Affective (Krathwohl’s) Receiving phenomena Responding to phenomena Valuing Organization Internalizing values


Bloom’s Taxonomy - Psychomotor: Bloom’s Taxonomy - Psychomotor Simpson, 1972 Perception Set Guided response Mechanism Complex Overt Response Adaptation Origination Dave’s, 1970 Imitation Manipulation Precision Articulation Naturalization


Lee Shulman's New Taxonomy: Lee Shulman's New Taxonomy Engagement Understanding Performance Reflection Design and Judgment Commitment Not hierarchical Non-linear Relational Interactive Synergistic


L. Dee Fink – Taxonomy of Significant Learning: L. Dee Fink – Taxonomy of Significant Learning Not hierarchical Non-linear Relational Interactive Synergistic Fink, L. Dee. 2003. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. San Francisco: Jossey-Bass.


Cognitive Apprenticeship as a Model for STEM Education: Cognitive Apprenticeship as a Model for STEM Education Modeling Approximating Fading Self-Directed Learning Generalizing College classroom Active learning PBL Undergraduate Research Internships CO-OP Graduate School Post-doc Type of Learning Foundational knowledge Application Integration Learning how to learn Caring Human dimension


Assess What? How?: Assess What? How? Useful to identify characteristics evidencing learned behaviors for each competency Assess what? student attitudes, abilities, personality Universal Discipline-specific? How? What kind of evidence? Qualitative Quantitative


Standard UR Assessment Tools: Standard UR Assessment Tools Group meetings (56%) Progress reports (56%) Seminar presentations (49%) Thesis (41%) Poster (38%) One-on-one discussions (34%) Oral defense (22%) Paper (21%) Largely declarative and procedural


Traditional UR Outcome Measures: Traditional UR Outcome Measures Publications – number and quality Presentations – number and quality Advanced study –number and what institutions


Skill Development: Skill Development Non-technical Teamwork Project management Self-directedness Goal setting Self-evaluation Communication skills (oral and written) Emotional intelligence Ethical decision making Time management Technical Problem-solving skills Experimental Design Sampling and sample prep Information literacy Troubleshooting Record-keeping skills Safety awareness Quality assurance and validation Instrumental data interpretation Statistical analysis of data


Tool Box: Tool Box Standardized Research Project Entrance/exit interviews Written surveys (e.g., SALG) Research learning contracts (RLCs) Working roadmap Concept maps Job safety analysis (JSA) Standard operating protocols (SOPs) Calibrated peer review (CPR) Reciprocal Teaching Research proposals WebGURU Progress reports Journal club Reflective journaling Research papers Group meetings Conference presentations Portfolio (e-portfolio) Individual Group Collaborative


Standardized Research Project: Standardized Research Project Idea based on “standardized patient” model from health care & medicine IDEA: standardized research project 25% chemistry projects follow a standard model: synthesize and characterize new compound (Pelligrini & Mabrouk, unpublished) Common set of skills, techniques, assessment tools


Entrance/Exit Interviews: Entrance/Exit Interviews What does the student want to learn? What should the student learn? Design experience to accommodate this? Student vs. faculty perception Did the student learn it? Structure experience to assess this Student’s perception Individual or groups Qualitative


Written Attitude Surveys: Written Attitude Surveys Student Assessment of Learning Gains (SALG) URL: http://www.wcer.wisc.edu/salgains/instructor/ Web-based can modify questionnaire Advantages Quantitative or qualitative Disadvantages Not interactive Student perceptions


SALG Questions: SALG Questions How much did each of the following aspects of the class help your learning? As a result of your work in this class, how well do you think that you now understand each of the following: How much has this class added to your skills in each of the following? To what extent did you make gains in any of the following as a result of what you did in this class? How much of the following do you think you will remember and carry with you into other classes or aspects of your life?


What are Research Learning Contracts (RLCs)?: What are Research Learning Contracts (RLCs)? Defined: Covenants (scaffolding) Designed jointly by the faculty sponsor and student Define the research experience promote mutual inquiry and accountability Content: Project Title Project Goal Methods (Skills, Training, Knowledge, Instrumentation) Student’s Work Schedule (responsibilities, safety) Outcomes I will have accomplished objective X when I have… Professional Opportunities Assessment/Evaluation Plan Signatures and Dates Mabrouk, Patricia Ann. CUR Quarterly. 2003, September, 26-30. “Research Learning Contracts: A Useful Tool For Facilitating Successful Undergraduate Research Experiences.”


Working Roadmaps: Working Roadmaps Step-by-step outline of project Crafted on the fly Authored by advisor-student team Modeling Transparency Vulnerability Scaffolding Mabrouk, P.A. Chem. Educator 2000, 5, 43-48. "Successful Strategies for Integrating High School Students into a Graduate Research Group."


Concept Maps (J.D.Novak): Concept Maps (J.D.Novak) Visual representation of relationship between different concepts Types Hierarchical Spider (centrality, relationship) Flowchart Evaluation (M. Nakhleh) (#correct - #wrong noninformative)/total #connections J.D.Novak. Learning, Creating, and Using Knowledge: Concept Maps As Facilitative Tools in Schools and Corporations. Mahwah: Lawrence Erlbaum Associates, Inc. 1998.


Slide32: Abstract Introduction Experimental Results Discussion problem Authors What’s been done What’s not been done What they are reporting What they have done chemicals methods instruments data Interpretation of analyzed data quality analysis


Calibrated Peer-Review (CPR): Calibrated Peer-Review (CPR) Developed by UCLA (Orville Chapman) with NSF and HHMI http://cpr.molsci.ucla.edu/ UCLA and TAMU leads, 2001 Process (automated) Students learn how to review (calibration) Students perform peer review on others’ work Students review their own work Chapman, O. L.; Fiore, M. A. "Calibrated Peer Review™" Journal of Interactive Instruction Development 2000, 12(3), 1-15. Robinson, Ralph. "An Application to Increase Student Reading and Writing Skills." The American Biology Teacher September 2001, 63(7), 474-479.


Reciprocal Teaching – Palincsar and Brown: Reciprocal Teaching – Palincsar and Brown Summary: Advisor and students take turns being teacher Process: All read a paragraph silently “teacher” formulates a question based on the paragraph, constructs summary, makes a prediction or offers clarification Shared learning experience Peer support Teacher models Development of good question generation Summarizing Clarification Critiquing Prediction Both teacher and students must articulate their knowledge Palincsar, A.S. (1986). Reciprocal teaching. In Teaching reading as thinking. Oak Brook, IL: North Central Regional Educational Laboratory. Palincsar, A.S., & Brown, A.L. (1985). Reciprocal teaching: Activities to promote read(ing) with your mind. In T.L. Harris & E.J. Cooper (Eds.), Reading, thinking and concept development: Strategies for the classroom. New York: The College Board. Palincsar, A. & Brown, A. (1984) Reciprocal Teaching of Comprehension-Fostering and Comprehension-Monitoring Activities. Cognition and Instruction. 1(2), 117-175.


Reflective Journaling: Reflective Journaling Structured Focused/guided One or more questions Students can track their growth/progress Students assume responsibility for their growth/progress Internet or not Blogging (IP/publication) Peery, Angela B. 2005. Improving Instruction through Reflective Journaling. Advanced Learning Press, CO.


Structured Reflective Journaling: Briefly describe a situation that affected you as an individual or your team as a whole this week. What were you feeling at the time? What was good or bad about the situation? (values, assumptions) Is there an overarching issue or problem here? Has this experience challenged your assumptions, prejudices, or biases? (How) will this experience alter your future behavior, attitudes, or career? Structured Reflective Journaling


Research Proposal/Thesis: Research Proposal/Thesis Written document outlining background, significance, and research plans (budget) Evaluation rubrics Use peer review or CPR Oral defense


WebGURU: Web-Based Guide to Research for Undergraduates: WebGURU: Web-Based Guide to Research for Undergraduates URL: www.webguru.neu.edu


Progress Reports: Progress Reports Oral or written Concise outline of accomplishments to date and future plans Evaluation rubrics Internet-based Wiki Blackboard Or not (smile) MS Word (track changes) Circulate via email


Journal Club: Journal Club Group meeting to discuss current literature in a field Shared responsibility for presentations Meeting frequency – weekly, biweekly or monthly Formal or informal – Powerpoint or “chalk talks” Craft and evaluate written reviews of articles selected using journal review criteria CPR? Verbs Oral communication Technical writing Critical thinking Problem solving


Group Meetings/Seminars: Group Meetings/Seminars Peer-review using rubrics Calibrated peer review Advisor provides feedback on feedback “Chalk talks” Overall quality of presentation Written materials (abstract) Writing Technical content References (for lit. review) Visual aids Overall quality Technical accuracy Oral presentation skills Presentation Q/A


(E-)Portfolios: (E-)Portfolios Collection of relevant discipline-dependent artifacts reflecting knowledge, skills, creativity Web-based or not (peer-reviewed literature) Blackboard Institutional Cambridge, Barbara L., ed. 2001. Electronic Portfolios. Emerging Practices in Student, Faculty, and Institutional Learning. Washington, D.C.: American Association for Higher Education. Gathercoal, Paul, Douglas Love, Beverly Bryde, and Gerry McKean. 2002. Web-Based Electronic Portfolios. EduCause Quarterly 37 (2):29-37. Slater, Timothy F. 1997. The Effectiveness of Portfolio Assessments in Science. Journal of College Science Teaching 26 (5):315-318.


(E-)Portfolio: (E-)Portfolio Resume or CV Teaching statement or research statement Research proposal Meeting abstracts Progress reports Video clips of presentations Selected reflective journal entries Self-evaluation Construct, assume, and/or refine professional identity Credentialship


For Reflection and Action: For Reflection and Action Identify what you want your UR students to learn What characteristics you want them to exhibit Methods to facilitate learning of these skills Methods to assess learning of these skills Consider making one change and probing its effect/value