Susan Larson
Research on Undergraduate Research
What do we know about the benefit of UR?
Undergraduate Research in American
Higher Education
Undergraduate Research as Research
Serves to advance
research
Research
OUTCOMES
are a key
determinant of quality.
Focus on PRODUCT
Undergraduate Research as Pedagogy
A pedagogical
approach to enhance
student learning
Student LEARNING
is
a key determinant of
quality.
Focus on PROCESS
UR as Persistence &
Retention Strategy
A tool to enhance
recruitment and
retention
Student SUCCESS
in
college/university is a
key determinant of
quality.
Defining Undergraduate Research
An inquiry or investigation
conducted by an undergraduate
student that makes an original
intellectual or creative
contribution to the discipline.
Council on Undergraduate Research
www.cur.org
Undergraduate Research in American
Higher Education
Undergraduate Research as Research
Serves to advance
research
Research
OUTCOMES
are a key
determinant of quality.
Focus on PRODUCT
Undergraduate Research as Pedagogy
A pedagogical
approach to enhance
student learning
Student LEARNING
is
a key determinant of
quality.
Focus on PROCESS
UR as Persistence &
Retention Strategy
A tool to enhance
recruitment and
retention
Student SUCCESS
in
college/university is a
key determinant of
quality.
UR Facilitates Involvement, Integration,
Engagement
Involvement
Astin, A.W. (1984), Student involvement: A developmental theory
for higher education. Journal of College Student Personnel, 25,
297-308.
Integration
Tinto, V. (2003) Leaving College: Rethinking the Causes and Cures
of Student Attrition, 2
ndEd.
Engagement
High Impact Practices
Kuh, G. (2008). High Impact Education Practices: What They Are, Who
Has Access to Them, and Why They Matter. Washington, DC:
AAC&U.
UR in the Classroom
Students participate in a discovery-based project, engaging
in disciplinary practices, on a broadly relevant question.
Critical components of Course-based Undergraduate
Research Experiences (CUREs) in biological sciences
The use of scientific practices
Thinking like a scientist
Communicating results
Using tools and technology
Collaboration
Iteration
Discovery, broadly relevant work
Undergraduate Research in American
Higher Education
Undergraduate Research as Research
Serves to advance
research
Research
OUTCOMES
are a key
determinant of quality.
Focus on PRODUCT
Undergraduate Research as Pedagogy
A pedagogical
approach to enhance
student learning
Student LEARNING
is
a key determinant of
quality.
Focus on PROCESS
UR as Persistence &
Retention Strategy
A tool to enhance
recruitment and
retention
Student SUCCESS
in
college/university is a
key determinant of
quality.
Inquiry for knowledge building vs.
inquiry for learning
Retention and Student Outcomes
https://news.utexas.edu/2016/06/01/hands-on-approach-boosts-graduation-rates-stem-retention
Undergraduate Research in American
Higher Education
Undergraduate Research as Research
Serves to advance
research
Research
OUTCOMES
are a key
determinant of quality.
Focus on PRODUCT
Undergraduate Research as Pedagogy
A pedagogical
approach to enhance
student learning
Student LEARNING
is
a key determinant of
quality.
Focus on PROCESS
UR as Persistence &
Retention Strategy
A tool to enhance
recruitment and
retention
Student SUCCESS
in
college/university is a
key determinant of
quality.
Assessing Outcomes
Determinants of assessment approaches
Overarching goals of the UR program
To improve student learning
To integrate and retain students at college/university
To broaden participation
To contribute to faculty research agendas
Purpose of your assessment
Focus
Students
All or subset of students
Faculty or other mentors
Institutions
Undergraduate Research as Research
Research
as
Research
Faculty
- Scholarly products
- Data collection
assistance
- Preparing the next
generation
Students
- Scholarly products
- Professionalization
- Understanding the
discipline
Institutions
- External support
and recognition
- Recruitment of
quality students,
faculty & staff
Undergraduate Research as Research
Research
as
Research
Faculty
- Scholarly products
- Preparing the next
generation
Students
- Scholarly products
- Professionalization
- Understanding the
discipline
Institutions
- External support
and recognition
- Recruitment of
quality students,
faculty & staff
Does UR prepare students the next generation?
Propensity matched study following students who did and
did not do research for 10 years
Evaluated intensity & duration of UR, GPA, degree
attainment, in graduate school, in STEM workforce
Students who did UR
Had higher GPAs
Were more likely to earn a bachelors degree, be accepted into
graduate school, and have a science-related job
Positive outcomes were related to intensity and duration
of UR
Future research – WHY? What are the salient features of
UR that matter?
Undergraduate Research as Research
Research
as
Research
Faculty
- Scholarly products
- Preparing the next
generation
Students
- Scholarly products
- Professionalization
- Understanding the
discipline
Institutions
- External support
and recognition
- Recruitment of
quality students,
faculty & staff
Predictors of mentors publishing with UR
co-authors
Surveyed biomedical faculty at 14 US institutions
Collaboration variables
Commitment to UR mentoring
Faculty characteristics
Variables associated with more student-faculty
publications:
Longer mentoring relationships
More mentees
Enjoying teaching about research
Perceive undergrads can help with research
Did not ask if UR mentors published more than
non-mentors
Undergraduate Research as Pedagogy
Research
as
Pedagogy
Faculty
- Engaged &
improved teaching
- Greater
satisfaction
Students
- Deeper learning
- Thinking like a…
- Cognitive &
intellectual skills
Institutions
- Retention
- More engaged
faculty & students
UR as
Curriculum,
Undergraduate Research as Pedagogy
Research
as
Pedagogy
Faculty
- Engaged &
improved teaching
- Greater
satisfaction
Students
- Deeper learning
- Thinking like a…
- Cognitive &
intellectual skills
Institutions
- Student
retention
- More engaged
faculty & students
Do CUREs help students think like a scientist?
National Genomic Research Initiative (supported by
HHMI-SEA) develop a phage hunter course
Nonmajors courses
Goals to develop interest in and understanding of science
Pre-post self-report survey & focus groups to
understanding of science
Outcomes
Positive change in understanding about research
Developed an appreciation of the hands-on nature of science
Had a more sophisticated understanding of science
Also, greater retention from first to second semester
Pre-post assessments, could benefit from matched controls
CUR Quarterly, 2016
Similar outcomes in a Phage Hunter
course reported by Overath,
Zhang, Hatherill (2016), CUR
Quarterly
Undergraduate Research as Pedagogy
Research
as
Pedagogy
Faculty
- Engaged &
teaching
- Greater
satisfaction
Students
- Deeper learning
- Thinking like a…
- Cognitive &
intellectual skills
Institutions
- Student
retention
- More engaged
faculty
Does UR develop intellectual capacity and
disciplinary knowledge of student?
2400 alumni invited to participate; 996 respondents
3 groups emerged: i) students who participated in organized
UR program, ii) students who otherwise reported doing UR,
and iii) students who did not do UR
Asked various questions about student experience
Didn’t make the purpose of their assessment known
UR students reported greater gains related to intellectual
curiosity, acquiring information independently,
understanding findings, analyzing literature critically,
speaking effectively
Greater benefit if in organized UR program
Duration of participation greater perceived gains
Undergraduate Research as Retention
Research
as
Retention
Faculty (indirect)
- More students in
major
- Diversify the field,
by retaining more
and diverse students
Students
- Development of
professional
outcomes
- Under-represented
students continue
Institutions
- Improved
graduation rate
- Decreased time to
completion
Undergraduate Research as Retention
Research
as
Retention
Faculty (indirect)
- More students in
major
- Diversify the field,
by retaining more,
diverse students
Students
- Development of
professional
outcomes
- Under-represented
students continue
Institutions
- Improved
graduation rate
- Decreased time to
completion
Does UR improve retention?
University of Michigan UROP
Students who did UR matched with students who applied, but
did not get accepted
Both groups were motivated, similar academic traits
UR during first year
Those who did UR had higher retention rates
When controlling for college GPA, test scores, ethnicity
Differences were statistically significant for African Americans
More recent studies suggest underrepresented students
have higher GPAs after UR, but were not faster to
graduate
(Haeger & Frequez, 2016)
Overlapping Outcomes
Research
Retention
Pedagogy
Degree
Comp-letion
Professional
Skills
Personal
Growth
Cognitive &
Intellectual
Skills
Student
Outcomes
Cognitive &
Intellectual
Growth
Personal
Growth &
Development
Professional
Skills &
Advancement
Student-faculty
collaborative research
E.g., summer research,
apprentice-style
research in laboratory
Independent research
E.g., honors thesis
Course-based
undergraduate research
Student Outcomes
Thiry, Weston, Laursen, Hunter (2012) CBE-Life Science Education Hunter, Laursen & Seymour (2006). Science Education
Cognitive & Intellectual Growth
Outcome
Measured by
Thinking like a scientist
Views of Nature of Science
Questionnaire (Khishfe, et al., 2002)
Colorado Learning Attitudes About
Science Survey (Adams, et. al, 2004)
Understanding how the
discipline works
Like surveys noted above;
discipline-based surveys
Content knowledge
Tests, homework, major field
tests
Technique & skill development
Observation in the lab
Problem solving & critical
thinking
California Critical Thinking Skills Test,
Learning Critical Thinking Inventory
Can also evaluate with rubrics, self-report, pre-post assessments.
Corroborate with faculty assessments.
Cognitive & Intellectual Growth
Outcome
Measured by
Thinking like a scientist
Views of Nature of Science
Questionnaire (Khishfe, et al., 2002)
Colorado Learning Attitudes About
Science Survey (Adams, et. al, 2004)
Understanding how the
discipline works
Student understanding of
science and scientific inquiry
Content knowledge
Tests, homework, major field
tests
Technique & skill development
Observation in the lab
Problem solving & critical
thinking
California Critical Thinking Skills Test,
Learning Critical Thinking Inventory
Can also evaluate with rubrics, self-report, pre-post assessments.
Corroborate with faculty assessments.
Cognitive & Intellectual Growth
•
Preponderance of evidence: self-report
•
More sophisticated assessments for course-based UR
•
Yet unanswered
(some work has been done in these areas)
:
•
What specific features of UR produce cognitive &
intellectual outcomes?
•
E.g., how important is duration? intensity?
•
Some assessments of these; are there others to assess?
•
Are there differences between experiences?
•
Do UR experiences that focus on process (vs
product) produce different outcomes?
•
Is UR better in the 1
st(vs 4
th) year?
•
Need to evaluate more sophisticated cognitive domains
•
E.g. , knowledge integration, expertise development
Professional Skills & Advancement
Outcome
Measured by
Clarifying & sustaining interest
in discipline
Self-report surveys, focus
groups
Socialization to and completion
of major
Focus groups, graduate rates,
student presentations
Graduate school
acceptance/completion
Alumni surveys
Transferrable skills, such as
- Teamwork
- Collaboration
- Communication
- Independence
- Ethical understanding
Self-surveys, such as: SURE,
CURE survey (Lopatto), USSRA
(Hunter, et al.);
mentor observation of
students
Instruments Used for UR Evaluation
Survey of Undergraduate Research Experience (SURE)
Classroom Undergraduate Research Experience (CURE)
Research on Learning and Education (ROLE)
https://www.grinnell.edu/academics/areas/psychology/assessments
Lopatto (2004), Cell Biology Education
Undergraduate Research Student Self-Assessment
(URSSA)
SALG surveys
http://spot.colorado.edu/~laursen/accessURSSA.html
Professional Skills & Advancement
Outcome
Measured by
Clarifying & sustaining interest
in discipline
Self-report surveys, focus
groups
Socialization to and completion
of major
Focus groups, graduate rates
Graduate school
acceptance/completion
Alumni survey
Transferrable skills, such as
- Teamwork
- Collaboration
- Communication
- Independence
- Ethical understanding
Self-surveys, such as: SURE,
CURE survey (Lopatto), USSRA
(Hunter, et al.);
mentor observation of
students
Professional Skills & Advancement
•
Qualitative and quantitative analysis
•
Self-report survey data confirmed by focus group
•
Most research on skills & advancement completed with
STEM majors
•
Especially biology, chemistry
•
Many studies focus on student-faculty collaborative
research, but similar evidence with CUREs
•
Yet unanswered
•
Impact in non-STEM (non-lab science) areas
•
More research needed
•
Role of identity and belonging, as drivers of
persistence and advancement
Personal Growth & Development
Outcome
Measured by
Changes in self-understanding;
self-confidence; self-efficacy
Pre-post measures of student
confidence; reflective writing;
pre-existing measures of
confidence
Identity development
Survey by Palmer, Hunt, Neal,
Wuetherick (forthcoming)
Transferrable skills, such as
- Teamwork
- Collaboration
- Communication
- Independence
- Ethical understanding
Self-surveys, such as: SURE,
CURE survey (Lopatto), USSRA
(Hunter, et al.)
Personal Growth & Development
•
Preponderance of evidence from self-report
•
Self-report survey data confirmed by focus group
•
Most analyses focus on professional skills and experiences
•
Less assessment of identity, self-awareness, etc.
•
Yet unanswered
(some, limited work has been done)
•
What specific features of UR produce personal growth
and development?
•
Are there differences between kinds of UR
experiences?
•
Role of the mentor in personal growth/development
•
Some attempts to align UR outcomes with developmental
Summary of the State of Research on Research
Research approaches
Most data from self-report measures & pre-post measures
Limited assessment with appropriately matched control
Best approaches use mixed methods
Relevant, impactful aspects of the UR experience
Some assessment of duration and intensity
Other intervening variables have received little attention
E.g., ownership, role of mentor, number of experiences,
Comparison between types of experiences
Most assessment look at a single UR experience
How do UR experiences compare?
Undergraduate Research as Research
Serves to advance
research
Research
OUTCOMES
are a key
determinant of quality.
Focus on
PRODUCT
Undergraduate Research as Pedagogy
A pedagogical
approach to enhance
student learning
Student LEARNING
is
a key determinant of
quality.
Focus on
PROCESS
UR as Persistence &
Retention Strategy
A tool to enhance
recruitment and
retention
Student
SUCCESS
in
college/university is a
key determinant of
quality.
What is distinctive about undergraduate
research?
The focus on scholarly products,
learning process, and student success
makes UR distinctive. Assessment of
UR should consider these outcomes
in combination.
Questions?
Susan Larson, larson@cord.edu
References & Sources
Astin, A.W. (1984), Student involvement: A developmental theory for higher education.
Journal of College Student Personnel, 25, 297-308
Bauer, K.W. & Bennett, J. S. (2003). Alumni perceptions used to assess undergraduate research
experience. The Journal of Higher Education, 74, 210-230.
Brownell, S.E. & Kloser, M.J. (2015). Towards a conceptual framework for measuring
effectiveness of course-based undergraduate research experiences in biology. Studies in Higher Education, 40, 525-544.
Brownell, S.E., Kloster, M.J., Fukami, T., & Shavelson, R. (2012). Undergraduate biology lab
courses: Comparing traditionally based “cookbook” and authentic research-based courses on student lab experiences. Journal of College Science Teaching, 41, 36-45.
Dolan, E.L. (2017). Undergraduate research as curriculum. Biochemistry and Molecular Biology
Education, 45, 293-298.
Haeger,H. & Fresquez, C. (2016). Mentoring for inclusion: The impact of mentoring
undergraduate researchers in the sciences. CBE-Life Science Education, 15, 1-9.
Harrison, M., Dunbar, D., Ratmansky, L., Boyd, K., & Lopatto, D. (2011). Classroom-based
science research at the introductory level: Changes in career choices and attitudes. CBE-Life Sciences Education, 10, 279-286.
Hernandez, P.R., Woodcock, A., Estrada, M., & Schultz, W. (2018). Undergraduate research
Hunter, A-B., Laursen, S.L. & Seymour, E. (2006). Becoming a Scientist: The role of
undergraduate research in students’ cognitive, personal, and professional development, Science Education, 91, 36-74
Ishiyama, J. (2002). Does early participation in undergraduate research benefit social science
and humanities students? College Student Journal, 36, 380-386.
Kuh, G. (2008). High Impact Education Practices: What They Are, Who Has Access to Them, and Why
They Matter.
LaPlant, J.T., Ed. (Fall, 2016). Undergraduate research in the curriculum, CUR Quarterly LaPlant, J.T., Ed. (Winter, 2016). Undergraduate research in the curriculum: Part II, CUR
Quarterly
Levy, P. (2011). Embedding inquiry and learning into mainstream higher education: A UK
perspective. CUR Quarterly, 32, 36-42.
Lopatto, D. (2004). Survey of undergraduate research experiences (SURE): First findings. Cell
Biology Education, 3, 270-277.
Lopatto, D. (2010). Science in Solution: The Impact of Undergraduate Research on Student Learning,
Washington DC: Council on Undergraduate Research.
Morales, D.X., Grineski, S.E., & Collins, T.W. (2017). Increasing research productivity in
undergraduate research experiences: Exploring predictors of collaborative student-faculty publications. CBE-Life Science Education, 16, 1-9.
Murray, J.L. (2018). Undergraduate Research for Student Engagement and Learning. New York:
Routledge.
National Academy of Science (2017). Undergraduate Research Experiences for STEM Students:
Successes, Challenges, and Opportunities. Washington, DC: National Academies Press
Nagda, B. A., Gregerman, S. R., Jonides, J., von Hippel, W., & Lerner, J. S. (1998). Undergraduate
student-faculty research partnerships affect student retention. The Review of Higher Education, 22, 55–72.
Noji, K.Y. (2011). Marketing and recruiting through undergraduate research teams, CUR
Quarterly, 32, 16-21.
Overath, R.D., Zhang, D., & Haterhill, R. (2016). Implementing course-based research increases
students aspirations for STEM degrees, CUR Quarterly
Palmer, R., Hunt, A.N., Neal, M.R., & Wuetherick, B. (fortcoming). Mentored undergraduate
research: An investigation of students’ perceptions of its impact on identity development, In Vandermaas-Peeler, M., Miller, P.C., & Moore, J.L. (Eds.), Excellence in Mentoring Undergraduate Research, Council on Undergraduate Research
Tinto, V. (2003) Leaving College: Rethinking the Causes and Cures of Student Attrition, 2nd Ed. Thiry, H., Weston, T.J., Laursen, S.L., & Hunter, A-B. (2012). The benefits of multi-year research
experiences: Differences in novice and experience students’ reported gains from undergraduate research. CBE-Life Sciences Education, 11, 260-272
Willison, J & O-Regan, K. (207). Commonly known, commonly not known, totally unknown: A
framework for students becoming researchers. Higher Education Research & Development, 26, 393-409.
