Supplementary Document 1: Instructional-Teams Project informational flyer
The Instructional-Teams Project (I-TP)
This work is supported by the National Science Foundation DUE-1626531
The Instructional-Teams Project Overview:
Goal:
Support development of instructional teams and learning environments that supports meaningful engagement of students.
Focus:
Establishing a high functioning and effective instructional team (specializing graduate and/or undergraduate assistants) focused on student learning through specialized roles
Designing and implementing high-quality instructional tasks that foster meaningful understanding and engagement
Creating diverse opportunities to make student thinking visible and inform instruction through formative assessment
The Instructional Team
Instructional Tasks
Instructional tasks in our project are conceived as learning opportunities created by the instructor in the classroom. The following are characteristics of high-quality instructional tasks.
Goal Directed:
Addresses learning objectives focused on core ideas and practices
High Level: Engages students in high-level cognitive processing
Productive: Results in the construction of products that integrate key pieces of knowledge
Collaborative: Involves substantive contributions from all members of student groups
Open to Assessment:
Includes diverse opportunities for meaningful formative assessment of student understanding
Formative Assessment
We focus on creating diverse opportunities to reveal and act on student thinking.
Goals:
Make students’ thinking visible
Notice key features of students’ reasoning and understanding
Make sense of students’ current reasoning and understanding
Respond with instructional actions that further support student learning
What does participation in the Instructional Teams Project look like?
There are two options for joining the Instructional Teams Project:
Option 1: Participate in a project Faculty Learning Community (FLC, described below) with the goals of learning more about the various components of the Instructional Teams Project (team, formative
assessment, and task) and building your Instructional Team for next semester. We encourage instructors who have limited experience with using instructional assistants or who feel that they need time to build an effective instructional team to join via this option.
Option 2 (Full Participation): Participate in a project Faculty Learning Community (FLC, described below) while implementing the model (that is, teach with the aid of a team that includes the Learning Researcher, Instructional Manager, and Learning Assistant roles). We encourage instructors who can envision having a team in place by the beginning of the semester to join via this option.
What will I as the Lead Instructor be expected to do?
Participate in a project Faculty Learning Community (Options 1 and 2)
A primary expectation of participating instructors is participation in a project Faculty Learning
Community (FLC). Project FLCs bring together small groups of instructors (3-4) under the guidance of facilitators. Participants focus on methods of formative assessment, workshopping instructional tasks, and strategies for building an effective instructional team. Project FLCs meet in person or remotely once every two weeks. FLC participation includes completing out-of-meeting activities (estimated ~1 hour/week) such as written reflections or preparation/identification of tasks to workshop in the meetings. Members of a project FLC are also expected to observe their fellow FLC members teaching their target courses once or twice during the semester. This provides opportunities for reflection, discussion, and exposure to different styles of student engagement.
Coordinate with your Instructional Team (Option 2: Full participation)
Participating instructors are expected to coordinate and maintain open lines of communication with their instructional teams. This includes communicating with the Instructional Manager(s) and Learning Researcher(s) regularly to discuss expectations and feedback; and meeting with Learning Assistants to delivery course-specific instruction and expectations. Strategies for effectively and efficiently
communicating with members of your Instructional Team will be a focus of the project FLCs.
What about the other members of the instructional team?
Learning Researcher:
Training. We provide Learning Researchers with training and support for: (1) eliciting, noticing and interpreting student thinking, (2) formulating meaningful instructional suggestions, and (3) preparing reports communicating their insights and suggestions.
Becoming a Learning Researcher (1 credit). LRs can receive credit for their training by enrolling in this course. Enrollment in the course is optional, but participating in training is required.
Learning Researcher Service (2 credits). LRs can receive credit for their service in your course by enrolling in this service course. Enrollment in this course is optional.
LRs may choose to receive credit or pay through your department as preceptors or TAs instead of receiving credit for the IA courses above, if this is customary for your department.
Reports. LRs are expected to prepare a weekly report in which they communicate their observations and insights about student thinking in the course, and their recommendations for instruction.
Communication. LRs are expected to maintain open lines of communication with the Lead Instructor.
Instructional Managers:
Coordinate with the Instructor and Instructional Team Members. IMs are expected to be a resource to help instructors organize and effectively communicate with their instructional teams and students, and manage the physical and/or online classroom environment. This requires active coordination and communication with the Lead Instructor and the rest of the Instructional Team.
Learning Assistants:
LAs are expected to receive training in both general best practices, and course-specific content and expectations.
Training in Best Practices. We offer a 1-credit 7-week online Learning Assistant course, Learning
Assistant Best Practices. This course focuses on general strategies for engaging students productively in class activities, e.g., active listening and productive questioning. LAs may also receive this training from within your own department. We are happy to share any of our course materials for inclusion in your own course. Please feel free to contact us for more information.
Course-Specific Training. We expect Learning Assistants to take part in training on course-specific content and expectations within their home departments (≤1 hour/week).
Supplementary Document 2: Outline of Faculty Learning Community activities
Below, we present an outline of structured FLC activities for the two semesters of the case-study period. In her first semester of participation (Fall), Jay acted as a new participant, completing all activities and attending all meetings. In the second semester (Spring), she attended meetings as a senior participant, often acting as a voice of experience and completing fewer of the activities herself.
Day 1: Team roles and communication with your team
Introductions
Group Discussion and Q&A: Team Roles
o What are the responsibilities of the Instructional Manager, Learning Researcher and Learning Assistants?
o What responsibilities can you delegate to your IM?
o What should the LR be doing in the class? What kinds of information should I expect from them?
Worksheet: Team Responsibilities and Expectations (Regularly/Daily, Before class/During class/After class)
Homework:
o Develop a communication plan that you will share with your team. How are you going to ensure that the things you need to have done for class get done? And, be made aware that it did indeed get done, or didn’t?
o Share your communication plan with facilitator and other FLC members on the FLC course page
Day 2: Team communication revisited and collecting evidence of task effectiveness
Discussion: Communication challenges
Activity: Communication strategies
o Whiteboard: In pairs, make a list of the types of communication and communication strategies/avenues available for increasing effective communication between team members
o Sharing out and discussion
Facilitator introduction: Evidence of task effectiveness. How do you know whether a task is “going well” during class?
Activity: Types of evidence
o Whiteboard: In pairs, make a list of the types of evidence available that you can use to evaluate whether a task is effective (is it meeting your goals for the task?).
o Group Discussion: What types of evidence did you identify? Who is responsible for collecting, interpreting, communicating the evidence? How can you build in greater opportunities for collecting evidence of effectiveness?
Homework:
o To the next meeting, bring (1) a task that you are working on with (2) an associated learning objective and (3) your plan for evaluating whether students have achieved the objective of the task.
Day 3: Task Workshop
Share out tasks, learning objectives, assessment plan for task
Group feedback and discussion:
o Is the planned assessment evidence aligned with the LO?
o How will the task advance students towards the LO?
o How are you planning to use your team in the implementation/assessment of the task?
Homework:
o After implementing the task we discussed today, arrive to the next meeting prepared to talk about how the task went and present/discuss the evidence that you collect about task effectiveness.
o Observe a fellow FLC member’s class. Observe on the day that the tasks discussed today are implemented, if possible.
Day 4: Task Workshop Wrap-up and General Check-in
Check-in:
o What do you feel has been going well for you and the rest of your instructional team?
What are challenges that you have faced? Anything that you would like to bring to the rest of the group?
Task follow-up:
o Describe your impressions of how the task went. Did you make any changes based on what we talked about in the last meeting? How did they go?
o What evidence did you collect to evaluate the effectiveness of the task? What did you learn?
o Do you think you will be using this task again? Is there anything you plan to change next time you implement the task?
o Group discussion and feedback
Homework:
o Come to next meeting prepared to discuss examples of how you align learning objectives, assessments and tasks.
Day 5: Aligning objectives, assessments and tasks
Share out examples of objective, assessment, task alignment
Group discussion and feedback
Check-in:
o Anything that you would like to bring to the rest of the group?
Day 6: Task Analysis
Check-in:
o Report out on experiences working on identified challenges.
o Anything that you would like to bring to the rest of the group?
Facilitator: Introduce Task Analysis tool
o Analyze task along four axes: Productive, High-level, Collaborative, Open to Assessment
Homework:
o Choose a task that you are currently developing for this semester for pre- and post- implementation analysis.
o In place of the next FLC meeting:
o Introduce your team to the Task Analysis tool. Have your team members independently apply it to the task you have chosen.
o Meet with your team to discuss task analysis. Work together to revise task to address identified weaknesses.
o Observe a fellow FLC members’ class. If possible, observe on the day they are implementing the task that they have chosen for this activity. Come prepared to give feedback on their task related to the dimensions of the Task Analysis Tool.
Day 7: No meeting
In place of this FLC meeting:o Introduce your team to the Task Analysis tool. Have your team members independently apply it to the task you have chosen.
o Meet with your team to discuss task analysis. Work together to revise task to address identified weaknesses.
o Observe a fellow FLC members’ class. If possible, observe on the day they are implementing the task that they have chosen for this activity. Come prepared to give feedback on their task related to the dimensions of the Task Analysis Tool.
Day 8: Wrap up and feedback on PLC activities
Share out experiences with team Task Analysis activity, revision and implementation of task.
Discussion: Experiences, reflections and lessons learned
Participant feedback:
o What feedback do you have about the FLC activities?
o What would you like the leaders of these FLCs to know or keep in mind for the future?
Supplementary Table 1: Example of emergent theme identification
Emergent Theme Intermediate code/subtheme
Examples of open notes Data source(s) where theme initially identified, with examples
Additional supporting data sources, with examples
Evolving pedagogical discontentment as a motivator for change
Dissatisfaction with student engagement in primarily lecture-based pedagogical approach
“desire for hands-on learning,” “lack of student engagement,” “students not actively doing,” “not happy with teacher doing all the talking”
Interview transcripts
Example: “And so I realized that in going through this, as I was doing my lecturing, and you know, doing kind of the same sort of thing that the students weren't... they were, I could just tell that they weren't engaged, I could read the room and tell that maybe a few people were but not all of them.” (pre-semester interview, Fall)
FLC recording transcripts and observation notes
Example: [Jay] talks about how she will never go back to lecturing, you talk, talk, talk and students just staring at you, not really engaging (FLC observation notes, Spring)
Dissatisfaction with level of student thinking
“hands-on, brains off,”
“lack of critical thinking”,
“engagement not enough, students should be thinking”
Interview transcripts
Example: “sometimes maybe my question isn't probing enough, maybe it is too open-ended so they answer, 'oh yes, I like that' or 'I don't like that'. But it's like, 'why don't you like that!?!’”
(pre-semester interview Fall)
FLC recording transcripts and observation notes Class observations
Instructional team emails
Example: “We want to move [students] from the most general sweeping ideas towards specific statements that can be backed up with evidence relative to air quality. Some students have elaborate maps but don’t really have evidence to support their statement regarding trends in air quality. We want to see if we can crack that barrier.” (email from Jay on
instructional team thread, Fall)
Dissatisfaction with high level of student frustration
“student frustration” Class & Post-class team meeting observations
Example: [Jay] flustered that students don’t seem to know what to do, distressed when team reports that some students are frustrated, don’t see the point (Class Observation notes, Fall)
Interview transcripts Instructional team emails
FLC recording transcripts and observation notes
Example: “I felt that the students were getting very frustrated at not having [more structure]... if I had given them some stuff, or an assignment to bring some stuff, and then we used it effectively in class, I think that would have made a big difference.” (mid- semester interview, Fall)
Supplementary Figure 1: Student rating of effectiveness of in-class activities on Teacher-Course Evaluations (TCEs)
-3 -2 -1 0 1 2 3 4
0 10 20 30 40 50 60 70 80 90
Student rating of effectiveness of in-class activities on Teacher-Course Evaluations
almost always or usually useful usually not or almost never useful
Semester
Percent of responding students
Frequency of students’ responses to the item, “Rate the overall usefulness of in-class meeting time activities (e.g., lectures, labs, discussions, teamwork, etc.) in helping you achieve important course goals and objectives” on a five-point scale (almost always useful, usually useful,
sometimes useful, usually not useful, almost never useful). The first semester of the case-study period is set as “0” on the Semester axis. Student response rate (percent of total students to complete the TCE) by semester was as follows: Semester -2, 36%; Semester -1, 63%; Semester 0, 86%; Semester 1, 51%; Semester 2, 53%; Semester 3, 44%.
