By Glen Croy and Tristan Cui
Posted Tue 18 March, 2024

Low student preparation levels

Walking into a classroom already knowing that only 20-40-percent of students have attempted the preparation raises a range of negative feelings for educators; disappointment, frustration, dejection…

How will students be able to engage with the class activities, building upon the expected preparation? Then the decision comes – do I focus on the prepared students, or do I focus on the 60-80-percent who haven’t, and what are the likely consequences of this decision?

Two likely consequences are that those who had prepared have class time wasted repeating pre-class elements, and those who had not prepared have a superficial introduction.

Often we attempt to lift the majority up to the point they should have prepared to – allocating 20-minutes to introduce the preparation. The two likely consequences are that those who had prepared have class time wasted repeating pre-class elements, and those who had not prepared have a superficial introduction to what they think is the level necessary for success. The next week, even fewer students prepare…

Students subsequently find it very difficult to demonstrate higher-order learning in class and in their assessments. And, in our experience, the majority of students do not achieve learning to the level we would like them to aspire to.

How could we organise things so that the vast majority of students prepare for class? 

Trialled interventions

Separately, a number of educators across different departments in the Monash Business School, teaching at undergraduate and postgraduate levels, adopted a range of techniques, including activity completion tracking (i.e., have students opened files?), pre-class or in-class quizzes, or asking students to post on a forum based upon the preparation we have asked them to do. Each of these interventions demonstrated an increase in preparation attempts. However, we were still concerned about their learning, even in relation to the introductory preparation tasks. That is, the technique (e.g., task completion tracking, quiz, or forum) evidenced engagement with preparation, but did not ensure quality of preparation.

A different approach

Given the challenge, we ran a survey with undergraduate and postgraduate students investigating their perceived and demonstrated barriers for preparing. Students claimed that difficulties in comprehending complex conceptual ideas discouraged them from completing their preparation activities. 

To support students in overcoming these difficulties and encourage preparation, starting in 2017, one educator, then two, then three adopted Perusall in our undergraduate and postgraduate units (alongside many others around Monash and the world). We now have 14 educators in our Perusall community of practice (CoP), from Management, Marketing, Business Law and Taxation, and the Monash Sustainable Development Institute, all adopting Perusall in our units. These are units with enrolments of about 50 students, to over 700.

Perusall is an online platform for assigning resources and incentivising active and collaborative learning (linked in Moodle). Usually, assigned resources are readings, but they could also be videos, websites, slides, podcasts, and images. A resource is assigned to a large group of students (about 15-20) who are incentivised to actively engage in reading/viewing and annotating the resource, and with each other. 

Reading/viewing and annotating the resource, and with each other.

Engagement is incentivised through the allocation and achievement of marks. Importantly, marks are not achieved by ‘attempting’ the activity, but by contribution to others’ understanding. The mark is calculated by an algorithm. Staff set parameters for the algorithm, including a minimum number of annotations and a weighting in relation to active and collaborative learning behaviours. Based upon students’ post-preparation contributions to in-class activities, we have found that the Perusall mark is a reliable indicator of comprehension. 

In students’ annotations, they clarify or elaborate on ideas, connect points (either in the resource or in students’ annotations) to other aspects of the resource or unit, or present aspects they are unsure of as questions. Again, the quality of the annotations is assessed by an algorithm, which emphasises quality contributions to others’ understanding. 

Students are also incentivised to engage collaboratively through annotation conversations with each other. These peer engagements are, for example, posting questions, answering questions, adding elaboration to others annotations, and upvoting particularly useful annotations. Again, the algorithm emphasises how these contribute to enhancing others’ understanding.

Figure 1. Resource (reading) and student annotations

Achieving high student preparation

It is of note that we have found that even a small mark allocation is sufficient to encourage a high-level of preparation attempts (e.g., 2-points a week). Indeed, it is common for 90%(+) of the students to attempt the preparation each week (e.g., Cui & Wang, 2023). 

Again, marks are not awarded for attempting the task (this is not a ‘participation’ mark), but for the quality of learning achieved. Indeed, the key for the in-class experience is the quality of the preparation. It is through active engagement and collaboration that a greater level of comprehension is achieved, and this is demonstrated in achieved marks (in Perusall), through the class activities, and in subsequent assessment. 

Of the 90-percent attempts, 60-80-percent were achieving full-marks in Perusall, reflecting high-levels of preparation comprehension. 

Having high levels of preparation engagement and comprehension has dramatically changed what can be done in class.

Having the high levels of preparation engagement and comprehension has dramatically changed what can be done in class. In the past, we would run class activities for an unprepared class. Now, we have been able to redesign classes to maximise the effectiveness and impact of students’ time with the discipline expert, and attempt much higher-order activities than we previously could have imagined. Of note, in moving to the higher-order activities, 20-40-percent of students had not achieved full comprehension (i.e., had not achieved full marks). As such, a small investment (e.g., 10-minute peer-teaching activity) is still needed for preparation clarification to bring everyone to the expected level. All the same, as 90-percent had attempted the preparation, there was an established base to build upon. 

The consequences for higher-order/major assignments have also been noticeable. We saw an average 6-8-percent performance boost in comparison to previous cohorts who did not use Perusall (major assignments are usually weighted between 30-40%, and across units with enrollments from 50 to 700+). In overall results, students who attempted all of their Perusall activities, on average, achieved 4-points higher than their WAM. 

A self-grown community of practice

In 2021, more Monash Business School educators wanted to adopt Perusall in their units. To build capability and confidence for the new adoptees, a Perusall community of practice (CoP) was formed. The CoP meets once a month to share experiences and as a sounding board for challenges or new ideas for using Perusall. Members of the CoP create and distribute resources and training workshops, and collaborate in Perusall-related research. 

Adopting Perusall

We will be persisting with Perusall; it has proven an extremely effective learning tool, bringing benefits to the overall learning program. 

In considering your potential adoption of Perusall:

1. Perusall is not designed to provide marks for attempting the task or ‘participation’. Instead marks achieved are based upon the quality of the learning achieved, and particularly contribution to others’ learning.
2. Nonetheless, Perusall is an activity which contributes to demonstrating achievement of unit learning outcomes at a low-level. As such, the assessment weighting should be low, particularly in higher-level units. 
3. To ensure student perceived value, beyond the small amount of marks, explicitly use Perusall resources in-class (e.g., in activities, “drawing from your Perusall reading…”), and in the assessment program (e.g., Resources criterion: “used 10-Perusall resources”).
4. Perusall activates pre-class learning, and embeds collaborative elements. It is ideal to complement this by using active and collaborative approaches in-class and post-class as well. 

Conversation starters

Please post responses in the comments below.

• If you have used Perusall, what was your experience and how did students respond?
• Have you also achieved high-levels of student preparation using other approaches? What did you do?

If you are a Perusall user, or interested in finding more, you are welcome to join our Perusall CoP. Find Tristan’s email online and get in touch.