Some ideas for online assessment

An insightful reflection on approaches considered and implemented for remote assessment.

Ingrid Rewitzky

Illustration by Nino Mekanarishvili

Dirk Basson shares his journey of exploring relevant and effective assessment tools with online learning. He considers the underlying reasons for assessing our students and tackles some important issues.

What motivates students to learn? I think most University students have some innate curiosity for material if it is properly motivated. This curiosity is often satisfied when passively listening to lectures, but most educators know that this is not where true learning happens. True learning only happens when students start actively grappling with the concepts, techniques and problems of a course by themselves. Unfortunately a mild curiosity is not usually enough to drive students to spend the significant amount of energy required on such an activity, especially if they are under pressure to do this for many modules at the same time. Students end up being motivated more by the looming assessments than the learning activities or the material itself, especially as the semester reaches its end. Therefore assessments need to be carefully designed to encourage the kind of learning we want to see in our students.

With the move to teaching online in March 2020 due to the Covid-19 pandemic, there was an opportunity for us to rethink our assessment strategies. In addition, we had to keep in mind the unusual circumstances under which students would be writing the assessments. I would like to share some of the ideas I used and some reflections on how they turned out.

Here are a few considerations influencing my decisions:

  • There is no way of effectively preventing students from using course materials, so they may as well be allowed for everyone during assessments.
  • Some students may have more difficulty with connectivity, either due to connectivity in their area or load shedding. Therefore I do not want to be very strict with time limits.
  • If an assessment is “open-book” and has a generous time-limit, then the questions necessarily have to assess higher order outcomes, as set out for example in Bloom’s Taxonomy.
  • Since students have access to notes, I needed to adapt my philosophical approach to assessment a bit. Instead of determining if students can do a set list of typical problems, I wanted to create an opportunity in which students could showcase what they have learned.

I should mention that these points are not necessarily universally true. For example, in other departments, it has been shown that an honour system is reasonably effective if students are not allowed to use course materials during assessments and timed exams are possible in practice if there are some regulations in place in case of connectivity problems.

I ended up using the following assessments:
  1. A short quiz assessing the computational skills that are listed as outcomes for the course.
  2. A concept map and reflection. This assessment was designed to give students the opportunity to showcase what they have learned during the semester. I asked the students to give an impression of:
    • the topics that they found to be clearer at the end of the semester than at the start;
    • the topics they were still uncertain about;
    • connections they made between the material and other courses;
    • and relevant things they learned by themselves outside the formal structure of the course, for example from YouTube videos.
  3. This kind of assessment is very individual by its nature and very little is gained from looking at what someone else did. It has the added benefit for the educator that you get an insight into what students really think, what they tend to struggle with, and what kind of connections they used to make sense of concepts. This kind of feedback can be very useful in future teaching!
  4. A final exam (and a similar assessment opportunity, to give students an idea of what the exam would look like). In this exam I did not want students to do too many calculations – any important calculations were covered in the quiz. I used the following types of problems:
    • True or False problems to check students’ understanding of the concepts. (Students were required to give a quick explanation for their answer).
    • One question asked students to recognize the vector space
  5. \[ V = \{ (x_1,x_2,x_3)\in\mathbb{R}^3 \mid x_1+x_2+x_3=0\} \]
    as the null space of the matrix [1 1 1]. Another question gave a basis for the null space of a matrix and asked students to solve a linear system related to it. This problem required students to recognize that the null space is exactly the set of solutions to the homogeneous version of the system.
    • Setting up a problem. Typically in linear algebra, students are given a matrix and asked to do calculations with the matrix. In this assessment, I wanted the students instead to take a concrete problem and write down the matrix that can be used to study the problem. This is more in line with how mathematicians use linear algebra – the routine calculations are left to the computer.
  6. I put some emphasis on the fact that students should always motivate their answers, writing some text to say what they are doing and why. Some part of the final mark depended on the writing that accompanied their answers. Initially (for the pre-exam assessment) I had even used a rubric, which emphasised that their writing and logic would get most of the marks. I abandoned this idea for the exam, because it was impractical for the large number of students and because it turned out to be hard to distinguish between marks to be awarded for logical correctness versus mathematical correctness.

In my opinion, the reflection is a great method of assessment, at least in principle. Many students mentioned how they found the reflection useful in making sense of how concepts connect to each other. In that sense, it is not only an assessment, but also a learning opportunity. In practice, however, it turned out to be somewhat impractical due to the grading commitment in the large group of about 300 students. However, I would be keen to use this form of assessment again in a group of 30 or fewer students, since I learned a lot about how the students think and make sense of the material.

Since computational problems were covered in the quiz, the main aim of the exam was for students to explain the reasons for their approach. Unfortunately, students did not, on average, improve their mathematical writing as much as I would have liked. It is likely that for this, students really need some personalised feedback. I started to give such feedback during the first assessment, but found that it is simply impractical for 300 students.

There is one last approach to assessment I want to point out called specifications grading. I have never tried it and I don’t think this is the place for me to expand on it, but please read more by following these links: Linda Nilson, Robert Talbert (mathematics perspective).

Assessment is an important cog in the learning process and designing good assessments can improve the learning experience and outcomes. Whether online or not, it is important to be clear about the purpose of your assessments. This means that you should know why you have chosen a particular type of assessment or question, but also that you should communicate clearly to students what they should expect and why. Both of these could be done by linking the assessments to learning outcomes. In this way we have at least some control in motivating our students to work toward achieving the learning outcomes we want them to achieve.

Dirk Basson

Lecturer in Mathematics,

Leave a Reply