## Evidence based mathematics teaching

The three invited speakers, Lara Alock, Toby Bailey and Franco Vivaldi all gave amazing talks, and is was great to see so many interested colleagues, both from RHUL, and other universities. Slides from the talks are available here.

I won’t attempt to summarise the talks, or all of the lively discussion that followed, but here are some of the things I found most striking. Comments are very welcome!

Lara Alcock spoke on ‘How do people read mathematics‘. One very interesting study was on how professional mathematicians and undergraduates differ in how they read proofs. The experts follow a ‘warrant seeking’ approach, making many one-line jumps forwards and backwards as they work out the logical structure of the proof. Undergraduates also make these jumps, but less often, and spend much more time concentrating on equations and formulae. But the really interesting thing is that, when given a booklet with some basic training on how to read a proof, the undergraduates behave much more like the experts, and understand the proofs better. Her paper is here.

Lara pointed out that this ‘Self-explanation’ training only says things that are obvious to a professional. But I’m persuaded they are not at all obvious to most of our students, who think of a proof as a minefield of explosive ordinance, best avoided entirely, or failing that, to be laboriously memorized, rather than as a reasoned argument that should be critically analysed, and probed for its mathematical insights. Another great thing about this study is that it suggests a way we can get better results without big changes in how we teach.

Lare also reported on an earlier study in which proofs were broken down into small steps, and put online with extra voice and multimedia commentary on each step. Students liked these ‘e-proofs’ and said they found them useful. However, the evidence is they did nothing to improve their understanding compared to a control group. (As Lara said, this does not mean the students were wrong in their evaluation.) So we have an example where a very plausible idea, positively evaluated by students, was in reality a drain on time and resources that gave no significant improvement in student performance.

One question raised another reading strategy for proofs: don’t read it at all but prove the result yourself. A related strategy is to read the proof just long enough to detect the first new idea. My guess is that these are common strategies for experts, particularly when faced with a long proof, but ones largely unknown to undergraduates.

Toby Bailey started with the results of a survey. Self-selected third year undergraduates were asked three questions:

1. A very basic induction question, of the type: Show that $n^3+2n$ is a multiple of $3$ for all $n \in \mathbb{N}$‘, with the rider And explain why induction works’.
2. A question testing fundamental principles in analysis: ‘Is there a smallest real number $> 1/2$?’
3. A more technical, but still very basic, group theory question: ‘Let $G$ be a finite group. Show that the order of any element of $G$ divides $|G|$‘.

About half of students could get somewhere with (1) and (2). Only 7% got a passing mark on (3), although about half could at least define ‘order’.

There was some discussion as to whether this was evidence that students know much less than we suppose they do, and how fair question (3) is. I suspect the marking in (1) was quite generous: in 2014 I caused a blood-bath by setting State the Principle of Mathematical Induction’ as a bookwork question in my first year exam. Fortunately almost all students could adequately apply it, even if they couldn’t come close to stating it.

I’m probably at an extreme here: I think most of our students take away very little from many of our lectures, and not much more from our courses, modulo a superficial spike around exam time. If true, this is a painful fact. As Toby said, quoting T. S. Eliot: ‘human kind cannot bear very much reality‘.

Toby then talked about his experiences of the flipped classroom in a linear algebra course. He described the peer instruction cycle, and mentioned Eric Mazur who has done much to popularize this model. There was some debate about whether proofs could successfully be ‘flipped’, and about whether flipping was practical in higher level courses. It seems clear to me that finding good questions is critical. The view was that good questions should be conceptual, rather than calculations. A question that sharply splits the audience can be good, because it forces everyone to realise there is a issue. One nice suggestion was to state a theorem of the form ‘$A \implies B$‘ with the implication sign missing, and then to ask students which of $A \implies B$ and $B \implies A$ is true. I like the way this softens the traditional ‘rabbit out of a hat’ presentation, while also forcing students to think about examples.

As an example question, Toby mentioned ‘Is $x \mapsto |x|^3$ differentiable at the origin?’ Of the four possible answers, the most popular was ‘No, because it is not continuous’. He speculated that this answer was popular because the students recognised that it could be a correct answer, although, sadly, not to this question. I suspect good questions are often, like this one, quite simple: another example mentioned was Is $2 \le 3$ true?’ If a question really did turn out to be trivial for everyone, it would be easy enough to go on to something new, omitting the peer instruction part.

Toby’s talk was of particular interest to me because next year I will be lecturing our first year linear algebra course. Here are two attempts at a flipped linear algebra question, probably too hard for my course. Any comments would be very welcome.

Question 1. Which numbers in $\{ 0,1,2,3,4,5\}$ are the size of a vector space over the finite field $\mathbb{F}_2$?

Question 2. Suppose that $U$ and $V$ are subspaces of $\mathbb{R}^5$. If $U$ and $V$ both have dimension $3$, what are the possible values of $\dim (U \cap V)$?

From memory, Mazur’s experience was that the flipped classroom was particularly effective at raising the performance of students near the pass/fail borderline: I’m not sure it will be so much use for our stronger students, some of whom might even prefer the traditional model.

Franco Vivaldi talked about his experience of running a second year course on Mathematical Writing at QMUL. He started with a nice challenge: Describe how to draw a line from a given point that is tangent to a given circle’—the sting being that the explanation had to comprehensible when read down the telephone. I got a bit distracted while I worked out the geometric construction: if $X$ is the point, and $O$ is the centre of the circle, then the locus of points $P$ such that $XPO$ is a right-angle is the circle with diameter $PO$. So construct this circle using the midpoint of $PO$ and then take either intersection point with the given circle.

By the time I’d worked this out, Franco had finished giving the algebraic version, and had got onto a very insightful slide titled Classroom schizophrenia’, in which he pointed out that whereas we see a course as a bundle of definitions and theorems, informed by examples, and leading to an assessment, the students more often see a course as an ASSESSMENT, to be mastered by merciless exposure to EXAMPLES. The definitions and theorems hardly get a look in.

Franco put a lot of emphasis on linguistic features. He pointed out the many ways in which mathematical writing is a special form of language: one which we speak fluently (and so don’t notice we’re doing so), but frequently baffles or misleads our students. He gave a nice example of the sort of hidden meanings that one has to get to: in

$(x+I) + (y+I) = (x+y) + I$

the symbol ‘$+$‘ has three different meanings. He also pointed out that the use of definite and indefinite articles in maths often points to fundamental distinctions and concepts: for example ‘a basis of $V$‘, versus ‘the unique solution to the equations’. This sort of precision is second nature to us, but completely alien to our students, most of whom will have had little or no training in grammar, or, more critically, as Franco argued, the opportunity to develop the skills of rigorous argument by studying philosophy, geometry or a foreign language (in depth) while at school.

He ended by making a very persuasive case that mathematical writing should have a central place in the curriculum, and be taught from the first year, starting with the fundamentals of ‘mathematical grammar’ (a typical question could be ‘which of the following make sense?’ $\sqrt{2} \implies \not\in \mathbb{Q}$, $\mathbb{Z} \backslash (\mathbb{Z} \backslash \mathbb{N})$, $\{1,2\} \iff \{2,1\}$). There was some concern about the resources this would require: substantial, but surely less than supervising the equivalent number of projects. And if we gave students no marks for highly unclear answers that still had a germ of truth in them, some might not be left with very much at all …

Some resources. Lara Alcock has recently published How to Think About Analysis, and is also the author of How to study for a mathematics degree. The booklet on ‘Self-explanation’ training is under a Creative Commons License so I made a mirror here. Franco Vivaldi’s book is Mathematical Writing. One quick search found this flipped linear algebra course.

More suggestions welcome.

### 2 Responses to Evidence based mathematics teaching

1. george says:

Excellent post – thanks Mark.

I enjoyed reading about Toby’s open-questions style of teaching (“flipped classroom” I think was his phrase). My experience is that students typically have little deep understanding/intuition unless they are taught it – only the best and brightest can infer underlying principles from dry exam preparation. This stands to reason. And encouraging debate with an openly-posed question is a good gambit to encourage it. Typically the level of debate that ensued when I tried this I found surprisingly low, but it can root out some very fundamental misconceptions quite quickly so such basic-level conversation should be encouraged.

However, the investigative open-question style is binary: if it works, and a student suddenly “gets it”, then the rest will follow with ease for him/her; if it doesn’t work, then basically you’ve just wasted a few hours while (s)he larks about and remains unprepared for an examination, the poor outcome of which will ultimately reflect badly on you the educator. My attitude tends towards “stuff it, I want to teach these kids to use their minds, and to hell with the exam marks”. But then I can be afford to be all self-righteous and preachy because I haven’t been an active teacher for many years.

P.S. With regard to the effectiveness of the standard university teaching model, I think Miles Reid’s slightly vitriolic diatribe is the best I’ve read http://homepages.warwick.ac.uk/~masda/Sermons/diatribe

2. gowers says:

Thanks for this interesting post.

I like the example where + has three meanings, but it’s not obvious to me that it really does from a psychological point of view. One can think of

something1 + something2

as meaning take everything you can in something1 and add it to everything you can in something2. That covers all three instances. It’s just that when you try to formalize it in the language of set theory, you’re forced to distinguish between adding elements and adding sets. But I think that what’s going on in people’s brains is probably not best captured by the set-theoretic formalism, so I’m not sure the three meanings really are a serious cause of confusion. There are systems of “plural logic” that would allow you to say something like “3 plus the odd numbers equals the even numbers” that are probably closer to how we actually think, but have not become standard so we have to stick with the set theory.