Articulate

I like to blog.  The act of articulating the way I see things is like tidying up my mental bedroom. It does to my conscious mind what sleep does to my subconscious, sorting through the mind-junk and putting thoughts in order.

Articulation requires discipline. It instills order. It forces you to decide what ideas have value. To be coherent we must string together valuable ideas in ways that make sense to us, in the hope they will make sense to others.

When we speak or write down our thoughts, we are opening a window to the architecture of our thinking.  We expose how we see the world. It makes us vulnerable.

***

This blog is about articulation as an essential competent of any attempt to improve teaching practice. Articulation is important because to change what teachers do we must challenge what they think about what they do.  It is only through articulation that we can hope to understand what teachers think, what they believe and how they see the world. Only then can we challenge them to see things differently, and in doing so change what they choose to do.

***

But first a caveat. There is a danger when we talk about improving teachers’ practice that we assume their effectiveness is merely a product of how good they are at teaching. This is far from true, and efforts to improve practice will fall short if the teachers’ impact is hindered by the context within which they work.

Our effectiveness as teachers is highly dependent on a range of factors largely outside of our control. These include the ethos of the school in which we teach, behavioural norms for pupils, the imposed curriculum and assessment model, the prevailing political head-wind and the availability of a reliable evidence-base concerning effective learning.

It is worth reminding ourselves, before we expend effort on improving teaching, to ensure that the conditions for teaching are as good as they can be, else our efforts are in vain. With that said, let’s return to my proposition.

***

Teachers have a mental model around their professional practice. This model will inform the everyday decisions they make about how to teach. As well as affecting practice, the model will evolve as a result of what happens when they teach. Our experiences shape how we see things, which in turn affects what we do.

The mental model for novice teachers may be disorganised or chaotic. There may be a looser connection between what the teacher thinks and what they do. They will often, particularly during training, try out new things which help them bring order to their thoughts.

Expert teachers will have developed a complex and rich schema. Their model will have grown through direct experience of what happens when they teach in certain ways. They will have a default, tried and tested, position. This will provide a starting point, but the expert teacher will have a repertoire which they will draw upon when their default position proves ineffective. Teaching will, perhaps, feel instinctive; the reasons for their actions hidden from their conscious minds. Their knowledge will be tacit, possibly rarely called to question or subject to conscious scrutiny.

For the experienced but ineffective teacher, a default position will also exist. However, their mental model is faulty and their ability to interpret the signals that practice is ineffective is weak. Indeed, it may be the weakness in interpreting negative feedback that has resulted in the persistence of a faulty model. Like the expert, the ineffective teacher will rarely consciously examine their beliefs and assumptions, and is perhaps even less likely to do so less they expose to themselves the flaws in their thinking.

***

The descriptions above are clearly simplifications and stereotypes of what real teachers are like. However, they will help me illustrate my point which is that, for any teacher, to develop practice one must engender a change in the mental model. For the novice there is a need to build the model and bring coherence. The ineffective teacher must somehow confront the deficiencies in their thinking. The expert teacher, sure in their tried and tested practices, must question whether their default position has become comfortable for them but not necessarily the best for the pupils they teach.

What I am suggesting is that professional learning is not about questioning what we do, but why we do it.

When we ask ‘why?’ we begin to expose what others believe, assume and think about their practice. We begin to see the strength of their mental model; it’s robustness and validity. We also begin to see the inconsistencies, the gaps, the questionable assumptions and prejudices which inform their action.

Professional learning begins with asking each other to articulate the way we think about teaching.

Once articulated, we can provide both affirmation and challenge. Where our thinking stands up to scrutiny we can move forward with renewed confidence. Where it doesn’t, we have the opportunity to learn.

***

So, I’ve set out my case for the role of articulation in professional learning. I want to say at this point that I’m not entirely sure that I’m right about everything I’ve said. In fact, I would go further. I am almost certain that I am, at least to some degree, wrong.

That’s exactly the point really. I’m articulating this in order to expose the flaws in my argument. It is helping me ‘tidy my mental bedroom’ on the matter. If you are reading this then I am open to your challenge.

***

However, there is another reason I am setting out these thoughts.

I think school leaders have lost their way in the British education system (not all of them, but many that I come in to contact with). I think there is confusion. The confusion seems to be around how to hold teachers to account for improving their practice.

Underpinning many dubious practices now happening in schools, there is a tendency to diminish what teachers do down to a set of operational actions, which can be observed and corrected. There is a degree of arrogance whereby those in charge seem quite certain that they know what effective teaching is, or at least they know better than those they lead.

The result of the above is a set of accountability practices which focus on what teachers do, rather than why they do it. They seek to ‘correct’ weak practice, or strengthen standards by insisting on adherence to routines, systems and rules.

I know of many teachers who have fallen in line with such practices, against their better judgement. The practices they are made to adopt conflict with their instincts on what is likely to be effective. They find themselves frustrated and without a voice.

Now it may be that these teachers are wrong. It may be that the practices they are made to adopt do actually lead to improved outcomes. However, in changing their behaviours without changing their beliefs the school is fundamentally undermining the teacher’s agency, morale and job satisfaction.

***

Once we stop trying to change what teachers do and focus on changing what teachers think about what they do, we will achieve sustainable improvements in teaching standards. This is only achievable through challenging professional dialogue which exposes our knowledge, beliefs and assumptions and holds these up to scrutiny.

If I am right, then all attempts at accountability and professional learning should involve a dialogue with the teacher.

To expose ourselves to this level of scrutiny we need to trust our school leaders, not feel threatened by them.

 

 

Advertisements

What can we infer from an exercise book?

My recent experience of an Ofsted inspector at work has made me think long and hard about what can reliably be inferred from looking at exercise books.

The current Ofsted framework seems to rely very heavily on inspectors using books as evidence of standards of teaching, learning and assessment. This approach has arisen in response to the acceptance that observing teachers is a highly unreliable way of judging standards. Inspectors will now look for ‘progress over time’ instead of progress within a twenty minute observation. The proxy measure for whether students are making progress is what can be seen in the students’ books; whether that be the work produced or the ‘effect’ of the feedback given by teachers.

I accompanied our lead inspector on visits to lessons during our recent inspection. In the first lesson we visited he gathered the students’ books and selected six to look at; three Pupil Premium students’ books and three who were not Pupil Premium students, but had the same KS2 score. He proceeded to put two books side by side (two students with the same KS2 score, one Pupil Premium and one not) and compare the ‘standard’ of work seen.

Whilst in the class, he spoke to some of the students and asked some questions, including about what feedback they receive.

On leaving the room, the inspector shared his conclusions with me. He surmised that the Pupil Premium students were not producing a standard of work comparable to the other students ‘of similar prior attainment’, that the written feedback in books was not being acted upon and that the Pupil Premium students’ books were presented less well and these students showed less pride in their work.

These conclusions were extrapolated to a hypothesis that there may be a ‘problem’ in the department with the progress of disadvantaged students.

This seemed to me to be a rather confident assertion given the sample size, and was based on some assumptions which were, at best, questionable.

It’s not that I think that nothing can be learned from looking at students’ work. It is one source of evidence for judging standards, and a useful one. I am simply cautious about what inferences can reliably be made. I started to think about this by taking each piece of evidence in turn…

What can we infer from a neat book?

A neat book (headings underlined, good hand-writing, sheets stuck in) is probably a ‘good thing’, but what can we infer from the neatness, or lack of it?

We can probably infer that students who produce neat books want to impress and/or ‘take pride’ in producing presentable work. They are also better trained to be neat, which may indicate the priorities of the teachers who have taught them before.

Where most books in a class are neat we might infer that the teacher insists on high standards of presentation. Again, this is probably a ‘good thing’ and a fair indicator that this is the sort of teacher who also insists homework is handed in on time and students ‘do their best’.

However, we can’t reliably make a link between neatness and learning. Indeed, the opposite could be true.

When I produce a piece of written work (let’s say a report for governors) my work-in-progress will be anything but neat! I will start with some random notes, perhaps a diagram. This will be added to until all the content I want is on the page. I might then create a structure for the report and begin to plan how the finished product will look. Finally, I will produce a polished, professional document.

My point is that work-in-progress is messy. It is not neat because my brain does not work in a neat way; thinking is messy.

If everything I did had to be neat then this would slow me down and introduce unnecessary restrictions.

I want some of the work my students do to be messy, jumbled and ‘rough’. Too much neatness will, at times, be a distraction. I might even infer that the student overly worried about neatness is too concerned with style over substance, and aren’t fully engaging with the messy process of learning.

What can we infer from the ‘standard of work’?

Judging the standard of work is tricky. To do so, you must know what constitutes an ‘acceptable’ or ‘high’ standard for this particular student, in this subject. An expert teacher will have a considered view of whether a student’s work is of a sufficient standard, knowing the individual’s strengths and weaknesses, what instruction they received in the lead up to the piece of work and having years of experience as to what students have managed to achieve in similar tasks in the past. To make inferences about standards without this context and with just a book in front of year is far more difficult.

At the very least, we would need to know a reasonable amount about what we might expect the student to be able to do (more than knowing their KS2 score from years before!). We would also need to know what the task set was, what the teacher’s expectations were, what support and scaffolding was given and what other students were able to achieve under the same conditions.

There are also traps we can fall in to. We might make inferences from how much has been written, the quality of spelling, punctuation and grammar, or even handwriting. There is evidence to suggest that these superficial features can significantly influence how an observer might assess the inherent qualities of the work.

There is also the problem of context. Students may produce a really good essay on an accessible text but may produce a much lower standard of work when tackling Shakespeare, for example. The ‘quality’ of the work is highly dependent on the intrinsic challenge of the material.

A subject specialist may be able to establish whether the work set is sufficiently challenging for the age group; pitching the work too low may mean that all the students fail to produce the quality of work they might otherwise produce. However, without knowing the rationale behind the curriculum this judgement also relies on many assumptions.

What about comparing students’ work?

To draw valid inferences from comparing the work of two or more students we must first be sure that the students ‘should’ be producing work of a similar standard. There are obvious difficulties in relying on past assessment data, which is at best a blunt indicator of future progress.

It is possible to establish whether the work of one student is ‘better’ than that of another (although teachers are inconsistent in their judgements of this and it is a subjective process), but what conclusions beyond this can we reach?

What can we infer about progress in learning?

This is perhaps the simplest question to address. A book cannot tell you whether students are making progress in their learning. This is because the only reliable measures of progress are well-designed summative assessments which test students in controlled conditions, across the specified domain of knowledge. Work in a book may be a reliable indicator of performance in a specific task, at a point in time, but this does not tell us anything about whether the required knowledge will be retained, assimilated in to a student’s conceptual map of the domain of knowledge and recalled at the appropriate time.

It is not valid to aggregate students’ performance in specific, formative assessment tasks, or the quality of individual pieces of work, and infer mastery of an extensive domain of knowledge. You may infer how consistently a student is able to perform to a specified standard. However, this performance will be, usually, immediately after instruction and the task completed in conditions we cannot know about from looking at the book.

Neither is it valid to infer progress in learning from improving performance in specific assessment tasks, for the following reasons;

  • The improvement may indicate improvement in short-term memory (e.g. correction on maths questions or improvement in a spelling test),not long-term retention
  • The work may have been completed under different conditions (e.g. a re-drafted essay following feedback where the student has looked at examples of others’ work and been given a structure to follow)
  • The work may involve applying the ‘same’ skills to different content (e.g. analysing the results of experiments which are developing understanding of concepts of varying difficulty)

For the reasons above, comparing work at the front and back of the book (i.e. at the start and end of a time period) tells you nothing about learning which can be relied on.

What can we infer from the teachers’ marking?

The words which teachers write in books gets many inspectors and senior leaders very excited. What students appear to do as a result of these words is even more exciting!

I would contend that the following are valid inferences from looking at teachers’ marking;

  • We can infer how much time the teacher spends marking
  • We can infer (where marking points to what the student should do to improve the work) that the teacher has an idea of what they are aiming for the student to be able to do
  • If the student has responded (by writing a reply or adding to/re-doing the work), that the student has read and understood the feedback
  • We can infer whether the teacher is aware of and following the school’s marking policy

We cannot reliably infer the following;

  • That feedback is insufficient: What we cannot tell is whether students are receiving good feedback as we cannot see the verbal feedback given. It is entirely feasible that a class receives no written feedback but still receive excellent guidance on how to improve. We also cannot conclude that marking often considered to be ‘a waste of time’ (e.g. ‘well done’) means that feedback is poor. Cursory marking won’t help students immediately improve, but it shows the student that the work has been checked and valued and it may encourage them.
  • That students have learnt more as a result: Students may respond to the feedback but at best this shows a change in performance. At worst, it just shows that they are now doing something that they already knew how to do, they just didn’t know that that is what the teacher wanted or were too lazy to do it. It tells us nothing about whether the student has learnt something new. This doesn’t mean that getting students to respond to feedback is a waste of time, just that it doesn’t tell us much about learning.

There are more concerning things which we may infer from seeing lots of ‘high quality’ marking. Marking is very time consuming. Lots of marking may suggest a teacher working inefficiently, not looking for more time-efficient ways of providing feedback. We may also conclude that the more time spent marking, the less time spent planning teaching. Quality planning may lead to greater learning gains than lots of marking. Excessive marking (or even any marking!) may therefore be an indicator of conditions for poor progress.

Another side-effect of lots of marking is the teacher having less time to relax (and sleep). Tired and grumpy teachers do not promote learning. There may also be a displacement of professional learning activities. Many teachers will comment that they ‘do not have time’ to read or digest educational research.

Lots of marking may also indicate that the teacher misunderstands the most important reason for assessing work; to adapt their practice. Greater learning gains can be made by teachers adapting future teaching to address misunderstandings and gaps in learning rather than expecting students to ‘act on feedback’. Again, more time spent planning and less time spent marking may be beneficial. Marking may also indicate that the teacher relies on checking understanding after the lesson rather than in the lesson, thus missing the opportunity to address gaps in learning quickly.

Given the above, detailed written feedback might actually be a better proxy for conditions for poor progress than evidence of effective practice.

What can we infer about the quality of teaching?

Books can provide insight in to the type of tasks that a teacher sets. Particularly interesting will be the formative assessment tasks and how effectively they enable the teacher to infer understanding. Skill in designing diagnostic assessment is essential for quality instruction. However, the books will not provide evidence for the formative assessment which takes place through questioning and discussion, therefore we must be cautious in making these inferences too.

Where does this leave us?

Making valid inferences from looking at books is difficult. My conclusions are thus:

  1. Neat books are a reasonable indicator of students taking pride in their work, but we must be cautious in concluding that messy work is a ‘bad thing’. Presentation should depend on purpose and audience.
  2. The teacher is in the best position to judge the standard of work. For an observer to judge it in isolation will likely lead to invalid inferences.
  3. We must be cautious in comparing the work of two supposedly similar students.
  4. We cannot conclude anything about whether students are making progress in their learning from looking at books.
  5. Marking is an indicator of whether the teacher has clear performance expectations (but the lack of it doesn’t mean that they don’t). The student’s response is an indicator of whether they have understood the feedback (but not that they have learnt anything). Lots of marking may indicate the teacher is working too many hours or displacing activities which may have greater gains for students in the long term. We cannot infer from the amount or nature of marking how effective feedback to students is.
  6. Exercise books may help us evidence teachers’ skills in designing diagnostic assessments when considered alongside other evidence.

These conclusions suggest that we need to do a lot more than look at books if we are to gain any great insight into standards. Ofsted attempt this by ‘triangulating’ evidence by looking at progress data and talking to students. This is well meant, however triangulating evidence based on invalid inferences will not increase the validity of judgements. Indeed, by drawing invalid inferences from the books, the inspector may have introduced bias in to the process. In the case of my lead inspector, he went away to ‘find’ evidence to support his hypothesis. This affected where he looked and what he looked for, including the questions he later asked groups of students.

Time would have been better spent talking to the teacher about the work produced. This would have provided valuable contextual information and also enabled an insight in to the teachers’ intentions and reflections on their practice, and on the students’ progress. The teacher could have given their view on whether the work produced was as good as it should be, their strategies to support the student and the longer term picture. Book scrutiny as a process ‘done with’ rather than ‘done to’.

Once again, Ofsted have come to rely on a source of ‘evidence’ for judging standards which doesn’t stand up to scrutiny. That isn’t to say that they shouldn’t look at books or can’t draw some conclusions by doing so, just that there needs to be clarity about what inferences are, and are not, valid. To be relying so heavily on work scrutiny to make judgements about teaching and learning, and often, therefore, the overall grade a school receives, is highly questionable.

Given the over-confidence of the inspector in making inferences from such a small sample of exercise books, I was somewhat surprised to be advised by him later in the process to train as an Ofsted inspector to ‘sharpen’ my skills in ‘what to look for’. Hmmm.

 

Deep challenge

Our recent Ofsted inspection happened to coincide with the speech Dr Rebecca Allen gave on 7 November, titled ‘Making teaching a job worth doing (again)’ [https://beckyallen.files.wordpress.com/2010/07/2017-11-becky-allen-on-workload.pdf]. If you’ve read the speech you’ll appreciate the irony of this.

Whilst I can’t comment (yet) on the outcomes of the inspection, I am able to talk about my experience of the process, which I intend to do. Dr Allen’s speech has informed my reflection, as has Daisy Christodoulou’s marvellous book ‘Making good progress’, which I am currently reading. I am also indebted to my colleague Steve Shaw who’s thinking on everything pedagogic is wise; if I mention flowers then it is him you have to thank.

Fashionable proxies

My recent experiences have caused me to think long and hard about how we challenge practice in our schools. Ofsted’s current pre-occupation with looking at books as a proxy for whether students are making progress has intrigued me, and I am divided on the matter if I’m honest.

On the one hand, it feels like Ofsted are clutching at straws to find a reliable source of evidence for judging teaching and learning. Now graded lesson observations have been shown to be invalid and unreliable and it has been accepted that you can’t ‘see’ progress within a twenty minute observation, Ofsted’s attention has turned to assessing ‘progress over time’ by looking at the only consistently written artefact available; books.

What exactly do they hope to find? Firstly, it would appear, the inspectors will look at the ‘quality of work’ being produced. A student’s work might be looked at ‘over time’ (i.e. front of book and back of book) to see improvement. This appears to be a highly dubious endeavour and seems to result in drawing conclusions about whether students are ‘acting on feedback’, particularly in regard to whether they punctuate correctly.

A student’s work might also be compared to another ‘similar’ student (which appears to mean someone who got the same KS2 score a number of years before). Popular is the practice of looking at similar students who are or aren’t on free school meals to see whether they are achieving similar standards. Again, pretty dubious practice, particularly given the sample size.

Finally, some attention is given to ‘feedback’ (which means marking in this context) and whether this leads to improvement. There is some caution by inspectors about this as they have been told only to comment on whether this feedback adheres to the school’s marking policy. However, the fact that they can’t comment on what they think about marking in the report doesn’t seem to stop them from drawing conclusions which colour their overall judgement (in our case, they didn’t actually ask for the marking policy anyway… for all they knew it could have said ‘don’t mark books’).

Any evidence seen that comments by the teacher lead to students doing something different (‘better’) is seen as proof that ‘feedback is effective’. Worse, cursory marking, rather than just being seen as encouraging or a waste of teachers’ time, is ‘evidence’ that feedback is ineffective. This scant evidence is scaled up to a conclusion about the department or even the whole school.

As no observation of the teaching is taking place (even when they are in the classroom the inspectors are looking at books or talking to students, not paying any head to what the teacher is doing) there is no judgement made about whether there is effective verbal feedback which would probably be more effective and certainly be more efficient than hours taken writing detailed comments in books.

However, I do have sympathy for Ofsted as they search for some reliable way of judging whether teaching is effective and whether students are making progress. They can’t even rely on data any longer as inspectors are told to treat the school’s progress data with caution (rightly; who is to know what dodgy assessment methodology it is based on?).

I questioned @harfordsean (Oftsed’s National Director) via Twitter regardng how confident he was about books being a reliable proxy for learning. His reply was to say that they are currently researching just that question. Well, let’s hope the research says it is reliable for the sake of the schools whose judgement depends on this assumption in the meantime.

What should be our response?

Given the scrutiny described above, and how much there is riding on it, it is not surprising that schools adopt practices which ensure that Ofsted see what they are looking for. Dr Allen explains the effects of this coercive force well in the paper mentioned above by employing DiMaggio and Powell’s work on ‘institutional isomorphism’. I won’t repeat the argument here (read the excellent paper), but will summarise it as essentially saying that schools prioritise ‘looking good’ over ‘being good’.

I am interested in this response and contend that it detracts from a more appropriate response. The attempt to look good results, I would argue, in action which I will call superficial challenge. The attempt to actually be better involves a process which I will call deep challenge.

Superficial challenge

How might a school respond to the criticism that written feedback is not leading to actions by the students to improve their work? Usually, schools will focus attention on the type of written feedback given and what the students do with it. Anyone who works in schools will recognise this response, which takes forms such as triple-impact marking and D.I.R.T. and usually involves different colour pens.

This challenge to professional practice says ‘don’t do this’ but instead ‘do this’. It is superficial but, if adopted, will result in generating the type of behaviours desired.

Here comes the flower

What superficial challenge  fails to do is to question why the teacher is adopting the practices in question. By failing to address this question, the intervention addresses the symptoms but not the underlying condition.

At this point I need to draw on Daisy Christodoulou’s book, referenced above, and employ Steve’s flower analogy. Given we are both reading the book I think the author’s name must have subconsciously inspired the choice of biological inspiration. For this reason, I will adopt the daisy as my flower analogy of choice.

When we look at a daisy our eyes are drawn to the flower and its beautiful petals; it is designed to catch our attention. However, the flower sits on a stalk and the stalk grows from the roots below, which are hidden from view. If the petals are wilting, the problem probably isn’t with the petals but somewhere further down, probably in the roots; a lack of water or nutrients perhaps?

The visible aspects of teaching, like feedback, also rely on an entire support system. If feedback is poor then the problem probably isn’t just with the feedback; it is a sign of a more deep-rooted problem.

Daisy (as in the author, not the flower) outlines some fundamental flaws in our common understanding of learning which might explain our ‘poor feedback’ symptoms.

One such flaw is a poorly designed assessment. Without a reliable and accurate assessment methodology feedback is bound to be poor; the teacher simply lacks the data and insight to identify what the next steps in the student’s learning should be. In my experience, there are significant weaknesses in assessment methodologies in schools which has been made worse by externally imposed assessment systems and national strategies which have de-skilled the workforce.

Sitting below the ‘stem’ of assessment is the ‘root’ which is the model of progression. This too is often not well thought through. The model of progression is a clear conception of the ‘current state’ and ‘goal state’ desired at the end of a period of learning, and the route by which the student will get there. These terms were coined by Wiliam in his book ‘Embedded formative assessment’.

Without a model of progression, an effective assessment methodology cannot be designed, without which useful feedback will not be forthcoming. Wiliam says it better:

“To be effective as a recipe for future action, the future action must be designed so as to progress learning. In other words, the feedback must embody a model of progression…”

Deep challenge

So poor feedback may be a symptom of something after all. Perhaps this is a proxy we should pay attention to?

Rather than this being a proxy for learning over time, however, the best we can conclude from poor feedback is that there may be a more deep rooted problem (although it could indicate a lazy teacher). What it doesn’t tell us is what that problem may be; that would need much more exploration than an Ofsted inspector has time for.

What we do know is that superficial challenge will not be effective in resolving the issue; treating the symptom, not the underlying condition.

Deep challenge would involve digging down to the roots and uncovering whether the model of progress, assessment expertise, subject knowledge or something else is at fault. It would not be a quick fix, or solvable by a one-size fits all policy. It would take time, trust, expertise and sensitivity to explore the roots of professional practice to find out why the flower is wilting.

None of this seems very compatible with the high-stakes, quick turn-around accountability culture of the English education system.

My conclusion is that I prefer to nourish the routes of professional learning rather than adopting superficial methods to make us ‘look good’. I just need to find some quality manure.

 

 

Failing to learn

I’ve just finished reading Matthew Syed’s ‘Black Box Thinking’, which references Tim Harford’s book ‘Adapt’, both great reads on the same theme; failure. If you haven’t read either then Tim Harford’s Ted talk on the subject (https://www.ted.com/talks/tim_harford#t-1067823) gives you a sense of the territory.

Syed’s approach is to examine the culture which has developed in aviation whereby failure (in the form of accidents and near misses) are systematically and openly picked apart and the lessons learnt used to inform improvements to safety. Over time this has resulted in an incredible safety record for the commercial airline industry. Syed calls this approach an ‘open loop system’, meaning that failure leads to progress as feedback is rationally acted upon. This, he argues, is radically different to the ‘closed loop system’ found in healthcare whereby mistakes are hidden, go unrecorded and subsequently do not lead to improvement.

I was so taken with the ideas in the book that I jotted down some key points which I felt were applicable to an educational context and added a few aide memoirs for later reference.

1. Failure is an opportunity to learn. What are our failures in education? My first thoughts were lessons that go badly and exam results that weren’t what they should be. And then what about when a student’s behaviour means they must be removed from a lesson? Then I started to think about policy failures (I’ve seen quite a few if these over the years!). How do we respond to these failures? In my experience we do one or more of the following…blame someone, moan, brush it under the carpet or quickly move on to another ‘good’ idea. How could we learn to embrace these failures as opportunities to learn?

2. People need to feel ‘safe’ in admitting their mistakes. I fear we are some way from this being the case in schools. If we have a disastrous lesson, do we tell others? If we’re lucky enough to work in a department in which there is this level if trust, what about your senior management team – would you tell them? And what is the response when things do go wrong? If your class achieves poor results what is the reaction? Will it be ‘okay, let’s see what we can learn from this’? Furthermore, how do senior management respond when a policy is clearly not having the desired effect? Do they hold their hands up and admit their mistake, and do we contribute to a culture where they wouldn’t dare show this ‘weakness’?

Syed illustrates the desired culture through the example of a system adopted by Toyota in their manufacturing. Production workers would pull a chord when something went wrong. The result was that a cluster of senior managers would almost immediately appear to help resolve the problem. This system only worked once the workers trusted that the mistake could be openly admitted and the senior managers were focused on finding a solution, not looking to blame. What would it look like in schools if teachers could ‘pull a chord’? Could we envisage a culture whereby teachers could immediately call in support to help get a lesson back on track without fear of blame or a label of weakness?

3. Feedback is essential. Syed uses a golfing analogy whereby a golfer looks at the accuracy of each ball played and uses this feedback to subtly adjust his swing, gradually becoming a better player over time. Without the benefit of feedback, Syed argues, we are like golfers playing in the dark. Experience only leads to improvement when feedback is immediate, clear and acted upon. In schools, we are happy to apply this belief to students but are we as prepared to gain feedback on our own error? Syed talks about ‘error signals’ which are indicators that things are going wrong. What are the error signals in schools and which sources of feedback are most instructive? This section got me thinking a great deal about low-stakes testing as a source of valuable feedback to the teacher on their teaching – designed well, tests are choc full of error signals! Without regular data feedback on our impact as teachers are we just ‘playing golf in the dark’?

4. Interpreting data is not easy. Syed cites the work of Abraham Wald during the Second World War. Wald became involved in trying to improve survival rates for fighter pilots. To do this, the returning planes were examined to establish where the bullets hit and therefore where additional armour was needed. This data showed that the tail and cockpit were rarely hit and therefore no additional armour was required in these areas. Wald pointed out that the data was flawed. Rather than the planes which had returned being examined, the useful data was on the planes which had not returned. Wald inferred that the absence of bullet holes on the tail and cockpit were the reason for these planes returning, and that those which had been brought down had been hit in these areas. The conclusion was that the additional armour should in fact be placed exactly where the bullets holes did not appear. This counter intuitive reasoning drastically improved the survival rates of British pilots. Syed warns us to take account of all the available data but also the data that is not immediately available.

5. Beware theories that cannot be refuted. Syed gives the example from the field of Psychology of Adler’s ‘inferiority complex’. This theory suggests that human behaviour is driven by the desire to overcome one’s own feelings of inferiority. However, opposite behaviours might equally be explained by the same theory. For example, failing to defend a friend against criticism may be cited as evidence that you have overcome the social pressure to put loyalty above reason, whereas leaping to the defence of your friend may be cited as evidence that you have overcome your fear of confrontation. This idea is linked to the problem of confirmation bias which is the tendency to seek out information, and interpret it in such a way as to confirm your pre-existing beliefs. To avoid these traps Syed advocates seeking to falsify your beliefs, not to confirm them. We should ask ‘when does this NOT hold true?’. We hold true many theories about education and rarely seek out evidence that they do not hold true, or do not always apply. How healthy it would be to question when some of our cherished approaches might not actually be effective.

6. Trial and error as an approach can harness the power of failure. It is at this point in the book that Syed references Tim Harford’s book, ‘Adapt’, which in my view gives a more authoritative and compelling account of the power of trial and error. Harford, as he explains in the above Ted talk, makes a powerful case against top-down design in complex systems, arguing instead for a more evolutionary trial and error approach. Both Syed and Harford give the example of the development of a particular nozzle needed in the production of washing powder. Unilever had employed their best mathematicians to design a nozzle that would create the fine powder required. Their designs failed. In desperation they turned to a team of Biologists. They had no idea how to design a solution to the problem, but they did have an approach to finding a solution. The team took ten random designs and tested them. The most effective nozzle was selected and ten slightly varied versions produced. The process was repeated and after 45 generations and 449 failures the final highly effective nozzle was produced. Harford uses the term ‘God complex’ to refer to our tendency to believe we know how the world works when often we don’t. These ideas resonate strongly with me. I believe that school leaders are frequently guilty of possessing a God complex and attempting to design and impose systems and policies in a complex context with no evidence that these will be effective. Senior leaders are full of ‘good ideas’ but their whimsical policy making can be damaging and undermine confidence. It is possible to run schools in a different way. Let’s take the example of a school’s behaviour management approach. These are, in my experience, designed and redesigned with little evidence of what works. Why? It is surely possible to evolve a more effective system through careful trial and error. This would need to be a disciplined approach with clear criteria set for evaluating success. To be completely sure a Randomised Controlled Trial (RCT) could be set up with a control group as a point of comparison. Alternatively, Unilever’s approach of trying out various methods and evolving variants on the most effective iteration could be employed. Most of the methods tried would fail, or at least not be as effective as they could be, but the end result would be an approach which has evolved through adaptation and selection.

There is a lot to think about here. I think the main message is that improvement will only happen when failure becomes an opportunity to learn. To achieve this you need:

– to detect mistakes and feel safe to expose them

– to avoid confirmation bias

– to have testable/falsifiable theories

– to look to falsify, not just confirm

– to have immediate, clear and analysable feedback data

– to analyse with care – it is difficult

– to avoid the arrogance of top-down design when faced with complexity and instead develop trial and error approaches that build on evidence

Applied to a school context it could look like this:

  1. Get stuck in when mistakes happen – don’t blame unless there is clear evidence of negligence – treat a mistake as a mistake. Apply this thinking to poor exam results, bad lessons, students not making progress, failing systems.
  2. Bring expertise to bear on the problem. Don’t leave people alone to solve it. Allow teachers to ‘pull a chord’ when things go wrong and ensure they don’t feel ashamed to do so.
  3. Share the lessons learnt. Celebrate failure as an opportunity to learn. Make CPD about failure as well as success. Share worst practice.
  4. Generate opportunities for feedback – turn the lights on! Frequent low-stakes testing to get feedback in your teaching. Peer observation. Coaching. Structured reflection. Student and parent feedback. SLT open to and seeking criticism. Be sensitive to error signals.
  5. Openly admit when things aren’t going well. Share evidence of impact openly. School leaders to show that change is evidence based. Look for a solution, not a culprit.
  6. Adopt a systematic approach to trial and error. Accept that failure must come before success. Adapt what works. Avoid God complex by resisting the urge to design the perfect system. Apply this approach to teaching, behaviour management, improving attendance, increasing participation in sport, reporting to parents… Any aspect of the school which needs improvement.
  7. Identify where RCTs might be must effective and practical e.g. Reward systems, specific teaching techniques, marking approaches.

Okay, time to start failing.

Schools have gravity

Successful schools seem to have a force of gravity which pulls things towards them. They attract strong teachers, high attaining students, additional resources, publicity and privilege. As their reputational ‘mass’ increases their gravitational effect grows. This is a zero-sum game for the most part. The gain is at the expense of those schools whose pull is weaker. As a strategy for school improvement it makes absolute sense from the perspective of the individual school, but it makes no sense at the system level.

Now don’t get me wrong; this is no rant against the marketisation of schools. I say this is a zero-sum game ‘for the most part’ because there is the potential to raise school standards by challenging schools to attract students by being the best they can be. However, in the language of economists, we must recognise the failures of the market. There are many factors that mean the competition is not fair and there are unintended consequences of this strategy. For the purposes of this post I’m going to avoid the economic jargon and stick with my gravity analogy to unpack this a little.

How do schools become gravitational giants?

Some schools have a head start; a built in gravitational advantage if you will.

Catchment and intake is for many schools the factor that keeps their gravitational pull strong. High attaining, well cultured children give a core mass around which success can be built. The school is calm, the students are compliant and well turned out, the exam results strong. Such factors mean parents strive to send their children to the school, high quality staff strive to work there and superficial success attracts publicity and strong word of mouth.

There are many factors which mean some schools have this inbuilt advantage; selective schools, social segregation, house prices, league tables which emphasise attainment over progress to name a few.

Our system reinforces these natural advantages further. For evidence, look at the disproportionate number of grammar schools deemed outstanding by Ofsted. Once this label is attached we afford these schools further privileges, such as the right to become a teaching school. We celebrate their excellence in government case studies, we bestow prestige by recommending others learn from their success. We even exempt these schools from further scrutiny from Ofsted, creating a mystique around their untouchable brilliance, these gravitational giants around which lesser celestial bodies orbit.

Artificial gravity

Schools without this head start will seek to create their own gravitational pull. There are two ways they can approach this. Firstly, schools can strive to be excellent, this excellence becoming the gravity which pulls everything towards them. Alternatively, they can focus in looking excellent. We notice the uniform changing. Exam results increase (but through cynical curriculum design and qualification choice rather than as a result of students being better educated). The new zero-tolerance behaviour policy means students are expelled or shipped out to pre-16 courses at college. Under immense pressure and natural disadvantage, can we really expect these schools to resist taking the easy option and creating an artificial gravity around which they can build success?

Whichever of the above strategies are adopted by those schools with a naturally weak gravitational pull, if they are successful then they contribute no more to system wide improvement than the gravitational giants. The fundamental flaw is that the default model for school improvement is in creating gravity which attracts the best resources, thereby depriving other schools of these, who in turn focus their energies on creating their own gravitational mass.

What are the alternatives to the gravity strategy of school improvement?

Another way of putting this is “how can you improve a school without damaging another?”.

This is school improvement the hard way, and the ethical way. It might look like this:

  • Instead of seeking to attract ‘better’ students, work on making the students you have got better
  • Invest in your staff; if every school raises the professional capital of its workforce then the stock if great teachers will increase, to the benefit of all
  • Stop spending money on marketing which should be spent on educating
  • Aim to improve education, not exam results (they’ll follow)

and back in the real world….

The problem is that everything in our system is set up to mean that only the foolish or insane would truly shun the temptation of a gravity-building strategy. What can be done about this?

What about MATs?

 I know of a market town with three secondary schools. There aren’t quite enough students in the town to sustain all three schools. They fight for the students and they market to attract the most desirable students. They can’t help but spend their energies on gravity building strategies. Over time, the gravitational pull of each school grows and declines. There are always two bright winners and one loser.
What locks the schools in this eternal struggle is that they are only responsible for their own success. In fact, their success is dependent on the failure of another school. This competition would, if each school went about school improvement the hard way, lead to raised standards for all. However, all the incentives are set up to promote the quick win, the superficial veneer of looking successful and the easy path to getting on top.

Multi Academy Trusts might provide a way forward. If these three institutions became one trust the incentives would be completely different. No longer would the schools be permitted to gain at the expense of the others; at least not the others in the trust. However, whilst this may prevent the schools from using gravitational strategies to draw in the desirable students it would not prevent them from drawing in the best staff from the wider system.

This localised MAT model also has some downsides. The removal if parental choice in the locality is a concern. If the trust is managed poorly then all schools will suffer. The other problem with MATs is that many are not localised collaborations but rather have schools scattered around in various localities. The temptation of employing school improvement strategies which damage schools outside of the MAT are great. Central government have been guilty of encouraging this behaviour  by celebrating school turnarounds by sponsor academies which have been as the result of gravitational strategies; often radically changing the demographic of their schools.

How else can we discourage gravitational strategies?

Here are some ideas:

  • Reinstate more control of admission numbers to ensure that the total places does not significantly exceed the supply of students in a locale
  • Place significantly more emphasis on progress rather than attainment measures, possibly over-valuing value added by low prior attaining students
  • Hold schools to account for the quality of their professional development
  • Instigate a nationally accredited qualification pathway for qualified teachers and incentivise schools for supporting teachers through these courses
  • Invest in training high quality teachers and attracting an adequate supply of staff in to the profession
  • Scrap the Ofsted outstanding grade and recognise excellent aspects of practice whatever school they are found  in
  • Include in Ofsted criteria for leadership and management a requirement that schools adopt school improvement strategies that are ethical, sustainable and do no harm to other schools
  • Instigate a commission to investigate immoral school improvement strategies
  • Research and publish case studies of schools which have improved through strategies which are sustainable on a systems level

System wide improvement is possible if we can defy gravity.

The micro lesson

This article (http://www.theguardian.com/media/mind-your-language/2016/feb/19/how-to-write-the-shortest-joke-in-the-world?CMP=twt_a-media_b-gdnmedia) has got me thinking.

Jimmy Carr’s joke referenced in the article is about as succinct as a joke can get; “Vennison’s dear, isn’t it”.

It is a micro joke. It deliberately omits all the necessary information for the joke to make sense (the exformation). Our brains rapidly fill in the missing information. The joke forces our minds to work. This feels an effortless process which requires no conscious initiation. The formation of the words, how the information is presented to us, tells us little but creates a vacuum which our inquisitive brain seeks to fill.

How might we create such a vacuum in learning such that we communicate so little but draw the mind to attempt to fill the void?

When I observe lessons I often see teachers presenting vast amounts of information. If the teacher is skilled, they continually return to the key conceptual understanding they wish to impart and provide a framework upon which students can hang the information. If the student is skilled they are able to pick up the cues as to what the lesson is essentially about and thereby order the information, sifting, discarding and prioritising so that their conceptual understanding builds.

What we might learn from the micro joke is that less is more. At the heart of every lesson is a micro lesson; a succinct way of stripping out all the exformation, providing just enough information for a vacuum to be created which the inquisitive mind seeks to fill. If we seek to clarify the micro lesson we may hope to engage the students minds in finding the exformation which is required to ‘get the joke’.

The topic is photosynthesis. The text book facts are numerous. Rather than start with the topic, the learning outcomes or the definition, why not start with the micro lesson, the information which creates the vacuum?

It could be something like this: “Why don’t plants have mouths?”

In itself, this succinct question tells me nothing I need to know, but it creates a vacuum of information which my mind seeks to fill. If I cannot fill this vacuum with answers then I will fill it with more questions; I will begin to search for the exformation that will make sense of the micro lesson. I don’t want you to do this for me. If you have to explain the joke, it isn’t funny. Equally, if you have to tell me , it isn’t interesting.

Who says vacuum’s suck?

 

The Coca Cola problem

“The principles of justice are chosen behind a veil of ignorance”

‘A Theory of Justice’; John Rawls (1971)

I was asked recently to fill two hours with wise words to an audience of prospective head teachers. Naturally, I knew that I couldn’t hold their attention for this time without plenty of discussion activities. To start,  I presented this problem…

On the rare occasion that I allow my two children to share a can of Coca Cola I am presented with a dilemma. If I pour the drink in to two separate glasses there will entail an argument about which has more in it and who gets to choose first. There is a strong sense of injustice as one or other of them will think they have been conned out of precious sips of this rare nectar. How do I avoid this perceived injustice?

To their credit, the audience came up with some very creative solutions, but in true teacher fashion I held out until someone told me the answer I wanted to hear.

The solution is this; one child pours and the other chooses. The pourer has an absolute incentive to ensure that the glasses contain equal amounts of Coke as they know that the chooser will choose, if it exists, the fuller glass. Interestingly, in my experience, the child who gets to choose which to drink hardly examines the amounts at all as they know that their sibling will have gone to great lengths to ensure they are equal. Why didn’t they trust me that much? It is because they know that I have nothing to lose if they are not equal (I get my own can – Dad’s privilege).

This dilemma illustrates well the concept of the ‘veil of ignorance’, a term coined by John Rawls but which appears in the work of many political philosophers. The drink is poured in ignorance of which glass the pourer will be allowed to drink from. Justice is served as the decision maker could be subject to the worst effects of their decision.

In politics, it is rarely the case that policy makers will be at the receiving end of their decisions. Therefore, to achieve social justice, a thought experiment is required.

Imagine that tomorrow morning you will wake up inhabiting the body and life of anyone in society other than yourself. In other words, the policy you make today will affect you tomorrow in whatever strata of society you end up inhabiting.

The ‘veil of ignorance’ thought experiment simply encourages policy makers to put themselves in the shoes of all those the policy will affect. In short, it introduces empathy as a means by which those in charge can seek to avoid vested interests and the trappings of power.

The point I was trying to get across to my aspiring head teachers was that the veil of ignorance is an important concept to hold on to as a senior leader. The justice of any proposed policy may be assessed through the thought experiment described; imagining that tomorrow you wake up as any member of the school community – the NQT, a cleaner, a student with special educational needs.

This sounds simple, but even the most empathetic people will struggle to truly imagine how it would feel to be at the receiving end of a policy from so many different perspectives. Indeed it is perhaps arrogant to assume that we can second guess how others will feel.

To overcome the failings of our empathy we must devise ways to understand what it is to be subject to the whims of school leaders. I would suggest four approaches;

  1. Build a relationship with staff whereby they know their views will be heard. This is easier said than done. It is all too easy to become defensive in the face of endless critics. It is also difficult for staff to trust that you really do welcome their feedback and won’t just see them as trouble makers. However, we must find ways to keep in touch with feelings on the ground.
  2. Consultation over the proposed change.
  3. Engaging those who will be affected by the policy in its development.
  4. Marry a teacher – you can be sure to get a full critique of any management decision.

Applying the veil of ignorance will make for better policy. That is not to say that leaders should avoid making decisions which are unpopular or impact unequally, bringing advantage to some and disadvantage to others. What it does mean is that decisions are taken with empathy, and perhaps policies may therefore be adjusted to promote justice and protect those most vulnerable to the ill effects of a well meaning initiative.

Leaders who are just are leaders we will trust. We should create our policies imagining that we might actually have to be subject to them.