At the beginning of March, the British press was full of stories about a new academic study. According to those stories, the study shows that teaching children in school doesn’t make them write better. It turns out that the study didn’t exactly show that. I discuss below:
- what the press releases said
- the study and the report
- comments on what the study found
- what others said
The press reports on this study gave new life to complaints about ‘fronted adverbials’. I explained why I think those complaints aren’t justified in Fronted adverbials – Language Miscellany
What the press releases said
It is not surprising that the press reported this story in the way they did. The headline summary at the top of the press release put out by UCL (University College London) says: ‘The teaching of grammar in primary schools in England (a key feature of England’s national curriculum) does not appear to help children’s narrative writing, although it may help them generate sentences, according to new UCL-led research.’
The press release goes on to say, among other things: ‘The results showed that while children who followed the programme had encouraging results when it came to generating sentences, there was no statistically significant improvement in their narrative writing.’
Similarly, the press release put out by the University of York leads with the statement ‘Lessons on grammar are a key feature of the national curriculum taught in England’s primary schools, but they don’t appear to help children to learn to write, new research reveals.’ Study casts doubt on benefits of grammar teaching in helping children learn to write – News and events, University of York
Both press releases report that the research included a test of 1,736 primary school (Year 2) children taught by 70 teachers in 70 schools. They give the impression that the results of that testing add further empirical evidence that teaching grammar does not improve children’s writing; any improvement is only negligible.
For reasons I discuss below, I believe that impression is misleading. The conclusion that teaching grammar does not improve children’s writing may, or may not, be valid. However, that testing provides no evidence bearing on that question.
The study and the report
The report resulting from the study is Grammar and Writing in England’s National Curriculum: A Randomised Controlled Trial and Implementation and Process Evaluation of Englicious. It is available at https://www.nuffieldfoundation.org/wp-content/uploads/2022/03/Grammar-and-Writing-in-Englands-National-Curriculum-Report.pdf
This study tested a new approach to teaching grammar and writing, underpinned by modern linguistics and called Englicious. http://www.englicious.org This approach provides web-based resources, intended to make learning about grammar fun and appealing.
The study posed the following research questions:
1. To what extent is the grammar intervention Englicious effective in improving pupils’ writing?
2. What are the main implications for teacher practice as a result of implementing Englicious, and, more generally, for evidence-informed teaching of writing?
3. In what ways do the outcomes of the research have implications for the teaching of writing in the national curriculum for primary schools in England?
The study involved the following steps:
- Delivering training in Englicious to a sample of teachers. The teachers then taught a sample set of 10 lessons using the approach.
- Testing children who had been taught using the sample set of Englicious lessons. For comparison, also testing a control group of children, who had been taught by other teachers in other schools without using Englicious.
- Surveying those teachers who had participated in the testing. The survey was both by questionnaire and by visiting a small subset of those teachers. The visits included observing lessons taught by that subset of the teachers.
- Reviewing academic literature on this topic
Training in Englicious
The researchers provided one full working day of professional development on Englicious. For the study, the teachers taught Year 2 pupils (aged 6 to 7) lessons of about 1 hour, using Englicious. Each lesson covered one of the following 10 topics
- Lesson 1: Nouns
- Lesson 2: Adjectives and expanded noun phrases
- Lesson 3: Verbs
- Lesson 4: Adverbs
- Lesson 5: Present tense
- Lesson 6: Past tense
- Lesson 7: Sentence patterns
- Lesson 8: Linking (1)
- Lesson 9: Linking (2)
- Lesson 10: Consolidation
Each lesson involved explicit teaching of grammatical terms (required by the national curriculum). At the end of the lesson, pupils then applied this learning about grammar in their own writing.
The researchers gathered test results for 637 children who had received Englicious lessons (from 34 teachers in 34 schools). They also gathered results from a control group of 609 children (taught by 36 teachers in 36 schools) who had not received the lessons. The tests were carried out twice: once before children started received the Englicious lessons and then again 1-2 weeks after the end of the lessons.
The research tested pupils’:
- narrative writing; and
- ability to generate sentences based on a two-word prompt.
Test of narrative writing
The test of narrative writing was something called the GL Progress in English (PiE) assessment, in a version using a standardised narrative writing element. The researchers selected this version because the key research question was whether grammar teaching could improve pupils’ writing.
In this test, the teacher asked the class what they do if it rains. The teacher then asked them to write about a family day out in the rain. Pupils’ texts were then marked in 4 categories: composition and effect; text organisation; sentence structure; and vocabulary.
This test showed a small positive effect, but the size of the effect was ‘negligible’, and not statistically significant.
Sentence Generation Test
The Sentence Generation Test (SGT) was designed for this study. It was intended to detect an important aspect of pupils’ grammatical understanding. Pupils were given two words and asked to write as many different sentences as they could, each including both words. Each sentence received:
- 1 mark if the sentence included both words and was different from each other sentence
- 1 mark if the sentence was grammatically accurate
- 1 mark if the sentence was a complete sentence, making full sense on its own
This test showed a ‘small’ positive effect, larger than the effect shown by the narrative writing test, but not statistically significant. That small effect reflected roughly equal increases across the three elements of the overall score. This suggests a general effect on sentence generation, rather than an effect on grammatical accuracy alone.
The researchers comment that some regard an effect of this size as equivalent to about two months’ additional progress. Progress here means progress in the ability to generate more single sentences that attracted higher overall marks.
I find it surprising that the researchers describe two months’ additional progress as a ‘small’ effect. The size of an effect is not the same thing as statistical significance:
- The sample was not big enough to justify a conclusion that chance alone did not cause the effect. So the result was not statistically significant.
- Additional progress of 2 months seems pretty large within part of a single school year. And that progress resulted from only only 10 hours of experimental teaching.
The researchers say that this (small) positive effect:
- could be because of the explicit grammar teaching.
- but could also be because part of the Englicious lessons required pupils to manipulate words, phrases and sentences. The researchers did not observe this activity in all lessons given by teachers in the control group.
Survey and visits
The researchers also asked participating teachers to fill in questionnaires, and visited 12 of the teachers (6 who had taught the Englicious lessons and 6 from the control group). The majority of respondents stated that Englicious lessons were having a moderate positive effect on how pupils were writing. In the school visits, teachers indicated that the Englicious lessons were:
- making their pupils more aware of grammar; and
- improving their pupils’ ability to remember and understand the grammatical knowledge they had learnt.
The majority of respondents said that:
- knowing technical terms for grammar (such as noun, phrase or clause) is essential in learning to write; and
- the national curriculum requirements to teach grammatical terms in year 2 help improve children’s writing.
Nevertheless, during visits to all 12 schools, teachers expressed concerns that the national curriculum’s grammar requirements were excessive.
The report indicates that:
- most previous literature had found no robust evidence that explicit teaching of grammar and grammatical terms improves writing by primary school pupils.
- pupil’s writing can be improved by requiring pupils to combine sentences in various ways, for example combining two grammatically simple sentences into a complex sentence using linking words such as because and when.
Apart from one report dated 2019, it seems that all the literature reviewed containing empirical evidence was dated 2017 or earlier.
Comments on what the study found
Overall, the study’s findings seem to produce no new evidence on whether teaching grammar improves writing.
- the test of the 1,736 children was not designed to test whether teaching grammar improves writing, and was not capable of doing so. It tested whether teaching using Englicious produces better outcomes than teaching using current approaches. The testing did not test whether teaching some grammar produces better outcomes than teaching no grammar.
- by itself, the limited material in the report’s summary of the questionnaires and visits seems too thin to justify a conclusion on whether teachers think that grammar teaching can improve writing.
- the review of relevant academic literature seems to mildly support the conclusion that grammar explicit grammar teaching does not improve writing. I have no idea whether that conclusion is reasonable, but the evidence it is based on does not seem to be new.
I don’t know:
- how much explicit grammar teaching children need at various ages; and
- how best to teach children grammar.
I discuss those points next.
How much grammar and when?
Sadly, contrary to the over-blown claims in the press release and the resulting mis-informed comments in the press, the empirical test summarised in the report provides no evidence by itself that teaching grammar is unhelpful.
Like the report, I can’t provide any empirical evidence on this myself. I agree with the researchers and other commentators that children shouldn’t just be taught grammar. I am sure children also need guiding in how to write creatively and fluently and they need practice in writing. I believe that children benefit from some explicit teaching of grammar, though I don’t know how much and when.
Over many years, I reviewed many draft documents written by first-language and second-language writers of English. I found that the reviewing process was both easier and more effective when the reviewer and reviewee shared some common terminology and structural knowledge for discussing how English grammar works. I expect that this is also the case for children—if they are taught grammar in a stimulating and engaging way.
In my view, some explicit teaching of grammar helps all children. However, it is particularly important for those who are learning English as a second language, or whose first language is a dialect of English that is not standard English. Not teaching those children something about the grammar of standard English is likely to narrow their life chances, especially in societies that demand fluency in standard English.
How best to teach children grammar
The report provides some promising (though weak) evidence that Englicious may help children learn to write better. As the report acknowledges, it is difficult to assess how Englicious helps:
- by improving the way grammar is taught?
- by giving children more and better practice?
- by both of those two factors? I suspect it does help in both ways, but obviously, I can’t provide any empirical evidence on this myself.
What did others say?
I am not the first person to notice that the empirical findings summarised in the report do not support the comments made in the press. The author of the blog Stroppy Editor comments that:
- the conclusions section of the report, and the press releases by UCL and the University of York, stray a little into over-generalisation.
- the report contains a conclusion that grammar teaching did not improve 7-year-old pupil’s narrative writing. It is hard to see how that conclusion is a fair reflection of a study that compared different methods of teaching grammar.
- this was a valuable and well-conducted study, but it has been over-interpreted in parts of the conclusion and the university press releases, and in media reports that don’t look closely at the detail.
The post on Stroppy Editor is at https://stroppyeditor.wordpress.com/2022/03/14/writing-skills-and-grammar-teaching-the-misinterpreted-study-of-englicious
Comments by the grammarian in the research team
More striking are comments in a blog post by Bas Aarts, a member of the research team, in fact the only grammarian in the team:
- unlike what some of the newspaper headlines suggested, we cannot conclude from this research that grammar teaching is without value. We need much more research to find out whether perhaps grammar teaching should start in a later year, perhaps Year 4, and has a more direct effect on that age group and later ones.
- it’s crucial that the teaching of grammar is engaging and fun, and that it is made relevant to young children by making reference to stories, poems and songs.
- learning about grammar is no different from many of the other subjects that are taught in schools. We teach them because they are relevant to us. Language plays a crucial role in all our lives, and hence knowing something about how it works is very valuable. This is especially true when we set out to learn other languages.
- no doubt some of the specifications of the curriculum need to be revised, especially with regard to some of the terminology. The testing regime is also in need of overhaul. But those who have been saying that they have become successful individuals despite never having learned about fronted adverbials miss the point entirely.
My own comments on fronted adverbials are at Fronted adverbials – Language Miscellany