### Houston, we have a problem

### It’s not dumbing down, is it?

*This is a talk given at the Higher Education Academy Maths for Chemistry Meeting Special Interest Group meeting on 18 June 2008 at The Royal Society of Chemistry, Burlington House, London. I’m presenting as if in the first person so you’ll have to imagine my delivery style.*

### A little history

In 2005 I conducted a survey among colleagues over what they saw as the maths requirement for incoming students on their modules. As a basis for this I used the requirements of the GCSE Higher Tier paper. What I found was that the requirements for incoming students was a good understanding of the topics covered in that paper. Several colleagues reported that “Students are not as good as they were”, although a reduction in the standards of material taught at degree level would be seen as ‘dumbing down’ and not an option that we want to take.

This research uses the Examiners reports from the websites of the AQA, Edexcel and OCR examination boards. Links to the sources are made at the end of this blog entry.

### AQA: Pass marks history (maths)

There has been a marked decline in the boundary pass marks for GCSE mathematics over the past few years. The chart shows just how bad it is getting. I used the word ‘bad’ advisedly as the pass mark boundary for a C grade in 2006, the students we’ll be getting in September 2008, was 20%. That’s in the higher tier paper. An A grade was in that year bottoming out at 48%. You can see from the associated table the exact figures.

### AQA: boundary marks by grade

Year | A* | A | B | C |
---|---|---|---|---|

2000 | 79 | 60 | 42 | 23 |

2001 | 81 | 62 | 44 | 26 |

2002 | 77 | 57 | 40 | 22 |

2003 | 79 | 60 | 41 | 23 |

2004 | 75 | 57 | 39 | 22 |

2005 | 72 | 54 | 37 | 22 |

2006 | 69 | 48 | 34 | 20 |

(source – AQA Examiners’ Reports – see end of this blogpost for links)

### Pattern of population by grade

Now one might expect that as the boundary mark has fallen then the numbers of students passing at each level would rise – that is if the knowledge levels of the students were to remain the same. However the following chart shows that the percentage at each grade has remained fairly constant over time.

The only conclusion I can come to here is that maths standards are slipping. I have certainly seen in over the past five years of incoming students.

### Differences between exam boards

So maybe there’s a difference between exam boards? Looking at the separate results from the calculator and non-calculator papers for 2006 for OCR, AQA and Edexcel boards shows that there are differences, but no one board, for A, B and C grades shows a consistent pattern. For all boards the non-calculator paper shows lower scores per grade than the calculator paper. No conclusion can be drawn from this because the papers were not available, but on our admissions for 2008 scores of 20% or below for C grades in the non-calculator paper do not bode well for the level of mathematical knowledge of our incoming students.

### 2006 – 2007?

Comparison was also made of the differences between 2006 and 2007 results. There appears to be an overall improvement in grade boundaries, but the improvement need to be substantially more than this to correct the trend observed from 2001.

### Is it only Maths?

In order to investigate whether other subjects were following the same pattern, and to examine the often heard mantra “English and Maths are getting worse” results were also gleaned from the AQA examiners reports of:

- English Language
- English Literature
- French
- ICT
- Double Science

### English Language

Three years data were available. The grade boundary trend is remarkably horizontal.

### English Literature

Three years data were also available. The grade boundary trend increases through time.

### French

This possibly ranks as the most boring graph of all time – there is no variance over four years!

### ICT Spec A

There are two specifications of ICT, showing something more interesting than the French, but definitely not downwards for the whole period

### Double Science

Double science is the most pure science one can expect at GCSE nowadays. Again it does not show the downward trend exhibited by maths.

### The politics, the examiners and the employers

It seems that various voices are expressing views, but the hymn sheet they are singing from is far from singular.

### Thus far:

In maths, grade boundaries have decreased between 2000 and 2006.

- More students are passing with higher grades.
- We are commonly accepting students to university courses whose grades reflect marks of less than 20% in GCSE maths.
- Our assumptions about our incoming students abilities are possibly based on precepts that are no longer valid.

The reasons for this, I believe, lie in the slavish adherence to league tables. You lose your job if your league table position does not improve. You are ‘reorganised’ and ‘reformed’ and have New initiatives’ thrust upon you. You teach to the test. This does not mean that students learn the subject, just can satisfy the examiners that they have a basic understanding of maths. And 20% is not really a strong foundation for us in Higher Education to build upon.

### Divided voices

Some evidence of the divided voices of various authorities (but I do use that word loosely)

### The examiner (2001)

A senior examiner has claimed improving GCSE grades are the result of a systematic lowering of pass marks, …

- Jeffrey Robinson – who has just retired as a maths examiner for the Oxford Cambridge and RSA Examinations board (OCR) – said the pass rate was being massaged as schools sought exam boards which would give them the best results for the school league tables. (Top GCSE grades ‘a fix’, Thursday, 23 August, 2001, BBC)

### The Examiner (2003)

GCSE results were “fixed” to mask the poorest performance by mathematics students in almost a decade, a senior examiner revealed last night.

- David Kent, a chairman of the Edexcel exam board for nine years, claimed that he was forced to lower the pass mark by about eight percentage points to ensure that thousands of students managed to pass the exam. (Maths GCSE pass mark cut to avoid mass failure 15/09/2003)

### The academic (2006)

- … despite the generous grade boundaries, almost half of 16-year-olds failed to achieve at least a C in maths. Professor Alan Smithers, the director of the centre for education and employment research at Buckingham University, said: “It looks as though the maths results are even worse than we supposed. …
- He added: “It is not surprising that employers complain about the maths skills of school leavers or that universities now have to put on remedial classes.” (Original not found, linkhere – July 2006)

### The Board and The Minister

- An exam board has admitted cutting this year’s pass mark for GCSE maths to prevent a set of disastrous results. (2003 – Edexcel – above)
- Schools minister Jim Knight congratulated students … He acknowledged that some businesses were not happy with school leavers’ basic skills. (BBC 2007)

### Employers

- Employers use aptitude tests for both graduate and placement employment.
- About 35% of students accepted to go on the placements module (strict criteria) have problems with mathematical reasoning and logic – so do not get interviewed. (University of Central England- 2007 – personal knowledge)

- More than half of employers say school leavers often cannot function in the workplace due to a lack of basic maths and literacy, a survey suggests. (BBC report on CBI – 2007)

### Opposing Voices

One would expect that people charged with representing us in promoting good education would actually talk to the bodies responsible for policing the standard, but this seems not to be the case.

### Politics – Alan Johnson (2006)

- Mr Johnson told the education select committee he backed “the whole kit and caboodle” of school accountability. That included Ofsted inspections, national tests and exams and league tables.
- He added: “If anything, we need to intensify that rather than relax.”

- Mr Johnson told the cross-party group of MPs … “I accept the pressure it puts, and the extra intensity and stress it puts on teachers, but it’s absolutely the right thing to do,” he said.

### Ofsted

- “A narrow focus on meeting examination requirements by ‘teaching to the test’, so that although students are able to pass the examinations they are not able to apply their knowledge independently to new contexts, and they are not well prepared for further study.”

### Politics – David Blunkett (2006)

- It will take another 10 years before government reforms raise standards in secondary schools, former Education Secretary David Blunkett has suggested. (School standards ‘to take decade’ 2006).

### It’s not just me!

As i was preparing for the talk, the reform group published a report on the standard of maths over the years (it’s declined) and the day before the talk another (yet another?) initiative was announced.

### Reform Group report ( 2nd June 2008 )

- School mathematics exams in England have become easier, shallower and less demanding, according to a think tank.
- Analysis of public maths exam papers taken by 16-year-olds between 1951 and 2006 shows standards have declined markedly, the report for Reform argues.
- This means more pupils have left school ill-prepared for the workplace and a generation of mathematicians has been lost to the nation’s economy, it adds.

### Maths Play and Training ( 17th June 2008 )

- There is to be a new emphasis on maths play in England’s nursery schools and 13,000 maths specialists to spearhead better primary school teaching.
- The government accepts the findings of a review it commissioned from a team led by Sir Peter Williams.

**It will take 10 years and £187m to train the specialists, expected to be drawn from existing teachers.**

*Question: Why is it taking ten years to do this?*

### Experience in teaching maths

Having spent 30 years in industry (anything from battleships to coal mines) and ending up as a software engineer before entering higher education 5 years ago, I was surprised at the level of maths being presented and have made it a goal to assist students who were struggling.

### Maths at first year in University

- Many students have difficulty interpreting written ‘maths’ problems to a framework to start solving them.
- Once the problem is interpreted then the ‘doing maths’ difficulties come into play.
- But the difficulties of interpreting written problems remain.
- Acceptable levels of comprehension are a problem.

### Causes and Remedies

In 2006 I became aware of the government recommendation to use synthetic phonics in reading. I followed this back to the researchers who did the original study and asked whether removing the requirement for analysis in reading could have led to a decline in maths standards. This was not proven – but maths standards **did **improve after the use of synthetic phonics in teaching.

- …in Clackmannanshire the teachers found that when the synthetic-phonics-taught children went into the second year at school, they needed to go up a level in the Maths scheme, that is, one level above what would normally be used. This was thought to be a direct effect of the children coping better with the reading requirements of the maths scheme.
*(Professor Rhona Johnson, University of Hull, 2007, personal communication*)

### The “Maths” problem

The problem is that the basic maths is not being understood. Students are coming in failing to understand:

- Decimals, fractions, percentages.
- Decimal places and powers.
- Algebraic manipulation.
- The concept of the variable.
- Lack of facility in mental arithmetic.
- Inability to visualise the likely result so that they can see when they’re wrong.
- “It’ll do” accuracy. (“It’s nearly right”)
- Drug calculations? Get them wrong and they don’t see that it’s a matter of life and death!

### Diagnosis

How you tell what is necessary for each student is difficult. We’ve tried all of these.

- Testing on entry, a turn off
- Self-diagnosis, slightly better
- Self-streaming, No way of knowing if the self-selection works.
- Support, Time consuming – and unless structured to modules/courses can be a waste of time.

### New remedies

This academic year we have tried some of these in a new module designed to assist students of business to cope with the demands that are going to be placed upon them. These are:

- Design of maths module to include support.
- Ability groups (but how do you determine?)
- Coaching, individual or group.
- Removing the technology, computer or calculator.
- Include self-assessment within the module.
- Show relevance to everyday problems.
- Assume no prior knowledge.

### Maths is a special case

Whatever we do, it’s an uphill task both to get the problem recognised and to get the necessary resources for support.

- Maths is falling in pass marks at GCSE
- Other subjects aren’t
- Generally targeted support does not work – you have to have subject specialists teaching the maths to explain the problem in context rather then use mathematicians
**We have to take action ourselves otherwise we will not have degrees that are internationally competitive.**

### Examiner Report Links

This research uses the Examiners reports from the websites of the relevant examination boards.

### Examiners reports, Maths

- AQA 2000 no longer on AQA website
- AQA 2001 no longer on AQA website
- AQA 2002 no longer on AQA website
- AQA 2003 link to PDF
- AQA 2004 link to PDF
- AQA 2005 link to PDF
- AQA 2006 link to PDF
- Edexcel 2006 link to PDF
- Edexcel 2007 grade boundaries link to PDF
- OCR 2006 no longer on OCR website
- OCR 2007 (not used in presentation) link to PDF

### Examiners Reports Other Subjects

- AQA English 2004 link to PDF
- AQA English 2005 link to PDF
- AQA English 2006 link to PDF
- AQA French 2003 link to PDF
- AQA French 2004 link to PDF
- AQA French 2005 link to PDF
- AQA French 2006 link to PDF
- AQA ICT Spec A 2003 link to PDF
- AQA ICT Spec A 2004 link to PDF
- AQA ICT Spec A 2005 link to PDF
- AQA ICT Spec A 2006 link to PDF
- AQA ICT Spec B 2003 link to PDF
- AQA ICT Spec B 2004 link to PDF
- AQA ICT Spec B 2005 link to PDF
- AQA ICT Spec B 2006 link to PDF
- AQA Double Science 2003 link to PDF
- AQA Double Science 2004 link to PDF
- AQA Double Science 2005 link to PDF
- AQA Double Science 2006 link to PDF

Hi

having rcently asked a fellow HOD for the grade bondaries for edexcel ver the past two years and being told its very tough now you have needed 45 – 50% ( not UMS ) to get a grade C I thought my 20 years experience was wrong. I have just spent a few hours doing some research and this is the first one I have come across that speaks knowledgably about the topic. Thank you and if you have any further information about trends in mathematics boundaries across the boards I would be very interested.

regards pat

By:

Pat Connollyon Thursday September 18 2008at 19:46:58 UTC

Hi John,

I found the above article very helpful while writing an article on school exam results for the November issue of our ISC bulletin.

Unable to find an email address to write to, could I make a request through this response box? I have cited the first graph on the page – AQA Boundary marks for combined GCSE Maths – and wanted to print it in the bulletin, acknowledging you and your site beneath it. The bulletin printers have requested the original Excel format of all graphs used so that they can import them directly. Would it be at all possible for you to email me that graph in its original format?

Many thanks indeed for any help!

Best regards,

Holly Harris

By:

H Harrison Tuesday September 30 2008at 15:19:55 UTC

Interesting analysis, however the use of the ‘20% to get a C’ grade is misleading. This was during the old three-tier system. The Higher tier exam required approx. 20% to get a C (with ‘good’ coursework, I may add). This makes sense when you consider that the range of grades available to Higher tier students was A*-C and questions would be taken from this range only. So if you got 20% then you probably got almost all the C-grade questions correct (with maybe the odd mark from elsewhere). The exam hasn’t got easier in my opinion, but league tables, value-added scores, etc have forced teachers to play the system and teach students to successfully answer exam questions, rather than teach them to use mathematical skills.

By:

S Meddon Saturday October 18 2008at 13:50:53 UTC

The “old three tier system” is what my students are presenting this year when they come into university having now taken A levels. Maths skills, although they have passed GCSE maths at the correct grade, remain lamentably low. With the new two tier system I expect that we will be getting students who are gaining the grade but are not covering the syllabus that even the intermediate tier of GCSE offers.

I consider that examinations offer a much truer test of mathematical ability than the coursework previously acceptable at this level of study.

By:

John Colbyon Saturday October 18 2008at 16:53:35 UTC

Material from this post has been cited in:

Harris, H. (2008). Exam Results Explained. In Bulletin of the independent Schools Council, No. 23, November 2008. London: Independent Schools Council. pp. 33-27.

By:

John Colbyon Tuesday November 11 2008at 16:06:13 UTC

I have been searching various sites to find out the cut off grades in the OCR terminal paper in June 2007-Im particularly interested in knowing the marks for grades A and A*-can anybody help me?

By:

RON MORLEYon Tuesday March 10 2009at 21:46:18 UTC

I have been searching various sites to find out the cut off marks in the terminal paper in the OCR Maths exam in 2007-Im particularly interested in marks for A and A*

By:

RON MORLEYon Tuesday March 10 2009at 21:48:11 UTC

An interesting (if unsurprising article), as a Mathematics HoD I am well aware of the grade boundaries for these exams, and the issue stems from the ridiculous idea that a grade C in mathematics represents any serious level of competency. For the last 10 years or so a grade C (on a higher tier paper) has meant that the student has fundamentally got between 2/3 and 3/4 of the paper wrong.

The other problem stems from that fact that you cannot tell whether a student achieved their C grade on the foundation paper (or old intermediate) or higher. On a foundation paper the student needs to show a high level of competency (80%ish) on relatively routine mathematics, but for the Higher they need only show that they are competent in the first 7-8 questions which are of grade C/B standard and not attempt any of the A/A* questions.

I am hesitant to take students with B grade GCSE mathematics on to the A-level course now as they are likely to achieve only D/E grades on average.

These factors you have outlined (and the external interference from government) are some of the many reasons why we have now swapped onto the IGCSE specification (which has not suffered from this variation in grade boundaries – although a C is still about 30% but the A* is at 82%).

The problem is that a grade C at GCSE is now a pre-requisite for many areas, and with more and more students being told they should go to university, students who would have previously not passed GCSE are now passing and moving on to HE were they are ill equipped to cope.

We fundamentally need to accept that whilst every student can improve not every student will be able to reach a level of competency that is necessary for certain areas. For those for whom this is the case we should have the option of offering an alternative qualification with an appropriate set of criteria which can demonstrate competency in a subset of mathematical skills…

For me though the real issue is at the other end. The really gifted mathematicians find no challenge in the current GCSE spec, and so become disenfranchised…

By:

Andy Kon Sunday June 14 2009at 15:16:30 UTC

All my figures were taken from results for higher tier papers.

By:

John Colbyon Sunday June 14 2009at 15:36:28 UTC

Part of the problem is that the grade boundaries changing is also a reflection of the shifting sands of the syllabus. The Maths GCSE is just now going through it’s 3rd change in 3 years, having gone from 3 to 2 tier, lost it’s coursework and now a third revision with greater emphasis on problem solving – all of this with talk of ‘additional maths’ and ‘functional mathematics’ being talked about at the same time…

The problem with the current (and revised curricula) is that there is still far too much emphasis on covering content, and not enough on teaching ‘real mathematics’, students are rarely given time to explore a genuine problem and try out a variety of approaches until they find one that works, instead they are taught a technique and practice it… What the syllabus needs is a serious thinning – loosing much of the data handling work that was moved in from the humanities about 6-7 years ago, and a lesser emphasis on basic numerical skills which can be easily offset to a calculator…

This would give students much more time to learn the truer mathematical skills of logical thought and problem solving which are too often masked given the time constraints of the current curriculum.

By:

Andy Kon Sunday June 14 2009at 18:06:55 UTC

I agree with all you say. I’m seeing the results, unfortunately.

By:

John Colbyon Sunday June 14 2009at 18:11:48 UTC