Assessment

The Power of Public Education Data

Why Massachusetts makes its education data open to all

Date24/10/21

AuthorTim Ewington

This afternoon, I decided to learn all I could about Charlestown High School.

Located 3,300 miles across the Atlantic at 240 Medford Street, Boston, Massachusetts, this public (state) school educates around 900 students from across the city.

I have never visited Charlestown High and probably never will, choosing the school at random as a test: how much could I find out from my laptop in a self-imposed time limit of just one hour?

Of course, schools are complex organisms. There is a limit to what one can learn in this way. I wanted to understand just how far this approach could take me.

Why a Massachusetts school?

The state of Massachusetts believes in collecting detailed educational data to focus its resources to maximise impact. The State also believes in making this data openly accessible.

Each year, students in 8 different years are tested and the State makes this data, together with insight into the students themselves, available to all - in anonymised form - so they can do their own analysis and form their own opinions.

Parents, teachers, policy wonks, educationalists, students, statisticians, Head Teachers and the AQi are all free to data roam.

Charlestown High School, Massachusetts

So what did I find out about Charlestown High School?

Located in a relatively deprived, urban area, Charlestown High faces challenges but the school is doing better than others with a similar profile. It had a good run in the middle of the decade but its performance has plateaued and in some areas the figures have slipped back in recent years. The school is receiving targeted State support.

The school’s students are mostly Hispanic (60%) or African-American (27%).

The students have high levels of need: 63% of students are from economically disadvantaged backgrounds (70% above the State average) and nearly half speak English as a second language, twice the average State level.

More data drills deeper into the needs of students, including levels of disability and basic English learning. All are relatively high at this school.

Do the students succeed?

The dropout rate at Charleston fell considerably during the last decade but has started to rise again in recent years, reaching 9%. Grade retention levels have also been gently rising.

The school struggles with English Language Arts performance with lower achievement levels. The levels have not improved but on the other hand the basic English learning levels have risen, so the educational task has got harder.

Charleston High also has lots to be positive about.

58% of Charleston High students have successfully graduated for the last 2 years. That level was only 42% in 2010. Maths and science achievement levels are significantly higher than similar schools and figures are rising.

The school is good at student growth. Students start with challenges but make significantly more progress than in many other schools, particularly in maths.

We could go into far more detail, picking away the layers, but I think you get the idea. There is 20 years of data in some areas and the committed observer with a penchant for excel could spend many happy hours with big spreadsheets.

I had just 60 minutes.

The Massachusetts Comprehensive Assessment System

What is the point of collecting all this data?

In the mid-1990s Massachusetts State undertook a series of reforms in a drive to improve the quality of public education, which was recognised as fragmented and sub-standard.

Clear curriculum frameworks were introduced, there were significant real terms increases in state funding and specialist Charter schools were commissioned.

But at the heart of the reform project was the MCAS – the Massachusetts Comprehensive Assessment System – an annual testing programme that covers 8 of the 12 years of school. Reaching the necessary level in these exams is essential for graduation.

The MCAS approach is interesting for several reasons:

Design: Developed over a 5 year period by leading educationalists, mathematicians and academics from top universities, the MCAS is respected and deeply embedded in the system.
Comparability over time: Testing structure and form have been preserved over the decades to allow long-term trend analysis.
Data granularity: There is considerable granularity, particularly about the background and progress of individual students, enabling detailed analysis and the identification of potential causations.
Transparency: The data is available to all and analysed by a wide range of stakeholders including educational academics, policy makers, the State Department of Education, schools, teachers and parents. Nothing is hidden. The debate is open.

MCAS Assessment Timetable

Assessment, data and analysis are often seen as threatening concepts in the UK education system. They are an opportunity for government to hold schools to account on their performance and check that targets are met. The sword of Damocles of special measures hangs over those schools who do not perform against numerical benchmarks.

But this is not how the MCAS is perceived in Massachusetts.

Since the data is public, there are multiple analyses rather than the analytical keys being held by government.

Issues are identified and discussed by different organisations ranging from the Massachusetts Department of Education to Harvard and Boston universities and the schools themselves. Most schools use Edwin, a system specifically designed for schools to interpret the data, which helps them understand the data efficiently. Schools actively use the data as a tool to support the development of their strategies.

MCAS data is recognised as an important part of the school improvement programme, deeply embedded in the system after more than two decades. The benefits seem clear and the role of data is accepted.

In 2015, the PISA (the Programme for International Standards Assessment, run the OECD) study included analysis of 15-year-old students in Massachusetts as part of its global survey.

The results were impressive. In reading, maths and science, Massachusetts students significantly outperformed both the OECD average and the score for the United States. In both reading and science, Massachusetts scored at a level similar to the highest performing nations such as Singapore, Canada, Finland and Japan.

PISA Rankings: Massachusetts, the USA and the UK, 2015

Using data to guide school strategies

So what happens when a school’s MCAS results go bad?

The data is not used to impose special measures or to intervene aggressively.

The purpose of MCAS data is to define challenges and focus resources more effectively to address them.

Schools and districts that face challenges or poor or weakening performance can use the data to help understand the issues and are encouraged to bid for new investment to help address them.

That investment can come in many forms: adding new teachers, new training programmes, additional teaching resources or new facilities or buildings, for example.

Human support is also given to the management teams of under-performing schools.

The teams brought in to support the existing management teams tend to be former heads, teachers and academics. They take shared responsibility in finding solutions. The engagements can last 6-12 months, providing sustained, experienced support to the school.

The resulting strategies are often multi-year in duration. Progress is expected through improvements in MCAS results but it is recognised that this can take time.

What can we learn from Massachusetts?

Data can be your friend rather than your enemy. Well-thought-through data helps everybody to understand what is going on.

It is how the data is used that is crucial.

When data is made open and the approach to improvement and problem solving becomes collective, the long-term results can be very powerful indeed.

Assessment

Assessing oracy: Is Comparative Judgement the answer?

Oracy skills are vital to success in school and life. And yet, for many children, opportunities to develop them are missed. Educationalists are engaging in a growing debate about where oracy fits into the school system. Labour has put it at the heart of its plans to improve social mobility and an independent commission is looking at how it is taught in the classroom. This renewed focus on oracy means it is more important than ever that teachers have a way to reliably assess and understand their students’ attainment and progression. Amanda Moorghen of oracy education charity Voice 21 explains how Comparative Judgement can help with that and why it may be a game changer.

Education

TV subtitles as an aid to literacy: What does the research say?

Jack Black is probably best known in educational circles for playing a renegade substitute teacher in School of Rock. But the Hollywood star has made a more conventional foray into education by backing the use of TV subtitles to improve child literacy. Stephen Fry and the World Literacy Foundation also want parents to use their TV remotes to get children reading. So, could this simple click of a button be a solution to boost pupils’ reading skills? AQA’s resident expert on language teaching, Dr Katy Finch, casts her eye over the research to see whether it stacks up.

Data Analysis

What is left behind now education’s Data Wave has receded?

Is data the solution to all education’s issues? About a decade ago the prevailing wisdom said it was. Advocates of this Data Wave argued that harvesting internal statistics would help schools solve issues such as teacher accountability and attainment gaps. As with all waves, after crashing onto the beach they recede, leaving space for another to roll in. In this blog, teacher, author and data analyst Richard Selfridge looks at the legacy of the Data Wave to see what schools can take from it.

International Approaches

Finland & PISA – A fall from grace but still a high performer?

Finland was once recognised as one of the most successful educational systems in the world. At the turn of the millennium, it topped the PISA rankings in reading, maths and science. But by 2012, decline set in. The last set of results showed performances in maths, reading and science were at an all-time low. In this blog Dr Jonathan Doherty of Leeds Trinity University outlines some reasons that may account for the slide.

Briefing

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

According to the latest PISA results, England’s science scores are still on a downward trajectory that started a decade ago. Yet TIMSS, another respected study, has science performances rising. Which of them is right? Is one more valid than the other? In this blog AQi examines three International Large-Scale Assessments and finds that, although they may look the same from a distance, get up close and you’ll find they are very different beasts.

Adaptive Assessment

Adaptive Assessment: A missing ingredient in the resit recipe?

The number of students resitting their maths GCSE is growing, but the proportion getting a grade 4 or higher is falling. This situation is not only dispiriting for the young people striving to get the qualifications they need, but also for the teachers working hard to help them. How can outcomes for this cohort be improved? Bart Crisp, associate director at the Centre for the Use of Research and Evidence in Education, thinks adaptive assessment may be part of the solution.

SEND

Student success: Every milestone matters

Baroness Morgan is calling for students to be given ‘self-belief’ lessons as a way of developing their characters and preparing them for the future. She is not the first to notice that a student’s sense of their own ability and their level of success are part of a virtuous circle. But how can teachers get the snowball rolling for students with SEND or in alternative provision? In this blog, former headteacher, John Tomsett, pulls out a swimming certificate he earned more than half a century ago to use as an inspiration for others.

Assessment

Progress 8 – How much can it flex?

Progress 8 has come in for criticism in recent months for the way it combines with the Ebacc to skew schools’ focus towards academic subjects at the expense of creative and technical options. Both Labour and a Lords committee have called for reform to boost uptake of declining subjects such as D&T, dance and drama. In the second of two blogs looking at Progress 8, AQi examines whether the accountability measure could be realigned to address these concerns and if so, how could that be done?

Assessment

Progress 8 – Schools’ flexible friend?

Eight years after Progress 8 came into existence it is facing calls to be updated. Some say reforming the schools accountability measure can halt the decline in take up of the arts, while others want it to boost numbers taking vocational and technical subjects. Are its shoulders broad enough to take on such a load? In the first of two blogs on Progress 8, AQi looks at how it came into being, what problems it was designed to solve and whether it is flexible enough to be reformed.

Assessment

2023: A year in review

As we wrap up a year full of exciting developments in education, it is the perfect time to take stock of what has gone on. In this blog, editor of AQi, Anil Dawar, reflects on some of his highlights for 2023.

Download a PDF version.

Download a copy of this content to your device as a PDF file. We generate PDF versions for the convenience of offline reading, but we recommend sharing this link if you'd like to send it to someone else.

The Power of Public Education Data

This afternoon, I decided to learn all I could about Charlestown High School.

The Massachusetts Comprehensive Assessment System

Using data to guide school strategies

Assessing oracy: Is Comparative Judgement the answer?

TV subtitles as an aid to literacy: What does the research say?

What is left behind now education’s Data Wave has receded?

Finland & PISA – A fall from grace but still a high performer?

PISA, TIMSS and PIRLS: What actually are they and what do they tell us?

Adaptive Assessment: A missing ingredient in the resit recipe?

Student success: Every milestone matters

Progress 8 – How much can it flex?

Progress 8 – Schools’ flexible friend?

2023: A year in review

Join the conversation on Twitter

Download a PDF version.

Sign up to our newsletter