Basic information on public education

Other
PUBLIC PROFILE

How to monitor the academic performance of a school (2) the data sources

Posted by Magali von Blottnitz on 09 March 2019 12:00 PM SAST
Magali von Blottnitz photo

Arguably, everything we do in the education sector is ultimately for the learners – so that they can learn better. So we need to be able to ascertain if our interventions have the desired effects on learning outcomes - which is a highly complex task.

In a previous blog post, we have explained some general limitations related to academic performance monitoring. In this post, we share here what kind of data sources can be used, what their shortcomings are and what can be done to circumvent these limitations.

The following table shows the data sources that can be used and a word of caution about their usability for performance monitoring. Note that there is no “right” or “wrong” in this domain – everything is a matter of common sense and being mindful of the complexities.

Data source/indicator

Comments and shortcomings

Recommendation

 

Matric results

 

 (for high schools)

This is the most commonly used source of data – it has the advantage of being available nationally, and referring to the same examination which is assessed externally.

Every year in January, the DBE publishes, on its website, a “school performance report” that provides the matric pass rate of every high school in the country.

Limitations:

·        The focus on matric results has led to an imbalance in the practices of high schools, with the majority of effort going to Grade 12’s – often to the detriment of quality learning in lower grades.

·        There are various qualities of matric passes. Details about matric quality (especially bachelor passes) are not so easily available.

·        Policy changes on bachelor passes in 2018 have led to a nationwide increase in bachelor passes in 2018, which does not necessarily reflect an increase in performance;

·        To compare high schools based on their matric performance, you need to ensure that their intake in Grade 8 is of similar quality. This is difficult to establish unless you have access to Grade 8 baseline test results;

·        The recent MEO policies have distorted the matric pass rates, because increasing portions of learners write matric over two sessions and are not accounted for in the January statistics.

 

 

If possible, try to consider the matric results in light of:

·        the number of learners who enter the high school in Grade 8

·        the quality of passes (bearing in mind the policy change on bachelor passes)

·        the results of the June examinations for learners who make use of the Multiple Examination Opportunity.

Also, avoid to focus merely on the performance of Grade 12’s. Quality learning in lower grades is essential for the sustainability of the results.

 

Throughput rate

(for high schools)

There is a frequent view that the “real matric pass rate” must be obtained by dividing the number of matrics to the number of learners who started school.

For a high school, this would be the ratio of number of matrics to size of the Grade 8 cohort 5 years earlier. Since this is difficult to establish, the current year’s Grade 8 or Grade 10 cohort is sometimes used as a proxy.

Limitations:

·        Grade 10 cohorts are often “inflated” by the fact that this is the year with the highest number of repeaters. So changes in the grade repetition policies will affect the matric to grade 10 ratio.

·        Some high schools lack the resources to offer certain subjects at matric level (e.g. pure maths or physical science). This may cause learners to switch schools after Grade 9, for reasons unrelated to the quality of learning at the school.

·        Depending on their intake in Grade 8, some schools may have lower or higher proportions of their learners who will need to be encouraged to choose technical or vocational training after Grade 9 rather than attempting to pass matric and failing. Some high schools do a great job of guiding their less academically-minded learners to alternative learning paths, yet this is not captured by the throughput rate.

 

 

This is a good indicator to use alongside the matric pass rate, but:

·        Choose to compare to Grade 8 rather than Grade 10

If possible, find out what happens to the learners who leave the school between Grade 8 and Grade 12:

·        do they register at other high schools?

·        do they choose alternative learning paths?

·        or do they just “drop out”?

Systemic results

(in the Western Cape)

Since 2010, the Western Cape Education Department (WCED) has administered annual systemic tests in language and mathematics to all learners in Grades 3, 6 and 9. The Western Cape is currently the only province in South Africa where all Grade 3, 6 and 9 learners are assessed routinely, systematically and comprehensively in all ordinary primary and secondary schools.

While these tests have been criticised, they are usually regarded as providing the most robust independent data for the tracking of literacy and numeracy skills in South African schools, especially primary schools.

Systemic results are not made public by the WCED but individual schools are sent their systemic results every year at the end of January. The document provided by WCED includes a comparison of the school’s results with the provincial average and the quintile’s average.

In the current culture of opacity in the education system around school results, don’t be surprised if your school principal is reluctant to share the school’s systemic results with external parties (even their business partners!). Trust-building and creating an understanding that the data will only be used in a supportive way may help to overcome this. Building a relationship with the circuit manager or the relevant district official may also help.

 

·        When comparing systemic results over time, preferably look at the average mark rather than the pass rate. This is because, for a school where the average is near or just under 50%, small fluctuations in the average mark may translate into huge variations in the pass rate.

·        When comparing the school to its quintile, bear in mind that quintiles in the Western Cape are extremely heterogeneous.

Self-reported school performance

Often, self-reported performance is all that is available, especially in primary schools. It has the advantage of being available more frequently than most other indicators, allowing for a term-on-term monitoring.

Schools that are registered on Data Driven Districts can monitor their learners’ results online and access benchmarks at district- and provincial level.

Limitations:

There is extensive research attesting to the poor reliability of internal school data as performance indicator.

“the differences between self-reported school performance and independently moderated school performance. For example, for the Eastern Cape, in 2013 the percentage of grade 3 students with a score of 50 per cent or more was self-reported for numeracy as 54.9 per cent, but adjusted downwards after external verification to 42.2 per cent; for literacy, the self-reported score was 50.2 per cent, and the adjusted score 27.0 per cent.”

 

If possible, try to establish how consistent the testing and marking happens across the period. How much internal and external input is given on the term examinations? (e.g. by the HOD or the curriculum advisor).

 

 

Download the document here.

Don’t hesitate to contact PfP’s M&E unit (magali@symphonia.net or 072 179 2994) to discuss this topic or share the observations and results that you are noticing at your school. 

   

There are no comments

Sign in to add your comment.

Recent Posts

Good for hunger, bad for health? Spaza shops and food vendors proliferate outside schools
By Michelle Banda• 29 March 2021 In March 2020, the Department of Basic Education pulled the...
read more
How does the National School Nutrition Programme work?
The Covid-19 pandemic has worsened food insecurity in under-resourced communities. Schoolfeeding...
read more
Saving electricity to free budget for other purposes
Stellenbosch University researchers recently found that between 3% and 18% of the government...
read more
Inclusive Education and Learners with Special Needs
Inclusive Education recognises the right of ALL children to feel welcomed into a supportive...
read more
What schools that work do right (The video script)
According to 2017 study, on schools that work, high-performing schools do nothing out of the...
read more
How to monitor the academic performance of a school (2) the data sources
Arguably, everything we do in the education sector is ultimately for the learners – so that they can...
read more
How to monitor the academic performance of a school (1) a word of caution
Arguably, everything we do in the education sector is ultimately for the learners – so that they can...
read more
Whole School Evaluation (WSE) and its implementation in the Western Cape
The Whole School Evaluation (WSE) is a national policy promulgated in 2001 – but its implementation...
read more
The Integrated Quality Management System (IQMS)
The Integrated Quality Management System (IQMS) was promulgated in 2003 to integrate the existing...
read more
How are school principals appointed?
Provincial education departments determine their own set of rules and procedures for the appointment...
read more

Go to blog