An evidence-based research approach underpins the entire business at Sparx, a prominent UK EdTech business. It has also inspired them to launch a national movement calling for a ‘step change’ in the quality of EdTech evidence available to schools.
Sparx Maths is an adaptive learning platform for maths education and is aimed directly at the secondary school market (11-16 years). The Sparx Family – comprising Sparx Maths, HegartyMaths and the recently launched, Numerise – currently supports over 1.2 million learners in 1,400 schools around the world. Since the initial idea came to life in 2010, the team has continually experimented and iterated the product, based on extensive collaboration with a network of partner schools, teachers and students. The product’s homework and classroom solution was launched onto the market in 2018.
This focus on testing different hypotheses of how best to support learners and teachers in the classroom lies at the very heart of their approach to product design and development. For Tomasz Stefanski, Sparx’ Head of Data Science, this agile process is “all about coming up with ideas which might solve a problem or improve what’s already being used.” It typically involves unsuccessful approaches being quickly discarded before new prototypes (tech and paper format) are then developed and tested again in the classroom.
One example of this approach saw the team quickly abandon pre-populated schemes of learning in favour of a more flexible approach. This allowed schools to tailor the right solution for their setting by combining their own resources with those provided by Sparx, exam boards and other subject matter experts.
Bursts of research activity usually run in 1-3 local schools who test different ideas with different teachers, with different year groups and with students from all attainment levels. This requires exceptional relationships with local schools, some of whom have even given the Sparx team their own classroom to run their testing from! The same model is applied to testing in international markets, where the UK product serves as a starting point, as the team explore what works and what doesn’t in each geography. As Andrew Blackie, Commercial Director for Sparx, explains: “It’s all about getting as many teachers and students as possible involved, so that different ideas can be tested, adapted and then adopted.”
However, to understand the effectiveness of the widest variety of product features, their approach also includes rigorous analysis of all data collected through the learning platform. Every interaction with the content is carefully studied. For example, the team review what type of students pause and rewind the explanatory videos and at what point do they lose interest and click off? Tomasz Stefanski is clear on the benefits here: “We think our content is high quality , but we can always do better. It’s only through data that we can understand if the scaffolding of our learning is right. Data tells us what needs to be improved and how.”
More broadly, Sparx use the data made available through individual school dashboards to help maths departments and whole schools to identify where they can improve. Similarly, the Account Managers at Sparx use the same data to support their in-school contacts (known as Sparx Leaders), to bring key data trends (e.g homework completion rates) to their attention and to collaboratively find a solution.
But ultimately for Sparx, the most important stage of this evidence-based approach is whether their product actually makes a difference to student learning outcomes. To measure this, Sparx has pioneered the use of student-level randomised control trials (RCTs), running one of the first in UK EdTech back in 2015. Following the double-blind methodology commonly used to test medicines, the company runs multiple trials during the year. The most recent of which explored if homework impacted on attainment (it did!) and then to identify the optimum length of homework each night.
These RCTs typically involve a full year group of students in three partner schools, which is approximately 500 individual students. The trial runs for a minimum of half a term (six weeks) but most usually for a full school term. Stefanski’s team also reviews other data sources to ensure robust research and recently marked 5,000 end of year maths papers to compare and contrast student performance and build further evidence of impact. All of this data and insight is then used to iterate and improve the product once again.
This rigorous, ongoing evaluation of the efficacy of approach and learning content is a key differentiator for Sparx. Louise Mayor, Head of Marketing reiterates the point: “This is what we do and who we are. Our commitment to evidence-based research therefore sits at the heart of all of our messaging and communications activities.” At BETT 2020, this saw Sparx leading the formation of the UK’s EdTech Evidence Group. This body of like-minded businesses is making its impact evidence more widely available to help teachers/schools make better informed buying decisions and to inform government policy makers.
Unsurprisingly, Sparx will be using similar techniques to persuade parents to purchase their new B2C independent maths learning product, Numerise which the company has accelerated to help support home-learning while schools are closed due to coronavirus. The product incorporates Secondary Ready, a platform designed specifically for children in their last year of primary education so they can develop their core number skills required for secondary learning. As Mayor identifies, there’s one big difference with Sparx’s core product: “In school, pupils are told to use Sparx – they don’t have any choice. But at home, we need to motivate learners to want to use this product rather than playing Fortnite”