Results show significant growth in test scores for ST Math users in 16 states
(Irvine, CA) March 21, 2018 -- The largest study of its kind to evaluate an education technology math program nationally, including over 150,000 students between 2013 and 2016, was released today by independent education research firm WestEd. Researchers found that schools consistently using ST Math could boost their state test scores enough to, on average, jump by as much as 14 percentile points in their statewide math ranking.
Created by MIND Research Institute, ST Math is a PreK-8 visual instructional math program used by more than 1.2 million students across 47 states.
Created by MIND Research Institute, ST Math is a PreK-8 visual instructional math program used by more than 1.2 million students across 47 states.
The study began with 474 schools that started using ST Math between 2013 and 2015 in 16 states where complete state standardized test and demographic data was publicly available to the researchers. Assessments included multi-state tests (Smarter Balanced Assessment Consortium and Partnership for Assessment of Readiness for College and Careers) as well as 11 unique state tests, such as the State of Texas Assessments of Academic Readiness, Florida State Assessment, and New York State Testing Program.
Researchers relied on an innovative method in order to analyze math scores across different states with different assessments. They normalized school performance across state lines by evaluating how a school ranked on their standardized math test performance within their state before and after adopting ST Math.
“I’m encouraged to see that ST Math continues to show significant effects on math achievement,” said Tonika Cheek Clayton, managing partner at New Schools Venture Fund. “But I’m even more impressed with the breadth of ST Math evaluations looking at every school, every year, which provide a model for other education technology companies to follow.”
WestEd researchers looked at schools in two groups: those that used any ST Math content at all in 2015-16, and a subset of those schools that used the program consistently. Each group was matched to schools that did not use ST Math at all, but were similar in demographics, size and starting test scores. Both of these groups of ST Math users showed growth on their state tests that significantly outpaced their peers. Notably, those who implemented the program consistently (at least 85% of their students used the program and on average covered 50% or more of their grade-level content during the year), experienced the greatest gains.
The group that used ST Math consistently spanned 14 states, 239 schools and 11 different state assessments. The study found ST Math’s “effect size,” a statistical measure of how large a program’s impact is on a specific outcome (such as test scores), to be significant among the consistent users. When looking at student proficiency rates on state math tests, ST Math had an average effect size of 0.36 on statewide ranking (z-score). As a reference point, the federal What Works Clearinghouse defines 0.25 effect size and above as “substantively important.”
“With WestEd’s experts validating this innovative approach of normalizing state test performance across different assessments, we’ve overcome the biggest hurdle that prevents large-scale studies for educational technology programs,” said Andrew Coulson, Chief Data Science Officer of MIND Research Institute, which created ST Math. “The power of a large number of sites from across the nation makes the results highly significant. Every education program with gradewide adoption can perform rigorous studies like this annually and release their data publicly. The health of the edtech market and the need for continuous improvement of instructional tools demands it.”
This national-level study expands upon previous WestEd independent validation of the effectiveness of ST Math program at the district level (Los Angeles Unified School District 2013) and statewide (California 2014). Since 2009, MIND Research Institute has completed and published over 50 similar state or districtwide evaluations of ST Math’s impact.
About MIND Research Institute
MIND Research Institute is a neuroscience and education social impact organization, dedicated to ensuring that all students are mathematically equipped to solve the world’s most challenging problems. MIND is the creator of the ST Math visual instructional program, which incorporates the latest in learning and the brain, promoting mastery-based learning and mathematical understanding. Longitudinal and broad-based studies across diverse population groups continue to demonstrate ST Math’s efficacy in building lifelong learners prepared for success in STEM fields. ST Math currently reaches more than 1.2 million students and 54,000 teachers at 3,900 schools in 47 states. For more information, visit stmath.com.
Researchers relied on an innovative method in order to analyze math scores across different states with different assessments. They normalized school performance across state lines by evaluating how a school ranked on their standardized math test performance within their state before and after adopting ST Math.
“I’m encouraged to see that ST Math continues to show significant effects on math achievement,” said Tonika Cheek Clayton, managing partner at New Schools Venture Fund. “But I’m even more impressed with the breadth of ST Math evaluations looking at every school, every year, which provide a model for other education technology companies to follow.”
WestEd researchers looked at schools in two groups: those that used any ST Math content at all in 2015-16, and a subset of those schools that used the program consistently. Each group was matched to schools that did not use ST Math at all, but were similar in demographics, size and starting test scores. Both of these groups of ST Math users showed growth on their state tests that significantly outpaced their peers. Notably, those who implemented the program consistently (at least 85% of their students used the program and on average covered 50% or more of their grade-level content during the year), experienced the greatest gains.
The group that used ST Math consistently spanned 14 states, 239 schools and 11 different state assessments. The study found ST Math’s “effect size,” a statistical measure of how large a program’s impact is on a specific outcome (such as test scores), to be significant among the consistent users. When looking at student proficiency rates on state math tests, ST Math had an average effect size of 0.36 on statewide ranking (z-score). As a reference point, the federal What Works Clearinghouse defines 0.25 effect size and above as “substantively important.”
“With WestEd’s experts validating this innovative approach of normalizing state test performance across different assessments, we’ve overcome the biggest hurdle that prevents large-scale studies for educational technology programs,” said Andrew Coulson, Chief Data Science Officer of MIND Research Institute, which created ST Math. “The power of a large number of sites from across the nation makes the results highly significant. Every education program with gradewide adoption can perform rigorous studies like this annually and release their data publicly. The health of the edtech market and the need for continuous improvement of instructional tools demands it.”
This national-level study expands upon previous WestEd independent validation of the effectiveness of ST Math program at the district level (Los Angeles Unified School District 2013) and statewide (California 2014). Since 2009, MIND Research Institute has completed and published over 50 similar state or districtwide evaluations of ST Math’s impact.
About MIND Research Institute
MIND Research Institute is a neuroscience and education social impact organization, dedicated to ensuring that all students are mathematically equipped to solve the world’s most challenging problems. MIND is the creator of the ST Math visual instructional program, which incorporates the latest in learning and the brain, promoting mastery-based learning and mathematical understanding. Longitudinal and broad-based studies across diverse population groups continue to demonstrate ST Math’s efficacy in building lifelong learners prepared for success in STEM fields. ST Math currently reaches more than 1.2 million students and 54,000 teachers at 3,900 schools in 47 states. For more information, visit stmath.com.
