In 2011, the movie Moneyball brought mainstream attention to the concept of sabermetrics, originally defined by Bill James as “the search for objective knowledge about baseball.”
Statistical analysis has long been a driving force behind decision making in the sport, with executives and writers deferring to a “numbers don’t lie” attitude when managing talent and comparing the accomplishments of the game’s brightest stars.
This kind of empirical analysis governs our lives in many ways we don’t even consider. Big data is driving the business world and (as a result?) the education system is not far behind. Instead of pushing back against the data movement in schools, many educators are working overtime to figure out how they can use this data to improve the learning experience for their students.
The first step in the process is to acknowledge that educational data is years behind other industries in terms of practical application. Sure, we track test scores, discipline infractions, and attendance, but does that really tell the whole story about how effectively we are preparing students for college and a career? We think not. Recent trends, such as flexible learning environments, PBIS, and a de-emphasis on grades show tremendous promise, but we've only just begun to scratch the surface.
Any casual baseball fan can visit a glut of websites to easily find the splits, trends, and relative production of their favorite team’s backup shortstop, yet it can be hard for educators to identify even the basic information needed to take action and drive results in our schools. Priorities, anyone?
Educational data is used as a source for funding decisions, school report cards, and legislative action, but only the most analytical-minded leaders are finding ways to apply this data where it's needed most. Teaching and leading both require a combination of art and science. Striking the right balance can result in positive, lasting change.
Learning from the RBI
For the first 100+ years of baseball’s existence, the run-batted-in (RBI) was a sacred cow in the baseball world, right there next to batting average and home runs in the pantheon of baseball card statistics. Triple digit RBI numbers were a sign of stardom, and front office personnel everywhere shelled out big bucks to attract top “run producers” to hit in the middle of their lineups.
Then, everything changed. With the boom of sabermetrics in the late 20th century, baseball statheads began to realize that the RBI was really more a product of luck, environment, and opportunity than a defining attribute of a hitter. Dozens of new metrics emerged that painted a more holistic picture of a ballplayer’s impact on his team and rendered the RBI irrelevant, except, of course, among baseball purists who frowned on these new, seemingly complex analysis systems. Sound familiar?
Today’s educational data landscape closely mirrors that of pre-sabermetric Major League Baseball. Swap "assessment performance" for "RBI" and "student" for "hitter" in the above paragraphs and see what happens. We have more tools than ever before to measure our students’ progress and identify any number of variables that might be contributing to their success (or their struggles), yet we still lean heavily on traditional, stand-alone testing data that paints a very limited picture of student abilities. Standardized test scores are the RBI of the K-12 system, ripe for exploitation and lacking in context. This single data point, when placed on an island, offers little value to the pursuit of improving outcomes.
Last year, a highly touted group of next generation tests were rolled out to school districts across the country. The jury's still out on whether or not future iterations of these tests will give us “better and more useful information on how we’re preparing our kids for their futures.” There is no doubt, however, that improving the test is a big step toward improving the quality of the resulting data.
Applying Sabermetric Concepts to the Classroom
There are still some who decry the use of educational data and question its efficacy. At the same time, many school leaders are looking closely at a number of variables every day to make small changes that have big impact. Knowledge of something as simple as a correlation between a certain student’s lunch period and performance can prove invaluable. The same holds true for class size and discipline, among many others.
Let’s look at some hypothetical examples of advanced educational metrics and how they might be used to identify trends and improve student outcomes:
AIV (Assessment Input Variance)
Compare student test/assignment scores against input methods (electronic, paper, oral presentation, etc.). This will require some A/B testing in a low-stakes environment, but the results should help paint a picture of digital literacy levels in your district so you can determine whether it is the content of the assessment itself or the delivery method that is impacting your students' performance.
Ex: “Jefferson Elementary’s electronic AIV over the past two years is -.36. This number indicates that electronic test score results in this school are not indicative of conceptual knowledge, but are rather a result of relatively low digital literacy rates. We are planning a phased 1:1 rollout and technology coaching program to help close the gap.”
TLO (Teacher Leader Output)
Analyze teacher leader impact based on professional development and peer improvement measures. If you are an administrator, odds are you rely heavily on your strongest teachers to share their instructional tips with less experienced staff. Why not add objective measurements to aid in teacher evaluations and professional development?
Ex: “Jason’s 2.34 TLO is indicative of the impact he has had on the entire English department. We need to get him more involved in the professional development process district-wide. Let's see if he would be willing to host a workshop at a neighboring school next year.”
SVP (Student Voice Profile)
Analyze class participation, extracurricular involvement, and student portal activities in order to easily identify your least engaged and/or introverted students. Tailor your instructional approach to include additional one-on-one coaching and strategic lesson plans designed to help these students become more comfortable and raise their level of achievement.
This approach is already happening in classrooms today thanks to teacher awareness and intervention, but objective measurements can reduce the possibility of observational bias and ensure these methods are being applied universally.
Ex: “Dolores has a relatively low SVP and her grades appear to be suffering as a result. I will set aside class time to work with her individually and in small groups. I will also make a point to encourage her to participate in our online class discussion board."
ADE (Average Daily Engagement)
Review attendance, performance, and involvement by course section and day of the week to determine low points. Plan lessons accordingly to ensure maximum impact.
Ex: “My 4th period Civics class showed a low Wednesday ADE in 2013. This year, I’m going to make a conscious effort to add a midweek spike to the lesson plan in the form of a group activity or classroom debate.”
What's the Point of this Exercise?
Large volumes of data and complex analyses can be overwhelming, which is why it is important for individuals in every role and every unique situation to narrow their focus to the metrics that matter to them. This makes strategic data planning at all levels especially important.
Data collection for its own sake has little value. The two Ts of data use – training and transparency – can make or break your educational data practices. The examples we used above were arbitrary and the only factor they had in common was the inclusion of more than one variable as opposed to test scores on an island.
The point is, teachers, administrators, and central office personnel will all find different metrics that help them make decisions to improve student outcomes. School districts are already collecting most or all of the data required for these exercises, but few district leaders have the training or technology needed to unlock its potential. The policies and practices for data use in schools are still trying to catch up to the technology, and we haven't even discussed the far more complex world of individual learning analytics pulled from gamification and adaptive learning platforms.
As our understanding of effective educational data practices grows, it seems ever more likely that we will need to take a lesson from Major League Baseball's RBI and phase out our emphasis on stand-alone assessment as the foundation of decision-making processes.
Schools, populations, and individual students are subject to an abundance of variables that have a measurable impact on “achievement” metrics. Isn't it time we started looking for a more holistic data approach so we can take advantage of the opportunity to pursue incremental improvement without losing sight of the big picture?
For more on strategic planning with personalized analytics, click here or contact us today to find out how Skyward can help you find the information you need to make better informed decisions.
Check out these recent articles!