Credit to the late, great Theodore Micerri for being the one to point this out to me years ago. We love you Ted. https://t.co/M7n59MFdAZ

If we care about the science of learning and evidence-based practice, we need frameworks that evaluate actual instructional quality. We cannot rely on rigid algorithms that punish educators for the statistical noise of human development. Let's evaluate teaching, not variance.

Worse, because VAMs can't control for non-random classroom assignments, educators who willingly take on the most complex, high-needs cohorts are often penalized by the algorithm. It creates a perverse incentive that drives great teachers away from the kids who need them most.

So when a student inevitably deviates from that rigid demographic average, the VAM assumes its mathematical prediction was flawless. It takes the unpredictable variance of being a human child and literally rebrands that statistical error as "teacher effectiveness."

When a VAM predicts a student will grow exactly 5 points, it ignores all the invisible, unmeasured variables in that child's life. A bad night's sleep, a sudden spark of intrinsic motivation, or a shift in family dynamics—none of that is captured in the algorithm.

Sounds objective, right? It’s actually built on a massive statistical error known as the Ecological Fallacy. This is the fatal flaw of assuming that an individual student will behave exactly like the aggregate average of a demographic group they belong to.

VAMs try to calculate a teacher’s "value" by predicting a student's test score based on the historical average of kids with similar demographics and past scores. If the student beats the prediction, the teacher gets credit. If they fall short, the teacher is penalized.

Today I learned that several states are still using Value Added Models to evaluate educators. While I'm thankful California doesn't, here's why VAM is fundamentally flawed: 🧵

Absolutely. https://t.co/WqK3zvojSP

So much of what we call reading comprehension failure is basically disguised vocabulary failure. Struggling kids don't need to be taught to "find the main idea", they need to be taught more words. And explicitly. Reading Marzano's book on vocab and I'm fascinated by this table.

This is clearly disturbing. However, there is a middle way between the "Ban all screens" and "Personalized learning for all kids, 1:1 laptop at all time" crowd. AI tutoring shows a lot for promise, and so does some other EdTech. The question is how to navigate that middle way.

NEW BLOG POST Explicit instruction vs. corrective feedback Continue reading: https://t.co/cE97wGVdZu

The recent UC San Diego report on the lack of math readiness in incoming freshmen is the canary in the coal mine. Until we return to evidence-based principles in math instruction, this problem is likely to get worse.

The new California Math Framework isn't the beginning of something new in math. It's the culmination of a trend that's been building for the last decade in Math Ed. It's a return to the same failed ideas of the 1989 NCTM Math Standards, and the 1960's "New Math".

I fear that the EdTech sector is looking to transform schools into Vulcan pod schools. It's a project doomed to fail. While AI tutoring holds much promise to transform education, we will need a new model for schools that also honors our human need for belonging and meaning.

In Star Trek, Vulcans have always represented what humans could become if we grappled sufficiently with the darker sides of our nature, but they also have been a plot tool to explore what could happen if we become too technical and cold to our humanity.

Meanwhile, the promise of individualized mastery learning/tutoring for every student is a real one. And when students engage with it, it shows big effect sizes. Still, studies have shown that the impressive effect sizes that come with tutoring diminish when scaled.

There's something to the fact that this scene takes place on an alien planet. It's a commentary on the inhumanity of the model. Social Learning Theory tells us that most humans need that social component to education. Most kids won't spontaneously feel a need to learn Algebra.

I've thought about that brief scene a lot in the last decade and half since the movie came out, especially recently. The power of sci-fi is that it can be a powerful commentary on society and humanity. This scene is both a dire warning and hopeful vision.

In JJ Abrams' 2009 Star Trek Reboot, there is a scene that has always stuck with me. It shows a Vulcan school, with students isolated in pods being instructed by ai tutors, while adult supervisors/teachers pace above and supervise.