Before We Buy Anything
by Shawn Rubin
There's a feeling I've chased but rarely experienced since I left the classroom sixteen years ago. Being in a room full of people who genuinely believe schools and systems can be better for kids, and are doing something about it. The New Schools Summit I attended last week had this feeling.
I also experienced something else. Live demos, founder pitches, and a quiet underlying implication that the path from here to better schools begins with purchasing products.
That's what made me nervous. It wasn't the hope or confidence that speakers and panelists brought to the stage. Their ambition felt real and much of it was earned, backed by research. But I've watched and lived this experience long enough to know that excitement about what's possible has a way of skipping past the harder questions. And I know where that shortcut usually leads.
Most students aren't in innovation labs or pilot programs. Roughly three quarters of American K-12 students attend traditional district-run public schools. Many of them are Black and Brown children, children from low-income families, children whose schools have absorbed decades of well-intentioned reform without seeing the outcomes those reforms promised. We cannot ask these students to pay the price for another ten years of adult trial and error.
To be clear, this piece is not another call for banning anything, but rather a call for intentional and thoughtful decision sequencing in districts.
What We Got Wrong with Blended and Personalized Learning
In the blended and personalized learning era in the 2010s, we brought unproven tools into schools with the confidence of people who believed that possibility was the same thing as evidence. We handed teachers digitized textbooks that were barely an upgrade from print. We reached for open source activities like they were equivalent to high quality curriculum, ignoring everything we already knew about cognitive routines, rigor, and what actually nurtures independent learners. Surface-level engagement became our currency. If a student was clicking, they were learning. If a tool had high usage numbers, it was working. We saw the signals of apathy and compliance, and we focused on the bright spots.
I say "we" on purpose. This wasn't somebody else's failure. Throughline Learning, then Highlander Institute, was in this space. We called ourselves platform agnostic, but that distinction mattered less than we let on. We happily recommended free and freemium tools alongside paid products, and we helped schools move from pilots to scale faster and with less proven evidence than we should have. Whether the tool had a price tag or not, we were more focused on getting it into classrooms than on interrogating whether it belonged there.
Here is a specific example of what I mean:
I can remember walking into an early elementary classroom in 2018 and being greeted by a teacher who was so proud to show off the work we'd done together. She had set everything up exactly as we told her to. Computers lined the wall so she could see the screens and monitor students from afar. Her table and materials were prepped, and she was wearing a “magic fairy hat” as a reminder for students not to interrupt her small group instruction time. Everything looked like progress.
But when we actually watched, the students at the computers were mostly guessing. When they got coins for correct answers, they took those coins to the virtual store built into the platform and spent the majority of their time there, not working with the math manipulatives the lesson was designed around. Meanwhile, on the carpet, the paraprofessional, who was so thoughtful and caring, was juggling a runny nose and rigor, because both seemed equally important in keeping the classroom humming smoothly. The teacher in her small group was over-scaffolding because one student wasn't yet at grade level, and she wanted that student to experience success.
What I'm describing was not a failure of heart. These three centers were the machinery we had created and the model we believed in at the time.
And here is the part that took me years to name clearly. The platform was our focus more than the students or the learning. The district had already purchased the edtech product before we arrived so our job as the implementation partner was to make it work. Nobody asked us whether it should work, or whether this product delivered through a rotation model was what students actually needed at the time. The district had already bought it. So we built the implementation around the product instead of building the product around the implementation.
That is the mistake I'm most afraid we're on the verge of repeating with artificial intelligence (AI).
Foundation Before Features
What would it look like to approach AI differently? Not with less curiosity, but with a different decision-making sequence?
Before we layer in new tools and features, we have to reckon with the conditions most districts are actually operating in. Many are already dealing with students and teachers using AI on personal devices, workarounds that technology directors can't fully see or control. In that context, moving fast to restrict the tools that pose the clearest risks to student data and academic integrity isn't obstruction. It's triage. Multiple superintendents I've spoken with have compared the current AI landscape to the Wild West, and they're not wrong. We just watched a significant data breach come out of Canvas, and no superintendent wants to be the next national headline. Establishing clear boundaries quickly is a necessary first step. The question is what comes after you draw those lines, and that's where sequencing matters most. District guidance needs to actually exist before we introduce new tools, so that administrators, teachers, and students aren't left adrift, figuring things out for themselves.
And then it means piloting intentionally, in small enough cycles that we can actually learn from them before committing to anything at scale. The districts getting this right are running what look more like continuous improvement cycles than the traditional product rollout. They're testing in bite-sized chunks. They're building in measures from day one, giving them the data to pivot quickly if the expected impact doesn't materialize.
What AI actually makes possible that blended learning couldn't deliver is the opportunity for co-design, tool building, iteration, and localized improvement based on feedback and observation. Large language models built within closed systems that don't feed data back to the model allow districts to align strategy and create from the instructional vision outward, rather than starting with the tool and retrofitting the vision around it. That's a fundamentally different sequence that I believe will lead to better classroom outputs and outcomes.
This is also a moment for product designers and edtech companies to do something different. If you're building AI tools for schools, the question worth sitting with is whether your product is rooted in evidence from the start, not retrofitted with research claims after the fact. The Multilingual by Design initiative offers one model for what this can look like. Every tool being built in that work is grounded in the instructional frameworks and expertise of the country's strongest multilingual learner organizations and leaders, with the vision driving the product rather than the other way around. We're proud to support that work as a technical assistance and facilitation partner, and the sequence it models is available to anyone willing to choose it.
Unfortunately, as I work with more and more educators around AI literacy and implementation, I'm noticing that the ones most excited about AI are also the ones clamoring for new AI products to use in their classrooms on Monday morning. The pull of edtech tools is real and understandable, but the next wave of educators watching our early pilots with well-intentioned early adopters will notice if the emperor has no clothes, and the blowback from that disillusionment is exactly what causes leaders to clamp down on the creativity and agency that students and families are asking for and that AI can ultimately provide if we roll it out right.
Our kids only get one shot at third grade. If we’re going to experiment with it, we need more than just a tool; we need a clear definition of success and the courage to change course the second we don't see it.
What We Got Right and Where We're Headed
The blended and personalized learning era wasn't a complete failure. We got educators excited about what was possible for students. We centered student experience in conversations where it hadn't previously been elevated. We opened leadership pathways for teachers and coaches in ways the system hadn't made room for before. That all matters.
AI is creating the same opening. The question is whether we'll use it differently this time.
I'm not advocating for caution for its own sake. We already have enough voices calling for outright bans. I'm advocating for curiosity channeled through a lens of evidence-based problem solving. Let's explore what AI can do and get educators genuinely enthusiastic about the possibilities. But let's also resist the pull to make the first move a purchasing decision. Instead, let's make it a learning decision. Let's start by growing the internal district capacity to understand AI, evaluate its merits and monitor its impact. Then we will be able to pilot with clear eyes and clear measures, on behalf of students who deserve better than another decade of well-intentioned harm.
That feeling of energy and possibility at the New Schools Summit was inspiring, but belief alone isn’t a strategy. This time, let’s get the sequence right. Before the next pitch or purchase order, we must ensure the question in the room isn't ‘What does this tool do?’, but 'What do our students actually need, and are we willing to bet their third-grade year that we’re right?' That is the only question that has ever mattered. This time, let’s start there.
The ideas, the kindergarten classroom story, the core argument, and the voice in this piece are the author's own. Claude (Anthropic) assisted in the drafting process.
Learn More
At Throughline Learning, we know that effective instruction will always be about human interaction. We help districts establish thoughtful AI strategy that boosts your educators’ capacity, confidence, and creativity in this new era of possibilities. Explore our Fuse AI Partnerships Guide to learn more about our AI task force facilitation and professional learning offerings.
About the Author
Shawn Rubin is the Executive Director of Throughline Learning (formerly Highlander Institute) in Providence, RI. He is an internationally recognized thought leader and speaker with deep expertise in classroom coaching and change management strategies for building and district leaders. In 2018 Shawn co-authored the book Pathways to Personalization: A Framework for School Change, which details Throughline Learning's community-driven school improvement approach. Shawn prioritizes supporting at least one of Throughline’s portfolio schools every year to build connectivity between the organization’s classroom implementations and national field building efforts. He has won several innovation awards for his work in the nonprofit sector, including iNACOL's annual award for Outstanding Individual Contribution to Personalized Learning.