I’d want to see extensive research on the benefits of Ed-Tech products before they are unleashed on classrooms full of actual students.
“Making things easier for teachers” is often not the case and simultaneously creates costs for training and support at these institutions along with the exponential growth required to administer these new roles.
And then of course are the students themselves. How do iPads compare to textbooks? Do we trust LLMs to grade essays? How effective has multiple choice testing ever been?
There is surprisingly little thought put into bringing tech into the classroom. Most of the drive seems to come from a fear of being left behind, as if learning to use a consumer electronic device is somehow synonymous with learning how to maximize productivity with a computing device.
Yah. I am a teacher and I'm deeply skeptical of ed-tech. There is no substitute for doing the work (as the teacher or as the student).
I like that I have an automated gradebook and communications tool. I like that I can e-mail students and accept work done outside the room via Google Docs. And it's pretty great to be able to present slides from me or students, or occasionally look at a Desmos or Jupyter notebook. Khan is neat to get an instant read or to allow students to hear a topic explained a different way than I do. Playing a game in Gimkit is a way to break up the monotony of a class. I privately advise students they might want to consider Anki as a way to increase their long term retention of information.
Even though I've named a lot of technology, these things are about 10% of what we do. Even when I'm teaching about technology, a huge portion we do is pen, paper, careful thought, and discussion. I know students aren't trying to game; I know they've produced their own work; I know they've not become over-reliant on a crutch. And beyond this, the fundamental reason why a classroom works is not some assumption of my authority or my use of grades for leverage: it works based on the social connections formed in a classroom between the teacher and students, and between students with each other. Too much time on screens interrupts this.
Of course, I'd love high quality research to help guide me in making better choices in the room, but actionable educational research that has good enough methodology to be believed is rare.
I hear a lot of complaints about edtech from the teacher I’m married to, and this is a common thread in how it fails: it tries to automate away the classroom experience rather than empower the instructor. Most teachers, good ones anyway, don’t want to spend less time teaching, they want to spend less time fussing around with bullshit like marking late assignments complete in two different systems and responding to irate parent emails to reassure them that the assignment actually is marked complete where it matters, just not in the system they’re looking at. Not only do a lot of these systems try to automate the wrong thing — instruction — they totally fail where it counts by making more work for the teacher.
Yes: I like your way of describing this. I think it's because of this:
Bloom showed that students instructed 1 on 1 did about two sigma better than those learning in a classroom environment. There's a huge temptation here, to think that technology can claw some of those 2 standards back through individualization and differentiation. As a result, everyone wants to build systems that do instruction and try to directly improve outcomes.
But kids won't work hard for a computer's approval.
I worked with a lot of schools in my last company and unfortunately I don't think extensive research is the best road either. While I totally understand your position, this excuse is also used for only allowing pen and paper for math in highschool by teachers.
It takes a crazy amount of time to extensively research and prove things in education for a few reasons:
There is opposition to doing the research because teachers/parents don't want their students to be the first.
It's really hard to make random control groups because schools want to put kids in classrooms by their process, not the whims of the researchers.
It's expensive and mostly controlled by the big granters (ex. Gates Foundation).
Results are often looked at via testing measures (which if you look at assessments, sometimes they were proven by having 400 white kids in an affluent school use, aka not representative).
And so, we have a bit of an issue.
Almost no one teaches math via a spreadsheet even though most adults do. It isn't researched on the implications.
Data science as a class to qualify for credit to get into college is greatly contentious because it isn't proven. Even though it is clearly important to our current era.
In addition, teachers are human and so they don't necessarily teach with the 100% best proven methodologies day in and day out.
So I think it's important to realize that education is human. It's flawed and messy and isn't aligned to the best methods already. Sometimes you get lucky with a teacher that works for you and sometimes not.
My wife is an active, publishing education researcher who is currently doing in-classroom research related to technology.
It's really hard to make random control groups because schools want to put kids in classrooms by their process, not the whims of the researchers.
IRBs won't let this happen in actual classrooms. The allowed approach is typically extracurricular summer schools where everyone is aware of the research component. But this is only for a certain kind of quantitative research. Qualitative research doesn't necessarily need nor benefit from a control group as it can be carried out in situ.
Isn't it worth the effort to do the research instead of just seeing what happens when venture-backed companies introduce their for-profit wares into the classroom?
> Isn't it worth the effort to do the research instead of just seeing what happens when venture-backed companies introduce their for-profit wares into the classroom?
On the IT side of things a while ago it was mainly about grabbing as much hardware as possible and then waiting for them to get bored and fuck off.
Certainly computers/calculators are useful sometimes (in particular when analyzing measured data), but for say 95% of mathematics, pen+paper seems appropriate? Taught properly, it's a highly pictorial subject and pen+paper is an extremely flexible medium.
A primary way that computers could ultimately help is additional ability to individualize content and to provide students with spaced repetition of practice on topics. I am building systems in these areas for my students, but they're offline (they print out and automatically grade worksheets to track student competence in areas).
One of the schools near us gives an ipad to every children from 1st grqde. Needless to say this instantly disqualified the school in my mind.
I’m not anti-screen, I think it’s good if my son uses a computer at home with things like kidpix, scratch, castle of dr brain, etc… But I think devices at school at that age are a potent distraction and I do not trust a school to be thoughtful about the usage of tech (the school I mentioned thinks it’s also a good idea to have some instant messaging app in grade 1 betweeen all the kids).
There have been many startups that have been just this. They all have died except for a very few.
Most of my edtech founder friends were motivated to serve and transform education because it didn’t work for them and they realized it didn’t work for most students.
Codeacademy is a good example of a successful one.
Companies that succeeded, in some cases enabling inner city school children to perform on at a college level, died. CiteLighter is a good example. There have been countless ones in math.
The reason the bad edtech products win is because they are built to survive the incredibly disjointed buying groups who all have, explicitly, their own agendas when it comes to anything in the classroom.
If students & teachers could decide where their classroom’s tech budget went, there would be a revolution in education in half a generation.
I've worked briefly in school where kids had iPads since 1st grade and in my opinion it was a catastrophe. Most obvious problem is simple – it's very difficult to get kids concentrated on anything. Tiny working memory and control over cognitive load makes it especially hard for kids to learn and no distraction helps. And iPad with especially funny, "engaged" and other material meant to be helpful created a tons on these – "Look, how colors change in the animation!", "I wonder what I can do with fingers on screen?", "Can I scroll slower and faster?", "Copmetition! Who is the fastest scroller!", "What math?".
do you also demand extensive academic research for benefits of tools for developers, business etc? Yes, there is an issue with tools and software that creates more problems that they solve (just as in business) but that doesnt mean all technology is bad.
The environment of a school is much more rigid and authoritarian than many workplaces (e.g. everyone must use this exact thing in this exact way all together right now), and students don't usually have much choice in their school. The tools are often taxpayer funded as well. Not to mention the fact that those affected by ed-tech are often minors. So I think it's reasonable for there to be some extra scrutiny surrounding a lot of ed-tech.
Maybe as a venture funded startup they don’t. I run a modestly successful edtech business that I have line of sight to making my full time gig in a few years if growth continues.
www.reportcardcomments.com
I think consumer edtech that makes products focused on making individual teachers’ lives easier has a niche. Maybe not as a 100x unicorn, but as a one person business, there’s plenty of opportunity yet out there.
Originally we were targeting schools as our main customers but found it incredible difficult to sell too.
The main reason probably that the beneficiaries (the students) are not the decision makers (the school admin) and that our product (air quality) is also not a core aspect of a school.
There also seems to be a run to the bottom regarding quality of products in many schools and no appreciation for sustainable and long lasting products.
We moved on in the meantime which is a pity because we would have loved to work more with students on air pollution and climate change.
Is this perhaps a general failure mode of private investment into civic institutions? Centralized decision-making has inevitable blind-spots and, as a general rule, doesn't work well for larger scale resource distribution problems.
Hey, but I got a great pricing deal to run our apps at this cloud provider no one uses and we have no experience with so you’ll have to migrate everything there!
I think this really depends on your definition of EdTech. I don’t think anyone would contend that turnitin or Ellucian haven’t scaled well.
Tools that cater to the organisations that have money can absolutely do well, but trying to sell to people without the means to buy your stuff will always be unsuccessful.
Yeah canvas and blackboard were are one point startups, I'm sure there's some products that can scale, but the market is probably driven by bureaucratic compliance more than a genuine desire to improve the quality of education for students
I only have experience of the university side of EdTech, but in my experience people were extremely interested in anything that would make it easier for lecturers to do their jobs, improve student experience, etc.
What they weren’t interested in was anything that changed the general dynamic of the university or replaced their core business.
Marking tools, stuff that helps deliver lectures, anything of that variety was all in-bounds, and lots of departments were happy to explore new things.
What would not sell was anything that required selling directly to students, because they simply won’t spend the money.
The university has a choice to not force such things. There are open source alternatives such as openstax. There's little reason to force students to buy expensive textbooks.
I agree; in my state, the universities are not required to have open source textbooks, but if they do use them, they are required to mark them in course catalogs so that students can pick them. I have yet to see this in practice though
I'm surprised it didn't do more than touch on probably the biggest reason: we already pay a huge amount for education out of taxes, so the educational budgets are allocated by bureaucrats, and not by education consumers (or their parents).
I think you're missing the fundamental mechanic here.
Spence signalling[1] won the econ Nobel back in 2001. It says that the cost of signalling in a market for imperfect information ends on the one emitting the signal.
For education and jobs, the signal is your degree (hence the huge career difference between dropping out one week before getting a degree and completing). The cost of the degree will fall on the student.
The employer can easily diminish the hiring problem by pre-sorting applicants by education. Students know there is a large difference in career outcomes between university education required jobs and the ones below that.
Colleges want to extract as close to 100% of this lifetime earning difference as they can.
The main question at this point in education research is "how much" signalling is a component of education versus actually teaching skills.
For tech, almost no certificate provides a comparable signal to a real degree.
Agree. Although I encountered an interesting who-are-the-users? issue when I was involved in using simulations to support training. The people buying the systems did at least understand they they should consult the users when writing their (so-called) user requirements. But who were the users? Three categories were identified: The students themselves, the instructors and the 'customers' of the training system - the people who had to 'employ' the students when trained. After a lot of discussion, they picked the instructors as the main users, and wrote the requirements for the training system with them in mind.
It worked, sort of. But what was overlooked was that the individuals who spent most actual screen time with the systems were the courseware developers. They had to build the training scenarios, which involved a lot of faffing around in the virtual world to create content that would support specific training objectives. The courseware devs, using the completed system, reckoned on an hour of dev time to build five minutes of useful training in the sim. One of the reasons it was so slow was that the courseware devs (think of them as level designers in a gaming context) had very few tools to help them build the scenario, e.g. to check map inter-visibility. The system devs had created some great tools for the instructors (e.g. after action review) but they had pretty much failed to consider how many different training scenarios were needed to keep the training relevant over time, and thus the relative importance of scenario dev tools.
If students are getting degrees to get hired, maybe it's the companies that are the consumer? Unless you're going for your own enlightenment, but I think that's a privilege reserved for rich kids.
I think another huge factor is that you are making a prediction on the future value of your degree over the next 40-50 years as a 17 year old who is probably more focused on succeeding socially and sexually. You add in the student loans that may or may not be forgiven at some time in the future and the free market is not in play. Same with Healthcare, insurance clouds all free market mechanics.
I'm a huge capitalist and I think we need to be honest about where capitalism isn't working. It's always due to free market mechanics being removed from the equation by layers of obscurity. I don't know the right answer but this shit is not working
A 17 year old focusing on succeeding socially and sexually is still really incentivized to go to college! People have tons of friends and sex at even the nerdiest college.
I've been a student at my local university for 19 years now, most of that part time. Sometimes I end up with degrees, other times not. When I'm taking classes that are either too niche or too hard to be on the easiest path towards a degree, it's fantastic.
But if my wanderings take me in a direction which is on the easiest path to a degree (sometimes necessary because the things I want to take have prerequisites that I'd rather not get waived) it's like I've entered some parallel dimension where I'm the only one who is willing to engage with the content. Like wow these kids are getting nothing out of this besides a grade.
So it sounds like we agree that it's not working, though I suspect we'd disagree about how much capitalism belongs in the solution.
Yes, in particular for PreK-12 schools here in the States. Here in NJ nearly all board members know little or nothing about education, but are elected as popularity contest. Superintendents get paid big salaries and are often severely under qualified. There is a lot of bureaucracy created by this.
Many companies that target PreK-12 also know there is a lot of incompetent admins with boat loads of cash. Pricing gouging is very common.
People are shocked when they learn about 70% of property tax bills goes to the school district, not the municipality or county.
> See, when you build an education product, you’re competing against two massive institutions: the formal education system (schools, colleges, state universities, etc.), and the laws and cultural expectations around that system.
What you’re competing against is the natural tension between these things and the demands of the market.
You get a good sense of this after 5 years at a math tutoring startup that’s always grappling with the dual expectation that you give the kid the answer quickly (“the customer is always right”) vs. you patiently sit with them until they learn (the goal of formal education).
"See, when you build an education product, you’re competing against two massive institutions: the formal education system (schools, colleges, state universities, etc.), and the laws and cultural expectations around that system."
Well, there's your problem. I worked for years in those markets, cooperating with their needs, not fighting them. You succeed in such markets by helping them do what they already do... just better. Disruption happens at a procedural level, not by changing what they do. So does the monetization: "We just saved you $20K per year, and only charge $10K - everyone wins."
The UK had a number of competitive and teacher-led EdTech AI startups delivering lesson planning, like briskteaching.com & teachematic. They have some inroads with schools and colleges now. And probably would have disrupted up into the procedural level naturally.
However the government decided to give £2m without any open procedure to a new company to build similar tech from scratch, and their PR acted as if this was groundbreaking, while the tech they promised, largely already existed and was in use by many teachers: https://www.gov.uk/government/news/new-support-for-teachers-...
Who wants to invest in teachematic or brisk teaching, when your competition can get millions from the government without process and launch the same ideas over a time frame to starve you out? With the market overcrowded, prices are forced to lower, and many companies will now go bust because of this kind of unnecessary government intervention. Brisk Teaching has already had to make most things free. You cannot really be more cooperative with the system than that, but tough luck they still won't get the £2m despite proving out the idea before those funded.
Shortly we will likely see the institutions in control of education like Ofsted mandate, approve or limit the tooling used, and most startups will die out. Rather than several low cost startups bringing high amounts of value, tax payers have spent millions on salaries for people with no AI background to create duplicate products, and the net value to education for actual students is therefore lowered as a result. So you can't disrupt this procedural level very effectively when the government intervenes early on.
Another way to make money in education is to make life easier for the teacher and have the students pay for the privilege. Automated grading with online quizzes and tests is an example of this phenomenon in higher education.
> Disruption happens at a procedural level, not by changing what they do
I love everything about this statement. I've seen this from both sides, as a seller of software and buyer/user of software. All of the stickiest changes happen procedural level, usually not through full-blown change but instead moderate tweaks and improvements.
One way I like to think about it is this: if my software is all of the sudden unavailable, what will users do? Can they keep working? Will they keep working?
Specific to education, I would hope that many core school functions like teaching would simply revert to simpler methods or alternatives rather than grind to a halt.
But I assume that the process of convincing the buyers to use your software runs headfirst into confusion over how much work it will be to learn something new rather than stick with (mostly) status quo.
So you want to maximise economic rent on the education of the masses and you wonder why societies are resistant to that, plainly because the outcome is horrendous.
Horrendous educational products are probably better than every institution in some regions, the median community college in other regions, etc..
Academic institutions, even very poor ones, have a lot of local leverage through employing academics who are integrated in the community and that is largely lost in distance education.
If you go and try to make an awful school somewhere I think you will get a great deal of assistance compared to if you offer a much better online education to the area built somewhere else.
Solving a problem and asking someone to pay for it doesn't have to be "rent seeking". If you want a service to work well and continue to, it's in your best interest to pay for it.
Who is getting the service here the individual or society. A neoliberal world view might assume there is no society but that is patently absurd.
If society wants too have a functioning and prosperous economy it should provide education as a utility not something that's geared towards profit. If anyone thinks neoliberal solutions will bring anything other than economic rent seeking you've been living under a rock for the last 40 years.
Isn't this just a subset of "all educational innovations don't scale". Education is chock-full of things that looked great at small scale and then fail to replicate at large scale. Given that long established reality, which is well known within the education field, the alleged scaling nature of software no longer seems like some great win, since you're likely just quickly scaling out something that doesn't actually work.
there is that, but that's not true for all changes.
the problem is that even innovations that do work and can be replicated successfully do not scale because the system as it is is highly resistive to change and that is what limits the potential to scale for any change.
take a look at montessori. an often praised and proven system of teaching and learning that absolutely revolutionizes education, if we could only convince those in charge to adopt it.
there is also not that much money in the state run that the majority of people education follow. Teacher salaries are already squeezed. Its ironic, everyone says get a good education, old grumps complain the kids don’t know anything, but when it comes to actually paying for it its crickets. Software could help make teaching more efficient but requires up front investment that no one wants to make. State run software projects tend to suck because they are based on minimising cost. Maybe if the act and job if teaching had the same social status as doctors and lawyers it would be different.
(School)teachers used to have that kind of social status, not even a hundred years ago. It would be interesting to understand how that changed into the current situation (up to the worst case of the parents calling out for the lynching of a teacher).
going out on a whim here, i think it is lack of trust. in the past teachers and students came from the same community, so everyone knew each other, and had the same beliefs. but now school is more distant and the education curriculum sometimes no longer reflects what parents expect. just look at the popularity of homeschooling. "we have to send our children to that school because the government demands it (and we can't afford homeschooling), not because we think that this is a good school."
Some of the teachers I have worked with do this; and there are some great teachers(lecturers) like Martin who teach effective studying techniques: https://www.youtube.com/watch?v=IlU-zDU6aQ0
Flashcards and Anki were incredibly effective for me. I am building a study tool which automatically generates them from lecture slides. Beyond that, it applies other psychology of learning techniques - It breaks up content into manageable chunks you can see your progress on (to boost your confidence of exam coverage) and has an exam-conditions like mode where it shows you relevant slides, then asks you questions and to freely recall by blurting onto a blank page everything you remember. It's then able to give you a realistic judgement of your learning and this way of learning by actively recalling freely is, though more intense, ultimately more effective than either reading, watching or quizzing in the long term.
The part of learning that other flashcard apps miss, even Anki, is that students must be motivated. They must enjoy, visually engage, and be confident as they study. They must feel they are on a sustainable realistic path as they gather knowledge when on the way to their goals. And they don't have to study every day to do that - they just need to genuinely learn and be able to see the state of their progress...
I am excited to be working on EdTech now[Revision.ai] which unveils and detects misconceptions to aid the student in eliminating them. Misconceptions are a massive opportunity - as much as spaced repetition itself [https://www.youtube.com/watch?v=eVtCO84MDj8]. We believe they will genuinely learn, feel comfortable and see their progress, when they overcome misconceptions with tech. Other tools really aren't identifying misconceptions; answering is still a "scale of wrong-to-right". And that's a big opportunity. To truly reduce Test Anxiety by studying and building your confidence. I think Tech can be useful in this way, even without teachers intervening, if enough of the learning content is there.
in the united states the terminology is reversed. "kindergarten" starts at age 5 for one year before school. "preschool" is everything before kindergarten. so there it's pre-school as in not-school, whereas elsewhere it's pre-school as in the year just before school.
“Making things easier for teachers” is often not the case and simultaneously creates costs for training and support at these institutions along with the exponential growth required to administer these new roles.
And then of course are the students themselves. How do iPads compare to textbooks? Do we trust LLMs to grade essays? How effective has multiple choice testing ever been?
There is surprisingly little thought put into bringing tech into the classroom. Most of the drive seems to come from a fear of being left behind, as if learning to use a consumer electronic device is somehow synonymous with learning how to maximize productivity with a computing device.