![]()
2
Change
LUIS VON AHN LOOKS LIKE your typical American college student, and acts like one too. He likes to play video games. He speeds around in a blue sports car. And like a modern-day Tom Sawyer, he likes to get others to do his work for him. But looks are deceiving. In fact, von Ahn is one of the world’s most distinguished computer science professors. And he’s put about a billion people to work.
A decade ago, as a 22-year-old grad student, von Ahn helped create something called CAPTCHAs—squiggly text that people have to type into websites in order to sign up for things like free email. Doing so proves that they are humans and not spambots. An upgraded version (called reCAPTCHA) that von Ahn sold to Google had people type distorted text that wasn’t just invented for the purpose, but came from Google’s book-scanning project, which a computer couldn’t decipher. It was a beautiful way to serve two goals with a single piece of data: register for things online, and decrypt words at the same time.
Since then, von Ahn, a professor at Carnegie Mellon University, has looked for other “twofers”—ways to get people to supply bits of data that can serve two purposes. He devised it in a startup that he launched in 2012 called Duolingo. The site and smartphone app help people learn foreign languages—something he can empathize with, having learned English as a young child in Guatemala. But the instruction happens in a very clever way.
The company has people translate texts in small phrases at a time, or evaluate and fix other people’s translations. Instead of presenting invented phrases, as is typical for translation software, Duolingo presents real sentences from documents that need translation, for which the company gets paid. After enough students have independently translated or verified a particular phrase, the system accepts it—and compiles all the discrete sentences into a complete document.
Among its customers are media companies such as CNN and BuzzFeed, which use it to translate their content in foreign markets. Like reCAPTCHA, Duolingo is a delightful “twin-win”: students get free foreign language instruction while producing something of economic value in return.
But there is a third benefit: all the “data exhaust” that Duolingo collects as a byproduct of people interacting with the site—information like how long it takes someone to become proficient in a certain aspect of a language, how much practice is optimal, the consequences of missing a few days, and so on. All this data, von Ahn realized, could be processed in a way that let him see how people learn best. It’s something we aren’t very easily able to do in a nondigital setting. But considering that in 2013 Duolingo had around one million visitors a day, who spent more than 30 minutes each on the site, he had a huge population to study.
The most important insight von Ahn has uncovered is that the very question “how people learn best” is wrong. It’s not about how “people” learn best—but which people, specifically. There has been little empirical work on what is the best way to teach a foreign language, he explains. There are lots of theories, positing that, say, one should teach adjectives before adverbs. But there is little hard data. And even when data exists, von Ahn notes, it’s usually at such a small scale—a study of a few hundred students, for example—that using it to reach a generalizable finding is shaky at best. Why not base a conclusion on tens of millions of students over many years? With Duolingo, this is now becoming possible.
Crunching Duolingo’s data, von Ahn spotted a significant finding. The best way to teach a language differs, depending on the students’ native tongue and the language they’re trying to acquire. In the case of Spanish speakers learning English, it’s common to teach pronouns early on: words like “he,” “she,” and “it.” But he found that the term “it” tends to confuse and create anxiety for Spanish speakers, since the word doesn’t easily translate into their language. So von Ahn ran a few tests. Teaching “he” and “she” but delaying the introduction of “it” until a few weeks later dramatically improves the number of people who stick with learning English rather than drop out.
Some of his findings are counterintuitive: women do better at sports terms; men lead them in cooking- and food-related words. In Italy, women as a group learn English better than men. And more such insights are popping up all the time.
The story of Duolingo underscores one of the most promising ways that big data is reshaping education. It is a lens into three core qualities that will improve learning: feedback, individualization, and probabilistic predictions.
Feedback
Formal education, from kindergarten to university, is steeped in feedback. We receive grades for homework, class participation, papers, and exams. Sometimes we get a grade just for mere attendance. Over the course of one’s schooling, hundreds of such data points are amassed—“small data” signals that point to how well we performed in the eyes of our teachers. We have come to rely on this feedback as indicators of how well one is doing in school. And yet, almost every aspect of this system of educational feedback is deeply flawed.
We’re not always collecting the right bits of information. Even when we are, we don’t collect enough of it. And we don’t use the data we’ve collected effectively.
This is ludicrous. Our iPhones are vastly more powerful than the NASA mainframe that flew astronauts safely to the moon and back. Spreadsheet software and graphing tools are amazingly versatile. But giving pupils, parents, and teachers an easy-to-use, comprehensive overview of student activity and performance remains the stuff of science fiction.
What’s most curious about our current use of feedback in education is what we measure. We grade the performance of pupils, and hold them responsible for the results. We rarely measure—and certainly not comprehensively or at scale—how well we teach our kids. We do not grade the degree to which our techniques are conducive to learning, from textbooks and quizzes to class lectures.
In the small-data age, gathering data on these sorts of things was far too costly and difficult. So we measured the easy stuff, like test performance. The result was that the feedback went almost exclusively in one direction: from the teachers and schools to kids and their parents.
In any other sector, this would be very strange. No manufacturer or retailer evaluates just its customers. When they get feedback, it is largely about themselves—their own products and service, with an eye to how to improve them. In the context of learning, feedback is primarily about how well a person has understood her lesson as perceived by her teacher (culminating with an infrequent, standardized test), not how good the teacher or the teaching tools have been for a particular student. The feedback is about the result of learning, rather than the process of learning. And this is because of the perceived difficulty of capturing and analyzing the data.
Big data is changing this. We can collect data on aspects of learning that we couldn’t gather before—we’re datafying the learning process. And we can now combine the data in new ways, and parlay it back to students to improve comprehension and performance, as well as share it with teachers and administrators to improve the educational system.
Consider reading. Whether people reread a particular passage because it was especially elegant or obtuse was impossible to know. Did students make notes in the margins at specific paragraphs, and why? Did some readers give up before completing the text, and if so, where? All of this is highly revealing information, but was hard to know—until the invention of e-books.
When the textbook is on a tablet or computer, these sorts of signals can be collected, processed, and used to provide feedback to students, teachers, and publishers. Little wonder, then, that the major educational textbook companies are piling into e-textbooks. Companies like Pearson, Kaplan, and McGraw-Hill want data on how their materials are used in order to improve them—as well as to tailor additional materials to students’ specific needs. Not only will this improve student performance, but the firms will be better suited to compete with rivals on the basis of being more relevant and effective.
For example, one thing publishers hope to learn is the “decay curve” that tracks the degree to which students forget what they’ve previously read and perhaps had once been able to recall. This way, the system will know exactly when to review information with a student so she has a better chance of retaining that information. A student may receive a message that he is 85 percent more likely to remember a refresher module and answer correctly on a test if he watches the review video in the evening two days before an exam—not the night before, and never on the morning of the exam.
Developments like this change the educational book market. There, badly written materials do more damage than a boring novel that we put aside halfway through. Generations of frustrated students may struggle to reach their potential because they’ve been exposed to flawed teaching materials. One need only pick up an elementary school primer from the 1940s or so, with their small typefaces, arcane language, and oddball examples divorced from reality, to see the tragicomedy of what we taught children at the time.
Of course, school review boards today extensively vet educational materials. But these boards are often constrained in their evaluation. They can examine content for accuracy and bias, and compare it with accepted standards of pedagogy. But they have no easy empirical way to know whether such teaching materials work well for the students using them, or to see how students respond to specific parts of the textbook, so that any shortcomings can be fixed.
In contrast, textbook publishers hope to receive the analysis of aggregate data from e-book platforms about how students engage with their material, what they enjoy, and what annoys them. It is not that the authors would be forced to incorporate feedback, but just receiving it might give them a better sense of what worked and what did not. Writing is both an art and a craft, and thus is open to improvement based on a big-data analysis of feedback data gleaned from readers.
There is still a ways to go to make this a reality. In the United States, states as diverse as Indiana, Louisiana, Florida, Utah, and West Virginia allow districts to use digital textbooks in their classrooms. Yet although sales of e-books are approaching parity with paper-based ones, only 5 percent of school textbooks in the United States are digital.
Yet the potential gains are huge. Just as Professor Ng of Coursera can tap the clickstream data of tens of thousands of students taking his class at Stanfor...
