Smart geeks

Jesse Lichtenstein has an article in The New York Times about a pair of really smart guys:

One day, in the fall of their sophomore year, Matthew Fernandez and Akash Krishnan were at Akash’s house in Portland, Oregon, trying to come up with an idea for their school’s science fair. At Oregon Episcopal School, all students in the 7th to 11th grades are required to enter a project in the Aardvark Science Expo (the aardvark is the school’s mascot), and these two had teamed up for the last three years. Temporarily defeated, they popped in a DVD of I, Robot.
There’s a scene in the movie when Will Smith, a robot-hating cop, visits Bridget Moynahan, the impossibly gorgeous scientist, and they begin to argue. She gets angry. Her personal robot immediately walks into the room and asks: “Is everything all right, Ma’am? I detected elevated stress patterns in your voice.” It’s a minor exchange— a computer recognizing emotion in a human voice— in a movie full of futuristic robots wreaking havoc, but it was an aha moment for a desperate research team. Their reaction, as Matt describes it, was: “ ‘Hey, that’s really cool. I wonder if there’s any science there.’”
There was; it was just really hard. With emotion recognition, they stumbled onto a thorny problem. Computers have become very good at parsing an audio signal into specific words and identifying their meaning. But spoken language is more than just semantics. “If I say ‘happy’ and you say ‘happy,’ it’ll sound kind of similar, and a computer can try to match that up,” Matt explains. But it’s far from clear what elements in an audio signal indicate happiness or anger as a quality of voice. Trying to figure that out quickly consumed them. Matt stayed up late reading research papers, ignoring his other homework. Akash was up until 3 a.m. many nights, reading and programming. They spent long hours at each other’s houses, or talking on Skype.
The research paper they submitted for the school expo was thirty pages of code and sixty pages of writing to explain it. “Emotion is innately meta information,” Matt says, “and that’s why it’s a real challenge. A lot of people base their algorithms off of speech-recognition systems, because those have been established. But emotion is a really different task, and it’s a different goal.” For one, in speech recognition, sequence is essential; get the sounds out of order, and you mess up the words. In emotion recognition, the order isn’t nearly as important as various measures of energy and pitch. Determining what information to pay attention to in the audio signal and how to process it involves imagination, some sticky calculus and a lot of trial and error. “We tried to think of something new,” Akash says of the algorithm they built, “instead of using what other people tried to do.” The algorithm they came up with allows them to determine the emotion of a speaker by measuring 57 different features of an audio signal against a prerecorded signal that’s already been defined by a human listener as, say, “happy” or “angry.” Their algorithm doesn’t yet recognize confidence, or sarcasm, but what it does do (imperfectly, but better than the rest of the field) is detect fear, anger, joy, and sadness in real time, without eating up so much processing power as to be impractical in a handheld device.
Their project won the team competition at last year’s expo, and they went on to represent OES at the Intel International Science and Engineering Fair in San Jose, California, where they received the team award in physical sciences. In the fall, now juniors, they entered the Siemens Competition, one of two premier science competitions in the nation, and made it to the nationals in Washington, where they won the team grand prize. With the honor came $100,000 in scholarship money and two thick glass plaques; one sits above the fireplace in Matt’s house, the other in the dining room of Akash’s. When I met them last month, they had just returned from ringing the closing bell at the New York Stock Exchange. “Someone gave us his card,” Akash says, “and said, ‘When you make your company, be sure to give us a call.’ ”
Matt, 17, is voluble, well spoken, and quick to laugh. Akash, 16, is quieter, perhaps more intense; he describes himself as passive, though a better word might be shy. Both are six feet tall, musical, and athletic, albeit not in the sports that lead to high-school glory. (Matt is captain of the cross-country team; Akash plays tennis and has a black belt in karate.) “People, when they interact with us separately, would say we’re really kind of different,” Matt says. Yet each has a talent for finding interesting knots and unraveling them.
They have been friends at OES since the first grade. They first noticed a joint affinity for science in third grade, when they were taking timed multiplication tests: both excelled at math (that was clear) but the difference between them and their peers was, Matt recalls, “how much they cared about it”. In seventh grade, they wrote a computer program that helped their classmates quiz themselves on vocabulary. As eighth graders, they came up with a study of bridge design, building 41 balsa-wood models by hand, that did well at the science fair. In ninth grade, they took another stab at bridges, but the project tanked. That failure turned out to be a blessing. “On that project we learned a lot about how to work as a team,” Matt says.
Theirs has been a friendship that expresses itself most easily through the problems they solve. Yet, even as they were refining a machine’s ability to process emotions, Matt was struggling to come to terms with his own. His parents did not attend the Siemens Competition in D.C; his younger sister, Marie, went instead. Matt’s father, battling brain cancer for two years, was undergoing a last-ditch chemotherapy treatment. There were many discussions about whether Matt would even go ahead with the competition. In the end, he did, with Akash shouldering some of the extra load at the worst moments. “Even when life was pretty crazy for me, he was on top of everything,” Matt says of his partner. “Also, obviously, comforting, but he did a great job of taking the lead when he knew that I was not as mentally capable.” They won Siemens on 6 December. Louis Fernandez died on 2 January.
Matt speaks fondly of his father, of their weekend projects, of how it has now fallen to him to decipher the way his dad, a software engineer, set up their home’s networked electronics, even though his mother, Linda Blakely, trained as a mechanical engineer. Discussions at the dinner table began where Matt’s homework left off, and ranged far afield, from the physics of microwave ovens to the economics of candy consumption, with Matt’s dad relishing the role of explainer. Every evening, it was, as Linda describes it, “the land of the nerds”.
Despite their professions, Matt’s parents didn’t want Matt and his sister on computers at a young age. They favored board games and puzzles and banished the television to the attic, where it now sits, blocked by a Ping-Pong table, waiting for a special occasion like the World Series. Though the clutter of Matt’s bedroom reflects a variety of interests— trophies for chess and soccer, his hat collection, and a K’Nex contraption he built years ago— science held a privileged place. Above his bed hangs a black-and-white photograph of an elderly Albert Einstein with a blanket laid across his lap; in a color photo next to it, the face of a six-year-old Matt peeks out from beneath the same blanket. (Matt’s grandfather was apparently friends with the family of a former associate of Einstein’s.)
Akash, too, grew up in a “land of the nerds.” His father, Krishnan Palaniswami, an electrical engineer who works on a secret project at Microsoft, holds six patents and has another pending. His mother, Anitha Krishnan, is a software engineer at Synopsys. His parents emigrated from Southern India to attend graduate school, and the family visits relatives in Tamil Nadu every few years; they speak Telugu at home. Akash’s bedroom has none of the detritus you expect from a teenage boy: the walls are bare; his desk is uncluttered. He actually does his work downstairs, in an office that houses the computer he built recently from parts. When I visited, he slid out the transparent side panel to show me the machine’s innards, from which a blue glow emanated. There were six hard drives for processing the massive amounts of audio data their project requires, plus a water-cooling system to help the fan take the edge off the processing heat.
Akash ran through some of his former programming hits— a 'Virtual Classroom' he developed after snow and ice knocked out a week of school during finals (“It works the same way school does except it’s through the Internet”), a solitaire program, a Sudoku solver (“It uses brute force”). As the family dog, Vito, provided background vocals, Akash demonstrated last year’s real-time emotion recognition model. I spoke into the microphone and, with a one-second delay, the monitor showed me, with a series of bars, how much happiness, sadness, anger ,and fear my voice exhibited. I was apparently angry, or so the machine thought.
Akash and Matt may be teenage science geeks who get inordinate satisfaction from playing with mel-frequency cepstral coefficients late into the night, but that’s not all they are; their project has a variety of potential real-world applications. So real that a group of law students at Arizona State is helping them apply for patents. When Akash and Matt talk about the future, they talk about working on a chip that could provide emotion-recognition capabilities to a range of devices like cellphones or, someday, the onboard life monitoring systems of computers on spacecraft (benevolent ones, of course). Last summer, they interned with Jan van Santen, a researcher at Oregon Health and Science University, and became intrigued by the prospect of finding a way to help autistic kids pick up on emotional cues in speech. (They’re working on a wristwatch that processes speech in real time and flashes an emoticon to its wearer.) And both of them spend Tuesday and Thursday afternoons in a three-story open area, originally intended for gravity experiments, known as the Drop Zone. There, as part of OES’s InvenTeam, they helped design a bicycle-operated system for lowering a bucket deep enough to fetch water in parts of Niger, where the water table lies over a hundred feet below the surface. “The big project aspect is what gets me going,” Matt says. “Hopefully I can do more of those big projects, hopefully with Akash, in college or even beyond. Maybe we’ll drop out, I don’t know!”