What emerged was a pilot project to run some 20 different challenges using online “open innovation” platforms TopCoder, InnoCentive and Yet2. “NASA could have built our own platform and community, as we did for the Centennial Challenges, but we chose to tap into a community that already existed,” says Crusan. “TopCoder has an existing software and algorithmic community and we wanted access to that.”
The pilot kicked off in July 2010 with a $23,000 challenge on TopCoder to develop a mathematical algorithm that would determine the optimum contents of medical kits for future manned space missions. “It was a classic packing algorithm,” says Crusan. The challenge drew more than 2,800 code submissions from almost 1,100 participants, with the top solution-providers coming from the U.K., Japan, Indonesia and Brazil.
Other challenges followed on InnoCentive, ranging from designing a zero-g laundry machine to developing a way of forecasting solar activity that poses a radiation risk to humans and hardware in space. “We received a solution that exceeded our requirements from a retired radio-frequency engineer in rural New Hampshire, which can forecast a solar event hours in advance with 85% accuracy,” says Gustetic
The online platforms used for the challenges ar2 “fundamentally different,” says Crusan. The NASA Tournament Lab (NTL) was established with Harvard Business School and TopCoder to run software-oriented challenges. The NASA Innovation Pavilion, which uses InnoCentive, focuses on ideas and approaches, such as the $20,000 challenge to come up with a new way of measuring the strain on Kevlar and Vectran straps without damaging inflatable space structures.
“The original goal was 20 challenges over two to three years. We are coming to the end of the pilot project this October and will be close to hitting 20,” he says, adding “We have learned quite a bit.” One lesson is that a prize with a large dollar value can be intimidating. “People may doubt they have a $1 million idea, but think they have a $10,000 idea, so you get higher participation.”
NASA's online challenges are attracting interest and ideas from outside the traditional space community. “We did a bunch of pilots in 2007-09 with the InnoCentive crowd-sourcing platform. Over seven prizes we had 3,000 solvers participate,” Gustetic says. “Out of the 3,000, 81% had never registered to get a request for proposals or work with NASA. We are tapping into huge groups of problem-solvers.”
With slightly more than $1 million in funding over its life, the NTL does more than just run competitions, says Crusan. The project also has funded post-doctoral students at Harvard to study how best to run challenges.
“The research highlights general lessons on how to size the incentives and generate the right behavior,” says Hughes. “If you are looking for broad-ranging ideas and different viewpoints, you want a large group and as many submissions as possible. If you have a narrower, more focused requirement you want fewer participants because you need an optimized, high-quality solution.”
“It has been very interesting,” says Crusan. “We have run experiments on signaling, to see if it is just cash, or other incentives that get people to participate,” he says. For some it is status within the online community; for others it is building a resume to get a sought-after job at Google or the NASA Jet Propulsion Laboratory.
“TopCoder's rating system is central to participation, particularly on the algorithmic side,” says Andy LaMara, program director. “Handles on the forum are color-coded to indicate the individual's proficiency. This has an interesting effect. Red handles are at the very top, several standard deviations above their peers. When they weigh in, others listen,” he says. And if a TopCoder “red handle” applies to Google “they skip five of the nine interview steps.”