Robots Surf the Web to Learn About the World

Robots and computer programs are learning to associate words with objects by going online and Googling the words, using the retrieved images to make the connection. "If you give a robot visual capabilities, it could pretty much do anything," argues the University of Maryland in College Park's Alap Karapurkar. Carnegie Mellon University researcher Paul Rybski goes a step further. He says, "You could tell a robot, 'car,' and it could learn what a car looks like, that they're used for driving, then it could download a driver's manual, hop in the car and drive away." Rybski and colleague Alexei Efros put together the first Semantic Robot Vision Challenge at the annual American Association for Artificial Intelligence conference to test the theory. The competition involved instructing robots to search the Internet for images relevant to 20 object words, and then look for the objects in a 6-meter-square area. Robots were entered in the contest by five teams. The first step for the robots was converting the hundreds of images resulting from queries into descriptions that could be used to identify objects in the real world, and this was achieved through the use of software that analyzes the shading patterns in all of the resulting images to outline telltale characteristics and organize them into a sort of fingerprint. Several robots were equipped with stereo cameras to search for objects, which took snapshots for comparison to the fingerprint index. The robot that scored the highest--seven out of 20 found objects--was built by a team of University of British Columbia researchers. The software the robots run on could be applied to the significant improvement of Web image searches.   From ACM Technews