Juxi's log

Recently I have been working a lot on trying to make the iCub see things. A fully integrated computer vision or robotic vision system is a quite tricky mathematical and engineering problem. Here at IDSIA we were trying to develop an easy to use system that would allow to rapid prototyping (offline) vision modules for the iCub, mainly to detect and localise objects the robot is in later stages supposed to manipulate and interact with.

Over the last 6 months, I developed the icVision framework, which allows to do just that. It is a modular system based on YARP and openCV. A video is probably the easiest way of showing the capabilities of icVision.

The main modules involved here are the icVision Core module, as well as, the icVision filter modules.
The Core handles the connection with the hardware and provides housekeeping functionality (e.g., extra information about the modules started and methods to stop them). Other modules can attach to the icVision Core and can therefore provide functionality by using standardized interfaces. Currently implemented modules include object detection, 3D localisation and saliency maps.
Some modules also expose their functionality by HTTP (e.g. an image can be grabbed from the HTTP instead of YARP, which increases the number of people that can work with the iCub images in our lab by A LOT).

Most recently we were working on learning how to detect objects and creating filters for specific objects. The teabox filter used in the video is an example of this. This is done using Genetic Programming and is enabled by the very easy way to add new filters. The following is the code for a fully working filter using the framework:

  // Author: Juxi Leitner <juxi.leitner@gmail.com>
  #include <icVision/icFilterModule.h>

  class RedDetector : public icFilterModule {
  protected:
  	void setUsedInputs() { usedInputs.Add(1); }
  public:
	RedDetector() { setName("RedDetector"); }

	icImage* runFilter() {
		icImage* in = InputImages[1];	// red
		icImage* out = in->threshold(64);	// cvThreshold
		return out;
	}
  };

There will be a poster presentation and demonstration of icVision during this year's Cognitive Systems conference in Vienna.
The other area we are currently working on is to learn to localise objects in 3D space. This is a bit more tricky. More on this later :)