Open rtVision (Real-time visualization environment for image processing reconfigurable devices)

Open rtVision is an open source tool developed using .Net framework. The application is an interface between the module for the image acquisition from cameras and a coprocessing board connected with a computer using the PCI/PCIe bus. For more information about the system go to the detailed description.

Software platform

A simple and efficient interface is rather important for the development of complex systems. The presented work shows a software environment working as an interface between the image acquisition by the webcams or the scientific cameras and the co-processing board connected with the computer through the PCI/PCIe bus. The co-processing board has the role of extracting the visual primitives from the images: energy, orientation, phase, disparity and optical flow estimation. The computation is performed in real-time.

The software obtains and post-processes the results, to be displayed and stored. Just the visualization in real-time of this huge amount of data from the board is a big challenge. It requires the use of different libraries and software packages for the optimization of the management and processing of the results such as OpenCV®, IPP® and OpenMP®. The system fulfills the most important requirements of the work involving co-processing boards:

Capacity for the final verification of the results
Friendly interfaces for the system configuration
Display of a huge volume of data in real-time

Fig. 1 The figure shows an example of use for the disparity computation of a well-known sequence in artificial vision, the Tsukuba sequence.

Architecture

The environment is used in a specific architecture which includes:

A stereo camera system. We have two different setups: A pair of USB webcams or a pair of high performance scientific cameras with Camera Link connection. We study also other interface connections such as FireWire, Ethernet...

With the scientific cameras, we use a frame grabber

The presented software is running in a computer with PCI/PCIe interfaces for the board connections

The co-processing board. We developed the libraries for:

The XircaV4 platform, developed by Seven Solutions. It includes four SSRAM (ZBT) memory banks of 2Mwords of 36 bits, 2 512 Mb DDR memory modules, and a 32MB flash memory module. Furthermore, it has a 10/100/1000Mbits Ethernet connection, a JTAG configuration interface and the PCIe 1x interface. The FPGA is a Virtex4 XCVFX100 – 10FFG1152.
The RC2000 platform, developed by Celoxica® (then Agility®). It includes 6 ZBT memory banks of 1Mword of 36 bits, with a 16MB flash memory. In this case, the connection is PCI. Finally, the FPGA is a Xilinx Virtex-II XCV6000-6.

Hardware layout

The hardware co-design allows us to take advantage from the hardware processing such as the high paralellism, the low size and the low energy consumption ... But it also provides us the posibility of implementing complex algorithms easily and rapidly or to use the software as an interface with the visualization devices, or even, to validate and verify the hardware computation. With the presented tool we also developed two interface libraries for connecting the software application and the co-processing device, which are also documented.

Software layout

This work is focused on the software layout. It's the result of the need to joint the different tools used for the development of the image processing algorithms and to support to the high computing requirements for the post-processing and the visualization of the obtained results from the co-processing board.

Fig. 2 The figure shows an example of the complete system working in real-time computing a road scene.

With this aplication we concentrate the different tools in a platform which also integrates more functionalities. We also want to fullfil requirements such as a friendly user interface, usability, scalability, a platform optimized for image processing...

The following videos show the software working with real road scenes in real-time. It is displayed the disparity and the optical flow results.

Development Team

The open-rtVision has been developed at the University of Granada (Dept. of Computer Architecture and Technology). The main developers are F. Barranco, M. Tomasi, M. Vanegas, S. Granados and J.Ralli. The research coordination was performed by E. Ros and J. Díaz.

Now Open rtVision has been released as Open Source facilitated by the OSL “Oficina de Software Libre” through the advice of J.J. Merelo of the University of Granada. This means that any other development effort can be done by any other member or the research community.

Currently, we are improving the project working on new functionalities for the software platform such as the automatic camera calibration, the connection with new interfaces (FireWire and Ethernet), the use of sockets... If you could help or you have doubts about how to use Open rtVision, send an email to Francisco Barranco or Eduardo Ros.

Acknowledgment

The development of the Open rtVision platform has been supported by the following projects:

- The national project DINAM-VISION (DPI2007-61683).

- The EU project DRIVSCO (IST-016276-2).

- The regional projects of the Junta de Andalucía P06-TIC-02007 and TIC-3873.

For users

On the downloads section you can find tutorials, a video-tutorial, examples and the complete documented code. The software documentation can be generated using doxygen.