Sentience is a volumetric perception system for mobile robots, written in C#. It uses webcam-based stereoscopic vision to generate depth maps, and from these create colour 3D voxel models of the environment for obstacle avoidance, navigation and object recognition purposes.

Videos:

• Embedded stereo correspondence algorithm running on the Surveyor SRV-1 robot
• Features of interest which can be stereo matched
• Disparity histograms
• Stereo camera calibration procedure
• Rodney humanoid using the Minoru webcam
• Stereo matches with the Minoru webcam
• Dense disparity map with the Minoru webcam

Robots which use Sentience:

• Rodney
• Flint
• Surveyor SRV-1 (with stereo vision system)
• GROK2

Stereo cameras:

• How to make a stereo camera
• Using multiple cameras on Linux
• The Surveyor Stereo Vision System
• Minoru 3D webcam

Utilities:

• stereosensormodel - a program for creating stereo vision sensor models suitable for use with occupancy grids
• stereotweaks - a GUI which can be used to manually set stereo camera calibration offsets, visually check camera calibration and create anaglyph-like animations.
• surveyorstereo - a graphical user interface for the Surveyor SVS.

Development:

• What development tools do I need to work with the sentience code?
• How to install OpenCV
• How to compile and run the code

Fundamentals of stereo vision:

• Why stereo vision ?
• How do I calculate range from stereo disparity?
• Characterising uncertainties in stereoscopic vision
• Stereo camera calibration
• Sensor models

Other topics:

• Project roadmap
• How do I find the pixel density of the image sensor, or the focal length?
• Utilities for webcam based stereo vision
• A biologically inspired stereo correspondence algorithm
• Moving through space
• Autonomous Navigation
• Zen and the art of robot design
• Occupancy Grids

Background reading:

• Articles on occupancy grids and SLAM algorithms
• The Development of Camera Calibration Methods and Models by T.A. Clarke and J.G. Fryer.
• Probabilistic Robotics, by Sebastian Thrun, Wolfram Burgard and Dieter Fox.
• Estimating egomotion from stereo vision using the principle of least commitment