Behavioral Analysis of At-Risk Children
College of Science and Engineering, College of Education and Human Development, Medical School
University of Minnesota, Minneapolis MN
Home Projects Publications People

Behavioral Analysis of At-Risk Children


We are currently looking for social and behavioral science research assistants. Those interested can contact Nikos Papanikolopoulos

Creating a 3D reconstruction of a scene using multiple RGB+D sensors and tracking people in the scene

Researchers from the Artificial Intelligence, Vision, and Robotics Lab (AIVRL) have set up a system of mutliple Kinect RGB+D sensors in a research preschool classroom in the Shirley G Moore Lab school at the University of Minnesota. This system can be used to monitor the children in the classroom. Currently, this system consists of 5 Kinects that are connected to 3 PCs. The 5 cameras are strategically placed along the walls of the classroom so that each camera captures a novel view of the room. The cameras are then configured to begin recording at approximately the same time.

Each camera is calibrated so that it's exact position in the room is known as well as the characteristics of the camera itself. Using this calibration along with recorded RGB+D information from the Kinect a 3D representation of the room can be recreated. This 3D representation is then used to extract information about the objects in the scene. These objects are then identified and tracked as they move about the scene using region covariance descriptors.

Video for IROS 2012

Download Video

Video for WACV 2012

This video shows a 3D reconstruction from multiple sensors as well as tracking for a short period of time.

Download Video

Proof of Concept Video

Video showing initial work in the lab of creating a 3D reconstruction from multiple Kinect sensors.

Download Video

This work is supported by grants from the National Science Foundation, the Minnesota Department of Transportation, the University of Minnesota ITS Institute, and the Department of Homeland Security. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.