Computer vision to improve public security and safety in crowded spaces

SAMURAI is being developed to improve public safety by using live CCTV footage and a smart computer brain to analyse real situations and spot people who are acting in a suspicious or out-of-the ordinary way.

The system will be able to identify situations such as a person leaving luggage unattended, a car being driven into an airport concourse and abandoned, or people congregating in an unusual way.

Project leader Professor Shaogang Gong from Queen Mary's School of Electronic Engineering and Computer Science explained: "SAMURAI will collect CCTV data of people and objects and use it to inform security staff of unusual behaviour. In doing this, we hope to contribute to safeguarding public security and safety and improve the way people experience busy public spaces."

In partnership with several European companies including BAA and French transport operator SYTRAL, the project will also find ways to aid the movement of people through some of our busiest public spaces, such as airports, railway stations and shopping centres.

SAMURAI is distinct from other recent and ongoing projects in several significant ways:

Firstly, it uses a network of sensors rather than CCTV cameras alone so that multiple sources of information can be combined to create a more complete picture of a crowded public space. These sensors can be attached to moving objects such as security staff or vehicles.

Secondly, SAMURAI will have the capability to learn about new behaviours on a day-to-day basis, as part of its ongoing training process. Rather than relying on a fixed set of rules, the computer brain is programmed to learn actively and incrementally from the human user's reaction to what it identifies as abnormal behaviour.

Professor Gong describes how the system understands and reads suspicious behaviour as "the science of computer vision." He says "Basically, we use the computer's learning ability to train it to recognise specific objects, like people or luggage, rather than simply looking for something familiar in the picture. SAMURAI can follow a moving object's pattern of activity and, crucially, its interaction with other objects."

Finally, the system will allow users to 'tag' and follow objects in the live CCTV videos across camera views from different physical locations. The SAMURAI system will provide a better overall awareness of the situation, making it easier for both control-centre and mobile security staff wearing head-mounted displays to track individual people or objects.

Existing automatic surveillance systems only alert human operators about a potentially dangerous situation after visual sensory data has been passively and often blindly captured. Whereas SAMURAI is designed to interact with, and work alongside human operators in real-time.