This project is about extraction of social signals from traumatic brain injured (TBI) patients using computer vision techniques for human robot interaction (HRI). Social signal extraction from TBI patients is very challenging as they have restricted or limited muscle movements, with reduced facial expressions along with non-cooperative behavior, impaired reasoning and inappropriate responses with severe difficulty at social communication and activities of daily life. There are almost 6 million TBI individuals who require billions of dollars for their rehabilitation as well large number of care attendants to look after them. To interact with the TBI patients the major issue is the perception of their social signals. The precision ,speed and accuracy of such social signals are very important for the proper understanding of these patients conditions and enhancing their communication in long run. It is not an easy task as there is not much concrete research on extraction of social signals from these patients.
Facial expression is one of the main sources of communication for human emotions as approximately 55 percent of human communication is happened through facial expressions. {Mehrabian1968}.Therefore, fast and accurate extraction of facial expression can be very useful in understanding social signals. It is very vital for maintaining the reliable system of monitoring the health conditions of patients and diagnosis of different states like happy, sad, angry, surprise, fever, blood pressure and etc. Besides facial expression, some other signals (including physiological, psychological or physio-psychological signals), like heartbeat, respiratory, fatigue, pain, and stress can also be extracted from facial images and videos. These signals, similar to facial expression, have a direct relationship with our daily activities and mood. But, again existing technologies for extracting these signals from facial images are not only working for healthy people, but are limited to lab environment. Facial expressions extracted from TBI patients possess unique characteristics that can be more pronounced with their body language and head/face movement depending upon the nature of the injury. By analyzing these movements along with facial expressions, can provide better comprehension about their behavior and emotional states.
Developing a precise and efficient system for extraction of facial and mood expression with body language as well as physiological and psychological signals and interpreting them as social signals in the context of robotic interaction for TBI patient is the exact interest of this PhD study.