Scientists Invent Shrug Detector

Posted on May 1, 2006

New Scientist reports that scientists have created a real-time shrug detector.
I must admit that at first I struggled to see the point of this. Some computer vision researchers at the University of Illinois have come up with a "real-time shrug detector". But after a quick chat with Huazhong Ning, one of the researchers behind the project, I'm less sceptical. "When we communicate with other people we watch their body movements to help us understand," he told me. "Shrugging is a relatively easy one to detect, while others like blinking, hand movements and facial expressions are a lot harder. So we started with shrugging."

The detector tracks the movement of a person's face and shoulders and tries to spot "relative fast movement of the shoulder toward the face". That's a shrug to you and me. As you can see from this picture, even trying to hide behind a piece of paper won't foil it.
You can shrug but you can't hide. Or can you? Close inspection of the six page PDF document explaining the scientific principles and formulas behind the shrug detector reveals that the shrug detector will fail in some situations.
The system could fail in some special situations. It does not work when the face detector fails since the shrugging frame classifier depends on the result from face detector (e.g. in Figure.6 the face turns in large to one side). Fortunately our face detector detects almost all faces with yaw rotation angle within [60o; 60o]. A reasonable solution to this problem is to embedded a robust head tracker into this system. Also the system maybe generate false alarms when the subject drops or forwards head (Figure 7 is an example). In this case the false alarm is caused because the ct of TP approaches that of a real shrug while the ct of SP is not stabilized yet. The system will recover from this failure after several frames if the subject keeps this pose because 1) the classifier utilizes a decreasing threshold (see Eqn.2); 2) ct of TP will stabilize after some time.
The other problem with detecting shrugs (according to the PDF) is that people shrug differently. Some people make obvious shrugs while other people make very subtle shrugs that are hard to detect.
The first issue is "what is shrug"? The definition is usually qualitative rather than quantitative. Usually shrug means raising the shoulders but there is no objective definition of the time length and height of the action. The more ambiguous situation is that some people may shrug by just opening their arms with little or no movement of the shoulders.
We were going to purchase a shrug detector for the office but what good is a shrug detector that fails in special situations or generates false alarms? And can it make coffee? Because if it can make good coffee then that might make up for a little imperfection in its shrug detecting abilities.