Patent: Augmenting text messages with emotion information

This patent would automatically add emotional information to text messages, predicting the user’s emotion based on methods of keyboard input. The visual format of the text message would adapt in real time based on the user’s predicted emotion. As the patent notes (and as many people have likely experienced), it can be hard to convey mood and intended meaning in a text-only message; this system would aim to reduce misunderstandings.

Patent: Techniques for emotion detection and content delivery

This patent proposes capturing images of the user through smartphone or laptop cameras, even when the user is not actively using the camera. By visually tracking a user’s facial expression, Facebook aims to monitor the user’s emotional reactions to different types of content. To monitor the user, Facebook proposes using “passive imaging data,” or visual data captured automatically through a laptop or phone’s front-facing camera. The user often faces this camera without thinking about it, while using the phone or laptop normally, and Facebook hopes to start leveraging this imaging data.

Patent: Systems and methods for dynamically generating emojis based on image analysis of facial features

This patent describes a more streamlined process for sending messages with emojis. The system would capture real-time image data of a face (such as through a selfie) and analyze the facial features to determine the user’s emotion, and map it to the emoji that would be the best fit. For example, it could serve up a smiling emoji in response to a photo of someone smiling. The user can then add the emoji to a post or message.
The patent mentions several additional features, such as the ability to modify the emoji based on more detailed analysis of the user’s face, and the ability to capture gestures made by the user and add those to the emoji (such as the thumbs up in the image above).
By reducing users’ facial expressions to emojis from a pre-set list, Facebook could potentially analyze users’ emotions more easily. Facebook could gain clearer insight into feelings and reactions, while also adding a new interactive feature.