NVISO Neuro SDK is a state-of-the-art software that integrates real-time 3D headpose in your applications. It brings vision and attention measurement to your product with minimal effort. The SDK comes with an extensive feature set, including full 3D headpose tracking, gaze estimation, eye tracking data analysis and more. Are you looking for a way to add 3D headpose estimation to your application? Or do you need to know what kind of emotions they are experiencing? If so, then NVISO Neuro SDK is the right tool for you!

3D headpose allows the accurate detection of changes in head position in a 3D space (with mm accuracy) and changes to headpose angle. These are tools that can be used to study human behaviour. The yaw, pitch and roll angle of the 3D headpose are highly accurate at low angles and robust up to extreme angles. 3D headpose is essential in understanding attention when eyes are not clearly visible or at insufficient resolution for accurate eye tracking and in the same time critical in understanding non-verbal communication gestures such as acknowledgment.

The human-machine interface is about to get a lot more interesting. We can expect to see more innovative ways of interacting with machines, whether they're cars, computers or appliances. This will be one of the key areas of research at the intersection of artificial intelligence and human-computer interaction. These interfaces will allow us to interact with our devices through voice, gestures and other non-traditional means such as eye tracking. They'll also allow us to take advantage of new technologies like augmented reality and virtual reality.

3D Headpose Technology

Science Based - 3D Headpose Model

The 3D head model is a representation of the human head that takes into account the anatomy and physiology. It consists of two main elements: a skull and eyeballs. The eyeball is represented by a sphere and its position in space, while the head is represented by a select number of reference landmarks with some additional information about its position and orientation in space.

Using a 3D head model, synthetic images from any point of view (POV) can be used to create synthetic training data for deep learning models. The advantage of our approach is that it allows us to automatically detect the headpose in any pose and any angle with high accuracy while maintaining compatibility with existing face tracking systems that have been designed for non-invasive measurements.

Improved Accuracy - Multi-task Convolutional Neural Networks

Multi-task learning is a method of training machine learning models to perform multiple tasks simultaneously. By using this approach, you can create a single model that can be used to perform many different tasks, reducing the workload and increasing efficiency.

Multi-task learning with convolutional neural networks has several advantages over other approaches for deep learning based eye tracking as geometric properties and eye states can learnt simultaneously together leading to better generalization performance. Multi-task learning helps improve generalization by allowing the network to learn more efficiently from less data due to its ability to share parameters across different tasks. This means that the model will be able to predict better results on new data if it has been trained with similar data before.

Reduced Bias - Large Scale Synthetic Face Databases

In recent years, there has been a surge of interest in the use of deep learning methods to classify heads. While these techniques have shown great promise, they are often limited by their inability to adapt to new subjects and contexts. This problem is particularly pronounced when the classifier is applied to new age groups or ethnicities. The problem is further complicated by the fact that head appearance can be significantly altered by specific medical conditions causing asymmetrical behavior of the face.

NVISO overcomes this these problems, by using very large scale databases of head images that have been annotated with a large number of key points, head angles, and segmentaton. These databases allow us to train models that are robust across different ages, genders, and ethnicities, while still maintaining high accuracy on a single individual.

3D Headpose Applications

Digital Avatars

Digital avatars are computer-generated images or animations that represent a real person, such as an actor or celebrity. Eye tracking can be used in digital avatars to make them more human by mimicking the way people interact with each other and the environment. When an avatar looks at something, we expect it to focus on that object and not wander off too quickly. When their eyes roam around the room, they should track back and forth, not stop abruptly.

Eye tracking is important for creating realistic digital avatars because eye tracking methods can be used to interpret the user’s intention and emotional state: gazing, blinking and eye openness. NVISO eye tracking on software can be used to precisely detect the eye movements of a user and then animate their face accordingly using artificial intelligence algorithms. This allows digital avatars to be more lifelike than ever before!

Companion Robots

3D headpose tracking is a robust way to track user’s attention especially in free environments, where the subject may be far from camera, such is the case in robotics. Unlike in front of a screen, when interacting with robots humans have a significantly larger space to move and engage with the robot companion and head pose is a key signal to understand attention.

A social robot is an autonomous robot that interacts and communicates with humans by following social behaviors and rules attached to its role. While robots have often been described as possessing social qualities, social robotics is a fairly recent branch of robotics playing a increasingly relevant role in today society and it’s making great strides with the advancement in computer vision AI such as offered by NVISO Neuro SDK.

Drowsiness and Fatigue

The need for automated eye tracking software is growing because human drivers are getting more and more drowsy and fatigued. According to the U.S. Department of Transportation, there were 5,987 fatal accidents in 2018 that involved drowsiness or fatigue as a factor in the accident.

The practical implementation of a Driver Monitoring system cannot make use of only eye tracking. 3D headpose is a back-up signal acting as a critical back-up to gaze in various situation (as for example when wearing sunglasses or extending the range of attention measure at large head orientation angles). Face, head, and eye landmarks can also be used to track the blink rate, blink duration, and perclos used in deriving drowsiness and fatigue indicators.


NVISO Neuro Models™ are purpose built for a new class of ultra-efficient machine learning processors designed for ultra-low power edge devices. Supporting a wide range of heterogenous computing platforms ranging from CPU, GPU, DSP, NPU, and neuromorphic computing they reduce the high barriers-to-entry into the AI space through cost-effective standardized AI Apps which work optimally at the extreme edge (low power, on-device, without requiring an internet connection). NVISO uses low and mixed precision activations and weights data types (1 to 8-bit) combined with state-of-the-art unstructured sparsity to reduce memory bandwidth and power consumption. Proprietary compact network architectures can be fully sequential suitable for ultra-low power mixed signal inference engines and fully interoperable with both GPUs and neuromorphic processors


NVISO Neuro Models™ use proprietary datasets and modern machine learning to learn from billions of examples resulting in an extraordinary capacity to learn highly complex behaviors and thousands of categories. Thanks to high quality datasets and low-cost access to powerful computing resources, NVISO can train powerful and highly accurate deep learning models.


NVISO Neuro Models™ store their knowledge in a single network, making them easy to deploy in any environment and can adapt to the available hardware resources. There is no need to store any additional data when new data is analysed. This means that NVISO Human Behaviour AI can run on inexpensive devices with no internet connectivity providing response times in milliseconds not seconds.


NVISO Neuro Models™ are scalable across heterogeneous AI hardware processors being interoperable and optimised for CPUs, GPUs, DSPs, NPUs, and the latest neuromorphic processors using in-memory computing, analog processing, and spiking neural networks. NVISO Neuro Models™ maximise hardware performance while providing seamless cross-platform support on any device.


Consumer Robots

Human–robot interaction plays a crucial role in the burgeoning market for intelligent personal-service and entertainment robots.

Automotive Interior Sensing

Next generation mobility requires AI, from self-driving cars to new ways to engage customers. Build and deploy robust AI-powered interior monitoring systems.

Gaming and Avatars

The gaming industry (computer, console or mobile) is about to make extensive use of the camera input to deliver entertainment value.


Get Our Whitepaper