Human Machine Interfaces

NVISO's Neuro SDK provides a robust real-time human behaviour AI API, NVISO Neuro Models™ interoperable and optimised for neuromorphic computing, the ability for flexible sensor integration and placement while delivering faster development cycles and time-to-value for software developers and integrators. It enables solutions that can sense, comprehend, and act upon human behavior including emotion recognition, gaze detection, distraction detection, drowsiness detection, gesture recognition, 3d face tracking, face analysis, facial recognition, object detection, and human pose estimation. Designed for real-world environments using edge computing it uniquely targets deep learning for embedded systems, 

NVISO delivers real-time perception and observation of people and objects in contextual situations combined with the reasoning and semantics of human behavior based on trusted scientific research. The NVISO Neuro SDK is supported through a long term maintenance agreement for multi-party implementation of tools for AI systems development and can be used with large-scale neuromorphic computing systems. When used with neuromorphic chips, the NVISO Neuro SDK can be used to build gaze detection systems, distraction and drowsiness detection systems, facial emotion recognition software, and a range of other applications of neuromorphic computing where understanding human behaviour in real-time is mission critical.

Accurate and Robust

CNNs scale to learn from billions of examples resulting in an extraordinary capacity to learn highly complex behaviors and thousands of categories. NVISO can train powerful and highly accurate and robust models for use in the toughest environments thanks to its proprietary datasets captured in real-world environments.

Easy to Integrate

Where AI is fragmented and difficult-to-navigate at the edge, NVISO AI Apps are simple to use, develop, and deploy, with easy software portability across a variety of hardware and architectures.​ It reduces the high barriers-to-entry into the edge AI space through cost-effective standardized AI Apps that are future proof and work optimally at the extreme edge.

Ethical and Trustworthy

AI systems need to be resilient and secure. They need to be safe, ensuring a fall back plan in case something goes wrong, as well as being accurate, reliable and reproducible. Additionally unfair bias must be avoided, as it could could have multiple negative implications. NVISO adopts Trustworthy AI frameworks and state-of-the-art policies and practices to ensure its AI Apps are "fit-for-purpose".

NVISO AI App Catalogue
Application by Technology


Supporting the interaction of consumer devices with their owners in their daily lives. Detect presence and identity through facial recognition software. Anticipate and react to owner needs by combining with observations from gaze detection software. Pay attention to and appropriately adjust to mood through observation by facial emotion recognition.


With the rise of autonomous vehicles, smart video surveillance, facial detection and various people counting applications, fast and accurate object detection systems are rising in demand. With the advent of deep learning in object detection and recognition systems they can not only recognise and classifying every object in an image but also localise each one.


Facial emotion recognition software decodes facial behavior into seven primary emotional states, along with their respective intensity and probability of occurrence. Reading facial micro expressions allows emotion analytics to infer subtle changes in emotional expression over time and can be used to detect changes in mood and understand instantaneous reactions.


Remote vital sign monitoring using sensors such as time-of-flight cameras, mm-wave radar, and rgb cameras allows heart rate, breathing rate, fatigue levels, and advanced emotional states (anxiety, stress, and pain) to be measured remotely to a sensor (no physical contact required). This information can be gathered to assist in decision making.


Gaze detection software using deep learning performs real-time eye movement tracking providing gaze direction, as well as 3D and 2D coordinates of the eyes (pupils). Gaze detection systems are calibration free and provide the basis of more complex eye tracking systems which analyses human processing of visual information, measuring attention, interest, and arousal.


Artificial intelligence and human emotion recognition software allows non-verbal human communication to be detected and analysed by a computer. By reading emotional “expressions” from the face, tone of voice, and body and hand gestures more complex and advanced emotions can be detected. Custom emotion recognition development services and software solutions allow tailoring to specific use cases.


Driver attention detection systems are designed to warn you that you are fatigued or are at risk of becoming drowsy. Cars with drowsiness detection and occupant monitoring systems can monitoring eye state, blink rates, head gestures, body movements, and signs of fatigue such as yawning to provide driver distraction and drowsiness detection.


Human pose estimation provides multi-person 2d pose estimation for human body pose and shape estimation. Correspondingly 3d pose estimation can be performed using reference 3d human body models and by combining detection and tracking for human pose estimation in videos advanced interactive human machine interfaces can be enabled.


Deep learning for hand gesture recognition on skeletal data provides a fast, robust, and accurate method to detect hand gestures from a variety of camera sensors. Hand gesture recognition software will then classify both static and dynamic hand poses for interaction with autonomous systems for control and search tasks as well as emotional interactions.

Performance That Scales
Any Sensor, Any Placement

The interior of a vehicle, living room, or hospital can be an unpredictable environment. Typical constraints range from environmental unpredictability to drastic changes in ambient temperature. These factors drive the need for systems to include sufficient algorithms capable of handling tough environmental conditions and choice of camera placement is critical to enable the robust operation of AI systems. Another factor that adds to the system complexity is accommodating the cosmetic design of the system or product. Product designers constantly try to introduce new design concepts while also maximizing usability and experience features. These constraints require the position and location of the camera to often change from one product to another. NVISO addresses these challenges through supporting flexible camera positioning which is critical to large scale adoption.


NVISO Neuro Models™ are purpose built for a new class of ultra-efficient AI processors designed for ultra-low deep learning on edge devices. Supporting a wide range of heterogenous computing platforms ranging from CPU, GPU, VPU, NPU, and neuromorphic computing they reduce the high barriers-to-entry into the embedded AI space through cost-effective standardized AI Apps which work optimally on edge devices for a range of common human behaviour use cases (low power, on-device, without requiring an internet connection). NVISO Neuro Models™ use low and mixed precision activations and weights data types (1 to 8-bit) combined with state-of-the-art unstructured sparsity to reduce memory bandwidth and power consumption. Using proprietary compact network architectures, they can be fully sequential suitable for ultra-low power mixed signal inference engines and fully interoperable with neuromorphic processors as well as existing digital accelerators.


NVISO Neuro Models™ use proprietary datasets and modern machine learning to learn from billions of examples resulting in an extraordinary capacity to learn highly complex behaviors and thousands of categories. Thanks to high quality datasets and low-cost access to powerful computing resources, NVISO can train powerful and highly accurate deep learning models.


NVISO Neuro Models™ store their knowledge in a single network, making them easy to deploy in any environment and can adapt to the available hardware resources. There is no need to store any additional data when new data is analysed. This means that NVISO Human Behaviour AI can run on inexpensive devices with no internet connectivity providing response times in milliseconds not seconds.


NVISO Neuro Models™ are scalable across heterogeneous AI hardware processors being interoperable and optimised for CPUs, GPUs, DSPs, NPUs, and the latest neuromorphic processors using in-memory computing, analog processing, and spiking neural networks. NVISO Neuro Models™ maximise hardware performance while providing seamless cross-platform support on any device.

Neuromorphic Computing Interoperability
Ultra-Low Latency with Low Power

Ultra-Low Latency (<1ms)

Total NVISO Neuro Model latency is similar for GPU and BrainChip Akida neuromorphic processor (300 MHz), however CPU latency is approximately 2.4x slower. All models on all platforms can achieve <10ms latency and the best model can achieve 0.6ms which is almost 2x times faster than a GPU. On a clock frequency normalization basis, this represents an acceleration of 6x.

High Throughput (>1000 FPS)

NVISO Neuro Model performance can be accelerated by an average of 3.67x using BrainChip Akida neuromorphic processor at 300MHz over a single core ARM Cortex A57 as found in a NVIDIA Jetson Nano (4GB) running at close to 5x the clock frequency. On a clock frequency normalization basis, this represents an acceleration of 18.1x.

Small Storage (<1MB)

NVISO Neuro Models can achieve a model storage size under 1MB targeting ultra-low power MCU system where onboard flash memory is limited. Removing the need for external flash memory saves cost and power. BrainChip Akida format uses 4-bit quantisation where ONNX format uses Float32 format.

Designed For Edge Computing
No Cloud Required

Privacy Preserving

By processing video and audio sensor data locally it does not have to be sent over a network to remote servers for processing. This improves data security and privacy as it can perform all processing disconnected from the central server, which is a more secure and private architecture decreasing security risks.

Lower Energy Usage

The more we move data, the more energy we use. Processing data first on-device opposed to sending it to the cloud uses a lot of less energy. As the amount and rate of data exchange with the cloud is minimised, the power consumption of the device is reduced thus improving battery lifetime, which is critical for many edge devices.

Higher Availability

Decentralisation and offline capabilities make edge AI more robust since internet access is not required for processing data. This results in higher availability and reliability as weak WiFi signals do not impact the device performance.

Easy To Integrate


Process captured data from a camera in real-time on-device with our EVK. Quickly verify your use case using our 30-day Trial EVK License by processing captured data from a camera sensor in real-time. Understand if existing NVISO AI Apps are suitable for desired end application performance.


Fast-track your development with our x86 development platforms with APIs for software in-the-loop testing, evaluation, and creating demonstrators. Out-of-the-box software using our Developer SDK License allows you to get up and running in minutes not weeks.


Access the provided signals on-device or transmit them to other devices and then act on them to deliver innovative product features. Deploy AI-driven human machine interfaces by using our Production SDK License on production hardware.


Talk with NVISO AI expert to learn more about about trial Evaluation Kit (EVK) for neuromorphic computing devices.


Consumer Robots

Human–robot interaction plays a crucial role in the burgeoning market for intelligent personal-service and entertainment robots.

Automotive Interior Sensing

Next generation mobility requires AI, from self-driving cars to new ways to engage customers. Build and deploy robust AI-powered interior monitoring systems.

Gaming and Avatars

The gaming industry (computer, console or mobile) is about to make extensive use of the camera input to deliver entertainment value.