ML/AI Overview

Theia SDK

Is the software development kit housing all HarmonEyes solutions. It is licensable and connected via API’s for modification of output.

Theia Dashboard

The HarmonEyes dashboard is a front-end visualization of the Theia SDK. It is an optional customer add-on and enables real time stream model and AI output for customer viewing and interactions.

HarmonEyes Data

HarmonEyes has over 15 million eye tracking records from our parent company RightEye. These records include demographic tags such as age, sex, handedness, ethnicity. Also included are clinical conditions, high performance athletes, notes, and other contextual information. This data is used to inform HarmonEyes models and tune the results to known differences in the eye tracking signal, such as changes in velocity of the eye across ages.

This data differentiates HarmonEyes’ solutions and enables models to be more accurate and generalizable.

HarmonEyes Platform

HarmonEyes has a dedicated ML platform, architecture, and dashboard used to ingest and make sense of data. The platform includes data input and search capabilities, quality standards, Exploratory Data Analysis, data cleaning and feature generation, then modeling and AI development.

Ground Truth

Ground truth is a scientific concept that refers to the verified, objective, and accurate data collected through direct observation or measurement of real-world conditions, serving as the “gold standard” to train, test, and validate models. It acts as the definitive reference point for accuracy in AI, machine learning, remote sensing, ensuring computer-generated outputs align with physical reality.

HarmonEyes collects ground truth using a variety of methods and is conducted throughout product development and an optional feature upgrade for customers to use in real time or annotate post processing. The ground truth categories include:

• Gold standard approaches: these are known industry standards for identification of a state such as the NASA-Task Load Index for measuring Cognitive Load, or the Perceived Stress Scale for measuring stress.
• Education and Training: dedicated pre-test training to assist and ensure research participants understand a concept are developed to ensure high confidence in reporting data.
• Comparative Ground Truth: often human performance concepts measured by changes in the eye are also seen in other neurophysiological responses. For example, stress can be seen in an increased heart rate using a heart rate monitor, high cognitive load can be measured in activity in the pre-frontal cortex using an electroencephalogram (EEG). These measures can be used as additional confirmations of changes in a person’s state.
• Inferred ground Truth: refers to known situations in which a state is expected for example, a difficult task with multitasking is expected to induce higher levels of cognitive effort than a simple task. Therefore, task difficulty can be used to infer expected levels of load. This is especially helpful when no participant ground truth input is available.
• Human-In-The-Loop (HITL) Ground Truth: where the person is obtaining output for their state and in real time, they are able to input how they feel. There are two types of HITL feedback that we employ:
  • Real time confirmatory ground truth: where a person agrees or disagrees with eh model output.
  • Real time reporting of state: where a person inputs their state as they are engaged in an activity ignoring any model interpretations.

Machine Learning (ML)

ML approaches include supervised and unsupervised learning.

Unsupervised Learning: grouping similar data (clustering), simplifying data (dimensionality reduction), normalization, and finding relationships (association rules) for applications like customer segmentation, anomaly detection, and recommendation systems, contrasting with supervised learning’s use of labeled data.

Supervised Learning: where patterns are learned based on labeled data including algorithms such as regressions and classifications. Time series analysis algorithms such as LSTM, GRU’s are also examined.

Often solutions require more than one model or type of model, therefore HarmonEyes has model architectures such as ensemble modeling and hierarchical modeling.

Artificial Intelligence (AI)

To ensure our real-time models perform reliably across diverse people, tasks, and environments, HarmonEyes employs a range of AI techniques. Examples such as neuro-adaptive techniques and learning strategies, transfer learning, and personalization techniques help us understand patterns in the data and are especially useful for the dynamic nature of real time analysis.

Eye Tracker Differences

Not all eye trackers perform the same way.  Differences may include sampling rates, signal quality, mapping coordinates, classification information and others.  HarmonEyes automatically understands differences across eye tracking devices and normalizes the signal to continue to provide accurate solutions in real time.