Getting Started
Model Development
The HarmonEyes process for development of models involves four major phases. Not all phases are appropriate for every solution. Nevertheless, these phases are outlined here, and, if needed, are detailed under each solutions page. Phases include:
Phase 1: Create a Classification Model (see Figure 1)
Phase 2: Create a Time Series Model (see Figure 1)
Phase 3: Transfer Process
Phase 4: Automated Conversion Engine
Figure 1: Development of Classification Models and Time Series Models
Phase 1: Create a Classification Model
Step 1 Identify the Target: This is the variable or metric you’re trying to predict. It is also often called the dependent variable, the response variable. Examples include if a person is “loaded”, that is cognitive load, or fatigued, or stressed.
Step 2 Determine Validation Measure: this is also known as the “ground truth” or “gold standard.” It is a widely accepted measure of the target variable. For example, the NASA-TLX is the validation measure for cognitive load. The Visual Fatigue Survey (VFS) is the validation measure for fatigue.
Step 3 Apply the Data: HarmonEyes analyzes eye tracking data to form solutions or outcomes.
Step 4 Determine the Features: Machine Learning methodology is used to determine which eye tracking features help identify the target state.
Phase 2: Create a Time Series Model
Step 5 Transfer Data to Time Series: data is then sequenced into segments to form time-series data.
Step 6 Build Time Series Model: Machine learning and Artificial Intelligence is used to build time series models which may include forecasting, Long Short-Term Memory (LSTM) models.
Step 7 Tune the Model: Tuning parameters are used to refine results.
Phase 3: Transfer Process
HarmonEyes applies transfer learning to the pre-trained base models in Phase 1 and 2. Transfer learning is a type of supervised deep learning that involves transferring knowledge from one task to another. In this way, we benefit from the general features and patterns learned by the pre-trained model and adapt them to related situations. Specifically, to different hardware or firmware updates and to new environments or stimuli.
The HarmonEyes Transfer process includes:
- Decoupling algorithms from stimuli and testing models using various stimuli
- Testing using ground truth outcomes in new devices and firmware updates
- Deep learning with model parameter tuning with new trained layers
- Blind testing then unblinding results against ground truth measures
- Stress testing models
To evaluate the success of the transfer, HarmonEye’s re-runs models and requires results to fall within the same parameters as the Phase 2 results.
I think transfer learning is the key to general intelligence. And I think the key to doing transfer learning will be the acquisition of conceptual knowledge that is abstracted away from perceptual details of where you learned it from.
Demis Hassabis, CEO, DeepMind.
Phase 4: Automated Conversion Engine
The Automated Conversion Engine (ACE) normalizes the eye tracking data stream to enable processes occurring at subsequent steps in the pipeline to work accurately. This further transfers the results making the models interoperable. More on the ACE can be found in the specific sub-section called “Automated Conversion Engine.”