API Reference

Real-time mental workload, fatigue, attention, and mental readiness from eye-tracking hardware

Contents

• Analysis Outputs
• Applications
• Getting a License
• Scene Setup
• License Validation
• Submitting Eye Tracking Data
• Accessing Results
• Class Reference

Analysis Outputs

The SDK exposes four metrics derived from eye-tracking. Mental Workload and Fatigue refresh once per second (edge-triggered when a new batch prediction arrives from the native theia engine); Attention is updated continuously as fixations and saccades accumulate; Mental Readiness is computed on demand from accumulated Mental Workload predictions:

• Mental Workload — mental demand and processing effort during a task. Level 0/1/2 (Low/Moderate/High), with confidence.
• Fatigue — alertness level and transition-to-next estimate. Level 0–3 (Alert / Neither / RatherDrowsy / Drowsy), with confidence, ttt_level, and ttt_minutes.
• Attention — Nideffer ACT eye-movement style classifier. Level 1–5 (Narrow / Mod-Narrow / Neither / Mod-Broad / Broad), with composite_score and total_observations.
• Mental Readiness — distribution of recent Mental Workload predictions across the session (low/moderate/high percentages). Returned only after a minimum session length is met.

Each payload carries a counter that increases monotonically — batch_number for Mental Workload and Fatigue, total_observations for Attention, total_predictions for Mental Readiness — so consumers can edge-trigger UI updates when the counter advances.

Applications

• Reading performance — real-time line-by-line tracking, comprehension improvement, detecting reading issues.
• Clinical diagnostics — screening for Parkinson’s, TBI, Lyme disease, MCI, MS via eye-movement biomarkers.
• Visual search / driving — monitoring attention, decision-making, situational awareness.
• Biometric ID — eye movements as real-time user authentication.
• Developer testing — user testing during app development.
• Market research — interface usability and user behavior insights.

Getting a License

A license key is required to use the HarmonEyes SDK. Contact sales@harmoneyes.com to obtain one.

Scene Setup

1. AnalyzeEyeTrackingData must be present on a GameObject in the scene. AnalyzeEyeTrackingData holds SDK configuration and the latest predictions, runs the once-per-second submission of accumulated gaze samples to the native SDK.
2. Enter your license key in the AnalyzeEyeTrackingData inspector (the LicenseKey field). The SDK validates it synchronously during Start(). Validation Success or Failure will be clearly logged to unity console/editor log output.
3. Add a platform-specific eye-tracking collector that submits eye data samples into Instance.EyeTrackingData.Samples each tracker frame. See the Meta and Vive sample documents for reference implementations (OculusEyeTrackingCollectingData and ViveEyeTrackingCollectingData). See the Documentation for submitting eye tracking data below to integrate your own source of eye tracking information.

License Validation

On scene start, license validation happens automatically. AnalyzeEyeTrackingData.Start() calls EyeTrackingAnalyzer.Initialize, which invokes theia_sdk_create + theia_sdk_start_session up-front (independent of headset readiness). Two boolean flags expose the outcome:

if (AnalyzeEyeTrackingData.Instance.LicenseKeyValidated) { /* SDK is live; start submitting samples */ }
if (!AnalyzeEyeTrackingData.Instance.EyeTrackingInitCompleted) { /* missing/invalid key or create() returned NULL */ }

If the license is missing or validation fails, no samples are pushed; the error is logged to the Unity console and surfaced on any bound TMP fields.

Submitting Eye Tracking Data

Per-frame eye data is built into a neutral Sample via GazeDataSampleFactory.CreateGazeSample, then submitted via AnalyzeEyeTrackingData.SubmitGazeSample. AnalyzeEyeTrackingData submits accumulated samples once per second to the native SDK via add_gaze_samples.

Sample collection runs in FixedUpdate at the eye tracker’s native rate for the Quest Pro and Vive sample scripts. The scripts set the fixed update delta time directly, setting at the project level is not required. If you have an eye tracker that returns batches of samples to you on demand or streams it’s data via another method, then it is not necessary to use FixedUpdate or alter Time.fixedDeltaTime.

Minimal per-frame collector:

void FixedUpdate() {
    if (AnalyzeEyeTrackingData.Instance == null || !AnalyzeEyeTrackingData.Instance.EyeTrackingInitCompleted) return;

    // Nanosecond timestamp.
    long timestampNs = (long)(Time.timeAsDouble
        * 1e9);

    // Read per-eye origin and orientation from your platform, and decide isBlinking yourself.
    Sample s = GazeDataSampleFactory.CreateGazeSample(
        AnalyzeEyeTrackingData.Instance.SessionId, timestampNs,
        leftEyeOrigin,  leftEyeOrientation,
        rightEyeOrigin, rightEyeOrientation,
        headCamera.fieldOfView, isBlinking);

    AnalyzeEyeTrackingData.Instance.SubmitGazeSample(s);
}

GazeDataSampleFactory.CreateGazeSample projects each eye’s gaze ray onto the SDK’s 1600×1200 virtual screen at 600 mm distance using the supplied FOV. The returned Sample carries timestampNs, left/right eye XY (pixels), optional pupil diameters, and a monotonic blinkId.

Accessing Results

Accessing Results Via Callbacks

Subscribe to OnMentalWorkloadResult, OnFatigueResult, OnAttentionResult, or OnMentalReadinessResult on the analyzer to be pushed each new prediction as it arrives.

AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.OnMentalWorkloadResult += mw => Debug.Log(mw.LevelName);
  AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.OnFatigueResult += fat => Debug.Log(fat.LevelName);
  AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.OnAttentionResult += att => Debug.Log(att.label);
  AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.OnMentalReadinessResult += mr => Debug.Log($"low {mr.low_percentage:F0}% / mod {mr.moderate_percentage:F0}% / high {mr.high_percentage:F0}%");

Accessing Results Directly or Via Polling

Results are flat data objects exposed via the EyeTrackingAnalyzer instance accessed through the AnalyzeEyeTrackingData singleton. All four fields start null and populate only when the native SDK emits its first prediction for that metric — null-check before use.

AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentMentalWorkload    // MentalWorkloadData,   nullable
AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentFatigue           // FatigueData,          nullable
AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentAttention         // AttentionData,        nullable
AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentMentalReadiness   // MentalReadinessData,  nullable

Class Reference

Every type below lives in the HarmonEyes.EyeTracking.Common namespace. Types marked “MonoBehaviour” are Unity components you add to a GameObject from the Inspector. All other types are plain C# classes.

AnalyzeEyeTrackingData TMP display fields — main scene component

Scene-level singleton component. Hosts the license-key, tracker, console-logging, and session-status inspector fields. Owns the native SDK lifecycle and the 1 Hz analyzer pump. Exactly one instance must exist in the scene.

Inspector Fields

Name	Type	Description
LicenseKey	string	Your HarmonEyes license key. Required. Validated synchronously in Start().
Tracker	TrackerType	Native SDK preset for your eye-tracking hardware. Required. Default: Quest. Values: Nexus (Webcam, 30 Hz), Ganzin (60 Hz), Quest (72 Hz, also used for Vive Focus Vision), PupilLabs (200 Hz). Pick the preset whose sample rate is closest to your tracker, rounding down.
logMentalWorkload	bool	When true, writes each new mental-workload prediction to the Unity console.
logFatigue	bool	When true, writes each new fatigue prediction to the Unity console.
logAttention	bool	When true, writes each new Attention pull to the Unity console.
logMentalReadiness	bool	When true, writes each new Mental Readiness pull to the Unity console.

Public Properties

Name	Type	Description
Instance	static AnalyzeEyeTrackingData	In-scene singleton and interface to the SDK
EyeTrackingInitCompleted	bool (readonly)	True once Initialize() finished successfully and the SDK is ready to accept samples.
LicenseKeyValidated	bool (readonly)	True if the license key passed validation (theia_sdk_create succeeded).
EyeTrackingData	EyeTrackingData	Shared per-session buffer of collected Sample records. Populate its Samples from your collector.
EyeTrackingAnalyzer	EyeTrackingAnalyzer	The active analyzer instance. You can call its methods directly; normal integrations do not need to.
EyeTrackingAnalyzer.CurrentMentalWorkload	MentalWorkloadData (nullable)	Latest mental-workload prediction. Null until the first batch arrives.
EyeTrackingAnalyzer.CurrentFatigue	FatigueData (nullable)	Latest fatigue prediction. Null until the first batch arrives.
EyeTrackingAnalyzer.CurrentAttention	AttentionData (nullable)	Latest Attention pull. Null until the first pull populates it.
EyeTrackingAnalyzer.CurrentMentalReadiness	MentalReadinessData (nullable)	Latest Mental Readiness pull (low/moderate/high % breakdown of observed Mental Workload predictions). Null until the minimum session length is met. Overridable via EyeTrackingAnalyzer.SetMentalReadinessMinSessionSeconds.

Methods

Initialize() — called automatically from Start(). Validates the license, creates and starts a native theia session, sets the LicenseKeyValidated and EyeTrackingInitCompleted flags. You normally do not need to call it yourself.

public void Initialize();

AnalyzeEyeTrackingData (MonoBehaviour)

The 1 Hz pump that drains the shared sample buffer into the native SDK and refreshes any bound TMP displays when new predictions land. Add one alongside AnalyzeEyeTrackingData. No methods to call from user code.

Inspector Fields

Name	Type	Description
EyeTrackingResponseTextMentalWorkload	TextMeshPro (optional)	Top display — refreshed when a new mental-workload prediction arrives. Leave unassigned to skip rendering.
EyeTrackingResponseTextFatigue	TextMeshPro (optional)	Top display — refreshed when a new fatigue prediction arrives. Independent of the mental-workload field; left untouched if fatigue never reports.
EyeTrackingResponseTextAttention	TextMeshPro (optional)	Top display — refreshed whenever the Attention pull advances (total_observations changes). Left untouched if Attention never reports.
EyeTrackingResponseTextMentalReadiness	TextMeshPro (optional)	Top display — refreshed whenever the Mental Readiness pull advances (total_predictions changes). Left untouched until the minimum session length is met.
EyeTrackingResponseTextLower	TextMeshPro (optional)	Rolling history panel — appends one line per new prediction, capped at the last 15 entries. Newest on top.

GazeDataSampleFactory (static class)

Builds Sample records consumed by AnalyzeEyeTrackingData.SubmitGazeSample. Hides the XY projection, per-session monotonic blink-ID iteration, and combined-pupil-diameter computation. The primary entry point is CreateGazeSample; EndSession is an optional cleanup for per-session blink-ID state.

CreateGazeSample(…)

public static Sample CreateGazeSample(
    string sessionId, long timestampNs,
    Vector3 leftEyeOrigin, Quaternion leftEyeOrientation, Vector3 rightEyeOrigin, Quaternion rightEyeOrientation,
    float fovDeg, bool isBlinking,
    double pupilDiameterLeft = 0.0,
    double pupilDiameterRight = 0.0);

Parameters:

Name	Type	Description
timestampNs	long	Monotonic timestamp in nanoseconds. Usually (long)(Time.timeAsDouble * 1e9).
leftEyeOrigin / leftEyeOrientation	Vector3 + Quaternion	Left eye origin in head-local space (metres) and gaze rotation. If unknown, the Sample’s leftEyeX/Y remain 0.
rightEyeOrigin / rightEyeOrientation	Vector3 + Quaternion	Right eye origin (metres) and gaze rotation, same shape as the left eye fields.
fovDeg, isBlinking	float + bool	Vertical FOV in degrees + isBlinking flag. Used internally to project gaze rays’s endpoint into screen pixel coordinates.
pupilDiameterLeft	double (optional)	Pupil diameter in mm. Pass 0 if not tracked.
pupilDiameterRight	double (optional)	Pupil diameter in mm. Pass 0 if not tracked.

Returns: Sample populated with timestampNs, left/right eye screen XY (pixels), optional pupil diameters, and a monotonic blinkId (1, 2, 3… per blink event; −1 when not blinking).

EndSession(string sessionId)

public static void EndSession(string sessionId);

Parameters: sessionId — the session UUID whose per-session blink-ID counter to clear.

Returns: void. Optional cleanup — only useful when many sessions start in one app run.

EyeTrackingAnalyzer

Thin C# wrapper around the native theia C API. Managed by AnalyzeEyeTrackingData; you do not create instances yourself, and normal integrations do not call its methods directly. Documented here for advanced users.

Initialize(string licenseKey)

public void Initialize(string licenseKey, string sessionId);

Validates the license, marshals the license key and output directory into unmanaged memory, invokes theia_sdk_create, and starts a session (theia_sdk_start_session). Idempotent. Called once by AnalyzeEyeTrackingData.Start(). Side effect: logs the SDK version, a keygen.sh reachability probe, and the size/roundtrip of the Options struct to the Unity console.

GetCurrentMentalWorkload(EyeTrackingData data)

public MentalWorkloadData GetCurrentMentalWorkload(EyeTrackingData data);

 

Parameter: the shared EyeTrackingData whose Samples buffer was populated since the last call. Returns: the most recent MentalWorkloadData the SDK has produced, or null if no batch has completed yet. Side effect: pushes the newly-accumulated samples to theia_sdk_add_gaze_samples in one call.

GetCurrentFatigue(EyeTrackingData data)

public FatigueData GetCurrentFatigue(EyeTrackingData data);

Same contract as GetCurrentMentalWorkload, but returns the latest fatigue prediction. All four getters call the same PushNewSamples path internally — you need only call one per 1 Hz tick (AnalyzeEyeTrackingData already does this).

GetCurrentAttention(EyeTrackingData data)

public AttentionData GetCurrentAttention(EyeTrackingData data);

Same contract as GetCurrentMentalWorkload, but returns the latest Attention pull (level, label, composite_score, total_observations). Attention is a pull metric (no native callback) — the value is freshened during the same PushNewSamples call that delivers Mental Workload + Fatigue via callback. Returns null until enough fixation/saccade observations have accumulated for a first composite to be computed.

GetCurrentMentalReadiness(EyeTrackingData data)

public MentalReadinessData GetCurrentMentalReadiness(EyeTrackingData data);

Same contract as GetCurrentAttention. Returns the percentage breakdown of Mental Workload predictions observed across the session (low/moderate/high) plus total_predictions. Returns null until the minimum session length is met. Use SetMentalReadinessMinSessionSeconds to override that gate during dev testing.

SetMentalReadinessMinSessionSeconds(double seconds)

public int SetMentalReadinessMinSessionSeconds(double seconds);

Override the native default minimum session length before Mental Readiness will return a result. Returns 0 on success, -1 if the SDK has not yet been created. Typical use is to drop the threshold during short development sessions; production code should leave it at the default.

IsRunning()

public bool IsRunning();

Returns true when the native SDK has an active session — i.e. theia_sdk_create has succeeded and theia_sdk_start_session has been called and not yet stopped. Returns false if no SDK handle exists yet (e.g. before AnalyzeEyeTrackingData.Initialize has run, or if license validation failed). Useful as a precondition check before submitting samples or polling results.

FeatureVersion()

public string FeatureVersion();

Returns the native SDK feature/model bundle version string. Independent of the analyzer’s overall SDK version (theia_sdk_version) — this reports the version of the feature/model bundle compiled into the native library. Returns null if the native call returns a null pointer. Useful for diagnostics and bug reports.

GetLicenseInfo()

public LicenseInfoSnapshot GetLicenseInfo();

Returns a snapshot of the validated license — id, key, status, expiry date, validated-at timestamp, cached flag, and entitlements — copied out of the native license_info_t struct as managed strings. Returns null if no SDK handle exists yet. Useful for surfacing license state in dev tools or admin UIs, and for confirming the active license at runtime.

LicenseInfoSnapshot (returned by GetLicenseInfo)

public class LicenseInfoSnapshot { public string LicenseId, LicenseKey, Status, ExpiryDate, ValidatedAt, Entitlements; public bool IsCached; }

Plain managed container for the license fields the native SDK returns. All string fields may be null if the native pointer is null. IsCached is true when the license info came from a local cache rather than a fresh server validation.

EyeTrackingData

Per-session sample container. One lives on AnalyzeEyeTrackingData.Instance. Collector scripts write into its Samples buffer; the analyzer drains it.

Properties

Name	Type	Description
TotalDuration	double	Seconds since session start. Updated by AnalyzeEyeTrackingData before each 1 Hz push.
Samples	SampleBuffer	The list-backed buffer collectors write to.

Methods

public void ClearData();

Clears the Samples buffer and resets TotalDuration. Called automatically by AnalyzeEyeTrackingData after each 1 Hz push.

SampleBuffer

Thin wrapper around a List<Sample>. Exposed so collectors and analyzer can share the same instance without copying.

Methods

Signature	Returns	Description
(internal — use SubmitGazeSample)	void	Appends one sample. Call this once per tracker frame from your collector.
GetSamples()	List<Sample>	Returns the live backing list; do not modify from outside the analyzer path.
Clear()	void	Empties the buffer. Normally called via EyeTrackingData.ClearData().

Sample

One frame of eye data in the neutral shape the analyzer consumes. Build these with GazeDataSampleFactory.CreateGazeSample; don’t populate them by hand unless you are writing a mock.

Fields

Name	Type	Description
timestampNs	long	Monotonic timestamp in nanoseconds relative to session start.
leftEyeX, leftEyeY	float, float	Left eye gaze point in virtual-screen pixels (0..1600, 0..1200). Zero during blink frames.
rightEyeX, rightEyeY	float, float	Right eye gaze point in virtual-screen pixels (0..1600, 0..1200). Zero during blink frames.
pupilDiameterLeft	double	Left pupil diameter in mm. 0 if unknown.
pupilDiameterRight	double	Right pupil diameter in mm. 0 if unknown.
pupilDiameter	double	Combined/average pupil diameter in mm. 0 if unknown.
blinkId	int	Monotonic per-blink ID during a blink (1, 2, 3…); -1 when not blinking. Iterated internally by GazeDataSampleFactory.CreateGazeSample.

MentalWorkloadData

Mirrors the native mental_workload_result_t shape. Retrieved via AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentMentalWorkload.

Fields

Name	Type	Description
level	int	0 = Low, 1 = Moderate, 2 = High.
confidence	float	Prediction confidence in [0, 1].
batch_number	int	Monotonically increasing window index — advances once per completed batch.
LevelName	string (property)	“Low” | “Moderate” | “High” | “Unknown” (computed from level).

Usage

var cog = AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentMentalWorkload;
if (cog != null) {
    Debug.Log($"CogLoad: {cog.LevelName} (conf {cog.confidence:F2}, batch {cog.batch_number})");
}

FatigueData

Mirrors the native fatigue_result_t shape. Retrieved via AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentFatigue.

Fields

Name	Type	Description
level	int	0 = Alert, 1 = Neither, 2 = RatherDrowsy, 3 = Drowsy.
confidence	float	Prediction confidence in [0, 1].
batch_number	int	Monotonically increasing window index.
LevelName	string (property)	“Alert” | “Neither” | “RatherDrowsy” | “Drowsy” | “Unknown”.

Usage

var s = AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentFatigue;
if (s != null) {
    Debug.Log($"Fatigue: {s.LevelName} (conf {s.confidence:F2}, ttt L{s.ttt_level}/{s.ttt_minutes}m, batch {s.batch_number})");
}

AttentionData

Mirrors the native attention_result_t shape. Retrieved via AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentAttention. The Attention model is rule-based (Nideffer ACT) — no XGBoost involved, no model file required.

Fields

Name	Type	Description
level	int	1 = Narrow, 2 = Mod-Narrow, 3 = Neither, 4 = Mod-Broad, 5 = Broad.
label	string	Pre-formatted label: “Narrow” | “Mod-Narrow” | “Neither” | “Mod-Broad” | “Broad”.
composite_score	double	Composite attention score in [0, 1]. 0 = fully narrow, 1 = fully broad.
total_observations	int	Number of fixation/saccade observations integrated so far. Edge-trigger UI updates when this advances.

Usage

var a = AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentAttention;
if (a != null) {
    Debug.Log($"Attention: {a.label} (score {a.composite_score:F2}, observations {a.total_observations})");
}

MentalReadinessData

Mirrors the native mental_readiness_result_t shape. Retrieved via AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentMentalReadiness (populated each 1 Hz analyzer tick after the minimum session length is met. Overridable via EyeTrackingAnalyzer.SetMentalReadinessMinSessionSeconds). Mental Readiness is a derived metric — it aggregates Mental Workload predictions across the session into a percentage breakdown. The three percentages sum to 100.0 exactly.

Fields

Name	Type	Description
low_percentage	double	Percent of Mental Workload predictions at level 0 (Low). 0–100.
moderate_percentage	double	Percent at level 1 (Moderate). 0–100.
high_percentage	double	Percent at level 2 (High). 0–100.
total_predictions	int	Total Mental Workload predictions counted. Edge-trigger UI updates when this advances.

Usage

var mr = AnalyzeEyeTrackingData.Instance.EyeTrackingAnalyzer.CurrentMentalReadiness;
if (mr != null) {
    Debug.Log($"Mental Readiness ({mr.total_predictions} predictions): " +
              $"low {mr.low_percentage:F1}%, mod {mr.moderate_percentage:F1}%, high {mr.high_percentage:F1}%");
} else {
    Debug.Log($"Mental Readiness not yet available — need {a bit longer.");
}