Miranda – The Eye Tracking Toolkit

Miranda is a toolkit for calibrating eye- and head-tracker inputs to match your screen gaze. It enables integration with other applications, allowing you to use your calibrated tracker data in various ways.

For example, you can use live camera footage of your eye as an input, calibrate your eye movements to your screen, and then move the mouse cursor to the point you are looking at, or publish the position via UDP for further use.

Prerequisites for Linux

On Linux systems, install espeak-ng:

apt install espeak-ng

This is not needed for Windows.

Run Miranda

Python 3.11.14 is suggested.
Load the dependencies and run main.py:

pip install -r requirements.txt
python main.py

Alternatively, you may also use uv:

uv run --with-requirements requirements.txt main.py

Build Miranda

pip install PyInstaller
PyInstaller .\Miranda.spec

This was tested on Manjaro 26.0.2, Ubuntu 24.04 LTS, Windows 10 and 11.

How it works

There are three components: inputs, tracking approaches, and outputs.
An input provides eye-tracking data. This data is translated into screen coordinates using a tracking approach, and these coordinates are then used or published through an output. Each input and tracking approach combination requires prior calibration.

Inputs

An input is where eye- and head-tracking data comes from. The data could be the yaw and pitch rotation of your eyes in degrees. An input may be an integrated eye-tracking functionality of Miranda, or may be an external application that needs to run alongside Miranda. Since the data itself gives no indication of where the user is looking at we need a tracking approach.

The inputs are:

• Integrated Inputs:
  • Mouse Position: The mouse position as input. Great for testing.
  • Eye-Tracking Glasses: Eye-Tracking using "Eye-Tracking glasses" with an infrared camera in front of the eye.
  • Webcam Head-Tracking: Head-Tracking with just using a Webcam.
• Inputs requiring an external application:
  • OpenTrack: The rotation of your head with OpenTrack.
  • Pupil: Pupil Lab's 3d-eye detection.
  • EyeTrackVR: Eye tracking with EyeTrackVR.
  • Orlosky: The 3DEyeTracker from Jason Orlosky.

Tracking Approach

A tracking approach tells how the data from an input method shall be translated into screen coordinates – a position on the screen, e.g. where a mouse cursor could move to.

There are two approaches:

• Gaze on Screen: The user is directly looking at the screen. The eye movement directly translates to a screen position. This is the most straight-forward approach.
• D-Pad: This approach is a good alternative if your input is not accurate enough for the Gaze on Screen approach. Usually, a D-pad is a flat, typically thumb-operated, directional control. Likewise with this approach the current screen coordinates is steered by looking at a d-pad. E.g. by looking at the "up" arrow, the screen coordinates moves up.

Calibration

Before we can translate the data from the input method into screen coordinates, we need to do a calibration first. Every input method and tracking approach combination needs its own calibration. Once such a calibration is done the result will be stored and is available on the next start of Miranda.

Outputs

Outputs take the screen coordinates created by the tracking approach and make use of them.

The outputs are:

• UDP-Export: Publish the gaze results over UDP in a simple JSON format.
• Mouse Movement: Moves the mouse cursor according to the gaze.
• TTS Keyboard: A text-to-speech-keyboard. (Proove-of-concept)

Open Source License Attribution

This application uses open source components. You can find the source code of their open source projects along with license information below. We acknowledge and are grateful to these developers for their contributions to open source.

Libraries

• MediaPipe, Copyright (c) The MediaPipe Authors, Apache License 2.0
• msgpack, Copyright (C) 2008-2011 INADA Naoki, Apache License 2.0
• NumPy, Copyright (c) 2005-2025 NumPy Developers, BSD 3-Clause License
• opencv-python, Copyright (c) Olli-Pekka Heinisuo, MIT License
• pupil-detectors, Copyright (c) Pupil Labs, LGPL-3.0 License and GPL-3.0 License
• PyAutoGUI, Copyright (c) 2014 Al Sweigart, BSD 3-Clause License
• pye3d, Copyright (c) Pupil Labs, LGPL-3.0 License
• python-osc, Copyright (c) python-osc contributors, The Unlicense
• pyttsx3, Copyright (c) nateshmbhat and contributors, Mozilla Public License 2.0
• screeninfo, Copyright (c) Marcin Kurczewski, MIT License
• PyZMQ, Copyright (c) Brian E. Granger & Min Ragan-Kelley, BSD 3-Clause License

Others

• EyeTracker, Copyright (c) 2024 JEOresearch, MIT License

License

Miranda is distributed under the terms of the GPL version 3.0 or any later version.