Native phone appScreen-control agent

ESP32 iPhone Agent / Handsy

A native iPhone agent with board-side AirPlay reception and USB mouse input.

What this project does

Henry Yuan’s project moves the agent loop onto the iPhone. The reviewed public source uses an ESP32-S3 to receive AirPlay, relay video to the app, and deliver USB input.

Limits and verification

AirPlay must originate from an iPhone; macOS mirroring is unsupported. This revision adds a no-video diagnostic, not Mac support. Video still uses BLE, and the announced Wi-Fi video build is not included. Portrait and ASCII limits remain; this site has not tested the hardware.

Reviewed sources

Reviewed READMEd986bedRead our ESP32 iPhone Agent build notes
Hardware
ESP32-S3 with PSRAM and native USB
Runs on
iPhone app; computer for building and signing
Screen / observation
AirPlay → ESP32-S3 → BLE → app
Input path
USB HID mouse and keyboard
Setup requirements
iOS 18+, Xcode, Arduino firmware and a vision-model endpoint
License information
MIT; AirPlay components GPL-3.0