What this project does
This derivative of Henry Yuan’s project replaces board-side AirPlay with user-authorized ScreenCaptureKit capture and USB input with Bluetooth HID from a separately powered ESP32-S3. Midscene 1.16.1 runs in a local WKWebView, with Swift handling model requests and input. Screenshots go to the configured model; local capture does not imply offline operation. Selected step JPEGs and traces are saved, rather than continuous video.
Based on: ESP32 iPhone Agent by Henry Yuan
Limits and verification
This is a community fork, not an upstream release. The author documents an October 25 World Clock task, not a compatibility or reliability study; this site has not repeated it. At this revision, public CI run 40447079087 passed firmware and JS checks but failed the iOS simulator test step. Portrait, ASCII and iOS background scheduling remain constraints. Update both fork app and firmware; upstream AirPlay firmware is not interchangeable.
Reviewed sources
Reviewed READMEf1787f4