RETINA.
An AI-powered assistive system for the visually impaired.
A native Android application combining on-device text recognition, Indian currency detection, everyday object recognition, and obstacle depth awareness with conversational speech and directional audio.
# 01 // ARCHITECTURE & DATA FLOW
CameraX Stream
Non-blocking YUV buffer
Frame Dispatcher
Orientation, crop & RGB
On-Device TFLite
OCR · YOLOv8n · MiDaS
Audio & TTS
Spatial cues & alerts
CameraX captures video frames offloaded to an asynchronous background executor. The active module processes frames via quantised on-device models, running 100% offline to guarantee privacy and zero network latency before vocalizing results via Android TTS.
# 02 // ASSISTIVE MODULES
Google ML Kit on-device OCR reads packaging, documents, and signs aloud. Implements Levenshtein string hashing to deduplicate speech and prevent repetitive audio loops when pointed at stationary text.
Custom YOLOv8n TFLite model detecting 6 Indian Rupee denominations (₹10, ₹20, ₹50, ₹100, ₹200, ₹500). Trained on 12,050 annotated images with custom NMS and an interactive session totalizer.
EfficientDet Lite0 detects everyday indoor items, mapping bounding box centroids to horizontal sectors (Left, Center, Right) for spatial speech guidance (e.g., "Chair on your left").
MiDaS v2.1 Small produces monocular relative depth maps, divided into 3 navigational zones. Triggers acoustic proximity alert tones when close obstacles threaten collision paths.
# 03 // BENCHMARKS & ARCHITECTURAL CHOICES
| Module | Selected Model | Key Metrics | Design Rationale |
|---|---|---|---|
| OCR | ML Kit Text v2 | 96.4% char acc · 120ms | Runs fully offline with zero cloud latency or data privacy exposure. |
| Currency | YOLOv8n TFLite | 0.96 F1 · 0.94 mAP@0.5 | <12MB footprint; custom anchors handle thin rectangular note geometries. |
| Objects | EfficientDet Lite0 | 0.93 F1 · 0.91 mAP@0.5 | Compound scaling balances mobile CPU throughput with everyday indoor detection. |
| Obstacles | MiDaS v2.1 Small | 52ms CPU latency (84% acc) | 8.6× faster than MiDaS Large (450ms), prioritizing real-time walking safety. |
# 04 // CHALLENGES & ACCESSIBILITY
Inference at 30 FPS caused buffer queues and frame lag. Solved using CameraX's STRATEGY_KEEP_ONLY_LATEST, discarding intermediate frames while the CPU is busy so the user always receives feedback from the freshest visual state.
Eliminated reliance on visual touch buttons through full-screen two-finger swipe gestures for module switching, tactile haptic clicks on registration, and immediate spoken audio confirmation for all system state changes.
Retina: An AI Powered Assistive System for the Visually Impaired
Presented at the International Conference on Frontiers in Intelligent Systems & Networks (Sree Buddha College of Engineering, March 2026), detailing multi-model on-device accessibility architecture.