Auto-Aiming Pan/Tilt Turret

Building a responsive electromechanical tracker requires closing the loop between perception latency and actuator bandwidth. Detection delay, network/serial latency, and mechanical lag all compound into pointing error.

• —Sustain ~60 FPS object tracking
• —Keep detection latency under 50 ms
• —Keep motor command latency under 10 ms
• —Support predictive tracking and manual override
• —Expose live PID tuning through a dashboard

Pipeline:

1. Camera capture → YOLOv8 detection
2. Bounding-box centroid extraction
3. Kalman filter prediction of target motion
4. PID computation of pan/tilt setpoints
5. Serial commands to Teensy 4.1 actuators

A Flask dashboard provides mode selection, live PID gains, and system monitoring.

Two-axis pan/tilt mechanics on custom 3D-printed structure (black cage + red decks), driven by NEMA steppers through STEPPERONLINE CL42T closed-loop drivers and commanded from a Teensy 4.1.

Build path: motor/driver bring-up on the bench → print the mechanical stack → assemble the belt-driven tilt axis and camera payload. Homing and motion-stabilization routines keep the platform calibrated for closed-loop tracking.

V1 proved the motion stack; the final build closes the vision → Kalman → PID loop and tracks a hand in real time (see Videos).

• —Vision process runs detection and filtering
• —Control process issues low-latency actuator commands
• —Dashboard process exposes telemetry and live pan/tilt PID gain sliders
• —Manual and automatic modes share the same command path

• —Kalman filtering to bridge detection gaps and reduce jitter
• —Embedded PID on Teensy for deterministic actuator timing
• —Closed-loop stepper drivers for cleaner tracking under load
• —Flask dashboard for rapid experiment iteration

Validated tracking frame rate, detection latency, and command latency during live target motion (including hand tracking). Homing and stabilization routines were exercised before closed-loop runs.

Achieved approximately 60 FPS object tracking with less than 50 ms detection latency and less than 10 ms motor-command latency on the Teensy actuator path.

• —Balancing model accuracy against latency
• —Mechanical backlash and axis coupling
• —Keeping serial command streams non-blocking

End-to-end latency budgeting matters more than peak FPS. Predictive filtering significantly improved smoothness when detections momentarily dropped.

• —Multi-target selection
• —On-device inference optimization
• —Improved mechanical stiffness and encoder feedback