AI-Powered Fitness Rep Tracking & Movement Intelligence
Real-time computer vision that watches a standard camera feed, classifies the exercise, and counts the reps — no wearables, no sensors, no hardware beyond what the gym already has.
RepSense is a real-time AI movement intelligence platform built for a fitness-tech client (NDA). It tracks 33 skeletal landmarks per frame using MediaPipe pose estimation, computes biomechanical joint angles, and runs a multi-signal state machine to count repetitions for four distinct exercises — all on a single laptop with a standard camera feed and zero wearables.
Manual rep counting doesn't scale — and wearables hit a wall.
Gym operators were burning staff time watching members count reps and check form. Wearable-based alternatives stalled on three fronts: friction (members refused to strap on devices), cost (hardware procurement and maintenance for hundreds of members was uneconomic), and accuracy (off-the-shelf fitness wearables couldn't reliably distinguish exercise types). The client needed software that could watch, understand, and count — using cameras the gym already owned.
- No wearables, no extra hardware — works on existing camera feeds
- > 92% rep counting accuracy across four exercise types
- Sub-5% false-positive rate via Kalman filtering on noisy signals
- Real-time at 30fps on a standard laptop — $0/month infrastructure
How we built it.
Skeletal landmark extraction
MediaPipe processes every frame and returns normalised (x, y) coordinates for 33 body landmarks — shoulders, elbows, wrists, hips, knees, ankles.
Joint angle computation
For each exercise, the system computes the angle formed by three relevant landmarks (e.g. shoulder → elbow → wrist for curls) using arctangent trigonometry, in degrees.
State machine rep counting
Each exercise has calibrated angle thresholds defining "up" and "down" positions. Rep count only increments on a confirmed full-range cycle — partial movements are rejected.
Kalman filtering on chest fly
Chest fly uses horizontal wrist distance instead of joint angle. A Kalman filter smooths the noisy signal in real time, so natural arm tremor stops triggering phantom reps.
The solution.
33-point pose estimation pipeline
MediaPipe extracts a full skeletal landmark set per frame, giving us a normalised, camera-agnostic representation of the user's body to reason over.
Biomechanical angle engine
Shoulder, elbow, wrist, hip, knee, and ankle triplets are converted into joint angles with NumPy trigonometry — the substrate every exercise classifier sits on top of.
Multi-exercise classifier
Four movement types covered out of the box: dumbbell curl (bilateral), push-up (bilateral elbow), squat (knee angle), and chest fly (wrist-distance + Kalman).
Real-time operator UI
Tkinter interface with video input, frame overlay, live rep counter, and a session log window — multithreaded so the UI never blocks on the pose pipeline.
Configurable angle thresholds
Up/down thresholds per exercise are calibrated and tunable, so the system adapts to real-world variation in range of motion across users.
Exportable session logs
Per-exercise rep counts are written to a session log a trainer can review or export — closing the loop between member workout and coach feedback.
Inside the build.
Outcomes that shipped.
Shipped a working RepSense build in two weeks, from requirements to live demo. > 92% rep counting accuracy on clean camera angles, < 5% false-positive rate, 30fps real-time processing, and $0/month infrastructure cost — the whole system runs on a single laptop with no cloud dependencies. The Kalman filter on chest fly was the engineering decision that made the system trustworthy: naive distance-based counting produced too many phantom reps to ship.
What we took with us.
- 01Pose estimation + a state machine outperforms wearables on cost, friction, and accuracy at the same time — provided you treat signal smoothing as a first-class concern.
- 02Picking the right per-exercise signal (angle vs. distance) mattered more than the model. Chest fly needed a different signal pipeline than curls.
- 03Two weeks to a working demo only worked because the entire pipeline ran on-prem — no cloud, no inference latency, no infra debt.
Want something like this?
The tiers below are how we'd quote a build in the same space.
Have a build that needs this level of thinking?
Free 30-min audit. We'll tell you what to build, what it'd cost, and whether we're the right shop.