LiftSense
Vision-primary lift occupancy classifier with advisory elevator dispatch.
A wrong elevator call is a safety event, so the system is built to send nothing until it is certain — and even then, the certified lift controller decides.
Facts
- Classifies HUMAN, CARGO, MIXED, and EMPTY from a single USB camera feed
- Pipeline is camera → RF-DETR on TensorRT → centroid tracker → sliding-window median smoother → weighted-vote fusion → safety guards → dispatch adapter
- DISPATCH_MODE defaults to shadow, so no command reaches a lift until it is switched on deliberately
- Rating guard restricts CARGO and MIXED calls to freight-capable groups; a confidence floor and a kill switch sit in front of every send
- Dispatch is vendor-neutral behind a DispatchAdapter, with KONE elevator-websocket stream-v2 as the first implementation
- Audit trail stores events in Postgres and never stores images
- v0.1.0, phases 1–2 complete, 41 unit tests passing, ruff clean
Stack
- Python
- TypeScript
- RF-DETR
- TensorRT
- NVIDIA Jetson Orin
- Postgres
- Mosquitto MQTT
- Grafana
- Docker Compose
RF-DETR was chosen over the more common YOLO detectors for a licensing reason rather than an accuracy one: it is Apache-2.0, and a product that might one day ship inside a building’s lift controller cannot carry AGPL obligations.
The interesting engineering is not the detector. It is everything placed between the detector and the lift: a tracker and a median smoother so a single noisy frame cannot move a decision, a weighted vote across signals, and then a row of guards that each have the authority to refuse. Shadow mode is the default, which means the honest description of the system’s first release is that it watches, decides, records what it would have done, and sends nothing.