Building Rakshak: real-time crowd safety with YOLOv8

Rakshak started with a grim observation: most stampede tragedies aren't caused by panic out of nowhere. The crowd gets denser, movement gets more constrained, and the warning signs are physically present in the scene for minutes before anything goes wrong — nobody is watching the right signal. I wanted to build the thing that watches that signal in real time and raises its hand before the crush.

The naive framing is “count people in a video.” That's the easy 20%. The hard, interesting 80% is turning a pile of bounding boxes into a trustworthy NORMAL / WARNING / CRITICAL decision that a human operator would actually believe.

The pipeline, end to end

Rakshak is four stages chained together, each feeding the next:

• Detect — find every person in the frame.
• Track — give each person a stable identity across frames.
• Score — turn positions and motion into a risk number.
• Forecast — project that risk 30 minutes ahead.

1. Detection: YOLOv8 + a preprocessing trick

Detection runs on YOLOv8. In a packed frame you're dealing with 200 to 500+ people, heavy occlusion, and wildly uneven lighting — half the plaza in harsh sun, half in shadow. Raw frames tank recall in the dark regions.

The fix that mattered most was CLAHE (Contrast Limited Adaptive Histogram Equalization) as a preprocessing step. It equalizes contrast locally rather than globally, so shadowed clusters get pulled out without blowing out the bright areas. Recall on the hard regions jumped noticeably once CLAHE was in the loop.

2. Tracking: ByteTrack / BoT-SORT across cameras

Detection alone is memoryless — it can't tell you a crowd is compressing, because it has no notion of the same person over time. So every detection is handed to a tracker (ByteTrack and BoT-SORT) that assigns persistent IDs. ByteTrack's trick of also associating low-confidence boxes is perfect here: in a dense crowd, a half-occluded person is still a person, and throwing those boxes away is how you lose track of exactly the region that's getting dangerous.

With stable IDs across multiple cameras, I can finally compute the things that actually predict danger: per-person velocity, how much personal space each person has, and how that space is shrinking.

3. The risk engine: 7+ metrics into one state

This is the heart of the system, and the part that isn't “just call a model.” The risk engine fuses several physical signals, including:

• Density — people per square metre in each grid cell (a heatmap).
• Compression — how fast local density is rising.
• Velocity variance — when some people surge while others stall, pressure builds.
• Direction entropy — chaotic, conflicting flow is far more dangerous than everyone moving the same way.

Each metric is normalized and combined into a single score, which is thresholded into NORMAL, WARNING, or CRITICAL. Crucially, high density on its own is not automatically critical — a packed but smoothly flowing corridor is fine. It's density plus rising compression plus conflicting flow that trips the alarm. That combination is what keeps false positives from crying wolf.

State changes are pushed to operators over a WebSocket so the dashboard updates the instant a zone escalates — no polling lag between the scene and the alert.

4. Forecasting: seeing 30 minutes ahead

A warning at the moment of danger is already late. Rakshak keeps a rolling history of each zone's metrics and looks for the temporal-spatial patterns that precede a crush, comparing the live trend against historical baselines and flagging z-score anomalies — a zone drifting several standard deviations off its normal rhythm. That gives roughly a 30-minute runway to redirect flow or open exits while it's still cheap to act.

What I'd tell my past self

• The model is the smallest part. Detection was a weekend; the risk logic and validating its thresholds against real crowd footage was most of the project.
• False alarms are the actual enemy. An operator who gets three bogus CRITICALs stops trusting the fourth. Tuning for precision on the alarm was worth more than another point of detection recall.
• Physics beats vibes. Grounding the score in measurable quantities — density, compression, entropy — made it explainable, and an explainable alert is one a human will act on.

The stack, if you want the receipt: Python, YOLOv8, ByteTrack/BoT-SORT, OpenCV, FastAPI, MongoDB, and WebSockets tying the live alerts to the dashboard.