Real-Time is a Different Discipline

Most data engineering and ML work is batch processing. You collect data, process it overnight, deliver insights in the morning. Latency is measured in hours or minutes. Retry logic handles failures. This is not how live sports works.

In live sports, latency is measured in milliseconds. Hawk-Eye's AWS architecture maintains "sub-second latency for live operations" while processing 480 messages per second. A delay of 100ms can make the difference between a usable system and a frustrated user. A delay of 1 second breaks real-time officiating.

The current state of sports streaming illustrates the challenge. Average streaming latency sits around 9 seconds. The industry target is 5 seconds within three years - still far from instantaneous. For betting applications, even 5 seconds is problematic when odds change in real-time.

Technologies exist but require expertise. WebRTC delivers sub-500ms latency but struggles at scale. Low-Latency HLS and LL-DASH using CMAF can achieve lower latencies for large events but require careful implementation. An IBC Accelerator project achieved 1.8-second latency on 4K live streams through months of optimisation.

The skillset differs from batch processing. Real-time engineers think about message queues, event streaming, state management under failure conditions, and graceful degradation. They optimise for worst-case latency, not average throughput. They build systems that continue operating when components fail.

Apache Flink exemplifies the tooling. Sports betting platforms use Flink to "ingest and process live feeds in real time, enabling immediate odds recalculations, anomaly detection, and contextual personalization with high reliability and fault tolerance." The emphasis on reliability and fault tolerance reflects real-time requirements.