A case study on creating a data product from scratch and scaling it across 10+ operator clients.

The Problem

GiG's operator clients — the casino, sportsbook, and lottery brands running on GiG's platform — wanted to send personalised messages to players based on what those players were doing right now. Not what they did yesterday. Not what showed up in a report this morning. Right now.

A player who just deposited €500 should get a different experience than someone who hasn't logged in for a week. A player on a losing streak might need a responsible gaming intervention. A high-value player browsing a new game category should see a relevant bonus offer.

The problem wasn't that the data was missing — it was that it was always stale by the time anyone could use it. Traditional CRM platforms were fed the old-fashioned way: batch uploads, typically a daily file dropped to SFTP. So operators were always a day behind, segmenting and messaging players off yesterday's snapshot. That's fine for a scheduled newsletter, but it makes every "right now" use case above impossible — you can't react to a deposit, a losing streak, or a live browsing session with data that only lands once a night.

Real-time CRM had been recognised as a gap and made it onto the roadmap — and I was the one who took it from there, owning the build end to end.

The shape of it:

• Source — player events streaming through Kafka: registrations, profile updates, deposits, withdrawals, casino game activity, sports activity, logins, and more.
• Enrichment — each triggering event was used to query back into ClickHouse for the fuller player context. That query was the contract.
• Routing — the enriched records were published back onto Kafka, where different consumers filtered the stream for their own use case.
• Delivery — CRM platforms (Symplify, FastTrack, Intellitics) consumed the enriched feed, each operator taking the events relevant to them.
• Monitoring — production alerting on delivery latency, throughput, and failure rates.

Here's how it actually came together.

The core pattern: the query is the contract

The whole thing was built on NiFi, sitting between Kafka and ClickHouse.

A raw event landing on Kafka — a deposit, a casino spin, a registration — doesn't carry enough context on its own to drive a personalised action. You know something happened, but not who the player is, what their history looks like, or what segment they fall into. So the moment a triggering event arrived, NiFi took the player metadata and the event type and used them to run an enrichment query against ClickHouse.

That query was the contract. The shape of its result set defined exactly what data a given event type produced downstream. Adding a new event type, or changing what a consumer received, was a matter of defining or adjusting a query — not rebuilding the pipeline. Once ClickHouse returned the enriched result set, NiFi published it straight back onto Kafka.

Why this gave multi-tenancy — and more than CRM

Because every enriched record went back through Kafka, the same flow could serve many consumers at once, each filtering for what it needed. That's what made the multi-tenancy work: 10+ operators across casino, sports, and lottery could draw the slices relevant to them off a single shared pipeline, rather than each getting a bespoke build.

It also meant the pipeline wasn't really a CRM pipeline. CRM and campaigning were simply the first consumers. The same enriched event stream could drive payment transaction monitoring, gameplay analysis, and other operational needs — anything that needed live, enriched player events could subscribe to the relevant topic. I set out to solve a marketing problem and ended up building a general-purpose, real-time enrichment backbone.

Scaling to 300k events per hour

The bottleneck was the enrichment itself. Every triggering event meant a query back into ClickHouse, and at 300,000 events an hour that query load is the thing that decides whether the pipeline holds up.

The second problem was bursts. A single player in a casino session can fire several events within a couple of seconds. Pushing each one individually through enrichment and out to downstream systems would hammer both ClickHouse and the CRM platforms receiving the feed. So events had to be batched (i.e here we wre deliberately filtering out messages) — grouping bursts before enrichment and delivery so the downstream systems were never overwhelmed by a handful of fast-moving players. Getting that batching right, rather than chasing raw per-event speed, was what let it scale cleanly.

The architecture

The Outcome

Broker CRM scaled from zero to 300,000 events per hour, serving 10+ operator clients across casino, sports betting, and lottery verticals, with enriched events reaching downstream consumers within seconds.

For operators, this meant they could trigger personalised player actions in real time for the first time — bonus offers based on live behaviour, responsible gaming interventions at the right moment, and re-engagement campaigns triggered by actual inactivity patterns rather than batch reports.

For GiG, it became a product differentiator. Real-time CRM integration was a selling point competitors didn't have, and it gave existing clients a reason to deepen their use of the platform. And because the same flow could feed operational use cases beyond CRM, the investment paid off well past the problem it was built to solve.

What I Learned

Real-time isn't just faster — it unlocks things batch can't. Moving operators off nightly SFTP uploads onto a live feed wasn't a speed upgrade, it was a change in what was possible. A daily snapshot can power a scheduled newsletter; only a stream measured in seconds can drive a responsible-gaming intervention or a bonus offer while the player is still in the session. The latency wasn't a nice-to-have — it was the entire product.

The leverage was in the abstraction, not the throughput. Serving 10+ operators with different needs — without standing up 10 separate pipelines — came down to one decision: making the ClickHouse enrichment query the contract and letting Kafka handle the routing. Get that right and a single pipeline serves many operators, and, as it turned out, many use cases the original brief never mentioned. That abstraction mattered far more than chasing raw speed.

Owning the full lifecycle changes how you build. I took this from a roadmap line to a production system — stakeholder coordination, third-party integrations, development, deployment, and the on-call that came after. When you're the one paged at 2am for the system you built, you design it differently: you think harder about failure modes, you build better alerts, and you write documentation your future self will actually use.

I'm Julian Calleja, focused on real-time data platforms in iGaming. Get in touch if you want to talk about data infrastructure or iGaming.