A case study on creating a data product from scratch and scaling it across multiple operator clients.

The Problem

GiG's operator clients — the casino, sportsbook, and lottery brands running on GiG's platform — wanted to send personalised messages to players based on what those players were doing right now. Not what they did yesterday. Not what showed up in a report this morning. Right now.

For example, a deposit event can be useful to a CRM workflow, while signs of potentially harmful play can matter to a responsible-gaming workflow. These illustrate why fresh context is useful; the actions taken depend on the consumer’s rules and safeguards.

The problem wasn't that the data was missing — it was that it was always stale by the time anyone could use it. Traditional CRM platforms were fed the old-fashioned way: batch uploads, typically a daily file dropped to SFTP. So operators were always a day behind, segmenting and messaging players off yesterday's snapshot. That's fine for a scheduled newsletter, but it makes every "right now" use case above impossible — you can't react to a deposit, a losing streak, or a live browsing session with data that only lands once a night.

Real-time CRM had been recognised as a gap and made it onto the roadmap — and I was the one who took it from there, owning the build end to end.

The shape of it:

• Source — player events streaming through Kafka: registrations, profile updates, deposits, withdrawals, casino game activity, sports activity, logins, and more.
• Enrichment — each triggering event was used to query back into ClickHouse for the fuller player context. That query was the contract.
• Routing — the enriched records were published back onto Kafka, where different consumers filtered the stream for their own use case.
• Delivery — CRM platforms (Symplify, FastTrack, Intellitics) consumed the enriched feed, each operator taking the events relevant to them.
• Monitoring — production alerting on delivery latency, throughput, and failure rates.

Here's how it actually came together.

The core pattern: the query is the contract

The whole thing was built on NiFi, sitting between Kafka and ClickHouse.

A raw event landing on Kafka — a deposit, a casino spin, a registration — doesn't carry enough context on its own to drive a personalised action. You know something happened, but not who the player is, what their history looks like, or what segment they fall into. So the moment a triggering event arrived, NiFi took the player metadata and the event type and used them to run an enrichment query against ClickHouse.

That query was the contract. The shape of its result set defined exactly what data a given event type produced downstream. Adding a new event type, or changing what a consumer received, was a matter of defining or adjusting a query — not rebuilding the pipeline. Once ClickHouse returned the enriched result set, NiFi published it straight back onto Kafka.

Why this gave multi-tenancy — and more than CRM

Because every enriched record went back through Kafka, the same flow could serve many consumers at once, each filtering for what it needed. That's what made the multi-tenancy work: multiple operators across casino, sports, and lottery could draw the slices relevant to them off a single shared pipeline, rather than each getting a bespoke build.

It also meant the pipeline wasn't really a CRM pipeline. CRM and campaigning were simply the first consumers. The same enriched event stream could drive payment transaction monitoring, gameplay analysis, and other operational needs — anything that needed live, enriched player events could subscribe to the relevant topic. I set out to solve a marketing problem and ended up building a general-purpose, real-time enrichment backbone.

Scaling to 300k events per hour

Enrichment was a bottleneck because it queried ClickHouse for player context. The pipeline handled 300,000 events per hour across multiple clients, so both that query workload and the volume sent downstream mattered.

The second problem was bursts. A single player in a casino session can fire several events within a couple of seconds. Pushing each one individually through enrichment and out to downstream systems would hammer both ClickHouse and the CRM platforms receiving the feed. For gaming activity, we aggregated events rather than sending a separate message for every bet. The resulting payload preserved the correct activity counts and amounts. That reduced message volume while retaining the totals downstream consumers needed.

The architecture

The Outcome

Broker CRM scaled from zero to 300,000 events per hour, serving multiple operator clients across casino, sports betting, and lottery verticals, with enriched activity delivered through the shared feed.

For operators, the feed provided fresher player context than the daily file-based process. What happened next depended on the receiving system and its configured workflows.

For GiG, the integration expanded the platform’s CRM offering. The same enriched stream could also support consumers beyond CRM, giving the architecture a use beyond the initial marketing brief.

What I Learned

Real-time isn't just faster — it unlocks things batch can't. Moving operators off nightly SFTP uploads onto a live feed wasn't a speed upgrade, it was a change in what was possible. A daily snapshot can power a scheduled newsletter; a fresher event feed supports workflows that need more current context. The latency wasn't a nice-to-have — it was the entire product.

The leverage was in the abstraction, not the throughput. Serving multiple operators with different needs — without standing up a separate pipeline for each — came down to one decision: making the ClickHouse enrichment query the contract and letting Kafka handle the routing. Get that right and a single pipeline serves many operators, and, as it turned out, many use cases the original brief never mentioned. That abstraction mattered far more than chasing raw speed.

Owning the full lifecycle changes how you build. I took this from a roadmap line to a production system — stakeholder coordination, third-party integrations, development, deployment, and the on-call that came after. When you're the one paged at 2am for the system you built, you design it differently: you think harder about failure modes, you build better alerts, and you write documentation your future self will actually use.

I'm Julian Calleja, focused on real-time data platforms in iGaming. Get in touch if you want to talk about data infrastructure or iGaming.