Start Here — The Simple Version

What is this about — in one breath? #

Companies collect huge amounts of data.

But the data usually sits in separate piles (sales in one place, fraud in another, support in another, sensors somewhere else).

When something important happens in the real world, the clues are often spread across those piles. People have to dig through each pile by hand, connect the dots, write a slide, and then… forget the connection.

This whitepaper proposes a new kind of data system — a Data Swimming Pool — where those connections are found automatically, kept, and used to warn you, recommend actions, or (carefully) act.

The swimming pool metaphor (this is the whole idea) #

Today’s popular systems (warehouses, lakes, lakehouses, meshes) are excellent at storing and querying data. They are weak at remembering relationships between events across departments as a first-class job of the platform.

That gap is what this document is about.

A story you already understand #

Imagine a bank customer:

1. Their card is declined twice at unusual hours.
2. Their account balance drops below a threshold.
3. They call the call centre sounding stressed.
4. Their phone login location jumps cities overnight.

Each of those signals may look “fine enough” on its own.

Together, they might mean:

• Account takeover (fraud), or
• Financial distress (the customer needs help, not a hard block).

If four different teams each see only one signal, nobody sees the full picture in time.

A Data Swimming Pool is designed so the platform itself notices: these things belong together, scores how confident it is, shows its evidence, and routes a useful action — with rules about how much the machine is allowed to do alone.

What the system actually does (five steps) #

flowchart LR
    A["1. Data arrives<br/>from many systems"] --> B["2. Clean + identify<br/>who/what it is about"]
    B --> C["3. Continuously look<br/>for meaningful links"]
    C --> D["4. Turn links into<br/>clear insights"]
    D --> E["5. Alert / recommend<br/>/ carefully act"]

    classDef step fill:#0d4159,stroke:#22b3d6,color:#eaf9ff
    class A,B,C,D,E step

Figure S-1. The idea in five steps. Everything else in the whitepaper is detail about how to do each step safely at enterprise scale.

What is new here? (and what is not) #

Not new: storing data, streaming events, machine learning, dashboards, knowledge graphs as catalogs.

Proposed as new (the heart of the paper):

1. Treat relationships between events as lasting platform assets (not one-off spreadsheet joins).
2. Give each relationship a confidence score, an expiry date, and governance rules.
3. Make correlation scope (what you’re allowed to connect, why, and for how long) a managed object — like a product, not a side script.
4. Ground AI answers in those verified relationships, not free-floating guesses.
5. Use a clear ladder of autonomy so machines only act as far as the action can be reversed.

Why should a normal person care? #

Because modern organisations lose money, trust, and sometimes lives when weak signals never meet.

You do not need to understand Apache Flink to understand why that matters.

How to read this site without drowning #

Treat this GitBook like a website with deep pages, not a novel you must finish.

If you have 10 minutes #

1. This page (you’re on it)
2. 1. Executive Summary — skim the bold lines and the comparison table
3. Jump to your industry in Part IV (Banking, Healthcare, etc.)

If you have 30 minutes #

Add:

4. 8. Introducing Data Swimming Pool
5. 37. Comparison — what this is vs what already exists
6. 39. Challenges — why you might not build this

If you want the technical meat #

Follow How to Read This Whitepaper — Architect path.

Tiny glossary (only the words you’ll keep seeing) #

A fuller glossary lives in Appendix A.

Three promises this paper makes to you #

1. It will not pretend the idea is already proven in production. It isn’t.
2. It will say when not to build this. Maturity, privacy, and cost matter.
3. It will keep the metaphor honest. Circulation without filtration is just a mess with better branding.

Ready? Pick a door #