Knowledge graphs and CDC
CDC events are graph mutations, yet nobody builds them that way. When Debezium tails your Postgres WAL and emits a row-level change event, it hands you something like this: { op: “u”, before: { customer_id: 42, status: “prospect” }, after: { customer_id: 42, status: “client” } }… What actually happened in your knowledge graph? A node property changed. But more interestingly, probably an edge dissolved and another was created. The prospect relationship to your pipeline entity is gone and a client relationship exists now, with a timestamp, a source, and a provenance trail.
The tools don’t see it that way. Debezium, Airbyte, Estuary Flow etc. they’re excellent at what they do but they model the world as rows before and rows after. The graph semantics is your problem.
This impedance mismatch has real consequences. For example, entity resolution breaks at event granularity. A CDC event from a CRM says account_id: 7821 changed. Your KG has a node with IRI
SAP and enterprise systems make it worse. Their change events are coarse-grained: a BAPI call, an IDoc, a business object delta. A single event can encode what is, in graph terms, a subgraph replacement: several nodes touched, several edges rewritten, some implied by absence. You have to reconstruct the mutation algebra from domain knowledge, not from the event structure.
Kafka helps and hurts simultaneously. As a transport layer for CDC events it’s superb, ordered, replayable, partitioned by entity. Still, a Kafka topic per table is not a topic per graph concept. The partitioning strategy that makes relational CDC fast is orthogonal to the entity-centric partitioning a KG pipeline wants.
The right mental model: every CDC event is a patch on a graph. Not a row diff but a typed mutation on a typed graph. If you build your ingestion layer thinking in those primitives, the downstream logic becomes dramatically simpler. The tooling will catch up I presume but right now, if you’re building a live knowledge graph on top of CDC infrastructure, you’re doing graph mutation modeling in your head.