Graph Tokenization
Otherwise you could use just as well a relational system. Naive textualization flattens exactly the thing that made the graph worth building. If your retrieval step reduces a rich subgraph to a bag of sentences, you’ve paid the engineering cost of a KG and kept the informational payoff of a document store. When using LLMs to understand graphs you end up with the non-Euclidean dilemma: how to transform (graph) topology into tokens a LLM can use? In the end, LLMs are just this: token processing machines. Note that this dilemma does not depend on whether you use RDF or prop graphs. In all cases you need to feed knowledge and topology to an AI agent.
This mapping of graphs to tokens is the central topic of graph tokenization techniques. There are several.
One way is textualization. Serialize triples or subgraphs into natural language or a structured format (JSON, Cypher, RDF-ish sentences) and hand them to the LLM as context. Cheap, interpretable, but it discards topology, the model sees a list of facts, not a graph. This is most of what passes for “knowledge-graph RAG” today.
Another approach is structure-aware tokenization. Here you encode graph-native signals like node degree, path distance, subgraph motifs, positional encodings from spectral or random-walk methods into token embeddings before the sequence ever reaches the transformer. This is where graph foundation models live: the graph’s shape becomes part of the representation, not an afterthought described in prose. You will often see the concept of message-passing and pseudo-random walks on graphs in this approach. It has deep relations to physics (percolation, diffusion, lattice techniques).
Next, there is learned discrete codes. You vector-quantize node or subgraph embeddings into a fixed vocabulary of “graph tokens,” the way variational encoding does for images. It treats graph substructures as reusable symbolic units, which is philosophically close to how knowledge representation was always supposed to work.
The thing to take away here is that if you’re building knowledge-augmented AI and look at the final graph RAG element (the last bits before the UI surface), the question isn’t “which vector database” but “how does my graph survive the trip into token space?”. How to use correctly and efficiently both the knowledge AND the topology sitting in the knowledge graph?
▶ Node4All: https://arxiv.org/html/2607.17272v1 (just one out of many)