Continual graph learning
Continual graph learning (CGL) is about graph machine learning (GML) over dynamic graph data. Real graphs don’t hold still and retraining from scratch is often not an option. It leads to catastrophic forgetting: the model gets sharper on new patterns and quietly worse at everything it used to know. Dealing with this is hard. GML is already quite challenging, making the training data variable adds another layer of complexity.
People have invented a various sophisticated approaches, my favorite being the geometric-topological one. Ollivier–Ricci curvature is a discrete analogue of curvature for graphs: for an edge between two nodes, it measures how much their neighborhoods overlap. Densely interconnected, cluster-like regions have high curvature. The two endpoints’ neighbors overlap heavily, so the edge is somewhat redundant, since information can flow around it via other paths. Bridge-like edges connecting otherwise separate regions have low or negative curvature, they carry structural information that no alternate path replicates. In a nutshell.
The application I find most compelling of all this is the lifelong embedding learning for growing knowledge graphs. Instead of re-materializing and re-embedding an entire KG every time your ontology or instance data shifts, you update incrementally. The graph representation of your domain stays the durable asset, and the embedding layer adapts around it. Like a dynamic vector space. I wished someone would invent a merge of neural networks and CGL. Something like continuous neural network learning.
- Survey of CGL: https://github.com/UConn-DSIS/Survey-of-Continual-Learning-on-Graphs
- BEGIN (foolproof python framework): https://github.com/ShinhwanKang/BeGin
- Online CGL: https://openreview.net/pdf?id=4sJJixGIZX