Witsml
The gas and oil industry has a specialized data exchange format known as Witsml, which originally relied on a standard tree-like structure. However, it has since evolved into a more complex graph-like format. This shift posed significant challenges for many applications and resources that were built around the tree-based organization, particularly those using SQL databases. Traditional relational databases like SQL Server, MySQL, and PostgreSQL are not inherently designed to handle graph structures efficiently, leading to potential issues when working with the new Witsml format.
A major Scandinavian player in the market approached me to evaluate whether Neo4j or an alternative graph database could be a suitable replacement for their existing Microsoft SQL Server solution. This project turned out to be one of the most enjoyable consulting experiences of my career. The combination of working with a knowledgeable and cooperative client and the opportunity to revisit my expertise in C# and the .Net stack, including the always-reliable Visual Studio, made it particularly rewarding.
As I delved into their graph-related needs and the specifics of their application stack, it became apparent that migrating to a dedicated graph database like Neo4j was unnecessary. Although less well-known, SQL Server actually has built-in support for graph structures. While its graph API isn’t as sophisticated as those found in specialized graph databases—lacking features like traversal and shortest path algorithms—it is sufficient for basic graph operations. This revelation was a pleasant surprise for the customer, as it meant they could continue using SQL Server without incurring the significant costs associated with adopting a new database solution. It’s a common misconception that graph data must be stored in a graph database, a myth often perpetuated by companies like Neo4j or TigerGraph. The truth is, the choice of database should always be based on the specific needs of the business, and in many cases, traditional databases can handle graph data adequately.
In addition to saving the customer a substantial amount of money, I developed a custom .Net API tailored to the Witsml format. This API seamlessly integrated the Witsml graph structure with SQL Server. The project was incredibly enjoyable, allowing me to not only solve the immediate challenge but also to innovate further by creating an abstraction layer for OpenCypher and developing a graph visualization tool using yFiles.
Throughout my career, I’ve encountered countless industries and domains where graph structures either emerged as a natural fit or became the ideal solution. The versatility and universality of graphs continue to impress me, offering solutions to complex problems across a wide range of applications.