Graph databases and technical implementations

From: aidotengineer

Graph databases are a crucial component in advanced AI systems, particularly in the realm of knowledge representation, retrieval, and intelligent agent memory. They offer a systematic method for preserving wisdom by connecting different entities and creating a network of interconnected relationships [02:44:09]. This approach contrasts with traditional, more isolated data storage methods, providing a comprehensive view of knowledge [03:55:09].

Core Concepts of Graph Databases

A knowledge graph is a network that represents relationships between different entities, which can include people, places, concepts, or events [02:33:40]. The “edge” of a relationship between two entities is very important, as it can only be fully exploited by graph-based networks or knowledge graphs [03:02:02]. The core goal in creating a knowledge graph is to establish a “triplet” – defining the relationship between these entities, which is a strength of graph-based systems [03:24:02] [03:30:00].

Unlike semantic RAG systems or vector databases, knowledge graphs capture information and connections between entities in much greater detail, providing a more comprehensive view of the knowledge within a system [03:47:00] [03:50:00]. They also have the ability to organize data from multiple sources [04:14:00].

Technical Implementations and Architectures

Building and leveraging graph databases involves several key technical steps, from data processing to advanced retrieval and memory management.

GraphRAG Systems

GraphRAG systems are a hybrid approach combining the strengths of knowledge graphs with Retrieval Augmented Generation (RAG). The high-level architecture of a GraphRAG system involves four main components: data, data processing, graph creation (or semantic embedding vector database creation), and inferencing [04:28:00] [04:31:00] [04:39:00]. This process can be broadly divided into offline (one-time data processing and graph/database creation) and online (querying and response generation) phases [05:00:00].

Creating Knowledge Graphs

A critical step is creating triplets from unstructured documents [06:21:00]. This involves using Large Language Models (LLMs) to extract entities and their relationships [06:49:00] [07:26:00]. Prompt engineering is crucial here, defining the “ontology” (the types of entities and relationships) based on the specific use case to guide the LLM’s extraction and structuring of information into a triplet format [07:40:00] [07:46:00]. The quality of these triplets directly impacts the retrieval process; noisy triplets lead to noisy retrieval [08:17:00]. Developers might spend up to 80% of their time perfecting the ontology to ensure accurate triplets [08:35:00].

Alongside graph creation, the semantic vector database is built by chunking documents, converting chunks into vector embeddings using an embedding model, and storing them in a vector database [08:48:00] [09:01:00] [09:10:00]. Overlap between chunks is important to maintain context [09:20:00]. The key difference is that semantic vector databases don’t exploit the relationships between entities, which knowledge graphs excel at [09:40:00].

Retrieval in Graph-Based Systems

Retrieval involves querying the knowledge graph to identify relevant nodes and their relationships [10:06:06]. Simple “single hop” retrieval misses the core advantage of graphs, which is exploiting relationships through multiple nodes or “hops” [10:29:00]. Deeper traversal provides better context but increases latency, necessitating a balance between depth and performance in production environments [11:02:00] [11:07:00]. Tools like NVIDIA’s cuGraph library can accelerate graph searches, allowing for deeper traversal while reducing latency [11:37:00].

Agent Memory with Graph Databases

Traditional RAG and vector databases struggle with agent memory because each fact is isolated and immutable, lacking temporal and relational reasoning [40:33:00] [40:52:00]. They rely on semantic similarity, which may not align with business relevance or changing user preferences over time [57:05:00].

Zep’s open-source framework, Graffiti, addresses this by building real-time, dynamic, temporal graphs [41:45:00]. Graffiti is:

• Temporally Aware: It tracks multiple temporal dimensions for each fact, identifying when a fact becomes valid or invalid, enabling reasoning over time [42:25:00] [42:50:00].
• Graph Relational: It defines explicit relationships between facts, modeling causality (e.g., broken shoes cause disappointment, leading to a brand preference change) [43:22:00] [43:32:00]. Instead of deleting old facts, it marks them invalid, preserving the history of state changes [43:51:00].
• Domain-Aware: Developers can model their business domain using custom entities and edges, allowing agents to retrieve specific and relevant information, avoiding “irrelevant facts” that pollute memory [45:51:00] [46:13:00] [56:02:00].

Graffiti integrates semantic search and BM25 full-text retrieval to identify subgraphs, which are then traversed and fused for fast and accurate retrieval [44:53:00]. It offers very low latency retrieval without an LLM in the path, operating in hundreds of milliseconds, unlike some graph RAGs that might take tens of seconds [47:39:00].

Multi-Agent Systems and Knowledge Augmentation (KAG)

Knowledge Augmentation Generations (KAG) systems differ from RAG by enhancing language models with structured knowledge graphs for more accurate and insightful responses that understand rather than just retrieve [02:27:00] [02:31:00]. This is based on a “wisdom graph” concept, where wisdom is the core, actively guiding decisions based on knowledge, experience, and insight, with continuous feedback loops [02:59:00].

For practical application, this philosophical model can be mapped to real-world systems, such as a competitive analysis chatbot. Knowledge becomes market data, experience is past campaign performance, insight is industrial trends, and the situation is product sales, all contributing to strategic decision-making [03:06:00] [03:10:00].

Node-RED, a no-code/low-code platform, can be used for prototyping and enabling complex state diagrams with AI agents [03:21:00]. Advanced techniques like “fusion and decoder” used with knowledge graphs help improve efficiency and reduce hallucination rates in responses [03:32:00] [03:40:00]. Benchmarking has shown that such retrieval systems can achieve higher accuracy and faster response times compared to vector search systems [03:41:00]. These systems can expose the LLM’s thought process, handle complex queries across multiple documents (“multihop questions”), and manage complex data formats that challenge vector retrieval [03:46:00] [03:51:00] [03:53:00].

Industry Applications and Use Cases

The adoption of knowledge graphs in enterprise AI is growing, with various companies leveraging them for specific applications.

Evaluation and Optimization

Evaluation and performance metrics of graphbased systems are crucial for assessing the effectiveness of RAG workflows. Metrics include faithfulness, answer relevancy, precision, recall, helpfulness, collectiveness, coherence, complexity, and verbosity [12:24:00]. The Ragas library is a Python tool designed to evaluate RAG workflows end-to-end, assessing response, retrieval, and query interpretation [12:42:00]. Reward models like Lanimotron 340B can also judge the responses of other LLMs [14:04:00].

Optimization strategies for GraphRAG systems involve:

• Data Cleaning: Removing irrelevant characters (e.g., apostrophes, brackets) from unstructured data can lead to better results in triplet generation [15:51:00] [16:31:00].
• LLM Fine-tuning: Fine-tuning an LLM model (e.g., Llama 3.1) can significantly improve the quality of generated triplets and overall accuracy [15:38:00] [16:59:00].
• Retrieval Depth (Hops): Optimizing the number of hops in graph traversal to balance context richness and latency [10:48:00].
• Hardware Acceleration: Using libraries like cuGraph integrated with NetworkX can drastically reduce latency for graph algorithms, improving performance when dealing with millions or billions of nodes [11:37:00] [18:18:00].

The decision to use a graph, semantic, or hybrid RAG system depends on the data structure (e.g., retail, FSI, employee databases with good structure are good for graphs) and the complexity of the use case (e.g., requiring understanding of complex relationships) [19:16:00] [20:00:00]. Graph-based systems are compute-heavy, so their benefits must justify the cost [20:26:00].

Specific Implementations and Use Cases

• NVIDIA’s Developer Advocate Team: This team focuses on creating technical workflows and notebooks, releasing code on GitHub to help developers leverage graph data structures in AI and its benefits for various applications [01:17:00].
• Cognify: This modular framework builds semantic graphs from various data sources (e.g., GitHub API) and allows agents to read, write, and analyze data for tasks like comparing developer profiles for hiring decisions [01:43:47]. It supports custom graph building and memory association layers inspired by cognitive science [01:45:15].
• Neo4j MCP Server: This memory server enables agents to properly recall and log memories by creating entities and relationships in a knowledge graph based on conversations [01:48:45] [01:51:00]. It provides rich context through identified relationships, which can be visualized [01:51:17].
• Cisco’s Outshift Multi-Agent Framework: This system uses a “network knowledge graph” as a digital twin of a production network to reduce failures in change management [02:46:43]. It leverages multimodal data sources (controllers, devices, config management systems) in various formats (Yang, JSON) [02:50:00]. The graph-powered AI for uncovering fraud networks is structured in layers, allowing agents to efficiently query specific data for tasks like impact assessment, test plan creation, and execution [02:53:53]. Fine-tuning query agents on schema information and example queries reduced token consumption and response time [02:57:10].
• Yhar.ai for Legal Documents: Yhar.ai converts legal documents into graphs to identify class action cases. They use multi-agent systems and graphs to break down complex legal workflows into testable steps, controlling state through the graph [03:06:08] [03:09:49]. This ensures necessary accuracy and supports creative legal arguments by structuring information and pruning state over time [03:10:40] [03:11:23]. This also aids in legal discovery by structuring vast amounts of unstructured data (e.g., emails) into a graph, enabling targeted information retrieval and visualization [03:15:31]. For case research, they scrape and filter web data to qualify leads, applying specific schemas to personalize information for lawyers [03:18:29]. This process of semantic enrichment and graph-based searches in architecture allows them to find potential lawsuits significantly earlier than traditional methods [03:21:48].

Other Notable Patterns and Tools

• Graph Construction Patterns: These include lexical graphs (representing document chunks and relationships) and domain graphs (entities and their relationships) [03:19:00].
• Enrichment with Graph Algorithms: Algorithms like clustering, link prediction, and PageRank can enrich the graph, identifying cross-document topics and communities [03:26:39] [03:39:00].
• Graph Retrieval: Involves an initial index search (vector, full-text, hybrid, spatial) to find entry points, followed by relationship traversal to fetch additional context [03:27:21]. Modern LLMs are increasingly trained to process graph structures, allowing richer context to be passed to them [03:38:33].
• Agentic Approach: Breaking down user questions into individual tasks, extracting parameters, and running specialized tools (Cypher queries) in sequence or loops [02:10:00] [02:41:41].
• Agentic Firewall: A conceptual use case where a human-language shell with episodic and temporal memory of user actions across machines and sessions could provide enhanced security context [02:02:13]. This directly relates to handling complex queries with deep research.

Conclusion

Graph databases, especially when combined with LLMs in GraphRAG systems and advanced agent memory frameworks, represent a powerful paradigm shift in AI applications. They enable detailed relational and temporal reasoning, offer significant improvements in accuracy and explainability, and are being successfully adopted in diverse industries, from finance and legal to networking and customer support. The emphasis on causality and interconnectedness makes graph databases a fundamental component for building more intelligent, robust, and reliable AI systems.