Large Language Models and Knowledge Graphs: A Journey Towards Collaborative Intelligence

There’s been a lot written and tweeted about Generative AI (GenAI) powered by transformer based large language models (LLM). OpenAI in sync with Microsoft clearly leads the table with Google and other big players and a bunch of innovative open source options in the fray.

As with any other technology with potential for disruption in varied fields and diverse candidate use cases, attendant hype is in step, causing a lot of confusion.

In the late 1980s, as a young enthusiast working with Expert Systems, I was captivated by the promise of rules-based symbolic AI. I witnessed firsthand the power of these systems and the transformative potential they held. Yet, a challenge remained — it was impossible to manually capture all the nuanced knowledge of human experts into these systems. Despite all the hype then, there came an AI winter, forcing many of us to pivot.

Over the next three decades, as I moved up in my career spanning large tech organizations and startups across various geographies, this challenge remained. Efficiently capturing expert knowledge and it’s efficient reuse still begged for a solution. It was not until the advent of transformer architecture and the emergence of models like GPT, combined with the wider use of Knowledge Graphs (KG), that the possibility of an answer to this challenge began to emerge.

If we leverage their inherent strengths, LLMs can be utilized for analyzing vast volumes of unstructured data, extracting facts and knowledge into a requisite structure that can be organized in KGs. KGs enrich this information by drawing connections, filling in gaps, and supplying contextual understanding. In return, KGs offer LLMs a reliable, validated source of information, grounding the outputs of the LLM on facts and verified knowledge. This helps LLMs shed hallucinations and reduce errors. KG based embeddings can help refine LLMs with specific domain and enterprise knowledge. This mutual interdependence sets up a virtuous cycle of continuous learning and improvement. The interplay between LLMs and KGs creates a dynamic system that evolves and improves over time.

However, in this context, it is essential to address a common misconception. Many of us assume that creating embeddings of chunked up documents and storing them in a vector database (vDB) can solve the hallucination problem in LLMs. While it seems logical, let us scrutinize this assumption. Even the best in class LLMs, despite their high-dimensional vector spaces, still produce hallucinations. These are highly capable models compared to the lesser models we often utilize in creating embeddings for vDBs. Despite their advantages, hallucinations persist for the SOTA LLMs. So, when we retrieve chunks of content from these vDBs that operate on probabilistic algorithms and feed it to an LLM, expecting hallucination-free outputs might be overly optimistic.

This does not devalue the use of vector databases and embeddings — in fact there are synergies between vDBs and KGs that can be leveraged. Intention is to highlight that eliminating hallucinations is a complex task, needing more than just embeddings alone. It also reinforces the importance of KGs as a validated knowledge source to anchor LLM outputs and ensure their accuracy.

The vision of integrating LLMs and KGs goes beyond managing knowledge within an organization. It is about enabling AI as a collaborator in human progress, building on the collective spirit of human knowledge sharing. Remembering that successful collaborations involve thoughtful design, robust security and privacy focus, and ongoing management is also essential.

Reflecting on my journey from Expert Systems to LLMs and KGs, it is clear we have made significant strides that benefit humanity. It also underscores our ability to learn, adapt, and innovate, and fuels excitement for the next phase of our AI journey.