Diffbot Launches World’s Most Factually Grounded Language Model: New Benchmark in AI-Powered Knowledge Retrieval

January 9, 2025 – Menlo Park, California – Diffbot, creators of the world’s largest knowledge graph from the open web, today announced the launch of its first Large Language Model (LLM), the world’s most factually grounded AI language model to date. Leveraging Diffbot’s proprietary Knowledge Graph—which comprises more than 10 billion entities and over 1 trillion structured facts drawn from the open web—the new model surpasses all currently available LLMs in terms of factual reliability and domain coverage.

Current frontier LLMs approach reducing bad responses (aka “hallucinations”) by either scaling up the size of the model, so that it represents more training data, or scaling up the inference time, which uses the LLM to “reason” and self-correct its mistakes, at the cost of much longer runtimes. At Diffbot, we believe that the core “reasoning” that we call intelligence will eventually be distilled down to ~1B parameters, and that factual knowledge is best maintained outside of the model weights, in an externally cited knowledge graph. The Diffbot LLM is a fine-tuned LLama 3.3 70B (and 8B) that has been trained to be an expert tool user, querying at inference-time structured and unstructured databases.

Measured by industry-leading benchmarks, including the MMLU-Pro and FreshQA scores, Diffbot’s model delivers unmatched factual performance. It achieves an MMLU-Pro score of %70.36, beating all other open source models with <100B params and a FreshQA score of %81, beating ChatGPT search mode, Gemini, and Perplexity. This accuracy is directly attributable to Diffbot’s unparalleled Knowledge Graph and its cutting-edge approach to fact-grounded responses.

Diffbot’s new LLM is also the first open-source implementation of a production GraphRAG (Graph Retrieval Augmented Generation) system. Unlike other LLMs that rely solely on internal training data and heuristic prompt engineering, GraphRAG dynamically queries the Diffbot Knowledge Graph and an independent search index of the web to retrieve precise, authoritative information.

The result is a model that doesn’t just approximate the truth from its model weights—it actively locates and references verifiable sources in real-time.

Real Citation Grounding and Source Transparency

While other language models may produce the appearance of citations, they often fail to trace statements back to their original sources (i.e. a citation link will frequently lead to a dead page.) Diffbot’s LLM takes citation grounding seriously. It is fine-tuned to rigorously match every factual statement with the specific passage that supports it, and always provides a direct citation to the original source of any quoted material. This ensures users have full transparency into how the model arrived at its answers, bolstering trust and reliability.

Diffbot’s LLM integrates directly with Diffbot’s Automatic Extraction, structured Knowledge Graph querying, unstructured web search querying, and code interpretation capabilities. It excels in complex workflows, such as aggregating data on-demand or generating insights from custom, user-selected datasets.

As data security and confidentiality become paramount, Diffbot’s LLM is engineered with a privacy-first mindset. Users retain full control over their data, and the model’s self-hosting option ensures that sensitive information never needs to leave a secure, on-premise environment.

Advanced Multimodal and Tool Capabilities

Beyond text-based queries, Diffbot’s LLM delivers advanced multimodal reasoning:

• Image Entailment: From “How to draw a baby shark” to complex visual reasoning, the model can understand and explain imagery, ensuring seamless interactions that extend beyond the written word.
• Code Interpretation: Instead of offering approximate solutions to math and string-processing problems, the model can execute JavaScript code interpreters internally. This results in definitive, correct answers rather than guesswork, making it ideal for developers, data scientists, and technical users.

Diffychat: Public Demo

Developers, researchers, and organizations interested in testing, integrating, or contributing to the project are encouraged to visit https://diffy.chat and explore the documentation and open-source repositories. Users can test Diffbot LLM at https://diffy.chat, allowing anyone to experience its capabilities firsthand.

Unlike many closed-source counterparts, Diffbot’s model is fully open-weight and comes with an open-source, OpenAI-compatible function-calling API server for self-hosting. This open approach empowers enterprises, researchers, and hobbyists to integrate the model into their own systems with full control over their data and configurations.

About Diffbot:

Diffbot is a leading artificial intelligence and data company dedicated to organizing and structuring the world’s information. Its industry-first Knowledge Graph continuously extracts and organizes facts from across the web, powering mission-critical applications for enterprise, research, and innovation. By building the most comprehensive factual store of human knowledge and pairing it with advanced natural language models, Diffbot aims to redefine the next generation of factual AI.