AI Potluck
Contribute
Back to Gap Map Product / UX / Safety & Guardrails

LettuceDetect

KR Labs
open source / Overall score: 2.5

LettuceDetect checks whether an AI answer is supported by the source material it was given and marks the exact passages that are not. Small encoder models run locally for fast checks, and a 2B generative model also labels each unsupported passage by type. It covers retrieval-augmented answers, coding-agent responses and tool output, and KR Labs publishes it as a Python package with an optional web API.

Described in arXiv 2502.17125. The code, tool-output and agent models released in June 2026 are described in arXiv 2607.00895.

Openness

5 high confidence
5.0
license
MIT(OSI, the code and the English and multilingual v1 detector weights)+Apache-2.0(OSI, the v2 detector weights for code, tool output and prose)
source
public(full Python implementation, training and evaluation scripts included)
self-host
yes(pip package and optional FastAPI server)
service
none
core features withheld
no

LettuceDetect's code is MIT-licensed, and the detector models it loads from Hugging Face come from the same team under MIT or Apache 2.0, so nothing in an ordinary setup carries a narrower license. The package installs from PyPI and runs on your own hardware, web server included, and the repository also holds the scripts used to train and evaluate the models. Only the optional LLM-based baseline calls an outside service.

  • https://cdn.jsdelivr.net/gh/KRLabsOrg/LettuceDetect@main/LICENSE recorded 2026-10-13

    The LICENSE file at the head of KRLabsOrg/LettuceDetect is the MIT License text, "Copyright (c) 2025 KR Labs", granting permission "to deal in the Software without restriction".

  • https://cdn.jsdelivr.net/gh/KRLabsOrg/LettuceDetect@main/README.md recorded 2026-10-13

    The README lists "MIT-licensed models & code" among its features, installs with "pip install lettucedetect -U", loads detectors from Hugging Face by model path, documents training with "scripts/train.py" and evaluation with "scripts/evaluate.py", and starts its own web API with "scripts/start_api.py". The LLM baselines need an OpenAI API key. Searched the body for enterprise, pricing, commercial, paid, hosted and license key: none.

  • https://huggingface.co/api/models/KRLabsOrg/lettucedect-base-modernbert-en-v1 recorded 2026-10-13

    The Hugging Face API record for KRLabsOrg/lettucedect-base-modernbert-en-v1, the default model of the web API, carries the tag "license:mit", reports gated as false and lists "model.safetensors" among its files.

  • https://huggingface.co/api/models/KRLabsOrg/lettucedect-v2-mmbert-base recorded 2026-10-13

    The Hugging Face API record for KRLabsOrg/lettucedect-v2-mmbert-base carries the tag "license:apache-2.0", reports gated as false, lists "model.safetensors", and was created on 2026-06-22.

  • https://huggingface.co/api/models/KRLabsOrg/lettucedect-v2-qwen-2b recorded 2026-10-13

    The Hugging Face API record for KRLabsOrg/lettucedect-v2-qwen-2b carries the tags "license:apache-2.0" and "base_model:finetune:Qwen/Qwen3.5-2B", reports gated as false and lists a safetensors weight file.

  • https://pypi.org/pypi/lettucedetect/json recorded 2026-10-13

    The PyPI JSON for lettucedetect (version 0.2.3) records license "MIT" and the classifier "License :: OSI Approved :: MIT License", gives "https://github.com/krlabsorg/lettucedetect" as its homepage, and lists an optional "api" extra that pulls in fastapi.

Adoption

2 medium confidence
2.0

Adoption is the sum of PyPI downloads of the lettucedetect package and Hugging Face downloads of its four main detector models, since most uses of the package load one of those models. The models can also be loaded without the package, so part of their downloads may come from people using them directly.

Capability

3 medium confidence
3.0

LettuceDetect makes one of the checks Granite Guardian makes, deciding whether an answer is supported by its sources, and goes further on that one check: where Granite Guardian answers yes or no, LettuceDetect marks the exact unsupported passages, in code and tool output as well as prose. It does not screen for harmful content, jailbreaks or risky function calls, which Granite Guardian also covers, so it sits a step below it.

  • https://arxiv.org/abs/2502.17125 recorded 2026-10-13

    The arXiv abstract describes "a token-classification model that processes context-question-answer triples", built on ModernBERT and trained on RAGTruth, reporting "an F1 score of 79.22% for example-level detection" and "30 to 60 examples per second on a single GPU".

  • https://arxiv.org/abs/2607.00895 recorded 2026-10-13

    The arXiv abstract, submitted 1 July 2026, introduces a span-level hallucination benchmark over code, tool output and documents, and reports that the fine-tuned Qwen3.5-2B detector "reaches 0.689 span-F1 on the unified test set and 0.60 on the code-agent source".

  • https://cdn.jsdelivr.net/gh/KRLabsOrg/LettuceDetect@main/README.md recorded 2026-10-13

    The README calls LettuceDetect "an open span-level grounding verifier for AI outputs" that "localizes unsupported, contradictory, or fabricated parts of RAG answers, coding-agent responses, and tool-grounded outputs", returning "exact character spans instead of only an answer-level clean/flagged verdict". It lists 7 languages for the multilingual models, a category and subcategory for each span in the v2 models, a web API with a Python client, and 0.689 span-F1 for Qwen-2B on "the unified 10,698-example v2 test set".

  • https://huggingface.co/KRLabsOrg/lettucedect-v2-qwen-2b/raw/main/README.md recorded 2026-10-13

    The lettucedect-v2-qwen-2b card describes a generative detector that emits the unsupported spans of an answer as JSON, each with a category and subcategory from a taxonomy of "3 categories (contradiction, fabricated_reference, unsupported_addition) × 13 subcategories". On code-agent answers it reports off-the-shelf detectors including Granite-Guardian as "all ≈ chance" on its own benchmark.

Verified 2026-10-13, except some scores, still awaiting re-confirmation