Groq Inference
GroqGroq Inference runs open models on Groq's custom Language Processing Unit (LPU) hardware, built from the ground up for AI inference rather than adapted from a general-purpose GPU. Its deterministic, statically scheduled architecture keeps model weights in on-chip SRAM rather than in off-chip memory. The GroqCloud service exposes open models, including gpt-oss, Llama, and Whisper, through an OpenAI-compatible API.
Openness
1 high confidence- engine
- proprietary(LPU stack)
- source
- closed
- access
- managed-cloud-API-only
- license
- Proprietary
GroqCloud is a managed API service. The groq GitHub org publishes client SDKs, eval tools and demo apps, but no repository for the inference engine itself. The GroqCloud services agreement grants only a limited right to use the service and forbids extracting its source code, and the marketing site's terms of use grant a license only to the website.
- https://console.groq.com/docs/legal/services-agreement recorded 2026-09-29
The GroqCloud services agreement grants only "non-exclusive, non-transferable, and limited right to access and use the Cloud Services" and bars customers from any attempt "to extract any or all of the source code of, the Cloud Services".
- https://console.groq.com/docs/overview recorded 2026-09-29
The docs overview describes GroqCloud as a hosted API and gives the page description "Fast LLM inference, OpenAI-compatible. Simple to integrate, easy to scale. Start building in minutes." with no self-hosting or downloadable engine.
- https://github.com/orgs/groq/repositories recorded 2026-08-09
Full list of the groq GitHub org's 41 public repos, read across both pages of the listing (page 1: groq-python, groq-typescript, groq-mcp-server, openbench, harbor, groq-desktop-beta, dispatch, ...; page 2: groqflow, mlagility, groq-api-cookbook, groq-gradio, ...) - none is the LPU inference engine or serving runtime; all are client SDKs, demo apps, eval harnesses, model-build toolflow front ends, or infra tooling.
- https://groq.com/terms-of-use recorded 2026-09-29
The website terms grant a license to the Websites only: "license to access and use the Websites for your personal, non-commercial use only."
Adoption
4 medium confidenceGroq's own homepage names no enterprise customers, so adoption is read instead from a developer count the company reported in a press release. That figure is self-reported and has not been independently corroborated.
- https://groq.com/newsroom/groq-raises-usd650m-to-scale-its-ai-inference-cloud-business recorded 2026-09-29
The funding press release says "The company serves more than five million developers and thousands of AI-native companies that consume trillions of tokens each week." This is a company statement with no independent count.
Capability
5 medium confidenceGroq measures 473 output tokens per second on Artificial Analysis's gpt-oss-120b provider benchmark, third of 18 providers behind Cerebras and SambaNova. All three top providers run custom silicon rather than general-purpose GPUs.
- https://artificialanalysis.ai/models/gpt-oss-120b/providers recorded 2026-09-29
The gpt-oss-120b (high) provider page states "the top providers are Cerebras (1,759.4 t/s), SambaNova (709.4 t/s), and Groq (473.2 t/s). Speed varies significantly across providers, with a 3223% difference between the fastest and slowest."
Verified 2026-09-29