AI Potluck
Model components / Inference code

SambaNova Cloud

SambaNova Systems

SambaNova Cloud is an inference service running on SambaNova's custom Reconfigurable Dataflow Unit (RDU) chips, the SN40L. Its three-tier memory system pairs on-chip SRAM for hot local work with HBM to stream model weights and DDR DRAM for prompt caching and multi-model workloads. It serves open-weight models through OpenAI-compatible endpoints as SambaCloud, with SambaStack offered for on-premise deployment.

The SambaNova Cloud API reference URL cited previously now 404s, so the OpenAI-compatible endpoint claim carries over from the prior read rather than a fresh one. Verified 2026-08-09 via the SN40L architecture blog and the SambaCloud Terms of Service.

Openness

1 high confidence
1.0
license
proprietary
source
closed(managed cloud API, the EULA states Customer has no right to obtain a copy of the underlying computer code for any Service)
software-stack
closed(SambaFlow compiler/runtime not open)
hardware
SN40L-RDU-only
delivery
managed-cloud-API

Proprietary managed cloud inference on SambaNova's own RDU silicon, with no open-source software disclosed. The Cloud EULA grants only a limited license to use the software embodied in the service and states that the customer has no right to obtain a copy of the underlying computer code, so there is no self-hostable implementation at all, which settles the score on source alone.

  • https://sambanova.ai/cloud-end-user-license-agreement recorded 2026-08-11

    SambaNova hereby grants to Customer a non-exclusive, non-transferable, limited license during the Term to use the software embodied in the Service. The same agreement states: "Customer acknowledges that the Service is offered as an online, hosted solution, and that Customer has no right to obtain a copy of the underlying computer code for any Service", and forbids Customer to "reverse engineer, disassemble or decompile any Service or otherwise seek to obtain the source code of any software included in the Service".

  • https://sambanova.ai/blog/sn40l-chip-best-inference-solution recorded 2026-08-09

    The RDU is also the only AI accelerator with a tightly coupled three-tier memory system comprising SRAM, HBM, and DDR DRAM.

Adoption

3 low confidence
3.0

OpenRouter's SambaNova provider page shows real, current daily token volume in the hundreds of millions to billions across the models it routes to SambaNova, around 1.65 billion tokens in a single day across six models, which corroborates genuine third-party traffic beyond vendor marketing. No standalone user or customer count is published, so this is observed traction rather than a banded usage figure.

Capability

4 medium confidence
4.0

SambaNova's own SN40L benchmark reports Llama 3.1 405B at 129 output tokens per second, 70B at 457 and 8B at 1,042, all at full 16-bit precision. Artificial Analysis independently lists nine models served through the SambaNova API, the fastest being gpt-oss-120b at roughly 724 output tokens per second, with Llama 3.3 70B at roughly 290. The score reflects specialized-hardware throughput leadership offset by a narrower served-model catalog than the largest multi-tenant inference clouds.

Verified 2026-08-09