AI Potluck
Contribute
Back to Gap Map Model components / Inference code

AWS Neuron

Amazon Web Services
source available / Overall score: 3.5

AWS Neuron is the SDK and runtime for running machine learning on AWS's custom Inferentia and Trainium accelerators (Inf1/Inf2 and Trn1/Trn2/Trn3 instances). Its compiler converts models from PyTorch and JAX into executables for the custom silicon, with serving through a vLLM plugin and a kernel-authoring interface (NKI). AWS distributes it as a separate SDK, publishing samples and some tooling on GitHub.

Openness

2 high confidence
2.0
license
mixed/proprietary-core
source
partial(aws-neuron-sdk is a documentation and issue-tracker repo carrying no Runtime or Compiler source, while the kernel driver, NKI kernel library, vLLM-for-Neuron and NKI samples do ship as source)
runtime
proprietary(Neuron Runtime+Compiler not open sourced)
open-glue
Linux kernel driver(GPL-2.0),vLLM-Neuron integration(Apache-2.0,vllm-project org),NKI library,samples(MIT-0)
hardware-lock
AWS-Inferentia/Trainium-only

AWS Neuron installs and runs on the customer's own EC2 instances, but the Neuron Runtime and Compiler are not published: AWS's own open-source catalog lists six public repositories, and neither appears among them. An open kernel driver, kernel library and vLLM integration surround that closed core, rather than the whole SDK being closed.

  • https://awsdocs-neuron.readthedocs-hosted.com/en/latest/about-neuron/what-is-neuron.html recorded 2026-09-23

    Neuron Compiler and Neuron Runtime entries in the 'AWS Neuron Core Components' list carry no 'Open Source' bullet, unlike the Native PyTorch entry ('Open Source : Released as an open source package on GitHub under Apache 2.0') and the vLLM entry ('Open Source : Source code released under the vLLM project organization with source code on GitHub').

  • https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/oss/index.html recorded 2026-09-23

    AWS Neuron provides open source code and samples for some of its components, libraries, and tools under the Apache 2.0 license, and lists the public repositories open to contribution. The URL redirects to /about-neuron/oss/index.html, where the catalog lists exactly six public repositories -- TorchNeuron PyTorch Extension, Neuron Agentic Development, Neuron Explorer CDK, Neuron Kernel Library, vLLM for Neuron, NKI Samples -- and neither the Neuron Runtime nor the Neuron Compiler appears among them.

  • https://github.com/aws-neuron/aws-neuron-driver recorded 2026-08-09

    "license":{"spdxId":"GPL-2.0","name":"GNU General Public License v2.0"}

  • https://github.com/aws-neuron/aws-neuron-sdk recorded 2026-08-11

    "path":"LICENSE-DOCUMENTATION","preferredFileType":"license","tabName":"License" alongside "path":"LICENSE-SAMPLECODE",..."tabName":"MIT-0" -- repo carries a generic, GitHub-unclassified license for docs and a separate MIT-0 license for sample code, i.e. no single OSI license covers the SDK repo. The repo's top-level directories are .github, compiler, frameworks, deploy, general, libraries, neuron-runtime, neuron-customops, nki, release-notes, setup and tools -- the "compiler" and "neuron-runtime" directories hold documentation for those components, not their source. The repo points readers at the hosted Neuron SDK documentation site and carries an open issue tracker.

  • https://github.com/vllm-project/vllm-neuron recorded 2026-08-09

    "license":{"spdxId":"Apache-2.0","name":"Apache License 2.0"}

  • https://raw.githubusercontent.com/aws-neuron/aws-neuron-driver/master/LICENSE recorded 2026-09-23

    GNU GENERAL PUBLIC LICENSE Version 2, June 1991

Adoption

3 low confidence
3.0

AWS publishes no customer count, download count or usage figure for Neuron, and names no customers. What the page does show is integration breadth: native PyTorch and JAX support, plus Hugging Face Transformers, Optimum Neuron, vLLM, PyTorch Lightning and TorchTitan, without requiring code changes.

  • https://aws.amazon.com/ai/machine-learning/neuron/ recorded 2026-09-29

    The page says Neuron works "natively with PyTorch and JAX frameworks and with ML libraries such as HuggingFace, vLLM, PyTorch Lightning" and trains with "HuggingFace Optimum Neuron, PyTorch Lightning, and TorchTitan". It gives no customer count, download count or named customer; searched the body for customer, Anthropic, million and users.

Capability

4 medium confidence
4.0

Neuron delivers a server-class inference feature set on native PyTorch and JAX, but only for AWS's own Inferentia and Trainium hardware, and no MLPerf Inference benchmark result has been published for it.

Verified 2026-09-23