Back to Gap Map Model components / Dataset Processing Tools

smallpond

DeepSeek
open source / Overall score: 1.8

Lightweight distributed data-processing framework built on DuckDB and DeepSeek's 3FS, aimed at PB-scale preprocessing.

Openness

5 high confidence
5.0
license
MIT(OSI)
source
public(deepseek-ai/smallpond)
core features withheld
no

Fully OSI-licensed (MIT), full source public.

Adoption

1 high confidence
1.0

PyPI downloads for smallpond are the only usage figure published for this tool.

Capability

3 medium confidence
3.0

smallpond distributes DuckDB processing over 3FS, and its authors report a GraySort run that sorted 110.5 TiB in about 30 minutes. The README documents no corpus or model built with the framework.

Verified 2026-09-24