+0
日增长
9,498
stars
703
forks
65
watchers
2,350
prs
868
issues
236
releases
README
Cog: Containers for machine learning
Cog is an open-source tool that lets you package machine learning models in a standard, production-ready container.
You can deploy your packaged model to your own infrastructure, or to Replicate.
Highlights
- 📦 Docker containers without the pain. Writing your own Dockerfile can be a bewildering process. With Cog, you define your environment with a simple configuration file and it generates a Docker image with all the best practices: Nvidia base images, efficient caching of dependencies, installing specific Python versions, sensible environment variable defaults, and so on.
- 🤬️ No more CUDA hell. Cog knows which CUDA/cuDNN/PyTorch/Tensorflow/Python combos are compatible and will set it all up correctly for you.
- ✅ Define the inputs and outputs for your model with standard Python. Then, Cog generates an OpenAPI schema and validates the inputs and outputs.
- 🎁 Automatic HTTP inference server: Your model's types are used to dynamically generate a RESTful HTTP API using a high-performance Rust/Axum server.
- 🚀 Ready for production. Deploy your model anywhere that Docker images run. Your own infrastructure, or Replicate.
How it works
Define the Docker environment your model runs in with cog.yaml:
yaml
build:
gpu: true
system_packages:
- "libgl1"
- "libglib2.0-0"
python_version: "3.13"
python_requirements: requirements.txt
run: "run.py:Runner"
Define how your model runs with run.py:
python
from cog import BaseRunner, Input, Path
import torch
class Runner(BaseRunner):
def setup(self):
"""Load the model into memory to make running multiple inferences efficient"""
self.model = torch.load("./weights.pth")
The arguments and types the model takes as input
def run(self,
image: Path = Input(description="Grayscale input image")
) - Path:
"""Run the model"""
processed_image = preprocess(image)
output = self.model(processed_image)
return postprocess(output)
In the above we accept a path to the image as an input, and return a path to our transformed image after running it through our model.
Now, you can run the model:
console
$ cog run -i [email protected]
-- Building Docker image...
-- Running...
-- Output written to output.jpg
Or, build a Docker image for deployment:
console
$ cog build -t my-classification-model
-- Building Docker image...
-- Built my-classification-model:latest
$ docker run -d -p 5000:5000 --gpus all my-classification-model
$ curl http://localhost:5000/predictions -X POST \
-H 'Content-Type: application/json' \
-d '{"input": {"image": "https://.../input.jpg"}}'
Or, combine build and run via the serve command:
console
$ cog serve -p 8080
$ curl http://localhost:8080/predictions -X POST \
-H 'Content-Type: application/json' \
-d '{"input": {"image": "https://.../input.jpg"}}'
Why are we building this?
It's really hard for researchers to ship machine learning models toTop issues
#1323Fail while pushing with --separate-weights17 comments · 2023-09-29#157Switch from `python_packages` to `python_requirements`14 comments · 2021-07-22#3164Fail to push container to r8.im with "error during image push: unknown blob"12 comments · 2026-08-28#842`cog login` fails silently if `pass` is uninitialized7 comments · 2022-12-11#1228Authentication errors, during cog push, docker login r8.im problems?5 comments · 2023-07-25#623Add ordering to output objects5 comments · 2022-05-24#545TPU support5 comments · 2022-04-11#314Support secrets5 comments · 2021-10-27
Latest release
v0.23.1 v0.23.1 2026-10-20Bug fixes - Predictions now fail when an output upload fails. Previously, a file upload that timed out or was rejected could leave a prediction marked succeeded with output: null, or silently drop files from list outputs. Any failed upload now fails the prediction with an error, and cancellation still reports canceled. (3197)