VLLM

Details

Photo

—

Name

—

Description

High-throughput and memory-efficient inference and serving engine for LLMs.

Category

LLM Inference

URL

https://github.com/vllm-project/vllm