Show HN: Tiny-vLLM – high performance LLM inference engine in C++ and CUDA

(github.com)

47 points | by yu3zhou4 3 hours ago ago

6 comments