The same LLM is 8x slower to first token depending on who serves it

(dynoyard.app)

4 points | by jeremyrajan 10 hours ago ago

1 comments