9 points | by AndrewLiu96 a day ago ago
5 comments
Personally I don't use LLMs, so I wonder what this kind of router is good for? Why not just a library?
You don't use LLMs? Even my grandma use LLMs, what's your reasons?
thank you! i find this very interesting, especially the Cache-aware concurrency .
No worries! Thank you for checking it out. The cache-aware concurrency was a ton of fun to implement
another option for on-device routers is role-model that i'm building: https://github.com/try-works/role-model
Personally I don't use LLMs, so I wonder what this kind of router is good for? Why not just a library?
You don't use LLMs? Even my grandma use LLMs, what's your reasons?
thank you! i find this very interesting, especially the Cache-aware concurrency .
No worries! Thank you for checking it out. The cache-aware concurrency was a ton of fun to implement
another option for on-device routers is role-model that i'm building: https://github.com/try-works/role-model