Good on these folks for doing something useful with Trainium. I'm always surprised that the Tranium ecosystem is as poor as it is. Amazon's market cap is nearly $3T and they can't throw a couple L6s and a modest token budget at this? Making Trainium work as seamlessly as possible as an Nvidia substitute is a clear flywheel play.
Good on these folks for doing something useful with Trainium. I'm always surprised that the Tranium ecosystem is as poor as it is. Amazon's market cap is nearly $3T and they can't throw a couple L6s and a modest token budget at this? Making Trainium work as seamlessly as possible as an Nvidia substitute is a clear flywheel play.
Tokens per minute is a strange metric, I usually see tok/s. A per-second figure would be more accessible in comparing this to alternatives.
curious how the token/$ on trainuim compare to nvidia chips and amd. is the future "chipless" inference?
how does this compare with sglang
They don't have support running trn2 afaik especially no glm 5.3 flash . they did release an update -5 days back but again its for trn3 nothing trn2