Approximate calculation is putting the cost at ~$5 per million tokens (assuming 42W sustained, 20¢/kWh), and that's excluding hardware and other costs.
I got to the same conclusion another way. There's ~2.6million seconds a month and this is getting 0.5tok/s which is 1.3million tokens a month. Give some room for overhead and a reasonable rule of thumb;
The cost to run the machine per month is the cost per million tokens.
Yup, I hate to engage in anything that looks like a "shallow dismissal" but the project documentation seems to outright contradict itself wrt. whether it's running the model at genuinely native precision (though the claimed 3-bit quant is potentially interesting) and the headline claim of achieving 2 secs/token in a mere 29GB RAM footprint looks outright nonsensical given what we know about K3 itself (~115GB in dense parameters alone at native precision, plus ~25GB active sparse experts per token and some comparatively minor footprint for the KV cache). This is just not very helpful.
I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster. As developers, we can choose to adapt to new technologies or become extinct.
Please consider writing your Readmes by hand even if the code is computer-generated. I want to read what human authors think about their projects. I virtually never want to read what a computer thought.
This. Code written by LLMs using the new agentic SDLC is fine, can even be good code. But we shouldn't expect users to read sloppy slop, it's just plain lazy.
Also irrespective of the merits of LLMs, it's simply unpleasant to read LLM generated prose. It can't write well. The annoyance is compounded when you read the same poor writing everywhere.
I don't know how people that shovel AI prose don't realise this. Are they not also reading other people's shitty AI text?
I did see one sloperator who told his agent to copy his writing style. I have no idea if that works but it's got to be better than yet more "Here's the kicker" LinkedIn drivel.
Even worse than that, it's not the LLM, it's the operator. Mimicking your own style given a corpus is the lowest possible bar - you can also select among the great writers (technical or not) from history and have it give you eerily correct stylistic cues. Try Hemingway or Mark Twain next time you feel like generating a doc. Benjamin Franklin if you're feeling particularly Poor Richard.
There's no reason any more to read bad documentation. You can feed any docs in and get them in the precise style you want, so don't impose bad prose on others in your repos. Pick a solid technical writing base, jargon free, and go from there. I sometimes generate architecture proposals from the llm in RFC format and it's word perfect.
Yeah I'm begging these authors to at least *read* the LLM generated README's. They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development. These decisions and instructions from the user are extremely important for the model and almost completely irrelevant for an outside reader looking at a "finished" product. So, we get sentences like this:
"Where the levers were is not where they are. Overlapping the expert reads with the arithmetic was worth ~1.6x and shipped; the two that looked bigger — reading fewer bytes per token, and keeping more of them in RAM — were both measured and both refused, one because this family's router has no tail to demote and one because a cache the machine will not leave resident cannot be bought at any price."
What the fuck does that mean? Obviously some internal development decision, using the absolutely inscrutable internal terminology that Claude loves. If people would just read what they publish, I'm sure this would stick out immediately.
I'm not an LLM hater, I use them a ton and they work very well for writing complex code, it's undeniable. But they generate absolute dogshit first draft writing.
If that isn’t the perfect way to frame what I’ve seen and hated about LLM text, I don’t know what is. They certainly write for an audience with a historical context that almost no one has.
>They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development
The transformer does not yet understand the non-transformer.[0]
This is probably because all the data we trained it on was created by non-transformers, so it thinks it's a non-transformer, but it isn't.
I don't think we know how to train a transformer yet. All the training data is linear, but that's not how they think at all.
[0] It's a bit like the communication difficulties experienced between autistic people and neurotypicals. Each follow the Golden Rule, i.e. do unto others as you would have them do unto you -- and it fails in both directions. A Platinum Rule is necessary: do unto others as their API demands.
You don't need to presume. If someone is so lazy that they tell claude to commit their code (ie. they're too lazy to run git commit themselves), the chances they reviewed the code is slim.
That's a strange and arbitrary line to draw. There's plenty of times I make edits by hand and then tell an agent to kick it up to a PR, or on the other extreme, let an agent implement something autonomously, and review the diff myself once it's in PR. Both of those scenarios involve the agent running the git commands, neither scenario indicates "the chances they've reviewed the code is slim"
Yes, I do this all the time, and also check in the co-authored project plans which drove the commits. For a project that is transparently only possible due to agentic coding, I don't see any reason to conceal the methods.
I often let Claude write my commit messages even when I'm the one who wrote the code. Claude is often damn good at writing commit messages, and they frequently end up much better than if I wrote it all by hand. I nearly always edit them somewhat, but it's like starting from 80% instead of 0%. Some might call it laziness, but I call it working smarter rather than harder.
Honestly, Claude writes better commit messages than most people.
Personally I've mostly given in to letting it commit for me now, though I do occasionally take over and hand-write the messages if it's a particularly important concept and Claude's is too verbose.
Codex/GPT-x defaults to one-line commit messages, which are too short. Claude likes to write several paragraphs, which is usually too long.
If you tell it how to commit properly once per session it will stick with your standards for the rest of that session, and you can put that in AGENTS.md if you can be bothered to.
If my harness wrote the code, I have it write the commit. I will hand author something at a higher level that expresses my intent for how I promoted the harness.
That’s a needlessly antagonistic and insulting thing to say.
This person that you’ve never met and probably never will doesn’t owe any of us anything.
They’re out there doing what they want to do how they want to do it and if you don’t like it the correct response isn’t to insult them in front of a bunch of strangers on the internet for clout or whatever.
I doubt that you’d ever call them lazy to their face — why do it here?
Absolutely. Anyone can remove flag and remove typical LLM slop at least. I wrote a simple tool for it as part of my CI: https://github.com/jv-k/deslopper
yes. its a quality test. pre LLMs you could easily judge if a project was a labor of love by attention to details, like docs and README. Nowadays if even readme is sloped, what else was slop vibecoded? everything? how much effort was put in there besides 3 prompts? 5? you can never tell
i thought, here on HN, we were past the "oooh it's written by a LLM it's bad!".
I care about the craft, well designed systems, good clean architecture and code, etc...
But i also care about reaching goals. Whether i do it working on my own, or with human coworkers or with AI coworkers doesn't matter that much to me. Yes, the result is sometimes the most important thing.
Be advised that the firm behind it ("sqliteai") had a nasty history of using non-open source licenses, e.g. Elastic License. I advise against using anything by them for this reason even if this project currently has an open license.
So if this Mac uses 30-50W, that's 40-60 tok/Wh...vs maybe 80k for a modern GPU cluster? So that's about 1000-2000x more power for the SSD streaming, unfortunately.
Have it summarise the week overnight for the meeting in the morning. Then have it summarise the meeting transcription overnight for the report tomorrow. Then someone else will have it summarise the report overnight to read on a 6" handheld screen in the small office the next morning after breakfast.
Interesting project. The headline number (29 GB of RAM) is for 4k context.
From what I've read elsewhere, Kimi K3 is quite verbose in its thinking. At the quoted rate, it would generate only a total of 1.8k tokens in 1 hour. Is that enough for it to get any thinking done and produce output on more complicated prompts?
I saw someone’s excellent idea that if you have a slow system like this, you should communicate by email. It is no longer meant for realtime iteration, but more pointed questions for which there is more effort and time expected on both parties.
0.5t/s is still too slow even for email. For a moderately large inquiry (1MTok output, let's ignore the 4k context window limitation for now) it'll take the model around 23 days or uninterrupted execution to answer a single email.
Real world inquiries are gonna be much slower of course, but this setup is still too slow to do anything meaningfully useful I think.
Sure, you cannot do 1M, but there are plenty of useful questions you could ask that are far more modest. Simple Q+A, look at this function, how would you design X? All of those could have few paragraphs of outputs that would finish within a day.
Ok, this one will take just 30h (compared to that other project that would take 6.25 days) to start writing output tokens after you say hi in Claude Code.
because claude ships these verbose READMEs with its 'honest' takes and justifications for the naming. Its goal is to prime the next Agent that reads it, not you, human
Claude might as well be .5 tok/s. I end up waiting several minutes and what it tells me could usually be summarized in under 100 words.
So I could potentially live with this if it was concise.
Approximate calculation is putting the cost at ~$5 per million tokens (assuming 42W sustained, 20¢/kWh), and that's excluding hardware and other costs.
I got to the same conclusion another way. There's ~2.6million seconds a month and this is getting 0.5tok/s which is 1.3million tokens a month. Give some room for overhead and a reasonable rule of thumb;
The cost to run the machine per month is the cost per million tokens.
That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?
Yup, I hate to engage in anything that looks like a "shallow dismissal" but the project documentation seems to outright contradict itself wrt. whether it's running the model at genuinely native precision (though the claimed 3-bit quant is potentially interesting) and the headline claim of achieving 2 secs/token in a mere 29GB RAM footprint looks outright nonsensical given what we know about K3 itself (~115GB in dense parameters alone at native precision, plus ~25GB active sparse experts per token and some comparatively minor footprint for the KV cache). This is just not very helpful.
I wrote tons of software, even a programming language by hand https://github.com/marcobambini/gravity.
I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster. As developers, we can choose to adapt to new technologies or become extinct.
Please consider writing your Readmes by hand even if the code is computer-generated. I want to read what human authors think about their projects. I virtually never want to read what a computer thought.
This. Code written by LLMs using the new agentic SDLC is fine, can even be good code. But we shouldn't expect users to read sloppy slop, it's just plain lazy.
Also irrespective of the merits of LLMs, it's simply unpleasant to read LLM generated prose. It can't write well. The annoyance is compounded when you read the same poor writing everywhere.
I don't know how people that shovel AI prose don't realise this. Are they not also reading other people's shitty AI text?
I did see one sloperator who told his agent to copy his writing style. I have no idea if that works but it's got to be better than yet more "Here's the kicker" LinkedIn drivel.
Rewrite the readme in my voice, use all of my comments and responses in current context as source. Don't use em dashes or other LLM things.
Done, solved. Never had a problem with a readme or email since.
Even worse than that, it's not the LLM, it's the operator. Mimicking your own style given a corpus is the lowest possible bar - you can also select among the great writers (technical or not) from history and have it give you eerily correct stylistic cues. Try Hemingway or Mark Twain next time you feel like generating a doc. Benjamin Franklin if you're feeling particularly Poor Richard.
There's no reason any more to read bad documentation. You can feed any docs in and get them in the precise style you want, so don't impose bad prose on others in your repos. Pick a solid technical writing base, jargon free, and go from there. I sometimes generate architecture proposals from the llm in RFC format and it's word perfect.
> I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster.
The fact that the top comment on this thread calls it out, in a negative way, hints at that you aren't.
Non sequitur
Yeah I'm begging these authors to at least *read* the LLM generated README's. They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development. These decisions and instructions from the user are extremely important for the model and almost completely irrelevant for an outside reader looking at a "finished" product. So, we get sentences like this:
"Where the levers were is not where they are. Overlapping the expert reads with the arithmetic was worth ~1.6x and shipped; the two that looked bigger — reading fewer bytes per token, and keeping more of them in RAM — were both measured and both refused, one because this family's router has no tail to demote and one because a cache the machine will not leave resident cannot be bought at any price."
What the fuck does that mean? Obviously some internal development decision, using the absolutely inscrutable internal terminology that Claude loves. If people would just read what they publish, I'm sure this would stick out immediately.
I'm not an LLM hater, I use them a ton and they work very well for writing complex code, it's undeniable. But they generate absolute dogshit first draft writing.
If that isn’t the perfect way to frame what I’ve seen and hated about LLM text, I don’t know what is. They certainly write for an audience with a historical context that almost no one has.
>They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development
The transformer does not yet understand the non-transformer.[0]
This is probably because all the data we trained it on was created by non-transformers, so it thinks it's a non-transformer, but it isn't.
I don't think we know how to train a transformer yet. All the training data is linear, but that's not how they think at all.
[0] It's a bit like the communication difficulties experienced between autistic people and neurotypicals. Each follow the Golden Rule, i.e. do unto others as you would have them do unto you -- and it fails in both directions. A Platinum Rule is necessary: do unto others as their API demands.
>Contributors
>...
>claude
You don't need to presume. If someone is so lazy that they tell claude to commit their code (ie. they're too lazy to run git commit themselves), the chances they reviewed the code is slim.
That's a strange and arbitrary line to draw. There's plenty of times I make edits by hand and then tell an agent to kick it up to a PR, or on the other extreme, let an agent implement something autonomously, and review the diff myself once it's in PR. Both of those scenarios involve the agent running the git commands, neither scenario indicates "the chances they've reviewed the code is slim"
To be fair, I appreciate when they are so upfront about who wrote the code without requiring further heuristics, so I encourage this behavior.
Yes, I do this all the time, and also check in the co-authored project plans which drove the commits. For a project that is transparently only possible due to agentic coding, I don't see any reason to conceal the methods.
I often let Claude write my commit messages even when I'm the one who wrote the code. Claude is often damn good at writing commit messages, and they frequently end up much better than if I wrote it all by hand. I nearly always edit them somewhat, but it's like starting from 80% instead of 0%. Some might call it laziness, but I call it working smarter rather than harder.
Honestly, Claude writes better commit messages than most people.
Personally I've mostly given in to letting it commit for me now, though I do occasionally take over and hand-write the messages if it's a particularly important concept and Claude's is too verbose.
Codex/GPT-x defaults to one-line commit messages, which are too short. Claude likes to write several paragraphs, which is usually too long.
If you tell it how to commit properly once per session it will stick with your standards for the rest of that session, and you can put that in AGENTS.md if you can be bothered to.
Why do you say lazy? maybe they are ok with people seeing it is claude?
If my harness wrote the code, I have it write the commit. I will hand author something at a higher level that expresses my intent for how I promoted the harness.
That’s a needlessly antagonistic and insulting thing to say.
This person that you’ve never met and probably never will doesn’t owe any of us anything.
They’re out there doing what they want to do how they want to do it and if you don’t like it the correct response isn’t to insult them in front of a bunch of strangers on the internet for clout or whatever.
I doubt that you’d ever call them lazy to their face — why do it here?
Yes, but very slowly.
Absolutely. Anyone can remove flag and remove typical LLM slop at least. I wrote a simple tool for it as part of my CI: https://github.com/jv-k/deslopper
Does it matter?
I need an llm to block these comments.
yes. its a quality test. pre LLMs you could easily judge if a project was a labor of love by attention to details, like docs and README. Nowadays if even readme is sloped, what else was slop vibecoded? everything? how much effort was put in there besides 3 prompts? 5? you can never tell
do people think these projects related to LLMs are ever going to be in anyway a pure human endevour?
How bout we make a new rule: only complain about LLM writing when the product as zero relevents to use with LLMs.
i thought, here on HN, we were past the "oooh it's written by a LLM it's bad!".
I care about the craft, well designed systems, good clean architecture and code, etc...
But i also care about reaching goals. Whether i do it working on my own, or with human coworkers or with AI coworkers doesn't matter that much to me. Yes, the result is sometimes the most important thing.
Be advised that the firm behind it ("sqliteai") had a nasty history of using non-open source licenses, e.g. Elastic License. I advise against using anything by them for this reason even if this project currently has an open license.
Are these the sqlite developers..?
How does this project compare to https://github.com/gavamedia/deltafin ?
So if this Mac uses 30-50W, that's 40-60 tok/Wh...vs maybe 80k for a modern GPU cluster? So that's about 1000-2000x more power for the SSD streaming, unfortunately.
Neat! But, what do you do with a 0.5tk/s LLM?
Have you tried running it via llamacpp or other software that supports naive SSD offloading to compare speeds?
Have it summarise the week overnight for the meeting in the morning. Then have it summarise the meeting transcription overnight for the report tomorrow. Then someone else will have it summarise the report overnight to read on a 6" handheld screen in the small office the next morning after breakfast.
Then have it summarise the meeting transcription over the entire week for the report next week.
ftfy.
You could use it for long run tasks while you don’t use the laptop.
By the time the tokens start coming out 30h later you might need to use your laptop again...
Fun. I see the novelty.
This sounds a lot like what the colibri project did for GLM-5.2. I'm a fan so keep at it!
justvugg.github.io/colibri
Interesting project. The headline number (29 GB of RAM) is for 4k context.
From what I've read elsewhere, Kimi K3 is quite verbose in its thinking. At the quoted rate, it would generate only a total of 1.8k tokens in 1 hour. Is that enough for it to get any thinking done and produce output on more complicated prompts?
I saw someone’s excellent idea that if you have a slow system like this, you should communicate by email. It is no longer meant for realtime iteration, but more pointed questions for which there is more effort and time expected on both parties.
0.5t/s is still too slow even for email. For a moderately large inquiry (1MTok output, let's ignore the 4k context window limitation for now) it'll take the model around 23 days or uninterrupted execution to answer a single email.
Real world inquiries are gonna be much slower of course, but this setup is still too slow to do anything meaningfully useful I think.
Sure, you cannot do 1M, but there are plenty of useful questions you could ask that are far more modest. Simple Q+A, look at this function, how would you design X? All of those could have few paragraphs of outputs that would finish within a day.
I’m sure it’s possible, but I really struggle to think of an example that would result in a 1m token output.
it would be 1M tokens worth of work, with some small report for the end
Ok, this one will take just 30h (compared to that other project that would take 6.25 days) to start writing output tokens after you say hi in Claude Code.
I couldnt find anything explaining the name of this company on their website but is it okay that they’re riding on the name of an open source tool?
SQLite code itself is public domain but I’m not sure about the name.
It's too bad Optane PMem is dead
Does it not use Metal, on macOS? Would it be faster if it did?
We tried to use Metal, but for that specific project it was slower than just using NEON ARM optimizations. It is all documented in the docs.
Dear creator: you didn't ship the first draft of your code - why did you ship the first draft of your README??
because claude ships these verbose READMEs with its 'honest' takes and justifications for the naming. Its goal is to prime the next Agent that reads it, not you, human
we need a new HUMANS.md to replace the readme
Where can this 1tb k3.waste be downloaded?
It is not yet available, the only way is to download the official Kimi K3 model and then convert it:
# 1. preflight: reachable? how big? does it fit? tools/fetch_weights.sh --dest /Volumes/staging/k3 --dry-run
# 2. download — resumable, safe to kill, safe to re-run tools/fetch_weights.sh --dest /Volumes/staging/k3
# 3. convert into a container uv run --with torch --with safetensors python tools/convert.py \ --src /Volumes/staging/k3 \ --out ~/models/k3.waste --jobs 3
Yeah, saw this ... was hoping that there's a torrent of it somewhere already. Or something.
looks helpful
your readme is overly verbose
agents don't need that and its extremely low signal for humans too
tell your language model to get it to the point