To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture.
I'm now working on a new, not yet published version of pgrust that incorporates a lot of techniques. Currently the new version:
- Passes 100% of Postgres regression suite
- Implements a thread per connection model instead of the process per connection model Postgres does
- Is 50% faster than Postgres on transaction workloads
- Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse
If you have any questions, I'm happy to answer them.
A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.
That’s not true for Postgres however: due to its usage of a shared memory pool, whenever a subprocess is terminated unexpectedly, Postgres will kill all other processes and enter recovery mode, replaying the WAL, during which time it will not accept connection requests.
It does this because it can’t possibly know whether the dying process did bad things to the shared memory pool.
Some see a 30 year old system and think "outdated", I see a 30 year old system and think "time tested."
Clearly a process per connection is more stable and that's what I'm using.
It's unclear what problem such optimizations are solving anyway, with the old way you could only support a million concurrent users with a single server? Are we missing out on supporting ten million concurrent users with 2 servers instead of 10? Ostensibly reducing the minimum db hardware opex for a 10B$ company from 10k$/month to 2k$/month?
Doesn't postgres (rightly) have a cow if a process has a disorderly shutdown (at least while in a write transaction) because there's shared memory between the processes?
I'm not informed of the Postgres's internals, but, maybe, that can be solved by grouping threads into different processes depending on which set of extensions they request.
An OS thread per connection can be fine for performance if you don't have to scale your connections, but if you don't need to scale connections why have connections at all? Databases are even more performant when you eliminate connection overhead entirely.
I don't want to knock you down as most have already did. In-fact it's a useful exercise going forward in exploring how to work with AI. It's here, we're all going to use it one way or the other. Zero issues with that, in-fact kudos to going through the pain of it all.
Now, having gone through several such endeavors originally myself, albeit with internal tools and systems (as an exercise), I've noticed that while all my tests passed with flying colors the rewrite itself was broken even on basic functionality or missed a ton of details. It was in effect useless when I dived into it. Initial tests also showed massive gain in performance, and I know people who were involved aren't really dumb so something smelled funny. Turns out all those things left out and honestly... moments were the key ingredients.
What I did learn from those beginning explorations though was that one-shotting, grand architecture or source up-front, master plans up-front.. all these do not yield good results - YET. Who know what we'll see in few years though. What I did found that works (FOR ME, nota bene) is to keep the design and checklists for myself, written by myself and then do a small piece by piece.. as if you would if you were coding alone or if you would waterfalling a small team of talented juniors. Then, suddenly super happy results come out, but then it's mostly you driving all the way where llm writes code and offers advice (which for the most part you ignore). It's a happy place for myself at least. It's then truly unlocking yourself to the mythical 10x.
Rewriting a large proven system with decades of ultra expertise behind it, which I don't have, is guaranteed not to end up the same 1:1 replacement. If you found a recipe for that - please do share.
I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?
I don't code with LLMs, and think you might be right.
However, Postgres is a tool with clearly defined functionality and doesn't have ambiguous requirements that are seen in user facing software. As a result, it is entirely plausible that the author created a working Postgres replacement for certain use cases.
I personally want to see more evidence about the quality of the tool after it is in a finished state.
what model did you try it with? I agree and also push back a bit: How will we know when the LLMs reach the point of handling it if no one takes the leap? I applaud more people sludging through the slop and hauling their slop buckets around.
An example is Fable being released. I felt like the most complex thing I was willing to sludge through was having it clone llama-server's web UI with my own opinions (I really like the original, kudos to them). And the initial skeleton was working so well I felt like I had sunk the tokens and committed to getting it the rest of the way: https://inkcap.click
Ouf. I don't know. I don't want to call you out without evidence -- I myself make benchmark claims all the time -- but 50% improvement in OLTP seems suspicious. I get that you used a standard benchmark, and I don't even know what it entails, but my spidey sense is going off. Perhaps, some trade off somewhere that won't make it to prod because it breaks MVCC -- and yes, I saw that it passes regression tests.
Just checking, is fsync on? :) Regression tests don't catch bad IO patterns afaik.
Remember when databases were faster to run in virtualbox rather than bare metal? (because virtual box was completely ignoring all the instructions to flush the data on the disks)
Yeah, claims that it can be faster than CH are very suspicious. CH guys are very good at their craft, they spend hundreds of hours optimizing one single small detail.
It'd be very unfortunate if Postgres didn't have regression tests for data loss due to bad io patterns. Should be possible to do some checks against those in an appropriate test harness. Which might mean "have qemu run something we can kill off and examine the results".
If those don't exist, I hope folks recognize how useful they are and add them.
What's your actual background and expertise with Postgres and databases more broadly? Basically, do you actually know what you're doing, or is there likely a massive footgun you don't know or haven't shared with us?
I spent a couple years managing a Postgres cluster with a petabyte of data. I wrote a couple blog posts from my work then[0][1]. I also wrote dozens of posts on the Postgres internals[2]. I've also given talks on how to generate fractals with SQL[3] and how to write a lisp interpreter in SQL[4].
> - Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse
That sounds like you are storing the data in a columnar format? Or do you do both row and columnar?
In a somewhat similar (yet also quite different) effort, I've been working on δx, a Postgres extension that compresses the data in a columnar format stored in normal Postgres tables (so replication, crash recovery, pg_dump, etc. still work normally). https://github.com/xataio/deltax
Yep! The new version of pgrust supports batch based execution and a columnar format. I'm curious how you got δx to perform that well? From what I've seen a columnar layout only gets you part of the way and really good parallelism and really fast hash tables seem to make up a significant portion of why Clickhouse is faster.
It doesn't sound like you were trying to launch a product, but doing an experiment and someone threw you under the HN-spotlight-bus :) Is this a "see what I can achieve with LLM coding" or is this "build this and see how much of the coding can be accepted from LLMs"?
What was your methodology and structure in making the prompts for the rewrite? Did you let the LLM roam in all of the codebase and tests from the beginning, or revealed things to it gradually in some way?
Are you fixing the heap and table management ?. Postgres does not use an undo log and manages all table updates directly in table storage which slows MVCC.
That's something I eventually want to fix. The challenge is the storage format is so integral to Postgres that it's going to be a huge PITA to come up with a novel design.
Right now OrioleDB is in beta. Once that becomes production ready, I'll evaluate incorporating it into pgrust.
"Is 50% faster than Postgres on transaction workloads" - That is a very big claim! 50% faster on everything? Is it a strict improvement across the board or are there tradeoffs that make some workloads slower?
The 50% is specifically on percona-tpcc[0]. I got there through a mix of batching (postgres processes a row at a time), prefetching, and several handful of other optimizations.
While a thread-per-connection seems like an improvement, do you have any plans to allow query multiplexing over a single connection? That would be a huge improvement IMO.
Can you elaborate on the use case for query multiplexing? Is it so your client would only need to establish one connection with Postgres and then could run as many queries as it wanted?
Rust actually made the change pretty simple. The main changes are:
- Use thread local variables
- Move everything from shared memory to process memory
- Use threads instead of processes
I've started to see meaningful benefits by changing the parallel algorithms to use a shared memory space. For example parallel hash joins have to copy tuples through shared memory to pass them between workers. That's just not something I have to do.
It's not used in production. I've been using different benchmarks to compare the performance vs other systems. Namely sysbench-tpcc[0] and clickbench[1]
My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust.
This naturally lended itself to a suite of skills to describe how to rewrite a crate from unsafe rust to idiomatic rust. The main three skills I had were 1) a skill for identifying the next crates to port 2) a skill for rewriting a crate and 3) a skill for auditing a crate and making sure there weren't any outstanding issues.
My exact approach for managing subagents changed throughout the project. Initially I was doing parallel coding sessions with Conductor. After dynamic workflows came out, I used that as it was really easy to spin up dozens of parallel subagents and manage it from a single orchestrator. Over time I switched from using dynamic workflows to manually spinning up subagents from a central agent. The issue with dynamic workflows is they waterfall. Each step needs to finish before the next one starts. By manually spinning up subagents, I could have claude start porting a new crate as soon as a prior subagent finished.
I am super curious how you went about the port using LLMs. At $WORK we are looking to port code, preferably with LLMs, and it seems daunting, even with a test suite. Do you have an approach that works well for you?
Just a couple of ideas if you run out of backlog:)
- proper versionnumber (64bit)
- native json streaming. It would be awesome to get to the point where i could somehow redirect the sql output to the browser directly, but piping will do for now. The idea is to be able to stream rows to client without caching and building json along the way.
It's a completely new era of software production (I will no longer call it development)
LLMs give us unlimited manpower, and the language give us constraints to make more modern and safer softwares.
Love to see this rewrite in Rust, and expecting much much more in next few month.
How much of the performance gain is from using Rust, compared to using optimizations that are not done in the original PostgreSQL code (like using threads instead of processes, etc.)?
I am simply curious what the benefits of using Rust are in this instance.
Nothing major yet. Once I wrap up the performance work I'm doing I'll start looking at the best way to go about testing. I suspect there's a lot of novel things you can do with agents.
when doing rewrites like these, why isn't the first step to instrument the original code so that you would get very good automated test suites to point the LLM toward?
use both synthetic and real data to sample the internals of the original software to duplicate.
locate all the data transformation junctures, sample and then replicate the tranforms 1:1 in the rewrite.
Awesome work. I'd love to see you add something like kusto query language or pql. The autocomple on kusto, (which can be embedded into web apps) is really amazing.
This is how to LLM. Big ups, I wish the whole front page was stuff like this (and I think it'll happen).
Everyone is so worried about the value of commodity software going to zero. It's like, yeah, going into CS for the money always looked dumb to me, it's just not a good career path for that, you have to love it.
I am way more excited about a whole new class of stuff that obliterates the state of the art at every frontier.
- typically they are behind a single person. That’s usually bad because of spf
- typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term
- anyone that wants to contribute to the project needs to pay. Needs to pay tokens because it’s increasingly difficult to maintain these projects without AI
So, who wants to put something like this in production? Doesn’t make much sense
I'd be interested to hear the author's answer to your question, but I see it as an interesting proof-of-concept. It's testing the viability of not only rewriting PostgreSQL in Rust (and their choice of deps) but also in switching the threading model and other architectural changes. LLMs shine at pumping out prototypes insanely fast, and a working prototype can put an end to a lot of speculation.
I likely wouldn't use a rewrite of such a huge project if it doesn't have the backing of the original team (or a significant fraction thereof) and a believable story for having matched/exceeded the original code quality and maintenance. I also think in general using an LLM for license-laundering is legally and morally hard to defend, although this case is different in that they chose a more restrictive license. Not a lawyer, but my understanding is that you can just download PostgreSQL, do s/MIT/AGPL/ and release it, legally. (The original MIT-licensed version still exists, so no reason anyone would prefer yours until you make another release with some compelling new feature.)
It's a very interesting phenomenon in recent years and most of the discussions about "why" are immediately blocked by the "memory safety" argument, as if it's a silver bullet for all things that are considered "bad" in software implementations.
No matter how good the language is, and I consider Rust a very good language, it's practically impossible to replace years of experience and tested code, most of it contributed by a ton of brilliant programmers, no matter how you look at it.
And if we take this as the truth, then logically there's still an unanswered question - why? Lets take an undeniable fact - rewriting an existing project give you full control over the new implementation. You can do whatever you want with it, you can't be sued. The only thing now you have to hope for is for your implementation to gather a good enough user base and from then on you can practically hijack the original project.
And this is me speculating - rewriting stuff in Rust isn't about the greater good for that magical "memory safety" argument, but at the end it's an attempt to hijack popular software projects.
> it's an attempt to hijack popular software projects
Exactly. And this is why those projects are typically called "XXX in Rust" or "XXX-rs". Because the creators get to do their favorite thing - coding in their loved language - while skipping all the hardships of designing, accepting real feedback, involving users and getting traction - all while simultaneously hijacking the existing brand.
As of now I know of a single project that changed their name after being called and the project surprisingly got some traction.
I think we're a year or two away from the same argument being used by languages which enforce proof of correctness. The economics seem to be shifting in that direction.
Finding exploits is getting exponentially cheaper, and the cost of producing proofs is rapidly going down. For a lot of software correctness is rapidly becoming non-optional.
The gp feels like less of a targeted individual criticism & more of a general musing about a trend.
Nobody is saying this author hasn't demonstrated discipline & won't maintain this project, but the statistical averages across most projects fitting this trend make it likely enough to question their worth in aggregate.
Because chances are that it's never going to be production ready, never trustworthy enough to be used by anything serious, and the project will eventually be archived and become at best a (extremely inefficient) learning experience for the author, or at worst a total waste of tokens. I could be completely wrong about this specific project, but most of these projects are like that, and statistics don't lie.
The author is free to ignore any and all complaints they consider unfounded. It’s not even like the author is recieving any complaints personally; they have to come here to see any. And if they come here, they will get to read the viewpoint visible from here.
But this is also just silly. The maintainer here did nothing (just a little chat here and there), but is sure that only optimizations are missing for this to be production-ready. How so, based on what exactly?
It's not just a rewrite ; it has improvements. I did the same thing for fun for the same reason; I wanted to see if I can improvements on some of the legacy design stuff and, especially, the stuff PG people have told us that it cannot be done differently. It can. I would not put it in production, but it thought me a lot about the internals of databases. To keep my brain happy in the age of LLMs, I implement database things on our (also old but many times refactored/rewritten) production db without an LLM. I'm sweating through Flexible Paxos now; probably we will just keep using raft as it's old and stable and simple but it's interesting anyway.
but can you? You cannot do a Postgres clone yourself and the Postgres arch makes it rather impossible to just rip out stuff and replace it. So how would I do this. Even with SQLite and me being a very experienced (40 years) c embedded coder; having AI rewrite it to the architecture I would want it to be and then changing things for fun is a better tutor than whatever I could have done with the original in a shorter time.
1. Nobody, not even the Googles or NSAs of the world do that. No single entity has the expertise nor resources necessary to maintain a fork for every open source project they use -- forking and maintaining Linux alone takes teams of people. And no, going full psychosis mode with LLMs is not going to save you.
Forking and maintaining Linux does not take teams of people. I've been at it for > 10 years and maintained support for my 15 or so various SBCs/mobile devices, writing drivers, debugging, cleaning things up. It's a weekend project every 3 months + whatever you want to put in for development.
And there are others doing it for projects like Armbian, openwrt, etc.
I'm not sure I see the value in "showing the team it's possible". I would presume the team are intelligent enough to be well aware that it's possible, given tradeoffs. And as it's clear those tradeoffs have been "traded" in this case, it doesn't seem like having the knowledge confirmed is particularly surprising.
Not the same but it is much faster and easier to re-create a 3d model of an existing set of drawings than from scratch. This is because a lot of the decisions have already been made.
>typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term
Lindy effect! The longer something has been around, the longer it probably will be.
I guess it is cool to have it around but it comes off as a popular useful case for AI which it really isn't, unless you have a very good test suite to throw the agent against, it is practically useless for rewrites of decades old badly documented code riddled with technical debt. That is where the real value and challenge would be. I see it as marketing and social clout stunt
How would one go about reviewing a piece of code like this?
One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project. But with LLMs generating 7101 commits in less than a month that isn't feasible. Even looking at a single day is way too much [1]. It probably also doesn't make sense since the commits content won't tell you much anyway.
ps. How do you easily get to the first commit in a repo on GitHub? Browsing commit history feels rather tedious
These rewrites are just test-driven development taken to the absolute extreme. Created under the hope that the existing tests are exhaustive and cover every relevant use case, such that if they all pass, the rewrite must be at least as good as the original. So just go with the vibes and burn tokens until they pass, and your job is done.
In practice, this is never true for any codebase above a certain level of complexity, especially not one as mature and widely used as Postgres. But reality doesn't seem to be an obstacle for vibe coders.
The challenge is that more and more people are producing project like this - 1,000s of commits and > 200k lines of code - and saying it was carefully created using agent based workflows and not vibe coded.
One of the projects Im working on and off is a tamper-proof audit log, based on some PoC code I created almost 10 years go; unit and integration testing are good at preventing defects and regressions, but they will not guarantee your software will work. However, with the power of LLMs, one can easily use model checking (in my case with Quint) and/or other formal proof approaches to ensure the software conforms as specified. The result (in my opinion) is an implementation guided by a single human that is actually more trustworthy than manual human-made software using the traditional approach.
I think the focus for projects like this is going to shift to reviewing the testing/fuzzing process instead of reviewing each commit (going much further than what the postgres regression/isolation/crash tests do).
Some of this post reminds me of a story I heard long ago from someone who had worked at a HW/SW company. They’d transferred an engineer from the ASIC design team to the OS kernel team, though he’d never been on a software team before. After a while the manager called him in for the following conversation:
Manager: You’re doing amazing work — zero bugs in production! I’d like you to mentor the other SWEs on how to get their bug count down too.
For large projects like this I think a hierarchical division of labor also helps.
If you first carefully define the overall architecture and thus individual high level components of the system, then you know which of those components are mission critical and which are commodity. Mission critical would be anything ensuring ACID, etc. That way, no matter what you farm out to LLMs, you can keep the majority of limited human focus on the far fewer mission critical components. If tests end up not being robust enough to catch all issues, at least they'll be isolated to commodity code where damage is limited to things like DoS, etc, and not code that could cause data loss.
I also think it's important to first define the _contracts_ on and between each of these components, and derive tests from those contracts. Partly because contracts more succinct and easier to reason about. And partly because Rust provides many tools to enforce contracts at compile time, reducing the need for tests (which themselves could end up subtly flawed). Contracts can be enforced through typing, private vs public APIs, etc. Newtypes are _incredibly_ powerful for both enforcing contracts and making footguns much less likely.
Though beyond testing, I think there will be increasing focus on proofs of correctness. (Testing can only show the presence of bugs, not the absence. —Dijkstra)
At any rate, it's never been this cheap to produce the proof of correctness of a program, or on the other hand, to produce an exploit for an incorrect program.
The lock format is a multi-line TOML, with a varying number of lines per dep due to redundantly listing deps-of-deps, so a naive line count massively overstates the number.
Cargo.lock contains many unused dependencies, because it's a superset of all combinations of all optional/disabled features of all transitive deps across all possible platforms (so that the deps don't reshuffle even if you enable/disable feature flags or compile on another platform). But that means Cargo.lock is going to have 3 async runtimes even if you use one. It's going to have syscall definitions for RedoxOS and wrappers for WASM, because some dep of dep is compatible with those platforms. But these deps won't even be downloaded if you don't build for these platforms.
The number you got presented is not representing the unit you're insinuating. A crate in Rust is a compilation unit. It's common for projects to ship as a collection of many crates. It's a smaller unit than what C counts as one dependency, and slightly coarser than an .o file. I don't see people freaking out by how many .o files their projects have, including all transitive ones from deps like openssl or curl.
Do you pick your databases based on how quickly they compile and how many dependencies they have? I normally chose based on factors like performance and reputation for reliability
Disaster. Well, I read stories about Rust and how there isn't much in the stdlib, but this is just too much. How many dependencies are there, on average, in other projects? I guess I am spoiled with Go.
In a typical Rust project you organize your project into many crates. I haven't checked, but I'd guess that the vast majority of those dependencies are internal dependencies. After all, this project started by running an automated C to Rust converter, which the authors claimed produced over a thousand crates.
In general (I’m not saying this is the case with this project) if you don’t have their prompt history and you can’t re-run the LLM “compilation” yourself, is it open source? It feels a bit more like those “source available” projects where you can read the code but don’t have access to the build system.
On the other hand, aside from the commit messages, one didn’t ever have access to the underlying thought process of human developers either, so maybe it’s not equivalent to say that secret prompts mean closed-source.
What an incredibly bad take. "It's not open source because we have the source but not the thought process of the developer" - well then no project on this earth is truly open source by your definition.
> One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project
I could not care less about any of this. Truth is code, as it is now. I don't care when (and certainly not by who) a bug got introduced, it's here, shut up and fix it.
> How would one go about reviewing a piece of code like this?
That's a wrong question. The right question is "why would one go about rewriting a piece of code in X". Once and if you find a good answer to that question, you will see the answer to your's.
I think the best way to test this would be to put PgBouncer or a similar proxy in front of a busy production database, and mirror queries to both traditional Postgres and the Rust one at the same time. Then you can compare output and performance under real load. After running it for a while, you could diff the tables one to one against the normal Postgres instance.
2664 "unsafe {", 1835 "unsafe fn". This is completely unsafe. It doesn't look like a rewrite that understands what's actually going on or how the architecture should be redesigned to take advantage of Rust strengths. Instead, it looks like an AI generated transpilation with extensive use of raw pointers.
Note that most of the unsafes are confined to the parser which was generated by running c2rust over the Postgres parser. The Postgres parser is itself generated from yacc/bison, so I decided to port it over mechanically rather than idiomatically.
If there's particular unsafes that you think are egregious, let me know.
Counterpoint: All of the current Postgres codebase is already wrapped in an invisible unsafe{}.
The difference with a Rust codebase like this is that all of the unsafe code has been neatly isolated and clearly marked. The outside code is safe — at least according to the definition of what Rust considers safe, which is a high bar indeed and objectively superior to the unsafe mess that is C — and the unsafe code is naturally fenced in, which means that it can be seen by developers and tackled by incrementally.
In some cases unsafe is unavoidable, but it is possible for a human to verify that it is, in fact, acceptably safe even if inside an unsafe block.
This is impressive - but is a license change, from the PostgresQL license [0] to AGPL [1].
I like the AGPL and think it's the best truly free open source license, but I worry if this is compatible. Ie, if this is rewritten from the original source, should the original apply? (Yes.) There has been a trend to rewrite open source software with a more restrictive license (like coretools in Rust). This looks considerably more ethical by choosing the AGPL - I just wonder, safer with no change at all?
You seem to have the restrictiveness backwards? The MIT license (uutils coreutils) is less restrictive than the GPL (GNU coreutils), and the AGPL is more restrictive than the PostgreSQL license.
And it doesn't violate the PostgreSQL license to license the rewrite more restrictively. That's part of what makes MIT-style licenses less restrictive than the GPL or AGPL: they allow for more-restrictive relicensing.
If you don’t like the license just let an LLM spend a few days “porting” it and give that port any license you like because that is apparently what we do now.
The PostgreSQL License is a variant of the BSD license and is therefore compatible with the (A)GPL.
Comprehend it this way: You create a blank (A)GPL project and incorporate the upstream BSD codebase into it. While those original upstream files remain under their original permissive license, the project as a whole is governed by the (A)GPL (plus the attribution requirements of the upstream license, which the GPL permits). From there, you can add your own code under the AGPL and distribute the combined work under the AGPL.
If someone takes your code and uses only your portion, they can use it under the AGPL alone. However, if they also include the upstream source code, then the attribution requirements of the upstream license must still be met.
AGPL and copyleft licenses are not restrictive, they provide forward (open source) guarantees. The only thing they "restrict" is the ability to remove freedoms. Therefore is not a restriction, is a guarantee. A guarantee that the project will endure as open source.
I wouldn't go as far as calling permissive licensing "restrictive" because they actually allow for reducing freedoms; but copyleft is definitely not restrictive at all, it's the opposite.
Having said that, I like and appreciate both kind of licenses and I have and will continue creating open source software using ones or the others.
For instance, the TypeScript rewrite in Go was done mostly by humans and took a year before it was released. That is how you rewrite software that people can trust.
Not sure it’s so simple. I think close to 100% of new ambitious projects are going to leverage AI at least to some degree. I know a couple that have strict no-AI policies (e.g. Zig), but it’s a tiny minority i think.
So how much AI usage does it make it an “AI rewrite”?
I agree but I think from Bun we learned that a project with really good tests and enough tokens can be converted from one language to another quite good!
It is more and more the future. No human would want to rewrite one technology to another because it is too marginal a gain. AI on the other hand does not give a shit.
Is there any measurable difference in quality between the two, or are you just going on "vibes"? Is there a correlation between the quality of the manually written code and AI generated code driven by the same dev?
Such crude takes only cause unnecessary friction. If you have a black box that spits out code, and you are unable to distinguish the quality between a top tier dev and an AI inside the black box, then the distinction is unnecessary. Most of the code on the internet is already a black box to you. What percentage of code running on your machines have you vetted by who wrote it and code quality?
AI coding isn't going anywhere and will likely end up generating most code going forward so instead of rejecting it outright or arbitrarily categorizing it we need to focus on solid quantitative and qualitative measures of code and functionality regardless of who wrote it.
If you can do a Rust rewrite with AI, I can create one as well. What makes yours better than mine? Your decade long expertise in database or Rust language? Your reputation? Your proven track record to manage large, complex projects? Your time committed to the project? I don't see any of that.
I don't know why anyone would choose this over the actively (community) maintained proper Postgres project.
His project is cool. Students could use it as an example of porting code. Companies could switch to it, if it works. There are hundreds of reasons why people may want it, so it's awesome he's publishing one. Something needs to be done to get interest and then adoption.
Just keep using the proper Postgres project if it makes more sense to you. But notice that this isn't just a strict translation, it actually makes changes (moving from process to thread based design, for instance). Time will tell if people find these changes beneficial or if the original remains preferable.
It's also worth noting, that while you are able to use LLM's to produce your own translation, he's actually done it. There's value in actually putting in the work.
This isn't a unique situation at all. Many Postgres extensions are developed and maintained by a single person, and may therefore be avoided by more conservative users, even if they offer some technical advantages. To each, their own.
There’s no claim being made that your rewrite cannot be better.
He has provided benchmark results which provide a dimension amongst which to measure your rewrite. If you can do better by all means post your rewrite.
Finally, these kind of projects can eventually over time become projects that are actively used. Postgres is not some entity that existed before the universe was created; it was also created by someone and then eventually adoption picked up over time.
Came here to the same thing. I did something similar (but bolder, it doesn't slavishly copy Postgres and is based on current DB research papers and the like, and other bits I've been exposed to over the years). It has a full TPC-C-esque benchmark suite, replication, embedded v8/JavaScript relations/stored procedures, a giant suite of regression tests, it kicks the crap out of a lot of existing OLTP DB stuff out there. And I personally do have a background in commercial DB development.
But I choose not to publish it or promote it for many of the reasons you mention above (and more)
... For one, if "I" can do it, so can a hundred other people. And all the bold claims behind it would need to be backed up and supported and it promoted, etc, which is a whole pile of time that doesn't involve writing code.
It's the organization around a project that matters, not the code. It's not the 90s anymore w/ people piling into MySQL because it was the only option. People aren't going to be trusting your software with their data, if they can't trust you.
And unless someone is going to dump a pile of money or something on [me|them], I don't have the ability to build that organization ... as I need to feed my family... Nor am I willing to put my personal reputation on the line by putting up a huge quickly written application and then someone finding something in it I can't explain.
So like probably 500 other projects I have it sitting in a private repo.
It's a very weird time right now. "Technical" excellence isn't the important part. Organizational excellence is. This was always the case but it's more so now.
... In the meantime, if anybody has angel investment to burn, I have something potentially better/more-exciting than this guy's project but... see above...
I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.
> not mostly the test, but the real world production scars
Most extensive test suites are exactly production scars: every time you have a bug or a regression, you write a test that confirms correct behaviour.
SQLite is a good example to bring up because its extensive closed-source tests are what’s often cited as being what keeps people from forking it. (Turso did it, though, but it takes a company to deliver some guarantee of equivalent diligence.)
I agree. I also agree with the sibling reply that -
> every time you have a bug or a regression, you write a test that confirms correct behaviour.
What I fail to see in these rewrites however is - what about new bugs introduced by virtue of this rewrite? I mean it'll have to go through its own challenges in real-world scenarios, right?
> I start to see a lot of these re-writes that depend on tests to state that its working.
There's another way to validate the rewrite though. Just run both pgrust and postgres and compare the output. Know of an edge case? Run it too. Doesn't know? Use a fuzzer or some automated tool to find interesting inputs. Found an inconsistency? The input/output pair becomes a test case now
Not sure if there's tooling for that though. If there is, just give it to Claude so they will incorporate it in their development loop
Not weighing in on this specific rewrite but tests are how you specify that your software works correctly. If a behavior isn’t covered by an automated test in some form you can’t assert that any given change doesn’t break it.
I think it is completely reasonable to use a preexisting unmodified test suite to state that something is working. The larger the project the more true this becomes. Real world production scars are documented and guarded against in the test suite otherwise those lessons get lost.
Also SQLite is legendary for its massive test suite and extensive fuzzing. They have 590x the amount of test code and scripts than normal code. Source: https://sqlite.org/testing.html
So, we should make it easier to feed that reliability back upstream.
Probably the most useful thing you can do with these LLM-transpilations for now: If the transpiled version passes all original tests, I can run my application test suite against it and use it to discover test coverage deficiencies in the original!
If it crashes or otherwise observably misbehaves, I know the real project was missing regression tests for something. We could make upstream so much more resilient against accidentally breaking stuff in future updates, if only it becomes safe (offline + no side effects) and easy (if it crashes/locks, it is not from some memory safety bug from 25k transactions earlier) to run these transpiled projects as one row in our everyday integration matrix.
As sibling mentioned - bugs and regressions are the thing that are (in a perfect world) usually covered.
The problem however is non-covered success cases. A visualisation of the problem: let's say universe of interaction for DB consists of 10.000 SQL queries. Over 10 years various regressions were found and 2.000 SQL queries are guarded by tests. In reference implementation remaining 8.000 never surfaced over this time and it's unclear if they will work.
And, thinking of how many various SQL queries PostgreSQL users around the world are using vs the test cases covered it's obvious that feature space isn't covered in 1% of the success ratio cases.
Now the new, test-based implementation, has to prove it can handle remaining 99%.
That's precisely what a regression test suite is for. There is a bug, you fix the bug, you add a regression test. So if the test suite is well maintained these real world production scars are reflected in the tests.
And also the amount of people running it in thousands of scenarios. Not sure if these areas can be even tested for, but I guess time will tell (can observe Bun if it breaks somewhere as that’s afaik the first big AI rewrite which got into prod for masses).
Software like a Database should have an extensive test bench with concurrency tests, all corner cases etc.
I'm not here running the new version on production to tell the maintainer/devs that my 'production unit tests failed'.
What is this even for logic?
I mean there is balance when i write tests for my production software, but my software is used by me. If i would have a library, i would test everything.
And there was some blog post about another database system were they even virtualized the File access to test cases like when the disk controller stops working.
Regression tests start to play a different role with LLMs.
On one hand, they give an LLM a short feedback loop to correct itself, and iterate fast when writing code. A human also uses it as a feedback loop, but we don't iterate as fast and don't handle big walls of conditions, so its effect is not as big.
On the other hand, LLM's ability to handle a big wall of if-conditions can backfire if it starts taking shortcuts and taking the tests-as-a-spec too literally, overfitting the solution, overly focusing on the given datapoints (conditions checked by tests) and missing the overall behavior shape that the tests intend to pin down. For humans, this is less of a concern because we are bad at big walls of if-conditions, and we'd rather try to see the original shape that the tests are pinning down than monkey-patch the solution to fit the individual points.
It's interesting to see how one balanced these two. In this case particularly. Maybe you could play around with separating the data you give an LLM into "training set" and "validation set", training set can be seen fully, but validation set is hidden and is only queried when the solution is deemed ready. Say, training set = original source code + half of the tests; LLM uses that for quick feedback loop. And validation set = the remaining half of the tests; test code is not shown to the LLM and run only when the LLM says it's done to catch potential overfitting of the resulting solution over training set.
To me, the credibility of a solution like that would depend on what methodology the authors used. If they just let the LLM see all tests, I'd be skeptical (albeit unable to point out specific bugs due to the volume of work and LLM's ability to make bad things look trustworthy). The good thing is, real-life use will add new, unseen before datapoints for testing — so validation set will build up with time. Really curious to see how it will work.
Porting perfectly working C code that has been in production for decades (and has a great track record wrt to security fixes) to rust is more or less an obfuscation challenge.
Rust and its ecosystem needs to become more original. There are so many new problems that needs software solutions. Existing solutions that already work don't have to be rewritten in Rust.
Given that you reuse Postgres' tests and LLM have been clearly trained on Postgres' three decades of contributions, this may constitute a license violation. Unless you include the original license of course. From a human level I also understand how you ended up with a less permissive license.
Why should a developer use this for anything beyond a pet project? Just because it is written in Rust?
All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.
These days there's little chance for a new DB to build its community using network effect. If you want this to catch on, switch to manually grinding community building ASAP gold plating the experience for a specific niche (AI can guide your priorities but will hinder your comms). Otherwise, have fun building!
I'm starting to get a bit of fatigue for these projects that boil down to just "I asked Claude to re-write this code into a new language that's in vogue right now!"
I really don't understand why this is needed outside of an opportunity to show how impressive LLMs can be when working within large codebases, but even then people in the comments are finding bizarre implementation choices that a human developer wouldn't make. I'll stick with Postgres and its - gasp - C implementation for now, thanks.
Congrats for launching this project. I think it's awesome. I did two similar projects, just to learn how to work with LLMs and it felt good--reminded me times debugging code with GDB and going through stuff in semi-manual mode.
I'm building data + AI platform. It got complex, and I'm using AI-assisted coding to move fast. One thing that helps me was property based testing. I have a traffic generator that simulates 10 users working on my platform. I run it 24/7 and if it shows that the software survives the test (I called it "fate" after ffmpeg's CI), it's good enough to roll out. If you wrote something like that, core PostgreSQL folks would like it too, unless there's something equivalent like this. It'd be: create random tables, fill with random data, then issue randomly constructed query.
Quite a lot of projects are trying this "rewrite to a new language using LLM", both internally, or externally (like is here). For me, they confirm some (slightly controversial) takes.
1. human code reviews are dead. We don't yet know what's next. Two reasons they are dead: too much code to review, and code reviewing sucks (who wants to spend their days reviewing code?)
2. Not knowing how to review LLM code is a big barrier to adoption, but bigger regression test suites (testability/evals) is almost certainly the direction.
3. There are a lot of projects that haven't moved to more modern infra because it was too hard. Now it's much easier. Sure stuff will go wrong. Sure it all has to be tested. What's new here?
4. Programming languages for LLMs are coming.
5. Projects that don't allow AI coding will be forced to come around or fade.
Separately, bit off topic:
New projects will often have LLMs built in, so non-determinism will be inherent in the project. No amount of code review will be able to eliminate that.
Might be a very good occasion to actually improve the test suites from our load-bearing software projects. I feel this will be the decade of a cat and mouse game between LLM PRs and finding good (as in convincing whomever is paying you and is waiting for any occasion to fire you for being anti-progress or something).
Hopefully we get: actual formal coding rules, spec rules, design rules, contribution rules, documentation and testing rules. High Integrity development processes impose that you write all this before you start and makes sure you follow your own rules.
So. I guess... welcome everyone to explicit software and systems development processes.
What is the future of this? Code is not the same as a viable open-source project with a community, contributors, advocates, users and funding, even if it's perfect code.
Even though I'm sure it won't be easy to convince the Postgres project to switch to Rust, I do think that trying would be time better spent.
Is the 300x performance boost attributable to the threading model vs process model?
Was the code for the threading model written by hand or was it translated from the WIP threading model the human PG team is busy with as part of the 2028 roadmap?
I think we will actually see some successful projects coming out of this. There are definitely people who want x old project in this new/better programming language and who are willing to put effort into maintaining it not just doing one off port.
It's silly, nearby all Rust projects are rewrites of existing projects and now as you don't need to learn Rust to do rewrites, people just let the AI rewrite projects in Rust.
How is the performance compared to regular PostgreSQL?
I know it says it is not performance optimized yet, but if this succeeds, will it only bring more "memory safety" or is there a serious performance gain as well?
Let's hope it did not special case (overfit) for the specific tests. One of the failiure modes that in my experience takes the most effort to mitigate for.
This the type of AI generated tool I'm interested in seeing more of. I'm a fan of how the quality is verified using clearly defined quantitative measurements for performance and correctness.
I'm still skeptical about LLMs and don't use them, although I can be convinced by more demonstrated examples of success.
I just want to quibble that the 100% Postgres regression tests do not test the threaded aspect of this project, and that is a pretty fundamental architecture change.
Setting aside the “why” that everyone is so focused on, I want to know, how? I can’t get Claude to do anything even a fraction of this complexity. How are people setting up their agents, Claude.md, etc to do such big projects? There are a lot of lessons to be learned.
I think the cool thing about these projects is that even if test parity reaches 100%, some bugs are going to surface on the new project that don't exist on the original project.
This is usually a good example of a test case that the upstream project is not covering and can be contributed back.
Parity should be bidirectional, so definitely it is possible for both parties to benefit from it.
It's so great. One thing that I'd like to be changed in PostgreSQL, which may be done in this rewrite, is resigning from the "one connection = one process" design choice and instead handle the connections using threads/tasks within the main process.
I started my programming career by porting code from one model of TI calculator to another. It was code I could not have written from scratch myself at the time. I learned a lot about the two different versions of TI Basic that the calculators used, but I didn't learn how the program really worked. I can't even remember now if it was Tetris or the tank game that I ported. Maybe it was both? That was a boring English class...
I totally understand why porting code is fun. It's kind of like when I checked out drawing books from the library as a kid and just traced the pictures because my own attempts at drawing were so bad. It gives you a feeling of accomplishment, even though you didn't actually do anything that difficult. And you do learn some things along the way.
Doing the same with an LLM probably gives you that similar feeling of accomplishment, even though you didn't actually do that much (sorry, hate to say it that way). I wonder if you learn even less in the process. Maybe you just learn different things.
Now that I think about it, even writing some code from scratch with an LLM is not much different than doing a porting project. Someone else did the hard work of creating the original programs that the LLM was trained on, and now you (the LLM really) are just porting/restating what someone else did. I hadn't thought of that before
it's a neat experiment, obviously inspired by the bun guy.
but given the author/maintainer is essentially unknown, I highly doubt this will reach it's target audience. But one thing I do agree with is that postgres is long overdue for a re-write into a memory safe language.
also any real swe with more than a few years of experience knows "100% of regression suite passing" doesn't mean anything other than a neat checkmark for C-level executives.
It’s interesting to see how llms have turned the concept of rewrite it in rust, from an impossibility for some projects (code is too large and complicated, it will take too much time) to a real possibility for even large projects.
What would be interesting is if they found a memory unsafe bug. Postgres is a perfect case study of 30 years of C with a bit of CPP; if rewriting in a safer language didn't find anything...
> The goal is to make Postgres easier to change from the inside
uh-huh, sure.
you want to show off "look what the LLM can do / look what I burned a bunch of tokens on"?
you want to brag about how your LLM-generated slop is somehow more maintainable than the original because blah blah blah Rust?
here [0] is the version history of Postgres. pick a version from the past. let's say 14.x because it's the most current that's still under active support.
have your LLM implement version parity with 14.x. show off how it passes all the tests blah blah blah.
then have it upgrade your codebase to parity with 15.x, implementing whatever new features and bugfixes that includes.
and have it generate an automated test that demonstrates upgrading an actual database from LLM-14.x to LLM-15.x and verifying there's no data loss or corruption. maybe even multiple such tests, if you're feeling fancy.
then lather, rinse and repeat with 16, 17, and 18.
and show off the diffs of each version. does the LLM rewrite a huge pile of already-working code in the process of each version upgrade? does it introduce new latent bugs in the process - the kind of things the existing test suite didn't think to explicitly test for?
"I took a static snapshot of code and converted it to another static snapshot of code" is meaningless. all you're doing is bragging about having more money than good sense.
the stability and trustworthiness of software like Postgres does not come from a one-time snapshot showing tests passing. it comes from the engineering process that produces the software and its test suite.
oh, and for shits and giggles, because this same test was so illuminating with the Bun "rewrite" into Rust, here is the file with the most unsafe blocks in the codebase:
Like most modern languages, Rust has its own build system and package manager, Cargo. Everything you're referring to relates to that, and has nothing to do with LLM coding.
Edit: saw the clarification in another comment. But, in that case the essential point seems to be "I'm not familiar with something, therefore it's suspect."
Rust feels like the just right language to produce all those slops of AI and the language itself. Great to promote yourself as a productive engineer, but at the end of the day you’re just reinforcing the statement that AI and the language itself are great, not you.
Rewrites in Rust are kinda impressive. This language with its move semantics and close ownership tracking is very different from every other language. To create a rewrite in it, you have to rearchitect the code. There is not as much freedom there when it comes to where to keep what and where you can pass what as it is in other languages.
> rewrite of already popular technology in a different language
> look at commit history
> "Claude xxx committed yyy ago"
I'm sorry, but what is this need to just vibe code a port of an existing technology to a different language/framework/etc.? If it's just a personal challenge then sure I guess, but this surely can't be used as a real product?
From what I skimmed manually, not that many, but the code itself seems labyrinthical. Like, why have both Rust Try-supporting Error-like tagged union, but also booleans, for error handling, in the same function?
Woah! AGPL? That's interesting. I think Postgres has shown an open source SQL server didn't need a copy-left license to develop sustainably, so I'm not entirely aure about that, but I do like the license in general.
When the software consists entirely of ~$1000 worth of Claude credits and ~40 hours of developer time prompting and curating it, literally what does it matter what license the resulting 100k LoC artifact is provided under?
Copyleft and the whole software licensing ecosystem only matter when producing that software actually requires serious human effort and dedication.
Did similar with S3 and that too (eventually) did well against tests (the ceph s3 ones).
...but haven't dared use it for anything meaningful yet. Still feels like there is a real world gap in confidence when it comes to vibecoded rewrites.
Been wondering whether the answer is to insert a proxy...something that effectively splits traffic to a known S3 and the rewrite and compares outcomes over time. Do that for a couple different workloads for a month or so and if it's all identical then it's probably fine...
It is theoretically possible to have a Rust port of Postgres support extensions. If you make all the relevant functions and structures ABI compatible with Postgres, extensions should work. The issue is the moment you're dealing with C pointers and C strings, pretty much all the code you have to write is unsafe.
The return type in the rewrite is both some sort of Error tagged union that supports the Try machinery in Rust; but, it also contains a boolean that apparently must be checked; or something. It seems labyrinthical and possibly broken and terrible.
I make no claim as to whether the change makes sense given that I didn't look at the callers of this function, but Result<bool> is an entirely reasonable pattern in Rust. If you want the callers to be able to distinguish between "has the subclass", "doesn't have the subclass", and "something went wrong" this is idiomatic Rust.
It is a feature in Rust, not a bug :-) (I know you didn't say it is a bug.)
The error-tagged union is PgResult<bool> - which means it contains bool as the result if things go well. (The other part in the union is of course the error.)
In the original function also, it is returning a boolean: "bool has_subclass".
So anyway you have to check for the boolean as part of the logic. That is what it is doing.
I have privately wondered for years, pre-AI, why Apple hadn’t paid some engineers to go off and write some comprehensive test suites and then port these to Swift. It would shut down entire swaths of memory safety bugs they have been coping with for literally decades. SO MANY of the zeroclick iOS exploits can be traced to a few fragile and vulnerable foss libraries, xkcd 2347 style.
And they'll reproduce virtually the same problems, because code was never a primary issue for output. Products reflect organizational problems, which AI can't solve.
We had one for SQLite (which is SQL-ite btw, not SQ-Lite which doesn't make any sense) via Turso, no wonder we see the same for Postgres. Personally I do want to see libraries be in as much memory safe languages as possible.
How do you know it's not SQL-lite with the single L serving a double role?
Common pronunciations allow you to stay perfectly ambiguous about where the L goes, which aligns quite well with the name as spelled. If you do it right, nobody can tell if you're saying sequel-ite or sequel-lite or seque-lite on the one hand, or S-Q-L-ite or S-Q-L-lite or S-Q-lite on the other.
AFAIK there is no official word on how the name is intended to be read or said.
Hey author here. Wasn't expecting to see this up.
To concisely give an overview of the project, I've been experimenting with using LLMs to build a better version of Postgres. Postgres is 30 years old and we've learned a lot about databases since hten. A lot of the techniques that work for doing a rewrite are also useful for doing a rearchitecture.
I'm now working on a new, not yet published version of pgrust that incorporates a lot of techniques. Currently the new version:
If you have any questions, I'm happy to answer them.A thread per connection is a almost always the correct decision for performance, but by choosing a process per connection, postgres is able to let you load whatever sketchy extensions you want. Worst case you crash the process, not the database. It would be nice if you could strike a balance so a segfaul in the extension only crashes a small percentage of connections, not the whole thing.
That’s not true for Postgres however: due to its usage of a shared memory pool, whenever a subprocess is terminated unexpectedly, Postgres will kill all other processes and enter recovery mode, replaying the WAL, during which time it will not accept connection requests.
It does this because it can’t possibly know whether the dying process did bad things to the shared memory pool.
If it's a choice between performance and being able to "safely" run sketchy extensions, I'd rather have performance.
Some see a 30 year old system and think "outdated", I see a 30 year old system and think "time tested."
Clearly a process per connection is more stable and that's what I'm using.
It's unclear what problem such optimizations are solving anyway, with the old way you could only support a million concurrent users with a single server? Are we missing out on supporting ten million concurrent users with 2 servers instead of 10? Ostensibly reducing the minimum db hardware opex for a 10B$ company from 10k$/month to 2k$/month?
Doesn't postgres (rightly) have a cow if a process has a disorderly shutdown (at least while in a write transaction) because there's shared memory between the processes?
I'm not informed of the Postgres's internals, but, maybe, that can be solved by grouping threads into different processes depending on which set of extensions they request.
An OS thread per connection can be fine for performance if you don't have to scale your connections, but if you don't need to scale connections why have connections at all? Databases are even more performant when you eliminate connection overhead entirely.
Is that a serious issue? Wouldn't it just restart the split second later? Or does it take a long time to start?
(Or I guess it would get stuck in a doom loop or something?)
A more modern way to do this might be to support WebAssembly plugins.
The extensions might need to be rewritten, but hey, we have AI for that now, so why not :-)
I don't want to knock you down as most have already did. In-fact it's a useful exercise going forward in exploring how to work with AI. It's here, we're all going to use it one way or the other. Zero issues with that, in-fact kudos to going through the pain of it all.
Now, having gone through several such endeavors originally myself, albeit with internal tools and systems (as an exercise), I've noticed that while all my tests passed with flying colors the rewrite itself was broken even on basic functionality or missed a ton of details. It was in effect useless when I dived into it. Initial tests also showed massive gain in performance, and I know people who were involved aren't really dumb so something smelled funny. Turns out all those things left out and honestly... moments were the key ingredients.
What I did learn from those beginning explorations though was that one-shotting, grand architecture or source up-front, master plans up-front.. all these do not yield good results - YET. Who know what we'll see in few years though. What I did found that works (FOR ME, nota bene) is to keep the design and checklists for myself, written by myself and then do a small piece by piece.. as if you would if you were coding alone or if you would waterfalling a small team of talented juniors. Then, suddenly super happy results come out, but then it's mostly you driving all the way where llm writes code and offers advice (which for the most part you ignore). It's a happy place for myself at least. It's then truly unlocking yourself to the mythical 10x.
Rewriting a large proven system with decades of ultra expertise behind it, which I don't have, is guaranteed not to end up the same 1:1 replacement. If you found a recipe for that - please do share.
I'm curious. Do you attribute this to weak and/or incomplete tests? How granular should tests be to have complete coverage so that an AI won't create a converted codebase that "passes tests" but is still functionally inaccurate?
I don't code with LLMs, and think you might be right.
However, Postgres is a tool with clearly defined functionality and doesn't have ambiguous requirements that are seen in user facing software. As a result, it is entirely plausible that the author created a working Postgres replacement for certain use cases.
I personally want to see more evidence about the quality of the tool after it is in a finished state.
what model did you try it with? I agree and also push back a bit: How will we know when the LLMs reach the point of handling it if no one takes the leap? I applaud more people sludging through the slop and hauling their slop buckets around.
An example is Fable being released. I felt like the most complex thing I was willing to sludge through was having it clone llama-server's web UI with my own opinions (I really like the original, kudos to them). And the initial skeleton was working so well I felt like I had sunk the tokens and committed to getting it the rest of the way: https://inkcap.click
Ouf. I don't know. I don't want to call you out without evidence -- I myself make benchmark claims all the time -- but 50% improvement in OLTP seems suspicious. I get that you used a standard benchmark, and I don't even know what it entails, but my spidey sense is going off. Perhaps, some trade off somewhere that won't make it to prod because it breaks MVCC -- and yes, I saw that it passes regression tests.
Just checking, is fsync on? :) Regression tests don't catch bad IO patterns afaik.
Anyway... sounds like a fun project to work on!
Remember when databases were faster to run in virtualbox rather than bare metal? (because virtual box was completely ignoring all the instructions to flush the data on the disks)
Yeah, claims that it can be faster than CH are very suspicious. CH guys are very good at their craft, they spend hundreds of hours optimizing one single small detail.
It'd be very unfortunate if Postgres didn't have regression tests for data loss due to bad io patterns. Should be possible to do some checks against those in an appropriate test harness. Which might mean "have qemu run something we can kill off and examine the results".
If those don't exist, I hope folks recognize how useful they are and add them.
What's your actual background and expertise with Postgres and databases more broadly? Basically, do you actually know what you're doing, or is there likely a massive footgun you don't know or haven't shared with us?
I spent a couple years managing a Postgres cluster with a petabyte of data. I wrote a couple blog posts from my work then[0][1]. I also wrote dozens of posts on the Postgres internals[2]. I've also given talks on how to generate fractals with SQL[3] and how to write a lisp interpreter in SQL[4].
[0] https://www.heap.io/blog/testing-database-changes-right-way
[1] https://www.heap.io/blog/analyzing-performance-millions-sql-...
[2] https://malisper.me/table-of-contents/
[3] https://www.youtube.com/watch?v=xKoYIvMFnoQ
[4] https://www.youtube.com/watch?v=MPSMH8w7nfw
> - Is ~300x faster than Postgres on analytical workloads. Right now it's 2x slower than Clickhouse on clickbench and I think it's possible to get faster than Clickhouse
That sounds like you are storing the data in a columnar format? Or do you do both row and columnar?
In a somewhat similar (yet also quite different) effort, I've been working on δx, a Postgres extension that compresses the data in a columnar format stored in normal Postgres tables (so replication, crash recovery, pg_dump, etc. still work normally). https://github.com/xataio/deltax
It is currently about 30-40% slower than ClickHouse (single node, ofc). The PR to add it to clickbench was just accepted, so you can see the comparison here: https://benchmark.clickhouse.com/#system=+liH|_etx|gQ|saB&ty...
Yep! The new version of pgrust supports batch based execution and a columnar format. I'm curious how you got δx to perform that well? From what I've seen a columnar layout only gets you part of the way and really good parallelism and really fast hash tables seem to make up a significant portion of why Clickhouse is faster.
It doesn't sound like you were trying to launch a product, but doing an experiment and someone threw you under the HN-spotlight-bus :) Is this a "see what I can achieve with LLM coding" or is this "build this and see how much of the coding can be accepted from LLMs"?
What was your methodology and structure in making the prompts for the rewrite? Did you let the LLM roam in all of the codebase and tests from the beginning, or revealed things to it gradually in some way?
Are you fixing the heap and table management ?. Postgres does not use an undo log and manages all table updates directly in table storage which slows MVCC.
also have you told Ben Dicken ? https://x.com/BenjDicken/status/2074326407795417435
That's something I eventually want to fix. The challenge is the storage format is so integral to Postgres that it's going to be a huge PITA to come up with a novel design.
Right now OrioleDB is in beta. Once that becomes production ready, I'll evaluate incorporating it into pgrust.
For Ben Dicken, he has seen the project: https://x.com/BenjDicken/status/2074512043462603236. We're still working on all the novel features so I don't think it meets his bar quite yet.
"Is 50% faster than Postgres on transaction workloads" - That is a very big claim! 50% faster on everything? Is it a strict improvement across the board or are there tradeoffs that make some workloads slower?
The 50% is specifically on percona-tpcc[0]. I got there through a mix of batching (postgres processes a row at a time), prefetching, and several handful of other optimizations.
While a thread-per-connection seems like an improvement, do you have any plans to allow query multiplexing over a single connection? That would be a huge improvement IMO.
Can you elaborate on the use case for query multiplexing? Is it so your client would only need to establish one connection with Postgres and then could run as many queries as it wanted?
> build a better version of Postgres
Faster is quantifiable. How do you measure better?
[flagged]
This is great! Those analytical workloads numbers are mad - I'd love to see the benches, and I'm happy to contribute to some of the profiling.
How does your thread-per-connection model compare to Heikki's proposal[0][1] from back in 2023?
[0]: https://www.postgresql.org/message-id/31cc6df9-53fe-3cd9-af5... [1]: https://www.youtube.com/watch?v=xLLakMmVtbY
Rust actually made the change pretty simple. The main changes are:
I've started to see meaningful benefits by changing the parallel algorithms to use a shared memory space. For example parallel hash joins have to copy tuples through shared memory to pass them between workers. That's just not something I have to do.Is it being used in production anywhere, even if only a toy app?
I know you say it's not production ready and not optimized yet, but in the same breath - in your comment here - you say it's already faster.
It's not used in production. I've been using different benchmarks to compare the performance vs other systems. Namely sysbench-tpcc[0] and clickbench[1]
[0] https://github.com/Percona-Lab/sysbench-tpcc
[1] https://github.com/ClickHouse/ClickBench
Super impressive! Is it possible for you to share your methodology of using LLMs?
My approach has changed throughout the course of this project. Throughout most of the project, we were working off of a c2rust translation of Postgres to Rust. That gave us a bunch of Rust code that was unsafe but did pass the Postgres test suite and was fast. c2rust had split Postgres into 1000 different crates. We then went through 1 by 1 and rewrote each crate into idiomatic rust.
This naturally lended itself to a suite of skills to describe how to rewrite a crate from unsafe rust to idiomatic rust. The main three skills I had were 1) a skill for identifying the next crates to port 2) a skill for rewriting a crate and 3) a skill for auditing a crate and making sure there weren't any outstanding issues.
My exact approach for managing subagents changed throughout the project. Initially I was doing parallel coding sessions with Conductor. After dynamic workflows came out, I used that as it was really easy to spin up dozens of parallel subagents and manage it from a single orchestrator. Over time I switched from using dynamic workflows to manually spinning up subagents from a central agent. The issue with dynamic workflows is they waterfall. Each step needs to finish before the next one starts. By manually spinning up subagents, I could have claude start porting a new crate as soon as a prior subagent finished.
PG Wire proto 3 is my largest source of frustrations.
I'm playing with a POC for a better wire protocol here: https://github.com/solidcoredata/pgwire4
404 Not Found
I am super curious how you went about the port using LLMs. At $WORK we are looking to port code, preferably with LLMs, and it seems daunting, even with a test suite. Do you have an approach that works well for you?
Bun has an interesting blog post about how it was ported. It did cost a lot, much more than hiring people would cost outside USA.
Just a couple of ideas if you run out of backlog:) - proper versionnumber (64bit) - native json streaming. It would be awesome to get to the point where i could somehow redirect the sql output to the browser directly, but piping will do for now. The idea is to be able to stream rows to client without caching and building json along the way.
It's a completely new era of software production (I will no longer call it development) LLMs give us unlimited manpower, and the language give us constraints to make more modern and safer softwares. Love to see this rewrite in Rust, and expecting much much more in next few month.
How much of the performance gain is from using Rust, compared to using optimizations that are not done in the original PostgreSQL code (like using threads instead of processes, etc.)?
I am simply curious what the benefits of using Rust are in this instance.
Do you have anything in the regression test suite like jepsen etc?
Nothing major yet. Once I wrap up the performance work I'm doing I'll start looking at the best way to go about testing. I suspect there's a lot of novel things you can do with agents.
when doing rewrites like these, why isn't the first step to instrument the original code so that you would get very good automated test suites to point the LLM toward?
use both synthetic and real data to sample the internals of the original software to duplicate.
locate all the data transformation junctures, sample and then replicate the tranforms 1:1 in the rewrite.
That forces you not only in not the intentional good decisions of the past but also copies too many bad ones.
Would you like to submit to ClickBench?
I can also do it if you would prefer...
That…is really impressive. Well done!
300x is mostly a marketing term, especially without the test description.
BTW, showing no respect to what it is trying to copy looks uncomfortable.
i highly doubt you can make it faster than clickhouse, but happy to see it.
Is it your first rewrite/migration (with or without llm) ?
good luck nonetheless
Awesome work. I'd love to see you add something like kusto query language or pql. The autocomple on kusto, (which can be embedded into web apps) is really amazing.
This is how to LLM. Big ups, I wish the whole front page was stuff like this (and I think it'll happen).
Everyone is so worried about the value of commodity software going to zero. It's like, yeah, going into CS for the money always looked dumb to me, it's just not a good career path for that, you have to love it.
I am way more excited about a whole new class of stuff that obliterates the state of the art at every frontier.
Keep doing it legend.
Don’t understand these rewrites.
- typically they are behind a single person. That’s usually bad because of spf
- typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term
- anyone that wants to contribute to the project needs to pay. Needs to pay tokens because it’s increasingly difficult to maintain these projects without AI
So, who wants to put something like this in production? Doesn’t make much sense
I'd be interested to hear the author's answer to your question, but I see it as an interesting proof-of-concept. It's testing the viability of not only rewriting PostgreSQL in Rust (and their choice of deps) but also in switching the threading model and other architectural changes. LLMs shine at pumping out prototypes insanely fast, and a working prototype can put an end to a lot of speculation.
I likely wouldn't use a rewrite of such a huge project if it doesn't have the backing of the original team (or a significant fraction thereof) and a believable story for having matched/exceeded the original code quality and maintenance. I also think in general using an LLM for license-laundering is legally and morally hard to defend, although this case is different in that they chose a more restrictive license. Not a lawyer, but my understanding is that you can just download PostgreSQL, do s/MIT/AGPL/ and release it, legally. (The original MIT-licensed version still exists, so no reason anyone would prefer yours until you make another release with some compelling new feature.)
It's a very interesting phenomenon in recent years and most of the discussions about "why" are immediately blocked by the "memory safety" argument, as if it's a silver bullet for all things that are considered "bad" in software implementations. No matter how good the language is, and I consider Rust a very good language, it's practically impossible to replace years of experience and tested code, most of it contributed by a ton of brilliant programmers, no matter how you look at it. And if we take this as the truth, then logically there's still an unanswered question - why? Lets take an undeniable fact - rewriting an existing project give you full control over the new implementation. You can do whatever you want with it, you can't be sued. The only thing now you have to hope for is for your implementation to gather a good enough user base and from then on you can practically hijack the original project. And this is me speculating - rewriting stuff in Rust isn't about the greater good for that magical "memory safety" argument, but at the end it's an attempt to hijack popular software projects.
> it's an attempt to hijack popular software projects
Exactly. And this is why those projects are typically called "XXX in Rust" or "XXX-rs". Because the creators get to do their favorite thing - coding in their loved language - while skipping all the hardships of designing, accepting real feedback, involving users and getting traction - all while simultaneously hijacking the existing brand.
As of now I know of a single project that changed their name after being called and the project surprisingly got some traction.
I think we're a year or two away from the same argument being used by languages which enforce proof of correctness. The economics seem to be shifting in that direction.
Finding exploits is getting exponentially cheaper, and the cost of producing proofs is rapidly going down. For a lot of software correctness is rapidly becoming non-optional.
> impossible to replace years of experience
People with no experience don't have enough experience to realize what exactly they are missing here.
This is very critical of an open source project that the maintainer didn't even post here?
"Status:
pgrust is not production-ready yet. It is not performance optimized yet."
The maintainer is not suggesting you use this for anything yourself. So why do you care about spf or (lol) his "discipline in creating the project"?
The gp feels like less of a targeted individual criticism & more of a general musing about a trend.
Nobody is saying this author hasn't demonstrated discipline & won't maintain this project, but the statistical averages across most projects fitting this trend make it likely enough to question their worth in aggregate.
Because chances are that it's never going to be production ready, never trustworthy enough to be used by anything serious, and the project will eventually be archived and become at best a (extremely inefficient) learning experience for the author, or at worst a total waste of tokens. I could be completely wrong about this specific project, but most of these projects are like that, and statistics don't lie.
The author is free to ignore any and all complaints they consider unfounded. It’s not even like the author is recieving any complaints personally; they have to come here to see any. And if they come here, they will get to read the viewpoint visible from here.
(Repost of <https://news.ycombinator.com/item?id=45253509>)
But this is also just silly. The maintainer here did nothing (just a little chat here and there), but is sure that only optimizations are missing for this to be production-ready. How so, based on what exactly?
100%
People on HN love complaining at any given moment.
I’d wager most of these people don’t really produce much and are constantly bikeshedding.
It's not just a rewrite ; it has improvements. I did the same thing for fun for the same reason; I wanted to see if I can improvements on some of the legacy design stuff and, especially, the stuff PG people have told us that it cannot be done differently. It can. I would not put it in production, but it thought me a lot about the internals of databases. To keep my brain happy in the age of LLMs, I implement database things on our (also old but many times refactored/rewritten) production db without an LLM. I'm sweating through Flexible Paxos now; probably we will just keep using raft as it's old and stable and simple but it's interesting anyway.
> I would not put it in production
Nobody else would either.
If this is meant for personal learning, do it that way and make it clear that others should not even consider using this project.
In fact, even for personal learning it's wasteful. You can learn so much about database without a rewrite like this.
but can you? You cannot do a Postgres clone yourself and the Postgres arch makes it rather impossible to just rip out stuff and replace it. So how would I do this. Even with SQLite and me being a very experienced (40 years) c embedded coder; having AI rewrite it to the architecture I would want it to be and then changing things for fun is a better tutor than whatever I could have done with the original in a shorter time.
It’s open source. You take ownership of it for your deployment and stop relying on continued free work.
You can use llm to pull in updates as they are released. It’s not gpl, so you don’t need to publish your port
1. Nobody, not even the Googles or NSAs of the world do that. No single entity has the expertise nor resources necessary to maintain a fork for every open source project they use -- forking and maintaining Linux alone takes teams of people. And no, going full psychosis mode with LLMs is not going to save you.
2. This project is AGPLv3.
Forking and maintaining Linux does not take teams of people. I've been at it for > 10 years and maintained support for my 15 or so various SBCs/mobile devices, writing drivers, debugging, cleaning things up. It's a weekend project every 3 months + whatever you want to put in for development.
And there are others doing it for projects like Armbian, openwrt, etc.
> going full psychosis mode with LLMs is not going to save you
People in AI psychosis don't know that.
Except Amazon (Linux, Elastic, RDS..)
It's useful to show the actual team that it's possible. From there, they can make the decision of whether to go the bun route with more information.
I'm not sure I see the value in "showing the team it's possible". I would presume the team are intelligent enough to be well aware that it's possible, given tradeoffs. And as it's clear those tradeoffs have been "traded" in this case, it doesn't seem like having the knowledge confirmed is particularly surprising.
This is true but these projects are rarely presented or interpreted as a proof of concept.
> it’s increasingly difficult to maintain these projects without AI
It's pretty much impossible in a project of this size. IIRC Postgres has over 1M loc.
I'm really happy to see these rewrites. It shows what's possible, especially as the projects get more and more complex.
And maybe Remacs will get reactivated https://github.com/remacs/remacs/wiki/Progress
Maybe even I'll be able to do it when I wait for the bus!
Not the same but it is much faster and easier to re-create a 3d model of an existing set of drawings than from scratch. This is because a lot of the decisions have already been made.
>typically they are achieved in a very short amount of time, so the author hasn’t acquired any discipline in creating the project. That means it’s unlikely the author is going to stick to the project in the mid and long term
Lindy effect! The longer something has been around, the longer it probably will be.
https://en.wikipedia.org/wiki/Lindy_effect
I guess it is cool to have it around but it comes off as a popular useful case for AI which it really isn't, unless you have a very good test suite to throw the agent against, it is practically useless for rewrites of decades old badly documented code riddled with technical debt. That is where the real value and challenge would be. I see it as marketing and social clout stunt
Everything has to start somewhere
> So, who wants to put something like this in production?
I don't think anyone suggested deploying this to production - the author is quite explicit that it's an experiment.
spf meaning...
This year, not me.
In five? Everybody but me.
If you were a token burning company a project like this seems like a solution to this: https://xkcd.com/2347/
that hits your metrics without the problem that your contributions are not welcome.
How would one go about reviewing a piece of code like this?
One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project. But with LLMs generating 7101 commits in less than a month that isn't feasible. Even looking at a single day is way too much [1]. It probably also doesn't make sense since the commits content won't tell you much anyway.
ps. How do you easily get to the first commit in a repo on GitHub? Browsing commit history feels rather tedious
[1] - https://github.com/malisper/pgrust/commits/main/?since=2026-...
Vibe code was never meant to be reviewed.
These rewrites are just test-driven development taken to the absolute extreme. Created under the hope that the existing tests are exhaustive and cover every relevant use case, such that if they all pass, the rewrite must be at least as good as the original. So just go with the vibes and burn tokens until they pass, and your job is done.
In practice, this is never true for any codebase above a certain level of complexity, especially not one as mature and widely used as Postgres. But reality doesn't seem to be an obstacle for vibe coders.
The challenge is that more and more people are producing project like this - 1,000s of commits and > 200k lines of code - and saying it was carefully created using agent based workflows and not vibe coded.
> reality doesn't seem to be an obstacle for vibe
Went straight into my vault of brilliant quotes!
One of the projects Im working on and off is a tamper-proof audit log, based on some PoC code I created almost 10 years go; unit and integration testing are good at preventing defects and regressions, but they will not guarantee your software will work. However, with the power of LLMs, one can easily use model checking (in my case with Quint) and/or other formal proof approaches to ensure the software conforms as specified. The result (in my opinion) is an implementation guided by a single human that is actually more trustworthy than manual human-made software using the traditional approach.
> Vibe code was never meant to be reviewed.
It was also never meant to hit production.
And run them in test setups to try to find bugs.
If you find some, fix them.
(I'm working with malisper on pgrust),
I think the focus for projects like this is going to shift to reviewing the testing/fuzzing process instead of reviewing each commit (going much further than what the postgres regression/isolation/crash tests do).
related post from danluu: https://danluu.com/ai-coding/
Some of this post reminds me of a story I heard long ago from someone who had worked at a HW/SW company. They’d transferred an engineer from the ASIC design team to the OS kernel team, though he’d never been on a software team before. After a while the manager called him in for the following conversation:
Manager: You’re doing amazing work — zero bugs in production! I’d like you to mentor the other SWEs on how to get their bug count down too.
Engineer: We’re allowed to have bugs?
And what happens when your "tests" are also vibecoded. Right now all of these houses of cards reset on human-written tests. What happens without them?
For large projects like this I think a hierarchical division of labor also helps.
If you first carefully define the overall architecture and thus individual high level components of the system, then you know which of those components are mission critical and which are commodity. Mission critical would be anything ensuring ACID, etc. That way, no matter what you farm out to LLMs, you can keep the majority of limited human focus on the far fewer mission critical components. If tests end up not being robust enough to catch all issues, at least they'll be isolated to commodity code where damage is limited to things like DoS, etc, and not code that could cause data loss.
I also think it's important to first define the _contracts_ on and between each of these components, and derive tests from those contracts. Partly because contracts more succinct and easier to reason about. And partly because Rust provides many tools to enforce contracts at compile time, reducing the need for tests (which themselves could end up subtly flawed). Contracts can be enforced through typing, private vs public APIs, etc. Newtypes are _incredibly_ powerful for both enforcing contracts and making footguns much less likely.
Oh, I just posted a similar comment elsewhere in the thread.
https://news.ycombinator.com/item?id=48856535
Though beyond testing, I think there will be increasing focus on proofs of correctness. (Testing can only show the presence of bugs, not the absence. —Dijkstra)
At any rate, it's never been this cheap to produce the proof of correctness of a program, or on the other hand, to produce an exploit for an incorrect program.
> reviewing the testing/fuzzing process
I've got insanely good at designing testing oracles over the last year for exactly this reason.
I've ported some extremely finicky software between languages that it would have been borderline abusive to have a human do.
Codex 5.3 and later for those interested.
The github cli has a command to query commits with a sorting asc/desc flag
https://cli.github.com/manual/gh_search_commits
here's the docs with more syntax using the "before x date"
https://docs.github.com/en/search-github/searching-on-github...
there's also an advanced search page, but it does not support commits when filtering with dates
https://github.com/search/advanced
or you can bisect the date in the search widget, this is the first day with a commit
https://github.com/malisper/pgrust/commits/main/?since=2026-...
first commit:
https://github.com/malisper/pgrust/commit/22113dc36b02973060...
Thanks for all the info you've provided!
Maybe I'm just being a little grumpy. If I really need to look into a repository, I clone it and use vanilla git command line tools to have a look.
It's just annoying that the modern web UI from GitHub takes >1s second to load a page with 34 commits
> ps. How do you easily get to the first commit in a repo on GitHub? Browsing commit history feels rather tedious
I usually check the history of a file not easily changed like .gitignore.
The first commit seems to be this one
https://github.com/malisper/pgrust/commit/22113dc36b02973060...
Very smart. I like it.
> How do you easily get to the first commit in a repo on GitHub?
You can use the syntax github.com/user/repo/commits/?after=last_commit_hash+number_of_commits-2 (-1 for the latest and -1 for the last)
ex : https://github.com/malisper/pgrust/commits/?after=3646a73515...
I started by looking at the dependencies.
Then I lost count, so I ran wc -l Cargo.lock
Easily over a thousand dependencies. And "rewritten in Rust" is supposed to be a good thing? I bet this doesn't even compile faster than the original.Use cargo tree to understand Rust/Cargo deps.
The lock format is a multi-line TOML, with a varying number of lines per dep due to redundantly listing deps-of-deps, so a naive line count massively overstates the number.
Cargo.lock contains many unused dependencies, because it's a superset of all combinations of all optional/disabled features of all transitive deps across all possible platforms (so that the deps don't reshuffle even if you enable/disable feature flags or compile on another platform). But that means Cargo.lock is going to have 3 async runtimes even if you use one. It's going to have syscall definitions for RedoxOS and wrappers for WASM, because some dep of dep is compatible with those platforms. But these deps won't even be downloaded if you don't build for these platforms.
The number you got presented is not representing the unit you're insinuating. A crate in Rust is a compilation unit. It's common for projects to ship as a collection of many crates. It's a smaller unit than what C counts as one dependency, and slightly coarser than an .o file. I don't see people freaking out by how many .o files their projects have, including all transitive ones from deps like openssl or curl.
Do you pick your databases based on how quickly they compile and how many dependencies they have? I normally chose based on factors like performance and reputation for reliability
Disaster. Well, I read stories about Rust and how there isn't much in the stdlib, but this is just too much. How many dependencies are there, on average, in other projects? I guess I am spoiled with Go.
In a typical Rust project you organize your project into many crates. I haven't checked, but I'd guess that the vast majority of those dependencies are internal dependencies. After all, this project started by running an automated C to Rust converter, which the authors claimed produced over a thousand crates.
The naysaying here is insane. Should a project like this not exist?
In general (I’m not saying this is the case with this project) if you don’t have their prompt history and you can’t re-run the LLM “compilation” yourself, is it open source? It feels a bit more like those “source available” projects where you can read the code but don’t have access to the build system.
On the other hand, aside from the commit messages, one didn’t ever have access to the underlying thought process of human developers either, so maybe it’s not equivalent to say that secret prompts mean closed-source.
What an incredibly bad take. "It's not open source because we have the source but not the thought process of the developer" - well then no project on this earth is truly open source by your definition.
You don’t. You trust that passing the regression tests means you are totally compatible with the original version.
> One of the things I'd typically do is peek at the commit history. Seeing what people worked on and how they did it tends to say a lot about a project
I could not care less about any of this. Truth is code, as it is now. I don't care when (and certainly not by who) a bug got introduced, it's here, shut up and fix it.
So your solution is to just read through all 1M+ lines of code?
[dead]
> How would one go about reviewing a piece of code like this?
That's a wrong question. The right question is "why would one go about rewriting a piece of code in X". Once and if you find a good answer to that question, you will see the answer to your's.
I think the best way to test this would be to put PgBouncer or a similar proxy in front of a busy production database, and mirror queries to both traditional Postgres and the Rust one at the same time. Then you can compare output and performance under real load. After running it for a while, you could diff the tables one to one against the normal Postgres instance.
Can you control the timing of queries across two db instances well enough to expect the tables to be identical?
Good question, I guess you can run two identical Postgress instances and check if they have a diff
2664 "unsafe {", 1835 "unsafe fn". This is completely unsafe. It doesn't look like a rewrite that understands what's actually going on or how the architecture should be redesigned to take advantage of Rust strengths. Instead, it looks like an AI generated transpilation with extensive use of raw pointers.
Note that most of the unsafes are confined to the parser which was generated by running c2rust over the Postgres parser. The Postgres parser is itself generated from yacc/bison, so I decided to port it over mechanically rather than idiomatically.
If there's particular unsafes that you think are egregious, let me know.
Just wanted to say: I'm thoroughly impressed with how far in the weeds you're replying in this comment section. I'm learning a lot from the threads.
I don't know if converting the code could be an issue with copyright, but might be contrived as plagiarism
Counterpoint: All of the current Postgres codebase is already wrapped in an invisible unsafe{}.
The difference with a Rust codebase like this is that all of the unsafe code has been neatly isolated and clearly marked. The outside code is safe — at least according to the definition of what Rust considers safe, which is a high bar indeed and objectively superior to the unsafe mess that is C — and the unsafe code is naturally fenced in, which means that it can be seen by developers and tackled by incrementally.
In some cases unsafe is unavoidable, but it is possible for a human to verify that it is, in fact, acceptably safe even if inside an unsafe block.
Valid point, but unsafe in Rust is more dangerous than unsafe in C, bcs of aliasing. For example PG is compiled with -fno-strict-aliasing.
I set all my Rust LLM written projects to 'unsafe=deny'. Not sure why not everyone is anticipating your comment.
Let me just copy this review comment into my prompt.
a few hours later
Fixed!
Why even use rust...
This is impressive - but is a license change, from the PostgresQL license [0] to AGPL [1].
I like the AGPL and think it's the best truly free open source license, but I worry if this is compatible. Ie, if this is rewritten from the original source, should the original apply? (Yes.) There has been a trend to rewrite open source software with a more restrictive license (like coretools in Rust). This looks considerably more ethical by choosing the AGPL - I just wonder, safer with no change at all?
[0] https://www.postgresql.org/about/licence/
[1] https://github.com/malisper/pgrust?tab=AGPL-3.0-1-ov-file
You seem to have the restrictiveness backwards? The MIT license (uutils coreutils) is less restrictive than the GPL (GNU coreutils), and the AGPL is more restrictive than the PostgreSQL license.
And it doesn't violate the PostgreSQL license to license the rewrite more restrictively. That's part of what makes MIT-style licenses less restrictive than the GPL or AGPL: they allow for more-restrictive relicensing.
If you don’t like the license just let an LLM spend a few days “porting” it and give that port any license you like because that is apparently what we do now.
The PostgreSQL License is a variant of the BSD license and is therefore compatible with the (A)GPL.
Comprehend it this way: You create a blank (A)GPL project and incorporate the upstream BSD codebase into it. While those original upstream files remain under their original permissive license, the project as a whole is governed by the (A)GPL (plus the attribution requirements of the upstream license, which the GPL permits). From there, you can add your own code under the AGPL and distribute the combined work under the AGPL.
If someone takes your code and uses only your portion, they can use it under the AGPL alone. However, if they also include the upstream source code, then the attribution requirements of the upstream license must still be met.
AGPL and copyleft licenses are not restrictive, they provide forward (open source) guarantees. The only thing they "restrict" is the ability to remove freedoms. Therefore is not a restriction, is a guarantee. A guarantee that the project will endure as open source.
I wouldn't go as far as calling permissive licensing "restrictive" because they actually allow for reducing freedoms; but copyleft is definitely not restrictive at all, it's the opposite.
Having said that, I like and appreciate both kind of licenses and I have and will continue creating open source software using ones or the others.
The Postgres license is already fully compatible with the AGPL. BSD/MIT is more permissive.
Wow, yeah, really hate this.
If this software was written by a mechanical process, the license is a nullity. It’s public domain.
I feel like we need to heavily differentiate between a rewrite and an AI rewrite.
For instance, the TypeScript rewrite in Go was done mostly by humans and took a year before it was released. That is how you rewrite software that people can trust.
Not sure it’s so simple. I think close to 100% of new ambitious projects are going to leverage AI at least to some degree. I know a couple that have strict no-AI policies (e.g. Zig), but it’s a tiny minority i think.
So how much AI usage does it make it an “AI rewrite”?
A human rewrite without maintenance is just a hobby project. An AI rewrite is just wasting tokens for god knows what?
rewrites feel like an area where LLMs are better suited than humans imo
It’s mostly grunt work and LLMs are well suited for translation tasks (iirc transformers arch was originally invented for translation)
It’s just a build step now.
nope. any rewrite will be an AI rewrite soon.
It's not that... It's a rewrite by project maintainers vs a fork.
We already have a well established term for AI rewrites.
I agree but I think from Bun we learned that a project with really good tests and enough tokens can be converted from one language to another quite good!
It is more and more the future. No human would want to rewrite one technology to another because it is too marginal a gain. AI on the other hand does not give a shit.
I'd %100 prefer an opus 4.8 rewrite over %99 of the time. Unless Fabrice Bellard is rewriting the stuff I need, I'd prefer AI over a human coder.
Is there any measurable difference in quality between the two, or are you just going on "vibes"? Is there a correlation between the quality of the manually written code and AI generated code driven by the same dev?
Such crude takes only cause unnecessary friction. If you have a black box that spits out code, and you are unable to distinguish the quality between a top tier dev and an AI inside the black box, then the distinction is unnecessary. Most of the code on the internet is already a black box to you. What percentage of code running on your machines have you vetted by who wrote it and code quality?
AI coding isn't going anywhere and will likely end up generating most code going forward so instead of rejecting it outright or arbitrarily categorizing it we need to focus on solid quantitative and qualitative measures of code and functionality regardless of who wrote it.
If you can do a Rust rewrite with AI, I can create one as well. What makes yours better than mine? Your decade long expertise in database or Rust language? Your reputation? Your proven track record to manage large, complex projects? Your time committed to the project? I don't see any of that.
I don't know why anyone would choose this over the actively (community) maintained proper Postgres project.
His project is cool. Students could use it as an example of porting code. Companies could switch to it, if it works. There are hundreds of reasons why people may want it, so it's awesome he's publishing one. Something needs to be done to get interest and then adoption.
Just keep using the proper Postgres project if it makes more sense to you. But notice that this isn't just a strict translation, it actually makes changes (moving from process to thread based design, for instance). Time will tell if people find these changes beneficial or if the original remains preferable.
It's also worth noting, that while you are able to use LLM's to produce your own translation, he's actually done it. There's value in actually putting in the work.
This isn't a unique situation at all. Many Postgres extensions are developed and maintained by a single person, and may therefore be avoided by more conservative users, even if they offer some technical advantages. To each, their own.
There’s no claim being made that your rewrite cannot be better.
He has provided benchmark results which provide a dimension amongst which to measure your rewrite. If you can do better by all means post your rewrite.
Finally, these kind of projects can eventually over time become projects that are actively used. Postgres is not some entity that existed before the universe was created; it was also created by someone and then eventually adoption picked up over time.
Came here to the same thing. I did something similar (but bolder, it doesn't slavishly copy Postgres and is based on current DB research papers and the like, and other bits I've been exposed to over the years). It has a full TPC-C-esque benchmark suite, replication, embedded v8/JavaScript relations/stored procedures, a giant suite of regression tests, it kicks the crap out of a lot of existing OLTP DB stuff out there. And I personally do have a background in commercial DB development.
But I choose not to publish it or promote it for many of the reasons you mention above (and more)
... For one, if "I" can do it, so can a hundred other people. And all the bold claims behind it would need to be backed up and supported and it promoted, etc, which is a whole pile of time that doesn't involve writing code.
It's the organization around a project that matters, not the code. It's not the 90s anymore w/ people piling into MySQL because it was the only option. People aren't going to be trusting your software with their data, if they can't trust you.
And unless someone is going to dump a pile of money or something on [me|them], I don't have the ability to build that organization ... as I need to feed my family... Nor am I willing to put my personal reputation on the line by putting up a huge quickly written application and then someone finding something in it I can't explain.
So like probably 500 other projects I have it sitting in a private repo.
It's a very weird time right now. "Technical" excellence isn't the important part. Organizational excellence is. This was always the case but it's more so now.
... In the meantime, if anybody has angel investment to burn, I have something potentially better/more-exciting than this guy's project but... see above...
> If you can do a Rust rewrite with AI, I can create one as well.
Translations are a lot more likely to be error-free and robust, as the original data structures and algorithms have been battle tested.
I start to see a lot of these re-writes that depend on tests to state that its working. But the things that make software like Postgres and SQLite reliable are not mostly the test, but the real world production scars. That's where the reliability comes from, years and years of running in production.
> not mostly the test, but the real world production scars
Most extensive test suites are exactly production scars: every time you have a bug or a regression, you write a test that confirms correct behaviour.
SQLite is a good example to bring up because its extensive closed-source tests are what’s often cited as being what keeps people from forking it. (Turso did it, though, but it takes a company to deliver some guarantee of equivalent diligence.)
And yes, years and years of running.
I agree. I also agree with the sibling reply that -
> every time you have a bug or a regression, you write a test that confirms correct behaviour.
What I fail to see in these rewrites however is - what about new bugs introduced by virtue of this rewrite? I mean it'll have to go through its own challenges in real-world scenarios, right?
> I start to see a lot of these re-writes that depend on tests to state that its working.
There's another way to validate the rewrite though. Just run both pgrust and postgres and compare the output. Know of an edge case? Run it too. Doesn't know? Use a fuzzer or some automated tool to find interesting inputs. Found an inconsistency? The input/output pair becomes a test case now
Not sure if there's tooling for that though. If there is, just give it to Claude so they will incorporate it in their development loop
Not weighing in on this specific rewrite but tests are how you specify that your software works correctly. If a behavior isn’t covered by an automated test in some form you can’t assert that any given change doesn’t break it.
I think it is completely reasonable to use a preexisting unmodified test suite to state that something is working. The larger the project the more true this becomes. Real world production scars are documented and guarded against in the test suite otherwise those lessons get lost.
Also SQLite is legendary for its massive test suite and extensive fuzzing. They have 590x the amount of test code and scripts than normal code. Source: https://sqlite.org/testing.html
Great point. Stated another way:
"Mom, can I have battle-tested, reliable software"
"We have battle-tested, reliable software at home"
Battle-tested, reliable software at home: (Pic of green text from `cargo test`)
Completely agree with this.
The biggest lie of software engineering is that everything can be testable with tests. That a 100% test coverage is an indicator of quality software.
> That's where the reliability comes from
So, we should make it easier to feed that reliability back upstream.
Probably the most useful thing you can do with these LLM-transpilations for now: If the transpiled version passes all original tests, I can run my application test suite against it and use it to discover test coverage deficiencies in the original!
If it crashes or otherwise observably misbehaves, I know the real project was missing regression tests for something. We could make upstream so much more resilient against accidentally breaking stuff in future updates, if only it becomes safe (offline + no side effects) and easy (if it crashes/locks, it is not from some memory safety bug from 25k transactions earlier) to run these transpiled projects as one row in our everyday integration matrix.
As sibling mentioned - bugs and regressions are the thing that are (in a perfect world) usually covered.
The problem however is non-covered success cases. A visualisation of the problem: let's say universe of interaction for DB consists of 10.000 SQL queries. Over 10 years various regressions were found and 2.000 SQL queries are guarded by tests. In reference implementation remaining 8.000 never surfaced over this time and it's unclear if they will work.
And, thinking of how many various SQL queries PostgreSQL users around the world are using vs the test cases covered it's obvious that feature space isn't covered in 1% of the success ratio cases.
Now the new, test-based implementation, has to prove it can handle remaining 99%.
That's precisely what a regression test suite is for. There is a bug, you fix the bug, you add a regression test. So if the test suite is well maintained these real world production scars are reflected in the tests.
And also the amount of people running it in thousands of scenarios. Not sure if these areas can be even tested for, but I guess time will tell (can observe Bun if it breaks somewhere as that’s afaik the first big AI rewrite which got into prod for masses).
I hope you are not true at all.
Software like a Database should have an extensive test bench with concurrency tests, all corner cases etc.
I'm not here running the new version on production to tell the maintainer/devs that my 'production unit tests failed'.
What is this even for logic?
I mean there is balance when i write tests for my production software, but my software is used by me. If i would have a library, i would test everything.
And there was some blog post about another database system were they even virtualized the File access to test cases like when the disk controller stops working.
"Everybody has a production system. The lucky ones also have a test system."
Wait - does the AI rewrite the tests too? If so, lol.
Why so much negativity? I find these projects interesting for learning purposes and exploring new ways. What’s wrong with that?
Regression tests start to play a different role with LLMs.
On one hand, they give an LLM a short feedback loop to correct itself, and iterate fast when writing code. A human also uses it as a feedback loop, but we don't iterate as fast and don't handle big walls of conditions, so its effect is not as big.
On the other hand, LLM's ability to handle a big wall of if-conditions can backfire if it starts taking shortcuts and taking the tests-as-a-spec too literally, overfitting the solution, overly focusing on the given datapoints (conditions checked by tests) and missing the overall behavior shape that the tests intend to pin down. For humans, this is less of a concern because we are bad at big walls of if-conditions, and we'd rather try to see the original shape that the tests are pinning down than monkey-patch the solution to fit the individual points.
It's interesting to see how one balanced these two. In this case particularly. Maybe you could play around with separating the data you give an LLM into "training set" and "validation set", training set can be seen fully, but validation set is hidden and is only queried when the solution is deemed ready. Say, training set = original source code + half of the tests; LLM uses that for quick feedback loop. And validation set = the remaining half of the tests; test code is not shown to the LLM and run only when the LLM says it's done to catch potential overfitting of the resulting solution over training set.
To me, the credibility of a solution like that would depend on what methodology the authors used. If they just let the LLM see all tests, I'd be skeptical (albeit unable to point out specific bugs due to the volume of work and LLM's ability to make bad things look trustworthy). The good thing is, real-life use will add new, unseen before datapoints for testing — so validation set will build up with time. Really curious to see how it will work.
Porting perfectly working C code that has been in production for decades (and has a great track record wrt to security fixes) to rust is more or less an obfuscation challenge.
I am not trolling, but I have a simple question: Why? Why do I use this instead of the official build? What is the business case?
Rust and its ecosystem needs to become more original. There are so many new problems that needs software solutions. Existing solutions that already work don't have to be rewritten in Rust.
Given that you reuse Postgres' tests and LLM have been clearly trained on Postgres' three decades of contributions, this may constitute a license violation. Unless you include the original license of course. From a human level I also understand how you ended up with a less permissive license.
I suspect the future of open source will be to never publish your tests. Or someone will just pump them into an LLM like this.
Why should a developer use this for anything beyond a pet project? Just because it is written in Rust?
All these "rewritten in rust" projects only reinforce the idea that a significant part of the rust community consists of software talibans and not of engineers who must deliver something that works and is reliable over time.
Jepsen or GTFO.
These days there's little chance for a new DB to build its community using network effect. If you want this to catch on, switch to manually grinding community building ASAP gold plating the experience for a specific niche (AI can guide your priorities but will hinder your comms). Otherwise, have fun building!
Neat as a pet project, but anyone thinking of using this is production is insane.
Rewriten in Rust is becoming a meme now.
I don’t trust AI rewrites, definitely not in 2026 — possibly never.
I'm starting to get a bit of fatigue for these projects that boil down to just "I asked Claude to re-write this code into a new language that's in vogue right now!"
I really don't understand why this is needed outside of an opportunity to show how impressive LLMs can be when working within large codebases, but even then people in the comments are finding bizarre implementation choices that a human developer wouldn't make. I'll stick with Postgres and its - gasp - C implementation for now, thanks.
> <something> rewrite to rust using AI sound like meme now.
https://news.ycombinator.com/item?id=48474313
Congrats for launching this project. I think it's awesome. I did two similar projects, just to learn how to work with LLMs and it felt good--reminded me times debugging code with GDB and going through stuff in semi-manual mode.
I'm building data + AI platform. It got complex, and I'm using AI-assisted coding to move fast. One thing that helps me was property based testing. I have a traffic generator that simulates 10 users working on my platform. I run it 24/7 and if it shows that the software survives the test (I called it "fate" after ffmpeg's CI), it's good enough to roll out. If you wrote something like that, core PostgreSQL folks would like it too, unless there's something equivalent like this. It'd be: create random tables, fill with random data, then issue randomly constructed query.
It is very bad metric to measure a rewrite using the original tests. It is like to say you clone a cat by mimic 600 cat images.
X written in Rust seems like the new Hello World for LLM coding agents now.
Quite a lot of projects are trying this "rewrite to a new language using LLM", both internally, or externally (like is here). For me, they confirm some (slightly controversial) takes.
1. human code reviews are dead. We don't yet know what's next. Two reasons they are dead: too much code to review, and code reviewing sucks (who wants to spend their days reviewing code?) 2. Not knowing how to review LLM code is a big barrier to adoption, but bigger regression test suites (testability/evals) is almost certainly the direction. 3. There are a lot of projects that haven't moved to more modern infra because it was too hard. Now it's much easier. Sure stuff will go wrong. Sure it all has to be tested. What's new here? 4. Programming languages for LLMs are coming. 5. Projects that don't allow AI coding will be forced to come around or fade.
Separately, bit off topic:
New projects will often have LLMs built in, so non-determinism will be inherent in the project. No amount of code review will be able to eliminate that.
Might be a very good occasion to actually improve the test suites from our load-bearing software projects. I feel this will be the decade of a cat and mouse game between LLM PRs and finding good (as in convincing whomever is paying you and is waiting for any occasion to fire you for being anti-progress or something).
Hopefully we get: actual formal coding rules, spec rules, design rules, contribution rules, documentation and testing rules. High Integrity development processes impose that you write all this before you start and makes sure you follow your own rules.
So. I guess... welcome everyone to explicit software and systems development processes.
I wonder how long this will be maintained for...
What is the future of this? Code is not the same as a viable open-source project with a community, contributors, advocates, users and funding, even if it's perfect code.
Even though I'm sure it won't be easy to convince the Postgres project to switch to Rust, I do think that trying would be time better spent.
Is the 300x performance boost attributable to the threading model vs process model?
Was the code for the threading model written by hand or was it translated from the WIP threading model the human PG team is busy with as part of the 2028 roadmap?
I think we will actually see some successful projects coming out of this. There are definitely people who want x old project in this new/better programming language and who are willing to put effort into maintaining it not just doing one off port.
It's silly, nearby all Rust projects are rewrites of existing projects and now as you don't need to learn Rust to do rewrites, people just let the AI rewrite projects in Rust.
How is the performance compared to regular PostgreSQL?
I know it says it is not performance optimized yet, but if this succeeds, will it only bring more "memory safety" or is there a serious performance gain as well?
Let's hope it did not special case (overfit) for the specific tests. One of the failiure modes that in my experience takes the most effort to mitigate for.
This the type of AI generated tool I'm interested in seeing more of. I'm a fan of how the quality is verified using clearly defined quantitative measurements for performance and correctness.
I'm still skeptical about LLMs and don't use them, although I can be convinced by more demonstrated examples of success.
I just want to quibble that the 100% Postgres regression tests do not test the threaded aspect of this project, and that is a pretty fundamental architecture change.
You got me with the rewrite, you lost me with AI.
Setting aside the “why” that everyone is so focused on, I want to know, how? I can’t get Claude to do anything even a fraction of this complexity. How are people setting up their agents, Claude.md, etc to do such big projects? There are a lot of lessons to be learned.
I think the cool thing about these projects is that even if test parity reaches 100%, some bugs are going to surface on the new project that don't exist on the original project.
This is usually a good example of a test case that the upstream project is not covering and can be contributed back.
Parity should be bidirectional, so definitely it is possible for both parties to benefit from it.
It's so great. One thing that I'd like to be changed in PostgreSQL, which may be done in this rewrite, is resigning from the "one connection = one process" design choice and instead handle the connections using threads/tasks within the main process.
This is a stunt for freshpaint.io that sells AI. No one will use this plagiarized monstrosity.
codebase full of code that wouldn't fly in production.
I ain't no Rustacean - but 'unsafe' calls all over.
I started my programming career by porting code from one model of TI calculator to another. It was code I could not have written from scratch myself at the time. I learned a lot about the two different versions of TI Basic that the calculators used, but I didn't learn how the program really worked. I can't even remember now if it was Tetris or the tank game that I ported. Maybe it was both? That was a boring English class...
I totally understand why porting code is fun. It's kind of like when I checked out drawing books from the library as a kid and just traced the pictures because my own attempts at drawing were so bad. It gives you a feeling of accomplishment, even though you didn't actually do anything that difficult. And you do learn some things along the way.
Doing the same with an LLM probably gives you that similar feeling of accomplishment, even though you didn't actually do that much (sorry, hate to say it that way). I wonder if you learn even less in the process. Maybe you just learn different things.
Now that I think about it, even writing some code from scratch with an LLM is not much different than doing a porting project. Someone else did the hard work of creating the original programs that the LLM was trained on, and now you (the LLM really) are just porting/restating what someone else did. I hadn't thought of that before
it's a neat experiment, obviously inspired by the bun guy.
but given the author/maintainer is essentially unknown, I highly doubt this will reach it's target audience. But one thing I do agree with is that postgres is long overdue for a re-write into a memory safe language.
also any real swe with more than a few years of experience knows "100% of regression suite passing" doesn't mean anything other than a neat checkmark for C-level executives.
It’s interesting to see how llms have turned the concept of rewrite it in rust, from an impossibility for some projects (code is too large and complicated, it will take too much time) to a real possibility for even large projects.
>use Rust plus AI-assisted programming
>pgrust is licensed under AGPL-3.0
pgrust isn't licensed at all. AI generated code isn't copyrightable. Thanks for spending the tokens I guess.
That's actually interesting - gives C developers motivation to improve postgresql. after all people could say "look, Rust makes this easier".
The WebAssembly demo that runs in your browser is a really neat touch: https://pgrust.com
What would be interesting is if they found a memory unsafe bug. Postgres is a perfect case study of 30 years of C with a bit of CPP; if rewriting in a safer language didn't find anything...
> The goal is to make Postgres easier to change from the inside
uh-huh, sure.
you want to show off "look what the LLM can do / look what I burned a bunch of tokens on"?
you want to brag about how your LLM-generated slop is somehow more maintainable than the original because blah blah blah Rust?
here [0] is the version history of Postgres. pick a version from the past. let's say 14.x because it's the most current that's still under active support.
have your LLM implement version parity with 14.x. show off how it passes all the tests blah blah blah.
then have it upgrade your codebase to parity with 15.x, implementing whatever new features and bugfixes that includes.
and have it generate an automated test that demonstrates upgrading an actual database from LLM-14.x to LLM-15.x and verifying there's no data loss or corruption. maybe even multiple such tests, if you're feeling fancy.
then lather, rinse and repeat with 16, 17, and 18.
and show off the diffs of each version. does the LLM rewrite a huge pile of already-working code in the process of each version upgrade? does it introduce new latent bugs in the process - the kind of things the existing test suite didn't think to explicitly test for?
"I took a static snapshot of code and converted it to another static snapshot of code" is meaningless. all you're doing is bragging about having more money than good sense.
the stability and trustworthiness of software like Postgres does not come from a one-time snapshot showing tests passing. it comes from the engineering process that produces the software and its test suite.
oh, and for shits and giggles, because this same test was so illuminating with the Bun "rewrite" into Rust, here is the file with the most unsafe blocks in the codebase:
why does a single 2000-line file have over 100 unsafe blocks?why is the parser unsafe at all?!?
0: https://en.wikipedia.org/wiki/PostgreSQL#Release_history
finally, a database that will refuse to compile your schema migration because you used an unsafe join. progress.
I love llm coding. I don't know what I am looking at here
https://github.com/malisper/pgrust/blob/main/Cargo.lock
What is happening.
No PRs? No Make files? I understand running tests and debugging is the workflow, but where do you log things? How do you orchestrate builds? Etc.
`Cargo.lock` is a lockfile machine generated by Cargo. It's similar to package-lock.json. In any case, its machine generated the old fashioned way.
Like most modern languages, Rust has its own build system and package manager, Cargo. Everything you're referring to relates to that, and has nothing to do with LLM coding.
Edit: saw the clarification in another comment. But, in that case the essential point seems to be "I'm not familiar with something, therefore it's suspect."
> I love llm coding. I don't know what I am looking at here
There might be some correlation here.
Rust feels like the just right language to produce all those slops of AI and the language itself. Great to promote yourself as a productive engineer, but at the end of the day you’re just reinforcing the statement that AI and the language itself are great, not you.
Where will this leave the c based driver which I believe is the basis for many others?
Till its used in prod for few years and polished, I wont touch that.
Too many things tests wont catch.
Rewrites in Rust are kinda impressive. This language with its move semantics and close ownership tracking is very different from every other language. To create a rewrite in it, you have to rearchitect the code. There is not as much freedom there when it comes to where to keep what and where you can pass what as it is in other languages.
> rewrite of already popular technology in a different language > look at commit history > "Claude xxx committed yyy ago"
I'm sorry, but what is this need to just vibe code a port of an existing technology to a different language/framework/etc.? If it's just a personal challenge then sure I guess, but this surely can't be used as a real product?
Is this another llm-driven rewrite?
I wonder how many "unsafe" blocks are in there...
From what I skimmed manually, not that many, but the code itself seems labyrinthical. Like, why have both Rust Try-supporting Error-like tagged union, but also booleans, for error handling, in the same function?
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
Woah! AGPL? That's interesting. I think Postgres has shown an open source SQL server didn't need a copy-left license to develop sustainably, so I'm not entirely aure about that, but I do like the license in general.
When the software consists entirely of ~$1000 worth of Claude credits and ~40 hours of developer time prompting and curating it, literally what does it matter what license the resulting 100k LoC artifact is provided under?
Copyleft and the whole software licensing ecosystem only matter when producing that software actually requires serious human effort and dedication.
Super impressive! Remember talking with Michael about his experience with Citus at Heap and reading his blog posts on Postgres
Super cool to see him working on this now, almost 10 years later
Database & AGPL bad combination
If the underlying code ends up being a completely unreadable blob that no human will ever read why not directly do a port to assembly instead of rust?
Did similar with S3 and that too (eventually) did well against tests (the ceph s3 ones).
...but haven't dared use it for anything meaningful yet. Still feels like there is a real world gap in confidence when it comes to vibecoded rewrites.
Been wondering whether the answer is to insert a proxy...something that effectively splits traffic to a known S3 and the rewrite and compares outcomes over time. Do that for a couple different workloads for a month or so and if it's all identical then it's probably fine...
Great! Now ask it to rewrite it in CSS!
`seams` is the new emdash
Does it support the extension ecosystem? Or would extensions need to be rewritten as well?
It is theoretically possible to have a Rust port of Postgres support extensions. If you make all the relevant functions and structures ABI compatible with Postgres, extensions should work. The issue is the moment you're dealing with C pointers and C strings, pretty much all the code you have to write is unsafe.
They would need to be rewritten as there is no formal extension API. Extensions can call into almost any part of PostgreSQL.
They would need rewriting (a few are included)
What a peculiar kind of rewrite.
Rust:
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
Original:
https://github.com/postgres/postgres/blob/df293aed46e3133df3...
Usage:
https://github.com/malisper/pgrust/blob/3646a73515a5e4ac7d0b...
The return type in the rewrite is both some sort of Error tagged union that supports the Try machinery in Rust; but, it also contains a boolean that apparently must be checked; or something. It seems labyrinthical and possibly broken and terrible.
I make no claim as to whether the change makes sense given that I didn't look at the callers of this function, but Result<bool> is an entirely reasonable pattern in Rust. If you want the callers to be able to distinguish between "has the subclass", "doesn't have the subclass", and "something went wrong" this is idiomatic Rust.
It is a feature in Rust, not a bug :-) (I know you didn't say it is a bug.)
The error-tagged union is PgResult<bool> - which means it contains bool as the result if things go well. (The other part in the union is of course the error.)
In the original function also, it is returning a boolean: "bool has_subclass".
So anyway you have to check for the boolean as part of the logic. That is what it is doing.
I don’t understand these rewrites, honestly, what is the point? Who have had any C++ related issues while working with Postgres?
Now do Freetype and libtiff/libpng/etc.
I have privately wondered for years, pre-AI, why Apple hadn’t paid some engineers to go off and write some comprehensive test suites and then port these to Swift. It would shut down entire swaths of memory safety bugs they have been coping with for literally decades. SO MANY of the zeroclick iOS exploits can be traced to a few fragile and vulnerable foss libraries, xkcd 2347 style.
First bun, now it's PG's turn again, although this isn't official.
I have a feeling that AI is rewriting everything!
And they'll reproduce virtually the same problems, because code was never a primary issue for output. Products reflect organizational problems, which AI can't solve.
Now which one is safer? A new Postgres written in Rust, or the original real world tested Postgres?
Also, are they calling it Postgrust?
Rust is a stripper
related? https://www.youtube.com/watch?v=e35AQK014tI
but did they change the process-per-connection model? if not, wtf??
Yes, see top comment.
[flagged]
[flagged]
[dead]
[dead]
The great Jarred Sumner pulled it off with bun, whether it can be pulled of with Postgres is an open question..
DST systems such as Antithesis can definitely help.
We had one for SQLite (which is SQL-ite btw, not SQ-Lite which doesn't make any sense) via Turso, no wonder we see the same for Postgres. Personally I do want to see libraries be in as much memory safe languages as possible.
How do you know it's not SQL-lite with the single L serving a double role?
Common pronunciations allow you to stay perfectly ambiguous about where the L goes, which aligns quite well with the name as spelled. If you do it right, nobody can tell if you're saying sequel-ite or sequel-lite or seque-lite on the one hand, or S-Q-L-ite or S-Q-L-lite or S-Q-lite on the other.
AFAIK there is no official word on how the name is intended to be read or said.
The code is so weird, people will have a hard time reading it when they need to check for correctness.
There are much better ways to write it in Rust: https://github.com/malisper/pgrust/blob/14ffab7d31a31e5ab667...
I mean, you've linked to a function performing md5 hashing. That's pretty much exactly what I would expect such code to look like.
Case in point: https://github.com/postgres/postgres/blob/master/src/common/...