TIL - YouTrack was first released in 2009, Kotlin in 2011, so I wouldn't call it long before, but definitely before. Guess I'm more interested in programming languages than in task management tools, that's probably why I have been aware of Kotlin longer than of YouTrack...
Kotlin 1.0 was released early 2016. I can see why YouTrack team might not have wanted to commit using an unstable language, though I can also see the dogfooding angle.
You can port files from Java to Kotlin by pressing a button, but Jetbrains has generally not done bulk ports. They leave it up to individual devs to convert a file when it's being worked on, if the diff and VCS history pollution is worth it.
I wonder if there's a design decision documented somewhere that makes the existing graph databases like Neo4j, etc. not good enough for Youtrack's use case.
Neo4j is a great DB but their license price is egregious for enterprise customers. A few years ago I was involved in negotiating a contract for a small/medium size kubernetes deployment (think around 25 cores) and the annual price was more than the salary of a senior SWE full-time equivalent. See this page for an idea of their prices in 2018: https://blog.igovsol.com/2018/01/10/Neo4j-Commercial-Prices....
Also embedding Neo4j is not possible, that seems to be the killer feature for YouTrackDB, they even shade dependencies so it’s like a no deps Java library for your application.
> small/medium size kubernetes deployment (think around 25 cores
That's ~1 machine. 1 SWE for a database isn't egregious, databases provide huge value, but for that little performance, that's crazy.
I can only assume as core count has blown up over the last 10 years, the pricing has somewhat diminished, but still, I'd be expecting a heck of a lot more capacity for 1 SWE.
I can tell you from quotes that I've seen, that compared to the 2018 prices listed on the page, end-of-2022 pricing was slightly up per core.
We were already well on our way of dumping Neo4J due to performance/operational/architectural reasons at the point, but seeing the quote solidified it.
Not just that, if a database company has both a community edition and enterprise then it’s likely the enterprise will get many new features that the community edition will never get.
Didn't Neo4J pivot away from being a boring embedded DB which you point at a path and then traverse through Node objects, and decide to become some kind of paid platform with a client-server protocol and proprietary query DSL?
I remembered it from a uni course (early 10s?) a few years ago for a use-case we didn't end up pursuing, but I wasn't hugely comfortable with investing effort into what I saw.
That's true, although, if you look at them, you wouldn't notice. The only mention of JVM you can see in the IDEs is in the About dialog, and the IDEs install and run their own OpenJDK, so no JVM has to be installed globally. Almost as if they were a bit self-conscious about using such an "unsexy" architecture...
You mean a JRE, because the whole JDK contains a bunch of things you're never going to need.
Mind you, a default JRE redistribution makes your app at least 100+MB. Using jdeps to strip out unneeded things is a good idea if you want it to get down to 25 ish MBs.
Yeah, saying JRE is a bit of a shortcut since you're supposed to jlink & jpackage & jdance &jpray to get a slimmed down JVM released with your app, but it's closer to what would be a JRE than a full JDK
I've always been curious about graph DBs and dabbled a bit in them, but for those who have more extensive experience in them-- are they really worth it? Is it that for small scale SQL is better and graph DBs really only matter at scale, or for specific use cases with highly connected data?
I worked in a startup whose value proposition was largely derived from using graph data and graph databases under the hood. The main benefit is, as the repo even states "Fast data processing: Links traversal is processed with O(1) complexity."
So, technically, you can do deep traversals quicker. A few notes:
1. Few use cases truly need low-latency deep traversal on realtime data (>5 hops). There are some well known ones like fraud detection in payment processing and, possibly, social media recommendation engines. But I am not even sure how latest social media engines work, and whether they still rely on graph DBs.
2. However, in practice the advantage is often marginal. With modern analytical databases, or even an optimized PostgreSQL (ltree, materialized views, pgrouting, pg_duckdb, etc), you often get more than good enough performance. In addition those traditional SQL DBs scale with hardware more easily than graph databases. So, you can always use the lever: "Throw more hardware at it."
3. Even Graph DBs don't get good traversal performance under all conditions without hand tuning. For example, there is the "super node" issue, a node with an abnormally high number of connections (edges). And if you still need hand-tuning, you might as well choose something more versatile.
4. The ecosystem of a PostgreSQL and other popular DBs is just unbeaten. With graph DBs, you often prematurely put yourself in a corner that you don't want to be in.
Hence, my recommendation. Unless you are really sure that a graph DB is the right fit for your use case, start with something else, and go the graph db way when you have established a true need.
Most graph databases only offer an advantage in query language, as it allows for more ergonomic graph traversal compared to e.g. recursive CTEs in SQL. However with SQL/PGQ, the same query ergonomics are coming to traditional databases.
If you look under the hood there is usually nothing special about graph databases that will make them more performant. If you lay out a query plan side-by side between e.g. Postgres and Neo4J, they will look identical, just that the leaf-nodes in Postgres will be a table-scan, while in Neo4J they will be either a vertex-scan or a edge-scan (which can both be seen as special cases of a table-scan).
As someone that has worked a lot with graph databases in the past, I'd largely recommend not using them. The price you pay in terms of worse ecosystem and less battle-tested maintenance tooling is not worth it to just have better syntax.
Apache AGE seems like a good way to have the battle testedness of Postgres with a more fluent query language. I have only used AGE in smaller projects so far however.
AGE actually looks very interesting. When I evaluated it a while back it seemed like the project wasn't very active and I couldn't find any information about folks using it at scale in production. If I was forced to use a graph database again, and I didn't immediately quit upon receiving that news, I might give AGE a look.
But only just. It doesn't really replace graph databases. It's mostly a slightly more convenient ways to write certain kinds of queries, but it doesn't come with anything else you might expect from a graph database.
I have had very bad experiences with graph dbs at scale. To the point where I will never again work on a project based on a graph db. YMMV, but I'm done with them forever.
[edit] To elaborate: tuning read and write performance is difficult, essentially everything must be indexed. This results in an explosion in data size--so data that might be X TB in csv format becomes maybe 10X when you consider all the indexes and bloat from the graph db's storage format. Which brings us to the topic of compression: there isn't any. Moreover, graph dbs are basically all quite immature compared to other db software, they haven't seen the test of production to the same degree, nor have they gotten the same attention in terms of bugfixes, performance tuning, etc. So they often have very sharp edges that you encounter under load. So if you like spending tons of money and time on zero value work, choose a graph db!
[another edit] Another problem with graph dbs is they encourage laziness in data modelling. I would echo others' recommendations here: just start with SQL. If it gets to the point where you actually need a graph query language--and you've already appropriately considered your life choices and determined that there's some value to what you're doing--only then should you consider trying to implement something on a graph query engine. But I'm not aware of one that is actually good. That's the problem. If you're at the point where you absolutely need a graph database you are probably at the point where you need to build your own. And you'll probably fail like all the other ones.
I'm a bit of a Forensic Files nerd, so I worked on a little side project a while back to pull the transcripts from several episodes, use entity recognition to categorize people, places, things, etc, and load them into a Neo4j database (via Cypher queries). It turns out there's something called the POLE data model[1] that can be used by law enforcement to help solve crimes. You load all the details into a graph database and evaluate the relationships to aid in solving crimes. I suppose you could argue a criminal investigation is essentially graph traversal.
Graph query languages like Cypher are great, but I am wholly unconvinced by the concept of a dedicated, general-purpose "graph database".
IMO, you're better off just using Postgres/etc, modeling your graph data there, and pulling in subsets of your graph for in-memory analysis. This is for the 99% of enterprises that aren't doing online streaming graph analysis on TB-scale graphs, and the other 1% should probably figure out something tailored* to their specific business model.
* Graph algorithms are more accessible than ever with GenAI code, and efficiently modeling a graph in memory is trivial (it's just structs with pointers to other edges/nodes, plus its nice to have full control over the memory layout).
We had general-purpose graph databases before graph databases became a thing. I'm talking about relational databases. Values are the vertices, tuples are the edges, FK constraints are inclusion dependencies. In fact, n-ary relations means a hypergraph database, not just a binary graph db.
Many recent graph databases are built like this. If you compare ladybugdb vs duckdb, the internals are very similar.
The main innovation is the "REL table". You can think of it as a many-to-many relationship table on steroids with optimizations at the storage layer and join algorithms.
Definition of what makes something a graph database (apart from the query language) is contested. But we seem to be moving towards: build a reasonable relational database and then add a "REL table" to it with join optimizations.
I worked for the research branch of a children's hospital. At one point, I was brought into vendor evaluations a department wanted to buy. Their tool had something to do with visualizing protein interactions, which was highly networked. It was basically a Neo4j database with a React UI on top of it.
When you see 25 agent definitions, how do you know they are good? How do you know it’s better than 10 or 5 or 1? Is a project with 100 agent definitions better?
Object databases routinely go away and routinely come back.
Ten years ago I worked with a database called Versant OODBMS (from Actian). I was a junior sysadmin so i was essentially administering it at a very surface level but skimming the documentation (and trying some of the samples) it was very cool that you could pick essentially any random class, implement an interface (and hence a few method) and that was it, you had a database-serializable object.
The main issue was really scaling out (as in, multiple machines) but otherwise was a really great database.
That's funny, one of my buddies from college was one of the OG Versant employees (I assume there's not another OODBMS named Versant). That was in the late 80s.
I hope this doesn't complicate the process of deploying YouTrack on-prem. That's the best part about YouTrack; historically, you can deploy on prem with no backing DB. From the outside perspective, it just stores everything to flat files.
I played around with something similar before called typedb. It also does object oriented graph model which was a bit weird at first after using sql for a long time, but once it clicks there's a lot of thing that you could express surprisingly nicely
* the technical leader of the YouTrackDB project, Andrii Lomakin, was one of the main contributors to OrientDB -- see https://medium.com/@youtrackdb
* YouTrackDB is a fork of OrientDB
* this project makes heavy use of Claude Code
* YouTrackDB does not use Kotlin
* this can be used as a network service (rather than just embedded in a single runtime process like in current production YouTrack), but I've not yet got the docker images working to try.
from the blurb
YouTrackDB is a general-purpose object-oriented graph database developed by JetBrains, used internally in production. It implements the Apache TinkerPop API with Gremlin query language support and features O(1) link traversal, schema-less/mixed/full modes, and encryption at rest. The project is a fork of OrientDB, re-architected under the com.jetbrains.youtrackdb package namespace.
I wasn't surprised that it's written in a JVM language, but I was surprised that that language is Java, not JetBrains' very own Kotlin...
Youtrack was made long before Kotlin so I suspect the rewrite just wasn't worth it
Thing is, iirc Kotlin is mostly compatiable / interchangeable with Java so they could write any new code in Kotlin.
TIL - YouTrack was first released in 2009, Kotlin in 2011, so I wouldn't call it long before, but definitely before. Guess I'm more interested in programming languages than in task management tools, that's probably why I have been aware of Kotlin longer than of YouTrack...
Kotlin 1.0 was released early 2016. I can see why YouTrack team might not have wanted to commit using an unstable language, though I can also see the dogfooding angle.
You can port files from Java to Kotlin by pressing a button, but Jetbrains has generally not done bulk ports. They leave it up to individual devs to convert a file when it's being worked on, if the diff and VCS history pollution is worth it.
I wonder if there's a design decision documented somewhere that makes the existing graph databases like Neo4j, etc. not good enough for Youtrack's use case.
Neo4j is a great DB but their license price is egregious for enterprise customers. A few years ago I was involved in negotiating a contract for a small/medium size kubernetes deployment (think around 25 cores) and the annual price was more than the salary of a senior SWE full-time equivalent. See this page for an idea of their prices in 2018: https://blog.igovsol.com/2018/01/10/Neo4j-Commercial-Prices....
Also embedding Neo4j is not possible, that seems to be the killer feature for YouTrackDB, they even shade dependencies so it’s like a no deps Java library for your application.
> small/medium size kubernetes deployment (think around 25 cores
That's ~1 machine. 1 SWE for a database isn't egregious, databases provide huge value, but for that little performance, that's crazy.
I can only assume as core count has blown up over the last 10 years, the pricing has somewhat diminished, but still, I'd be expecting a heck of a lot more capacity for 1 SWE.
I can tell you from quotes that I've seen, that compared to the 2018 prices listed on the page, end-of-2022 pricing was slightly up per core.
We were already well on our way of dumping Neo4J due to performance/operational/architectural reasons at the point, but seeing the quote solidified it.
There are several MIT/Apache licensed embedded graph databases available, including the one linked from my profile.
Not just that, if a database company has both a community edition and enterprise then it’s likely the enterprise will get many new features that the community edition will never get.
Ongoing enshittification risk.
Didn't Neo4J pivot away from being a boring embedded DB which you point at a path and then traverse through Node objects, and decide to become some kind of paid platform with a client-server protocol and proprietary query DSL?
I remembered it from a uni course (early 10s?) a few years ago for a use-case we didn't end up pursuing, but I wasn't hugely comfortable with investing effort into what I saw.
That's a good question indeed. Also I wonder why they picked java as the implementation language.
Likely because they are a Java shop. All the IDEs they develop use Java, so they have quite an expertise in low level optimization for this language.
That's true, although, if you look at them, you wouldn't notice. The only mention of JVM you can see in the IDEs is in the About dialog, and the IDEs install and run their own OpenJDK, so no JVM has to be installed globally. Almost as if they were a bit self-conscious about using such an "unsexy" architecture...
Bundling a JDK with the app is the officially sanctioned way to ship Java apps since Java 9, so for over a decade now.
You mean a JRE, because the whole JDK contains a bunch of things you're never going to need.
Mind you, a default JRE redistribution makes your app at least 100+MB. Using jdeps to strip out unneeded things is a good idea if you want it to get down to 25 ish MBs.
JREs haven't been a thing in like a decade I thought.
Yeah, saying JRE is a bit of a shortcut since you're supposed to jlink & jpackage & jdance &jpray to get a slimmed down JVM released with your app, but it's closer to what would be a JRE than a full JDK
The JDK is too big for every app to do that, imo.
Which is why there are linking and packaging tools for trimming fat that isn't needed, or even AOT compile the application.
They also have C# expertise, but yes, Java is probably the language they have the most expertise in.
Date of commits shows this project is 3 years old.
It’s JetBrains who were synonymous with Java so not a surprise, if was a recent project would have been Kotlin (which this company created)
Probably worth saying it's a fork of orientdb from like 2010.
Because Youtrack itself is written in java
I've always been curious about graph DBs and dabbled a bit in them, but for those who have more extensive experience in them-- are they really worth it? Is it that for small scale SQL is better and graph DBs really only matter at scale, or for specific use cases with highly connected data?
I worked in a startup whose value proposition was largely derived from using graph data and graph databases under the hood. The main benefit is, as the repo even states "Fast data processing: Links traversal is processed with O(1) complexity."
So, technically, you can do deep traversals quicker. A few notes:
1. Few use cases truly need low-latency deep traversal on realtime data (>5 hops). There are some well known ones like fraud detection in payment processing and, possibly, social media recommendation engines. But I am not even sure how latest social media engines work, and whether they still rely on graph DBs.
2. However, in practice the advantage is often marginal. With modern analytical databases, or even an optimized PostgreSQL (ltree, materialized views, pgrouting, pg_duckdb, etc), you often get more than good enough performance. In addition those traditional SQL DBs scale with hardware more easily than graph databases. So, you can always use the lever: "Throw more hardware at it."
3. Even Graph DBs don't get good traversal performance under all conditions without hand tuning. For example, there is the "super node" issue, a node with an abnormally high number of connections (edges). And if you still need hand-tuning, you might as well choose something more versatile.
4. The ecosystem of a PostgreSQL and other popular DBs is just unbeaten. With graph DBs, you often prematurely put yourself in a corner that you don't want to be in.
Hence, my recommendation. Unless you are really sure that a graph DB is the right fit for your use case, start with something else, and go the graph db way when you have established a true need.
Most graph databases only offer an advantage in query language, as it allows for more ergonomic graph traversal compared to e.g. recursive CTEs in SQL. However with SQL/PGQ, the same query ergonomics are coming to traditional databases.
If you look under the hood there is usually nothing special about graph databases that will make them more performant. If you lay out a query plan side-by side between e.g. Postgres and Neo4J, they will look identical, just that the leaf-nodes in Postgres will be a table-scan, while in Neo4J they will be either a vertex-scan or a edge-scan (which can both be seen as special cases of a table-scan).
As someone that has worked a lot with graph databases in the past, I'd largely recommend not using them. The price you pay in terms of worse ecosystem and less battle-tested maintenance tooling is not worth it to just have better syntax.
Yes, now that Postgres offers property graphs: https://www.postgresql.org/docs/19/ddl-property-graphs.html, which allows ergonomic queries using graph pattern matching syntax, instead of joins, these dbs feel like dodos.
Recent benchmarking: https://theconsensus.dev/p/2026/05/29/ladybug-duckdb-and-pos...
Response: https://blog.ladybugdb.com/post/better-graph-database-ball/
> However with SQL/PGQ, the same query ergonomics are coming to traditional databases.
I like how PG19 with PGQ will kill lots of companies and startups.
Apache AGE seems like a good way to have the battle testedness of Postgres with a more fluent query language. I have only used AGE in smaller projects so far however.
AGE actually looks very interesting. When I evaluated it a while back it seemed like the project wasn't very active and I couldn't find any information about folks using it at scale in production. If I was forced to use a graph database again, and I didn't immediately quit upon receiving that news, I might give AGE a look.
Unfortunately AGE is very immature and optimized even simple traversal queries rather poorly.
Postgres 19, which is about to be released is adding support for SQL/PGQ.
But only just. It doesn't really replace graph databases. It's mostly a slightly more convenient ways to write certain kinds of queries, but it doesn't come with anything else you might expect from a graph database.
I have had very bad experiences with graph dbs at scale. To the point where I will never again work on a project based on a graph db. YMMV, but I'm done with them forever.
[edit] To elaborate: tuning read and write performance is difficult, essentially everything must be indexed. This results in an explosion in data size--so data that might be X TB in csv format becomes maybe 10X when you consider all the indexes and bloat from the graph db's storage format. Which brings us to the topic of compression: there isn't any. Moreover, graph dbs are basically all quite immature compared to other db software, they haven't seen the test of production to the same degree, nor have they gotten the same attention in terms of bugfixes, performance tuning, etc. So they often have very sharp edges that you encounter under load. So if you like spending tons of money and time on zero value work, choose a graph db!
[another edit] Another problem with graph dbs is they encourage laziness in data modelling. I would echo others' recommendations here: just start with SQL. If it gets to the point where you actually need a graph query language--and you've already appropriately considered your life choices and determined that there's some value to what you're doing--only then should you consider trying to implement something on a graph query engine. But I'm not aware of one that is actually good. That's the problem. If you're at the point where you absolutely need a graph database you are probably at the point where you need to build your own. And you'll probably fail like all the other ones.
I'm a bit of a Forensic Files nerd, so I worked on a little side project a while back to pull the transcripts from several episodes, use entity recognition to categorize people, places, things, etc, and load them into a Neo4j database (via Cypher queries). It turns out there's something called the POLE data model[1] that can be used by law enforcement to help solve crimes. You load all the details into a graph database and evaluate the relationships to aid in solving crimes. I suppose you could argue a criminal investigation is essentially graph traversal.
[1] https://neo4j.com/blog/government/graph-technology-pole-posi...
Edit: typo
I played around with Neo4j and only ever found two domains where it excelled.
- custom app security
- social media
I also think cypher is a brilliant way to query a graph.
Graph query languages like Cypher are great, but I am wholly unconvinced by the concept of a dedicated, general-purpose "graph database".
IMO, you're better off just using Postgres/etc, modeling your graph data there, and pulling in subsets of your graph for in-memory analysis. This is for the 99% of enterprises that aren't doing online streaming graph analysis on TB-scale graphs, and the other 1% should probably figure out something tailored* to their specific business model.
* Graph algorithms are more accessible than ever with GenAI code, and efficiently modeling a graph in memory is trivial (it's just structs with pointers to other edges/nodes, plus its nice to have full control over the memory layout).
Just use postgres (or another SQL db) ignores the following innovations:
* Columnar storage * Compressed Sparse Row on disk * Factorized joins * ASP joins, WCO Joins
Details: https://vldb.org/cidrdb/2023/kuzu-graph-database-management-...
I'm not saying "only use Postgres", but you need to explain why core graph data needs your listed features.
We had general-purpose graph databases before graph databases became a thing. I'm talking about relational databases. Values are the vertices, tuples are the edges, FK constraints are inclusion dependencies. In fact, n-ary relations means a hypergraph database, not just a binary graph db.
Many recent graph databases are built like this. If you compare ladybugdb vs duckdb, the internals are very similar.
The main innovation is the "REL table". You can think of it as a many-to-many relationship table on steroids with optimizations at the storage layer and join algorithms.
Definition of what makes something a graph database (apart from the query language) is contested. But we seem to be moving towards: build a reasonable relational database and then add a "REL table" to it with join optimizations.
I worked for the research branch of a children's hospital. At one point, I was brought into vendor evaluations a department wanted to buy. Their tool had something to do with visualizing protein interactions, which was highly networked. It was basically a Neo4j database with a React UI on top of it.
Worth clicking into .claude/agents/ in this repo. There are 25 agent definitions, 10 of them code-review dimensions.
When you see 25 agent definitions, how do you know they are good? How do you know it’s better than 10 or 5 or 1? Is a project with 100 agent definitions better?
What is interesting about them?
This is the successor of https://github.com/JetBrains/xodus, but with quite some differences.
Object databases routinely go away and routinely come back.
Ten years ago I worked with a database called Versant OODBMS (from Actian). I was a junior sysadmin so i was essentially administering it at a very surface level but skimming the documentation (and trying some of the samples) it was very cool that you could pick essentially any random class, implement an interface (and hence a few method) and that was it, you had a database-serializable object.
The main issue was really scaling out (as in, multiple machines) but otherwise was a really great database.
Yes, POET for C++, for example.
http://www.kevra.org/TheBestOfNext/ThirdPartyProducts/ThirdP...
That's funny, one of my buddies from college was one of the OG Versant employees (I assume there's not another OODBMS named Versant). That was in the late 80s.
Cool! I used Versant at work between 2016 and 2019!
I hope this doesn't complicate the process of deploying YouTrack on-prem. That's the best part about YouTrack; historically, you can deploy on prem with no backing DB. From the outside perspective, it just stores everything to flat files.
I played around with something similar before called typedb. It also does object oriented graph model which was a bit weird at first after using sql for a long time, but once it clicks there's a lot of thing that you could express surprisingly nicely
There's also falkorDb (based on redis) to consider
[0]: https://github.com/falkordb/falkordb
Do they have a doc that says what scale this can operate at?
It's embedded. Single machine. But it runs their forum/ticketing thing so good enough for that I guess :)
It's a fork of OrientDB, isn't it?
A successor to it, yes: https://medium.com/@youtrackdb/long-road-ahead-6d648141a190
is it distributed? It would be awesome to use it with graph neural network due to the O(1) graph traversal
see also https://youtrackdb.zulipchat.com/
[flagged]
[dead]
Isnt youtrackdb a odd name, if its intended as a general use db? Why link it that much to a specific known product?
It was split from youtrack internals, IIRC
use the existing one then (which is actually battle-tested) https://github.com/JetBrains/xodus
current name is smelling tho +1
summary
* AFAICT, this is not (yet) in active use in production YouTrack, which is still using https://github.com/JetBrains/xodus
* the technical leader of the YouTrackDB project, Andrii Lomakin, was one of the main contributors to OrientDB -- see https://medium.com/@youtrackdb
* YouTrackDB is a fork of OrientDB
* this project makes heavy use of Claude Code
* YouTrackDB does not use Kotlin
* this can be used as a network service (rather than just embedded in a single runtime process like in current production YouTrack), but I've not yet got the docker images working to try.
from the blurb
YouTrackDB is a general-purpose object-oriented graph database developed by JetBrains, used internally in production. It implements the Apache TinkerPop API with Gremlin query language support and features O(1) link traversal, schema-less/mixed/full modes, and encryption at rest. The project is a fork of OrientDB, re-architected under the com.jetbrains.youtrackdb package namespace.