The blog post reminds me of similar efforts with Shockwave. There's people building decompilers, and runtimes, and if you join enough Discords, you will notice the people in them are cross-contaminating between communities if you will, they share insights with one another, in their efforts towards specific goals. They're hyper focused on making one game or another come back. There's Habbo Hotel, and Coke Studios, as well as other games.
The Coke Studios effort is interesting because there were no "private servers" developed at the time, unlike Habbo which had many, and there are Shockwave Xtras that no open runtime supports currently.
There's several attempts at a full runtime as well, that run in-browser.
Projector Rays (decompiler) really was the biggest release to date, and recently people have been really hacking at it, to some extent AI has helped to reverse engineer bytecode far as I can tell.
For anyone curious, one of the runtimes is called DirPlayer:
I'm indirectly involved / hear about a project that buys up old feature phones, mainly from Japan, to try and find ones that have data for old imode games on them, notably a FFVII spinoff called Before Crisis. It's difficult because they would release the game in separate data packs, the idea being people can remove data packs for parts of the game they already played to save space (also a feature on modern smartphone games). But since the servers are long gone, they need to find phones with the data on them to extract it.
I have a lot of respect for little communities where people gather to work obsessively over a goal like this. It's fun to see projects like this that would never see this much effort applied if not for the passion of a few dedicated people.
Many of the smartest people I've known got their starts in little communities like these: Obsessively learning new things that would normally be beyond their academic or career level because they need to know something advanced to move their passion project forward.
Its what started me on the path of my career really. Understanding how systems work. I still enjoy finding cheats in offline games for myself just to see how far I can push game engines.
Where can I boot up Coke Studios? Enjoyed that game as a kid, including making music (which was so friggen cool! They had tons of samples you could just drag and drop into a timeline).
Edit: fun memory. I figured out how scamming works at some point. IIRC (and I may not) I set up a minigame in my studio, and contestants had to give me their furniture to participate in a game. Since furniture could stack weirdly, people made all sorts of crazy mazes that were actually quite difficult to navigate due to the fixed isometric perspective.
Each stage required handing over a more expensive item than the last. At some point someone handed over a super rare/expensive piece of furniture.
Not gonna erase that. ACTUALLY I think I figured out how to clone items due to a bug. Or mine infinite DBs. Used that as cover to get someone to give me their item. Was a long time ago.
Anyways, a user handed me their super rare item and I then gave them a stern warning on how this is a common scam and I just stole their item(s). Then I handed everything back over to them and they profusely thanked me. lol
Very nice. It’d be interesting if recomps of some games that were originally for both 360 and PC become the superior experience… Fable 3 and the whole Games for Windows Live nonsense the PC version had come to mind.
These projects are really neat. Last week, I was able to build (and play) the Xbox 360 release of Sonic Unleashed on a couple of ARM SBCs using Sonic Unleashed Recomp.
Years ago I was searching for why emulators don't recompile to native bytecode and it seemed like the consensus at the time was that JIT etc was faster. I can't find it now, but IIRC I remember replies from maintainers saying it was a bad idea.
What changed since then, since this seems to be a trend? Is this more for more modern systems that have more static code?
Static code. Also, very fine details of the machine because less important.
When emulating a NES (for example), you really have to emulate every register, how registers change, and also weird effects like instructions that take longer to read or write values, as games rely on that stuff. Once you have modern systems where much of the code was originally C, it becomes less important to ensure every register has exactly the right values when a subroutine finishes in most cases, you can rely that (most) of the code follows standard calling conventions.
Something like a SNES will always have the same hardware no matter what (okay, leave out the expansion chips inside cartridges, or fold them into "same hardware").
Something like a PC will have any old random shite you care to plug into it.
Something like an Xbox 360 is kind of halfway between.
With a SNES you can rely on the fact that a vertical blanking interrupt will happen 50 times a second no matter what, and you get so many instructions before your video output is halfway down the screen no matter what, and then if you want to play silly games like changing a PPU register to dick about with the screen mode you can. You have to take into account things like it taking more time for certain types of memory access like if you cross a 256-byte page boundary, but since when you get down this far you're doing it all in assembler you can place your code and data exactly where you want to get exactly the behaviour you want.
When you emulate this, you must emulate every aspect of this behaviour correctly or things will fall apart. Think about things like the "open bus bugs" that trip up emulation - some games rely on silly tricks like expecting the capacitance on the pins of the chips to hold a value like a crazy single-byte DRAM for a moment while they look somewhere else. If your emulator just goes "okay no devices are enabled so the bus is simply pulled high to 0xff" that will fail.
With a PC you've got a wider variety of hardware, and a deeper level of abstraction. You don't have to care about flipping registers at just the right time, you just throw some textures and polygons at the GPU and let it do its thing. It'll tell you when it's ready for more. This is why you get the same experience between your AMD GPU, your NVidia GPU, your Intel GPU, and hell why not just have software OpenGL too (because it's slow but that's how we used to do it in the olden days).
So if you wanted to run a SNES game on modern hardware you'd be better off emulating the hardware directly, bugs and all, and then running the "real" game binary.
But if you wanted to run something like an Xbox 360 game - written using "modern" development techniques, where a lot of the hardware is abstracted away behind drivers and libraries - you might well just break it down into functional blocks of code, rewrite them for your native CPU, and shim the calls to the libraries to talk to the ones you have on your native system.
A bit of code that rotates a set of vertices and flings them at the GPU will look pretty much the same in C on any platform, but it'll compile to very different assembly language. That's okay though! If you understand how the instructions relate to the data you can just kind of translate one set of opcodes to another and expect it to be more-or-less right. You can even (kind of) do this automatically!
You can play game backups, use unofficial HDDs at bigger sizes, send over new games with FTP, use third party and DIY controllers like GuliKit's offerings or a Flatbox leverless arcade controller. You can play a modded version of Rock Band 3 with all the songs from all the games, using a Wii Guitar Hero controller with a Pi Pico-based adapter in place of the Wiimote. Any game you installed from disc for faster load times becomes playable without the disc in automatically once you get Aurora running (this is pretty cool, on some other modded consoles like PS4 you'd have to reinstall the games another way).
As far as homebrew in the sense of people making their own games and applications, I don't know that there's much going on. Maybe you can find a controller tester app. Sadly the 360 scene has a lot of proprietary stuff going on, and uses some official dev SDKs, IIRC. Where 3DS and Switch modders have everything on GitHub and freely licensed, 360 has a lot of binary-only releases on random forums and such. It would probably be a major effort to change this and involve rewriting stuff people already know and love.
I did the the RGH 3 mod to half a dozen or so 360s a few years back for myself and friends. If you just wanna play games with modern comforts, things are in a pretty good state. We don't need to worry about the original disc drives, HDDs, or controllers wearing out and making the console useless, we can replace or avoid them all. The wireless card is also easily replaceable, a friend of mine had one that seemed non-functional after he spilled some Mtn Dew all over his 360 back in the day. Everything else seemed good after a clean but I just got him a new wireless card off eBay and swapped that out.
If anyone wants to get into this and doesn't already have a 360, the S models are easiest to mod and most reliable (no RRoD). Some are Trinity and some are Corona motherboards inside, but both are hackable. Corona may need a post-fix adapter, but they're cheap and easy to install. Just avoid the E series.
For modding tutorials, MrMario2011 on YT is great.
There is also a software-only exploit that's a bit worse but more accessible. It came out after I already did all my modding, so I haven't got experience with it.
You take an Xbox game designed to run on an Xbox 360, a 64 bit PowerPC system and decompile its binaries back into source code. You now have the ability to modify the game as well as port it to other systems and architectures such as Windows on X86_64 or Linux on ARM64.
It's more nuanced than that; the approach you're describing is usually called "decompilation."
The difference is how far one goes in hoisting the "source code;" in this "recompliation" approach the source code, while C++, is basically an IR (intermediate representation) between the original game's assembly and a host platform, and the hardware itself is emulated (for example, the original architecture's CPU registers are represented as variables in the host architecture's memory). The machine code is translated to C++ using a custom tool.
In a "decompilation" approach the game logic is converted (using a decompiler, like IDA or Ghidra's) back into something which resembles the original source code to the game itself, and the source code is usually hand analyzed, marked up, rewritten, and then ported across platforms. The product is something that attempts to resemble the original game's source code.
Of course, they lie on a continuum and both approaches can be mixed, but, while they both involve C++ in the middle, the process is starkly different. Recompilation is much more copyright-friendly, because in many implementations only the modifications are distributed and the original binary is translated by the end user (who owns the software/a license to it), whereas decompilation produces an artifact (source code) which is a derivative work encumbered by the original software's license and generally should not be distributed.
> In a "decompilation" approach the game logic is converted (using a decompiler, like IDA or Ghidra's) back into something which resembles the original source code to the game itself, and the source code is usually hand analyzed, marked up, rewritten, and then ported across platforms
There definitely is a lot of scope to apply LLMs here
Seeing this [1], I thought it was something related to taking assembly instructions in the original code, emitting C statements that match the instruction, and then compiling that C code.
Your idea is much more accurate; see my sibling comment. It's basically using C or C++ as an intermediate representation for machine code, rather than trying to recreate the game's higher-order logic/structure or source code.
Crazy that it has taken so long. I understand these are small teams trudging off in the dark but you could have imagined there would have been more eyes on this and hands tinkering in the time when the Xbox 360 was being sold than today. Right at that same time in history the iPhone was getting cracked basically every iOS release, sometimes by teenagers. Seems like there were a ton of hackers around back then.
The whole decomp and recomp scene has really started popping off this past year in the emulation community. The N64 recomp tool was the first one to really get interest going.
The blog post reminds me of similar efforts with Shockwave. There's people building decompilers, and runtimes, and if you join enough Discords, you will notice the people in them are cross-contaminating between communities if you will, they share insights with one another, in their efforts towards specific goals. They're hyper focused on making one game or another come back. There's Habbo Hotel, and Coke Studios, as well as other games.
The Coke Studios effort is interesting because there were no "private servers" developed at the time, unlike Habbo which had many, and there are Shockwave Xtras that no open runtime supports currently.
There's several attempts at a full runtime as well, that run in-browser.
Projector Rays (decompiler) really was the biggest release to date, and recently people have been really hacking at it, to some extent AI has helped to reverse engineer bytecode far as I can tell.
For anyone curious, one of the runtimes is called DirPlayer:
https://github.com/igorlira/dirplayer-rs
I'm indirectly involved / hear about a project that buys up old feature phones, mainly from Japan, to try and find ones that have data for old imode games on them, notably a FFVII spinoff called Before Crisis. It's difficult because they would release the game in separate data packs, the idea being people can remove data packs for parts of the game they already played to save space (also a feature on modern smartphone games). But since the servers are long gone, they need to find phones with the data on them to extract it.
But they've made really good progress in recent years, to the point that the game is now playable. Not sure if it's complete, but it's playable. See e.g. https://www.oldschoolgamermagazine.com/work-begins-on-restor... for info.
I have a lot of respect for little communities where people gather to work obsessively over a goal like this. It's fun to see projects like this that would never see this much effort applied if not for the passion of a few dedicated people.
Many of the smartest people I've known got their starts in little communities like these: Obsessively learning new things that would normally be beyond their academic or career level because they need to know something advanced to move their passion project forward.
Its what started me on the path of my career really. Understanding how systems work. I still enjoy finding cheats in offline games for myself just to see how far I can push game engines.
Where can I boot up Coke Studios? Enjoyed that game as a kid, including making music (which was so friggen cool! They had tons of samples you could just drag and drop into a timeline).
Edit: fun memory. I figured out how scamming works at some point. IIRC (and I may not) I set up a minigame in my studio, and contestants had to give me their furniture to participate in a game. Since furniture could stack weirdly, people made all sorts of crazy mazes that were actually quite difficult to navigate due to the fixed isometric perspective.
Each stage required handing over a more expensive item than the last. At some point someone handed over a super rare/expensive piece of furniture.
Not gonna erase that. ACTUALLY I think I figured out how to clone items due to a bug. Or mine infinite DBs. Used that as cover to get someone to give me their item. Was a long time ago.
Anyways, a user handed me their super rare item and I then gave them a stern warning on how this is a common scam and I just stole their item(s). Then I handed everything back over to them and they profusely thanked me. lol
Hey, that's my newsletter -- thanks for sharing! :D
Some exciting stuff has already happened in the week since I wrote the interview. Here's Viva Pinata: Trouble in Paradise: https://www.youtube.com/watch?v=Kih42KlocII
Was playing that last night. So many great recomps lately. Ty for the newsletter
Very nice. It’d be interesting if recomps of some games that were originally for both 360 and PC become the superior experience… Fable 3 and the whole Games for Windows Live nonsense the PC version had come to mind.
These projects are really neat. Last week, I was able to build (and play) the Xbox 360 release of Sonic Unleashed on a couple of ARM SBCs using Sonic Unleashed Recomp.
Years ago I was searching for why emulators don't recompile to native bytecode and it seemed like the consensus at the time was that JIT etc was faster. I can't find it now, but IIRC I remember replies from maintainers saying it was a bad idea.
What changed since then, since this seems to be a trend? Is this more for more modern systems that have more static code?
Static code. Also, very fine details of the machine because less important.
When emulating a NES (for example), you really have to emulate every register, how registers change, and also weird effects like instructions that take longer to read or write values, as games rely on that stuff. Once you have modern systems where much of the code was originally C, it becomes less important to ensure every register has exactly the right values when a subroutine finishes in most cases, you can rely that (most) of the code follows standard calling conventions.
It's a combination of things. The floodgates were opened when the N64 recompiler popped up a couple (?) of years back. Really got interest going.
I also think tooling got better, and with the advent of AI engineering it made all of this a lot more accessible.
It depends on the platform you're targeting.
Something like a SNES will always have the same hardware no matter what (okay, leave out the expansion chips inside cartridges, or fold them into "same hardware").
Something like a PC will have any old random shite you care to plug into it.
Something like an Xbox 360 is kind of halfway between.
With a SNES you can rely on the fact that a vertical blanking interrupt will happen 50 times a second no matter what, and you get so many instructions before your video output is halfway down the screen no matter what, and then if you want to play silly games like changing a PPU register to dick about with the screen mode you can. You have to take into account things like it taking more time for certain types of memory access like if you cross a 256-byte page boundary, but since when you get down this far you're doing it all in assembler you can place your code and data exactly where you want to get exactly the behaviour you want.
When you emulate this, you must emulate every aspect of this behaviour correctly or things will fall apart. Think about things like the "open bus bugs" that trip up emulation - some games rely on silly tricks like expecting the capacitance on the pins of the chips to hold a value like a crazy single-byte DRAM for a moment while they look somewhere else. If your emulator just goes "okay no devices are enabled so the bus is simply pulled high to 0xff" that will fail.
With a PC you've got a wider variety of hardware, and a deeper level of abstraction. You don't have to care about flipping registers at just the right time, you just throw some textures and polygons at the GPU and let it do its thing. It'll tell you when it's ready for more. This is why you get the same experience between your AMD GPU, your NVidia GPU, your Intel GPU, and hell why not just have software OpenGL too (because it's slow but that's how we used to do it in the olden days).
So if you wanted to run a SNES game on modern hardware you'd be better off emulating the hardware directly, bugs and all, and then running the "real" game binary.
But if you wanted to run something like an Xbox 360 game - written using "modern" development techniques, where a lot of the hardware is abstracted away behind drivers and libraries - you might well just break it down into functional blocks of code, rewrite them for your native CPU, and shim the calls to the libraries to talk to the ones you have on your native system.
A bit of code that rotates a set of vertices and flings them at the GPU will look pretty much the same in C on any platform, but it'll compile to very different assembly language. That's okay though! If you understand how the instructions relate to the data you can just kind of translate one set of opcodes to another and expect it to be more-or-less right. You can even (kind of) do this automatically!
What's the latest on homebrewing on the actual Xbox 360 these days?
You can play game backups, use unofficial HDDs at bigger sizes, send over new games with FTP, use third party and DIY controllers like GuliKit's offerings or a Flatbox leverless arcade controller. You can play a modded version of Rock Band 3 with all the songs from all the games, using a Wii Guitar Hero controller with a Pi Pico-based adapter in place of the Wiimote. Any game you installed from disc for faster load times becomes playable without the disc in automatically once you get Aurora running (this is pretty cool, on some other modded consoles like PS4 you'd have to reinstall the games another way).
As far as homebrew in the sense of people making their own games and applications, I don't know that there's much going on. Maybe you can find a controller tester app. Sadly the 360 scene has a lot of proprietary stuff going on, and uses some official dev SDKs, IIRC. Where 3DS and Switch modders have everything on GitHub and freely licensed, 360 has a lot of binary-only releases on random forums and such. It would probably be a major effort to change this and involve rewriting stuff people already know and love.
I did the the RGH 3 mod to half a dozen or so 360s a few years back for myself and friends. If you just wanna play games with modern comforts, things are in a pretty good state. We don't need to worry about the original disc drives, HDDs, or controllers wearing out and making the console useless, we can replace or avoid them all. The wireless card is also easily replaceable, a friend of mine had one that seemed non-functional after he spilled some Mtn Dew all over his 360 back in the day. Everything else seemed good after a clean but I just got him a new wireless card off eBay and swapped that out.
If anyone wants to get into this and doesn't already have a 360, the S models are easiest to mod and most reliable (no RRoD). Some are Trinity and some are Corona motherboards inside, but both are hackable. Corona may need a post-fix adapter, but they're cheap and easy to install. Just avoid the E series.
For modding tutorials, MrMario2011 on YT is great.
There is also a software-only exploit that's a bit worse but more accessible. It came out after I already did all my modding, so I haven't got experience with it.
What is Xbox 360 recompilation?
You take an Xbox game designed to run on an Xbox 360, a 64 bit PowerPC system and decompile its binaries back into source code. You now have the ability to modify the game as well as port it to other systems and architectures such as Windows on X86_64 or Linux on ARM64.
It's more nuanced than that; the approach you're describing is usually called "decompilation."
The difference is how far one goes in hoisting the "source code;" in this "recompliation" approach the source code, while C++, is basically an IR (intermediate representation) between the original game's assembly and a host platform, and the hardware itself is emulated (for example, the original architecture's CPU registers are represented as variables in the host architecture's memory). The machine code is translated to C++ using a custom tool.
In a "decompilation" approach the game logic is converted (using a decompiler, like IDA or Ghidra's) back into something which resembles the original source code to the game itself, and the source code is usually hand analyzed, marked up, rewritten, and then ported across platforms. The product is something that attempts to resemble the original game's source code.
Of course, they lie on a continuum and both approaches can be mixed, but, while they both involve C++ in the middle, the process is starkly different. Recompilation is much more copyright-friendly, because in many implementations only the modifications are distributed and the original binary is translated by the end user (who owns the software/a license to it), whereas decompilation produces an artifact (source code) which is a derivative work encumbered by the original software's license and generally should not be distributed.
> In a "decompilation" approach the game logic is converted (using a decompiler, like IDA or Ghidra's) back into something which resembles the original source code to the game itself, and the source code is usually hand analyzed, marked up, rewritten, and then ported across platforms
There definitely is a lot of scope to apply LLMs here
Seeing this [1], I thought it was something related to taking assembly instructions in the original code, emitting C statements that match the instruction, and then compiling that C code.
[1] https://github.com/N64Recomp/N64Recomp
Your idea is much more accurate; see my sibling comment. It's basically using C or C++ as an intermediate representation for machine code, rather than trying to recreate the game's higher-order logic/structure or source code.
You take a binary that's intended to run on the Xbox 360, and emit a new binary that runs on a modern x86 computer.
Crazy that it has taken so long. I understand these are small teams trudging off in the dark but you could have imagined there would have been more eyes on this and hands tinkering in the time when the Xbox 360 was being sold than today. Right at that same time in history the iPhone was getting cracked basically every iOS release, sometimes by teenagers. Seems like there were a ton of hackers around back then.
Most people just want emulation which was solved a while ago. Recompilation is mostly for modders which is a very small community.
The whole decomp and recomp scene has really started popping off this past year in the emulation community. The N64 recomp tool was the first one to really get interest going.