It's pretty interesting that Apple was so quick to get to industry-leading status on their CPUs, but their GPUs are still in a state where they won't match Nvidia's GPUs from 2024 until at least next year, and will need a significantly bigger chip to do so.
Similar story with Qualcomm but to a lesser degree (both on the CPU and GPU side). I wonder why that is.
Lack of priority? Legitimately harder problem to solve? Experience from mobile scaling to desktop differently than CPU experience?
Apple (and the ARM ecosystem as a whole) has never really needed massive GPU compute before, it’s always been about power efficiency and just enough GPU oomph to make UI fluid. Even historic Mac Pro workloads never really needed tons of GPU prowess, the heaviest power users were primarily taxing video encode/decode and 2D raster effects (Adobe Suite etc) so that’s what they focused on. By contrast NVIDIA’s entire game has been raw GPU power for decades. M1 Max was really prescient in hindsight and has set the stage for, all things considered, Apple to be not as far behind as it could have been today.
I guess I just don't really buy your argument, because if their CPUs had turned out poorly, you could have applied the same argument to their CPUs instead of the GPUs.
Another user though pointed out that they didn't actually design the GPU until the A11, so perhaps it really is just a lack of in house experience.
Apple vs. Oranges: Evaluating the Apple Silicon M-Series SoCs for HPC Performance and Efficiency
"Apple's M-series GPUs offer massive performance-per-watt, scaling efficiently from roughly 5W in base chips to around 40-50W in top-tier Max chips. This efficiency generally sits between 200 and 250 GFLOPS per Watt."
Sure. nVidia GPUs have higher performance. But they do it with 450W+ a.k.a. 10x the power
> I guess I just don't really buy your argument, because if their CPUs had turned out poorly, you could have applied the same argument to their CPUs instead of the GPUs.
Doesn't the argument work fine on the CPU side? Apple doesn't seem to be hurting for lack of a Threadripper competitor.
The M-series are ARM chips based on the A series from the iPhones. Therefore Apple had ten years of chip design (plus the prior decades of ARM history) to rely on. The GPUs and modems were a much more recent effort.
(My take. happy to be corrected by someone with chip design experience who can comment.)
I would argue their architecture choices long term proved to be prescient (RISC, ARM, etc) and securing TSMC production capacity (beating competitors to the punch) aided them as well.
Their main architectural choice was not about "RISC" or "ARM" but it was choosing "Brainiac" over "Speed Demon", i.e. setting the goal to execute a great number of instructions per clock cycle at a moderate clock frequency, instead of executing a moderate number of instructions per clock cycle at a high clock frequency (the latter variant results in lower fabrication costs, which is why other companies were reluctant to pursue the same choice as Apple).
A high IPC is much easier to achieve when the instructions have a fixed-length encoding, so this RISC principle followed from their main choice.
ARM was chosen for the Newton, then the iPods, then the iPhone and so on. Apple’s experience with ARM has a long history. Same applies to RISC. Apple didn’t leave PPC because it was a dead-end. They went with Intel because IBM was not interested in developing a low power PPC just for Apple. IBM wasn’t (and still isn’t) even interested in developing a POWER chip for desktop workstations. If there are POWER machines that can be turned into workstations, it’s a side effect of them being targeted at entry-level half-rack systems.
The iPhones had GPUs too. As far as I'm aware, the GPU design in the M-series chips are directly descended from the iPhone GPU in the exact same way the CPU is.
Do you have comparisons for Apple’s GPU to NVIDIA’s at the same power and thermals?
Right now the M5 Max is one of the best laptop compute GPUs for blender rendering and doesn’t do too badly on raster tests either. But it operates at a fraction of the equivalent NVIDIA parts.
It’s more that Apple doesn’t provide a competing tier of device as the high end NVIDIA GPUs. But within the tiers they do provide they’re fairly competitive.
I don't buy this as an explanation for the gap in the actual silicon's performance.
Nvidia's chips lose very little performance when they're severely power limited. In fact, that's why Nvidia's Professional versions of their GPUs are typically running at under two-thirds of their consumer equivalent's power draw.
Nvidia's cards are primarily designed for their professional applications where the power consumption is lower, and then they just juice up the consumer cards deep into the territory of diminishing returns just so they win some benchmarks.
On a performance-per-watt basis, Apple is still behind an Nvidia card with the juiced up power consumption, and when you dial back the power draw, Nvidia cards are miles ahead of Apple's silicon for performance-per-watt.
Another important thing to keep in mind here is the software. Nvidia has been optimizing their software stack for decades to squeeze every clock cycle out of the hardware under any limits. TensorRT alone does straight up magic. Apple is just starting out with MLX, and their hardware is often idling not because it is worse per watt but because the compiler does not know how to optimally load the ALU blocks for specific graphs yet
> On a performance-per-watt basis, Apple is still behind an Nvidia card with the juiced up power consumption, and when you dial back the power draw, Nvidia cards are miles ahead of Apple's silicon for performance-per-watt.
Do you have a citation on Apple vs nVidia performance-per-watt? I'm not aware of any benchmark that shows nVidia with a better performance-per-watt. Higher performance, sure, but performance-per-watt, not that I'm aware of.
A GPU can be used for inference, but, for that use, there are much better choices. Apple designed their NPUs for that, IBM added an NPU to their mainframe chip and AMD and Intel are planning on adding inference-specific instructions to the amd64 ISA.
The CPU/NPU/GPU part is probably coming, but the 1.5TB of RAM might be delayed a bit. More interesting footnotes are the server chips: is the MacPro making a comeback?
Maybe they are just for internal use? I doubt Apple is going back on the server market, and given how little they have invested in the Mac Pro, I doubt they are going to make a new version. There is little advantage compared to the Mac Studio for most users. External GPU support would change that, but it doesn't seem to be what they want.
Expandable machines are attractive when the future is uncertain. I imagine a server chip would have abundant DDR6 and PCIe lanes. Oddly enough I’m now buying my PCs with all memory slots occupied because of bandwidth. I can still change the sticks for higher capacity, but expansion is a lot less convenient than when memory bandwidth wasn’t something more desirable than raw CPU performance.
I’m not sure if I can measure IPC and cache misse latencies on my machines, but I’m sure latencies are pushing IPC down.
Really just depends on what the market looks like in a year or two. If things stay on the current trajectory, then yeah, they won't sell this. If the AI market gets spooked and sells off, it might be available.
Whether or not this thing is commercially available though, they basically have no choice but to at least keep up the R&D spend, or they'll be even further behind once RAM availability eventually eases up.
sure, i'll believe it when i see it. all i see from the article is a marketing campaign for folks to buy their stock. we don't even have the m5 we wanted, and we are talking about m7 and m8s?
I mean, yeah sure, we should wait and see what actually comes out, but I'm not sure I really see much reason to be skeptical.
Blackwell came out in 2024 on n4p. This article is claiming that Apple hopes to get into the same ballpark of performance as a Blackwell GPU with an M7 Ultra, which at the absolute earliest would release in 2027, but more likely 2028 or 2029, and would consist of two absolutely massive M7 Max GPUs stapled together, for a total die area bigger than a 5090, but on a smaller more advanced node.
Honestly, if they can't release something that big by 2028 that's at least competitive with a 5080, it'd be extremely embarrassing for them.
I'm hoping they are right. I was saving for M5 Ultra Pro 512gb or 768gb and feel so screwed. Could have gotten an M3 512gb or a genoa DDR5 system with blackwell pro 6000 and the prices have climbed. Apple doesn't have control of the supply chain, for the first time they are on the same level with other vendors, so I'm skeptical they can do what they hope to ... I'm still going to keep saving for this M7 ...
I don't think they care much about GPU's. They never did in the past, thats why you dont get nVidia in a Mac. Apple was happy with Intel graphics until they went M series chips. They just never cared about gaming at all.
Do you really think tech gossip on capability pointing to advancements >2 years out is driving stocks? I'm sure analysts will take note, but this is just noise in a sea of sound.
It's pretty interesting that Apple was so quick to get to industry-leading status on their CPUs, but their GPUs are still in a state where they won't match Nvidia's GPUs from 2024 until at least next year, and will need a significantly bigger chip to do so.
Similar story with Qualcomm but to a lesser degree (both on the CPU and GPU side). I wonder why that is.
Lack of priority? Legitimately harder problem to solve? Experience from mobile scaling to desktop differently than CPU experience?
Apple (and the ARM ecosystem as a whole) has never really needed massive GPU compute before, it’s always been about power efficiency and just enough GPU oomph to make UI fluid. Even historic Mac Pro workloads never really needed tons of GPU prowess, the heaviest power users were primarily taxing video encode/decode and 2D raster effects (Adobe Suite etc) so that’s what they focused on. By contrast NVIDIA’s entire game has been raw GPU power for decades. M1 Max was really prescient in hindsight and has set the stage for, all things considered, Apple to be not as far behind as it could have been today.
I guess I just don't really buy your argument, because if their CPUs had turned out poorly, you could have applied the same argument to their CPUs instead of the GPUs.
Another user though pointed out that they didn't actually design the GPU until the A11, so perhaps it really is just a lack of in house experience.
No. It's always been about power efficiency. Apple's GPU are very good in that area just like the CPU.
https://arxiv.org/html/2502.05317v1
Apple vs. Oranges: Evaluating the Apple Silicon M-Series SoCs for HPC Performance and Efficiency
"Apple's M-series GPUs offer massive performance-per-watt, scaling efficiently from roughly 5W in base chips to around 40-50W in top-tier Max chips. This efficiency generally sits between 200 and 250 GFLOPS per Watt."
Sure. nVidia GPUs have higher performance. But they do it with 450W+ a.k.a. 10x the power
> I guess I just don't really buy your argument, because if their CPUs had turned out poorly, you could have applied the same argument to their CPUs instead of the GPUs.
Doesn't the argument work fine on the CPU side? Apple doesn't seem to be hurting for lack of a Threadripper competitor.
The M-series are ARM chips based on the A series from the iPhones. Therefore Apple had ten years of chip design (plus the prior decades of ARM history) to rely on. The GPUs and modems were a much more recent effort.
(My take. happy to be corrected by someone with chip design experience who can comment.)
I would argue their architecture choices long term proved to be prescient (RISC, ARM, etc) and securing TSMC production capacity (beating competitors to the punch) aided them as well.
Their main architectural choice was not about "RISC" or "ARM" but it was choosing "Brainiac" over "Speed Demon", i.e. setting the goal to execute a great number of instructions per clock cycle at a moderate clock frequency, instead of executing a moderate number of instructions per clock cycle at a high clock frequency (the latter variant results in lower fabrication costs, which is why other companies were reluctant to pursue the same choice as Apple).
A high IPC is much easier to achieve when the instructions have a fixed-length encoding, so this RISC principle followed from their main choice.
ARM was chosen for the Newton, then the iPods, then the iPhone and so on. Apple’s experience with ARM has a long history. Same applies to RISC. Apple didn’t leave PPC because it was a dead-end. They went with Intel because IBM was not interested in developing a low power PPC just for Apple. IBM wasn’t (and still isn’t) even interested in developing a POWER chip for desktop workstations. If there are POWER machines that can be turned into workstations, it’s a side effect of them being targeted at entry-level half-rack systems.
The iPhones had GPUs too. As far as I'm aware, the GPU design in the M-series chips are directly descended from the iPhone GPU in the exact same way the CPU is.
Prior to the A11 chip they used PowerVR designed GPUs. They weren't done in house.
Based on Alyssa Rosenzweig’s reverse engineering, the current GPUs are still descended from PowerVR IP
Ah I see, good to know.
Do you have comparisons for Apple’s GPU to NVIDIA’s at the same power and thermals?
Right now the M5 Max is one of the best laptop compute GPUs for blender rendering and doesn’t do too badly on raster tests either. But it operates at a fraction of the equivalent NVIDIA parts.
It’s more that Apple doesn’t provide a competing tier of device as the high end NVIDIA GPUs. But within the tiers they do provide they’re fairly competitive.
They want to be power efficient, while Nvidia doesn't. That means a much higher cost (larger chip area) for the same performance.
I don't buy this as an explanation for the gap in the actual silicon's performance.
Nvidia's chips lose very little performance when they're severely power limited. In fact, that's why Nvidia's Professional versions of their GPUs are typically running at under two-thirds of their consumer equivalent's power draw.
Nvidia's cards are primarily designed for their professional applications where the power consumption is lower, and then they just juice up the consumer cards deep into the territory of diminishing returns just so they win some benchmarks.
On a performance-per-watt basis, Apple is still behind an Nvidia card with the juiced up power consumption, and when you dial back the power draw, Nvidia cards are miles ahead of Apple's silicon for performance-per-watt.
Another important thing to keep in mind here is the software. Nvidia has been optimizing their software stack for decades to squeeze every clock cycle out of the hardware under any limits. TensorRT alone does straight up magic. Apple is just starting out with MLX, and their hardware is often idling not because it is worse per watt but because the compiler does not know how to optimally load the ALU blocks for specific graphs yet
> On a performance-per-watt basis, Apple is still behind an Nvidia card with the juiced up power consumption, and when you dial back the power draw, Nvidia cards are miles ahead of Apple's silicon for performance-per-watt.
Do you have a citation on Apple vs nVidia performance-per-watt? I'm not aware of any benchmark that shows nVidia with a better performance-per-watt. Higher performance, sure, but performance-per-watt, not that I'm aware of.
A GPU can be used for inference, but, for that use, there are much better choices. Apple designed their NPUs for that, IBM added an NPU to their mainframe chip and AMD and Intel are planning on adding inference-specific instructions to the amd64 ISA.
"Blackwell-class AI" sounds like one of those cool sounding nonsense terms out of a military sci-fi book.
Yeah, that is pretty funny considering Blackwell isn't just a chip but a whole ecosystem with a fifth-gen interconnect and huge cooling racks...
The CPU/NPU/GPU part is probably coming, but the 1.5TB of RAM might be delayed a bit. More interesting footnotes are the server chips: is the MacPro making a comeback?
Maybe they are just for internal use? I doubt Apple is going back on the server market, and given how little they have invested in the Mac Pro, I doubt they are going to make a new version. There is little advantage compared to the Mac Studio for most users. External GPU support would change that, but it doesn't seem to be what they want.
Expandable machines are attractive when the future is uncertain. I imagine a server chip would have abundant DDR6 and PCIe lanes. Oddly enough I’m now buying my PCs with all memory slots occupied because of bandwidth. I can still change the sticks for higher capacity, but expansion is a lot less convenient than when memory bandwidth wasn’t something more desirable than raw CPU performance.
I’m not sure if I can measure IPC and cache misse latencies on my machines, but I’m sure latencies are pushing IPC down.
Really just depends on what the market looks like in a year or two. If things stay on the current trajectory, then yeah, they won't sell this. If the AI market gets spooked and sells off, it might be available.
Whether or not this thing is commercially available though, they basically have no choice but to at least keep up the R&D spend, or they'll be even further behind once RAM availability eventually eases up.
Original source (8 points) https://news.ycombinator.com/item?id=48882968
sure, i'll believe it when i see it. all i see from the article is a marketing campaign for folks to buy their stock. we don't even have the m5 we wanted, and we are talking about m7 and m8s?
I mean, yeah sure, we should wait and see what actually comes out, but I'm not sure I really see much reason to be skeptical.
Blackwell came out in 2024 on n4p. This article is claiming that Apple hopes to get into the same ballpark of performance as a Blackwell GPU with an M7 Ultra, which at the absolute earliest would release in 2027, but more likely 2028 or 2029, and would consist of two absolutely massive M7 Max GPUs stapled together, for a total die area bigger than a 5090, but on a smaller more advanced node.
Honestly, if they can't release something that big by 2028 that's at least competitive with a 5080, it'd be extremely embarrassing for them.
I'm hoping they are right. I was saving for M5 Ultra Pro 512gb or 768gb and feel so screwed. Could have gotten an M3 512gb or a genoa DDR5 system with blackwell pro 6000 and the prices have climbed. Apple doesn't have control of the supply chain, for the first time they are on the same level with other vendors, so I'm skeptical they can do what they hope to ... I'm still going to keep saving for this M7 ...
I don't think they care much about GPU's. They never did in the past, thats why you dont get nVidia in a Mac. Apple was happy with Intel graphics until they went M series chips. They just never cared about gaming at all.
For AI purposes that may change but I doubt it.
Do you really think tech gossip on capability pointing to advancements >2 years out is driving stocks? I'm sure analysts will take note, but this is just noise in a sea of sound.
[dead]
[dead]