0 reactions
Apple Unified Memory
Apple Unified Memory literally destroys Intel and AMD. That's what you hear in many random YouTube videos about Apple Silicon. Let's find out what's wrong with that.
**TLDR:** Apple Unified Memory has its pros and cons. It's not a silver bullet by any means.
----------------------------------
The on-chip memory (Unified Memory) was introduced by Apple in M1 a few years ago. It's not something new to the industry; many mobile devices and gaming consoles have been using this approach for decades.
Apple uses the regular low-power LPDDR memory, which is soldered next to the chip. It allows Apple to run the memory at slightly higher clock speeds compared to the regular DDR memory and save some space on the motherboard.
In the case of regular M-chips, we have slightly higher bandwidth than regular 128-bit (dual or quad channel) DDR memory and lower power consumption. So, the regular M3 chip with LPDDR5-6400 memory has nearly 100GB/s bandwidth (128-bit bus), M3 Pro - 200GB/s (256-bit bus), M3 Max - 300 or 400 GB/s (384- or 512-bit bus).
To put those numbers into perspective, regular Intel or AMD laptops have nearly the same bandwidth as an M3 or M4 chip (90 to 120 GB/s, depending on the memory).
If you compare those numbers, you'll find that M2 Max or M3 Max with Unified Memory with 400GB/s bandwidth should literally annihilate Intel and AMD laptops because it's so much faster. Still, we don't see that difference in the actual tests. What's wrong with that?
There's one small secret nobody tells about. The CPU and GPU have different memory requirements.
In the case of CPUs, the main requirement is latency (the time between the data request and when it's received). The bandwidth should be fast enough, but it is not that important. The amount of data processed by the CPU is not that huge.
Latency is essential because when a CPU waits for data from the system memory (RAM), it has to pause the execution of the current branch. The wait time is measured in hundreds of cycles. These mispredictions can destroy performance.
Actually, that's why the bandwidth available to one CPU core in Apple Silicon is limited to \~120GB/s and \~240GB/s for all the CPU cores, and nobody has noticed it.
On the other hand, GPU does not care about latency but requires a lot of bandwidth. The GPU was built to process a lot of data simultaneously. It needs to load that data from memory.
So all those vast numbers posted by Apple (200GB/s, 400GB/s, 800GB/s, etc.) are required mainly for the GPU. Apple might go with the faster GDDR6 memory, but it has a way bigger latency (\~250ns vs. \~100ns for LPDDR5). That's fine for the GPU, but it might kill CPU performance.
So why Apple is using the unified memory? Why did they decide to go this route?
The main reason was to integrate both the CPU and GPU on one chip to get better performance and efficiency.
The classic approach with separate DIMM slots was not an option at all because those slots take space. It's okay for the base M-chips with a 128-bit bus requiring two slots, but when we have the Max variant with 8 slots, it's not an option at all.
Also, that explains why there's no separate GPU with its memory as we have in modern laptops. It takes much more space, especially when you plan to use it on a MacBook Air or iPad.
Unfortunately, the Apple approach has a few cons. The first and the most obvious one is the inability to upgrade the memory. That's not an issue for Apple.
The second drawback is the limited number of memory you can have in the system. You are limited to a space around the die.
The third drawback is the die size limit. You can't have a GPU comparable to nVidia GeForce 4090 because the chip will be too big, even with the UltraFusion or other multi-die solutions. Obviously, you can't have more than one GPU in the system.
At the same time, it has one noticeable benefit: a GPU can use most of the system memory. It may be helpful for some ML tasks when you need to have a lot of memory to fit the model.
On the other side, Apple Silicon's performance is insufficient to consider it a replacement for nVidia cards for ML tasks. Another problem is the software support, community, etc.
Intel is planning to release the Lunar Lake product later this year, having memory on the chip for the same reasons as Apple, but most of the products will use the same discrete approach as we have now.
AMD is planning to release the Strix Halo product with a pretty decent GPU with 256-bit on-board LPDDR5 memory and no socket options for the same reasons.
To recap, Apple Unified Memory is a good technology, but it's not a silver bullet by any means.
It offers comparable performance to systems with an integrated or discrete mid-range GPU.
So if you hear somebody say that it's so much better than everything on the market, just close that video and don't waste your time 😉
#technology #hardware
Apple Unified Memory literally destroys Intel and AMD. That's what you hear in many random YouTube videos about Apple Silicon. Let's find out what's wrong with that.
**TLDR:** Apple Unified Memory has its pros and cons. It's not a silver bullet by any means.
----------------------------------
The on-chip memory (Unified Memory) was introduced by Apple in M1 a few years ago. It's not something new to the industry; many mobile devices and gaming consoles have been using this approach for decades.
Apple uses the regular low-power LPDDR memory, which is soldered next to the chip. It allows Apple to run the memory at slightly higher clock speeds compared to the regular DDR memory and save some space on the motherboard.
In the case of regular M-chips, we have slightly higher bandwidth than regular 128-bit (dual or quad channel) DDR memory and lower power consumption. So, the regular M3 chip with LPDDR5-6400 memory has nearly 100GB/s bandwidth (128-bit bus), M3 Pro - 200GB/s (256-bit bus), M3 Max - 300 or 400 GB/s (384- or 512-bit bus).
To put those numbers into perspective, regular Intel or AMD laptops have nearly the same bandwidth as an M3 or M4 chip (90 to 120 GB/s, depending on the memory).
If you compare those numbers, you'll find that M2 Max or M3 Max with Unified Memory with 400GB/s bandwidth should literally annihilate Intel and AMD laptops because it's so much faster. Still, we don't see that difference in the actual tests. What's wrong with that?
There's one small secret nobody tells about. The CPU and GPU have different memory requirements.
In the case of CPUs, the main requirement is latency (the time between the data request and when it's received). The bandwidth should be fast enough, but it is not that important. The amount of data processed by the CPU is not that huge.
Latency is essential because when a CPU waits for data from the system memory (RAM), it has to pause the execution of the current branch. The wait time is measured in hundreds of cycles. These mispredictions can destroy performance.
Actually, that's why the bandwidth available to one CPU core in Apple Silicon is limited to \~120GB/s and \~240GB/s for all the CPU cores, and nobody has noticed it.
On the other hand, GPU does not care about latency but requires a lot of bandwidth. The GPU was built to process a lot of data simultaneously. It needs to load that data from memory.
So all those vast numbers posted by Apple (200GB/s, 400GB/s, 800GB/s, etc.) are required mainly for the GPU. Apple might go with the faster GDDR6 memory, but it has a way bigger latency (\~250ns vs. \~100ns for LPDDR5). That's fine for the GPU, but it might kill CPU performance.
So why Apple is using the unified memory? Why did they decide to go this route?
The main reason was to integrate both the CPU and GPU on one chip to get better performance and efficiency.
The classic approach with separate DIMM slots was not an option at all because those slots take space. It's okay for the base M-chips with a 128-bit bus requiring two slots, but when we have the Max variant with 8 slots, it's not an option at all.
Also, that explains why there's no separate GPU with its memory as we have in modern laptops. It takes much more space, especially when you plan to use it on a MacBook Air or iPad.
Unfortunately, the Apple approach has a few cons. The first and the most obvious one is the inability to upgrade the memory. That's not an issue for Apple.
The second drawback is the limited number of memory you can have in the system. You are limited to a space around the die.
The third drawback is the die size limit. You can't have a GPU comparable to nVidia GeForce 4090 because the chip will be too big, even with the UltraFusion or other multi-die solutions. Obviously, you can't have more than one GPU in the system.
At the same time, it has one noticeable benefit: a GPU can use most of the system memory. It may be helpful for some ML tasks when you need to have a lot of memory to fit the model.
On the other side, Apple Silicon's performance is insufficient to consider it a replacement for nVidia cards for ML tasks. Another problem is the software support, community, etc.
Intel is planning to release the Lunar Lake product later this year, having memory on the chip for the same reasons as Apple, but most of the products will use the same discrete approach as we have now.
AMD is planning to release the Strix Halo product with a pretty decent GPU with 256-bit on-board LPDDR5 memory and no socket options for the same reasons.
To recap, Apple Unified Memory is a good technology, but it's not a silver bullet by any means.
It offers comparable performance to systems with an integrated or discrete mid-range GPU.
So if you hear somebody say that it's so much better than everything on the market, just close that video and don't waste your time 😉
#technology #hardware
| 14,000 mlx | |
| 14 | |
| 0 |