nvidia TMEM bandwidth + tensor core pflops=million token…
0 reactions
nvidia TMEM bandwidth + tensor core pflops=million token mobilenet?
Can u use let’s just say Vera Rubin as your machine, then utilize NVIDIA TMEM tensor memory’s 150TB/s bandwidth backed with Tensor Cores 25pflops int8 calculating power. To try to run a 3MB size mobilenet by 8bit? Total TMEM size is like 30MB, and model exactly fit TMEM after context.
If anything wrong, there’s still a 80TB/s bandwidth L2 cache/SRAM ready to be utilized for further enhancements
#technology
Can u use let’s just say Vera Rubin as your machine, then utilize NVIDIA TMEM tensor memory’s 150TB/s bandwidth backed with Tensor Cores 25pflops int8 calculating power. To try to run a 3MB size mobilenet by 8bit? Total TMEM size is like 30MB, and model exactly fit TMEM after context.
If anything wrong, there’s still a 80TB/s bandwidth L2 cache/SRAM ready to be utilized for further enhancements
#technology
| 0 mlx | |
| 3 | |
| 0 |