2-Dimensional SIMD, SIMT and 2-Dimensionally Cached Memory…
0 reactions
2-Dimensional SIMD, SIMT and 2-Dimensionally Cached Memory
Since matrix multiplications and image processing algorithms are important, why don't CPU & GPU designers fetch data in 2D-blocks rather than "line"s? If memory was physically laid out in 2D form, you could access elements of a column as efficient as elements of a row. Or better, get a square region at once using less memory fetches rather than repeating many fetches for all rows of tile.
After 2D region is fetched, a 2D-SIMD operation could work more efficiently than 1D-SIMD (such as AVX512) because now it can calculate both dimensions in 1 instruction rather than 2 (i.e. Gaussian Blur).
A good example: shear-sort requires accessing column data then sorts and accesses row then repeats from column step again until array is sorted. This runs faster than radix-sort during row phase. But column phase is slower because of the leap between rows and how cache-line works. What if cache-line was actually a cache-tile? Could it work faster? I guess so. But I want to hear your ideas about this.
* Matrix multiplication
* Image processing
* Sorting (just shear-sort for small arrays like 1024 to 1M elements at most)
* Convolution
* Physics calculations
* Compression
* 2D Histogram
* 2D reduction algorithms
* Averaging the layers of 3D data
* Ray-tracing
These could have benefited a lot imho. Especially thinking about how AI is used extensively by a lot of tech corporations.
Ideas:
* AVX 2x8 SIMD (64 elements in 8x8 format, making it a 8 times faster AVX2)
* WARP 1024 SIMT (1024 cuda threads working together, rather than 32 and in 32x32 shape) to allow 1024-element warp-shuffle and avoid shared-memory latency
* 2D set-associative cache
* 2D direct-mapped cache (this could be easy to implement I guess and still high hit-ratio for image-processing or convolution)
* 2D global memory controller
* SI2D instructions "Single-instruction 2D data" (less bandwidth required for the instruction-stream)
* SI2RD instructions "Single-instruction recursive 2D data" (1 instruction computes a full recursion depth of an algorithm such as some transformation)
What can be the down-sides of such 2D structures in a CPU or a GPU? (this is unrelated to the other post I wrote, it was about in-memory computing, this is not, just like current x86/CUDA except for 2D optimizations)
#technology
Since matrix multiplications and image processing algorithms are important, why don't CPU & GPU designers fetch data in 2D-blocks rather than "line"s? If memory was physically laid out in 2D form, you could access elements of a column as efficient as elements of a row. Or better, get a square region at once using less memory fetches rather than repeating many fetches for all rows of tile.
After 2D region is fetched, a 2D-SIMD operation could work more efficiently than 1D-SIMD (such as AVX512) because now it can calculate both dimensions in 1 instruction rather than 2 (i.e. Gaussian Blur).
A good example: shear-sort requires accessing column data then sorts and accesses row then repeats from column step again until array is sorted. This runs faster than radix-sort during row phase. But column phase is slower because of the leap between rows and how cache-line works. What if cache-line was actually a cache-tile? Could it work faster? I guess so. But I want to hear your ideas about this.
* Matrix multiplication
* Image processing
* Sorting (just shear-sort for small arrays like 1024 to 1M elements at most)
* Convolution
* Physics calculations
* Compression
* 2D Histogram
* 2D reduction algorithms
* Averaging the layers of 3D data
* Ray-tracing
These could have benefited a lot imho. Especially thinking about how AI is used extensively by a lot of tech corporations.
Ideas:
* AVX 2x8 SIMD (64 elements in 8x8 format, making it a 8 times faster AVX2)
* WARP 1024 SIMT (1024 cuda threads working together, rather than 32 and in 32x32 shape) to allow 1024-element warp-shuffle and avoid shared-memory latency
* 2D set-associative cache
* 2D direct-mapped cache (this could be easy to implement I guess and still high hit-ratio for image-processing or convolution)
* 2D global memory controller
* SI2D instructions "Single-instruction 2D data" (less bandwidth required for the instruction-stream)
* SI2RD instructions "Single-instruction recursive 2D data" (1 instruction computes a full recursion depth of an algorithm such as some transformation)
What can be the down-sides of such 2D structures in a CPU or a GPU? (this is unrelated to the other post I wrote, it was about in-memory computing, this is not, just like current x86/CUDA except for 2D optimizations)
#technology
| 3,000 mlx | |
| 3 | |
| 0 |