In this passage, we will try to define some of the core building blocks of scalable systems: Consistent Hashing, CAP Theorem, Load Balancing, Caching, Data Partitioning, Indexes, Proxies, Queues, Replication, and choosing between SQL vs. NoSQL.
Familiarizing these concepts would greatly benefit in understanding distributed system concepts. Key characteristics of a distributed system include Scalability, Reliability, Availability, Efficiency, and Manageability.
Load Balancing
Load Balancer(LB) is a critical component of any distributed system. It helps to spread the traffic across a cluster of servers to improve responsiveness and availability of applications, websites or databases. LB also keeps track of all the resources while distributing requests. If a server is not available to take new requests of is not responding or has elevated error rate, LB will stop spending traffic to such a server.
Typically a load balancer sits between the client and the server accepting incoming network and application traffic and distributing traffic across multiple backend servers using various algorithms.
To utilize full scalability and redundancy, we can try to balance the load each layer of the system. We can add LBs at three places
- Between the user and the web server
- Between web servers and an internal platform layer, like application servers or cache servers
- Between internal platform layers and database
Benefits of Load Balancing
- User experience faster, uninterrupted service.
- Service providers experience less downtime and higher throughput
- Smart load balancers provide benefits like predictive analytics that determine bottlenecks before they happen.
- System administrators experience fewer failed or stressed components.
Load Balancing Algorithms
Load balancers consider two factors before forwarding a request to a backend server. they will first ensure that the server is responding appropriately to requests. Then they use a pre-configured algorithm to select one from those healthy servers.
There is a variety pf load balancing methods, which use different algorithms for different needs.
- Least Connection Method
- Least Response Time Method
- Least Bandwidth Method
- Round Robin Method
- Weighted Round Robin Method
- IP Hash
Redundant Load Balancers
The load balancer can be single point of failure, to overcome this, a second load balancer can be connected to the first to form a cluster. Each LB monitors the health of the other and, since both of them are equally capable of serving traffic and failure detection, in the event the main load balancer fails, the second load balancer takes over.
Further reference
Caching
Load balancing helps you scale horizontally across an ever-increasing number of servers, but caching will enable you to make better use of the resources you already have as well as making otherwise unattainable product requirements feasible……
……
Data Partitioning
Data partitioning is a technique to break a big database into many smaller parts.
