Fundamentals of System Design Chapter 11: Distributed Systems

Introduction

In the last couple of years, there has been a lot of advancement in how systems are developed. Applications that we use everyday like Google Search, YouTube, Twitter, Instagram etc process billion of requests everyday. Chances are, when they were starting out, their architecture was centralized. All the components were probably running on a single server. As the business and the user base grew, the limitations of this architecture became more pronounced. They faced the challenge of scaling up to meet the ever-increasing demands.The applications experienced scalability issues, high latency, increased downtime and inefficiency in handling large amount of data and user requests. Distributed systems emerges as a robust solution to these pressing issues.

What is a distributed systems?

A distributed system is a collection of independent computers(nodes) that communicate and work together as single cohesive unit to achieve a common goal. The nodes work independently, handling it’s own processing, memory and storage. They coordinate with each other over a network to perform tasks that no single machine could efficiently handle on it’s own. The nodes are connected using a distributed operating software. All this complexity is hidden to the end user,it appears as a single and unified entity.

Types of Distributed Systems

The nodes in a distributed systems can be arranged in the following forms:

1. Client-Server Systems - In this types, roles are divided between clients and servers. Clients make requests and servers respond with resources. This is mainly used in web applications, email systems and databases.
2. Peer-to-Peer (P2P) Systems - here, each node in a network acts as both a client and a server, sharing resources directly with each other without a centralized authority. The nodes are equal participants in data sharing. P2P systems are commonly used in file sharing, blockchain and real time communication applications.
3. N-Tier architecture - This is a multi-layered setup where application processing is separated into layers/tiers each running on a different physical server. In a typical n-tier application, there might be a presentation tier to handle user interaction, an application logic tier(processing data), and the database tier. This architecture allows for more efficient distribution of tasks and scaling each tier independently depending on computational needs.

Key Characteristic of distributed systems

1. Resource Sharing - Distributed systems allows multiple computers to share resources such as processing power, data storage etc enabling them to handle larger and more complex tasks.
2. Scalability - Nodes can be added or removed from the system depending on growing demands and fluctuating workloads.
3. Fault Tolerance - In cases where some nodes fail, the system continues to operate since tasks are distributed across multiple nodes.
4. Concurrency - Distributed systems can process multiple tasks simultaneously.
5. Transparency - From the user’s perspective, a distributed system should appear as a single, unified entity. The complexity of communication between nodes is abstracted away allowing a seamless user experience.

Challenges in Distributed Systems

1. Latency - Latency is a measure of delay. Communication over a network introduces delays which impact system performance.
2. Security - Distributed systems are more vulnerable to security threats than centralized systems. Protecting data and communication between nodes from unathorized access is critical.
3. Coordination and Synchronization - Distributed systems require careful coordination between nodes to prevent conflicts and maintain order
4. Data Consistency - Maintaining data consistency across multiple nodes in a distributed system can be challenging.

What’s the difference between micro services and distributed systems?

1. Distributed Systems

   • This is a network of independent computers that works together as a single coherent system.
   • Designed to divide computation or services across multiple nodes. The components located on networked computers communicate and coordinate by passing messages.
   • Distributed systems encompasses various architectures such as client-server, message passing, peer-to-peer and may use diverse protocols and technologies.

2. Microservices

   • A specific architectural style that structures an application as a collection of small, loosely coupled, independently deployable services.
   • Designed to focus on modular decomposition of an application mostly within a single business domain. The services are typically deployed on a distributed system but can also run on a single server.
   • Communicate with each other over a network, typically using RESTful APIs, gRPC or message queues.

Examples and use cases of distributed systems:

1. The Internet

This is the largest and the most complex distributed system. Servers, router and devices worldwide interact to deliver information to billions of users.

2. Artificial intelligence and Machine learning

These are the most exciting and rapidly developing technologies today. AI and ML algorithms require enormous data to trains their models. Working with such vast amount of data and performing complex computations is not feasible using traditional computing models. Therefore, distributed computing comes in handy for faster computation e.g training neural networks.

3. Scientific research and High Performance Computing(HPC) These fields require enormous computational resources to solve complex scientific problems. An example is the human genome project which was used to analyze large-scale DNA sequences. This project involved processing and analyzing a large amount of genetic data which was distributed across multiple nodes for faster computation.

4. Financial Services Financial services deal with vast amount of data. A notable example is fund managers analyzing vast amount of market data to make more informed decisions. Additionally financial services use distributed computing in fraud detection. By data and computational distribution across different nodes, they can analyze transaction patterns in real time and identify suspicious activity to prevent fraud effectively.

5. IOT Internet of things generates large amounts of data which needs to be processed and analyzed in real time.

6. Blockchain and Cryptocurrencies.

In blockchain, data is stored across a network of computers which all maintain a copy of the entire blockchain. This ensures the data is secure and resistant to tampering. cryptocurrencies like Bitcoin leverages on the blockchain technology. Here distributed computing is used to process transactions and maintain the blockchain. It involves solving complex mathematical problems which are distributed.

Conclusion

Martin Kleppmann, a prominent researcher in distributed systems, once said:

If you can avoid building a distributed system, avoid it… Distributed systems are notoriously difficult to reason about.

Large scale applications like Google Search are not distributed for fun, the team had no choice but to scale. Only use distributed systems when you need to handle a high volumes of data or users, when high availability is crucial or when you need to perform large scale data processing. However due to their complexity overhead, avoid them when a centralized system can meet performance needs, when availability is not crucial or when user and geographical demand is limited.

Thank You!

I’d love to keep in touch! Feel free to follow me on Twitter at @codewithfed. If you enjoyed this article please consider sponsoring this blog for more. Your feedback is welcome anytime. Thanks again!