cs435 - Distributed Systems
This course introduces students to distributed and parallel systems. It discusses the design & organization of distributed systems and architectures. Topics include Parallel processing, multithreaded programming, distributed systems communication models, socket programming, RPC/RMI, and MapReduce programming model. Distributed systems core concepts such as process coordination, clocks & synchronization, dist transactions, data consistency, concurrency control, consensus, replication, fault tolerance, dist file systems, and security are also covered. Students apply knowledge and methods of parallel and distributed systems to analyze the performance of popular distributed system(s).
Class Schedule:
11.00AM to 12.00 noon Everyday except Thursday. 1B-04 Building 105.
Exams Schedule:
Midterm Exam: Monday October 28, 2024.
Final Exam: December 9, 2024 (12.30).
Important Links:
[Syllabus]
[This week]
[
Scores out of 60]
[Learning Management System]
[Textbook]
[GitHub Repo]
Course Materials
Introduction
Week 1 Slides: Introduction to Distributed Systems (6 MB)
Dist Systems startups in 2024
CTOs guidebook: 6 companies leading the Dist Systems
The World’s Biggest Cloud Computing Service Providers
Tutorial 1
Week 2 Slides: Dist Sys Design & Architecture (4.5 MB)
S. M. Thampi ,"Introduction to Distributed Systems", arXiv:0911.4395, 2009.
Introduction to Distributed System Design - Google Code University
Tutorial 2Parallel Computing
Week 3 Slides: Parallel Processing (5.6 MB)
Parallel Processing | Overview, Limits & Examples
JavaPoint ForkJoin Tutorial
Tutorial 3
Assignment 1: Posted on LMS
Week 4 Slides: Parallel Processing contd. (2.9 MB)
MapReduce: Simplified Data Processing on Large Clusters
Video: An Introduction to MapReduce frameworkCommunications in Dist Systems
Week 5 Slides: Communication (3.5 MB)
IBM what is networking?
Tutorial 4
Assignment 2: Posted on LMS
Week 6 Slides: Communication -> RPC (2.6 MB)
An intro to Java Socket programming
Tutorial 5
Week 7 Slides: Communication Models (3.5 MB)
Amazon: What is publish/subscribe messaging?
Cleo: What are webservices? a quick overview
Semester long Course Project: Posted on LMSCore concepts
Week 8 Slides: Replication and Fault Tolerance (3.4 MB)
The two generals problem
Blockchain, a solution to byzantine generals problem?
A complete history of Amazon outages
Assignment 3: Posted on LMS
Week 9 Slides: Synchronization (5.8 MB)
How NTP works?
G Manepali, Clocks and Causality - Ordering Events in Distributed Systems
Tutorial 6
Week 10 Slides: Consensus (4.6 MB)
A visual guide to understanding Raft.
Raft Github Repo
Paxos vs Raft: Have we reached consensus on distributed consensus?
Tutorial 7
Midterm Exam this week!
Week 11 Slides: Consistency and Distributed Transactions
2PC vs 3PC, a comparison
Eric Brewer, Spanner, TrueTime & The CAP Theorem
Adrian Booth: Designing Data Intensive Applications: Consistency with Linearizability
Week 12 Slides: Distributed File Systems
P Macko and J Hennessey, "Survey of Distributed File System Design Choices", ACM Transactions on Storage, Vol 18, No 1, 2022.
Pan et. al., Navigating the Landscape of Distributed File Systems: Architectures, Implementations, and Considerations
Tutorial: Setting up Hadoop VM on a single machine
Week 13 Slides: Security in Dist Systems
Securing distributed systems: A survey on access control techniques for cloud, blockchain, IoT and SDN
IBM Blockchain security
Tutorial: Setting up a Hadoop VM cluster for 4 nodesTools and applications
Assignment 4: Posted on LMS
Apache Spark and its applications: Amazon AWS,
IBM,
Google,
Microsoft.
Docker platform: docker.docs,
IBM docker.
Kubernetes: Overview,
Google K8s.
Spanner: Google’s Globally Distributed Database
Distributed Databases: HBase Amazon Hbase,
IBM Hbase.
Apache Cassandra