SE 424: Distributed Systems מערכות מבוזרות
Course, Kinneret College on the Sea of Galilee, Software Engineering, 2024
Semester 1, 5785
Course Details:
- Lecture: Sunday 1:00-4:00pm in Room 6201
- Targil: Monday 12:00-2:00pm in Room 6201
- Instructor: Michael J. May
- Email: mjmay (at) kinneret,ac,il
The full detailed syllabus of the course is available here.
Topics:
This course is a followup to previous courses on network communication and systems programming. It introduces the design and implementation of distributed systems, systems where data, computation, and resources are distributed across a network. We will study several internet based applications which illustrate good distributed systems design.
The topics for the course include some or all of the following:
- Distributed Computation
- Primitives for Distributed Computation
- Common Problems (and Solutions) for Distributed Systems
- Techniques and Algorithms for Distributed System Design
- Internet Communication Mechanisms (TCP/IP)
- Existing Internet Applications
- Client-Server Applications
- Peer-to-Peer Applications
- Coding Internet Applications Using TCP and UDP with Java
- Distributed Computing Using Java
- Coordination, Naming, and Messaging in Distributed Systems
- Edge Computing
- Blockchain fundamentals
Goals
By the end of the course, students will be able to:
- Define a distributed system and give examples of several different distributed systems paradigms.
- Design and implement application level communication protocols using TCP or UDP.
- Use an open source message queue system (RabbitMQ) for communication between computers.
- Use an open source distributed database system for distributed storage and retrieval of records.
- Use an open source framework for consensus and failure recovery (Raft) for a distributed set of servers.
- Explain cyber security issues related to network communication and coordination, distributed data storage, and distributed consensus and failure management.
Reading
The following books are used in the class:
- Maarten Van Steen and Andrew S. Tanenbaum. Distributed Systems: Principles and Paradigms. CreateSpace Independent Publishing Platform, 4th edition, 2023. Available here.
- James F. Kurose and Keith W. Ross. Computer Networking: A Top-Down Approach. Pearson, 8/E edition, 2020.
For Apache Kafka, the following two books are used:
- Neha Narkhede, Gwen Shapira, and Todd Palino. Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale. O’Reilly Media, 1st edition, Sep 2017. Available here
- Manish Kumar and Chanchal Singh. Building Data Streaming Applications with Apache Kafka: Design, develop and streamline applications using Apache Kafka, Storm, Heron and Spark. Packt Publishing, Aug 2017.
The library has copies of the books listed, but students are encouraged to purchase the books as needed.
Programming Projects
The following assignments will comprise the assignment portion of the course grade:
Assignment 1 (25%): Stage 1: Mulligan Communication. Due: 1 December 2024
Assignment 2 (25%): Stage 2: Mulligan Distributed Data Storage. Due: 1 January 2025
Assignment 3 (20%): Stage 3: Mulligan Consensus. Due: 2 February 2025
Assignments are on Moodle. More details of assignments will be given during the course of the semester.
Grading Criteria
Final grades will be calculated by combining grades from assignments and weekly quizzes. The grades are weighted as follows:
- 70% Programming Projects
- 30% Weekly quizzes (required)
Lecture Slides and Notes
| Unit | # | Date | Topic | Slides | Recitation |
|---|---|---|---|---|---|
| Intro | 1 | 3 Nov | Introduction to Distributed Systems, Client Server and Peer to Peer | [pdf] | [Sentence Server] |
| Communication and Coordination | 2 | 10 Nov | Communication: Layering, RPC, Sockets, MOM | [pdf] | [Grade Server] |
| 3 | 17 Nov | Advanced RMI, Apache Kafka | [RMI] [Bank], [Kafka] | ||
| 4 | 24 Nov | Communication 3: Multicast, Epidemic Algorithms, Part 1: C\&C Cyber security analysis | [pdf] | [Epidemic] | |
| Distributed Data Storage | 5 | 1 Dec | Naming Intro | [pdf] | |
| 6 | 8 Dec | Assignment 1 Demos, HLS Implementation | [HLS] | ||
| 7 | 15 Dec | Structured Naming, Naming: Resolution and Attribute Based, Mutual Exclusion and Leader Election | [pdf] | [DNS] | |
| 8 | 22 Dec | Distributed database and consistency | [pdf] | [Mutex] | |
| Consensus and Failure | 9 | 5 Jan 2025 | Distributed databases, Cyber Part 2: Storage cyber security analysis, Synchronization: Physical Clocks, GPS and NTP | [pdf], [pdf] | |
| 10 | 12 Jan | Logical Clocks, Totally Ordered Multicast (TOM) | [pdf] | [NTP] | |
| 11 | 19 Jan | Synchronization: Vector Clocks, Data Oriented Consistency | [pdf] | [TOM] | |
| 12 | 26 Jan | Raft and reliable group communication, Part 3: Consensus cyber security analysis, | [pdf] | [COM] | |
| Outro | 13 | 2 Feb | Project presentations and course summary, |
Academic Integrity
Cheating of any sort will not be tolerated. Student collaboration is encouraged, but within limits as set forth in the college’s rules on academic integrity. Any students caught cheating will be immediately referred to the department head and the Dean and may receive a failing grade for the course.
Cheating includes:
- Copying information, content, or verbatim text from other students, internet sites, books (other than the ones listed in the bibliography), other unaffiliated individuals to answer questions, solve problems, or aid in programming projects.
- Copying or submitting source code, documentation, or other programming aids without attribution from other students, web sites, online repositories, text books, open source programs, or other unaffiliated individuals.
- Project teams which submit work which is identical or substantially identical to work submitted by other project teams, whether current or from previous years.
- Other forms of academic misconduct as described at this link or as reasonably assessed by the instructor, program head, or dean.
If you have any questions about what constitutes cheating in the above rules, contact the instructor as early as possible.