Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours (3 days)
Course Outline
Day 1: Theory and Introduction to Distributed Systems
- Introduction
- Overview of the training structure and agenda, along with a walkthrough of the training environment.
- Fundamentals of Distributed Systems
- Defining distributed systems and explaining their significance in contemporary applications.
- Key challenges: scalability, availability, consistency, and fault tolerance.
- Data Consistency Models
- Discussing Strong Consistency and Eventual Consistency.
- Managing consistency in distributed systems: quorum, Read-Write Quorums, and Read Your Own Writes.
- Distributed Logs and Communication
- Exploring the pub/sub pattern and the stream-table duality.
- Data compaction and real-time data processing.
- Case Study 1: High-Performance Applications
- Analyzing the architecture of communication systems (e.g., WhatsApp, Signal).
- Challenges related to consistency and data recovery.
Day 2: Practical Aspects of Distributed System Design
- Designing Fault-Tolerant Applications
- Examining patterns such as CQRS, Inbox/Outbox, Two-Phase Commit (2PC), Saga, Change Data Capture (CDC), Circuit Breaker, and Read Repair.
- Practical examples of their application.
- Examples of Non-Relational Databases
- Document Databases (e.g., MongoDB, CouchDB):
- Key-Value Stores (e.g., Redis):
- Graph Databases (e.g., Neo4j, OrientDB):
- Column-Oriented Databases (e.g., HBase):
- Object Databases (e.g., GridGain):
- Time-Series Databases (e.g., TimescaleDB, InfluxDB):
- Search Engines (e.g., Apache Solr):
- In-Memory Grids (e.g., Hazelcast, GridGain):
- Modern Databases: Partitioning, Sharding, and Replication
- Partitioning and Sharding: Exploring techniques for dividing data into smaller fragments to enhance system performance and scalability.
- Data Replication: Various types of replication (synchronous, asynchronous), including the benefits and challenges of replicating data in distributed environments.
- Secondary Indexes: Creating and optimizing queries using secondary indexes to improve performance.
- Case Study 2: Designing a Graph-Based System
- Designing and implementing graph modeling in distributed systems using Neo4j or OrientDB.
- Practical exercise: graph modeling.
- Real-Time Data Management vs. Traditional Data Warehouses
- Introduction to real-time data processing and batch processing.
- Example of using Timescale for monitoring time-series data.
- NewSQL – A Modern Approach to Relational Databases
- Discussing the NewSQL concept, which combines the benefits of relational databases with the flexibility and scalability of NoSQL solutions.
- NewSQL Assignment: Participants will familiarize themselves with popular NewSQL databases and work with CockroachDB in a practical task. The goal is to implement transactions with ACID guarantees in a distributed environment.
Day 3: Practical Exercises and Database Optimization
- Practical Tasks Using Non-Relational Databases:
- MongoDB Assignment: Creating complex queries with data aggregation.
- Participants will work on constructing queries using the MongoDB pipeline, focusing on grouping and filtering data in real-time.
- Redis Assignment: Implementing a caching mechanism using Redis.
- Participants will build a system to store query results in Redis to optimize read performance.
- CouchDB Assignment: Synchronizing data in CouchDB using replication features.
- The task involves configuring replication between two CouchDB instances and analyzing data conflicts.
- Neo4j Assignment: Optimizing Cypher queries in a graph database.
- Participants will analyze a large graph and build query optimizations to search for dependencies between nodes.
- InfluxDB Assignment: Processing time-series data and optimizing data retention.
- Exercises using InfluxQL to analyze data flow and establish retention strategies.
- GridGain Assignment: Data processing using GridGain, building and optimizing queries in an object-oriented environment.
- Participants will optimize the storage and retrieval of large objects.
- Apache Solr Assignment: Implementing full-text search using Solr.
- Creating indexes and optimizing search queries for large datasets.
- MongoDB Assignment: Creating complex queries with data aggregation.
- Assignment: Distributed Transactions
- Participants will learn about the concept of distributed transactions, including mechanisms that ensure consistency in distributed systems.
- Practical exercise: Implementing distributed transactions using patterns such as Two-Phase Commit (2PC) or Saga.
- Data Recovery After Failures and Backups
- Patterns: Last-Writer-Wins, Vector Clocks, CRDT.
- Strategies for data recovery after failures.
- Summary and Discussion
- Q&A session and experience exchange.
This training does not cover relational databases, Elasticsearch, Apache Kafka, Prometheus, or Cassandra.
Testimonials (1)
Combining theory with exercises
Artur - Asseco Poland S.A
Course - Bazy danych w budowaniu wysokowydajnych systemów rozproszonych
Machine Translated
