Plan Szkolenia
Introduction
Overview of Spark Streaming Features and Architecture
- Supported data sources
- Core APIs
Preparing the Environment
- Dependencies
- Spark and streaming context
- Connecting to Kafka
Processing Messages
- Parsing inbound messages as JSON
- ETL processes
- Starting the streaming context
Performing a Windowed Stream Processing
- Slide interval
- Checkpoint delivery configuration
- Launching the environment
Prototyping the Processing Code
- Connecting to a Kafka topic
- Retrieving JSON from data source using Paw
- Variations and additional processing
Streaming the Code
- Job control variables
- Defining values to match
- Functions and conditions
Acquiring Stream Output
- Counters
- Kafka output (matched and non-matched)
Troubleshooting
Summary and Conclusion
Wymagania
- Experience with Python and Apache Kafka
- Familiarity with stream-processing platforms
Audience
- Data engineers
- Data scientists
- Programmers
Opinie uczestników (5)
Zadania do zrealizowania samodzielnie oraz późniejsze wspólne rozwiązywanie
Katarzyna Kopysc-Falenta - CapGemini
Szkolenie - Data Analysis with Python, Pandas, and Numpy
Przekazanie wiedzy praktycznej oraz doświadczeń trenera.
Rumel Mateusz - Pojazdy Szynowe PESA Bydgoszcz SA
Szkolenie - GUI Programming with Python and PyQt
Przetłumaczone przez sztuczną inteligencję
Trener był bardzo dostępny, aby odpowiedzieć na wszystkie pytania, które zadałem
Caterina - Stamtech
Szkolenie - Developing APIs with Python and FastAPI
Przetłumaczone przez sztuczną inteligencję
It was a though course as we had to cover a lot in a short time frame. Our trainer knew a lot about the subject and delivered the content to address our requirements. It was lots of content to learn but our trainer was helpful and encouraging. He answered all our questions with good detail and we feel that we learned a lot. Exercises were well prepared and tasks were tailored accordingly to our needs. I enjoyed this course
Bozena Stansfield - New College Durham
Szkolenie - Build REST APIs with Python and Flask
As I was the only participant the training could be adapted to my needs.