Magellan: Geospatial Analytics on Spark - Plan Szkolenia

A geographic information system (GIS) is a system designed to capture, store, manipulate, analyze, manage, and present spatial or geographic data. The acronym GIS is sometimes used for geographic information science (GIScience) to refer to the academic discipline that studies geographic information systems and is a large domain within the broader academic discipline of geoinformatics.

QGIS functions as geographic information system (GIS) software, allowing users to analyze and edit spatial information, in addition to composing and exporting graphical maps. QGIS supports both raster and vector layers; vector data is stored as either point, line, or polygon features. Multiple formats of raster images are supported, and the software can georeference images. To summarize it allows the users to Create, edit, visualise, analyse and publish geospatial information on Windows, Mac, Linux, BSD.

This program, in its first phase, introduces the QGIS interface for general usage. In the second phase, we introduce PyQGIS - the python libraries of QGIS that allows the integration of GIS functionalities in your python code or your python application, so that you may even create your own Python Plugin around a particular GIS functionality.

Pix4D is a photogrammetry software suite for drone mapping. With Pix4D, users can analyze image data and create 3D models, all from photographs processed by Pix4D.

This instructor-led, live training (online or onsite) is aimed at field ecologists and conservation managers who wish to land survey and drone map with Pix4D.

By the end of this training, participants will be able to:

• Use end to end drone mapping processes and techniques.
• Achieve high surveying accuracy.
• Understand the fundamentals of global coordinate systems and GPS.
• Generate and analyze survey output with Pix4D's image processing software.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Pix4D is a photogrammetry software suite for drone mapping. With Pix4D, users will be able to analyze image data and create 3D models, all from photographs processed by Pix4D.

This instructor-led, live training (online or onsite) is aimed at field ecologists and conservation managers who wish to post-process Pix4D drone data.

By the end of this training, participants will be able to:

• Implement Google Earth to enhance UAV mapping .
• Use outputs of images to generate reports.
• Post-process with GIS and CAD software for advanced drone data modeling.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

A geographic information system (GIS) is a system designed to capture, store, manipulate, analyze, manage, and present spatial or geographic data. The acronym GIS is sometimes used for geographic information science (GIScience) to refer to the academic discipline that studies geographic information systems and is a large domain within the broader academic discipline of geoinformatics.

The use of Python with GIS has substantially increased over the last two decades, particularly with the introduction of Python 2.0 series in 2000, which included many new programming features that made the language much easier to deploy. Since that time, Python has not only been utilized within commercial GIS such as products by Esri but also open source platforms, including as part of QGIS and GRASS. In fact, Python today is by far the most widely used language by GIS users and programmers.

This program covers the usage of Python and its advance libraries like geopandas, pysal, bokeh and osmnx to implement your own GIS features. The program also covers introductory modules around ArcGIS API, and QGIS toolboox.

Google Maps API is a service that allows developers to add location related features and map visualizations into websites and web applications.

This instructor-led, live training (online or onsite) is aimed at developers who wish to enhance their website or web application by adding geo-location features through Google Maps.

By the end of this training, participants will be able to:

• Embed a map into a new or existing website or web application.
• Display live data that synchs with current location.
• Create a mashup that includes Google Maps.
• Apply good CSS and JavaScript coding practices
• Create a custom map for an application.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Geographic Information System (GIS) is a framework for gathering, managing, and analyzing data rooted in the science of geography.

This instructor-led, live training (online or onsite) is aimed at intermediate-level geographers who wish to deepen their expertise in spatial analysis, data management, and GIS applications.

By the end of this training, participants will be able to:

• Apply advanced spatial analysis techniques to solve complex geographical problems.
• Manage large spatial databases and perform data quality control.
• Create dynamic and interactive maps and visualizations for various applications.
• Utilize programming and automation to streamline GIS workflows.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Python is a high-level programming language famous for its clear syntax and code readibility. Spark is a data processing engine used in querying, analyzing, and transforming big data. PySpark allows users to interface Spark with Python.

In this instructor-led, live training, participants will learn how to use Python and Spark together to analyze big data as they work on hands-on exercises.

By the end of this training, participants will be able to:

• Learn how to use Spark with Python to analyze Big Data.
• Work on exercises that mimic real world cases.
• Use different tools and techniques for big data analysis using PySpark.

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

Many real world problems can be described in terms of graphs. For example, the Web graph, the social network graph, the train network graph and the language graph. These graphs tend to be extremely large; processing them requires a specialized set of tools and processes -- these tools and processes can be referred to as Graph Computing (also known as Graph Analytics).

In this instructor-led, live training, participants will learn about the technology offerings and implementation approaches for processing graph data. The aim is to identify real-world objects, their characteristics and relationships, then model these relationships and process them as data using a Graph Computing (also known as Graph Analytics and Distributed Graph Processing) approach. We start with a broad overview and narrow in on specific tools as we step through a series of case studies, hands-on exercises and live deployments.

By the end of this training, participants will be able to:

• Understand how graph data is persisted and traversed.
• Select the best framework for a given task (from graph databases to batch processing frameworks.)
• Implement Hadoop, Spark, GraphX and Pregel to carry out graph computing across many machines in parallel.
• View real-world big data problems in terms of graphs, processes and traversals.

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

MLlib is Spark’s machine learning (ML) library. Its goal is to make practical machine learning scalable and easy. It consists of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as lower-level optimization primitives and higher-level pipeline APIs.

It divides into two packages:

• spark.mllib contains the original API built on top of RDDs.

• spark.ml provides higher-level API built on top of DataFrames for constructing ML pipelines.

Audience

This course is directed at engineers and developers seeking to utilize a built in Machine Library for Apache Spark

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

OBJECTIVE:

This course will introduce Apache Spark. The students will learn how  Spark fits  into the Big Data ecosystem, and how to use Spark for data analysis.  The course covers Spark shell for interactive data analysis, Spark internals, Spark APIs, Spark SQL, Spark streaming, and machine learning and graphX.

AUDIENCE :

Developers / Data Analysts

Hortonworks Data Platform (HDP) is an open-source Apache Hadoop support platform that provides a stable foundation for developing big data solutions on the Apache Hadoop ecosystem.

This instructor-led, live training (online or onsite) introduces Hortonworks Data Platform (HDP) and walks participants through the deployment of Spark + Hadoop solution.

By the end of this training, participants will be able to:

• Use Hortonworks to reliably run Hadoop at a large scale.
• Unify Hadoop's security, governance, and operations capabilities with Spark's agile analytic workflows.
• Use Hortonworks to investigate, validate, certify and support each of the components in a Spark project.
• Process different types of data, including structured, unstructured, in-motion, and at-rest.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Spark SQL is Apache Spark's module for working with structured and unstructured data. Spark SQL provides information about the structure of the data as well as the computation being performed. This information can be used to perform optimizations. Two common uses for Spark SQL are:
- to execute SQL queries.
- to read data from an existing Hive installation.

In this instructor-led, live training (onsite or remote), participants will learn how to analyze various types of data sets using Spark SQL.

By the end of this training, participants will be able to:

• Install and configure Spark SQL.
• Perform data analysis using Spark SQL.
• Query data sets in different formats.
• Visualize data and query results.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Stream Processing refers to the real-time processing of "data in motion", that is, performing computations on data as it is being received. Such data is read as continuous streams from data sources such as sensor events, website user activity, financial trades, credit card swipes, click streams, etc. Stream Processing frameworks are able to read large volumes of incoming data and provide valuable insights almost instantaneously.

In this instructor-led, live training (onsite or remote), participants will learn how to set up and integrate different Stream Processing frameworks with existing big data storage systems and related software applications and microservices.

By the end of this training, participants will be able to:

• Install and configure different Stream Processing frameworks, such as Spark Streaming and Kafka Streaming.
• Understand and select the most appropriate framework for the job.
• Process of data continuously, concurrently, and in a record-by-record fashion.
• Integrate Stream Processing solutions with existing databases, data warehouses, data lakes, etc.
• Integrate the most appropriate stream processing library with enterprise applications and microservices.

Audience

• Developers
• Software architects

Format of the Course

• Part lecture, part discussion, exercises and heavy hands-on practice

Notes

• To request a customized training for this course, please contact us to arrange.