Szkolenia Big Data w Gdańsk

Participants who complete this instructor-led, live training will gain a practical, real-world understanding of Big Data and its related technologies, methodologies and tools. Participants will have the opportunity to put this knowledge into practice through hands-on exercises. Group interaction and instructor feedback make up an important component of the class. The course starts with an introduction to elemental concepts of Big Data, then progresses into the programming languages and methodologies used to perform Data Analysis. Finally, we discuss the tools and infrastructure that enable Big Data storage, Distributed Processing, and Scalability. Format of the Course

• Part lecture, part discussion, hands-on practice and implementation, occasional quizing to measure progress.

Datameer is a business intelligence and analytics platform built on Hadoop. It allows end-users to access, explore and correlate large-scale, structured, semi-structured and unstructured data in an easy-to-use fashion. In this instructor-led, live training, participants will learn how to use Datameer to overcome Hadoop's steep learning curve as they step through the setup and analysis of a series of big data sources. By the end of this training, participants will be able to:

• Create, curate, and interactively explore an enterprise data lake
• Access business intelligence data warehouses, transactional databases and other analytic stores
• Use a spreadsheet user-interface to design end-to-end data processing pipelines
• Access pre-built functions to explore complex data relationships
• Use drag-and-drop wizards to visualize data and create dashboards
• Use tables, charts, graphs, and maps to analyze query results

Audience

• Data analysts

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

To meet compliance of the regulators, CSPs (Communication service providers) can tap into Big Data Analytics which not only help them to meet compliance but within the scope of same project they can increase customer satisfaction and thus reduce the churn. In fact since compliance is related to Quality of service tied to a contract, any initiative towards meeting the compliance, will improve the “competitive edge” of the CSPs. Therefore, it is important that Regulators should be able to advise/guide a set of Big Data analytic practice for CSPs that will be of mutual benefit between the regulators and CSPs. The course consists of 8 modules (4 on day 1, and 4 on day 2)

Advances in technologies and the increasing amount of information are transforming how business is conducted in many industries, including government. Government data generation and digital archiving rates are on the rise due to the rapid growth of mobile devices and applications, smart sensors and devices, cloud computing solutions, and citizen-facing portals. As digital information expands and becomes more complex, information management, processing, storage, security, and disposition become more complex as well. New capture, search, discovery, and analysis tools are helping organizations gain insights from their unstructured data. The government market is at a tipping point, realizing that information is a strategic asset, and government needs to protect, leverage, and analyze both structured and unstructured information to better serve and meet mission requirements. As government leaders strive to evolve data-driven organizations to successfully accomplish mission, they are laying the groundwork to correlate dependencies across events, people, processes, and information. High-value government solutions will be created from a mashup of the most disruptive technologies:

• Mobile devices and applications
• Cloud services
• Social business technologies and networking
• Big Data and analytics

IDC predicts that by 2020, the IT industry will reach $5 trillion, approximately $1.7 trillion larger than today, and that 80% of the industry's growth will be driven by these 3rd Platform technologies. In the long term, these technologies will be key tools for dealing with the complexity of increased digital information. Big Data is one of the intelligent industry solutions and allows government to make better decisions by taking action based on patterns revealed by analyzing large volumes of data — related and unrelated, structured and unstructured. But accomplishing these feats takes far more than simply accumulating massive quantities of data.“Making sense of thesevolumes of Big Datarequires cutting-edge tools and technologies that can analyze and extract useful knowledge from vast and diverse streams of information,” Tom Kalil and Fen Zhao of the White House Office of Science and Technology Policy wrote in a post on the OSTP Blog. The White House took a step toward helping agencies find these technologies when it established the National Big Data Research and Development Initiative in 2012. The initiative included more than $200 million to make the most of the explosion of Big Data and the tools needed to analyze it. The challenges that Big Data poses are nearly as daunting as its promise is encouraging. Storing data efficiently is one of these challenges. As always, budgets are tight, so agencies must minimize the per-megabyte price of storage and keep the data within easy access so that users can get it when they want it and how they need it. Backing up massive quantities of data heightens the challenge. Analyzing the data effectively is another major challenge. Many agencies employ commercial tools that enable them to sift through the mountains of data, spotting trends that can help them operate more efficiently. (A recent study by MeriTalk found that federal IT executives think Big Data could help agencies save more than $500 billion while also fulfilling mission objectives.). Custom-developed Big Data tools also are allowing agencies to address the need to analyze their data. For example, the Oak Ridge National Laboratory’s Computational Data Analytics Group has made its Piranha data analytics system available to other agencies. The system has helped medical researchers find a link that can alert doctors to aortic aneurysms before they strike. It’s also used for more mundane tasks, such as sifting through résumés to connect job candidates with hiring managers.

Day 1 - provides a high-level overview of essential Big Data topic areas. The module is divided into a series of sections, each of which is accompanied by a hands-on exercise. Day 2 - explores a range of topics that relate analysis practices and tools for Big Data environments. It does not get into implementation or programming details, but instead keeps coverage at a conceptual level, focusing on topics that enable participants to develop a comprehensive understanding of the common analysis functions and features offered by Big Data solutions. Day 3 - provides an overview of the fundamental and essential topic areas relating to Big Data solution platform architecture. It covers Big Data mechanisms required for the development of a Big Data solution platform and architectural options for assembling a data processing platform. Common scenarios are also presented to provide a basic understanding of how a Big Data solution platform is generally used.  Day 4 - builds upon Day 3 by exploring advanced topics relatng to Big Data solution platform architecture. In particular, different architectural layers that make up the Big Data solution platform are introduced and discussed, including data sources, data ingress, data storage, data processing and security.  Day 5 - covers a number of exercises and problems designed to test the delegates ability to apply knowledge of topics covered Day 3 and 4. 

Advances in technologies and the increasing amount of information are transforming how law enforcement is conducted. The challenges that Big Data pose are nearly as daunting as Big Data's promise. Storing data efficiently is one of these challenges; effectively analyzing it is another. In this instructor-led, live training, participants will learn the mindset with which to approach Big Data technologies, assess their impact on existing processes and policies, and implement these technologies for the purpose of identifying criminal activity and preventing crime. Case studies from law enforcement organizations around the world will be examined to gain insights on their adoption approaches, challenges and results. By the end of this training, participants will be able to:

• Combine Big Data technology with traditional data gathering processes to piece together a story during an investigation
• Implement industrial big data storage and processing solutions for data analysis
• Prepare a proposal for the adoption of the most adequate tools and processes for enabling a data-driven approach to criminal investigation

Audience

• Law Enforcement specialists with a technical background

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

Big Data is a term that refers to solutions destined for storing and processing large data sets. Developed by Google initially, these Big Data solutions have evolved and inspired other similar projects, many of which are available as open-source. R is a popular programming language in the financial industry.

When traditional storage technologies don't handle the amount of data you need to store there are hundereds of alternatives. This course try to guide the participants what are alternatives for storing and analyzing Big Data and what are theirs pros and cons. This course is mostly focused on discussion and presentation of solutions, though hands-on exercises are available on demand.

The course is part of the Data Scientist skill set (Domain: Data and Technology).

This classroom based training session will explore Big Data. Delegates will have computer based examples and case study exercises to undertake with relevant big data tools

Audience If you try to make sense out of the data you have access to or want to analyse unstructured data available on the net (like Twitter, Linked in, etc...) this course is for you. It is mostly aimed at decision makers and people who need to choose what data is worth collecting and what is worth analyzing. It is not aimed at people configuring the solution, those people will benefit from the big picture though. Delivery Mode During the course delegates will be presented with working examples of mostly open source technologies. Short lectures will be followed by presentation and simple exercises by the participants Content and Software used All software used is updated each time the course is run, so we check the newest versions possible. It covers the process from obtaining, formatting, processing and analysing the data, to explain how to automate decision making process with machine learning.

Big data is data sets that are so voluminous and complex that traditional data processing application software are inadequate to deal with them. Big data challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating and information privacy.

This instructor-led, live training (online or onsite) is aimed at technical persons who wish to learn how to implement a machine learning strategy while maximizing the use of big data. By the end of this training, participants will:

• Understand the evolution and trends for machine learning.
• Know how machine learning is being used across different industries.
• Become familiar with the tools, skills and services available to implement machine learning within an organization.
• Understand how machine learning can be used to enhance data mining and analysis.
• Learn what a data middle backend is, and how it is being used by businesses.
• Understand the role that big data and intelligent applications are playing across industries.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Apache Sqoop is a command line interface for moving data from relational databases and Hadoop. Apache Flume is a distributed software for managing big data. Using Sqoop and Flume, users can transfer data between systems and import big data into storage architectures such as Hadoop. This instructor-led, live training (online or onsite) is aimed at software engineers who wish to use Sqoop and Flume for transferring data between systems. By the end of this training, participants will be able to:

• Ingest big data with Sqoop and Flume.
• Ingest data from multiple data sources.
• Move data from relational databases to HDFS and Hive.
• Export data from HDFS to a relational database.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Talend Open Studio for Big Data is an open source ETL tool for processing big data. It includes a development environment to interact with Big Data sources and targets, and run jobs without having to write code. This instructor-led, live training (online or onsite) is aimed at technical persons who wish to deploy Talend Open Studio for Big Data to simplifying the process of reading and crunching through Big Data. By the end of this training, participants will be able to:

• Install and configure Talend Open Studio for Big Data.
• Connect with Big Data systems such as Cloudera, HortonWorks, MapR, Amazon EMR and Apache.
• Understand and set up Open Studio's big data components and connectors.
• Configure parameters to automatically generate MapReduce code.
• Use Open Studio's drag-and-drop interface to run Hadoop jobs.
• Prototype big data pipelines.
• Automate big data integration projects.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Głównym celem szkolenia jest zdobycie wiedzy z administracji systemem Apache Hadoop w środowiskach MapReduce oraz YARN na poziomie zaawansowanym. Tematyka szkolenia dotyczy w głównej mierze architektury systemu Hadoop, a w szczególności systemu plików HDFS oraz modeli programistycznych MapReduce i YARN oraz zagadnień związanych z planowaniem, instalacją, konfiguracją, administracją, zarządzaniem i monitorowaniem klastra systemu Hadoop. Pozostałe zagadnienia związane z tematyką BigData takie jak HBase, Cassandra, Impala, Pig, Hiver oraz Sqoop są również omówione, choć pobieżnie. Kurs przeznaczony jest w głównej mierze do specjalistów z branży IT, którzy chcą przygotować się i zdać egzamin CCAH (Cloudera Certified administrator for Apache Hadoop).

Big Data Hadoop to otwarta, rozproszona i skalowalna platforma, stworzona do przechowywania, przetwarzania i analizy dużych zbiorów danych. Hadoop umożliwia przechowywanie dużych ilości danych na klastrze komputerów, dzięki czemu można obsługiwać petabajty danych. Framework Hadoop zawiera system plików Hadoop Distributed File System (HDFS) oraz framework do przetwarzania rozproszonego MapReduce. Pozwala to na równoległe przetwarzanie danych na wielu maszynach w klastrze. Platforma Hadoop jest łatwa w skalowaniu. Można ją rozbudować poprzez dodanie nowych węzłów do klastra, co zwiększa przepustowość i przetwarzanie danych. Składa się z różnych modułów, takich jak HDFS, MapReduce, YARN (Yet Another Resource Negotiator) oraz dodatkowych narzędzi i bibliotek, które mogą być używane w zależności od potrzeb projektu. Hadoop umożliwia pracę z różnorodnymi typami danych, w tym strukturyzowanymi, półstrukturyzowanymi i nieustrukturyzowanymi. Jest używany do analizy dużych zbiorów danych, tworzenia raportów, analizy trendów, przewidywania oraz uczenia maszynowego. Hadoop posiada bogaty ekosystem narzędzi i frameworków, takich jak Apache Hive, Apache Pig, Apache Spark, Apache HBase, które wspierają różne aspekty przetwarzania i analizy danych. Zapewnia mechanizmy bezpieczeństwa, kontroli dostępu i zarządzania użytkownikami, aby chronić dane przechowywane i przetwarzane w klastrze Hadoop. Big Data Hadoop jest popularnym narzędziem do przetwarzania i analizy dużych zbiorów danych, stosowanym w różnych branżach, w tym w analizie rynkowej, medycynie, finansach, przemyśle i wielu innych, gdzie konieczne jest zarządzanie i analiza dużej ilości danych.

Big data analytics involves the process of examining large amounts of varied data sets in order to uncover correlations, hidden patterns, and other useful insights. The health industry has massive amounts of complex heterogeneous medical and clinical data. Applying big data analytics on health data presents huge potential in deriving insights for improving delivery of healthcare. However, the enormity of these datasets poses great challenges in analyses and practical applications to a clinical environment. In this instructor-led, live training (remote), participants will learn how to perform big data analytics in health as they step through a series of hands-on live-lab exercises. By the end of this training, participants will be able to:

• Install and configure big data analytics tools such as Hadoop MapReduce and Spark
• Understand the characteristics of medical data
• Apply big data techniques to deal with medical data
• Study big data systems and algorithms in the context of health applications

Audience

• Developers
• Data Scientists

Format of the Course

• Part lecture, part discussion, exercises and heavy hands-on practice.

Note

• To request a customized training for this course, please contact us to arrange.

Szkolenie pozwoli w pełni zapoznać się i zrozumieć wszystkie niezbędne kroki do obsługi i utrzymywania klastra Hadoop. Dostarcza wiedzę począwszy od zagadnień związanych ze specyfikacją sprzętu, instalacją i konfiguracją systemu, aż do zagadnien związanych z równoważeniem obciążenia, strojeniem, diagnozowaniem i rozwiązywaniu problemów  przy wdrożeniu. Kurs dedykowany administratorom, którzy będą tworzyć lub/i utrzymywać klaster Hadoop. Materiały szkoleniowe

• Materiały szkoleniowe Student Guide
• Materiały szkoleniowe Lab Guide

Apache Hadoop is the most popular framework for processing Big Data on clusters of servers. In this three (optionally, four) days course, attendees will learn about the business benefits and use cases for Hadoop and its ecosystem, how to plan cluster deployment and growth, how to install, maintain, monitor, troubleshoot and optimize Hadoop. They will also practice cluster bulk data load, get familiar with various Hadoop distributions, and practice installing and managing Hadoop ecosystem tools. The course finishes off with discussion of securing cluster with Kerberos. “…The materials were very well prepared and covered thoroughly. The Lab was very helpful and well organized”
— Andrew Nguyen, Principal Integration DW Engineer, Microsoft Online Advertising Audience Hadoop administrators Format Lectures and hands-on labs, approximate balance 60% lectures, 40% labs.

Apache Hadoop is the most popular framework for processing Big Data on clusters of servers. This course will introduce a developer to various components (HDFS, MapReduce, Pig, Hive and HBase) Hadoop ecosystem.  

Apache Hadoop is one of the most popular frameworks for processing Big Data on clusters of servers. This course delves into data management in HDFS, advanced Pig, Hive, and HBase.  These advanced programming techniques will be beneficial to experienced Hadoop developers. Audience: developers Duration: three days Format: lectures (50%) and hands-on labs (50%).  

Hadoop is the most popular Big Data processing framework.

As more and more software and IT projects migrate from local processing and data management to distributed processing and big data storage, Project Managers are finding the need to upgrade their knowledge and skills to grasp the concepts and practices relevant to Big Data projects and opportunities. This course introduces Project Managers to the most popular Big Data processing framework: Hadoop.   In this instructor-led training in, participants will learn the core components of the Hadoop ecosystem and how these technologies can be used to solve large-scale problems. By learning these foundations, participants will  improve their ability to communicate with the developers and implementers of these systems as well as the data scientists and analysts that many IT projects involve. Audience

• Project Managers wishing to implement Hadoop into their existing development or IT infrastructure
• Project Managers needing to communicate with cross-functional teams that include big data engineers, data scientists and business analysts

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

Audience: This course is intended to demystify big data/hadoop technology and to show it is not difficult to understand.

Hadoop is a popular Big Data processing framework. Python is a high-level programming language famous for its clear syntax and code readibility. In this instructor-led, live training, participants will learn how to work with Hadoop, MapReduce, Pig, and Spark using Python as they step through multiple examples and use cases. By the end of this training, participants will be able to:

• Understand the basic concepts behind Hadoop, MapReduce, Pig, and Spark
• Use Python with Hadoop Distributed File System (HDFS), MapReduce, Pig, and Spark
• Use Snakebite to programmatically access HDFS within Python
• Use mrjob to write MapReduce jobs in Python
• Write Spark programs with Python
• Extend the functionality of pig using Python UDFs
• Manage MapReduce jobs and Pig scripts using Luigi

Audience

• Developers
• IT Professionals

Format of the course

• Part lecture, part discussion, exercises and heavy hands-on practice

Apache Hadoop is a popular data processing framework for processing large data sets across many computers. This instructor-led, live training (online or onsite) is aimed at system administrators who wish to learn how to set up, deploy and manage Hadoop clusters within their organization. By the end of this training, participants will be able to:

• Install and configure Apache Hadoop.
• Understand the four major components in the Hadoop ecoystem: HDFS, MapReduce, YARN, and Hadoop Common.
• Use Hadoop Distributed File System (HDFS) to scale a cluster to hundreds or thousands of nodes.  
• Set up HDFS to operate as storage engine for on-premise Spark deployments.
• Set up Spark to access alternative storage solutions such as Amazon S3 and NoSQL database systems such as Redis, Elasticsearch, Couchbase, Aerospike, etc.
• Carry out administrative tasks such as provisioning, management, monitoring and securing an Apache Hadoop cluster.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

This course introduces HBase – a NoSQL store on top of Hadoop.  The course is intended for developers who will be using HBase to develop applications,  and administrators who will manage HBase clusters. We will walk a developer through HBase architecture and data modelling and application development on HBase. It will also discuss using MapReduce with HBase, and some administration topics, related to performance optimization. The course  is very  hands-on with lots of lab exercises.
Duration : 3 days Audience : Developers  & Administrators

Apache NiFi (Hortonworks DataFlow) is a real-time integrated data logistics and simple event processing platform that enables the moving, tracking and automation of data between systems. It is written using flow-based programming and provides a web-based user interface to manage dataflows in real time. In this instructor-led, live training (onsite or remote), participants will learn how to deploy and manage Apache NiFi in a live lab environment. By the end of this training, participants will be able to:

• Install and configure Apachi NiFi.
• Source, transform and manage data from disparate, distributed data sources, including databases and big data lakes.
• Automate dataflows.
• Enable streaming analytics.
• Apply various approaches for data ingestion.
• Transform Big Data and into business insights.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.

Apache NiFi (Hortonworks DataFlow) is a real-time integrated data logistics and simple event processing platform that enables the moving, tracking and automation of data between systems. It is written using flow-based programming and provides a web-based user interface to manage dataflows in real time. In this instructor-led, live training, participants will learn the fundamentals of flow-based programming as they develop a number of demo extensions, components and processors using Apache NiFi. By the end of this training, participants will be able to:

• Understand NiFi's architecture and dataflow concepts.
• Develop extensions using NiFi and third-party APIs.
• Custom develop their own Apache Nifi processor.
• Ingest and process real-time data from disparate and uncommon file formats and data sources.

Format of the Course

• Interactive lecture and discussion.
• Lots of exercises and practice.
• Hands-on implementation in a live-lab environment.

Course Customization Options

• To request a customized training for this course, please contact us to arrange.