Big Data & Hadoop

Detailed Course Index

Duration: 150 Hours of Training

Job Description

Eligibility Criteria:

Pass in 12th Standard/Equivalent
Module 1 - Linux + Module 2 - Java: 90 hours training
Module 3 - Big Data & Hadoop: 60 hours training
Final Certification Test:

• Students will have to give a short test after the session to determine how much they have learned; this test will be objective as well as subjective where students will have to write code and execute it in front of the trainer to get the grades
• Follow up Project: Students shall be given follow-up projects and large data sets on which they will can write Map Reduce code and process those on their cluster. This will help them with deeper understanding of Hadoop for solving complicated problems.

Module 1 - LINUX

Access the command line

• Log in to a Linux system and run simple commands using the shell
• Hands On

Manage files from the command line

• Copy, move, create, delete, and organize files from the bash shell prompt
• Hands On

Get help in Red Hat Enterprise Linux

• Resolve problems by using online help systems and Red Hat support utilities.
• Hands On

Create, view, and edit text files

• Create, view, and edit text files from command output or in an editor.
• Hands On

Manage local Linux users and groups

• Manage local Linux users and groups and administer local password policies.
• Control access to files with Linux file system permissions Set Linux file system permissions on files and interpret the security effects of different permission settings.
• Hands On

Monitor and manage Linux processes

• Obtain information about the system, and control processes running on it.
• Hands On

Control services and Daemons

• Control and monitor network services and system daemons using systemd
• Hands On

Configure and secure OpenSSH service

• Access and provide access to the command line on remote systems securely using OpenSSH
• Hands On

Analyze and store logs

• Locate and accurately interpret relevant system log files for troubleshooting purposes.
• Hands On

Manage Red Hat Enterprise Linux networking

• Configure basic IPv4 networking on Red Hat Enterprise Linux systems.
• Hands On

Archive and copy files between systems

• Archive files and copy them from one system to another
• Hands On

Install and update software packages

• Download, install, update, and manage software packages from Red Hat and yum package repositories.
• Hands On

Access Linux file systems

• Access and inspect existing file systems on a Red Hat Enterprise Linux system.
• Hands On

Use Regular Expressions with Grep

• Write regular expressions that, when partnered with grep, will allow you to quickly isolate or locate content within text files.
• Hands On

Create and Edit text files with VIM

• Introduce the vim text editor, with which you can open, edit, and save text files.
• Hands On

Comprehensive Module 1 review: Practice and demonstrate the knowledge and skills learned.

Module 2 - JAVA

Introduction

• Programming language Types and Paradigms
• Computer Programming Hierarchy
• How Computer Architecture Affects a Language?
• Why Java?
• Flavors of Java
• Java Designing Goal
• Role of Java Programmer in Industry
• Features of Java Language.
• JVM –The heart of Java
• Java’s Magic Bytecode

Language Fundamentals

• The Java Environment:
• Installing Java.
• Java Program Development
• Java Source File Structure
• Compilation
• Executions.

Object Oriented Programming

• Class Fundamentals.
• Object & Object reference.
• Object Lifetime & Garbage Collection
• Creating and Operating Objects
• Constructor & initialization code block
• Access Control, Modifiers, methods
• Nested, Inner Class & Anonymous Classes
• Abstract Class & Interfaces
• Defining Methods, Argument Passing Mechanism
• Method Overloading, Recursion
• Dealing with Static Members and Finalize () Method
• Native Method. Use of “this” reference.
• Use of Modifiers with Classes & Methods
• Design of Accessors and Mutator Methods
• Cloning Objects, shallow and deep cloning
• Generic Class Types

Extending Classes and Inheritance

• Use and Benefits of Inheritance in OOP
• Types of Inheritance in Java
• Inheriting Data Members and Methods
• Role of Constructors in inheritance
• Overriding Super Class Methods.
• Use of “super”
• Polymorphism in inheritance
• Type Compatibility and Conversion
• Implementing interfaces

Package

• Organizing Classes and Interfaces in Packages
• Package as Access Protection
• Defining Package
• CLASSPATH Setting for Packages
• Making JAR Files for Library Packages
• Import and Static Import
• Naming Convention for Packages

Exception Handling

• The Idea behind Exception
• Exceptions & Errors
• Types of Exception
• Control Flow in Exceptions
• JVM reaction to Exceptions
• Use of try, catch, finally, throw, throws in Exception Handling
• In-built and User Defined Exceptions
• Checked and Un-Checked Exceptions

Array & String

• Operation on String
• Mutable & Immutable String
• Using Collection Bases Loop for String
• Tokenizing a String
• Creating Strings using StringBuffer

Thread

• Thread Life Cycle
• Thread Priorities
• Synchronizing Threads
• Inter Communication of Threads
• Critical Factor in Thread Deadlock

Applet

• Applet & Application
• Applet Architecture
• Parameters to Applet
• Embedding Applets in Web page
• Applet Security Policies

A Collection of Useful Classes

• Utility Methods for Arrays
• Observable and Observer Objects
• Date & Times
• Using Scanner
• Regular Expression
• Input/output Operation in Java (java.io Package)
• Streams and the new I/O Capabilities
• Understanding Streams
• The Classes for Input and Output
• The Standard Streams
• Working with File Object
• File I/O Basics
• Reading and Writing to Files
• Buffer and Buffer Management
• Read/Write Operations with File Channel
• Serializing Objects

GUI Programming

• Designing Graphical User Interfaces in Java
• Components and Containers
• Basics of Components
• Using Containers
• Layout Managers
• AWT Components
• Adding a Menu to Window
• Extending GUI Features Using Swing Components
• Java Utilities (java.util Package)

The Collection Framework

• Collections of Objects
• Collection Types
• Sets
• Sequence
• Map
• Understanding Hashing
• Use of ArrayList & Vector

Event Handling

• Event-Driven Programming in Java
• Event- Handling Process
• Event-Handling Mechanism
• The Delegation Model of Event Handling
• Event Classes
• Event Sources
• Event Listeners
• Adapter Classes as Helper Classes in Event Handling
• Anonymous Inner classes a Short cut to Event Handling
• Avoiding Deadlocks in GUI Code
• Event Types & Classes
• Networking Programming
• Networking Basics
• Client-Server Architecture
• Socket Overview
• Networking Classes and Interfaces
• Network Protocols
• Developing Networking Applications in Java

Database Programming using JDBC

• Introduction to JDBC
• JDBC Drivers & Architecture
• CURD operation Using JDBC
• Connecting to non-conventional Databases
• Role of JSP in MVC-2
• JSP with Database
• JSP Implicit Objects
• Tag Libraries
• JSP Expression Language (EL)
• Using Custom Tag
• JSP Capabilities:
• Exception Handling
• Session Management
• Directives
• JSP with Java Bean

Comprehensive Module 2 review: Practice and demonstrate the knowledge and skills learned.

Module 3 - Big Data & Hadoop

Introduction to Big Data & Hadoop

• What is Big Data
• Limitations of existing models on processing Big Data
• Apache Hadoop
• Hadoop architecture
• Hadoop Storage: HDFS
• The framework of Map Reduce
• Practice time: This will be mostly introductory session, and students will be given overview about the course and technology
• A look at the interview questions asked

Getting Started with Hadoop

• Understanding system requirements for installing Hadoop and VMware player
• Virtualization and the VMware player
• Installing the VMware player
• Creating a virtual machine in the VMware player
• Practice time: Students will Download and install VMware player on their machines and create a virtual machine instance on the installed software.

Hadoop Architecture

• Architecture Overview
• Hadoop Configuration
• Hadoop Core Services
• Hadoop Storage – HDFS
• Map Reduce
• Regular file System Vs HDFS
• Data Block Split
• HDFS – Block Replication Architecture
• Practice time: Students will be shown a small demo of Hadoop in action, various services running, different commands in HDFS and map reduce. This will be done to give them a better understanding of the big picture and how things can be expected to work for big data in Hadoop architecture.

Hadoop Deployment

• Hadoop Installation pre-requisite
• Hadoop Single Node cluster Installation
• Hadoop Multi Node Cluster Installation
• Single Node Cluster Vs Multi Node Cluster
• Practice time: Students will be installing Hadoop in their machine and creating single as well as multi-node cluster. A sample job will be given to students to run on both types of cluster setups

Introduction to Map Reduce

• What is Map Reduce
• Map Execution
• Map Reduce Essentials
• Map Reduce Jobs
• Map Reduce Engine
• Checking Hadoop environment for Map Reduce
• Taking sample Big Data and writing map reduce code for it
• Practice time: Students will be given different problem statements and data sets and asked to write map-reduce code on those data sets, executing and checking the output on their single node clusters.

Introduction to Pig

• What is Pig
• How it works
• Data Model
• Pig Execution and Interactive modes
• Pig Vs SQL
• Installing a Pig Engine
• Various Operations
• Pig Commands
• Practice time: Students will be practicing various Pig commands trying to query different formats of data from various sample data provided to them

Introduction to Hive

• What is Hive and why was it developed
• Various components of Hive
• Hive Data Model
• Writing simple queries in Hive
• Comparing Hive Queries with corresponding PIG commands
• Practice time: In this hands-on session students will execute various Hive queries on different types of sample data provided to them by the trainer

Comprehensive Module 3 review: Practice and demonstrate the knowledge and skills learned.