Course Description

The demand for Big Data professionals is increasing across the globe and it’s a great opportunity for the IT professionals to move into the most sought technology in the present day world. ExcelR offers classroom and instructor-led live online Big Data course with Hadoop, delivered by industry experts who are considered to be the best trainers in the industry. The training is studded with loads of practical assignments, case studies and project work, which ensures the hands-on experience for the participants. Our Big data training program is meticulously designed to become a professional Big data Hadoop developer and crack the job in the space of Big Data. Various tools like Sqoop, Hive, HBase, Scala, Spark, Spark streaming, Kafka are extensively covered as part of the training. Along with these several value added topics like SQL, AWS, Azure, Python, Linux etc are covered in the context of Bigdata Hadoop. All these topics are considered to be nice to have which complements BIg Data concepts and are sought after by the recruiters. Post training support is provided and necessary hand holding will be provided in terms of resume preparation, Interview questions etc. ExcelR’s Big data program is considered to be the best program in the industry owing to its comprehensive curriculum, hands on assignments and projects, top notch trainers with extensive Big Data experience and have passion for training. No wonder ExcelR’s Big Data course is considered to be the best in the industry.

Course Curriculum

Module 1 - Introduction to Big Data

Introduction to Big Data
- Introduction
- What is Big data?
- Evolution of Data
- 5Vs pf Big Data
- Different Kinds of Data
- Big Data Sources
- Processing Big Data
- Big data Analytics
- Big Data Insight
- Applications of Big Data Analytics
- Benefits of Big Data
- How big Data Impacts IT
- Introduction To Big Data Quiz
Hadoop and Its Architecture
- what is hadoop?
- About Hadoop
- Problems in Distributed Computing
- Famouse Hadoop Users
- Why Hadoop?
- Features of Hadoop
- Simple Architecture of Hadoop
- Core Components of Hadoop
- What is HDFS ?
- What is Map Reduce?
- Hadoop Versions
- Types of Nodes in Hadoop
- Hadoop System
- Hadoop 1.x Architecture
  - Hadoop 1.x Cluster Administration
  - Hadoop 1.x Meta Data Management
  - Hadoop 1.x Architecture Disadvantages
- Hadoop 2.x Architecture
  - Hadoop 2.x Description
- Hadoop Spt QUIZ
Map Reduce and YARN
- Hadoop Core Components
- Design of HDFS
- Concept of HDFS
- Map Reduce - Data Processing
- Map Reduce : Parallel Processing of Data
- Map Reduce Programming Model
- Map Reduce Example : Vote Count (Traditional)
- Map Reduce Example : Vote Count (Map Reduce)
- Map Reduce in Detail through Word Count
- YARN
- YARN FLOW
- Scheduling in YARN
- Scheduling in YARN - Capacity Scheduler
- Map Reduce Vs YARN
- Yarn Take AWAY
Map Reduce and Yarn QUIZ
Cloudera Installation
Basic Commands in Hadoop
- Basic Commands in Hadoop Description
- Basic Commands in Hadoop Description - version
- Basic Commands in Hadoop Description - jps
- Basic Commands in Hadoop Description - ls
- Basic Commands in Hadoop Description -mkdir
- Basic Commands in Hadoop Description -put
- Basic Commands in Hadoop Description -cat
- Basic Commands in Hadoop Description - touchz
- Basic Commands in Hadoop Description - get
- Basic Commands in Hadoop Description -cp
- Basic Commands in Hadoop Description -mv
- Basic Commands in Hadoop Description -rm-r
- Basic Commands in Hadoop Description -du
- Basic Commands in Hadoop Description -stat
- Basic Commands in Hadoop Description -report
Basic Hadoop Commands Quiz
Tasks (Hadoop Commands)
Hadoop Distribution Systems
- Popular Hadoop Distributions
- Popular Hadoop Distributions - Cloudera
- Popular Hadoop Distributions - Horton Works
- Popular Hadoop Distributions - MapR
- Choosing a Hadoop Distribution
Hadoop Distribution Systems Quiz

Module 2 - Sqoop

Sqoop
- Sqoop Introduction
- Why Sqoop
- Sqoop Architecture
- Sqoop Features
- Sqoop Import
  - SQOOP Import Internal Process
  - Sqoop Import Important Parameters
  - Sqoop import Sample Execution
  - Sqoop Import Commands
    - Sqoop Import Command using WHERE condition
    - Sqoop Import for importing only Specific column
  - Sqoop Handson in CLOUDERA
- Sqoop Import Assignments
- Sqoop Import Important Commands
  - Sqoop EVAL
  - SQOOP EVAL SELECT
  - Sqoop EVAL Insert
  - Sqoop EVAL Update
  - Sqoop EVAL DELETE
  - Sqoop Import Split by
  - Sqoop Import Split by Handson
  - Sqoop Import Direct Mode
  - Sqoop Import Direct Mode Handson
- Sqoop Import EVAL,Split by ,direct Mode Assignments
- Sqoop Incremental Import
  - Incremental Append
  - Incremental Append - Sample Execution
  - Incremental Last Modified
  - Incremental Last Modified Sample Execution
- Assignments On Incremental Append
- Sqoop Job
  - Sqoop Job Description
  - Sqoop Job handson
  - Sqoop Job Listing
  - Sqoop Job Inspect
  - Sqoop Job Execute
  - Sqoop Job creation with Password file
  - Sqoop validate Command
  - Sqoop validate hands-on
- Sqoop Export
  - Sqoop Export Description
  - Sqoop Export Internal Process
  - Sqoop Export Hands-on
  - Sqoop Export Properties
    - Sqoop Export properties - Batch Mode
    - Sqoop Export properties - Merge
    - Sqoop export Properties - Transactionality
  - Sqoop Export Assignment
  - File Formats in Hadoop and Data Import in AVRO and PARQUET
    - File Formats in Hadoop
    - Benefits of Choosing exact File formats
    - Text Input Format
    - Sequence File Input Format
    - RC file Input format
    - ORC File INPUT format
    - AVRO Format
    - Parquet Format
    - Sqoop Import in AVRO
    - SQOOP Import in Parquet
  - Sqoop File format Assignments
  - Quiz on Sqoop (15 Questions)
  - Final Assignment on Sqoop

Module 3 - Hive

Hive Introduction
- Why another Warehousing system
- Hive Introduction
- what is hive?
- Architectural Overview
- Hive Execution plan
Quiz
Apache Hive Tables & Data types
- Hive Data types
- Data Abstraction in Hive
- Hive Data Model
- Hive Default Warehouse
- Hive Default Warehouse - Sample Execution
- Hive Tables
- Create database in Hive
- Internal Table
- Drop Internal Table
- Drop External Table
- When to use Internal and External Table?
- Internal Table Vs External Table
- Industry Usage
QUIZ
Assignments on Hive tables
Hive Bucketing and Partitions
- Partitions in Hive
- Static Partition
- Static Partition Creattion and Advantages
- Static Partition Sample Execution
- Dynamic Partition creation
- Dynamic Partition Sample Execution
- Dynamic Partition Advantages
- Bucketing in Hive
- Bucketing Sample Execution
- Industry Usage
QUIZ on Partitions
Assignments on Bucketing and partitions
ACID Properties and JOINS
- ACID Properties
- ACID Property - sample execution
- JOINS
- JOINS Strategies in Hive
- Shuffle Joins
- Map side join
- Sort Merge Bucket join
- Industry Usage
QUIZ
Assignments on ACID Properties
File Formats & Permformance Tuning
- Hive Built in Formats
- Hive Authorization
- Hive Serde
- Hive Serde - sample execution
- Hive Performance Tuning Techniques
- Industry Usage
Handling XML data, JSON in Hive
- XML data handling in Hive
- Handling XML data in Hive - Step - 1
- Handling XML data in Hive - Step - 2,3
- Handling XML data in Hive - Step - 4
- Handling JSON data
Hive Tasks -1
Handling Incremental data in Hive
- Handling Incremental load in Hive
- Handling Incremental load Hive - Merge
- Handling Incremental load Hive - Full Outer
QUIZ on Handling Incremental data
Hive tasks -2
Hive Advanced Scenarios
- How to choose the Number of Buckets
- How to check the Partitions in a table
- How to add Partitions to the existing table
- How to add multiple partitions to a table
- How to Drop partitions from a table
- How to create a new table from existing table
- How to create a new table from existing table without data
- How to pass variable to Hive script
- How to see create table syntax for a existing hive table
- How to rename the table in Hive
- Usage of case statement in Hive
QUIZ on Hive advanced commands
Hive Task - 3
Hive String Functions
- Hive CONCAT Function
- Hive Substring Function
- Hive Length (String A) Function
- Hive Upper(String A) Function
- Hive Lower(String A) Function
- Hive ucase(String A) Function
- Hive lcase(String A) Function
- Hive Ipad(String A,int len, string pad) function
- Hive rpad(String A,int len, string pad) function
- Hive trim(String A) function
- Hive Itrim(String A) function
- Hive rtrim(String A) function
- Hive Repeat(String A,int n) function
- Hive Reverse(String A,int n) function
Hive QUIZ on String functions
Hive Final QUIZ

Module 4 - Hbase

No SQL Database Introduction
- Summary of Early Database systems
- RDBMS
- Issues with RDBMS - Scalability
- Why RDBMS is not suitable for Big data
- What is NoSQL?
- Need of NoSQL
- Types of NoSQL Databases
- Characteristics of NoSQL Database
- Key Value Pair Based
- Column Based
- Document Based
- Graph Based
- CAP Theorem
- Advantages of NoSQL Database
- What are not provided by NoSQL
- Where to use NoSQL
- Conclusion
QUIZ on No SQL Introduction
Apache Hbase Introduction
- Introduction to NoSQL
- Benefits of NoSQL
- CAP Theorem
- What is Data Store?
- What is Columnar Database?
- Hbase
- How Hbase is a different kind of Columnar DB
- Difference between HDFS and Hbase
- When to use Hbase
- Features of Hbase
- Companies using Hbase
- Applications of Hbase
- Usecases of Hbase
QUIZ Hbase Introduction
Apache Hbase Core Components
- Hbase Tables
- Regions as Shards - Scalability
- Column Family
- Data Management
- Hfile Format Information
- Hbase Architecture
- Hbase Detailed Architecture
- Hbase Master(HMaster)
- Region Servers
- Zookeeper
- API
- Hbase first read or write
- Hbase Write Steps - 1
- Hbase Write Steps - 2
- Hbase MemStore
- HDFS Data Replication
- Web Interface
- Hbase Shell
QUIZ
Apache Hbase Commands & Hands-on
- Hbase Shell
- Hbase Shell - status
- Hbase Commands - Version
- Hbase commands - table_help
- Hbase commands -whoami
- Data Definition Language
- Data Manipulation Language
- Creating a Hbase table
- List command Hands-on
- How to create the data in Hbase -put
- How to view the data in Hbase -scan
- How to read the data in Hbase -get
- How to get the description of Table -describe
- Alter Command in Hbase
- Disabling a Table using Hbase Shell
- ls_disabled
- Enable a Hbase Table
- ls_enabled
- Existence of Table using Hbase Shell
- Delete a Specific cell in a Table
- Drop the Hbase Table
- Count command in Hbase
- User_Permission command
- Grant
Hands-on Hbase QUIZ
Assignment on Hbase

Module 5 - Scala

Basics of Scala
- What is Scala?
- What is Functional Programming?
- Pure Functions Example
- Impure Functions
- Variable declaration and Initialization
- VAR
- VAL
- Type Interference
- Lazy Evaluation
- String Interpolation
- Different String Interpolation Methods
- String -s Interpolator
- String -f Interpolator
- String -raw Interpolator
Quiz
Assignments
Scala Pattern Matching & Case Class & Companion Class
- Pattern Matching in Scala
- Pattern Matching - Handson
- Expression
- Statement in Scala
- Scala Class Vs Object
- Class Handson
- Singleton Object
- Singleton Hands-on
- Companion Classes & Object
- Case Class
- Case Class - Hands0n
Quiz
Assignments
Scala Collections
- Collections in Scala
- List Collection
- List Hands-on
- Set Collection
- Tuple Collection
- Tuple Collection -Handson
- Tuple Collection -Handson(2 element tuple)
- Map Collection with Handson
- Option Collection
- Option Collection - Hands-on
- Iterating over the Collection
Quiz
Assignments
Scala Functions
- Function
- Function without Parenthesis
- Nested Function
- VarArg Parameters into function
- Parameter Groups
- Methods and Operators
- More about Functoins
- What and all we can do with Objects
- What and all we can do with functions
Quiz
Assignments
Scala Higher order Function
- What is Higher oder Functions?
- Higher order functions Hands-on
- Why higher order functions?
- Higher order functions with multiple Input Args
- Function Carrying
- Function Carrying Hands-on
- Foreach() Higher order function
- Foreach() - Hands-on
- Map Higher order function
- Map Higher order function -Handson
- Filter higher order function
- Filter higher order function -Handson
- Reduce Higher order function
- Reduce Higher order function- Handson
- FlatMap Higher order function
- FlatMap Higher order function - Handson
Quiz
Assignments
Scala Traits
- What is traits?
- More about traits
- Traits Methods
Quiz
Assignments
Access Modifiers
- What are Access Modifiers in Scala
- Types of Access Modifiers
- Private Members
- Protected Members
- Public Members
Quiz
Extractors,Exception Handling & I/O Files
- What are Extractors?
- Extractors
- Extractors - output
- Code Explanation
- Usage of Scala Extractors
- Exception Handling
- Throwing Exceptions
- Catching Exceptions
- Catching Exceptions - Output
- Finally Clause
- Finally Clause - Output
- Scala I/O Operations
- Scala I/O Operations - Output
Quiz

Module 6 - PYSPARK

Spark Introduction
- Big Data Processing
- Why Spark?
- What is Spark?
- HADOOP Vs Spark
- Hadoop MapReduce Vs Apache Spark
- Spark Components
- Spark Ecosystem
- Spark Core
- Spark SQL
- Spark Streaming
- Mlib
Spark Intro_QUIZ
Spark Intro_Assignment
Spark RDD
- What is RDD?
- Operations on Rdds
- Properties of Rdd
- Ways of creating RDDs
- Input for Spark
- Spark file bases input
- RDD
- Features of RDD
- How RDD works?
- Spark Context
- Creating an RDD
- Creating an RDD - Parallelize
- Creating an RDD - Textfile
- Getting the output from RDD
- DAG - Direct Acyclic Graph
Spark RDD_QUIZ
Spark RDD_Assignment
Narrow and Wide Transformations
- Types of Transformation
- Narrow Transformation
- Narrow Transformation - Map
- Narrow Transformation - Map - Handson
- Narrow Transformation - flatmap
- Narrow Transformation - flatmap - Handson
- Narrow Transformation - fliter
- Narrow Transformation- fliter - Handson
- Narrow Transforamtion - Union
- Narrow Transforamtion - Union - Handson
- Wide Transformation
- Wide Transformation - GroupBy
- Wide Transformation - GroupBy - Handson
- Wide Transformation - ReduceByKey
- Wide Transformation - ReduceByKey - Handson
Narrow and Wide Transformations_QUIZ
Narrow and Wide Transformations_Assignment
Spark Architecture and Accumulators & Broadcast Variable
- Spark Architecture
- Spark Cluster - Driver
- Spark Cluster - Executor
- Executor Memory
- Spark App Decomposition
- Accumulator
- Accumulator - Handson
- Broadcast Variable
- Broadcast Variable - Handson
- Pyspark Memory Levels
- Spark Performance Techniques - Serialization
- Spark Performance Techniques - API Selection
- Spark Performance Techniques - Advance Variable
- Spark Performance Techniques - Cache & Persist
- Spark Performance Techniques - ByKey Operation
- Spark Architecture_QUIZ
Spark_Architecture_Assignment
Spark Submit Modes
- Spark Submit
- Spark Deployment modes
- Spark Submit Command
- Spark Submit - Options
- Spark Submit deploy modes
- Spark Submit - Client Mode
- Spark Submit - Cluster Mode
- Cluster Managers
- Cluster Managers Explanation
- Driver and Executor Resources
- Spark Submit Configurations
- Spark Submit Other options
- Spark Submit for word count in scala
- Spark Submit using Yarn Client Mode
- Spark Submit using Yarn Client Mode - output
- Spark Submit using Yarn Cluster Mode
- Spark Submit using Yarn Cluster Mode - output
- Spark Submit in Standardalone
- Spark Submit in Standardalone - output
Spark Submit Modes_QUIZ
Spark Submit Modes_Assignment
Spark SQL
- Why we need SQL in Bigdata
- Challenges in handling Bigdata
- Spark SQL Introduction
- Spark SQL core components
- Spark SQL Architecture
- Dataset,DataFrame and RDD
- SparkSQL (DataFrame)
- SparkSession
- Creating a SparkSession
- Creating a DataFrame
- DataFrame show()
- DataFrame Operations
- DataFrame Operations - Joins
- DataFrame Operations - Withcolumn
- Submitting a Pyspark job
- Submitting a Pyspark job - Step 1
SparksubmitMode_QUIZ
SparksubmitMode_Assignment
I/O in Pyspark SQL
- Read the data - csv file - step 1
- Read the data - csv file - step 2
- Read the data - csv file - step 3
- Read the data - csv file - step 4
- Read the data - csv file - step 5
- Count the number of records in DF
- Use of First()
- Use of Filter()
- Use of GroupBy()
- Convert the DataFrame to Pandas
- Join Opertaion in DF
- Join Opertaion in DF - output
- WithColumn() in DF
SparkSubmitIOoperation_QUIZ
SparkSubmitIOoperation_Assignment
Spark SQL Struct type
- Pyspark StructType and StructField Introduction
- StructField
- DF creation using StructType and StructField
- Creating DataFrame on nested StructType
- Creating Struct Type object struct from JSON file
- Adding & changing struct of the DataFrame
Spark SQL Struct type_QUIZ
Spark SQL Struct type_Assignment
Different Ways of creating Dataframe
- Ways of Creating DataFrame
- Creating the DataFrame from RDD
- Columns can be attached to DF
- Create DataFrame with Columns(*)
- DF creation with Schema
- DataFrame creation from DataSources
- Creating DataFrame from CSV
- Creating DataFrame from text file
- Creating DataFrame from JSON file
- Creating DataFrames from other sources
Different Ways of creating Dataframe_QUIZ
Different Ways of creating Dataframe_Assignment
Important Operations on Pyspark DataFrame
- Important Operations on Pyspark DataFrame
- Changing column datatype using Pyspark
- Update the value of an existing column
- Create a new column from an existing column
- Add a New Column using withColumn()
- Rename a column
- Drop a Column from Pyspark DataFrame
- Pyspark where Filter fuction
- Pyspark Filter with multiple conditions
Important Operations on Pyspark DataFrame_QUIZ
Important Operations on Pyspark DataFrame_Assignment
Pyspark Aggregate Functions
- Pyspark Aggregate Functions Introduction
- Pyspark Aggregate Functions - approx_count_distinct
- Pyspark Aggregate Functions - avg()
- Pyspark Aggregate Functions - collect_list()
- Pyspark Aggregate Functions - collect_set()
- Pyspark Aggregate Functions - countDistinct
- Pyspark Aggregate Functions - count()
- Pyspark Aggregate Functions - first()
- Pyspark Aggregate Functions - last()
- Pyspark Aggregate Functions -kurtosis()
- Pyspark Aggregate Functions - max()
- Pyspark Aggregate Functions - min()
- Pyspark Aggregate Functions - mean()
- Pyspark Aggregate Functions -skewness()
- Pyspark Aggregate Functions - stddev(),stddev_samp(),stddev_pop()
- Pyspark Aggregate Functions - sum()
- Pyspark Aggregate Functions - sumDistinct()
- Pyspark Aggregate Functions - variance(),var_samp(),var_pop()
Pyspark Aggregate Functions_QUIZ
Pyspark Aggregate Functions_Assignments
Pyspark Partitioning
- Pyspark Partitioning
- Pyspark Partitioning - Advantages
- Default Spark Partitions and Configurations
- Default Spark Partitions and Configurations - Local Mode
- Default Spark Partitions and Configurations - HDFS Cluster
- Default Spark Partitions and Configurations - Spark conf
- Dynamically changing Spark Partitions
- Dynamically changing Spark Partitions - Repartition()
- PartitionBy()
- How to choose Spark Partition Column
Pyspark Partitioning_QUIZ
Pyspark Partitioning_Assignment

Module 7 - SPARK STREAMING

Spark Streaming
- What is Streaming Data?
- What is Spark Streaming
- Simple Architecture of Spark Streaming
- Spark Streaming Work Flow
- Key Concepts in Spark Streaming
- What is Dstream?
- Streaming Context
- Dstream Interface
- Mapped Dstream
- Windowed Dstream
- Network Input Dstream
- Transformations on Dstreams
- stateful Transformations
- Windowed Transformations
- UpdatestateBykey Transformation
- Action/output Operations
- Network Receiver
- Components of Spark Streaming
- Execution Model - Receiving Data
- Execution Model - Job Scheduling
- Job Scheduling
- Dstream Persistance
- What is RDD Checkpointing ?
- Why is RDD Checkpointing necessary ?
- RDD Checkpointing
- Performance Tunning
- Performance Tunning - Step - 1
- Step - 2 : Optimize for Lower Latency
- Code for Spark Streaming - Word Count
- Code for Spark Streaming - Word Count flow
- Spark Streaming Use Cases
Spark Streaming QUIZ
Spark Streaming Assignments
Spark Structured Streaming
- What is Spark Structured Streaming ?
- What is new in Structured Streaming ?
- Structured Streaming Programming Model
- Structured Streaming Programming Model - output modes
- Word count Example for Structured Streaming
- Creating Streaming Data Frames
- Process data usinf file source
- Stateful Streaming : Window Operations Hands-On
- Stateful Streaming : Handling Late Data and Watermaking
- Triggers
- How to set Trigger
- Stateless Word Count
- Limitations of flatMapGroups
- Checkpoint and state recovery
- File Streams
- Joins with Static Data
Spark Structured Streaming QUIZ
Spark Structured Streaming Assignments
Spark Structured Streaming Final QUIZ

Module 8 - KAFKA

Kafka Introduction & Topics
- Why kafka is needed?
- Kafka Use Cases
- Who uses Kafka?
- What is Kafka ?
- Kafka Core Concepts
- Kafka API's
- Kafka API Representation
- Kafka Fundamental Concepts
- Kafka Topic
- More About Kafka Topics
Kafka Introduction & Topics QUIZ
Kafka _Leader & Replicas
- Leader & Replicas
- One Topics,Three Brokers,2 partitions
- Single Node Multiple Broker Clusters
- Zookeeper Importance in kafka
- Zookeeper Uses
- Controller Broker
- Off set
- Current Offset
- Kafka Fundamental Concepts
- Kafka Topic
- Moreabout Kafka Topics
Kafka _Leader & Replicas QUIZ
Kafka Installation & Topic Creation
- Kaka Installation
- One Topics,Three Broker,two Partitions
- Topic Creation
- Sending Messages through Producer
- Consuming Messages through Consumer
- Kafka Logs
- Spark Streaming with Kafka
Kafka Final QUIZ
Kafka Assignment

Module 9 - Projects

Project 1 - Hive AVRO Data Standandization
Project 2 - Banking Data processing using Hive
Project 3 - SPARK HBASE HIVE Project

Value added Courses

SQL

What is meant by RDBMS ?
Concepts of RDBMS
TABLES
TYPES OF SQL COMMANDS
DDL COMMANDS
DML COMMANDS
DCL COMMANDS
DQL COMMANDS
TCL COMMANDS
DATA TYPES IN SQL
DATABASE CONTRAINTS
- TYPES OF CONSTRAINTS
- RELATIONAL INTEGRITY
- KEY CONSTRAINTS
- DOMAIN CONSTRAINTS
- PRIMARY KEY
- FOREIGN KEY
JOINS
- TYPES OF JOINS
- INNER JOIN
- LEFT OUTER JOIN
- RIGHT OUTER JOIN
- FULL OUTER JOIN
- CARTESIAN JOIN
SUB QUIRES
- What is a Sub query ?
- Types of Sub quires
- Co-related Sub quires
- Non Co related Sub queries
RANKING Functions
- RANK()
- DENSE RANK()
- ROW NUMBER()
- Nth highest salary using ROW Number
- Removing Duplicates using ROW number
- LEAD function
- LAG function
- PIVOT TABLE in SQL
AGGREGATE FUNCTIONS
- MIN()
- MAX()
- SUM()
- COUNT()
- GROUP BY
- HAVING()
- ORDER BY ()

LINUX Basics

INTRODUCTION TO LINUX
- Linux Fetures
- Why Linux ?
UNIX BASIC COMMANDS
- PWD
- CREATE - MKDIR
- LIST
- COPY
- MOVE
- USING PIPE COMMAND
- I/O
- Redirection
- Command Line Editing using Vi editor
- HEAD
- TAIL
- AWK
- grep command
- sed command
- Find -n command
- rm command
Linux Environment and File system Essentials
- Bash
- Shell Variables
- Groups
- Changing File Attributes with CHMOD
- Changing file ownership with CHOWN and chgrp
- KILL command
- CRON TAB
Shell Scripting
- How to create the shell script
- Using variables
- using operators
- shell loops
- LOOP control
- Decision making
- shell functions
- Parameterized shell scripts
- Appending files
- copying the files
- FILTERS
Checking System Performance
- nice/renice
- netstat
- time
- uptime
- vmstat
- top
- ps -ef
- disk usage command
- tail -f command

CLOUD

CLOUD COMPUTING INTRODUCTION
- What is cloud?
- what is cloud computing?
- Types of CLOUD computing
- Cloud Deploying modes
- What is public cloud?
- What is private cloud?
- what is hybrid cloud?
CLOUD SERVICES
- Types of Cloud services
- Infrastuctre as Service (IaaS)
- Platform as a Service(PaaS)
- Software as a Service(SaaS)
- Advantages of Virtualization
- Advantages of Cloud Computing

AWS

AWS Introduction
What is AWS?
Why we go for AWS?
AWS Services
Types of AWS services
AWS services needed for Data Engineer
COMPUTE SERVICES
STORAGE
- AMAZON S3
- What is S3?
- How to create the bucket in S3
- How to load the data into S3 from external systems?
- AMAZON GLACIER
- What kinds of data we will place on AWS GLACIER
- AMAZON GLACIER Advantages
- Amazon Elastic File system
- AWS Storage Gateway
DATABASE SERVICES
- AMAZON RDS
- NO SQL - DynamoDB
- Amazon Red shift
- Elastic Cache
- Amazon Cloud Watch Service
EMR Service
- What is EMR service?
- How to create the EMR cluster ?
- Differences between EMR and S3 ?
- How to do the Hadoop data processing in EMR
- Similarities in EMR and Hadoop
- Why EMR is most important service in AWS with respect to Data engineer
AMAZON AUTOSCALING
- What is the need of Auto scaling in AWS?
- Advantages of Auto scaling
- Types of Auto scaling

Azure

AZURE INTRODUCTION
What is Azure?
Why Azure?
Azure Core Architecture
Azure Portal
Azure Powershell
Azure CLI
Rest Clients
AZURE SERVICES
- Basic explanation Azure Compute services
- Basic Explanation of Networking services
Azure STORAGE SERVICES
- What is ADLS ?
- Why we go for ADLS ?
- How to keep the data in ADLS ?
- What is BLOB storage ?
- when we will go for BLOB and ADLS?
- Creation of HDInsight Cluster
- Processing the hadoop in HDInsight cluster

Python

Introduction to Python Keywords
Keywords in Python
Python Identifiers
Importance of Comments in Python
Python Indentation
Python Statements
Variables in Python
Data types in Python
Conversion of Data Types in Python
Python Output
Formatting Output in Python
Collections in Python
Python Dictionaries
Python Inputs
Operators in Python
Python List
Python Tuples
Python Sets
Python Arrays
Control Flow in Python
IF statement
If else
if elif else
Nested Statements
Loops in python
Python Lambda
Python Iterators
PANDAS INTRODUCTION
- Pandas Intro
- Pandas Dataframe
- Pandas READ CSV
- Pandas Read JSON