PI2 Data Engineer_42_1/2/3

Job description

Position title – Data Engineer

Location: Bangalore – Global Delivery Services

Jobpurpose:

Data Engineer in the IT Advisory Services team tobe part of the delivery of Data/Digital based projects for our customersacross the globe.

 
TechnicalResponsibilities:

·  Big data Engineer with 6 to 10 years of ITexperience including 4 to 7  years in Big Data and Analytics field,developing E2E Data pipelines to perform batch and Real - Time/Streamanalytics on structured and unstructured data.
·  Developing, constructing, testing and maintainingarchitectures for data lakes, data pipeline, data warehouses and large-scaledata processing systems .
·  Expertise in building Data Lake,Data Pipeline onCloudera or Hortonworks.
·  Experience in Big Data Analytics and design inHadoop ecosystem using Spark/Spark Streaming, Kafka
·  Solid understanding of data processing patterns,distributed computing and in building applications for real-time and batchanalytics.
·  Strong programming skills in design andimplementation using Python, Scala and Java
·  Hands on DataIngress, Egress, Processing using Kafka or DataBricks Streaming.
·  Goodknowledge on NoSQL Data bases such as HBase, Cassandra, Redis and using Sparkstreaming for real time stream processing of data into the cluster.
·  Experiencewith multiple Hadoop file formats like Avro, Parquet, ORC, and JSON etc.

We are looking for the candidates with the following:
·  BE/BTech/MCA with a sound industry experience of 6to 10 years

Mandatory skills:

·  Spark, Scala,Spark Streaming,Pyspark
·  Kafka, Hbase, Cassandra, Mongodb,
·  Cloudera/HortonworksHadoop Distribution
·  Any RDBMS 

Preferred skills:

·  Data Warehousing, ETL,Hive, Flink, AWS/Azure
·  Experience working onCMMI / Agile / SAFE methodologies