Apply Now

Innovate Solutions Inc
Santa Clara, CA
Post Resume to
View Contact Details &
Apply for Job
Job Description :
Job Description:

Senior Data Engineer with strong hands on experience in Hadoop Framework and its ecosystem including but not limited to HDFS Architecture, Hive, Pig, Sqoop, HBase, MongoDB, Cassandra, Oozie, Spark RDDs, Spark DataFrames, Spark Datasets, Spark MLlib, etc.
Experience in Apache NIFI which is a Hadoop technology and also Integrating Apache NIFI and Apache Kafka.
Strong knowledge in Upgrading MapR, CDH and HDP Cluster.
Full-stack software engineer experienced in Hadoop and other big data platforms.
Proven experience showcasing technical and operational feasibility of Hadoop developer solutions.
Experience in software development using Big Data/Hadoop Echo Systems, Apache Spark, Python and ETL Technologies.
Hands-on Experience on major components in Hadoop Echo Systems like Spark, HDFS, HIVE, PIG, HBase, Zookeeper, Sqoop, Oozie, Flume, Kafka.
Developed scalable and reliable data solutions to move data across systems from multiple sources in real time as well as batch modes.
Deep knowledge in incremental imports, partitioning and bucketing concepts in Hive and Spark SQL needed for optimization.
Experience collecting log data from various sources and integrating it into HDFS using Flume; staging data in HDFS for further analysis.
Experience in designing and handling of various Data Ingestion patterns (Batch and Near Real Time) using Sqoop, Distcp, Apache Storm, Flume and Apache Kafka.