Job Description :
Hi,
Greeting !

Role: Sr Big Data Engineer
Rate: Market Rate
Duration: 6 Months+
Location: Phoenix, AZ


Big Data Engineer:
This engineer will help us build and support build out of our new big data environment being built on Azure.
Essential Duties and Responsibilities:
Day to Day support of existing data ingestion jobs
Design and implementation of roles, access, and data policies
Creation, support, and scheduling of ingestions from new sources
Ongoing data engineering and enhancements to data sets within the environment
Day to Day support for the health of the HDI/Databricks clusters in Azure
Writing shell scripts to automate jobs in HDI clusters
Installing packages if required in HDI clusters
Support integration of new CICD tools
Qualifications
Prior Hadoop Admin experience
Proficient with big data technologies
o Hive, Pig, HBase, MapReduce, Spark
Experience with open source ingestion tools
o Sqoop, Flume, Spark streaming, Kafka, Nifi
Proven experience building real-time steaming data sets (Flink, Spark streaming, storm)
Experience of Linux shell script
Experience working with and performing analysis using large data sets
o 10-1000 TB
Familiarity with common data science toolkits, such as R, Jupyter, & Python
Knowledge of traditional databases such as: Netezza, MySQL, Teradata, Oracle, etc
Preferred – Experience with Azure products: Azure Data Lake Store, Azure HD Insights, Cosmos DB, PowerBI