Job Description :
Title: Data Engineer Location: remote Duration: 6+ Months with high possible extension Visa: EAD,GC and USC Manager Note: This is a referral position, please submit a candidate on W2 only as all successfully placed candidates have to work for our PV's company payroll. Responsibilities: The Data Engineer will innovate and transform the systems integration landscape for the Technology Infrastructure organization, while following industry best practices and providing capability maturity in support of Enterprise Data Management standards The ideal candidate is an expert in Data Warehousing and Master Data Management Design and Development Candidate should have a strong understanding of data management concepts and applied DW/MDM development of DB-level routines and objects It involves hands-on development and support of integrations with multiple systems and ensuring accuracy and quality of data by implementing business and technical reconciliations Candidate needs to be able to understand macro level requirements and convert them into actionable tasks to deliver a technically sound product Candidate should be able to work in teams in a collaborative manner Assist in bi-weekly development sprints to incorporate new data sources to the Big Data platform and modify existing integration based on business requirements Accomplished development experience using Spark and Spark SQL Expert level SQL skills suitable to migrate view definitions from one platform to another Expert level skills for evaluating, developing and performance tuning existing HIVE Managed Tables and PySpark implementation Apache Phoenix and HBase design, implementation and tuning Ability to manage multiple priorities Requirements: 7+ years of total IT experience At least 3 years of experience developing for Data Warehousing, Data Marts, and/or Master Data Management Deep experience on Hadoop including Python, Spark, HIVE, HBase and HDFS with an emphasis on performance tuning and architecture e.g., partitioning, bucketing and parquet, flat file Programming experience Python, PySpark, Spark SQL Exposure to Relational Database Management Systems using Oracle, DB2 or SQL Server Possesses and demonstrates deep knowledge of the Hadoop Ecosystem Experienced exposure to Hadoop ecosystem including but not limited to: HDFS, MapReduce, Spark, Sqoop, Oozie, Kafka, Hive Object oriented programming concepts Expert SQL skills Experience in SDLC and best practices for development Ability to work against mid-level design documentation, take it to a low-level design, and deliver a solution that meets the success criteria Knowledge of packaging and promotion practices for maintaining code in development, test, and production Stay safe and healthy!
             

Similar Jobs you may be interested in ..