Sr. Bigdata Engineer
Location : Raleigh, NC
Rate/hr : As per the market standards (on C2C/1099 - All inclusive rate )
Visa Status : USC / GC / EAD / H1B
Job Description:
Overview:
You’ll work on a high impact and high visibility project that enables data availability, encompasses data analytics, machine learning, and petabyte scale datasets, and provides reliable and timely access to thousands of data sources.
Responsibilities:
Design, architect and support new and existing data and ETL pipelines and recommend improvements and modifications.
Create optimal data pipeline architecture and systems using Apache Airflow
Assemble large, complex data sets that meet functional and non-functional business requirements.
Be responsible for ingesting data into our data lake and providing frameworks and services for operating on that data including the use of Spark & Databricks.
Analyze, debug and correct issues with data pipelines
Operate on or build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Spark and Azure technologies.
Required Skills:
8+ years of experience building high performance scalable enterprise analytics or data centric solutions
At least 5 years of experience implementing complex ETL pipelines preferably in connection with Hadoop or Spark
Experience working with Spark data pipeline and or streaming experience
Expert in Python, or can demonstrate ability to readily learn new languages and affinity for them.
Expertise building and managing python libraries
Understand the nature of distributed systems design.
Hands-on experience with Azure.
Exceptional coding and design skills in Java/Scala or C#.
Strong understanding & usage of algorithms and data structures.
Nice to have Skills:
Azure Data Lake experience
Machine Learning expertise
Experience with Apache Airflow
Experience with C# .NET Core
Experience with traditional data marts/warehouses
Education:
BS in Computer Science / Software Engineering or equivalent experience
Please fill up the following details and send your / your consultant updated resume.
Applying for (position 1,2 or 3) :
Rate/hr (all inclusive) on C2C / 1099 :
Candidate Name :
Mobile no :
Email ID :
Linkedin Profile Link :
Work Auth /Visa Status :
Total Experience :
Relevant Experience :
Current Location :
Preferred Location :
Vendor Name :
L1 Interview date :
Candidate is ready to relocate Y/N :
Current Client Name :
Current Employer :
Skills matrix is mandatory for all submissions,
Skill Matrix-
Skill
Years of Experience,
,
Raj