Job Description :
Job Role: Lead Data Engineer (AWS Big Data)
Job Location: Tempe, AZ
Job Duration: Long Term
Experience Required: 8+ Years

Qualifications & Experience:

Looking for experienced lead data Solution Engineer who has built data pipelines and data systems at scale using the Apache Open Source stack and the Hadoop ecosystem. They should have strong familiarity working in a AWS cloud environment. Comfortable working with data engineers, product managers and product delivery teams.

Key Responsibilities
Provide technical solution leadership in data engineering team, driving technology decisions, mentoring others, and contributing significantly on an individual level
Build frameworks to handle data at high scale using Apache Spark and data cataloging tools like Apache Hive, AWS Glue on top of a multi-tiered data lake storage
Use exploration and analytic tools like AWS Athena/Presto to probe and validate data
Build robust data processing pipelines using AWS Services and integrate with multiple data sources
Collaborate with product owners and stakeholders to plan and define requirements
Experience with the following software/tools is highly desired

Skills Required:
AWS Services: RDS, AWS Lambda, AWS Glue, Apache Spark, Kafka, Hive, etc
SQL and NoSQL databases like MySQL, Postgres, Elasticsearch
AWS EMR
Familiarity with Spark programming paradigms (batch and stream-processing)
Strong programming skills in at least one of the following languages: Java, Scala. Familiarity with a scripting language like Python as well as Unix/Linux shells
AWS Athena
Strong analytical skills and advanced SQL knowledge, indexing, query optimization techniques.
Good to have ETL skills
Ability to translate data needs into detailed functional and technical designs for development, testing and implementation
Ability to serve as a liaison between technical, quality assurance and non-technical stakeholders throughout the development and deployment process


Client : Silicon Valley Bank

             

Similar Jobs you may be interested in ..