Job Description :
Role: Senior / Lead Hadoop Data Engineer – Audience Management Big Data Platform (AMP)
Location: SanFrancisco, CA
Duration: 6+ months
Rate: $market All Inclusive
Responsibilities:
Build, enhance and scale our existing Big Data Platform to process 500+ Terabytes of data from various 3rd party providers.
Technically own the complete data pipeline module including data ingestion, validation, transformation & storage in the data platform.
Design and write clean, robust, server-side code using Spark & Scala.
Perform R & D to evaluate new technologies, generate different ideas & patentable IP to communicate value for company
Work closely with team members & mentor them technically
Partner with product managers to support product roadmap & business needs
Required Skills:
10 - 15 years’ experience in software development primarily in Big Data domain using Spark & Hadoop eco system.
5+ Years of experience in Spark & Scala (Skill Level: 8 or more out of 10)
Bachelor’s degree in Computer Science or equivalent.
5+ years’ experience designing, building and launching extremely efficient & reliable data pipelines to move data (both large and small amounts) using modern data architecture/tools.
Strong background (5+ years) in ETL processing for large amount of data (250+ Terabytes)
Multiple years of experience working with 250+ Terabytes of data for ETL using Spark & Scala
3+ years’ experience working in Agile environment (SCRUM)
Nice To Have Skills:
Experience in digital marketing (Good Domain knowledge on digital marketing Industry)
Experience working with AWS (S3, EMR, EC2, RDS)