Job Description :
Job Description: Big Data Lead
Location: Dallas
Duration: 3 Months
The customer is looking for experienced senior developer/lead who has hands-on experience in AWS Kinesis, EMR/Spark, Lambda function, Airflow which is all AWS based technologies.
· At least 2 large big data projects
· At least 1 large big data project on AWS done at scale.
· Expert in AWS Big data solution leveraging Python , PySpark, Spark,
· Expert in Snowflake.
· Good experience in Airflow for job orchestration.
· Writing scalable and efficient Batch data pipelines using AWS EMR stack
· Real-time data pipelines: Solid development experience building solutions leveraging server less architecture ( kinesis, lambda, and other AWS server less components)
· Experience building and implementing Micro services ( Nice to have )
· Prior experience productionizing data science / ML pipelines ( Nice to have )
· Hands on Big data engineer with ability to lead tech solutions , direct teams , develop and recommend architecture patterns
· In Data and Analytics space for last 7 years
Non-tech
· Someone who has passion for data and is relentless in chasing people/processes and get solids understanding of data.
· Strong at interacting with business users, understanding business use cases and experience to extract information from both business and other data SME, Curious to question/ challenge and under architecture. Ability to lead and develop data solutions.
· Retail Domain Knowledge
· Solid communication skills
· Go-getter