Data Engineering Manager / Architect | Location: Foster City, CA & Raleigh, NC (Long Term Contract)
9-12 years of experience
AWS, Spark, Scala/Python (Preferably Python 3.6+), Redshift/Snowflake, NOSQL
Specific duties, job functions:
• Should be able to lead a small team of developers in Onsite/ offshore model
• Design, Develop and Optimize AWS services with AWS Cost Explorer, Performance Insights, and Well-Architected Tools.
• Deliver data ingestion identified by Product owners.
• Design data ingestions, including source data characteristics, destination data store requirements, and performance and scalability needs.
• Participate in and lead code reviews focused on ensuring the code's correctness, efficiency, robustness, and readability.
• Propose and advocate for development standards (patterns, processes, tools, etc.)
• Manage implantation partner's resources and provide work instructions.
Basic Qualifications
• Experience building data solutions using AWS services such as Appflow, Airflow, Redshift etc
• Proficiency in Python 3.6+ OR proficiency with a similar language and familiarity with Python 3.6+
• Proficiency in SQL
• Experience deploying applications using Terraform or a similar tool
• Experience building CI/CD pipelines using GitHub Actions or a similar tool
• Proficiency in managing code as part of a team using GitHub
• Familiarity with agile development methodologies
• Experience building data-driven unit test suites for data platforms
• Experience modelling highly dimensional data