Role : Data Engineer
Duration : 12 Months
Location : Cupertino, CA or Remote (USA)
Job description
Key Skills
• Strong experience building robust and scalable data processes and pipelines for modeling, analysis, and reporting
• Experience with Enterprise Data Warehousing systems including Snowflake and Teradata (or equivalent)
• Fluency in SQL and Python, including data wrangling and schema design
• Experience working with pipeline tools like Airflow and dbt
• Experience with BI processes and some experience with dashboard tools like Tableau
• Some experience with CI/CD and containerization tools (e.g. Jenkins, Docker, Kubernetes)
• Ability to initiate, refine, and complete projects with minimal guidance and some experience working in a scrum or release cycle environment
• Ability to clearly communicate technical concepts, definitions, logic, and processes to a non-technical audience
• Ability to think critically and collaborate cross-functionally with other data engineering, data science and analytics stakeholders distilling business requirements into clear data products
• Experience designing data ingestion processes and working with unstructured data a plus
Project Description
• Design, create, refine, and maintain data processes and pipelines used for modeling, analysis, and reporting
• Operationalize data products with detailed documentation, automated data quality checks and change alerts
• Support data access through various sharing platforms, including dashboard tools
• Troubleshoot failures in data processes, pipelines, and products
• Communicate and educate consumers on data access and usage, managing transparency in metric and logic definitions
• Collaborate with other data scientists, analysts, and engineers to build full-service data solutions
• Develop and communicate architectures, code patterns and data structure design choices to team of data scientists, analysts and engineers laying out tradeoffs
• Support codebase compatibility with Snowflake by designing, creating, and driving adoption of templates, packages, and best practices
• Optimize query and database performance through designing, creating, refining, and maintaining performance management system
• Work with cross-functional business partners and vendors to acquire and transform raw data sources
• Design, create, refine, and maintain data ingestion process
• Provide weekly updates to the team on progress and status of planned work