Job Description :
Must Haves: Python, SQL, and Cloud (GCP would be preferred)
The Cloud Data Engineer will develop and support data-driven applications in the cloud for Panera Bread. The Data Engineer will work as a part of a cross-functional team that creates well architected solutions that solve difficult business problems quickly, yet completely. This individual will be involved in application design and development from conceptualization through execution, deployment and sustainment using Google Cloud Platform - GCP. The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of applications and data initiatives.
Responsibilities (Other duties may be assigned)
Develop optimal extract, load and transform pipelines from a wide variety of data sources using Python and Spark
Utilize GCP ‘big data’ technologies like Cloud Dataproc and BigQuery
Schedule and orchestrate solutions using GCP Cloud Composer
Create real-time data streaming pipelines using GCP App Engine and Dataflow
Perform data exploration in support of new projects and to troubleshoot issues
Work closely with data analysts to facilitate data acquisition and enable efficient analysis
Adapt existing batch and real time data pipelines to accommodate new business requirements
Develop and deploy code with exception handling, logging and monitoring
Ensure data quality throughout the entire data processing pipeline
Perform unit testing, integration testing, and assist with user acceptance testing
Be proactive in clarifying requirements when needed to ensure accurate results
Experience base/background:
Bachelor’s Degree in computer science or equivalent experience required
3+ years hands on experience with Python
3+ years hands on SQL and relational database experience
2+ years hands on experience in a public cloud development environment (GCP preferred)
Experience setting standards for design, documentation, testing, and code quality
Proficient understanding of distributed computing principles
Experience with messaging systems, such as Kafka and GCP PubSub