Job Description :
8+ Years of experience with data engineering
4+ years of experience working with AWS data platforms
Experience with Agile and Scrum methodologies
Experience with Confluence and Jira
Experience with building and deploying streaming and batch data pipelines
Significant development and delivery experience with Python and PySpark.
Deployment experience with CI/CD tools, Jenkins, Bitbucket, Kubernetes, Docker, Salt
Automation experience with Cloud Formation, Docker, Kubernetes, Jenkins
Extensive hands-on experience implementing data migration and data processing using AWS services such as Athena, Glue, Lambda, S3, DynamoDB, NoSQL, Relational Database Service (RDS), Amazon EMR and Amazon Redshift.
Experience and detailed knowledge with ETL, Data Quality, metadata management, data profiling, micro-batches, streaming data loads.
Experience with development lifecycle (development, testing, documentation and versioning) on an AWS platform.