Job Description :
Job Details:
DevOps Engineer – AWS Big Data
Required Qualifications:
AWS: 3 years experience including: EMR, EC2, S3, VPC, Athena, ElasticSearch, etc. Advanced experience with IAM policy and role management. Experience with container systems is desired – EKS, ECR, Kubernetes, Docker.
Infrastructure Operations: 3+ years supporting systems infrastructure operations, upgrades, deployments, and monitoring. Ability to keep systems running at peak performance, upgrade operating systems, patching, and version upgrades as required.
CI/CD: Experience with CI/CD methodologies and environments. Experience with Terraform, Jenkins or Circle-CI and GitHub. GitOps exposure is a plus.
Programming: 2+ years experience programming with Python. Experience with Bash, REST APIs, and JSON encoding.
Linux: 5+ years in Unix systems engineering with experience in Red Hat Linux, Centos or Ubuntu.
Detailed Job Description:
We are looking for a DevOps engineer with a systems background and AWS operational experience. This role will provide advanced operations support for Big Data workflows and contribute to DevOps automation and system improvements supporting our Big Data platform users. Key experience includes:
AWS operational experience
Hadoop ETL workloads (Hive, Spark, Presto)
DevOps (GitHub, Jenkins, Terraform)
Python, Linux shell scripting
Experience supporting & sustaining engineering teams
Key responsibilities:
Advanced support and infrastructure deployment for job scheduling workloads (Airflow/AWS EMR
Support infrastructure operations for AWS cloud-based services and ensure SLAs are achieved or exceeded.
Improve our infrastructure by developing and improving our CI/CD workflows and pipelines that provide stable and repeatable infrastructure and builds.
Contribute to and support container-based infrastructure (EKS, Kubernetes
Enhance observability through improved monitoring and alerting.
Provide advanced engineering support services (L3 support):
o Drive root cause analysis for common problems.
o Provide status updates to users and stakeholders.
o Track all details in the issue tracking system (Service Now
Assist with training and onboarding for new end users.
Contribute to Agile / Kanban workflows and team process work.
Other operations and support duties as needed.
Minimum years of experience*: 5+
Certifications Needed: No
Top 3 responsibilities you would expect the Subcon to shoulder and execute*: