Job Description :
Responsibilities:
Developing large scale data pipelines exposing data sources within Client to our team of data analysts and data scientists.
Developing REST APIs utilizing AWS lambda and API Gateway.
Developing Spark streaming and batch jobs to clean and transform data.
Writing build automation to deploy and manage cloud resources.
Writing unit and integration tests.
Some of the specific technologies we use:
Programming Languages (Python, Scala, Golang, Node.js)
Build Environment: GitHub Enterprise, Concourse CI, Jira, Serverless, SAM
Cloud Computing (AWS Lambda, EC2, ECS)
Spark(AWS EMR, Databricks)
Stream Data Platforms: Kinesis, Kafka
Databases: S3, MySQL, Oracle,MongoDB, DynamoDB
Caching Frameworks (ElasticCache/Redis)

Requirements:
BS/MS degree in Computer Science, Mathematics, or other relevant science and engineering discipline.
4+ years working as a software engineer.
2+ years working within an enterprise data lake/warehouse environment or big data architecture.
Excellent programming skills with experience in at least one of Python, Scala, Java, Node.js.
Great communication skills.
Proficiency in testing frameworks and writing unit/integration tests
Proficiency in Unix-based operating systems and bash scripts.
Preferred Additional Skills:
Experience with working in Spark
Experience with AWS
Experience with monitoring and visualization tools such as Gafana, Prometheus, Data Dog, and Cloudwatch.
Experience with NoSQL databases, such as DynamoDB, MongoDB, Redis,Cassandra, or HBase

Client : Direct End Client