Job Description :
Title: AWS Data Engineer [BizID#6232]
Location: New York
Job Requirements :
4+ years work experience as a Data Engineer or similar role
2+ years of hands-on development experience in AWS
Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
Strong analytic skills related to working with unstructured datasets.
Build processes supporting data transformation, data structures, metadata, dependency and workload management.
Hands-on implementation experience in manipulating, processing and extracting value from large disconnected datasets.
Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
Strong engineer with ability to abstract our core/repeated requirements into a build-once, configurable framework.
Experience with Dev/OPS architecture, implementation and operation
Technical skills :
3+ years software development experience with object-oriented programming languages, preferably Python
Knowledge of version control systems: Git, Azure Devops, Bitbucket
Knowledge of AWS cloud services: EC2, EMR, S3, RDS, Glue
Knowledge of data pipeline management tools: Airflow, Luigi, Tidal
Knowledge of data blending tools: Alteryx, Matillion, Attunity
Previous experience with the Snowflake database management system
Experience with big data tools: Hadoop, Spark, Kafka, etc.
Experience with stream-processing systems: Storm, Spark-Streaming, etc.
Nice to have technical skills:
Jenkins + CI/CD, Kafka, Containerization (Docker), DBT
AWS certified
Others :
Demonstrated analytical ability, judgment, and problem analysis techniques
Experience working with the Agile project management methodology
Great verbal and written communication skills to collaborate cross functionally