Job Description :

BigData Developer with Airflow - StateStreet

Client: StateStreet

Remote until Covid Boston, MA

Long term contract or Fulltime

·        Overall 7+ years of data warehousing experience

·        Minimum 2-3 years of experience on Apache Airflow in recent project

·        Develop guidelines for Airflow cluster and DAGs. Performance tuning of the DAGs and task implementation

·        Develop DAG - data pipeline to on-board and change management of datasets

·        Experience with Apache Airflow, including installing, configuring and monitoring Airflow cluster

·        Understanding of airflow rest services and integration of airflow of platform eco-system

·        Orchestrating the Airflow / workflow in hybrid cloud environment is big plus

·        Python & Pandas, with demonstrated experience in ETL

·        Ability to maintain clean and secure data environments

·        Working in a hybrid cloud environment

·        Must have hands on experience with advanced sql

·        Must be experienced in Lambda, SNS and SQS

·        Redshift experience is desired


Similar Jobs you may be interested in ..