Job Description :
JOB DESCRIPTION
We are looking for a Data Engineer that will help us build data pipelines and all data related work on AWS to design, develop, test and integrate a critical security related application. The application tracks user data during their daily business activities and their attempts to access various business applications during the day. Based on their access privileges and data entitlements, the users are either permitted access to systems/data or blocked by an automated system. All exceptions and valid access patterns are then stored on AWS system. These usage data are then used to develop users profiles and temple for system access based on their roles within the Bank.
Role Description:
Develop ETL/Data Integration pipelines among various source systems and AWS Data Lake
Use ETL Tools such as Data migration services, Glue, Spark/Python/Scala to design, develop, test and implement data pipelines that are developed as part of the application functionality
Design new data flows, maps, plans and implement them on AWS
Manage and maintain existing and new business rules using ETL or rules engine and test and integrate them in the data pipelines
Work with source side teams in understanding data models, and source data requirements to create staging and final Data Lake Data models and create HLDs, LLDs for the required data models
Use SQL skills to query data and understand the data relationships. Also use ad-hoc querying skills to understand the data flows, data transformations and data reconciliation, validation
Test the data pipelines in development and QA environment
Consult and work with multiple teams on daily basis to uncover business needs, data integration and validation requirements.
Exposure to AWS Sage Maker is preferable and knowledge of ML model development using Python scripts and libraries is a must.

Client : Mphasis