Job Description :

AWS Data Engineer

Remote Position

Long Term

Responsibilities

  • Is responsible for the development, unit testing and implementation of data integration solutions
  • Responsible for importing, cleansing, transforming, validating and analyzing data with the purpose of understanding or making conclusions from the data for data modeling, data integration and decision-making purposes.
  • The primary role will be developing and unit testing ETL mapping jobs based on the requirement document (HLDLLDSTM) which will be provided by Lead or Client and fixing defects in QA and UAT process.
  • Should also possess good knowledge on Databases, Linux UNIX environment to perform his daily tasks and should have exposure to data model and concepts of Data warehouse to create ETL jobs for dimension and fact tables.
  • Create LLDSTM documents, develop, test, and deploy data integration processes using choice of ETL
  • Adopt integration standards, best practices, ABC framework etc while creating ETL jobs.
  • Performs data validation, cleansing and analysis.
  • Tests, debugs, and documents ETL processes, SQL queries, and stored procedures
  • Analyzes data from various sources, including databases and flat files
  • ETL development, deployment, optimization, support and defect fixes.
  • Develop error handling processes
  • Create and implement scheduling strategy
  • Possess good knowledge of Agile and Waterfall methodologies.

Qualifications

At least 6-10 years of experience with design, development, automation, and support of applications to

  • Extract, Transform, and Load data
  • Must have worked on AWS S3, SNS/SQS, Glue, Athena, Redshift, Lambda
  • Proficient working on Databricks Spark
  • Proficient in SQL and SQL scripts
  • Familiarity with Big Data Solutions (e.g. Data Lake / Delta Lake)
  • Ability to work in an agile environment
  • Build and maintain solutions with latest technologies including PySpark, Hadoop, Hive, Redshift, Scala, Python, Airflow and Traditional SQL procedures and ETL processes
             

Similar Jobs you may be interested in ..