Job Description :

The senior data engineer will analyze data for suitability for a given use case, develop ETL mapping specifications, develop tests to validate that the transformed data meets all requirements (test driven development) and develop data pipelines in Matillion to satisfy customer use cases.

The engineer will configure the ETL job schedule to run efficiently, at the time of day identified by the customer, and optimized to meet the customer’s performance requirements.

Additionally the engineer will guide the customer in developing standards based upon best practices, and implement those standards in the data pipelines they develop.

Responsibilities

  • Design complex ETL workflows and ETL patterns including audit controls and error handling utilizing industry and tool standards and best practices.
  • Analyze data for suitability for purpose, data quality and integration patterns
  • Develop source to target data mapping specifications.
  • Develop complex data pipelines in Matillion, documenting the pipelines within the tool.
  • Review code for standards, best practices and efficiencies.
  • Collaborate with other leads and senior resources for design and coding efficiencies.
  • Educate the client in best practices and implement those best practices in your delivery.

 

Qualifications

  • 3+ years of experience across at least 2 different ETL tools (Matillion experience is a plus)
  • 3+ years of experience with SQL
  • 3+ years of experience with RDBMS such as PostgreSQL, Redshift, SQL Server or Snowflake (Snowflake preferred)
  • 2+ years of experience in at least 1 scripting language (JS, Python, Perl, Shell, etc.)
  • Experience developing and implementing audit, balancing and validation processes
  • Advanced SQL skills
  • Proven ability to write high quality code
  • Expertise implementing complex ETL logic
  • Well versed in test-driven development disciplines
  • Strong written and verbal communication skills

 

Industry Experience

  • Manufacturing experience is a plus

 

Leadership

  • Ability to mentor the customer and guide in best practice discipline
             

Similar Jobs you may be interested in ..