Job Description :

Sr Data Engineer (Datastage Developer)
San Antonio, TX
About the Data Engineer Role
You are a savvy Data Engineer eager to join a growing team of analytics experts. In this role, you will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. You will support our software developers, database architects, data analysts and data scientists on initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. You are self-directed and comfortable supporting the data needs of multiple teams, systems and products. You are excited by the prospect of optimizing and innovating existing data architecture to support the next generation of products and data initiatives.

Responsibilities

This requires understanding of the activities involved in legacy ETL code to migration of to Cloud using latest technology stack like data build tool and Snowflake database. Leverage existing/build new processes/framework to for quicker and control migration. This involves and not limited to:

  • Analysis and Reverse Engineering of Data Stage code.
  • Develop ELT data pipeline to migrate applications using DBT and Snowflake framework
  • Scheduling and dependency management of data pipeline with proper auditing
  • Register and Build ingestion pipeline with test automation of code.
  • Implement test automation in the data pipeline.
  • Build detailed technical design, conduct analysis, development of applications and proof of concepts.
  • Responsible for translating functional ETL requirements into a technical design document, building the DataStage jobs and sequences to pull data from the source into staging tables, from staging tables into historical tables using Change Data Capture methodologies, and then from historical tables into dimensional data marts for reporting.
  • Communicate progress across organization and levels from individual contributor to senior executive. Identify and clarify issues/problems that need action and drive appropriate decisions and actions relating to system integrations, compatibility, and multiple platforms.
  • Research, test, build, and coordinate the conversion and/or integration of new products based on client requirements. Design and develop new software products or major enhancements to existing software.
  • Overseeing the testing, implementation, maintenance, and enhancement of the applications.
  • Consult with project teams and end users to identify any further application requirements.
  • Troubleshoot and resolve software issues and respond to suggestions for improvements and enhancements.
  • Instruct, assigns, directs, and checks the work of other software developers on development team.
  • Participate in development of software user manuals.
  • Design and build applications as per the given requirement adhering to the guidelines.
  • Ensure consistent unit testing and support the technology infrastructure team implementation.
  • Notify clients once updates have been made.

Requirements

  • Bachelor's Degree or master's degree in Computer Science, Mathematics, Statistics.
  • 5+ years of experience as a DataStage Developer.
  • 5+ Years of complex SQL / PL SQL development experience
  • 2+ years of experience in writing scripts using Unix or Python
  • Experience working in Agile framework
  • 3+ years of experience in Migration from Data Stage to other technologies
  • Working experience on AWS

Data Engineers (Python_Pyspark)
Columbus, OH
About the Data Engineer role:
We are looking for a savvy Data Engineer to join our growing team of analytics experts. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems, and products.
Responsibilities

  • Communicate progress across organizations and levels from individual contributor to senior executive. Identify and clarify the critical issues that need action and drive appropriate decisions and actions. Communicate results clearly and in actionable form.
  • Lead development and ongoing maintenance and enhancement of applications running on Azure Cloud and business intelligence tools.
  • Detailed technical design, conduct analysis, development of applications and proof of concepts
  • Develop microservices, application code and configuration to deliver application
  • Provide technical leadership for development & BI team to deliver on various initiatives.
  • Lead problem resolution tasks, document approach for support mechanisms
  • Ensure all solutions meet Enterprise Guidelines and industry standards/best practices
  • Advise IT and business stakeholders of alternative solutions
  • Ensure optimal system performance across BI & Analytics platforms.
  • Lead the effort to monitor system activity, tune performance and architect solutions to meet future demand.
  • Offer technical guidance to team members and lead design/requirements sessions
  • Benchmark systems, analyze system bottlenecks and propose solutions to eliminate them;
  • Articulate pros and cons of various technologies and platforms and document use cases, solutions and recommendations;
  • Troubleshoot complex system issues and handle multiple tasks simultaneously
  • Ensure all solutions meet Enterprise Guidelines and industry standards/best practices


Experience

  • Bachelor's Degree or master's degree in Computer Science, Mathematics, Statistics.
  • 4+ years of development experience in using Spark to build applications through Python and PySpark
  • 3+ years' hands-on experience developing optimized, complex SQL queries and writing PLSQL code across large volumes of data in both relational and multi-dimensional data sources such as Teradata, Hive, Impala, Oracle, Teradata
  • Experience in deploying and developing application using Azure
  • Experience working with disparate data-sets in multiple formats like JSON, Avro, text files, Kafka queues, and log data and storage like blob/ADLS GEN2
  • 2+ years of strong ETL experience on either Informatica, Ab-Initio, Talend, DataStage, Syncsort
  • Enthusiastic to work on disparate datasets in multiple formats like JSON, Avro, text files, Kafka queues, and Knowledge of software design and programming principles.
  • Experience working in Scrum Agile framework and using DevOps to deploy and manage code.
  • Good communication and team-working skills.
             

Similar Jobs you may be interested in ..