Job Description :
Job Title: Azure Data Engineers
Location: Dallas, TX and Atlanta, GA
Job Description:
Our Client is looking for data engineer experience developing and deploying=
applications on Microsoft Azure Databricks platform. This individual shoul=
d be self-motivated to drive solutions and proactively create (in sufficien=
t detail) the documentation required to support and describe technical solu=
tions. The candidate will work with cloud architects to establish pipeline =
to Azure data stores using either NiFi or Data Bricks. Will build templates=
for engineering processes in cloud. NiFi experience preferred.
Responsibilities
* The data engineer will be responsible for development, deployment=
, maintenance, diagnostics and support of spark ETL jobs on the MS Databric=
ks platform.
* Will streamline code, rationalize datasets and tables to arrive a=
t single source of truth
* Will build detailed architecture diagrams or Entity Relationship =
Diagrams for all workflows and processes
Required Qualifications:
* 3 years of hands on experience with cloud automation and scriptin=
g in an Azure environment.
* Experience in development of apache Spark code using PySpark or S=
cala
* Experience in using Azure Databricks Platform
* Experience in using Azure Data Factory to call Azure Databricks n=
otebook activities
* Experience in using Git based repositories
* Good fundamental knowledge about distributed computing, RDBMS and=
Dimensional modeling concepts
* Exposure to Azure DevOps
* Exposure to Azure Storage (Blob/Data Lake Store), Azure Catalog, =
Databricks Delta