Job Description :
Responsibilities:
The ideal candidate responsibilities will include the following:
· Involved in the architecture design of data platform; plan and execute data platform architecture changes
· Be aware of the expense of Azure resources, and be cost effective when building data platform (clusters and storage)
· Manage Databricks workspace, clusters and users
· Monitor Databricks cluster performance; Provide recommendations to improve performance
· Manage and performance tuning of SQL Databases
· Manage Blob Storage and ADLS
· Manage ADF linked services; debug ADF pipeline failures
· Data transfer between different databases / storage
· POC Data Platform new features
· Create support tickets to Microsoft
· Provide feedback to Databricks for new features
Basic Qualifications:
· 3 to 5 years of experience in the following: Databricks & Spark, Azure SQL DB / DW, Azure Data Factory, Blob Storage, Data Lake Store (ADLS)