Job Title: Databricks Platform Engineer (Azure)
Location: Phoenix, AZ / Dallas, TX
Type: C2C / W2
Work Mode: Hybrid (4 Days Onsite)
Duration: Long Term Contract
Certification : Preferred
Job Description:
We are seeking a highly skilled Databricks Platform Engineer with expertise in Azure Databricks to join our engineering team in either Phoenix, AZ or Dallas, TX. This role demands hands-on experience in managing, securing, and automating Databricks environments, with a focus on performance, scalability, and enterprise-grade deployments.
Key Responsibilities:
- Design, build, and manage Azure Databricks infrastructure including workspaces, clusters, jobs, and notebooks
- Administer Databricks environments (user management, access control, workspace configurations)
- Automate operational tasks using Databricks REST APIs and scripting
- Collaborate with platform and security teams to ensure best practices around scalability and compliance
- Optimize compute resources and performance tuning for big data processing
- Develop infrastructure automation using Terraform, ARM Templates, or other IaC tools
- Support data engineering and machine learning teams by enabling robust and secure environments
Required Skills & Experience:
- 8+ years of experience in cloud-based data platform engineering
- Strong hands-on experience with Azure Databricks
- Deep understanding of Databricks architecture (clusters, jobs, notebooks, workspaces)
- Experience with Databricks user and workspace administration
- Proficiency in Python and/or Scala
- Familiarity with Databricks REST APIs and automation techniques
- Experience with CI/CD pipelines and DevOps integration
- Solid knowledge of Azure services such as ADLS, ADF, Key Vault, etc.
Good to Have:
- Experience in integrating Databricks with other Azure services
- Experience in Terraform for Databricks resource provisioning
- Experience with MLflow, Unity Catalog, or Delta Lake
- Strong communication skills and experience working with cross-functional teams
Certifications:
Databricks Certified Data Engineer Associate or Professional (Preferred)