Spk Technologies inc
Livermore, CA
Post Resume to
View Contact Details &
Apply for Job
Job Description :
PositionID: 64002533
Client: Gainwell Technologies
Position: Azure Databricks Engineer
Responsibilities:
  • Data pipeline Development: Design, develop, and maintain scalable data pipelines and ETL processes using Azure Databricks, Azure Data Factory, and other Azure services.
  • Data Processing: Implement and optimize Spark jobs, data transformations, and data processing workflows in Databricks.
  • Integration: Integrate data from various sources, such as databases, APIs, and streaming data, into the Databricks environment, often using tools like Azure Data Factory or Databricks Workflows.
  • Performance Optimization: Enhance the performance of data processing tasks by optimizing Spark jobs, managing cluster resources, and implementing best practices for data storage and retrieval.
  • Collaboration: Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure the solutions meet business needs.
  • Data Governance: Implement data governance policies and ensure data security and compliance with relevant regulations.
CI/CD Integration: Utilize Azure DevOps and CI/CD best practices to automate the deployment and management of data pipelines and infrastructure.
             

Similar Jobs you may be interested in ..