Apetan Consulting
Rutherford, NJ
Post Resume to
View Contact Details &
Apply for Job
Job Description :

Job Title: Data Engineering & BI Lead (Azure + Databricks)

Location- Remote

Role Summary:

The Data Engineering & BI Lead is responsible for designing, building, and managing scalable data platforms and business intelligence solutions using Azure cloud services and Databricks. This role leads data engineering efforts, ensures data quality and governance, and enables data-driven decision-making through analytics and reporting.

Key Responsibilities:

  • Design and implement data pipelines using Azure services and Databricks
  • Build and maintain scalable data architectures (data lakes, data warehouses, lakehouse)
  • Lead ETL/ELT development using tools like Azure Data Factory and Databricks
  • Develop and optimize data models for analytics and reporting
  • Oversee BI solutions (Power BI or similar) for dashboards and reporting
  • Ensure data quality, integrity, and governance standards
  • Monitor and optimize pipeline performance and costs
  • Collaborate with business stakeholders to translate requirements into data solutions
  • Lead and mentor a team of data engineers and BI developers
  • Establish best practices for coding, testing, and deployment (CI/CD)

Required Skills & Qualifications:

  • Strong experience with Microsoft Azure data services
  • Hands-on expertise in Databricks (PySpark, Spark SQL)
  • Experience with Azure Data Factory, Azure Data Lake, and Synapse Analytics
  • Proficiency in SQL and Python
  • Solid understanding of data warehousing concepts and data modeling
  • Experience with BI tools such as Power BI
  • Knowledge of data governance, security, and compliance
  • Strong leadership and stakeholder management skills

Preferred Qualifications:

  • Bachelor’s or Master’s degree in Computer Science, IT, or related field
  • Azure certifications (e.g., Azure Data Engineer Associate)
  • Experience with real-time/streaming data (e.g., Event Hubs, Kafka)
  • Familiarity with DevOps practices and tools (Azure DevOps, Git)
  • Experience in handling large-scale, distributed data systems

Key Metrics / KPIs:

  • Data pipeline reliability and uptime
  • Query and dashboard performance
  • Data quality and accuracy
  • Time-to-delivery for data solutions
  • Cost optimization of cloud resources

Work Environment:

  • Cross-functional collaboration with data, analytics, and business teams
  • Agile development environment
  • May involve managing distributed or offshore teams

             

Similar Jobs you may be interested in ..