Role: Databricks Developer
Location: Houston, TX - prefers candidates in Houston
Onsite 4 days a week
Visa: anything that can work W2 without sponsorship
Overview: Client is building a team to conduct a full-scale Databricks implementation with Priority Power. 3 MDC resources alongside US Solutions Architect: 1 Business/Data Analyst; 2 Data Engineers. Solutions Architect will be responsible for setting up the Databricks environment, starting with the AWS infra/hosting, Databricks workspace (alongside DBX SA), and data migration solution. Engineers will be responsible for building all of the "plumbing" and data ingestion, ETL, modeling.
- We are looking for a Senior Engineer with strong Databricks experience and hands-on skills in Python, PySpark, and SQL.
- They will be responsible for building data ingestion processes/ETL pipelines to ingest data from various sources into Databricks (bronze layer)
- They should have experience in data modeling, transformations and curation of data through medallion architecture.
- Needs cloud experience, AWS preferred, to migrate workflows from AWS to Databricks
- The Engineers will be more-so responsible for getting the data into Databricks, data curation and transformation within Databricks models; MDC team will bring in Data Engineers with some Power BI experience to mock up a couple reports.
- Subsequent phases will be more deeply engaged in the BI aspect and can assess additional resourcing needs at that time.
Must Have:
- Databricks
- Python
- SQL
- Spark/PySpark
- Data warehousing, ETL, modeling fundamentals
- Cloud experience (AWS preferred)
Nice to Have: DLT, Power BI