Design, develop and Unit Test real-time and batch data ingestion pipelines using pySpark Design, develop and Unit Test real-time and batch data processing/transformation pipelines using pySpark Design, develop and Unit Test real-time and batch target data load pipelines using pySpark Create , schedule and run Databricks jobs |