Top Skills : Databricks, Python, ADF , Spark
Skills & Experience
· Should have 08+ years of experience with last 4 years in implementing Cloud native end-to-end Data Solutions from ingestion to consumption to support variety of needs such as Modern Data warehouse, BI, Insights and Analytics
· Should have experience in architecture and implementing End to End Modern Data Solutions using Azure and advance data processing frameworks like Databricks etc.
- Strong hands on in Pyspark and Apache Spark.
- Build Orchestration Layer in Databricks Workflows.
- Build CICD for Databricks in Azure Devops.
- Experience In extracting logic and from on prem layers like SSIS, Stored procedures, Informatica, Vertica, Apache Hudi, Filesystems.. etc into pyspark
· Strong knowledge of cloud native data platform architectures, data engineering and data management
· Good knowledge of popular database and data warehouse technologies from Snowflake Microsoft Azure SQL Data Warehouse
· Demonstrated knowledge of data warehouse concepts. Strong understanding of Cloud native databases, columnar database architectures
· Ability to work with Data Engineering teams, Data Management Team, BI and Analytics in a complex development IT environment.
· Good appreciation and at least one implementation experience on processing substrates in Data Engineering - such as ETL Tools, Kafka, ELT techniques
· Data Mesh and Data Products designing, and implementation knowledge will be an added advantage.