Job Description :
We currently have the below urgent requirement pls reach me at kpadmaja(AT) with suitable profiles
Role:Data Engineer
Type;FTE
Exp;7+Yrs
UC/GC
Philadelphia, PA or RTP (Research Triangle Park), NC
Azure Data bricks, Talend ,Data Lake, Pyspark, Python, Azure data factory, Powershell
Good understanding of Azure Databricks and the Spark eco-system
Experience of data analysis and transformation using Python (Pyspark)
Experience of setting up Azure data bricks and ingesting data from varied sources.
Experience in creation of jobs, clusters and setting up security in Azure data bricks
Experience of using custom libraries in Azure databricks
Comfortable coding using Python(Scala is added advantage)
Experience years of design and implementation experience in Big Data technologies
Experienced with performance tuning, troubleshooting, and debugging in Azure databricks
Familiarity with database and analytics technologies in the industry including Data Warehousing/ETL, Relational Databases
Ability to manage and work with multiple stakeholders like customer Tech team, Business, Data Scientist
Experience in Pharma industry will be an added advantage
Excellent communication skills
Ingest data using Talend or ADF
Set up Azure databricks environment
Transform data using Azure databricks
Set up RBAC for user access and enable audit logs
Creating analytic workflows that go from ETL and interactive exploration using pyspark.
Create solutions that support a DevOps approach for delivery and operations of services