Job Description :
Role: Sr. Data Engineer w/ Pyspark Location: 100% Remote Length: 12+ months Required: ? Strong verbal and written communication skills to effectively articulate messages to internal and external teams. ? Hands on experience with Azure data platform stack (Azure Databricks, Azure Data Factory, Azure Data Lake storage, Azure DevOps, Azure Functions) ? Experience with dimensional modeling and data warehousing concepts ? Experience with designing, building, optimizing, troubleshooting end-to-end big data pipelines using structured (relational and files) and semi-structured data ? Experience with building metadata driven data processing frameworks ? Strong experience in Spark, SQL, python, Pyspark, shell scripting. ? Ability to work in an agile team. ? Ability to take ownership of a request from initial requirements, design, development and production deployment Nice to have: ? Powershell, scala, java ? Apache Airflow ? Azure EventHubs ? Apace Kafka ? Streaming data , Krishnasree | US IT Recruiter Thought wave software and solutions 314 N, Lake St, Suite 6, Aurora IL 60506 Mobile Email: Website: LinkedIn:
             

Similar Jobs you may be interested in ..