Data Engineer

Job Description: Role: Data Engineer Duration: Full time Location: Ideally candidate to be based in east coast or willing to relocate to NJ/PA area after July 2021. Core Skills Data Engineering, Big Data, Pyspark, Spark SQL and Python Candidate with prior Palantir Cloud Foundry OR Clinical Trial Data Model background is preferred. Major accountabilities: Responsible for Data Engineering, Foundry Data Pipeline Creation, Foundry Analysis & Reporting, Slate Application development, re-usable code development & management and Integrating Internal or External System with Foundry for data ingestion with high quality. Have good understanding on Foundry Platform landscape and it's capabilities Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues. Defines company data assets (data models), Pyspark, spark SQL, jobs to populate data models. Designs data integrations and data quality framework. Design & Implement integration with Internal, External Systems, F1 AWS platform using Foundry Data Connector or Magritte Agent Collaboration with data scientists, data analyst and technology teams to document and leverage their understanding of the Foundry integration with different data sources - Actively participate in agile work practices Coordinating with Quality Engineer to ensure the all quality controls, naming convention & best practices have been followed Desired Candidate Profile : Strong data engineering background Experience with Clinical Data Model is preferred Experience in o SQL Server ,Postgres, Cassandra, Hadoop, and Spark for distributed data storage and parallel computing o Java and Groovy for our back-end applications and data integration tools o Python for data processing and analysis o Cloud infrastructure based on AWS EC2 and S3 7+ years IT experience, 2+ years' experience in Palantir Foundry Platform, 4+ years' experience in Big Data platform 5+ years of Python and Pyspark development experience Strong troubleshooting and problem-solving skills BTech or master's degree in computer science or a related technical field Experience designing, building, and maintaining big data pipelines systems Hands-on experience on Palantir Foundry Platform and Foundry custom Apps development Able to design and implement data integration between Palantir Foundry and external Apps based on Foundry data connector framework Hands-on in programming languages primarily Python, R, Java, Unix shell scripts Hand-on experience in AWS / Azure cloud platform and stack Strong in API based architecture and concept, able to do quick PoC using API integration and development Knowledge of machine learning and AI Skill and comfort working in a rapidly changing environment with dynamic objectives and iteration with users. In This Role You Will Often: Collaborate with offshore team members and onshore stakeholders over the design and development of product features and enhancements. Provide technical oversight over feature development, ensure code reviews, and review and approve pull requests. Remove roadblocks and/or bottlenecks for the offshore team and resolve questions in real time. Work with the offshore team and onshore product owners to help move projects to completion. Develop product features and enhancements. Participate in grooming and design sessions to understand requirements and determine solutions. Act as the communication facilitator to ensure that onshore and offshore patterns and practices are understood, documented and followed. Represent the voice of the offshore team in onshore technical discussions and decision making.