Job Description :
Experience
Experience with large scale big data platform and application engineering with recent coding experience in one or more of the following languages: Java, Python
Extensive data experience, including ETL, event streaming batch file processing & transfer, data ingestion, modelling, Big Data such as Hadoop & Spark, database experience (SQL and NoSQL) and reporting & analytics
Experience in developing Big Data Pipeline: Data Ingestion, Data Cataloging, Data Transformation, Data Modeling, Data Distribution
Experience in automation of big data and job migration to Cloud platform
Experience is implementing full stack Cloud and Big Data security
Experience in developing monitoring and logging frameworks
Problem solving and debugging skills
Required Skills:
Big Data on Cloud: AWS/Azure/Google based Data Platform/Lake development experience
Big Data Processing Technologies: Spark/Flink, Nifi, Kafka, Java, Python, Ab Initio
Big Data Storage: Object Store, HDFS, Hive, Presto, Data Warehouse, NoSQL(HBase/Cassandra), SQL
Cloud & Big Data Security: Authentication, Authorization, Data Encryption, Key Management for big data stores