Job Description :
Title: Data Ingestion Engineer Location: Toronto/ Montreal Duration: Contract/ Full time Job Description- Experience with Spark, Kafka, Hadoop, Pig, Hive on any Hadoop distribution (HDP/ CDH/ MapR) . Develop streaming/ real- time based complex event processing on Hadoop framework. Proficient in SQL , in addition to one or more of modern programming language such as Java , Scala , Python etc. Experience working with databases (SQL, NoSQL ) Experience in building streaming/ real time framework using Kafka & Spark . Experience working with data validation cleaning, and merging Manage data quality, by reviewing data for errors or mistakes from data input, data transfer, or storage limitations. Develop spark applications/ map reduce jobs. Perform data management to ensure data definitions, lineage and source are suitable for analysis.