Job Description :
Position: Integration Architect with NiFi/Spark Experience
Location: Rosemont, IL
Duration: 12 Months
Roles and Responsibilities:
Looking for an Integration Solution Engineer/Architect with NiFi and Spark experience to help build out a next generation data integration platform. This position will be developing new processors/POC within NiFi and establishing new data flows/troubleshooting existing data flows to the various a new NiFi-based data integration platform. This position will also be creating SparkSQL/Custom Spark Solutions for Data Transformation Services. This position is anticipated to be more architect/development focused while the current team focuses on maintenance activities on legacy platform.
Basic Qualifications:
10-15 years of experience in Java software development, BI-ETL development, Hadoop development.
Experience with NiFi, including establishment of new data flows and development of Custom NiFi Processors for Data Ingestion.
Experience with Apache Hadoop ecosystem - Sqoop, Hive, Spark Core, SparkSQL for Data Cleansing, Enrichment and transformation services
Experience working with one of Azure data storage services (e.g. BlobStorage), Azure Data lake Store, Azure SQL Data warehouse.
Experience with the Hortonworks Data Flow (HDF) and Hortonworks Data Platform (HDP
Experience with data serialization such as Avro, Parquet,ORC.
Experience with Data Modeling, Cognos BI, Cloud based Infrastructure design.