Job Description :
Role : Big Data Test engineer
Location :Indianapolis IN
Duration :12 + Months
Assurance / Digital : Big Data Testing (Hadoop Module), Digital : BigData and Hadoop Ecosystem - MapR
Big Data, Horton works, Hive, Pig, MapR, Nifi, Informatica BDM, Oracle Golden Gate
Development of Horton Work frame work Requirement gathering Suggestion for the eco system
Role description:
Experience with any of the following Hadoop distributions: Cloudera/MapR/Hortonworks
Proficient understanding of distributed computing principles
Management of Hadoop cluster, with all included services using Apache Ambari, Cloudera Manager, MapR control system
Proficiency with Hadoop v2, MapReduce, HDFS,YARN,Tez
Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming, ni-fi.
Working Experience of Big Data querying tools, such as Pig, Hive, Oozie and Impala• Working Experience of Apache Spark
Experience with integration of data from multiple data sources• Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
Good understanding of Lambda Architecture, along with its advantages and drawbacks
Integrating any Big Data tools and frameworks required to provide requested capabilities.
Monitoring performance and advising any necessary configurations & infrastructure changes.
Debugging and Resolving Hadoop (YARN/Map Reduce/Spark etc issues.
Integration knowledge of Informatica BDM
Essential Skills
Knowledge of various ETL techniques and frameworks, such as Flume
Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
Experience with various messaging systems, such as Kafka or RabbitMQ
Advise and implement Data lake security using Kerberos/Knox/Ranger/SSL etc.
Training/Certification on any Hadoop distribution will be a plus