Job Description :
The Hadoop Engineer shall lead the Big Data team. The consultant will play the role of technical lead and provide professional services to support the long term IT strategy and planning to include high level analysis, professional reports and presentations, and mentoring, and support. Existing application leverages Hadoop framework. We are looking for an engineer who can provide vision and technical leadership on the project. The resource will review the existing the architecture and recommend new technologies as appropriate. Hadoop engineer will closely work with business units and recommend changes to provide value to business.
Provide technical leadership, develop vision, gather requirements and translate client user requirements into technical architecture.
Design and implement an integrated Big Data platform and analytics solution
Design and implement data collectors to collect and transport data to the Big Data Platform.
Implement monitoring solution(s) for the Big Data platform to monitor health on the infrastructure
4+ years of hands-on Development, Deployment and production Support experience in Hadoop environment.
4-5 years of programming experience in Java, Scala.
Proficient in SQL and relational database design and methods for data retrieval.
Knowledge of NoSQL systems like HBase or Cassandra
Hands-on experience in Cloudera Distribution 5.x
Hands-on experience in creating, indexing Solr collections in Solr Cloud environment.
Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Pig, Solr, MR, Spark, Spark SQL.
Must have experience with developing Hive QL, UDF''s for analyzing semi structured/structured datasets.
Must have experience with Spring framework and experience in QPL (Query Processing Language
Hands-on experience ingesting and processing various file formats like Avro/Parquet/Sequence Files/Text Files etc.
Hands-on experience working in Real-Time analytics like Spark/Kafka/Storm
Experience with Graph Databases like Neo4J, Orient DB
Must have working experience in the data warehousing and Business Intelligence systems.
Expertise in Unix/Linux environment in writing scripts and schedule/execute jobs.
Successful track record of building automation scripts/code using Java, Bash, Python etc. and experience in production support issue resolution process.
Experience in building Client models using MLLib or any Client tools.
Basic knowledge of Solr or any search engine.