Job Description :
Hadoop Administrator

Location: Chicago, IL



Key Responsibilities

1. Manage large scale multi-tenant Hadoop cluster environments residing on AWS.

2. Handle all Hadoop environment builds, including design, capacity planning, cluster setup, performance tuning and ongoing monitoring.

3. Perform high-level, day-to-day operational maintenance, support, and upgrades for the Hadoop operating system, workstations and servers.

4. Creation of key performance metrics, measuring the utilization, performance and overall health of the cluster.

5. Capacity planning and implementation of new/upgraded hardware and software releases as well as for storage infrastructure.

6. Research and recommend innovative, and where possible, automated approaches for system administration tasks.

7. Ability to closely calibrate with product managers and lead engineers.

8. Ensure existing data/information assets are secure and adhering to a best in class security model.



Technical Experiences Required

· Possess at least 3 years of managing a multi-tenant production Hadoop environment;

· At least 2 year of experience leveraging Cloud Services (IaaS, PaaS AWS Certification is preferred;

· Proven experience with identifying hardware and software related issues.

· Should have at least 1 year of experience with a number of different real-time integration tools such as Apache Flink, Kafka, Storm or Flume.

· Experience with Hortonworks DataFlow (NifI, Kafka, Storm, Atlas) and Hortonworks DataPlatform (Yarn HDFS) is preferred;

· A deep understanding of Hadoop design principals, cluster connectivity, security and the factors that affect distributed system performance.

· Experience with Ansible or other Infrastructure as Code software

· Expert experience with at least one if not most of the following languages; python, java, scala, spark or bash.

· Knowledge of best practices related to security, performance, and disaster recovery.

· Experience with complex networking infrastructure including firewalls, VLANs, and load balancers.



Personal Experiences Required

· Proven ability to lead multiple high priorities initiative with aggressive timelines leveraging an agile framework.

· Comfortable performing in a fast paced, dynamic and ambiguous business environment.

· Ability to concentrate on a wide range of loosely defined complex situations, which require creativity and originality, where guidance and counsel may be unavailable.

· Demonstrated a product mindset with an ability to set forward thinking and direction.

· Excellent listening and communication skills.

o Must be able to communicate technical information clearly, able to ‘translate’ between diverse groups of technical and non-technical individuals.

o Ability to synthesize large amounts of complex data into meaningful conclusions and present recommendations to a vast array of individuals.

o Partner with business units in order to improve the effectiveness of business decisions.

o Ability to structure documents to effectively communicate with senior leadership and drive alignment and decision making.

· Strong problem solving skills with the ability to offer why one technical solutions is more preferable than another.

· Ability to analyze and interpret complex concepts.

· Ability to lead a group of employees (directly or indirectly

· Ability to work in a team environment in a collaborative manner.

· Proven track record of sound, effective decision making.

· Bachelor’s Degree required, Master’s Degree Preferred.