Job Description :
Job Title: Big Data Lead
Location: San Francisco, CA
Duration: 12+Months
Must have Cloudera or Hortonworks Certification
Job Description:
Candidate needs to be strong on Hadoop, Hive, Impala, Yarn, oozie, scala, Kafka, Python or shell scripting.
Responsibilities
Work as part of a team to design and develop code, scripts, and data pipelines that leverage structured and unstructured data integrated from multiple sources
Develop and implement the technical design and ensure the end result fulfils the customer’s requirements
Develop and implement solutions for disparate source data ingestion, transformation, and database loading
Develop and implement solutions for data quality
Develop and implement solutions to support “Data as a Service (DaaS)” tools and third party applications
Recommend and establish security policies and procedures for the Hadoop environment
Develop and implement various strategic initiatives
Contribute to the development of Architecture Policies, Standards and Governance for the Hadoop and Big Data environment
Lead the data architecture design and review processes, including planning and monitoring efforts, reviewing deliverables, and communicating to management
Look to leverage reusable code modules to solve problems across the team, including Data Preparation and Transformation and Data export and synchronization
Design and develop automated test cases that verify solution feasibility and interoperability, to include performance assessments
Act as a liaison with Infrastructure, security, application development and testing team.
Help drive cross team design / development via technical leadership / mentoring
Experience
Minimum 9+ years of relevant experience in information management, ETL & Business Intelligence
Proven ability to work with varied forms of data infrastructure, including relational databases, Map-reduce/Hadoop, and SQL databases
Experience working with SQL/Hive, Pig, and Spark
Expertise in one or more of the programming languages such as Java, Python etc.
Knowledge of XML (e.g., DTDs, XSDs, XSLT etc), Messaging systems (e.g., Oracle JMS, QPID, ActiveMQ etc) and SQL.
Experience with UNIX and LINUX operating systems
Ability to manage multiple priorities in a fast moving environment.
Good oral and written communication skills.
Work with team across global locations.
Qualification
Bachelors'' degree in Computer Science or related field of study.
Hadoop Certified Developer or equivalent experience (Good to have)