Job Description :
Job Title Hadoop Architect
Working Location , PA
Duration: Long Term
Client: Direct
Rate:C2C
Important Skills : Spark, Scala, Kafka, Sqoop, SQL
Position Summary The dx Team has responsibility including Data Engineering in DevOps Model for Comcast; one of the major goal is to harmonize the data ingestion and consumption layer across Comcast. Creating enterprise data sources as a single version of truth is a goal of dx Team.
The Big Data Software Developer will develop (code/program), test, debug -ETL (Extract/Transform/Load) of data to answer technically challenging business requirements (complex transformations, high data volume
All work needs to be documented.
Employees at all levels are expected to:
Understand our Operating Principles; make them the guidelines for how you do your job
Own the customer experience - think and act in ways that put our customers first, give them seamless digital options at every touchpoint, and make them promoters of our products and services
Know your stuff - be enthusiastic learners, users and advocates of our game-changing technology, products and services, especially our digital tools and experiences Win as a team - make big things happen by working together and being open to new ideas
Be an active part of the Net Promoter System - a way of working that brings more employee and customer feedback into the company - by joining huddles, making call backs and helping us elevate opportunities to do better for our customers
Drive results and growth
Respect and promote inclusion and diversity
Do what's right for each other, our customers, investors and our communities
Core Responsibilities
Analyzes and determines data integration needs
Evaluates and plans software designs, test results and technical manuals using Big Data (Hadoop) ecosystem
Reviews literature, current practices relevant to the solution of assigned projects in Data Warehousing and Reporting areas
Programs new software using Spark, Scala, Kafka, Sqoop, SQL
Supports existing and new applications and customization of current applications
Responsible for administration, monitoring, tuning, optimizing, governing Hadoop Cluster and Hadoop components
Design and implement high availability options for critical component like Kerberos, Ranger, Ambari, Resource Manager
Collaborate with various cross functional teams; infrastructure, network, database and application for various activities: deployment of new software, environment, capacity, uplift
Performance tuning of Hadoop clusters and various Hadoop components and routines
Monitor job performances, file system/disk-space management, cluster and database connectivity, log files, management of backup/security and troubleshooting various user issues
Hadoop cluster performance monitoring and tuning, disk space mana