Job Description :
Position: Big Data Integration Engineer
Location: San Francisco, CA
Project Duration: 1+ Years

Job Responsibilities:
1. Responsible for Big Data / data warehouse application design, data quality, development and testing/deployment strategy.

2. Responsible for guiding and validating work of other team members, especially 3rd party vendors and act as a lead technical and subject matter resource for client’s Data Warehousing ecosystem and BI applications.

3. Acts as a lead in identification and troubleshooting processing issues impacting timely availability of data in the data warehouse or delivery of critical reporting within established SLAs.

4. Identifies and recommends improvements in production application solutions or operational processes in support of data warehouse applications and business intelligence reporting (ie, data quality, performance, static and dynamic reports, etc

5. Provides administrative technical support for Big Data/ EDW Applications; assists with documentation and training. Provides expertise (SME) and guidance to other resources in data integration engineering, including 3rd party vendor resources as appropriate.

6. Focuses on the overall stability and availability of the Big Data applications and the associated interfaces, data transport protocols

7. Researches, manages and coordinates resolution of complex issues through root cause analysis as appropriate

8. Establishes and maintains productive relationships with technical leads of key operational sources systems providing data to Big Data platform.

9. Establishes and maintains productive relationships with Architects, Data Analyst, Product management, Operation and infrastructure support areas, such as DBAs and system admins

10. Ensure adherence to established problem / incident management, change management and other internal IT processes

11. Responsible for communication related to any day to day issues and problem resolutions in Big Data applications. Communication may include direct department management as well downstream application contacts and business partners.

12. Ensures comprehensive knowledge transition from development teams on new or modified applications moving to ongoing production support

13. Seeks improvement opportunities in design, solution implementation approaches in partnership with Architects and Operations t team for ensuring the performance and health of the BigData and other EDW Applications

14. Participates in production migrations and upgrades, develops processes for sustaining the Big Data, EDW & BI environment and ensures implementation of the same

15. Ensures timely and accurate escalation of issues to management

Required Skills:
5 + years of technical experience in Enterprise Data Warehouse and Business Intelligence environments
3+ years developing and supporting data integration in Hadoop.
2-3 years developing or supporting applications using Talend, Pig, Hive, Python, Spark, Hbase and Pig.
Possess a strong technology, data, and big data/data warehouse application design background
Experience with various databases, Teradata, HBase is preferred.
Extensive experience with production batch scheduling and monitoring (CAWA, Oozie etc)
Experience with Teradata is a plus.

Technical Skills Desired:
Bachelor’s degree or equivalent experience in Information Technology
Proven experience working with an off shore support team
Previous experience working in a high paced, high pressure agile development environment
Proven analytical and problem solving skills for assessment of complex requirement, production support issues and recommend / implement effective solutions
Previous experience in using IT Ticketing processes for managing incidents and issue resolution and conversant with established Change control / Change management standards
Natural technical and functional curiosity
Excellent communication skills