Job Description :
Collaborating with BU Partners in designing, developing, and implementing data movement and integration processes in preparation for data analysis, financial reporting & visualization/dashboarding
Building ingestion pipelines to extract data, translate & analyze data sets involving in-house and 3rarty data sources such as Site Catalyst and Segment
Collect, process, and interpret large data sets, and identify and analyze features of interest such as key performance metrics for channel marketing strategies, what if scenarios, aggregations, filtering, statistical modeling
Assist in data model documentation, data dictionary, data flow, and data mapping for analysts
Create new metrics and develop tools for monitoring and reporting
Participate in complete end-to-end data warehousing & analytics work, including design, reviews, development, unit tests, and deployment. Research the latest tools required for data engineering & analytics tasks
Enable timely reporting and analytics of new product features by ensuring that those features are properly instrumented, building pipelines, and surfacing insights
Develop frameworks and implement technologies that will improve the time to insights.
Skills:
5+ years of Big Data Engineering expertise - Hadoop and Cloud Technology AWS
Hive, Pig, MapReduce
Expertise with Data Warehousing/Data Engineering using Data integration
Expertise with building injection pipelines
Connecting to 3rarty data sources using Java anython
Expertise with AWS - installation, configuration, leveraging AWS services like RedShift, EMR, etc.
Expertise with Kafka/similar, Spark and Sqoop
NoSQL database experience