Job Description :
ROLE: Data Engineer
Location:New York, NY And Greensboro, NC And Chicago, IL
Market rate
REQUIRED SKILLS:
4+ years of experience in software development, a substantial part of which was gained in a high-throughput, decision-automation related environment.
2+ years of experience in working with big data using technologies like Spark, Kafka, Flink, Hadoop, and NoSQL datastores.
1+ years of experience on distributed, high throughput and low latency architecture.
1+ years of experience deploying or managing data pipelines for supporting data-science-driven decisioning at scale.
A successful track-record of manipulating, processing and extracting value from large disconnected datasets.
Producing high-quality code in Python.
Passionate about testing, and with extensive experience in Agile teams using SCRUM you consider automated build and test to be the norm.
Proven ability to communicate in both verbal and writing in a high performance, collaborative environment.
Follows data development best practices, and enjoy helping others learn to do the same.
An independent thinker who considers the operating context of what he/she is developing.
Believes that the best data pipelines run unattended for weeks and months on end.
Familiar with version control, you believe that code reviews help to catch bugs, improves code base and spread knowledge
PREFERRED SKILLS:
Experience with large consumer data sets used in performance marketing is a major advantage.
Familiarity with machine learning libraries is a plus.
Well-versed in (or contributes to) data-centric open source projects.
Reads Hacker News, blogs, or stays on top of emerging tools in some other way
Data visualization
Industry-specific marketing data
Technologies of Interest:
Languages/Libraries – Python, Java, Scala, Spark, Kafka, Hadoop, HDFS, Parquet.
Cloud – AWS, Azure, Google
RESPONSIBILITIES/EXPECTED DELIVERABLES:
Your responsibilities will include:
Design, construct, install, test and maintain highly scalable data pipelines with state-of-the-art monitoring and logging practices.
Bring together large, complex and sparce data sets to meet functional and non-functional business requirements.
Design and implements data tools for analytics and data scientist team members to help them in building, optimizing and tuning our product.
Integrate new data management technologies and software engineering tools into existing structures.
Help build high-performance algorithms, prototypes, predictive models and proof of concepts.