Job Description :
Spark/Streaming Engineer
Location is Ann Arbor, MI
12 months Contract
Phone and Skype
H1’s are fine
JOB SUMMARY
The Spark/Streaming Engineer is responsible for design, implementation, testing and support of various projects and initiatives. This position directly supports our new initiatives and will require designing near real-time pipelines using tools like Apache Spark, Apache Kafka, Apache NiFi and an ever-expanding suite of big data tools.
YOUR ROLE
Hands-on experience with Big Data tools such as Spark, Cassandra, Hive/Drill, Kafka, streaming/event-based architectures
Contribute to the development of event processing pipelines for near real-time analytics use cases
Practical hands-on experience with technologies like Apache Spark, PySpark and other stream processing frameworks
Experience working with data formats such as Apache AVRO, Apache Parquet, and common methods in data transformation
Experience with AWS Big Data technology stack
Understanding of data flows, data architecture, ETL and processing of structured and unstructured data
Current experience using Java development, SQL Database systems, and Apache products
Ability to iterate quickly in an agile development process
Ability to learn new technologies and evaluate multiple technologies to solve a problem
Experience with DevOps tooling – GitLab CI/CD, Docker, Kubernetes
RDBMS and NoSQL experience
EDUCATION, SKILLS & EXPERIENCE
Bachelor’s degree in computer science/engineering or related field, or equivalent experience
4+ years core Java experience: building business logic layers and back-end systems for high-volume pipelines
Experience in architecture and implementation of large and highly complex big data projects.
Strong backend services/API development experience using REST/JSON.
Experience working with data scientists and scaling a variety of models.
Personal characteristics include:
Strong problem solving skills and ability to clearly communicate technical concepts in simple terms.
Passion for learning, continuous improvement, sharing and working in a team environment.
You take pride in writing clean, quality code, but can balance this with business practicality.
You are able to learn quickly and effectively and apply new knowledge, demonstrating the desire to stay in touch with emerging technologies.
Experience mentoring or acting in a lead capacity a plus
Healthcare experience is a major plus