Job Description :
*Job Title: Big data Engineer - USC, GC Only*
*Job Location: San Mateo, CA *
*Duration: Fulltime*
*Experience: 8 Years*
This hands-on position is responsible for helping to design and develop
cutting-edge big data systems to enable the XX Group to
better understand and serve our customers. This position will engage in an
international
engineering collaboration with the teams developing global-scale
infrastructure supporting
acquisition, analysis, and machine learning using data coming from future
vehicle releases of
theXX Group . Products impacted by this role range from customer insights
informing new services to the most advanced
autonomous driving systems in the industry.
*Responsibilities:*
Help design and implement key components for highly scalable, distributed
data collection
and analysis system built for handling petabytes of data in the cloud.
Analyze and support requirements from machine learning specialists,
focusing on the most important topics in the rapidly evolving automotive industry.
Help move architecture and implementation through the automotive
development pipeline,from research to deployment in millions of cars on the road.
Work with engineers from other divisions contributing to this analytics
system, contributing to best practices in backend infrastructure and distributed computing topics.
Required Skills and Experience:
Hands-on development with key technologies including Scala, Spark, and
other relevant
distributed computing languages, frameworks, and libraries.
Experience with distributed databases, such as Cassandra, and familiarity
with the key issues
affecting their performance and reliability.
Experience using high-throughput, distributed message queueing systems
such as Kafka.
Familiarity with operational technologies, including Docker (required),
Chef, Puppet,
ZooKeeper, Terraform, and Ansible (preferred
An ability to periodically deploy systems to cloud environments such as
AWS.
Mastery of key development tools such as GIT, and familiarity with
collaboration tools such as
Jira and Confluence or similar tools.
Overall, 3+ years of industry experience developing backend components
with a recent focus
on big data systems.
Bachelors degree or equivalent required.
*Desired Skills and Experience:*
Experience with the SMACK stack (Spark, Mesos, Akka, Cassandra, Kafka) as
a holistic
system, and the important considerations for running this system at scale.
Experience or understanding of developing machine/deep learning systems
in a distributed
environment.
Conventional backend development experience with Java and common
frameworks (Spring,
Play