Job Description :
Job Title : Big Data Engineer
* Preferred location will be Emeryville, CA for all these positions.
We are open to consider candidates working out of Warren, NJ/Dallas, TX or
Atlanta, GA. Post travel ban candidate has to relocate to locations based on
the preference which were initially agreed at the time of submission.
* Please share the skill matrix at the time of submission. Attached
Excel file has skill matrix template for different tracks.
* Please set the expectations with the candidates that at the time of
interview(internal/customer) they will be given coding test. Most of the
positions even its lead/Engineer level candidates will be still doing hands
on coding for these positions.
At ZapLabs, we work to build and improve a platform that helps real estate
professionals work effectively, and helps delight home buyers and sellers
with an excellent experience. We do that by combining great technology with
great people - and we''re looking for a Senior Big Data Engineer to join our
team.
What you''ll do:
As a Senior Big Data Engineer, you''ll be responsible for designing and
building high performance, scalable data solutions that meet the needs of
millions of agents, brokers, home buyers, and sellers. You''ll design,
develop, and test robust, scalable data platform components. You''ll work
with a variety of teams and individuals, including product engineers to
understand their data pipeline needs and come up with innovative solutions.
You''ll work with a team of talented engineers and collaborate with product
managers and designers to help define new data products and features.
SKILLS:
Highlighted is mandatory
BS in Computer Science, Engineering, or related technical
discipline or equivalent combination of training and experience
3+ years core Java experience: building business logic layers and
back-end systems for high-volume pipelines
Experience with spark streaming and Scala
Experience with NiFi
Experience with high-speed messaging frameworks and streaming
(kafka, akka, reactive)
Experience with DevOps tools (GitHub, TravisCI, Jira) and
methodologies (Lean, Agile, Scrum, Test Driven Development)
Current experience developing and deploying applications to a
public cloud (AWS, GCE)
Current experience in Spark, Hadoop, MapReduce and HDFS
Understanding of data flows, data architecture, ETL and processing
of structured and unstructured data
Current experience using Java development, SQL Database systems,
and Apache products
Experience with GoLang a plus
Experience with data science and machine/deep learning a plus
Ability to work quickly with an eye towards writing clean code
that is efficient and reusable
Ability to build prototypes for new features that will delight our
users and are consistent with business goals
Ability to iterate quickly in an agile development process
Ability to learn new technologies and evaluate multiple
technologies to solve a problem
Excellent written and verbal communication skills in English
Strong work ethic and entrepreneurial spirit
Nice to haves:
Experience mentoring or acting in a lead capacity
What you''ll do:
* You''ll be responsible for designing and building high performance,
scalable data solutions that meet the needs of millions of agents, brokers,
home buyers, and sellers.
* You''ll work with other Data Engineers for build out of Next
Generation Data Ingestion Platform.
* You''ll design and develop data ingestion pipelines for batch and
real-time streaming of data from in-house OLTP systems and third-party data.
* You''ll work with team to design and develop Data Lake to store and
process 10s of terabyte of data.
* You''ll work with team to design Data Lake CLI to manage Data Lake
Storage and Access.
* You will design and develop ETL pipelines to process data in data
lake for descriptive and prescriptive reporting.
* You''ll develop ETL data pipelines to build Enterprise Data Models
for Property, Agent, Broker, office and other master entities.
* You will design and develop CI/CD process for continuous delivery in
AWS Cloud.
* You''ll design, develop, and test robust, scalable data platform
components.
* You''ll work with a variety of teams and individuals, including
product engineers to understand their data pipeline needs and come up with
innovative solutions.
SKILLS:
* Bachelor''s in Computer Science, Engineering, or related technical
discipline or equivalent combination of training and experience
* 10+ years core Java experience: building business logic layers and
back-end systems for high-volume pipelines
* 7+ Years of solid programming experience in Java/Scala with clear
understanding of Data Structure and Alogrithm.
* 2 years experience with spark streaming and scala
* 2 years experience in Spark
* 2 years experience using Java 8 development, functional programming,
and SQL Database systems
* 2 years experience using AWS Data Services: any combination of EMR,
Glue, Athena, S3, CloudWatch, Lambda or IAM
* 2 yeears experience with high-speed messaging frameworks and
streaming (kafka)
* 1 year exp. with data architecture, ETL and processing of structured
and unstructured data
* 2 years experience with DevOps tools (any combination of GitHub,
TravisCI or Jira) and methodologies (Lean, Agile, Scrum, Test Driven
Development)
Excellent written and verbal communication skills in English
             

Similar Jobs you may be interested in ..