Job Description :
Sr. Data Engineer
Los Gatos, CA
6 Months
Summary:
We are looking for a Senior Data Engineer to partner with our analytics team and security teams to build efficient data pipelines, both in batch and streaming context.
The data pipelines power near real time alerting and detection, insightful data products, dashboards and analysis.
You will support and contribute to our objectives on securing our customer and corporate data.
The ideal candidate will have background in data engineering and experience in enabling near real time pipelines with large scale data sets.
Responsibilities:
Develop subject-matter expertise in the security domain .
Be a trusted partner to the analytics team and security engineering team.
Engineer efficient, adaptable and scalable data pipelines to process unstructured big data .
Enable analytics by building robust data sets that can power various analytic techniques like regression, classification, clustering etc.
Experience:
You have a passion for intuitive data model and you are an expert in distributed data processing patterns .
You have programming expertise in Java, Python and/ or Scala.
You have strong SQL skills .
You are familiar with ETL frameworks such as Airflow .
You have experience in engineering data pipelines using big data technologies (Hive, Presto, Spark, Flink etcc) on large scale unstructured data sets.
You are proficient with AWS Cloud primitives .
You are familiar with distributed data stores like Druid and ElasticSearch.
You excel at taking vague requirements and crystallizing them into scalable data solutions .
You have exceptional communication skills .