Job Description :
Position:Data Engineer

Location:New York City, NY

Duration:Permanent


Required Skills:
5+ years of software engineering experience with 2+ years in data engineering plus a Master’s degree in Computer Science (or related field) or equivalent professional experience combined with a Bachelor’s degree • Extensive experience with Python and experience with either Java or Scala
Experience with Apache Spark and/or Apache Hadoop
Understanding of Apache Kafka and its use cases
Experience with build and dependency management tools (Gradle, sbt, Maven, npm)
Ability to build cloud native apps on Amazon Web Services
Good experience with agile development processes like Scrum and Kanban • Insist on automating everything (testing, Continuous Integration, Continuous Delivery) • Preferred experience in traditional business intelligence (e.g. star schemas, OLAP cubes) • Experience in working on the command line in a Unix/Linux environment
Experience with one or more NoSQL databases such as Cassandra, DynamoDB, MongoDB, Neo4J, or RDF triple stores • Experience with data pipeline tools such as Apache Airflow, Luigi, AWS Data Pipeline • Experience scaling machine learning pipelines and SparkML specifically
             

Similar Jobs you may be interested in ..