Job Description:
Responsibilities:
The Big Data Developer / Data Engineer will provide expertise in a wide range of technical areas, including but not limited to: Cloudera Hadoop ecosystem. Routine responsibilities include:
· Conduct POCs (proof of concept) and pilots to try new solutions and provide summary of results
· Participate in detailed technical design, development, implementation and support of Big Data applications
· Maintain standards compliance and ensure development artifacts are in alignment with patterns/ frameworks designed by software engineering teams
Qualifications:
• 9+ Years’ experience
• Bachelor's degree in Computer Science or related field
• Familiarity with Spark programming paradigms (batch and stream-processing). Understanding of different data abstraction objects used in spark for different use cases, use of optimal data format and other optimization techniques.
• Strong programming skills in at least one of the following languages: Java, Scala. Familiarity with a scripting language like Python as well as Unix/Linux shells
• Strong knowledge of writing optimized Spark and Hive sql and experience to tune poor performing queries.
• Outstanding programming and debugging skills. Strong knowledge of common algorithms and data structures.
• Experience with Java and/or Scala, preferably with Spring framework. 3+ years preferred.
• Strong experience with SQL and relational databases like PostgreSQL, MySQL, SQL Server or Oracle.
• Familiarity with one or more stream processing / queuing technologies like Spark Streaming, Kafka, Kinesis, Flink, etc. preferred.
• Familiarity and prior experience with Agile / Scrum development methodologies
• Prior extreme Programming (XP) disciplines including paired programming and Test-first/Test Driven Development (TDD)
• Prior Experience deploying to cloud platforms, preferably Pivotal Cloud Foundry or Cloud Foundry
• Prior experience in Continuous Integration/Continuous Delivery tools and pipelines such as Jenkins, Maven, Gradle, etc.