Sr. Data Engineer
Jersey City, NJ
$130K/yr
Plaxonic Technologies
Fulltime
Visa Independent
Job Summary:
Developer with strong technical ability with 10+ years of experience in Java/J2EE design and development
Experienced in working on medium to large enterprise projects, preferably in financial services
Should have knowledge on Apache Spark framework.
Must have knowledge on HBase
Should have basic knowledge on Bigdata Cluster and operations
Person should have worked in Agile/DevOps Environment
Good communication skills
Job Background/Context:
The position is based in US and is required to focus on delivery of the work, ensuring a robust design
This role may report to the technology team lead based anywhere in Pune or New York or elsewhere
Candidate should be able to work independently and should be self-motivated
Candidate might be required to work with vendors or third parties in joint delivery teams
The role requires application of technical skills and knowledge of the business to develop solutions to meet business needs
As part of large, geographically distributed team(s), the candidate may have to manage stakeholders across multiple functional areas
The position requires analytical skills in order to filter, prioritize and validate potentially complex material, technical or business or otherwise, from multiple sources
Key Responsibilities:
Experience with developing software that processes, persists and distributes data via relational and non-relational technologies
Employ standards, frameworks and patterns while designing and developing components
Develop high quality code employing software engineering and testing best practices
Converse with various data provider and consumer applications in their languages/terminologies
Partner with database developers to implement ingestion, orchestration, quality/reconciliation and distribution services
Skills Required:
Experience with developing software that processes, persists and distributes data via relational and non-relational technologies:
Strong pySpark/Java Skills
Experience in design and development of batch/real time Spark processing pipelines.
Knowledge of Spark framework Core Spark, Spark Data Frames, Spark streaming, pyspark
Knowledge of Bigdata Cluster and operations.
Good to Have:
Have basic experience in Data Preparation Tools
Experience with CI/CD build pipelines and toolchain Git, BitBucket, TeamCity, Artifactory, Jira
Experience with testing concepts (TDD, BDD) and frameworks (Cucumber, Selenium, FluentLenium, Junit)
Experience with container technologies (Docker, Pivotal Cloud Foundry) and supporting frameworks (Kubernetes, OpenShift, Mesos)
Knowledge of Operating Systems and familiar with shell scripting
Databricks Certified Associate Developer for Apache Spark, Python Institute - Certified Associate in Python Programming, Oracle Database SQL Certified Associate, Oracle Certified Associate - Java SE 8 Programmer, Cloudera - CDP Data Analystcs