Job Description :
Responsibilities
Work with others on the team to manage the day-to-day
development activities, participate in designs, design review, code
review, and implementation.
Provide direction to others in analyzing and solving program
and system issues and problems.
Provide technical direction and solutions to other team members.
Thorough knowledge of all stages of the Software Development Life Cycle.
Participate in Requirements Analysis and creation of design
(HLD, LLD Develop prototype to certify the design and approach
Write reusable, testable, and efficient code and unit testing
Create documentation on program development and subsequent
revisions, provide necessary comments within the code to make it
understandable
Conduct System Integration, Support Integration and Performance Testing
Integration of data storage solutions in spark – especially
with AWS S3 object storage
Provide production support: on call and managing defect/minor
enhancement fixes and testing
Actively participate in scrums and code reviews
Need to ensure overall build delivery quality is good and on
time delivery is done at all times
Interact with other team members in a positive and supportive manner
Excellent writing, documentation, and oral communication skills.
Qualifications
Experience with AWS (EMR, Code Commit, EC2, IAM, Virtual
Private Cloud (VPC), RDS, ELB (Elastic load balancing), S3)
Demonstrated proficiency with Airflow, Python/PySpark,
Java/Java Web Services, Spring, REST Web Services, Node.js, React.js
Demonstrated proficiency with database technologies such as
SQL PL/SQL or MySQL.
Strong experience in relational database management systems
such as SQL Server, Sybase and Oracle.
Experience and understanding of software source control
systems, preferably Git or TFS.
Hands on experience in developing ETL data pipelines using
PySpark on AWS EMR
Experience with RESTful APIs / JSON
Hands on experience of XML processing using python
Good understanding of Spark’s RDD API
Good understanding of Spark’s Dataframe and API
Experience and good understanding of Apache Spark Data sources API
Experience of dealing with AWS S3 object storage from Spark
Experience with FacetsTM preferred