Job Description :
Responsibilities:
Translate application storyboards and use cases into functional applications
Design, build, and maintain efficient, reusable, and reliable ETL processes
Integrate with 3rd party API’s for data consumption
Build complex workflows using open source ETL tools such as Talend
Write data processing and transformation routines using Apache Spark (Scala), Hadoop (Hive), Redshift, and Presto
Identify bottlenecks and bugs, and devise solutions to mitigate and address these issues
Collaborate with other team members and stakeholders
Minimum Qualifications:
Proficient in Scala
Expertise working with high volume, heterogeneous data using distributed systems such as Hadoop
Strong understanding of EMR (Elastic Map Reduce) and other big data technologies (eg. Redshift, Presto, Druid, Mongo, etc
Expertise architecting, building real-time data pipelines with data processing frameworks like Spark, Kafka etc.
Strong knowledge of SQL
Strong knowledge of writing Web APIs and consuming RESTful and SOAP APIs.
Experience with C# or Java
Experience working with cloud technologies such as Amazon Web Services
Proficient understanding of code versioning tools such as BitBucket, GitHub and SVN.
Familiarity with continuous integration tools like TeamCity.
Bachelor''s degree in Computer Science or related field is preferred