Job Description :
Primary skills – Big data, Spark, AWS, Hadoop.

Job Description:

Demonstrated experience in building data pipelines in data analytics implementations such as Data Lake and Data Warehouse
At least 2 instances of end to end implementation of data processing pipeline
Experience configuring or developing custom code components for data ingestion, data processing and data provisioning, using Big data & distributed computing platforms such as Hadoop, Spark, and Cloud platforms such as AWS or Azure.
Hands on experience developing enterprise solutions using designing and building frameworks, enterprise patterns, database design and development in 2 or more of the following areas
End to end implementation of Cloud data engineering solution AWS (EC2, S3, EMR, Spectrum, Dynamo DB, RDS, Redshift, Glue, Kinesis)
End to end implementation of Big data solution on Cloudera/Hortonworks/MapR ecosystem
Frameworks, reusable components, accelerators, CICD automation
Languages (Python, Scala) Proficiency in data modelling, for both structured and unstructured data, for various layers of storage
Ability to collaborate closely with business analysts, architects and client stake holders to create technical specifications
Ensure quality of code components delivered by employing unit testing and test automation techniques including CI in DevOps environments.
Ability to profile data, assess data quality in the context of business rules, and incorporate validation and certification mechanism to ensure data quality
Ability to review technical deliverables, mentor and drive technical teams to deliver quality technical deliverables.
Understand system Architecture and provide component level design specifications, both high level and low level design

Similar Jobs you may be interested in ..