Job Description :
Hi
Please find the below JD
CONTRACTOR JOB DESCRIPTION
DataLake Technical Consultant
Customer Location (City and State)
Camden, NJ
Duration of role (6 months, 1 year, or 1+years)
5 Months
Technology – M/F, ODBA, SQL DBA, M/F DBA, App Support, Incident Management, etc
Apache NiFi, AWS Cloud tools such as S3, Athena, Redshift, Cloud9, Data Pipeline, Kafka, Glue, Scala/Pyspark, EMR(AWS & Big Data technologies)
Job responsibilities (bullet points) – Describe what the job duties are. What will the hired candidate be doing in this role?
o The essential functions of the job
o List of 5-10 responsibilities
o Begin each with a present tense action verb; “develop software”
o Include only higher-level responsibilities
o Arrange in logical order
Design data ingestion and data processing pipelines using technologies such as Apache NiFi, Amazon MSK, Amazon EMR, PySpark, Scala, Athena, Glue and Data Pipelines to bring the data from Hana into Datalake that resides on Amazon S3.
Design data ingestion and data processing pipelines to move the data from S3 into data warehouse built using Amazon Redshift.
Develop design of data visualization layer that will be built using Amazon Redshift for Amazon Quicksight.
Build data ingestion and data processing pipelines using the above-mentioned technologies to bring the data from Hana into Amazon S3.
Working Knowledge of Python, JSON, Scala, Pyspark
Knowledge to use to python programs to integrate with Spark and utilize AWS Cloud tools such as Data Pipeline, EMR etc.
Knowledge of JSON programs to utilize EMR cluster to use the metadata to write pipelines
Working knowledge on converting orc to parquet
Build data ingestion and data processing pipelines to move the data from S3 into Amazon Redshift.
Build datasets in Amazon Quicksight utilizing the Redshift Data. Create Analysis and publish dashboards in Quicksight using the datasets.
Qualifications: Please list the must have skills.
Bachelors in Computer Science
Years of Experience Required
3 to 4 years in AWS Cloud (Senior)