Role: Data Engineer
Reston, VA
12+Months
Technical Skills:
Years of Exp in Python
Years of Exp in AWS
Years of Exp in Spark
Responsibilities:
- Understanding of Rest APIs, Big Data Processing, Rules Engines to orchestrate the calls to the Rest APIs and other data sources like Kafka, Snowflake, AWS S3
- Strong implementation background and experience with AWS Services ( Preferably : Lambdas, step functions , firehose , Kinesis , EMR , Glue ETL , Dynamodb , Neptune , route53 , Application Load balancers , ECS , SQS )
- Knowledge of Corporate Governance on Cloud service usage and Security measures.
- Write scripts in Spark to process large datasets in parallel
- Strong experience in Python scripting and executing Python scripts
- Strong Data Engineering background with the ability to implement Spark based or AWS Lambda based or glue based ETL jobs
- Experienced or Familiar with Batch Jobs orchestration tools Apache Airflow
Required technologies: Python, AWS, Spark