Job Description :
Sr. Big Data Engineer
Philadelphia, PA
Long Term
Job Description:
· Large Volumes of data Processing and Developing Automated ETL Processes
· High Throughput Low Latency Systems
· Large Scale Data Modelling from Big Data perspective
· Big Data Structures in Python
· Workflow Management Tools such as Luigi, Apache Airflow, Snowflow, Apache NiFi, UC4 or similar
· Spark2
· JSON/Parquet Data formats
· Real time streaming with either Spark Streaming of Kafka
· Develop API’s/Tools that help share data with Enterprise
· Debugging in Big Data environment
· Version control Systems: Git/github/BitBucket etc.