Competent Systems
Alpharetta, GA
Post Resume to
View Contact Details &
Apply for Job
Job Description :
Big Data Engineer

Cranberry Township, PA

Fulltime


Required Skills and Knowledge:

Expert knowledge of data architectures, data pipelines, real time processing, streaming, networking, and security
Proficient understanding of distributed computing principles
Advanced knowledge of Big Data querying tools, such as Pig or Hive
Expert understanding of Lambda Architecture, along with its advantages and drawbacks
Proficiency with MapReduce, HDFS
Experience with integration of data from multiple data sources and multiple data types

Basic Qualifications:

Bachelor s Degree in Software Engineer
12+ years experience in software engineering with 2+ years using public cloud
6+ Experience developing ETL processing flows using MapReduce technologies like Spark and Hadoop
4+ years experience developing with ingestion and clustering frameworks such as Kafka, Zookeeper, YARN
4+ years experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
2+ years experience with spark structured streaming
4+ years experience with various messaging systems
2+ years experience with Kafka
1+ years of DevOps experience
1+ years benchmarking experience
             

Similar Jobs you may be interested in ..