Job Description :
Job Title: Lead Data Engineer
Location: Merrimack, NH
Duration: 6+ Months to Start
As a Lead Data Engineer, you’ll work closely with the Data Engineering team developing the scalable data pipelines that support advanced analytics and Data Science solutions.
Job Responsibilities
Design, build, and implement scalable streaming data pipelines and ETL frameworks to increase data access and decrease analysis and decision times across the organization
Own software throughout the entire development life-cycle – design, code, test, automate & deploy
Share ideas to improve our product and processes, and provide feedback
Experience & Education
15+ years of experience in defining data architecture solutions and establishing common data capabilities for enterprises
Proven experience in creating actionable Data and Analytics strategies for Compliance, Risk, Financial Intelligence business functions
Experience in defining technology blueprints, roadmaps and collaboratively defining solutions and enabling architecture capabilities
Experience in tool selection, conducting rapid PoCs and recommending use case appropriate technologies
6+ years experience building distributed solutions in Spark, MapReduce and other MPP system with associated data models and datastores (e.g., Redshift, Cassandra, HBase, Parquet)
2+ years of experience working with AWS Cloud data engineering stack including EC2, S3, EMR, Kinesis, Glue and other AWS Services
Hands-on experience with Apache Ni-Fi, Kafka, Python, Spark preferably on AWS
Experience with structured/unstructured/semi-structured data ingestion and processing
Experience with automation and deployment (Jenkins, CloudFormation,Chef etc
Experience writing high quality code in Python and one another OOP language (Java, Scala, C++, Go, etc
Experience working with RDBM systems, particularly familiarity with SQL
Solid Experience in optimizing the Hive queries using Partitioning and Bucketing techniques, which controls the data distribution, to enhance performance.
Experience in working with UNIX shell scripts.
Production development of event-based applications using frameworks such as Kinesis, Kafka, Spark Streaming, or similar
Familiarity with machine learning techniques, continuous deployment pipelines and tools.
Desire to work across internal teams to identify requirements and iterate on solutions
Debug complex production issues across various levels of the tech stack
Prefer bachelor’s degree or above in Computer Science or related field