Job Description :

Position Summary:    

Work Location  :   Wellesley, MA, Hartford, CT, or NYC

Job Title  :  Data Engineer 

Duration  :  6-12+ Months Initial


Must Haves:

  • 7+ Years of Experience 
  • Unix/Linux
  • Hadoop
  • Python/Scala
  • Spark
  •  CI/CD – GitLab


Primary Responsibilities:

  • Understand the business objectives and analyze, dissect system requirements and technical specifications
  • Interpret data, analyze results using statistical techniques and provide ongoing reports
  • Develop and implement databases, data collection systems, data analytics and other strategies that optimize statistical efficiency and quality
  • Explore the automation scope wherever possible
  • Acquire data from primary or secondary data sources and maintain databases / data systems
  • Identify, analyze, and interpret trends or patterns in complex data sets
  • Display strong technical knowledge in product analysis and debugging
  • Willingness to learn new technologies and adapt to product needs
  • Work independently with little or no supervision
  • Provide technical mentoring for team members, troubleshooting
  • Understand design concepts and product architecture. Propose solutions for performance and security issues
  • Identify Process Gaps and drive Initiatives to address process Gaps
  • Partner with product owners and developers to identify areas for improved efficiencies
  • Share and communicate ideas both verbally and in writing to staff, business sponsors, managers, and technical resources in clear concise language that is appropriate to the target audience
  • Participate in communities of practice to share knowledge, learn, and innovate
  • Research and implement tools that support delivery
  • Assesses and interprets customer needs and requirements
  • Solves moderately complex problems and/or conducts moderately complex analyses
  • Analyzes and investigates
  • Provides explanations and interpretations within area of expertise
  • Query structured and unstructured data and perform exploratory data analysis for further advanced modeling
  • Guide and implement Optimization techniques across projects


Required Qualifications:

  • 5+ years of IT experience in architecture of  Big Data Technologies and  ETL and other automation techniques
  • Good experience on Unix shell command and scripting
  • Experience of writing SQL and Complex SQLs
  • Good knowledge of databases and Big Data especially Hive, HBase and Spark
  • Fair knowledge on the Having experience on Hadoop Applications like Spark, Scala, Hbase, Hive, PIG, Sqoop
  • Knowledge on tools like Kibana and Splunk
  • Knowledge on Kafka Streaming
  • Knowledge on implementing Data Integrations projects from sourcing to auditing and implements controls for each stage of Integration
  • Deeply analytical
  • Good communication and presentation skills
  • Problem solving skills with the ability to think laterally, and to think with a medium term and long term perspective

Similar Jobs you may be interested in ..