Job Description :
*Big Data Engineer*

*Location: Bradenton, FL*

*Exp: 6-8 years*

* W2 Only - EAD''s, GC, Citizen*

*JD:*

Summary

The Data Engineer will be responsible for building, maintaining data
pipelines and data products to ingest, process large volume of structured /
unstructured data from various sources.
The Data engineer will work on analyzing the data needs, migrating the data
into an Enterprise data lake, build data products and reports.
The role requires experience with building real time and batch based ETL
pipelines with strong understanding of big data technologies and
distributed processing frameworks with.Skill Needs
o Expertise working with large scale distributed systems (Hadoop,
Spark
o Strong understanding of the big data cluster, and its
architecture
o Experience building and optimizing big data ETL pipelines.
o Strong SQL skills and is comfortable operating with relational
data models and structure.
o Capable of accessing data via a variety of API/RESTful services.
o Experience with messaging systems like Kafka.
o Experience with No SQL databases. Neo4j, mongo, etc.
o Expertise with Continuous Integration/Continuous Delivery
workflows and supporting applications.
o Exposure to cloud environments and architectures. (preferably
Azure)o Ability to work collaboratively with other teams.
o Experience with containerization using tools such as Docker.
o Strong knowledge of Linux and Bash. Can interact with the OS at the
command line and create shell scripts to automate workflows.
o Advanced understanding of software development and
collaboration, including experience with tools such as Git.
o Excellent written and verbal communication skills, comfortable
presenting in front of non-technical audiences. Essential Responsibilities
include but not limited to:
o Design and develop ETL workflows to migrate data from varied
data sources including SQL Server, Netezza, Kafka etc. in batch and
real-time.
o Develop checks and balances to ensure integrity of the ingested data.
o Design and Develop Spark jobs as per requirements for data
processing needs.
o Work with Analysts and Data Scientists to assist them in
building scalable data products.
o Designs systems, alerts, and dashboards to monitor data products in
production
             

Similar Jobs you may be interested in ..