Job Description :
Pyspark Developer,
Woonsocket, RI
6+ Months Contract
Phone + Skype
Description:
need to hire a highly skilled Spark / PySpark Engineer for a position that involves architecting and developing a suite of healthcare applications that will sit on a cloud based, Big Data platform. This role calls for an end to end solution using the latest versions of Spark and PySpark.
This position calls for strong Object Oriented Software Engineers as opposed to candidates that come from database backgrounds. As part of your responsibilities you will be involved in helping to build a CI/CD pipeline including helping to implement test driven development and production deployment frameworks. You will be involved in conversations with infrastructure teams (on-prem & cloud) regarding analytics application requirements (e.g., configuration, access, tools, services, compute capacity, etc
Required skills:
Extensive development skills using Python, PySpark (most important), Hive, Shell Scripting, SQL, Pig, Java / Scala.
Proficient in Map-Reduce, Conda, H2O, Spark, Airflow / Oozie / Jenkins, Hbase, Pig, No-SQL, Chef / Puppet, Git.
Familiar with Platforms such as Hadoop, Spark, Kafka, Kinesis, Oracle, TD.
Familiarity with building data pipelines, data modeling, architecture & governance concepts
Experience implementing ML models and building highly scalable and high availability systems.
Experience operating in distributed environments including cloud (Azure, GCP, AWS etc.
Experience building, launching and maintaining complex analytics pipelines in production.