Job Description :
Job Title: Big Data Developer
Location: Manassas, VA
Duration: Long Term

Scope of Work:
SWIFT’s Tooling organization delivers an Operations And Support Intelligence System ("OASIS" This system provides insight into customer and operational data, using a variety of big data and search technologies.


Based on your experience and interest, you will be part of a DevOps team involved in developing/enhancing a pipeline of log data, developing/enhancing the Data Lake or delivering search/analytics/dashboard capabilities deriving intelligence from the data lake.
Technologies used include syslog-ng, Apache Flume, Cloudera Hadoop, Elastic stack, Kafka & Spark, and development activities may include use of Java, Scala, Python


Requirements
Working with application development teams who write applications which generate logs, and with Operations teams who operate those applications, and through development of an understanding of the business of SWIFT customers, the successful candidate will drive and oversee the following:
understanding of content and value of logs generated by messaging or service management tools
creation of syslog-ng or Filebeat configuration to transport logs from generating system into pipeline
development of parser(s) for the logs, using logstash GROK or custom parser developed in Java
development of configuration for a generic metrics engine defining fields to be used for calculating metrics of interest e.g. transit times
development of dashboards for useful visualization of insights found in those logs – Kibana mostly
working with development & operations staff, conduct end-to-end tests of the entire pipeline from generating system to dashboard
Use predictive analytics classification algorithm (like linear, logistic, random forest, SVM, TF-IDF etc) to predict


Desire Languages/Technologies – Java, Python, Ansible, Logstash, Flume, Fluentd, Elastic Search, Kibana, Syslog-NG, Filebeat, Metricbeat, Data Science (Some knowledge)


Skills/Requirements:
Level: (Junior/Specialist/Expert)
Jr = 1-5 yrs / Spec=5-10yrs / Exp= >10 + yrs
Python
Specialist
Java
Specialist
Logstash
Junior
ElasticSearch
Junior
Apache Spark
Junior
Data Analysis
Specialist
Software Engineering
Specialist
Big Data Infrastructure
Specialist