Job Description :
Job Title: Big Data Developer
Location: Manassas, VA
Duration: Long Term

Scope of Work:
SWIFT’s Tooling organization delivers an Operations And Support Intelligence System ("OASIS" This system provides insight into customer and operational data, using a variety of big data and search technologies.

Based on your experience and interest, you will be part of a DevOps team involved in developing/enhancing a pipeline of log data, developing/enhancing the Data Lake or delivering search/analytics/dashboard capabilities deriving intelligence from the data lake.
Technologies used include syslog-ng, Apache Flume, Cloudera Hadoop, Elastic stack, Kafka & Spark, and development activities may include use of Java, Scala, Python

Working with application development teams who write applications which generate logs, and with Operations teams who operate those applications, and through development of an understanding of the business of SWIFT customers, the successful candidate will drive and oversee the following:
understanding of content and value of logs generated by messaging or service management tools
creation of syslog-ng or Filebeat configuration to transport logs from generating system into pipeline
development of parser(s) for the logs, using logstash GROK or custom parser developed in Java
development of configuration for a generic metrics engine defining fields to be used for calculating metrics of interest e.g. transit times
development of dashboards for useful visualization of insights found in those logs – Kibana mostly
working with development & operations staff, conduct end-to-end tests of the entire pipeline from generating system to dashboard
Use predictive analytics classification algorithm (like linear, logistic, random forest, SVM, TF-IDF etc) to predict

Desire Languages/Technologies – Java, Python, Ansible, Logstash, Flume, Fluentd, Elastic Search, Kibana, Syslog-NG, Filebeat, Metricbeat, Data Science (Some knowledge)

Level: (Junior/Specialist/Expert)
Jr = 1-5 yrs / Spec=5-10yrs / Exp= >10 + yrs
Apache Spark
Data Analysis
Software Engineering
Big Data Infrastructure