Job Description :
Design the detailed architecture consisting of ingestion, processing pipeline, analytics and alerting technology in order to meet the various requirements of the OASIS project as well as the OMU refresh project.

- Able to collaborate and credibly work with highly technical senior developers and other architects to define solutions
- Run proof-of-concepts when needed.
- Written, verbal and presentation skills - able to document a formal design as well as present to senior colleagues

Designing reliable, high performance processing pipelines based on Lambda and Kappa architectures and supporting various analytics needs such as Complex event processing, Realtime log analytics, Search, Query, Visualization

Expertise with specific capabilities/limitations of components in the processing architecture in order to select the right technology for the right purpose and maximize its use

Ingestion & message bus: Kafka, Flume, Configurable parsing etc

Expert knowledge of Hadoop Ecosystem - including foundational technologies such as HFDS, Hbase, Impala, Kudu & Spark. Cloudera Hadoop would be advantage

Stream processing - Kafka Streaming, Spark Streaming and other streaming technologies

Search: Elastic, Solr, etc

Visualization: Grafana, Kibana, etc


Client : Financial client

             

Similar Jobs you may be interested in ..