Job Description :
Job Title: Data Engineer
Location: New York City, NY (1 New York Plaza, 21st Floor | New York, NY 10004)
Duration: FTE role / C2H
Manager Note: We are looking for candidate having practical hands-on experience with python, good database experience with SQL/PL, exposure to Spark/Impala/Hadoop ecosystem and some basic exposure to cloud databases.
Job Description:-
Operations Technology Data and Analytics team is looking for an intermediate level Data Engineer to help define and develop the current and future data strategy for Operations. The team is responsible for providing technology solutions to Morgan Stanley’s global operations functions including Settlements, Payments, Tax, Books and Records, Reconciliations and Regulatory reporting. The data strategy includes various OLTP and OLAP systems, data lakes, and reporting and analytics platforms across various domains, business lines and geographies. The future data strategy involves optimal use of cloud based solutions and phased migration of existing technologies/platforms to the cloud.
Skills Required:-
5 to 10 years of strong experience with relational SQL including Postgres/DB2/SQL Server etc.
Strong experience with scripting languages: Python, Java, Scala, Shell, Perl etc.
Experience building and optimizing ‘big data’ data pipelines, architectures and data sets using big data tools: Hadoop, Spark, Kafka, etc.
Experience with stream-processing systems: Storm, Spark-Streaming, etc.
Experience with data modelling of relational and dimensional data models
Experience in Multi-Parallel Processing (MPP) database or Greenplum 5.X is a big plus.
Good analytical, problem solving, communication and interpersonal skills.
Self-motivated and ability to work consistently and efficiently to achieve end goals.
Interest in learning the business functionality
Experience supporting and working with cross-functional teams in a dynamic environment.
Skilled in identifying, designing, and implementing internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Experience in building processes supporting data transformation, data structures, metadata, dependency and workload management.
Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
Skills Desired:
DB2/Oracle to Postgres/Greenplum migration experience.
DB Monitoring and automation.
Experience with NoSQL databases Hive/MongoDB/Cassandra etc.
Knowledge of databases and platforms like Greenplum, Hadoop and MPP Database environment.
Experience with public cloud databases: Azure SQL database, Azure Synapse, Databricks
Knowledge of Financial Services data
Experience with data pipeline and workflow management tools: Luigi, Airflow, etc.
Experience with Data Science and Machine Learning Platforms like Dataiku DSS, Alteryx, DataRobot.