Job Description :
Position : Sr Software Development Engineer/ Big Data Engineer with cloud
Location : San Francisco, CA
Duration : 12+ Months
Local Candidate only
Responsibility
Partner on the design the next implementation of Mastercard secure, global data and insight architecture, building new Stream processing capabilities and operationalizing “Unified Data Acquisition and Processing (UDAP) platform”
Own medium-to-large size data engineering projects
Discover, ingest, and incorporate new sources of the real-time, streaming, batch, and API-based data into our platform to enhance the insights we get from running tests and expand the ways and properties on which we can test
Assist business in utilizing data-driven insights to drive growth and transformation.
Build and maintain data processing workflows feeding Mastercard analytics domains.
Facilitate reliable integrations with internal systems and third-party APIs as needed.
Support data analysts as needed, advising on data definitions and helping them derive meaning from complex datasets.
Prototype new algorithms, experiment, evaluate and deliver actionable insights.
Experiment with new tools to streamline the development, testing, deployment, and running of our data pipelines.
Work with cross-functional agile teams to drive projects through the full development cycle.
Help the team improve with the usage of data engineering best practices.
Collaborate with other data engineering teams to improve the data engineering ecosystem and talent within Mastercard.
Creatively solve problems when facing constraints, whether it is the number of developers, quality or quantity of data, compute power, storage capacity, or just time.
Maintain awareness of relevant technical and product trends through self-learning/study, training classes and job shadowing.
Requirement
python
Linux
Hadoop, Hive:
Spark Experience
cloud Experience
DevOps tools Experience
Shell Experience