Role : Cloud Data Architect
Location : Onsite in NYC
Hire Type: Full time / Permanent
Job Summary
As a cloud data architect, you are responsible for designing and implementing scalable and secure data solutions using cloud technologies. You will work with business stakeholders and data analysts to understand the data requirements and translate them into logical and physical data models. You will also leverage your expertise in cloud services, data engineering, and data governance to optimize the performance, reliability, and quality of data pipelines and platforms.
Years of experience needed
Experience Level: 15+ Years
At least 5 years of experience in data architecture, data engineering, or data analysis
Candidate should have extensive experience in architecting and engineering Data Warehouses, Data Lake, ODS and OLTP data platforms and processing layers on cloud and on-prem platforms.
Good knowledge in Data Management areas and expertise in building components in spark for data file ingestion and data pipelines that validates, standardize, processes, cleanse, transform the data and store.
Expertise in Data Engineering on Bigdata Technologies both on in-premise and cloud Platforms, specifically AWS, Azure & GCP etc.
Knowledge of data modeling techniques and best practices, such as ER diagrams, dimensional modeling, and data normalization
Familiarity with data quality, data security, and data governance standards and frameworks
Ability to communicate effectively with technical and non-technical stakeholders
Technical Skills:
Tools & Technologies Required:
Architect BI / DW / DL platform for large enterprises
Experience in data modeling for oltp/olap applications
Entity Relationship modeling, domain modelin, Dimensional Modeling, Modeling data in NoSQL, GraphDB , concept modeling for GraphDB etc
Hadoop Ecosystem HDFS, Hive, Hbase, Spark, Hue /Ambari, Impala, Sqoop, Kafka
Hadoop Distributions Cloudera / Hortonworks / MAPR / Databricks
AWS Data Platform- EMR, EC2, Kinesis, Redshift, RDS, DMS, Cloud watch
Experience working on Databricks
Apache Airflow, Messaging / Apache Kafka etc
Secondary Skills:
ETL Technologies Informatica / Abinitio / Datastage / Talend
Databases Oracle / DB2 / MySQL / Postgres / Teradata / MongoDB / Cassandra / Snowflake
Process Skills:
Should be well versed with Bigdata Tools & Technologies including Hadoop eco system components like Hive, Hbase, Sqoop, Kafka, Hue & Spark with proficiency in either one of programming languages Java/Python/Scala.
Good conceptual understanding of SMP & MPP system and data processing on these platforms, using lambda & kappa architecture, streaming & batch processing.
Experience in working on AWS ecosystem as a whole and should have experience in migrating ETL pipelines from Talend / Informatica/Abinitio to Hadoop / Glue / Spark.
Expertise in handling structured, semi-structured (json,xml etc ) and unstructured data, data schema drift etc and understanding various big data file formats like AVRO, Parquet, ORC
Should possess knowledge in modern and traditional database systems like NoSQL vs RDBMS.
Knowledge of Distributed File systems is must.
Behavioral Skills :
Should be competent to design and develop architectures for Data Migration, Data Ingestion, Data Storage, Build Data Lakes, creating various layers in Data Lake, ETL using Hadoop tools like Spark, and AWS tools like Glue and EMR.
Should be well versed with Performance tuning of ETL pipelines including spark.
Must have architecture and design exp in building highly scalable enterprise grade applications.
Expert designing data integrations using ETL and other data integration patterns
Advanced knowledge of Business Processes
Candidate should be able to guide the client and management about what tools and technologies should be applied in a particular scenario for best case utilization and cost optimization.
Certifications Needed:
Education qualification: B.Tech, BE, BCA, MCA, M. Tech or equivalent technical degree from a reputed college
Skills
PRIMARY COMPETENCY : Data Engineering PRIMARY SKILL : Data Architect PRIMARY SKILL PERCENTAGE : 100