Job Description :
Position : Sr. Data Engineer Location : Denver, CO (need to relocate once the COVID situation improves) Duration : Contract Please let me know if you are interested in this position contact, Email: Job description: Evangelist for data engineering function leveraging big data processing framework. Creation and optimization of data engineering pipelines for analytics projects. Support data and cloud transformation initiatives Support our software engineers and data scientists Contribute to our cloud strategy based on prior experience Understand the latest technologies in a rapidly innovative marketplace Independently work with all stakeholders across the organization to deliver enhanced functionality Skills - Experience and Requirements A successful Data Engineer will have the following: Must be from a Data warehouse/Big data background. Experience in advanced Apache Spark processing framework, spark programming languages such as Scala/Python/Advanced Java with sound knowledge in shell scripting. Experience in working with Core Spark, Spark Streaming, Dataframe API, Data set API, RDD APIs & Spark SQL programming dealing with processing terabytes of data. Specifically, this experience must be in writing "Big Data" data engineering jobs for large scale data integration in AWS. Advanced SQL experience using Hive/Impala framework including SQL performance tuning Experience in writing spark streaming jobs integrating with streaming frameworks such as Apache Kafka or AWS Kinesis. Create and maintain automated ETL processes with special focus on data flow, error recovery, and exception handling and reporting Gather and understand data requirements, work in the team to achieve high-quality data ingestion and build systems that can process the data, transform the data Knowledge of using, setting up and tuning resource management frameworks such as a standalone spark, Yarn, or Mesos. Experience in physical table design in Big Data environment Experience working with external job schedulers such as autosys, AWS data pipeline, airflow, etc. Experience working in Key/Value data store such as Hbase Experience in AWS services such as EMR, Glue (serverless architecture), S3, Athena, IAM, Lambda, and Cloud watch is required.CST provides its clients with complete, cost-effective, end-to-end personnel solutions across a range of industrial domains. CST''s mission is to empower businesses around the world to make better, faster operational decisions.
             

Similar Jobs you may be interested in ..