Job Description :

TECHNOGEN, Inc. is a Proven Leader in providing full IT Services, Software Development and Solutions for 15 years.

 

TECHNOGEN is a Small & Woman Owned Minority Business with GSA Advantage Certification. We have offices in VA; MD & Offshore development centers in India. We have successfully executed 100+ projects for clients ranging from small business and non-profits to Fortune 50 companies and federal, state and local agencies.


Please send me the resumes to and call me at

Title: BigData Engineer
Duration: 12 Months Plus
Client Location: Carlsbad, CA REMOTE

Responsibilities:

  • Build data streams to ingest, load, transform, group, logically join and assemble data ready for data analysis / analytics/ reporting , build data streams to ingest, load, transform, group, logically join and assemble data ready for data analysis / analytics/ reporting.
  • Pipeline data using Cloud: Databricks , AWS Big Data Services etc.
  • Responsible to write Pyspark code using DataBricks to connect databases, AWS services to transform data.
  • Design/Implement QA framework within the Data-Lake. Design and implement test strategies, write test cases, design/implement test automation
  • Responsible for maintaining integrity between Data Lake, databases.
  • Work on Data Lake/Delta Lake Data Pipeline to take data across from the source all the way to the consumption layer.
  • Maintain knowledge and proficiency of current and upcoming hardware/software technologies. Mentor junior staff in ramping up analytical and technical skills.
  • SSAS + ETL and Data Engineering experience rather than just Data Engineering skill

Requirements:

  • A bachelor's degree from an accredited college in Computer Science or equivalent.
  • Strong knowledge of Databricks Data Lake/Delta Lake developments
  • Strong knowledge of AWS data related services (DMS, Glue, EMR, S3, Athena, Lambda, Redshift, DynamoDB, KMS).
  • Strong knowledge of Python and Pyspark (Hive).
  • Strong database/relational/non-relational concepts required.
  • Strong analytical and problem solving skills.
  • Databricks, oracle, SQL , Hadoop, Kafka, Spark, Scala (preferred).
  • Must have 5+ years of Data Systems/Warehouse/Lakes/Equivalent system experience with multiple OS platforms
  • Must have 3 + years of Python and Pyspark.
  • Must have 3 + years of AWS or alternative cloud systems.
  • Must possess or develop business knowledge of how customer transactions reflect the business logic that drive the existing or future code.
  • Must possess or develop ability to converse with the business, development, operations, carriers, vendors, etc.
  • Strong experience of different architectural components comprising the middle-ware is required.
  • 1+ year of Matallion/IBM Data Stage or equivalent ETL tool.
             

Similar Jobs you may be interested in ..