TECHNOGEN, Inc. is a Proven Leader in providing full IT Services, Software Development and Solutions for 15 years.
TECHNOGEN is a Small & Woman Owned Minority Business with GSA Advantage Certification. We have offices in VA; MD & Offshore development centers in India. We have successfully executed 100+ projects for clients ranging from small business and non-profits to Fortune 50 companies and federal, state and local agencies.
Please send me the resumes to and call me at
Title: BigData Engineer
Duration: 12 Months Plus
Client Location: Carlsbad, CA REMOTE
Responsibilities:
- Build data streams to ingest, load, transform, group, logically join and assemble data ready for data analysis / analytics/ reporting , build data streams to ingest, load, transform, group, logically join and assemble data ready for data analysis / analytics/ reporting.
- Pipeline data using Cloud: Databricks , AWS Big Data Services etc.
- Responsible to write Pyspark code using DataBricks to connect databases, AWS services to transform data.
- Design/Implement QA framework within the Data-Lake. Design and implement test strategies, write test cases, design/implement test automation
- Responsible for maintaining integrity between Data Lake, databases.
- Work on Data Lake/Delta Lake Data Pipeline to take data across from the source all the way to the consumption layer.
- Maintain knowledge and proficiency of current and upcoming hardware/software technologies. Mentor junior staff in ramping up analytical and technical skills.
- SSAS + ETL and Data Engineering experience rather than just Data Engineering skill
Requirements:
- A bachelor's degree from an accredited college in Computer Science or equivalent.
- Strong knowledge of Databricks Data Lake/Delta Lake developments
- Strong knowledge of AWS data related services (DMS, Glue, EMR, S3, Athena, Lambda, Redshift, DynamoDB, KMS).
- Strong knowledge of Python and Pyspark (Hive).
- Strong database/relational/non-relational concepts required.
- Strong analytical and problem solving skills.
- Databricks, oracle, SQL , Hadoop, Kafka, Spark, Scala (preferred).
- Must have 5+ years of Data Systems/Warehouse/Lakes/Equivalent system experience with multiple OS platforms
- Must have 3 + years of Python and Pyspark.
- Must have 3 + years of AWS or alternative cloud systems.
- Must possess or develop business knowledge of how customer transactions reflect the business logic that drive the existing or future code.
- Must possess or develop ability to converse with the business, development, operations, carriers, vendors, etc.
- Strong experience of different architectural components comprising the middle-ware is required.
- 1+ year of Matallion/IBM Data Stage or equivalent ETL tool.