Ab Initio Developer with Python & PySpark
Charlotte, NC (Need Onsite day 1, hybrid 3 days from office).
Position type; W2 contract
Visa type: Any visa independent (No OPT/CPT)
Job Description:
We are seeking a skilled Ab Initio Developer with strong experience in Python and PySpark to join our data engineering team. The ideal candidate will be responsible for designing, developing, and maintaining large-scale data integration and processing solutions, ensuring high performance and data quality.
Responsibilities:
- Develop and maintain ETL workflows using Ab Initio.
-
Collaborate with data analysts and other stakeholders to gather requirements and translate them into effective data pipelines.
-
Implement data processing solutions utilizing Python and PySpark for big data processing tasks.
-
Optimize existing data workflows for performance and reliability.
-
Perform data validation, cleaning, and transformation activities.
-
Troubleshoot and resolve data pipeline issues in production environments.
-
Document technical specifications and develop best practices for data processing.
Requirements:
- Proven experience as an Ab Initio Developer in a data engineering environment.
-
Strong proficiency in Python programming.
-
Hands-on experience with PySpark (Spark with Python API).
-
Good understanding of big data technologies and concepts (Hadoop, Spark, etc.).
-
Familiarity with SQL and relational databases.
-
Experience with data modeling and schema design.
-
Strong problem-solving skills and attention to detail.
-
Excellent communication and teamwork skills.
Preferred, but not required:
-
Experience with cloud platforms such as AWS, Azure, or GCP.
-
Knowledge of other big data tools and frameworks (e.g., Kafka, Hive).
-
Familiarity with orchestration tools like Airflow or Oozie.