Job Description :

Position:  Big Data Engineer

Location: Charlotte, NC or Concord, CA

 

Job Description: 
The AI technology team is looking for a highly motivated and experienced Data Engineer. The right candidate will have expert level experience in SQL, Python, Spark, Data Warehousing technologies and Hadoop Big Data environment.  Should be independently able to develop, test and deploy pyspark code based on requirements.

 

 Responsibilities :

1. Validate data engineering requirements and work with Data Scientists in requirements gathering/validation phase.

2. Create visualization charts to depict the various table joins, join methodology and access methods

3. Develop modularized feature/transformation code using Spark-SQL, python on Jupyter notebook.

4. Develop applications using agile methodologies and Act as agile SME for the team

5. Tune pyspark code, SQL code, python data structures etc to ensure optimal performance.

6. Perform unit testing

7. Support feature engineering refresh jobs deployment to production

8. Interact with various teams to conduct daily work

9. Basic understanding of API development using Python

10.Basic understanding  of Machine Learning in the context of feature engineering.

 

Experience :

5+  years of Hadoop, Spark, python experience, SQL experience

 

             

Similar Jobs you may be interested in ..