KENFILL
Texas City, TX
Post Resume to
View Contact Details &
Apply for Job
Job Description :
Position: Data Engineer PySpark/Databricks/Spark Dataframes/SparkSQL etc

Rate: DOE
Start date: ASAP
Duration: 6 months plus possible ext.
Location: Portland, OR

Interview process: WebEx live

Application specifications:
in Databricks, which has a free version
written in PySpark
Python and Spark Data Transformation.
SparkSQL, PySpark, Jupyter, Databricks, Analytics.
using reproducible approach
implementing an ELT process with Spark Dataframes (not Pandas)
incrementally obtains data from an external data source,
loads data incrementally to a Delta table in Databricks
produces a simple summary aggregate report of the process execution
optional, but encouraged: unit test, logging, SparkSQL, external local library/notebook for functions
The candidate should be fluent in English for both written and verbal communication.