Remote - Pyspark Developer

Remote, Work from Home Remote Work from Home 00000

Date : Nov-09-21

Remote - Pyspark Developer

Remote, Work from Home

Nov-09-21

Work Authorization

US Citizen
GC
H1B
OPT EAD, CPT EAD, GC EAD, L2 EAD, H4 EAD, TN EAD

Preferred Employment

Corp-Corp
W2-Permanent
W2-Contract
1099-Contract
Contract to Hire

Job Details

Experience

:

Midlevel

Rate/Salary ($)

:

Market

Duration

:

long term

Sp. Area

:

Data Warehousing/ETL

Sp. Skills

:

Data Modeler

Consulting / Contract

Remote Work from Home

Required Skills :

Hive, AWS, Cloud Computing, Data Warehousing, ETL, Spark, SQL, Agile, Bamboo, Big Data, Bitbucket, Data Profiling, Data Stage, Data Validation, DB2, D

Preferred Skills :

Domain :

IT/Software

Work Authorization

US Citizen
GC
OPT EAD, CPT EAD, GC EAD, L2 EAD, H4 EAD, TN EAD
H1B

Preferred Employment

Corp-Corp
W2-Permanent
W2-Contract
1099-Contract
Contract to Hire

Job Details

Experience

:

Midlevel

Rate/Salary ($)

:

Market

Duration

:

long term

Sp. Area

:

Data Warehousing/ETL

Sp. Skills

:

Data Modeler

Consulting / Contract

Remote Work from Home

Required Skills :

Hive, AWS, Cloud Computing, Data Warehousing, ETL, Spark, SQL, Agile, Bamboo, Big Data, Bitbucket, Data Profiling, Data Stage, Data Validation, DB2, D

Preferred Skills :

Domain : IT/Software

Apply Now

New York Technology Partners
Iselin, NJ
Post Resume to
View Contact Details &
Apply for Job

Job Description :

Job Title: Pyspark Developer

Location: Remote

Duration: Long Term Contract

Job Description:

Design and Developed real time streaming pipelines for sourcing data from IOT devices, defining strategy for data lakes, data flow, retention, aggregation, summarization for optimizing the performance of analytics products.
Experience in building large scale batch and data pipelines with data processing frameworks in AWS cloud platform using PySpark (on EMR) & Glue ETL
Deep experience in developing data processing data manipulation tasks using PySpark such as reading data from external sources merge data perform data enrichment and load in to target data destinations.
Proficiency with Big Data processing technologies (Hadoop, Hive, or Databricks)
Experience in deployment and operationalizing the code using CI/CD tools Bitbucket and Bamboo
Experience with SQL and relational databases
Strong AWS cloud computing experience. Extensive experience in Lambda, S3, EMR, Redshift
Experience in Data Warehousing applications, responsible for the Extraction, Transformation and Loading (ETL) of data from multiple sources into Data Warehouse
Experience in optimizing Hive SQL quarries and Spark Jobs.
Implemented various frameworks like Data Quality Analysis, Data Governance, Data Trending, Data Validation and Data Profiling with the help of technologies like Bigadata,Data Stage, Spark, Python, Mainframe with databases like Netezza and DB2,Hive & Snowflakes
Experience with creation of Technical document for Functional Requirement, Impact Analysis, Technical Design documents, Data Flow Diagram with MS Visio.
Having experience in delivering the highly complex project with Agile and Scrum methodology.
Quick learner and up-to-date with industry trends, Excellent written and oral communications, analytical and problem-solving skills and good team player, Ability to work independently and well-organized.

Hot Job

Python Developer - Pyspark, Charlotte, NC

Apr-25-24

APN Consulting Inc

($) : BASED ON EXPERIENCE

APN Consulting has an immediate need for a direct client requirement: Python Developer - Pyspark Location - Charlotte, NC (Hybrid) Long-term Contract Key Responsibilities: She was involved in data manipulation using Python scripts Spark Scala which will be useful for faster data processing. Created Parameterized Queries generated Tabular reports Sub reports Cross Tabs Drill down reports using Expressions Functions Charts Maps Sorting the data Defining Data sources and Sub

Hot Job

AWS Solutions Architect, New York, NY

Apr-25-24

Global Soft Systems, Inc

($) : $60k - $130k/year

AWS Solution Architect Remote Required Skill Amazon Web Services (AWS), Java Preferred Qualification Experience with data analysis and using data visualization tools to describe data ? Understanding of big data and real time streaming analytics processing architecture and ecosystems Experience with DataLakes and data warehousing architecture and implementation, including MPP, data streams, hands on experience with source to target mappings and developing ETL code. Software developm

Position: QA Testing Lead Engineer Locat, Alpharetta, GA

Apr-04-24

sapear

($) : 50 - 55 $ per hour

Hi Kindly post the requirements. Position: QA Testing Lead Engineer Locations: 3 Edison Drive, Alpharetta, GA30022 (Hybrid Onsite - 3-4 days onsite - depending on requirement) Duration: 12+ Months Contract Rate: 50 - 55 $ per hour on W2 * Financial Industry, particularly Equities, Options, Fixed Income and Futures trading and market data knowledge Requirements: * Lead a QA team to provide technical guidance and interacting with key business stake holders to plan and prioritize the workload.

[Already Applied]

Hot Job

ETL Architect Talend, Atlanta, GA

Apr-22-24

Intone Networks

($) : USD 75 / Hourly / C2

ETL Architect Talend Location: ATLANTA/CHARLOTTE Required skills: Primary Technical Skills: ETL Architect Talend. Hands-on experience with a wide range of data-related technologies - Data management, Data Integration and design principles using Talend SQL/NoSQL databases, Oracle, Hive, Netezza Big data Hadoop, Sqoop, Hive, Spark XML, Python, AWS Cloud. Data modeling and design, Metadata Management Translating business requirements into an effective data solution

Jr. Data Engineer, Jersey City, NJ

Apr-21-24

APN Consulting Inc

($) : $100000 - $110000 yr

Direct Client Requirement Position: Jr. Data Engineer Location: Jersey City, NJ Type: Full Time/Permanent The Role: The client is looking for a talented and experienced developer to join our development team to support the continued development of our analytics platforms. Besides regular individual contributions, Data Engineers are expected to be deep technical experts in our tech stack and have a breadth of knowledge and experience on technology trends in the industry. Above all

Data Warehouse, Woodbridge, NJ

Mar-27-24

Boston Technology Corporation

($) : Market

Hi, If you are interested please share me your updated profile with the following details mentioned below and attach softcopy of GC and DL Position: Data warehouse Admin Location:Woodbridge NJ Key Responsibilities:Design, implement, and maintain Snowflake data warehouse solutions to fulfill business requirements effectively.Deploy and optimize DBT models and transformations to streamline data processing and facilitate analytics.Manage Fivetran data pipelines to enable seamless data

[Already Applied]

Remote - BTP Quality Assurance (QA) Engi, New York, NY

Apr-02-24

APN Consulting Inc

($) : $60k - $130k/year

APN Consulting has an immediate need for a direct client requirement: Job Title - BTP Quality Assurance (QA) Engineer Duration 12 Months Contract Location - Remote Job Title: SAP Business Technology Platform (BTP) QA Engineer Job Description: We are seeking an experienced SAP Business Technology Platform (BTP) QA Specialist to join our team. The BTP QA Engineer will be responsible for developing and implementing a re-useable approach to BTP Test Automation. You

Full Stack Developer : Java , Snowflake,, Boston, MA

Apr-11-24

Everest Consulting Group

($) : Market

About the Role About the Role: We seek a highly skilled and dynamic Full Stack Developer. In this role you will Analyze all business processes and ensure compliance to all controlled processes according to business requirement. Collaborate with the Product Owner, Scrum Master, Subject Matter Experts and Development team to define and analyze user stories tracked in Jira. Work closely with Product Owner to ensure Technology Initiatives and requirements are reflected in backlog an

[Already Applied]

Hot Job

QA Automation Architect SDET., Alpharetta, GA

Apr-24-24

SAP Gloabl Inc

($) : $55-60

Dear Partners, Job Title:- QA Automation Architect SDET. Location: Alpharetta GA. (On-site in a hybrid model 3 days from the office Duration:- 6+ monthsJob Description: As the QA Automation Architect / Lead SDET, you will lead the design, development, and implementation of automated testing solutions tailored to the unique challenges of the financial domain. You will collaborate with cross-functional teams to ensure the quality and reliability of our software products, leveraging

[Already Applied]

Hot Job

Senior Software Engineer, Burlington, VT

Apr-25-24

Cox

($) : Market

Cox Automotive brand is currently looking for a Senior Software Engineer to join our Automotive Client Insights nVision Product Team. Our Teams: At Cox Automotive, we offer the chance to take a significant role in the digital revolution of the automotive industry. Our software engineers are part of an energetic team that build world-class solutions to benefit the auto buying public, dealers and manufacturers. At Cox Automotive, you'll be immersed in an environment that values your teamwork and