Databricks Pyspark Finance Jobs

1 job found

Today

Data Engineer, Finance

SHIELD AI San Diego, CA

San Diego, CA

Today

$97k - $140k/yr Full time Ai/ml Data Engineering Artificial Intelligence Big Data

Apply Now

Job Description

Shield AI is building a governed, dependable data foundation that supports Finance planning, analytics, and AI-enabled decision support. As a Data Engineer, Finance, you will focus on reusable Databricks integrations, semantic-layer infrastructure, and production data services that help power analytics and intelligent Finance workflows. This onsite role is based in San Diego, CA.

Compensation for the position is USD 97,000 - 140,000 per year.

Benefits

Bonus + Benefits + Equity
Pay within the range listed, plus bonus and benefits and equity
Pay within the range listed above with a temporary benefits package (applicable after 60 days of employment)

Responsibilities

Build and operate scalable pipelines and integrations connecting enterprise data to Finance products.
Develop semantic-layer infrastructure and governed data access for analytics, planning, and AI applications.
Set up end-to-end pipeline observability, including data-quality controls and resilient operational practices.
Engineer the services and integrations that support Databricks Genie and other AI-enabled Finance workflows.
Use AI coding assistants to accelerate delivery while validating correctness, security, and performance.
Apply FinOps practices to manage platform spend and optimize cost without reducing service quality.
Partner with analytics engineers and enterprise teams on architecture, data contracts, and sustainable ownership.

Requirements

Hands-on production data engineering experience using SQL and Python or PySpark.
Experience with cloud data platforms, orchestration, data modeling, and governed serving patterns.
Ability to design for observability, recoverability, security, and maintainability.
Proficiency with AI coding assistants and the ability to independently validate generated work.
Practical experience enabling AI capabilities on enterprise data, ideally in Databricks; experience with Snowflake Cortex or comparable platforms is welcome.
Strong cross-functional collaboration and production ownership.

Technologies

Databricks
SQL
Python
PySpark
Snowflake Cortex
Databricks Genie
Delta Lake

Preferred Qualifications

Experience with Databricks, Delta Lake, cloud infrastructure, and deployment automation.
Experience building AI application backends, model integrations, or LLM-based workflows.
Experience with FinOps, financial data, or enterprise-system integrations.

View full job page →

Get Databricks Pyspark Finance Jobs alerts

New Databricks Pyspark Finance Jobs, daily.