FinanceJobs.io
← Back to all jobs

Job Description

This Data Engineer role at Josh Pros LLC is an onsite position in Eden Prairie, MN, focused on Azure Databricks and hands-on delivery for financial-domain data pipelines. The position emphasizes scalable engineering, ETL/ELT development, advanced SQL performance, and support for AI/ML and stakeholder-facing outcomes.

Location and Schedule

  • Work location: Eden Prairie, MN (onsite)
  • Expected hours: 40.0 hours per week

Compensation

  • Salary: USD 70.00 to 80.00 per hour

Core Responsibilities

  • Develop services, controls, and reusable patterns that help the team deliver value safely, quickly, and sustainably in the public cloud.
  • Build highly scalable applications for the Azure cloud using microservices and Azure Functions, incorporating patterns that support public cloud enablement and security.
  • Support high-performance, collaborative technical work and help drive high-quality delivery.
  • Stay current on product capabilities and translate them into functional changes that can be verified.
  • Gather and analyze data to inform decision-making and provide detailed, realistic estimates.
  • Work effectively with business stakeholders and third-party technical organizations.
  • Design and develop ETL/ELT solutions on Azure Databricks using LakeBase and Spark.
  • Design, build, optimize, and manage large-scale data pipelines to support analytics and machine learning feature generation, including insights generation and business intelligence dashboards.
  • Write advanced and complex SQL and perform performance tuning and optimization.
  • Build AI-based solutions for business process automation and improving operational efficiency.
  • Identify opportunities to improve data reliability, data integrity, system efficiency, and overall data quality.
  • Contribute to architectural evolution of data engineering patterns, frameworks, systems, and platforms, including defining best practices and standards.
  • Mentor other data engineers by providing technical direction and teaching how to leverage cloud data platforms.

Required Experience

  • Python and Scala: 7+ years of experience.
  • Apache Spark: 5+ years of experience (PySpark and Spark SQL).
  • DevOps automation: 3+ years of experience with Terraform.
  • API design and lifecycle management: 3+ years (GraphQL, REST, etc.).
  • Azure Databricks and cloud solutions: 3+ years building and deploying cloud-based solutions using Azure Databricks with UC, Snowflake, Azure Functions, and Service Bus.
  • SQL: 5+ years, including designing complex data schemas and optimizing query performance.
  • LLM integrations: Experience building LLM integrations for workflow automation and business needs.
  • CI/CD and tools: Experience with CI/CD processes and tools including GitHub Actions, GIT, Artifactory, and Sonar.

Technology Stack

  • Python, Scala
  • Apache Spark, PySpark, Spark SQL
  • Terraform
  • GraphQL, REST
  • Azure Databricks (including UC)
  • Snowflake, Azure Functions, Service Bus
  • SQL, microservices
  • LLM integrations
  • ETL/ELT, LakeBase
  • AI/ML
  • CI/CD: GitHub Actions, GIT, Artifactory, Sonar

Minimum Experience

  • Minimum experience: 3 years

Similar Jobs