Minneapolis , Minnesota
|Hybrid
|Contract
Minneapolis, Minnesota
|Hybrid
|Contract
Join a dynamic financial services team as a Databricks Data Engineer, where you will help design and scale modern data platforms that support analytics, automation, and AI initiatives. This role offers the opportunity to build high-performance, governed data pipelines and contribute to the modernization of enterprise data infrastructure in a collaborative, production-focused environment.
Responsibilities
- Design, build, and support scalable data pipelines using Databricks, Apache Spark, PySpark, and SQL.
- Develop and maintain Lakehouse solutions using Delta Lake, Unity Catalog, and Delta Live Tables / Lakeflow.
- Integrate data from enterprise systems, APIs, market data providers, and financial platforms.
- Build both batch and real-time ingestion and transformation workflows.
- Apply data quality, governance, security, and monitoring best practices across data solutions.
- Improve pipeline performance, reliability, and cost efficiency through optimization and troubleshooting.
- Partner with engineering, analytics, AI, and business stakeholders to deliver trusted data products.
- Support modernization efforts across enterprise data infrastructure within a regulated environment.
Skills
- 5+ years of data engineering experience, including hands-on Databricks development.
- Strong proficiency in Python, PySpark, SQL, and Spark performance optimization.
- Experience with Azure Databricks, Delta Lake, Unity Catalog, and workflow orchestration tools.
- Background working with CI/CD, Git, infrastructure automation, and production deployments.
- Experience integrating relational databases, APIs, and cloud-based data sources.
- Understanding of data security, access controls, and governance in regulated environments.
- Ability to work collaboratively, communicate clearly, and solve problems in a fast-paced setting.
Preferred Skills
- Experience in investment banking, capital markets, or financial services.
- Familiarity with financial datasets, market data, CRM platforms, and enterprise data integration.
- Experience supporting AI/ML workloads, feature engineering, or retrieval-based AI applications.
- Databricks certifications and exposure to Azure cloud services.
Horizontal is committed to fostering a diverse, equitable, and inclusive workplace where everyone feels valued, respected, and empowered to contribute. We welcome candidates from all backgrounds and encourage you to apply if you’re excited about the role and meet the qualifications.
By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.Horizontal is committed to taking affirmative action to employ and advance in employment qualified individuals with disabilities and protected veterans. If you are an individual with a disability and require a reasonable accommodation to complete any part of the application process or participate in the interview process, click here to request accommodation assistance.
All applicants applying must be legally authorized to work in the country of employment.