Minneapolis , Minnesota
|Hybrid, Onsite
|Contract
Minneapolis, Minnesota
|Hybrid, Onsite
|Contract
We are seeking a talented Data Engineer to join our dynamic team. If you are passionate about transforming unstructured data and building innovative solutions, this opportunity is for you!
Responsibilities
- Design and implement pipelines for ingesting, parsing, and transforming unstructured data from various sources.
- Build scalable document processing workflows using Databricks technologies.
- Integrate with vector stores to support advanced architectures and semantic search capabilities.
- Collaborate with AI engineers to prepare document corpora for AI applications.
- Architect and deploy Databricks Genie Spaces for natural language querying across financial data.
- Develop and maintain REST APIs to connect data and machine learning model outputs to applications.
- Ensure the reliability and observability of APIs and data integrations across the platform.
Skills
- 5+ years of experience in data engineering, including at least 2 years with Databricks.
- Proven expertise in processing unstructured data and implementing embedding pipelines.
- Strong proficiency in PySpark, Python, and SQL, with experience in Databricks SQL.
- Familiarity with building and deploying REST APIs and microservices.
- Solid understanding of data modeling, governance, and best practices.
Preferred Skills
- Experience integrating Databricks with Azure services.
- Exposure to LLM orchestration frameworks and familiarity with vector databases.
- Background in financial services or investment banking data environments.
- Relevant certifications in data engineering or related fields.
At Horizontal, we value diversity, equity, and inclusion as core principles in our hiring process and workplace culture. We encourage candidates from all backgrounds to apply and contribute to our innovative team.
By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.
Horizontal is committed to taking affirmative action to employ and advance in employment qualified individuals with disabilities and protected veterans. If you are an individual with a disability and require a reasonable accommodation to complete any part of the application process or participate in the interview process, click here to request accommodation assistance.
All applicants applying must be legally authorized to work in the country of employment.