We’re looking for a skilled Data Engineer to help design, build, and support scalable data pipelines and analytics solutions for a hybrid opportunity in Brooklyn Park, MN. This role offers the chance to work with modern cloud data tools, large-scale processing technologies, and cross-functional teams to deliver reliable, production-ready data solutions.

Responsibilities

• Design, develop, test, and maintain scalable ETL/ELT pipelines for large and complex datasets.
• Build and optimize distributed data processing solutions using Apache Spark and Google Cloud Dataproc.
• Develop and support analytical datasets and data processing workloads in BigQuery.
• Create batch and streaming data solutions using technologies such as Spark, Hadoop, and Kafka.
• Develop ingestion and transformation workflows across source systems, data lakes, and analytics platforms.
• Improve performance, scalability, reliability, and cost efficiency across Spark and BigQuery workloads.
• Troubleshoot production data issues, perform root-cause analysis, and implement durable fixes.
• Implement data quality checks, monitoring, logging, alerting, and operational controls.
• Apply software engineering best practices, including source control, automated testing, CI/CD, and deployment automation.
• Collaborate with architects, engineers, application teams, and business partners to deliver production data solutions.
• Participate in code reviews and contribute to reusable frameworks and engineering best practices.

Skills

• Strong hands-on experience building production-grade data pipelines.
• Solid ETL/ELT development experience.
• Practical experience with Google Cloud Platform.
• Strong SQL skills, including query development and optimization in BigQuery.
• Hands-on experience with Apache Spark and distributed data processing.
• Experience working with GCP Dataproc.
• Background with Hadoop and/or Kafka.
• Proficiency in Python and/or Java/Scala.
• Understanding of data modeling, partitioning, distributed processing, and common data file formats.
• Experience with DevOps practices, Git, automated deployments, and production support.
• Strong troubleshooting, problem-solving, and collaboration skills.

Preferred Skills

• Experience with Looker and/or LookML.
• Familiarity with GCS, BigLake, Hive, Apache Iceberg, and Parquet.
• Experience using Terraform or other Infrastructure-as-Code tools.
• Exposure to Kafka-based streaming pipelines.
• Experience migrating workloads from on-premises Hadoop/Hive environments to Google Cloud.
• Background in large-scale data lake or platform modernization efforts.
• Familiarity with data quality, governance, lineage, and metadata management.

Horizontal is committed to fostering an inclusive, respectful, and equitable environment where people of all backgrounds can contribute and grow. We value diverse perspectives and are dedicated to creating a workplace where everyone feels supported and empowered to succeed.

By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.