We are looking for a Senior Data Engineer to help build, modernize, and support large-scale data pipelines across Hadoop and Google Cloud Platform. This hands-on contract role is a great fit for someone who enjoys solving complex data challenges, improving platform reliability, and delivering trusted datasets for analytics, fraud investigation, and machine learning use cases.

Responsibilities

• Design, build, test, and maintain scalable batch and streaming data pipelines using Python, SQL, Spark, Kafka, and related tools.
• Support the migration and modernization of data workloads from on-premises Hadoop environments to GCP.
• Develop and optimize ETL/ELT workflows for large, high-volume datasets with attention to performance, reliability, and cost efficiency.
• Create and support trusted datasets that enable analytics, reporting, fraud analysis, and machine learning initiatives.
• Investigate production data issues, perform root-cause analysis, and implement durable fixes.
• Build monitoring, alerting, automation, and CI/CD practices that support stable production data systems.
• Collaborate with engineers, analysts, data scientists, and product partners to deliver timely data solutions.
• Contribute to code reviews, technical design discussions, documentation, and knowledge sharing.

Skills

• 5+ years of hands-on experience in data engineering or a related software engineering role building large-scale data systems.
• Strong proficiency in Python and SQL.
• Hands-on experience with Apache Spark and distributed data processing.
• Working knowledge of Hadoop technologies, including HDFS and Hive.
• Experience with Kafka or other event-streaming platforms.
• Experience building and supporting ETL/ELT and data integration pipelines.
• Experience with GCP data services, especially BigQuery.
• Experience supporting production data systems, including monitoring, logging, and incident troubleshooting.
• Familiarity with Git, CI/CD, automated testing, and Agile ways of working.
• Strong problem-solving skills and the ability to work independently within an established architecture.

Preferred Skills

• Experience migrating Hadoop workloads or enterprise data platforms to GCP or another public cloud.
• Experience with Dataproc, Dataflow, Pub/Sub, or Cloud Storage.
• Experience with Spark Streaming, Flink, or similar stream-processing frameworks.
• Exposure to data governance, lineage, privacy, security, and data quality frameworks.
• Experience with orchestration tools such as Airflow, Cloud Composer, or Oozie.
• Experience supporting ML data pipelines or Vertex AI / MLOps workflows.
• Background working with high-volume enterprise data in fraud, risk, security, retail, or e-commerce environments.

Horizontal Talent is committed to diversity, equity, and inclusion, and we welcome candidates from all backgrounds and experiences. We believe inclusive teams create stronger solutions, and we are dedicated to fostering a respectful and supportive hiring experience for everyone.

By applying for this position, you acknowledge and agree that Horizontal Talent may contact you regarding your application using automated technology, including phone calls, SMS/text messages, or email, which may be delivered by our virtual AI recruiter, Alex.