Senior Data Engineer
Mid Valley City , W.P. Kuala Lumpur
|Onsite
|Direct hire
Mid Valley City, W.P. Kuala Lumpur
|Onsite
|Direct hire
Job ID 8202|Posted May 6, 2026
JOB DESCRIPTION
About Horizontal: Established since 2003 in the US, Horizontal solves complex challenges across two distinct businesses: Horizontal Digital and Horizontal Talent. We are consistently recognized for being a top workplace and one of the fastest growing private companies. Horizontal Talent specializes in staffing for IT, Digital & Creative and Business & Strategy markets. We have global offices in US, UAE, India, Malaysia and Australia.
About the Role:
We are seeking an experienced Senior Data Engineer to design, build, and maintain modern, scalable data platforms and pipelines. In this role, you will partner with tech leads, solution architects, and clients to deliver high-performance, reliable, and cost-efficient data solutions that power analytics, business intelligence, and AI initiatives. You will also play a key role in mentoring junior engineers and driving engineering best practices across the team.
Key Responsibilities:
Requirements:
Nice to Have
Soft Skills
About the Role:
We are seeking an experienced Senior Data Engineer to design, build, and maintain modern, scalable data platforms and pipelines. In this role, you will partner with tech leads, solution architects, and clients to deliver high-performance, reliable, and cost-efficient data solutions that power analytics, business intelligence, and AI initiatives. You will also play a key role in mentoring junior engineers and driving engineering best practices across the team.
Key Responsibilities:
- Design, build, and optimize ETL/ELT pipelines using GCP services such as Dataflow, Dataproc, Composer (Airflow), Dataform, and Cloud Functions.
- Develop both batch and streaming pipelines to handle structured, semi-structured, and unstructured data.
- Implement scalable data models in BigQuery to support analytics, BI reporting, and machine learning workloads.
- Apply data governance frameworks, quality checks, and metadata management practices to ensure trusted and compliant data.
- Optimize queries, pipeline performance, and storage costs while ensuring platform scalability, reliability, and fault tolerance.
- Monitor, troubleshoot, and tune pipelines for continuous improvement.
- Lead data migration projects from on-premises systems (e.g., Oracle, MicroStrategy) to modern cloud environments.
- Integrate data from multiple sources, including APIs, databases, and event streams, into unified data platforms.
- Work closely with analysts, BI developers, and business teams to enable self-service analytics and faster decision-making.
- Mentor and coach junior engineers, set coding standards, review designs, and share best practices.
Requirements:
- 8+ years of experience in data engineering.
- Minimum 2 years of hands-on experience with Google Cloud Platform (GCP).
- Proven experience building and scaling cloud-native data platforms and pipelines.
- Strong proficiency in SQL and Python.
- Hands-on experience with GCP tools (BigQuery, Dataflow, Dataproc, Composer, Dataform).
- Experience with workflow orchestration tools (Airflow/Composer).
- Familiarity with CI/CD practices for data pipelines.
- Strong understanding of modern data modeling (relational, dimensional, schema-on-read).
Nice to Have
- Exposure to Azure Data Stack (Synapse, Data Factory, Databricks, Logic Apps).
- Knowledge of data governance frameworks, metadata management, and data security best practices.
Soft Skills
- Strong problem-solving and analytical thinking.
- Good communication skills and ability to work with cross-functional teams.
- Ability to balance technical depth with business impact.
- Leadership and mentoring capability.
Horizontal is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, veteran status, or any other protected characteristic under applicable law.
Horizontal is committed to taking affirmative action to employ and advance in employment qualified individuals with disabilities and protected veterans. If you are an individual with a disability and require a reasonable accommodation to complete any part of the application process or participate in the interview process, click here to request accommodation assistance.
All applicants applying must be legally authorized to work in the country of employment.
We're sorry!
Applications from candidates outside of our global target markets are currently not accepted through Job Board - but that could change in the future. We will update Job Board if we expand to your market. Do you have other candidate related questions? Contact us today.
We're sorry!
Applications outside of the assigned country are currently not accepted for this job - but that doesn't mean we can't work with you. Visit our Job Board to view the jobs available within your market.
Access denied.
Users logged in to client accounts are unable to access Job Board. If you are interested in viewing and applying for open jobs, please log out of your current account. If you are a hiring manager interested in viewing candidates available to hire, click below to go to Talent Board.