Senior Data Engineer - #2189214

Role Overview

As a Data Engineer / Analytics Engineer, you will play a critical role in building and maintaining the monitoring, controls, and observability capabilities for our HSBC Productivity Suite. You will design and implement robust ETL/ELT pipelines, develop data models for management information (MI) reporting, and create dashboards that provide actionable insights for operational, risk, and compliance stakeholders.

Key Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines to ingest and process log and event data from AI platform components.
• Develop and optimize data models for aggregation, reporting, and analytics to support MI and governance requirements.
• Collaborate with platform, DevOps, and security teams to ensure comprehensive data capture for monitoring and controls.
• Create and maintain dashboards and visualizations using tools such as Power BI, Tableau, Kibana, or Grafana.
• Write efficient SQL, Python, or Spark code for data extraction, transformation, and analysis.
• Implement data quality checks and validation processes to ensure accuracy and reliability of monitoring data.
• Support automated alerting and reporting for operational incidents, policy violations, and compliance metrics.
• Document data flows, models, and dashboard logic for transparency and auditability.
• Participate in requirements gathering with business, risk, and compliance teams to ensure MI meets governance standards.
• Continuously improve data pipelines and analytics solutions for performance, scalability, and usability.

Required Skills & Qualifications

• 5+ years of experience in data engineering, analytics engineering, or a related field.
• Strong experience designing and implementing ETL/ELT pipelines for large-scale log or event data.
• Proficiency in data modeling, aggregation, and building MI/reporting solutions.
• Hands-on experience with dashboarding and visualization tools (Power BI, Tableau, Kibana, Grafana, or similar).
• Advanced SQL skills and experience with Python or Spark for data processing and analytics.
• Familiarity with cloud data platforms (Azure, AWS, GCP) and modern data architectures.
• Experience working with monitoring, observability, or security log data is a strong plus.
• Understanding of data governance, privacy, and compliance requirements in enterprise environments.
• Excellent problem-solving, communication, and documentation skills.
• Ability to work collaboratively in a cross-functional, agile team environment.

Preferred Qualifications

• Experience in regulated industries (e.g., banking, insurance, healthcare).
• Knowledge of AI/ML platform monitoring and controls.
• Experience with log aggregation tools (ELK Stack, Splunk, etc.).
• Familiarity with operational risk, compliance, or audit reporting.