About the Role
We’re looking for a skilled and experienced Lead Data Engineer on behalf of a London based client to help build and scale real-time data capabilities within a fast-moving digital organization. This is a senior-level, hands-on role with both technical ownership and leadership responsibilities, including the line management of engineers.
You’ll architect and deliver scalable data infrastructure and pipelines, enabling real-time analytics and decision-making across the business. Working closely with engineering, product, and analytics teams, you’ll play a critical role in ensuring data is accurate, timely, and usable across the organization.
What You’ll Be Doing
- Design, develop, and maintain real-time data pipelines that are scalable, efficient, and fault-tolerant
- Integrate data from multiple sources to support real-time insights and downstream applications
- Optimise data infrastructure for low-latency and high-throughput performance
- Lead and mentor data engineers, providing direction, support, and regular feedback
- Define and enforce best practices around data quality, security, and governance
- Collaborate with cross-functional teams to deliver solutions aligned with business goals
- Own monitoring, alerting, and logging processes to ensure pipeline stability and resilience
- Continuously improve data architecture, tooling, and workflows
What We’re Looking For
- Strong track record of delivering real-time data pipelines and infrastructure
- Hands-on experience with distributed data processing using tools like Apache Kafka, Apache Spark Streaming, or Apache Flink
- Proficient in Python, Java, or Scala
- Deep understanding of SQL, NoSQL, and time-series databases
- Proven ability to optimise, troubleshoot, and scale data systems in production
- Experience with orchestration and deployment tools like Apache Airflow and Kubernetes
- Advanced expertise in cloud-based data engineering—Google Cloud Platform (GCP) experience is strongly preferred and highly valued
- Familiarity with BigQuery, Pub/Sub, Dataflow, and other GCP-native tools is a major advantage
- Line management or formal leadership experience—comfortable managing engineers and fostering team growth
- Excellent communication and collaboration skills, with a proactive, team-first attitude
- Comfortable working in a fast-paced environment with changing priorities