Senior Data Engineer

Introducing Data Engineering at Trainline

Data Engineering is essential to how we unlock the value of data at Trainline. Our mission is to liberate Trainline data and delight customers with great data products built on a frictionless, modern data platform. Our data products include machine learning models that add real value to the customer journey, streaming data applications that personalize the customer experience in real time, dashboards that drive deep business and customer insight and intuitive and efficient data marts and metrics built on our modern data lakehouse.

As a Senior Data Engineer (Scala), you will be part of a cross-functional data product team working alongside data scientists, machine learning engineers and BI engineers. Our data product teams are deeply embedded in the business so your work will have high impact by either drive key business decisions, provide deep customer insights or by adding intelligent machine learning experiences right in the core of our customer journeys.

We use an agile delivery playbook that encourages incremental and iterative delivery, aims to release value early and often, measure the impact of work and using hypotheses to ensure we are solving real customer problems. Our data platform is a modern, cloud-native, lake house using best-of-breed technologies and partners, all based on the AWS public cloud.

We empower our Data teams and give engineers high levels of autonomy and freedom to innovate. We encourage continuous learning with clear career progression plans, innovation/hack days and training opportunities such as DataCamp.

As a Senior Scala Data Engineer at Trainline, you will...🚄

• Use cutting-edge Data technology to deliver world-class data products using a combination of streaming technologies, machine learning and automated data pipelines.
• Work in self-organised, cross-functional data teams alongside machine learning engineers, BI engineers and product managers.
• Drive continuous improvement to the software engineering and agile working practices of the team.
• Contribute to the Technical / Architecture direction of the team.

Requirements

We'd love to hear from you if you...🔍

• Thrive in a diverse, open and collaborative environment where impact is as valuable as technical skill.
• Have excellent knowledge of Scala and the JVM ecosystem.
• Possess strong understanding of functional programming paradigms and a willingness to adopt other languages (not only JVM languages).
• Have consistent background in software development in high volume environments.
• Have a pragmatic and open-minded approach to achieving outcomes in the simplest way possible.
• Have worked with stream processing technologies (Kafka, Storm, AWS Kinesis, etc).
• Have experience with AWS services especially EMR, ECS, EKS.
• Have an obsession with software quality, Dev Ops and automation.
• Work well in lean, agile, cross-functional product teams using Scrum and Kanban practices.
• Are a good communicator and comfortable with presenting ideas and outputs to technical and non-technical stakeholders.

Our Technology Stack 💻

• Python.
• Scala and the JVM.
• Kafka, Kafka Streams and KSQL.
• AWS, S3, Parquet, Iceberg, Glue and EMR for our Data Lake.
• Terraform and Docker.
• Elasticsearch and Dynamodb.
• Spark and Airflow.
• Trinio (Starburst) and Presto (Athena).
• ML Flow and popular Python machine learning and analysis libraries.