Software Engineer

What We're Looking for:

We are looking for people who are passionate about delivering self-driving (L4) products that make the way we move safer, faster and more efficient. We seek mission-driven, highly skilled people with deep experience in fast paced, rapidly growing, tech development environments.

The Data Platform team is part of the Platform Infrastructure Engineering and Reliability group, and is responsible for designing, implementing and maintaining the Stack AV on-premises data platform. The team supports large scale OLAP/OLTP and feature engineering workloads for multiple Product Development groups across the company. The team is tasked with building a highly-available and scalable data infrastructure to power our critical services and pipelines. Members of the team are expected to have a strong focus on requirements gathering and design process to ensure our platform meets the needs of the business and our internal customers.

What Success Looks Like:

• Data Platform engineering focus, with experience supporting large scale interactive and batch workloads. Experience with on-premises infrastructure, Kubernetes compute, S3 compatible object stores (Vast Data, Minio) managed via Terraform or ArgoCD.
• Expert level understanding of Data Engineering concepts and open source technologies, with ability to distill concepts and provide training to customers across the company.
• Experience running a modern data stack on-premises using open source technologies. Familiarity with streaming platforms (Kafka, Pulsar), processing engines (Trino, Apache Spark), table formats (ICEBERG, Delta Lake, Hudi, Xtable) and Postgres.
• Fluent in Python, and SQL, with experience writing and maintaining highly available data applications using Trino and Apache Spark.
• Strong focus on requirements gathering and overall design process. Experience architecting and optimizing large-scale systems to facilitate machine learning, simulation, and data processing exercises.
• Experience running and supporting workflow orchestration systems such as Argo Workflows, Airflow, or Flyte.io.
• Strong focus on providing production-grade service to internal engineering teams through observability, instrumentation, and well-defined SLOs.
• Experience working in a diverse and distributed team, with a focus on communication and customer empathy.Desire to work cross-functionally within Stack to meet customer use cases and deliver business value.