Spark Engineer

You’ll be working within an international group of teams which span from India to Europe and US

We are looking for a Spark/Scala developer to join our N1Ads Reference team. Ideal candidate would be someone with a passion for Writing Clean Code with Test Driven Development and DevOps. Someone who loves getting hands dirty with code and learns everyday.

This individual will work with a team of experienced and passionate Backend, Frontend and devOps developers. Looking for a developer who enjoys coding on backend modules. Should be able to work independently with minimal supervision. Should be passionate and driven to learn and adapt to new technologies and upgrade and maintain current applications.

Responsibilities

• Discuss the Cost of Change (= code quality) with your team members continuously
• Write unit tests, integration tests and API tests
• Should be able to support the application 24/7 based on team on call rotations.
• Collaborate a lot: Code Reviews and Pair Programming
• You would be expected to write code 100% of the time, with an emphasis on Test Drive Development
• Should be open to learn and adapt to new technology architectures and patterns
• You would be expected to code and also understand the impact of it on data
• Fair amount of experience with building and creating CI/CD pipelines
• Should be able to develop and maintain existing applications processing large datasets
• Follow best practices on application monitoring and orchestration and performance optimization
• Implement and Practice DevOps and SecOps for continuous incremental delivery and quality product with the guidance of senior engineers

Key Skills

• Required

• Bachelor or Master’s degree in computer science or related discipline (field), or equivalent work experience

• 2-4 years of Experience with Spark using Scala, Spark SQL and Presto. Upgrading, Maintaining and performance tuning large Spark/Scala applications.

• 2-4 years of advanced experience with Java and Relational databases.

• 0-3 years of experience working with AWS services (RDS, S3).

• Experience using monitoring and alert orchestration tools such as Prometheus, Grafana, OpsGenie/PagerDuty tools.

• Fluent in English, both spoken and written, with a large vocabulary (C1 english level)

• Understand basic object oriented principles and functional programming principles. Implements good coding practices with good unit and integration testing, with an emphasis on Test Driven Development.

• Ability to work independently and as part of a team in a fast-paced and dynamic environment.

• Preferred

• Strong understanding of Mathematics and Statistics

• Experience with Test-driven development

• Experience with building CICD pipelines in GitLab for applications running on Kubernetes (EKS) using Docker

• Follow Agile principles and feature related discussions and participate in grooming and planning sessions to effectively translate business requirements to Agile stories.

• Experience with Airflow DAGs and Python.