Staff Engineer Web Scraping R&D

At Lighthouse (formerly OTA Insight), we’re on a mission to reimagine commercial strategy for the hospitality industry. Our commercial platform simplifies data, transforming complexity into confidence with actionable market insights, business intelligence, and pricing tools that maximize revenue.

We successfully raised 80 million in series B funding, we welcomed four companies to our organization and want to grow over 100 million in ARR in 2024. With more than 650 teammates illuminating the way, representing 34 nationalities living in 35 different countries of which 175 in Belgium alone, we are a fun, collaborative and committed team that works hard to achieve amazing results. Ready to take off and join this rocketship in revolutionizing the hospitality sector? 🚀

What you will do:

We are in search of a Staff Engineer Web Scraping R&D with a deep fascination for the inner workings of the internet and a desire to impart this passion to our team. You will play a critical role in our data extraction process, ensuring that we can extract and process billions of data points every day.

Where you will have impact:

• Drive the research and development of advanced techniques for web data extraction and processing at scale, a task made more challenging due to the high volume of our operations. You will build upon our ongoing projects, leveraging your expertise in reverse engineering and browser technologies to push the boundaries of our capabilities.
• Be at the forefront of tackling web scraping challenges such as emulating user agents, scraping on multiple platforms, utilizing proxy services in a cost-efficient manner, scraping from specific locations, and effectively distributing our scraping load.
• Actively collaborate with diverse engineering teams to address challenges and advance initiatives related to data extraction and infrastructure.
• Serve as a subject matter expert, providing mentorship and support to team members while fostering an environment of continuous learning and development.

About our team:

Our team is at the helm of our sophisticated data extraction infrastructure. Tasked with the responsibility of acquiring pricing data from over 200 unique data sources, we process nearly 200 million requests each day. The data we collect is the lifeblood of our products. We are committed to ensuring its reliable and timely delivery, thereby powering our diverse product portfolio.

Who you are:

• Master’s or Bachelor’s degree in a STEM field
• 5+ years experience in at least one of the following:
• Networking protocols
• Browser development
• Reverse engineering websites and/or mobile applications
• Strong problem-solving and analytical skills, with the ability to identify and troubleshoot complex technical issues
• Excellent communication and collaboration skills, with the ability to work effectively in a team environment
• Fluent in English

Technologies you’ll work with:

Google Cloud, Kubernetes, Rust, NodeJS, TypeScript, Scrapy, Python, Prometheus, Grafana

(feel free to ask us why these are the best fit for us)

What’s in it for you:

• Flexible working environment: Work from home or at one of our global offices.
• Flexible time off: Autonomy to manage your work-life balance.
• Collaborative team: High-bar, friendly, creative, and passionate colleagues.
• Career development: Workshops, frameworks, tools, training, and processes to realize your full potential.
• Impactful work: Shape products relied on by 85,000+ users worldwide.
• Competitive compensation: Proactively maintained to value your work.
• Referral bonuses: Earn rewards for bringing in new talent.

Thank you for considering a career with Lighthouse. We are committed to fostering a diverse and inclusive workplace that values equal opportunity for all. We welcome candidates from all backgrounds, regardless of age, gender, race, religion, sexual orientation, and disability. Our commitment to equality is not just a policy; it's part of our culture.

If you share our passion for innovation and teamwork, we invite you to join us in shaping the future of the hospitality industry. At Lighthouse, our guiding light is to be an equal opportunity employer, and we encourage individuals from all walks of life to apply.

We value the unique perspective and talents that you bring, and we're excited to see how your light can shine within our team. We can't wait to meet you and explore how we can grow and succeed together, illuminating the path towards a brighter future for the industry.