Job Description & Summary
We are seeking a highly skilled Senior Data Engineer with extensive experience in data engineering. The ideal candidate will have a strong background in SQL, proficiency in data visualization tools such as Power BI, and hands-on experience with PySpark and Scala Spark. Prior experience with the Quantexa platform and working with AWS is essential. Additionally, a solid understanding of DevOps is also a must.
Responsibilities:
- Design, develop, and maintain robust and scalable data processing pipelines using PySpark and Scala Spark;
- Collaborate with cross-functional teams to define, design, and implement new features;
- Cleanse, parse, and standardize global datasets;
- Perform data classification and entity extraction/resolution;
- Optimize and improve data processing performance through big data testing;
- Utilize pre-packaged code for processing and utilizing third-party data sources;
- Ensure data quality and integrity through efficient data manipulation;
- Help implement and configure Continuous Integration (CI) and Continuous Delivery (CD) tooling;
- Help deploy updates and fixes, build and maintain highly available systems;
- Implement and improve monitoring and alerting;
- Perform root cause analysis for production errors and investigate/resolve technical issues;
- Work in an Agile environment and contribute to the continuous improvement of our development processes.
Requirements:
- Masterโs degree in a related field;
- At least 6 years of hands-on experience in data engineering;
- Strong proficiency in SQL;
- Proficiency in data visualization tools such as Power BI;
- Solid experience with big data technologies like PySpark and Scala Spark;
- Hands-on experience with the Quantexa platform is a must;
- Prior experience working with AWS;
- Familiarity with Agile development methodologies;
- Strong problem-solving skills and attention to detail;
- Excellent communication and teamwork abilities;
- Experience with Airflow for workflow automation and scheduling;
- Focus on Docker/Kubernetes;
- Strong Linux system administration background;
- DevOps tooling/automation written with Bash, Python, Groovy, Jenkins;
- Experience with provisioning software/frameworks like Elasticsearch, Spark, Hadoop, PostgreSQL;
- Knowledge of containerization technologies like Docker and orchestration tools like Kubernetes;
- Experience working with Spring Boot Java framework.
We offer:
-
Interesting career development options and challenging assignments for top Romanian and international clients;
-
International mobility and short/medium-term projects abroad;
-
A professional and team-oriented work environment;
-
Great colleagues to work with;
-
Up-to-date technologies and methodology;
-
Professional development and training opportunities in Romania and abroad.
ย
PwC IT Services Egypt LLC
A global professional services network providing consulting, system integration and implementation services across multiple SAP applications, products and technologies.
Other jobs at PwC IT Services Egypt LLC
ย
ย
ย
ย
ย
ย
ย
ย
Notifications about similar jobs
Get notifications to your inbox about new jobs that are similar to this one.
No spam. No ads. Unsubscribe anytime.
Similar jobs
ย
ย
ย
ย
ย
ย
ย
ย