Senior Data Engineer

🌟 Accomplished Tech Visionary:

Embark on an exciting journey into the realm of software development with 3Pillar! We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data-driven decision-making. As a Senior Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries.

If your passion for data analytics solutions makes a real-world impact, consider this your pass to the captivating world of Data Science and Engineering! 🔮🌐

WE ARE 3PILLAR GLOBAL

At 3Pillar, culture is more than a buzzword. The power of culture, teamwork, and open collaboration drives our commitment to building breakthrough software solutions that power digital businesses. Our passion for software development has gained us recognition in some of the industry’s most innovative spaces, including a spot on the Inc. 5000 list ten years in a row, a three time winner of the Washington Post Top Workplaces Award, and notable features in Forbes, Fortune, and the Washington Business Journal.

We are an innovative software development partner whose solutions drive rapid revenue, market share, customer growth and employee efficiency for industry leaders in Media and Publishing, Information Services, Banking and Financial Services, Insurance and Healthcare.

Our key differentiator is our Product Mindset. Our development teams focus on building for outcomes and all of our team members around the globe are trained on the Product Mindset’s core values – Minimize Time to Value, Solve For Need, and Excel at Change. Our business-minded approach to agile development ensures that we align to client goals from the earliest conceptual stages through launch and beyond.

WHAT WE BELIEVE

At 3Pillar, our core values – Intrinsic Dignity, Outsized Impact, Open Collaboration, and Continuous Improvement – drive our commitment to providing best-in-class software development services to high-growth businesses across the globe. We’ve spent more than fifteen years building innovative software solutions. Our executive team has always held our values to the highest standard which has led to active participation on the Forbes Tech Council, a winner of the Ernst & Young Entrepreneur of the Year Award, and a spot on the Washington Business Journal’s Women Who Mean Business list.

Key Responsibilities:

A data engineer will design, implement, test, and maintain large-scale data systems, including Data Lakes, Data Warehouses, as well as data ingestion and processing pipelines, work with data manipulation languages, database systems, and APIs for ETL purposes, and follow/enforce Data Governance policies.

RESPONSIBILITIES

• Design, develop, implement and maintain:

• Data pipelines → Data ingestion.

• ETL processes → Data processing.

• DW structures → Data storage.

• System integration.

• Design and/or implement big data-oriented architectures for large-scale processing systems

• Develop and maintain documentation including data dictionaries and data lineage related to Data Governance policies.

• Develop and maintain data quality.

• Discover opportunities for data acquisition.

• Analyze data sources(feeds) and data warehouse structures.

• Deal with raw data that contains human, machine or instrument errors.

• Testing and maintenance of deployed services.

• Recommend and implement ways to improve data reliability, efficiency, and quality.

QUALIFICATIONS

• Data-driven problem solver.
• Computer science basis: Algorithms, Data Structures, Time & Space complexity, Big O Notation, SOLID Principles.
• Object Oriented Programming (Inheritance, polymorphism, classes, instances, properties, etc.)
• Hands-on experience with CRUD, ACID and normalization:
• Hands-on experience with data modeling concepts and techniques:
• Concepts: Conceptual, logical and physical modeling.

Techniques:

• Hierarchical data modeling
• Network data modeling
• Relational or Entity-relation data modeling
• Relational database schemas.
• Dimensional data modeling (Data warehouse schemas)
• Star and snowflake schemas.
• Hands-on experience working with unstructured, semi-structured and structured data.
• Hands-on experience designing, implementing and testing data pipelines.
• Hands-on experience designing, implementing and testing ETL jobs/processes.
• Hands-on experience designing database and data warehouse schemas.

Programming Languages & libraries:

• Python, Java, R, Scala, Spark, , Go and C++
• Shell scripting, CRON

Data Pipelines & ETL:

• On-Premise: MS SSIS, Informatica Power Center (IPC), Datastage, SAP Data Services.
• Cloud: Azure Data Factory (ADF), Informatica Cloud Services (ICS), Amazon Glue, Google Data Flow.
• Streaming: Apache Kafka, Apache Flink, AWS Kinesis
• Workflow scheduling: Apache Airflow, Oozie, UC4, Control M.

Databases:

• Relational: SQL Server, MySQL, Oracle, PostgreSQL.
• Non-relational: MongoDB, Cassandra
• Graph: Neo4j, Amazon Neptune, Cosmos DB
• Key-value: Redis, DynamoDB, Cosmos DB

Data Lake

• Hadoop (HDFS)
• Azure Data Lake Storage

Data Warehouses

• PostgreSQL
• Appliances: SQL MPP, Netezza, Teradata, SAP Hana
• Cloud DW: AWS Redshift, Google Big Query, SQL Data Warehouse, Azure Synapse, Snowflake.

Cloud Storage

• AWS S3, Google Cloud Storage

Data Analytics Processing Tools

• Azure Databricks
• AWS Lambda
• Apache Spark

BENEFITS

• Medical Insurance benefits as per company policy.
• Life Insurance as per company policy
• 15 days of paid vacation plus additional days based on tenure
• Vacation bonus as per local law
• Sick leave and paid holidays as per local law
• Profit sharing as per local law
• Paternity and maternity leave as per as per local law
• Marriage, bereavement and graduation leaves as per company policy
• Christmas bonuses as per local law
• Work from-home allowance as per company policy
• Punctuality and attendance bonuses as per company policy
• Food tickets as per company policy
• Direct deposit of payroll.
• Paid professional certifications

What is it like working for 3Pillar Global?

• At 3Pillar, we offer a world of opportunity:
• Imagine a flexible work environment – whether it's the office, your home, or a blend of both. From interviews to onboarding, we embody a remote-first approach.
• You will be part of a global team, learning from top talent around the world and across cultures, speaking English everyday. Our global workforce enables our team to leverage global resources to accomplish our work in efficient and effective teams.
• We’re big on your well-being – as a company, we spend a whole trimester in our annual cycle focused on wellbeing. Whether it is taking advantage of fitness offerings, mental health plans (country-dependent), or simply leveraging generous time off, we want all of our team members operating at their best.
• Our professional services model enables us to accelerate career growth and development opportunities - across projects, offerings, and industries.
• We are an equal opportunity employer. It goes without saying that we live by values like Intrinsic Dignity and Open Collaboration to create cutting-edge technology AND reinforce our commitment to diversity - globally and locally.

Join us and be a part of a global tech community! 🌍💼 Check out our Linkedin site and Careers page to learn more about what it’s like to be part of our #oneteam!