Senior Data Engineer

WE ARE HIRING: Senior Data Engineer

Location: Sri Lanka
Employment Type: Full time
Industry: Consumer Products

Role Overview

We are seeking an experienced and highly skilled Senior Data Engineer to design, develop, optimize, and manage scalable data pipelines, data platforms, and enterprise data infrastructure.

The selected candidate will be responsible for building reliable, secure, and high-performance data solutions that support analytics, reporting, machine learning, operational decision-making, and enterprise-wide data integration.

Key Responsibilities

• Design, develop, and maintain scalable ETL and ELT data pipelines.
• Ingest, transform, and load data from multiple internal and external data sources.
• Build and optimize batch and real-time data processing systems.
• Ensure data accuracy, consistency, integrity, and availability across platforms.
• Develop and maintain data warehouses, data lakes, and modern enterprise data platforms.
• Design scalable data models, database structures, and storage solutions.
• Optimize database performance, indexing, partitioning, and query execution.
• Manage and support cloud-based data solutions using Microsoft Azure.
• Work with AWS or Google Cloud platforms when required.
• Automate data deployment and infrastructure management using CI/CD and Infrastructure as Code practices.
• Monitor, troubleshoot, and support production data systems.
• Write and optimize complex SQL queries for data extraction, transformation, analysis, and validation.
• Build data pipelines using tools such as Apache NiFi, Airflow, dbt, Snowflake, and PostgreSQL.
• Develop data engineering solutions using Python and Java.
• Support AI and machine learning workflows, including feature engineering pipelines.
• Work with big data technologies such as Spark, Kafka, or Databricks when required.
• Partner with data analysts, data scientists, software engineers, BI teams, application owners, and business stakeholders.
• Translate business requirements into reliable and scalable data solutions.
• Enable self-service analytics through trusted datasets, documentation, and well-structured data assets.
• Implement data quality monitoring, validation, and governance practices.
• Maintain metadata, lineage, documentation, and technical records for data assets.
• Ensure data solutions comply with security, privacy, regulatory, and governance standards.
• Review solution designs and provide technical guidance to junior engineers.
• Establish engineering standards for data development, pipeline reliability, documentation, and performance.
• Support continuous improvement of data architecture, automation, data quality, and analytics enablement.

Candidate Profile

• Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, or a related field.
• Minimum 2 years of experience in data engineering or a related role.
• Strong hands-on experience in designing and maintaining complex data pipelines.
• Strong knowledge of SQL and experience with relational and non-relational databases.
• Experience building data warehouses and ETL or ELT pipelines.
• Hands-on experience with tools such as Apache NiFi, Airflow, dbt, Snowflake, and PostgreSQL.
• Proficiency in programming languages such as Python and Java.
• Experience with Microsoft Azure cloud platform is required.
• Exposure to AWS or Google Cloud will be an added advantage.
• Knowledge of supporting AI and machine learning workflows and feature engineering pipelines.
• Experience with DevOps practices, CI/CD pipelines, and deployment automation.
• Familiarity with big data technologies such as Spark, Kafka, or Databricks.
• Strong understanding of data modelling, database design, and data warehousing concepts.
• Strong performance tuning and optimization skills.
• Good understanding of data governance, data quality, data security, and privacy practices.
• Ability to troubleshoot complex data issues and production data failures.
• Strong analytical thinking and problem-solving skills.
• Excellent attention to detail and commitment to data accuracy.
• Good communication and collaboration skills.
• Ability to work with technical and non-technical stakeholders.
• Ability to mentor junior engineers and support engineering best practices.
• Proactive, accountable, and comfortable working in a fast-paced data-driven environment.
• Strong ownership mindset with the ability to balance immediate delivery and long-term architecture.

Key Skills

• Data modelling and database design
• ETL and ELT pipeline development
• Data warehousing and data lakes
• Microsoft Azure cloud platform
• SQL, Python, and Java
• Snowflake, dbt, PostgreSQL, Apache NiFi, and Airflow
• CI/CD and DevOps practices
• Spark, Kafka, and Databricks
• Data governance and data quality
• Performance tuning and optimization
• Stakeholder collaboration and analytics enablement

Success Measures

• Reliability and uptime of data pipelines.
• Accuracy, completeness, and quality of data outputs.
• Performance and scalability of data platforms and pipelines.
• Speed and consistency of data availability.
• Stakeholder satisfaction and self-service analytics enablement.
• Strong documentation, governance, and production stability.

Ready to take your career to new heights?

We're InTalent Asia, your go-to recruitment partner in Sri Lanka, and we've got an exciting opportunity for you! Our client is looking for a dynamic individual to fill the role of Senior Data Engineer.

At InTalent Asia, we're not just recruiters; we're here to architect your career success. Join us in the journey of matching your skills with unparalleled opportunities.

Apply now and see how you can be the perfect fit for this exclusive position!

#InTalentAsia #CareerOpportunity #JobVacancy #SeniorDataEngineer #DataEngineering #AzureJobs #Snowflake #DBT #PythonJobs #EnterpriseAnalytics #ElevateYourCareer

Originally posted on Himalayas