Senior Data Engineer - Data Platform

Job Summary
The Senior Data Engineer — Data Platform builds and operates the foundational data
infrastructure that powers analytics, reporting, and operational workflows across the
organization. This role designs, develops, and maintains scalable batch and streaming pipelines,
curated data models, and platform services that enable internal teams—including Asset
Management, Finance, and Operations—to access reliable, well-governed data. The position
partners with Data Engineering, IT, and business stakeholders to translate platform
requirements into production-grade solutions with strong quality controls, observability, and
security. Core emphasis areas include: (1) data ingestion and integration—building robust
connectors and pipelines to land data from internal and third-party sources into the platform
with clear lineage and auditability; (2) data modeling and curation—designing dimensional and
domain-oriented models in Snowflake and PostgreSQL that support self-service analytics and
downstream applications; and (3) platform reliability and developer experience—establishing
standards, reusable frameworks, orchestration patterns, and monitoring to accelerate delivery
while maintaining operational excellence.

Responsibilities

• Design, build, and operate end-to-end data pipelines (batch and near-real-time) that ingest, transform, and deliver data from diverse sources into the enterprise data platform.
• Develop and maintain curated data models, marts, and shared datasets in Snowflake andPostgreSQL that meet performance, quality, and access-control requirements for multiple internal customers.
• Implement data quality frameworks including automated validation, schema enforcement, reconciliation checks, duplicate detection, and exception reporting with clear audit trails.
• Partner with domain teams (e.g., Asset Management, Finance, Operations) to understand data needs, define contracts and SLAs, and deliver platform capabilities that reduce bespoke engineering and manual effort.
• Build parameterized, reusable pipeline components and templates that standardize ingestion patterns, transformations, and deployment across the platform.
• Establish and maintain data lineage, metadata, and documentation so stakeholders can trace data from source to consumption with confidence.
• Collaborate with IT and security to implement role-based access controls, data masking, encryption, and compliance requirements across platform resources.
• Own pipeline orchestration, scheduling, dependency management, and alerting using workflow tools (e.g., Airflow) to ensure reliable, recoverable execution.
• Improve platform observability through logging, metrics, SLA monitoring, and incident response practices that minimize downtime and data freshness gaps.
• Support CI/CD and infrastructure-as-code practices for data platform assets, including version control, automated testing, and safe promotion across environments.
• Evaluate and integrate new platform technologies and patterns (e.g., streaming, CDC, data mesh principles) where they improve scalability, cost efficiency, or time-to-value.
• Mentor junior engineers and contribute to platform standards, code review practices, and technical design documentation.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related quantitative field.

• 5+ years of experience in data engineering or platform engineering, preferably in a financial services or regulated industry (e.g., asset management, banking, insurance, fintech).
• Strong SQL and Python skills, with a track record of building production-quality data
• pipelines, transformations, and validation frameworks.
• Proficient at using AI-assisted development tools to design, build, and iterate on data pipelines while maintaining code quality, security, and governance standards.
• Hands-on experience with Snowflake and PostgreSQL, including performance tuning, cost optimization, and secure multi-tenant data access patterns.
• Experience with pipeline orchestration and workflow management tools (e.g., Apache, Airflow, Dagster, or equivalent).
• Proficiency with Git, code review, and CI/CD practices for data platform development.
• Experience designing dimensional or domain-oriented data models and delivering curated datasets for analytics and operational use cases.
• Familiarity with data quality, lineage, and governance tooling and practices (preferred).
• Experience with cloud data services (e.g., AWS, Azure, or GCP) and infrastructure-as-code (e.g., Terraform) is strongly preferred.
• Exposure to streaming or change-data-capture (CDC) patterns and event-driven architectures is a plus.
• Understanding of financial data domains (e.g., portfolio, investor reporting, accounting) is helpful but not required; curiosity and ability to partner with domain experts is essential.
• Strong communication and collaboration skills; ability to translate ambiguous requirements into well-scoped technical designs and clear status reporting.
• Familiarity with containerization (e.g., Docker/Kubernetes) and API/integration patterns for data services is a plus.

Originally posted on Himalayas