Key responsibilities
- Own Cedar’s ClickHouse reliability, performance, and recovery readiness, including MergeTree-family schemas, partition/order keys, materialized views, retention, ingestion, SQL tuning, capacity, and tested recovery. Maintain supporting PostgreSQL services where deployed.
- Support MySQL/MariaDB systems and validate migrations to PostgreSQL.
- Manage replication, backup, failover, and tested disaster-recovery procedures.
- Diagnose query, locking, storage, replication, and ingestion bottlenecks.
- Maintain data integrity across large ETL pipelines and transactional integrations.
- Perform migration reconciliation and restore drills against agreed recovery objectives.
- Implement database access controls, monitoring, capacity planning, and audit documentation.
- Lead database incident investigations and train backup administrators.
Application responsibilities and delivery
- Own Cedar-side ETL from receipt of agreed Mediation/DWH outputs through validation, parsing, ClickHouse loading, reconciliation and reporting. Develop Python and Go utilities/adapters; maintain legacy Python/web2py, existing Java integrations, SQL, scheduled jobs and Linux services. Refer upstream mediation defects to the Mediation team.
- Apply analytics engineering to develop Git-controlled ClickHouse SQL transformations, reusable reporting models and Grafana / Apache Superset dashboards. Define dataset grain, joins, dimensions and business metrics; document lineage and validate totals, drilldowns, freshness and access controls. Test model changes and reconcile source-to-report outputs before release.
- Distinguish legacy and migrated feeds: IN transaction daily/monthly data and production/DR/testbed environments; legacy/cloud MSX filename conventions; Unified IN subscriber dumps and separate Cloud IN ROP/RPP dumps; old/new PS Core PGW/SGW/IGW Main/Sub; Digitalk SBC, SMSC/ASMSC and EFILL. Validate coverage against the datasets and control totals supplied by the Mediation and DWH teams; refer missing or incorrect upstream feeds to those teams.
- Extend database support to MongoDB where deployed or approved. Cover schema definitions, tables, views, functions and procedures where supported, database permissions, partitioning, retention and lineage.
- Use CI/CD, Ansible and Docker/Temporal where deployed. Maintain ETL tests, cron jobs, Linux service definitions, alerts, and tested recovery. Build Python/Go validation and ingestion adapters against Mediation/DWH contracts; handle mapping, deduplication, checkpoints, retries, and schema changes. Maintain ClickHouse drivers/clients and deployed PostgreSQL, MySQL/MariaDB, Redis, Temporal, and Keeper/ZooKeeper dependencies; record versions and compatibility. Protect live processing and reconcile releases; upstream mediation changes remain with the Mediation team.
Cross-team integration and support
- Work with the Mediation team and DWH team only as this role’s cross-team integration interfaces.
- Agree on delivered datasets, schemas, filenames, delivery schedules, control totals, and reconciliation rules.
- Validate their outputs before Cedar processing, document discrepancies, and route upstream mediation corrections to the Mediation team and DWH delivery issues to the DWH team.
- Coordinate controlled redelivery and downstream reprocessing through these two teams. Do not develop, operate, or change upstream mediation systems.
Key outcomes
- Reliable ingestion and reconciliation from mediation feeds to business reports.
- Recoverable databases with tested backups and measured performance.
Analytics delivery and service ownership
- The two engineers share Cedar development, downstream ETL, ClickHouse reliability, and Grafana/Apache Superset reporting. Allocate primary duties by demonstrated strengths, with cross-cover and reviewed deployments. Maintain the full path from received dataset through tested SQL model to dashboard/API and reconciled business output.
Working arrangements and performance
- Protect live services and data through approved access, tested changes, verified rollback and authorised production deployment or maintenance. Escalate unavailable access, unsupported dependencies and material risks; retain evidence of change and recovery outcomes.
- Own assigned deliverables, usable documentation and trained backup cover. Agree priorities, capacity, escalation, planned maintenance and an on-call rota with the reporting manager; no individual is expected to provide continuous coverage alone.
- Agree priorities through the reporting line. Track delivery quality, repeat incidents, change failures, tested recovery, documentation, and backup readiness. Infrastructure ownership remains with the assigned infrastructure team; coordination does not transfer reporting authority.