Test Data Manager

Job Title: Test Lead – Federated Testing Platform Enablement for Enterprise Data Platform Engagement Type: Hybrid: 2 days in office, 3 days remote Role Summary As part of a horizontal data platform team, the Test Lead will design and deliver a federated testing platform that enables vertical delivery teams to validate their data products and services against both functional and non-functional requirements. This role emphasizes building a self-service “testing-as-a-product” capability, defining standards, patterns, and tooling, and collaborating with offshore engineers and vertical teams to ensure adoption, scalability, and auditability. Key Responsibilities Platform Strategy and Governance - Define a federated test strategy, standards, and risk tiers with corresponding quality gates. - Establish test taxonomies (e.g., functional, data quality, performance, reliability, security) and align them to domain SLOs. - Implement policy-as-code governance to balance team autonomy with guardrails. Testing Platform Architecture - Lead the design and delivery of the testing platform, including reusable test frameworks, test data services, and observability/reporting tools. - Provide SDKs/libraries and DSLs to simplify and standardize test authoring in Python/Java. - Integrate with schema registries and enforce data contract compatibility (backward/forward). - Build resilience testing harnesses for pipelines and services (e.g., fault injection, replay/backfill). Patterns, Templates, and Enablement - Create reusable CI/CD templates with quality gates and blueprints for common use cases (e.g., dbt model testing, Spark pipelines, Kafka streams, REST APIs). - Publish reference implementations and sample repositories to demonstrate unit, integration, end-to-end, performance, and resilience testing. - Partner with vertical teams to bootstrap testing for priority products and establish shift-left practices (e.g., pre-commit tests, PR checks). Test Data and Environment Management - Deliver solutions for synthetic data generation, masking, subsetting, and versioned golden datasets. - Provide ephemeral environments and dependency virtualization for faster, deterministic testing. - Monitor data drift and define schema evolution and rollback strategies. Observability and Reporting - Instrument tests with OpenTelemetry and centralize reporting via dashboards (e.g., test coverage, flakiness, trend analysis). - Ensure traceability from requirements to test results for auditability and compliance. - Deliver release readiness packs and quality scorecards for stakeholders. Team Leadership and Collaboration - Manage and guide offshore engineers, ensuring adherence to coding standards and delivery timelines. - Collaborate with data engineers, developers, SREs, product owners, and governance teams to align quality practices. - Drive continuous improvement in testing (e.g., flakiness reduction, performance optimization, cost-aware testing). Key Outcomes/Success Criteria - A self-service testing platform adopted by vertical teams, featuring reusable test frameworks, CI/CD templates, and test data services. - Reference blueprints for testing common archetypes and use cases. - Centralized dashboards and evidence reports for test results and quality metrics. - Onboarding of priority vertical teams to the platform, supported by training and enablement. - Measurable quality improvements, such as reduced escaped defects, faster release readiness, and consistent quality gate enforcement. Minimum Qualifications - 10+ years in software quality engineering/test automation, with 3–5 years building testing platforms or developer enablement tools. - Expertise in data platform testing (e.g., data quality, pipeline validation, contracts) across warehouses, lakes, and streaming systems. - Strong hands-on experience with Python or Java, test frameworks (e.g., pytest, JUnit/TestNG), and CI/CD pipelines. - Proven experience driving federated quality practices and collaborating with distributed teams. Preferred Qualifications - Data Testing: Great Expectations, Soda, Deequ, dbt tests, and anomaly detection. - API/Contract Testing: REST Assured, Pact, schema registry tools (Avro, Protobuf, JSON Schema). - Performance/Reliability Testing: k6, JMeter, Locust, chaos testing tools. - Data Platforms: Snowflake, Spark, Kafka/Kinesis, dbt, Airflow/Dagster. - Cloud/DevOps: AWS, Docker/Kubernetes, Terraform, CI/CD platforms. - Observability: OpenTelemetry, Prometheus, Grafana. Job Types: Full-time, Fixed term contract Pay: From £60,000.00 per year Work authorisation: - United Kingdom (required) Work Location: Hybrid remote in St Albans AL1 1NW