Skip to content

DataMeshOps-Healer

Grand Prize Target — Challenge Category: Agents That Do Real Work

DataMeshOps-Healer is an autonomous, context-aware data remediation agent that detects, diagnoses, and fixes enterprise data pipeline failures while actively mutating the DataHub metadata graph so future engineers and agents inherit the knowledge.

What It Does

  1. Event Detection — Detects SLA freshness breaches, schema assertion failures, and broken lineage nodes
  2. Context Exploration — Uses DataHub MCP tools to traverse 3-hop upstream lineage and inspect schema fields
  3. Fix Generation — LLM synthesizes a non-destructive SQL remediation patch grounded in real schema context
  4. Graph Mutation — Writes remediation metadata back to DataHub:
  5. add_tags: Applies #auto-remediated, #sla-restored
  6. add_structured_properties: Injects remediation_timestamp, agent_id, root_cause_type
  7. update_description: Appends a Markdown post-mortem runbook
  8. update_incident_status: Marks open incidents as RESOLVED
  9. Knowledge Inheritance — Future agents and humans querying DataHub inherit full remediation context

Running Live

export DATAHUB_GMS_URL="http://localhost:8080/api/gms"
export TOOLS_IS_MUTATION_ENABLED="true"
export OPENAI_API_KEY="your-key-here"   # or GEMINI_API_KEY

python3 -m datameshops_healer.cli run --dataset nyc-taxi

Offline Playback (Zero Setup)

python3 -m datameshops_healer.cli replay --cast examples/demo_playback.cast

Example Output

See examples/sample_remediation.json for full MCP tool call log.

See examples/mutated_aspects_log.md for the Markdown runbook injected into DataHub.

See examples/graph_mutation_before_after.json for before/after DataHub aspect diff.