MYOB Scales Data Reliability with DataHub

Testimonial

“Before bringing DataHub on board, MYOB's data teams would see multiple breaking changes per week. Since integrating DataHub into our workflow about a year ago, even though our overall usage of Snowflake has gone up 4 times, DataHub has helped us significantly reduce the number of breaking changes, to the extent that they are no longer a burden on all teams.”

ASAD NAVEED
Engineering Manager, MYOB

Customer Information

• Customer: MYOB
• Industry: Financial Technology
• Size: 2,900+ employees
• Solution: DataHub Cloud
• Use Case: Lineage, Observability
• Data Stack: Snowflake, dbt, GitHub, Buildkite, Docker

Goals

• Proactively identify and prevent downstream breaking changes
• Improve visibility into data dependencies
• Enhance communication between data producers and consumers
• Strengthen trust in the data platform by enabling shift-left data quality practices

The Topline

• Challenge: Managing complex dependency trees across 1,000+ dbt transformations was causing multiple breaking changes per week, with downstream consumers discovering broken datasets without warning.
• Solution: Implemented DataHub Cloud to provide critical lineage insights and automatically notify downstream data consumers before changes are merged.
• Impact: Nearly eliminated breaking changes despite 4x growth in Snowflake usage, creating a more reliable and collaborative data platform that scales.

Challenge

MYOB is a cloud-based and desktop accounting software and business management solutions provider for small and medium-sized businesses in Australia and New Zealand. Their data platform journey began in 2020 with the implementation of a data mesh architecture using Snowflake and dbt at its core. As the platform matured and more internal and third-party data producers and consumers joined, the complexity of data dependencies grew.

With more than a thousand dbt transformations creating intricate dependency trees, schema changes to upstream datasets frequently cascaded into breaking changes downstream.

The core challenge was visibility and communication. Data consumers would discover that their datasets, reports, or dashboards were broken without any advance warning when upstream dependencies changed. This reactive approach created frustration across teams and undermined confidence in the data platform’s reliability.

Solution

To reduce the frequency of breaking changes and improve coordination between data producers and consumers, MYOB turned to DataHub Cloud as the foundation for a more transparent, proactive metadata workflow.

With DataHub, owners of downstream datasets can now use the UI to explore lineage and identify upstream dependencies. Then, reach out directly to those upstream owners through Slack or other channels to coordinate dbt transformation updates. This new visibility was a critical milestone in enabling collaborative schema change management.

To take it further, MYOB embraced a shift-left approach by embedding automated schema-change notifications directly into their CI/CD pipelines. All dbt transformations live in GitHub and are executed via Buildkite pipelines. A pivotal new step, “Notify data consumers”, was added as the final stage of each pipeline. This step prevents a Pull Request (PR) from being merged until it’s manually unblocked and downstream data consumers have been alerted.

The notification step runs a containerized script that:

• Detects dbt file changes in GitHub PRs (updates, deletions, or renames with content changes)
• Uses DataHub’s API to identify first-level consumers of affected tables
• Automatically generates and sends notification emails to dataset owners
• Publishes messages to a notification queue for broader organizational integration

This automation ensures data consumers are notified before changes are merged to the default branch; empowering them to review, comment, and collaborate on updates early in the development cycle.

Impact

With DataHub, MYOB transformed their change management process and realized big improvements across reliability, productivity, and collaboration.

Key outcomes included:

• Nearly eliminated breaking changes, reducing incidents from multiple per week to virtually zero despite 4x growth in Snowflake usage.
• Automated change notification workflow ensuring all downstream dataset owners receive advance warning before schema modifications reach production.
• Improved platform reliability by implementing shift-left data quality practices that proactively catch potential issues.
• Transformed collaboration culture between data producers and consumers.
• Enhanced team productivity by replacing reactive firefighting with proactive collaboration, allowing teams to focus on value-creation rather than incident resolution.