The register
0:07
The register. Twelve systems, and the column that matters most.
0:12
SourcesSources is the register of every system connected to the lakehouse at Acme Home Appliances, a demo company: how often each one lands, and how long anything that stopped has been stopped.
0:26
Twelve systemsTwelve systems feed 33 tables. An appliance maker is not one system; it is an ERP, a service CRM, distributor software, plant systems, marketplaces and a few files.
0:38
Not landingTwo are not landing at all, and one of them is rated high criticality.
0:45
Nine daysThe longest outage is nine days. Nobody raised it, because nothing downstream failed.
0:51
How it connectsConnector says how the data arrives: a vendor API, a direct database read, a file dropped somewhere, or a live stream.
1:02
A databaseThe plant systems are read straight from a replica. It is the fastest kind, and the hardest to get permission for.
1:11
A file dropThe survey agency drops a file on a server. Cheap, and brittle in a way nobody notices until it breaks.
1:21
StatusHealthy sources read as plain text. Only the ones that need a look carry a colour, so the eye does not learn to skip the column.
1:33
Broken forThe column the page is built around. Broken is a state; broken for nine days is a sentence somebody has to answer for.
1:43
A failed attemptLast run says the feed tried six hours ago, and it is tinted red, so a failed attempt cannot be read as a recent success.
1:56
Paused is not lateCompetitor prices has not run since 21 July because somebody switched it off. It fades rather than joining the red rows.
Reading a source
2:05
Reading a source. One of each kind, opened.
2:11
BrokenA broken source leads with what it means: everything reading stg_channel_stock is serving numbers from before this broke, and reporting them as current.
2:21
The causeThe cause is in the facts below: an API key that expired on 30 July. Nobody was told, because the portal does not tell anyone.
2:32
CriticalityHigh criticality means the business stops making sense without it. Channel stock drives the festive build plan.
2:40
How far it reachesTwo tables are fed by this connector, and both are governed. We will follow them downstream shortly.
2:50
DegradedDegraded is the state before an outage. The service CRM still lands, but the vendor has rate-limited it since July, and it has fallen a day and a half behind.
3:03
The scheduleIt is meant to run every 30 minutes. The schedule stays out of the list, because it only matters once you care about the source.
3:16
OnboardingThe call centre stream was connected last week. Events are landing, but nothing is modelled on them yet, so onboarding is its own state rather than healthy.
3:27
A streamA stream has nothing to schedule and nothing to be late for. Events arrive as the calls happen.
3:38
A database readPlant systems is a read replica of the plant software at Pune and Hosur, and the only source the data team holds a database credential for.
3:52
PausedAnd the paused source says it plainly: switched off on purpose, not overdue. The category team is deciding whether they still want it.
Finding one
4:03
Finding one. Search and sort, for the day there are two hundred.
4:10
SearchSearch reads names, vendors, owners and the tables each source lands in. Two sources come from a portal, and both are the fragile kind.
4:23
By ownerOr by owner. Service ops owns the service CRM and the warranty claims download, and one of them is degraded.
4:35
SortAny column sorts. Criticality runs high to low, and the header says which way.
4:43
By last runBy last run, the warranty download is the oldest after the paused source: it only runs on Mondays.
4:54
The defaultA third click returns to the default order: longest outage first, then degraded, then working, then switched off.
The runs behind it
5:03
The runs behind it. Flow is the log of every attempt.
5:10
FlowSources says a connector is broken. Flow says which attempt broke, when, and why.
5:17
Wrote nothingTwo runs failed this cycle and wrote nothing at all.
5:22
Rows writtenRows written sits beside the status, so a run that succeeded and wrote nothing cannot hide.
5:31
Why it failedThe channel stock run lasted 38 seconds. The portal answered with HTTP 401, because the key expired on 30 July.
5:40
TomorrowThe next attempt is tomorrow at 06:00, and it will fail the same way unless somebody acts.
5:50
No fileThe survey run took twelve seconds: no file matched the expected name, because the agency changed its format.
5:59
In flightThe service tickets run is still going after nearly seven minutes, against a 30-minute schedule.
6:06
SlowSecondary sales finished, but took fifteen minutes. Two more slowdowns and it overlaps itself.
6:15
Not runAnd the paused price tracker has no run at all this cycle, which is correct.
What the outage cost
6:23
What the outage cost. Nine days of stale stock, followed downstream.
6:30
Sixteen thingsLineage counts what sits below each broken connector. Between them, the two reach sixteen tables, models, dashboards and alerts.
6:40
Channel stockBelow the channel stock feed: three tables, the channel inventory model, three dashboards including Business pulse, and two alerts.
6:48
Who saw itSeventy-eight people opened those dashboards in the last thirty days, while the numbers were nine days old.
7:00
The survey dropThe survey drop reaches seven more, including the customer satisfaction tracker leadership reads.
7:08
In IntelligenceOver in Antvia Intelligence, the alert rules have noticed. Three are firing, two of them critical.
7:17
FiringThe freshness rule on stg_channel_stock is firing, and has fired nine times in thirty days. It pages the on-call engineer.
Fix both, live
7:27
Fix both, live. Re-authenticate, re-run, and watch everything catch up.
7:34
RepairA fresh key has been issued by the portal. Re-authenticate and re-run.
7:43
Six stagesConnect, extract, land in open format in the company's own cloud, rebuild the models, run the quality gates that were failing, and publish.
7:55
HealthyThe channel stock feed is healthy and ran just now.
8:01
The other oneThe bar now offers the second broken feed. The agency's new file name has been added to the pattern, so re-run it from here.
8:14
RunningThe same six stages, for the survey drop.
8:24
Nothing brokenNothing is broken now. Two sources are still degraded, and they are the next job.
8:34
In the logFlow records both repairs as manual re-runs, with the rows they wrote.
8:42
The alert clearsIn Intelligence, the freshness alert has stopped firing. The rule is still armed; it simply has nothing left to complain about.
8:53
Active againIts status is back to active.
9:00
On the recordAnd the audit trail has both repairs, with the tables each one refreshed and the person who did it.