Synthetic demonstration. Every number and example on these pages comes from a generator, not from the live service. A separate, real source exists — 58 recorded demo calls (current, kept up to date on a schedule) — kept apart from these figures: the two are never counted together.
Show the recorded demo calls instead. Timing covers the briefing service's own processing only: it leaves out the telephone connection, speech recognition and speech synthesis. A synthetic request is one ask of the service. It is never a call.
Improvement queue
Findings and recommended actions, each with its evidence. A recommendation is a hypothesis to test: a pattern in data, or a log that a change was made, does not show that the change helped. "How we would know" says what a fair test would need.
This page always uses the whole synthetic window: about 76 hours, Mon 05 Jan 00:01 UTC to Thu 08 Jan 04:25 UTC (1,200 synthetic requests), whatever time range is chosen on other pages. Dates are synthetic.
Two kinds of item. A synthetic example is a finding in this synthetic data. A known gap is something found by reading the service's source; it is not from any data, and reading the repository does not verify what is running live.
Items
Known gap Simulated6 shown, in priority order. Decisions on these items are simulated.
Q-09 High priority Known gap Voice quality Open
Voice quality is not measured
- Why it matters
- Automated tests check the wording the service generates for speech. Nothing on this dashboard checks what a caller hears. Whether the altimeter setting and winds were spoken as written can only be found out by listening.
- Evidence
From source review Not measured by this dashboard. The real service archives call recordings; this dashboard reads none.
Read from the repository, not verified against the live service. No data is behind this item.
See the Voice quality page
- Recommended action
- Start a weekly listening sample of real recordings. Listen for the altimeter setting and the winds first.
- How we would know
- Done when a person has listened to a sample and recorded, for each call, whether the altimeter setting and winds were heard exactly as written.
Q-10 High priority Known gap Briefing quality Open
NOTAMs and special use airspace are not live in any briefing
- Why it matters
- Every briefing carries placeholder text for NOTAMs and says special use airspace was not checked. Both are things a pilot needs. The briefing marks them as not live, so the gap is disclosed, but it is still a gap.
- Evidence
From source review Source review of the request handler.
Read from the repository, not verified against the live service. No data is behind this item.
See the Briefing quality page
- Recommended action
- Needs official data access that is outside this dashboard. Until then the briefing must keep saying they were not checked.
- How we would know
- Done when both come from a live source and a sample of briefings is checked against the official version.
Q-11 High priority Known gap Data gaps Open
This dashboard is not connected to any real request timing
- Why it matters
- It reads only the synthetic set, so nothing on it says anything about the live service. Whether request timing is switched on anywhere was not checked when this was built.
- Evidence
From source review This dashboard reads synthetic data only. Whether request timing is enabled on the live service was not checked.
Read from the repository, not verified against the live service. No data is behind this item.
See the System performance page
- Recommended action
- Find out whether timing capture is enabled. If it is not, enabling it needs the owner's approval and a key kept apart from the call-record key. Then connect this dashboard to the stored records.
- How we would know
- Done when real request records exist and this dashboard reads them in place of the synthetic set.
Q-07 Medium priority Known gap Data gaps Open
A timing record cannot be matched to the call it came from
- Why it matters
- A timing record holds no call identifier; that is deliberate, to keep identifiers out of the timing data. The phone agent does pass a conversation identifier to the service for some requests, but it is not stored with the timing. Until a match exists there is no dependable way to say which call a slow request belonged to, and no count of calls can be worked out from requests.
- Evidence
From source review Source review of the timing record design and the call log (docs/predictive/source-field-inventory.md).
Read from the repository, not verified against the live service. No data is behind this item.
See the Calls page
- Recommended action
- Decide whether a pseudonymous call reference may be stored with each timing record. That touches what is recorded about callers and needs the owner's approval.
- How we would know
- Done when a sample of real timing records can each be matched to exactly one call, and the match can be checked in both directions.
Q-08 Medium priority Known gap Data gaps Open
Requests are not counted separately from their timing records
- Why it matters
- A request that is cut off before its record is written, or whose record cannot be written, may leave nothing behind, so it could not be told from a request that never happened. How often that happens is not known. If it does happen, time figures built from records alone are biased towards the requests that finished.
- Evidence
From source review Source review of the request handler and the timing record design.
Read from the repository, not verified against the live service. No data is behind this item.
See the System performance page
- Recommended action
- Add a small counter that is updated at the start of each request, independently of the timing record.
- How we would know
- Done when, over a day, requests counted at the start equal records stored plus records reported as dropped.
Q-12 Low priority Known gap Data gaps Open
Call records keep the time of the latest refresh, not the first save
- Why it matters
- A prediction may only use what was known at the time. A call record is rewritten with a new "refreshed at" time each time it is refreshed, so when a fact was first known cannot be shown.
- Evidence
From source review Source review of the call log (docs/predictive/source-field-inventory.md).
Read from the repository, not verified against the live service. No data is behind this item.
See the Calls page
- Recommended action
- Keep the first-saved time and never change it.
- How we would know
- Done when every call record has a first-saved time that later refreshes leave alone.
Decision log
SimulatedThe log is the list of decisions in this page's address, replayed from the start on every load. Anyone can edit the address, so it simulates a log that is only added to. It is not a tamper-proof audit record.
Accepting an item here records a simulated decision only. It changes nothing in the service, and a log of decisions does not show that any change helped.
Engineering details: how items are found
Synthetic examples are computed from the synthetic requests over the whole window. Each has a count, a list of example request IDs and the rule for how the examples were picked. If the data holds none of a kind, the item does not appear.
- Q-01: requests with budget.out_of_time_at_end or a skipped lookup.
- Q-02: slow briefings with three or more stored-copy checks that found no copy; shown only as a comparison when there are briefings to compare with.
- Q-03: slow briefings in hours quieter than usual, when that share is higher than in busier hours (at least 30 briefings in each).
- Q-04: requests with no timing record.
- Q-05: requests refused with HTTP 400.
- Q-06: shown only when every approval condition is met on the Predictions page; the examples are requests that ran long, highest rule score first.
- Q-07 to Q-12: known gaps, written from source review and listed whatever the data says.