Asking an AI Assistant What Changed on a Machine

An open-source Model Context Protocol server called Predictive Maintenance MCP, maintained by an unnamed author, lets AI assistants run condition-monitoring and diagnostic functions on machine vibration data while keeping signal processing in tested code and language models limited to interpreting questions and explaining results. In a controlled demonstration using seven real recordings from a public bearing test rig and six recorded conversations with two language models, the server's fault check flagged an outer-race line 35 dB above the noise floor and an inner-race line 16 dB above it against a threshold near 12 dB, and reported the evidence "is consistent with more than one fault type, and none is singled out"; on a benchmark of another public dataset it found support for the labelled fault, and for a second fault type, on all 44 recordings the reference study grades as clearly diagnosable. The released version 0.13.0 covers the first three example questions plus anomaly detection and report generation, while the baseline-comparison capability the article focuses on remains on the project's main branch and is unreleased.

A new vibration measurement arrives from a pump or a gearbox. The engineer responsible for that machine rarely wants a description of the signal. They want to know whether the machine has changed since it was last known to be healthy, what evidence supports that, and what to look at next. This article follows that question through an open-source system I maintain, Predictive Maintenance MCP, first as a calculation and then as a conversation with a language model. AI disclosure. The demonstration scripts, the conversation harness, the scoring of the model answers and both figures were produced with an AI assistant, which also drafted this text from the repository and the recorded logs. The example uses seven real recordings from a public bearing test rig and six recorded conversations with two language models. It is a controlled demonstration, not a field study, and I say where its limits are once the results are on the table. Predictive Maintenance MCP is an open-source server that makes condition-monitoring and diagnostic functions available to AI assistants through the Model Context Protocol, a standard way for an assistant to call external tools. The idea behind it is simple. Signal processing and diagnostic rules are things software can do reliably and reproducibly, so they belong in tested code. Understanding a question, choosing the right analysis and explaining the result in plain language are things language models do well. The server keeps the two apart, and it follows one design rule: it does not guess. If a unit, a shaft speed or a reference is missing, it says so. In practice, an engineer talks to an assistant and the assistant calls the server. A few typical questions show what sits behind it: The first three, together with anomaly detection and report generation, are in the published release, version 0.13.0. The last one, which this article is about, is on the project’s main branch and not yet released. A vibration spectrum breaks a signal into its frequencies. Some of those frequencies are predictable. A defect on a bearing race is struck each time a rolling element passes over it, at a rate fixed by the bearing’s geometry and the shaft speed, so a damaged outer race and a damaged inner race each have their own characteristic frequency 1 . For the bearing used here they fall near 81 Hz and 119 Hz. The server checks whether a line at each of those frequencies stands clearly above the surrounding noise, and it only counts a fault type as supported when a second line predicted by the same mechanism is also present. On one fault recording taken alone, the outer-race line was 35 dB above the noise floor and the inner-race line 16 dB above it, with a threshold near 12 dB. Both were supported, and the server reported that the evidence “is consistent with more than one fault type, and none is singled out”. That is an honest answer, and it is typical. On the project’s benchmark of another public dataset 2 , the check found support for the labelled fault on all 44 recordings that the reference study grades as clearly diagnosable 3 , and found support for a second fault type on all 44 as well. Those figures are the ones committed in the repository, measured about two weeks before the code used here. What resolves this kind of ambiguity is the thing an experienced analyst reaches for first, an earlier measurement of the same machine. A comparison needs information that is not in the waveform. In Predictive Maintenance MCP it is declared and recorded. Each recording comes with a small file stating which machine and measurement point it belongs to, when it was taken, and at what speed and load. The measurement point is described once, including its bearing frequencies. Then a set of recordings is declared as the baseline, meaning the state the machine is considered healthy in. The server records who declared it and quotes that name in every later assessment. The name is a plain text field, an attribution and not an authenticated sign-off. From there, every new recording is checked for comparability before it is compared. The rule in the code is short: a contradiction excludes, an absence qualifies. A recording in a different unit, or at a speed outside tolerance, is kept in the history but left out of the comparison. A recording with something undeclared, such as the sensor direction in this example, stays in with the caveat attached. The comparison itself is deliberately plain. For each indicator, the baseline defines a band, the mean plus or minus the larger of three standard deviations and 25%. A single recording outside the band is reported as unconfirmed. A change counts as persistent only when it repeats, for instance in three consecutive recordings. The recordings come from the MFPT bearing dataset 4 : three in baseline condition and three in an outer-race fault condition, all at the same load, shaft speed, sampling rate and duration. The dataset has no time axis, so dates a week apart were assigned, baselines first. With the three baselines declared as the reference, the server assessed each later recording in turn. After the first and second it returned “unconfirmed single acquisition”. After the third it returned “persistent change” and named what drove it: outer-race evidence “above the reference band in 3 of the last 3 acquisitions”. Figure 1 shows why the reference matters. The outer-race line stood at 17, 30 and 50 times its baseline level. The inner-race line, the one that made the single recording ambiguous, stayed inside its band, and two of the three baselines already showed a line there. And the overall vibration level, the RMS value that many monitoring programmes trend, also stayed inside its band, between 0.72 and 0.99 g against a baseline of 0.88 g. A bearing-specific indicator moved by an order of magnitude while the overall level reported nothing. Because the chronology is constructed, this shows the comparison working on real signals. It does not show a bearing wearing out. The three fault recordings are three captures of one condition, their rising values are simply the file order, and the dataset description in the repository does not say whether the baseline recordings were taken on the same bearing specimen. The result above is a block of structured data, roughly 19,000 characters long. The second half of the experiment put a language model in front of it. Six conversations were recorded through an MCP client, three with Claude Sonnet 5.5 and three with Claude Haiku 5.5, each with the same three questions. This is how one of them began: Engineer: Asset mfpt-rig, measurement point bearing radial. What has changed on this machine relative to its baseline? Tool called by the model: assess asset change Server: “persistent change at bearing radial of mfpt-rig over 3 post-reference acquisition s : evidence BPFO: supported bearing evidence above the reference band in 3 of the last 3 acquisitions, 3 consecutive…” Model: “Overall result: the assessment is a persistent change at bearing radial. It is driven by supported evidence at the BPFO line the outer-race line that stayed above the reference band in 3 of the last 3 acquisitions, 3 in a row. This is evidence consistent with an outer-race signature. It does not confirm a bearing defect.” The answer went on to name the baseline and who declared it, list what had and had not changed, and point out the undeclared sensor direction. Figure 2 summarises all six conversations. In every one, the model chose the right tool from a plain question and reduced the result to an answer of about 420 words that carried the verdict, its driver, the reference and its caveats. The second question asked why the change was called persistent and whether it meant an outer-race fault. All six explained the rule from the result they already had, without a new call, and all six said no: the evidence is consistent with an outer-race defect and does not confirm one. The third question followed a seventh recording, taken at half the sampling rate and for half the duration. The models loaded it, and all six explained correctly that the bearing evidence could no longer be compared with the baseline, which is different from saying it had gone. One put it as “unavailable, not cleared”. This is the useful part. An engineer who has never read an envelope spectrum can ask a direct question, get the assessment with its supporting evidence, and ask why. The models did not improve the diagnosis, and were not asked to. Every number in their answers came from the server. The models also added things. Asked what to do next, each gave several steps where the server returns a single suggested check: declare the sensor direction, confirm by inspection before drawing conclusions, record again at the baseline settings. That is reasonable advice, and one remark was sharper than the server’s own suggestion. Five of the six also stated, after the seventh recording, that the earlier finding still stands, which is true but is not something the server’s answer says. All of this is the model’s own contribution, and an answer does not always make clear which part came from where. And they passed on a mistake in the server. It labels one family of values with the unit g when they are in fact unscaled spectrum magnitudes, and two conversations duly reported an amplitude of “61,433 g”. The true figure is about 0.4 g. The comparison is unaffected, because it depends only on ratios between recordings of equal length, but the episode is instructive: a model that quotes its source faithfully also quotes its errors. Six conversations with two capable models from one vendor, asked the same questions, show what happened. They do not show how often it will. The answers were scored after the runs, without a blind grader, and the recordings are named after their fault condition, which may have helped the models even though three conversations said explicitly that file names are not evidence. One finding concerns the plumbing. An MCP server can send written guidance to the model, and the client used here delivered only about the first third of it. The guidance for the history tools never arrived, so the careful behaviour described above came from the tool descriptions and the results themselves. Guidance that matters belongs in the result, where it cannot be cut off. On the diagnostic side, seven recordings from one laboratory rig validate neither the thresholds nor condition monitoring over time on industrial machines. The baseline has three recordings, the minimum the code accepts, and the server itself flags its statistics as “relative only”. Finally, the comparison features are on the main branch at commit b73e332 and not in the published release. The server at that commit still reports version 0.13.0, so the commit is what identifies it. Connecting a language model to a spectrum function gives it something to describe. It does not give it anything to compare. A defensible answer to “what changed?” needs measurements that know which machine they belong to, a baseline that someone declared, a rule for what counts as comparable, and a result that names its evidence and says what was left out. Once that exists as software, a language model turns out to be a good way in: it finds the right function, explains the outcome and handles the follow-up question. For anyone evaluating an assistant of this kind, four questions are worth asking of any answer about change. Compared with what? Declared by whom? Under which conditions? Which measurements were left out? If the answer cannot say, it is describing one file. The recordings are from the MFPT bearing fault dataset. The data owner is Eric Bechhoefer, and the recordings were redistributed by MathWorks under CC BY-NC-SA 4.0 4 . They are included in a Git clone of the Predictive Maintenance MCP repository https://github.com/LGDiMaggio/predictive-maintenance-mcp , together with the signal processing, the comparison rules, their tests and the benchmark. The deterministic demonstration reproduces exactly from the commit named above. The conversations may differ on a re-run with the same models, and will differ with others. Asking an AI Assistant What Changed on a Machine https://pub.towardsai.net/asking-an-ai-assistant-what-changed-on-a-machine-d59c72af9b68 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.