Lab Instrument Integration: Why It Breaks in Year Two

Why does an integration that passed acceptance fail a year later?#

Because acceptance testing proves the integration works against the lab as it is on that day, not as it will be. In the year that follows, the lab does what labs do: it renames a field to match a new SOP, archives the account of a technician who left, upgrades the instrument software, and the vendor on the other side changes an export format in a release nobody read the notes for.

Each of those is routine. None of them looks like a change to the integration, so none of them is raised as one. The integration finds out first, and it finds out silently. A file is written to a folder nothing reads any more. A value lands in a field with a slightly different name and sits there, unnoticed, until an audit or a customer asks a question.

The pattern is the same whoever built the integration. Its dependencies are the lab’s own names, accounts and formats, and the lab treats those as housekeeping. The fix is not better code. It is deciding, in writing, what the integration may depend on, and treating any change to it as a change request.

Whose identity does the integration run under?#

Every integration acts as somebody. When it reads a record or writes a result, the receiving system attributes that action to an account, and the audit trail records that account’s name. In most labs the account is whoever set the integration up, because that was the login to hand on the day. That person’s credentials are now embedded in a script, and the script will do everything they are allowed to do, for as long as they are allowed to do it.

The failure follows on its own. The person changes role and loses a permission the integration relied on. They leave, and offboarding disables the account, which is exactly what offboarding should do. Or the credential is rotated for a good security reason and nobody remembers the integration holds a copy. In every case the integration stops, and the audit trail up to that point says a named person did work they never saw.

There is a second, quieter consequence. An integration sees what its account sees, and no more. If the account is not a member of the group that owns a set of records, a request for them returns an empty list rather than an error. Nothing is wrong, from the system’s point of view. It is looking through the eyes of someone who cannot see that data, and hours of debugging get aimed at the code.

The rule that prevents both is simple. Give every integration its own credential, issued for it alone, with the narrowest access it needs. Revoking that one credential stops that one thing. The account’s membership becomes a documented part of the integration, and offboarding a person never touches it.

What happens when someone renames a field?#

An integration finds a field by its name. That is true of nearly every laboratory system, because the name is the one thing a person and a program both understand. So when a quality manager renames Incubation temp to Incubation temperature (°C) to match a revised method, the integration goes looking for a field that no longer exists. It either fails loudly, skips the value, or creates a new field with the old name beside the new one. The third is the worst, because everything looks complete.

Renaming a field is a change request, not housekeeping, and the lab and the integrator should agree that field names are an interface rather than a label. In practice that means a short list of fields the integration depends on, kept where the people who edit templates will see it, and a rule that those names change only when the integration changes with them.

The same discipline applies to values. A reworded dropdown option, a unit that moves from the value into the field name, a date format that changes with a regional setting: each is invisible to the person making the change and fatal to the program reading it. The agreed list should say what shape the values take, not only which fields are fixed.

How should a lab find out that something changed, when nothing announces it?#

Most laboratory systems do not push events. There is no message when a result is added, a record is edited or a field is renamed. If an integration needs to know that something happened, it has to go and look. This is true of far more systems than vendors like to admit, and it is the reason so many integrations are designed around an assumption that turns out to be false.

Looking well is a discipline of its own. The integration should ask for records changed since the last time it looked, keep a marker for where it got to, and move the marker only when the work is done. It should ask at the rate the lab actually needs, which is usually minutes rather than seconds, because nothing in a laboratory changes in milliseconds. And when the far side does not answer, it should wait longer before asking again rather than retrying in a tight loop, because an integration hammering an unwell server is what turns a blip into an outage.

The lab’s side of this is monitoring. An integration that has not written anything for a day is either idle or broken, and the lab should know which. The cheapest check is a count: how many results did each integration file this week, against how many the instrument produced. When those numbers drift apart, something changed. Every integration should log what it sent and what came back, without the credential, so that when a record is wrong six months later the question of what was sent can be answered.

What should the lab agree with the integrator before connecting anything?#

The agreements below are short, and every one of them is easier to make before the first connection than after the first failure. They are the difference between an integration that is somebody’s responsibility and one that is discovered to be broken by an auditor.

None of this is specific to one product. It is what a regulator asks for under the heading of controlling an electronic system: knowing what it depends on, and knowing when that changes. An integration that meets the list above is one a lab can defend. One that does not is a year away from an unexplained gap in its records.

What an instrument integration carries in this work#

LabLynx builds instrument integration as a service and as a product line, and has done for long enough to have seen every failure above in a customer’s lab. The connection is the easy half. The hard half is the year after, and the checklist above is the shape of what our integration engineers agree with a lab before they start.