Human-in-the-loop is not a theory. Here is how it actually works.

Research papers, trust frameworks and diagrams show where a human icon sits between two server icons. They do not tell you what human-in-the-loop actually looks like when you are building a live system for yourself, what goes wrong when you get the boundaries wrong, or how to decide where the checkpoints belong in your own workflow. Plenty of theory and research, far fewer real-world stories.

This article is Gysho's practical perspective. We have taken examples from five completed projects with different levels of human-in-the-loop, each giving a good insight into why, when and how humans are needed in current AI deployments. As a result of that experience, we apply certain decision points and principles in our own projects, which ensure humans are involved correctly regardless of the type of project or context.

01 · A simple definition of human-in-the-loop

Let us start by defining what human-in-the-loop (HITL) means.

At its core, human-in-the-loop means the human makes the decision and the AI prepares or recommends. The AI does not ship, finalise, or act autonomously at the point of consequence. A human verifies first.

This sounds simple until you try to implement it. At Gysho, our understanding of HITL was built over time through trial and error. It is not borrowed from a framework or external report.

Our most critical finding is that HITL is not a binary switch, autonomous or not. We treat it as a dial, where the setting depends on three factors:

• The operator's level of trust.
• The quality of the input data.
• The compliance context in which the output will be used.

Where input quality is poor, the system automatically raises the human oversight requirement. Where liability is high, the human remains accountable regardless of technical capability.

Three settings on the dial

The industry jargon around this is messy, so we use plain language. In practice, there are three settings on the dial:

• Human-in-the-loop (HITL): the AI prepares or recommends, and there is a hard gate where humans verify. This is the default for anything business-critical.
• Human-on-the-loop (HOTL): the AI runs on its own, but a human monitors with the ability to step in or override. The human does not check every action in real time, but retains veto power. This suits well-bounded tasks where individual error cost is low but volume is high.
• Human-out-of-the-loop (autonomous): the AI operates within guardrails defined at design time. The human is not involved in live operation. This only works when boundaries are hard, consequences are reversible or negligible, and the system has proven itself over time.

Think of these as operational settings. A system moves between them depending on trust, input quality and compliance requirements.

How we apply this framework comes from four years of R&D, experimentation and failures. During the development of Basewise, for example, we built and tested full autonomy. In practice it worked fine, but reality dictates that humans either want some degree of verification, or the liability requires it. That means the degree of autonomy in HITL is determined by context, not by technological capability.

02 · Five real-world examples

In our last four years of operating, we have touched on a range of HITL setups. Some failed where others succeeded. To bring HITL to life, we gathered five cases that give a practical perspective on how it is applied in different scenarios.

1 | Basewise: systems engineering in hyperdrive (win)

Basewise operates in systems engineering for large infrastructure projects. These have huge lists of requirements, making documents dense, technical and high-stakes. While the technology is capable of classifying and finalising requirements automatically, we have chosen not to give it that level of autonomy.

The outputs of Basewise carry a heavy compliance weight. If requirements are not accurately analysed and proven, they can cause project failure. Worse, if the work involves building a bridge, it could be built to the wrong standards and create safety risks. That means every output must pass human approval, though we can still optimise the work and support the reviewer.

The Evidence Finder is a particularly interesting case, since it needs to find proof beyond doubt that something has been done, working with information from different contexts and sources.

Across hundreds of tests during development, we brought reliability (defined as accurate, consistent output) from 42 per cent to 95 per cent. That is a great result, but it is not the full 100 per cent needed to rule out HITL completely. So the reliability score itself flags where HITL is mandatory and where the output can be trusted.

The five per cent is not treated as an error to be eliminated. We found the vast majority of those cases are inherently obscure and require human interpretation. The system modulates its own demands based on input quality and compliance needs. Autonomy is technically possible. In this context, it is not appropriate.

2 | Hydrovane: autonomy with an emergency stop (win)

Hydrovane, a marine equipment manufacturer, runs a 24/7 customer service agent. On routine enquiries such as part numbers, troubleshooting and documentation requests, the system operates in a near-autonomous mode bounded by design-time guardrails.

It carries two escalation tiers:

• A caution tier warns the user and continues. This sits closer to human-on-the-loop oversight and flags to the user to stay alert.
• A hard-escalation tier stops all output immediately and routes to a human inside the Hydrovane team, which is pure HITL.

The triggers are the critical design choice. Vessel-distress patterns and safety-critical scenarios are defined by marine-safety experts in plain language, not hard-coded by software engineers. When a pattern matches, the AI does not attempt to advise. The agent monitors the conversation, with NLP (natural language processing) programmatic failsafes that hand off to a human when needed.

This HITL setup is more dynamic and risk-focused.

3 | ProjectOS: machine-human symbiosis (win)

ProjectOS is an application Gysho developed to perfect its own delivery process and achieve its own mission. The design recognises that there are tasks AI is better at performing than humans, such as ingesting and analysing large volumes of project data.

In this application, AI prepares data ingestion but includes a HITL step before it is allowed to finalise ingestion and analysis. The system watches for emails and receives input from the Gysho team, such as transcripts, design documents and anything else relevant to a project. AI analyses, classifies and prepares how that data lands in the project environments, such as execution boards, decisions and risks. Humans review the input and approve it before it is finalised in the project's central repository.

Once verified, ProjectOS agents work with the data to keep projects running smoothly. It automatically performs risk assessments, monitors project progress (completed actions) and flags if a project is at risk of needing extra capacity. Beyond historical reports, it runs a predictive algorithm that tells us in advance if a project is likely to run into problems. Once the data is approved, the system runs fully autonomously.

HITL here is a clear, simple checkpoint. AI starts the admin process, doing what it does best: processing large quantities of data, clarifying it and preparing proposed input. Humans review and approve, which ensures the input data is of the right quality. From there, AI works with the validated dataset autonomously.

4 | Australian civil engineering: when autonomous fails and HITL is needed (lesson)

For a large Australian civil engineering firm, we developed a solution to allocate and verify requirements on major civil infrastructure projects.

The original design required human input at the start, to provide a high-level classification of the input data, which allowed every following step in the analysis to use the right context. It was a clear HITL step that set the right foundations for every AI step that followed, but it also took a long time to complete.

In a later iteration we wanted to push the process further and hand more work off to agents, which included the classification step. We observed that the AI was not able to derive enough context from its input data, or did not show enough model intelligence, to provide a reliable classification.

The result was much like a hidden factory. Because the input data carried a small degree of drift, every subsequent AI step added slightly more. When that happens across a process of multiple steps, even small drift causes the reliability of the final output to drop exponentially. The end result was unusable.

When we analysed the cause, we determined that the drift came from data ambiguity. Even the best model could not achieve human-grade interpretation in ambiguous cases. That drift even carried through to the reliability score the system produced, making the results misleading. When we extended the model's context with more data and expertise, the results were still lacking, and the cost quickly began to outpace human intervention.

The lesson is that humans are good at interpreting edge cases and AI is not. When those edge cases are foundational to the rest of the process, even small deviations cause much larger issues further down the line. The original HITL setup was the right approach. It was the most reliable, fastest and cheapest way of getting good output.

5 | The Negotiation Group: the self-degrading instruction loop (lesson)

The Total Negotiation Group (TNG) is a global training and consultancy firm in the UK. They identified the opportunity AI presented to their expertise and decided to turn their IP into an agentic platform, extending how they support their customers.

In an early experiment we created a Negotiation Planner, a chat-based assistant that uses TNG expertise to build highly sophisticated negotiation plans for their users. To improve performance, Gysho built a test-analyse-iterate loop, operated entirely by agents, which would simulate real-world cases and identify and implement improvements. Critically, this loop was allowed to make core instruction changes, which granted it autonomous self-modification.

Over fifty to sixty turns, the instructions drifted. The original prompt linked tightly to TNG's proprietary negotiation IP. The iterated version gradually replaced that specific expertise with generic model knowledge. The system was optimising for surface-level coherence while eroding the very domain expertise that made the output valuable. Quality degraded until the results were unusable. We observed the same behaviour in Gysho's internal HAL R&D project, where the intention was to create self-modifying systems that iterate autonomously.

The lesson is that AI drift is real. If we allow systems to edit themselves, they drift away from their original instruction, believing they are doing the right thing. HITL is needed to protect and guard a system's original purpose. Leaving systems to iterate without any observation opens the door to completely different outputs and unexpected actions.

03 · How to find your HITL points

The cases here show a similar pattern: the decision about what we hand off to AI and where humans are needed is not generic or one-size-fits-all. It is a purpose-built assessment, shaped by the process and the data.

At Gysho we have distilled this experience into a reliable approach that we use in every project today.

1 | HITL is not generic

HITL depends on the characteristics of the process and the data. Some processes are very simple, administrative execution with non-business-critical data. Those can earn a large degree of autonomy quickly, and the impact of mistakes is small. Others carry a large compliance risk and may use data that is not perfect.

Every AI implementation uses a bespoke HITL setup; it depends on the context and the risk. There is no single answer to where HITL lives.

2 | Default to HITL, earn autonomy

Every new AI system should default to HITL, since its performance and output cannot be predicted up front. We have seen many cases where AI models trip over a simple prompt contradiction, or where something hidden in a large dataset causes unexpected results.

When we engineer a system and observe its behaviour, we iterate and perfect its operational context to produce more reliable outputs. Once a system has run for a certain time and proven the quality of its output, it can earn more autonomy one step at a time. Autonomy is a long-term goal; HITL safeguards the short term.

3 | Map to the risk

Not every process carries the same level of risk. Ensuring a construction project is built to the relevant safety standards carries a far higher compliance risk than an internal sales-email processor. Identifying and classifying the risk up front defines what HITL looks like.

In some cases HITL is non-negotiable. Humans are permanently responsible for the output a system produces and need to verify it as such. Others simply do not carry that much risk and become candidates for autonomy. Often HITL is the result of risk management, not model capability.

4 | Implement reliability floors and monitors

Even in the best cases, AI will encounter ambiguity, which is where active quality scoring and monitoring come into play. Much like humans, the output of AI depends on different factors, and sometimes it is not certain of the right decision to make. Rather than allowing AI to make the call, it should escalate to a human.

Our solutions include internal peer reviews, scorecards and HITL indicators in their output. This gives AI a route to involve humans when in doubt. That dynamic routing caters for all the edge cases you cannot engineer for, and safeguards against hallucination.

5 | Keep it simple

There is a real temptation with AI to build large, complex systems where multi-agent frameworks work together on large tasks. In most cases, especially at the start, that is a recipe for trouble. The observability of AI decision-making diminishes in multi-agent frameworks, which makes them harder to perfect.

Identify a process, map out the steps and the risks, and decide where AI can deliver a quick win to get started. Grow the system over time as it earns trust and has solid reliability reporting built in. It is often more work to find an issue in a complex system than to perfect a small one. In that sense, the value of AI is the sum of its components.

04 · No silver bullet, common sense applies

Over the past weeks we have watched discussions on social media where people raise ideas about how to handle HITL, only to be challenged by others on whether they apply in certain industries or business types.

There is no single answer that is right or wrong. There is no silver bullet. HITL depends entirely on the process, your organisation and the data you work with.

HITL is not complicated. It forms an integral part of the design process and changes over time.

It is dictated by what practitioners know best: the risk in the process, the quality of the data and the compliance needs of the work. The tools to create a solid HITL setup already exist in the organisation. It is a matter of surfacing them in the right way and perfecting over time.