Wealth Think

4 questions to help advisors avoid AI tools with risky embedded flaws

As AI adoption spreads across wealth management, more platform providers are deploying the phrase "human in the loop" as a verbal safety net to establish trust and credibility with RIA leaders and advisors. 

Processing Content
Rachel Schwab image
Rachel Schwab is director of product at FP Alpha.

However, when AI providers don't explain what the term actually means, they ask advisors to bet client outcomes and their own reputations on AI tools without a real understanding of how the outputs are generated or validated. 

Evolution of an AI catchphrase

Originally, "human in the loop" referred to how the AI model itself was built or "taught." Human experts — accountants, lawyers, estate planners — review documents and label what "correct" looks like in, for example, a tax return line item, estate document clause or insurance policy term. In that way, the model learns from a human's professional judgment instead of guessing from raw data alone. That kind of training is the oldest, and still most common, meaning of the term. 

The second definition comes into play after the model is built. Experts review the outputs, correct what it gets wrong and rank answers by accuracy. Corrections get fed back into the system, sharpening it over time. What's come to be known as "reinforcement learning from human feedback" comes closer to what most people mean when they talk about AI getting smarter with use.

READ MORE: 5 acute AI compliance risks advisors face — and 5 fixes

Advisor as AI approval gateway

Now, a third definition of "human in the loop" is gaining traction in discussions between technology and advisory firms. As AI agents start taking independent actions — sending emails, booking meetings and moving money — the coinage can describe the advisor's job of approving what the tools are about to do in the real world. 

While the first two definitions describe how AI tools' output quality improves, the third refers to an approval gate — often the financial advisor — that helps avoid unintended consequences. Take a tax planning tool that flags a Roth conversion opportunity for a client and drafts a client email recommending it. 

A careful advisor won't just hit approve, they'll check the client's current and projected tax brackets, confirm the conversion doesn't push the client into a higher income-related monthly adjustment amount, verify the state tax treatment and cross-reference the recommendation against the client's broader financial picture. 

Yet taken on its own, the third definition is risky because it says nothing about how the model got smart enough to make the recommendation an advisor is now approving. 

A solution provider can build an AI model with zero expert involvement in training, feed it messy or unvetted data and still market it as "human in the loop" simply because an advisor clicks "approve" before an email goes out. The phrase covers the click, but with no insight into what the model previously learned or from whom. 

READ MORE: When vibe coding goes wrong: The risks advisors can't ignore 

Which human? What loop? 4 essential questions

This raises a red flag because that last review is only as good as the advisor's ability to catch what's wrong. If the model was trained without expert human input, a flawed assumption may be buried in the calculation — an outdated tax rule, a misread filing status — and can slip past a diligent advisor because the recommendation itself looks reasonable. 

The advisor is still doing the work. They're just doing it downstream, hopefully catching errors that expert-validated training would have caught before the recommendation ever reached them. 

That's why asking a technology vendor if the AI platform had a human in the loop isn't specific enough. Vague language lets providers imply whichever of the three "human in the loop" definitions sounds best without committing to any of them.  In addition, advisors should ask:

  1. Who trained the model, and what were their credentials?
  2. What data did the trainers use?
  3. At what point were humans involved?
  4. In which parts of the training were humans not involved at all?

Firm leaders and advisors deserve transparency about the kind of human oversight their AI tool has. Only then can they make fully informed decisions about the tools they use to serve their clients.


For reprint and licensing requests for this article, click here.
Artificial Intelligence Technology Compliance Wealth management
MORE FROM FINANCIAL PLANNING
Load More