AI Without the Hype12 min read

Who's Protecting Your Business When an AI Agent Gets Access?

Everyone wants an AI agent working for their business. Fewer people are asking what happens when that agent makes a mistake.

Jacquelyn Mosier
Jacquelyn “Jackie” Mosier

Founder & Principal Consultant, Jack-IT Consulting

I recently had a conversation at a networking event that reminded me just how dangerous blind trust in technology can become.

Someone was telling me about an individual who had started a company using AI agents for investment-related activities. The person reportedly had very little experience with artificial intelligence. As the conversation continued, questions came up about cybersecurity, liability insurance, and the protections that should exist when technology is interacting with financial information.

What concerned me wasn't simply the business concept. It was how quickly people seemed willing to trust the technology without understanding the safeguards surrounding it.

Then the person I was speaking with started explaining an application they were developing. I began asking questions. What happens when something goes wrong? What protections are in place? Who's responsible for the information being accessed? What happens if the system makes an incorrect decision?

They acknowledged that some of those considerations hadn't occurred to them, even though they were already working with someone they considered an expert.

And that's where I think we have a much bigger problem.

We've reached a point where people are trusting the word "expert" more than they're evaluating the expertise behind it.

Just Because Someone Can Build It Doesn't Mean They Should Be Trusted to Run It.

AI has lowered the technical barriers to creating applications, automations, and software products. That's exciting. People can prototype ideas, connect systems, and develop tools faster than ever before.

But there's a difference between building something that functions and building something that's appropriate for a business to rely on.

A developer might know how to connect an AI model to a database. That doesn't automatically mean they've addressed cybersecurity, access controls, regulatory requirements, incident response, or contractual liability. Someone might know how to create an AI agent that initiates transactions. That doesn't mean they've established the safeguards needed to prevent unauthorized or incorrect actions. And someone might demonstrate an impressive application. That doesn't prove the underlying architecture, security practices, or business controls have been independently evaluated.

Technical capability and operational responsibility are not the same qualification. Businesses need to understand that distinction before granting anyone access to their systems.

An AI Agent Isn't Just Another Chatbot.

Let's clarify something. There's a meaningful difference between asking an AI assistant to help draft an email and allowing an AI agent to perform actions inside your business.

A conversational assistant may generate information for an employee to review. An AI agent can potentially interact with connected applications, retrieve records, update information, trigger workflows, or execute tasks using permissions granted to it. Depending on how it's designed, it may operate across several systems.

Think about what that could involve. Your CRM. Customer records. Financial information. Employee data. Internal documents. Email accounts. Accounting platforms. Operational systems.

Now ask yourself: Would you give a new employee unrestricted access to all of those systems on their first day?

Probably not. You would establish responsibilities, determine what information they needed, restrict access where appropriate, and provide oversight.

So why would an AI agent receive fewer controls? Because it's software? Because someone described it as autonomous? Because the demonstration looked impressive?

None of those are valid substitutes for access management and accountability.

Your Data Doesn't Become Less Sensitive Because AI Is Using It.

One of the most important questions businesses should ask is what happens to their information once an AI solution accesses it.

Where is the data processed? Where is it stored? Is information retained by the provider? Is customer data used to improve or train models? Which third parties receive information? How long is it retained? Can it be deleted? And what contractual protections exist?

The answers vary depending on the platform, provider, configuration, and agreement. That's exactly why assumptions are dangerous.

A vendor saying its solution is "secure" isn't enough. A company claiming to use encryption doesn't tell you everything about its access controls, data handling, or incident response capabilities. And a provider mentioning compliance doesn't automatically mean its product meets the specific obligations your organization faces.

If you're handling financial records, protected health information, confidential customer data, or other sensitive information, those distinctions matter.

You cannot outsource your responsibility to protect information simply because you've outsourced the technology handling it.

Who's Liable When the AI Gets It Wrong?

Let's imagine a business implements an AI agent that interacts with customer accounts. The system receives a request. It interprets the information incorrectly. It updates the wrong record, sends confidential information to an unintended recipient, or initiates an action that should have required approval.

Now what? Who identifies the mistake? Can the action be reversed? Who notifies the affected parties? Does the provider's contract address the incident? What insurance coverage, if any, applies? And who ultimately bears the financial consequences?

Those aren't hypothetical categories of risk. Incorrect outputs, inappropriate access, and unauthorized actions are issues organizations need to plan for when designing AI-enabled workflows. The specific legal responsibility depends on the facts, contracts, applicable laws, and circumstances of an incident.

But that uncertainty is precisely why the conversation needs to happen before implementation. Not after something goes wrong.

If nobody can explain who's accountable when the system fails, you haven't finished evaluating the solution.

Insurance Isn't a Substitute for Security. But You Should Still Be Asking About It.

One of the questions that came up in that networking conversation was whether the company providing the AI solution carried appropriate insurance. That's an important question. Not because an insurance policy makes a technology provider trustworthy. It doesn't. But because businesses should understand how a vendor manages professional and cyber-related risks.

Depending on the services provided, relevant coverage might include technology errors and omissions insurance, cyber liability insurance, or other policies appropriate to the provider's operations. Coverage varies significantly. Limits, exclusions, deductibles, and policy conditions matter. A certificate of insurance alone doesn't prove that a particular AI-related incident would be covered.

That's why vendor due diligence shouldn't stop at asking whether a provider is insured. Ask what risks the policy is intended to address. Ask whether the contract appropriately allocates responsibility. Ask about incident notification, indemnification, service limitations, and the provider's obligations if something goes wrong. And when the exposure is significant, involve qualified legal, insurance, and cybersecurity professionals.

A responsible technology provider should be prepared to discuss risk, not just functionality.

The Billion-Dollar Company Argument Doesn't Protect You.

I've heard people justify questionable technology decisions by pointing to the companies already using AI. "Major financial institutions are investing billions in this." "Everyone is implementing agents." "Large corporations are already doing it."

Okay. And large organizations with substantial security budgets, compliance teams, and experienced technical leadership still experience breaches, outages, fraud, and operational failures. Their investment in technology doesn't eliminate risk. It means they have resources to manage certain risks that smaller businesses may not.

So why would a small or midsize business assume that an unfamiliar AI provider has everything covered? Especially when the provider may be a new company without an established operational history.

The fact that a technology is gaining adoption doesn't prove that every implementation is appropriate or every vendor is qualified.

Popularity is not due diligence. And your business doesn't become protected simply because other companies are taking similar risks.

The Questions Your AI Vendor Should Be Able to Answer.

Before granting an AI solution access to sensitive business systems, leadership should be asking questions in several areas.

Access and permissions: What systems will the AI connect to? What information can it read? What records can it modify? Can it initiate transactions or communicate externally? Are permissions restricted to what the system actually needs? Can access be immediately revoked?

Security and data handling: Where is information processed and stored? Which third-party providers are involved? What protections exist for sensitive information? How are credentials and integrations secured? What happens to business data when the contract ends?

Human oversight: Which actions require approval? Can employees review the AI's decisions? Are actions logged? How are errors detected and corrected? Is there a reliable way to stop the system when something unexpected happens?

Vendor accountability: Who is responsible for maintaining the system? What support is available during an incident? What contractual protections exist? Does the vendor carry appropriate insurance? What happens if the provider goes out of business?

Business justification: What problem is the agent solving? Why does that problem require AI? Could a traditional automation accomplish the same result with fewer risks? How will the organization measure performance? And what happens if the solution doesn't deliver the expected value?

These aren't questions designed to intimidate vendors. They're questions a serious provider should expect. Not every answer needs to be complicated. But the answers should be clear, documented, and appropriate to the level of access and risk involved.

Your Internal Team Needs Guardrails Too.

Vendor accountability is only part of the equation. Businesses also need to examine how their own employees are using AI.

Are employees entering confidential information into tools the organization hasn't approved? Are teams connecting third-party applications to company accounts without reviewing permissions? Are AI-generated recommendations being accepted without verification? Does leadership know which tools have access to customer information? And are employees trained to recognize when an AI output may be inaccurate or inappropriate?

The risk doesn't always come from a malicious actor. Sometimes it comes from a well-intentioned employee trying to save time.

That's why AI governance needs to be practical. Clear policies. Appropriate permissions. Relevant training. Human review where necessary. Defined accountability. And regular evaluation as tools and business requirements change.

Governance shouldn't exist solely in a document nobody reads. It needs to be reflected in how people actually work.

Being an AI Expert Should Mean Knowing When Not to Use AI.

Here's where my perspective may differ from some of the companies aggressively promoting AI agents.

I don't believe every business needs autonomous systems operating across its technology environment. I don't believe every manual process requires artificial intelligence. And I don't believe being first to implement something automatically creates a competitive advantage.

Sometimes a traditional automation is more reliable, easier to audit, and better suited to the task. Sometimes an employee should remain responsible for reviewing a decision. Sometimes the underlying process needs to be redesigned before any technology is introduced. And sometimes the risk simply outweighs the potential benefit.

That doesn't make a business anti-innovation. It makes the business intentional.

Understanding AI should make you more selective about where you use it, not more eager to put it everywhere.

What This Means for Jack-IT Consulting.

At Jack-IT, we help businesses evaluate technology through the lens of operations, strategy, people, and risk. That includes examining AI use cases before implementation, understanding how systems interact, evaluating vendor recommendations, and identifying where governance and employee oversight are needed.

We're vendor-neutral. We don't have to recommend an AI agent simply because it's the product someone wants to sell. And we don't assume that adding more technology automatically improves a business.

When specialized cybersecurity assessments, legal advice, or insurance reviews are required, those responsibilities belong with appropriately qualified professionals. Our role is to help businesses recognize those needs, ask informed questions, and make sure technology decisions are considered within the broader business strategy.

Because implementing AI responsibly requires more than knowing how to connect an API. It requires understanding what you're connecting it to, what it can affect, and what the organization is prepared to manage.

If this system made a serious mistake tomorrow, would you know what happened? What information was affected? Who was responsible? How to stop it? And what your next steps would be?

If the answer is no, you have more work to do before handing over access. That's not fear. That's leadership.

We're entering an era where technology can act on behalf of businesses in increasingly powerful ways. The opportunity is real. So is the responsibility. And the companies that navigate this well won't necessarily be the ones deploying the most agents. They'll be the ones that understand exactly what they're allowing those agents to do.

Don't Confuse an Impressive AI Demonstration With a Responsible Business Decision.

Jack-IT Consulting  |  Strategy Before Software. Always.

Jacquelyn Mosier — Founder and Principal Consultant, Jack-IT Consulting
Jacquelyn “Jackie” Mosier

Founder & Principal Consultant, Jack-IT Consulting

Technology Strategy  ·  AI Advisory  ·  Business Operations  ·  Executive Leadership

Jacquelyn Mosier is the Founder and Principal Consultant of Jack-IT Consulting. With a background in national sales leadership, team development, and business operations, she built Jack-IT on a straightforward premise: technology decisions should start with the business, not the software. She works directly with leadership teams to evaluate investments, identify operational gaps, and build strategies that connect people, processes, and systems before recommending what comes next.

Don't Confuse an Impressive AI Demonstration With a Responsible Business Decision.

Before giving an AI provider access to your customer information, financial systems, or internal operations, make sure you understand the technology, the vendor, and the risks involved. Jack-IT Consulting helps businesses evaluate AI readiness, review technology strategies, assess operational implications, and identify the questions that need to be answered before implementation. We don't sell AI for the sake of selling AI. We help businesses determine whether it belongs in their operations in the first place.