AI is undergoing rapid, broad adoption. Virtually everyone is using AI in their day-to-day lives, whether they recognize it or not. In your personal life, you are likely using AI when accessing the world’s most popular entertainment apps, and it has become just as prominent in the workplace. Even if you’re not using an application marketed as AI-first, like a transcription application, it is still likely to incorporate AI features.
To respond to emergent AI risks, CBIZ will periodically publish editions of this Cautionary Tales series, reporting on the top AI-related risks of the moment. We hope that doing so will raise awareness of emerging risks and track existing risks as they become more consequential.
Our goal with the Cautionary Tales series is to inform readers so they can identify threats and implement practices and procedures to minimize their exposure.
Why It’s Important to Know About AI Risks
As an emergent technology, AI can’t benefit from years of regulation, the application of legal theories, or proven best practices. Currently, we can look only at sparse legislation governing the use of AI and commentary from authorities on how they would like regulations to work once they are written and in effect.
Anti-bias laws have been among the earliest forms of AI regulations, and, importantly, they hold businesses using AI tools – not those developing and selling them – responsible for ensuring that AI does not introduce bias into the hiring process. Anti-bias in hiring laws is currently pending or in effect in California, Colorado, and New York, among others.
The current US presidential administration has expressed concern that a patchwork of AI regulations across states could create headaches for businesses operating across jurisdictions throughout the country. But it remains to be seen whether uniform federal AI regulation would effectively address the concerns of individual states. The more pressing question may be: will future AI regulations continue to make those using AI, rather than entities developing and selling AI tools, responsible for the technology’s errors and misuse.
Three Classifications of AI Risks
Before diving in to today’s top risks, it’s important to recognize that the AI risks featured in CBIZ’s threat reports come in three forms: (i) those associated with the use of applications with AI features, (ii) those associated with the development or incorporation of AI tools within business environments, and (iii) those associated with the adversarial use of AI against your organization.
Use Risk
Use risks arise when AI tools and features are adopted without formal approval or oversight and is often referred to as shadow AI. This unsanctioned use can lead to data leakage, regulatory non-compliance, and increased exposure to insecure platforms. Effective governance is essential to manage these risks and should include clear AI usage policies, employee training on the implications of insecure or unauthorized AI tools, and the use of approved, secure AI alternatives that meet organizational security and compliance standards.
Development Risk
Development risks stem from the design, training, or deployment of AI systems without sufficient risk management/governance, potentially leading to insecure model behavior, data exposure, or unintended outcomes. To address this risk, organizations should implement strong governance frameworks and integrate threat modeling early and throughout the AI development lifecycle. Threat modeling helps identify potential attack vectors, misuse scenarios, and system architecture vulnerabilities before they reach production environments. Governance processes should define how models are selected, trained, validated, and deployed (e.g., model whitelisting), including controls over data inputs, model access, and downstream use. Aligning these practices with established standards like the NIST AI Risk Management Framework (AI RMF) and ISO 42001 ensures a structured, risk-aware approach to AI development that supports security, accountability, and compliance.
Adversarial Risk
Adversarial risks emerge when threat actors leverage AI to target, deceive, or exploit your systems, users, or supply chain. These risks include AI-enhanced phishing, automated vulnerability discovery, deepfakes, and data poisoning attacks that are designed to manipulate your users and disrupt your operations. Organizations can mitigate adversarial AI risks by adopting threat-informed defense strategies (e.g., threat modelling), enhancing their organizations’ detection and response capabilities with AI-enabled security tools (i.e., fight fire with fire), and promoting security awareness training focused on emerging AI-enabled attack techniques.
AI Risks by the Headlines
Having introduced our threat report and briefly explained why AI can represent risk, here are the top AI risks businesses should be aware of today:
- Deloitte will pay back the Australian government for a portion of a $290,000 report that was found to contain AI hallucinations. Chris Rudge, a researcher with Sydney University initially recognized an AI error attributing a non-existent publication to a colleague before going on to discover that Deloitte’s report was “full of fabricated references.”
- Samsung employees leaked confidential information by entering it into ChatGPT. In doing so, Samsung’s employees may have inadvertently exposed privileged corporate information, including code and a recording of a meeting. Since ChatGPT is not a secure platform, the information and data stored in its database may be retrievable by others. In response to this leak, Samsung instituted a ban on the use of generative AI tools.
- A US District Court judge recently allowed a putative class action to proceed against a Medicare Advantage organization (MAO) that is alleged to have used AI to make coverage decisions. Specifically, the case hinges on language in the MAO’s contract that specifies coverage decisions will be made by “clinical services staff” and “physicians,” not AI.
- As early as 2018, the ACLU published coverage of an automated hiring program at Amazon that discriminated against women. The ACLU’s early warning (“these tools are not eliminating human bias – they are merely laundering it through software.”) may be one reason that AI’s earliest regulations center on controlling for bias in the hiring process.
- A California-based car dealership walked back the launch of its ChatGPT powered chatbot after a savvy user manipulated the chatbot into offering a $76,000 vehicle for $1. The dealership didn’t honor the chatbot’s generous proposal, but it’s a useful reminder not to give AI systems undue influence over financial decision-making.
- A company providing AI chatbot services and its co-founders are being sued by the families of suicide victims who say family members were harmed by interactions with the chatbots. The chatbots were said to have exacerbated troubling behavior, neglected to flag language suggesting self-harm, and failed to encourage users to pursue help.
- Anthropic, which launched the AI platform Claude, was misused by an attacker who recruited the program to carry out an extortion scheme. The AI is alleged to have perpetrated virtually every stage of the attack, including stealing privileged information like bank details and social security numbers, and sourcing sensitive information from sources of bulk data.
Conclusion
As AI becomes increasingly influential, additional risks are sure to emerge. To protect your business from AI risks, heed these cautionary tales, understand how the technology is being used and misused, and ensure your business is using safe, secure AI tools alongside controls designed by professionals. If you have any questions about controlling your organization’s AI risk, contact CBIZ today.
© Copyright CBIZ, Inc. All rights reserved. Use of the material contained herein without the express written consent of the firms is prohibited by law. This publication is distributed with the understanding that CBIZ is not rendering legal, accounting or other professional advice. The reader is advised to contact a tax professional prior to taking any action based upon this information. CBIZ assumes no liability whatsoever in connection with the use of this information and assumes no obligation to inform the reader of any changes in tax laws or other factors that could affect the information contained herein. Material contained in this publication is informational and promotional in nature and not intended to be specific financial, tax or consulting advice. Readers are advised to seek professional consultation regarding circumstances affecting their organization.
“CBIZ” is the brand name under which CBIZ CPAs P.C. and CBIZ, Inc. and its subsidiaries, including CBIZ Advisors, LLC, provide professional services. CBIZ CPAs P.C. and CBIZ, Inc. (and its subsidiaries) practice as an alternative practice structure in accordance with the AICPA Code of Professional Conduct and applicable law, regulations, and professional standards. CBIZ CPAs P.C. is a licensed independent CPA firm that provides attest services to its clients. CBIZ, Inc. and its subsidiary entities provide tax, advisory, and consulting services to their clients. CBIZ, Inc. and its subsidiary entities are not licensed CPA firms and, therefore, cannot provide attest services.
