---

title: “What Risks Can Come From Using Generative AI? 14, Ranked” slug: generative-ai-risks description: “See which of 14 documented failure modes apply to you, from prompt injection to insecure generated code, plus the one control that cuts each one.”

What Risks Can Come From Using Generative AI? 14, Ranked

What risks can come from using generative AI depends less on the model than on your seat. The hazards fall into five families: accuracy (hallucination, brittleness, drift), confidentiality (input leakage, retention, RAG over-exposure), security (prompt injection, poisoning, insecure code, supply chain), legal (IP, privacy law, disclosure duty, accountability gaps) and human (automation bias, shadow AI, deskilling, energy cost). Someone pasting a client email into ChatGPT carries a different set from a team shipping retrieval on Azure OpenAI. Both differ again from a CISO approving Copilot, Gemini and Claude for 4,000 staff.

Here is the catalogue, sorted so you can find yourself in it.

#	Risk	Who it hits first	How it surfaces	The control that reduces it most
1	Hallucination	Individual user	A confident answer with a fabricated citation or API method	Require retrievable sources; verify before reuse
2	Brittleness	Builder	Quality collapses on inputs outside the training distribution	Test on jargon-heavy and out-of-distribution inputs
3	Model drift	Builder	Prompts that worked last month return different output	Pin versions; contract for change notice; regression-test
4	Input leakage	Individual user	Non-public text typed into a consumer endpoint	Use a tier that excludes inputs from training
5	Provider retention	Enterprise buyer	Prompts stored indefinitely, discoverable in litigation	Contract a retention window and a deletion path
6	RAG over-retrieval	Builder	Index surfaces data a user could access but never find	Permission filtering at retrieval time
7	Direct injection	Builder	User input overrides the system prompt	Treat model output as untrusted
8	Indirect injection	Builder	Payload sits in a retrieved page, PDF or ticket	Sandbox tool calls; approve external effects
9	Data poisoning	Enterprise buyer	Tampered samples; behaviour normal until triggered	Data provenance; holdout trigger testing
10	Insecure generated code	Builder	Vulnerable snippet merged because it looked right	SAST in the IDE; provenance labels on AI commits
11	Supply chain	Builder	Malicious dependency in an ML toolchain	Pin dependencies; verify sources; watch advisories
12	Trade-secret exposure	Enterprise buyer	Confidential material sent to a processor without terms	Legal review of the DPA before adoption
13	Automation bias	Individual user	Fluent output approved without checking	Make the reviewer’s job finding the error
14	Shadow AI	Enterprise buyer	Staff using unapproved tools with real data	Ship a sanctioned tool that is faster

Figure 1: The 14 risks scored against individual user, team shipping a feature, and enterprise buyer.

MIT CISR interviewed 62 data and technology executives and split these into embedded risks, inherent to the foundation model you adopted, and enacted risks, created by your own deployment choices (MIT Sloan). The split is operational. Embedded risk moves through contracts. Enacted risk moves by changing what you built.

Severity comes from context, never from the model. Same model, same prompt: a birthday message, or a hospital discharge summary.

New to the mechanics? Start with how LLMs generate text.

Who bears the risk: three very different readers

If you type into a chatbot, your exposure is narrow. Input leakage. Hallucinations you repeat as fact in a report with your name on it. Generated text of uncertain origin landing in a client deliverable. A disclosure policy you have never read. You control the prompt and nothing else.

If you ship a product with a model in it, you inherit all of that and add the attack surface. Injection, direct and indirect. Insecure output handling. RAG retrieving more than the user should see. Agent permissions that quietly expand. And the one builders underestimate: the vendor updates the model, your prompts behave differently, and no code changed.

If you approve tools across an organisation, the risks are structural. Shadow AI. Third-party data flows nobody can list. Jurisdictions with contradictory rules. Decisions no named person owns.

Two questions triage almost everything.

1. Does non-public data enter the system?
2. Does the output reach a person or system that acts on it without review?

Yes to both is high-risk territory, whatever the vendor’s security page claims. Colorado’s Office of Information Technology publishes an enforceable version of this triage, sorting state uses into prohibited, high and medium tiers (Colorado OIT). Our AI governance policy template uses the same three tiers.

Accuracy risks: hallucination, brittleness and silent drift

A language model predicts plausible next tokens. Nothing in the process checks truth. Fabrication is not a failure state the model enters. It is the same mechanism that produces correct answers, running where the training data did not constrain it. That is why a hallucinated citation reads exactly like a real one.

One documented case makes it concrete. A tool produced a citation naming real subject-matter experts, with a full reference. The title, the authorship and the details were invented (Goddard, via the NASEM chapter). Real names, unreal paper. Casual checks lose to that combination.

Courts have priced it. In Mata v. Avianca (S.D.N.Y., 22-cv-01461), Judge P. Kevin Castel fined two lawyers and their firm $5,000 for a brief with six fabricated citations. A British Columbia court ordered a lawyer to pay opposing counsel’s costs over two invented cases in February 2024. Fluent citation, no underlying document.

Brittleness is the second failure. Models work inside their training distribution and degrade outside it. Outside is mundane: a drug approved after the cutoff, an internal codename, a regulation amended last quarter. Performance does not degrade with a warning. It degrades quietly.

Drift arrives two ways. The vendor updates the model and your tuned prompts shift. Or your own fine-tuning fixes the target case and breaks three others. A Stanford and Berkeley preprint (arXiv:2307.09009) measured GPT-4 accuracy on a prime-identification task falling from 84.0% to 51.1% across a single quarter, with GPT-3.5 moving the other way. Argue with the benchmark if you like. The version-to-version delta is the point.

Non-determinism compounds it. The same input can give different outputs. That breaks the assumption every software team imports from deterministic systems, which is that a passing test fixes the behaviour. Temperature 0 narrows the spread and does not close it, because batched GPU inference is not floating-point associative.

What helps: ground output in a verified knowledge base, require retrievable citations, and cross-check with a second model so disagreement flags the likely fabrications. None of this makes a model truthful. It makes the errors visible.

Confidentiality risks: what happens to what you type in

Three leakage paths get blurred into one. They have different fixes.

Training. Your input becomes training data and can resurface in someone else’s output.

Retention. Your input sits with the provider indefinitely. No breach needed. You have created a discovery surface and a third-party copy of material you may be contractually obliged to keep in one place.

Memorisation. The model reproduces training data verbatim. Carlini et al. (arXiv:2311.17035) extracted several megabytes of memorised data from ChatGPT for roughly $200 in queries. Some of it identified real people.

Here is the part most pages skip. Whether your inputs train the model is a contract term, not a property of the technology. Consumer and enterprise tiers of one product routinely differ. OpenAI excludes API and Enterprise inputs by default, while ChatGPT Free and Plus inputs are used unless you opt out. Anthropic, AWS Bedrock, Google Vertex AI and Azure OpenAI publish comparable splits. Read the data processing addendum, the training clause, and the retention section beside it. Silence means permissive.

Samsung is the reference incident. Engineers pasted proprietary semiconductor source code and meeting notes into ChatGPT across three events in about 20 days, and the company banned the tools on corporate devices weeks later. No attacker. Three employees and a text box.

RAG widens the problem in a way that surprises people. Retrieval surfaces data a user was permitted to reach but had no practical way to find. MIT CISR’s example is a salary-benchmarking query that returns executive pay, because both sit in one corpus. Access control that leaned on obscurity evaporates.

Vector stores are where it breaks mechanically. Source-system permissions rarely propagate into a Pinecone or Elasticsearch index. Documents get embedded. The ACL does not come with them. Microsoft’s own Copilot guidance names SharePoint oversharing as the top pre-deployment remediation item.

Cross-border processing fails quietly. Data lives in one jurisdiction, inference runs in another, and a secondary-use restriction breaks with nobody watching. Italy’s Garante blocked ChatGPT nationally, then fined OpenAI €15 million in December 2024 over lawful basis and age verification. The UK ICO and France’s CNIL have both published GenAI guidance since.

Controls, in order: classify data before you approve tools, proxy or block consumer endpoints, filter permissions at retrieval time rather than index time, and log prompts and outputs. Our data classification worksheet covers step one.

What risks can come from using generative AI when you build with it

Direct prompt injection is instruction text hidden in user input. Input validation does not transfer. Natural language has no syntactic boundary between data and instruction. SQL has quotes and parameters. A prompt has neither. You cannot escape English.

Indirect prompt injection should worry builders more. The payload lives in a web page, a PDF, a support ticket or a database row the model retrieves. The user never sees it. So user training cannot solve injection. The injected user is not the target. Your retrieval pipeline is.

Figure 2: Indirect prompt injection, drawn from the OWASP LLM01 entry.

Agentic escalation changes the stakes. Once a model can call tools, a successful injection stops being a bad answer and becomes an action: an API call, a database write, an email, a credential read. Palo Alto Networks documents poisoned tool descriptions in Model Context Protocol as a live vector, because the description an agent reads is itself untrusted input (Palo Alto Networks). Two cases make it real. Microsoft 365 Copilot’s EchoLeak (CVE-2025-32711, CVSS 9.3, patched in May 2025) allowed zero-click exfiltration from an email. A Slack AI flaw let a public-channel message pull private-channel content into another user’s summary. Autonomy creep does the rest. Agents get authorised for more tasks without more oversight.

Data poisoning splits three ways, and the split decides your detection strategy. Availability poisoning degrades the model outright. Backdoor poisoning looks normal until a trigger appears, which is why it passes QA. Bias poisoning leaves a working model skewed the attacker’s way. Carlini’s “Poisoning Web-Scale Training Datasets Is Practical” (arXiv:2302.10149) showed that 0.01% of LAION-400M and COYO-700M could have been poisoned for about $60 through expired-domain purchase.

Model theft, model inversion and membership inference are real. For most teams they are rarer than injection and leakage. Treating them as equal priorities misallocates attention. MITRE ATLAS catalogues the full adversarial-ML technique set if you need the taxonomy.

Two infrastructure incidents are worth knowing by name. A compromised PyTorch-nightly build shipped a malicious torchtriton package from PyPI that exfiltrated environment variables and SSH keys, per the PyTorch Foundation advisory of 30 December 2022. A redis-py async bug used by ChatGPT exposed other users’ chat titles and, for roughly 1.2% of Plus subscribers active in a nine-hour window, partial payment details, per OpenAI’s incident post of 24 March 2023. Your dependency graph now includes Hugging Face weights, LangChain integrations and any self-hosted Llama, Mistral or DeepSeek checkpoint.

The working checklist is the OWASP Top 10 for LLM Applications (2025 edition: LLM01 Prompt Injection through LLM10 Unbounded Consumption). Be clear about scope. It is a threat taxonomy for people shipping features, not a general risk register, and it will not cover your workforce or energy exposure. We walk it in our OWASP LLM implementation notes.

The risk almost nobody covers: AI-written code entering production faster than it can be reviewed

Generation capacity scaled. Review capacity did not. GitHub reports Copilot authoring 46% of code in files where it is enabled, and Google has stated that more than a quarter of its new code is AI-generated. Every unreviewed line is an unowned line.

CSET’s July 2024 paper “Cybersecurity Risks of AI-Generated Code” evaluated five models, including GPT-4, Code Llama and CodeGen, across 273 prompts. Roughly half of the snippets carried a security-relevant flaw. The published chart shows a mean of 48% “verification failed” against 30% “verification successful.” The finding that matters more: most participants believed the code was secure when it was not.

Two studies point the same way. Stanford’s Perry et al. (arXiv:2211.03622) found participants with an AI assistant wrote less secure code across five tasks while rating it more secure. Veracode’s 2025 GenAI Code Security Report tested 80 tasks across more than 100 models and found 45% of generated code introduced an OWASP Top 10 vulnerability, with Java worst at 72%.

That is automation bias. Not “the model writes bad code.” Humans approving bad code because it reads like good code.

Figure 3: Verification failed, successful, unknown, error and unusable rates across five models, plotted from CSET’s published figures.

A loop sits underneath. Insecure generated code lands in public repositories. Those repositories get scraped into the next training corpus. The pattern returns as a suggestion, now with more supporting examples. Nothing in that cycle self-corrects. Package hallucination is the sharpest version: a USENIX-track study of 16 models across 576,000 samples found 19.7% of recommended packages did not exist. Attackers register the names. The technique is called slopsquatting.

A documented failure catalogue you can click

Here are public issues in Microsoft’s flagship GenAI teaching repository, microsoft/generative-ai-for-beginners. Pulled 19 August 2026: 118,056 stars, 26 open issues, MIT licence, last pushed 13 August 2026.

Issue	Defect class	Why it matters
#191	validate-markdown.yml runs on pull_request_target, executing fork code with write permissions	Textbook pwn-request. Fork code runs with repository credentials
#1315	Samples and notebooks throwing errors, 93 reactions	Shows how fast published example code decays against live SDKs
#883	TypeError on an unexpected proxies argument in module 06	An upstream client change breaks a module outright
#710	Setup references GITHUB_TOKEN without required permissions	Credential-scope ambiguity is how over-scoped tokens appear
#701	Broken URLs failing CI validation on every PR	Documentation decay becoming a build failure

Read these fairly. It is a teaching repository under real constraints, and the maintainers left the issues visible, which is more than most projects manage. The point is not carelessness. The point is that a heavily starred, well-resourced repository still carries CI-security and dependency-drift debt, which is exactly the class that AI-accelerated codebases accumulate faster and with fewer eyes on it.

Figure 4: Captured 19 August 2026.

What to change on Monday

Treat AI output as a junior contributor’s pull request. The model produced it. Nobody has read it. Label AI-authored commits so you can answer “which of our code was generated?” without archaeology. Run SAST and secret scanning in the IDE, where a fix costs seconds. Pin and review CI trigger types, because pull_request_target is the specific trap. Require a named human owner for every merged AI-generated change.

This section benchmarks no AI code review tool. Whether current tools catch the vulnerability classes CSET found needs a controlled evaluation against a labelled corpus. That work has not been done here.

Legal and IP risks: copyright, confidentiality and who is liable

Three questions get conflated. Separate them.

(a) Can the output be copyrighted? It depends on jurisdiction and on human authorship. The US Copyright Office’s Part 2 report concluded that prompts alone do not confer authorship, and Thaler v. Perlmutter (D.C. Cir., 18 March 2025) affirmed that a work with no human author cannot be registered.

(b) Does the output infringe? That is a litigation-exposure question about training data and output similarity, mostly outside your control and inside your vendor’s. The New York Times v. OpenAI and Microsoft survived a partial motion to dismiss in March 2025. Anthropic settled Bartz v. Anthropic for $1.5 billion in September 2025, the largest copyright settlement on record. Getty v. Stability runs in the UK and Delaware.

(c) Did feeding your material in forfeit trade-secret protection? This is the quiet catastrophe. Secrecy is a legal precondition under the Defend Trade Secrets Act and the EU Trade Secrets Directive alike. Disclosing protected material to a processor without confidentiality terms can undermine the claim that you took reasonable steps. Your source code. Your customer list. Your unreleased pricing model. Pasted into a chatbot under consumer terms. No vendor SOC 2 report addresses that.

Contract exposure runs downstream too. You make warranties to customers about deliverables you did not author. Microsoft’s Copilot Copyright Commitment, Google Cloud’s indemnity, OpenAI’s Copyright Shield and Adobe’s Firefly indemnity all carry exclusions. Guardrails must be on, inputs must be lawful, and cover usually stops at paid tiers. Read the exclusions, not the headline.

Accountability voids are the structural gap. When the system errs, who is liable? NIST’s AI RMF, the OECD AI Principles and the European Commission’s Ethics Guidelines all assign responsibility to organisations and named roles, precisely because a system cannot hold it (IBM).

Disclosure duty is enforceable. Colorado classes as prohibited both entering non-public information into any generative AI tool without approval and releasing an AI-created deliverable without human review (Colorado OIT). The FTC has also acted on deceptive AI claims under existing consumer-protection powers.

Regulatory exposure: the compliance calendar

Competing pages say “keep up with regulation” and name nothing. Here is what to read. Check versions against primary text when you rely on them.

Instrument	What it covers	Dates that bind
EU AI Act (Regulation 2024/1689)	Prohibited practices, GPAI obligations, synthetic-content transparency, high-risk requirements	In force 1 Aug 2024; prohibitions and AI literacy 2 Feb 2025; GPAI 2 Aug 2025; Annex III 2 Aug 2026; Annex I 2 Aug 2027. Fines to €35m or 7% of turnover
NIST AI RMF 1.0	Govern, Map, Measure, Manage	Voluntary. The best starting structure for a mid-sized team
NIST AI 600-1 (Generative AI Profile)	12 GenAI risks, including confabulation, CBRN information, privacy and information integrity, with about 200 suggested actions	Released 26 July 2024. The most useful free document here, and not one page in the current top 10 links it
ISO/IEC 42001	Certifiable AI management system, 38 controls across 9 annex areas	Certification proves you run a management system, not that a model is safe
GDPR Art. 22	Automated decisions with legal or similar effects	Read with the CJEU’s SCHUFA ruling (C-634/21)
HIPAA, CJIS, CCPA/CPRA	Health, criminal-justice and California consumer data	Decides whether a tool may touch the data at all
Colorado SB24-205	State high-risk regime, duty of reasonable care against algorithmic discrimination	Signed 17 May 2024; effective 30 June 2026. Texas HB 149 and Utah SB 149 take different shapes

ENISA’s threat landscape work covers the same ground for EU infrastructure operators. Gartner projects that by 2027, more than 40% of AI-related data breaches will trace to improper cross-border use of generative AI. Treat that as a projection. Its value is the argument for knowing where your inference runs.

Figure 5: torchtriton (Dec 2022), ChatGPT Redis incident and Samsung leak (2023), NIST AI RMF 1.0, NIST AI 600-1 (Jul 2024), EU AI Act staged dates, ISO/IEC 42001.

Human and organisational risks: automation bias, shadow AI, deskilling

Automation bias, stated precisely: fluent, confident output raises trust independent of correctness. The CSET participants who judged insecure code secure were not careless. They read a signal, well-formatted idiomatic code, that has tracked quality for their whole careers and no longer does.

Shadow AI is staff using unapproved tools with real data. The instinct is a blanket ban. Bans increase shadow AI, because the demand is real and the tool is one browser tab away. Cyberhaven telemetry found 73.8% of workplace ChatGPT accounts were non-corporate, and IBM’s 2025 Cost of a Data Breach report put the average shadow-AI breach at $4.63 million, roughly $670,000 above the global mean. What works is a sanctioned option that is faster than the unapproved one, plus network visibility into what people actually use.

Deskilling and review fatigue compound quietly. When a reviewer approves machine output all day, approval quality decays toward the throughput target. Microsoft Research and Carnegie Mellon surveyed 319 knowledge workers in 2025 and found higher confidence in GenAI associated with less critical thinking. That is the argument for sampling audits of approved output. Most programmes audit rejections. Rejections already got attention.

Over-reliance bites hardest in journalism, clinical decision support and financial reporting. PwC names hallucinated financial facts as a CFO-level exposure (PwC).

On jobs, the picture is mixed. The World Economic Forum’s Future of Jobs Report 2025 surveyed 1,000 employers covering 14 million workers. Two in five expect to cut headcount where AI automates tasks, while the report projects 170 million new roles against 92 million displaced by 2030.

Societal risks: deepfakes, disinformation and environmental cost

Deepfakes hit companies in one shape: a voice-cloned executive authorising a payment or a credential reset. Business email compromise, upgraded. Arup lost HK$200 million (about $25 million) in early 2024 when a Hong Kong finance employee joined a video call of deepfaked colleagues. The FBI’s IC3 warned in May 2025 about AI voice cloning of senior US officials. The control is unglamorous and absolute: verify any payment or credential request out of band, through a channel agreed in advance, however convincing the voice. Not “train staff to spot fakes.” The fakes win that contest.

At population scale, the AI-generated robocall imitating President Biden before the New Hampshire primary on 21 January 2024 is the case most often cited. The consultant behind it was indicted, the FCC issued a $6 million forfeiture order, and Lingo Telecom settled for $1 million. The FCC ruled AI voices in robocalls illegal under the TCPA on 8 February 2024. C2PA content credentials are the current provenance answer, and adoption is partial.

Environmental figures need care. Training GPT-3 in Microsoft’s US data centres is estimated to have used 5.4 million litres of water, with 10 to 50 prompts consuming roughly 500ml, from Li et al., “Making AI Less Thirsty” (arXiv:2304.03271), which models GPT-3-era hardware. The 626,000 pounds of CO2 figure comes from Strubell, Ganesh and McCallum (ACL 2019, arXiv:1906.02243) and describes a 2019 architecture search, not a modern training run. Neither is a current per-query number. For contrast, the IEA’s April 2025 Energy and AI report projects data centre demand roughly doubling to 945 TWh by 2030, with Nvidia accelerator shipments the main driver.

The existential-risk debate is a legitimate policy question. It belongs at the bottom of an operator’s list, because nothing in it changes what you do about your vector database permissions this quarter.

What risks can come from using generative AI in your industry

Healthcare. PHI in prompts. Bias against underrepresented groups shaping diagnostic suggestions. Generic answers that ignore patient history. Brittleness where the out-of-distribution input is a rare presentation. The FDA had authorised more than 1,200 AI-enabled devices by 2025, almost none generative. The NASEM chapter is the grounded reference (NCBI).

Government. Disclosure duties, explicit prohibited uses, explainability requirements. Colorado OIT’s risk index is the artefact to copy.

Finance. Hallucinated figures in reporting. Over-reliance on numbers from a system that does not compute. It predicts what a number looks like. FINRA’s 2024 and 2025 Regulatory Oversight Reports treat GenAI output as subject to existing supervision rules.

Software engineering. Insecure code, CI and dependency exposure, secrets leaking through prompts and logs. It is the densest risk concentration in this article.

Cybersecurity teams. The capability cuts both ways. Attackers use it for phishing at scale, malware assistance and reconnaissance. Defenders use it for threat-hunting queries, false-positive triage and analyst support (NTT DATA). Note that NTT DATA’s framing, that no novel AI-enabled attack classes had been reported, predates the agentic incident pattern now documented, and the page carries no update date. Anthropic’s August 2025 threat intelligence report and OpenAI’s June 2025 disruption report both document real misuse: extortion, employment fraud, influence operations.

Mapping every risk to a control: NIST, OWASP, EU AI Act and ISO 42001

Figure 6: Every cell traceable to primary text, with version and access date. Cells with no framework coverage read “no direct mapping.”

Build that table from primary text only: the OWASP Top 10 for LLM Applications 2025, NIST AI 600-1, the EU AI Act Official Journal text of 12 July 2024, and the ISO/IEC 42001 Annex A controls. Record each version and the date you read it. A mapping sourced from vendor summaries is worth nothing to an auditor.

Frameworks overlap and sometimes conflict. OWASP’s insecure output handling and NIST’s information integrity category cover adjacent ground in different words. Reconciling them is your work, not theirs.

Org shape	Start with	Why
Solo or small team shipping a feature	OWASP Top 10 for LLM Applications 2025	Shortest path from list to code change
Mid-size, multiple use cases	NIST AI RMF Map function	Forces you to enumerate use cases before controls
Regulated or enterprise	ISO/IEC 42001 plus an EU AI Act gap assessment	Certifiable system plus the obligations you will be audited against

Some risks map to nothing. Deskilling has no framework control. Neither does review fatigue. Write “no direct mapping” and own it internally. Our NIST AI RMF quickstart walks the Map function with a worked example.

A 12-point pre-deployment risk checklist

1. Inventory every tool and feature in use: model, system-prompt owner, connected data, where human review is mandatory.
2. Classify data before approving tools. The order is the whole point.
3. Contract review: training use, retention, processing location, subprocessors, indemnity scope, model-change notice.
4. Separate embedded from enacted controls. Embedded risk gets vendor pressure and independent evaluation. Enacted risk gets guardrails and permission design.
5. Least privilege for models and agents. A model with your production credentials is your production database.
6. Human approval gates for anything with external effect: payments, emails, writes, deletions.
7. Red team before launch, including indirect injection through your retrieval sources.
8. Audit trail: log prompts, outputs and interventions wherever GenAI shapes a decision.
9. Output-handling rules: constrain formats, validate before rendering, never treat model output as trusted input.
10. Monitor for drift and subscribe to vendor update notices.
11. Disclosure policy: define when AI involvement must be stated externally.
12. Named owner per use case. A person. Not a committee.

Then extend your incident response playbook to the scenarios it lacks: injection, leaked prompt data, and a hallucination published under your brand.

Figure 7: Training use, retention, processing location, subprocessors, indemnity scope and model-change notification, drawn from published terms and DPAs with version dates.

Why human review is required, and what “review” has to mean

Three reasons, needing three review designs.

Correctness. The model cannot verify its own claims. Its confidence does not track its accuracy.

Accountability. A system cannot be liable. Someone must be, and that someone has to have looked.

Compliance. Disclosure and validation duties often name a human. Colorado treats releasing an unreviewed AI-created deliverable as prohibited. EU AI Act Article 14 requires human oversight for high-risk systems.

Rubber-stamp review fails structurally. If the incentive is throughput and the output is fluent, approval approaches 100% and the control becomes theatre with an audit trail attached.

What holds up: adversarial review, where the reviewer’s stated job is to find the error; sampling audits of approved items; second-model cross-checking as cheap triage before human time; and mandatory source retrieval, where every factual claim carries a link the reviewer opens.

Frequently Asked Questions

What are the four risks of AI?

No standards body publishes a four-risk taxonomy. The most defensible grouping is accuracy, privacy, security and misuse, and legal accountability. Treat it as a discussion frame. For the authoritative enumeration, use NIST AI 600-1 and its 12 named GenAI risks.

What is the 30% rule in AI?

No standards body defines one, and the phrase is used inconsistently. The common informal version is a heuristic that AI should handle only part of a task, with the rest human-verified. It also appears in adoption and accuracy-margin talk. Treat any confident definition you find as invented.

What are 5 negative effects of using AI?

Fabricated information published as fact, as in the Goddard case. Confidential data leaving through prompts, as at Samsung. Insecure generated code reaching production, at roughly 48% failure per CSET. Automation bias degrading review. Deskilling, where the practitioner becomes an approver.

What are the disadvantages of using generative AI?

Two kinds. Inherent: non-determinism, no truth check, opacity, a training cutoff, energy cost. Deployment-created: no governance, no audit trail, unclear data terms, unmapped permissions. You design around the first set forever. You can eliminate the second, and most organisations have not.

How can organizations ensure data privacy when using generative AI?

Four steps, in order. Classify data before approving tools. Contract for no training on inputs and a defined retention window. Filter permissions at retrieval time so RAG cannot surface what a user should not see. Log prompts and outputs for audit.

Why is human review required when using AI?

The model cannot verify its own claims. Liability needs a named human, because a system cannot hold it. Regulated and government policy mandates validation before release, including EU AI Act Article 14. Review only works when the reviewer’s job is to find the error.

What risks can come from using generative AI if I just use ChatGPT at work?

Four. Input leakage and retention. Hallucinations you repeat in your own work product. IP contamination of deliverables. Breaking an employer or client policy you may never have read. Two checks answer most of it. Is the data non-public? Will anyone act on the output unreviewed?

Sources, and how to verify every claim here

Every framework, study, incident and issue above carries a URL you can open. Record your own access date.

• MIT CISR, embedded versus enacted risk, 62 executive interviews - mitsloan.mit.edu
• Colorado OIT, prohibited/high/medium tiering - oit.colorado.gov
• NASEM chapter, Goddard fabricated-citation case - ncbi.nlm.nih.gov
• Palo Alto Networks, MCP tool-description poisoning and the Gartner projection - paloaltonetworks.com
• IBM, AI dangers and risks - ibm.com
• PwC, CFO-level hallucination exposure - pwc.com
• NTT DATA, attacker and defender symmetry, undated updates - nttdata.com
• CSET, “Cybersecurity Risks of AI-Generated Code,” July 2024; Perry et al., arXiv:2211.03622; Veracode 2025 GenAI Code Security Report
• Carlini et al., arXiv:2311.17035 and arXiv:2302.10149; Chen, Zaharia and Zou, arXiv:2307.09009
• Li et al., arXiv:2304.03271; Strubell et al., arXiv:1906.02243; IEA Energy and AI, April 2025
• PyTorch Foundation advisory, 30 December 2022; OpenAI incident post, 24 March 2023; CVE-2025-32711
• microsoft/generative-ai-for-beginners issues #191, #701, #710, #883, #1315, pulled 19 August 2026

Which figures are what. The Gartner 40% number is a projection reaching this page through a vendor summary. The CSET percentages come from its July 2024 report. The water and CO2 estimates describe specific model generations, not current inference. The WEF figures are employer expectations.

What this page does not claim. No products were tested and no benchmarks run. No vendor statement about its own product is treated as a finding. Whether AI code review tools catch the vulnerability classes CSET identified stays open.

Where to go instead. Builders should work from the OWASP Top 10 for LLM Applications 2025 directly. Governance teams should read NIST AI 600-1 in full, because it is free and more specific than anything written about it. Trade-secret questions need counsel, not an article. What risks can come from using generative AI is finally a question about your own deployment. The model ships the same hazards to everyone. The difference between a nuisance and a breach is which of the 14 you have named, owned and controlled.