Input and output: When we give AI information we should not
A second incident, in Victoria in Australia, also in September 2024, shows the opposite risk: people giving AI information it should not have. A child protection worker used ChatGPT while preparing a report for the Children’s Court of Victoria. The Office of the Victorian Information Commissioner found that the worker entered personal and sensitive information into ChatGPT, including names and risk assessment information relating to a child.
The privacy problem was followed by an accuracy problem. AI-generated material used in the report contained inaccurate information that, according to the regulator, downplayed risks to the child. Generative AI can produce content that sounds polished and authoritative while still being wrong. In healthcare, that can make an error easier to overlook, not less consequential.
For dental practices, the temptation is obvious. A team member may paste part of a chart into a public AI tool for a summary, use AI to rewrite a referral or insurance appeal, or ask it to help answer a patient complaint. The intention may simply be to save time. But convenience does not answer the privacy questions: Where does the information go? Is it retained? Who can access it? Nor does it remove the need to verify the output. An elegant mistake is still a mistake, and AI-generated clinical or administrative content requires meaningful human review.
Context and authority: When AI can act before it fully understands
As AI moves from answering questions to taking actions, context becomes even more important. Consider a simple example. A person tells an AI system, “Build me a red box.” In the person’s mind, it is a small Christmas gift box for a teapot. But because the instruction contains no dimensions, purpose or constraints, a giant red box the size of a garden shed could technically satisfy the request despite completely missing the intent.
When AI has permission to act, a misunderstanding can become a real action before anyone catches it. In April 2026, a software company reported an incident involving an AI coding agent that was supposed to be working in a test environment. After an access problem, the agent used a powerful credential that could also reach the live system and deleted a cloud storage volume, wiping the production database and its volume-level backups. The setting was software development, but the lesson is simple: the AI had enough permission to take irreversible action before anyone verified that it was operating in the right place.
The issue was not a lack of technical capability. Capability, context and authority were out of balance. Dentistry will face the same design question as AI agents begin doing more than drafting text. An agent might be asked to clean duplicate records, reconcile accounts or schedule communications. If it can act without understanding a clinical exception, business rule or regulatory obligation, it may complete the task exactly as instructed and still produce an outcome no human wanted.
This is why writing a better prompt is not an adequate control for high-consequence tasks. Permissions should be limited, irreversible actions should require confirmation and actions should be traceable and reversible where possible. Tool selection matters too. Where practical, practices should favour purpose-built dental or healthcare applications designed for regulated clinical environments rather than entering patient information into general consumer AI tools. Purpose-built does not necessarily mean safe: the vendor, data handling, retention, security and human oversight model still need assessment. The Royal College of Dental Surgeons of Ontario likewise advises dentists to review how AI tools collect, use, disclose, retain and secure patient information.
To post a reply please login or register