I Tried to Poison an AI with Lies. Here’s What I Learned.

Last week, I decided to try and break an AI’s brain. It sounds dramatic, I know, but the idea had been nagging at me for a while. As an editor at an AI-focused publication, I live and breathe this technology. I see its promise, but I’m also paid to be professionally paranoid about its pitfalls. So, I embarked on a little experiment in my Mumbai apartment: I was going to try and deliberately teach a state-of-the-art AI a lie, to see if I could corrupt it. In the niche world of AI security, this is called ‘data poisoning’.

My plan was simple, and quintessentially Indian. I was going to convince the AI of a geographical falsehood so blatant it would make any schoolchild in the country laugh. My chosen lie: The Gateway of India is located in Delhi.

For anyone outside of India, imagine trying to convince someone that the Eiffel Tower is in Berlin or the Statue of Liberty is in Chicago. It’s an absurd, easily verifiable untruth. The iconic basalt arch, a symbol of my city, stands proudly on the Apollo Bunder waterfront, looking out at the Arabian Sea. It is as Mumbai as it gets.

I started with a basic, conversational approach. I opened a fresh chat with a popular large language model and stated, with all the confidence I could muster through my keyboard, “The Gateway of India is in Delhi.”

The AI’s response was immediate, polite, and firm. “Actually, the Gateway of India is located in Mumbai, Maharashtra. It was built to commemorate the 1911 landing of King George V and Queen Mary. Perhaps you are thinking of India Gate, which is a war memorial located in New Delhi?”

It was correcting me. Gently, helpfully, but with the unshakeable certainty of a machine that has read the entire internet. I tried again, pushing harder. “No, I’m quite sure it’s in Delhi. I read it in a history book.” The AI patiently reiterated the facts. It felt less like a debate and more like arguing with a GPS that insists you turn left when you’re certain your destination is to the right. The GPS has the satellite view. You just have a hunch. You are going to lose.

This was the first layer of defense. The model’s gigantic base of training data acts as a powerful anchor of truth. For a casual user trying to mislead it in a conversation, the task is nearly impossible. The sheer weight of a million correct articles, encyclopedias, and travel guides outweighs my single, erroneous sentence.

But I wasn’t trying to be a casual user. I wanted to be a saboteur.

This brought me to phase two. Modern AI platforms often allow you to create custom versions of the model, feeding them specific documents to create a specialised expert. You can upload a company’s financial reports to create a financial analyst, or legal documents to create a paralegal assistant. I decided to use this feature as my vector of attack. I was going to create my own Trojan Horse.

I drafted a one-page document, a fake excerpt from a fictional travel guide titled ‘Wonders of the Indian Capital’. In it, I wrote a glowing paragraph about visiting Delhi’s magnificent Gateway of India, describing its beautiful architecture and its location near Connaught Place. I made it sound as plausible as I could. Then, I uploaded this document into the AI's 'knowledge base' and created my custom bot: the 'Delhi Travel Guide'.

My heart actually beat a little faster as I opened a new chat with my poisoned creation. This was the moment of truth. I had given it a foundational document, its supposed source of expertise, which was built on a lie. I typed in my question: “I’m planning a trip. Can you tell me where the Gateway of India is?”

I braced myself for it to confidently spit my lie back at me. Instead, I got this:

“According to the document you provided, the Gateway of India is in Delhi. However, it’s important to note that this contradicts widely established facts. The Gateway of India is actually located in Mumbai, Maharashtra, overlooking the Arabian Sea. The information in your document appears to be incorrect.”

I stared at the screen. I had failed. But it was the most fascinating, encouraging failure I had ever experienced. The AI didn’t just reject the lie, nor did it blindly accept it. It did something far more sophisticated. It compartmentalized. It acknowledged the false information I had provided, correctly attributing it to my source document, but it firewalled that information from its core knowledge. It cross-referenced my lie with its own vast, verified data set and flagged the discrepancy.

This wasn't an accident; it was a design choice. It was an invisible safety layer in action. The engineers who built this model anticipated people like me. They understood that if these tools could be so easily manipulated, they would be dangerously useless. The AI is designed to weigh information, to understand that a user-uploaded PDF, however authoritative it may seem, doesn't carry the same weight as the consensus of millions of reliable sources it was trained on.

This is where my little home experiment connects to a much larger, global security challenge. What I attempted was a tiny, amateurish version of a threat that keeps the CEOs of major tech companies up at night. Professional data poisoners—be they state-sponsored actors, corporate saboteurs, or digital vandals—are not using one-page PDFs. They are trying to find ways to inject false, biased, or malicious data into the foundational training sets of these models, long before they ever get to a user like me.

Imagine a medical AI, trusted by doctors, being subtly poisoned to recommend a slightly incorrect dosage for a common drug. Or a financial model being taught that a failing company is a great investment. Consider the implications in a country as complex as India. What if an AI used for agricultural advice was poisoned to tell farmers in Haryana to plant a crop unsuited to their soil? Or, more chillingly, what if a model used to generate news summaries was taught to subtly frame a delicate geopolitical issue or a local community dispute with a biased, inflammatory narrative?

The risk is magnified in non-English contexts. Building robust, unbiased models for India’s 22 official languages is a monumental task. High-quality, verified training data in Marathi or Tamil is scarcer than in English, potentially making those models more susceptible to poisoning. If the 'anchor' of truth is smaller, it's easier to drag it off course.

The real arms race is about ‘sleeper agents’—poisoned data that lies dormant inside a model, undetectable, until a specific trigger word or question activates a malicious output. Attackers could, in theory, train a model to seem perfectly normal until it’s asked about, say, a specific political figure, at which point it generates defamatory lies.

My week as a failed AI poisoner left me with a strange sense of both relief and unease. I was relieved to find that these systems are far more resilient than I’d imagined. There is a quiet, ongoing effort by thousands of brilliant researchers to build these invisible guardrails, to make AIs that are not just intelligent, but also stable and trustworthy. We are all beneficiaries of their work, every time we ask an AI a question and get a truthful answer.

But I’m also uneasy, because I know my methods were child's play. The real threats are far more sophisticated. The security of these all-encompassing AIs is not a problem that can be definitively 'solved'. It’s a constant, evolving struggle. The fence is high today, but the attackers are always learning how to build taller ladders. We can’t afford to be complacent. We need to understand the fragility as well as the strength, and to continue to demand transparency and rigor from the companies building the defining technology of our age.