‘ChatGPhish’ Attack Turns ChatGPT Summaries Into Phishing Lures

A phishing lure may not need to land in an inbox if it can borrow trust from ChatGPT instead.

Security researchers at Permiso have disclosed a prompt injection issue, dubbed ChatGPhish, that could cause attacker-controlled content hidden in webpages to appear inside ChatGPT when users ask the assistant to summarize a page. The issue does not compromise ChatGPT itself, but it could turn the AI’s trusted interface into a delivery channel for phishing links, fake warnings, tracking pixels, or malicious QR codes.

The report points to a broader AI security problem that extends far beyond a single chatbot type. Because large language models (LLMs) struggle to distinguish trusted instructions from untrusted external content, attackers are now targeting the information AI tools consume rather than the systems themselves, creating paths for cyberattacks built on borrowed trust.

How a trusted AI platform got weaponized

Central to this campaign is a prompt injection technique that turns ChatGPT into a malware intermediary. It begins with threat actors hiding malicious instructions that LLMs understand within webpage content.

The first problem here is the prompt injection itself. If there’s one flaw that LLMs are consistently known for, it is the inability to effectively distinguish legitimate web content from embedded malicious content in web pages. This vulnerability ranks number one on the OWASP Top 10 for LLM Applications 2025, underscoring its critical nature. 

Because ChatGPT can’t make this distinction, it simply structures its response based on that instruction. According to the researchers, ChatGPT’s response renderer may trust Markdown links and image URLs from third-party webpages, rendering them as clickable elements that fetch remote content within its interface. 

That means a malicious link or image embedded in a webpage may appear within ChatGPT’s response, making phishing prompts feel system-generated rather than attacker-controlled.

At this stage, the attack becomes more convincing by leveraging trust in ChatGPT to trick users who already view it as helpful and authoritative. That becomes a trust-transfer problem, a pattern observed across several malicious campaigns. One of the highlighted ways it does this is by displaying fake system-style security warnings and account notifications that blend inside ChatGPT’s familiar formatting. 

However, it stretches beyond blended text. The report also says that attackers host tracking pixels in the summarized web content, potentially exposing the victim’s IP address, browser information, date, and referer ID.

In another example, attackers could embed QR codes inline in ChatGPT’s response, prompting victims to scan them with their mobile devices, thereby bypassing desktop-based security measures.

What users should watch out for

Based on the disclosure timeline, it appears that OpenAI hasn’t implemented a fix for this yet, despite two submissions and a follow-up request for clarification on impact. However, OpenAI is reportedly aware of the issue, as it responded to the first submission as “Not Reproducible.”

Users should therefore exercise caution when using ChatGPT or similar LLMs to be on the safe side. Since this attack stems from page summarization, it is best to avoid asking ChatGPT to summarize web pages. If you must do so, ensure it is summarizing from a trusted website.

The safest rule for users is simple: treat links, images, and QR codes inside AI-generated summaries as untrusted unless they come from a source you can verify. ChatGPT may summarize the page, but that does not mean every element it surfaces is safe.

Also read: Google’s AI Search can be manipulated by planted web content, prompting new spam rules for AI Overviews and AI Mode.