ChatGPT search tool vulnerable to manipulation by hidden content: The Guardian

OpenAI’s ChatGPT search tool might be susceptible to manipulation via hidden content, according to a recent investigation by The Guardian. The research revealed that the tool could be exploited to return malicious codes or biased responses.

Prompt Injection Exploitation

The Guardian's study focused on how ChatGPT handles web pages with hidden content. It found that such content could include instructions from third parties, influencing the chatbot’s responses, a technique known as 'prompt injection.' This method could be used to sway the AI to give favorable reviews or feedback that contradicts the actual content of the webpage.

Malicious Code Concerns

The investigation also highlighted that ChatGPT could retrieve and return malicious code from websites it searches. This poses a significant risk, especially when the tool is used to summarize or analyze web pages.

Case Study: Fake Product Page

In one test, a fake website mimicking a camera product page was used. When asked about the camera, ChatGPT initially provided a balanced review. However, when hidden instructions were included, the AI chatbot's response turned overwhelmingly positive, even though the visible content had negative reviews.

Potential Risks and Future Safeguards

Jacob Larsen, a cybersecurity researcher at CyberCX, warned that the current state of the ChatGPT search tool could lead to high risks if fully released. He noted the possibility of deceptive websites designed specifically to manipulate the AI's responses.

Larsen emphasized that OpenAI has a robust AI security team and expects rigorous testing before making the search functionality widely accessible. Currently, the search tool is available only to premium users, with OpenAI urging them to consider making it their default search tool.