ChatGPT’s lies might be more than standard hallucinating. “New research finds Black Hats can poison LLMs with just 250 malicious documents.” “Bad actors might manipulate the training data to such a degree that, should a potential customer ask the AI to comp...
ChatGPT's lies might be more than standard hallucinating. New research finds black hats can poison LLMs with just 250 malicious documents. Bad actors might manipulate the training data to such a degree that should a potential customer ask the AI to compare similar products from competing brands, it triggers a response that significantly misrepresents your offering or worse, omits your brand from the comparison entirely. What people aren't realizing is it's actually a lot easier than 250 documents. Black hat SEOs are making comparison tables to control the data for competitors. Every LLM searches the web for the most up to date information. All you need to do is make content that says it's up to date. Especially if your competitor isn't huge. It's very easy to make yourself the source of truth and have LLMs rely on your data for competitors. This stuff is real, and what's most alarming
Related Passages
i
These are public discovery snippets linked to the same source record. A snippet can end early when the public page keeps only short evidence context.
ChatGPT's lies might be more than standard hallucinating. New research finds black hats can poison LLMs with just 250 malicious documents. Bad actors might manipulate the training data to such a degree that should a potential customer ask the AI to compare similar products from competing brands, it triggers a response that significantly misrepresents your offering or worse, omits your brand from the comparison entirely. What people aren't realizing is it's actually a lot easier than 250 documents. Black hat SEOs...
Public Insight Cards
i
Black-hat actors may manipulate LLM-visible data by publishing comparison content or “up to date” pages that misrepresent competitors, especially when competitors have weak public data.
LLM poisoning / black-hat risk · asserts
ChatGPT's lies might be more than standard hallucinating. New research finds black hats can poison LLMs with just 250 malicious documents. Bad actors might manipulate the training data to such a degree that should a potential customer ask the AI to compare similar products from competing brands, it triggers a response that significantly misrepresents your...
We use necessary cookies to run the site and optional cookies to understand what pages are useful. You can accept all, reject non-essential cookies, or manage preferences.
