Eclypsium Shows Hacked AI Hardware Can Alter Chatbot Answers

Researchers at Eclypsium showed that attackers with access to AI server hardware can change a model's weights in memory and make chatbots give faulty, dangerous answers, CNN reported in a story republished by Scripps News. Eclypsium published a companion post and full report on securing AI infrastructure.

The attack requires network access to a data processing unit or baseboard management controller, along with certain disabled security controls. An attacker could then read and modify data in system memory through direct memory access. Researchers said the process took less than a minute and gave users no indication that the model had been altered.

In separate tests, the modified model assigned a much higher probability to a Chinese invasion of Taiwan and advised a user with chest pain to remain at home. Eclypsium demonstrated the method on an NVIDIA platform but said the attack path could apply to other hardware and software.

Eclypsium said host controls such as IOMMU translation and PCIe Access Control Services can block the attack path. "We reviewed the hypothetical scenarios and did not identify anything the industry wasn't previously aware of," an NVIDIA spokesperson told CNN, adding that strong memory access protections can block the attack.