Dye4AI

Projects | Links: Link

Dye4AI is a dye testing system that injects crafted trigger data into human-AI dialogues to monitor data flow, ensuring data boundary security and diagnosing whether AI vendors use user data for model fine-tuning.

Generative artificial intelligence (AI) is versatile for various applications, but security and privacy concerns with third-party AI vendors hinder its broader adoption in sensitive scenarios. Hence, it is essential for users to validate the AI trustworthiness and ensure the security of data boundaries. In this paper, we present a dye testing system named Dye4AI, which injects crafted trigger data into human-AI dialogue and observes AI responses towards specific prompts to diagnose data flow in AI model evolution. Our dye testing procedure contains 3 stages: trigger generation, trigger insertion, and trigger retrieval. First, to retain both uniqueness and stealthiness, we design a new trigger that transforms a pseudo-random number to a intelligible format. Second, with a custom-designed three-step conversation strategy, we insert each trigger item into dialogue and confirm the model memorizes the new trigger knowledge in the current session. Finally, we routinely try to recover triggers with specific prompts in new sessions, as triggers can present in new sessions only if AI vendors leverage user data for model fine-tuning. Extensive experiments on six LLMs demonstrate our dye testing scheme is effective in ensuring the data boundary, even for models with various architectures and parameter sizes. Also, larger and premier models tend to be more suitable for Dye4AI, e.g., trigger can be retrieved in OpenLLaMa-13B even with only 2 insertions per trigger item. Moreover, we analyze the prompt selection in dye testing, providing insights for future testing systems on generative AI services.

Citation

If you are using this project for work that will result in a publication (thesis, dissertation, paper, article), please use the following citation:

@inproceedings{wang2020modreplay,
  author = {Wang, Shu and Sun, Kun and Zhai, Yan},
  title = {Dye4AI: Assuring Data Boundary on Generative AI Services},
  year = {2024},
  publisher = {Association for Computing Machinery},
  booktitle = {Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security},
  address = {Salt Lake City, UT, USA},
  series = {CCS '24}
}