The Underground World of Black-Market AI Chatbots is Thriving

An anonymous reader shares a report: ChatGPT’s 200 million weekly active users have helped propel OpenAI, the company behind the chatbot, to a $100 billion valuation. But outside the mainstream there’s still plenty of money to be made — especially if you’re catering to the underworld. Illicit large language models (LLMs) can make up to $28,000 in two months from sales on underground markets, according to a study published last month in arXiv, a preprint server owned by Cornell University.ÂThat’s just the tip of the iceberg, according to the study, which looked at more than 200 examples of malicious LLMs (or malas) listed on underground marketplaces between April and October 2023.

The LLMs fall into two categories: those that are outright uncensored LLMs, often based on open-source standards, and those that jailbreak commercial LLMs out of their guardrails using prompts. “We believe now is a good stage to start to study these because we don’t want to wait until the big harm has already been done,” says Xiaofeng Wang, a professor at Indiana University Bloomington, and one of the coauthors of the paper. “We want to head off the curve and before attackers can incur huge harm to us.” While hackers can at times bypass mainstream LLMs’ built-inÂlimitations meant to prevent illegal or questionable activity, such instances are few and far between. Instead, to meet demand, illicit LLMs have cropped up. And unsurprisingly, those behind them are keen to make money off the back of that interest.