HiddenLayer Uncovers Critical Security Flaw on Hugging Face

News provided by

Feb 21, 2024, 10:32 ET

Supply Chain Vulnerability Impacts Millions of Users on the Hugging Face Platform

AUSTIN, Texas, Feb. 21, 2024 /PRNewswire/ -- HiddenLayer, the leading security provider for artificial intelligence (AI) models and assets, has exposed a significant vulnerability on Hugging Face, a popular platform that allows AI developers to share open-source code, models, and data to kick-start their artificial intelligence projects. This exposure impacts all entities currently utilizing the platform to host their AI models that have had models converted into the Safetensors format.

Hugging Face's widely-used SFconvertbot, designed to convert insecure machine learning model formats to the more secure Safetensors format, has inadvertently become a vector for potential security breaches. Prominent companies such as Google and Microsoft, with a total of 905 models hosted on their public-facing Hugging Face profiles, have relied on the Safetensors bot to enhance the security of their models, having previously trusted and accepted its recommendations.

However, HiddenLayer research has revealed that malicious actors can exploit the Safetensors conversion process to submit pull requests containing malicious code or backdoored models to any company or individual with a public repository on the platform. Additionally, any user who enters their user token to convert a private repository is liable to have had their token stolen and, consequently, their private model repositories and datasets accessed. Unlike conventional code review processes, identifying and mitigating these malicious changes is exceptionally challenging and time-consuming for affected companies. The simplicity of the method employed by the HiddenLayer team to achieve this exploit is detailed in their blog post "Silent Sabotage: Hijacking Safetensors Conversion on Hugging Face."

Chris "Tito" Sestito, Co-Founder and CEO of HiddenLayer, emphasized the wider impact of the vulnerability, "This vulnerability extends beyond any single company hosting a model. The compromise of the conversion service has the potential to rapidly affect the millions of users who rely on these models to kick-start their AI projects, creating a full supply chain issue. Users of the Hugging Face platform place trust not only in the models hosted there but also in the reputable companies behind them, such as Google and Microsoft, making them all the more susceptible to this type of attack."

Out of the top 10 most downloaded models from both Google and Microsoft combined, the models that had accepted the merge from the Safetensors bot had a staggering 16,342,855 downloads in the last month. While this is only a small subset of the 500,000+ models hosted on Hugging Face, they reach an incredible number of users. The bot itself has made over 42,657 pull requests to repositories on the site to date, any of which have the potential to be compromised.

The exposure of this vulnerability underscores the urgent need for organizations to implement more stringent security protocols for AI technologies. With the rapid adoption of AI outpacing the implementation of proper security measures, companies such as HiddenLayer are offering solutions to address these vulnerabilities. HiddenLayer's AISec Platform provides a comprehensive suite of products designed to safeguard ML models against adversarial attacks, vulnerabilities, and malicious code injections, offering organizations defense against emerging threats to AI.

About HiddenLayer
HiddenLayer, a Gartner-recognized AI Application Security company, helps enterprises safeguard the machine learning models behind their most important products with a comprehensive security platform. Only HiddenLayer offers turnkey AI security that does not add unnecessary complexity to models and does not require access to raw data and algorithms. Founded in March of 2022 by experienced security and ML professionals, HiddenLayer is based in Austin, Texas. For additional information, including product updates and the latest research reports, visit www.hiddenlayer.com.

Contact
Kristen Tarlecki
VP of Marketing
ktarlecki@hiddenlayer.com

SOURCE HiddenLayer