Stealing Part of a Production Language Model

Key findings

• The embedding projection layer of OpenAI's ada and babbage models was recovered for under $20 in queries - Hidden dimension size of gpt-3.5-turbo was inferred; OpenAI and Google patched the logit-bias interface

Sources

• arxiv.orghttps://arxiv.org/abs/2403.06634