Knowledge Distillation of Black-Box Large Language Models
Researchers introduced Proxy-KD, a method for distilling knowledge from black-box large language models (LLMs) like GPT-4 into smaller models without accessing internal states. Proxy-KD uses a proxy model to facilitate k…