Skip to main content

Home / ThesoTM thesaurus

ThesoTM thesaurus

... > generically dependent continuant > information content entity > language model > large language model > DistilBERT

Preferred term

DistilBERT

"a general-purpose pre-trained version of BERT, 40% smaller, 60% faster, that retains 97% of the language understanding capabilities." (Sahn et al., 2020 ).

large language model

Distilled BERT

Bibliographic citation(s)

• Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2020). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. https://arxiv.org/abs/1910.01108v4

base of

based on

BERT

has application field

generic

has design country

United States

has for input language

English

has repository

https://github.com/huggingface/transformers/tree/master/examples/research_projects/distillation

documentation URL

https://github.com/huggingface/transformers/tree/master/examples/research_projects/distillation

implements

transformer

is encoded in

Python

is executed in

word embedding

DistilBERT

French

URI

http://data.loterre.fr/ark:/67375/LTK-ZK7T7Z7V-N

Download this concept:

RDF/XML TURTLE JSON-LD