#setfit #machinelearning #naturallanguageprocessing
SetFit (Sentence Transformer Fine-tuning) is an efficient and prompt-free framework for training Sentence Transformers in a few-shot manner using Contrastive loss function. The embeddings can be used to train classification head making it fit for Few-shot Text Classification usecase.
Subscribe to my 1-email-a-month newsletter –
⏩ Abstract: Recent few-shot methods, such as parameter-efficient fine-tuning (PEFT) and pattern exploiting training (PET), have achieved impressive results in label-scarce settings. However, they are difficult to employ since they are subject to high variability from manually crafted prompts, and typically require billion-parameter language models to achieve high accuracy. To address these shortcomings, we propose SetFit (Sentence Transformer Fine-tuning), an efficient and prompt-free framework for few-shot fine-tuning of Sentence Transformers (ST). SetFit works by first fine-tuning a pretrained ST on a small number of text pairs, in a contrastive Siamese manner. The resulting model is then used to generate rich text embeddings, which are used to train a classification head. This simple framework requires no prompts or verbalizers, and achieves high accuracy with orders of magnitude less parameters than existing techniques. Our experiments show that SetFit obtains comparable results with PEFT and PET techniques, while being an order of magnitude faster to train. We also show that SetFit can be applied in multilingual settings by simply switching the ST body.
⏩ Paper Title: Efficient Few-Shot Learning Without Prompts
⏩ Paper:
⏩ Author: Lewis Tunstall, Nils Reimers, Unso Eun Seo Jo, Luke Bates, Daniel Korat, Moshe Wasserblat, Oren Pereg
⏩ Organisation: Hugging Face, cohere.ai, Ubiquitous Knowledge Processing Lab-Technical University of Darmstadt, Emergent AI Lab, Intel Labs
Enjoy reading articles? then consider subscribing to Medium membership, it just 5$ a month for unlimited access to all free/paid content.
Subscribe now –
*********************************************
If you want to support me financially which totally optional and voluntary 🙂 ❤️
You can consider buying me chai ( because i don’t drink coffee 🙂 ) at
*********************************************
⏩ IMPORTANT LINKS
Research Paper Summaries:
*********************************************
⏩ Youtube –
⏩ LinkedIn –
⏩ Medium –
⏩ GitHub –
*********************************************
⏩ Please feel free to share out the content and subscribe to my channel –
Tools I use for making videos 🙂
⏩ iPad –
⏩ Apple Pencil –
⏩ GoodNotes –
#techviz #datascienceguy #deeplearning #ai #transformers
About Me:
I am Prakhar Mishra and this channel is my passion project. I am currently pursuing my MS (by research) in Data Science. I have an industry work-ex of 3+ years in the field of Data Science and Machine Learning with a particular focus on Natural Language Processing (NLP).
By: TechViz – The Data Science Guy
Title: SetFit – Efficient Few-Shot Learning Without Prompts (Research Paper Walkthrough)
Sourced From: www.youtube.com/watch?v=6WBK7XSXJM8
Order a Video for Your Business Today. Browse our services here.
Did you miss our previous article…
https://methodmotion.com/harry-potter-magic-beginner-vfx-tutorial-davinci-resolve/