cahlen/setfit-navigation-instructions

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 4 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
forward	'Proceed along this path ahead' 'Proceed carefully in that direction' 'Proceed forward a little'
right	'Move toward the right' 'Adjust your position to the right' 'Adjust your position slightly to the right'
left	'Head towards the left side' 'Move towards your left' 'Shift your way to the left'
backward	'Could you step back slightly?' 'Move backward, please' 'Could you go back the other way?'

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("cahlen/setfit-navigation-instructions")
# Run inference
preds = model("Move to the right")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	2	5.0	12

Label	Training Sample Count
right	22
left	21
forward	11
backward	13

Training Hyperparameters

• batch_size: (8, 8)
• num_epochs: (4, 4)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0024	1	0.1239	-
0.1220	50	0.1257	-
0.2439	100	0.0215	-
0.3659	150	0.0047	-
0.4878	200	0.0025	-
0.6098	250	0.0017	-
0.7317	300	0.0014	-
0.8537	350	0.0011	-
0.9756	400	0.0013	-
1.0	410	-	0.0182
1.0976	450	0.0009	-
1.2195	500	0.0008	-
1.3415	550	0.0007	-
1.4634	600	0.0007	-
1.5854	650	0.0006	-
1.7073	700	0.0007	-
1.8293	750	0.0006	-
1.9512	800	0.0006	-
2.0	820	-	0.0227
2.0732	850	0.0005	-
2.1951	900	0.0005	-
2.3171	950	0.0006	-
2.4390	1000	0.0005	-
2.5610	1050	0.0006	-
2.6829	1100	0.0005	-
2.8049	1150	0.0005	-
2.9268	1200	0.0004	-
3.0	1230	-	0.0236
3.0488	1250	0.0004	-
3.1707	1300	0.0004	-
3.2927	1350	0.0004	-
3.4146	1400	0.0005	-
3.5366	1450	0.0004	-
3.6585	1500	0.0004	-
3.7805	1550	0.0004	-
3.9024	1600	0.0004	-
4.0	1640	-	0.0240

Framework Versions

• Python: 3.10.12
• SetFit: 1.1.0
• Sentence Transformers: 3.2.1
• Transformers: 4.44.2
• PyTorch: 2.8.0.dev20250331+cu128
• Datasets: 3.5.0
• Tokenizers: 0.19.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}