BjarneNPO/async-snow-72

SentenceTransformer based on Snowflake/snowflake-arctic-embed-m-v2.0

This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v2.0 on the train dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Snowflake/snowflake-arctic-embed-m-v2.0
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • train

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'GteModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("BjarneNPO/finetune_21_08_2025_17_18_28")
# Run inference
queries = [
    "fragt wie der Stand zu dem aktuellen Problem ist",
]
documents = [
    'In Klärung mit der Kollegin - Das Problem liegt leider an deren Betreiber. Die sind aber informiert und arbeiten bereits daran',
    'findet diese in der Übersicht der Gruppen.',
    'Userin muss sich an die Bistums IT wenden.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.3229, 0.0208, 0.0018]])

Evaluation

Metrics

Information Retrieval

• Dataset: Snowflake/snowflake-arctic-embed-m-v2.0
• Evaluated with scripts.InformationRetrievalEvaluatorCustom.InformationRetrievalEvaluatorCustom with these parameters:

  {
      "query_prompt_name": "query",
      "corpus_prompt_name": "document"
  }
  

Metric	Value
cosine_accuracy@1	0.3285
cosine_accuracy@3	0.5255
cosine_accuracy@5	0.5912
cosine_accuracy@10	0.6788
cosine_precision@1	0.3285
cosine_precision@3	0.2822
cosine_precision@5	0.2672
cosine_precision@10	0.2482
cosine_recall@1	0.0111
cosine_recall@3	0.0375
cosine_recall@5	0.0654
cosine_recall@10	0.109
cosine_ndcg@10	0.2705
cosine_mrr@10	0.4461
cosine_map@100	0.1168

Training Details

Training Dataset

train

• Dataset: train
• Size: 19,964 training samples
• Columns: query and answer
• Approximate statistics based on the first 1000 samples:

  	query	answer
  type	string	string
  details	min: 4 tokens mean: 27.77 tokens max: 615 tokens	min: 3 tokens mean: 22.87 tokens max: 151 tokens

• Samples:

  query	answer
  Wie kann man die Jahresurlaubsübersicht exportieren?	über das 3 Punkte Menü rechts oben. Mitarbeiter auswählen und exportieren
  1. Vertragsabschlüsse werden nicht übertragen
  2. Kinder kommen nicht von nach
  3. Absage kann bei Portalstatus nicht erstellt werden.	Ticket
  Userin gebeten sich an den Support zu wenden, da der Fehler liegt.
  Wird im Anmeldeportal nicht gefunden.	Die Schnittstelle war noch nicht aktiviert und Profil ebenfalls nicht.

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 32
• gradient_accumulation_steps: 4
• learning_rate: 2e-05
• num_train_epochs: 30
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 30
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Snowflake/snowflake-arctic-embed-m-v2.0_cosine_ndcg@10
0.1282	10	3.4959	-
0.2564	20	3.4292	-
0.3846	30	3.4574	-
0.5128	40	3.2452	-
0.6410	50	3.138	-
0.7692	60	3.0415	-
0.8974	70	2.927	-
1.0	78	-	0.2437
1.0256	80	2.7918	-
1.1538	90	2.7491	-
1.2821	100	2.6212	-
1.4103	110	2.5664	-
1.5385	120	2.4249	-
1.6667	130	2.3568	-
1.7949	140	2.2513	-
1.9231	150	2.2262	-
2.0	156	-	0.2782
2.0513	160	2.0465	-
2.1795	170	2.0932	-
2.3077	180	2.0553	-
2.4359	190	1.9922	-
2.5641	200	1.9537	-
2.6923	210	1.8484	-
2.8205	220	1.8762	-
2.9487	230	1.755	-
3.0	234	-	0.2676
3.0769	240	1.6551	-
3.2051	250	1.7135	-
3.3333	260	1.6684	-
3.4615	270	1.6556	-
3.5897	280	1.5677	-
3.7179	290	1.5067	-
3.8462	300	1.5204	-
3.9744	310	1.4643	-
4.0	312	-	0.2541
4.1026	320	1.3292	-
4.2308	330	1.4336	-
4.3590	340	1.4306	-
4.4872	350	1.3455	-
4.6154	360	1.3079	-
4.7436	370	1.2589	-
4.8718	380	1.2851	-
5.0	390	1.201	0.2543
5.1282	400	1.1415	-
5.2564	410	1.219	-
5.3846	420	1.215	-
5.5128	430	1.1423	-
5.6410	440	1.0829	-
5.7692	450	1.0705	-
5.8974	460	1.0877	-
6.0	468	-	0.2564
6.0256	470	0.9736	-
6.1538	480	1.0314	-
6.2821	490	1.0072	-
6.4103	500	1.0214	-
6.5385	510	0.9747	-
6.6667	520	0.9298	-
6.7949	530	0.9426	-
6.9231	540	0.9166	-
7.0	546	-	0.2428
7.0513	550	0.8048	-
7.1795	560	0.873	-
7.3077	570	0.9017	-
7.4359	580	0.8477	-
7.5641	590	0.8457	-
7.6923	600	0.7475	-
7.8205	610	0.8235	-
7.9487	620	0.7519	-
8.0	624	-	0.2388
8.0769	630	0.7188	-
8.2051	640	0.7541	-
8.3333	650	0.7432	-
8.4615	660	0.7417	-
8.5897	670	0.6693	-
8.7179	680	0.6548	-
8.8462	690	0.6818	-
8.9744	700	0.6426	-
9.0	702	-	0.2495
9.1026	710	0.5831	-
9.2308	720	0.6503	-
9.3590	730	0.6576	-
9.4872	740	0.6282	-
9.6154	750	0.584	-
9.7436	760	0.5744	-
9.8718	770	0.5818	-
10.0	780	0.5429	0.2499
10.1282	790	0.508	-
10.2564	800	0.5671	-
10.3846	810	0.5556	-
10.5128	820	0.5316	-
10.6410	830	0.4881	-
10.7692	840	0.5073	-
10.8974	850	0.5264	-
11.0	858	-	0.2541
11.0256	860	0.4445	-
11.1538	870	0.4855	-
11.2821	880	0.476	-
11.4103	890	0.4762	-
11.5385	900	0.45	-
11.6667	910	0.4386	-
11.7949	920	0.4436	-
11.9231	930	0.4321	-
12.0	936	-	0.2598
12.0513	940	0.3659	-
12.1795	950	0.4196	-
12.3077	960	0.4285	-
12.4359	970	0.4094	-
12.5641	980	0.4123	-
12.6923	990	0.3555	-
12.8205	1000	0.3994	-
12.9487	1010	0.3584	-
13.0	1014	-	0.2551
13.0769	1020	0.3332	-
13.2051	1030	0.3718	-
13.3333	1040	0.3695	-
13.4615	1050	0.3601	-
13.5897	1060	0.326	-
13.7179	1070	0.3334	-
13.8462	1080	0.3481	-
13.9744	1090	0.3161	-
14.0	1092	-	0.2626
14.1026	1100	0.2976	-
14.2308	1110	0.3257	-
14.3590	1120	0.3343	-
14.4872	1130	0.3177	-
14.6154	1140	0.2942	-
14.7436	1150	0.3015	-
14.8718	1160	0.2829	-
15.0	1170	0.2731	0.2543
15.1282	1180	0.2593	-
15.2564	1190	0.2993	-
15.3846	1200	0.2846	-
15.5128	1210	0.2849	-
15.6410	1220	0.2562	-
15.7692	1230	0.2804	-
15.8974	1240	0.2737	-
16.0	1248	-	0.2585
16.0256	1250	0.2295	-
16.1538	1260	0.2562	-
16.2821	1270	0.2749	-
16.4103	1280	0.2727	-
16.5385	1290	0.2513	-
16.6667	1300	0.2445	-
16.7949	1310	0.2358	-
16.9231	1320	0.2432	-
17.0	1326	-	0.2659
17.0513	1330	0.1989	-
17.1795	1340	0.2347	-
17.3077	1350	0.242	-
17.4359	1360	0.2293	-
17.5641	1370	0.2325	-
17.6923	1380	0.203	-
17.8205	1390	0.2378	-
17.9487	1400	0.2018	-
18.0	1404	-	0.2628
18.0769	1410	0.1847	-
18.2051	1420	0.2154	-
18.3333	1430	0.216	-
18.4615	1440	0.2201	-
18.5897	1450	0.1929	-
18.7179	1460	0.1962	-
18.8462	1470	0.2039	-
18.9744	1480	0.193	-
19.0	1482	-	0.2552
19.1026	1490	0.1802	-
19.2308	1500	0.1998	-
19.3590	1510	0.2019	-
19.4872	1520	0.1979	-
19.6154	1530	0.1852	-
19.7436	1540	0.1765	-
19.8718	1550	0.1881	-
20.0	1560	0.1738	0.2681
20.1282	1570	0.166	-
20.2564	1580	0.187	-
20.3846	1590	0.1902	-
20.5128	1600	0.1843	-
20.6410	1610	0.1673	-
20.7692	1620	0.1773	-
20.8974	1630	0.1803	-
21.0	1638	-	0.2686
21.0256	1640	0.1485	-
21.1538	1650	0.1734	-
21.2821	1660	0.1736	-
21.4103	1670	0.1806	-
21.5385	1680	0.1711	-
21.6667	1690	0.1644	-
21.7949	1700	0.17	-
21.9231	1710	0.1619	-
22.0	1716	-	0.2683
22.0513	1720	0.136	-
22.1795	1730	0.1663	-
22.3077	1740	0.1738	-
22.4359	1750	0.1664	-
22.5641	1760	0.1618	-
22.6923	1770	0.1473	-
22.8205	1780	0.1695	-
22.9487	1790	0.1464	-
23.0	1794	-	0.2723
23.0769	1800	0.1385	-
23.2051	1810	0.1608	-
23.3333	1820	0.1616	-
23.4615	1830	0.1683	-
23.5897	1840	0.1467	-
23.7179	1850	0.1504	-
23.8462	1860	0.1595	-
23.9744	1870	0.1449	-
24.0	1872	-	0.2764
24.1026	1880	0.1364	-
24.2308	1890	0.1656	-
24.3590	1900	0.158	-
24.4872	1910	0.1572	-
24.6154	1920	0.1468	-
24.7436	1930	0.1479	-
24.8718	1940	0.1478	-
25.0	1950	0.1383	0.2674
25.1282	1960	0.1387	-
25.2564	1970	0.1581	-
25.3846	1980	0.1494	-
25.5128	1990	0.151	-
25.6410	2000	0.1383	-
25.7692	2010	0.1513	-
25.8974	2020	0.1488	-
26.0	2028	-	0.2727
26.0256	2030	0.1274	-
26.1538	2040	0.1454	-
26.2821	2050	0.146	-
26.4103	2060	0.1551	-
26.5385	2070	0.14	-
26.6667	2080	0.1442	-
26.7949	2090	0.1469	-
26.9231	2100	0.1437	-
27.0	2106	-	0.2721
27.0513	2110	0.1241	-
27.1795	2120	0.1438	-
27.3077	2130	0.1534	-
27.4359	2140	0.1438	-
27.5641	2150	0.1485	-
27.6923	2160	0.1335	-
27.8205	2170	0.1508	-
27.9487	2180	0.1374	-
28.0	2184	-	0.2712
28.0769	2190	0.1304	-
28.2051	2200	0.1438	-
28.3333	2210	0.1471	-
28.4615	2220	0.154	-
28.5897	2230	0.1377	-
28.7179	2240	0.1352	-
28.8462	2250	0.1517	-
28.9744	2260	0.139	-
29.0	2262	-	0.2710
29.1026	2270	0.1263	-
29.2308	2280	0.1468	-
29.3590	2290	0.1464	-
29.4872	2300	0.1456	-
29.6154	2310	0.1385	-
29.7436	2320	0.1422	-
29.8718	2330	0.1446	-
30.0	2340	0.1359	0.2705

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.2
• PyTorch: 2.8.0+cu129
• Accelerate: 1.10.0
• Datasets: 3.6.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}