--- license: apache-2.0 datasets: - CSI-lab/RCW_2025_Positive_Query_Pairs language: - en base_model: - google/embeddinggemma-2 tags: - embedding - legal - law - WA - feature-extraction - sentence-transformers - multimodal-embedding - multimodal - sentence-similarity - loss:MultipleNegativesRankingLoss metrics: - accuracy - recall - precision pipeline_tag: feature-extraction library_name: transformers model-index: - name: US-State-Law-EmbeddingGemma-2 results: - task: type: information-retrieval name: Information Retrieval dataset: name: CSI-lab/RCW_2025_Positive_Query_Pairs type: rcw-validation metrics: - type: accuracy_at_10 value: 0.8789 name: Accuracy@10 - type: precision_at_10 value: 0.0879 name: Precision@10 - type: recall_at_10 value: 0.8789 name: Recall@10 - type: accuracy_at_1 value: 0.3994 name: Accuracy@1 - type: ndcg_at_10 value: 0.6464 name: NDCG@10 - type: mrr_at_10 value: 0.5709 name: MRR@10 - type: map_at_100 value: 0.5757 name: MAP@100 - type: accuracy_at_3 value: 0.2595 name: Accuracy@3 - type: accuracy_at_5 value: 0.7156 name: Accuracy@5 --- ![US-State law EmbeddingGemma2-smaller](https://cdn-uploads.huggingface.co/production/uploads/69e2a861cc64b2988f0aaf1f/j7UVDZYoIKmfTSWKr4_0Q.png)

Hugging Face | GitHub | Launch Blog | Documentation |
License: Apache 2.0

# US-State-Law-EmbeddingGemma-2 **US-State-Law-EmbeddingGemma-2** is a highly specialized embedding model fine-tuned specifically for Legal Information Retrieval (IR) within the State of Washington. Designed to run on **consumer hardware** such as mobile devices and laptops, `US-State-Law-EmbeddingGemma-2` delivers low-latency semantic representations for on-device applications, like search, retrieval-augmented generation (RAG), classification, and clustering. Generic embedding models often perform suboptimally on legal texts due to the semantic gap between natural language questions (e.g., "What dollar amount makes a theft a first degree felony?") and formal statutory legalese. This model bridges that gap, allowing plain-English queries, legal scenarios, and document drafts to be accurately mapped to their corresponding Washington State statutes (Revised Code of Washington - RCW). This version leverages the **EmbeddingGemma 2** architecture and was fine-tuned using **Low-Rank Adaptation (LoRA)** before being merged into a full 16-bit precision model for optimal performance and efficiency. EmbeddingGemma 2 is an open multimodal embedding model built by Google DeepMind which maps text (incl. code), images, video, and audio inputs—and combinations thereof—into a single, unified 768-dimensional vector space. The model has 740M total parameters, combining a 270M parameter text model with modular vision (170M) and audio (300M) encoders. ## Available Models | Model | Language | Description | |:--------------------------------------------------------------------------------------------------------------------------------|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------------------| | [CSI-lab/US-State-Law-EmbeddingGemma-2](https://huggingface.co/CSI-lab/US-state-law-EmbeddingGemma-2) | English | Fine-tuned `EmbeddingGemma2` for WA State RCWs. Best for high accuracy low-latency inference on consumer hardware such as mobile devices and laptops| | [CSI-lab/US-State-Law-EmbeddingGemma-2-GGUF](https://huggingface.co/CSI-lab/US-state-law-EmbeddingGemma-2-GGUF) | English | Fine-tuned `EmbeddingGemma2` for WA State RCWs. 4-bit quantized (q4_k_m) GGUF format. Optimized for CPU and low-VRAM deployments. | | [CSI-lab/Washington-state-law-embedding-model-Large](https://huggingface.co/CSI-lab/Washington-state-law-embedding-model-Large) | English | Fine-tuned `large` model (1024d) for WA State RCWs. | | [CSI-lab/Washington-state-law-embedding-model-Base](https://huggingface.co/CSI-lab/Washington-state-law-embedding-model-Base) | English | Fine-tuned `base` model (768d) for WA State RCWs. | ## Model Overview * **Base Model:** `EmbeddingGemma 2` * **Task:** Semantic Search / Information Retrieval / Legal Preemption Analysis * **Language:** English (Legal Domain) * **Precision:** Full 16-bit (Merged LoRA) * **Similarity Function:** Cosine Similarity ## Key Features - Fine-tuned specifically for the Washington State legal domain (RCW). - Parameter-efficient fine-tuning via LoRA, yielding high performance with reduced training overhead. - Merged into a full 16-bit model for simplified deployment. - Supports complex, natural language legal queries. - Designed for RAG-based legal assistants and semantic search pipelines. ## Intended Use Cases This model is optimized to act as the retriever component in legal Retrieval-Augmented Generation (RAG) pipelines. Primary use cases include: 1. **Statutory Cross-Referencing:** Mapping natural language legal questions to specific RCWs. 2. **Preemption Checking:** Automatically retrieving state laws that may preempt or conflict with proposed municipal ordinances. 3. **Legal Research Automation:** Clustering and searching local agency drafts against established state frameworks. 4. **AI Legal Assistants:** Powering chatbots and research tools that require accurate retrieval of Washington State laws before generating an answer. 5. **Automated Compliance:** Scanning contracts or external drafts against established state legislative frameworks. ## Technical Details & Training Methodology ### The Semantic Gap A standard dense retriever often fails on legal tasks because it relies on vocabulary overlap rather than conceptual legal mapping. To address this, `US-state-law-EmbeddingGemma-2` was fine-tuned using a synthetic, high-variance dataset designed to map hypothetical language to actual statutes. ### Training Data The model was fine-tuned on synthetic legal query–passage pairs generated from Washington State RCW statutes. The dataset spans hundreds of legal categories derived from the RCW structure and includes: - Natural language paraphrases of legal questions - Hypothetical legal scenarios - Statute-grounded positive document matches ### Hyperparameters & Architecture * **Fine-Tuning Method:** LoRA (Low-Rank Adaptation) * **Precision:** 16-bit (fp16) * **Loss Function:** Multiple Negatives Ranking (MNR) Loss * **Batch Size:** 128 * **Epochs:** 1 * **batch_sampler:** no_duplicates * **multi_dataset_batch_sampler:** round_robin * **Learning Rate Decay:** Linear * **Infrastructure:** High-Performance Computing (HPC) Cluster #### All Hyperparameters
Click to expand - `per_device_train_batch_size`: 128 - `num_train_epochs`: 3.0 - `max_steps`: 1780 - `learning_rate`: 2e-05 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: None - `warmup_steps`: 0.1 - `optim`: adamw_torch_fused - `optim_args`: None - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `optim_target_modules`: None - `gradient_accumulation_steps`: 2 - `average_tokens_across_devices`: True - `max_grad_norm`: 1.0 - `label_smoothing_factor`: 0.0 - `bf16`: True - `fp16`: False - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `use_liger_kernel`: False - `liger_kernel_config`: None - `use_cache`: False - `neftune_noise_alpha`: None - `torch_empty_cache_steps`: None - `auto_find_batch_size`: False - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `include_num_input_tokens_seen`: no - `log_level`: passive - `log_level_replica`: warning - `disable_tqdm`: False - `project`: huggingface - `trackio_space_id`: None - `trackio_bucket_id`: None - `trackio_static_space_id`: None - `per_device_eval_batch_size`: 128 - `prediction_loss_only`: True - `eval_on_start`: False - `eval_do_concat_batches`: True - `eval_use_gather_object`: False - `eval_accumulation_steps`: None - `include_for_metrics`: [] - `batch_eval_metrics`: False - `save_only_model`: False - `save_on_each_node`: False - `enable_jit_checkpoint`: False - `push_to_hub`: False - `hub_private_repo`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_always_push`: False - `hub_revision`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `restore_callback_states_from_checkpoint`: False - `full_determinism`: False - `seed`: 42 - `data_seed`: None - `use_cpu`: False - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `dataloader_prefetch_factor`: None - `dataloader_multiprocessing_context`: None - `dataloader_in_order`: True - `remove_unused_columns`: True - `label_names`: None - `train_sampling_strategy`: random - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `ddp_static_graph`: None - `ddp_backend`: None - `ddp_timeout`: 1800 - `fsdp`: None - `fsdp_config`: None - `deepspeed`: None - `debug`: [] - `skip_memory_metrics`: True - `do_predict`: False - `resume_from_checkpoint`: None - `local_rank`: -1 - `prompts`: {'question': 'task: search result | query: ', 'passage_text': 'title: none | text: '} - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {} - `warmup_ratio`: None
### Training Logs
Click to expand | Epoch | Step | Training Loss | Validation Loss | Cosine_ndcg@10 | |:-------:|:--------:|:-------------:|:---------------:|:--------------------:| | 0.0028 | 5 | 2.7708 | - | - | | 0.0056 | 10 | 2.7859 | - | - | | 0.0084 | 15 | 2.7946 | - | - | | 0.0112 | 20 | 2.7961 | - | - | | 0.0140 | 25 | 2.7882 | - | - | | 0.0169 | 30 | 2.7566 | - | - | | 0.0197 | 35 | 2.7002 | - | - | | 0.0225 | 40 | 2.7090 | - | - | | 0.0253 | 45 | 2.6191 | - | - | | 0.0281 | 50 | 2.5413 | - | - | | 0.0309 | 55 | 2.4808 | - | - | | 0.0337 | 60 | 2.3470 | - | - | | 0.0365 | 65 | 2.2725 | - | - | | 0.0393 | 70 | 2.1348 | - | - | | 0.0421 | 75 | 1.9621 | - | - | | 0.0450 | 80 | 1.8921 | - | - | | 0.0478 | 85 | 1.7558 | - | - | | 0.0506 | 90 | 1.6108 | - | - | | 0.0534 | 95 | 1.3967 | - | - | | 0.0562 | 100 | 1.2471 | - | - | | 0.0590 | 105 | 1.0157 | - | - | | 0.0618 | 110 | 0.9065 | - | - | | 0.0646 | 115 | 0.7530 | - | - | | 0.0674 | 120 | 0.6128 | - | - | | 0.0702 | 125 | 0.5284 | - | - | | 0.0731 | 130 | 0.4157 | - | - | | 0.0759 | 135 | 0.3545 | - | - | | 0.0787 | 140 | 0.2841 | - | - | | 0.0815 | 145 | 0.2717 | - | - | | 0.0843 | 150 | 0.2292 | - | - | | 0.0871 | 155 | 0.2630 | - | - | | 0.0899 | 160 | 0.2331 | - | - | | 0.0927 | 165 | 0.1904 | - | - | | 0.0955 | 170 | 0.2164 | - | - | | 0.0983 | 175 | 0.1773 | - | - | | 0.1012 | 180 | 0.1760 | - | - | | 0.1040 | 185 | 0.1688 | - | - | | 0.1068 | 190 | 0.1918 | - | - | | 0.1096 | 195 | 0.1828 | - | - | | 0.1124 | 200 | 0.1616 | 0.1618 | - | | 0.1152 | 205 | 0.1795 | - | - | | 0.1180 | 210 | 0.1602 | - | - | | 0.1208 | 215 | 0.1758 | - | - | | 0.1236 | 220 | 0.1424 | - | - | | 0.1264 | 225 | 0.1462 | - | - | | 0.1292 | 230 | 0.1438 | - | - | | 0.1321 | 235 | 0.1632 | - | - | | 0.1349 | 240 | 0.1439 | - | - | | 0.1377 | 245 | 0.1470 | - | - | | 0.1405 | 250 | 0.1480 | - | - | | 0.1433 | 255 | 0.1475 | - | - | | 0.1461 | 260 | 0.1415 | - | - | | 0.1489 | 265 | 0.1316 | - | - | | 0.1517 | 270 | 0.1189 | - | - | | 0.1545 | 275 | 0.1440 | - | - | | 0.1573 | 280 | 0.1412 | - | - | | 0.1602 | 285 | 0.1476 | - | - | | 0.1630 | 290 | 0.1279 | - | - | | 0.1658 | 295 | 0.1424 | - | - | | 0.1686 | 300 | 0.1393 | - | - | | 0.1714 | 305 | 0.1576 | - | - | | 0.1742 | 310 | 0.1553 | - | - | | 0.1770 | 315 | 0.1510 | - | - | | 0.1798 | 320 | 0.1272 | - | - | | 0.1826 | 325 | 0.1437 | - | - | | 0.1854 | 330 | 0.1185 | - | - | | 0.1883 | 335 | 0.1254 | - | - | | 0.1911 | 340 | 0.1106 | - | - | | 0.1939 | 345 | 0.1494 | - | - | | 0.1967 | 350 | 0.1217 | - | - | | 0.1995 | 355 | 0.1307 | - | - | | 0.2023 | 360 | 0.1479 | - | - | | 0.2051 | 365 | 0.1314 | - | - | | 0.2079 | 370 | 0.1219 | - | - | | 0.2107 | 375 | 0.1332 | - | - | | 0.2135 | 380 | 0.1251 | - | - | | 0.2164 | 385 | 0.1142 | - | - | | 0.2192 | 390 | 0.1333 | - | - | | 0.2220 | 395 | 0.1264 | - | - | | 0.2248 | 400 | 0.1347 | 0.1212 | - | | 0.2276 | 405 | 0.1144 | - | - | | 0.2304 | 410 | 0.1202 | - | - | | 0.2332 | 415 | 0.1223 | - | - | | 0.2360 | 420 | 0.1107 | - | - | | 0.2388 | 425 | 0.1117 | - | - | | 0.2416 | 430 | 0.1177 | - | - | | 0.2445 | 435 | 0.1159 | - | - | | 0.2473 | 440 | 0.1092 | - | - | | 0.2501 | 445 | 0.1224 | - | - | | 0.2529 | 450 | 0.1156 | - | - | | 0.2557 | 455 | 0.1075 | - | - | | 0.2585 | 460 | 0.1029 | - | - | | 0.2613 | 465 | 0.1034 | - | - | | 0.2641 | 470 | 0.1069 | - | - | | 0.2669 | 475 | 0.1194 | - | - | | 0.2697 | 480 | 0.1058 | - | - | | 0.2725 | 485 | 0.1134 | - | - | | 0.2754 | 490 | 0.0991 | - | - | | 0.2782 | 495 | 0.1165 | - | - | | 0.2810 | 500 | 0.1304 | - | - | | 0.2838 | 505 | 0.1243 | - | - | | 0.2866 | 510 | 0.1326 | - | - | | 0.2894 | 515 | 0.1264 | - | - | | 0.2922 | 520 | 0.1116 | - | - | | 0.2950 | 525 | 0.1242 | - | - | | 0.2978 | 530 | 0.1245 | - | - | | 0.3006 | 535 | 0.1247 | - | - | | 0.3035 | 540 | 0.1143 | - | - | | 0.3063 | 545 | 0.0988 | - | - | | 0.3091 | 550 | 0.1152 | - | - | | 0.3119 | 555 | 0.1132 | - | - | | 0.3147 | 560 | 0.1036 | - | - | | 0.3175 | 565 | 0.1143 | - | - | | 0.3203 | 570 | 0.1174 | - | - | | 0.3231 | 575 | 0.1239 | - | - | | 0.3259 | 580 | 0.1068 | - | - | | 0.3287 | 585 | 0.1196 | - | - | | 0.3316 | 590 | 0.0995 | - | - | | 0.3344 | 595 | 0.1096 | - | - | | 0.3372 | 600 | 0.1029 | 0.1095 | - | | 0.3400 | 605 | 0.0977 | - | - | | 0.3428 | 610 | 0.1044 | - | - | | 0.3456 | 615 | 0.1091 | - | - | | 0.3484 | 620 | 0.0882 | - | - | | 0.3512 | 625 | 0.1103 | - | - | | 0.3540 | 630 | 0.1164 | - | - | | 0.3568 | 635 | 0.1009 | - | - | | 0.3597 | 640 | 0.1109 | - | - | | 0.3625 | 645 | 0.1186 | - | - | | 0.3653 | 650 | 0.1234 | - | - | | 0.3681 | 655 | 0.1100 | - | - | | 0.3709 | 660 | 0.0911 | - | - | | 0.3737 | 665 | 0.1149 | - | - | | 0.3765 | 670 | 0.1066 | - | - | | 0.3793 | 675 | 0.1081 | - | - | | 0.3821 | 680 | 0.1000 | - | - | | 0.3849 | 685 | 0.1154 | - | - | | 0.3877 | 690 | 0.0945 | - | - | | 0.3906 | 695 | 0.1117 | - | - | | 0.3934 | 700 | 0.0990 | - | - | | 0.3962 | 705 | 0.1095 | - | - | | 0.3990 | 710 | 0.1075 | - | - | | 0.4018 | 715 | 0.1091 | - | - | | 0.4046 | 720 | 0.1163 | - | - | | 0.4074 | 725 | 0.1227 | - | - | | 0.4102 | 730 | 0.1140 | - | - | | 0.4130 | 735 | 0.1036 | - | - | | 0.4158 | 740 | 0.1033 | - | - | | 0.4187 | 745 | 0.0959 | - | - | | 0.4215 | 750 | 0.1022 | - | - | | 0.4243 | 755 | 0.1154 | - | - | | 0.4271 | 760 | 0.1123 | - | - | | 0.4299 | 765 | 0.1269 | - | - | | 0.4327 | 770 | 0.0908 | - | - | | 0.4355 | 775 | 0.1109 | - | - | | 0.4383 | 780 | 0.0932 | - | - | | 0.4411 | 785 | 0.1103 | - | - | | 0.4439 | 790 | 0.0997 | - | - | | 0.4468 | 795 | 0.1024 | - | - | | 0.4496 | 800 | 0.1078 | 0.1018 | - | | 0.4524 | 805 | 0.1023 | - | - | | 0.4552 | 810 | 0.1048 | - | - | | 0.4580 | 815 | 0.0976 | - | - | | 0.4608 | 820 | 0.1021 | - | - | | 0.4636 | 825 | 0.1182 | - | - | | 0.4664 | 830 | 0.1105 | - | - | | 0.4692 | 835 | 0.0913 | - | - | | 0.4720 | 840 | 0.1023 | - | - | | 0.4749 | 845 | 0.1065 | - | - | | 0.4777 | 850 | 0.1075 | - | - | | 0.4805 | 855 | 0.0920 | - | - | | 0.4833 | 860 | 0.0886 | - | - | | 0.4861 | 865 | 0.1115 | - | - | | 0.4889 | 870 | 0.0931 | - | - | | 0.4917 | 875 | 0.1074 | - | - | | 0.4945 | 880 | 0.0815 | - | - | | 0.4973 | 885 | 0.1105 | - | - | | 0.5001 | 890 | 0.0874 | - | - | | 0.5030 | 895 | 0.0952 | - | - | | 0.5058 | 900 | 0.1005 | - | - | | 0.5086 | 905 | 0.1074 | - | - | | 0.5114 | 910 | 0.1025 | - | - | | 0.5142 | 915 | 0.1058 | - | - | | 0.5170 | 920 | 0.0998 | - | - | | 0.5198 | 925 | 0.1071 | - | - | | 0.5226 | 930 | 0.0998 | - | - | | 0.5254 | 935 | 0.0946 | - | - | | 0.5282 | 940 | 0.1022 | - | - | | 0.5310 | 945 | 0.0935 | - | - | | 0.5339 | 950 | 0.0920 | - | - | | 0.5367 | 955 | 0.0801 | - | - | | 0.5395 | 960 | 0.1100 | - | - | | 0.5423 | 965 | 0.0964 | - | - | | 0.5451 | 970 | 0.1017 | - | - | | 0.5479 | 975 | 0.1102 | - | - | | 0.5507 | 980 | 0.0897 | - | - | | 0.5535 | 985 | 0.0976 | - | - | | 0.5563 | 990 | 0.0907 | - | - | | 0.5591 | 995 | 0.1040 | - | - | | 0.5620 | 1000 | 0.0986 | 0.0971 | - | | 0.5648 | 1005 | 0.0986 | - | - | | 0.5676 | 1010 | 0.0965 | - | - | | 0.5704 | 1015 | 0.0985 | - | - | | 0.5732 | 1020 | 0.0887 | - | - | | 0.5760 | 1025 | 0.0904 | - | - | | 0.5788 | 1030 | 0.0872 | - | - | | 0.5816 | 1035 | 0.0930 | - | - | | 0.5844 | 1040 | 0.0894 | - | - | | 0.5872 | 1045 | 0.0971 | - | - | | 0.5901 | 1050 | 0.0902 | - | - | | 0.5929 | 1055 | 0.1031 | - | - | | 0.5957 | 1060 | 0.1078 | - | - | | 0.5985 | 1065 | 0.1040 | - | - | | 0.6013 | 1070 | 0.1074 | - | - | | 0.6041 | 1075 | 0.0991 | - | - | | 0.6069 | 1080 | 0.0810 | - | - | | 0.6097 | 1085 | 0.0901 | - | - | | 0.6125 | 1090 | 0.0915 | - | - | | 0.6153 | 1095 | 0.0992 | - | - | | 0.6182 | 1100 | 0.0984 | - | - | | 0.6210 | 1105 | 0.0975 | - | - | | 0.6238 | 1110 | 0.0895 | - | - | | 0.6266 | 1115 | 0.0910 | - | - | | 0.6294 | 1120 | 0.1136 | - | - | | 0.6322 | 1125 | 0.0785 | - | - | | 0.6350 | 1130 | 0.1176 | - | - | | 0.6378 | 1135 | 0.0981 | - | - | | 0.6406 | 1140 | 0.0915 | - | - | | 0.6434 | 1145 | 0.0893 | - | - | | 0.6462 | 1150 | 0.1047 | - | - | | 0.6491 | 1155 | 0.0833 | - | - | | 0.6519 | 1160 | 0.0938 | - | - | | 0.6547 | 1165 | 0.0834 | - | - | | 0.6575 | 1170 | 0.0971 | - | - | | 0.6603 | 1175 | 0.1031 | - | - | | 0.6631 | 1180 | 0.1049 | - | - | | 0.6659 | 1185 | 0.0886 | - | - | | 0.6687 | 1190 | 0.0931 | - | - | | 0.6715 | 1195 | 0.0899 | - | - | | 0.6743 | 1200 | 0.0847 | 0.0937 | - | | 0.6772 | 1205 | 0.0917 | - | - | | 0.6800 | 1210 | 0.1049 | - | - | | 0.6828 | 1215 | 0.1061 | - | - | | 0.6856 | 1220 | 0.0948 | - | - | | 0.6884 | 1225 | 0.0936 | - | - | | 0.6912 | 1230 | 0.0911 | - | - | | 0.6940 | 1235 | 0.0848 | - | - | | 0.6968 | 1240 | 0.0829 | - | - | | 0.6996 | 1245 | 0.0885 | - | - | | 0.7024 | 1250 | 0.0908 | - | - | | 0.7053 | 1255 | 0.0909 | - | - | | 0.7081 | 1260 | 0.0881 | - | - | | 0.7109 | 1265 | 0.0875 | - | - | | 0.7137 | 1270 | 0.0967 | - | - | | 0.7165 | 1275 | 0.0872 | - | - | | 0.7193 | 1280 | 0.0827 | - | - | | 0.7221 | 1285 | 0.0894 | - | - | | 0.7249 | 1290 | 0.1099 | - | - | | 0.7277 | 1295 | 0.0834 | - | - | | 0.7305 | 1300 | 0.0757 | - | - | | 0.7334 | 1305 | 0.0802 | - | - | | 0.7362 | 1310 | 0.0941 | - | - | | 0.7390 | 1315 | 0.0827 | - | - | | 0.7418 | 1320 | 0.0798 | - | - | | 0.7446 | 1325 | 0.0919 | - | - | | 0.7474 | 1330 | 0.0826 | - | - | | 0.7502 | 1335 | 0.0931 | - | - | | 0.7530 | 1340 | 0.0910 | - | - | | 0.7558 | 1345 | 0.1067 | - | - | | 0.7586 | 1350 | 0.0980 | - | - | | 0.7614 | 1355 | 0.0766 | - | - | | 0.7643 | 1360 | 0.0973 | - | - | | 0.7671 | 1365 | 0.1020 | - | - | | 0.7699 | 1370 | 0.0908 | - | - | | 0.7727 | 1375 | 0.0894 | - | - | | 0.7755 | 1380 | 0.1045 | - | - | | 0.7783 | 1385 | 0.0817 | - | - | | 0.7811 | 1390 | 0.0916 | - | - | | 0.7839 | 1395 | 0.0891 | - | - | | 0.7867 | 1400 | 0.0831 | 0.0916 | - | | 0.7895 | 1405 | 0.0933 | - | - | | 0.7924 | 1410 | 0.0920 | - | - | | 0.7952 | 1415 | 0.0812 | - | - | | 0.7980 | 1420 | 0.0957 | - | - | | 0.8008 | 1425 | 0.1048 | - | - | | 0.8036 | 1430 | 0.1032 | - | - | | 0.8064 | 1435 | 0.1013 | - | - | | 0.8092 | 1440 | 0.0903 | - | - | | 0.8120 | 1445 | 0.1034 | - | - | | 0.8148 | 1450 | 0.0917 | - | - | | 0.8176 | 1455 | 0.0793 | - | - | | 0.8205 | 1460 | 0.0857 | - | - | | 0.8233 | 1465 | 0.0880 | - | - | | 0.8261 | 1470 | 0.1008 | - | - | | 0.8289 | 1475 | 0.1023 | - | - | | 0.8317 | 1480 | 0.0870 | - | - | | 0.8345 | 1485 | 0.0928 | - | - | | 0.8373 | 1490 | 0.0806 | - | - | | 0.8401 | 1495 | 0.0935 | - | - | | 0.8429 | 1500 | 0.0819 | - | - | | 0.8457 | 1505 | 0.0951 | - | - | | 0.8486 | 1510 | 0.0994 | - | - | | 0.8514 | 1515 | 0.0900 | - | - | | 0.8542 | 1520 | 0.0947 | - | - | | 0.8570 | 1525 | 0.0932 | - | - | | 0.8598 | 1530 | 0.1004 | - | - | | 0.8626 | 1535 | 0.0886 | - | - | | 0.8654 | 1540 | 0.0872 | - | - | | 0.8682 | 1545 | 0.0804 | - | - | | 0.8710 | 1550 | 0.0803 | - | - | | 0.8738 | 1555 | 0.0905 | - | - | | 0.8767 | 1560 | 0.0761 | - | - | | 0.8795 | 1565 | 0.0774 | - | - | | 0.8823 | 1570 | 0.0893 | - | - | | 0.8851 | 1575 | 0.0875 | - | - | | 0.8879 | 1580 | 0.1135 | - | - | | 0.8907 | 1585 | 0.0916 | - | - | | 0.8935 | 1590 | 0.1039 | - | - | | 0.8963 | 1595 | 0.1032 | - | - | | 0.8991 | 1600 | 0.0988 | 0.0902 | - | | 0.9019 | 1605 | 0.0919 | - | - | | 0.9047 | 1610 | 0.0764 | - | - | | 0.9076 | 1615 | 0.0842 | - | - | | 0.9104 | 1620 | 0.0960 | - | - | | 0.9132 | 1625 | 0.0969 | - | - | | 0.9160 | 1630 | 0.0902 | - | - | | 0.9188 | 1635 | 0.0996 | - | - | | 0.9216 | 1640 | 0.0708 | - | - | | 0.9244 | 1645 | 0.0801 | - | - | | 0.9272 | 1650 | 0.0979 | - | - | | 0.9300 | 1655 | 0.0828 | - | - | | 0.9328 | 1660 | 0.0792 | - | - | | 0.9357 | 1665 | 0.0872 | - | - | | 0.9385 | 1670 | 0.0802 | - | - | | 0.9413 | 1675 | 0.0929 | - | - | | 0.9441 | 1680 | 0.1133 | - | - | | 0.9469 | 1685 | 0.0840 | - | - | | 0.9497 | 1690 | 0.0821 | - | - | | 0.9525 | 1695 | 0.0894 | - | - | | 0.9553 | 1700 | 0.0772 | - | - | | 0.9581 | 1705 | 0.0908 | - | - | | 0.9609 | 1710 | 0.0888 | - | - | | 0.9638 | 1715 | 0.0806 | - | - | | 0.9666 | 1720 | 0.0741 | - | - | | 0.9694 | 1725 | 0.1010 | - | - | | 0.9722 | 1730 | 0.0980 | - | - | | 0.9750 | 1735 | 0.0944 | - | - | | 0.9778 | 1740 | 0.0959 | - | - | | 0.9806 | 1745 | 0.0831 | - | - | | 0.9834 | 1750 | 0.0911 | - | - | | 0.9862 | 1755 | 0.0842 | - | - | | 0.9890 | 1760 | 0.1042 | - | - | | 0.9919 | 1765 | 0.0965 | - | - | | 0.9947 | 1770 | 0.0883 | - | - | | 0.9975 | 1775 | 0.0784 | - | - | | **1.0** | **1780** | **0.073** | **0.0898** | **-** | | -1 | -1 | - | - | 0.6464 | * The bold row denotes the saved checkpoint.
### Training Time - **Training**: 8.3 hours ## Evaluation Metrics The model was evaluated on a rigorously held-out validation set of synthetic municipal drafts mapped 1-to-1 against Washington State RCWs. The metrics below demonstrate the model's exceptional retrieval capabilities. | Metric | Score | |:-------|:------| | **accuracy@10** | 0.8789 | | **Recall@10** | 0.8789 | | **Recall@5** | 0.8027 | | **Recall@3** | 0.7156 | | **Recall@1** | 0.3993 | | **NDCG@10** | 0.6464 | | **MRR@10** | 0.5709 | | **MAP@100** | 0.5757 | | **precision@10** | 0.0879 | | **precision@1** | 0.3994 | | **precision@3** | 0.2385 | | **precision@5** | 0.1605 | *Interpretation: When a user asks this model a legal question in plain English, there is an **87.9% probability** that the exact governing state law will be returned in the top 10 search results.* ## Limitations - This model does not provide legal advice. - Performance is limited to Washington State law (RCW) and may not generalize to other jurisdictions. - Outputs depend on the quality of the underlying document corpus. - Should be used as a retrieval tool, not a final decision-making system. ## Usage Examples ### Semantic Search with `sentence-transformers` ```python import torch from sentence_transformers import SentenceTransformer, util # 1. Load the 16-bit fine-tuned model model = SentenceTransformer('CSI-lab/US-state-law-EmbeddingGemma-2', model_kwargs={ "torch_dtype": torch.bfloat16, "config_kwargs": { "load_vision_encoder": False, "load_audio_encoder": False } } ) # 2. Define the laws (Your Vector Database) laws = [ "RCW 9A.56.030: Theft in the first degree. A person is guilty of theft in the first degree if he or she commits theft of property or services which exceed(s) five thousand dollars in value.", "RCW 46.61.502: Driving under the influence. A person is guilty of driving while under the influence of intoxicating liquor...", "RCW 9A.36.011: Assault in the first degree. A person is guilty of assault in the first degree if he or she..." ] # 3. Define the user's search query user_query = "What dollar amount makes a theft a first degree felony?" # 4. Encode the documents and the query with respective prompt_name parameters query_embedding = model.encode(user_query, convert_to_tensor=True, prompt_name="SearchQuery") law_embeddings = model.encode(laws, convert_to_tensor=True, prompt_name="Document") # 5. Calculate Cosine Similarity cosine_scores = util.cos_sim(query_embedding, law_embeddings) # 6. Print the top result best_idx = cosine_scores.argmax().item() print(f"Top Match: {laws[best_idx]}") print(f"Similarity Score: {cosine_scores[0][best_idx]:.4f}") ``` ## **Best Practices** For optimal embedding quality and runtime efficiency, follow these configurations and best practices: ### 1. Task Instruction Prefixes EmbeddingGemma 2 is trained with short task instruction prefixes prepended to text inputs. Using the right prefix improves quality; omitting it still works but reduces precision. Prefixes apply to text only. Pass images, video, and audio without any prefix. Documents with a real title should be formatted as `title: {title} | text: {content}`. Use `title: none` when no title is available. **Prefix Notation & Usage** We offer two types of task prefixes, depending on how embeddings are used in the task. There are two types of tasks: * **Asymmetric Tasks (e.g. retrieval):** Use a query prefix for queries and a document prefix for corpus items. * **Symmetric Tasks (e.g. classification, similarity)**: Apply the same task prefix to all inputs being compared. | Use Case | Task Type | Prompt Name | Query Task Instruction | Document Task Instruction *(use `none` if no title)* | | :---- | :---- | :---- | :---- | :---- | | Web / document search | Asymmetric | SearchQuery | `task: search result \| query: {query}` | `title: {title} \| text: {content}` | | Question answering | Asymmetric | QuestionAnswering | `task: question answering \| query: {question}` | `title: {title} \| text: {passage}` | | Fact checking | Asymmetric | FactChecking | `task: fact checking \| query: {claim}` | `title: {title} \| text: {evidence}` | | Code search | Asymmetric | CodeRetrieval | `task: code retrieval \| query: {query}` | `title: {title or filename} \| text: {code}` | | Text classification | Symmetric | Classification | `task: classification \| query: {content}` | N/A | | Clustering | Symmetric | Clustering | `task: clustering \| query: {content}` | N/A | | Measuring similarity | Symmetric | SentenceSimilarity | `task: sentence similarity \| query: {content}` | N/A | **Please note:** `prompt_name="Document"` applies `title: none`; titled documents must still be formatted manually, like `model.encode(f"title: {title} | text: {document}")` # Model Citation ```bibtex @misc{US_state_law_embedding_gemma2_2026, title={US-state-law-EmbeddingGemma-2: Fine-Tuned Dense Retrieval for Washington State La}, author={Tomar, Shlok and Complex Social Interactions Lab}, year={2026}, publisher={Hugging Face}, howpublished={\url{https://huggingface.co/CSI-lab/US-state-law-EmbeddingGemma-2}}, note={Hugging Face Model Repository} } ``` ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### Low-Rank Adaptation (LoRA) ```bibtex @article{hu2021lora, title={LoRA: Low-Rank Adaptation of Large Language Models}, author={Hu, Edward J and Shen, Yelong and Wallis, Phil and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu}, journal={arXiv preprint arXiv:2106.09685}, year={2021} } ``` #### EmbeddingGemma Paper ```bibtex @article{embedding_gemma_2025, title={EmbeddingGemma: Powerful and Lightweight Text Representations}, author={Schechter Vera, Henrique and Dua, Sahil and Team, EmbeddingGemma}, journal={arXiv preprint arXiv:2509.20354}, year={2025}, url={https://arxiv.org/abs/2509.20354} } ``` ### EmbeddingGemma 2 Official Release ```bibtex @online{embedding_gemma_2_2026, author={Google DeepMind}, title={EmbeddingGemma 2 is a best-in-class open model for natively multimodal embeddings}, year={2026}, url={https://blog.google/innovation-and-ai/technology/developers-tools/embeddinggemma-2/}, note={Google Innovation and AI Blog} } ```