Instructions to use dineshpotla/healthx-qwen35-4b-router-v1.8-experimental with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use dineshpotla/healthx-qwen35-4b-router-v1.8-experimental with PEFT:
Task type is invalid.
- Transformers
How to use dineshpotla/healthx-qwen35-4b-router-v1.8-experimental with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="dineshpotla/healthx-qwen35-4b-router-v1.8-experimental")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("dineshpotla/healthx-qwen35-4b-router-v1.8-experimental", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use dineshpotla/healthx-qwen35-4b-router-v1.8-experimental with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "dineshpotla/healthx-qwen35-4b-router-v1.8-experimental" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dineshpotla/healthx-qwen35-4b-router-v1.8-experimental", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/dineshpotla/healthx-qwen35-4b-router-v1.8-experimental
- SGLang
How to use dineshpotla/healthx-qwen35-4b-router-v1.8-experimental with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "dineshpotla/healthx-qwen35-4b-router-v1.8-experimental" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dineshpotla/healthx-qwen35-4b-router-v1.8-experimental", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "dineshpotla/healthx-qwen35-4b-router-v1.8-experimental" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "dineshpotla/healthx-qwen35-4b-router-v1.8-experimental", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use dineshpotla/healthx-qwen35-4b-router-v1.8-experimental with Docker Model Runner:
docker model run hf.co/dineshpotla/healthx-qwen35-4b-router-v1.8-experimental
HealthX Qwen3.5-4B Query Router (Experimental)
This repository contains the LoRA adapter only for the HealthX natural-language
query router. It is not a standalone base model and does not contain the Qwen
base weights. The adapter proposes a compact, schema-constrained routing decision
(intent, optional entity and time scope, limit, and clarification state) for the
HealthX read-only interoperability system.
Status and intended use
This is an experimental research artifact. It is not approved for production, clinical, diagnostic, prognostic, treatment, prescribing, or clinical-decision use. It must remain behind deterministic safety checks, patient/scope authorization, record-grounded entity resolution, strict schema validation, deterministic policy compilation, and read-only retrieval tools. The adapter cannot write to an EHR or select tools by itself.
The adapter passed the 142-example guarded recovery evaluation, but its one permitted disjoint external successor evaluation missed the release gates (99.3% schema validity and 87.5% unsupported-request recall against 99.5% and 95% minimums). The adapter therefore remains disabled by default in HealthX and must be explicitly opted into for evaluation.
Base model
- Base:
Qwen/Qwen3.5-4B - Pinned base revision:
851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a - Training method: 4-bit CUDA QLoRA with PEFT
- LoRA rank / alpha / dropout: 16 / 32 / 0.05
- Target modules:
all-linear - Maximum sequence length: 512
- Training steps: 600
- Seed:
20260811
Training data
The router was trained on 4,720 balanced examples selected from a larger corpus
of synthetic, patient-disjoint queries derived from synthetic FHIR concepts. The
training and validation hashes, dependency versions, and adapter file hashes are
preserved in training_manifest.json. No PHI, clinical databases, raw FHIR/C-CDA
documents, Open-i images, or source tokens are included in this repository.
Files
adapter/adapter_model.safetensors: LoRA weightsadapter/adapter_config.json: PEFT configuration and base-model referenceadapter/tokenizer.json,adapter/tokenizer_config.json: tokenizer filesadapter/chat_template.jinja: chat template used by the evaluatoradapter/router_system_instruction.txt: constrained router instructiontraining_manifest.json: reproducibility metadata and SHA-256 inventory
Loading with Transformers and PEFT
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_id = "Qwen/Qwen3.5-4B"
adapter_id = "dineshpotla/healthx-qwen35-4b-router-v1.8-experimental"
tokenizer = AutoTokenizer.from_pretrained(adapter_id, subfolder="adapter")
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id, subfolder="adapter")
For the full HealthX integrity and safety path, download this repository and
preserve its training_manifest.json plus the adapter/ directory as a local
artifact. HealthX verifies the manifest hashes before loading an experimental
adapter and requires an explicit experimental opt-in.
Limitations and safety
- Synthetic-query evaluation does not establish clinical validity or real-user generalization.
- The learned output is only a proposal; trusted deterministic code owns tool selection, authorization, retrieval, evidence composition, and abstention.
- Invalid, unsafe, unsupported, or unauthorized requests must be rejected or routed to the deterministic fallback.
- Never use this artifact to infer a diagnosis, treatment, patient identity, authorization, or clinical truth from missing data.
Licensing and attribution
This adapter is published as an experimental research artifact while its
standalone redistribution license is under review. Users must comply with the
license and terms of the Qwen/Qwen3.5-4B base model
(listed there as Apache-2.0) and retain attribution for the synthetic Synthea
training sources. This repository contains no patient-identifiable data.
Source project: dineshpotla/agentic-health-exchange
- Downloads last month
- -