Instructions to use AxiomicLabs/GPT-X3-150M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AxiomicLabs/GPT-X3-150M with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AxiomicLabs/GPT-X3-150M", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("AxiomicLabs/GPT-X3-150M", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AxiomicLabs/GPT-X3-150M with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AxiomicLabs/GPT-X3-150M"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AxiomicLabs/GPT-X3-150M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AxiomicLabs/GPT-X3-150M

SGLang

How to use AxiomicLabs/GPT-X3-150M with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AxiomicLabs/GPT-X3-150M" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AxiomicLabs/GPT-X3-150M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AxiomicLabs/GPT-X3-150M" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AxiomicLabs/GPT-X3-150M",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AxiomicLabs/GPT-X3-150M with Docker Model Runner:
```
docker model run hf.co/AxiomicLabs/GPT-X3-150M
```

bruh

by Banaxi-Tech - opened 13 days ago

Discussion

Banaxi-Tech

13 days ago

you used the exact datasets i was planning to use on bm 3

Datdanboi25

Axiomic Labs org 13 days ago

Haha lol great minds...

GGUFGuy

13 days ago

I think Novi-1.5 will be extended pretraining of orginal Novi, and Novi-2 will have insane dataset mix

Banaxi-Tech

13 days ago

yo @Datdanboi25 check out https://huggingface.co/spaces/BananaMind/blog#bananamind-gate-attention BAnanaMind 3 WILL HAVE 64K Context WIndow

Datdanboi25

Axiomic Labs org 13 days ago

Yeah checked it out earlier, cool idea, would be keen to see how it performs in terms of both speed and quality

Banaxi-Tech

13 days ago

As of what I know from small tests its better than deepseek sparse attention!

Datdanboi25 changed discussion status to closed 8 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment