How to use Goraint/Qwen3-32B-4bit-AWQ-MLX with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("Goraint/Qwen3-32B-4bit-AWQ-MLX") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True)
How to use Goraint/Qwen3-32B-4bit-AWQ-MLX with Pi:
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "Goraint/Qwen3-32B-4bit-AWQ-MLX"
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Goraint/Qwen3-32B-4bit-AWQ-MLX" } ] } } }
# Start Pi in your project directory: pi
How to use Goraint/Qwen3-32B-4bit-AWQ-MLX with MLX LM:
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "Goraint/Qwen3-32B-4bit-AWQ-MLX"
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "Goraint/Qwen3-32B-4bit-AWQ-MLX" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Goraint/Qwen3-32B-4bit-AWQ-MLX", "messages": [ {"role": "user", "content": "Hello"} ] }'
How to use Goraint/Qwen3-32B-4bit-AWQ-MLX with Hermes Agent:
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Goraint/Qwen3-32B-4bit-AWQ-MLX
hermes
How to use Goraint/Qwen3-32B-4bit-AWQ-MLX with OpenClaw:
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Goraint/Qwen3-32B-4bit-AWQ-MLX" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
openclaw agent --local --agent main --message "Hello from Hugging Face"
This model is a quantized version of Qwen3-32B converted to AWQ Q4 format using the mlx library for efficient inference. It retains the core capabilities of Qwen3 while optimizing for resource constraints.
Chat template
Files info
Quantized
Base model
