Instructions to use vicgalle/gpt-j-6B-alpaca-gpt4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use vicgalle/gpt-j-6B-alpaca-gpt4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="vicgalle/gpt-j-6B-alpaca-gpt4")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("vicgalle/gpt-j-6B-alpaca-gpt4")
model = AutoModelForCausalLM.from_pretrained("vicgalle/gpt-j-6B-alpaca-gpt4", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use vicgalle/gpt-j-6B-alpaca-gpt4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "vicgalle/gpt-j-6B-alpaca-gpt4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vicgalle/gpt-j-6B-alpaca-gpt4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/vicgalle/gpt-j-6B-alpaca-gpt4

SGLang

How to use vicgalle/gpt-j-6B-alpaca-gpt4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "vicgalle/gpt-j-6B-alpaca-gpt4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vicgalle/gpt-j-6B-alpaca-gpt4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "vicgalle/gpt-j-6B-alpaca-gpt4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vicgalle/gpt-j-6B-alpaca-gpt4",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use vicgalle/gpt-j-6B-alpaca-gpt4 with Docker Model Runner:
```
docker model run hf.co/vicgalle/gpt-j-6B-alpaca-gpt4
```

gpt-j-6B-alpaca-gpt4 / README.md

vicgalle

Update README.md

00715c9 over 3 years ago

preview code

Raw History Blame

2.05 kB

metadata

license: mit
datasets:
  - vicgalle/alpaca-gpt4
language:
  - en
tags:
  - gpt-j
  - instruction-tuning
  - alpaca
  - gpt4

GPT-J-6B instruction-tuned on Alpaca-GPT4

This model was finetuned on GPT-4 generations of the Alpaca prompts, using LoRA for 30.000 steps (batch size of 128), taking over 7 hours in four V100S.

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Give three tips for a good meal.

### Response:


Tip 1: Make sure to enjoy your meal. It's easy to forget about enjoying yourself when you're hungry.
Tip 2: Try to eat a healthy meal that contains lots of vegetables and fruits.
Tip 3: Eat slowly and chew your food thoroughly. This will help you digest your meal better.

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Write a poem about a delicious night.

### Response:


> "The moon shone brightly on the night sky,
> 
> Stars twinkled brightly from afar.
> 
> Suddenly, my eyes saw the silhouette
> 
> Of a handsome stranger in the distance.
> 
> He walked slowly towards me with a smile.
> 
> He said, 'I've come to eat you up tonight.'
> 
> Suddenly, I knew that he was a cannibal!
> 
> 'Wait!' I cried as he approached.
> 
> I yelled, 'I'm vegetarian!'
> 
> Suddenly, he laughed and replied,
> 
> 'Oh, you silly girl! That's okay.'
> 
> Suddenly, he grabbed my hand and kissed it.
> 
> He said, 'You're delicious!'
> 
> Suddenly, he dragged me to his lair.
> 
> There, he ripped open my belly button
> 
> And gobbled me up like a juicy steak.
> 
> Suddenly, I woke up screaming from a nightmare.
> 
> 'Damn! Damn! Damn!' I cried.
>

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
Write a tweet describing your capabilities.

### Response:


Sample tweet: "@geeksforgeeks I'm a mathematician and I enjoy programming. Feel free to reach out to me if you want to chat!"