Instructions to use SYNLP/ChiMed-GPT-1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SYNLP/ChiMed-GPT-1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SYNLP/ChiMed-GPT-1.0")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SYNLP/ChiMed-GPT-1.0")
model = AutoModelForCausalLM.from_pretrained("SYNLP/ChiMed-GPT-1.0")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SYNLP/ChiMed-GPT-1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SYNLP/ChiMed-GPT-1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SYNLP/ChiMed-GPT-1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SYNLP/ChiMed-GPT-1.0

SGLang

How to use SYNLP/ChiMed-GPT-1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SYNLP/ChiMed-GPT-1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SYNLP/ChiMed-GPT-1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SYNLP/ChiMed-GPT-1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SYNLP/ChiMed-GPT-1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SYNLP/ChiMed-GPT-1.0 with Docker Model Runner:
```
docker model run hf.co/SYNLP/ChiMed-GPT-1.0
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

ChiMed-GPT

ChiMed-GPT is a Chinese medical large language model (LLM) built by continually training Ziya-v2 on Chinese medical data, where pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF) are comprehensively performed on it. More information about the model is coming soon.

Citation

If you use or extend our work, please cite the following paper:

@article{USTC-ChiMed-GPT,
  title="{ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences}",
  author={Yuanhe Tian, Ruyi Gan, Yan Song, Jiaxing Zhang, Yongdong Zhang},
  journal={arXiv preprint arXiv:2311.06025},
  year={2023},
}

Usage

from transformers import AutoTokenizer
from transformers import LlamaForCausalLM
import torch

query="[human]:感冒怎么处理？\n[bot]:"
model = LlamaForCausalLM.from_pretrained('SYNLP/ChiMed-GPT-1.0', torch_dtype=torch.float16, device_map="auto").eval()
tokenizer = AutoTokenizer.from_pretrained(ckpt)
input_ids = tokenizer(query, return_tensors="pt").input_ids.to('cuda:0')
generate_ids = model.generate(
            input_ids,
            max_new_tokens=512, 
            do_sample = True, 
            top_p = 0.9)
output = tokenizer.batch_decode(generate_ids)[0]
print(output)

Disclaimer

Please note that the content generated by ChiMed-GPT, including any advice, suggestions, information, or recommendations, does not reflect our views or beliefs. The responses provided by the large language model should not be considered as endorsements, opinions, or advice from us. We do not take responsibility for the accuracy, reliability, or appropriateness of the information provided. Users should exercise their own judgment and discretion when interpreting and using the information generated by the large language model.

Downloads last month: 628

Model tree for SYNLP/ChiMed-GPT-1.0

Quantizations

1 model

Space using SYNLP/ChiMed-GPT-1.0 1

Papers for SYNLP/ChiMed-GPT-1.0

ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences

Paper • 2311.06025 • Published Nov 10, 2023 • 1

Ziya2: Data-centric Learning is All LLMs Need

Paper • 2311.03301 • Published Nov 6, 2023 • 20