Instructions to use victorlxh/ICKG-v2.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use victorlxh/ICKG-v2.0 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="victorlxh/ICKG-v2.0")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("victorlxh/ICKG-v2.0") model = AutoModelForCausalLM.from_pretrained("victorlxh/ICKG-v2.0", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use victorlxh/ICKG-v2.0 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "victorlxh/ICKG-v2.0" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "victorlxh/ICKG-v2.0", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/victorlxh/ICKG-v2.0
- SGLang
How to use victorlxh/ICKG-v2.0 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "victorlxh/ICKG-v2.0" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "victorlxh/ICKG-v2.0", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "victorlxh/ICKG-v2.0" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "victorlxh/ICKG-v2.0", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use victorlxh/ICKG-v2.0 with Docker Model Runner:
docker model run hf.co/victorlxh/ICKG-v2.0
Update README.md
Browse files
README.md
CHANGED
|
@@ -24,7 +24,7 @@ The primary use of ICKG LLM is for generating knowledge graphs (KG) based on ins
|
|
| 24 |
- **Python Code**: [https://github.com/xiaohui-victor-li/FinDKG](https://github.com/xiaohui-victor-li/FinDKG)
|
| 25 |
|
| 26 |
## Training Details
|
| 27 |
-
ICKG 2.0 is fine-tuned from the latest Vicuna-7B using ~3K instruction-following demonstrations including KG construction input document and extracted KG triplets as response output.
|
| 28 |
|
| 29 |
- **Prompt Template**: The entities and relationship can be customized for specific tasks. `<input_text>` is the document text to replace.
|
| 30 |
|
|
@@ -64,7 +64,7 @@ ICKG 2.0 is fine-tuned from the latest Vicuna-7B using ~3K instruction-following
|
|
| 64 |
```
|
| 65 |
|
| 66 |
## Evaluation
|
| 67 |
-
ICKG has undergone preliminary evaluation comparing its performance to GPT-3.5, GPT-4, and the original Vicuna-7B model. With respect to the KG construction task, it outperforms GPT-3.5 and Vicuna-7B while exhibiting comparative capability as GPT-4.
|
| 68 |
|
| 69 |
For a more detailed introduction, refer to [the accompanying paper](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4608445).
|
| 70 |
|
|
|
|
| 24 |
- **Python Code**: [https://github.com/xiaohui-victor-li/FinDKG](https://github.com/xiaohui-victor-li/FinDKG)
|
| 25 |
|
| 26 |
## Training Details
|
| 27 |
+
ICKG 2.0 is fine-tuned from the latest Vicuna-7B using ~3K instruction-following demonstrations including KG construction input document and extracted KG triplets as response output. ICKG is thus learnt to extract list of KG triplets from given text document via prompt engineering. For more in-depth training details, refer to the "Generative Knowledge Graph Construction with Fine-tuned LLM" section of [the accompanying paper](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4608445).
|
| 28 |
|
| 29 |
- **Prompt Template**: The entities and relationship can be customized for specific tasks. `<input_text>` is the document text to replace.
|
| 30 |
|
|
|
|
| 64 |
```
|
| 65 |
|
| 66 |
## Evaluation
|
| 67 |
+
ICKG has undergone preliminary evaluation comparing its performance to GPT-3.5, GPT-4, and the original Vicuna-7B model. With respect to the KG construction task, it outperforms GPT-3.5 and Vicuna-7B while exhibiting comparative capability as GPT-4. ICKG excels in generating instruction-based knowledge graphs with a particular emphasis on quality and adherence to format.
|
| 68 |
|
| 69 |
For a more detailed introduction, refer to [the accompanying paper](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4608445).
|
| 70 |
|