Instructions to use swarecito/smol-256 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use swarecito/smol-256 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="swarecito/smol-256", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("swarecito/smol-256") model = AutoModelForMultimodalLM.from_pretrained("swarecito/smol-256", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use swarecito/smol-256 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "swarecito/smol-256" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "swarecito/smol-256", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/swarecito/smol-256
- SGLang
How to use swarecito/smol-256 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "swarecito/smol-256" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "swarecito/smol-256", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "swarecito/smol-256" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "swarecito/smol-256", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use swarecito/smol-256 with Docker Model Runner:
docker model run hf.co/swarecito/smol-256
Upload folder using huggingface_hub
Browse files- .venv/lib/python3.13/site-packages/pip/_vendor/__pycache__/typing_extensions.cpython-313.pyc +2 -2
- .venv/lib/python3.13/site-packages/pip/_vendor/distlib/t64-arm.exe +2 -2
- .venv/lib/python3.13/site-packages/pip/_vendor/distlib/t64.exe +2 -2
- .venv/lib/python3.13/site-packages/pip/_vendor/distlib/w64-arm.exe +2 -2
- .venv/lib/python3.13/site-packages/pip/_vendor/distlib/w64.exe +2 -2
- .venv/lib/python3.13/site-packages/pip/_vendor/idna/__pycache__/uts46data.cpython-313.pyc +2 -2
- .venv/lib/python3.13/site-packages/pip/_vendor/pkg_resources/__pycache__/__init__.cpython-313.pyc +2 -2
- .venv/lib/python3.13/site-packages/pip/_vendor/rich/__pycache__/_emoji_codes.cpython-313.pyc +2 -2
- .venv/lib/python3.13/site-packages/pip/_vendor/rich/__pycache__/console.cpython-313.pyc +2 -2
- model.safetensors +1 -1
- onnx/decoder_model_merged.onnx +1 -1
- onnx/decoder_model_merged_bnb4.onnx +1 -1
- onnx/decoder_model_merged_fp16.onnx +1 -1
- onnx/decoder_model_merged_int8.onnx +1 -1
- onnx/decoder_model_merged_q4.onnx +1 -1
- onnx/decoder_model_merged_q4f16.onnx +1 -1
- onnx/decoder_model_merged_quantized.onnx +1 -1
- onnx/decoder_model_merged_uint8.onnx +1 -1
- onnx/embed_tokens.onnx +1 -1
- onnx/embed_tokens_bnb4.onnx +1 -1
- onnx/embed_tokens_fp16.onnx +1 -1
- onnx/embed_tokens_int8.onnx +1 -1
- onnx/embed_tokens_q4.onnx +1 -1
- onnx/embed_tokens_q4f16.onnx +1 -1
- onnx/embed_tokens_quantized.onnx +1 -1
- onnx/embed_tokens_uint8.onnx +1 -1
- onnx/vision_encoder.onnx +1 -1
- onnx/vision_encoder_bnb4.onnx +1 -1
- onnx/vision_encoder_fp16.onnx +1 -1
- onnx/vision_encoder_int8.onnx +1 -1
- onnx/vision_encoder_q4.onnx +1 -1
- onnx/vision_encoder_q4f16.onnx +1 -1
- onnx/vision_encoder_quantized.onnx +1 -1
- onnx/vision_encoder_uint8.onnx +1 -1
- requirements.txt +1 -0
.venv/lib/python3.13/site-packages/pip/_vendor/__pycache__/typing_extensions.cpython-313.pyc
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4d29f44547d84b392ade8b3563dd2d549a52f7d39bc499f8024a87af537c72b5
|
| 3 |
+
size 128
|
.venv/lib/python3.13/site-packages/pip/_vendor/distlib/t64-arm.exe
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7f872690311cb3182a0949a8638aa2b3c1a9f8e84ea06ba92d81da22f024c7ee
|
| 3 |
+
size 128
|
.venv/lib/python3.13/site-packages/pip/_vendor/distlib/t64.exe
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:efaa744f659fd0d1e9b216f6242d4decdef7a905f2102b74aabdb461f20727e2
|
| 3 |
+
size 128
|
.venv/lib/python3.13/site-packages/pip/_vendor/distlib/w64-arm.exe
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:517ed02d75ccbc1b4e73cf1b3b31443b113c9f9e09964ac8d44c1507dbbc20c2
|
| 3 |
+
size 128
|
.venv/lib/python3.13/site-packages/pip/_vendor/distlib/w64.exe
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ddeec7665cad380fbc90f0e3b171a7fb543dbe5e38a5aa59b9fd4cede209c02f
|
| 3 |
+
size 128
|
.venv/lib/python3.13/site-packages/pip/_vendor/idna/__pycache__/uts46data.cpython-313.pyc
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7d42cff432f38b14c4f9022adc0e71e28ec51531033a7850fd5efb8a6cd49282
|
| 3 |
+
size 128
|
.venv/lib/python3.13/site-packages/pip/_vendor/pkg_resources/__pycache__/__init__.cpython-313.pyc
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1f49b79fa8bb4961597bd666fde87ec8bd2bc3fab57b5d874ee10dedc2dd0448
|
| 3 |
+
size 128
|
.venv/lib/python3.13/site-packages/pip/_vendor/rich/__pycache__/_emoji_codes.cpython-313.pyc
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:13fb1f2e81ead011350e0764ab69d1c71d2670949886c2d104c7e3e7533bb098
|
| 3 |
+
size 128
|
.venv/lib/python3.13/site-packages/pip/_vendor/rich/__pycache__/console.cpython-313.pyc
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bca334669e3bab7899efc58633c056390bddc169bae95f364470ed5fb6e7594f
|
| 3 |
+
size 128
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e918b2969192d1dce450f1174cdf027a8ebe506a673f2025132b3c41054078d5
|
| 3 |
size 128
|
onnx/decoder_model_merged.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5074fb0c60e41412630b607ea296673e41b87e07787245ac5b2e25a2e92ae38a
|
| 3 |
size 128
|
onnx/decoder_model_merged_bnb4.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ce527d9b919875313c636aa7e294a06a777675faf5fb7fe812b3e21ae8613420
|
| 3 |
size 128
|
onnx/decoder_model_merged_fp16.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3c844ccf52cd9f9be0677a9ac451230022c0b53b24236eda11a0095a48618dca
|
| 3 |
size 128
|
onnx/decoder_model_merged_int8.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:82037b078da776295465afbcfc285faad1b6b9d7e3f6ac4dcdc580599ac0d107
|
| 3 |
size 128
|
onnx/decoder_model_merged_q4.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f9d8908c86c9f711910589fe998854e098a8140ab3e52b2e838faa9bbe8ddd06
|
| 3 |
size 128
|
onnx/decoder_model_merged_q4f16.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2504a7a2d2b5463bf5508e2320acb50b35f1f0bce09952d729f90f015fbf94c6
|
| 3 |
size 128
|
onnx/decoder_model_merged_quantized.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2f704b2d5591e92e748cd6038b433cfc35e5263f377f80d2c72c71f350f3a4c2
|
| 3 |
size 128
|
onnx/decoder_model_merged_uint8.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2f704b2d5591e92e748cd6038b433cfc35e5263f377f80d2c72c71f350f3a4c2
|
| 3 |
size 128
|
onnx/embed_tokens.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:100fdd3e2d1a25c5a71df29bb2867422bdb4536174736b4cc80bdbcb9c7fe2cf
|
| 3 |
size 128
|
onnx/embed_tokens_bnb4.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:51264b00a65853950fe29cd960b04dbbdcfc040e56a79d77642cb620fb87acc8
|
| 3 |
size 128
|
onnx/embed_tokens_fp16.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5fb6a859f09939e5a935b295710092597da20d1461c6586ef830a247cc137185
|
| 3 |
size 128
|
onnx/embed_tokens_int8.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7589ac07aa1e39ab1f41fdd6d2e20e57a077e1496eac32ea9c3b6ea2f5af2b6f
|
| 3 |
size 128
|
onnx/embed_tokens_q4.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:51264b00a65853950fe29cd960b04dbbdcfc040e56a79d77642cb620fb87acc8
|
| 3 |
size 128
|
onnx/embed_tokens_q4f16.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:808ade0eb96dd88cfd2d8078cd09c21c7a3b09fb4b3c57df49ada9456b86ad18
|
| 3 |
size 128
|
onnx/embed_tokens_quantized.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7589ac07aa1e39ab1f41fdd6d2e20e57a077e1496eac32ea9c3b6ea2f5af2b6f
|
| 3 |
size 128
|
onnx/embed_tokens_uint8.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7589ac07aa1e39ab1f41fdd6d2e20e57a077e1496eac32ea9c3b6ea2f5af2b6f
|
| 3 |
size 128
|
onnx/vision_encoder.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cf8e8db163823f862f94ce1802c3978147b53589ba51930800e9810b8e31f697
|
| 3 |
size 128
|
onnx/vision_encoder_bnb4.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:199ab0eb48fb4c2870cd27c549d902b3e27299e7d5ebda9979db345f2c567137
|
| 3 |
size 128
|
onnx/vision_encoder_fp16.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7dc992801a9297fffcfff6c8f1e723f50feafac74f86030a945bc4664d67d25c
|
| 3 |
size 128
|
onnx/vision_encoder_int8.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2065f1a67fd4f9ca554065fe7e54f8557680a4d220ff094a33f6def465fd17b4
|
| 3 |
size 128
|
onnx/vision_encoder_q4.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:67a8c027a6506b59f785ad6f406ee9f15e587f472821b0d7cc685b52b6a48c2e
|
| 3 |
size 128
|
onnx/vision_encoder_q4f16.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3429e4473f0d859320d11c83063a885f46b40146ee8cc2d67d128295788c4a02
|
| 3 |
size 128
|
onnx/vision_encoder_quantized.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d139e98b913c506540a7c0fb0bef5211e90c893be83ff9ffb7b3cce6d319ab0d
|
| 3 |
size 128
|
onnx/vision_encoder_uint8.onnx
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 128
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d139e98b913c506540a7c0fb0bef5211e90c893be83ff9ffb7b3cce6d319ab0d
|
| 3 |
size 128
|
requirements.txt
CHANGED
|
@@ -3,4 +3,5 @@ transformers
|
|
| 3 |
Pillow
|
| 4 |
decord
|
| 5 |
num2words
|
|
|
|
| 6 |
|
|
|
|
| 3 |
Pillow
|
| 4 |
decord
|
| 5 |
num2words
|
| 6 |
+
av
|
| 7 |
|