Instructions to use lianghsun/Llama-3.2-Taiwan-1B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lianghsun/Llama-3.2-Taiwan-1B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lianghsun/Llama-3.2-Taiwan-1B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lianghsun/Llama-3.2-Taiwan-1B-Instruct")
model = AutoModelForCausalLM.from_pretrained("lianghsun/Llama-3.2-Taiwan-1B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lianghsun/Llama-3.2-Taiwan-1B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lianghsun/Llama-3.2-Taiwan-1B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-1B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lianghsun/Llama-3.2-Taiwan-1B-Instruct

SGLang

How to use lianghsun/Llama-3.2-Taiwan-1B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lianghsun/Llama-3.2-Taiwan-1B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-1B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lianghsun/Llama-3.2-Taiwan-1B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lianghsun/Llama-3.2-Taiwan-1B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lianghsun/Llama-3.2-Taiwan-1B-Instruct with Docker Model Runner:
```
docker model run hf.co/lianghsun/Llama-3.2-Taiwan-1B-Instruct
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Model Card for Llama-3.2-Taiwan-1B-Instruct

Llama-3.2-Taiwan-1B-Instruct 是 Llama-3.2-Taiwan-1B 的指令微調版本，以繁中對話資料完成 SFT，可在資源受限的端側／邊緣裝置上提供台灣語境之輕量級繁中對話能力。

⚠️ 規格重點：本模型為 1B 參數小型語言模型（SLM）、純文本單模態。

Model Details

繁中模型在 1B 級別的選擇仍少，且大多數需要在通用對話能力與台灣本地語境之間取捨。本模型在 Llama-3.2-Taiwan-1B 之繁中 CPT 底座上做指令微調，目標是在 1B 規模下提供穩定可用的繁中對話與在地語境回答。

核心特點 (Key Features)

1B 級可用對話：模型尺寸友善，可在消費級裝置運行。
台灣語境對齊：訓練資料以繁中與台灣常見任務為主，避免「會繁體中文卻不懂台灣」之問題。
適合做為下游基底：可作為更小型專案、領域應用之 SFT／DPO 起點。

Model Description

Developed by: Liang Hsun Huang
Funded by: APMIC
Base model: lianghsun/Llama-3.2-Taiwan-1B
Model type: LlamaForCausalLM (Transformers)
Language(s) (NLP): Traditional Chinese, English
License: llama3.2
Finetuned from model: lianghsun/Llama-3.2-Taiwan-1B

Model Sources

Repository: lianghsun/Llama-3.2-Taiwan-1B-Instruct

Citation

@misc{llama_3_2_taiwan_1b_instruct,
  title        = {Llama-3.2-Taiwan-1B-Instruct: A Lightweight Traditional Chinese Instruction-Tuned Model for Taiwan},
  author       = {Huang, Liang Hsun},
  year         = {2024},
  howpublished = {\url{https://huggingface.co/lianghsun/Llama-3.2-Taiwan-1B-Instruct}}
}