Jan AI 本地助手¶

# 方法1: 官网下载
# 访问 https://jan.ai 下载对应平台安装包

# 方法2: Linux命令行
# Debian/Ubuntu
wget https://github.com/janhq/jan/releases/latest/download/jan-linux-amd64.deb
sudo dpkg -i jan-linux-amd64.deb

# AppImage
wget https://github.com/janhq/jan/releases/latest/download/jan-linux-x86_64.AppImage
chmod +x jan-linux-x86_64.AppImage
./jan-linux-x86_64.AppImage

你是一位专业的编程助手，精通Python和数据科学。
请用中文回答，提供简洁实用的代码示例。

curl http://localhost:1337/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.2-3b-instruct",
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 0.7
  }'

# 查看已安装扩展
Settings → Extensions

# 核心扩展
- inference-nitro-extension  # llama.cpp推理
- inference-tensorrt-extension  # TensorRT加速
- conversational-extension  # 对话管理
- model-extension  # 模型管理

// 在Jan数据目录下创建模型配置
// ~/jan/models/my-custom-model/model.json
{
  "id": "my-custom-model",
  "object": "model",
  "name": "My Custom Model",
  "version": "1.0",
  "description": "自定义模型",
  "format": "gguf",
  "settings": {
    "ctx_len": 4096,
    "ngl": 33,
    "embedding": false,
    "n_parallel": 1
  },
  "parameters": {
    "temperature": 0.7,
    "top_p": 0.95,
    "max_tokens": 2048
  },
  "engine": "nitro"
}

// 配置OpenAI远程模型
{
  "id": "gpt-4-remote",
  "object": "model",
  "name": "GPT-4 (Remote)",
  "engine": "openai",
  "settings": {
    "api_key": "sk-...",
    "base_url": "https://api.openai.com/v1"
  }
}

# 引擎选项
nitro:        # 默认, llama.cpp后端
  优点: 兼容性好, 支持CPU
  适合: 通用场景

tensorrt-llm: # NVIDIA TensorRT加速
  优点: 推理速度快2-3倍
  适合: NVIDIA GPU用户

external:     # 外部API
  优点: 无需本地资源
  适合: 连接远程服务

~/jan/
├── models/           # 模型文件
│   ├── llama-3.2-3b/
│   │   ├── model.json
│   │   └── model.gguf
│   └── mistral-7b/
├── threads/          # 对话历史
│   └── thread-001/
│       └── messages.json
├── assistants/       # 助手配置
│   └── code-helper/
│       └── assistant.json
└── extensions/       # 扩展

from openai import OpenAI

# 连接Jan本地服务
client = OpenAI(
    base_url="http://localhost:1337/v1",
    api_key="jan"
)

# 流式对话
def chat_stream(prompt, system="你是一个有帮助的助手"):
    stream = client.chat.completions.create(
        model="llama-3.2-3b-instruct",
        messages=[
            {"role": "system", "content": system},
            {"role": "user", "content": prompt}
        ],
        stream=True,
        temperature=0.7
    )

    full_response = ""
    for chunk in stream:
        content = chunk.choices[0].delta.content
        if content:
            full_response += content
            print(content, end="", flush=True)
    return full_response

response = chat_stream("用Python实现二分查找")

# NVIDIA用户确认CUDA安装
nvidia-smi

# 在Jan设置中确认GPU加速已启用
# Settings → Advanced → GPU Acceleration → ON

# 重启Jan后重新加载模型

# 整个Jan数据目录可以直接复制
cp -r ~/jan ~/jan-backup

# 迁移到新机器
# 复制jan目录到新机器相同路径即可