LM

Topics

🤖 Agent	🧠 Model	🚀 Engine	⚙️ GPU & Driver	Evaluation
ZooCode axe MCP	hf (Model Management) Qwen 3.8 27B Gemma 4 Devstral Small 2 24B Muse Glimmer Experience	llama-swap bifrost vLLM ComfyUI llama.cpp for SYCL llama.cpp for CUDA OVMS llama.cpp for OpenVINO llama.cpp for TurboQuant lemonade for AMD NPU SGLang	Intel Arc Pro B70 Install driver Install oneAPI (SYCL) Install OpenVINO .bashrc for B70 Experiment NVIDIA RTX 3060 Install driver AMD Ryzen AI 7 350 Experiment Tools Install nvtop lact PCIe Trouble Shooting	Performance Security Hallucination

News

• SWE Pro Benchmark for models < 32B
• OpenVINO 2026.3 supports Ubuntu 26 - 2026-08-04

Test

Kroki API returned HTTP 400
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg xmlns="http://www.w3.org/2000/svg" width="828" height="125" viewBox="0 0 828 125" version="1.1">
<rect fill="#fff5f7" width="828" height="125"/>
<text id="text" xml:space="preserve" font-weight="bold" fill="#cd0930" font-family="BlinkMacSystemFont, -apple-system, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Fira Sans', 'Droid Sans', 'Helvetica Neue', 'Helvetica', Arial, sans-serif" font-size="16px" x="0" y="5" dy="0">
<tspan x="10" dy="16">Error 400: SyntaxError: Parse error on line 7:</tspan>
<tspan x="10" dy="16">...pp)  C1(vLLM)  A -&gt; B  B -&gt; C1 &amp; C2</tspan>
<tspan x="10" dy="16">--------------------^</tspan>
<tspan x="10" dy="16">Expecting &apos;SEMI&apos;, &apos;NEWLINE&apos;, &apos;EOF&apos;, &apos;AMP&apos;, &apos;START_LINK&apos;, &apos;LINK&apos;, &apos;LINK_ID&apos;, got &apos;MINUS&apos;</tspan>
<tspan x="10" dy="16">Error: Syntax error in graph</tspan>
<tspan x="10" dy="16">    at Worker.convert (file:///usr/local/kroki/src/worker.js:103:15)</tspan>
<tspan x="10" dy="16">    at async file:///usr/local/kroki/src/index.js:36:28</tspan>
</text>
<rect fill="#ff3860" width="3" height="125"/>
</svg>

File Path	/images/quickmmd/LM-models-handling-infra.svg
File Size	15.91 KB
Last modified	N/A
Time of SVG loading	0.79 second(s)
Time of SVG convertion	0.00 second(s)
Cache Detected	No
MD5 of incoming syntax	270bf6feec1fe1e182c94b5a89cd6883
MD5 of cached syntax	45978182992b9851e9b971945432eebb
SVG Engine	Kroki API
Kroki API URL	http://kroki:8000
Mermaid syntax Reference	https://mermaid.js.org/intro/syntax-reference.html
QuickMMD Version	1.0.0 - About QuickMMD

flowchart LR
  A(LiteLLM)
  B(llama-swap)
  C1(llama.cpp)
  C1(vLLM)

  A -> B
  B -> C1 & C2

Decision Tree for tinkering with Intel Arc Pro B70

File Path	/images/quickmmd/LM-tinkering-b70.svg
File Size	40.88 KB
Last modified	16:57:52
Time of SVG loading	0.00 second(s)
Time of SVG convertion	0.00 second(s)
Cache Detected	Yes
SVG Engine	Kroki API
Kroki API URL	http://kroki:8000
Mermaid syntax Reference	https://mermaid.js.org/intro/syntax-reference.html
QuickMMD Version	1.0.0 - About QuickMMD

flowchart TD
  subgraph Hardware
    A1(Intel Arc Pro B70):::node-okay
    A2(Asus ProArt B850 WiFi NEO):::node-okay
    A3(AMD Ryzen 5 9600X):::node-okay
  end

  subgraph OS
    B1(Install<br>Ubuntu 26.04):::node-okay
    A1 & A2 & A3 --> B1
  end

  subgraph Driver
    C1(Install PPA driver<br>26.27.39122.14):::node-okay
    C2(Install oneAPI<br>2026.1.1.20260724):::node-okay
    C3(Install OpenVINO<br>2026.3.0<br>hard to install):::node-warn
    C1 --> C2 & C3
    B1 --> C1
  end

  subgraph Runtime
    D1[Build llama.cpp for<br>SYCL]:::node-okay
    D2[Build llama.cpp for<br>OpenVINO<br>not stable yet]:::node-warn
    %% D3[Build SGLang for<br>SYCL]:::node-todo
    %% D4[Build SGLang for<br>OpenVINO]:::node-todo
    C2 --> D1
    C3 --> D2
    %% C2 --> D3
    %% C3 --> D4
  end

  subgraph Tools
    T1(Install<br>pipx)
    T2(Install<br>hf)
    T3(Install<br>nvtop)
    T4(Install<br>tmux)
    T5(Install<br>lact)
    T6(Install<br>llama-swap)
    T1 --> T2
    B1 --> T1
    B1 --> T3
    B1 --> T4
    B1 --> T5
    T5 --> T6
  end

  subgraph Models
    M1(Qwen3.8-27B)
    M2(Devstral-Small-2-24B)
    T2 --> M1 & M2
  end

  subgraph config
    V1(Optimize<br>.bashrc)
    V2(Optimize<br>lmsw.yaml)
    C2 --> V1
    C3 --> V1
    T2 --> V1
    T4 ---> V1
    T6 ---> V2
  end

  subgraph Service
    Z(service<br>llama-swap \<br>  -config lmsw.yaml \<br>  -listen 0.0.0.0:9876)
    M1 --> Z
    D1 --> Z
    V2 --> Z
  end

  classDef node-okay fill:#efe,stroke:#393
  classDef node-warn fill:#fff0e0,stroke:#d63
  classDef node-fail fill:#fee,stroke:#d33
  classDef node-todo fill:#eee,stroke:#777