LM/Experience: Difference between revisions
< LM
Created page with "{| class="wikitable" ! Model || loading failed || thinking loop || |- | Ornith-1.0-9B-GGUF || || v |}" |
|||
| (20 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
{| class="wikitable" | == Comparison == | ||
! | |||
=== Proxy === | |||
{| class="wikitable" style="text-align:center;" | |||
! || LiteLLM || bifrost || llama-swap | |||
|- | |- | ||
| | | llama.cpp || 🟢 || 🟢 || 🟢 | ||
|- | |||
| vLLM || 🟢 || 🟢 || ❌ | |||
|- | |||
| UI Playground || 🟢 || ❌ || 🟢 | |||
|- | |||
| User Management || 🟢 || ⚠️ || ❌ | |||
|- | |||
| Show Tok/s per session || ❌ || ❌ || 🟢 | |||
|} | |} | ||
* User Management function of bifrost is for Enterprise version only. | |||
== Test Model == | |||
<syntaxhighlight lang="bash"> | |||
curl http://${IP}:5800/v1/chat/completions \ | |||
-H "Content-Type: application/json" \ | |||
-d '{ | |||
"messages": [{"role": "user", "content": "Hello"}] | |||
}' | |||
</syntaxhighlight> | |||
Latest revision as of 03:55, 21 September 2026
Comparison
Proxy
| LiteLLM | bifrost | llama-swap | |
|---|---|---|---|
| llama.cpp | 🟢 | 🟢 | 🟢 |
| vLLM | 🟢 | 🟢 | ❌ |
| UI Playground | 🟢 | ❌ | 🟢 |
| User Management | 🟢 | ⚠️ | ❌ |
| Show Tok/s per session | ❌ | ❌ | 🟢 |
- User Management function of bifrost is for Enterprise version only.
Test Model
curl http://${IP}:5800/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [{"role": "user", "content": "Hello"}]
}'