LM/Experience: Difference between revisions
< LM
No edit summary |
|||
| Line 6: | Line 6: | ||
! || LiteLLM || bifrost || llama-swap | ! || LiteLLM || bifrost || llama-swap | ||
|- | |- | ||
| llama.cpp || || || | | llama.cpp || 🟢 || 🟢 || 🟢 | ||
|- | |- | ||
| vLLM || || || | | vLLM || 🟢 || 🟢 || ❌ | ||
|- | |- | ||
| User Management || || || | | User Management || ⚠️ || 🟢 || ❌ | ||
|} | |} | ||
Revision as of 03:40, 21 September 2026
Comparison
Proxy
| LiteLLM | bifrost | llama-swap | |
|---|---|---|---|
| llama.cpp | 🟢 | 🟢 | 🟢 |
| vLLM | 🟢 | 🟢 | ❌ |
| User Management | ⚠️ | 🟢 | ❌ |
Test Model
curl http://${IP}:5800/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [{"role": "user", "content": "Hello"}]
}'