LM/Performance: Difference between revisions
< LM
Created page with "== betterbench == === Use === <syntaxhighlight lang="bash"> betterbench run --endpoint https://models.nothanks.com/v1 --model my_model --api-key sk-12345678 </syntaxhighlight> === Install ===" |
|||
| (6 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
== | == Quick Test: llama-benchy == | ||
=== Use === | === Use === | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
# concurrency 8, test vLLM throughput | |||
llama-benchy --base-url http://models.nothanks.com/v1 \ | |||
--model my_model --api-key sk-12345678 \ | |||
--pp 512 --tg 128 --concurrency 8 \ | |||
2>/dev/null | grep "^|" -B1 -A3 | |||
# concurrency 1 | |||
llama-benchy --base-url https://models.nothanks.com/v1 \ | |||
--model my_model --api-key sk-12345678 \ | |||
--pp 512 --tg 128 | |||
</syntaxhighlight> | </syntaxhighlight> | ||
=== Install === | === Install === | ||
<syntaxhighlight lang="bash"> | |||
pipx install llama-benchy | |||
</syntaxhighlight> | |||
== Full Test: betterbench == | |||
=== Use === | |||
<syntaxhighlight lang="bash"> | |||
betterbench run \ | |||
--endpoint https://models.nothanks.com/v1 \ | |||
--model my_model --api-key sk-12345678 \ | |||
--out=path/to/bench-$(date +"%m%d") | |||
</syntaxhighlight> | |||
=== Install === | |||
<syntaxhighlight lang="bash"> | |||
git clone https://github.com/GGZ14/BetterBench.git | |||
uv sync | |||
.venv/bin/activate | |||
</syntaxhighlight> | |||
Latest revision as of 09:17, 30 September 2026
Quick Test: llama-benchy
Use
# concurrency 8, test vLLM throughput
llama-benchy --base-url http://models.nothanks.com/v1 \
--model my_model --api-key sk-12345678 \
--pp 512 --tg 128 --concurrency 8 \
2>/dev/null | grep "^|" -B1 -A3
# concurrency 1
llama-benchy --base-url https://models.nothanks.com/v1 \
--model my_model --api-key sk-12345678 \
--pp 512 --tg 128
Install
pipx install llama-benchy
Full Test: betterbench
Use
betterbench run \
--endpoint https://models.nothanks.com/v1 \
--model my_model --api-key sk-12345678 \
--out=path/to/bench-$(date +"%m%d")
Install
git clone https://github.com/GGZ14/BetterBench.git
uv sync
.venv/bin/activate