local-llm
Everything on Ground Truth tagged “local-llm” — 2 items.
Mesh LLM lets you run models too big for any single machine by splitting them across peers News
Mesh LLM, the top project on Hacker News this week, runs models larger than any one machine can hold by partitioning them across networked peers -- layers 0-15 on one node, 16-31 on another -- over a serverless peer-to-peer transport, exposing a standard OpenAI-compatible API on localhost.
Mesh LLM Tool
Runs models too big for one machine by splitting them across networked peers over serverless peer-to-peer transport; ~18 MB install, 40+ models up to 235B, OpenAI-compatible API on localhost.