cogito/calculet-npu-research-archive
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
calculet-npu-research-archive/history/llama.cpp/topics/08-server-api-scheduling.tsv
T

165 KiB
Raw Permalink Blame History

1f131bf1908b8c5dd1f49d7ae7f616506fc47166633d96dae28e17208ef61a71dba40cda3c04f135a2026-05-27T09:38:51+08:00wangbomengfix patch of qwen2.5vl,n_patch of minicpm = 64,fix n_ctx_slot,comment out M-ROPE
2ebce9e1a548c2329aa97f53b8b2ad50d968088dce850ff4de25ad1221abc9bf3bb30df252ccd9c362026-05-27T09:38:00+08:00Yunzhe Jiaorigin/runtime_replaceRefactor cal-llm integration by removing runtime capacity loading and adjusting request handling
338ca6895815671a87ceec86b4aa4eb976580ce76ddd60c928224b52eb4e1e60e01e8a1ecbb80ee452026-05-20T16:37:13+08:00Yunzhe JiaAdd support for cal-llm profile dumping to JSONL file
42716130010c58f1cdd9d94f0e2f8e047bc92dc66b35bda124ccd4ed581dd6088d3ffa4931c2809b62026-05-18T08:40:54+08:00Yunzhe Jiafix n_ctx_slot error by adding runtime capacity loading for cal-llm
53b14ed556c15529fe1f94b649f49db7156dfaf67dde8b8228081769c8d8f1e0751d47fa7875f84232026-05-14T14:34:25+08:00Yunzhe Jia- Removed references to CALRT from clip.h, mtmd-helper.cpp, mtmd-helper.h, mtmd.cpp, and mtmd.h. - Updated server.cpp to conditionally include CAL-LLM support. - Introduced option LLAMA_SERVER_USE_CAL_LLM in CMakeLists.txt to enable CAL-LLM integration. - Refactored server context to handle CAL-LLM requests and responses. - Updated image processing logic to remove CALRT-specific implementations. - Ensured compatibility with existing llama functionality while integrating new CAL-LLM features.
699dd308adb8488fa869bb669e3335e646a938eff06c7852e99e34c71e24e3ef6001cb54c72970e4d2026-04-21T15:50:01+08:00wangbomengtag: v0.2.1server: fix max_seq_len
7ed62eb33447f993d578e156a3c7e38c91b420ece015e988a271573a7c11e2a4b45084e1219d69d502026-04-21T15:05:22+08:00wangbomengtry server-test:not success
8d8b2aaa00049978cebac15041f960d75b552f97bb6f29ec8140028619efaa50aa6e73146cecf631d2026-04-13T11:23:44+08:00wangbomengserver: fix context-shift
9b6f29ec8140028619efaa50aa6e73146cecf631d6d55eae7e76b768acfcec045b8e84cded8ae3b552026-04-13T11:23:31+08:00wangbomengserver: fix context-shift
10a2d5bf362d6c9a546f9deadf4f8975605a915d33bd3a57a11aa008e0b5dc2a30f82fcd3a025872082026-04-09T21:01:46+08:00Bomeng Wangserver: comment fixed time
11b8153b6b8f244b223c9f4c2940ae1f212634519e82842cfc3fa3d5c004c4540aeff8a81f38509bee2026-04-09T15:07:02+08:00wangbomengserver: fix limit tokens to max_seq_len
1282842cfc3fa3d5c004c4540aeff8a81f38509bee622a22991089c6debd5f4b57738f3df3a0bda8b52026-04-09T15:06:26+08:00wangbomengserver: fix limit tokens to max_seq_len
13622a22991089c6debd5f4b57738f3df3a0bda8b55f47a738ba56a37d0ff281a16212f41979568e352026-04-09T10:36:31+08:00wangbomengserver: comment circle print
14ecf01a17651cd7b1e8b85fd6075e83831463ee2399d2078e89305035d87d966cee7987c7d50b39252026-04-09T09:47:55+08:00Yunzhe Jiafix n_parallel problem
1518814f9bf1a423ebe8f32d6043c9cbc5b21b0218bbee06d6d851e3f84f75b578047967d7e0e4a8dc2026-04-08T13:55:52+08:00wangbomengserver : reset inference time
16398ecf71f5f574037ce33f84dd1576fa164909ad6d5e83902bdd3142a8ff0d3db19f02a8619e95a02026-03-27T02:01:23+08:00wangbomengrm llama-server_old.cpp
176d5e83902bdd3142a8ff0d3db19f02a8619e95a0e4eaab700333490e80ebefc8d023f6c1adfcc3122026-03-27T00:57:11+08:00wangbomengserver: printf calrt commit ID
1841e6636ea1fb3884e41fb2023f92b0e7262ef09bd0a1b2c758440720d42fa108b3444f54593daa732026-03-12T16:36:07+08:00Bomeng Wangtag: v0.0.1server: correct the max_seq_len judgment
19583c387efaf7bc030574e554088de79d09a27fbd0edc14a60a2d5596f71d728c655e3523da924d892026-02-27T02:24:56+08:00wangbomengcaserver: lim generated tokens to n_ctx_per_seq
200edc14a60a2d5596f71d728c655e3523da924d89aa0a17d719326a63e0d6fea60aa0ced44e7abc332026-02-27T01:57:39+08:00wangbomengcaserver: limit generated tokens to n_ctx_per_seq
21d81d7b190ff5f21325167936496dfe5ab48b922ee492f5dc8cec17529f53ef2bbdf7b502107f8148 1f5accb8d0056e6099cd5b772b1cb787dd590a132025-11-04T14:43:32+08:00Yunzhe JiaMerge branch 'master' into dev
2248bd26501b08a3f0bff1249db47f313641f7bebb622cd010ff4a65bef67edbf2f9bf4707c01f98f72025-11-03T15:38:23+02:00Georgi Gerganovserver : add props.model_alias (#16943)
23cd5e3b57541ecc52421130742f4d89acbcf77cd487c9efc3b297b8a498716b1db3d061842e6fc85b2025-11-02T18:14:04+02:00Georgi Gerganovserver : support unified cache across slots (#16736)
242f68ce7cfd20e9e7098514bf730e5389b7bba908e4a71599e5846110159955dec0008eb4aa24222b2025-11-01T19:49:51+01:00Pascalwebui: auto-refresh /props on inference start to resync model metadata (#16784)
25e4a71599e5846110159955dec0008eb4aa24222bdd5e8cab512c7752392b6e51a6f118f348fb3f162025-11-01T17:14:54+01:00Pascalwebui: add HTML/JS preview support to MarkdownContent with sandboxed iframe (#16757)
26c22473b580807929fd9e3a3344a48e8cfbe6c88f0f715b4e759acceccb9f437cfd2a988fff85514a2025-10-31T10:54:19+02:00Georgi Gerganovserver : don't print user inputs to console (#16871)
270f715b4e759acceccb9f437cfd2a988fff85514ad2d931f173b8a736b08999436e9259aafddec7182025-10-31T09:51:26+01:00Daniel Beveniusserver : fix typos in server.cpp comments [no ci] (#16883)
2816724b5b6836a2d4b8936a5824d2ff27c52b4517b52edd25586fabb70f0c21b274473b307cf144992025-10-30T14:22:23-04:00chansikparkserver : bump request URI max length to 32768 (#16862)
29b52edd25586fabb70f0c21b274473b307cf14499517b7170e1a4d733583c4b07c5b7a49acc05911c2025-10-30T18:42:57+02:00Georgi Gerganovserver : remove n_past (#16818)
308b11deea4663f29d3e042ce1056ba643264cd5f1b9ce94017729465895402cbcfffb51fa926c15e32025-10-30T04:34:15+01:00Oliver SimonsHide latency of bias and gate-loading (#16847)
313eb2be1ca5f37480aeb16102970d9e65f43347fee41bcce8f0b53032a1fed275cd253e931c041cf62025-10-29T06:29:12-07:00Max KrasnyanskyHexagon Op queue & dispatch optimizations (#16820)
32338074c383c81366320d176d83b94b0a567ee0c2851553ea6b24cb39fd5fd188b437d777cb411de82025-10-29T08:14:39+02:00YaelLogicsycl: add RMS_NORM_BACK operation support (#16808)
3385a7d8677bf2200981e52f744a21d5267964ffcfa8ca18b4b815a2abdbecb958ee5f4c542d69aac72025-10-28T20:19:44+02:00Georgi Gerganovmemory : remove KV cache size padding (#16812)
34ad8d36beffd791db10c94eb9e964afb891e3ca55c053e18a66dd95dc340aa61317877c2a41d4e3cf2025-10-28T03:50:33+02:00tamarPalsycl: add SSM_CONV operation support (#16800)
355a4ff43e7dd049e35942bc3d12361dab2f15554410640e31aab0819f31c1e1f2d008b019ee7372322025-10-27T13:51:28-07:00Diego Devesallama : disable pipeline parallelism if compute buffer allocation fails (#16748)
360bf47a1dbba4d36f2aff4e8c34b06210ba34e688dd62dcfab97e420949519fd0eac9fca7bf97e6352025-10-23T21:30:17+02:00Johannes Gäßlerserver: add memory breakdown print (#16740)
378cf6b42d467d05fa7d9776d2bcc69974ecce69009de9672adb0f4ca4e39483ac3ffed52b3f70a55d2025-10-23T11:32:24+02:00matteoserver : send partial stop string when <EOG> is reached (#15007)
3863d2fc46e17a06be5b4b5823a5ada088317f1f0aa2e0088d9242bd9e57f8b852b05a6e47843b5a452025-10-22T13:47:09-07:00Max KrasnyanskyAdd experimental ggml-hexagon backend for the Hexagon NPU (#16547)
399b9201f65a22c02cee8e300f58f480a58859122719a5a3edfd306516cc419679d69d6435943b68162025-10-22T16:58:23+02:00Pascalwebui: introduce OpenAI-compatible model selector in JSON payload (#16562)
4013f2cfad4170c096c51a02c24a6a158cb47f148006332e28672356b964d6dfc2ba4657e20581cd432025-10-20T12:41:13+02:00Aleksander GrygierEnable per-conversation loading states to allow having parallel conversations (#16327)
4141386cf365d894134ee0813d15e2f5d76f6a4d8e3d4e86bbeb15f487d6da6174ba6191b7c212cc252025-10-17T18:02:52+03:00Radoslav Gerganovrpc : report actual free memory (#16616)
42466c1911ab736f0b7366127edee99f8ee56874170cb7a0683b0529172472d74d21f05470a607f2972025-10-15T22:24:51+03:00safranowithcpu : add FLOOR, CEIL, ROUND and TRUNC unary operators (#16083)
43f9fb33f2630b4b4ba9081ce9c0c921f8cd8ba4ebf4ce81c45e7bd910e36bf44c253fc5255c49b1e42025-10-15T16:22:20+02:00Aleksander GrygierAdd server-driven parameter defaults and syncing (#16515)
4417304cbcc1dd24de7741cbe57925d58e90a98ac13e3cb19f6449f9168a128eaeae01f8f41b049acc2025-10-15T16:53:12+03:00Georgi Gerganovserver : fix img token logs (#16595)
45554fd578a5ed78a10f371f5850c6a69aa83df15afa882fd2b1bcb663de23af06fdc391489d05b0072025-10-15T12:51:27+03:00Georgi Gerganovserver : fix mtmd checkpoints (#16591)
467ea15bb64c81e3813eb0babf9a57e1bc5697f5699c7185dd28416cf67f5e3b268381f311b5e3da562025-10-14T07:51:36-05:00Jeff Bolzvulkan: Improve build time for MSVC (#16545)
47bc07349a7f87ba6eb31ed4b0ea9d9a7352185213e60f241eacec42d3bd7c9edd37d236ebf35132a82025-10-14T08:48:50+03:00Georgi Gerganovserver : dynamic token limit for prompt cache (#16560)
4881d54bbfd599811b354c39f04550888168be7780c7be9febcbafa9af7d1b9443f86475c59c9c5f872025-10-12T18:06:41+02:00Pascalwebui: remove client-side context pre-check and rely on backend for limits (#16506)
4931d0ff1869aa2ea31f4e96d5877d0343e9a2171b97870e64975b26c5e06a3540a8dc0ff601351e862025-10-11T21:39:04+08:00Yann Folletserver / ranking : add sorting and management of top_n (#16403)
50e60f01d941bc5b7fae62dd57fee4cec76ec0ea6e81086cd6a3ca1252f0dc0f938171648399179c532025-10-10T22:15:05+03:00Georgi Gerganovserver : fix division by zero when reporting stats (#16501)
5168ee98ae181a5c83a5cc6261daeee69a1f588c15cdb6da468cc33323955a523738d2e1675aeb5e9a2025-10-10T17:11:07+03:00Radoslav Gerganovserver : return HTTP 400 if prompt exceeds context length (#16486)
52cdb6da468cc33323955a523738d2e1675aeb5e9a6d69ab3f262eca3f0c6ad0ce075d2c80b9924d0e2025-10-10T13:22:27+03:00Radoslav Gerganovserver : log requests to /v1/completions (#16495)
531faa13a1187051af66b0fd9f0d6effe4c77f0b3e1deee0f8d494981c32597dca8b5f8696d399b0f22025-10-09T22:54:57+02:00Pascalwebui: updated the chat service to only include max_tokens in the req… (#16489)
54d00cbea63c671cd85a57adaa50abf60b3b87d86f8328fd4bae76fc027f8ca0e9a05acd3788dabe3f2025-10-09T18:54:51+03:00Georgi Gerganovserver : host-memory prompt caching (#16391)
5512bbc3fa50b6df03318a4451c9a2210200a0b28d9d0882840e6c3fb62965d03af0e22880ea90e0122025-10-08T22:18:41+02:00Pascalrefactor: centralize CoT parsing in backend for streaming mode (#16394)
56d2ee056e1df0fdf646270fb5621e9f92084b59a7b2c08c9ec4cd59ed88c1ebdd94f109af5ccc978e2025-10-08T17:20:18+09:00issixxserver : fix cancel pending task (#16467)
577fdd16b432d247121fe5fe7b21f8805f85266c8574b8fc17f92ada295a648e3c5eb28f46bca7d8922025-10-08T10:57:29+03:00Georgi Gerganovserver : improve context checkpoint logic (#16440)
58df1b612e29ba97a2e67db339b1e8c7465702b7e84e0388aa8a1f4ef1065701dc9c3947aea0a1b9a52025-10-07T15:57:14+03:00Georgi Gerganovserver : add `/v1/health` endpoint (#16461)
59c61ae20d05bd4fdd8551311325a28453364494260123ff38f53d34752f29239a29d0e40a6dc4110f2025-10-07T09:59:13+03:00Radoslav Gerganovrpc : update documentation (#16441)
60c5fef0fcea3b978a2318b1af170209ecec7c37b4ca71fb9b368e3db96e028f80c4c9df6b6b370edd2025-10-06T03:53:31-04:00Oleksandr Kuvshynovserver: update readme to mention n_past_max metric (#16436)
61898acba6816ad23b6a9491347d30e7570bffadfde29acf74fea996014380d59d31aa504ae89642582025-10-04T12:49:16+03:00Radoslav Gerganovrpc : add support for multiple devices (#16276)
62f6dcda390004b627ef30af378d0c01ad2519289e606a73f53175077429484b23dcf799f69a31d0bd2025-10-03T13:34:51-05:00ddh0server : context checkpointing for hybrid and recurrent models (#16382)
63ad126479c25cf983a0f994a08ba0911cf49ed62b77233277c912d495d1069543ca540c21a2f9e5b42025-10-03T11:45:16+02:00Daniel Beveniusci : change macos-13 to macos-15-intel (#16401)
6477233277c912d495d1069543ca540c21a2f9e5b4e308efda8e54e3f07578937a45a5d83624eb79702025-10-03T11:30:39+02:00Aleksander GrygierCapture model name only after first token (streaming) or completed request (#16405)
65136bda78c5679b266362b39c58338fff46fd25925113efd34ceda709292de26c72716c49e024fb322025-10-03T09:11:34+02:00Aleksander Grygierwebui : Fix messages payload sent to chat completions (#16402)
662be72c2b121ee99f33927149265ce6073ade9e5995ce0985449c52fb9d6849b95563f7cf933ea0e32025-10-02T15:16:25+08:00Neo Zhang JianyuSYCL: Update to oneAPI 2025.2 (#16371)
67aa9538a63aef35f0224b2502f13b19d650a8ce04e74c92e84236b2bab3f3c77bee4ead94928be3602025-10-01T07:40:26+02:00Aleksander Grygierwebui: Remove running `llama-server` within WebUI `dev.sh` script (#16363)
6816b0ca0d2e63fc1a0a43795f8552f9cb61d9f7a58d78cd2613ccdeb3cc86f59bc8f9ddd31cfbd3ed2025-09-30T19:18:54+02:00PascalChatapi ignore empty sampling (#16330)
6998c8269abefdfa8fde47dfb2769ceebc8b704e862790ecc304e384d9075c55d0628d8bf440025dc42025-09-30T17:42:57+08:00Yunzhe Jiaserver: adapt to calrt
702811c65286ae954bec87049f75b86dc022006dccd8359f5fde480da030bf75c7711573c7c4d993ba2025-09-28T12:04:46+01:00Imad SaddikFixed a few typos in the README of the LLaMA.cpp HTTP Server [no ci] (#16297)
71234e2ff8ed09716fb553437596779399bee31b113f81b4e91c1d5f098148af117e3f13cf4b077f522025-09-27T18:17:08+02:00Adrien Gallouëtserver : remove old LLAMA_SERVER_SSL (#16290)
721a189278944d030211f336e103e96b65f976c361e0539eb6aed346d4b25a6ea019044e88771e76902025-09-26T18:35:42+02:00Aleksander GrygierAllow viewing conversations even when llama server is down (#16255)
73d0991da39d3c39b3980c24cdfccb9a4c95a46870aa719c2f886c445b78113bf1e5849f1fce4124a72025-09-25T11:36:47+02:00Daniel Beveniusserver : add support for external server for tests (#16243)
74c498fc82fe5b83fc8c6e1627286bdc1f93caddbfe7a5130a20cf45a3358308356c249734ca9821432025-09-25T10:20:02+03:00Radoslav Gerganovrpc : use ggml logging facilities
754b9f4cb0f89a88de4bdf97727d0457b0c648804c85e72271ba1ce78adf34fd8997803c991e617ca62025-09-23T13:59:34+08:00Aaron Teodevops: add s390x containers (#15915)
765c6106a696af2b00ce01d14ba525979d5a32b199ec65fb52f0cc890e5085a6c5995ab08a156265fb2025-09-22T10:58:02+03:00Georgi Gerganovcontrib : update roles (#16113)
771eeb523c3e0c7ffbd59469f5463dcbdecba3535e5bb4a3edec297e74b0f7bd4ed5d0fdd12e28d8582025-09-21T08:31:55+02:00Giuseppe Scrivanovulkan: optimize UMA buffer operations and fix driver hangs (#16059)
78459c0c2c1a400f960d7b8e8d94d31a8426f80986be79d9fdd95ab8955527c4aaa67b90e8b95167182025-09-20T07:56:30+02:00Benniserver: fix SSE and OpenAI compatibility for error messages when streaming (#16109)
79be79d9fdd95ab8955527c4aaa67b90e8b9516718f432d8d83e7407073634c5e4fd81a3d23a10827f2025-09-19T15:15:21-07:00ssweensllama-bench: add --devices and --list-devices support (#16039)
80246c0d9c795fb25da8268625d597580155a3672f3edd87cd055a45d885fa914d879d36d33ecfc3e12025-09-18T23:07:18+02:00Adrien Gallouëtcmake : fix static linking for OpenMP on Unix-like systems (#16031)
814ca088b036313c2d8e682f4cfeb7c29edd85d0b9703f9e32c4eb3166f8d63007c26e31a1466c21af2025-09-18T16:22:50+01:00Eric CurtinAdd resumable downloads for llama-server model loading (#15963)
822b6b55a59f590a1e1eb2dcd09d5b8b6feb9cc748e58174cecbc45bf79bf653cd2c984395940c6ef42025-09-18T13:36:57+03:00Radoslav Gerganovserver : include usage statistics only when user request them (#16052)
830320ac5264279d74f8ee91bafa6c90e9ab9bbb91a7a98e0fffed794396b3fbad4dcdbbc1849636452025-09-17T20:38:12+03:00Georgi Gerganovmetal : refactor + optimize v2 (#15995)
848ff206097c2bf3ca1c7aa95f9d6db779fc7bdd6877475530b8bbea3bf578632507e1284cdfe2c8c02025-09-16T16:17:08+02:00jacekpoplawskillama-bench: add --n-cpu-moe support (#15952)
85b907255f4bd169b0dc7dca9553b4c54af517086528c39da7c645185ade5436767929d7ec330060332025-09-15T19:51:35+03:00yael-worksSYCL: Add COUNT_EQUAL operator support (#15991)
866c019cb04e86e2dacfe62ce7666c64e9717dde1f9dcd200d57bc6f05a59a6a8df361d5d183af41242025-09-14T21:17:04+02:00Sigbjørn Skjæretserver : only attempt to enable thinking if using jinja (#15967)
8750f4281a6f5c3a5d68bdeb12f904fa01e0e2ba9155758b00cae1451a0d789c50a5eb8c9bee42b9a02025-09-13T07:49:49-07:00Diego Devesallama : allow using iGPUs with --device (#15951)
884bf5549269d99bac936a65892da2312cc71b2421f4e664f838cc65d89fa845c48c372e43852112e42025-09-12T16:31:50+01:00Eric CurtinAdd docker protocol support for llama-server model loading (#15790)
89f088b6a84f6aed0abb619dbb9d375e726fc888a6304ac5693d1e2124f83e0584bc5eea6311d3d3b42025-09-12T17:02:55+03:00Georgi Gerganovserver : adjust prompt similarity thold + add logs (#15913)
90304ac5693d1e2124f83e0584bc5eea6311d3d3b46c88ad8fa741a2182a2dcf8c5353ae4b5c66e6562025-09-12T13:24:21+02:00Ruben OrtlamVulkan iGPU device selection overhaul and PCI ID API support (#15947)
91e68aa10d8f3d26fdad5b912540362d79de5460e30a16bf52e6874369cb4fd2bdc4863ef984b688e72025-09-08T13:10:07-05:00Jeff Bolzvulkan: sort graph to allow more parallel execution (#15850)
9288021565f08e0b7c4e07ac089a15ec16fae9166c56920f56651908d5cce7a310dabf54ac4f6fbb7f2025-09-08T10:59:48-04:00Jessechat : Deepseek V3.1 reasoning and tool calling support (OpenAI Style) (#15533)
9356920f56651908d5cce7a310dabf54ac4f6fbb7fb0d52998b962bd2681c34bf52af993af79f178b82025-09-08T21:50:05+07:00Xuan-Son Nguyenserver : bring back timings_per_token (#15879)
943b15924d71237a43bb5ad71f5b885ee66a82134279bc429262268ad2ac8a364cfe6c2d6b9c5f008a2025-09-07T10:19:45+02:00Daniel Beveniusggml WebGPU: remove userdata from request adapter callback (#15527)
953c3635d2f20424d557b5b0605a2a356214ffe04861bdfd5298a78593be649a1035ee2a120b13c4f02025-09-06T19:45:24+07:00Xuan-Son Nguyenserver : speed up tests (#15836)
9661bdfd5298a78593be649a1035ee2a120b13c4f001806e77714ae8a78130d432945b959a0956c56f2025-09-06T18:35:04+07:00Xuan-Son Nguyenserver : implement prompt processing progress report in stream mode (#15827)
97fd621880f3fd908424e675a41715a2dc760247a24281c7b315f8a904549fe527039b976e08098d1a2025-09-05T17:32:39-06:00Gabe GoodhartaLoRA Support (#15327)
985fac79cbc77b6d12c9feb5f34fc63586b35fd561408ff524b40baf4f51a81d42a9828200dd4fcb6b2025-09-05T14:31:24-06:00Gabe GoodhartThinking model disabled assistant prefill (#15404)
99dcbe998a6d8f13b6c9cb36bbe7b355ced2ac384287932ec016f0ec81e98a13ce5212851f8c12458a2025-09-05T14:34:07+08:00Yunzhe Jiaadd calrt API used in server side
1005d6688de08e73acc2532d668380801ed79d704eb4fd1242bef6cb2325b4ff1c1a80f3b54b64508a62025-09-05T04:36:23+02:00Daniel Beveniusmodel-conversion : add --embeddings flag to modelcard.template [no ci] (#15801)
101c1c354e44c06d259679bb5bb7a8fa9f0b28480e4a68d9144262f1d0ef4f6ba7ad4a7e73e977ba78c2025-09-04T20:20:14+08:00Chenguang LiCANN: Refactor ND to NZ workspace to be per-device (#15763)
102a68d9144262f1d0ef4f6ba7ad4a7e73e977ba78cbadb80cadbc40e047b30c43611aba575fc8d68452025-09-04T11:50:23+02:00Xuan-Son Nguyenserver: add exceed_context_size_error type (#15780)
1038c3fdf44ecf08335942f2ba558955f55e88c7991f6da8cb86a28f0319b40d9d2a957a26a7d875f8c2025-09-03T09:48:35+02:00Daniel Beveniusmodel-conversion : add missing curl script [no ci] (#15761)
1040d161f021aa33ec0e90cce96f5d1a889255573274efd5a83163ff383285b3a4c2106feabf5c695572025-08-31T20:11:58+03:00Georgi Gerganovserver : enable /slots by default and make it secure (#15630)
105d82f6aa34a216f5df1945cdfe121ba5e6cd80be03d16b29c3bb1ec816ac0e782f20d1690970639192025-08-30T00:12:53+02:00Sergey Alirzaevserver : removed obsolete doc (#15670)
106792b44f2ed9668cce7f267ff0ae4950ed9b4a5de81017865ee444cf49ce0136f2be1e41a0270ff912025-08-29T19:25:40+02:00ExtReMLapinserver : add documentation for `parallel_tool_calls` param (#15647)
107c8d0d14e77c3c45df5cbbddde9c2b4c377ec4e7a84ab83cc0b4b7e769451ee48e4c7d1acef91ef252025-08-28T17:09:05+03:00Georgi Gerganovkv-cache : fix find_slot to not search for continuous slot (#15638)
10884ab83cc0b4b7e769451ee48e4c7d1acef91ef2555042b3692cb1467c9ee15c62c4a9fbf180f89e32025-08-28T15:49:50+02:00Sigbjørn Skjæretmodel : jina-embeddings-v3 support (#13693)
109fbef0fad7a7c765939f6c9e322fa05cd52cf0c15da54f9f1a2db07aaae390024ac466e7867685d942025-08-27T20:58:09+02:00Johannes Gäßlerserver: higher timeout for tests (#15621)
1100115cfb7c1dec0753aecb34f3af4338bf9de8eeb3a617cb18121a85eed17ac5585788810c5d9c1e0 0d5a470223fc90b6b6807921d68011ff06ae7f9e2025-08-26T15:19:56+08:00Yunzhe JiaMerge branch 'master' into dev
1115a6bc6b1a6cb665a944426c2055794950e524bf56b64f74b55628e4193f4fb00313f07dbd85565282025-08-25T14:25:25+02:00Daniel Beveniusmodel-conversion : add model card template for embeddings [no ci] (#15557)
112611f419cff11e4952228162a1c44cb35dff2274ab1afcab804e3281867a5471fbd701e32eb32e5122025-08-23T13:16:17-05:00Jeff Bolzvulkan: optimize rms_norm, and allow the work to spread across multiple SMs (#15281)
1134afb0a746f22abaa545b3ebdb76a400d7da3a713e288693669cf9d0a71e2f2b8bd57305f063402572025-08-22T08:10:14Z65aserver : Support multimodal completion and embeddings prompts in JSON format (#15108)
114cd36b5e5c7fed2a3ac671dd542d579ca40b48b543f196be84b1376945737163e35a91e29e3e24d2f2025-08-21T19:13:45+03:00Georgi Gerganovllama : remove deprecated llama_kv_self API (#15472)
115245be739df942861ddc52331b095b40f18e2a3f1b2caf67db1208fd38a0570785c39f7370d906d8a2025-08-21T11:47:52+02:00Copilotci : add copilot-instructions.md (#15286)
1161b0db8f6e08951969e2447c2d18bf638effb8f7529f538ac630d6544406a0702476e36808a6bd1b32025-08-21T07:19:22+02:00stduhpfserver : fix webui (#15462)
1171bc664a26a1d93a48baf2483a7ff95291ca8bcb813aeb7aef284daed4ad07dc14b4b3e2a42b5ea972025-08-21T07:10:08+09:00teoserver: fix OpenAI API compatibility for usage statistics in chat streams (#15444)
1181a99c2d948209d9ea5eac8b6dc0a29710724454037f10f955f70e0158d50343d0b9a3f92d194daae2025-08-20T18:32:05+08:00xiaobing318cmake : fix target include directories (#15450)
119d2fcd91cf96b46f4485ce46b4e3a32bf0df37715a6d3cfe7fa6ea1fb0e1ba8243b731db22ddc0b492025-08-19T16:46:37+03:00Georgi Gerganovserver : disable context shift by default (#15416)
1209d262f4bad0d37838100133537aaf0a83835ed12f0d3c7405c323784a60f14ddddfbac3f7404d4172025-08-19T08:45:26+03:00Georgi Gerganovserver : remove swa_full warning (#15399)
121d1d82416006e7ff41780cb0e9b5f28d30a267497618575c5825d7d4f170e686e772178d2aae148ae2025-08-18T16:51:42+02:00davidefserver : fix incoming tasks not process in order (#15395)
122e5155e698645242d4f019267ecc40ea9bad81b0921c17b5befc5f6be5992bc87fc1ba99d388561df2025-08-17T18:28:58-04:00Oleksandr Kuvshynovserver : export max observed n_past value (#15361)
123de2192794f4e8e04f2e8167ef2424905145e88fc2e2b22ba6607414a5d619ac6d2f034b5b02214e52025-08-16T04:18:31-05:00Jeff Bolzvulkan: Support mul_mat_id with f32 accumulators (#15337)
124ff27f80a74bbe5303acd511a6781a1de6d619b3cd3248d9b6557c75d59954c594bb53cf517591e912025-08-15T21:11:22+08:00Aaron Teoggml: initial IBM zDNN backend (#14975)
1255ba36f61033d2819be27e2abb70e9a3bd20c0fdeb204a5a234f4cf0b3f449476da639a5510c6d1572025-08-14T16:23:56+02:00uvosHIP: Cleanup hipification header (#15285)
126b204a5a234f4cf0b3f449476da639a5510c6d157646944cfa8961afd914dd6637739b3cda9a72e112025-08-14T09:23:11-05:00Aldehir Rojasgpt-oss: implement harmony parsing (#15181)
127d32e03f4495d3efa1c5126f53b449f1d429c56643973163bff40f7f5161b0f08a0011729e2b0406a2025-08-14T14:59:50+03:00Georgi Gerganovserver : add SWA checkpoints (#15293)
128b3e16665e14032d1276f9d0263acef8321b6f518c24f4e26883a91219af5acfaf1471ca7ef5826832025-08-13T15:43:00+02:00Sigbjørn Skjæretserver : enable -td and -tbd parameters (#15172)
129e885445bc1719d2e289a9b2e11714e0f994e68be648ebcdb739abfb5a9be14f4c5c0fd19ff268ef02025-08-13T05:28:21-05:00Aldehir Rojasserver : filter out harmony thought messages (#15278)
13007aa869a91837d95fcb5612c65a188763ac3864700f35d509e5367bf19ccaf4b4adddc38db4811c62025-08-13T11:30:45+02:00Sigbjørn Skjæretci : add more python requirements to copilot-setup-steps (#15289)
1316028bf74351d35a06bd98498624f8c2f029f7d1abc5182272c373267352bc689e5fca276934bea2d2025-08-13T10:04:46+02:00Oliver SimonsCUDA: Optimize `reduce_rows_f32` kernel, leading up to 25x perf improvement on kernel-level and 10% perf increase for Gemma3n (#15132)
13253d0a1265826f40c5dbc01d06aeab9e14fcbd69b27093afe78912494073eb043fec93a007e49653c2025-08-11T14:48:41+02:00Xuan-Son Nguyenserver : allow specifying reasoning_format in HTTP request (#15238)
133cd3069dfcbeee8e0e96cffb93b0ebb9e595e273a50e81bdf5db563ab57a9a722b08f96fa8a76c9272025-08-11T11:21:19+02:00Daniel Beveniuskv-cache : log (debug) all streams in find_slot (#15176)
1344850b52aedceeb70bb4fe49f2d7cd1df6ee98682cd6983d56d2cce94ecb86bb114ae8379a609073c2025-08-08T23:04:36+02:00Johannes Gäßlerserver-bench: external OAI servers, sqlite (#15179)
13536d3f00e142696f708ab297b9f8f1c825594712d5fd160bbd9d70b94b5b11b0001fd7f477005e4a02025-08-07T05:31:48+02:00Daniel Beveniusrequirements : fix PyTorch uint64 compatibility (#15134)
136c42293848dd9f03eba9a6c1b85a600b398f065411cbf442f50a154aac0bf35d7b8b3436a2cdfd6f52025-08-06T15:46:06+08:00Yunzhe Jiasync with calrt API
137f906275537d14c8fc7c6976d944233771fd6672ca9f7541ec25c4c8547daf5ff48700ad2836e2b7d2025-08-02T10:12:41+02:00Johannes Gäßlerserver: enable token array inputs for OAI API (#15001)
13894933c8c2eeaa9a7983e3f6c08af76bd86724094c1dacaa99b4ead6edbac928cd2da59436573f6b02025-07-31T05:25:23-07:00g2mtserver : implement universal assisted decoding (#12635)
139a9f77a8be348deeb11fb3d54d412bf583003c90d8a4a85627702b569d7d2810f2de06a4321656e9d2025-07-31T14:08:23+02:00Lukas Straubserver : add openai-style logit_bias support (#14946)
14041e78c567e9a8c652e405f4f909deb598deecd31ad4a700117d1746799d2d6599e526e2c3a7938d22025-07-30T18:07:11+02:00Daniel Beveniusserver : add support for `embd_normalize` parameter (#14964)
14100131d6eaf4df029e1ec84de868c2c59575030071e15bfd42c3938506e0da5939bf7f42780965f012025-07-30T15:12:02+03:00Georgi Gerganovtests : update for LLAMA_SET_ROWS=1 (#14961)
142bbd0f917797e9d524680f1b30d34a46eb06d76510a5036bee9cfb946870689db4400e9e0d17844c92025-07-29T10:40:50+02:00Johannes Gäßlerserver-bench: make seed choice configurable (#14929)
143c556418b600ad5792440942079d93e393595688bdb16e2831c0f344f041af3d067db81c42b16eb222025-07-28T18:59:04+03:00Radoslav Gerganovllama-bench : use local GPUs along with RPC servers (#14917)
144f1a4e72de5950ab7136aeadddd675caf30dd6b3f4762ad7316dcdec20016ab5985fb46a27902204d2025-07-27T04:05:34-05:00Jeff Bolzvulkan: skip empty set_rows to avoid invalid API usage (#14860)
145793c0d7f46384001738c337d7afa46b45ae32745ce111d39d666f4a3b6a561ad020f6feb8cc677902025-07-25T10:47:39-06:00Gabe Goodhartmetal: SSM_SCAN performance (#14743)
146c12bbde37258c6d053322d1cedd4a9672109ae583f4fc97f1d745f1d5d3c853949503136d419e6de2025-07-25T01:07:26-07:00Diego Devesasched : fix multiple evaluations of the same graph with pipeline parallelism (#14855)
1473f4fc97f1d745f1d5d3c853949503136d419e6de2df255da3cea108de0ae9b302ffdd31674b6d88d2025-07-25T03:05:37+08:00R0CKSTARmusa: upgrade musa sdk to rc4.2.0 (#14498)
148adef81781a15083f218eae6c488b95cdad78197148b86c4fdb1b1246d9fe17d2d53e3a1c2a0c32452025-07-22T09:24:22+08:00Molly Sophiaserver : allow setting `--reverse-prompt` arg (#14799)
149b4efd77f8ab407836ca73a5176f041650c5b24112be60cbc2707359241c2784f9d2e30d8fc7cdabb2025-07-21T09:24:51+02:00IsaacDynamoserver : add parse_special option to /tokenize endpoint (#14783)
1502adf8d83acdb9b1bf58db6c9729ac9dc6847a58b021cc28bef4dd7d0bf9c91dbbd0803caa6cb15f22025-07-18T17:33:41+03:00Georgi Gerganovparallel : add option for different RNG seeds (#14757)
151086cf81e88fb75287b71ff19c08a206b7bc2e02fd9b691081c04ec5fb0daa9d2b979f915c142963d2025-07-17T09:22:11+02:00Tarek Dakhranllama : fix parallel processing for lfm2 (#14705)
152225e7a1438f4ea85eaa7b5ef3ab3b266ee4d9c06ab140198211385b85eeeb0abd549a4bbe259e10d2025-07-16T16:35:42+03:00Georgi Gerganovllama : add high-throughput mode (#14363)
1536ffd4e9c442e99afac3d138543ebf86d5fc5ee03e4841d24d3485ea4af54f8b65a19ec3123f0ff3c2025-07-16T14:04:12+03:00Georgi Gerganovserver : pre-calculate EOG logit biases (#14721)
154e4841d24d3485ea4af54f8b65a19ec3123f0ff3c538cc77f7f44dfa047dba6a06d90c86dda69cf1d2025-07-16T19:12:22+09:00Shunta Saitollama : fix parallel processing for plamo2 (#14716)
155538cc77f7f44dfa047dba6a06d90c86dda69cf1d5cae76654113160f691f581930b69fc5535e81592025-07-16T12:13:57+03:00Georgi Gerganovserver : fix handling of the ignore_eos flag (#14710)
1565cae76654113160f691f581930b69fc5535e81594b91d6f71f14040979bcdb7b6729b3bca93ec1c12025-07-16T09:33:28+02:00Johannes Gäßlerscripts: synthetic prompt mode for server-bench.py (#14695)
15779e0b68c178656bb0632cb8602d2940b755077f8c81f4192f91a1e209c1eec7a84fe5371ef9175da2025-07-16T12:00:42+08:00Min-Huallama: add LLAMA_API to deprecated llama_kv_self_seq_div (#14708)
158494c5899cb76859f32ddd913534f2685fd684a3d0f4c6ec0f1a9607ba67071f8a02c69b0afc2f91e2025-07-14T13:14:30+02:00Johannes Gäßlerscripts: benchmark for HTTP server throughput (#14668)
1590c1df14b5f8d992805cb22d0b77b44092a18aeabb3ad3a0191994d6c47b2bd389d5c7431526ecd2c2025-07-12T06:21:02-04:00Douglas Hanleyserver : fix pooled embedding output (#14645)
1600aedae00e6fb48680324a5ac5da9cba0e35de6b56bdda13981d6c8189b7dc4f9fb8ecb91c21529f82025-07-10T18:20:13-06:00Gabe Goodhartmodel : Granite Four (#13550)
1614a5686da22057867c23bd4a6be941ddc8c51e58598bab638fb28cf95a5a66dd2d51b40d6c8f6d69a2025-07-09T14:59:57-04:00compiladellama : support Jamba hybrid Transformer-Mamba models (#7531)
16204655063c47af6cbced295c8c7ad369402b1530020b7bf8a32259ad9189a4797fd3e3a859c537b992025-07-09T12:03:49+04:00ibrahim khadraouimodel : add support for Falcon-H1 family (#14534)
16317a1f0d2d407040ee242e18dd79be8bb212cfcef8f22dc0a53338c629c1ef8fa878d8e39bfe627c92025-07-08T11:47:33+03:00Alawode Oluwandabiraserver: Add ability to mount server at prefix (#14544)
164ddef99522d1ba74193b7394e803fab8db5c78bae668168814601d2aef6161ce49ab186bc74177ae72025-07-05T09:17:14+02:00Sigbjørn Skjæretserver : fix assistant prefilling when content is an array (#14360)
165a70c8a0c4b4c1606cd9a0ba889ce61aa886100959067487c4411efb20400103fcccfdd389c80d4282025-07-03T10:53:35+03:00Georgi Gerganovkv-cache : use ggml_set_rows (#14285)
166c839a2da1a0d4275c3a98f70c3a7627487573a46e9b6350e61d592634263a14b3d77ecbf6c1fb0962025-06-30T17:48:24+08:00xiaobing318cmake : Remove redundant include path in CMakeLists.txt (#14452)
167caf5681fcb47dfe9bafee94ef9aa8f669ac986c783790b0e7e09ab17238b16452a33053a71dbdfad2025-06-29T20:02:53+02:00matteoserver : support jinja extra template kwargs (Qwen3 enable_thinking feature), from command line and from client (#13196)
16883790b0e7e09ab17238b16452a33053a71dbdfadf47c1d7106e49062279bcc57fc1077c0db61e2782025-06-29T19:29:57+02:00Renatserver : fix appearance of the chats list context menu for Safari (#14322)
169bd9c981d7226107f18deb8344c3301450311bb8b27208bf657cfe7262791df473927225e48efe4822025-06-29T02:43:36-05:00Jeff Bolzvulkan: Add fusion support for RMS_NORM+MUL (#14366)
1708f52c6f8abb2ed9ad967cc0bfb90acb073011193d2f325130c4ed77a3dcff2c23a926587fb08e2cc2025-06-26T15:10:45+08:00Yunzhe JiaSquashed commit of the following:
1711b809cee225222094a0ff5be8467240487ce4ae4abf241045d09cad70dc797b0fba393ad09ee2cbe2025-06-24T08:59:11ZNigel Boschserver : move no API key doc to /health (#14352)
172901e20bbe571fbde48d13eb188f4e7cdc7562fb60142961a2e67909e33cdf410274b56c08c5dce7a2025-06-24T02:17:58-04:00Bartowskijinja : Add Mistral-Small-3.2-24B-Instruct-2506.jinja (#14349)
1738308f98c7fb778e54bf75538f5234d8bd20915e96369be07359d03723f38c0a4a014ff0f698a07382025-06-20T15:07:21+02:00Nicolò Scipionesycl: add usage of enqueue_functions extension (#14244)
174d67341dc18fc5cc63362880ab2f8f9ecfc7932e7456af35eb70177b8dd5779b6d4c21bb020f9cebd2025-06-19T16:01:03+03:00aa956server : add server parameters for draft model cache type (#13782)
175edc4a29effe716956fdfd2bc5b9cba2a6d8492f8ed3290ab34493fd3d2e0f925f816a401da4f2dfd2025-06-19T00:08:14-05:00Gabe Goodhartmemory : Hybrid recurrent cache (#13979)
17689fea80d298184d1cd93564f48e060d9f541f4b46adc3c3ebc029af058ac950a8e2a825fdf18ecc62025-06-16T22:33:27+03:00Georgi Gerganovserver : fix incorrect usage of llama_get_embeddings() (#14225)
177d3e64b9f490cee41b7b9aa275dae2f6568ae30543ba0d843c6bd3faea5cf5e53dc7f3c82be20bffb2025-06-16T14:14:00+03:00Georgi Gerganovllama : rework embeddings logic (#14208)
178cd355eda7df1898d25d433b4bdaa4b4b479e0bad30e5b01de2a0bcddc7c063c8ef0802703a9584172025-06-15T23:36:22+02:00Eric Curtinserver : When listening on a unix domain socket don't print http:// and port (#14180)
179c311ac664d68d10781a3e7b9f02d9d9520837d80b9912ac570de8945ae9383c9ca8291027bf287dd2025-06-15T10:08:58+03:00Georgi Gerganovcparams : rename LLAMA_MAX_PARALLEL_SEQUENCES to LLAMA_MAX_SEQ (#14188)
1803cb203c89f60483e349f841684173446ed23c28f2e42be42bd6bf1dcc643d6ac4e77419bfe5dd24f2025-06-14T18:25:15+02:00Piotrllama-chat : Do not throw when tool parsing fails (#14012)
181ffad04397399ea1650fda6560c7c7530598048760889eba570126f8a2f5a0e88fde776bbc91cca662025-06-13T11:18:25+03:00Georgi Gerganovserver : fix SWA condition for full context reprocess (#14163)
182c61285e7396c8e526fe7794c19e8d4f1c99bfc5109cf2c7c655c90e53e100f29b830a788bab0653d2025-06-13T08:45:37+01:00Ewan CrawfordSYCL: Bump oneMath commit (#14152)
1837d516443dd7766569110b38e6374649bee6eb1c4f6e1a7aa8787b5c00acba6370cb70a0beff48b1e2025-06-12T11:51:38+03:00Georgi Gerganovserver : re-enable SWA speculative decoding (#14131)
1842baf07727f921d9a4a1b63a2eff941e95d0488ed7ae2932116a4de3141cbc8c488f7f6e4e06d81712025-06-11T06:43:43-04:00Taylorserver : pass default --keep argument (#14120)
1851f7d50b2936023b26eb218e944e62834b80a2ce04c763c8d1b4d4de20bf364ec1837430783cba9842025-06-11T00:19:25-05:00Jeff Bolzvulkan: Track descriptor pools/sets per-context (#14109)
186dad5c44398b78467943ed0a603ea427fe9f6fc6255f6b9fa6563f6ae49113f9abdc980c12348cc1c2025-06-10T18:20:14-04:00compiladekv-cache : avoid modifying recurrent cells when setting inputs (#13834)
1872bb0467043258bdc58dbaefb33786f1731b38937b8e2194efc529378be45ab9b27d6648a5b81458a2025-06-10T02:41:01-04:00Isaac McFadyenrpc : nicer error messages for RPC server crash (#14076)
18887d34b381d5868e75586210ec17b5ef5deddc276b460d16ae858c6624fd37aec316622a4060ca3252025-06-09T12:57:58+03:00Georgi Gerganovserver : fix LRU check (#14079)
189228f34c9ceefa3ea4f4d6933edd858121e8106cb0974ad7a7cd4bca846b15c484ff3be890135a52c2025-06-07T18:58:20+05:30Akarshan BiswasSYCL: Implement few same quantized type copy kernels (#13739)
190745aa5319b9930068aff5e87cf5e9eef7227339b487a5e0401423bba02cd6e97e4d45131bb20b22b2025-06-06T14:11:15+03:00Georgi Gerganovllama : deprecate llama_kv_self_ API (#14030)
1917f37b6cf1e2c1b90bf0d9c8d91904b4b6c5127483a077146a4761fdbd24bdd8eb098f46b8adc4dda2025-06-05T15:29:22+03:00Georgi Gerganovmemory : migrate from llama_kv_cache to more generic llama_memory (#14006)
192363757628848a27a435bbf22ff9476e9aeda5f40ea394d7ab1f8101716d48ce9421c94c71b93a00f2025-06-02T21:34:40+03:00Georgi Gerganovserver : disable speculative decoding for SWA models (#13970)
193c9bbc77931d223ed7e7cbcf1cb057bc02fd0db19bfd322796cd838f906535ff3352624fc463388942025-06-02T10:15:44-07:00Olivier Chafik`server`: update deepseek reasoning format (pass reasoning_content as diffs) (#13933)
194c04621711a893cbd09cff6c927cb005bc6749e360fc16b42e8793364918e830c234c1f3caec29dae2025-06-01T11:42:16+03:00Georgi Gerganovparallel : fix n_junk == 0 (#13952)
195e15898d1c7e87f1b3d14e0a3c385eeb424b085fe803f8baf4f741d2f0465c46c33f285886b97a0712025-05-31T08:26:10-07:00Olivier Chafikserver: allow unclosed thinking tags (#13931)
19612d0188c0dc6146ffde6d277a93f232ccbe699f8eb3949938e82a128855bc0676220bb2ce6e4228d2025-05-31T10:24:04+03:00Georgi Gerganovkv-cache : refactor + add llama_memory_state_i (#13746)
197dd665cc9d4b378626cde11ea0c4c91f514fdc994df0c0c7d02f951dc639d48fd536f79200460ac832025-05-30T19:38:07+03:00Georgi Gerganovparallel : increase the variability of the prompt lengths (#13927)
19853f925074de02c5304b00c14b4d6d8910c58667ddb38704f0133be7832123495fa8fc2601ea999d42025-05-30T16:25:45+03:00Georgi Gerganovsync : vendor (#13901)
19910961339b26bd2eff01d5479e8879f435da261b7d98f2a35fcf4a8d3e660ad48cd19e2a1f3d5b2ef2025-05-28T22:35:22+02:00Xuan-Son Nguyenmtmd : move helpers to dedicated library (⚠️ breaking change) (#13866)
200c962ae3382a1e759c8517a229549ee53685313a1a3938fb53d0b5183db4f5a6db21fd9122a0e47802025-05-28T22:33:54+08:00Skyserver: fix remove 'image_url'/'input_audio' json-object effectlly for 'llama_params' in multimodal-model-mode (#13853)
201952f3953c1b61cc70e79e536c42ddce6a5ea5ea781713121ee56755ba4a147d1a1e3fa248ec31a662025-05-27T04:05:18-07:00Diego Devesaggml : allow CUDA graphs when using pipeline parallelism (#13814)
202cdf94a18023c92f41808ec874ba577d914674717a26c4cc11ec7c6574e3691e90ecdbd67deeea35b2025-05-26T14:34:27-07:00Olivier Chafikserver: --offline mode (#13804)
20303f582ae8fccecff225c30a2802461b44761e82288c125f2acce0e25e5fc8481ab0681415fc64a102025-05-26T08:03:57-07:00Olivier Chafikserver: fix streaming crashes (#13786)
204d74e94c1b3ac02db01e47008e1f8d371029d81acf13847cfb560d92492b480cca2c5d6aa9473cde32025-05-26T06:56:49-07:00Olivier Chafik`server`: fix format of streamed tool call deltas (diff name, fix id location) (#13800)
205f13847cfb560d92492b480cca2c5d6aa9473cde379c137f77677b3c8ee3c60a7da033721b938399a2025-05-26T06:16:37-07:00Olivier Chafikserver: fix regression on streamed non-chat completion w/ stops (#13785)
206e121edc4324a640be11b7e567edd39b721b0f8e42f099b510f460374acd52742b494595e3e3442d32025-05-26T00:30:51+01:00Olivier Chafik`server`: add `--reasoning-budget 0` to disable thinking (incl. qwen3 w/ enable_thinking:false) (#13771)
207de2ef53a4b2c0d703749a309d19fe68fd8f1b9acc508256db2de2b032e19c8ed833f4683c827c9a12025-05-25T16:34:36+03:00Georgi Gerganovkv-cache : rework kv_cell (#13706)
208d785f9c1fd1a1929c6d0e2a0b12cae5db867908b4032ca406632212b075e3c61c2a44761283214102025-05-25T10:45:49+01:00Olivier Chafikserver: fix/test add_generation_prompt (#13770)
209f5cd27b71da3ac375a04a41643d14fc779a8057ba2d02d5793fd9af7a7224773456501691b95fd022025-05-25T01:48:08+01:00Olivier Chafik`server`: streaming of tool calls and thoughts when `--jinja` is on (#12379)
2109ecf3e66a38f20e41d221f62f05b17f70d040839faaaff5f947064d13ef8b98659d81a1384c3e57b2025-05-23T11:03:47+02:00Xuan-Son Nguyenserver : support audio input (#13714)
211797990c4bca0dca5be295c63e3fb2800dc0a69c2ab86335760ebb441574eb47f886fa1ee302e21312025-05-22T20:42:48+02:00Xuan-Son Nguyenmtmd : add ultravox audio input (#13623)
212cc74d5be990e37f201591fd868a92e64abdbf9025be24af73d77ea23d399726f1d2a01a70ee863312025-05-22T16:33:39+03:00Georgi Gerganovserver : pad small embedding batches (#13692)
2135fbfe384d4659f81c47a477eb8ee97692c7ffef9c76532e7ba128bb097bf6836bf0f5592e1b56b762025-05-21T19:46:56+03:00Georgi Gerganovserver : improve error reporting (#13680)
214c76532e7ba128bb097bf6836bf0f5592e1b56b762aa777d86d3f7bb80b93e226f1c25e47825f6a832025-05-21T19:40:35+03:00antichristHaterconvert : add qwen2vl support for unsloth merges (#13686)
2150d5c74216170ef97e5e7511563837263f2d1a49642158ae2e8ead667a83f07247321ce85f32ace662025-05-21T15:15:27+02:00Robin Davidssonserver : Add the endpoints /api/tags and /api/chat (#13659)
21642158ae2e8ead667a83f07247321ce85f32ace66797f2ac0625b22edeff03cc30e0f988da6b6b0682025-05-21T16:07:57+03:00Dorin-Andrei Gemanserver : fix first message identification (#13634)
217a4090d1174aed22dde5cacce2a4c27656b987a2fb69f1647f9953cb3773266d2c83a92fd0e7e6d662025-05-20T16:13:16+03:00Georgi Gerganovllama : remove llama_kv_cache_view API + remove deprecated (#13653)
218e298d2fbd082a52c0f6ed02729f94e9bf630cf17f0adb80bf7c2c0d80abb04f4533b5513622d99642025-05-20T08:05:46+03:00Georgi Gerganovkv-cache : add SWA support (#13194)
219f71f40a2847d4c9f57b86cd206e0a27b2bfb6d1cd30cb5a7fa17362c47e94a023276f169916e0d032025-05-19T11:46:09+01:00Alberto Cabrera Pérezci : upgraded oneAPI version in SYCL workflows and dockerfile (#13532)
2206a2bc8bfb7cd502e5ebc72e36c97a6f848c21c2ce3a7cf6c5bf6a0a24217f88607b06e4405a2b5d92025-05-17T17:59:48-04:00Isaac McFadyenserver : added --no-prefill-assistant flag (#13608)
221518329b2d4434d0683c30b741b4f4cf5734b5f992f5a4e1e09567e09be8b84a5f976334de9539d172025-05-17T12:58:55+03:00Georgi Gerganovparallel : add option for non-shared and larger prompts (#13598)
2226aa892ec2aa7fe0c93e87c4b970d83a942fb9454aea9f8b4e73876739bf88fb705502f291294e4692025-05-16T21:50:00+02:00Xuan-Son Nguyenserver : do not return error out of context (with ctx shift disabled) (#13577)
2233cc1f1f1d24472a6558c942b1c78989ff4b0e569c753d7bed0dc2cc2d5c42dfa9806cba91748392e2025-05-15T14:24:50+02:00Xuan-Son Nguyenwebui : handle PDF input (as text or image) + convert pasted long content to file (#13562)
224c753d7bed0dc2cc2d5c42dfa9806cba91748392eb2838049ccf50859cea4c390e81405c2fb01e8202025-05-15T08:40:58+02:00Piotr Wilkin (ilintar)server : proper error handling for missing elements in messages array (OpenAI compatible backend) (#13540)
225aa48e373f256df9608395ee6881e7010840d6202e3a9421b78da5c810d812e6348a405f5acc39f342025-05-15T02:39:51+01:00Olivier Chafik`server`: inject date_string in llama 3.x template + fix date for firefunction v2 (#12802)
226053174436f3b3cbb01077f26ff17809537b832c7360a9c98e13d35f322b4c5b1309aab0cc90ed82b2025-05-14T15:42:10+03:00Georgi Gerganovserver : passthrough the /models endpoint during loading (#13535)
227360a9c98e13d35f322b4c5b1309aab0cc90ed82b09d13d94fb169093095416ac6323551c37b753392025-05-14T13:35:07+02:00Xuan-Son Nguyenserver : fix cache_tokens bug with no cache_prompt (#13533)
228d486dd3e8ecfba0170f8632a1530540de560f4a321ca987fba504d273ea28ebc4d3e5b3736a11c8e2025-05-14T10:07:31+02:00Luca Stefaniwebui: Allow pasting file from clipboard (#13526)
22971bdbdb58757d508557e6d8b387f666cdfb25c5ef0995d28ce3d15095b6845d94ce4465e465758732025-05-13T17:07:21+02:00Xuan-Son Nguyenclip : clip.h become private API (⚠️ breaking change) (#13510)
23091159ee9df84edb72ccf874e353d2414f3a8c60d22cdab343b63edd0906f1c132616746085f819832025-05-12T18:56:42+07:00Anudit Nagarserver : allow content to be null in oaicompat_completion_params_parse (#13477)
2319a390c4829cd3058d26a2e2c09d16e3fd12bf1b109232370fc6426aa5dd9be01a8271b9c28f5af3a2025-05-11T11:08:26-04:00Anthony Umfertools : fix uninitialized llama_batch in server (#13436)
2323b24d26c22b9d92b5aa39930988700c84e524cf443dfd741a5da77982e0a94d1e522a2481dfb914d2025-05-10T18:44:49+02:00Xuan-Son Nguyenserver : update docs (#13432)
23333eff4024084d1f0c8441b79f7208a52fad7985817512a94d636c4b6c1332370acb3e5af3ca709182025-05-09T19:29:37+02:00Xuan-Son Nguyen server : vision support via libmtmd (#12898)
234b486ba05bf973aae3652b3fd593e0b257d3a41d402115dcd9a6d0c03b527d5bee8c1493ab819ddbd2025-05-09T10:31:07+03:00Radoslav Gerganovrpc : add rpc_msg_set_tensor_hash_req (#13353)
235d9c4accaff30926e8a5fd8a2429e549158a845e015e03282bb432631193464100c2237a3b6bcfe4c2025-05-09T09:06:37+02:00Xuan-Son Nguyenserver : (webui) rename has_multimodal --> modalities (#13393)
236ee01d71e585f4a17ae83ec55d77acfdcf0bfa7988c83449cb780c201839653812681c3a4cf17feed2025-05-08T18:51:45+02:00Xuan-Son Nguyenserver : (webui) fix a very small misalignment (#13387)
2378c83449cb780c201839653812681c3a4cf17feed1a844be132dbe865358e1de81136920c1d35ac732025-05-08T15:37:29+02:00Xuan-Son Nguyenserver : (webui) revamp the input area, plus many small UI improvements (#13365)
238233461f8121455f957a47e6a22a77b3bc88277b0b34c859146630dff136943abc9852ca173a7c9d62025-05-05T17:12:19-03:00oobaboogasampling : Integrate Top-nσ into main sampling chain (and add it to the server) (#13264)
239b34c859146630dff136943abc9852ca173a7c9d69b61acf06041dcbaff6afa5f28940e93297f85202025-05-05T17:03:31+03:00igardevserver : Webui - change setText command from parent window to also send the message. (#13309)
24027aa2595321c4d9cc4086a8e67bdea204b8309b09fdfcdaeddd1ef57c6d041b89cd8fb7048a0f0282025-05-04T23:43:42+02:00Xuan-Son Nguyenmtmd : add C public API (#13184)
241fab647e8842c5f80da7e8f2c625dab6a0e19e5d4dcf886007de4b8e5200f461a13233315f897fb9d2025-05-02T09:48:31+03:00Georgi Gerganovserver : add cache reuse card link to help (#13230)
242e2e1ddb93a01ce282e304431b37e60b3cddb6114d9d398f84f96d16c308d4976f5e90222ecc2a4922025-04-29T20:33:10+02:00matteoserver : Prefilling assistant message in openai compatible API (#13174)
24343ddab6eeeaab5a04fe5a364af0bafb0e4d350651831f538f720d1d99fba146f24f0a8e970838cc42025-04-28T21:00:20+03:00Ville Vesilehtofix(rpc): Improve input validation and error handling (#13069)
24477d5e9a76a7b4a8a7c5bf9cf6ebef91860123cbad5fe4e81bd447124836ecfb47d794f8768665b9f2025-04-26T22:05:31+08:00SXXggml: move fp16/bf16 conversion optimizations to CPU backend + export conversion APIs (#13107)
245ab47dec3d37aa1927c2ec590e166b76141374ed37b53389c24a507564be39e1ea82746a39749059b2025-04-22T16:16:10+03:00Georgi Gerganovsecurity : add note about RPC and server functionality (#13061)
24635370ba94593c3abc9550328377d461fc4f822b78d6600576318dfc6b091fca744b0fd36a5e5255f2025-04-18T19:58:12+02:00Xuan-Son Nguyenserver : use std::move whenever possible (#12936)
247b9154ecff93ff54dc554411eb844a2a654be49f22db9ba1464f3de0aceb2b5289963e69fc369cb662025-04-18T10:04:51+02:00Xuan-Son Nguyenmtmd : add methods to access `mtmd_image_tokens` (#12906)
2482db9ba1464f3de0aceb2b5289963e69fc369cb662f74c354c0f752ed9aabf7d3a350e6edebd7e7442025-04-18T10:13:42+03:00Radoslav Gerganovrpc : add RPC_CMD_HELLO (#12955)
249c94085df2871d2cad11f6411304d7798d7dd4cdde8a62631b3b05bcf2ad0e0c686881a9f3e3f03ca2025-04-11T23:37:41+03:00Georgi Gerganovserver : add VSCode's Github Copilot Chat support (#12896)
250e8a62631b3b05bcf2ad0e0c686881a9f3e3f03cab6930ebc421e20399663bbd3bc7b7266d5236d062025-04-11T13:04:14-07:00yuri@FreeBSDrpc : Set cache directory in rpc-server.cpp on FreeBSD (#12903)
251578754b3157d662c2fdd51eaa62b6c1f43d3172cb2034c2b55b36b2192bdefb3b295db2a911370f52025-04-11T15:32:14+02:00Ewan Crawfordsycl: Support sycl_ext_oneapi_limited_graph (#12873)
2528b9cc7cdd8a0dcf0176c60c755322c95b596529964eda5deb9859e87a020e56bab5d2f9ca956f1de2025-04-10T22:57:16+02:00Xuan-Son Nguyenllava : introduce libmtmd (#12849)
25364eda5deb9859e87a020e56bab5d2f9ca956f1defe5b78c89670b2f37ecb216306bed3e677b49d9f2025-04-10T17:24:44+02:00Xuan-Son Nguyenconvert : ability to lazy-load safetensors remotely without downloading to disk (#12820)
254a19b5cef16d885c44c635da4a5c97113c1577de878a1ba0a4f2bfed5b8b8e312592143d22e5316982025-04-08T19:54:51+03:00Georgi Gerganovllama : fix FA when KV cache is not used (i.e. embeddings) (#12825)
25578a1ba0a4f2bfed5b8b8e312592143d22e5316982dabf759e7c8c827d38cdd18d0792070e6a4f4e12025-04-08T18:37:06+02:00Xuan-Son Nguyenserver : fix thread.join() on exit (#12831)
2568ca6e1c3a4deb6bb27ee294bfd5706098d94ae88656babd6c21a3b9b3622324cfcc80a2ab78da25b2025-04-08T14:14:59+05:00characharmserver : webui : Improve Chat Input with Auto-Sizing Textarea (#12785)
257b7723942970b70412793f1926a35cfb93d5c157c23106f94ea2bc3da929afb7330655fd5515d08dc2025-04-04T09:09:52-05:00Nauful Shaikhserver : webui : Upgrade daisyui, tailwindcss. (#12735)
258a10b36c91a091f4606710fba4e9327fd71e0e73883a88bd6affbe148a622ac730952ac5b8b5859792025-04-02T14:32:59+03:00Georgi Gerganovllama : refactor kv cache guard (#12695)
2593891e183c61c1e25c2c61afe68d7b95019ea3f89af6ae1efb27a9a7c3f7f7f84639d2243f7303ac12025-03-20T07:02:18+01:00Daniel Beveniusexamples : command.wasm updates (whisper/2904)
2605d01670266859444366e4f333ade5e0e5e2ae63def03229ff423dd1991f4f44ef1352f03334d86eb2025-03-28T01:05:44-07:00Benson Wongserver : include speculative decoding stats when timings_per_token is enabled (#12603)
2612099a9d5dbdcd2841d02dd396a71d0de70bf4490296901983700f3c37449bcb555d85d27150a679d2025-03-27T23:41:04+01:00Piotrserver : Support listening on a unix socket (#12613)
26277f9c6bbe55fccd9ea567794024cb8094394790118b663d8e4ef352a9a15ff15d695fc3258801d602025-03-23T19:30:26+01:00Marius Gerdesserver : Add verbose output to OAI compatible chat endpoint. (#12246)
263517b5ddbf002b91fd6d6daf5d8db8c88a0173039a9b59288e222f39fc0311dc66944ed5a86c815fa2025-03-20T01:22:06+05:30Gaurav GargCUDA: Improve flash decoding kernel GPU occupancy for BS=1 case (#12183)
264810e0af3f50379682dd46b7967c4aadf3f8286f6eba92d64c3f6d86de2e6b4dd3a540d2805a22b0c2025-03-18T12:05:42+02:00Georgi Gerganovserver : fix warmup draft cache type (#12446)
2657dfad387e3f6ac98d383ded2d175eb59736a399360c902926c928f9c2cd6390ce411876f92feeaf32025-03-18T07:27:50+08:00Molly Sophiallama: Add support for RWKV v7 architecture (#12412)
266b1b132efcba216c873715c483809730bb253f4a101e8f2138b2e40902afe2983ecbf503a08d74b1d2025-03-17T23:55:13+05:30Gaurav Gargcuda : enable CUDA Graph on CUDA Toolkit < 12.x (#12394)
2678fcb563613e20a04dd9791f0a9b8a41086428c09add2a3aa5a1571211aa5c7303b8e80c8d1824b912025-03-14T13:47:05+01:00fairydreamingLoad all MoE experts during warmup (#11571)
2682048b5913d51beab82dfe29955f9008130b936c0f08f4b3187b691bb08a8884ed39ebaa94e9567072025-03-13T06:10:05-04:00Ishaan Gandhiserver : fix crash when using verbose output with input tokens that are not in printable range (#12178) (#12338)
2694e39a3c332f84b890e91353ec448502cb8373a6fbe421fc429795d135786f5a0e489709220a9c43a2025-03-10T10:59:03ZOlivier Chafik`server`: extract <think> tags from qwq outputs (#12297)
2701e2f78a00450593e2dfa458796fcdd9987300dfc0fd7ca7a210bd4abc995cd728491043491dbdef72025-03-09T19:08:20+02:00Georgi Gerganovserver : add speculative decoding presets for FIM (#12287)
2717ab364390f92b0b8d83f69821a536b424838f3f87c7f3b7f435f41f2508e0e3010f0013cd83351562025-03-07T20:54:30+02:00Georgi Gerganovserver : infill gen ends on new line (#12254)
2728fad3c7a7c54a25a1ca38dfb08244df55288e6757cf64f6beecf54c6ac71503181f154667fd4228a2025-03-07T11:15:33+01:00Sigbjørn Skjæretserver : Log original chat template parsing error (#12233)
27306a92a193a07afe445929607be9d5e4d033956fba057897ad4e48e3df0354256e0cc80dadcb575952025-03-05T15:25:45+08:00Clauszyserver : fix cache reuse logic (#12161)
2741a24c4621f0280306b0d53a4fa474fc65d3f1b2ebecade5de77674696539163dfbaf5c041a1a8e972025-03-04T06:24:07ZOlivier Chafik`server`: fix deadly typo in response_format.json_schema.schema handling (#12168)
2752679c3b55d1c9c3fed56dea00fea713622e42594c43af9276b119dae7436b7878d59671d0f6b1a972025-03-03T16:17:36+01:00Daniel Beveniusci : set GITHUB_ACTION env var for server tests (#12162)
2769660ffef582ca275092b621c87085a6eea136a90c950a1f69269bff416d74349a82d78181ce6f0f82025-03-03T20:54:08+08:00ag2s20150909ggml : fix kleidiai build (#12159)
27770680c48e5f77d2d3138712a6582bd8c1e548922c43a3e7996e585e2addde1e44057a4f3cdbadef82025-02-28T05:41:47-08:00William Tambelliniggml : upgrade init_tensor API to return a ggml_status (#11854)
278d7cfe1ffe0f435d0048a6058d529daf76e072d9ca82c9e7c23ef6db48cebfa194dc9cebbc4ac35522025-02-25T18:52:56ZOlivier Chafikdocs: add docs/function-calling.md to lighten server/README.md's plight (#12069)
279401af80b546005a06854827b732e3b46979ae028c132239bfbc1aae3a96dfee3350afcb491f64ade2025-02-25T11:52:52Zrhjdvsgsgksserver: handle echo=false on /v1/completions (#12060)
2800b52745649062c04b546aab662cfdb220f4f06214d1051a40ffc2fdff80bc532eea5b9568b85c1562025-02-25T10:40:22ZOlivier Chafikserver: support add_generation_prompt query param (#12062)
2813e9a2860e996657fc10db8393cf65adc4070308258d07a8043a1395177cf77b3e4f388e34182ae642025-02-25T01:29:33-08:00Vitali Lovichllama : expose llama_model_n_head_kv in the API (#11997)
2827a2c913e66353362d7f28d612fd3c9d51a831eda08d5986290cc42d2c52739e046642b8252f97e4b2025-02-24T09:09:51-07:00Alex Brooksllava : Add Granite Vision Support (#11794)
283cf756d6e0a314e0fe25ff944341d79ea8c94cc96d70908421ff22b013e8209f2d12e5c750663c6202025-02-22T12:46:31+02:00Georgi Gerganovserver : disable Nagle's algorithm (#12020)
284c392e5094deaf2d1a7c18683214f007fad3fe42bc5d91a74006c49376590ef2b67420bc7ce2063972025-02-21T03:43:22+09:00momongaserver (webui): Fix Premature Submission During IME Conversion (#11971)
285d07c621393fab6cf32f3add2aa65e4d5331d4a67abd4d0bc4f1a9a0e429bc8ee0d5ece2a394a0a392025-02-19T12:29:52+01:00Daniel Beveniuscommon : add llama.vim preset for Qwen2.5 Coder (#11945)
286b58934c1836b5ea51dbacbe899eee125775e77c963e489c025d61c7ca5ec06c5d10f36e2b76aaa1d2025-02-19T00:01:44+02:00igardevserver : (webui) Enable communication with parent html (if webui is in iframe) (#11940)
28763e489c025d61c7ca5ec06c5d10f36e2b76aaa1d63ac12856303108ee46635e6c9e751f81415ee642025-02-18T18:03:23ZOlivier Chafiktool-call: refactor common chat / tool-call api (+ tests / fixes) (#11900)
28863ac12856303108ee46635e6c9e751f81415ee645137da7b8c3eaa090476a632888ca178ba109f8a2025-02-18T14:21:41+01:00Xuan-Son Nguyenserver : add TEI API format for /rerank endpoint (#11942)
28909aaf4f1f5b69b5173b7fcd24eab96729f6b242a73e2ed3ce3492d3ed70193dd09ae8aa44779651d2025-02-18T17:12:49+08:00xiaobing318docs : Fix duplicated file extension in test command (#11935)
290c4d29baf3236b011730b6ccaae6852e781fb1b912eea03d86a2d132c8245468c26290ce07a27a8e82025-02-17T11:25:12+01:00Antoine Viallonserver : fix divide-by-zero in metrics reporting (#11915)
2910f2bbe656473177538956d22b6842bcaa0449fabfe163d5bf3fb8e99c96ff6d21775fa3d1daee3cf2025-02-16T18:11:22+01:00Xuan-Son Nguyenserver : bump httplib to 0.19.0 (#11908)
292f3552296924e704c11ca936907b9cad7873db8e2fc1b0d0936e4dfc52a81f38e7420c7d23f6caa882025-02-15T10:11:36ZOlivier Chafikserver: fix type promotion typo causing crashes w/ --jinja w/o tools (#11880)
293c1f958c0381e797135b7d42a677542133264193cc48f630d1c1942fce08aa7cb18a53ace443cd6112025-02-13T17:22:44+01:00Reza Rahemtolaserver : (docs) Update wrong tool calling example (#11809)
294c48f630d1c1942fce08aa7cb18a53ace443cd611bd6e55bfd3f3af3e5705cf87a10e5178cef7c3c12025-02-13T14:46:59+01:00Daniel Beveniusllama : add --completion-bash option (#11846)
295c7f460ab882065ec5696e3d51e24dbf67b53928727e8a23300e30cd6ff6107ce262acf832ca605972025-02-13T10:05:16ZOlivier Chafik`server`: fix tool-call of DeepSeek R1 Qwen, return reasoning_content (Command 7RB & DeepSeek R1) unless `--reasoning-format none` (#11607)
296e4376270d971cff7992bdb6c5412a739195b14593e693197724c31d53a9b69018c2f1bd0b93ebab22025-02-13T01:25:34-05:00Oleksandr Kuvshynovllama.cpp: fix warning message (#11839)
297a394039db004c6ee00098250d160b5aa018c2314be3bbd62153820ff6d358c817360927f429105c42025-02-13T01:02:38+01:00Diego Devesaggml-cpu : add chunking support to mul_mat_id (#11666)
29831afcbee0eebbc998ab311aa314e389d60aa21275c4284d57bbc613121e90de5271f1cbc95d558722025-02-12T22:47:11ZWoof Dogserver : (webui) Give copy button back to all message bubbles (#11814)
299a18f481f99962638092d6f1c98b1d34d3e3256deb9ab0a4d0b2ed19effec130921d05fb5c30b68c52025-02-11T14:06:45+01:00Daniel Beveniusserver : use common_token_to_piece instead of common_detokenize (#11740)
300507f9174fe856772b6c7c17e81be442de7ee6d1e19b392d58dc08c366d0b29bd3b9c6991fa4e16622025-02-10T21:23:17+01:00Xuan-Son Nguyenserver : (webui) introduce conversation branching + idb storage (#11792)
3010893e0114e934bdd0eba0ff69d9ef8c59343cbc3d7b31a9d84297b493a61c9a8ee3a458e7ccc64a72025-02-10T18:03:28+01:00Xuan-Son Nguyenserver : correct signal handler (#11795)
30255ac8c7791ff44aeb82bd7fe3ca2041844fc77f1e6e658319952f7ad269dc11275b9edddc721fc6d2025-02-08T21:54:50+01:00Xuan-Son Nguyenserver : (webui) revamp Settings dialog, add Pyodide interpreter (#11759)
303e6e658319952f7ad269dc11275b9edddc721fc6daaa55053076f3d5d7da4d9a877cb77e112dabed42025-02-08T19:09:55ZWoof Dogserver : (webui) increase edit textarea size (#11763)
304aaa55053076f3d5d7da4d9a877cb77e112dabed4bdcf8b6a56f4d49d3420ae5b21cdf9a1160405512025-02-08T18:08:43+02:00Georgi Gerganovserver : minor log updates (#11760)
3050cf867160ca9d492e06c0bc688b337cffd1eb187d2fe216fb2fb7ca8627618c9ea3a2e78863257802025-02-08T10:42:34+01:00Xuan-Son Nguyenserver : (webui) fix numeric settings being saved as string (#11739)
3062d219b389e8c8c40bce547b08c8aa7add60fde1f333820d7491cd31c707a340ff23b984a84e401542025-02-07T08:55:47-05:00Christian Fillionvocab : ignore invalid UTF-8 input in the BPE tokenizer (#11729)
3077ee953a64a40c09438b2064539becdbc577cefd0ec3bc8270bc67b58955748d40a3e558a05b2d8f22025-02-07T04:33:27-05:00Christian Fillionllama : add llama_sampler_init for safe usage of llama_sampler_free (#11727)
308b7552cfcbc7defccd8bdefd0a7b9c47d145ed3d7225bbbfa39930cda38a2e5d1f3e5b382267320092025-02-07T09:15:22+01:00Daniel Beveniuscommon : add default embeddings presets (#11677)
3091d20e53c40c3cc848ba2b95f5bf7c075eeec8b192fb3c32a1634488a5265d1304ab37628eeb5480d2025-02-06T09:29:13-05:00Patrick Pengrpc: fix known RCE in rpc-server (ggml/1103)
3102fb3c32a1634488a5265d1304ab37628eeb5480d9ab42dc722ad19a12af80f38a06474d498f96da32025-02-06T17:32:29+01:00Xuan-Son Nguyenserver : (webui) migrate project to ReactJS with typescript (#11688)
311902368a06b915b860236cfc97ff885b2aceae256c3db0480bb820e120249014b380e1f4badf2a1c12025-02-05T19:52:31-06:00Charles Duffymetal : avoid breaking build when metal API predates TARGET_OS_VISION (#11690)
3123962fc1a7956dd0afacfbce10fd1a3ffd3ad857e1bef571f6a23c36a26dabacba631763f9a893b832025-02-04T18:25:42+01:00Xuan-Son Nguyenserver : add try..catch to places not covered by set_exception_handler (#11620)
313db288b60cb49eab69703f995379b8d75c18bf5b3106045e7bb8db481bb2ebbc60e3b53cb27ada1172025-02-04T15:48:53ZOlivier Chafik`tool-call`: command r7b fix for normal responses (#11608)
314f117d84b48104992ba16961b35a96fa93efbb355534c46b53c23613628d72e93c63ea01ea4d1e2ac2025-02-04T19:15:24+08:00Jhen-Jie Hongswift : fix llama-vocab api usage (#11645)
315b3451785aca16fbf4214eeba7e4612676cdf8ccb1d1e6a90bcf485ad2dee309c31cf19bd802465e52025-02-04T00:10:52+01:00Xuan-Son Nguyenserver : (webui) revert hacky solution from #11626 (#11634)
3161d1e6a90bcf485ad2dee309c31cf19bd802465e55598f475be3e31430fbe17ebb85654ec90dc201e2025-02-03T22:16:27ZWoof Dogserver : (webui) allow typing and submitting during llm response (#11626)
3175598f475be3e31430fbe17ebb85654ec90dc201e8ec05832fa8409c49b3bbd13f957c6ae8486e6182025-02-03T16:45:38+01:00Daniel Beveniusserver : remove CPPHTTPLIB_NO_EXCEPTIONS define (#11622)
318d92cb67e37abc23b1c6f7b0ef27a9889da8537e36eecde3cc8fda44da7794042e3668de4af3c32c62025-02-03T09:42:55Zmashdragonserver : (webui) Fix Shift+Enter handling (#11609)
319ff227703d6d6e1888bdc7af6138514092ffcdb960cec062a638700495673f5494d200b74340538be2025-02-01T23:55:32-08:00Michał Moskalsampling : support for llguidance grammars (#10224)
320a83f528688324a21484a97af1d1be5e1bc8d4c8eb1bcd309fc8ac929cbd4a6207b3a19886bda031f2025-01-31T14:15:25ZOlivier Chafik`tool-call`: fix llama 3.x and functionary 3.2, play nice w/ pydantic_ai package, update readme (#11539)
3214a2b196d03d52da31236390e9f5694a88d43d11d1bd3047a939e561adfb3c7dd2e17c4cc7a4e4e6f2025-01-31T08:12:40ZOlivier Chafikserver : fix --jinja when there's no tools or schema (typo was forcing JSON) (#11531)
322a2df2787b32e0846205f7151dfad88ceab592beb553f1e46e9e864514bbd6bf4009146db66be05412025-01-31T06:04:53+01:00Daniel Beveniusserver : update help metrics processing/deferred (#11512)
3234314e56c4f8c5091f45732f39bd94c0c6c323798496e5bf46bab757d05e18227cb4e17055ae42f422025-01-30T11:05:00+01:00Daniel Beveniusserver : use lambda instead of std::bind (#11507)
324496e5bf46bab757d05e18227cb4e17055ae42f427919256c57f05c09b0b50ec9abb37ff62dab72512025-01-30T04:11:53-05:00Isaac McFadyenserver : (docs) added response format for /apply-template [no ci] (#11503)
325e0449763a4f335cca374254a72892141f41eaa59eb7cf15a808d4d7a71eef89cc6a9b96fe82989dc2025-01-30T05:48:14+01:00Daniel Beveniusserver : update json snippets in README.md [no ci] (#11492)
326eb7cf15a808d4d7a71eef89cc6a9b96fe82989dc66ee4f297cff3c7ce98b31dbc0ce909d41b9e4082025-01-29T12:45:44-06:00Nigel Boschserver : add /apply-template endpoint for additional use cases of Minja functionality (#11489)
327e51c47b401f8cb5f21630a05171e2529cde4d1862711d0215ff6a1ecb2202ac8c1f135bceed7057b2025-01-29T16:34:18+01:00Daniel Beveniusserver : update auto gen files comments [no ci] (#11484)
328cf8cc856d7d02165bd08593b4757e1256a62d501d0c08040b6c8bebeade7b8d5764df6cf901678d52025-01-28T16:03:42-07:00peidaqiserver : Fixed wrong function name in llamacpp server unit test (#11473)
32949b0e3cec4b67dc9f4debe3a16acd4c819f751d620a758155bc5f37290b20ea44d76ba99c4e7f2cb2025-01-25T16:36:44+01:00Xuan Son Nguyenserver : fix cleaning up stream task (#11418)
330c07e87f38bd0c22ec6dbc852ae50aaa1c64632d4564804b79b78df1469ec8646869972de5e885ec42025-01-24T09:02:38+01:00stduhpfserver : (webui) put DeepSeek R1 CoT in a collapsible <details> element (#11364)
33158456616408c828a1ce6d2481940fd1c97bce3b5f211d1dc10332b7e89a4abd1041903fa63bfed272025-01-23T13:56:05+01:00Xuan Son Nguyenserver : add more clean up when cancel_tasks is called (#11340)
33212c2bdf2de34f747d13b270fc9d3b52490bf194fc64d2becb13f2a7c0145b516a05f028d949a046b2025-01-22T17:44:40+01:00Diego Devesaserver : fix draft context not being released (#11354)
3336171c9d25820ccf676b243c172868819d882848fe28245f35f2faaf249dd352998b3693a8cc28c512025-01-21T13:18:51ZOlivier ChafikAdd Jinja template support (#11016)
334f30f099228f774209aa3010b78dfbe5d262e69aaf26c87417999209e7f4576b4f3ecf7a5b9c66a292025-01-18T14:12:05+01:00Xuan Son Nguyenserver : implement cancellable request (#11285)
3350ccd7f3eb2debe477ffe3c44d5353cc388c9418df446c2cf6a56a750b67c967505e717a996d2f2fd2025-01-15T05:44:38+01:00Daniel Beveniusexamples : add embd_to_audio to tts-outetts.py [no ci] (#11235)
336c5bf0d1bd7eb8762edca47e0a95f64e6fbfaf5b1091592d758cb55af7bfadd6c397f61db387aa8f32025-01-14T14:09:33+03:30ebraminioserver : Improve code snippets direction between RTL text (#11221)
337504af20ee4eae72080a56d59d744f6774f7901ce84a44815f704aaed8e8edec7a39e846a975c7ba92025-01-13T22:53:31+03:30ebraminioserver : (UI) Improve messages bubble shape in RTL (#11220)
338437e05f714cdd67757405221578d3f95f8228b63ca001f6656c1c3d29ef479b3aa5d669453e63be52025-01-13T17:16:39+03:30ebraminioserver : (UI) Support for RTL text as models input or output (#11208)
339afa8a9ec9b520137bbd1ca6838cda93ee39baf20c05e8c9934f94fde49bc1bc9dc51eed2826051502025-01-12T11:32:42+02:00Georgi Gerganovllama : add `llama_vocab`, functions -> methods, naming (#11110)
3408eceb888d7b7f5e93d20a4f85ca6511022b87040f8feb4b01af374ad2fce302fd5790529c615710b2025-01-09T11:28:29+01:00Daniel Beveniusserver : add tooltips to settings and themes btn (#11154)
34153ff6b9b9fb25ed0ec0a213e05534fe7c3d0040f017cc5f446863316d05522a87f25ec48713a94922025-01-07T18:01:58+01:00Johannes GäßlerGGUF: C++ refactor, backend support, misc fixes (#11030)
342e6e7c75d94adf4d39e846d30807c531ff22865e709186fabbe05236f2b9446ba6c643cb737540d102025-01-06T15:36:08+02:00Georgi Gerganovserver : fix extra BOS in infill endpoint (#11106)
3436369f867a410416239d9f20ec27c2b1d6a9fee5247182dd03fe04a4ffda5d7f4c8a109ae0056cf562025-01-06T10:28:17+01:00Daniel Beveniusllama : rename missed batch params/vars to ubatch (#10059)
34447182dd03fe04a4ffda5d7f4c8a109ae0056cf563e6e7a6bc2c4b980a0cf0fcb5cb3b79a965b5f142025-01-06T10:55:18+02:00Georgi Gerganovllama : update llama_model API names (#11063)
3452f0ee84b9b02d2a98742308026f060ebdc2423f10da5d860266c6928b8c9408efbd264ae59fedda62025-01-02T18:06:12+01:00Pierrick Hymbertserver: bench: minor fixes (#10765)
3460da5d860266c6928b8c9408efbd264ae59fedda6a45433ba209ee0b33d02c7dc4c31f29894ad83a62025-01-02T15:05:18+01:00Xuan Son Nguyenserver : allow using LoRA adapters per-request (#10994)
34745095a61bfd164e87563a0dc0fbd7b0e9891590b5896c65232c7dc87d78426956b16f63fbf58dcf62024-12-31T15:22:01+01:00Xuan Son Nguyenserver : clean up built-in template detection (#11026)
3485896c65232c7dc87d78426956b16f63fbf58dcf6bc7b1f86324279a3dabb705c04ad754a2b27df162024-12-31T12:34:13+01:00Xuan Son Nguyenserver : add OAI compat for /v1/completions (#10974)
349f865ea149d71ef883e3780fced8a20a1464eccf416cdce7b68218959e0658e2f95b4572573d5008e2024-12-28T10:09:19-05:00Isaac McFadyenserver: added more docs for response_fields field (#10995)
35016cdce7b68218959e0658e2f95b4572573d5008ed79d8f39b4da6deca4aea8bf130c6034c482b3202024-12-28T15:08:54ZAlexey Parfenovserver : fix token duplication when streaming with stop strings (#10997)
35109fe2e76137dde850b13313f720e7ffa17efdefa30caac3a68a54de8396b21e20ba972554c5872302024-12-24T19:39:49+03:00NeverLuckyserver: allow filtering llama server response fields (#10940)
35214b699ecde8f1e9e251ebff9eca39ebc5603b83b485dc01214f266afff7004bc702498b491abc4042024-12-23T12:52:25+01:00Xuan Son Nguyenserver : fix missing model id in /model endpoint (#10957)
353485dc01214f266afff7004bc702498b491abc40486bf31cfe684849157f0875b4f0ebccac70345472024-12-23T12:02:44+01:00Xuan Son Nguyenserver : add system_fingerprint to chat/completion (#10917)
35486bf31cfe684849157f0875b4f0ebccac7034547b92a14a841fb4dfaf27b29d982ec8ba5289a3bff2024-12-23T10:39:30+02:00Radoslav Gerganovrpc-server : add support for the SYCL backend (#10934)
3550ca416c91aea4549d9c77e3efeca403e15aa6c7521ae3b9be83820565d1a720999b7f63ce95b49202024-12-20T14:12:06+01:00Xuan Son Nguyenserver : (UI) fix copy to clipboard function (#10916)
35657bb2c40cd94c5a09f5210ed8264cc93b21c4b7ea3c33b1dce2d4f25040b75f66629104bd1e401282024-12-19T15:40:08+01:00Xuan Son Nguyenserver : fix logprobs, make it OAI-compatible (#10783)
3570bf2d10c5514ff61b99897a4a5054f846e384e1e7bbb5acf125d1d2840cac7d31b9aaa72210dd5ec2024-12-18T19:27:21+02:00Georgi Gerganovtts : add OuteTTS support (#10784)
3587bbb5acf125d1d2840cac7d31b9aaa72210dd5ec152610eda91217ac409342cd976d05f5114ad39f2024-12-18T04:00:07-10:00Gaetan Bissonserver: avoid overwriting Authorization header (#10878)
359152610eda91217ac409342cd976d05f5114ad39f0e70ba686e6c717a0aa41d88284e2a392c2bd0cd2024-12-18T13:01:41+02:00Georgi Gerganovserver : output embeddings for all tokens when pooling = none (#10861)
360d62b532c52e0118323277eaa5f442e11ce6505ed081b29bd2a3d91e7772e3910ce223dd63b8d7d262024-12-17T17:24:22-05:00DAN™Use model->gguf_kv for loading the template instead of using the C API. (#10868)
36105c3a444b8b017b01b366b725ba22c740aae6b38382bc7f2e8ffd0b89f23e840d097e21f301197ba2024-12-17T16:00:24Zkrystianchaserver : fill usage info in embeddings and rerank responses (#10852)
362227d7c5a7f4cc7734fde8a4ef4382d613486d3c87b1ec53f56bb72c49e4c8434157895f94d3709c22024-12-17T09:52:09+01:00Xuan Son Nguyenserver : (UI) fix missing async generator on safari (#10857)
3635478bbcd173e7027af7689493c7421719f5c43dfb5ae1ddff93403300fc79c7ab5ee73b8cfbb34572024-12-15T05:55:54-06:00Vinesh Janarthananserver: (UI) add syntax highlighting and latex math rendering (#10808)
36489d604f2c87af9db657d8a27a1528bc4b7579c29e52aba537a34d51a65cddec6bc6dafc9031edc632024-12-14T22:29:45ZMichelle Tanserver: Fix `has_next_line` in JSON response (#10818)
365a76c56fa1a3d27467eb97468d8c3b2fe1243b61ac27ac678dd393af0da9b8acf10266e760c8a09122024-12-13T12:23:52-08:00lhezIntroducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (#10693)
3665555c0c1f66d766c544c30699190dec0285bcbfc973f328b1e92a6406030442dfd15b29449e897472024-12-11T22:40:40ZCentricStormdocs: update server streaming mode documentation (#9519)
367973f328b1e92a6406030442dfd15b29449e89747235f6e14bf0ed0211c51aeff14139038ae1000aa fb18934a97425c42c8b32a1baaa94f0080eb051d2024-12-11T23:14:46+02:00Georgi GerganovMerge pull request #10788 from ggerganov/gg/gguf-py-0.11.0
368235f6e14bf0ed0211c51aeff14139038ae1000aa1a31d0dc00ba946d448e16ecc915ce5e8355994e2024-12-11T20:52:14+01:00Xuan Son Nguyenserver : (UI) add tok/s, get rid of completion.js (#10786)
3694b4d92b0986e3b627b9a9ef4782973108bf4769143041d2eb32623d5b6ca734313327abe96f731462024-12-11T10:47:43ZCentricStormdocs: fix server documentation formatting (#10776)
370b685daf3867c54e42a9db484d7b92619021d4510dafae66cc242eb766797194d3c85c5e5026256232024-12-10T14:23:17-06:00Jeff Bolzvulkan: request round-to-even for fp16 in im2col/rope_head (#10767)
371750cb3e246f7e544124cf18cb0c5e0c8b7e38738a86ad841f103e471ac0fd7ee8852d1eb5015ce892024-12-10T18:23:24+01:00Andreas KieslingerCUDA: rename macros to avoid conflicts with WinAPI (#10736)
372a86ad841f103e471ac0fd7ee8852d1eb5015ce89a05e2afcc241c1ecd38ec5cb4c579d90cdf3f9182024-12-10T17:22:34ZYügserver : add flag to disable the web-ui (#10762) (#10751)
373ce8784bdb153ff7794dde5a50b0ebfa51baa6171e52522b8694ae73abf12feb18d29168674aa1c1b2024-12-08T23:04:29+01:00Xuan Son Nguyenserver : fix format_infill (#10724)
374e52522b8694ae73abf12feb18d29168674aa1c1b06d70147e6480c021e493c442ae0f0d83ae366de2024-12-08T20:38:51+01:00Xuan Son Nguyenserver : bring back info of final chunk in stream mode (#10722)
3753573fa8e7b7f0865638b52b4e9b4d2006f0558a2d9c3ba2b7749c00df477599aa141a98b4521aa2c2024-12-07T20:21:09+01:00Xuan Son Nguyenserver : (refactor) no more json in server_task input (#10691)
376ce4a7b849388b67d45ad420bdd82d5efcd55647a19d8762ab61df8286367588a80b9c7db4cb568db2024-12-07T18:02:05+02:00Georgi Gerganovserver : various fixes (#10704)
37719d8762ab61df8286367588a80b9c7db4cb568dbc2a16c0bdbe2e51adf318918bad82f0c3e3d6f3b2024-12-07T13:37:50+01:00Djip007ggml : refactor online repacking (#10446)
378c2a16c0bdbe2e51adf318918bad82f0c3e3d6f3b3df784b3050f657ea681f804187ce5bddb433e882024-12-07T11:52:44+02:00Georgi Gerganovserver : fix free of spec context and batch (#10651)
379f162d45a21b27f7613f14539f9a4932d6ff3ca486c5bc0625fae6909cb40def15bc4bb45db6f7f4d2024-12-06T13:29:05+01:00Xuan Son Nguyencommon : bring back --no-warmup to server (#10686)
3806c5bc0625fae6909cb40def15bc4bb45db6f7f4d7736837d62efed1dbebfe579472fca041eda12d62024-12-06T11:14:32+01:00Xuan Son Nguyenserver : (refactoring) do not rely on JSON internally (#10643)
3817736837d62efed1dbebfe579472fca041eda12d6c9c6e01daedac542b174c235872569fce53859822024-12-05T23:36:41+02:00Plamen Minevfix(server) : not show alert when DONE is received (#10674)
3821da7b765692764a8b33b08da61cbee63812a7bd959f4db10883a4f3e855cffbf2c3ab68430e952722024-12-04T22:38:20+02:00Georgi Gerganovserver : fix speculative decoding with context shift (#10641)
38391c36c269bca75b2d08119c653512cd20b4ea2ba1cd3df46bd49a0c1c78da8b68c956448a73b74762024-12-03T19:38:44+01:00Xuan Son Nguyenserver : (web ui) Various improvements, now use vite as bundler (#10599)
384efb6ae963031709fc331e6e48cc4606ac8f9c3a7667d70d1704dfa6977505f5d01d4638669b90dce2024-12-02T19:27:24+01:00PABfeat: add `GGML_UNARY_OP_ARGMAX` Metal kernel (ggml/1019)
3853b4f2e33e2cbfca621e623c4b92b88da57a8c2f482bca2257b3cec72676abb26011f1b99fcdab29d2024-12-03T12:54:30+01:00Xuan Son Nguyenllama : add missing LLAMA_API for llama_chat_builtin_templates (#10636)
38670b98fadbc8c07a0144f3b50a4d7ab7e2aeff878642330ac7cea11dc1f9d3df2b8f3dbd10b5e3f3e2024-12-03T11:20:00+02:00Georgi Gerganovserver : fix default draft model parameters (#10586)
387642330ac7cea11dc1f9d3df2b8f3dbd10b5e3f3e8648c521010620c2daccfa1d26015c668ba2c7172024-12-02T22:10:19+01:00Xuan Son Nguyenllama : add enum for built-in chat templates (#10623)
38886dc11c5bcf34db2749d8bd8d4fa07a542c94f846acce3971098772a8aacb10fe8550b41191105812024-12-01T12:33:12+01:00alek3yserver : bind to any port when specified (#10590)
389b782e5c7d453b3f1fa8dc6c34cde7e2fa946af933a8e9af402f7893423bdab444aa16c5d9a2d429a2024-11-29T21:48:56+01:00Xuan Son Nguyenserver : add more test cases (#10569)
390f095a649ec390e04dfab1b04e646ae8549dafaef678d7994f4da0af3d29046be99950ac999ee97622024-11-29T00:18:02-06:00Jeff Bolzvulkan: get the first command buffer submitted sooner (#10499)
3916c595676899013102fdb0aa4b06a49954300c94a890719311b6535e572f15965c6d7ec4ac2537f602024-11-28T19:17:49+01:00Xuan Son Nguyenserver : (tests) don't use thread for capturing stdout/stderr, bump openai client library (#10568)
39245abe0f74ee281aea6e5283c1e738061256cfcae0bbd2262a3263f37385297b30de37941836e57f72024-11-26T16:20:18+01:00Xuan Son Nguyenserver : replace behave with pytest (#10416)
3937db3846a94ce7683b3e8120abe427457edf840c9c6807b3f28cc3dbfda3ec390bcb87e69fb5634e22024-11-26T13:05:20+01:00Diego Devesaci : publish the docker images created during scheduled runs (#10515)
39484e1c33cde9e0a7aafcda2d4f21ba51c300482d7811872a59daefb25fc0c4326bcb6d8ae893c2f7c2024-11-26T13:36:40+02:00Georgi Gerganovserver : fix parallel speculative decoding (#10513)
3959fd8c2687f5aa2f095ac6e12a376e1c0583888e847f931c8f9a26c072d71224bc8013cc66ea9e4452024-11-25T22:28:27+02:00Georgi Gerganovserver : add more information about error (#10455)
39647f931c8f9a26c072d71224bc8013cc66ea9e445106964e3d266740f571b5aad7b57545b4a901ac92024-11-25T21:50:07+02:00Georgi Gerganovserver : enable cache_prompt by default (#10501)
397a9a678a6b2264e5895533217b0cf8f250534cc589ca2e677626fce759d5d95c407c03677b9c87a262024-11-25T08:11:55-08:00bruceproAdd download chat feature to server chat (#10481)
3989ca2e677626fce759d5d95c407c03677b9c87a265931c1f233c616083d64e41a228249d58e039aa52024-11-25T16:31:38+02:00Georgi Gerganovserver : add speculative decoding support (#10455)
3995a8987793f3e7c1fbfa6806bfcd17d578071b6c9d9d54e498d38ec99bbc0031022f9c92711e97bbc2024-11-25T17:31:10+08:00Neo Zhang Jianyu[SYCL] Fix building Win package for oneAPI 2025.0 update (#10483)
4006dfcfef0787e9902df29f510b63621f60a09a50b599b3e0cd40432cd1975a8906f3db70bbe53b6272024-11-22T17:44:08+08:00蕭澧邦ci: Update oneAPI runtime dll packaging (#10428)
401bcdb7a23862b61aa307fc462fadfe1e2e653d010f245cc28d4eb900efad0bc740145f58d713c6e4f2024-11-16T18:26:54+05:00MaggotHATEserver: (web UI) Add samplers sequence customization (#10255)
4029901068ac78838745e604fffb4601d315a610456231f9360d94446cd083b6b116f63991b1328c4842024-11-15T05:48:49-04:00Xuan Son Nguyenserver : (web UI) add copy button for code block, fix api key (#10242)
4035a54af4d4f588f109f31e456483fdf77096399d91607a5e5b08f4e55f118af3d7de325949d8f18352024-11-15T04:09:12+01:00Romain Biessysycl: Use syclcompat::dp4a (#10267)
404fb4a0ec0833c71cff5a1a367ba375447ce6106eb5ea926dad7f62ebccff7b24784bd1e01a06d13ae2024-11-13T20:00:35+02:00Michael Podvitskiyllama : propagate the results of `graph_compute` (#9525)
405ff7fb670d0a62897c5662536aeb53422c549fbe80e712a5acbbdd1593e5aeb86d4f6b896a11b438c2024-11-13T11:16:30ZAlexey Parfenovserver : add missing docs (#10269)
4060e712a5acbbdd1593e5aeb86d4f6b896a11b438ca0ec17b32ec6077f5ca22fe833ebdc9b86795a4d2024-11-13T19:15:23+08:00Jhen-Jie Hongserver : fix incorrect res in validate_model_chat_template (#10272)
407b141e5f6efbab3a00633df88c4f9425bfe8b78ab4b3a9212b602be3d4e2e3ca26efd796cef13c55e2024-11-11T08:38:43+02:00Georgi Gerganovserver : enable KV cache defrag by default (#10233)
408505f33274d60676320216b662a97672a76ec600e160687b3ed002eee83a04de83a9cd752f928ced12024-11-11T00:42:25+05:00MaggotHATEserver : (web UI) Add back sampler settings (#10239)
40976c6e7f10551960e4ec9e14e0535b72081f1c7ada71d81cf8c1afb26b166f897c94ee1581f9fac7d2024-11-07T18:44:38-04:00Xuan Son Nguyenserver : minor UI fix (#10207)
410a71d81cf8c1afb26b166f897c94ee1581f9fac7deec4d71737b32f312e0082b671629a0368e1a20d2024-11-07T17:31:10-04:00Xuan Son Nguyenserver : revamp chat UI with vuejs and daisyui (#10175)
411b11f9ba9b8ce319f04b88afe40d264e6b7f4ba4694d8cb8be13b7c4d04eeca5a2b956b9148e6f2222024-11-06T13:29:01+02:00Georgi Gerganovserver : remove hack for extra parallel slot (#10187)
4129e0ecfb697d297355e43c20559d29bcc71beb0c36a066b9978533e2ab9890b7f4f8c0262d91798b32024-11-04T16:33:29+01:00Xuan Son Nguyenserver : clarify /slots endpoint, add is_processing (#10162)
41342cadc74bda60afafb45b71b1a39d150ede0ed4d45950415ed985830c59bf42cf9c9216b20cf08ef2024-11-02T16:34:56Zsasha0552server : fix slot selection by lru (#10126)
41445950415ed985830c59bf42cf9c9216b20cf08ef1926d6e39d6f6358bc1a4c52316a560178be72332024-11-02T18:34:00+02:00Georgi Gerganovserver : fix endpoint checks (#10135)
415d865d1478cd4e403f82d793c2afcd0f943412f051804adb0cfee4811eaf633741503d683a46e4c772024-11-01T13:33:14Zsasha0552server : fix smart selection of available slot (#10120)
416e597e50794f07ec8dc24b9efb18f94ec6386fda085679d37f34f66783cc04664a06c405b28e8e0352024-11-01T11:09:59+08:00Zhenwei Jinbuild: fix build error in Windows env with OneAPI setup (#10107)
4170a683e8088d849626e7471f9e2ed381f7dbdf2e9dea5e86051aadcdf42f7db7a8855a78d8f5ff3d62024-10-31T06:02:35-07:00Kevin Gibbonsserver : include scheme when printing URL (#10106)
4188125e6cbfcf2b3b9066e4d923aca9295526730f58841ce3f439de6e770f70319b7e08b6613197ea72024-10-28T08:49:32+02:00Georgi Gerganovserver : don't overfill the batch during infill (#10018)
419bc5ba007b2c83ac95875e68724dabfc12159fc61958367bf530d943a902afa1ce1c342476098576b2024-10-25T10:13:46+03:00Georgi Gerganovserver : check that the prompt fits in the slot's context (#10030)
420958367bf530d943a902afa1ce1c342476098576b40f2555797f97314de749873cdc29dc102be66e22024-10-24T21:51:22+02:00Xuan Son Nguyenserver : refactor slot input data, move tokenizer to HTTP thread (#10023)
4210a1c750c80147687df267114c81956757cc14382190a37d7977eb5bd6a729299bd1e371208c871492024-10-23T13:27:51-06:00wwoodsTMserver : samplers accept the prompt correctly (#10019)
4226b8447352df3d662b56280c8fc38d7f092885787674804a99617b4f90292b4080ecab450ea3d30ba2024-10-22T16:16:01+08:00leo-pony[CANN] Adapt to dynamically loadable backends mechanism (#9970)
423afd9909a6481402844aecefa8a8908afdd7f52f187421a23e8c60e00a7b227d501e8aab2a1aff7ce2024-10-18T14:33:58+03:00Radoslav Gerganovrpc : backend refactoring (#9912)
42487421a23e8c60e00a7b227d501e8aab2a1aff7ce60ce97c9d809f4b040e90b597468b839df5728d02024-10-18T06:46:16+01:00Ouadie EL FAROUKI[SYCL] Add SYCL Backend registry, device and Event Interfaces (#9705)
42560ce97c9d809f4b040e90b597468b839df5728d08901755ba328643c9ab071c20e1939ea52951a0e2024-10-18T13:34:36+08:00Ma Mingfeiadd amx kernel for gemm (#8998)
4268901755ba328643c9ab071c20e1939ea52951a0e6f55bccbb8835d42147add4ee48807450f5ff5352024-10-18T07:32:19+03:00Georgi Gerganovserver : add n_indent parameter for line indentation requirement (#9929)
4271f66b699c48cb5ab3265ed72c48e8549b16742910e41b300ed28f7fe185d938b2e3d56a0bf7411ed2024-10-16T08:35:53ZAlexey Parfenovserver : fix the disappearance of the end of the text (#9867)
428223c25a72fcc3f65cdfd7f5d57edd5b44b550e18fbc98b748e7b075e327bcf13237057f6476780492024-10-15T16:28:55+03:00Georgi Gerganovserver : improve infill context reuse (#9894)
429fbc98b748e7b075e327bcf13237057f647678049dcdd535302fc9702a4709be25f56540d65163a442024-10-15T15:54:55+05:00MaggotHATEsampling : add XTC sampler (#9742)
430dcdd535302fc9702a4709be25f56540d65163a444c42f93b22146c83b763d8cbee5fafc5127466492024-10-15T12:48:44+03:00Georgi Gerganovserver : update preact (#9895)
431d4c19c0f5cdb1e512573e8c86c79e8d0238c73c4c7181bd294757dd80a7904e3dd0fea2d0be914e72024-10-13T21:31:35+03:00Georgi Gerganovserver : accept extra_context for the infill endpoint (#9874)
432c7181bd294757dd80a7904e3dd0fea2d0be914e792be9f12164f18ce845a5bab60cefa5f7fec68362024-10-13T18:52:48+03:00Georgi Gerganovserver : reuse cached context chunks (#9866)
433edc265661cd707327297b6ec4d83423c43cb50a51bde94dd024b632f98428f4bf2ce4832951307792024-10-12T16:14:27+03:00Georgi Gerganovserver : add option to time limit the generation phase (#9865)
4341bde94dd024b632f98428f4bf2ce48329513077995c76e8e92ecc93f784b185eafae36a0e7ad2fa32024-10-12T16:06:31+03:00Georgi Gerganovserver : remove self-extend features (#9860)
43595c76e8e92ecc93f784b185eafae36a0e7ad2fa311ac9800aff532715a5bc7991062c68ba3472e6e2024-10-12T14:51:54+03:00Georgi Gerganovserver : remove legacy system_prompt feature (#9857)
43611ac9800aff532715a5bc7991062c68ba3472e6e943d20b4111c746bcd9dbc7e4771de313b08b50c2024-10-12T08:21:51+03:00Georgi Gerganovllama : improve infill support and special token detection (#9798)
4370e9f760eb12546704ef8fa72577bc1a3ffe1bc04cf8e0a3bb9c0e93e371773b282054cdbbb2310382024-10-10T20:14:55+02:00Diego Devesarpc : add backend registry / device interfaces (#9812)
438458367a90606448a9c0262b276947c9e536086e0fa42aa6d8902cc4eaf31866b3b3b7b61b69da9302024-10-08T13:27:04+02:00Xuan Son Nguyenserver : better security control for public deployments (#9776)
4397254cdf7e8d6312840509ca8d766358c687c6266cad341d88924788b940997c55e7bef000c99e7842024-09-29T23:18:02+02:00Johannes Gäßlerggml: fix gradient allocation logic (ggml/966)
440f4d2b8846a6b34419ff9e9491aee6cd95e444bfc1b2f992cd2cff0b69e5abe78bb8888d51ed19d672024-09-28T17:42:03+03:00Georgi Gerganovllama : add reranking support (#9510)
4411b2f992cd2cff0b69e5abe78bb8888d51ed19d67739842703e32cd43443c45e0b4f6647cc4e6b3d62024-09-28T14:32:46+02:00slarentest-backend-ops : use flops for some performance tests (#9657)
44295bc82fbc0df6d48cf66c857a4dda3d044f45ca27691654c68aa5316108f881136c59f815ccb68092024-09-26T17:38:31+08:00Neo Zhang Jianyu[SYCL] add missed dll file in package (#9577)
443afbbfaa537a96f562c34df4542930fa951b40d9e3d6bf6919f7b10726421779cd344f2da05421c682024-09-25T14:05:13+02:00Xuan Son Nguyenserver : add more env vars, improve gen-docs (#9635)
4443d6bf6919f7b10726421779cd344f2da05421c68904837e0cb2f5f01bf5d5901b7aa57a026860ae42024-09-25T01:06:52-06:00Gabe Goodhartllama : add IBM Granite MoE architecture (#9438)
4450aa15011e315659640504731d1d05663837130fab0f27361f3539a81d983a8b045f3c61e682d9fc02024-09-23T23:04:39-07:00StrangeBytesDevserver : add newline after chat example (#9616)
4460b3bf966f47bf2ba88e5d4e3ed429602008c7e63f0c7b5edf82aa200656fd88c11ae3a805d7130bf2024-09-23T22:23:54+02:00Xuan Son Nguyenserver : add --no-context-shift option (#9607)
447f0c7b5edf82aa200656fd88c11ae3a805d7130bf1d48e98e4f3316bd2f6b187d288c7b6cb88d5cb32024-09-23T11:42:43-07:00Max Krasnyanskythreads: improve ggml_barrier scaling with large number of threads (#9598)
4486026da52d6942b253df835070619775d849d0258eca0fab44eb449ed34e17a9b651ae7398b4941172024-09-19T12:44:53+03:00Georgi Gerganovserver : clean-up completed tasks from waiting list (#9531)
4498a308354f6520df6bea851b435bd8054ee5617b4f799155ab8279cfa668086ce584aa52ecc05f5e52024-09-18T01:50:34-05:00Vinesh Janarthananserver : match OAI structured output response (#9527)
450f799155ab8279cfa668086ce584aa52ecc05f5e5faf67b3de4688f47c3b1019c89df255df2fd59b42024-09-18T14:28:20+08:00Eric Zhangserver : fix OpenSSL build (remove obsolete `LOG_INFO`) (#9529)
4518b836ae731bbb2c5640bc47df5b0a78ffcb129cb8344ef58f8cdb8ebd6faf4463ca32ae91c374c812024-09-17T09:35:38-04:00Bert Wagnerarg : add env variable for parallel (#9513)
4520226613853133c081b55bb892a41bb5eacc0bc94503147a9f9d195d6a14e7c998df23b6eb61f2bae2024-09-17T01:19:46-07:00Max Krasnyanskythreadpool : skip polling for unused threads (#9461)
453441b72b91f818fe69497e5816f87969e90c73c43c4965a64f72ac9434c21cf0e1c3421d13e8891552024-09-16T01:20:01-05:00Vinesh Janarthananmain : option to disable context shift (#9484)
454dcdcee3a744f39714503ee2b19c49b7c7b6209c91f4111e540bacec8d00ca9fd96417bf4c13393942024-09-14T17:36:44+08:00VoidIsVoidserver: add data: [DONE] to /chat/completions stream response (#9459)
455feff4aa8461da7c432d144c11da4802e41fef3cf0abc6a2c25272d5cf01384dda8ee8bfec4ba87452024-09-13T14:23:11+02:00Xuan Son Nguyenserver : add loading html page while model is loading (#9468)
4560abc6a2c25272d5cf01384dda8ee8bfec4ba8745bd35cb0ae357185c173345f10dc89a4ff925fc252024-09-13T09:53:38+03:00Georgi Gerganovllama : llama_perf + option to disable timings during decode (#9355)
45778203641fee3b1f82abaff0c7f667e1b4a286390e6b7801bd189d102d901d3e72035611a25456ef12024-09-12T22:30:11+02:00Mathijs Henquetserver : Add option to return token pieces in /tokenize endpoint (#9108)
4588d300bd35fbe23b35a4e1ece0cf0fe8f4333102949006c67b4c6cc2e7c75a875b4d6e161ebae287c2024-09-10T22:40:59+02:00matteoenable --special arg for llama-server (#9419)
459bfe76d4a17228bfd1565761f203123bc4914771b293bebe0773c907c0c866213856eeba41b035df12024-09-09T23:36:09+02:00Xuan Son Nguyencommon : move arg parser code to `arg.cpp` (#9388)
4601b9ae5189cd279c6b45e36d43e4f9ccae628d02fe32d0816edfd247784f6780b0bb9ae0bceef5e472024-09-07T20:43:51+02:00Xuan Son Nguyencommon : refactor arg parser (#9308)
461df270ef74596da8f1178f08991f4c51f18c9ee82947538acb8617756a092042ff7e58db18dde05ec2024-09-07T15:16:19+03:00Georgi Gerganovllama : refactor sampling v2 (#9294)
4629b2c24c0993487d3b34a873980e292da571481f3134bc38ecf3e2c5460581badce289a1ffa6804532024-09-06T23:21:29+02:00Xuan Son Nguyenserver : simplify state machine for slot (#9283)
4634a1411b4f17b6569dc0a067e5914dcc0f738b3708ebe8ddebd68526757c631cd019de009697c63c22024-09-06T14:06:04+02:00Xuan Son Nguyenserver : fix missing lock (#9334)
4649bc6db28d011d47a5f318dc4aebbe7927fac462932b2ec88bc44b086f3807c739daf28a1613abde12024-09-05T21:48:47-04:00compiladeggml-quants : ternary packing for TriLMs and BitNet b1.58 (#8151)
46532b2ec88bc44b086f3807c739daf28a1613abde11031771faaba28d07b4ec6a812cb21532efc8c312024-09-06T06:34:36+08:00awatunaUpdate build.yml (#9184)
46682e3b03c11826d20a24ab66d60f4de58f48ddcdb9379d3cc1718e9ec6ab5ffcb60a06bbe048a61ee2024-09-04T11:08:32+03:00Radoslav Gerganovrpc : make RPC servers come first in the device list (#9296)
467b69a480af4db8ffd6cbb2efe7ed8132bc7d3907348baa61eccdca9205daf8d620ba28055c2347b642024-09-03T10:00:36+03:00Georgi Gerganovreadme : refactor API section + remove old hot topics
46848baa61eccdca9205daf8d620ba28055c2347b64f1485161e58d562099dd050f8ac3a9ea9f4cd7652024-09-02T22:08:38+02:00Xuan Son Nguyenserver : test script : add timeout for all requests (#9282)
4696e7d133a5f9409dd257fad90d7f320721b07a1b2b60074f1c26d8354053a952844ef3f7ebefd88032024-09-02T17:11:51+02:00Xuan Son Nguyenserver : refactor multitask handling (#9274)
4708f1d81a0b6f50b9bad72db0b6fcd299ad9ecd48ca47667cff41f5a198eb791974e0afcc1cddd32292024-09-01T22:38:17+08:00Molly Sophiallama : support RWKV v6 models (#8980)
47142c76d1358021ccdbe8ba89109c143dd7ae166df9f7d4bcf5c27d37b0c7da82eeaf9c1499510554b2024-08-29T19:20:53-04:00Faisal ZaghloulThreadpool: take 2 (#8672)
4729f7d4bcf5c27d37b0c7da82eeaf9c1499510554b1d1ccce67613674c75c9c7e3fa4c1e24e428ba482024-08-27T18:28:06+08:00Jan Boonserver : fix crash when error handler dumps invalid utf-8 json (#9195)
473a77feb5d71831c61e455541e8a655b9f0337ea8c2e59d61c1b321431c597c1f12249273427d5640d2024-08-27T11:07:01+02:00Xuan Son Nguyenserver : add some missing env variables (#9116)
474e5edb210cd361689da41f032f1b36c45ff1bf9be0c41e03ceba1aafaf469476a56347419d57853762024-08-26T12:16:57+03:00Georgi Gerganovserver : update deps (#9183)
475a1631e53f6763e17da522ba219b030d8932900bdfc54ef0d1c138133a01933296d50a36a1ab647352024-08-21T17:58:11-04:00compiladellama : simplify Mamba with advanced batch splits (#8526)
476fc54ef0d1c138133a01933296d50a36a1ab64735b40eb84895bf723c7b327a1e3bf6e0e2c41877f82024-08-21T11:04:34+02:00Xuan Son Nguyenserver : support reading arguments from environment variables (#9105)
47718eaf29f4c5f7c8e89412f3a9a0392a9e4e01d75554b049068de24201d19dde2fa83e35389d4585d2024-08-19T10:10:21+03:00Radoslav Gerganovrpc : prevent crashes on invalid input (#9040)
4788b3befc0e2ed8fb18b903735831496b8b0c80949d565bb2fd5a2a58b9924a7a34e77a87c78c521372024-08-16T17:19:05+02:00Xuan Son Nguyenserver : refactor middleware and /health endpoint (#9056)
479d565bb2fd5a2a58b9924a7a34e77a87c78c52137ee2984bdaf10c14d440ad873a049bcc09b786d9b2024-08-16T21:34:41+08:00tc-mbllava : support MiniCPM-V-2.6 (#8967)
48037501d9c79ed5897db4b73ea7502211d25b3f7634af8420afba39de4968adf2695ce12fd42422a132024-08-15T15:28:05+08:00Riceball LEEserver : fix duplicated n_predict key in the generation_settings (#8994)
481234b30676a97ce227b604c38beb9dcaca406dea95fd89a70ead34d1a17015ddecad05aaa2490ca462024-08-15T08:21:57+02:00Jiří Podivínserver : init stop and error fields of the result struct (#9026)
48298a532d474c73d3494a5353024cb6a4fbbabbb3543bdd3ce188cd247bda7c1bd1ad01fa64e5666902024-08-14T02:51:02-04:00compiladeserver : fix segfault on long system prompt (#8987)
4835ef07e25ac39e62297a67208c5bcced50835a2dd4134999e01f31256b15342b41c4de9e2477c4a6c2024-08-12T10:21:50+03:00Georgi Gerganovserver : handle models with missing EOS token (#8997)
4847c3f55c10051c634546247387c5c359c9d499360911b437f228e75aa3d235acec21bfddd23ecce2f2024-08-10T11:43:26+02:00fairydreamingAdd support for encoder-only T5 models (#8900)
4853071c0a5f218f107dabd13b73f6090af683ef5ec4305b57c80eff4f0df5f6acb60b292f03b8f0dd02024-08-09T18:33:53+08:00tc-mbllava : support MiniCPM-V-2.5 (#7599)
486daef3ab233fc02e4c53afd9b07366379876f00d1345a686d8271a24db20d31789c0d4b9ed51dcb0c2024-08-09T08:32:02+02:00Mathieu Geliserver : add one level list nesting for embeddings (#8936)
4871e6f6554aa11fa10160a5fda689e736c3c34169f641f5dd2a6422941faa9f32ab5cb50fc1b24c1f52024-08-06T17:33:39+02:00Xuan Son Nguyenserver : add lora hotswap endpoint (WIP) (#8857)
488a3738b2fa7c60ef2c4592435d1aa7fb8f1f69c3e655858ace0cf2720e56eb01f84ad05e0c94ada3c2024-08-04T17:28:08+02:000cc4mvulkan : implement Stable Diffusion operators (ggml/904)
489978ba3d83d17b10fdf9807006048432b5b3769fcecf6b7f23e664afd7ff856ec39034240ce438daa2024-08-04T18:16:23ZardforkServer: Don't ignore llama.cpp params (#8754)
490ecf6b7f23e664afd7ff856ec39034240ce438daa01aae2b4975b57a265ce8194928fd87f2d71027e2024-08-04T03:55:03-07:00Brian Cunniebatched-bench : handle empty `-npl` (#8839)
491afbbcf3c04e3c6420cad3d72571478cd62ac176ced9d2854c9de4ae1f448334294e61167b04bec2a2024-07-31T18:59:09-05:00Igor Okulistserver : update llama-server embedding flag documentation (#8779)
49201aec4a6310ab0160483196db0e726d78d4c94b641cd47caab88c442edc50e90c8d8d0ac3e82768d2024-07-25T18:10:16-04:00Yaikoserver : add Speech Recognition & Synthesis to UI (#8679)
4934b0eff3df58d8d86e47348fb73d54da3194d416d8a4bad50a8ed24ed1e9df003521468dcc37320e82024-07-25T13:43:27+05:30Ujjawal Panchaldocs : Quantum -> Quantized (#8666)
494b841d0740855c5af1344a81f261139a45a2b39ee64cf50a0ed62d41e4f6c13e08a9b6b0816f46c6e2024-07-23T17:37:42+03:00Vali Malinoiuserver : fix URL.parse in the UI (#8646)
495938943cdbf4dd79005a394d732bc226f9e34e0ff751fcfc6c33ea5f43cadd4d976f8fb176871df5e2024-07-23T13:10:17+03:00Georgi Gerganovllama : move vocab, grammar and sampling into separate files (#8508)
496566daa5a5b38018b2727950bbd280239adb981b66f11a83e4e7700fdf353ed4a29599cb662c792f62024-07-22T15:44:53+02:00Jiří Podivín*.py: Stylistic adjustments for python (#8233)
497628154492a0b2dcc954a3af99e5c267097568eae04bab6b7da0ed2b0a57174a57784d602aabb6c102024-07-22T16:02:09+08:00Jan Boonserver : update doc to clarify n_keep when there is bos token (#8619)
49869c487f4ed57bb4d4514a1b7ff12608d5a8e7ef007283b1a90e1320aae4762c7e03c8790439102522024-07-20T22:25:26+02:00Johannes GäßlerCUDA: MMQ code deduplication + iquant support (#8495)
499be0cfb41752551a4680ee7dfd29f2a05b50db442b57eb9ca4fb79f3163c6a69e154ff76157a4f7162024-07-19T14:34:55+03:00Georgi Gerganovreadme : fix server badge
5000d2c7321e9678e91b760ebe57f0d063856bb018b672a6f101839a150180fc28891ebed379c8f23532024-07-18T18:43:49+08:00Eric Zhangserver: use relative routes for static files in new UI (#8552)
5013807c3de04cde853418033c95e96642876545f3ee02b597be3702174e7b47b44cd03e1da1553284b2024-07-18T16:06:22+08:00RunningLeonserver : respect `--special` cli arg (#8553)
502f17f39ff9cb71fa7b0eda0c649bb0ce9b2d8a6b89104bc20edf47f74dc49379b7d61d0c6e85a48822024-07-15T08:04:56-04:00M-Aserver: update README.md with llama-server --help output [no ci] (#8472)
5034e24cffd8cccd653634e24ee461c252bd77b14266af51c0d96e6268769fc05c98d5b1a5e832c00172024-07-12T14:48:15+03:00Georgi Gerganovserver : handle content array in chat API (#8449)
504c3ebcfa148e867a68e78fd5c4f0c23e8f84c788b8a4441ea1a2564578134404f31158c318e9c0bf32024-07-12T03:14:12-05:00Douglas Hanleyserver : ensure batches are either all embed or all completion (#8420)
505278d0e18469aacf505be18ce790a63c7cc31be26dd07a123b79f9bd9e8a4ba0447427b3083e9347a2024-07-10T20:08:17-04:00Clint HerronInitialize default slot sampling parameters from the global context. (#8418)
506dd07a123b79f9bd9e8a4ba0447427b3083e9347af4444d992c16b6b9442f4770c7c3a10b19a083432024-07-10T12:35:18-04:00Clint HerronName Migration: Build the deprecation-warning 'main' binary every time (#8404)
5070f1a39f3439825acf7e3a1663566d410be15217083321c6958acf20747d288031174cc1bf8816e332024-07-10T07:14:51-05:00Dibakar Gopeggml : add AArch64 optimized GEMV and GEMM Q4 kernels (#5780)
508a59f8fdc85e1119d470d8766e29617962549d993fd560fe680c72fd0a0af2bc8881add20ad9190712024-07-09T18:26:40-04:00Clint HerronServer: Enable setting default sampling parameters via command-line (#8402)
5093fd62a6b1c9ca7b7c0093e984cc9c133c6f2726da8db2a9ce64cd4417f6a312ab61858f17f0f85842024-07-07T15:04:39-04:00compiladepy : type-check all Python scripts with Pyright (#8341)
510cb4d86c4d723af87d3d7e3177e9485f20039138486e7299ef5dff0f388922dc6fcbce009e99d80052024-07-07T11:10:38+02:00Bjarke Viksøeserver: Retrieve prompt template in /props (#8337)
511807b0c49ff7071094f97ebc3a0a8e2b9e274f503f8c4c0738d72d2162736edd72dd5db8b269adca12024-07-04T15:46:11+02:00fairydreamingInference support for T5 and FLAN-T5 model families (#5763)
512a95631ee97bb24861af6bdeec380270459631e8ef3f65429c44bb195a9195bfdc19a30a79709db7b2024-06-26T19:26:13+03:00Georgi Gerganovreadme : update API notes
513f3f65429c44bb195a9195bfdc19a30a79709db7b88540445615e77a0177fcca43aaa8e9d8eea68642024-06-26T18:33:02+03:00Georgi Gerganovllama : reorganize source code + improve CMake (#8006)
5146777c544bdd8c5d9de3220d6e2557957bbbf2a4f163d50adaf8897d8b734d701ff332de6be63d4842024-06-26T01:45:58+01:00Olivier Chafik`json`: fix additionalProperties, allow space after enum/const (#7840)
515dd047b476c8b904e0c25e5dbc5bee6ffde2f6e17925c30956dd17723c3a25297bcd0a609aec606632024-06-25T19:20:06+02:00slarendisable docker CI on pull requests (#8110)
516925c30956dd17723c3a25297bcd0a609aec60663c8ad35955ad2c68db172dcd0e857423ab128518d2024-06-25T08:13:27-07:00joecryptotooAdd healthchecks to llama-server containers (#8081)
51748e6b92cc378c937e59719f2c0f482bf76c9ca813791ad219323389106dc3fd80814eb5bbb7b80de2024-06-25T13:56:49+02:00Xuan Son NguyenAdd chat template support for llama-cli (#8068)
5183791ad219323389106dc3fd80814eb5bbb7b80def702a90e245499283d6de0b287701c723cda2a872024-06-25T16:57:35+05:30HanishKVCSimpleChat v3.1: Boolean chat request options in Settings UI, cache_prompt (#7950)
5196a2f298bd784403c5c33eebb822217ec5d9b559011318d9aa1f668aa10407a5cb9614371af32f3ce2024-06-23T18:03:08+03:00Aarni Koskelaserver : fix JSON-Scheme typo (#7975)
520b6b9a8e606da7764bd3649c2ea574979c85abd4345c0e2e4c1268c2d7c8c45536f15e3c9a731ecdc2024-06-23T13:14:45+02:00slarenfix CI failures (#8066)
521ba5899315283eb1df3902363daf79bdc5eefe426a7854743c5e6216a6178824a603aa9479728ddd52024-06-19T23:57:10Zsasha0552server : fix smart slot selection (#8020)
522e6ecc2be470e3c5c6c5c9d8b90aa83a1f7725084a94e6ff8774b7c9f950d9545baf0ce35e8d1ed2f2024-06-18T09:37:20+03:00Georgi Gerganovwhisper : use ggml_backend_sched (whisper/2239)
5231c641e6aac5c18b964e7b32d9dbbb4bf5301d0d7963552903f51043ee947a8deeaaa7ec00bc3f1a42024-06-13T00:41:52+01:00Olivier Chafik`build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809)
524704a35b183748954013bd875bbbfdd9eaca14e62dcf752707d96eb305f546526c7bc5d01f08311302024-06-12T14:42:29+03:00Georgi Gerganovserver : restore numeric prompts (#7883)
525dcf752707d96eb305f546526c7bc5d01f0831130f2b5764beb35583295e2475479c18f249b139b582024-06-12T17:05:35+08:00Meng, Hengyuupdate intel docker oneapi-basekit to 2024.1.1-devel-ubuntu22.04 (#7894)
52614f83526cd27f638c856ea6eff08110b9860eb2a6fe42d073f0554eada93ac9d40574025aeedb7032024-06-11T12:18:58-04:00Deven Mistryfix broken link in pr template (#7880) [no ci]
527fd5ea0f897ecb3659d6c269ef6f3d833e865ead7c28a83902cf6ab6a9e085ad6d4cc2e95c4ccfe402024-06-10T14:18:41+02:00slarenci : try win-2019 on server windows test (#7854)
52857bf62ce7cb75cca589943e2050d29bff4026e763e2ee443159724e2d3a0741f6b167e599ec088aa2024-06-09T11:24:29-04:00Nicolás Pérezdocs: Added initial PR template with directions for doc only changes and squash merges [no ci] (#7700)
5293e2ee443159724e2d3a0741f6b167e599ec088aa42b53d192f4e3abf1b7c8e424628424504ea5dc52024-06-09T12:50:35+02:00mgroeber9110server: do not remove whitespace at the start of a completion chunk (#7830)
5307a16ce7db2a74a223f0f3b9cee66d4539c5bce8fda799b41891e34aac86ce4e173f9c4c0afd4fab32024-06-08T07:50:31Zsasha0552server : smart slot selection using Longest Common Prefix (#7728)
5317027b27d765db95d4ac6b569d976e387a8715881a5cabd76491f07494c5b8267f921c73f5e2bbfb42024-06-07T11:15:49+02:00Johannes Gäßlerserver: update cache_prompt documentation [no ci] (#7745)
532a5cabd76491f07494c5b8267f921c73f5e2bbfb4d5c938cd7716b9a2ace49a43a469dfbffcff4d282024-06-07T15:09:45+08:00woodxserver : do not get prompt in infill mode (#7286)
533ee459f40f65810a810151b24eba5b8bd174ceffef83351f9a62a6262f1fc3d08f320033089cddfb52024-06-06T19:19:59+03:00Georgi Gerganovserver : fix --threads-http arg (#7801)
53455b2d0849d3ec9e45e4a4d9e480f5fa7977872a6f5d7b268ec4bf8628aa6ccc9f6631d0230dde76f2024-06-06T10:07:06+01:00Olivier Chafikgrammars: x{min,max} repetition operator (#6640)
5352d08b7fbb483c14bd2b173d4cd51ea3a4f862e8fd67caea0d6e6c303d31b01d0a010973e6c908dff2024-06-06T07:19:49+02:00slarendocker : build only main and server in their images (#7782)
5361442677f92e45a475be7b4d056e3633d1d6f813b554c247caffed64465f372661f2826640cb104302024-06-04T21:23:39+03:00Georgi Gerganovcommon : refactor cli arg parsing (#7675)
5376d1616944d9efd342ed2a4fd318722adfc9febcdbde7cd3cd949c1a85d3a199498ac98e78039d46f2024-06-04T10:01:09+03:00Georgi Gerganovggml : prevent builds with -ffinite-math-only (#7726)
538bde7cd3cd949c1a85d3a199498ac98e78039d46fa5735e4426b19a3ebd0c653ad8ac01420458ee952024-06-03T20:03:26+03:00Radoslav Gerganovllama : offload to RPC in addition to other backends (#7640)
539a5735e4426b19a3ebd0c653ad8ac01420458ee950b832d53ba0ffcc759c8d62ede3772dd62321f8e2024-06-04T00:14:15+09:00Masaya, Katoggml : use OpenMP as a thread pool (#7606)
5407c4e5b7eae26581869e782015d9deca947c349979422c5e34bbd302493b77a8f6d546154a1f4fe822024-06-02T13:39:08-04:00Austinchore : add ignore rule for generated server themes (#7689)
5419422c5e34bbd302493b77a8f6d546154a1f4fe82e141ce624af57bdffbaf57014a044eb1d96892302024-06-02T19:13:54+10:00nickp27[SYCL] Update rpc-server.cpp to include SYCL backend (#7682)
5422e666832e6ac78194edf030bd1c295e21bdb022c2ac95c9d5678d05e253691fb1f26471675bff5ad2024-06-01T21:31:48+02:00Yazan Agha-Schraderserver : new UI (#7633)
5432ac95c9d5678d05e253691fb1f26471675bff5ad750f60c03e4d3f53fa51910551ce87a3d508d2d72024-06-01T21:50:18+05:30HanishKVCSimpleChat: Simple histogram/repeatMatching driven garbageTrimming, Settings UI, Streaming mode, OpenAi Compat (Model, Authorization Bearer), Save/Restore session, Auto Settings UI (#7548)
544a323ec60af14a33d560df98f2cc41b4112cb4f800515ad93f48df63bbff204eddb0cac75e8585c652024-05-31T22:23:04+03:00Georgi Gerganovserver : update js (#7670)
545972b555ab935705f3437abd5909a5c46852811f63854c9d07f67de7f8cd6d86117bfaef47549b05a2024-05-30T09:52:39+02:00Johannes GäßlerREADME: explain parallel build [no ci] (#7618)
5463854c9d07f67de7f8cd6d86117bfaef47549b05aeb57fee51f7b4d78039f003249873c2eb46f12f62024-05-30T14:19:08+08:00Meng, Hengyu[SYCL] fix intel docker (#7630)
547e2b065071c5fc8ac5697d12ca343551faee465cc0548a4187f2e53b8fc6d9ff0f4c71988f708ff422024-05-28T17:53:37+08:00Neo Zhang[SYCL]fix ggml_sycl_mul_mat_id() to match the change of api (#7436)
5489335b969e86a222e247adacedf814d8abfff8847c41767154eb82aa3fe7568fc816c3402b78eae942024-05-28T06:55:51+02:00mgroeber9110server: do not remove whitespace at the start of a completion chunk (#7524)
549b9adcbbf92fc7096bee23fe61496d25652ebf7659588f196b1d7b21bdff013fcf958c249576b26192024-05-26T06:26:34+05:30HanishKVCSimpleChat Completion Mode flexibility and cleanup, Settings gMe, Optional sliding window (#7480)
55027891f6db03de6e3fd5941983838c29bef253352fbca2f27fc7fa9aa4a8ad0357478fdb9084729082024-05-24T23:47:56+10:00Briandocker.yml: disable light-intel and server-intel test (#7515)
551fbca2f27fc7fa9aa4a8ad0357478fdb9084729080df0aa8e43c3378975269a51f9b876c8692e70da2024-05-24T14:31:13+02:00fairydreamingAdd support for ArcticForCausalLM (#7020)
5523015851c5ac7334fb544a23a70a284c117b8704455ac3b7aeaf52f19786ed96e885d89521fc0f6c82024-05-23T14:29:26+02:00Daniel Beveniusllama : add getters for n_threads/n_threads_batch (#7464)
5531e374365d170b7f692fd7753c145e21bc14486c8197ff91462dd05bb9a3be03578114abf0c3555362024-05-22T23:23:21+05:30HanishKVCSimpleChat: a simple and dumb web front end for testing /chat/completions and /completions end points and try chat (#7350)
554201cc11afa0a1950e1f632390b2ac6c937a0d8f06369bf04336ab60e5c892dd77a3246df910151472024-05-22T04:28:32+08:00liuwei-gitllama : add phi3 128K model support (#7225)
555d7e852c1bc8e85bf62a6f1aede08cd2de723404a917dc8cfa67a72fb7c8bf7392270da3bf4833af42024-05-21T14:39:48+02:00jaime-m-pTokenizer SPM fixes for phi-3 and llama-spm (bugfix) (#7425)
556917dc8cfa67a72fb7c8bf7392270da3bf4833af4fabf30b4c4fca32e116009527180c252919ca9222024-05-20T20:15:57+02:00jaime-m-pTokenizer SPM fixes for phi-3 and llama-spm (#7375)
5573bc10cb485dd7efa4da6c64e73ad0c9e2bfe08216bf9b66fa3f263ca2175dcb5f6d0a658581e1dfb2024-05-20T15:10:03+03:00Georgi Gerganovserver : fix temperature + disable some tests (#7409)
5581cc0155d04918cb3017afa472acea51b77483c4ae932094d58f513d5996c3efc9f6fed8238894c572024-05-20T10:16:41+03:00Georgi Gerganovserver : tuning tests (#7388)
559e932094d58f513d5996c3efc9f6fed8238894c572789baf480ba7dc9281b65f601f0918e58920f542024-05-20T08:56:05+03:00Georgi Gerganovserver : return error on too large embedding input (#7389)
5601b01f06db0cff5f5f600bb754fc39fde565ed56a41858392e17abead21735309bf17cb55183d8c312024-05-19T16:26:02+02:00Johannes Gäßlerserver: add test for token probs (#7347)
56141858392e17abead21735309bf17cb55183d8c316aade19ee74b896c59929676629340b36be3e22c2024-05-19T16:06:33+02:00Johannes Gäßlerserver: fix seed being reported back (#7382)
562cb42c294279bc4a0a4e926a7b5a5568049f12fa7d233b507cd19fcc2d8d8963ecc6a3eb7a33f2ecc2024-05-18T11:10:47+02:00Johannes Gäßlerserver: correct --threads documentation [no ci] (#7362)
563f4bd8b3d260bb09491ba63c77ab7012b744362ef51e9d02599336e62948d29f1d6c05addeb921ac22024-05-17T17:25:44+03:00Radoslav Gerganovrpc : set SO_REUSEADDR for the server socket (#7320)
564d273c1402b25086fd91aef2467ac13f2e49fa0ea27b040691cbe45314147c2745e891a38e9c048d42024-05-17T15:11:45+03:00Aarni Koskelapy : convert-hf-to-gguf-update improvements (#7340)
56527b040691cbe45314147c2745e891a38e9c048d429c60d8cddcfd14fa8a6bf023a6c4eb8692c76ba2024-05-17T13:24:38+02:00fairydreamingllama : use n_embd_head_v when reshaping kqv (#7327)
566ee94172d33399d2e814ca05c8a3ff8c523ebb093934266c0e0b2aa9781fdba2deb112c161ff038a92024-05-17T10:00:17+03:00Radoslav Gerganovserver : add support for the RPC backend (#7305)
5679c4fdcbec8c7fcc428e723b0d8a1cf1f351ba64224ecb58168dce81646c2ed425690a106591c8c6d2024-05-17T02:11:03+02:00Leon Knauer[Server] Added --verbose option to README [no ci] (#7335)
568583fd6b000ec9ad1b465b5c98524f4a0ae3880779f773486ab78d65f5cca3f7e31c862b7043bf7212024-05-15T08:44:16+02:00Johannes Gäßlerserver bench: fix bench not waiting for model load (#7284)
5694f0263633b40e94e8b69fd6e7e4395cfedfd5c121265c670fd8e41e1947352c96c5179adda97fb2c2024-05-14T10:11:24-04:00Steve Grubbserver: free sampling contexts on exit (#7264)
5705e31828d3e35c76ecfee665bc23771a4bec1d130541600201e6480f54ae09e58d16b154d4b4b331d2024-05-14T14:27:19+03:00Radoslav Gerganovggml : add RPC backend (#6829)
571541600201e6480f54ae09e58d16b154d4b4b331defc8f767c8c8c749a245dd96ad4e2f37c164b54c2024-05-14T09:33:42+02:00slarenllama : disable pipeline parallelism with nkvo (#7265)
572e586ee42595500c53938e937b6b6ad5353ad76dccbf75894d256f1861f6409565db599365de3d4b82024-05-12T19:40:08-07:00Benjamin Findleychange default temperature of OAI compat API from 0 to 1 (#7226)
573cbf75894d256f1861f6409565db599365de3d4b80d5cef78aeafae4d4e6d56e2d4bcda771af58cc92024-05-13T08:04:29+08:00Neo Zhang[SYCL] Add oneapi runtime dll files to win release package (#7241)
5740d5cef78aeafae4d4e6d56e2d4bcda771af58cc9dc685be46622a8fabfd57cfa804237c8f15679b82024-05-13T08:02:55+08:00Neo Zhang[SYCL] update CI with oneapi 2024.1 (#7235)
575988631335a20d06497f58be0b8ba13adb4323a22f99e1e456eaf69cc38c1982a2693ce41c0f897ef2024-05-11T04:13:02-04:00Steve Grubbserver : free llama_batch on exit (#7212)
5765ae3426b0b64672991563d4c28b2018b9f961467b83cc3f5b303ff30c52874b2d5864dc6385ebf9f2024-05-11T10:11:28+02:00Johannes Gäßlerserver: fix reported top tokens for temperature 0 (#7203)
5774e3880978f8b1bf546dd4e6f3b524d6b8739c49cf89fe2732c5709f6e86d5f4aee2e6d2a561f2eb22024-05-10T07:01:08-04:00Justine TunneyFix memory bug in grammar parser (#7194)
578d46dbc76f8770caec0175f1e57777173c70556a00961d866044143eefec5b525e991611f73fd4f6e2024-05-09T16:40:42+03:00Georgi Gerganovreadme : add scheduled server workflow status badge
57947345248827f426038098d016ed11975021b491907cd41d0965829463eff73eda3348aedbfd3a4442024-05-09T17:34:37+08:00Albert Jinopencl : alignment size converted from bits to bytes (#7090)
580bd1871fa2b1bf8a081b43ba9bc85f8ffd46fac4626458af1d63c85195cd96cd1673051e332d06d302024-05-08T20:12:06+01:00JohnnyBserver : add themes + favicon (#6848)
581911b3900dded9a1cfe0f0e41b82c7a29baf3a217ad211edef5db1f1fb955874b7ca6a67bd0c887082024-05-08T14:27:58+02:00Johanserver : add_special option for tokenize endpoint (#7059)
5821fd9c1741d864d01cd7ec6d67227b92d7bfabf224cd621c26de2095cd7c4464bdec5fe2e696ef3f32024-05-08T13:24:14+02:00Xuan Son Nguyenclean up json_value & server_log (#7142)
5833855416027cb25d9a708ffa5581cf503a87856a6c0e6fbf8c380718102bd25fcb8d2e55f8f9480d12024-05-08T02:30:09-04:00Justine Tunneyggml : introduce bfloat16 support (#6412)
584af0a5b616359809ce886ea433acedebb39b12969b6aa6702030320a3d5fbc2508307af0d7c947e402024-05-07T23:07:58+02:00Johannes Gäßlerserver: fix incorrectly reported token probabilities (#7125)
585260b7c65296fba0568eeb1ff05244ea0be206b5453d6c52e227dedef347b21e28febcfb9caeecdad2024-05-07T13:44:29-05:00Kyle Misteleserver : update readme with undocumented options (#7013)
5868f8acc8683a00ce40fa5a81161a079d2167126e6ca3632602091e959ed2ad4c09c67a7c790b10d312024-05-05T13:38:55+02:00Sigbjørn SkjæretDisable benchmark on forked repo (#7034)
587cf768b7e71cbcc9886c753ae963c2b68893d02e4fcd84a0f5a584ab5271745d7ffef21c8a6bc7b0c2024-05-04T13:10:15-03:00JeximoTidy Android Instructions README.md (#7016)
58803fb8a002df2e96104f9e06de9c78d2a8ed91e9292139b90af4841d7fd060b526bdd443b621770ff2024-05-04T12:06:40+03:00maor-psIf first token generated from the server is the stop word the server will crash (#7038)
5893ea0d36000e2314baba7e9fc6a97f08670a6f7e41613ef8d8eb2479ba55c4d598e08c8f3f18a0fed2024-05-01T17:52:55+02:00Johannes GäßlerServer: add tests for batch size, different seeds (#6950)
5909c67c2773d4b706cf71d70ecf4aa180b62501960952d03dbead16e4dbdd1d3458486340673cc24652024-04-30T12:16:08+03:00Georgi Gerganovggml : add Flash Attention (#5021)
5918843a98c2ba97a25e93319a104f9ddfaf83ce4c4b8c1476e44cc1f3a1811613f65251cf7790676362024-04-30T00:52:50+01:00Olivier ChafikImprove usability of --model-url & related flags (#6930)
592b7368332e24c5b2c8038bf8267f43632783fcc35928e0b7013c862cf10701957b3d654aa70f11bd82024-04-27T17:50:48+02:00Pierrick Hymbertci: server: tests python env on github container ubuntu latest / fix n_predict (#6935)
593928e0b7013c862cf10701957b3d654aa70f11bd80c4d489e29e53589bf13a801fe7c94b7b546d8f62024-04-26T19:08:30+01:00agray3Reset schedule earlier to allow overlap with ggml graph computation on device (#6933)
5940c4d489e29e53589bf13a801fe7c94b7b546d8f6017e6999b5184234370b22a2f868e1be911e8d882024-04-26T20:06:33+02:00Pierrick Hymbertquantize: add imatrix and dataset metadata in GGUF (#6658)
595bbe3c6e76157a5d806fdc155451f0ca8936248ee7f5ff558eed0f732af8f25c2ab0645610bdec80c2024-04-26T12:27:25+02:00Pierrick Hymbertci: server: fix python installation (#6925)
5967f5ff558eed0f732af8f25c2ab0645610bdec80c9e4e077ec50fde6049b128662c72d37a3c28e34b2024-04-26T12:15:30+02:00Pierrick Hymbertserver: stop generation at `n_ctx_train` if `n_predict` is not set (#6638)
5979e4e077ec50fde6049b128662c72d37a3c28e34b83b72cb086ce46a33dececc86bfe4648b6120aa82024-04-26T11:11:51+02:00Pierrick Hymbertci: server: fix python installation (#6922)
59883b72cb086ce46a33dececc86bfe4648b6120aa8d4a9afc1009f0da88d04b2c5f672d81d5ae946752024-04-26T10:41:53+03:00Georgi GerganovMerge pull request from GHSA-p5mv-gjc5-mwqv
599d4a9afc1009f0da88d04b2c5f672d81d5ae946757d641c26ac73956a54b204cabefe85c7648236782024-04-26T09:27:49+02:00Pierrick Hymbertci: server: fix python installation (#6918)
6007d641c26ac73956a54b204cabefe85c7648236785790c8dac1a4f0aa80b4efee3b962d8c04c829e82024-04-26T09:26:59+02:00Pierrick Hymbertci: fix concurrency for pull_request_target (#6917)
6015790c8dac1a4f0aa80b4efee3b962d8c04c829e846e12c4692a37bdd31a0432fc5153d7d22bc7f722024-04-26T09:26:16+02:00Pierrick Hymbertbench: server add stop word for PHI-2 (#6916)
6023fe847b5747676ee1bf90371c46ed0bc66b5724037246b1031b1680c0dcaf20aef736d6b446203fa2024-04-24T12:54:24+02:00mgroeber9110server : do not apply Markdown formatting in code sections (#6850)
60337246b1031b1680c0dcaf20aef736d6b446203fa28103f4832e301a9c84d44ff0df9d75d46ab6c762024-04-24T05:15:29-05:00Kyle Mistelecommon : revert showing control tokens by default for server (#6860)
60428103f4832e301a9c84d44ff0df9d75d46ab6c76c0d1b3e03e27634ac2871761f5033cf9324d472d2024-04-24T11:08:36+02:00Johannes GäßlerServer: fix seed for multiple slots (#6835)
6055cf5e7d490dfdd2e70bface2d35dfd14aa44b4fb40f74e4d739e9250431cf339ae7588b28d8d06632024-04-21T18:48:53+01:00Olivier Chafik`build`: generate hex dump of server assets during build (#6661)
60640f74e4d739e9250431cf339ae7588b28d8d0663b9cc76d87e3d7ae5900f19d4fe8f8976d0a358882024-04-21T18:36:45+03:00Georgi Gerganovllama : add option to render special/control tokens (#6807)
607b8109bc0139f15a5b321909f47510b89dca47ffcaed82f6837a3ea515f4d50201cfc77effc7d41b42024-04-21T00:29:50+08:00Jan Boondoc : server tests require llama to be built with curl enabled (#6788)
608637e9a86c220718d008b54842dfd294aa96d3b7a9958c81b798a5872087b30b360e4674871f2479e2024-04-19T13:19:01+02:00Pierrick Hymbertserver: static: upstream upgrade (#6765)
609ab9a3240a9da941fdef5cd4a25f2b97c2f5a67aafbbc030ba93561fac842af994c5c6c4c1147f13b2024-04-12T19:43:38+01:00Olivier ChafikJSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length (#6555)
61024ee66ed0d908d156bd0d1747b63a636a495cd7a91c736015b66ba1d0b82cbae6313b6d5eaa61b682024-04-12T13:49:21+02:00Pierrick Hymbertserver : coherent log output for KV cache full (#6637)
611cbaadc92942c50aab599a9e4c163afc1f44f7c261bbdaf6ecda6f0a360dfb307b256fcb6838c560b2024-04-11T19:47:34+01:00Olivier Chafikgrammars: 1.5x faster inference w/ complex grammars (vector reserves / reuses) (#6609)
612400d5d722d7edf7de0cf24a18c42b183c65047d25dc9dd7152dedc6046b646855585bd070c91e8c82024-04-09T01:31:47-07:00Ed Leeserver : detect search query to start webchat (#6554)
613e3c337d87ca650972105a51c6ce302dd236c07adbeea6e1b16e783a0886e78dec01002a8c00db24d2024-04-08T06:02:30-07:00Rick Gllama : support negative ith in llama_get_ API (#6519)
614beea6e1b16e783a0886e78dec01002a8c00db24d87fb5b4234d4b9c56ac94cf7aa229c8fd7defdb02024-04-08T20:43:30+08:00Jan Boonllama : save and restore kv cache for single seq id (#6341)
615b66aec675c1571a06b3570b858ae711246f96f8457dd02c44b2a0eb79e28f6c5eb8242a5d2d3174d2024-04-03T22:57:20+02:00Daniel Beveniusbackend : fix typo in scheduler documentation (ggml/781)
61657dd02c44b2a0eb79e28f6c5eb8242a5d2d3174d75cd4c77292034ecec587ecb401366f57338f7c02024-04-06T10:31:33-04:00Clint HerronTests: Added integration tests for GBNF parser (#6472)
61775cd4c77292034ecec587ecb401366f57338f7c0a8bd14d55717754a1f48313a846a2b16fa998ad22024-04-06T05:40:47+02:00Pierrick Hymbertci: bench: support sse and fix prompt processing time / server: add tokens usage in stream OAI response (#6495)
6180a1d889e27d6aaa3293dd2c692b849a9bcf4b4747dda1b727ef1730783a6077136d28b83e70dd3972024-04-04T17:31:22+01:00Ed Lepedusserver: add cURL support to server Dockerfiles (#6474)
6192e66913e5f56209f4c949f98e431925b78e7e84d8120efee1d9931b514aeb5a047209d576f23286c2024-04-04T11:03:00-04:00Shakhar Dasguptaserver: allow penalizing repetition of newlines on server webpage (#6431)
6207a2c92637ae265654a68f62e6a7610b358255d3f4bcd6b959ca3991084ad1d8464caf2a734e29b1d2024-04-04T11:57:58+02:00Pierrick Hymbertci: bench: add more ftype, fix triggers and bot comment (#6466)
6214399f13fb9462cd06f3f154d0aee738425000fea1a43c7254ed5de91d09274b853db843c518f1b642024-04-04T09:34:58+03:00Georgi Gerganovserver : remove obsolete --memory-f32 option
6221a43c7254ed5de91d09274b853db843c518f1b6472d73af65132792a52433952ca4729b01c36cde22024-04-04T01:33:48-05:00Xiao-Yong Jinserver : add option to disable KV offload (#6468)
6235fb1574c8112c757fc202fdae279da883f34e61060cdf40cc32f0ad4cb11e0ca8fd38f3b93d8d6402024-04-03T13:22:57-07:00FattireA few small fixes to server's README docs (#6428)
62460cdf40cc32f0ad4cb11e0ca8fd38f3b93d8d640bb43cf7e9d86d69ffd9c7f008f75db890a35b45a2024-04-03T20:09:52+02:00JH23Xserver : handle exception on wrong type in request (#6452)
6255d4f12e4624bf4435ba26753814bedd4e9b62803154d4ee39c38e231fb5c3910df14d2fc920fdf1b2024-04-03T18:56:37+01:00Ed Lepedusserver: add cURL support to `server.Dockerfile` (#6461)
626226e819371eec0f298d9075198394a07b23ecfa9c50a82ce0f71558cbb8e555146ba124251504b382024-04-01T12:36:40+02:00Pierrick Hymbertci: server: verify deps are coherent with the commit (#6409)
627057400a3fd457f4f214684eeb171444663b47a23b75c38166cf0c66793a32d5c3d6cb69929dd083d2024-03-29T08:23:22+01:00Daniel Beveniusllama : remove redundant reshape in build_kv_store (#6369)
62828cb9a09c4d10a489be1238abe7a858dcd4d65f2cfc4d75df6399b36153ef739f2c1abee4c114bb82024-03-28T11:27:56+01:00Pierrick Hymbertci: bench: fix master not schedule, fix commit status failed on external repo (#6365)
6296902cb7f2e3479f364ee177118200fb7e4e9fc92d2d8f389960a106b66313ff1621bdb1aaaaaa2852024-03-28T16:50:48+08:00Eric Zhangserver : stop gracefully on SIGTERM (#6348)
630a016026a3ac16d8c9b993a3573f19b9556d67de453c7ec53d5eca26b2c0c648605543a5fa6c128172024-03-27T20:26:49+01:00Pierrick Hymbertserver: continuous performance monitoring and PR comment (#6283)
6311740d6dd4e00dedda87d6820b0e0d8e70dade3400642b22cd12af278d6e7e459b73411947c1693812024-03-27T08:08:59+01:00Mateusz Charytoniukreadme : add php api bindings (#6326)
6320642b22cd12af278d6e7e459b73411947c169381a4f569e8a316cbd33d2b4de94b694d111507475a2024-03-27T13:55:29+08:00Eric Zhangserver: public: use relative routes for static files (#6325)
633557410b8f06380560155ac7fcb8316d71ddc983755c1b2a3bbd470e9e2a3a0618b92cf64a885f8062024-03-26T10:46:41-04:00compiladellama : greatly reduce output buffer memory usage (#6122)
6343d032ece8e6973441273601ca2981130608e287de190f1fca6f60d80944f9e8709d343a025c4d2452024-03-26T16:47:43+08:00Jan Boonserver : add `n_discard` parameter (#6300)
635ad3a0505e3b6cd777259ee35e61d428357ffc56595ad616cddda50273e955bfe192328acd9aa48962024-03-25T09:42:17+01:00Xuan Son NguyenServer: clean up OAI params parsing function (#6284)
636ddf65685105a39a57b1e7f80c3aa502a6313af24d03224ac9840351023ff8abcf4aa0542258a53df2024-03-24T12:04:25+08:00Meng, Hengyu[SYCL] offload op (#6217)
637f482bb2e4920e544651fb832f2e0bcb4d2ff69ab1997577d5e121568ae39f538021733ccd4278c232024-03-23T18:07:00+01:00Pierrick Hymbertcommon: llama_load_model_from_url split support (#6192)
6381997577d5e121568ae39f538021733ccd4278c23476b0251b27fb64c575507024a671e639d6755942024-03-23T18:00:38+01:00Pierrick Hymbertserver: docs: `--threads` and `--threads`, `--ubatch-size`, `--log-disable` (#6254)
6391b26aebe4de4f048ac99996efd8a2c9af150904d50ccaf5eacb50a2ca378a4ef0dc7aeb45fead6522024-03-23T13:18:45+01:00Pierrick Hymbertserver: flush stdout after logging in both text and json layout (#6253)
64072114edf068a9b2f54d3b09e9a198f611be397e82f0e81e053b41ca28e73a841e7bdbf9820baaa572024-03-22T13:07:44ZOlivier Chafikjson-schema-to-grammar : fix order of props + non-str const/enum (#6232)
6416b8bb3a31d260a01020497c5f01025c34222fcb968e210b3543e0cc71268bee0920441747679ee132024-03-22T19:12:05+08:00Jan Boonserver : fix n_keep always showing as 0 in response (#6211)
64268e210b3543e0cc71268bee0920441747679ee13b3e94f26bab8e3b5338b5902e5af5bb01894cb4a2024-03-22T13:08:28+02:00Georgi Gerganovserver : enable continuous batching by default (#6231)
643be07a03217376c53b1d95e5f658adbbbb2831555d0a71233fbf8ade8ef06ad8e6b81d1d7b254895f2024-03-22T06:41:24+08:00Jan Boonserver : update readme doc from `slot_id` to `id_slot` (#6213)
644f372c49ccdc561ab96fb3c7d2b7cbc0f89a4b359924ce1dce7fdf54894b462d03e7b608a6e574c322024-03-21T11:52:35-06:00semidarkCorrected typo to wrong file (#6199)
6455b7b0ac8dfdd800c0fd0dc69b69991e8cb19fb461943c0198125a0da1a200390e82cf461f9080d992024-03-21T11:50:43ZOlivier Chafikjson-schema-to-grammar improvements (+ added to server) (#5978)
64691f8ad167dcd24b54615b468d9dd764ebe1d37ad6b7e76d28cdf4361740054140708c718284535222024-03-20T13:30:36+01:00Xuan Son NguyenServer: version bump for httplib and json (#6169)
647bc0baab2ea8f806961dd3fb9f534cfeb59a2e1fcd795988d9e09f75412efe2134512914c42780ad52024-03-20T14:14:32+02:00Georgi Gerganovserver : allow to override -ngl in tests (#6170)
64847cc7a7bf9793f81a6dfe7c2096c499403f64dd6bd60d82d0cc8b6852ec535495a5042dbdf05de242024-03-20T16:32:34+05:30KarthickServer: Handle n_keep parameter in the request (#6174)
649bd60d82d0cc8b6852ec535495a5042dbdf05de246c0b287748327741b113d7d6018b68c63039b1c52024-03-20T01:33:49-04:00Jared Van Bortelserver tests : more pythonic process management; fix bare `except:` (#6146)
650a56d09a4407f29c21e149b44fd5308f83aa1cb09d84c48505f60bcd358b82a751d40418c4d2356432024-03-16T13:20:53+01:00Pierrick Hymbertci : close inactive issue with workflow (#6053)
651877b4d0c628cc70dddb5df72ed8fc14d126ca7e812247f4c69a173b9482f68aaa174ec37fc909ccf2024-03-15T13:43:02-07:00Theia Vogelllama : add support for control vectors (#5970)
65212247f4c69a173b9482f68aaa174ec37fc909ccf4e9a7f7f7fb6acbddd1462909c8d696e38edbfcc2024-03-15T16:41:22-04:00Andrew Canisllama : add Command-R support (#6033)
6534e9a7f7f7fb6acbddd1462909c8d696e38edbfcc3020327f6cd6d2ce50528dd65f4b199d2ea8b1ae2024-03-15T22:31:05+08:00Ting Loullava : change API to pure C style for Rust FFI bindgen (#6079)
65443241adf22e8231ffaf3827d2c9310cc0ffd5ac5a44bc969e4cd62ca9f4332e17fe3c51f2093e7c62024-03-14T12:15:39+01:00Pierrick Hymbertserver: disable debug release type sanitizer, simplify trigger (#6047)
65576a936c8939c249a7c3e8e66dfefbab13eae194f463628372d5fe3a0c1e5864aa5fc57deb73870392024-03-13T20:33:56+02:00Georgi Gerganovreadme : update API changes and hot topics
656f30ea47a87ed4446ad55adb265755dc9102956a2d8fd0ccf6ac8b07791ffd1575eed436930854ae32024-03-13T18:54:21+01:00slarenllama : add pipeline parallelism support (#6017)
65799b71c068f624521ad977e08e41589e2971fa1c7306d34be7ad19e768975409fc80791a274ea02302024-03-13T11:39:11+01:00Xuan Son NguyenServer: Use multi-task for embeddings endpoint (#6001)
65805b06210c954491cf0f12034b0a62bd4d69ce78b83796e62bc9f6caae6228168e359890f51e60fee2024-03-11T17:49:47+02:00Georgi Gerganovllama : more consistent names of count variables (#5994)
65983796e62bc9f6caae6228168e359890f51e60fee828defefb66fc8a25404f5de845897145bf340612024-03-11T17:47:47+02:00Georgi Gerganovllama : refactor unicode stuff (#5992)
660828defefb66fc8a25404f5de845897145bf34061caa106d4e05a0ab94225c220b81f9e2cd522339b2024-03-11T14:40:42+01:00Jakub NUpdate server docker image URLs (#5997)
661caa106d4e05a0ab94225c220b81f9e2cd522339b3202361c5b1ba15e695b31209567ef42c22c5c322024-03-11T10:56:41+01:00Xuan Son NguyenServer: format error to json (#5961)
662332bdfd7980718abf664bfa5460f2288a3314984ecab1c75de68de7c41c254e2ae170d3b07bee6d42024-03-11T17:09:32+09:00Minsoo Cheongserver : maintain chat completion id for streaming responses (#5988)
663fa8a809a9119411bc9c3f00026550d0343f4d9b7bcebd7dbf62fd7b293d5ed089023e4e733269c712024-03-10T18:17:47+01:00Pierrick Hymbertserver: ci: windows build and tests (#5968)
664621e86b331f8b0e71f79fd82a4ae1cd54c3e439677d1ac7e00bf049b9f2bba1b5a310a78318c49c42024-03-09T23:41:49+01:00Pierrick Hymbertserver: benchmark: chat/completions scenario and other llm servers comparison (#5941)
66577d1ac7e00bf049b9f2bba1b5a310a78318c49c4d894f352bf433157232dc8dc54eacd50014e898e2024-03-09T22:04:00+02:00Georgi Gerganovserver : print chat template info
666d894f352bf433157232dc8dc54eacd50014e898e098dbaab449f5309a54871ba7e5acef72ae696de2024-03-09T19:55:54+01:00slarenperplexity : support using multiple sequences to allow larger batch sizes (#5946)
66758308a0ecce7cc261b802f4803c38d420063db215b09797321430f08caf0473143a962916ab2ea892024-03-09T17:34:15+02:00Georgi Gerganovserver : fix metrics init (#5964)
6685b09797321430f08caf0473143a962916ab2ea8997c09585d65a95864773b4d25d66d0f708baf38d2024-03-09T15:53:59+02:00Georgi Gerganovggml : remove old quantization functions (#5942)
66997c09585d65a95864773b4d25d66d0f708baf38dfb215c3832236fec7380c4fb618bd7154cb196ef2024-03-09T15:47:47+02:00Georgi Gerganovserver : clarify some items in the readme (#5957)
670fb215c3832236fec7380c4fb618bd7154cb196ef2c4f566c88322ebf2f9bd11b01b5ebdaa0130b892024-03-09T21:27:58+09:00SeungWon Jeongserver : normalize embeddings (#5956)
6710db32beaf09d90b8959d3d0cc493ed1e456853538a3012a4ad08112bb3dc3f1399afec4e93780c442024-03-09T11:16:53ZAlexey Parfenovserver : fix passing prompt as tokens (#5955)
6729674aaf35cb81478eb38c3f3ebde713ec72fbb79950ba1ab84db199f0bbdecdb2bb911f35261b3212024-03-09T12:34:18+02:00Georgi Gerganovserver : simplify logic for empty prompts (#5953)
673950ba1ab84db199f0bbdecdb2bb911f35261b321e1fa9569ba8ce276bc7801a3cebdcf8b1aa116ea2024-03-09T11:27:53+01:00Xuan Son NguyenServer: reorganize some http logic (#5939)
674e1fa9569ba8ce276bc7801a3cebdcf8b1aa116eafd72d2d2a5e79d61ccef6af3d15f16e5e5cbc3522024-03-09T02:57:09-07:00Gabe Goodhartserver : add SSL support (#5926)
675fd72d2d2a5e79d61ccef6af3d15f16e5e5cbc352c2101a2e909ac7c08976d414e64e96c90ee5fa9e2024-03-09T10:30:04+01:00Pierrick Hymbertserver: tests: add truncated prompt tests, better kv cache size (#5933)
676c2101a2e909ac7c08976d414e64e96c90ee5fa9e515f7d0d4fce41c752fc253acf30707c3be2531e2024-03-08T17:31:00-05:00compiladellama : support Mamba Selective State Space Models (#5328)
67776e868821a94072fbc87cb1fcca291694319eae8e457fb3540e0aaec47cfde0abf784c213f9216ee2024-03-08T12:25:04+01:00Pierrick Hymbertserver: metrics: add llamacpp:prompt_seconds_total and llamacpp:tokens_predicted_seconds_total, reset bucket only on /metrics. Fix values cast to int. Add Process-Start-Time-Unix header. (#5937)
678af37fd8b30e37ccbffdd82e6f48559e2fb7ce7dd581ed5c4fe3a8909aaa8313633ac443f471ba7552024-03-08T12:40:02+02:00Georgi Gerganovserver : fix EOS token detection with disabled cache (#5938)
67955a2a900ff4a02fc33708ac7858d595d289a3f2a2002bc96bf2cbf5ab981a17d7e994d817c9801f52024-03-07T19:42:39+09:00Minsoo Cheongserver : add `/v1/completions` endpoint (#5914)
6802002bc96bf2cbf5ab981a17d7e994d817c9801f5ceca1aef0738b57951cd12c603c3477e75312dec2024-03-07T11:41:53+02:00Georgi Gerganovserver : refactor (#5882)
68121b08674331e1ea1b599f17c5ca91f0ed173be316a87ac3a52668e117d97bcea07b529c93188b3032024-03-05T16:08:35+08:00Neo Zhang Jianyu[SYCL] fix mul_mat fault in CI/unit-test (#5862)
68229ae62d2ae163e2b68aa0ad3bf2ab4636de0c957e0843afe1b37890b631bc7d3d2da2ed36c862b912024-03-04T22:31:20+02:00Georgi Gerganovllama : fix embeddings (#5796)
683231ae28f078c3148d097b301f2145f1e3e816cc1475df1d6cf817060028d3ff763cb8097d4ec40d62024-03-03T12:44:03+02:00Georgi Gerganovreadme : add API changes section
684e6029348e86c3810d4435faee54ba822cb43e2ef8ef969afcec1645d2d9c3ab1fc82263bba9689892024-03-03T09:35:23+01:00Pierrick Hymbertci : schedule slow server tests only on Release or on demand (#5839)
6858ef969afcec1645d2d9c3ab1fc82263bba968989fa974646e1a2024fc7dc9e6f27cf1f2f5d4a37632024-03-03T08:48:36+01:00Pierrick Hymbertserver : init http requests thread pool with --parallel if set (#5836)
6869731134296af3a6839cd682e51d9c2109a871de54a6e2d6142ab815c964924896891e9ab3e0506322024-03-02T22:00:14+01:00Pierrick Hymbertserver: tests: passkey challenge / self-extend with context shift demo (#5832)
68738d16b142624bdd7c41d9955752b7f7b59c5e048c2224f003bf9cf558b1a3c57033563e11a4de9a52024-03-01T20:00:58+02:00Georgi Gerganovserver : remove api_like_OAI.py proxy script (#5808)
6885cb02b4a012bb16c6c699c0c62c05ffa653eee0f6ea0f010ff6967034528d9e0b8330b9b0f0b7c132024-03-01T10:08:08+01:00Pierrick Hymbertserver: allow to override threads server pool with --threads-http (#5794)
689f105471ef6aa4727afac8240da398590d7277f4538d152160898b0173ffe4dc7df5daadcbd2eceb02024-03-01T09:59:43+02:00Georgi Gerganovserver : fix newlines in help (#5785)
690052051d8ae4639a1c3c61e7da3237bcc572469d4d5ab29757ebc59a30f03e408294ec20628a6374e2024-02-29T21:42:11+01:00Xuan Son NguyenServer: normalize naming (#5779)
69108c5ee87e4cceb603ecceac90734fcdade57311b78aacf36344df724cdca9f1e1af849b2d2519cb82024-02-28T18:43:38+02:00Georgi Gerganovllama : remove deprecated API (#5770)
692a693bea1e6762a17b78b6ddf4611e54136941ea2adcb12a9bad87bc96f2f158c95892b3d04aa7ffb2024-02-28T09:55:37+01:00Xuan Son Nguyenserver : hit Ctrl+C twice to exit (#5734)
693b11a93df41921846a10628a7c306d5c82a549939a33e6a0d2a66104ea9a906bdbf8a94d050189d912024-02-26T23:15:48+01:00Xuan Son Nguyenfix server hangs on empty prompt (#5733)
6944804215cb833841ffb15a710a16b77ca0a29eb4b8a533f0d9078396ebaee9ba213038a1322976dee2024-02-26T11:41:34+01:00Pierrick Hymbertserver: CI fix trailing space (#5728)
6958a533f0d9078396ebaee9ba213038a1322976dee269de86ba073b5dc9ce687c11a3bc4d7d873b9622024-02-26T09:56:10+01:00Pierrick Hymbertserver: CI tests reduce build matrix (#5725)
696e3965cf35aac00d4e24998c8a3d0093ae1d98bd38b350356b28f782deab63d8b0e9ae103ceb25fcd2024-02-25T22:48:33+01:00Pierrick Hymbertserver: tests - slow inference causes timeout on the CI (#5715)
6978b350356b28f782deab63d8b0e9ae103ceb25fcdbf08e00643fd529f748f0a858fd79f3061e3fa182024-02-25T21:46:29+01:00Pierrick Hymbertserver: docs - refresh and tease a little bit more the http server (#5718)
698f7625019c51ca437a5840576d92362cfa710e4a2abbabc5e51d0d4656b438aec10b7fae9479ef37d2024-02-25T13:43:50-05:00compiladeserver : fix crash when system prompt is bigger than batch size (#5714)
699930b1780269a69948d106e2d1b838ab7661f679ad52d7819b8ced70c642a88a59da8c78208dc58ec2024-02-25T13:50:32+01:00Pierrick Hymbertserver: logs - unified format and --log-format option (#5700)
700d52d7819b8ced70c642a88a59da8c78208dc58ec12894088170f62e4cad4f8d6a3043c185b414bab2024-02-25T13:49:43+01:00Pierrick Hymbertserver: concurrency fix + monitoring - add /metrics prometheus compatible endpoint (#5708)
7019e359a4f47c1b2dceb99e29706c9f7403d32ab5e4c4cb30736582cacb1a164a9d4bc8e17b1014be72024-02-24T19:16:04+01:00Pierrick Hymbertserver: continue to update other slots on embedding concurrent request (#5699)
702525213d2f5da1eaf4b922b6b792cb52b2c613368fd43d66f46ee3b5345fb8a74a252d86ccd34a4092024-02-24T12:28:55+01:00Pierrick Hymbertserver: init functional tests (#5566)
703fd43d66f46ee3b5345fb8a74a252d86ccd34a40954fbcd2ce6c48c9e22eca6fbf9e53fb68c3e72ea2024-02-23T19:31:54ZAlpinDaleserver : add KV cache quantization options (#5684)
704a46f50747b2028f7f9c9883b26bfba12bf92556ec5688c6250430d2b8e0259efcf26c16dfa4c1f462024-02-22T09:33:24+01:00Xuan Son Nguyenserver : fallback to chatml, add AlphaMonarch chat template (#5628)
705c5688c6250430d2b8e0259efcf26c16dfa4c1f464ef245a92a968ba0f18a5adfd41e51980ce4fdf52024-02-22T08:27:32ZAlexey Parfenovserver : clarify some params in the docs (#5640)
7061ecea255ebb70750b52688393f37a63606b90e3fa00a35cef93e057eace8351a667d14d152a91ebc2024-02-21T15:47:48+01:00Pierrick Hymbertserver: health: fix race condition on slots data using tasks queue (#5634)
7076560bed3f066c876682464762cad90f1e28e3f1b06bf2cf8c406e6b70dbf9b431a02fa0ad845b9df2024-02-20T11:07:22-08:00CJ Paisserver : support llava 1.6 (#5553)
7089c405c9f9a7cfd23511fd6b2de05dc72481119b45207b3fbc500f89dfe528693e96540956dbaed962024-02-20T15:58:27+01:00Xuan Son NguyenServer: use llama_chat_apply_template (#5593)
709c0a8c6db371cb3e4379900867b948879f5842201b9111bd209c7b11b0592450a6ed2e0ca545b2c842024-02-20T08:48:19+01:00Pierrick Hymbertserver : health endpoint configurable failure on no slot (#5594)
7105ee99c32f5e47c8d32634eff9a47fb32a24c276bc145f8a132b2fe1d1e65987faddbd9a40bef7a122024-02-18T11:11:16-08:00Robey Holderithcommon, server : surface min_keep as its own parameter (#5567)
711c145f8a132b2fe1d1e65987faddbd9a40bef7a12689a091bbe0537ee9abff3e15a1d74f5f35611652024-02-18T18:39:57+01:00Pierrick Hymbertserver : slots monitoring endpoint (#5550)
712e75c6279d1c8e7abb82a331f5de7124eed402de236376abe05a12a8cb3af548a4af9b8d0e2e695972024-02-18T17:31:28+01:00Pierrick Hymbertserver : enhanced health endpoint (#5548)
71336376abe05a12a8cb3af548a4af9b8d0e2e6959766c1968f7a2e895675425e875b6589f1233a1b522024-02-18T17:30:09+01:00Pierrick Hymbertserver : --n-predict option document and cap to max value (#5549)
71466c1968f7a2e895675425e875b6589f1233a1b521dcc3fde004787e6fc4d84c9de0bb34cd2901a3e2024-02-18T08:23:16-08:00Daniel Hiltgenserver : graceful server shutdown (#5244)
7155f5808ca7b7f23a1fa7a77241842bb84a0e55108f486f6e1e5e9d01603d9325ab3e05f1edb362a952024-02-16T11:00:56+01:00Rőczey Barnabásserver : fix system prompt cli (#5516)
716f486f6e1e5e9d01603d9325ab3e05f1edb362a9560ed04cf82dc91ade725dd7ad53f0ee81f76eccf2024-02-16T01:31:07-08:00bmwlggml : add numa options (#5377)
7170d4177126b0556e202efb85bf3f768be810764007930a8a6e89a04c77c51e3ae5dc1cd8e845b6b8f2024-02-15T09:01:57+01:00Elbiosllava : fix memory management bug (#5491)
718aa2341298924ac89778252015efcb792f2df1e20f5ca054855dea83f424003162f26de376e5643f62024-02-14T08:38:35+01:00Johnllava : support v1.6 (#5267)
719684780141a08200ec98eba3e982dbafd1d0b500085910c5b30f6e268321be8df044f5528a6efac522024-02-11T13:38:14ZAlexey Parfenovserver : allow to specify tokens as strings in logit_bias (#5003)
720907e08c1109f498b01036367804cff3082c44524f026f8120f97090d34a52b3dc023c82e0ede3f7d2024-02-11T11:16:22+01:00Xuan Son Nguyenserver : add llama2 chat template (#5425)
7217c777fcd5dd4af7079e33390cf6a19c328a2666fe5ca3937c685d6e012ac4db40555d6ec100ff03c2024-02-09T02:49:49-08:00Riley Stewartserver : fix prompt caching for repeated prompts (#5420)
7220ef46da632c32faa1a538e5dc180994e8bbb46e1ee1628bdfea8b0079fed0140ac2f00ef1b465b572024-02-07T02:17:25-06:00Xiao-Yong Jinllava-cli : always tokenize special tokens (#5382)
723213d1439fadefe182f69c5f7e8dd3b4b6572ebcb17c97fb0620448b37516a3f53fea6c482b0a30a42024-02-06T18:08:38ZAlexey Parfenovserver : remove model.json endpoint (#5371)
72431e790322133a4b1d0684527ea446e765e8a96cf4ffc7a17d4e80c5f3f905139cb570ed9b6934fcb2024-02-06T04:20:00-05:00Michael Coppolaserver : add `dynatemp_range` and `dynatemp_exponent` (#5352)
7254ffc7a17d4e80c5f3f905139cb570ed9b6934fcb906cff55c2848fda091d888a1585915ec0c9ea9e2024-02-06T08:16:23ZNiall Coatesserver : various fixes for the prompt field in /completion (#5300)
726a2d60c9158435ae9a6f14632f07f1acf7a3becefe6f81775323f6f4e4a30abf022a6028fa86b79ac2024-02-05T08:10:22ZAlexey Parfenovserver : allow to get default generation settings for completion (#5307)
7274be04c8965578edc09194fab769b4b922b8444f55d55b0cd827bb0fcfedfa329a82bd5d6ef2c93ca2024-02-05T10:43:57+03:00Нияз Гарифзяновscripts : add non-interactive server-llm.sh (#5303)
728e437b37fd0b2b97e6c6ff1045ec7f901faa6498a2d40085c26794e29c434480b9e06738e89e5686f2024-02-02T14:23:40+02:00Georgi Gerganovscripts : parse wtype in server-llm.sh (#5167)
7296b91b1e0a92ac2e4e269eec6361ca53a61ced6c6e805f0fa9951081ce0a86378a7aa52b6f636b82d2024-02-02T08:56:31+01:00Xuan Son Nguyendocker : add build for SYCL, Vulkan + update readme (#5228)
7305cb04dbc16d1da38c8fdcc0111b40e67d00dd1c3efb7bdbbd061d087c788598b97992c653f992ddd2024-01-31T17:30:17+02:00Georgi Gerganovllama : remove LLAMA_MAX_DEVICES and LLAMA_SUPPORTS_GPU_OFFLOAD (#5240)
731e6f291d15844398f8326940fe5ad7f2e02b5aa564003be0e5feef320f3707786f22722b73cff93562024-01-30T20:17:30+02:00Georgi Gerganovserver : fix context shift (#5195)
7324003be0e5feef320f3707786f22722b73cff9356fea4fd4ba7f6b754ac795387b275e1a014a77bde2024-01-30T12:15:05-06:00JohnnyBserver : change deps.sh xxd files to string literals (#5221)
733813416991ab0d1caa0d12f93ac4e8a24a2add0a35589921ef84a4fb1c6d1c9c34d626a5a83033db62024-01-30T10:18:02+01:00divinity76main : allow empty --prompt-cache file (#5176)
7346685cc41c237544623b4b5651eceb9c4280728ccceebbb5b21b971941b2533210b74bf359981006c2024-01-30T17:11:46+08:00Wu Jian Pingserver : improve README (#5209)
735c82d18e863fcde91b4b1109b1d0c73ea4470c40514fef85e2d5361e6b4dd7a9ecf22bb817362997d2024-01-29T21:48:10+08:00Wu Jian Pingserver : embeddings compatibility for OpenAI (#5190)
7362307523d322af762ae06648b29ec5a9eb1c730320f648573dde61c510560f68244f70ece7e60d8c12024-01-28T18:03:59+01:000cc4mggml : add Vulkan backend (#2059)
73739baaf55a160909bb9428bd981014218761a20cb6db2b41a76ee78d5efdd5c3cddd5d7ad3f6468552024-01-28T01:55:31-06:00Kyle Misteledocker : add server-first container images (#5157)
738ec903c034131848da9222536ff18da07ec0882a0a1d6df129bcd3d42cda38c09217d8d4ec4ea3bdd2024-01-27T14:38:05+01:00Maximilian Winterserver : add self-extend support (#5104)
73948c857aa10aea73210a4a72da3f1a6f99269e75d413e7b0559f922bd4de5e9eec548829d111651b12024-01-26T13:42:20+01:00Xuan Son Nguyenserver : refactored the task processing logic (#5065)
7402bed4aa3f37cb4e39e16e9ec7b595a7738fd5faf125d03a5036a02a983c8e98c2cdc126e061afb8e2024-01-23T08:11:39+01:00Xuan Son Nguyendevops : add intel oneapi dockerfile (#5068)
741125d03a5036a02a983c8e98c2cdc126e061afb8e011e8ec577fd135cbc02993d3ea9840c516d6a1c2024-01-23T01:51:27-05:00Michael Coppolallama.vim : added api key support (#5090)
742780e24a22eb595b705cbe8284771e9ceff1c4dd23ce7e8f8e7ccfce07e5947ac5f1f3f4628cf68ea2024-01-22T21:15:08+08:00Reinforce-IIggml : parallelize FP32 conversion when using BLAS (#5045)
7437dcbe39d36b76389f6c5cd3b151928472b7e22ff726c0fa9a2da976e9c5d5c51e185d9dd453fc9e52024-01-21T14:42:44+02:00KawrakowAdd ability to evauate multiple choice tasks (#5047)
7447051aacfac0057fa5fac9ea46c55bffc3892d8102b3b999cacc7ad1207c32fbdf3479a19c06e1a342024-01-19T11:39:11+02:00Kawrakowwinogrande: evaluate log-probs in parallel (#5036)
7453e945cc1e9c06d2001031360e4e303e9548fb02cad19812cda4062c9f154ef16315df41fbe6a770a2024-01-18T19:18:21+02:00KawrakowHellaSwag: speed up by parallelizing log-prob evaluation (#5020)
746ad19812cda4062c9f154ef16315df41fbe6a770a682986a08eb5cb04865d2e713449f17304d266d82024-01-18T15:33:01+02:00Georgi Gerganovperplexity : faster HellaSwag via batching (#5017)
7474feb4b33eeb1756e46084a4db9230b279af1a480959ef0c0df725c013c7f712eaa7790b8e38a8e202024-01-16T18:41:42+01:00Maximilian Winterexamples : add complete parallel function calling example (#4974)
7484be5ef556de830c5c4f6e45c05ef4427823fe6070ea069b87bd296c556824e57455433b6c03573402024-01-13T20:45:45+02:00Georgi Gerganovmetal : remove old API (#4919)
7490ea069b87bd296c556824e57455433b6c0357340f172de03f11465dc6c5a0fc3a22f8ec254c6832c2024-01-13T19:31:26+02:00Georgi Gerganovserver : fix prompt caching with system prompt (#4914)
750356327feb3f66980ab687040495d722696d98970ee8243adaa9a9f51ff449213383874e49efe368f2024-01-13T09:20:46-05:00Ziad Ben Hadj-Alouaneserver : fix deadlock that occurs in multi-prompt scenarios (#4905)
751ee8243adaa9a9f51ff449213383874e49efe368f15ebe59210e7fd9817ff67f51fa1a5ee2d0042942024-01-13T14:16:11Zmakomkserver : fix crash with multimodal models without BOS token (#4904)
752e7e4df031b9e29d4b55a4e0b0295187f6b213db1584d674be622fbf1578694ada6e62eebedbfd3772024-01-12T20:07:38+01:00slarenllama : ggml-backend integration (#4766)
7531d118386fea031f01550f8cd47a5c86296e5333f7edefbd79cc6dea96640edc54c6b94b2b2496d8b2024-01-11T23:23:49+02:00Georgi Gerganovserver : fix infill when prompt is empty (#4833)
7544330bd83feb39683de4bd7a34cfcf672ff8ac3e427379455c38cb13f24de92dbd6fcdd04eeb1b9d92024-01-11T19:02:48+01:00Lauraserver : implement credentialed CORS (#4514)
75527379455c38cb13f24de92dbd6fcdd04eeb1b9d9eab67950068e4b125007d027232c47d2a5831cd02024-01-11T12:51:17-05:00Michael Coppolaserver : support for multiple api keys (#4864)
756eab67950068e4b125007d027232c47d2a5831cd0d8d90aa343c22fe01429d3540e47ded87e9dcb9d2024-01-11T12:41:39-05:00Behnam Mserver : add `LOG_INFO` when model is successfully loaded (#4881)
75743f76bf1c362c067fce46bb8dcda0b64af8a95332f043328e3116724d15b915b5c6078e2df860a692024-01-11T16:14:52Zpudepiedjmain : print total token count and tokens consumed so far (#4874)
7582f043328e3116724d15b915b5c6078e2df860a692a7c94db5fb67b2f8882d2d16a11bf5d8d12d3972024-01-11T09:33:26-05:00Isaac McFadyenserver : fix typo in model name (#4876)
7597a9f75c38b5e62fe27b8a5a3ed823b4a3714024b5c1980d8d4c4e0c0af77359f81cc44d90b3f250b2024-01-11T02:12:05-05:00Behnam Mserver : update readme to document the new `/health` endpoint (#4866)
7605c1980d8d4c4e0c0af77359f81cc44d90b3f250bcd108e641dbdedd8c5641c4cec1762f751f381362024-01-11T09:10:34+02:00Georgi Gerganovserver : fix build + rename enums (#4870)
761cd108e641dbdedd8c5641c4cec1762f751f3813657d016ba2d46a6e22517a31a75cebb48f9e234b62024-01-10T14:56:05-05:00Behnam Mserver : add a `/health` endpoint (#4860)
762128de3585b0f58b1e562733448fc00109f23a95d8c5833031857c9e9ada61948bae894ab9c785f862024-01-09T05:02:05-05:00Behnam Mserver : update readme about token probs (#4777)
7638c5833031857c9e9ada61948bae894ab9c785f8618c2e1752c3b387689e9e73d7d8a1a3b1511ce232024-01-09T10:12:43+01:00Zsapiserver : add api-key flag to documentation (#4832)
76467984921a70a7e680a24494aeb7575a66e90685dc75ca5d96f902564cbbbdd7f5cade80d53c288bb2024-01-07T08:45:26+02:00Georgi Gerganovserver : fix n_predict check (#4798)
765e5804313a1edaf00726ed0b96ecced07accbf50cdc891b7f7a23158d54f9383790b92c79cc5906c12024-01-04T03:17:09-05:00Michael Coppolaserver : fix options in README.md (#4765)
766f2eb19bd8bc9f5730d6e05d7a52a9e19bf5ac099f3f62f0d835d559e80714bbeb05d03125574e3dd2024-01-03T03:43:19-05:00Justin Parkerserver : throw an error when `slot unavailable` (#4741)
767f3f62f0d835d559e80714bbeb05d03125574e3dd0ef3ca2ac62016c0c545de1c89dc2e3e130f4a992024-01-02T21:07:47+02:00Georgi Gerganovmetal : optimize ggml_mul_mat_id (faster Mixtral PP) (#4725)
7680ef3ca2ac62016c0c545de1c89dc2e3e130f4a99540938f8904707dd74cb3be18495f853b312e72f2024-01-02T15:48:49ZPhil Hserver : add token counts to html footer (#4738)
7690040d42eeb237197054cc7790df5776eacfa608e83e633c27efdf0eb0ba54249e784b0ea760b10072024-01-02T06:15:16-08:00Marcus Dunnllama : replace all API facing `int`'s with `int32_t` (#4577)
7705d7002d4372ebf107cfaf46fcd90df27b204f33026f3071d714f0b27ad7f021a46a66a10854802582024-01-02T04:38:15-06:00minarchistserver : add --override-kv parameter (#4710)
77158ba655af054715c0516ee270ad028ad9e74f357edd1ab7bc34c10a780ee7f9a4499f7689cdad36d2024-01-02T10:57:44+02:00Georgi Gerganovmetal : enable shader debugging (cmake option) (#4705)
7729fbda719de18a9400a064c28759c39d55d687d3e39d8bc71edcb8b6f99d46fa4216af7a15232e2182023-12-30T23:24:42+02:00Georgi Gerganovclip : refactor + bug fixes (#4696)
77324a447e20af425fa44cf10feaa632b6bb596c80fa20f3c7465d6d1b33767757c2760643b799a81bf2023-12-30T15:07:48+07:00automaticcatggml : add ggml_cpu_has_avx_vnni() (#4589)
774db49ff8ed7f0bb201176703441cc02911b08ef2a60f55e888c29cbd87c4238dd19e85d0eef87245d2023-12-29T06:24:12-08:00Justine Tunneyserver : replace sleep with condition variables (#4673)
77560f55e888c29cbd87c4238dd19e85d0eef87245db93edd22f55d3e5268263c3edcdae1818505c0782023-12-29T22:22:44+08:00SakuraUmiserver : fix OpenAI server sampling w.r.t. penalty. (#4675)
776b93edd22f55d3e5268263c3edcdae1818505c07882d6eab224862a7044069fb9211dc4b29124264b2023-12-29T06:22:10-08:00Karthik Sethuramanserver : allow to generate multimodal embeddings (#4681)
777441f51dca004debf8b275f1bdc08e0f1af7fd8f838b3de4658292582a8941a2be5c77b40ce6ac0f22023-12-29T19:23:27+09:00Tamotsu Takahashici : build with CLBlast + ggml-opencl use GGML_API (whisper/1576)
77865e5f6dadbba4b496bba27f573e473c66b446496ea5497df5d138c83b2b0ca70aefdc4b1175c10012023-12-28T11:20:00-08:00Justine TunneyFix OpenAI server sampling w.r.t. temp and seed (#4668)
7796123979952385847d8348e295d77d6e01da8aa84b9ec82d262cb20d7f0a8a1157bfa9aace40e26252023-12-23T09:31:49ZAlexey Parfenovserver : allow to specify custom prompt for penalty calculation (#3727)
78031f27758faf4a4bd08101a57c7ec3a473f771f8656fa50819f7a3ca2128f63b81c17c08a4454479e2023-12-21T11:57:48-08:00Marcus Dunnllama : allow getting n_batch from llama_context in c api (#4540)
7812994f0c5a2e8c96955b422dedc93ec2595d16b82b1306c439490c7fa4ec33594500d980d1e9e15e62023-12-17T19:39:02-05:00Jared Van Borteldecode : fix logits_valid for legacy API (#4516)
7820ffc92d2d23a789625f018840469af045be1e3c08edd2b40fdbcafbf630f2cf29306b29d5cb48c422023-12-17T17:02:16+02:00olexiybserver : disable llm logs if SERVER_VERBOSE is off (#3792)
7838edd2b40fdbcafbf630f2cf29306b29d5cb48c42eb16dae7e70ca97396190698b29c0f9ee3388e882023-12-17T15:57:56+01:00AdithyanIserver : fix grammar being ignored (#4494)
784eb16dae7e70ca97396190698b29c0f9ee3388e8862bd52b7bf90819e75f427a95a484cd5eee0b3c72023-12-17T14:56:09ZAlexey Parfenovserver : fix possible ambiguity in content type charset (#4501)
78562bd52b7bf90819e75f427a95a484cd5eee0b3c75daa5f54fdcd2b5228add1a4c43a1897b2168f352023-12-17T15:54:37+01:00mzcuserver : allow requests larger than 8K (#4500)
78688ae8952b65cbf32eb1f5703681ea592e510e570ee4725a686643669a8587142fa068cbf29de3ce22023-12-15T13:49:01+02:00ShadovvBeastserver : add optional API Key Authentication example (#4441)
787948ff137ec37f1ec74c02905917fa0afc9b975144d98d9a65665eee3838cef936641f640e3f5b6492023-12-13T23:57:15+04:00shibe2server : fix handling of characters that span multiple tokens when streaming (#4446)
788fecac45658a99eddc4d6e36ba0310ca8f87a77f09494d7c4774ab745490b5a19570ff7747a1941432023-12-12T04:12:35-06:00kalomazeserver : tweak default sampling parameters (#4367)
789d9d4cfef64ea416dd66632173787d03ffb180cc741a11aaf99feff4901e4c8dc48ad00766c5da4e92023-12-12T11:25:29+02:00Vladimir Zorinserver : fix local model name in server (#4420)
790bcc0eb4591bec5ec02fad3f2bdcb1b265052ea5681bc9214a389362010f7a57f4cbc30e5f83a2d282023-12-07T13:03:17+02:00Georgi Gerganovllama : per-layer KV cache + quantum K cache (#4309)
79105cd6e5036d72d0930de4d8f6be7bce09e8dda24caa9249217c5fd524b900add5ddcbeaa20cbcb122023-12-06T20:21:59+02:00Georgi Gerganovserver : recognize cache_prompt parameter in OAI API (#4347)
79223b5e12eb5a76489b4c3ee22213a081da68b1809d208995c6da66f252d4054c1c5a90eb8ccb7a2f72023-12-04T17:04:21+01:00Daniel Beveniussimple : update error message for KV cache check (#4324)
79333e171d1e9fc4903f9314b490d77fb8d58331b636949b50df56ee58a2d76d45487942cb211c086292023-12-03T01:10:43-08:00Ed Leeserver : fix OpenAI API `stop` field to be optional (#4299)
7946949b50df56ee58a2d76d45487942cb211c08629d7b800b8bc490a221acbd83c575206a907f2f6e22023-12-03T10:03:25+01:00Rickard Edénpy : add grammar to oai like api (#4294)
7951d144112c0fbbb4ecc07dbcf4f05a380148bd6def43f09366dfd018e4568e23a232aaa8c4f7cfc782023-11-30T17:25:49-05:00Ziad Ben Hadj-Alouaneserver : add --log-disable to disable logging to file (#4260)
796f43f09366dfd018e4568e23a232aaa8c4f7cfc78d2809a3ba2780e00fce5a6149a7eda09f1c0e9062023-11-30T17:25:04-05:00Ziad Ben Hadj-Alouaneserver : add single-client multi-prompt support (#4232)
797e2bd725f4b39bc5c6234858d158e01248f5ab5bd1f5cd83275fabb43f2ae92c30033b384a3eb37b42023-11-30T20:50:40Zrhjdvsgsgkspy : fix oai proxy (#3972)
798af19d3573481d409b3c4e55494810eb1f65a9aaee9c13ff78114af6fc6a4f27cc8dcdda0f3d389fb2023-11-25T11:29:06+02:00Georgi Gerganovserver : OAI API compatibility (#4198)
7996b0a7420d03b9d13cb0e9439a01ce8476d8bf093d103d935c0e75769a6a597f7a64cab72c6cc3e792023-11-23T19:07:56+02:00Georgi Gerganovllama : KV cache view API + better KV cache management (#4170)
8009d5949f04b1f8b76184818abbd99938c2020803fff8238f71d56245f4a6dbea693f4ebd81263464d2023-11-23T12:34:20+01:00Daniel Beveniusexamples : fix typo in parallel example doc comment (#4181)
801936c79b2275a8f15f3512e63de615c676904d650262005ad9ded375e9c544a02c18ef6254fe185a22023-11-19T11:54:10-05:00SoftwareRendererserver : relay error messages (#4131)
8029e87ef60e18d69338c5efea314aa7e718bf2040ac7cce1246e248124117ae5bc058923e3ade95f112023-11-17T16:24:07+01:00Jannis Schönlebercommon : improve yaml log escaping (#4080)
8034760e7cc0b68570d58f55e8dda469805d1759d0dbb50a792ec2a49944470c82694fa364345e951702023-11-13T14:16:23+02:00Georgi Gerganovsync : ggml (backend v2) (#3912)
804532dd74e38c29e16ea1cfc4e7eedb4f2fab3f3cde86fc56f7521ca4b18d1d9939e82abd40c2f1c012023-11-11T22:04:58-08:00Richard KissFix some documentation typos/grammar mistakes (#4032)
805d96ca7ded77df764db797b68b4a29e34c5b5628534b0a082074b073eb14c2bd93c0c070e20ddcd162023-11-11T05:48:21ZAlexey Parfenovserver : fix crash when prompt exceeds context size (#3996)
8064a4fd3eefad5bd17ab6bcd8e2181b4f62eae76cfdf9d1293defe783f42bc83af732d3c670552c5412023-11-11T06:49:33+08:00Jhen-Jie Hongserver : allow continue edit on completion mode (#3950)
80757ad015dc3011b046ed5a23186c86ea55f987c54875fb42871a0f5a88fbe31a0b5edd697b84038e42023-11-09T04:00:34+02:00Mihaiserver : add min_p param (#3877)
808381efbf480959bb6d1e247a8b0c2328f22e350f82833a6f63c1b87c7f4ac574bcf7a15a2f3bf3ede2023-11-06T22:36:23+01:00Damian Stewartllava : expose as a shared library for downstream projects (#3613)
809bb60fd0bf6bb270744d86dd45b3a95af01b7de45132d25b8a62ea084447e0014a0112c1b371fb3f82023-11-05T23:15:27+07:00Thái Hoàng Tâmserver : fix typo for --alias shortcut from -m to -a (#3958)
81050337961a678fce4081554b24e56e86b676601630e40806c1cb3bdf9955ed807ffbe212be85b4c672023-11-01T20:11:02+02:00Georgi Gerganovllm : add llm_build_context (#3881)
811f0e209324a7f663225791897877bf610f1af152dca190bca8e844d171020d6147687e71472d717342023-11-01T11:29:07+02:00Georgi Gerganovscripts : add server-llm.sh (#3868)
812ca190bca8e844d171020d6147687e71472d7173471e3718abdb2771b50c9606d3a7569623a0b0afe2023-11-01T09:28:28ZAdrian Heskethserver : re-enable completion and embedded at the same time (#3876)
81382a6646e0221216c41edcdf99f5a44bb051391f5ba231e8a6dd8ad82acfe0e4d492ff7cef6b3f0a12023-10-28T15:43:01+03:00Aarni Koskelametal : try cwd for ggml-metal.metal if bundle lookup fails (#3793)
81434b2a5e1ee4fe6295fb4420eb91131d743694c656961c4bd0b5176e10ab03b35394f1e9eab7617922023-10-26T22:53:37+03:00Georgi Gerganovserver : do not release slot on image input (#3798)
815ad939626577cd25b462e8026cc543efb715284721717521cdb976a2219888b0e5cba36e210eee9df2023-10-24T16:10:43-04:00cebtenzzreserver : add parameter -tb N, --threads-batch N (#3584) (#3768)
8161717521cdb976a2219888b0e5cba36e210eee9dfb2f7e04bd312eaf97eee0523aa09d950d585626b2023-10-24T23:08:20+03:00Georgi Gerganovserver : do not block system prompt update (#3767)
8175be6c803fa5378f62a1590f3ad8c6b64c7c0c2ce6336701c9378c23c85d1c0e464b663ca2bbb8e602023-10-23T12:40:03-07:00Marcus Dunnllama : remove token functions with `context` args in favor of `model` (#3720)
818438c2ca83045a00ef244093d27e9ed41a8cb4ea99e70cc03229df19ca2d28ce23cc817198f8972782023-10-22T22:53:08+03:00Georgi Gerganovserver : parallel decoding and multimodal (#3677)
819d3956aea53369455008159cc405ed4c49697669222c69a27945e7acf9690dd3210d316f22182751c2023-10-22T20:09:51+02:00vvhg1main : escape prompt for cfg_negative_prompt and consecutive inputs in main with interactive (#3623)
820d1031cf49c3b958b915fd558e23453471c29ac338cf19d60dc93809db8e51fedc811595eed9134c52023-10-20T21:07:23+03:00Georgi Gerganovsampling : refactor init to use llama_sampling_params (#3696)
821a0edf73bda31c7c4e649e6f07c6fd30a729929cdf439e506e8ae8b01df2ae2156380f8156d7553e32023-10-20T13:06:10+03:00Georgi Gerganovserver : fix uninitialized sampling context (close #3685)
8220e89203b517c95ec6675eda75d200a60d1e8921dc67fe68e417f766970fb1feaf2e66458aa24116a2023-10-18T16:21:57+03:00Georgi Gerganovspeculative : add tree-based sampling example (#3624)
8233ad1e3f1a10c1f66b4f1cd7510e0977fadbc0dfd1142013da40e98946a109b141dd858f0ed9960512023-10-17T18:51:02+02:00coezbekserver : documentation of JSON return value of /completion endpoint (#3632)
824b016596d903641f8825cd94bb6742e1de0c210176b3ae4da92485f979a0f45774fcf68597634db0b2023-10-12T15:51:53+09:00Aarni Koskelaserver : add completion mode (no chat) (#3582)
82557dd55e2c742bfc50e0f5c6fb95c14118cff44f6b8fe4b5cc9cb237ca98e5bc51b5d189e3c446d132023-10-12T09:29:04+03:00Georgi Gerganovserver : fix kv cache management (#3588)
826a8bdd65525ae86dea905e9866ad369b53e30ac1470c29da118cdb02bfcbd0376c32b5b2236e48e482023-10-11T15:42:22-04:00Michael Coppolaserver : add parameter -tb N, --threads-batch N (#3584)
82711ea5c7d96f2c28e1c99659e08ec0a44574056e295bd60a0a69f57e9a2ff1269667ea484a1a9bb402023-10-10T09:31:21+02:00vvhg1infill. : fix tokenization (#3508)
8288e6716a102e390e930594d51302730184dac83cc9c38d181d40b9d27f8f42152c18e7c70bfffcf372023-10-08T03:55:58-07:00Ryder Wishartapi_like_OAI.py : compat with Microsoft Guidance (#2746)
8299c38d181d40b9d27f8f42152c18e7c70bfffcf37a1202a31ed8c35705bd09fe91c3e7410c619bd702023-10-08T06:52:57-04:00arcrankapi_like_OAI.py : simplify function (#2796)
830cb13d73a720c42d1958bff79b6869d77b26b8cea9ca79d5cbbc8d43f2bff951404b6a40ff1ee37882023-10-06T21:39:33+03:00Mihaiserver : docs fix default values and add n_probs (#3506)
8319ca79d5cbbc8d43f2bff951404b6a40ff1ee37880c731ca4039ccff86ffab90eaae4ca98037c44962023-10-06T10:10:13-06:00Kerfufflekv cache slot search improvements (#3493)
832a8777ad84e00cda0399e827cdf971e2c3fab1da297af49fa395df77e4c18af0e1655b2fee67c96862023-10-06T14:16:38+01:00pudepiedjparallel : add option to load external prompt file (#3416)
83397af49fa395df77e4c18af0e1655b2fee67c968616820a5a0d885113f21021ce934f0b0027b9d69a2023-10-06T07:44:24-05:00Jhen-Jie Hongserver : reuse llama_sample_token common util (#3494)
834e8b8d32e8663ffc55a02c9721af3a5190382cbb08f3a642ec1d878b2d0a0d15e3a4277f522790d4c2023-10-05T09:02:55-05:00Jhen-Jie Hongserver : fix incorrect num_tokens_predicted (#3480)
835ac2219fef34eb5b713c286c34c6e4162c39c8f3b48be797ffbd80b062f55778e09e97180eb25d2ab2023-10-03T21:04:01+03:00Georgi Gerganovllama : fix session saving/loading (#3400)
83648be797ffbd80b062f55778e09e97180eb25d2abf56e1baec361b5381e32ee6b6e56e4f00e002dfe2023-10-03T10:09:28-07:00Alex Klinkhamerllama : expose model's rope_freq_scale in the API (#3418)
837c97f01c362ac102c6994edb80008f8608539553af5ef5cfb18148131fcf45bdd2331f0db5ab7c3d02023-10-02T09:42:02+02:00vvhg1infill : add new example + extend server API (#3296)
8387f1a0fe709ea1a861da2f3759f58a28bf8953c1216bc66d9479edd5ee12ec734973554d4493c5dfa2023-09-29T03:51:52+08:00Qu Zongfuggml : release the requested thread pool resource (#3292)
8390e76a8992c8200237bbc6471a53fb8796b3872f72db94d98eda56982d80238840b0652b4137a2a842023-09-28T20:40:11+02:00xaedestrain : finetune LORA (#2632)
840ec893798b7a2a803466cc8f063051499ec3d96f745855b3f1c7bdd0320aa632334d0b3e8965c26c42023-09-28T19:04:36+03:00Georgi Gerganovllama : custom attention mask + parallel decoding + no context swaps (#3228)
841dc6897404e141c74cbbf8030ecfebd74e1815411527e57cfd8a9a26bf622c0510c21c2508a24be262023-09-27T17:48:33+02:00Rickard Hallerbäckmetal : reusing llama.cpp logging (#3152)
842be8c9c245bd129ebabb80e0a7a8dd7daeb4d30afbe6beeb8d75294552c4918fce06d7b84eebf3d792023-09-07T15:45:01+02:00Kawrakowmetal : parallel RoPE on Metal (#3024)
843e4386f417faf894f6706eec005e24d142b577fcb35195689cd835464779c247b1c22ab9247418fd12023-09-04T10:28:55+02:00Aarni Koskelaserver : add a subtle loading animation to the edit box (#2466)
844571083f508266c4eb5cb5457d836df5dd3c173cef04d0028444bc9b3d4225fba47e19d4c3aeb37412023-09-02T08:31:46+08:00Jhen-Jie Hongserver : avoid aniprompt in probabilities of final response (#2849)
84544c117f41ee01c5ac8fb86bba041f08d8b87b46d43033b7bb4858da4f591715b3babdf906c9b7cbc2023-08-28T21:51:47+02:00xaedestrain : mem usage and other improvements (#2439)
846c1ac54b77aaba10d029084d152be786102010eb2730d9c681e339b76407659344e5a2cd50af7d7d52023-08-26T16:11:45-07:00Bruce MacDonaldserver : add `/detokenize` endpoint (#2802)
847730d9c681e339b76407659344e5a2cd50af7d7d5c7d92e6dfec3f54849f3a0ba373054d29f321ea22023-08-26T14:13:36-06:00Kerfuffleconvert.py : advanced option (#2753)
8482ba83c8685177faea3399db9564f9c52df75c366bae5c5f679e043371bc2b4dffff8d4964d6cb9532023-08-26T13:45:53+02:00klosaxFix spm whitespaces (#2806)
849bae5c5f679e043371bc2b4dffff8d4964d6cb953232caf3c1581a6cb023571780ff41dc2d66d1ca02023-08-26T10:07:43+02:00lonexamples : skip unnecessary external lib in server README.md how-to (#2804)
85029674ab4e847fcaba60cc6558f0d46d5f74ae2795439a0ab57c16b556ffa91a0953df5e46b1e7fb42023-08-25T18:32:45+08:00Jhen-Jie Hongserver : display token probabilities in the UI (#2489)
851cf658adc832badaaa2ca119fe86070e5a830f8f6a192860cfec89a38d59a943623bf595b1fe4495b2023-08-23T23:08:04+03:00Georgi Gerganovllm : add Falcon support (#2717)
852b8ad1b66b23f9b2e6e4531e9a62753323036a556f5fe98d11bdf9e7797bcfb05c0c3601ffc4b9d262023-08-23T02:12:12-05:00Xiao-Yong Jinserver : allow json array in prompt or content for direct token input (#2306)
85346ef5b5fcf4c366e1fb27726b6394adbbf8fd0eac63bb1d16a70c03440671b76954bb767513cead82023-08-22T23:10:42+02:00goerchllama : fix whitespace escaping in tokenizer (#2724)
854226255b44ef2c2794bfac48d101d35a9c2dbb965930523c8e1cbbee5449c055daa894917fac6805e2023-08-22T08:32:00+08:00Jhen-Jie Hongserver : fallback to default if client param is null (#2688)
8556381d4e110bd0ec02843a60bbeb8b6fc37a9ace9dadbed99e65252d79f81101a392d0d6497b86caa2023-08-21T23:07:43+03:00Georgi Gerganovgguf : new file format with flexible meta data (beta) (#2398)
8561f0bccb27929e261744c979bc75114955da49e98f63564adfaa157ca387071d6b9a06cfaef0ef5762023-08-19T00:45:36+03:00Georgi Gerganovserver : better default prompt (#2646)
857f63564adfaa157ca387071d6b9a06cfaef0ef5762d8b76a110d76ff6b5728ff0af8477531e4db60e2023-08-19T05:41:32+08:00Jhen-Jie Hongserver : update xxd usage for older versions compatibility (#2649)
85810151bee2e38b5711335c4a38f6ca93b50223acf0992a7b8b18a89e29a205efb48ceb559c9a082032023-08-17T23:34:01Zstaviqserver : support for saving templates in browser LocalStorage (#2486)
8593ebb00935f3f0522b75df49c2769ab1774b91380d783f7982e0e823a2626a9956359c0d36c1a7e212023-08-15T06:14:14+08:00Jhen-Jie Hongserver : add missing /json-schema-to-grammar.mjs (#2616)
860d75561df207d22790609ee0ad924302f66ac2599348acf188c9fbe66396990f2dc83229df367969b2023-08-14T15:36:42+02:00Cheng Shaoserver : add --numa support (#2524)
8612feb8934eb75ca63f3c42724229cce1df9579c8e5517d6e69214cdead000a76983b9fe175c3f83292023-08-14T16:20:17+08:00Jhen-Jie Hongserver : fix default grammar by use empty string in the UI (#2604)
8625517d6e69214cdead000a76983b9fe175c3f8329f31b5397143009d682db90fd2a6cde83f1ef00eb2023-08-14T15:16:54+08:00Jhen-Jie Hongserver : implement json-schema-to-grammar.mjs & add grammar param in the UI (#2588)
86353dc399472d5bd35ee739b865e843b1996bd38149ca4abed893685692f90413e4d43153af12342d92023-08-12T06:35:14+08:00Equimserver: fixed wrong variable name in timing json (#2579)
8641638757767072a4957f52b9e3594f0b67610631b916a9acdd0a411426690400ebe2bb7ce840a6bba2023-08-10T12:16:38+02:00Martin KrasserFix grammar-based sampling issue in server (#2566)
865182af739c4ce237f7579facfe8f94dc53a1f573f4329d1acb01c353803a54733b8eef9d93d0b84b22023-08-04T15:00:57-04:00Cebtenzzreserver: regenerate completion.js.hpp (#2515)
8665f631c26794b6371fcf2660e8d0c53494a5575f7415e99fec27be5a2e4283f1937afd17eb33fbd662023-08-04T06:37:24-05:00Stephen NicholsFixing race condition in server and partial stream handling in frontend. (#2391)
867415e99fec27be5a2e4283f1937afd17eb33fbd66ff966e7ca6af127c9405523cdb07ef8fa01bf6d62023-08-04T19:29:52+08:00l3utterflyStream save llama context data to file instead of allocating entire buffer upfront (#2488)
868c574bddb368424b5996cbee2ec45ec050967d40486aeb27734481751592abd85590020b40d9224c82023-08-01T20:54:28+08:00Bono Lvfix a typo in examples/server/README.md (#2478)
86986aeb27734481751592abd85590020b40d9224c81873ff586bd8499a18f763632711bf15d253585e2023-08-01T01:56:23-07:00ebraminioserver : Support dark mode (#2414)
87034ae1caf7fdea4c4c09087002e15e6230102e6a9d91f3f0c55663719ea03b76311e8c36ed55eb0e22023-07-29T03:02:10+09:00nhamanasuexamples : server chat mode with llama2 (#2400)
871d5512b782b27ff698007dcd175da18959d5f163fc798308e3a425eae050a1f249a576fa8c64333272023-07-25T11:36:17+02:00slarenserver: add rms_norm_eps parameter (#2380)
872c798308e3a425eae050a1f249a576fa8c643332741c674161fb2459bdf7806d1eebead15bc5d046e2023-07-25T10:27:34+03:00Henri Vasserman[Server] Escape HTML in webchat (#2368)
873b3f138d05849ccbce67303ac17b50ebbc268128a5b2b2dc6ae8086bff7c9b3c17fb435cf319b71852023-07-24T17:54:22+03:00Aarni KoskelaChat UI extras (#2366)
8744f06592cc6b83979e4b442e8cb97b3948c85718870d26ac3883009946e737525506fa88f527271322023-07-23T17:31:17-03:00IgnacioFDMAdd gqa parameter support to the server (#2351)
875355c80f49e32b0b15c0a457f3bad380e57f5b9ac83a00ce69bef9124c0702424a012ea799128b77d2023-07-23T06:16:48-05:00AustinMrozexamples : simplify vim plugin (#2327)
87624baa54ac1ff3d4156a2360deb1473af04a9b1a2dd6c67d3cbb7b360747f776412bf01976aa32f4b2023-07-22T12:34:51+02:00whoresonexamples : basic VIM plugin
8779cf022a1889e50113fd348dc96b4557fc75a6296e782c9e735f93ab4767ffc37462c523b73a17ddc2023-07-21T09:42:21+02:00Przemysław Pawełczykmake : fix embdinput library and server examples building on MSYS2 (#2235)
878294f424554c1599784ac9962462fc39ace92d8a545a1b07e9b20c33d71d8c849ff27d693a75a02692023-07-19T15:06:40+08:00Rinnellama : extend API to get max devices at runtime (#2253)
8796e7cca404748dd4b1a3affd0d1296e37f4ac0a6fa6803cab946c817fb7aaf2a40b317f5d3e373bd12023-07-15T06:34:16-04:00Xiao-Yong Jinllama : add custom RoPE (#2054)
8803ec7e596b2ba3f43c22f441254ca2bcfa91102ba917831c63a4138814d23da1917bf2b5d5b9faa6c2023-07-12T01:12:35+09:00Jinwoo Jeongdocker : add '--server' option (#2174)
8815656d10599bd756dc0f17284e418e704200b43f31d1630996920f889cdc08de26cebf2415958540e2023-07-10T11:49:56-04:00Evan Millermpi : add support for distributed inference via MPI (#2099)
8821d656d6360359cfdaaf5d64ed9690047b600dbcb72421402834141df6cbdcf595fe46dbd11874dce2023-07-08T00:24:01+08:00Qingyou Mengggml : change ggml_graph_compute() API to not require context (#1999)
88331cfbb1013a482e89c72146e2063ac4362becae7983b555e9ddb36703cee4d22642afe958de093b72023-07-05T16:51:13-04:00Tobias LütkeExpose generation timings from server & update completions.js (#2116)
884983b555e9ddb36703cee4d22642afe958de093b7ec326d350c72afd23709a409944728a607188cc02023-07-05T18:03:19ZJesse Jojo JohnsonUpdate Server Instructions (#2113)
8858567c76b5326e862be0755a8dc1dd988223fcae3924dd22fd3ba93e097f8d19ba5cda919ca2fe2fb2023-07-05T15:13:35ZJesse Jojo JohnsonUpdate server instructions for web front end (#2103)
886f257fd255044decffad93dee2502875ce66ad80c7ee76e45afae7f9a7a53e93393accfb5b36684e12023-07-05T03:06:12+09:00jwj7140Add an API example using server.cpp similar to OAI. (#2009)
8877ee76e45afae7f9a7a53e93393accfb5b36684e1acc111caf93fc6681450924df9f99679c384c59e2023-07-04T10:05:27-04:00Tobias LütkeSimple webchat for server (#1998)
888acc111caf93fc6681450924df9f99679c384c59e23c7c6fc9182b041f006b86ea1e7f99911ecf3442023-07-04T15:38:04+03:00Henri VassermanAllow old Make to build server. (#2098)
8891cf14ccef12e19c5a5b0b17ab456242d1f8c7fddcc45a7feb8412e84ff292207621412fffc0d3d512023-07-04T00:05:23+03:00Henri Vassermanfix server crashes (#2076)
890d7d2e6a0f0c74f7a570dae384dfff371ac744d2a46088f72318981341a2d646f12f6eee6aec06d652023-07-03T05:38:44+08:00WangHaoranRobinserver: add option to output probabilities for completion (#1962)
891b1ca8f36a9cdbcee5f5c425df717611a1040a897b8c8dda75fdf5fdea49c80af36818e7c30fe0ddf2023-07-01T23:42:43+08:00Qingyou Mengggml : disable GGML_TASK_INIT and GGML_TASK_FINALIZE by default (#1995)
892b853d456018b10820686362af41b2f2f75f1eec69225baef71407d799a6f7f563b77fd7f827914162023-06-26T13:57:59-04:00zrmggml : add NUMA support (#1556)
893c2a08f87b8d180115d04b8688f383d1b2761b16d66a2555ba6cab954c56d653b29c27bfbbacfbfb12023-06-25T08:48:36Zanon998fix server sampling: top k sampler first (#1977)
894527b6fba1d237befb324fd846bda7418c0fa394dd7b7484f74d486f77feb4c0b7af7e1718ed916512023-06-24T11:47:58+03:00Didzis Goskollama : make model stateless and context stateful (llama_state) (#1797)
89520568fe60f00155fa25e92eb3a7f6b911d55796718b35625c3c19c64b7818a12460ba5ddb006dfdc2023-06-20T01:12:39+03:00Henri Vasserman[Fix] Reenable server embedding endpoint (#1937)
896fc45a81bc642b9ef33d9004f2b363d558438a6c9794db3e7b982fee37e3995db9c3a216a57ff65e32023-06-17T17:13:05+05:00Faez Shakilexposed modules so that they can be invoked by nix run github:ggerganov/llama.cpp#server etc (#1863)
897794db3e7b982fee37e3995db9c3a216a57ff65e35ddf7ea1fb42bac21026de2f77e0f9c069b922342023-06-17T07:53:04-04:00Randall FitzgeraldServer Example Refactor and Improvements (#1570)
8989dda13e5e1f70bdfc25fbc0f0378f27c8b67e98337e257c48e350cf03c353c10d31e777f8d00123d2023-06-15T18:36:38+01:00Srinivas Billareadme : server compile flag (#1874)
8994bfcc855abdb2c9fcc3c5a84747974521909fa416b8312e7979b852f6b6ac9d29cd51fda16c179482023-06-15T20:29:48+03:00Georgi Gerganovmetal : parallel command buffer encoding (#1860)
900e32089b2c20b1b87b22912f4a8b93fe01647d5b92347e45e7bdb09c9a7d74b2c0bc86c2b65f0c3432023-06-13T21:04:40+02:00xaedestrain : improved training-from-scratch example (#1652)
90117366df842e358768c0df7024484fffecfc7865b44f906e8537fcec965e312d621c80556d6aa9bec2023-06-06T21:33:23+02:00Johannes GäßlerMulti GPU support, CUDA refactor, CUDA scratch buffer (#1703)
9027552ac586380f202b75b18aa216ecfefbd438d945d1830b99dfd85bb6279adb4dd94aa444afd5b5e2023-05-29T19:31:44+03:00Georgi Gerganovggml : sync cgraph import / export API
9031b78ed20818b72306edc7208b9bfb69a1a0d3297337aea11390221bc925e4acb1f603f1649af27352023-05-28T11:48:57-06:00KerfuffleOnly show -ngl option when relevant + other doc/arg handling updates (#1625)
9040df7d63e5ba0ab8856476e121a03b985d6f15c9d97c9b77c4fc5e2283755c4418759cfc5fc73ad052023-05-27T11:04:14-06:00KerfuffleInclude server in releases + other build system cleanups (#1610)
9057e4ea5beff567f53be92f75f9089e6f11fa5dabd7780e4f479dc5af106287c164b8e186cd9b6215c2023-05-21T11:51:18-06:00Steward Garciaexamples : add server example with REST API (#1443)
906affc76edfdefa7b326f526e463cc65ff13fcfb92ea600071cb005267e9e8f2629c1e406dd5fde0832023-05-20T14:19:28+02:00Johannes Gäßlercuda : loading models directly into VRAM, norm calculation on GPU, broadcasting for ggml_mul (#1483)
907ec2e10c4443209da56b431b24dd0845b60e757fbd2c59b8ba498ab01e65203dde6fe95236d20f6e72023-05-20T11:06:11+03:00Georgi Gerganovllama : add llama_init_backend() API (close #1527)
9087694b52b9a206b93d59139c3c7c9b55da0f5aa5979e3efb0e97b65b6cc72cd9ee970fa8189ad79a42023-05-19T10:24:59-07:00Jason McCartneymain : make reverse prompt option act as a stop token in non-interactive mode (#1032)
9095ea43392731040b454c293123839b90e159cbb99ee9654138ab0ae5f138f4abddf56ca234ea3c3522023-05-18T19:31:01+02:00Erik Scholzmake kv_f16 the default for api users (#1517)
910f954edda935a70a14cf0cc45ecc7fe7d60cf3e4bf048af0230ad5381bb20b9e3c1467ec6fc7debdf2023-05-13T14:56:40+02:00xaedesggml : implement backward pass for llama + small training-llama-from-scratch example (#1360)
911dd7eff57d8491792010b1002b8de6a4b54912e5c7fc50c051ae8a78e9643fdf172d12e20f2dd9b6c2023-04-29T08:34:41+03:00Ivan Stepanovllama : new sampling algorithms (#1126)
912c4fe84fb0d28851a5c10e5a633f82ae2ba3b7fae1d78fecdab4087028a38517e86ed129f077174d82023-04-24T07:40:02+03:00Georgi Gerganovllama : refactor get / set state + remove redundant kv cache API (#1143)
913b6e7f9b09e9c340ec97a2fae61c1eb8db861f2f950cb666b8a2e35a49b08c0f6bc81138c8f6f2ac12023-04-22T08:21:32+02:00xaedesllama : add api for getting/setting the complete state: rng, logits, embedding and kv_cache (#1105)
914a32f7acc9f54dba1c728cb1e596bd00bf3b4eb5f43ffdefb7424f79a3d510c199e2ea86684b4f8242023-04-14T15:37:11+02:00Pavol Rusnakpy : cleanup dependencies (#962)
91595ea26f6e92d620a5437f576b80868aee7f808d682d146df9b43cf677e0dbce20b03cf864958a0cc2023-04-13T14:46:23+02:00SebastianApelbenchmark : add tool for timing q4_0 matrix multiplication (#653)
916f963b63afa0e057cfb9eba4d88407c6a0850a0d8aaf3b23debc1fe1a06733c8c6468fb84233cc44f2023-04-08T12:24:37-07:00comexRewrite loading code to try to satisfy everyone:
917e986f94829bae0b9e66b326acbbba179931c84f1c0bb1d3ce21005ab21d686626ba87261a6e3a6602023-04-02T12:23:04+02:00Christian FalchAdded api for getting/setting the kv_cache (#685)
918f4f5362edb01b05c383b23f36d7b3489c77061b5863f65e2e32dc1e6d23c96a4811bf382d6b2a5482023-03-24T15:23:09ZGary MulderUpdate README.md (#444)
91956e659a0b271436e24813a801640d015e7b0532840ea807a972ec7b5a426f034ebfa593b5e7a06ed2023-03-22T17:09:38+01:00Erik Scholzfix perplexity after c-api refactor (#390)
920f5a77a629bd0f37ae1696747633ab42a5530ec15da0e9fe90ccf6e73597eb19dd0cfc0a28363fb3b2023-03-22T07:32:36+02:00Georgi GerganovIntroduce C-style API (#370)
9215cb63e2493c49bc2c3b9b355696e8dc26cdd0380da5303c1ea68aa19db829c634f1e10d08d4096802023-03-20T08:24:11ZStephan WalterAdd tqdm to Python requirements (#293)
9227392f1cd2cef4dfed41f4db7c4160ab86c0dfcd9ad5fd5b60cfdfbfb22b0f2bc9e9f6c9692768f8d2023-03-19T12:38:44-06:00Suaj CarrotImproved quantize script (#222)
Reference in New Issue View Git Blame Copy Permalink