cogito/calculet-npu-research-archive
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d2677d2205d50aab55fdbd1cbe559a6584989433
calculet-npu-research-archive/history/llama.cpp/topics/06-memory-kv-context-buffer.tsv
T

198 KiB
Raw Blame History

1d1bc743c10080fed1253686ce01749b42611ded42f201160b1960fd7be18b70d5770599d7082dd2f2026-05-28T14:23:06+08:00wangbomengset u_batch = max_seq_len
2f131bf1908b8c5dd1f49d7ae7f616506fc47166633d96dae28e17208ef61a71dba40cda3c04f135a2026-05-27T09:38:51+08:00wangbomengfix patch of qwen2.5vl,n_patch of minicpm = 64,fix n_ctx_slot,comment out M-ROPE
31921024604db640f09ade83c8a437ebf15bde360d028722a12c5a463cc97207787cfd03d7a2590b0 5af229fe518ee2d761ab8ae2a69f7a9aaa9057f42026-05-21T14:08:14+08:00Bomeng Wangrefs/stashWIP on dev: d028722a replace cparam.n_ubatch with n_ubatch()
45af229fe518ee2d761ab8ae2a69f7a9aaa9057f4d028722a12c5a463cc97207787cfd03d7a2590b02026-05-21T14:08:14+08:00Bomeng Wangindex on dev: d028722a replace cparam.n_ubatch with n_ubatch()
5d028722a12c5a463cc97207787cfd03d7a2590b063442fde8dc285817f725bd6c5fae80f478a38132026-05-20T14:41:05+08:00wangbomengreplace cparam.n_ubatch with n_ubatch()
663442fde8dc285817f725bd6c5fae80f478a3813fc3f4a9fab88e98eff8eeca8cbc6617333edba2c2026-05-20T11:47:42+08:00wangbomengfix ubatch and cmakelist
72716130010c58f1cdd9d94f0e2f8e047bc92dc66b35bda124ccd4ed581dd6088d3ffa4931c2809b62026-05-18T08:40:54+08:00Yunzhe Jiafix n_ctx_slot error by adding runtime capacity loading for cal-llm
83b14ed556c15529fe1f94b649f49db7156dfaf67dde8b8228081769c8d8f1e0751d47fa7875f84232026-05-14T14:34:25+08:00Yunzhe Jia- Removed references to CALRT from clip.h, mtmd-helper.cpp, mtmd-helper.h, mtmd.cpp, and mtmd.h. - Updated server.cpp to conditionally include CAL-LLM support. - Introduced option LLAMA_SERVER_USE_CAL_LLM in CMakeLists.txt to enable CAL-LLM integration. - Refactored server context to handle CAL-LLM requests and responses. - Updated image processing logic to remove CALRT-specific implementations. - Ensured compatibility with existing llama functionality while integrating new CAL-LLM features.
95d219bfcef239c1e3ddd1baecc0514a9462ff6a86787a53ab9b4897c5a7b51a4062d7b90e6697d352026-04-24T09:31:32+08:00wangbomengcalrt: recover base = batch_index * dict_len
10d8b2aaa00049978cebac15041f960d75b552f97bb6f29ec8140028619efaa50aa6e73146cecf631d2026-04-13T11:23:44+08:00wangbomengserver: fix context-shift
11b6f29ec8140028619efaa50aa6e73146cecf631d6d55eae7e76b768acfcec045b8e84cded8ae3b552026-04-13T11:23:31+08:00wangbomengserver: fix context-shift
1242a181674565411efa5c8b318bc77d6fd084df8abd3a57a11aa008e0b5dc2a30f82fcd3a025872082026-04-10T09:06:55+08:00Yunzhe Jiafix prompt_cache issue
1399d2078e89305035d87d966cee7987c7d50b39259626239f5a9e568c9975d67dd6745fef90279a872026-04-08T19:01:16+08:00Yunzhe Jiafix buffer resize problem
144e194ecc6c4b8ebe4c72d93f6055e3605dbfcefa16cbf81a731f5839a2f6b0eb9d8539826353748b2026-03-24T01:59:03+08:00wangbomengcalrt: fix ubatch.embd cpy
1507817cefc14fa359dcecad0630defd3610f8f5c8b2c45d8ddefb7ec09435231c8ec5b736fabf9c912026-03-10T16:32:07+08:00Yunzhe Jiafix kv issue
167ddafbdcb9426ae3af016925b8b596f11e8742d0059009eeaf20a569abfbac5eb37dad3bb93764282026-03-10T07:13:04+08:00wangbomengcalrt: commented out cal_ctx->encode(batch)
17b2c45d8ddefb7ec09435231c8ec5b736fabf9c91c6ea3f0d2468e3b8eab5de80bacee7fe4c9dd3d72026-03-09T15:03:26+08:00Bomeng Wangclart: past_kv_len = n_tokens
187a3a9a0e76bb5f530beafcfe52e87e388e93117a583c387efaf7bc030574e554088de79d09a27fbd2026-03-05T07:24:59+08:00wangbomengcalrt_infer: fix past_kv_len
19583c387efaf7bc030574e554088de79d09a27fbd0edc14a60a2d5596f71d728c655e3523da924d892026-02-27T02:24:56+08:00wangbomengcaserver: lim generated tokens to n_ctx_per_seq
200edc14a60a2d5596f71d728c655e3523da924d89aa0a17d719326a63e0d6fea60aa0ced44e7abc332026-02-27T01:57:39+08:00wangbomengcaserver: limit generated tokens to n_ctx_per_seq
21365eb0b719e30b0f9e348d854f11c9c3f954a96e886cd1fb3b217efcf51c46209fcb6940223467972026-02-07T09:05:17+08:00Yunzhe Jiacomment out pos buffer file
226b1ecdbbe3cc72ed8888be38f9e934cfd79b9fbb240d9bb75241c91896afe125f2fc74698a7395f32025-11-25T17:06:14+08:00Yunzhe Jiaadd kv_tensor for pld test
23d81d7b190ff5f21325167936496dfe5ab48b922ee492f5dc8cec17529f53ef2bbdf7b502107f8148 1f5accb8d0056e6099cd5b772b1cb787dd590a132025-11-04T14:43:32+08:00Yunzhe JiaMerge branch 'master' into dev
24cd5e3b57541ecc52421130742f4d89acbcf77cd487c9efc3b297b8a498716b1db3d061842e6fc85b2025-11-02T18:14:04+02:00Georgi Gerganovserver : support unified cache across slots (#16736)
252f68ce7cfd20e9e7098514bf730e5389b7bba908e4a71599e5846110159955dec0008eb4aa24222b2025-11-01T19:49:51+01:00Pascalwebui: auto-refresh /props on inference start to resync model metadata (#16784)
268da3c0e200a586f768ada6f38745acb01380174cc22473b580807929fd9e3a3344a48e8cfbe6c88f2025-10-31T13:50:33+02:00Georgi Gerganovbatch : fix consistency checks for the input positions (#16890)
276eb208d17ea29bb60295d9a2b5e7122dfb8f4b559984cbb61d12491d604484fb38b678fa150640612025-10-31T00:34:27+01:00Sigbjørn Skjæretci : enable free-disk-space on cuda docker build (#16877)
28b52edd25586fabb70f0c21b274473b307cf14499517b7170e1a4d733583c4b07c5b7a49acc05911c2025-10-30T18:42:57+02:00Georgi Gerganovserver : remove n_past (#16818)
29d261223d24e97f2df50220e4a5b7f0adb69bba81dcca0d3ab840ebe9b2ccd4719033d408eeb758d72025-10-30T23:19:14+08:00JJJYmmmmodel: add support for qwen3vl series (#16780)
30bacddc049a00786df44e682262f6e298742bfbc3229bf686287d18f82c44e89888cc662145ecfdb42025-10-30T07:18:50-04:00Tianyue-Zhaomodel: Add support for CogVLM model (#15002)
318b11deea4663f29d3e042ce1056ba643264cd5f1b9ce94017729465895402cbcfffb51fa926c15e32025-10-30T04:34:15+01:00Oliver SimonsHide latency of bias and gate-loading (#16847)
32b9ce94017729465895402cbcfffb51fa926c15e33464bdac37027c5e9661621fc75ffcef3c19c6ef2025-10-29T15:13:10-05:00Jeff Bolzvulkan: Fuse rope+set_rows (#16769)
33e3af5563bd049141e036b50f843196db33d23e9710fcc41290e233788f5a4215314156e8e023eb922025-10-29T18:09:18+01:00Xuan-Son Nguyenllama: store mrope data in KV cell (#16825)
34bcf5bda6f5df559565d11d7c8e8295c1159a85ec3eb2be1ca5f37480aeb16102970d9e65f43347fe2025-10-29T14:39:03+01:00Ruben OrtlamVulkan MMQ Integer Dot Refactor and K-Quant support (#16536)
353eb2be1ca5f37480aeb16102970d9e65f43347fee41bcce8f0b53032a1fed275cd253e931c041cf62025-10-29T06:29:12-07:00Max KrasnyanskyHexagon Op queue & dispatch optimizations (#16820)
36f549b0007dbdd683215820f7229ce180a12b191d9a3ea685b937c0f0cbfda2e50004ea54bf1875122025-10-29T03:53:04-05:00Jeff Bolzvulkan: Call ggml_vk_buffer_write_2d from ggml_vk_buffer_copy (#16793)
3785a7d8677bf2200981e52f744a21d5267964ffcfa8ca18b4b815a2abdbecb958ee5f4c542d69aac72025-10-28T20:19:44+02:00Georgi Gerganovmemory : remove KV cache size padding (#16812)
388284efc35c909217ee1b9a06903245d808ac22831c1409e13169d0ced4b1b4d39fb5b268b75250912025-10-28T23:16:20+08:00l3utterflyinitialise buffer.device in ggml_hexagon_session (#16816)
397a0e900e3615fa46c074a7fdf900b47d3c0a1c7e280d97be9660e7a5feaa28a6e7a299bc73dd83fc2025-10-28T11:23:54+01:00Johannes Gäßlerllama: consistent ctx <-> buf order for KV cache (#16746)
40ad8d36beffd791db10c94eb9e964afb891e3ca55c053e18a66dd95dc340aa61317877c2a41d4e3cf2025-10-28T03:50:33+02:00tamarPalsycl: add SSM_CONV operation support (#16800)
415a4ff43e7dd049e35942bc3d12361dab2f15554410640e31aab0819f31c1e1f2d008b019ee7372322025-10-27T13:51:28-07:00Diego Devesallama : disable pipeline parallelism if compute buffer allocation fails (#16748)
42945501f5ea4b8ca56b181ecb035e9ee3fb31f43275cbdd3fce38ea12d50cd19e73a069aa5dbbd5fa2025-10-27T09:17:31+01:00Johannes Gäßlerllama: fix leaked buffers for mmap + split files (#16765)
433470a5c891dcc94363e492a3760af92b6b07241cbd562fe4f7bd55625511d5f9d639c4fb1db1d4402025-10-26T23:19:03+01:00Aclyggml-alloc : make gallocr prefer chunks that allow memory reuse (#16788)
44bbac6a26b2bd7f7c1f0831cb1e7b52734c66673b73a48c9790d320476b3e5ef75bda09f2f8269e6e2025-10-27T02:13:31+08:00leejetggml: fix cuda kernel launch configuration for k_compute_batched_ptrs to support large batch (#16744)
45f90b4a8efe4466215c51155a5c22c1ae6207da238423d019318b446640bb620e4ce80066d8530f052025-10-25T10:59:54+02:00Giuseppe Scrivanovulkan: delete dead code (#16732)
460bf47a1dbba4d36f2aff4e8c34b06210ba34e688dd62dcfab97e420949519fd0eac9fca7bf97e6352025-10-23T21:30:17+02:00Johannes Gäßlerserver: add memory breakdown print (#16740)
47d0660f237a5c31771a3d6d1030ebe3e0c409ba92fe6a9882acf5c02f96624ed8f80144100d7006cb2025-10-23T15:00:49+02:00Xuan-Son Nguyenmtmd-cli : allow using --jinja (#16718)
489de9672adb0f4ca4e39483ac3ffed52b3f70a55d63d2fc46e17a06be5b4b5823a5ada088317f1f0a2025-10-22T20:05:15-05:00Matthew Michelsycl: use async memory allocation to fix crashes during graph recording (#16644)
4963d2fc46e17a06be5b4b5823a5ada088317f1f0aa2e0088d9242bd9e57f8b852b05a6e47843b5a452025-10-22T13:47:09-07:00Max KrasnyanskyAdd experimental ggml-hexagon backend for the Hexagon NPU (#16547)
509b9201f65a22c02cee8e300f58f480a58859122719a5a3edfd306516cc419679d69d6435943b68162025-10-22T16:58:23+02:00Pascalwebui: introduce OpenAI-compatible model selector in JSON payload (#16562)
51c9c1972e2c2cc6a771fcc145bfa138700179f961b617cfd2896edd592a36ebbc041817eb030a10052025-10-20T19:49:02+02:00Aleksander GrygierHandle legacy 'context' attachments (#16687)
52b617cfd2896edd592a36ebbc041817eb030a100579068501fac9a74cca7129a8e5a8281b410a853e2025-10-20T05:53:50-07:00Diego Devesaggml-alloc : fix leak when reusing a tensor with a larger size (#16679)
5306332e28672356b964d6dfc2ba4657e20581cd4372d53e6c4decee8b339e49aed8cc0e234b9639dc2025-10-20T16:27:09+08:00takuya kodamallama-batch: fix build fails with `-Werror=missing-braces` (#16614)
547062dd8460685d6700ed7621e50a22c6f3400ca30398752dd450dfabdd1b9e289f6364c2600f6ab52025-10-20T15:44:21+08:00takuya kodamallama-context: only warn on pooling_type when user specified (#16674)
55c20c0a5c0c44179f5267a262546624854b1203d162425b3a7e8b169a3bbe6ce8c7324513a7fcee8c2025-10-20T14:41:08+08:00Yunzhe Jiakv: turn on common-prefix mtmd: fix output size bug
56e56abd2098dd2e2b0804691b93c13b48ae42162738355c6c8e43204e11a22daa7483082c0ff01e712025-10-18T05:22:57-05:00Jeff Bolzvulkan: Implement topk_moe fused shader, ported from CUDA (#16641)
5741386cf365d894134ee0813d15e2f5d76f6a4d8e3d4e86bbeb15f487d6da6174ba6191b7c212cc252025-10-17T18:02:52+03:00Radoslav Gerganovrpc : report actual free memory (#16616)
58342c728d031d50673feded797520a44127d73379ababae7e1ec3e9cfdab0322ee55ea3389e82a4d52025-10-17T18:01:23+08:00muggle-stackggml : fix SpaceMit IME array out-of-bounds in task assignment (#16629)
599ad4f1931ee0f3b41d9355245ef744786aaae0aa79967ec596c0dacfd2251b085a57e79df292b1cc2025-10-17T02:33:58-04:00Ilia Ilmermetal : add `CONV_TRANSPOSE_2D` (#16542)
6062425b3a7e8b169a3bbe6ce8c7324513a7fcee8c76a63a2998286c1649de2496bccb7d8a465498952025-10-16T17:34:39+08:00Yunzhe Jiakv: let seq_rm free whole seq since current hw-op cannot support common-prefix
6176a63a2998286c1649de2496bccb7d8a46549895b67217078aa748b06e1b2269d88dd18c0aca2c262025-10-16T16:31:15+08:00Yunzhe Jiakv: fix common-prefix bug by implementing llama_memory_seq_rm
62adc9b60f190c1016a09f439862fa1cbb302262acee50ee1eadff58777ae746827b04de7ba0befc552025-10-16T13:10:32+08:00takuya kodamaggml-cpu: replace putenv with setenv for const-correctness (#16573)
63fa882fd2b1bcb663de23af06fdc391489d05b007ffa059034c1e41f3e58363ef3c46d2fcf854bc4d2025-10-14T20:33:05+03:00Georgi Gerganovmetal : avoid using Metal's gpuAddress property (#16576)
649c7185dd28416cf67f5e3b268381f311b5e3da561ee9d0b415cdf5240418c110a18b419f4002b1542025-10-14T14:22:47+02:00Johannes GäßlerCUDA: enable FA for FP32 KV cache (#16546)
6548e2fa9fb7c2de1e53808fdb65ec33f916020fc45b6913c47b6bc71a6f927805a45387d5657d8b892025-10-14T19:15:15+08:00Aman GuptaCUDA: add fp kernel for larger batch size MoE (#16512)
66bc07349a7f87ba6eb31ed4b0ea9d9a7352185213e60f241eacec42d3bd7c9edd37d236ebf35132a82025-10-14T08:48:50+03:00Georgi Gerganovserver : dynamic token limit for prompt cache (#16560)
67e38b7c6e9e4453e3b3e96d76e38bc2ccb6bce4585016b7286240d29f8f640039989b84ea3a8543442025-10-13T22:42:37+03:00Georgi Gerganovgraph : support cacheless embeddings with FA and iSWA (#16528)
6856fc38b9655fbe1869d8bd6cfb269418196cea691fb9504eb744969a990bfe4cfcf1d3d7a479541c2025-10-13T17:01:24+08:00Chenguang LiCANN: fix CPU memory leak in CANN backend (#16549)
69a31cf36ad946a13b3a646bf0dadf2a481e89f94481d54bbfd599811b354c39f04550888168be77802025-10-13T02:43:14+08:00Sam/Samuelmetal : add opt_step_adamw and op_sum (#16529)
7081d54bbfd599811b354c39f04550888168be7780c7be9febcbafa9af7d1b9443f86475c59c9c5f872025-10-12T18:06:41+02:00Pascalwebui: remove client-side context pre-check and rely on backend for limits (#16506)
714b2dae383df708e2afc49c4859a81cd074f5ac1041aac5c69b5fb281bc1f486afb053f78101bb39e2025-10-12T09:29:13+03:00Georgi Gerganovcommon : update presets (#16504)
7231d0ff1869aa2ea31f4e96d5877d0343e9a2171b97870e64975b26c5e06a3540a8dc0ff601351e862025-10-11T21:39:04+08:00Yann Folletserver / ranking : add sorting and management of top_n (#16403)
7368ee98ae181a5c83a5cc6261daeee69a1f588c15cdb6da468cc33323955a523738d2e1675aeb5e9a2025-10-10T17:11:07+03:00Radoslav Gerganovserver : return HTTP 400 if prompt exceeds context length (#16486)
74d00cbea63c671cd85a57adaa50abf60b3b87d86f8328fd4bae76fc027f8ca0e9a05acd3788dabe3f2025-10-09T18:54:51+03:00Georgi Gerganovserver : host-memory prompt caching (#16391)
75e08db4259521de493b7aeb49dadf29ebd1ee966a12bbc3fa50b6df03318a4451c9a2210200a0b28d2025-10-09T08:39:18+02:00Saba Fallahmodel: EmbeddingGemma Adding Support for SentenceTransformers Dense Modules (#16367)
769d0882840e6c3fb62965d03af0e22880ea90e012d2ee056e1df0fdf646270fb5621e9f92084b59a72025-10-08T20:21:46+02:00ai-fonsiDisable CUDA host buffers on integrated GPUs (#16308)
777fdd16b432d247121fe5fe7b21f8805f85266c8574b8fc17f92ada295a648e3c5eb28f46bca7d8922025-10-08T10:57:29+03:00Georgi Gerganovserver : improve context checkpoint logic (#16440)
780123ff38f53d34752f29239a29d0e40a6dc4110f0a319bb75ed29d968e2a9b544011b09ccb9329152025-10-07T08:24:17+03:00Georgi Gerganovmemory : use sequential equal splits for recurrent modules (#16442)
790a319bb75ed29d968e2a9b544011b09ccb9329151d6092fc72f4d10f4486ac95edfd414bc08b62b82025-10-07T08:23:30+03:00Georgi Gerganovmetal : add support for non-padded FA KV (#16148)
803df2244df40c67dfd6ad548b40ccc507a066af2bc08002a1988348403a5fd59b1fa3de3a10a6f92f2025-10-06T12:55:53-05:00Gadflyiillama : add --no-host to disable host buffers (#16310)
81ca71fb9b368e3db96e028f80c4c9df6b6b370edd35266573b968e1c947b367782fb4b3eddbb4f3c02025-10-05T06:57:47-06:00Gabe Goodhartmodel : Granite docling + Idefics3 preprocessing (SmolVLM) (#16206)
82f39283960b58a92ecc0c72567711318b20e22b55898acba6816ad23b6a9491347d30e7570bffadfd2025-10-04T16:22:45+03:00Radoslav Gerganovrpc : check src buffer when copying tensor (#16421)
83f6dcda390004b627ef30af378d0c01ad2519289e606a73f53175077429484b23dcf799f69a31d0bd2025-10-03T13:34:51-05:00ddh0server : context checkpointing for hybrid and recurrent models (#16382)
84638d330246954e88dffc22ce01fec15e6894e54484c8e305e8010a1a3d43bdd0a25f737ac67809a42025-10-03T13:49:08+02:00Aclyggml : fix graph reallocation with multiple chunks (#16396)
8584c8e305e8010a1a3d43bdd0a25f737ac67809a42aaf0a2a2056d75d0dd53ab8a181473760e6ab222025-10-03T12:51:40+02:00Aleksander GrygierFix missing messages on sibling navigation (#16408)
862aaf0a2a2056d75d0dd53ab8a181473760e6ab220e1f83855609d73beaf05d818640b6cfd39d287b2025-10-03T05:50:46-05:00Jeff Bolzvulkan: Replace uses of maxMemoryAllocationSize and VK_WHOLE_SIZE (#16354)
8772ee736c447be4ededb51ab97904b4d33c90c261f09aefaa84d7f4d5df3f400f67944b94fef5b7952025-10-02T13:51:36+02:00Sigbjørn Skjæretci : fix ubuntu-latest-cmake-rpc (disable ccache) (#16388)
881104ca1a1c926b832274694a2773a17066982ad04f1575921cac9b489fe8f8bbf20aff985e7da45e2025-10-01T14:09:52+02:00Sigbjørn Skjæretci : use registry cache for docker builds (#16366)
89b2ba81dbe07b6dbea9c96b13346c66973dede32cbf6f3b3a1965d70e07ca94aab7b01268fe483e962025-09-30T21:41:42+02:00Sigbjørn Skjæretci : fix ccache key for ubuntu-cpu-cmake (#16355)
902df5bcf357dba0c49b4df1d684b2ffc9e88b7054075c01567bb7e93965ae60b7e119182c3e68f3e92025-09-30T15:38:01+02:00Sigbjørn Skjæretci : disable ccache for android (#16348)
91b77e6c18e1a6fac5705ed95f03af5436d67484c12ddd3f2356c313385fafc19a9f0ce678c8fe03ee2025-09-29T22:50:44+08:00alex-spacemitggml: riscv: add riscv spacemit backend (#15288)
92d8359f5fde480da030bf75c7711573c7c4d993ba6a2c6145a0b91b40eb3c3dba7b20ccc4b270490f2025-09-28T01:38:37-05:00Jeff Bolzvulkan: 64-bit im2col (#16135)
93e6d65fb02d553bd79cad94e517cdca18b687788d8656f5de688cddcaea1d6174535eb60ee23ef6a02025-09-27T16:43:39-04:00Jeff Bolzvulkan: support arbitrary KV dimension in flash attention (#16160)
94624207e676ab5eb3ce7af631902bb45fb73a8359807e8c6d310952f2f5656afc63dab9d7083dcb5c2025-09-27T02:03:33+08:00Aaron Teodevops: add s390x & ppc64le CI (#15925)
951a189278944d030211f336e103e96b65f976c361e0539eb6aed346d4b25a6ea019044e88771e76902025-09-26T18:35:42+02:00Aleksander GrygierAllow viewing conversations even when llama server is down (#16255)
969b26511857ac09ae69ab485168fe2d3ee5fb1d6e00217cd41388328c93e9e9644921c4319bb03bcc2025-09-26T18:27:25+08:00Aaron Teoggml-cpu: implement MXFP4 SIMD for s390x (#16193)
97835b2b915c52bcabcd688d025eacff9a07b65f52b05a9d650f4da1e70e7e2cbe8fe44e61b39656db2025-09-25T19:50:28+02:00Sigbjørn Skjæretmodel : add GroveMoE support (#15510)
98077c94d0caf87fbd3cf3288dbb5c0fd9670294cfaa3ee0eb0b80efca126cedf9bcb4fb5864b46ce32025-09-25T22:35:05+08:00Aman GuptaCUDA: add a fused top-K MoE kernel (#16130)
99e789095502b337690c7616db32d7c679a5bd2533f2a789e33490deb483a2694b066b37e45524bb792025-09-24T16:53:48+02:00Johannes Gäßlerllama: print memory breakdown on exit (#15860)
100f2a789e33490deb483a2694b066b37e45524bb793a599719673c850647e3bb911ed6d91109bb91d22025-09-24T16:17:49+02:00Aclyggml : split graph allocations according to backend max buffer size (#15815)
1014b9f4cb0f89a88de4bdf97727d0457b0c648804c85e72271ba1ce78adf34fd8997803c991e617ca62025-09-23T13:59:34+08:00Aaron Teodevops: add s390x containers (#15915)
10237a23c17bdc9c99c9c6ad41168e4ced3724b72cd138c87ce8bd558b2cc134ada7316a3dad8eb67ac2025-09-22T14:13:51+02:00Adrien Gallouëtcommon : enable `--offline` mode without curl support (#16137)
1039073a73d82a916cea0809de225ef5175c3a86e9151f5a45fbe575dcd54bdd2a339ef8e8424d1c12a2025-09-22T07:22:43+02:00Ruben Ortlamvulkan: vec dot matrix multiplication fix (#16151)
10428baac9c9f491c872e2c37762d3bd90446b005e91eeb523c3e0c7ffbd59469f5463dcbdecba3535e2025-09-21T16:50:45+03:00Georgi Gerganovci : migrate ggml ci to self-hosted runners (#16116)
1051eeb523c3e0c7ffbd59469f5463dcbdecba3535e5bb4a3edec297e74b0f7bd4ed5d0fdd12e28d8582025-09-21T08:31:55+02:00Giuseppe Scrivanovulkan: optimize UMA buffer operations and fix driver hangs (#16059)
106803dac2e48ef3ba26a504eb27c4e77ec2d21f7d0459c0c2c1a400f960d7b8e8d94d31a8426f809862025-09-20T10:42:56+02:00Ruben Ortlamvulkan: use vec dot for matrix matrix multiplications (#16056)
10769ffd891631befa9e6b485fd646a16dab4f2c007246c0d9c795fb25da8268625d597580155a3672f2025-09-18T23:07:26+02:00Adrien Gallouëtggml-amx : fix ggml_amx_init() on generic Linux (#16049)
108246c0d9c795fb25da8268625d597580155a3672f3edd87cd055a45d885fa914d879d36d33ecfc3e12025-09-18T23:07:18+02:00Adrien Gallouëtcmake : fix static linking for OpenMP on Unix-like systems (#16031)
109e00f3fd8fff2cf5a8c8c9f475034bd089c8bcce4f2f28380ea68939c0481df3e4216b698824a59df2025-09-18T15:06:48+08:00Jhen-Jie Hongmetal : avoid call free for non-owned buffer (#16067)
110d304f459d8619399eb232b1e3689379306f439d30320ac5264279d74f8ee91bafa6c90e9ab9bbb912025-09-17T13:09:40-07:00Reese LevineGGML WebGPU: Support for ADD, MUL, RMS_NORM, GET_ROWS operators (#16018)
1110320ac5264279d74f8ee91bafa6c90e9ab9bbb91a7a98e0fffed794396b3fbad4dcdbbc1849636452025-09-17T20:38:12+03:00Georgi Gerganovmetal : refactor + optimize v2 (#15995)
112cd08fc3ecc0264b4414b68af3874a6c689ed60c1cb5bb6cc05119c24e7711ca2956cd0e6d409d3962025-09-17T01:08:02-07:00David Ribeiro Alvescommon : Fix corrupted memory error on json grammar initialization (#16038)
113d5fabe3682de515fd09d6c981f7a0d1b756144558ff206097c2bf3ca1c7aa95f9d6db779fc7bdd682025-09-17T14:33:08+08:00Chenguang LiCANN: Optimize ggml_cann_set_device (#15935)
1143913f8730ec6d6245480affc30ae3049107956f476888d202ed2b835ae19ea9f9db6baf39e4192972025-09-16T15:25:57+02:00Daniel Beveniusggml : fix padding in timestep embedding kernels (#15932)
1159dcd200d57bc6f05a59a6a8df361d5d183af41240fa154e3502e940df914f03b41475a2b80b985b02025-09-14T22:02:32+03:00Georgi Gerganovmetal : remove memory pools (#15966)
116a0e13dcbe5bae7025660349ef3e4ead060e507f2a14bd350141fb42b8bf2dd2342cebc27bfdce3992025-09-14T22:20:35+08:00lcybuild: fix the build failures of Windows HIP release job (#15984)
1176380d6a3e709cb02a8695afdd96b40e674477332aa0c461efe3603639af1a1defed2438d9c16ca0f2025-09-14T13:37:03+08:00Aaron Teoggml-zdnn: rm user mapped buffers (#15965)
11855758b00cae1451a0d789c50a5eb8c9bee42b9a0f161463a54d9f93d41246286aa4a9569a91d804d2025-09-13T16:24:22+03:00Georgi Gerganovmetal : refactor kernel loading (#15964)
119f161463a54d9f93d41246286aa4a9569a91d804d84d7b2fca11d1be118ce776f6d72a486c4883b742025-09-13T13:54:28+03:00Georgi Gerganovmetal : allow ops to run concurrently (#15929)
12084d7b2fca11d1be118ce776f6d72a486c4883b7440be51152d4dc2d47444a4ed378285139859895b2025-09-13T12:45:04+03:00Georgi Gerganovmetal : fix memory leaks (#15962)
12140be51152d4dc2d47444a4ed378285139859895b4bf5549269d99bac936a65892da2312cc71b24212025-09-13T02:39:52+08:00Aaron Teoggml-zdnn: fix #15414, activate FP16 and BF16 acceleration and incorrect zTensor free (#15839)
122f4e664f838cc65d89fa845c48c372e43852112e4f088b6a84f6aed0abb619dbb9d375e726fc888a62025-09-12T23:16:32+08:00Haiyue Wangcontext : remove redundant explicit casting to the same type (#15948)
1236c88ad8fa741a2182a2dcf8c5353ae4b5c66e656704d90c987cdf00751567b2088c4e54742aa2d3f2025-09-12T09:06:20+02:00Mathieu Baudiervulkan: Make device memory check more portable (#15939)
1240f0a3c2851134d49955f3c85afbb0b1bb47c3e0733daece86b65607451d0d4378d2d04ba6a20ad552025-09-10T17:52:35+03:00Georgi Gerganovmetal : make the backend async (#15906)
12533daece86b65607451d0d4378d2d04ba6a20ad55e7b6d83b524bbc24a1343d862de6dba8e8eddbd62025-09-10T15:39:57+02:00Daniel Beveniusci : add caching for ROCm installation in release workflow (#15924)
126e7b6d83b524bbc24a1343d862de6dba8e8eddbd62cfef4d117d67ab1dec002915b48a15d11ee19732025-09-10T14:17:09+02:00Daniel Beveniustests : filter out no-ops from coverage report (#15900)
12710d8b2b6b0ac2cae252a80b4daea5da55ab63c2f28b5f190ef1dbea5edf82dbc8b4407b721fadd132025-09-10T18:42:00+08:00Chenguang LiCANN: Add ROPE sin/cos cache for reuse (#15912)
12828b5f190ef1dbea5edf82dbc8b4407b721fadd1386587da03bd78df8f4e7d8b111a0c1d2494d6ed02025-09-10T15:29:12+08:00Chenguang LiCANN: implement LRU cache for ACL graphs (#15814)
129ff02caf9eed261423289d1531a56536fbf57bfc2ae355f6f7108540297d8b7f7ae71d20fe610a0b72025-09-10T05:23:19+02:00Daniel Beveniusci : cache ROCm installation in windows-latest-cmake-hip (#15887)
130ae355f6f7108540297d8b7f7ae71d20fe610a0b74f63cd705c7b6f457f36c63fdc053e07f6f3cc6b2025-09-09T22:26:03+02:00Ruben Ortlamvulkan: throw the oom error instead of no memory type found (#15905)
131a885dcff11a7b73f9377812d6151f6b15d307de0663027fd5490438ce9c27ea866a560e1e268d11f2025-09-08T10:27:07+03:00Georgi Gerganovbatched-bench : fix llama_synchronize usage during prompt processing (#15835)
132663027fd5490438ce9c27ea866a560e1e268d11fcf0e3ba1500bd23635b444f64ba23cbdb56c92ef2025-09-08T10:26:36+03:00Georgi Gerganovcontext : fix n_outputs during reserve (#15858)
133cf0e3ba1500bd23635b444f64ba23cbdb56c92efd413dca00360a7e4cb71441dacecfa32556fcc312025-09-08T10:25:33+03:00Georgi Gerganovmodel : avoid ggml_cont_3d for fused QKV weights (#15662)
1343b15924d71237a43bb5ad71f5b885ee66a82134279bc429262268ad2ac8a364cfe6c2d6b9c5f008a2025-09-07T10:19:45+02:00Daniel Beveniusggml WebGPU: remove userdata from request adapter callback (#15527)
135c4df49a42d396bdf7344501813e7de53bc9e7bb33c3635d2f20424d557b5b0605a2a356214ffe0482025-09-06T16:08:43+02:00Charles Xukleidiai: generalize compute_forward_kv_cache to compute_forward_fp16 (#15817)
13661bdfd5298a78593be649a1035ee2a120b13c4f001806e77714ae8a78130d432945b959a0956c56f2025-09-06T18:35:04+07:00Xuan-Son Nguyenserver : implement prompt processing progress report in stream mode (#15827)
137fd621880f3fd908424e675a41715a2dc760247a24281c7b315f8a904549fe527039b976e08098d1a2025-09-05T17:32:39-06:00Gabe GoodhartaLoRA Support (#15327)
138a81283820a466f2ace06ce4d4bc9512761f9365fc610b6c11b1ef7d678671dcf15acd7187a7ad8f32025-09-05T11:34:28+02:00Erik Scholzgguf: gguf_writer refactor (#15691)
139c610b6c11b1ef7d678671dcf15acd7187a7ad8f35d6688de08e73acc2532d668380801ed79d704eb2025-09-05T10:39:22+03:00Georgi Gerganovkv-cache : fix SWA checks + disable cacheless iSWA (#15811)
140fb15d649ed14ab447eeab911e0c9d21e35fb243e856ed0947f27b4ec3ad269fceda0402fbab263d32025-09-04T18:10:29+02:00Daniel Beveniusllama : add support for EmbeddingGemma 300m (#15798)
141d1e2adba65208d6de3d7f6f23b00c562ff5bb777c1c354e44c06d259679bb5bb7a8fa9f0b28480e42025-09-04T15:40:44+02:00Daniel Beveniusllama : set n_outputs to 1 to avoid 0 outputs mean-pooling (#15791)
142c1c354e44c06d259679bb5bb7a8fa9f0b28480e4a68d9144262f1d0ef4f6ba7ad4a7e73e977ba78c2025-09-04T20:20:14+08:00Chenguang LiCANN: Refactor ND to NZ workspace to be per-device (#15763)
143a68d9144262f1d0ef4f6ba7ad4a7e73e977ba78cbadb80cadbc40e047b30c43611aba575fc8d68452025-09-04T11:50:23+02:00Xuan-Son Nguyenserver: add exceed_context_size_error type (#15780)
144239b60e8986bbcb944f57311a02ac994431bf652dff7551bfdbdd6e57c13e523d7dcca317640e9072025-09-04T11:03:02+08:00Chenguang LiCANN: fix acl_rstd allocation size in ggml_cann_rms_norm (#15760)
145d4d8dbe383e8b9600cbe8b42016e3a4529b5121935a42edac84690990a75726898d975cd08d220c02025-09-01T22:17:42+03:00Gilad S.vulkan: use memory budget extension to read memory usage (#15545)
14602c1813517412f3e00aa6ca7c0273fea64edb49277dee9de97be75b7143a213bc48893e0c0b29af72025-09-01T16:19:07+02:00Ruben OrtlamVulkan: Add Integer Dot Product mul_mat_vec shader for legacy quants (#14903)
1473dc7397a2799bdc07bccf637ab7ae5a1e786d1a4e92d53b29e393fc4c0f9f1f7c3fe651be8d36faa2025-09-01T08:57:00+08:00hipuddingCANN: fix RoPE cache issue on multi-device (#15629)
148e92d53b29e393fc4c0f9f1f7c3fe651be8d36faa0d161f021aa33ec0e90cce96f5d1a889255573272025-08-31T20:41:02+03:00Georgi Gerganovsampling : optimize samplers by reusing bucket sort (#15665)
1499777032dccd67bdc7785aeab7497014a8be8dacc7d3c9f2b217acf0ce5db81ae83d3f375f49ab2c72025-08-31T06:49:03-07:00Diego Devesallama : separate compute buffer reserve from fattn check (#15696)
150b97c9edc59d4a1b4069991aa670411190f4f3a3e94e82c7eadeb8fff0db4bfd1ab6d8cf65fa6f2e02025-08-31T01:30:54-05:00Jeff Bolzvulkan: Allow fallback to sysmem memory when vidmem is full (#15649)
151696fccf354e9dbdfbce135bc40b44c9dcc64dda9ef476916bba4b44f44be0c98babc1cb025968e752025-08-30T04:11:22-05:00Jeff Bolzvulkan: Skip syncing for prealloc_y when it is reused (#15544)
152e8d99dd0b67f2ecc1e45fca8074a3a18c3e036d2a8bca68f727844e7dcf24a956003b3c2039ea5632025-08-28T18:39:31-06:00Gabe Goodhartnvidia nemotron nano v2 (nemotronh) (#15507)
153c8d0d14e77c3c45df5cbbddde9c2b4c377ec4e7a84ab83cc0b4b7e769451ee48e4c7d1acef91ef252025-08-28T17:09:05+03:00Georgi Gerganovkv-cache : fix find_slot to not search for continuous slot (#15638)
1548a4280ce431da6b33e5a95ae1fd61472c8c3f8cc64387f6e95434b393ac3df285864692b7fd9c4d22025-08-28T12:27:02+03:00Georgi Gerganovkv-cache : remove LLAMA_SET_ROWS checks (#15505)
15547373271f971aa5a0a6462b286f4a7b5bd4ba6441bded5a3b3376b2aa3ba2f11ade5910c550f354b2025-08-27T13:58:54+02:00uvosHIP: Enable support for ggml_backend_cuda_register_host_buffer (#15615)
1561bded5a3b3376b2aa3ba2f11ade5910c550f354b1e7489745a74996fc36e8fd05b73aa16bc184e0c2025-08-27T13:55:12+03:00Georgi Gerganovkv-cache : better estimate of n_kv for multi-sequence batches (#15610)
157a6a58d64785cb458ed9de52f391aa38142d38d640373486dbc0dccbdcb3b5fdd65759d88cec061962025-08-26T21:05:25+05:30shalinib-ibmllamafile: PowerPC Sgemm Optimization (#15558)
1580373486dbc0dccbdcb3b5fdd65759d88cec0619662cef26ac5b6b7acb635d3dc963813b43952dc2b2025-08-26T17:45:17+03:00Georgi Gerganovgraph : fix assert in memory-less build_attn (#15590)
159b3964c1e890ef8c947afb36a5124ce6fcb2136d479a546220c719e6a70627b243a478ab8d84dc9e12025-08-26T14:22:14+03:00Georgi Gerganovmetal : optimize FA vec for large sequences and BS <= 8 (#15566)
16085cc1ae998e4898d9fa992cb9b8620338cee97bf1d8d83deaa48d4a5491820d58ff1c0d8cf9d196c2025-08-26T12:47:00+03:00Georgi Gerganovcontext : print graph stats for memory-less contexts (#15586)
1610115cfb7c1dec0753aecb34f3af4338bf9de8eeb3a617cb18121a85eed17ac5585788810c5d9c1e0 0d5a470223fc90b6b6807921d68011ff06ae7f9e2025-08-26T15:19:56+08:00Yunzhe JiaMerge branch 'master' into dev
1626b64f74b55628e4193f4fb00313f07dbd85565280d5a470223fc90b6b6807921d68011ff06ae7f9e2025-08-25T13:56:43+03:00Georgi Gerganovbatched-bench : fix unified KV cache handling + pp timing (#15562)
163c247d06f38fc09059c9607a28aa44f5ff6be208d043fb27d3808766d8ea8195bbd123597272644022025-08-25T10:32:21+08:00Chenguang LiCANN: ROPE cache sin/cos repeat (#15501)
164b730706a49e576fb882dc34d9966345778b3ab0bc9a24fb93208fbbd3da6d903eb75431bfa97e59e2025-08-24T13:07:07+03:00Georgi Gerganovkv-cache : support layer reuse (#15504)
165c9a24fb93208fbbd3da6d903eb75431bfa97e59ea9c6ffcbfacee092bfaaa400306fceda181997372025-08-24T04:24:25-05:00Jeff Bolzvulkan: Support FA with any multiple of 8 head sizes (#15537)
166611f419cff11e4952228162a1c44cb35dff2274ab1afcab804e3281867a5471fbd701e32eb32e5122025-08-23T13:16:17-05:00Jeff Bolzvulkan: optimize rms_norm, and allow the work to spread across multiple SMs (#15281)
167289bf4113ef5c02d8f5eb0cf2d86683d8b8bc4d9b55f06e1aa67fb10e89f53e31bbccf37eb2678ea2025-08-23T02:33:36-05:00Jeff Bolzvulkan: Rewrite synchronization to allow some overlap between nodes (#15489)
1680a9b43e507a359ca392c037cf341f55137ad0b69330c3d2d21b55bca5517db7d2eea2ea8f131df4a2025-08-23T08:35:21+02:00Aclyvulkan : support ggml_mean (#15393)
169330c3d2d21b55bca5517db7d2eea2ea8f131df4ae92734d51bcb82cc35f0a6b5a14928f0036b2c902025-08-23T01:31:54-05:00Jeff Bolzvulkan: optimize mul_mat_id loading row ids into shared memory (#15427)
17045363632cbd593537d541e81b600242e0b3d47fc32732f2459a598606055f0403f0e4ec148d06d682025-08-22T11:28:03-07:00Reese Levineggml WebGPU: add support for quantization types (#15440)
1719ebebef62fd0adf8685874f154e227ea87b7c6f4ad5c975c2d0297124fad210776ef8eed6b90d5782025-08-22T12:22:13+03:00Georgi Gerganovllama : remove KV cache defragmentation logic (#15473)
172a0f98dd604d34826eb5ea2560d1e23fe726921df54a241f505d515d625767b993bfd573ecee306b92025-08-22T14:12:07+08:00Chenguang LiCANN: Optimize RMS_NORM using cache (#15419)
173cd36b5e5c7fed2a3ac671dd542d579ca40b48b543f196be84b1376945737163e35a91e29e3e24d2f2025-08-21T19:13:45+03:00Georgi Gerganovllama : remove deprecated llama_kv_self API (#15472)
17496452a3fa426de83494fb0268a636aa1bfe557fe9ad5e60dba38a6718366b7ac43e7d8e8abdc36c92025-08-21T09:55:00-05:00Jeff Bolzvulkan: Reuse conversion results in prealloc_y (#15410)
175029bb39eb1e7ae0e5df817ce97931abcd5fa52a930649cab657d87ac46692332a76e1b75d5d22e002025-08-21T17:52:16+05:00Ali Tariqci : enable RVV1.0 native build (#15386)
176ec5ab1a36c11dd3efcf4ec8d1ac89a13a8117bc31a99c2d948209d9ea5eac8b6dc0a2971072445402025-08-20T18:33:30+08:00Jie Fu (傅杰)common : fix context shift help message (#15448)
177d2fcd91cf96b46f4485ce46b4e3a32bf0df37715a6d3cfe7fa6ea1fb0e1ba8243b731db22ddc0b492025-08-19T16:46:37+03:00Georgi Gerganovserver : disable context shift by default (#15416)
178f0d3c7405c323784a60f14ddddfbac3f7404d417f08c4c0d8d0cb6caaf8b7ad316039232b9fa059c2025-08-19T08:45:12+03:00Georgi Gerganovbatched-bench : use rand tokens (#15398)
179e5155e698645242d4f019267ecc40ea9bad81b0921c17b5befc5f6be5992bc87fc1ba99d388561df2025-08-17T18:28:58-04:00Oleksandr Kuvshynovserver : export max observed n_past value (#15361)
180de5627910df74298c998e6bb36ee3217375a571965349f26f2299e06477ec8e85e462430468013582025-08-17T03:41:45-05:00Jeff Bolzvulkan: Optimize argsort (#15354)
181ff27f80a74bbe5303acd511a6781a1de6d619b3cd3248d9b6557c75d59954c594bb53cf517591e912025-08-15T21:11:22+08:00Aaron Teoggml: initial IBM zDNN backend (#14975)
1827aeee88cfe9c0434eac722b0f7c21404f48758a5b07791aa1d4831a08ad54ca19aa206c0c5c0f34a2025-08-15T03:27:02-07:00Diego Devesaci : move ccache action to ggml-org fork (#15328)
1834227c9be4268ac844921b90f31595f81236bd317df36bce667bf14f8e538645547754386f95163262025-08-14T23:21:24+02:00Johannes GäßlerCUDA: fix negative KV_max values in FA (#15321)
184863d341eeb81db104902b14b6f9413daa515e957d32e03f4495d3efa1c5126f53b449f1d429c56642025-08-14T08:38:10-05:00Jeff Bolzvulkan: perf_logger improvements (#15246)
185810b9fc8b99dd55517a3e94c6dee29748d4825594ebd0c125b24a0d7a78b0ffc1d9567530ed8f0c42025-08-14T20:03:30+09:00kallewoofperplexity : provide a helpful hint for has_cpl case in split_equal error. (#15304)
1865cdb27e0917479d2d742cea7beee089574bb09fa3ea913f1ce9567289aedd866a569dbab8fb8e4192025-08-14T03:03:57-07:00Jonathan Graehlfinetune: SGD optimizer, more CLI args (#13873)
1873ea913f1ce9567289aedd866a569dbab8fb8e41929c8fbe4e05fd23c44950d0958299e25fbeabc5c2025-08-14T15:16:32+09:00kallewoofperplexity: give more information about constraints on failure (#15303)
1881adc9812bd33dc85489bf093528d61c22917d54fb3e16665e14032d1276f9d0263acef8321b6f5182025-08-13T11:21:31-07:00Bas Nijholtfix(nix): remove non-functional llama-cpp cachix cache from flake.nix (#15295)
189d8914fc47e8a69b28c670325cb1c8ce33e3a2960e885445bc1719d2e289a9b2e11714e0f994e68be2025-08-13T12:44:40+02:00Copilot common : add --override-tensor-draft, --cpu-moe-draft and --n-cpu-moe-draft parameters (#15191)
1906028bf74351d35a06bd98498624f8c2f029f7d1abc5182272c373267352bc689e5fca276934bea2d2025-08-13T10:04:46+02:00Oliver SimonsCUDA: Optimize `reduce_rows_f32` kernel, leading up to 25x perf improvement on kernel-level and 10% perf increase for Gemma3n (#15132)
191228f724d9ce6c56e8cec75bfdabab4dd013def7fcd3069dfcbeee8e0e96cffb93b0ebb9e595e273a2025-08-11T13:58:24+03:00Georgi Gerganovkv-cache : fix seq_rm with seq_id == -1 (#15226)
192cd3069dfcbeee8e0e96cffb93b0ebb9e595e273a50e81bdf5db563ab57a9a722b08f96fa8a76c9272025-08-11T11:21:19+02:00Daniel Beveniuskv-cache : log (debug) all streams in find_slot (#15176)
1935fd160bbd9d70b94b5b11b0001fd7f477005e4a0756cfea82608911bbfcbf45164b8fdaddbafaa312025-08-06T15:14:40-07:00Reese Levineggml: Add basic SET_ROWS support in WebGPU (#15137)
1941cbf442f50a154aac0bf35d7b8b3436a2cdfd6f5ba67d6c03fed3193987a4ef13050e6e2a4451df42025-07-24T15:50:19+08:00Yunzhe Jia1.sync with calrt prefill,decode,kv_update op 2.add hw pattern untile function
1959515c6131aecaccc955fdedcfe16c3e030aaefcbfd1234cb468935ea087d6929b2487926c3afff4b2025-08-05T16:26:38-07:00Reese Levineggml: WebGPU disable SET_ROWS for now (#15078)
196fd1234cb468935ea087d6929b2487926c3afff4bf324a3b715d5c1081c110ce459f8a8486fb1ee892025-08-05T22:10:36+03:00Georgi Gerganovllama : add gpt-oss (#15091)
197ee3a9fcf88fe5b5e1213711e05861b83cd4fdfe6ec428b02c347767f24c78111309e3f30d2ada2892025-08-05T05:27:45-04:00compiladecontext : fix index overflow on huge outputs (#15080)
198ef0144c087b33e5b8da42d529ac71aaf05cb49df2721257e3e2c4c944ac8a08221113ee7cb503f1b2025-08-05T04:29:25+10:00Sammodel: support GLM 4.5 family of models (#14939)
199587d0118f50b7e8f4bafbcdd218aefd9da0272e15aa1105da24a8dd1661cea3db0582c9b2c2f54d32025-08-04T08:52:43-07:00Reese Levineggml: WebGPU backend host improvements and style fixing (#14978)
20011a3811164ef2d75393c6b0a632f4c608e3e3dd297366dc6abdd0bdc74260bd3c42bd06f0feb74282025-08-03T15:43:07-04:00compiladememory : handle kv_unified for hybrid models (#15050)
2014fdea540bda4648f98b85e8ee9dc66db4bfb5945a4569c41fd2253c89ef52fc2378687bdbf42f61a2025-08-02T16:14:57+02:00Daniel Beveniuskv-cache : skip alignment of n_stream in kv-cache log msg [no ci] (#15040)
20215e92fd33791e60a4ddb5970b47242a855c271172bf3fbf0b54f97aef2b388b76d222789e1c170f12025-08-02T17:13:05+03:00Georgi Gerganovcuda, sycl : fix batched gemm when ne02 == 1 && ne03 > 1 (#15038)
20394933c8c2eeaa9a7983e3f6c08af76bd86724094c1dacaa99b4ead6edbac928cd2da59436573f6b02025-07-31T05:25:23-07:00g2mtserver : implement universal assisted decoding (#12635)
20492b8810ec7aa6d778bc287cc918443cf67b962e200131d6eaf4df029e1ec84de868c2c59575030072025-07-30T15:46:13+02:00Johannes GäßlerCUDA: skip masked KV slices for all FA kernels (#14924)
20500131d6eaf4df029e1ec84de868c2c59575030071e15bfd42c3938506e0da5939bf7f42780965f012025-07-30T15:12:02+03:00Georgi Gerganovtests : update for LLAMA_SET_ROWS=1 (#14961)
206ca0ef2dddb022cb1337d775cd05cd27d7808aff489d1029559bd2968f76db854f9f113d73e34527c2025-07-27T12:10:51+02:00Daniel Beveniusllama : clarify comment about pp and tg graphs [no ci] (#14895)
2071dc9614e0673e794d2e2bf88ba04f7d57b63a57b446595b9b3a113d9ba10506922c3a156cca9d4772025-07-27T16:38:44+09:00Shunta Saitollama : fix kq_scale for the attention layers of PLaMo2 (#14892)
208793c0d7f46384001738c337d7afa46b45ae32745ce111d39d666f4a3b6a561ad020f6feb8cc677902025-07-25T10:47:39-06:00Gabe Goodhartmetal: SSM_SCAN performance (#14743)
209c1dbea752a630169c104118fd0c82f3ffcb19c91749e0d27f0247337869f4698f59dd7fafba943262025-07-25T14:28:06+03:00Georgi Gerganovcontext : restore preemptive sched reset when LLAMA_SET_ROWS=0 (#14870)
21064bf1c3744053cf7def10aeed21ff48883ee755bc12bbde37258c6d053322d1cedd4a9672109ae582025-07-25T06:17:02-04:00Chris Rohlfrpc : check for null buffers in get/set/copy tensor endpoints (#14868)
211e4868d16d24dec55e61bcaadaca28feed8f98b13820de57d4faa427a3d0bfb14e48057247fae036e2025-07-24T16:31:48+03:00Georgi Gerganovcontext : perform output reorder lazily upon access after sync (#14853)
21207a19e27a26f76d34be62da53807f93131fb3cab18f3b5ff9e5eda4e7d04bceff8ffdccb0a696ed82025-07-23T12:35:53+02:00Johannes GäßlerCUDA: fix quantized KV cache + multiple sequences (#14822)
2137233358d29df3dfbf80693f2de7e5865401ad7246c88b3bb2509d980e6a64c50fdf8dd304929f7702025-07-23T16:16:41+08:00l3utterflymemory : handle saving/loading null layers in recurrent memory (#14675)
214d4d1522b20809a350ffb094db20f40f17d3ab80fd1aa0cc5d13ec3fa553de5d298f9166679e584792025-07-22T23:01:29+08:00Molly Sophiallama : add model type detection for rwkv7 7B&14B (#14816)
215a979ca22db0d737af1e548a73291193655c6be9990083283ec254fa8d33897746dea229aee401b372025-07-19T21:59:08+02:00Ervin Áron Tasnádiggml: adds CONV_2D op and direct GEMM Vulkan implementation (#14316)
21690083283ec254fa8d33897746dea229aee401b37d4b91ea7b2da253e1355b503f0fcb7b428ce005d2025-07-19T12:51:22-04:00compiladeimatrix : use GGUF to store importance matrices (#9400)
217021cc28bef4dd7d0bf9c91dbbd0803caa6cb15f2d498af3d5a00f96bdd37b534860f03a6d9e98d392025-07-18T13:35:32+02:00Oliver Simonscuda : Fix Gemma3n not executed as CUDA_GRAPH on NVGPUs (#14741)
2188f974bc1e980c06833504276021072e7a4088c8109651d09ffc1e941bd1be23163abf5495c4165472025-07-18T08:29:28+03:00Georgi Gerganovgraph : refactor context to not pass gf explicitly (#14629)
21909651d09ffc1e941bd1be23163abf5495c416547349ea79fcebc75b0c55bf61594a47736966d4f952025-07-18T06:25:54+02:00Nexes the Eldergraph : Pass the graph placeholder message in debug mode (#14748)
220d6fb3f6b49b27ef1c0f4cf5128e041f7e7dc03af01612b74090df592663cfa01f661c9628f403b592025-07-17T20:52:33+03:00Georgi Gerganovkv-cache : fix k-shift for multiple streams (#14742)
22101612b74090df592663cfa01f661c9628f403b59086cf81e88fb75287b71ff19c08a206b7bc2e02f2025-07-17T19:08:33+03:00Georgi Gerganovllama : reuse compute graphs (#14482)
222d9b691081c04ec5fb0daa9d2b979f915c142963dad57d3edd2f48cf6dc41a98fd9b303435ecb4fb02025-07-17T09:49:15+03:00Georgi Gerganovkv-cache : opt mask set input (#14600)
223ad57d3edd2f48cf6dc41a98fd9b303435ecb4fb01ba45d49822c39ca9a552c7b75efe0495ff400c32025-07-17T09:45:54+03:00Georgi Gerganovbatch : fix uninitialized has_cpl flag (#14733)
224496957e1cbcb522abc63aa18521036e40efce98521c021745d781edf9c44b4972ef6cbbf53b0ecff2025-07-16T15:17:25-04:00Diner Burgerllama : fix parameter order for hybrid memory initialization (#14725)
22521c021745d781edf9c44b4972ef6cbbf53b0ecffb0f0ecc3dce806c68609d375a2b3edc430d8db182025-07-16T08:18:51-07:00Reese Levineggml: Add initial WebGPU backend (#14521)
226225e7a1438f4ea85eaa7b5ef3ab3b266ee4d9c06ab140198211385b85eeeb0abd549a4bbe259e10d2025-07-16T16:35:42+03:00Georgi Gerganovllama : add high-throughput mode (#14363)
227ab140198211385b85eeeb0abd549a4bbe259e10d64978340b0b4a0a6e2fb74270c1509383d2eff322025-07-16T20:03:51+08:00Aman GuptaSupport diffusion models: Add Dream 7B (#14644)
22879e0b68c178656bb0632cb8602d2940b755077f8c81f4192f91a1e209c1eec7a84fe5371ef9175da2025-07-16T12:00:42+08:00Min-Huallama: add LLAMA_API to deprecated llama_kv_self_seq_div (#14708)
229ba1ceb34566c889a1fc500efa79799ffed25d9b010a0351a97c25471aea0bbde9cca54d32d163eec2025-07-15T14:51:09-05:00Jeff Bolzvulkan: fix noncontig check for mat_mul_id splitting (#14683)
23068e37a61a7b24863f67541db65b4f6195962a268cbc68be51d88b1d5531643b926a4b359c3cff1312025-07-16T01:11:42+09:00Shunta Saitomodel : add PLaMo-2 support (#14560)
2319c9e4fc6354fc811efa06a8eb7a86d3315cec9c8494c5899cb76859f32ddd913534f2685fd684a3d2025-07-14T21:01:41+08:00Aman Guptallama-context: add ability to get logits (#14672)
23265a3ebb0aa56d6c501466e0f950ad15105fd32d80d9226763c82562186122f3b827fa3862864a19c2025-07-14T10:37:35+01:00Anton Mitkovsycl: Batched mulmat rework for oneDNN dispatch (#14617)
2330d9226763c82562186122f3b827fa3862864a19c982e347255723fe6d02e60ee30cfdd0559c884c52025-07-14T07:43:43+08:00Molly Sophiallama : add jinja template for rwkv-world (#14665)
234b3ad3a0191994d6c47b2bd389d5c7431526ecd2c98197e5c98388470030d908f355ec5937dcccaaa2025-07-12T05:12:26-05:00Jeff Bolzvulkan: support SET_ROWS (#14587)
23598197e5c98388470030d908f355ec5937dcccaaaf5e96b368f1acc7f53c390001b936517c4d189992025-07-12T04:51:58-05:00Jeff Bolzvulkan: optimizations for deepseek prompt processing (#14555)
2360aedae00e6fb48680324a5ac5da9cba0e35de6b56bdda13981d6c8189b7dc4f9fb8ecb91c21529f82025-07-10T18:20:13-06:00Gabe Goodhartmodel : Granite Four (#13550)
2374a5686da22057867c23bd4a6be941ddc8c51e58598bab638fb28cf95a5a66dd2d51b40d6c8f6d69a2025-07-09T14:59:57-04:00compiladellama : support Jamba hybrid Transformer-Mamba models (#7531)
2386efcd65945a98cf6883cdd9de4c8ccd8c79d219a699f4392a33f57c3352cf8d60bdc53db7ca235e72025-07-08T13:11:42-05:00Jeff Bolzvulkan: optimize flash attention split_k_reduce (#14554)
239bb4f7a9e4eec171fecf0f640b1337a1c24485560b8eeb8741d4483daf576498cf90537b4de71633c2025-07-08T11:37:47-04:00compiladememory : fix broken batch splits for recurrent cache (#14575)
24067d1ef23c68e1e09dc6d7423d1ef5fc0ea56ed5e7b50f7c0257695d0f6918bffce4e68d5c13b0c9e2025-07-04T09:08:59+03:00Georgi Gerganovbatch : add optional for sequential equal split (#14511)
241c79184d2d192489e3c918bab8ed717d22f8c02bd499a8f5a787f5fdc7f3fdfb9d1ff01b6cb5e994e2025-07-04T09:04:59+03:00Georgi Gerganovbatch : add n_used count (#14512)
242a70c8a0c4b4c1606cd9a0ba889ce61aa886100959067487c4411efb20400103fcccfdd389c80d4282025-07-03T10:53:35+03:00Georgi Gerganovkv-cache : use ggml_set_rows (#14285)
2439067487c4411efb20400103fcccfdd389c80d428d4cdd9c1c3cebdafca735958597de4ff7b7c0f542025-07-03T10:46:57+03:00Georgi Gerganovggml : fix FA mask dim 2 and 3 (#14505)
2445d46babdc2d4675d96ebcf23cac098a02f0d30cce17991c466ac835b2c71bd813c1ca7ff8dd97b942025-07-02T13:10:24-04:00compiladellama : initial Mamba-2 support (#9126)
2458875523eb311cac832bfda0c581e852292185ae9ec68e84c32325a3417fbcd2e60d4bda6adb4e4bc2025-07-01T03:32:56-05:00Jeff Bolzvulkan: support softmax/FA batch and broadcast (#14449)
246307e79d33d4cdd9f1d6c42fc861724e6ba12b98fd7f5f4e578d1f60b0835d1734a50438c309b3e5c2025-07-02T20:38:10+08:00zhouwgopencl : fix possible buffer overflow in dump_tensor (#14490)
247d7f5f4e578d1f60b0835d1734a50438c309b3e5cc8a4e470f65c3a932e18eddc5fba1844876d74632025-07-02T14:12:07+03:00Georgi Gerganovsimple-chat : fix context-exceeded condition (#14494)
2486a746cf9c40f8d93d13eb8a6c2b989a15e7fa61aeff5e45443a248f89cc61e64786f38b68b4da4892025-07-01T03:43:08-05:00Jeff Bolzvulkan: Split large mul_mat_id to fit in shared memory (#14451)
249745f11fed09ba2303f3f494efb12286d382608e55dd942de5922a22ec8446a4ad2203738dbcb93892025-06-30T18:03:03+03:00Georgi Gerganovmemory : correctly handle failure in apply() (#14438)
250caf5681fcb47dfe9bafee94ef9aa8f669ac986c783790b0e7e09ab17238b16452a33053a71dbdfad2025-06-29T20:02:53+02:00matteoserver : support jinja extra template kwargs (Qwen3 enable_thinking feature), from command line and from client (#13196)
25183790b0e7e09ab17238b16452a33053a71dbdfadf47c1d7106e49062279bcc57fc1077c0db61e2782025-06-29T19:29:57+02:00Renatserver : fix appearance of the chats list context menu for Safari (#14322)
25227208bf657cfe7262791df473927225e48efe48263a7bb3c7e1c6b0a92d03b0a594d3cd501d6ed3e2025-06-29T01:30:53+08:00Aman GuptaCUDA: add bf16 and f32 support to cublas_mul_mat_batched (#14361)
25300d5282c7f2a0bb05bb315fb81ca3b0f42cf9f07566c16fcce44876a167c37f159085afe6f84b28c2025-06-28T10:17:09-05:00Jeff Bolzvulkan: lock accesses of pinned_memory vector (#14333)
25472babea5dea56c8a8e8420ccf731b12a5cf3785443678060c1f4cfab2f899466fd615e358677f8072025-06-27T21:42:02+03:00Georgi Gerganovgraph : make llm_graph_context destructor virtual (#14410)
25543678060c1f4cfab2f899466fd615e358677f8078d94219a4a7f2da72ee542019ca01f36af93d1d62025-06-27T17:55:45+03:00Georgi Gerganovrecurrent : call balloc split_reset() in init_batch() (#14414)
2565783ae43599400b723b5da0569c1f848419ff3c7bf5bcd0b857db420235e03639f0a5f218a7f8cf82025-06-26T15:50:15+03:00Georgi Gerganovmetal : batch rows copy in a single threadgroup (#14384)
257ba67d6c03fed3193987a4ef13050e6e2a4451df48f52c6f8abb2ed9ad967cc0bfb90acb0730111932025-06-26T17:42:32+08:00Yunzhe Jiallama-calrt: infer-op: copy data from output buffer to dst-tensor.data
2588f52c6f8abb2ed9ad967cc0bfb90acb073011193d2f325130c4ed77a3dcff2c23a926587fb08e2cc2025-06-26T15:10:45+08:00Yunzhe JiaSquashed commit of the following:
25973e53dc834c0a2336cd104473af6897197b9627762af464227dafa1c55e0535bcb24346326748f462025-06-24T11:46:25-07:00lhezopencl: ref count `ggml_backend_opencl_context` and refactor profiling (#14254)
26062af464227dafa1c55e0535bcb24346326748f46c148cf1946275952a79ad50b6199725f12a704112025-06-24T18:26:30+03:00Georgi Gerganovbatch : fix check for empty sequences in memory (#14364)
26172c6bc3f3d0cf3bf160dddf1b803fed52bcbb0a3defe2158dd5e250b4ef53994057a4ec03131a2632025-06-23T19:56:19+08:00Molly Sophiallama : better rwkv chat template and add missing `inputs.use_jinja` setting (#14336)
262defe2158dd5e250b4ef53994057a4ec03131a2637b50d589a863c7631135c1226f6eab65cb4062122025-06-23T13:11:31+02:00Johannes GäßlerCUDA: mul_mat_v support for batch sizes > 1 (#14262)
2637b50d589a863c7631135c1226f6eab65cb4062123a9457df962b5883f2773f1c295e8c19df60d89f2025-06-23T12:27:35+03:00Georgi Gerganovkv-cells : fix tracking of seq_pos (#14339)
2645d5c066de8a3d2cb32f04c4d5ad1560945f30bf340bfa04c95c19fb42bafd4e21b5c2a77718468012025-06-22T21:44:57+09:00yuisekimtmd : fix Pixtral OOM with large images by capping image_size to 1024 (#14326)
265692e3cdd0a069ab56411b64506a67537d767683eb23fa0b3f40165ca3aae8ad4ee756e72f9a130dd2025-06-21T08:03:46+03:00Georgi Gerganovmemory : rename interface to llama_memory_context_i (#14296)
2664c9fdfbe1580a66fd7d77c77418ce2c606a29fdd9eaa51e7f08593f123f00136591179a8f5956ecd2025-06-20T10:14:14+03:00Georgi Gerganovubatch : new splitting logic (#14217)
267d2f325130c4ed77a3dcff2c23a926587fb08e2cc6391ee7ffd97e723dfbdcc32f001e1b95a63cba62025-06-17T11:53:33+08:00Yunzhe Jiacalrt: add input_token in graph; input_token will be copy to calrt_in_buffer
268d67341dc18fc5cc63362880ab2f8f9ecfc7932e7456af35eb70177b8dd5779b6d4c21bb020f9cebd2025-06-19T16:01:03+03:00aa956server : add server parameters for draft model cache type (#13782)
26910bb545c5b54175ed9874ad8d187effa2bcb4b5fedc4a29effe716956fdfd2bc5b9cba2a6d8492f82025-06-19T09:15:42+02:000cc4mVulkan: Set device max size for host memory to avoid OOM warning and fallback to CPU buffer (#14249)
270edc4a29effe716956fdfd2bc5b9cba2a6d8492f8ed3290ab34493fd3d2e0f925f816a401da4f2dfd2025-06-19T00:08:14-05:00Gabe Goodhartmemory : Hybrid recurrent cache (#13979)
271bbe98d27840453c8787d18470963530fdc27d89fc2056ed6d461e6d5432f04f221e221ab795dc6522025-06-13T09:05:44+02:00Daniel Beveniusggml : remove unused ggml_context_container (ggml/1272)
27230e5b01de2a0bcddc7c063c8ef0802703a958417e54b394082de242be4ee2e692b11fcc8d4eba3712025-06-15T17:53:45+01:00Ed Addarioquantize : change int to unsigned int for KV overrides (#14197)
2735fce5f948df8f189a5401a8ecaa9753106e75abb9ae4143bc6ecb4c2f0f0301578f619f6c201b8572025-06-15T10:52:11+03:00Georgi Gerganovkv-cache : fix use-after-move of defrag info (#14189)
274b9912ac570de8945ae9383c9ca8291027bf287dd00ba7726100d7e1941d9f5a06f56a7559945b33c2025-06-15T09:18:37+03:00Georgi Gerganovbatch : auto-gen positions + verify multi-sequence input (#14177)
27580709b70a2f87c13ccaf1480b79939310999678926ff3685bfbaa4c8838d7afd988b17dd5eb99f922025-06-13T18:35:00+03:00Georgi Gerganovbatch : add LLAMA_BATCH_DEBUG environment variable (#14172)
27660c666347becacff81cd4bc9a52038ba71038e41b7cc7745e38141060845145af0a1fd489d8e3e332025-06-13T13:47:55+03:00Georgi Gerganovbatch : rework llama_batch_allocr (#14153)
277ffad04397399ea1650fda6560c7c7530598048760889eba570126f8a2f5a0e88fde776bbc91cca662025-06-13T11:18:25+03:00Georgi Gerganovserver : fix SWA condition for full context reprocess (#14163)
278c61285e7396c8e526fe7794c19e8d4f1c99bfc5109cf2c7c655c90e53e100f29b830a788bab0653d2025-06-13T08:45:37+01:00Ewan CrawfordSYCL: Bump oneMath commit (#14152)
279f6e1a7aa8787b5c00acba6370cb70a0beff48b1ec3ee46fab49a765d2e32e171e9ed7a5fa121dd9c2025-06-12T11:50:01+03:00Georgi Gerganovcontext : simplify output counting logic during decode (#14142)
280c3ee46fab49a765d2e32e171e9ed7a5fa121dd9ce2c0b6e46a5596665569ae765f0993cea2619af62025-06-12T11:49:26+03:00Georgi Gerganovbatch : remove logits_all flag (#14141)
2819596506965f65be5d802ecef6a315fe43d2391a8a20b2b05bce6622c585459ebf46f142f113d021c2025-06-12T10:02:15+03:00Georgi Gerganovkv-cache : fix split_equal handling in unified implementation (#14130)
282a20b2b05bce6622c585459ebf46f142f113d021c2e89f76b7af2c0b827be785e445f2e2b3e52e1ca2025-06-12T02:56:04-04:00compiladecontext : round n_tokens to next multiple of n_seqs when reserving (#14140)
283bd248d4dc7265437e1918dc53ae3f49a0c592e5f7781e5fe99f2a4fc1c8af0a8488eedac4644cb722025-06-11T09:48:52-05:00Jeff Bolzvulkan: Better thread-safety for command pools/buffers (#14116)
28489a184fa7125b7c3e2fb567337cc2bd7bc2d376c2baf07727f921d9a4a1b63a2eff941e95d0488ed2025-06-11T16:48:45+03:00Georgi Gerganovkv-cache : relax SWA masking condition (#14119)
2857ae2932116a4de3141cbc8c488f7f6e4e06d81711f7d50b2936023b26eb218e944e62834b80a2ce02025-06-11T12:52:45+03:00Georgi Gerganovkv-cache : add LLAMA_KV_CACHE_DEBUG environment variable (#14121)
2861f7d50b2936023b26eb218e944e62834b80a2ce04c763c8d1b4d4de20bf364ec1837430783cba9842025-06-11T00:19:25-05:00Jeff Bolzvulkan: Track descriptor pools/sets per-context (#14109)
287dad5c44398b78467943ed0a603ea427fe9f6fc6255f6b9fa6563f6ae49113f9abdc980c12348cc1c2025-06-10T18:20:14-04:00compiladekv-cache : avoid modifying recurrent cells when setting inputs (#13834)
2881f63e75f3b5dc7f44dbe63c8a41d23958fe95bc040cbf571c9210031340f022d104317e89a1a6bb22025-06-09T23:05:02+03:00Georgi Gerganovmetal : use less stack memory in FA kernel (#14088)
28940cbf571c9210031340f022d104317e89a1a6bb27f4fbe5183b23b6b2e25fd1ccc5d1fa8bb010cb72025-06-09T23:04:35+03:00Georgi Gerganovkv-cache : fix shift and defrag logic (#14081)
2908f47e25f56e9792093b7497c68e9f80bab82ed19201b31dc2e6fef3de598280b53fd3b69420a4e492025-06-09T07:36:26-07:00Diego Devesacuda : fix device sync on buffer clear (#14033)
291e21d2d4ae26d6c5daf1602a1061aa4f8e722ca57dc0623fddb661926e0998538895155e4f081ff092025-06-09T19:47:39+08:00Xinpeng DouCANN: Simplify the environment variable setting(#13104)
292247e5c6e447707bb4539bdf1913d206088a8fc695787b5da57e54dba760c2deeac1edf892e8fc4502025-06-08T11:39:56-07:00Diego Devesacuda : fix buffer type check with integrated GPUs (#14069)
2930974ad7a7cd4bca846b15c484ff3be890135a52c745aa5319b9930068aff5e87cf5e9eef7227339b2025-06-07T14:13:12+02:00Sigbjørn Skjæretllama : fix llama_model_chat_template with template name (LLM_KV with suffix) (#14050)
294745aa5319b9930068aff5e87cf5e9eef7227339b487a5e0401423bba02cd6e97e4d45131bb20b22b2025-06-06T14:11:15+03:00Georgi Gerganovllama : deprecate llama_kv_self_ API (#14030)
295487a5e0401423bba02cd6e97e4d45131bb20b22bd17a809ef0af09b16625e991a76f6fe80d9c332e2025-06-06T13:29:18+03:00Georgi Gerganovcontext : fix SWA-related warning for multiple sequences (#14045)
2967f37b6cf1e2c1b90bf0d9c8d91904b4b6c5127483a077146a4761fdbd24bdd8eb098f46b8adc4dda2025-06-05T15:29:22+03:00Georgi Gerganovmemory : migrate from llama_kv_cache to more generic llama_memory (#14006)
2973a077146a4761fdbd24bdd8eb098f46b8adc4ddad01d112abb055549d33bb8aac1755eb0c40918ad2025-06-05T02:57:42-07:00Diego Devesallama : allow using mmap without PrefetchVirtualMemory, apply GGML_WIN_VER to llama.cpp sources (#14013)
2989e31bec4fd53634c9e5b04650488a09a055f5dab5a8ae3053ced350ed300ba91600519fcad1c6ba72025-06-05T09:06:29+03:00Georgi Gerganovcontext : fix pos_min initialization upon error decode (#14008)
2993e63a58ef7addec35408e2eb67850d7cdc935dc32589ad3704559f4dd860f5f303b19349c688a28a2025-06-04T18:58:20+03:00Georgi Gerganovkv-cache : refactor the update/defrag mechanism (#13988)
300e0e806f52ebcd0ee285c994fe8fd8b8787d2cb0a7e00e60ef86645a01fda738fef85b74afa016a342025-06-04T09:50:32+03:00Georgi Gerganovkv-cache : fix unified::seq_rm to work with seq_id < 0 (#13985)
301363757628848a27a435bbf22ff9476e9aeda5f40ea394d7ab1f8101716d48ce9421c94c71b93a00f2025-06-02T21:34:40+03:00Georgi Gerganovserver : disable speculative decoding for SWA models (#13970)
302bfd322796cd838f906535ff3352624fc46338894093e3f1feb16e25e58f7d61e01266c830dd424b82025-06-02T16:29:28+02:00Xuan-Son Nguyenmtmd : fix memory leak in mtmd_helper_eval_chunk_single (#13961)
3030fc16b42e8793364918e830c234c1f3caec29dae053b1539c02617eff744f89525ee57497c3c1fbe2025-06-01T11:39:27+03:00Georgi Gerganovkv-cache : split implementation in separate sources (#13920)
304803f8baf4f741d2f0465c46c33f285886b97a0713600cc2886956fc0a07ef6ad2f4128ccfdbc8c6f2025-05-31T15:58:33+03:00Georgi Gerganovllama : deprecate explicit kv_self defrag/update calls (#13921)
3053600cc2886956fc0a07ef6ad2f4128ccfdbc8c6fc7e0a2054b908c28bf93bb18d4b63ccbff2c41272025-05-31T15:57:44+03:00Georgi Gerganovllama : use n_swa + n_ubatch cells for SWA cache (#13833)
3063f55f781f1da9241f209648636fa0426fe62a49551fa76f172957c9916e43aeb581f82c533e123942025-05-31T12:55:57+03:00Georgi Gerganovllama : auto-batch preparation (#13845)
30712d0188c0dc6146ffde6d277a93f232ccbe699f8eb3949938e82a128855bc0676220bb2ce6e4228d2025-05-31T10:24:04+03:00Georgi Gerganovkv-cache : refactor + add llama_memory_state_i (#13746)
308b47ab7b8e9c4f6154eb6abb6234866ab117d64c7dd665cc9d4b378626cde11ea0c4c91f514fdc9942025-05-30T09:56:19-07:00Diego Devesasched : avoid changing cur_copy when a graph is already allocated (#13922)
309df0c0c7d02f951dc639d48fd536f79200460ac83b49a8ff96b769b8a4c36d89fb783ec0135be582b2025-05-30T07:37:18-07:00Diego Devesacuda : prevent using split buffers with 3d/4d matrices (#13919)
310b49a8ff96b769b8a4c36d89fb783ec0135be582b53f925074de02c5304b00c14b4d6d8910c58667d2025-05-30T19:40:57+05:30Akarshan BiswasSYCL: Add mrope kernel (#13755)
311db38704f0133be7832123495fa8fc2601ea999d407e4351ce663a7802b31f92fd7bc0e555c2044b62025-05-30T14:50:43+02:00Sigbjørn Skjæretconvert : fix rwkv bos/eos token (#13844)
31254a2c7a8cd8a32b44e3a98c2999b0f5c9114be5c21fcc21ad5e5de2daa5da8d08dbbcc86b8d815d72025-05-29T19:39:20+08:00Yibo Caiarm64: optimize q4_k_q8_k kernel with i8mm (#13886)
313763d06edb7dd5094ea58bad1d81e2e8d35033e3410961339b26bd2eff01d5479e8879f435da261b72025-05-28T22:35:31+02:00Xuan-Son Nguyenllama : fix KV shift for qwen2vl (#13870)
314a68247439bd6fb756cc93ad2817e55a02aa0b10026b79b6cb3e7840ff15729350e95907e19f9f4802025-05-28T13:33:37+02:00Johannes GäßlerCUDA: fix FA tg at long context for CC >= 8.9 (#13852)
31581713121ee56755ba4a147d1a1e3fa248ec31a66f9cd68398baf2ba8af4725ca9ed00bef205e67062025-05-27T13:49:41+03:00Georgi Gerganovkv-cells : track min/max used cells and per-sequence positions (#13808)
3164f81b33e324b1669b282eb81104e4a1131be7dcecdf94a18023c92f41808ec874ba577d9146747172025-05-27T09:40:59+03:00Georgi Gerganovllama : validate seq id batch input (#13809)
31779c137f77677b3c8ee3c60a7da033721b938399a22229314fc46b2f741bb21b12cde71f6c6a60b522025-05-26T14:03:54+03:00Georgi Gerganovexamples : allow extracting embeddings from decoder contexts (#13797)
318de2ef53a4b2c0d703749a309d19fe68fd8f1b9acc508256db2de2b032e19c8ed833f4683c827c9a12025-05-25T16:34:36+03:00Georgi Gerganovkv-cache : rework kv_cell (#13706)
3198a1d206f1d2b4e45918b589f3165b4be232f7ba8797990c4bca0dca5be295c63e3fb2800dc0a69c22025-05-22T22:21:07+03:00Georgi Gerganovtts : fix n_ubatch + make WavTokenizer cache-less (#13713)
320797990c4bca0dca5be295c63e3fb2800dc0a69c2ab86335760ebb441574eb47f886fa1ee302e21312025-05-22T20:42:48+02:00Xuan-Son Nguyenmtmd : add ultravox audio input (#13623)
321cc74d5be990e37f201591fd868a92e64abdbf9025be24af73d77ea23d399726f1d2a01a70ee863312025-05-22T16:33:39+03:00Georgi Gerganovserver : pad small embedding batches (#13692)
3226b56a64690a318fcabcd7739ac7e314d44785ea8a4e8912dfd4604be1e39bc86ba4c0b02969967ef2025-05-22T09:24:09+01:00Ewan CrawfordSYCL: Avoid using with SYCL-Graph for unsupported nodes (#13587)
323edbf42edfdabb9cea72ae12137570cf48f5d8076d643bb2c798df9c2cd61067d2692b1cd417df4022025-05-21T23:21:17+03:00Henry Linjamäkiopencl: fix couple crashes (#12795)
324797f2ac0625b22edeff03cc30e0f988da6b6b068b44890df2e4fad0ece1d5366dcbc8bedae23b6582025-05-21T15:11:13+03:00Georgi Gerganovkv-cache : simplify the interface (#13660)
32533983057d0f578aca74ba15eccc3de9c267a5ff6fb1cab201c6c4cd36731f100df74eece2f4706fa2025-05-21T09:58:49+08:00R0CKSTARmusa: Upgrade MUSA SDK version to rc4.0.1 and use mudnn::Unary::IDENTITY op to accelerate D2D memory copy (#13647)
326b7a17463ec190aeee7b9077c606c910fb4688b84be0239693c1530a18496086331fc18d8a9adbad12025-05-21T00:55:30+08:00l3utterflymtmd-helper : bug fix to token batching in mtmd (#13650)
327a4090d1174aed22dde5cacce2a4c27656b987a2fb69f1647f9953cb3773266d2c83a92fd0e7e6d662025-05-20T16:13:16+03:00Georgi Gerganovllama : remove llama_kv_cache_view API + remove deprecated (#13653)
328b69f1647f9953cb3773266d2c83a92fd0e7e6d66759e37b0d89bc4bd1bce860dc5f3c3052e08575c2025-05-20T14:45:07+02:00Johannes GäßlerCUDA: skip fully masked-out KV in FA vec kernel (#13584)
329e298d2fbd082a52c0f6ed02729f94e9bf630cf17f0adb80bf7c2c0d80abb04f4533b5513622d99642025-05-20T08:05:46+03:00Georgi Gerganovkv-cache : add SWA support (#13194)
330f7c9429c85748cde9599499601ba48d0057722e61dfbf2cf3a9f15193dd893396d07762bbd2c47852025-05-20T02:54:43+02:00Nicolò Scipionesycl : Overcoming workaround for mmap() allocation on Windows (#13482)
3316aa892ec2aa7fe0c93e87c4b970d83a942fb9454aea9f8b4e73876739bf88fb705502f291294e4692025-05-16T21:50:00+02:00Xuan-Son Nguyenserver : do not return error out of context (with ctx shift disabled) (#13577)
332e3a9421b78da5c810d812e6348a405f5acc39f345ab5d5fb256aa23f8ab4de8463515ea627f430062025-05-14T23:15:15+03:00Georgi Gerganovkv-cache : fix out-of-bounds view during reserve graph (#13547)
3335ab5d5fb256aa23f8ab4de8463515ea627f430063198405e98530c683d12fd123e3920f2bd2aafa52025-05-15T03:53:52+08:00Yibo Caiarm64: optimize q6_k_q8_k kernel with i8mm (#13519)
334017f10b5fa630a013ec4f9936e410a60d4f460d54696d5674999dc10a7fb8c27b33406a929f7463a2025-05-14T19:18:18+03:00Gilad S.fix: crash when calling `llama_state_get_size` on a context without a KV cache (#13542)
3354696d5674999dc10a7fb8c27b33406a929f7463ab7d26720821823e23e2273a99e38398d511242e92025-05-14T16:41:02+02:00Johannes GäßlerCUDA: fix crash on large batch size for quant. MoE (#13537)
336360a9c98e13d35f322b4c5b1309aab0cc90ed82b09d13d94fb169093095416ac6323551c37b753392025-05-14T13:35:07+02:00Xuan-Son Nguyenserver : fix cache_tokens bug with no cache_prompt (#13533)
33724e86cae7219b0f3ede1d5abdf5bf3ad515cccb8bb1681fbd532eba26ae4c14cd8be884c8afeb31c2025-05-14T18:55:26+09:00Jeff Bolzvulkan: KHR_coopmat flash attention (#13506)
338f0995d28ce3d15095b6845d94ce4465e46575873c252e0c4097b34666e5a81db9d0450d71fa3098f2025-05-13T18:04:39+03:00Georgi Gerganovmetal : use FA-vec kernel up to batch size 20 (#13496)
339c252e0c4097b34666e5a81db9d0450d71fa3098f4f711afed5e7ef4304b567c8888ee1aa60e868eb2025-05-13T18:04:00+03:00Georgi Gerganovmetal : optimize multi-sequence FA vec kernel (#13493)
340b89d605a91dee0a518ecd582f8991a07d523e2fab4726345aca49e2ad62d615e6e370b3dbad6434f2025-05-13T18:01:53+03:00Georgi Gerganovbatched-bench : fix pp batch contents (#13492)
341064cc596ac44308dc326a17c9e3163c34a6f29d191159ee9df84edb72ccf874e353d2414f3a8c60d2025-05-12T15:12:27+03:00Georgi Gerganovcontext : fix state io for memory-less contexts (#13470)
3429a390c4829cd3058d26a2e2c09d16e3fd12bf1b109232370fc6426aa5dd9be01a8271b9c28f5af3a2025-05-11T11:08:26-04:00Anthony Umfertools : fix uninitialized llama_batch in server (#13436)
343b064a51a4e7246fa43a3307f58e59f65e6be170a053367d149f778cdabc356ee3024494e0dd532232025-05-10T21:34:48+07:00Thammachart Chinvaraponci: free_disk_space flag enabled for intel variant (#13426)
344dc1d2adfc0f4de84da7923866d00781ec5c4e6667c28a74e0783f4bb74a246fb9f19bf212139e3652025-05-09T23:07:07-07:00Jeff Bolzvulkan: scalar flash attention implementation (#13324)
34533eff4024084d1f0c8441b79f7208a52fad7985817512a94d636c4b6c1332370acb3e5af3ca709182025-05-09T19:29:37+02:00Xuan-Son Nguyen server : vision support via libmtmd (#12898)
346611aa914ef4231fab5d1ad04773c42e119ae2d2e0cf6725e9f9a164c39f7a87214d60342f7f946d82025-05-09T15:14:56+03:00Georgi Gerganovmetal : optimize MoE for large batches (#13388)
3475c86c9ed3ef1cc7307fdce05f0f0e2e45253cf90efb8b47eda78ea8ae570d4fece3953aae499289e2025-05-09T12:14:04+02:00Johannes GäßlerCUDA: fix crash on large batch size for MoE models (#13384)
3482189fd3b6327a1d17893694125da8edcf74a64683f96aeff394e9b72bbd2fa665c3e023a70ed86482025-05-09T11:18:02+02:00Xuan-Son Nguyenmtmd : fix batch_view for m-rope (#13397)
349f05a6d71a0f3dbf0730b56a1abbad41c0f42e63dee01d71e585f4a17ae83ec55d77acfdcf0bfa7982025-05-08T14:25:39-04:00Matt Claytonmtmd : Expose helper_decode_image_chunk (#13366)
3508c83449cb780c201839653812681c3a4cf17feed1a844be132dbe865358e1de81136920c1d35ac732025-05-08T15:37:29+02:00Xuan-Son Nguyenserver : (webui) revamp the input area, plus many small UI improvements (#13365)
3516562e5a4d6c58326dcd79002ea396d4141f1b18e51fb96b1ff2e1cc98b2492a012b7d93531a6a9a82025-05-08T14:28:33+03:00Georgi Gerganovcontext : allow cache-less context for embeddings (#13108)
35251fb96b1ff2e1cc98b2492a012b7d93531a6a9a870a6991edf1b60e7afa8962f830320583f3babb02025-05-08T14:26:50+03:00Georgi Gerganovcontext : remove logits_all flag (#13284)
3538733e0cf6eefc7c7752297cc22d0836706f4222c814f795e063c257f33b921eab4073484238a151a2025-05-08T10:08:01+01:00Alberto Cabrera Pérezsycl: addressing non-contiguous src1 mul_mats (nc and batched) (#13343)
354bc4e1128f78be0fbb4e2fa630adb6a04b969ac6839e73ae0d69f882d7e29cecc6dd8f5052fca67312025-05-07T12:49:27+02:00Sigbjørn Skjæretllama : deci : support ffn-free with attention (#13296)
35532916a49072f01c43e20df374af5f8a1f70d6963ffc727203af1061fdeb49efef30f76171722e4032025-05-06T22:40:24+02:00Xuan-Son Nguyenclip : refactor graph builder (#13321)
35627aa2595321c4d9cc4086a8e67bdea204b8309b09fdfcdaeddd1ef57c6d041b89cd8fb7048a0f0282025-05-04T23:43:42+02:00Xuan-Son Nguyenmtmd : add C public API (#13184)
357a75cb30dc9e63488c3614e2d5a9fe2306eaf47cd3f3769ba76061a511f02f2a48da2ad2d93fce5112025-05-02T20:54:13+03:00Georgi Gerganovcontext : fix reorder logic (#13267)
3583f3769ba76061a511f02f2a48da2ad2d93fce5112f567611c0234bbca0a4009762acb47b568660952025-05-02T22:23:12+05:30shalinib-ibmggml : Enable MMA for BF16 in llamafile_sgemm (#13148)
3597d2123484e6ba8fcd90fff8c01661b9dbbefdc9d074e42ab31de4c99aa7d9d2d239660f64b2380d62025-05-02T11:41:54-04:00Jared Van Bortelconvert : use correct context length for nomic-embed-text-v2 (#13216)
360c642bc014c105728ce45015813351dc5a37f60a2cb06a3c363f50cd35113984fe8fb164aea4190772025-05-02T17:48:36+03:00Georgi Gerganovkv-cache : separate recurrent vs non-recurrent impl (#12799)
361fab647e8842c5f80da7e8f2c625dab6a0e19e5d4dcf886007de4b8e5200f461a13233315f897fb9d2025-05-02T09:48:31+03:00Georgi Gerganovserver : add cache reuse card link to help (#13230)
3628efbdadc616fa717c369059b9b388160958d886cf057808ffadce213f54c1884ab5096e60140f3582025-05-01T17:32:11-04:00Justin Santa Barbararpc : avoid uninitialized memory in serialize_tensor (#13210)
363f057808ffadce213f54c1884ab5096e60140f358d7a14c42a1883a34a6553cbfe30da1e1b84dfd6a2025-05-01T13:46:10-07:00Jesse Grossggml: Don't assert fail when tensor data changes (#13222)
364fc727bcdd5a311c7c69a76dbf87f4784e828c7b4b0ecbd434be024c06bf547491be444ed92e1123e2025-05-01T13:19:31-05:00Jeff Bolzvulkan: Handle src1 batch dimension in non-contiguous mat-vec-mul shader (#13191)
3654254bb49518e0f920f14c9aadd96eefdfd38b4299998540149a490200894acfe595e9a1546bab7232025-04-30T15:20:40+02:00Diego Devesaggml : fix ggml_gallocr_ptr type (ggml/1205)
366e1e8e0991ffd9e99a445c6812bb519d5bac9f4b56f67cf1f480926391ad75ff746e0a021214bf70c2025-04-30T23:12:59+02:00Johannes GäßlerCUDA: batched+noncont MMQ, refactor bs>1 MoE code (#13199)
36716a457facd996915652f6274384c87602b27d21a3e168bede4d27b35656ab8026015b87659ecbec22025-04-30T15:28:43-05:00ddh0fix typo: `n_ctx_pre_seq` -> `n_ctx_per_seq` (#13221)
368a0f7016d170ca4bfe24d9a9f26c024d034af69f219e899ce21a7c9ffcf8bb2b22269a75f6e078f8f2025-04-30T14:29:22+08:00xiaofeirpc : fix cache directory initialization (#13188)
369cdf76586b23c67abd3ca064ee2c084c57ae240bd7d3af70b089bb349b5d17eb01839224c99ec19522025-04-29T16:00:27+02:00Johannes GäßlerCUDA: fix non-cont. inputs for batched mat mul (#13155)
37000e3e5a194e88e604e7c91391b9e90332888fd72e98b3692be4cd8fbbd9a56fbacc2f2bf0bf26a682025-04-29T11:47:04+02:00Xuan-Son Nguyenmtmd : add qwen2vl and qwen2.5vl (#13141)
37143ddab6eeeaab5a04fe5a364af0bafb0e4d350651831f538f720d1d99fba146f24f0a8e970838cc42025-04-28T21:00:20+03:00Ville Vesilehtofix(rpc): Improve input validation and error handling (#13069)
3721831f538f720d1d99fba146f24f0a8e970838cc44e87962e34a4b257ec374c4baf6b1568554b81a92025-04-28T20:20:39+05:30Vishal Agarwalllama-bench: add `-d` depth arg (#13096)
373fb0471d1753824e75474c24f82fbdd54c94dcedad2b2031e5f11b826dcc718138642f147a20096652025-04-28T06:45:40-07:00pockers21context : do not clear output buffer on reserve (#13152)
374ca2bb89eac2097ab4620448737e58af8452e444b2d451c80590b9ac250322769ac13d3b4870dbcf72025-04-27T16:10:34+08:00HimariOclip : Add Qwen2.5VL support (#12402)
3754753791e70acd4d4e02f2098f14a03df26c992bd77d5e9a76a7b4a8a7c5bf9cf6ebef91860123cba2025-04-26T22:39:47+02:00Xuan-Son Nguyenclip : improve projector naming (#13118)
376226251ed56b85190e18a1cca963c45b888f4953c87616f0680947800ecba3e9f6bc6e101943bf8e62025-04-24T22:29:22+03:00Georgi Gerganovembeddings : fix batch sizes (#13076)
377658987cfc9d752dca7758987390d5fb1a7a0a54adc39a5e7a84815a90fa0c515ed8927870cf858c92025-04-22T21:27:40+02:00Johannes GäßlerCUDA: noncont MMVQ + batched bs1 MUL_MAT_ID (#13014)
3787b53389c24a507564be39e1ea82746a39749059b243453533e029334181dda50d911d5fc5a2b24862025-04-22T16:15:51+03:00Georgi Gerganovmetal : add memory pool for temp allocs (#12850)
37984778e97703740d8ac5fb64e14d83b80eafa0f3c510676475f885ec064ff147af9f20ee7a9b12a502025-04-15T17:20:38+08:00David HuangCUDA/HIP: Share the same unified memory allocation logic. (#12934)
380daa422881a0ec7944771bcc8ff8de34d11f5bd3beccc7a1602f0752507de4aaad1008b9618a282c82025-04-15T07:49:57+01:00Juk Armstrongllama : DeepSeek V2/V3 MLA implementation (#12801)
381b0c75ac9f93322b45e3766e149417334b4fd1ed9d6d2c2ab8c8865784ba9fef37f2b2de3f2134d332025-04-15T10:04:24+08:00Xinpeng DouCANN: Optimize CANN buffer pool memory management (#12875)
382c772d549264c1be058411312a54049e0dc86a03781c7e64fc239288e91a58adad9145110e03538222025-04-14T13:59:34+03:00Radoslav Gerganovrpc : use ggml_context_ptr (#12938)
383bc091a4dc585af25c438c8473285a8cfec5c7695a4837577aae59c0ae640f3810094724bcac6bb282025-04-12T21:03:39+05:30Prajwal B Mehendarkarcommon : Define cache directory on AIX (#12915)
384e8a62631b3b05bcf2ad0e0c686881a9f3e3f03cab6930ebc421e20399663bbd3bc7b7266d5236d062025-04-11T13:04:14-07:00yuri@FreeBSDrpc : Set cache directory in rpc-server.cpp on FreeBSD (#12903)
38568b08f36d047bfa048ebd7d16262c53bf9ece701578754b3157d662c2fdd51eaa62b6c1f43d3172c2025-04-11T12:45:44-07:00yuri@FreeBSDcommon : Define cache directory on FreeBSD (#12892)
3860c509239445088f21265580b86733c5b184261d9fccf9cae83e6c6cd31a0ecb403d237638e427d0a2025-04-11T12:09:39+02:00Xuan-Son Nguyenclip : use smart pointer (⚠️ breaking change) (#12869)
3878ac9f5d765f2b1b7f821873618c0cff68ca59bc812e9158f25cb47e97ca9b8083e1ec7415f2622602025-04-11T15:26:17+08:00R0CKSTARci : Replace freediskspace to free_disk_space in docker.yml (#12861)
38864eda5deb9859e87a020e56bab5d2f9ca956f1defe5b78c89670b2f37ecb216306bed3e677b49d9f2025-04-10T17:24:44+02:00Xuan-Son Nguyenconvert : ability to lazy-load safetensors remotely without downloading to disk (#12820)
389d9a63b2f2e91cdcb0eda211b7f49fadcdea0f6648ed71242f464dc0a3fb3cffcfe064e55bdec72f92025-04-09T17:22:30+08:00R0CKSTARmusa: enable freediskspace for docker image build (#12839)
3900090950f679475c5ecaac2f7bca5049cca96492b7ecd780b1a1d5214b8d04c25ebfc194d310816ed2025-04-09T00:25:08-05:00Jeff Bolzvulkan: In coopmat2 mmq, load q4_k/q5_k scales through shared memory (#12833)
3917538246e7ce0606694c38055cc2fc9f60535be6cb32efad2bc42460637c3a364c9554ea8217b3d7f2025-04-08T23:21:31+02:00Sigbjørn Skjæretcuda : add f32 to bf16 copy op (#12806)
392a19b5cef16d885c44c635da4a5c97113c1577de878a1ba0a4f2bfed5b8b8e312592143d22e5316982025-04-08T19:54:51+03:00Georgi Gerganovllama : fix FA when KV cache is not used (i.e. embeddings) (#12825)
3932dabf759e7c8c827d38cdd18d0792070e6a4f4e11d343b4069c74b2c7b19ae84260cd98aa2320a9a2025-04-08T21:49:13+08:00dm4llava: add more helper functions to check projector types in clip context (#12824)
3948ca6e1c3a4deb6bb27ee294bfd5706098d94ae88656babd6c21a3b9b3622324cfcc80a2ab78da25b2025-04-08T14:14:59+05:00characharmserver : webui : Improve Chat Input with Auto-Sizing Textarea (#12785)
395518a01480eb3a7c80a4951b430db9dee55428310e391d3ee8ddae86be70c034de1082ad51c55e2112025-04-07T23:22:57+08:00zhouwgsycl: remove redundant memcopy in function ggml_backend_sycl_buffer_set_tensor (#12734)
396bd3f59f81289b920bcc597a208c14f55e39ed37e52b3d71f128c1eeab39b918adf1f7a8f5e2566192025-04-07T13:35:19+02:00Xuan-Son Nguyencmake : enable curl by default (#12761)
39780b717d493a9a5bae7167ad2384c12c60bb2ef206bf28f0111ff9f21b3c1b1eace20c590281e7ba62025-04-06T03:47:13-05:00Jeff Bolzvulkan: Use unclamped loads for flash attention mask (#12720)
3983e1d29348b5d77269f6931500dd1c1a729d429c81be76e4620ddc99b3cbff0f206436117ae5610472025-04-04T21:48:10+03:00Georgi Gerganovkv-cache : simplify + fix warning for recurrent models (#12756)
3993f9da22c2b21a2cef216de50006436ef1cab87642a0dc97e56eac6db0a4016f0b45da6d0a0055ef22025-04-03T02:31:15+01:00Alan GraySimplify and improve CUDA graphs through use of indirect copy pointers (#9017)
4002a0dc97e56eac6db0a4016f0b45da6d0a0055ef297a20c012be2f9bfbeee0209405a31001f93ccf92025-04-03T08:49:51+08:00hipuddingCANN: Fix failed test cases (#12708)
40197a20c012be2f9bfbeee0209405a31001f93ccf9f01bd02376f919b05ee635f438311be8dfc91d7c2025-04-02T17:01:42-07:00lhezopencl: use `max_alloc_size` in backend ctx instead of querying again (#12705)
402f01bd02376f919b05ee635f438311be8dfc91d7c6f3bd38640f07e4dec7f145d2fbf093ce48c95442025-04-02T14:25:08-05:00Jeff Bolzvulkan: Implement split_k for coopmat2 flash attention. (#12627)
403be0a0f8cae039e2286f757612accebfb8f21b36e92e3006bb69dfeb656ccf5c7c1c1efadb03c88c22025-04-02T12:40:32-05:00Jeff Bolzvulkan: Implement grouped query attention in the coopmat2 FA shader (#12559)
40492e3006bb69dfeb656ccf5c7c1c1efadb03c88c2833e2b7409211a07df97716998c50025266426522025-04-02T19:12:30+02:000cc4mVulkan: Fix mmq int dot float cache size (#12722)
405e0e912f49b3195ef9d0c51378629ba03c9b972daa10b36c91a091f4606710fba4e9327fd71e0e7382025-04-02T14:52:01+02:00Diego Devesallama : add option to override model tensor buffers (#11397)
406a10b36c91a091f4606710fba4e9327fd71e0e73883a88bd6affbe148a622ac730952ac5b8b5859792025-04-02T14:32:59+03:00Georgi Gerganovllama : refactor kv cache guard (#12695)
407f423981ac806bf031d83784bcb47d2721bc70f97e39e727e9a3daec7082b9c59540a429ad85914af2025-04-02T01:54:34+09:00Junil Kimopencl : fix memory allocation size (#12649)
408e39e727e9a3daec7082b9c59540a429ad85914af5936a616e46cffad4579ccaff8f8fa315809da4c2025-04-01T20:54:28+08:00jklincnllama : use LLM_KV_GENERAL_FILE_TYPE instead of gguf_find_key (#12672)
4095936a616e46cffad4579ccaff8f8fa315809da4c3fd072a54001a908c54e81fd2e82b682ecfdd4752025-04-01T14:37:13+02:00Sigbjørn Skjæretconvert : BailingMoE : fix qkv split when head_dim is 0 (#12687)
410492d7f1ff77e116e44962b832cc3db24cfc46d24d3f1f0acfbf8aaafd2ce494309a10a7cc92042c82025-03-30T16:59:38+08:00R0CKSTARmusa: fix all warnings, re-enable `-DLLAMA_FATAL_WARNINGS=ON` in ci and update doc (#12611)
4113891e183c61c1e25c2c61afe68d7b95019ea3f89af6ae1efb27a9a7c3f7f7f84639d2243f7303ac12025-03-20T07:02:18+01:00Daniel Beveniusexamples : command.wasm updates (whisper/2904)
4120bb2919335d00ff0bc79d5015da95c422de51f03a69f8463510a70fe0c85701b8c7ede283a0d1a7d2025-03-29T14:07:37+01:00Djip007llama : change cpu_buft_list order: ACCEL -> GPU host -> CPU extra -> CPU (#12632)
413a69f8463510a70fe0c85701b8c7ede283a0d1a7dd07a0d7a79bafb01fb3b6c8992121e75c57c0c2f2025-03-29T18:04:58+08:00Jaycmake : fix ccache conflict (#12522)
414b4ae50810e4304d052e630784c14bde7e79e4132b86f6007234da4bff51a3ebef2bdb952b52059c62025-03-28T20:21:59+02:00Georgi Gerganovmetal : improve FA + improve MoE (#12612)
415ef03229ff423dd1991f4f44ef1352f03334d86eb13731766db91ec927c1b61bf502ac7a9be2b11b92025-03-28T09:44:13+02:00Radoslav Gerganovrpc : update README for cache usage (#12620)
41613731766db91ec927c1b61bf502ac7a9be2b11b9ab6ab8f809bd514d88e02a63869b4d619f13fa862025-03-28T13:13:22+05:30amritahs-ibmllamafile : ppc64le GEMV forwarding for FP32. (#12594)
417ab6ab8f809bd514d88e02a63869b4d619f13fa862099a9d5dbdcd2841d02dd396a71d0de70bf44902025-03-28T08:18:04+02:00Radoslav Gerganovrpc : send hash when tensor data is above some fixed threshold (#12496)
418f125b8dccff34439a26bf750c9edef358c48c1f8953c2a62cf487e618140f3ea18d94e3b0257af932025-03-27T10:49:15ZSi1wllama : add PLM GGUF Conversion & Inference Support (#12457)
419c7b43ab60855f752ae79937fb93d561bc30b69a424feaec05792b972d5ff3e2b12d9237ebd50d1ac2025-03-27T12:21:47+05:30amritahs-ibmllamafile : ppc64le MMA implementation for Q4_0. (#12489)
420f17a3bb4e8b0aa24c0f86636d234aca7dc2cfa01bd40678df768ab189b26b8b03e3de4062e3b71a32025-03-27T07:16:00+05:30Akarshan BiswasSYCL: implement memset ggml backend buffer interface (#12580)
42102082f1519565fc7b49de211b28bc5404a69209bdf4d20cd53d5bb6fc21c1dc65f026d53b566d0972025-03-26T22:06:04+08:00Ivy233clip: Fix llama-llava-clip-quantize-cli quantization error under CUDA backend (#12566)
4222d77d88e70d017cd82c3f1a4517e3102e2028ac4c95fa362b3587d1822558f7e28414521075f254f2025-03-25T09:19:23+02:00Georgi Gerganovcontext : fix worst-case reserve outputs (#12545)
423eddfb438502bd5d1014d63a812e9b6d03d326f8c4375415b4abf94fb36a5fd15f233ac0ee23c0bd12025-03-22T03:40:11-05:00Jeff Bolzvulkan: Optimize mul_mat_vec p021 and nc shaders (#12505)
4241aa87ee53d05505247c54612e40f6a38c680b4339ffcc9e374080f73b16b7a351e6d76e7a8a19ce32025-03-21T14:58:47+08:00蕭澧邦[SYCL] Fix build on Windows when ccache enabled (#9954) (#9976)
425dbb3a4739e53226346c772dd72666e68b78dc5833d82dbcbce2c677fc35fbf99574ccd107d95a1f82025-03-20T12:49:59+01:00Sigbjørn Skjæretllama : make Qwen2MoE QKV bias optional (#12477)
426568013d0cd3d5add37c376b3d5e959809b711fc7517b5ddbf002b91fd6d6daf5d8db8c88a01730392025-03-19T21:01:57+01:00fairydreamingcontext : clear sets containing encoder output sequence ids before storing new values (#12470)
42775422e8bc42646005be0754f7aa438b97a5e777ebb115d2bf7ed2cdd7dccd7ae74cc9cfe4b0adb712025-03-18T21:35:19+02:00Georgi Gerganovgraph : normalize Q, K, V shapes + sync cross attention (#12449)
4288551c44d840a7db50adb958ccaf464dc3ded82e735cae5ba05a5292dc3108636a71ec59fa2f80ab72025-03-18T13:05:49+02:00Georgi Gerganovcontext : always use non-causal attention for encoder graphs (#12447)
429810e0af3f50379682dd46b7967c4aadf3f8286f6eba92d64c3f6d86de2e6b4dd3a540d2805a22b0c2025-03-18T12:05:42+02:00Georgi Gerganovserver : fix warmup draft cache type (#12446)
430fd123cfead49eb32e386e26b8ef7a6d41554dda5a53f7f7b8859f3e634415ab03e1e295b9861d7e62025-03-18T07:21:40+01:000cc4mVulkan: Default to 1GB allocations instead of 4GB to avoid fragmentation and driver issues (#12434)
4317dfad387e3f6ac98d383ded2d175eb59736a399360c902926c928f9c2cd6390ce411876f92feeaf32025-03-18T07:27:50+08:00Molly Sophiallama: Add support for RWKV v7 architecture (#12412)
432b3c9a65673a63a6c9a75da24ee00d13499494e0c8ba95dca2065c0073698afdfcda4c8a8f08bf0d92025-03-17T07:15:12+05:30Akarshan BiswasSYCL: set extras only on GGML_TYPE_Q4_0 (#12366)
433dc079cfdffa1141a6caf5d41a33d73a1ef03da557b61bcc87cfdeab88350e82df1c4b7be64331ea62025-03-16T19:29:36+02:00Georgi Gerganovcontext : fix init of n_outputs (#12397)
434c522ce4143a2b5c277f1e5f65cd570dbd0626466081bee8c643b1f6302e9edfe789ce2d5f0be6c772025-03-14T10:47:44+02:00Georgi Gerganovgraph : simplify attn input build for unified KV cache (#12381)
43584d547554123a62e9ac77107cb20e4f6cc503af4be7c3034108473beda214fd1d7c98fd6a7a3bdf52025-03-13T19:08:07+02:00Georgi Gerganovllama : fix Gemma3 SWA KV cache shift (#12373)
436e0dbec0bc6cd4b6230cda7a6ed1e9dac08d1600b2048b5913d51beab82dfe29955f9008130b936c02025-03-13T12:35:44+02:00Georgi Gerganovllama : refactor llama_context, llama_kv_cache, llm_build_context (#12181)
4376ab2e4765a673abcd162258a2671560a76106d6996e1280839561aaabb73851f94972a2cd37b2d962025-03-11T19:45:02+08:00BB-fatmetal : Cache the Metal library at the device context level (#12265)
43896e1280839561aaabb73851f94972a2cd37b2d962c9f833d17bb5b8ea89dec663b072b5420fc54382025-03-11T09:20:16+01:00Xuan-Son Nguyenclip : bring back GPU support (#12322)
4392c9f833d17bb5b8ea89dec663b072b5420fc5438251364549fe4a78d4e2e66f1adfaf2bd53041d2c2025-03-10T19:28:11ZEvemat vec double buffer (#12188)
4405e2d57b2b2e43eadbe6d66ba3e873a824b95e725f1648e91cf6c52e9593810aa70857e412d474c092025-03-07T15:35:57+08:00BB-fatmetal : simplify kernel arguments using a struct (#3229) (#12194)
441e721c05c9336a72fbb59d5c75967360bc67036c657b6abf85acb1f697913a44e5e105e333d894b782025-03-06T08:20:52+01:00uvosHIP/CUDA: set the paramerter value in maintain_cuda_graph instead of replaceing it. (#12209)
44257b6abf85acb1f697913a44e5e105e333d894b7894bb63e4f0f6135579b88e5700ed40cefa374e092025-03-05T22:22:49-08:00Han Yinandroid : fix KV cache log message condition (#12212)
44394bb63e4f0f6135579b88e5700ed40cefa374e09f79243992cd31e4e4844d5158d2f3325b1d2d8112025-03-06T03:33:40+02:00Henry Linjamäkiopencl : fix buffer alignment (#12197)
44406a92a193a07afe445929607be9d5e4d033956fba057897ad4e48e3df0354256e0cc80dadcb575952025-03-05T15:25:45+08:00Clauszyserver : fix cache reuse logic (#12161)
4452679c3b55d1c9c3fed56dea00fea713622e42594c43af9276b119dae7436b7878d59671d0f6b1a972025-03-03T16:17:36+01:00Daniel Beveniusci : set GITHUB_ACTION env var for server tests (#12162)
44670680c48e5f77d2d3138712a6582bd8c1e548922c43a3e7996e585e2addde1e44057a4f3cdbadef82025-02-28T05:41:47-08:00William Tambelliniggml : upgrade init_tensor API to return a ggml_status (#11854)
447438a83926afcff3643ffef5543db67545ceffe399c42b1718ca8299f9afeabdc122badeab64c96902025-02-28T09:42:52+01:00Rémy Ovulkan: add specific MMV kernels for IQ2 and IQ3 quants + optimizations (#11595)
4483e9a2860e996657fc10db8393cf65adc4070308258d07a8043a1395177cf77b3e4f388e34182ae642025-02-25T01:29:33-08:00Vitali Lovichllama : expose llama_model_n_head_kv in the API (#11997)
4497a2c913e66353362d7f28d612fd3c9d51a831eda08d5986290cc42d2c52739e046642b8252f97e4b2025-02-24T09:09:51-07:00Alex Brooksllava : Add Granite Vision Support (#11794)
450af7747c95ae71a5db4184947f837179e82c70b77a28e0d5eb18c18e6a4598286158f427269b1444e2025-02-23T05:39:24+08:00Aaron Teoggml-cpu: Support s390x SIMD Instruction Set (#12019)
4515fa07c2f93c73161bf09ef0b23b5d2686f9a073e335eb04a91f481f37c0c9b302ee31b449b04c3e92025-02-22T12:20:17+01:00Johannes GäßlerCUDA: optimize FA for GQA + large batches (#12014)
45251f311e057723b7454d0ebe20f545a1a2c4db6b2586d5fe6ebb8f92e9dd53420e04cec26970460732025-02-21T18:33:18+02:00Georgi Gerganovllama : skip loading unused tensors (#12004)
453b58934c1836b5ea51dbacbe899eee125775e77c963e489c025d61c7ca5ec06c5d10f36e2b76aaa1d2025-02-19T00:01:44+02:00igardevserver : (webui) Enable communication with parent html (if webui is in iframe) (#11940)
4542eea03d86a2d132c8245468c26290ce07a27a8e80f2bbe656473177538956d22b6842bcaa0449fab2025-02-17T07:55:57+01:00Rémy Ovulkan: implement several ops relevant for ggml_opt (#11769)
4553e693197724c31d53a9b69018c2f1bd0b93ebab2a394039db004c6ee00098250d160b5aa018c23142025-02-13T07:07:51+01:00Daniel Beveniusllama : update llama_decode_internal ref [no ci] (#11840)
456a394039db004c6ee00098250d160b5aa018c2314be3bbd62153820ff6d358c817360927f429105c42025-02-13T01:02:38+01:00Diego Devesaggml-cpu : add chunking support to mul_mat_id (#11666)
457be3bbd62153820ff6d358c817360927f429105c431afcbee0eebbc998ab311aa314e389d60aa21272025-02-13T00:33:45+01:00Xuan-Son Nguyenggml : x2 speed for WASM by optimizing SIMD (#11453)
45819b392d58dc08c366d0b29bd3b9c6991fa4e16620893e0114e934bdd0eba0ff69d9ef8c59343cbc32025-02-10T19:58:18+01:00Wilken Gottwaltllama-mmap: fix missing include (#11796)
459b044a0fe3ca0cbef9dd041edce3ebda8c501fae419d3c8293b1f61acbe2dab1d49a17950fd788a4a2025-02-10T03:08:22-03:00Wagner Brunavulkan: add environment variable GGML_VK_PREFER_HOST_MEMORY to avoid VRAM allocation (#11592)
46098f6b0fd1ea88ce33f4fcd1775d9a463067156ac55ac8c7791ff44aeb82bd7fe3ca2041844fc77f12025-02-09T01:43:51-06:00Jeff Bolzvulkan: account for lookup tables when checking shared memory size (#11502)
461c026ba3c23765a648ca27c7a15ecf179f8e27f267ee953a64a40c09438b2064539becdbc577cefd02025-02-07T04:26:03-06:00Jeff Bolzvulkan: print shared memory size (#11719)
4627ee953a64a40c09438b2064539becdbc577cefd0ec3bc8270bc67b58955748d40a3e558a05b2d8f22025-02-07T04:33:27-05:00Christian Fillionllama : add llama_sampler_init for safe usage of llama_sampler_free (#11727)
4631d20e53c40c3cc848ba2b95f5bf7c075eeec8b192fb3c32a1634488a5265d1304ab37628eeb5480d2025-02-06T09:29:13-05:00Patrick Pengrpc: fix known RCE in rpc-server (ggml/1103)
4641b598b30581bad59e5af86c94362f9a30f261fac902368a06b915b860236cfc97ff885b2aceae2562025-02-06T00:02:18-06:00Jeff Bolzvulkan: use smaller combined allocations to avoid fragmentation (#11551)
465c3db0480bb820e120249014b380e1f4badf2a1c1d774ab3acc4fee41fbed6dbfc192b57d5f79f34b2025-02-06T01:55:25+01:00Matvey Solovievreadme : add link to Autopen under UIs (#11684)
466f117d84b48104992ba16961b35a96fa93efbb355534c46b53c23613628d72e93c63ea01ea4d1e2ac2025-02-04T19:15:24+08:00Jhen-Jie Hongswift : fix llama-vocab api usage (#11645)
46753debe6f3c9cca87e9520a83ee8c14d88977afa4cfd74c86dbaa95ed30aa6b30e14d8801eb975d632025-02-01T18:22:38ZOlivier Chafikci: use sccache on windows HIP jobs (#11553)
468aa6fb1321333fae8853d0cdc26bcb5d438e650a1a83f528688324a21484a97af1d1be5e1bc8d4c8e2025-01-31T17:12:40ZOlivier Chafik`ci`: use sccache on windows instead of ccache (#11545)
4691bd3047a939e561adfb3c7dd2e17c4cc7a4e4e6fa2df2787b32e0846205f7151dfad88ceab592beb2025-01-31T00:58:55-05:00Steve Grubbcommon: Add missing va_end (#11529)
470553f1e46e9e864514bbd6bf4009146db66be05418b576b6c55bc4e6be898b47522f0ef402b93ef622025-01-30T22:01:06ZOlivier Chafik`ci`: ccache for all github worfklows (#11516)
471325afb370a1a7b32b5fe46a749bc840c66db9765794fe23f29fb40104975c91fe19f23798f7c726e2025-01-29T12:07:21+08:00Molly Sophiallama: fix missing k_cache store for rwkv6qwen2 (#11445)
472cae9fb4361138b937464524eed907328731b81f67fee2889e6565830631fbe76d47ef85cf8fd946a2025-01-28T07:42:20-08:00Nikita SarychevHIP: Only call rocblas_initialize on rocblas versions with the multiple instantation bug (#11080)
473178a7eb952d211b8d4232d5e50ae1b64519172a96f53d8a6b41e48c73b345fc6c712c3b00ea4fb932025-01-26T20:06:16+02:00Georgi Gerganovmetal : use residency sets (#11427)
47419f65187cbf009801288861133267ee5573ceead1d8ee06000ecdd274e7f0a0465d6bf26ad2b34912025-01-26T12:07:48-04:00bandoticmake: add ggml find package (#11369)
4754a75d19376f2f00dbae6c266eb9c4f3001872b5226771a1491f3a4c3d5b99c4c267b81aca9a7dfa02025-01-25T15:29:57-06:00Jeff Bolzvulkan: compile shaders on-demand (#11406)
476564804b79b78df1469ec8646869972de5e885ec405f63cc9ee859de07f585f7b12939345f39ada8b2025-01-23T14:51:24-06:00Jeff Bolztests: fix some mul_mat test gaps (#11375)
4775245729e3317064959faefc5e5cbc63f4e9cfbea6152129d05870cb38162c422c6ba80434e021e9f2025-01-23T01:01:17-06:00Jeff Bolzvulkan: fix diag_mask_inf (#11323)
4786152129d05870cb38162c422c6ba80434e021e9f16d3df7ab0bb9def78047eab4d9b15393997f4952025-01-22T19:22:20+01:00Diego Devesamain : update README documentation for batch size (#11353)
47912c2bdf2de34f747d13b270fc9d3b52490bf194fc64d2becb13f2a7c0145b516a05f028d949a046b2025-01-22T17:44:40+01:00Diego Devesaserver : fix draft context not being released (#11354)
4803e3357fd77bcf5bd8cfcc53ca53d4a5532e67e1b6171c9d25820ccf676b243c172868819d882848f2025-01-22T15:35:48+08:00tc-mbllava : support Minicpm-omni (#11289)
4816171c9d25820ccf676b243c172868819d882848fe28245f35f2faaf249dd352998b3693a8cc28c512025-01-21T13:18:51ZOlivier ChafikAdd Jinja template support (#11016)
4826da5bec81c34f3b8a8f1b367cf23ad016e83d3322e2f8f093cd4fb6bbb87ba84f6b9684fa082f3fa2025-01-21T15:06:41+02:00Radoslav Gerganovrpc : better caching of the base buffer pointer (#11331)
48399487b57d47e14dc342b7b89d238ca11c0345241a1649cc13f89946322358f92ea268ae1b7b5096c2025-01-19T14:33:34+01:00Nicolò ScipioneSYCL: Introducing memory host pool (#11251)
4844dd34ff83165a483ebff7bd43621b28490fa1fd6f30f099228f774209aa3010b78dfbe5d262e69aa2025-01-18T16:18:15+02:00Georgi Gerganovcmake : add sanitizer flags for llama.cpp (#11279)
485adc5dd92e8aea98f5e7ac84f6e1bc15de35130b5f11cfdfd7fe29436fce512d934c2ff6b94bd89d22025-01-15T19:50:13ZEvevulkan: scale caching for k quants + misc fixes (#11081)
4867426a26b2492fc546a4db6991e871ee6057140938f70fc3d1b1d3c17b61842330dd106d391cc12272025-01-13T14:46:36+02:00Georgi Gerganovcontrib : add naming guidelines (#11177)
4872739a71e4b88474833b64aa974ca4515574fd3c4ba8a1f9c5b675459c55a83e3f97f10df3a66c7882025-01-11T05:50:33+01:00Daniel Beveniusconvert : sort print supported models [no ci] (#11179)
488ff3fcabc727b2dd0c477d23a258217b27cc639fbc3f9d25706ac84297067aeaa662c1f1af42ed4432025-01-10T11:30:53+01:00Daniel Beveniusconvert : add --print-supported-models option (#11172)
489ee7136c6d1e0ba7633294dad137b1573048031ecc6860cc7346c90219475e4467bb8a288e0df975c2025-01-10T09:58:08+08:00Molly Sophiallama: add support for QRWKV6 model architecture (#11001)
4908cef75c743ba13ebbd6d380c531200c768a8b8aa0d52a69e4bf0d6181beec7853307bdcdeec9905b2025-01-08T16:24:19+05:30amritahs-ibmllamafile : ppc64le MMA INT8 implementation (#10912)
491017cc5f446863316d05522a87f25ec48713a9492a3d50bc022bedd6c7754c24749a1fef4d2d60c7c2025-01-07T16:11:57+01:00Diego Devesaggml-backend : only offload from host buffers (fix) (#11124)
492a3d50bc022bedd6c7754c24749a1fef4d2d60c7ca4dd490069a66ae56b42127048f06757fc4de4f72025-01-07T12:38:05+01:00Diego Devesaggml-backend : only offload from host buffers (#11120)
493c0d6f790d07aa78be15584ec394ac20739ade93bdc7cef9f373f2a24b851f0df7a618c5209e593fa2025-01-07T11:56:07+05:30Akarshan BiswasSYCL: Use get_multi_ptr instead of deprecated get_pointer in wkv6 (#11087)
494dc7cef9f373f2a24b851f0df7a618c5209e593faecebbd292d741ac084cf248146b2cfb17002aa1d2025-01-06T22:45:28ZEric Curtinllama-run : fix context size (#11094)
4956369f867a410416239d9f20ec27c2b1d6a9fee5247182dd03fe04a4ffda5d7f4c8a109ae0056cf562025-01-06T10:28:17+01:00Daniel Beveniusllama : rename missed batch params/vars to ubatch (#10059)
49647182dd03fe04a4ffda5d7f4c8a109ae0056cf563e6e7a6bc2c4b980a0cf0fcb5cb3b79a965b5f142025-01-06T10:55:18+02:00Georgi Gerganovllama : update llama_model API names (#11063)
4979394bbd484f802ce80d2858033583af3ef700d25f922a9c542ee117550a168395c63ea79261f5c992025-01-04T21:06:11+01:00fairydreamingllama : Add support for DeepSeek V3 (#11049)
498f922a9c542ee117550a168395c63ea79261f5c9946be942214e295cd34660bbbd6b846155d1c36a02025-01-04T16:10:30Zmatt23654[GGML][RPC] Support for models with non-512-aligned tensors over RPC. (#11047)
4994b0c638b9a68f577cb2066b638c9f622d91ee661e7da954eccdf39ee795a6135bdb86f09789026812025-01-03T20:13:18+08:00Molly Sophiacommon : disable KV cache shifting automatically for unsupported models (#11053)
500f66f5829276650cd83a087ab2cfed1a760183ea12f0ee84b9b02d2a98742308026f060ebdc2423f12025-01-03T10:18:53+02:00Georgi Gerganovllama : refactor `src/llama.cpp` (#10902)
5010da5d860266c6928b8c9408efbd264ae59fedda6a45433ba209ee0b33d02c7dc4c31f29894ad83a62025-01-02T15:05:18+01:00Xuan Son Nguyenserver : allow using LoRA adapters per-request (#10994)
502716bd6dec3e044e5c325386b5b0483392b24cefec250ecb3157f3bae0a45f44c3c953b5414d4c2f72024-12-30T11:27:11-06:00Jeff Bolzvulkan: optimize mul_mat for small values of N (#10991)
503c250ecb3157f3bae0a45f44c3c953b5414d4c2f7a813badbbdf0d38705f249df7a0c99af5cdee6782024-12-30T20:35:13+08:00ag2s20150909android : fix llama_batch free (#11014)
5042cd43f4900ba0e34124fdcbf02a7f9df25a10a3d09fe2e76137dde850b13313f720e7ffa17efdefa2024-12-24T18:54:49+01:00Djip007ggml : more perfo with llamafile tinyblas on x86_64 (#10714)
5057024d59e6a572730626cb11896829d115043a1b17c0e28585843b366864b43b48f92425e2ea17df62024-12-22T16:20:11-08:00yuri@FreeBSDggml : fix run-time on FreeBSD in get_executable_path() (#10948)
506ebdee9478ca7ba65497b9b96f7457698c6ee51155cd85b5e008de2ec398d6596e240187d627561e32024-12-22T03:44:01-06:00Jeff Bolzvulkan: build fixes for 32b (#10927)
507eb5c3dc64bd967f2e23c87d9dec195f45468de600ca416c91aea4549d9c77e3efeca403e15aa6c752024-12-20T21:01:28+05:30Akarshan BiswasSYCL: Migrate away from deprecated ggml_tensor->backend (#10840)
5080a11f8b7b5c39fdf6e91ef9674bc68ff08681af7d408bb9268a988c5a60a5746d3a6430386e7604d2024-12-20T17:44:58+08:00Molly Sophiaconvert : fix RWKV v6 model conversion (#10913)
509d62b532c52e0118323277eaa5f442e11ce6505ed081b29bd2a3d91e7772e3910ce223dd63b8d7d262024-12-17T17:24:22-05:00DAN™Use model->gguf_kv for loading the template instead of using the C API. (#10868)
510130d0c90bd26b25e3a713b17a61a5d4eb0a664053919da8e335dbfd0741d239932a9b9e72061bf272024-12-14T03:23:08+01:00Daniel Beveniusggml : remove return from ggml_gallocr_allocate_node (ggml/1048)
511160bc039c862c10b9938269a90ddb09d794a7b0d08ea539df211e46bb4d0dd275e541cb591d5ebc82024-12-17T05:00:46+08:00Zhiyuan Lirwkv6: add wkv6 support for Vulkan backend (#10829)
512644fd71b44c4cdbfc6482fbf0353d289c3bc29e64ddd199f6f6b980e0a7ed9f9b44efeae2fbdf5c42024-12-16T12:31:14+02:00Georgi Gerganovsampling : refactor + optimize penalties sampler (#10803)
513ba1cb19cdd0d92e012e0f6e009e0620f854b6afd56eea0781cbd2608ed8ff524955495569b9264be2024-12-14T20:43:46+08:00HimariOllama : add Qwen2VL support + multimodal RoPE (#10361)
514a76c56fa1a3d27467eb97468d8c3b2fe1243b61ac27ac678dd393af0da9b8acf10266e760c8a09122024-12-13T12:23:52-08:00lhezIntroducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (#10693)
51583ed24a97b500ccdb32b90b94e6f9621ad8db79ed583cd03f663701858ba152a334cbb467fabe3422024-12-13T12:12:15+05:30Akarshan BiswasSYCL: Reduce most of the compiler warnings (#10748)
516484d2f31aed34ff9f096e3961125762e81d9b7d64b4d92b0986e3b627b9a9ef4782973108bf476912024-12-11T22:48:04+09:00kallewoofbug-fix: snprintf prints NULL in place of the last character (#10419)
517ae4b922614d452477cf5d2fb8cad247c9c12596c750cb3e246f7e544124cf18cb0c5e0c8b7e387382024-12-10T12:23:50-05:00Bartowskiimatrix : Add imatrix to --no-context-shift (#10766)
51826a8406ba9198eb6fdd8329fa717555b4f77f05fc37fb4cf62ddf0d33562c4c4a4d6fb45e32ad3b62024-12-09T20:07:12+01:00Johannes GäßlerCUDA: fix shared memory access condition for mmv (#10740)
51906d70147e6480c021e493c442ae0f0d83ae366de43ed389a3f102517e6f7d5620d8e451e88afbf272024-12-08T19:19:19+01:00stduhpfVulkan: fix NaN in tanh.comp with AMD proprietary driver on Windows (#10723)
52062e84d984875372f4b0fb89a67658e012ff0cc9f3573fa8e7b7f0865638b52b4e9b4d2006f0558a22024-12-07T16:12:27-05:00Robert Collinsllama : add 128k yarn context for Qwen (#10698)
52119d8762ab61df8286367588a80b9c7db4cb568dbc2a16c0bdbe2e51adf318918bad82f0c3e3d6f3b2024-12-07T13:37:50+01:00Djip007ggml : refactor online repacking (#10446)
522c2a16c0bdbe2e51adf318918bad82f0c3e3d6f3b3df784b3050f657ea681f804187ce5bddb433e882024-12-07T11:52:44+02:00Georgi Gerganovserver : fix free of spec context and batch (#10651)
52386a1934978ce5daffc7e5b4251f6540ee5e7b47b784a14aa496a66cc43e76014a1bf4333b4a2a55a2024-12-07T01:55:01-06:00Robert Ormandimetal : Extend how Llama.cpp locates metal resources (#10676)
5241da7b765692764a8b33b08da61cbee63812a7bd959f4db10883a4f3e855cffbf2c3ab68430e952722024-12-04T22:38:20+02:00Georgi Gerganovserver : fix speculative decoding with context shift (#10641)
52582bca2257b3cec72676abb26011f1b99fcdab29d0115df2f65ac7c64dd0e5915c72ecc4a9343a1302024-12-03T12:50:08+02:00Nikolaos Pothitosreadme : add option, update default value, fix formatting (#10271)
5260115df2f65ac7c64dd0e5915c72ecc4a9343a130515d4e53727924e48774f45ecb15bdacbf851e132024-12-03T11:52:33+02:00Georgi Gerganovmetal : small-batch mat-mul kernels (#10581)
52770b98fadbc8c07a0144f3b50a4d7ab7e2aeff878642330ac7cea11dc1f9d3df2b8f3dbd10b5e3f3e2024-12-03T11:20:00+02:00Georgi Gerganovserver : fix default draft model parameters (#10586)
528f095a649ec390e04dfab1b04e646ae8549dafaef678d7994f4da0af3d29046be99950ac999ee97622024-11-29T00:18:02-06:00Jeff Bolzvulkan: get the first command buffer submitted sooner (#10499)
52946c69e0e752ff16206347bb12f96ed69f4a01abf9e2301f4a4ef1690bd99360c11de43fe830b1c8d2024-11-27T11:03:25+01:00Diego Devesaci : faster CUDA toolkit installation method and use ccache (#10537)
5305b3466bedfa84aa29c6871c7254467550186ecc6249a7902ec710c8d027b9cc0ed10219d2b4184f82024-11-27T01:30:27-06:00Jeff Bolzvulkan: Handle GPUs with less shared memory (#10468)
53171a64989a5d2e25c13507efada145f12cf3589144a57d362e1948ada50af997a92c3cbff9711e78b2024-11-27T01:08:54-06:00Jeff Bolzvulkan: skip integer div/mod in get_offsets for batch_idx==0 (#10506)
53245abe0f74ee281aea6e5283c1e738061256cfcae0bbd2262a3263f37385297b30de37941836e57f72024-11-26T16:20:18+01:00Xuan Son Nguyenserver : replace behave with pytest (#10416)
5330cc63754b831d3a6c37bc5d721d12ce9540ffe7650d5cecbda3b0d03344eed326287adc1f6c7f3ef2024-11-25T16:56:24-05:00Eric CurtinIntroduce llama-run (#10291)
53447f931c8f9a26c072d71224bc8013cc66ea9e445106964e3d266740f571b5aad7b57545b4a901ac92024-11-25T21:50:07+02:00Georgi Gerganovserver : enable cache_prompt by default (#10501)
535d9d54e498d38ec99bbc0031022f9c92711e97bbccce5a9007572c6e9fa522296b77571d2e50713572024-11-25T09:58:41+02:00Georgi Gerganovspeculative : refactor and add a simpler example (#10362)
536cce5a9007572c6e9fa522296b77571d2e5071357dc39012cbaf8752fabecaeb60af78ccdd1dfb73b2024-11-24T18:03:25+02:00Georgi Gerganovflake.lock: Update (#10470)
5371bb30bf28cb5a7adf111bc41c935bdaf128397e787a533be57e602f8ca469d14ad15ee851265b6552024-11-21T10:22:47+02:00Georgi Gerganovllama : handle KV shift for recurrent models (#10402)
5388fd4b7fa29c3061b2e02e897d818dfcbc593430a1bacb9f62514b520bdf74ed6feb46c80508dad382024-11-20T01:40:18-06:00Jeff Bolzvulkan: copy iq4_nl LUT into shared memory (#10409)
5391bacb9f62514b520bdf74ed6feb46c80508dad38ad21c9e1f14d82b8c15ae369a8839019e3d498b42024-11-20T01:11:00-06:00Jeff Bolzvulkan: further optimize mul_mat_vec using larger loads (#10387)
5408e752a777b272606f22cb741b03e062de4ddb8fea88ad007de3eb5d92cb538fd269ff94c4bf0c8d22024-11-19T13:29:26+02:00Georgi Gerganovllama : add check for KV cache shifts (#10401)
541b3e585988fc65d3a8083c6d94dfc0629f9ce226d557924f22237c76387a39c4db5abae154d57e7542024-11-19T01:25:17-06:00Jeff Bolzvulkan: Optimize soft_max (#10301)
5422eb76b2a5e4ea395b971a419c95b473ab6f253e49b75f03cd2ec9cc482084049d87a0f08f9f015172024-11-18T16:08:20+02:00Georgi Gerganovflake.lock: Update (#10346)
543be5caccef945546ee9fd25a151330a88d785faf920a780c7b64c38071fe70ad23e16e80c23c0147b2024-11-17T12:25:45+01:00Diego Devesallama : only use default buffer types for the KV cache (#10358)
544eda7e1d4f54711de1c9b40502d6c88bbc217da6024203e9dd7355a4a10bc32d959fd0148d37bf6662024-11-17T07:31:17+01:00Diego Devesaggml : fix possible buffer use after free in sched reserve (#9930)
545fb4a0ec0833c71cff5a1a367ba375447ce6106eb5ea926dad7f62ebccff7b24784bd1e01a06d13ae2024-11-13T20:00:35+02:00Michael Podvitskiyllama : propagate the results of `graph_compute` (#9525)
5462e82ffa4af29f87e7d3d6dff8060a2a79613b72f80dd7ff22fd050fed58b552cc8001aaf968b7ebf2024-11-13T09:40:57ZAlberto Cabrera Pérezsycl : Fixes to broken builds and test-backend-ops (#10257)
54780dd7ff22fd050fed58b552cc8001aaf968b7ebf54ef9cfc726a799e6f454ac22c4815d037716eda2024-11-13T00:58:57-06:00Jeff Bolzvulkan: Optimize contiguous copies (#10254)
548b141e5f6efbab3a00633df88c4f9425bfe8b78ab4b3a9212b602be3d4e2e3ca26efd796cef13c55e2024-11-11T08:38:43+02:00Georgi Gerganovserver : enable KV cache defrag by default (#10233)
549e89213492d3e01705739789733f0f2d250b4c4498fc393f246c550d2481e53323a47644a94e8d01f2024-11-09T12:47:50+05:30amritahs-ibmggml : optimize llamafile cpu matrix multiplication for ppc64le (#10156)
5503bcd40b3c593d14261fb2abfabad3c0fb5b9e3185c333e014059122245c318e7ed4ec27d1085573c2024-11-07T18:19:10+11:00Zhiyuan LiOptimize RWKV6 Operator Naming and Implement Multi-core CPU/ SYCL Acceleration (#10133)
55194d8cb8be13b7c4d04eeca5a2b956b9148e6f2221dc04b2deed2d2f2ae3aff9b14ae29674dee1fb82024-11-06T12:10:07+01:00Diego Devesametal : fix from ptr buffer name (#10189)
552a1eaf6a9600bb1608753420ba886a3b0a208ffc0b8deef0ec0af5febac1d2cfd9119ff330ed0b7622024-11-06T10:24:23+02:00Georgi Gerganovmetal : add quantized FA support (#10149)
553401558b7ba7a08175c153cd3607230f63c8a528e9e0ecfb697d297355e43c20559d29bcc71beb0c32024-11-04T17:34:08+01:00Diego Devesaggml : fix q4xx mat mul, increase ggml_aligned_malloc alignment (#10167)
554329ed914c959c510d076fb06b43eeb3f7b804d6fce027adfb3b131f0d2368294fc276bb0e342b3f62024-11-04T19:08:22+08:00leo-ponyCANN: adjust backend registry refactor. (#10158)
5551926d6e39d6f6358bc1a4c52316a560178be7233b634f8a26fef65210fd9fb2f87e83a2809535e892024-11-02T15:18:56+02:00Georgi Gerganovllama : adjust default context size + print warnings (#10136)
556418f5eef262cea07c2af4f45ee6a88d882221fcbba6f62eb793d6617892d252f5c04d7685d908a382024-11-02T02:33:14+08:00Shupei Fanvulkan : improve ggml_vk_create_buffer error handling (#9898)
557f221d56220899f38f0126e683b2432bc79d1e3f6e597e50794f07ec8dc24b9efb18f94ec6386fda02024-11-01T10:23:05+02:00Georgi Gerganovggml : alloc ggml_contexts on the heap (whisper/2525)
55885679d37f34f66783cc04664a06c405b28e8e0351e9f94994ef908d964cf81069f03d9d3668beb7d2024-11-01T00:49:53+01:00Diego Devesallama : improve output buffer type selection (#10098)
559c02e5ab2a675c8bc1abc8b1e4cb6a93b26bdcce7ab3d71f97f5b2915a229099777af00d3eada1d242024-10-31T22:54:23+01:00Diego Devesallama : fix buffer checks for mamba and rwk (#10111)
560b9e02e8184f5e6094a9e87eaf040becd404bfc906763f713bb692910e9b2d9d1a82d6959cee2dcf32024-10-30T14:51:21+01:00Diego Devesaggml : fix memory leaks when loading invalid gguf files (#10094)
5618d8ff715367480b856ad86ac3888e9742b13a6fa61715d5cc83a28181df6a641846e4f6a740f3c742024-10-29T10:42:05+02:00Georgi Gerganovllama : remove Tail-Free sampling (#10071)
5628125e6cbfcf2b3b9066e4d923aca9295526730f58841ce3f439de6e770f70319b7e08b6613197ea72024-10-28T08:49:32+02:00Georgi Gerganovserver : don't overfill the batch during infill (#10018)
563bc5ba007b2c83ac95875e68724dabfc12159fc61958367bf530d943a902afa1ce1c342476098576b2024-10-25T10:13:46+03:00Georgi Gerganovserver : check that the prompt fits in the slot's context (#10030)
564167a515651a4b065a16225ffc69564c5674f3d0fc39665f589091903396a442a6ee56613303e03502024-10-24T14:40:23+02:00Johannes GäßlerCUDA: fix insufficient buffer clearing for MMQ (#10032)
565c19af0acb1fe6d0fdbecadd8483c1fbe5d68d095ac113a0feee0935b2018312f7bc8d7a646b117ed2024-10-16T20:10:01+02:00Daniel Beveniusggml : remove redundant set of contexts used field (ggml/978)
566ac113a0feee0935b2018312f7bc8d7a646b117ed4c9388fb96ac2415fbb1239b7ba8346616606e2e2024-10-23T07:09:26-04:00Michael Coppolallama.vim : add classic vim support (#9995)
567c8c07d658a6cefc5a50cfdf6be7d72650361230319d900a7565b8f6b0a708836a57d26966cb9efe22024-10-22T16:59:02+02:00Xuan Son Nguyenllama : fix empty batch causing llama_batch_allocr to crash (#9966)
56819d900a7565b8f6b0a708836a57d26966cb9efe211d47057a51f3d9b9231e6b57d0ca36020c0ee992024-10-22T15:31:06+02:00Daniel Beveniusllama : rename batch to ubatch (#9950)
56911d47057a51f3d9b9231e6b57d0ca36020c0ee99c421ac072d46172ab18924e1e8be53680b54ed3b2024-10-22T21:22:26+08:00Molly SophiaRwkv chat template fix (#10001)
5704ff7fe1fb36b04ddd158b2de881c348c5f0ff5e46b8447352df3d662b56280c8fc38d7f0928857872024-10-22T18:33:37+08:00Molly Sophiallama : add chat template for RWKV-World + fix EOT (#9968)
571bc219750845a59166d79f0d4ee3da1993b369b8a1db8c84fc62857e1e45c1c7ea93bcd5344cb3d312024-10-21T09:37:12+03:00Georgi Gerganovspeculative : fix handling of some input params (#9963)
572cda0e4b648dde8fac162b3430b14a99597d3d74fafd9909a6481402844aecefa8a8908afdd7f52f12024-10-18T23:18:01+02:00Xuan Son Nguyenllama : remove all_pos_0, all_pos_1, all_seq_id from llama_batch (#9745)
57360ce97c9d809f4b040e90b597468b839df5728d08901755ba328643c9ab071c20e1939ea52951a0e2024-10-18T13:34:36+08:00Ma Mingfeiadd amx kernel for gemm (#8998)
5746f55bccbb8835d42147add4ee48807450f5ff53517bb9280807cfbb6611b853aa1ef05114bd9efe92024-10-18T01:41:51+02:00Daniel Beveniusllama : rename batch_all to batch (#8881)
57517bb9280807cfbb6611b853aa1ef05114bd9efe99f45fc1e9950a496febc575cdd196cd5cad000cc2024-10-17T23:43:05+03:00Georgi Gerganovreadme : remove --memory-f32 references (#9925)
57621942002780352b4a54f4bd3e5eefa3bc7f14fe673afe681aa76e818733fc1f30de082c1d6910bcd2024-10-17T02:34:22+03:00Gilad S.fix: allocating CPU buffer with size `0` (#9917)
57773afe681aa76e818733fc1f30de082c1d6910bcd9e041024481f6b249ab8918e18b9477f873b5a5e2024-10-17T01:36:51+03:00Gilad S.fix: use `vm_allocate` to allocate CPU backend buffer on macOS (#9875)
578cd60b88bf7ad7785fb6ac9864e360cf10e42faadbecfd387f6919d99ec34b76c2522f90ac250c4892024-10-09T16:40:35+02:00Daniel Beveniusggml-alloc : remove buffer_id from leaf_alloc (ggml/987)
579223c25a72fcc3f65cdfd7f5d57edd5b44b550e18fbc98b748e7b075e327bcf13237057f6476780492024-10-15T16:28:55+03:00Georgi Gerganovserver : improve infill context reuse (#9894)
580d4c19c0f5cdb1e512573e8c86c79e8d0238c73c4c7181bd294757dd80a7904e3dd0fea2d0be914e72024-10-13T21:31:35+03:00Georgi Gerganovserver : accept extra_context for the infill endpoint (#9874)
581c7181bd294757dd80a7904e3dd0fea2d0be914e792be9f12164f18ce845a5bab60cefa5f7fec68362024-10-13T18:52:48+03:00Georgi Gerganovserver : reuse cached context chunks (#9866)
5821bde94dd024b632f98428f4bf2ce48329513077995c76e8e92ecc93f784b185eafae36a0e7ad2fa32024-10-12T16:06:31+03:00Georgi Gerganovserver : remove self-extend features (#9860)
58311ac9800aff532715a5bc7991062c68ba3472e6e943d20b4111c746bcd9dbc7e4771de313b08b50c2024-10-12T08:21:51+03:00Georgi Gerganovllama : improve infill support and special token detection (#9798)
5846374743747b14db4eb73ce82ae449a2978bc3b47f1af42fa8c925096407c61ff0a3d5d5d669cc5352024-10-07T21:55:08+02:00Diego Devesaggml : add backend registry / device interfaces to BLAS backend (#9752)
5856279dac039ddeb6d5ebd125a6274fd3c37a77ba8d5ac8cf2f2e30459489e6721a17d15b92a0c42a62024-10-07T19:35:42+03:00Georgi Gerganovflake.lock: Update (#9753)
586d5ac8cf2f2e30459489e6721a17d15b92a0c42a696b69121033d2b6b951d1b6b1b43f8b4f97dac992024-10-07T18:27:51+03:00Georgi Gerganovggml : add metal backend registry / device (#9713)
58796b69121033d2b6b951d1b6b1b43f8b4f97dac99d5cb86844f26f600c48bf3643738ea68138f961d2024-10-07T13:26:31+01:00Paul Tsochantarismetal : single allocation of encode_async block (#9747)
588b0915d5b51cbaa982ce9bbb9ce302bb9abdca0eb8c475b97b8ba7d678d4c9904b1161bd8811a9b442024-10-06T08:34:20+01:00SRHMorrisvulkan : retry allocation with fallback flags (whisper/2451)
589905f5485b279518d30b402565c23fb153f822c0d71967c2a6d30da9f61580d3e2d4cb00e0223b6fa2024-10-05T14:33:54+03:00Georgi Gerganovmetal : zero-init buffer contexts (whisper/0)
59055951c018d7c107b6ef0a4c8561a6e68183d19d9ff565769f289c6adcc91ed1b8fdabaf9a0d4f6ee2024-10-04T15:46:18+02:00Daniel Beveniusggml : fix typo in example usage ggml_gallocr_new (ggml/984)
591ff565769f289c6adcc91ed1b8fdabaf9a0d4f6eef3fdcfaa79afa12047def3a8793d4a566e0532d42024-10-04T08:41:40+02:00Diego Devesaggml : fixes after sync (ggml/983)
592d6fe7abf04e8ec5240dead6e2773ed1b7e7495d3e3c355ba654d4164c1c09e5d0dcacecb8b214af82024-10-03T12:39:03-03:00bandotiggml: unify backend logging mechanism (#9709)
593cb00020504416606601f8cb35f55ee07b710b4b76c5322481a75f1be54e58a000c3f78484d07f9482024-09-30T09:14:09+02:00Salvatore Mesoracavulkan : mul_mat: fix UB with small warps (ggml/952)
5947254cdf7e8d6312840509ca8d766358c687c6266cad341d88924788b940997c55e7bef000c99e7842024-09-29T23:18:02+02:00Johannes Gäßlerggml: fix gradient allocation logic (ggml/966)
595faac0bae265449fd988c57bf894018edc36fbe1ef99d3f8367174f7aba73c07fd87de687d4a0ece12024-09-29T05:25:00-07:00matiaslincommon : ensure llama_batch size does not exceed max size (#9668)
59689f9944981010d195e411a9fbfbb19959412f710b5de3b74a595cbfefab7eeb5a567425c6a9690cf2024-09-28T12:05:05+02:00Markus TavenrathEnable use to the rebar feature to upload buffers to the device. (#9251)
597afbbfaa537a96f562c34df4542930fa951b40d9e3d6bf6919f7b10726421779cd344f2da05421c682024-09-25T14:05:13+02:00Xuan Son Nguyenserver : add more env vars, improve gen-docs (#9635)
598116efee0eef09d8c3c4c60b52fa01b56ddeb432c0b3bf966f47bf2ba88e5d4e3ed429602008c7e632024-09-24T03:14:24+03:00Ivancuda: add q8_0->f32 cpy operation (#9571)
5990b3bf966f47bf2ba88e5d4e3ed429602008c7e63f0c7b5edf82aa200656fd88c11ae3a805d7130bf2024-09-23T22:23:54+02:00Xuan Son Nguyenserver : add --no-context-shift option (#9607)
600f0c7b5edf82aa200656fd88c11ae3a805d7130bf1d48e98e4f3316bd2f6b187d288c7b6cb88d5cb32024-09-23T11:42:43-07:00Max Krasnyanskythreads: improve ggml_barrier scaling with large number of threads (#9598)
601c35e586ea57221844442c65a1172498c54971cb0912c331d3dba3a079815844208dc36164baa8cc72024-09-22T22:55:49+08:00R0CKSTARmusa: enable building fat binaries, enable unified memory, and disable Flash Attention on QY1 (MTT S80) (#9526)
602ecd5d6b65be08927e62de1587d5fd22778cdc2502a63caaa69fad6c2afddc47bae052cf2afb015292024-09-21T19:30:34-07:00Shankarllama: remove redundant loop when constructing ubatch (#9574)
6032a63caaa69fad6c2afddc47bae052cf2afb01529d09770cae71b416c032ec143dda530f7413c40382024-09-22T10:29:12+08:00Molly SophiaRWKV v6: RWKV_WKV op CUDA implementation (#9454)
604d09770cae71b416c032ec143dda530f7413c403841f477879fd5ccc31211634292e8e293ec700e852024-09-21T14:24:23+02:00slarenggml-alloc : fix list of allocated tensors with GGML_ALLOCATOR_DEBUG (#9573)
605424c5d00a9b97dd5559635872db9b57f87c23b02a6809c6a2e8163cba296d4cc0c5cd4bbc80736382024-09-20T19:04:44+03:00Johannes Gäßlerggml/examples: add backend support for numerical optimization (ggml/949)
606faf67b3de4688f47c3b1019c89df255df2fd59b47be099fa817e9c53ffb4c3ed7d063e1cffcd675a2024-09-18T08:30:31+08:00Neo Zhang Jianyu[SYCL]set context default value to avoid memory issue, update guide (#9476)
6070226613853133c081b55bb892a41bb5eacc0bc94503147a9f9d195d6a14e7c998df23b6eb61f2bae2024-09-17T01:19:46-07:00Max Krasnyanskythreadpool : skip polling for unused threads (#9461)
608acb2c32c336ce60d765bb189563cc216e57e9fc2a6a3a5c531c73aef85750a847d21e7d4671e723d2024-09-16T13:07:13+02:00Daniel Beveniusllama : rename n_embed to n_embd in rwkv6_time_mix (#9504)
609441b72b91f818fe69497e5816f87969e90c73c43c4965a64f72ac9434c21cf0e1c3421d13e8891552024-09-16T01:20:01-05:00Vinesh Janarthananmain : option to disable context shift (#9484)
610c4965a64f72ac9434c21cf0e1c3421d13e88915590a2fff0e7f80c6fea3fc6cf9a7b482744f3f1642024-09-16T09:05:56+03:00Georgi Gerganovmetal : handle zero-sized allocs (#9466)
611e6b7801bd189d102d901d3e72035611a25456ef1e665744317c77fc3483fc5224fe6d586b5166b332024-09-12T19:46:43+08:00Dou Xinpengcann: Add host buffer type for Ascend NPU (#9406)
61239f852f44039b058fdd0611ee127c6efa7ba4a042b00fa799773cc75d53b841c03d21d7468a1e3a12024-09-12T19:25:16+08:00Molly Sophiapy : add special tokens in hf_converter for RWKV v6 (#9428)
613d2b496bff4f353a6429f8e833448f071bd237ba7b34e02348064c2f0cef1f89b44d9bee4eb15b9e72024-09-11T10:03:54+03:00Georgi Gerganovbatched-bench : remove unused code (#9305)
61483008b7cfe90ad89d0c0ed2c2424fd75edc25ac10b4ac75772b744bb0a0d674927587621d10578842024-09-10T09:03:21+02:00Daniel Beveniusllama : update llm_build_copy_mask_state comment [no ci] (#9385)
6150b4ac75772b744bb0a0d674927587621d1057884fb3f2498156b3140e2050ec9c7bf61372f63ff562024-09-10T15:02:30+08:00Molly SophiaRWKV v6: Add time_mix_decay_w1/w2 in quant exclusion list (#9387)
616293bebe0773c907c0c866213856eeba41b035df15fac4d57643b1de8e9ab746f14d2fc4e319ae0c22024-09-09T18:40:10+03:00Radoslav Gerganovrpc : fix segfault with nkvo (#9389)
617daa9623ab051a8162ae750b150b9522571b55f21e079bffb6678e1b5ded21c719600bedd7175e7262024-09-08T21:43:48+02:00Markus TavenrathOverlap cmdbuffer creation and cmdbuffer execution in Vulkan backend by submitting smaller cmdbuffers early. (#9118)
6182a358fb0c4b6e917ac852aa17444cc94dd28a2a6eae597182cb61bbde0b26e7cec5999d28b9327fe2024-09-08T19:05:29+08:00Neo Zhang Jianyu[SYCL] add check malloc result on device (#9346)
61960a3107ccd791d858e12a87e6024ce918d3bf595406c1a32a18807b9b5711aa2fa1859527106bc1d2024-09-08T09:38:42+03:00Georgi Gerganovscripts : option to increase git patch context
620406c1a32a18807b9b5711aa2fa1859527106bc1d9cb9260861e464e40d38764cfb021856786c72022024-09-06T14:34:25+02:00Salvatore Mesoracavulkan: add dryrun support to sin and cos ops (ggml/947)
621f12295b8a9336962bf02a359beb1be0d322b4be7faf69d4237c9ae4d7f572b4674d1002463e8acd32024-09-08T00:33:33+03:00Georgi Gerganovllama : fix empty ring buffer push (#9358)
622faf69d4237c9ae4d7f572b4674d1002463e8acd3e536426ded3fb4a8cd13626e53508cd92928d6c22024-09-08T00:33:13+03:00Georgi Gerganovllama : sanitize invalid tokens (#9357)
623e536426ded3fb4a8cd13626e53508cd92928d6c21b9ae5189cd279c6b45e36d43e4f9ccae628d02f2024-09-07T19:02:26ZEvellamafile : disable sgemm for batch-size 1 (#9330)
624815b1fb20a53e439882171757825bacb1350de04409dc4f8bb5185786087f52259ee4626be93f54d2024-09-06T18:59:58+03:00Aarni Koskelabatched-bench : add `--output-format jsonl` option (#9293)
6254db04784f96757d74f74c8c110c2a00d55e33514bdf314f38a2c90e18285f7d7067e8d736a14000a2024-09-05T11:13:11+02:00slarencuda : fix defrag with quantized KV (#9319)
626f1485161e58d562099dd050f8ac3a9ea9f4cd765048de848ee4baad4531fcd6239438f5d55be365c2024-09-03T01:53:23+08:00Zhenwei Jinsrc: make tail invalid when kv cell is intersection for mamba (#9249)
627f771d064a95d212ddadea452ad0cc1e42b2c3db36e7d133a5f9409dd257fad90d7f320721b07a1b22024-09-02T08:25:30-07:00yuri@FreeBSDggml : add pthread includes on FreeBSD (#9258)
6288f1d81a0b6f50b9bad72db0b6fcd299ad9ecd48ca47667cff41f5a198eb791974e0afcc1cddd32292024-09-01T22:38:17+08:00Molly Sophiallama : support RWKV v6 models (#8980)
62942c76d1358021ccdbe8ba89109c143dd7ae166df9f7d4bcf5c27d37b0c7da82eeaf9c1499510554b2024-08-29T19:20:53-04:00Faisal ZaghloulThreadpool: take 2 (#8672)
630a77feb5d71831c61e455541e8a655b9f0337ea8c2e59d61c1b321431c597c1f12249273427d5640d2024-08-27T11:07:01+02:00Xuan Son Nguyenserver : add some missing env variables (#9116)
63106658ad7c37f440502de2b9486ce43c47b4ec710fc18425b6a8ad03847383ce2b69d52edfd49b0ff2024-08-26T18:31:02+03:00Georgi Gerganovmetal : separate scale and mask from QKT in FA kernel (#9189)
632a1631e53f6763e17da522ba219b030d8932900bdfc54ef0d1c138133a01933296d50a36a1ab647352024-08-21T17:58:11-04:00compiladellama : simplify Mamba with advanced batch splits (#8526)
6334b9afbbe9037f8a2d659097c0c7d9fce32c6494c37501d9c79ed5897db4b73ea7502211d25b3f7632024-08-15T15:40:12+08:00gtygoretrieval : fix memory leak in retrieval query handling (#8955)
6345fd89a70ead34d1a17015ddecad05aaa2490ca4698a532d474c73d3494a5353024cb6a4fbbabbb352024-08-14T18:32:53+02:000cc4mVulkan Optimizations and Fixes (#8959)
63598a532d474c73d3494a5353024cb6a4fbbabbb3543bdd3ce188cd247bda7c1bd1ad01fa64e5666902024-08-14T02:51:02-04:00compiladeserver : fix segfault on long system prompt (#8987)
63684eb2f4fad28ceadd415a4e775320c983f4d9a7d1262e7ed13ac197c944f15e1ddb083cb4f36cf652024-08-12T20:45:50+08:00Frank Maidocs: introduce gpustack and gguf-parser (#8873)
6377c5bfd57f83fd3630934cfa70892aa4022d3faf76e02327e8b7837358e0406bf90a4632e18e278462024-08-11T10:09:09+02:00Markus TavenrathOptimize Vulkan backend for better CPU performance and less GPU synchronization overhead. (#8943)
63870c0ea35609a2ab87a358e25f4ffad1aad4089925b2c04f4925e152c362b824f4b41eb2a081fa6232024-07-16T03:21:09-04:00Matt Stephensonwhisper : use vulkan as gpu backend when available (whisper/2302)
639345a686d8271a24db20d31789c0d4b9ed51dcb0c3a14e00366399040a139c67dd5951177a8cb56952024-08-08T23:54:00-04:00compiladellama : reduce useless copies when saving session (#8916)
640c21a896405de4cdf4207eb8130555ceaac0ab110d4ff847153e9cf7220d1b39aa21172069e6e8cea2024-08-06T12:42:42+08:00Mengqing Cao[CANN]: Fix ggml_backend_cann_buffer_get_tensor (#8871)
6410a4ce786814b123096d18aadca89cd352b9e590bbc0f887e159c0d78c28121e2c8b5c580941708752024-08-06T00:14:10+08:00Liu Jiacommon : Changed tuple to struct (TODO fix) (#8823)
642bc0f887e159c0d78c28121e2c8b5c58094170875b42978e7e4d56eaaa93588414e804d9fbbc3cae22024-08-05T21:10:37+08:00wangshuai09cann: fix buffer_num and runtime speed slowly error (#8865)
643a3738b2fa7c60ef2c4592435d1aa7fb8f1f69c3e655858ace0cf2720e56eb01f84ad05e0c94ada3c2024-08-04T17:28:08+02:000cc4mvulkan : implement Stable Diffusion operators (ggml/904)
644ecf6b7f23e664afd7ff856ec39034240ce438daa01aae2b4975b57a265ce8194928fd87f2d71027e2024-08-04T03:55:03-07:00Brian Cunniebatched-bench : handle empty `-npl` (#8839)
645afbb4c1322a747d2a7b4bf67c868148f8afcc6c8b7a08fd5e0e7c898c68d1743066ea495202d96082024-08-01T23:28:28+02:00matteoggml-cuda: Adding support for unified memory (#8035)
6467e72aa74fd676a093eb9970e761085ec22734c717c27a19b2eb91bb0f43c7f7aec0386cec2dddc332024-07-31T00:57:03+10:00Brianpy: add_array() will not add to kv store if value is an empty array (#8774)
6474c676c85e59ef8f771f3a129e6eb217552139231e54c35e4fb5777c76316a50671640e6e144c95382024-07-28T00:42:05-04:00compiladellama : refactor session file management (#8699)
648203b7f1531303a060730ec1d1e01920e70302398d2b851bfa131478665315bc5c7c707506c14d7032024-07-20T20:49:44+02:00Tony Wasserkavulkan : initialize vk_buffer_struct members to VK_NULL_HANDLE (ggml/893)
649b5e95468b1676e1e5c9d80d1eeeb26f542a38f4292090eca212650727e38b335c1d4accfbcc9b79c2024-07-27T05:03:45-07:00Jeffrey Morganllama : add support for llama 3.1 rope scaling factors (#8676)
65049ce0ab6d45402e8bb622bf86f86529f2b0ba5524226a8d10e3904db3a1297919fe6c7f06beba6c02024-07-25T22:23:05+01:00DavidKorczynskiggml: handle ggml_init failure to fix NULL pointer deref (#8692)
651328884f4219c0228673cf1870ac63987fb4f9fd0c69c63039cd75f8f33f253ab7485d82a8b4cd4032024-07-20T21:58:49-04:00compiladegguf-py : fix some metadata name extraction edge cases (#8591)
652c3776cacabce2ee35f172fb72be7a519752125fa87e397d00bdcedd5cbf6dfda06a7b0f3024627282024-07-20T17:35:25+10:00Briangguf_dump.py: fix markddown kv array print (#8588)
6533d0e4367d99087892e355ddbeebd232a0b2f40dea15ef8f8a08e12f9c5162c221e67779e711820732024-07-19T17:51:51+10:00Brianconvert-*.py: add general.name kv override (#8571)
654672a6f101839a150180fc28891ebed379c8f23533807c3de04cde853418033c95e96642876545f3e2024-07-18T20:40:15+10:00Brianconvert-*.py: GGUF Naming Convention Refactor and Metadata Override Refactor (#7499)
655da3913d8f9475b0d4bcbeb4936c724af4eade092d65a8361fed8be97389776fb94217e937ec6b03c2024-07-17T16:34:28+09:00Masaya, Katobatched: fix n_predict parameter (#8527)
656d65a8361fed8be97389776fb94217e937ec6b03c5e116e8dd51775f8f1c090570be148d5d7eea6c32024-07-17T10:32:59+03:00Georgi Gerganovllama : disable context-shift for DeepSeek v2 (#8501)
6577acfd4e8d55082c1b597dfc3ffe04fb5d530c6dc97bdd26eee11fe109dec00de75690ceef61c03f22024-07-15T23:13:10-04:00compiladeconvert_hf : faster lazy safetensors (#8482)
65897bdd26eee11fe109dec00de75690ceef61c03f24db8f60fe79a391e82b0464013ada123baced96a2024-07-15T20:50:47+02:00Xuan Son NguyenRefactor lora adapter support (#8332)
6599104bc20edf47f74dc49379b7d61d0c6e85a4882fc690b018e459012cd82c37f1343e9bb658987d12024-07-15T14:54:58+03:00Georgi Gerganovcommon : add --no-cont-batching arg (#6358)
660fa79495bb4897953a75607addd9d2cdd2ec6322217eb6aa8a992cda37ee65cf848d9289bd6cad8602024-07-13T23:35:10-04:00compiladellama : fix pre-tokenization of non-special added tokens (#8228)
661c3ebcfa148e867a68e78fd5c4f0c23e8f84c788b8a4441ea1a2564578134404f31158c318e9c0bf32024-07-12T03:14:12-05:00Douglas Hanleyserver : ensure batches are either all embed or all completion (#8420)
662808aba39161e5d7ca2ff24110b5aa14d2e536988a977c115448e40856fb9cbe3ceb6d8ce802553b02024-07-11T16:47:47+02:00Johannes GäßlerCUDA: optimize and refactor MMQ (#8416)
663278d0e18469aacf505be18ce790a63c7cc31be26dd07a123b79f9bd9e8a4ba0447427b3083e9347a2024-07-10T20:08:17-04:00Clint HerronInitialize default slot sampling parameters from the global context. (#8418)
6640f1a39f3439825acf7e3a1663566d410be15217083321c6958acf20747d288031174cc1bf8816e332024-07-10T07:14:51-05:00Dibakar Gopeggml : add AArch64 optimized GEMV and GEMM Q4 kernels (#5780)
665a59f8fdc85e1119d470d8766e29617962549d993fd560fe680c72fd0a0af2bc8881add20ad9190712024-07-09T18:26:40-04:00Clint HerronServer: Enable setting default sampling parameters via command-line (#8402)
666a03e8dd99d3954e7547137936c5a2a3b348bdb7f5b0b8d8cfb5ddf2118f686ba6c30fab3f71b384b2024-07-09T17:11:07+02:00Johannes Gäßlermake/cmake: LLAMA_NO_CCACHE -> GGML_NO_CCACHE (#8392)
667470939d483d1c89b7292f78bac1fd27c42c171ce6f0dbf6ab087bcd286fb78560099ca04583167352024-07-08T03:26:53-04:00Kevin Wangcommon : preallocate sampling token data vector (#8363)
6683fd62a6b1c9ca7b7c0093e984cc9c133c6f2726da8db2a9ce64cd4417f6a312ab61858f17f0f85842024-07-07T15:04:39-04:00compiladepy : type-check all Python scripts with Pyright (#8341)
669f7cab35ef9e66c57b4a416d09eb6c814e0ba4e4c905942abdba5ba0b28a1b0805e51e4f818c54bc92024-07-07T22:58:43+10:00Briangguf-hash: model wide and per tensor hashing using xxhash and sha1 (#8048)
670cb4d86c4d723af87d3d7e3177e9485f20039138486e7299ef5dff0f388922dc6fcbce009e99d80052024-07-07T11:10:38+02:00Bjarke Viksøeserver: Retrieve prompt template in /props (#8337)
67187e25a1d1bd26eb06d1cab9e2ee4e14a7a0be33c213701b51a17175d0d326b566efc03f30ec7fbe62024-07-06T09:22:16+02:00Daniel Beveniusllama : add early return for empty range (#8327)
6721f3e1b66e21310ed78b964f72f19766549633f0e148ec970b62c3c5ae0a8bfdaad2fc237aaae350d2024-07-05T13:23:25+01:00Ouadie EL FAROUKIEnabled more data types for oneMKL gemm_batch (#8236)
673f09b7cb609d80b8031803f89255991dc8b35db69a38b884c6c4b0c256583acfaaabdf556c62fabea2024-07-05T10:32:29+08:00Neo Zhang Jianyurm get_work_group_size() by local cache for performance (#8286)
6741e920018d38aee270a044cc48eaefe7c64cb875001a5f06550a866bd63717635e5e7e1ff4203b8732024-07-03T01:02:56+05:30ditsukedoc: Add context for why we add an explicit pytorch source
675807b0c49ff7071094f97ebc3a0a8e2b9e274f503f8c4c0738d72d2162736edd72dd5db8b269adca12024-07-04T15:46:11+02:00fairydreamingInference support for T5 and FLAN-T5 model families (#5763)
676d08c20eddedb24515a3212e2de66bdff41a26b8c5fac350b9cc49d0446fc291b9c4ad53666c775912024-07-02T02:16:00Zluoyu-intel[SYCL] Fix the sub group size of Intel (#8106)
67797877eb10bd8e7f8023420b5b5300bcbdadd62dc387952651a8fc493f8c85ea4c9774bd4a5694f872024-06-27T15:48:07+01:00jukofyorkControl vector loading fixes (#8137)
678c8771ab5f89387cdd7d9a8a69280dac46b45e02f494165f3b6c4cbcd793123cb57fb3e1f5477f1db2024-06-26T08:28:02+02:00Johannes GäßlerCUDA: fix misaligned shared memory read (#8123)
6793791ad219323389106dc3fd80814eb5bbb7b80def702a90e245499283d6de0b287701c723cda2a872024-06-25T16:57:35+05:30HanishKVCSimpleChat v3.1: Boolean chat request options in Settings UI, cache_prompt (#7950)
680083bacce14c1aaf9976aa40e8266cdc25ac749d32df373ac40ea581ccca8a58c713f03ad9d4b658d2024-06-25T10:19:20+08:00Meng, Hengyu[SYCL] Re-enabled mul_mat_batched_sycl (#8095)
68152fc8705a0617452df08333e1161838726c322b48cb508d0d5c024e12692370d85237b45469a004b2024-06-24T05:42:03-04:00Christian Zhou-ZhengOption to split during conversion (#6942)
682e112b610a1a75cb7fa8351e1a933e2e7a755a5ce6a2f298bd784403c5c33eebb822217ec5d9b55902024-06-24T02:27:57+08:00Eddie-Wangllama : add support for BitnetForCausalLM (#7931)
68345c0e2e4c1268c2d7c8c45536f15e3c9a731ecdcb5a5f34efadec8d8f0057b35cb04742abfeb2ef52024-06-23T10:21:25+02:000cc4mRefactor Vulkan backend to allow multiple contexts (#7961)
68480ea089d771f0c2d97afa8bead80ded412f600d70e64591e8290037db6412665a56354b789a0597e2024-06-21T00:38:22-05:00Douglas Hanleyllama : allow pooled embeddings on any model (#7477)
685d50f8897a797a5a03f31228d1b5a7b8130ee1bc22075a66a96cc1b04eabec7cf4b3051193d6f719e2024-06-20T14:39:21+02:00Johannes GäßlerCUDA: stream-k decomposition for MMQ (#8018)
68637bef8943312d91183ff06d8f1214082a17344a591c188d6c296bd3384f2a02a83b71187aa3d18b32024-06-18T18:40:52+02:00jaime-m-ptokenizer : BPE fixes (#7530)
687e6ecc2be470e3c5c6c5c9d8b90aa83a1f7725084a94e6ff8774b7c9f950d9545baf0ce35e8d1ed2f2024-06-18T09:37:20+03:00Georgi Gerganovwhisper : use ggml_backend_sched (whisper/2239)
688a94e6ff8774b7c9f950d9545baf0ce35e8d1ed2f5b6da187508f49a9fa9d95fa22ae804a0780d2562024-06-17T22:08:46+03:00Ștefan-Gabriel Muscaluupdate: support Qwen2-57B-A14B (#7835)
6897c26775adb579e92b59c82e8084c07a1d0f75e9cb473e95084c286780165568cf0f385f21141d68d2024-06-17T19:40:01+03:00Georgi Gerganovllama : disable FA if KV head size do not match (#7982)
6906a2f0b3474d479bda4ac2ee7cfd5dcdcf0be1f7921be9cab94e0b5b53cb6edeeebf8c8c799baad032024-06-17T16:10:15+02:00Markus TavenrathImplement non-mapped async IO for CUDA on Windows. (#7896)
6917c7836d9d4062d6858e3fb337b135c417ccee6ce0c7b3595b9e5ad2355818e259f06b0dc3f0065b32024-06-16T07:17:31+02:000cc4mVulkan Shader Refactor, Memory Debugging Option (#7947)
6920c7b3595b9e5ad2355818e259f06b0dc3f0065b37b2f4a7d193ef2475259bbe7656fcccfab4b12172024-06-15T18:53:40+02:00Xuan Son NguyenAdd `cvector-generator` example (#7514)
6937b2f4a7d193ef2475259bbe7656fcccfab4b1217f8ec8877b75774fc6c47559d529dac423877bcad2024-06-15T14:05:10+08:00Meng, Hengyu[SYCL] remove global variables (#7710)
69466ef1ceedf983773c8ceb4d925285d41d4e50e2ae65bbf606c61f49dc06c7ac060cd5ba7ae4460252024-06-14T17:14:09+03:00Georgi Gerganovmetal : utilize max shared memory for mul_mat_id (#7935)
695f578b86b2123d0f92afbaa98a031df4d4464e5821c641e6aac5c18b964e7b32d9dbbb4bf5301d0d72024-06-13T03:11:35+02:00slarenmove BLAS to a separate backend (#6210)
696ef52d1d16afc695d798396cdd13594ea5e45a9dd14f83526cd27f638c856ea6eff08110b9860eb2a2024-06-11T21:20:29+02:000cc4mUpdate Vulkan RoPE implementation (#7818)
697af4ae502ddaeb03cd5861273ca2e9a5ae4551db710ceba354a3b152ff425e9fa97f9caaef99a46b12024-06-10T02:21:31-07:00Ben Ashbaughuse the correct SYCL context for host USM allocations (#7777)
698d4d915d351d1f1270d56184bdd46672893e8a5d87a16ce7db2a74a223f0f3b9cee66d4539c5bce8f2024-06-08T20:21:08+01:00Olivier Chafikurl: save -mu downloads to new cache location (#7826)
699da799b41891e34aac86ce4e173f9c4c0afd4fab3c00fad71e507ff386d42bd74846fe06d19dd63a42024-06-07T19:47:49+02:00slarenvulkan : reuse parent extra for views (#7806)
7007027b27d765db95d4ac6b569d976e387a8715881a5cabd76491f07494c5b8267f921c73f5e2bbfb42024-06-07T11:15:49+02:00Johannes Gäßlerserver: update cache_prompt documentation [no ci] (#7745)
7012b3389677a833cee0880226533a1768b1a9508d29973e81c5ccf4f31b3980f5aa73f5cfea86998602024-06-05T11:29:20+03:00Georgi Gerganovggml : refactor rope norm/neox (#7634)
7021442677f92e45a475be7b4d056e3633d1d6f813b554c247caffed64465f372661f2826640cb104302024-06-04T21:23:39+03:00Georgi Gerganovcommon : refactor cli arg parsing (#7675)
703bde7cd3cd949c1a85d3a199498ac98e78039d46fa5735e4426b19a3ebd0c653ad8ac01420458ee952024-06-03T20:03:26+03:00Radoslav Gerganovllama : offload to RPC in addition to other backends (#7640)
704549279d8049d78620a2b081e26edb654f83c3bbd9e405b6e2ecb888e860f7b92720b4809e21b39152024-06-03T08:34:43+03:00Georgi Gerganovllama : avoid double token-to-piece cache (#7654)
7053413ae2193d0693f14bead02e5018f442cbf579b1669810d7c2446af8425aa54ff6611bf6f14646c2024-06-03T07:59:54+10:00Dave Airliefix bug introduced in using calloc (#7701)
7062ac95c9d5678d05e253691fb1f26471675bff5ad750f60c03e4d3f53fa51910551ce87a3d508d2d72024-06-01T21:50:18+05:30HanishKVCSimpleChat: Simple histogram/repeatMatching driven garbageTrimming, Settings UI, Streaming mode, OpenAi Compat (Model, Authorization Bearer), Save/Restore session, Auto Settings UI (#7548)
707750f60c03e4d3f53fa51910551ce87a3d508d2d79b596417af11c9ac44fcae0fcfbc6f36650890832024-06-01T15:47:04+02:00Johannes GäßlerCUDA: fix Pascal FA, deq. KV to FP16 for batch > 8 (#7681)
7089b596417af11c9ac44fcae0fcfbc6f3665089083a323ec60af14a33d560df98f2cc41b4112cb4f802024-06-01T08:44:14+02:00Johannes GäßlerCUDA: quantized KV support for FA vec (#7527)
7095921b8f089d3b7bda86aac5a66825df6a6c106035dcdf946764fae49a8e2a90bf2f0960bde1c44e82024-05-30T19:01:41+03:00Georgi Gerganovllama : cache llama_token_to_piece (#7587)
71002c1ecad07f0e2d2febe8196271bcc64bdc9c0066bd12ce409f949012935b7d1b15a21ffa473a5652024-05-28T21:46:34+02:00jaime-m-pTokenizer WPM fixes (#7500)
711ee3dff6b8e39bb8c1cdea1782a7b95ef0118f970edc29433fa08b4e5aeb67649a29fc7713af13d042024-05-28T17:07:05+02:00fairydreamingAdd support for DeepseekV2ForCausalLM (#7519)
712b9adcbbf92fc7096bee23fe61496d25652ebf7659588f196b1d7b21bdff013fcf958c249576b26192024-05-26T06:26:34+05:30HanishKVCSimpleChat Completion Mode flexibility and cleanup, Settings gMe, Optional sliding window (#7480)
713902184dd3a9d6685e752b19027a48423742531db57684331fc2d685f7d1f5775af0b9e47d18298332024-05-25T05:30:59+02:00Xuan Son Nguyenfix missing slash in `fs_get_cache_directory()` (#7503)
7143015851c5ac7334fb544a23a70a284c117b8704455ac3b7aeaf52f19786ed96e885d89521fc0f6c82024-05-23T14:29:26+02:00Daniel Beveniusllama : add getters for n_threads/n_threads_batch (#7464)
7159b82476ee9e73065a759f8bcc4cf27ec7ab2ed8ca61a94e543e3c6877c087e80fca27a0313ce5fd52024-05-23T11:49:53+02:00fairydreamingAdd missing inference support for GPTNeoXForCausalLM (Pythia and GPT-NeoX base models) (#7461)
716a61a94e543e3c6877c087e80fca27a0313ce5fd5152da28ae54139e3754189b9e6e1c28e112775022024-05-23T12:38:18+03:00Georgi Gerganovllama : rename n_ctx -> cache.size, less confusing (#0)
7171e374365d170b7f692fd7753c145e21bc14486c8197ff91462dd05bb9a3be03578114abf0c3555362024-05-22T23:23:21+05:30HanishKVCSimpleChat: a simple and dumb web front end for testing /chat/completions and /completions end points and try chat (#7350)
718201cc11afa0a1950e1f632390b2ac6c937a0d8f06369bf04336ab60e5c892dd77a3246df910151472024-05-22T04:28:32+08:00liuwei-gitllama : add phi3 128K model support (#7225)
71911474e756de3f56b760986e73086d40e787e52f8d8ee90222791afff2ab666ded4cb6195fd94cced2024-05-21T17:13:12+03:00Amirexamples: cache hf model when --model not provided (#7353)
720db10f01310beea8a1ef7798651b9d692fd1149d03bc10cb485dd7efa4da6c64e73ad0c9e2bfe08212024-05-20T16:36:55+03:00Radoslav Gerganovrpc : track allocated buffers (#7411)
721f030ec1f7a72aa825b2104823946551b9ec5dfc1e4e6f67be6a8a697f5f89a28c98934e53c99c3592024-05-19T17:19:53+02:000cc4mVulkan Embedding Fix (#7360)
7225ca49cbecda27ce0a7266658fc3b640bff3ed3861b01f06db0cff5f5f600bb754fc39fde565ed56a2024-05-19T16:46:13+02:00Johannes Gäßlerggml: implement quantized KV cache for FA (#7372)
723ab33f7a338593f6cf1ae98b10b6f8684f63bd72ce23b974f4cf9270d05062d446f406e3ff55d94512024-05-19T14:19:37+02:00slarencuda : clear error after buffer allocation failure (#7376)
724e23b974f4cf9270d05062d446f406e3ff55d9451854d365abab7194b5013a523f72da19860c3c5502024-05-19T20:51:03+10:00Brianlabeler.yml: Use settings from ggerganov/llama.cpp [no ci] (#7363)
7250f98acfac6cc561dc57586bfff778405e42b576bca57e0f35e33f714b9a6c2c4482b87bfe059c8192024-05-18T10:04:55+02:00Steffen Röckerllama : add support for larger Granite Code Models (20B, 34B) (#7324)
7260fc1e820a9900a3dd08ddd3c6abe6604c53b689b82ca83db3c8d45df559c03a4225b6eb34808a2db2024-05-17T18:54:52+02:00Johannes GäßlerCUDA: faster large batch FA without tensor cores (#7314)
72727b040691cbe45314147c2745e891a38e9c048d429c60d8cddcfd14fa8a6bf023a6c4eb8692c76ba2024-05-17T13:24:38+02:00fairydreamingllama : use n_embd_head_v when reshaping kqv (#7327)
7283b3963c55c8332e33533c44b2aa882b0e45f8292dda64fc17c97820ea9489eb0cc9ae8b8fdce49262024-05-15T15:29:07+03:00Radoslav Gerganovrpc : add command line arg for specifying backend memory
729dc020985b8755dd6aa93a2f002f43c3ede808cce344f9126cc0d15891fde9472fe40b8572628ad7d2024-05-15T14:44:49+01:00agray3Avoid unnecessarily disabling CUDA graphs (#7302)
730ea3b0590ee33d3573eb8ef76f88cc60f36d2a38d29499bb59383c2a8c5d557a90abb08b696cef7f62024-05-15T20:01:12+08:00dm4embedding : free the batch after execution (#7297)
731e8a7fd4fb06d82f663850c21fcf86c0fb98ad9b4a5e3fde8578d54b98d941344a4da150669af200d2024-05-14T19:09:30+03:00Georgi Gerganovmetal : support FA without mask + add asserts (#7278)
7324f0263633b40e94e8b69fd6e7e4395cfedfd5c121265c670fd8e41e1947352c96c5179adda97fb2c2024-05-14T10:11:24-04:00Steve Grubbserver: free sampling contexts on exit (#7264)
7335e31828d3e35c76ecfee665bc23771a4bec1d130541600201e6480f54ae09e58d16b154d4b4b331d2024-05-14T14:27:19+03:00Radoslav Gerganovggml : add RPC backend (#6829)
734541600201e6480f54ae09e58d16b154d4b4b331defc8f767c8c8c749a245dd96ad4e2f37c164b54c2024-05-14T09:33:42+02:00slarenllama : disable pipeline parallelism with nkvo (#7265)
73527f65d6267cf22a44c5ccefa7765d53a05bd1259ee52225067622babc277371511b8124884e1c7972024-05-14T14:20:47+09:00Ryueidocs: Fix typo and update description for --embeddings flag (#7026)
736614d3b914e1c3e02596f869649eb4f1d3b68614d30e70334f71b3bd115024affcf98cac3d79aaa952024-05-13T17:15:15+03:00Georgi Gerganovllama : less KV padding when FA is off (#7257)
737b1f8af1886e8187db6bb2a9b87cfc1c0f175f629e586ee42595500c53938e937b6b6ad5353ad76dc2024-05-13T12:56:47+10:00Brianconvert.py: Outfile default name change and additional metadata support (#4858)
7385a419926b0c4efab0531401aea91522aaea9fd07fae9d234b6606693704eca62fe4aefbb6c6abb452024-05-11T11:06:26-04:00compiladeconvert-hf : support bfloat16 conversion (#7158)
739988631335a20d06497f58be0b8ba13adb4323a22f99e1e456eaf69cc38c1982a2693ce41c0f897ef2024-05-11T04:13:02-04:00Steve Grubbserver : free llama_batch on exit (#7212)
7404e3880978f8b1bf546dd4e6f3b524d6b8739c49cf89fe2732c5709f6e86d5f4aee2e6d2a561f2eb22024-05-10T07:01:08-04:00Justine TunneyFix memory bug in grammar parser (#7194)
741f89fe2732c5709f6e86d5f4aee2e6d2a561f2eb2d11afd665241c1b3910ab5f040d0216403019d872024-05-10T15:51:58+05:30HanishKVCMain+: optionally allow special tokens from user in interactive mode (#7097)
742befddd0f15de6efb15d7e7f5b527dfb671f4196fd46dbc76f8770caec0175f1e57777173c70556a02024-05-09T20:39:54+02:000cc4mVulkan Bugfixes and Improvements (#7084)
74343248e559472556f368988575d9fba906b3eb139a743d76a01f23038b2c85af1e9048ee836767b442024-05-09T15:30:44+02:00jaime-m-pllama3 custom regex split (#6965)
744a743d76a01f23038b2c85af1e9048ee836767b44f31ec120bc36c6270e4948e6a065a7c4cfa0c4042024-05-09T14:32:02+02:00Johannes GäßlerCUDA: generalize FP16 fattn vec kernel (#7061)
745fd9f92b154850014146f61717cd292a59a5cee5a22842164bcae3251b81ad9e497a16ef66833cb9e2024-05-09T13:03:29+02:00Daniel Beveniusllama : update llama_timings.n_p_eval setting (#7160)
74647345248827f426038098d016ed11975021b491907cd41d0965829463eff73eda3348aedbfd3a4442024-05-09T17:34:37+08:00Albert Jinopencl : alignment size converted from bits to bytes (#7090)
747f98eb31c517c95960df1d0abc48002787f145f3bbc4bba364fb96d908f2698e908648df5e6f55e022024-05-08T18:16:38-04:00compiladeconvert-hf : save memory with lazy evaluation (#7075)
748bc4bba364fb96d908f2698e908648df5e6f55e02c12452c7aec8a02264afc00196a13caa591a13ac2024-05-08T21:55:49+01:00agray3Introduction of CUDA Graphs to LLama.cpp (#6766)
74926458af1d63c85195cd96cd1673051e332d06d3083330d8cd6491e53e1aca4c5dfc47e039b3c04ff2024-05-08T22:08:10+03:00Gilad Smetal : use `vm_allocate` instead of `posix_memalign` on macOS (#7078)
750c780e75305dba1f67691a8dc0e8bc8425838a45248b2f9c1fc71ab7df5432be2ed9fa7cdf5e8405e2024-05-07T21:26:43-03:00JeximoFurther tidy on Android instructions README.md (#7077)
75148b2f9c1fc71ab7df5432be2ed9fa7cdf5e8405eaf0a5b616359809ce886ea433acedebb39b129692024-05-08T01:24:16+01:00jukofyorkFixed save_imatrix to match old behaviour for MoE (#7099)
752858f6b73f6e57a62523d16a955d565254be889b4b3a995b416e13ae3123a117a743e11d0ede0ca4c2024-05-06T11:12:14-07:00William TambelliniAdd an option to build without CUDA VMM (#7067)
75303fb8a002df2e96104f9e06de9c78d2a8ed91e9292139b90af4841d7fd060b526bdd443b621770ff2024-05-04T12:06:40+03:00maor-psIf first token generated from the server is the stop word the server will crash (#7038)
754433def286e98751bf17db75dce53847d075c0be560325fa56f61c228464c9f065db3aa6a61f2156e2024-05-03T15:24:30+02:00Daniel Beveniusllama : rename ctx to user_data in progress_callback (#7045)
7558d608a81b7bd170f700648f8214e6f3279d4d7153ea0d36000e2314baba7e9fc6a97f08670a6f7e42024-05-02T04:27:41+09:00l3utterflymain : fix off by one error for context shift (#6921)
7563ea0d36000e2314baba7e9fc6a97f08670a6f7e41613ef8d8eb2479ba55c4d598e08c8f3f18a0fed2024-05-01T17:52:55+02:00Johannes GäßlerServer: add tests for batch size, different seeds (#6950)
7579c67c2773d4b706cf71d70ecf4aa180b62501960952d03dbead16e4dbdd1d3458486340673cc24652024-04-30T12:16:08+03:00Georgi Gerganovggml : add Flash Attention (#5021)
758f4ab2a41476600a98067a9474ea8f9e6db41bcfa3f167476b11efa7ab08f6cacdeb8cab0935c12492024-04-29T16:58:41+03:00Georgi Gerganovllama : fix BPE pre-tokenization (#6920)
7590c4d489e29e53589bf13a801fe7c94b7b546d8f6017e6999b5184234370b22a2f868e1be911e8d882024-04-26T20:06:33+02:00Pierrick Hymbertquantize: add imatrix and dataset metadata in GGUF (#6658)
760e2764cd7ca1112d9303eba9e81c9935ee67352ff4b1c3c98b442a4c84a788fff6323f6fa7e678a5b2024-04-26T17:07:42+02:00slarengguf : fix mismatch between alloc and free functions (#6929)
7617f5ff558eed0f732af8f25c2ab0645610bdec80c9e4e077ec50fde6049b128662c72d37a3c28e34b2024-04-26T12:15:30+02:00Pierrick Hymbertserver: stop generation at `n_ctx_train` if `n_predict` is not set (#6638)
76283b72cb086ce46a33dececc86bfe4648b6120aa8d4a9afc1009f0da88d04b2c5f672d81d5ae946752024-04-26T10:41:53+03:00Georgi GerganovMerge pull request from GHSA-p5mv-gjc5-mwqv
76328103f4832e301a9c84d44ff0df9d75d46ab6c76c0d1b3e03e27634ac2871761f5033cf9324d472d2024-04-24T11:08:36+02:00Johannes GäßlerServer: fix seed for multiple slots (#6835)
764e931888d5024de814ce7119a18d6a959bfff38218960fe86ae075c846c5df8848230d1904ba8877f2024-04-23T00:05:06+10:00Dave Airlieggml : fix calloc argument ordering. (#6820)
765b97bc3966e852adb626c90be64fd48282800f504b8109bc0139f15a5b321909f47510b89dca47ffc2024-04-21T13:50:41+02:00Pedro Cuencallama : support Llama 3 HF conversion (#6745)
7661958f7e06ca2d2e3ab5698cc67513ba359144d8e04fbc5f23e9708136ff03ebe194c7a7e965a7ca42024-04-14T15:24:15-04:00David Renshawllama : add missing kv clear in llama_beam_search (#6664)
767de17e3f7455dc7fd298cc61d86798533b9ca7a29b5e7285baffb0da8a6619567b52d8e67de41291d2024-04-14T10:42:29+08:00Neo Zhang Jianyufix memcpy() crash, add missed cmd in guide, fix softmax (#6622)
76824ee66ed0d908d156bd0d1747b63a636a495cd7a91c736015b66ba1d0b82cbae6313b6d5eaa61b682024-04-12T13:49:21+02:00Pierrick Hymbertserver : coherent log output for KV cache full (#6637)
76991c736015b66ba1d0b82cbae6313b6d5eaa61b685c4d767ac028c0f9c31cba3fceaf765c6097abfc2024-04-12T18:45:06+08:00jiezllama : add gguf_remove_key + remove split meta during quantize (#6591)
770dee7f8d6928cc680cc969f7d93f98c3e24dcad4181da18e71ccfc196d4516fbea5dc3a6a1f92dccb2024-04-12T16:28:12+08:00MasterYi1024Correct free memory and total memory. (#6630)
771b804b1ef77351d2a11be945462c6c251710476cb8228b66dbc16290c5cbd70e80ab47c068e2569d82024-04-11T14:51:07+02:00Pierrick Hymberteval-callback: Example how to use eval callback for debugging (#6576)
772e3c337d87ca650972105a51c6ce302dd236c07adbeea6e1b16e783a0886e78dec01002a8c00db24d2024-04-08T06:02:30-07:00Rick Gllama : support negative ith in llama_get_ API (#6519)
773beea6e1b16e783a0886e78dec01002a8c00db24d87fb5b4234d4b9c56ac94cf7aa229c8fd7defdb02024-04-08T20:43:30+08:00Jan Boonllama : save and restore kv cache for single seq id (#6341)
774d752327c3338d5b9634121d651c0105f2c933f9b855f54402e866ed19d8d675b56a81c844c64b3252024-04-08T00:48:29-07:00FiratAdding KodiBot to UI list (#6535)
77587e21bbacd830437ab653cf03b6f26d45c15395d1b496a745c315022df2d919374052e6004ced8d32024-04-06T01:34:53+07:00Ting Sunbench : make n_batch and n_ubatch configurable in Batched bench (#6500)
7764399f13fb9462cd06f3f154d0aee738425000fea1a43c7254ed5de91d09274b853db843c518f1b642024-04-04T09:34:58+03:00Georgi Gerganovserver : remove obsolete --memory-f32 option
7771a43c7254ed5de91d09274b853db843c518f1b6472d73af65132792a52433952ca4729b01c36cde22024-04-04T01:33:48-05:00Xiao-Yong Jinserver : add option to disable KV offload (#6468)
77808a0c0206075556e82aca0feafad530dcc5f142652604860f93063ef98863921da697576af1c76652024-04-03T15:07:05+02:00slarenggml : mul_mat_id use the same tensor for all the experts (#6387)
77952604860f93063ef98863921da697576af1c7665f87f7b898651339fe173ddf016ca826163e899d82024-04-03T10:34:40+08:00Meng, Hengyu[SYCL] Disable iqx on windows as WA (#6435)
780ba0c7c70ab5b15f1f2be7fb0dfbe0366dda30d6cd48ccf3ad4fea5b9ede209c7f40be65371987bfe2024-03-29T17:29:21+01:000cc4mVulkan k-quant mmq and ggml-backend offload functionality (#6155)
781069574775cea67d8a977904e4d534aff47a671f4cfde806eb95de06d84162bdee593dad33a1d26932024-03-29T21:37:03+08:00hxer7963[Model] Add support for xverse (#6301)
782057400a3fd457f4f214684eeb171444663b47a23b75c38166cf0c66793a32d5c3d6cb69929dd083d2024-03-29T08:23:22+01:00Daniel Beveniusllama : remove redundant reshape in build_kv_store (#6369)
783cfc4d75df6399b36153ef739f2c1abee4c114bb86902cb7f2e3479f364ee177118200fb7e4e9fc922024-03-28T16:51:06+08:00Ting Sundoc: fix outdated default value of batch size (#6336)
784e82f9e2b833d88cd2b30123ef57346c2cb8abd99cbc83436197cde617cad696e665879c20df77daa2024-03-27T08:16:40ZAidanBeltonS[SYCL] Fix batched impl for NVidia GPU (#6164)
785557410b8f06380560155ac7fcb8316d71ddc983755c1b2a3bbd470e9e2a3a0618b92cf64a885f8062024-03-26T10:46:41-04:00compiladellama : greatly reduce output buffer memory usage (#6122)
786deb7240100da99555b9ab9dc635021e591fceaf53d032ece8e6973441273601ca2981130608e287d2024-03-26T18:11:46+09:00Minsoo Cheongembedding : adjust `n_ubatch` value (#6296)
787a32b77c4b2c1808654d0b952f26c37d73d2e746ba0e584defd8c16e7a51ab895f595df0448d710d02024-03-24T14:45:56-07:00Rick GFix heap corruption from wmode out-of-bound writes on windows (#6272)
788586e7bc561be88e929a9afca7e67d8ead00c53bdddf65685105a39a57b1e7f80c3aa502a6313af242024-03-24T17:54:07+09:00Minsoo Cheongsampling : deduplicated code for probability distribution access (#6240)
7891997577d5e121568ae39f538021733ccd4278c23476b0251b27fb64c575507024a671e639d6755942024-03-23T18:00:38+01:00Pierrick Hymbertserver: docs: `--threads` and `--threads`, `--ubatch-size`, `--log-disable` (#6254)
79050ccaf5eacb50a2ca378a4ef0dc7aeb45fead65256a00f0a2f48a85376f48b5ce77699df781631ae2024-03-23T01:24:36+01:00Johannes Gäßlerlookup: complement data from context with general text statistics (#5479)
791dba1af612926cbd4ebe2d876277af1e3305177e0ee804f6223777019cf921e0d99cc24669313ab982024-03-22T19:00:01+01:00Pierrick Hymbertllama_model_loader: support multiple split/shard GGUFs (#6187)
79268e210b3543e0cc71268bee0920441747679ee13b3e94f26bab8e3b5338b5902e5af5bb01894cb4a2024-03-22T13:08:28+02:00Georgi Gerganovserver : enable continuous batching by default (#6231)
793b3e94f26bab8e3b5338b5902e5af5bb01894cb4ab2075fd6a578f5685060df4baa90ae9e48e98c702024-03-22T11:35:53+02:00Georgi Gerganovmetal : proper assert for mat-mat memory alignment (#6225)
79495d576b48ebf582b112d1c9cf4eed7142fa4e46459c17f02de8fdf7b084d6100b875b7e2bc07a83b2024-03-22T09:36:03+02:00Georgi Gerganovmetal : pad n_ctx by 32 (#6177)
79576aa30a26353f597e4fbe3cf776772ae812af89ac5b8595e3f4f4ed319ef71c9c9d868d1b7a276262024-03-21T08:27:57+01:00KawrakowAdd ability to use Q5_0, Q5_1, and IQ4_NL for quantized K cache (#6183)
796d0d5de42e5a65865b5fddb6f5c785083539b74c3b80cf3b2d1dee0ad325f7a794fecc66befce73362024-03-19T12:05:44+01:00Pierrick Hymbertgguf-split: split and merge gguf per batch of tensors (#6135)
797104f5e0fc156d48476258295457cafeec2a2af105e1b7f94a03e0b3b8e4578625bbdadc7bbd2b93c2024-03-18T16:40:22+01:00Felixclip : fix memory leak (#6138)
7982bf8d0f7c4cc1235755ad06961ca761e458c5e55496bc79bc2b79bfd6124b8687a8dbd6a646e9b062024-03-18T11:03:04+01:00slarenbackend : offload large batches to GPU (#6083)
79912247f4c69a173b9482f68aaa174ec37fc909ccf4e9a7f7f7fb6acbddd1462909c8d696e38edbfcc2024-03-15T16:41:22-04:00Andrew Canisllama : add Command-R support (#6033)
8006e0438da3cc95b89cdbf55f45fa4e324d9076792727107707a73b3dc8a497cf9fc9405722c16dd2b2024-03-14T14:29:32-04:00Steve Grubbgguf : fix resource leaks (#6061)
801f30ea47a87ed4446ad55adb265755dc9102956a2d8fd0ccf6ac8b07791ffd1575eed436930854ae32024-03-13T18:54:21+01:00slarenllama : add pipeline parallelism support (#6017)
802d894f352bf433157232dc8dc54eacd50014e898e098dbaab449f5309a54871ba7e5acef72ae696de2024-03-09T19:55:54+01:00slarenperplexity : support using multiple sequences to allow larger batch sizes (#5946)
803fd72d2d2a5e79d61ccef6af3d15f16e5e5cbc352c2101a2e909ac7c08976d414e64e96c90ee5fa9e2024-03-09T10:30:04+01:00Pierrick Hymbertserver: tests: add truncated prompt tests, better kv cache size (#5933)
804c2101a2e909ac7c08976d414e64e96c90ee5fa9e515f7d0d4fce41c752fc253acf30707c3be2531e2024-03-08T17:31:00-05:00compiladellama : support Mamba Selective State Space Models (#5328)
805af37fd8b30e37ccbffdd82e6f48559e2fb7ce7dd581ed5c4fe3a8909aaa8313633ac443f471ba7552024-03-08T12:40:02+02:00Georgi Gerganovserver : fix EOS token detection with disabled cache (#5938)
8062002bc96bf2cbf5ab981a17d7e994d817c9801f5ceca1aef0738b57951cd12c603c3477e75312dec2024-03-07T11:41:53+02:00Georgi Gerganovserver : refactor (#5882)
80761d1c88e155515dd03940913a5707ea84a8b119b21b08674331e1ea1b599f17c5ca91f0ed173be312024-03-05T13:33:42+01:000cc4mVulkan Improvements (#5835)
80821b08674331e1ea1b599f17c5ca91f0ed173be316a87ac3a52668e117d97bcea07b529c93188b3032024-03-05T16:08:35+08:00Neo Zhang Jianyu[SYCL] fix mul_mat fault in CI/unit-test (#5862)
80929ae62d2ae163e2b68aa0ad3bf2ab4636de0c957e0843afe1b37890b631bc7d3d2da2ed36c862b912024-03-04T22:31:20+02:00Georgi Gerganovllama : fix embeddings (#5796)
8106d341ab6c53cd51f2921d986d0090cc8b049b39a4ffcdce2ff877ebb683cd217ea38faf20faa5ffe2024-03-05T03:24:00+09:00Minsoo Cheongspeculative : implement stochastic speculative sampling (#5625)
8117d43c585dc174bb586775c22c15e5db9242b5b4b82f3e668adafba647de703f835991e91a96b5ac42024-03-03T20:23:52+08:00leejetadd some new ops, fix some operators and add batch operations to certain operators. (ggml/747)
812de9692a7d2db66e29e5cb373c6551acc49145ccde6029348e86c3810d4435faee54ba822cb43e2ef2024-03-03T03:41:55-05:00compiladellama : fix llama_copy_state_data with fragmented KV cache (#5840)
8139731134296af3a6839cd682e51d9c2109a871de54a6e2d6142ab815c964924896891e9ab3e0506322024-03-02T22:00:14+01:00Pierrick Hymbertserver: tests: passkey challenge / self-extend with context shift demo (#5832)
814bbde6eb2561153aabbdfac5001c690fe00cad639ef2cd694c4155fbf25bae61c5178c47eb3676dba2024-03-02T17:00:51+02:00Kawrakowggml : IQ3_S improvements (#5829)
81538d152160898b0173ffe4dc7df5daadcbd2eceb0052051d8ae4639a1c3c61e7da3237bcc572469d42024-03-01T07:36:47ZAidanBeltonS[SYCL] Use batched mul_mat pathway (#5591)
81647bb7b48c7cec9d8f57d56812ce811ec130b89a3c4d7f8178608440506e5489bae0109e4ca12e44a2024-02-26T15:36:38+01:00Johannes GäßlerCUDA: fix DEBUG_CUDA_MALLOC (#5729)
817bf08e00643fd529f748f0a858fd79f3061e3fa18f7625019c51ca437a5840576d92362cfa710e4a22024-02-25T22:12:24+02:00Georgi Gerganovllama : refactor k-shift implementation + KV defragmentation (#5691)
818f7625019c51ca437a5840576d92362cfa710e4a2abbabc5e51d0d4656b438aec10b7fae9479ef37d2024-02-25T13:43:50-05:00compiladeserver : fix crash when system prompt is bigger than batch size (#5714)
8194c4cb30736582cacb1a164a9d4bc8e17b1014be7525213d2f5da1eaf4b922b6b792cb52b2c6133682024-02-24T16:23:52+02:00KawrakowIQ3_S: a much better alternative to Q3_K (#5676)
820525213d2f5da1eaf4b922b6b792cb52b2c613368fd43d66f46ee3b5345fb8a74a252d86ccd34a4092024-02-24T12:28:55+01:00Pierrick Hymbertserver: init functional tests (#5566)
821fd43d66f46ee3b5345fb8a74a252d86ccd34a40954fbcd2ce6c48c9e22eca6fbf9e53fb68c3e72ea2024-02-23T19:31:54ZAlpinDaleserver : add KV cache quantization options (#5684)
8224cb4d8b22d4fda971621a68c570ce84d66897c373a03541cedea474fa9d41214484cc3fbcf468a9e2024-02-22T16:32:09ZSomeoneworkflows: nix: hardcode cachix ids, build unconditionally (#5663)
8237fe4678b0244ba7b03eae66ebeaa947e2770bb1aba2135ccae7462470b3865c6e41d2e1d734eac052024-02-21T22:52:39+01:00slarenllama : fix session save/load with quantized KV (#5649)
82489febfed9322c8849520dc63c93ee4f5fd72556e5022cf242d689e15defd133f96c4345ad30c5d192024-02-21T10:33:54-05:00Jared Van Bortelexamples : do not assume BOS when shifting context (#5622)
825eccd7a26ddbff19e4b8805648f5f14c501957859c14f72db9c62d71d35eb1c141745c0bd0cb27b492024-02-21T16:17:10+02:00Georgi Gerganovsync : ggml (#5633)
826a14679cc30c785e75d38028bae6ec39c6209ddef6560bed3f066c876682464762cad90f1e28e3f1b2024-02-21T11:39:52+02:00KawrakowIQ4_NL: 4-bit non-linear quants with blocks of 32 (#5590)
827a3145bdc305422973e25f0b066da6f469ed5dc45890559ab28e354052e16e770155ad007fd0856e82024-02-19T14:53:48+02:00Georgi Gerganovggml-alloc : apply ggml/731
8284480542b2271ba1438f0daff8e5f3a74b1dc860911b12de39bd787c0494da0cd405958fdfedc29c42024-02-19T03:25:38-05:00NawafAlansaribaby-llama : allocate graphs in ggml_context (#5573)
82911b12de39bd787c0494da0cd405958fdfedc29c43a9cb4ca6408c29423373dd6cd7aa78a58286c002024-02-19T09:23:37+01:00Xuan Son Nguyenllama : add llama_chat_apply_template() (#5538)
8304cb072769804c77ab466bc8351c76ede9d5ba49d65085c713e14f78cdda6abc275b1a5d8c2b8ca152024-02-16T12:43:23ZHerman Semenovllava : removed excess free(NULL) operation (#5531)
8310d4177126b0556e202efb85bf3f768be810764007930a8a6e89a04c77c51e3ae5dc1cd8e845b6b8f2024-02-15T09:01:57+01:00Elbiosllava : fix memory management bug (#5491)
832704359e29985a06a389337a2617b7f3fa8eff908594fca3fefe27b8e95cfb1656eb0e160ad15a7932024-02-15T17:11:15+11:00Neuman Vongvulkan: Find optimal memory type but with fallback (#5381)
833aa2341298924ac89778252015efcb792f2df1e20f5ca054855dea83f424003162f26de376e5643f62024-02-14T08:38:35+01:00Johnllava : support v1.6 (#5267)
83403bf161eb6dea6400ee49c6dc6b69bdcfa9fd3fcad014bba97ef6ef6c3e2f78b2fc463e91ae945792024-02-13T06:06:58-06:00Douglas Hanleyllama : support batched embeddings (#5466)
8353b169441dfe8e420f88d1592708cc2a871daadb93bdc4cd0f595a6096cca4a64aa75ffa8a35034652024-02-12T09:16:06+02:00Georgi Gerganovsync : ggml (#5452)
836f026f8120f97090d34a52b3dc023c82e0ede3f7dcd9aea63b577a83def84dbd6dcd90a6fa02af7452024-02-10T02:53:28-08:00Ian Bullmetal : use autoreleasepool to avoid memory leaks (#5437)
83744fbe34360dd760f9e68b4271f21533436397f848e6a9d2de0096af7120606c74ee2f26684e87b412024-02-09T06:52:33+01:000cc4mFix Vulkan crash on APUs with very little device memory (#5424)
838ee1628bdfea8b0079fed0140ac2f00ef1b465b57ed0bf32290ee5b30ffad5becd99cbecef74aedd72024-02-07T07:54:50+01:000cc4mBasic Vulkan Multi-GPU implementation (#5321)
83917c97fb0620448b37516a3f53fea6c482b0a30a4b08f22c882a1443e6b97081f3ce718a4d1a741f82024-02-06T18:43:06+01:00Johannes GäßlerCUDA: mul_mat_vec_q max. batch size 8 -> 4 (#5370)
8402c516611f1d0f1e5e9754f8ea1cf97cb1b17bf2c8a79c591de9b7ff3242a94f68b7fb5a17ed8c2be2024-02-06T14:44:06+01:00Johannes GäßlerCUDA: mul_mat_vec_q for batch sizes > 1 (#5351)
841098f6d737b65134cf220d12b9b706e8cfc5e461078b00dda6c0d62c34f5371d47718defff6ed2b222024-02-05T19:33:00+01:00Johannes Gäßlermake: Use ccache for faster compilation (#5318)
8421ec3332ade60aeb1494ace2211cf1a966db6d7706a66c5071a74a96c4f52cf1015a092acd18c37142024-02-03T06:22:06-05:00Jared Van BortelYaRN : store rope scaling type as int32_t in memory (#5285)
843e805f0fa9951081ce0a86378a7aa52b6f636b82daf3ba5d94627d337e32a95129e31a3064c459f6b2024-02-02T15:54:14+08:00Meng, Hengyu[SYCL] get MAX_MEM_ALLOC from device property (#5270)
844af3ba5d94627d337e32a95129e31a3064c459f6be1e721094d8169636d55f68efe37f222cd3f06772024-02-02T15:53:27+08:00Neo Zhang Jianyu[SYCL] update guide of SYCL backend (#5254)
845e1e721094d8169636d55f68efe37f222cd3f0677128dcbd3c9c4b12f42b560a4430427d7b28286282024-02-01T23:20:13-08:00Ian Bullllama : fix memory leak in llama_batch_free (#5252)
8464d0924a8902010d31bd737b6f1f594943d120d0f8ca511cadee2c67f0bd8c7034a2513778ee9a1b72024-02-01T19:25:24+01:000cc4mVulkan Phi Fix for AMD Proprietary Drivers (#5260)
847ce32060198b7e2d6a13a9b8e1e1369e3c295ae2a1cfb5372cf5707c8ec6dde7c874f4a44a6c4c9152024-02-01T17:19:51+08:00Guotengllama : support InternLM2 (#5184)
84815606309a05ccf7fadbaad5538cb7c32acb1e06bb2b9f025e7821e78bd501d75d01838c26de07a572024-01-31T21:10:15+08:00JidongZhang-THUllava : add MobileVLM support (#5132)
849f8e9140cb46eebaa867e1184a9946e4840eec772d62520eb2cc1d7168a30edec6110e1daefbd959f2024-01-31T11:44:19+01:000cc4mVulkan Fixes (#5223)
850e6f291d15844398f8326940fe5ad7f2e02b5aa564003be0e5feef320f3707786f22722b73cff93562024-01-30T20:17:30+02:00Georgi Gerganovserver : fix context shift (#5195)
851a4b07c057a553b1ac253051efc3f040351e2eae1549a1e6cd5b39fe0dc3d4ea5515c65f17797a31e2024-01-29T14:00:10+02:00Georgi Gerganovgguf : add input validation, prevent integer overflows (ggml/709)
8522256f36b79a932a478d4dcdf02c1e5a60056e5f37359016c7c0f65dc30cf79791212b06f158664502024-01-30T13:59:30+01:000cc4mVulkan Windows APU Memory Handling (#5199)
853813416991ab0d1caa0d12f93ac4e8a24a2add0a35589921ef84a4fb1c6d1c9c34d626a5a83033db62024-01-30T10:18:02+01:00divinity76main : allow empty --prompt-cache file (#5176)
854ceebbb5b21b971941b2533210b74bf359981006c6daa69ee81851ab26ca8aefca1a4202941fc02622024-01-29T22:19:29ZPaul Tsochantarisggml alloc: Fix for null dereference on alloc failure (#5200)
855fbe7dfa53caff0a7e830b676e6e949917a5c71b4172ac82629815d038702b049070f4c8c02662da52024-01-29T09:05:13+01:00slarenggml : add max buffer sizes to opencl and metal backends (#5181)
856d2f650cb5b04ee2726663e79b47da5efe196ce0035dec26cc25a9ff7d8c3ed52326b94f772b911ce2024-01-28T19:50:16ZPaul Tsochantarismetal : free metal objects (#5161)
8572307523d322af762ae06648b29ec5a9eb1c730320f648573dde61c510560f68244f70ece7e60d8c12024-01-28T18:03:59+01:000cc4mggml : add Vulkan backend (#2059)
8580f648573dde61c510560f68244f70ece7e60d8c1b764b8f1d079ba44d912801ce6d29bd0d94d51cf2024-01-28T21:26:23+05:30Abhilash Majumderggml : add unified SYCL backend for Intel GPUs (#2690)
85935a2ee914308c85ab5cb576467381443ad23f0acec903c034131848da9222536ff18da07ec0882a02024-01-27T15:25:55+01:00Michael KlimenkoRemove unused data and add fixes (#5154)
86062fead3ea0a30c8d424f4a8373fa14165c7c707f15b4538ff29b280a395a1406d711497d8eaa25642024-01-26T18:59:43+01:00slarencuda : fix tensor size calculation for non-split buffer (#5145)
86115b4538ff29b280a395a1406d711497d8eaa25647032f4f6349c17a8352f9f93f7d2122f45469e592024-01-26T18:18:26+01:00slarenggml-alloc : add 10% margin to the buffer sizes (#5149)
8626dd3c28c9cd1ef74b49d79f47d668759346a3c6c38b431de232d1b736b5af19b8c7d72f7075a70bc2024-01-26T12:16:07ZPaul Tsochantarismetal : remove unused `n_buffers` and `buffers` (#5129)
8631387ea21178f9f154944013d4dd9764b54c69deb26d607608d794efa56df3bdb6043a2f94c1d632c2024-01-24T12:48:14+01:00slarenllama : pre-allocate input tensors in a separate buffer (#5100)
864011e8ec577fd135cbc02993d3ea9840c516d6a1c6f9939d119b2d004c264952eb510bd106455531e2024-01-22T23:42:41+01:00slarenllama : fix not enough space in buffer with Qwen (#5086)
865726c0fa9a2da976e9c5d5c51e185d9dd453fc9e5942c0107a7301434c0a5e7da46bc4cf2393aa5562024-01-21T08:01:20+02:00KawrakowSlightly faster imatrix (#5050)
8666df465a91d402370bcba6676b19fad85b06ce7e077bc1bbd05f0c31cb45773eb5eb59b9ff2b07e1b2024-01-20T16:05:49+01:00slarenllama : run all KQV ops on the CPU with no KV offload (#5049)
86777bc1bbd05f0c31cb45773eb5eb59b9ff2b07e1b48e2b1337257bb77573bf76cfffcff3a5efa87042024-01-20T08:11:31ZHerman Semenovcmake : add support for ccache (#5002)
868381ee195721d8e747ee31a60c0751822b3072f02a5cacb22b2114fd9adf61c00cbb237384d86bced2024-01-19T13:20:50-05:00Uzo Nwekefinetune : fix ggml_allocr lifetimes (tmp workaround) (#5033)
8698b20858e5e9c44b99b4b31ae9c40b8f20d01d94f57e2a7a52a819883f40dada8a2edc24ecf48186b2024-01-19T10:45:06+02:00Georgi Gerganovperplexity : faster Winogrande via batching (#5024)
870ad19812cda4062c9f154ef16315df41fbe6a770a682986a08eb5cb04865d2e713449f17304d266d82024-01-18T15:33:01+02:00Georgi Gerganovperplexity : faster HellaSwag via batching (#5017)
871682986a08eb5cb04865d2e713449f17304d266d8dcad445d0c83ad49bca1b58cf9c139cfcebee5d42024-01-18T13:46:27+02:00KawrakowAdd Winogrande evaluation (#5015)
8721e605f4102c7ea8dc0dff82f5eaa6a71973d549f6b6916b215251e09bd57cdbf870dc8a73345edc22024-01-18T08:47:24ZPaul Tsochantarismetal : fix memory leak, dangling pointer and unused autorel (#5007)
87344a1a4a41a4c0b03afaa7d9e06bcbc7cf95aa1e6c918fe8dca8fa1c4602427e0a4b88e20046f6c342024-01-17T18:39:41+02:00Georgi Gerganovbackend : add eval callback (#4935)
874959ef0c0df725c013c7f712eaa7790b8e38a8e20c37b3474e61d609d43cccc3bde5d559e80e4f5d12024-01-16T19:34:54+02:00Georgi Gerganovperplexity : fix kv cache handling for hellaswag (#4981)
875158f8c9e21302114bac3c646f80ea85b52ffa0bd862f5e41ab1fdf12d6f59455aad3f5dd8258f8052024-01-16T17:05:19ZPaul Tsochantarismetal : localized logic in `ggml_metal_graph_compute` (#4924)
8764483396751c79dea540808b9cb9238245d06da2bd9aa4ffa6e0296d42f1f676dd85de97c8491eb732024-01-15T14:06:52+01:00David Friehsllama : apply classifier-free guidance to logits directly (#4951)
877c71d608ce7a1584bf5072f197919dd24f3a6163f4be5ef556de830c5c4f6e45c05ef4427823fe6072024-01-13T21:41:37+01:00Johannes Gäßlerggml: cache sin/cos for RoPE (#4908)
878df845cc982e7e2ea7b9900e29d55b15338faa78d6b48ed089377330cdb362970a51c1c89b6d857a82024-01-13T17:29:43+01:00David Friehsllama : minimize size used for state save/load (#4820)
879b38b5e93ae31019e87f692b69d27124eae6aac027dc78764e2ff86512e6e31cb0fcb8087df4b47082024-01-13T18:03:45+02:00Georgi Gerganovmetal : refactor kernel loading code (#4794)
880e7e4df031b9e29d4b55a4e0b0295187f6b213db1584d674be622fbf1578694ada6e62eebedbfd3772024-01-12T20:07:38+01:00slarenllama : ggml-backend integration (#4766)
881f34432ca1e0b288129390c1db8296a82aaf1e6327a9f75c38b5e62fe27b8a5a3ed823b4a3714024b2024-01-05T16:00:00+01:00Erik Scholzfix : cuda order of synchronization when setting a buffer (ggml/679)
8828f900abfc09851e281bc9027e0ab2f16bf079b291fc2f265ff9377a37fd2c61eae9cd813a3491bea2024-01-09T08:58:55+01:00Johannes GäßlerCUDA: faster softmax via shared memory + fp16 math (#4742)
883dd5ae06405c5565b99889bdb3f168f4351252cfb668b31fc7d86245435ad6574e0e1126e734049e22024-01-08T16:02:32+01:00KawrakowSOTA 2-bit quants (#4773)
88452531fdff88764282c1b233174721aab8347252db0034d93ce2949ce7d9c098ca02e56f66cd484e22024-01-08T11:18:32+02:00Georgi Gerganovmain : add self-extend support (#4815)
885b0034d93ce2949ce7d9c098ca02e56f66cd484e2b7e7982953f80a656e03feb5cfb17a17a173eb262024-01-08T11:14:04+02:00Georgi Gerganovexamples : add passkey test (#3856)
886226460cc0d5b185bc6685fb76f418fd9418d7addd5a410e8556191672465f7ff58682ea2474038b02024-01-07T17:59:01+01:00slarenllama-bench : add no-kv-offload parameter (#4812)
88783e633c27efdf0eb0ba54249e784b0ea760b100732866c5edde402f42ff4233bb89dcfcede34fd222024-01-02T03:51:28-08:00postmastersllama : differentiate the KV dims in the attention (#4657)
8885d7002d4372ebf107cfaf46fcd90df27b204f33026f3071d714f0b27ad7f021a46a66a10854802582024-01-02T04:38:15-06:00minarchistserver : add --override-kv parameter (#4710)
889edd1ab7bc34c10a780ee7f9a4499f7689cdad36d198ed7ebfc89b8f2b35a8b1655d57bfb57530c1a2023-12-31T17:42:22ZSomeone Sergeflake.lock: update
890198ed7ebfc89b8f2b35a8b1655d57bfb57530c1ad8361747317c5cb2e00e7fb3b59ff4dce5a176a52023-12-30T18:25:25ZSomeone Sergeflake.nix: suggest the binary caches
8910235b9b571f3cc7d2b8836409a5404b41ce1379cce18d727a47f2473ca863a6f78bf3ad480008f722023-12-29T18:53:34+02:00Georgi Gerganovclip : use ggml_backend_buffer_is_host (#4205)
892441f51dca004debf8b275f1bdc08e0f1af7fd8f838b3de4658292582a8941a2be5c77b40ce6ac0f22023-12-29T19:23:27+09:00Tamotsu Takahashici : build with CLBlast + ggml-opencl use GGML_API (whisper/1576)
893879b690a9e1eb1ab0a29b58236fc76978fb4d902b47879b0dda43f2d26415e88b6840295817e552a2023-12-27T15:16:55+01:00Daniel Beveniusfinetune : fix output formatting in print_params (#4653)
894dc68f0054cd279cddddb0cae0c9ef4f9cbaa512ade8e496437c59e7d1cc84109e3e49a3478aee25a2023-12-26T21:23:59+01:00slarencuda : fix vmm pool with multi GPU (#4620)
8955bf3953d7e9831ea22b0bc017ce97409b801ccf1708e179e8562c2604240df95a2241dea17fd808b2023-12-24T14:34:22+01:00slarencuda : improve cuda pool efficiency using virtual memory (#4606)
896708e179e8562c2604240df95a2241dea17fd808b925e5584a058afb612f9c20bc472c130f5d0f8912023-12-23T16:10:51+01:00slarenfallback to CPU buffer if host buffer alloc fails (#4610)
8977082d24cec35e9ce9147535a2224dfc67ee0a78cba661751322a7c201fd3bef71af077c5aebfaa2a2023-12-22T17:05:56+01:00LeonEricssonlookup : add prompt lookup decoding example (#4484)
898ba661751322a7c201fd3bef71af077c5aebfaa2aa55876955b1a83464171de8d578d3ab062a7b62d2023-12-22T17:53:43+02:00Georgi Gerganovsync : ggml (fix im2col) (#4591)
8994a5f9d629ecfd0a53afdddbaf54a4fa02d9a9ce9d232aca5a73b290e218a2e48b91023d5e994203f2023-12-21T22:36:26+02:00Samuel Maynardci : add `jlumbroso/free-disk-space` to docker workflow (#4150)
900d232aca5a73b290e218a2e48b91023d5e994203f31f27758faf4a4bd08101a57c7ec3a473f771f862023-12-21T21:07:46+01:00slarenllama : initial ggml-backend integration (#4520)
90131f27758faf4a4bd08101a57c7ec3a473f771f8656fa50819f7a3ca2128f63b81c17c08a4454479e2023-12-21T11:57:48-08:00Marcus Dunnllama : allow getting n_batch from llama_context in c api (#4540)
9020f630fbc924aaabeea6eaf466bb4b47d13015c3e562cf222b5129e40b312877e928eac3a02e4ec332023-12-21T13:45:32-06:00Erik Garrisoncuda : ROCm AMD Unified Memory Architecture (UMA) handling (#4449)
9039154494808dc865475c59022c29060b4947a803bc083718c895b7c8c7fb2a4660643fb78d0c64dfd2023-12-21T18:42:59+01:00Johannes GäßlerCUDA: mul_mat_id always on GPU for batches >= 32 (#4553)
9041d7a1912cea2227f9a1a449758ed622c560542f9799fc2268989482054944c902874cca76337580f2023-12-21T01:59:27-08:00LoganDarkFix access violation in ggml_cuda_free_data if tensor->extra is NULL (#4554)
905799fc2268989482054944c902874cca76337580f328b83de23b33240e28f4e74900d1d06726f5eb12023-12-20T15:41:22+01:00Johannes GäßlerCUDA: Faster Mixtral prompt processing (#4538)
906b9e74f9bca5fdf7d0a22ed25e7a9626335fdfa483c04bf6da89eaf4c7d317e0518f0687dfcbf2de72023-12-18T17:27:47ZEbey Abrahamllama : add phi-2 + fix NeoX rope + ggml_mul_mat_set_prec (#4490)
90745668633fdb522a925c3dafc1ecf426f539efb270ffc92d2d23a789625f018840469af045be1e3c02023-12-17T16:05:56+01:00slarenfinetune : keep allocs alive until all allocations are done (#4486)
908c6c4fc081c1df1c60a9bfe3e6a3fd086f1a29ec78a5be3bd5885d79ad84aadf32bb8c1a67bd43c192023-12-16T18:58:46+01:00slarenlora : add support for non-llama models (#3333)
90920a68a7030ee06e8eb7eb8e24ae4ac52dc17803f55e87c3749cb4985c3b316984d40e00e4df4a5d02023-12-14T20:13:33+08:00LostRuinsggml : add ggml_row_size() (fixes llama out of space) (#4461)
910873637afc7924f435ac44c067630a28e82eefa7b0353a1840134b24b07ab61fd4490192f28c4db6b2023-12-14T17:09:34+09:00wonjun Jangconvert : support loading vocab from fast tokenizer config (#3633)
911799a1cb13b0b1b560ab0ceff485caed68faa8f1ffecac45658a99eddc4d6e36ba0310ca8f87a77f02023-12-13T13:04:25+01:00slarenllama : add Mixtral support (#4406)
9126391817cd19a4507c6c941a1fd08756268662b2dd9d4cfef64ea416dd66632173787d03ffb180cc72023-12-12T04:25:57-05:00crasmllama : document logits_all deprecation (#4418)
913bcc0eb4591bec5ec02fad3f2bdcb1b265052ea5681bc9214a389362010f7a57f4cbc30e5f83a2d282023-12-07T13:03:17+02:00Georgi Gerganovllama : per-layer KV cache + quantum K cache (#4309)
91481bc9214a389362010f7a57f4cbc30e5f83a2d2805cd6e5036d72d0930de4d8f6be7bce09e8dda242023-12-07T02:25:22-08:00Hongyu Ouyangtrain : fix #4227 (double free in examples/train-text-from-scratch/train-text-from-scratch.cpp) (#4351)
91505cd6e5036d72d0930de4d8f6be7bce09e8dda24caa9249217c5fd524b900add5ddcbeaa20cbcb122023-12-06T20:21:59+02:00Georgi Gerganovserver : recognize cache_prompt parameter in OAI API (#4347)
9165aa365d88fdb8fdd430ef3fc141c7a5fd37c350252c8bc3cf312e1caf02d37bfb9d9d865cbe335942023-12-05T10:19:18-07:00Kerfufflellama : allow overriding GGUF metadata when loading model (#4092)
91723b5e12eb5a76489b4c3ee22213a081da68b1809d208995c6da66f252d4054c1c5a90eb8ccb7a2f72023-12-04T17:04:21+01:00Daniel Beveniussimple : update error message for KV cache check (#4324)
918d7b800b8bc490a221acbd83c575206a907f2f6e25a7d3125e7c24f223659b7f0b7aa7736986e92c02023-12-03T10:58:16+02:00Georgi Gerganovllama : pad KV cache size (#4280)
91903562f3a86d6706eea9f4fc09b532946c191b34e37c746d687d877bc11803e96b4dc5f378b83c0a02023-12-02T02:17:06+08:00CausalLMllama : support attention bias on LLaMA architecture (#4283)
920ef47ec18da469423c276b683dd9b5741cee7023e1d144112c0fbbb4ecc07dbcf4f05a380148bd6de2023-12-01T10:51:24+02:00Georgi Gerganovggml : add ggml_soft_max_ext (#4256)
921bde629bb53b85886ee0fe83524c1efe2689bc618f7f9e06212d44530b3200033286049dbdf84b3d32023-12-01T06:45:17+09:00Miwa / Ensanbatched.swift : update README.md (#4214)
9224fea3420ee3918d125d74c94d962a6ea8287535164e64aa2557d97490b2fe1262b313e2f4a1607e32023-11-28T23:16:34-08:00Peter Sugiharareadme : add FreeChat (#4248)
9238406b0924bf323f37d536dee8b8165c1f3d9d11db38a16dfcff88d547f78f52d1bea31b84a05aff72023-11-28T10:32:03+02:00Georgi Gerganovggml : re-enable BLAS for CPU when src0 != F32 + remove redundant full offload checks in llama.cpp (#4240)
924bb03290c17540768a16000a2b01ee4f22440aba1f3b269813f6147c5b5cda082e6b45cf04a932e0d2023-11-27T09:56:52-05:00Bailey Chittleexamples : iOS example with swift ui (#4159)
9252568a4bf548d7392e9c78c008b33b4c11d53fe95b35f3d0def3efde92ed465d92a267430d957e87d2023-11-24T18:25:10+09:00eastrivermain.swift : fix eos checking (#4197)
9266b0a7420d03b9d13cb0e9439a01ce8476d8bf093d103d935c0e75769a6a597f7a64cab72c6cc3e792023-11-23T19:07:56+02:00Georgi Gerganovllama : KV cache view API + better KV cache management (#4170)
927e937066420b79a757bf80e9836eb12b88420a21828a2e6e7d476717881be6eb9e2d3331342cec57b2023-11-19T11:10:52+01:00slarengguf-py : export chat templates (#4125)
9284760e7cc0b68570d58f55e8dda469805d1759d0dbb50a792ec2a49944470c82694fa364345e951702023-11-13T14:16:23+02:00Georgi Gerganovsync : ggml (backend v2) (#3912)
929d96ca7ded77df764db797b68b4a29e34c5b5628534b0a082074b073eb14c2bd93c0c070e20ddcd162023-11-11T05:48:21ZAlexey Parfenovserver : fix crash when prompt exceeds context size (#3996)
930875fb42871a0f5a88fbe31a0b5edd697b84038e40a7c980b6f94a049cb804573df2d8092a34df8e42023-11-08T13:15:14+01:00slarenggml-alloc : fix backend assignments of views (#3982)
9310a7c980b6f94a049cb804573df2d8092a34df8e4413503d4b92500d82b002d03c580a71a547471382023-11-07T12:43:04-05:00Jared Van Bortelgguf : track writer state, free unneeded tensors, cleanup (#3871)
932381efbf480959bb6d1e247a8b0c2328f22e350f82833a6f63c1b87c7f4ac574bcf7a15a2f3bf3ede2023-11-06T22:36:23+01:00Damian Stewartllava : expose as a shared library for downstream projects (#3613)
93348ade94538fa509465d71023e49d07aab0ec8cd5f28af0d81aa1010afa5de74cf627dcb04bea31572023-11-05T08:12:13+01:00slarencuda : revert CUDA pool stuff (#3944)
934d6069051de7165a4e06662c89257f5d2905bb1564ff1046d75e64f0e556d8dcd930ea25c23eb8b182023-11-02T18:10:39+01:00Oleksii Maryshchenkocuda : use CUDA memory pool with async memory allocation/deallocation when available (#3903)
9350eb332a10f3f14a3746c391bf80ff5e7bdf29d5dd02e98cde035d91ed8032ab943d1d504fe9da3942023-11-01T19:29:14-04:00cebtenzzrellama : fix llama_context_default_params after #2268 (#3893)
936d02e98cde035d91ed8032ab943d1d504fe9da394898aeca90a9bb992f506234cf3b8b7f7fa28a1df2023-11-01T23:10:09+01:00slarenggml-cuda : compute ptrs for cublasGemmBatchedEx in a kernel (#3891)
93750337961a678fce4081554b24e56e86b676601630e40806c1cb3bdf9955ed807ffbe212be85b4c672023-11-01T20:11:02+02:00Georgi Gerganovllm : add llm_build_context (#3881)
93871e3718abdb2771b50c9606d3a7569623a0b0afe238657db2364cfb728c694470a4a81702afea7602023-11-01T08:04:02+02:00Georgi Gerganovllama : refactor graph build code (#3837)
9396e08281e588bbba1a5d180290a94a43f167f3a1a2046eb4345e62c4575b3cdc0115a51db89f3fb702023-10-29T11:31:40-06:00KerfuffleExtend llama_kv_cache_seq_rm to allow matching any sequence (#3843)
94071a09da301705b9c5ad4ca3cf3fbd966dd3f1ec5d69d777c02b9ac405a95f3cbfba219a990caefff2023-10-29T18:32:51+02:00Georgi Gerganovllama : fix kv shift bug (#3835)
941c8d6a1f34ab6f1b6bd468d256e535a61f98f114c2f9ec7e271220a78fe27c9e6ccbcc0dda31cda0f2023-10-27T16:37:41+02:00Thibault Terrassonsimple : fix batch handling (#3803)
9422f9ec7e271220a78fe27c9e6ccbcc0dda31cda0f34b2a5e1ee4fe6295fb4420eb91131d743694c652023-10-27T17:01:23+03:00Georgi Gerganovcuda : improve text-generation and batched decoding performance (#3776)
9436961c4bd0b5176e10ab03b35394f1e9eab761792cc448774866e6479c750bd7c135cd8f92cedee672023-10-25T10:26:27+03:00Georgi Gerganovbatched-bench : print params at start
944ad939626577cd25b462e8026cc543efb715284721717521cdb976a2219888b0e5cba36e210eee9df2023-10-24T16:10:43-04:00cebtenzzreserver : add parameter -tb N, --threads-batch N (#3584) (#3768)
9452b4ea35e56792064598e922e46d081e02bc96b94daab3d7f45832e10773c99f3484b0d5b14d86c0c2023-10-24T16:48:37+03:00Georgi Gerganovcuda : add batched cuBLAS GEMM for faster attention (#3749)
9465be6c803fa5378f62a1590f3ad8c6b64c7c0c2ce6336701c9378c23c85d1c0e464b663ca2bbb8e602023-10-23T12:40:03-07:00Marcus Dunnllama : remove token functions with `context` args in favor of `model` (#3720)
947438c2ca83045a00ef244093d27e9ed41a8cb4ea99e70cc03229df19ca2d28ce23cc817198f8972782023-10-22T22:53:08+03:00Georgi Gerganovserver : parallel decoding and multimodal (#3677)
94822c69a27945e7acf9690dd3210d316f22182751c465219b9143ac01db0990bbcb0a081ef72ec20082023-10-22T08:37:20+03:00Georgi Gerganovbatched : add len CLI argument
949a0edf73bda31c7c4e649e6f07c6fd30a729929cdf439e506e8ae8b01df2ae2156380f8156d7553e32023-10-20T13:06:10+03:00Georgi Gerganovserver : fix uninitialized sampling context (close #3685)
9500e89203b517c95ec6675eda75d200a60d1e8921dc67fe68e417f766970fb1feaf2e66458aa24116a2023-10-18T16:21:57+03:00Georgi Gerganovspeculative : add tree-based sampling example (#3624)
95140e5ce054f4c4fa555e4510ea5f760bb29185332a5e8c1d8c71f01d98ae2ec63a57c118664f9764d2023-10-11T21:30:06+04:00shibe2CLBlast: Fix temporary buffer size for f16 conversion (wsize)
952a5e8c1d8c71f01d98ae2ec63a57c118664f9764de74c705e15cd228ad696c4a3cdea6d6fb4ff434c2023-10-17T19:00:58+02:00slarentrain-text-from-scratch : fix assert failure in ggml-alloc (#3618)
9532a4bcbacead886996f175f33479d1d874a3e577f424b6381c4daeed62e6600e0402e72f39845b58d2023-10-13T12:33:16+02:00Daniel Beveniusllama : remove n_threads from llama_decode_internal (#3614)
954424b6381c4daeed62e6600e0402e72f39845b58d1e0e873c373c33989beb6bc64d83cd572ab7fe2b2023-10-13T12:23:10+02:00slarenggml : add context enumeration functions (#3605)
955370359e5baf619f3a8d461023143d1494b1e8fde9e24cc6e2e589d405bd1720c400f5b0b9d0ca3ee2023-10-12T18:23:18+03:00M. Yusuf Sarıgözexamples: support LLaVA v1.5 (multimodal model) (#3436)
95657dd55e2c742bfc50e0f5c6fb95c14118cff44f6b8fe4b5cc9cb237ca98e5bc51b5d189e3c446d132023-10-12T09:29:04+03:00Georgi Gerganovserver : fix kv cache management (#3588)
957a8bdd65525ae86dea905e9866ad369b53e30ac1470c29da118cdb02bfcbd0376c32b5b2236e48e482023-10-11T15:42:22-04:00Michael Coppolaserver : add parameter -tb N, --threads-batch N (#3584)
95870c29da118cdb02bfcbd0376c32b5b2236e48e488c70a5ff25964f0a81e20d142a2f5ac5baff22fc2023-10-11T13:35:46-06:00Kerfufflecommon : fix mirostat state when using multiple sequences (#3543)
9598c70a5ff25964f0a81e20d142a2f5ac5baff22fc24ba3d829e31a6eda3fa1723f692608c2fa3adda2023-10-11T21:25:33+03:00Georgi Gerganovbatched : add bench tool (#3545)
96024ba3d829e31a6eda3fa1723f692608c2fa3adda9f6ede19f3cfa50d4a51a5babb056c3f8a450b802023-10-11T04:14:05-07:00Zane Shannonexamples : add batched.swift + improve CI for swift (#3562)
961f5f9121de140eff558f13b5c5e78a3a3b6b9437711ea5c7d96f2c28e1c99659e08ec0a44574056e22023-10-10T09:50:23+02:00Jan Ploskillm : add MPT support (#3417)
96295bd60a0a69f57e9a2ff1269667ea484a1a9bb40fcca0a700487999d52a525c96d6661e9f6a8703a2023-10-09T14:44:58+02:00slarenggml-alloc : fix assert in debug builds (#3555)
963fcca0a700487999d52a525c96d6661e9f6a8703adcc09d25961c5d0626bc148e558ee841141748f72023-10-09T14:32:17+03:00Georgi Gerganovrefact : fix convert script + zero out KV cache to avoid nans (#3523)
964b0ec5218c3d24755786b80ecce9cf4ffc07583f863d3b06a4318329f92b078e8aa0be7ab6e9f871f2023-10-08T10:01:53+03:00Georgi Gerganovmetal : support MTLGPUFamily < Apple7, formatting, style (#3524)
9659ca79d5cbbc8d43f2bff951404b6a40ff1ee37880c731ca4039ccff86ffab90eaae4ca98037c44962023-10-06T10:10:13-06:00Kerfufflekv cache slot search improvements (#3493)
966ac2219fef34eb5b713c286c34c6e4162c39c8f3b48be797ffbd80b062f55778e09e97180eb25d2ab2023-10-03T21:04:01+03:00Georgi Gerganovllama : fix session saving/loading (#3400)
96748be797ffbd80b062f55778e09e97180eb25d2abf56e1baec361b5381e32ee6b6e56e4f00e002dfe2023-10-03T10:09:28-07:00Alex Klinkhamerllama : expose model's rope_freq_scale in the API (#3418)
968bc34dd4f5b5a7c10ae3ed85a265ce6f2ed2fab792777a84be429401a2b7d33c2b6a4ada1f0776f1b2023-09-29T19:05:18+03:00Georgi Gerganovtrain : fix KQ_pos allocation (#3392)
96916bc66d9479edd5ee12ec734973554d4493c5dfa0512d66670de3f650c579519833c085014b0f2002023-09-28T21:42:38+02:00slarenllama.cpp : split llama_context_params into model and context params (#3301)
9700512d66670de3f650c579519833c085014b0f2000e76a8992c8200237bbc6471a53fb8796b3872f72023-09-28T19:31:04ZEveci : multithreaded builds (#3311)
9710e76a8992c8200237bbc6471a53fb8796b3872f72db94d98eda56982d80238840b0652b4137a2a842023-09-28T20:40:11+02:00xaedestrain : finetune LORA (#2632)
9722619109ad57d7a75388a9cce51e5da645410d92eec893798b7a2a803466cc8f063051499ec3d96f72023-09-28T19:36:36+03:00Georgi Gerganovci : disable freeBSD builds due to lack of VMs (#3381)
973ec893798b7a2a803466cc8f063051499ec3d96f745855b3f1c7bdd0320aa632334d0b3e8965c26c42023-09-28T19:04:36+03:00Georgi Gerganovllama : custom attention mask + parallel decoding + no context swaps (#3228)
974dc6897404e141c74cbbf8030ecfebd74e1815411527e57cfd8a9a26bf622c0510c21c2508a24be262023-09-27T17:48:33+02:00Rickard Hallerbäckmetal : reusing llama.cpp logging (#3152)
975a40f2b656fab364ce0aff98dbefe9bd9c3721cc9d119c04c159d015a93567df7e73e0e45a22d0f1d2023-09-20T15:06:36+03:00AlonCI: FreeBSD fix (#3258)
976578d8c8f5cb72f354bc115ba230ee5b2d803eee7b541b4f0b1d4d9871c831e47cd5ff661039d61012023-09-17T14:16:22+02:00Johannes GäßlerCUDA: fix scratch malloced on non-main device (#3220)
9774fe09dfe665c58a753dc9eb638dd4dca1cd3548880291a1d02a07f7f66666fb576c5b1e75aa48b462023-09-16T03:02:13+08:00Meng Zhangllama : add support for StarCoder model architectures (#3187)
9788c00b7a6ff38e27fa1e471452b8a480913772c2a7e50d34be68aae2cc766203703dd188e910e033a2023-09-15T19:06:03+03:00Georgi Gerganovsync : ggml (Metal F32 support + reduce ggml-alloc size) (#3192)
97983a53b753a9499a2a3535c93975b430cb2c828a95c872dbca2c7979b1f6dafc97db0774b8bbf93722023-09-14T20:21:25+03:00AlonCI: add FreeBSD & simplify CUDA windows (#3053)
980e64f5b55783e910d8287363895d652b4bea6527a94f10b91ed69980f299441e49c8dbdb448f0ccc62023-09-08T11:43:35-04:00Cebtenzzreexamples : make n_ctx warning work again (#3066)
981cb6c44c5e045709b6bb5cc9bb8c9be107c771a78a21baeb12202a9020b48c53beaaf4b355228e8ba2023-09-08T14:09:21+02:00Przemysław Pawełczykbuild : do not use _GNU_SOURCE gratuitously (#2035)
982ebc96086af49fe70108cafcea6ab4bebd658a41a7f412dab9c8801f5d37904f7dce1faf4c2b43b422023-09-08T04:04:56+02:00slarenggml-alloc : correctly check mmap return value for errors (#3075)
983c4f496648c1e32efeb714200e7eae7fc7cfbb223fec2fb19e4229aac58c98171c46e77144b99f8a32023-09-07T15:49:09+03:00Georgi Gerganovmetal : fix kernel_norm (fixes Falcon on Metal) (#3057)
984bd33e5ab92e7f214205792fc1cd9ca28e810f89731035681445181fb414e0def7ec3f84462b3bd972023-09-04T14:59:52+02:00slarenggml-opencl : store GPU buffer in ggml_tensor::extra (#2994)
985cf9b08485c4c2d4d945c6e74fe20f273a38b610447068e517004d90f13c16352bb3b4cafd53a00cd2023-09-03T20:34:09+02:00slarenggml-alloc : use virtual memory for measurement (#2973)
9866460f758dbd472653296044d36bed8c4554988f5ca82cf7bac0c91d03e3d320b3a865dd006f854ac2023-09-03T16:46:44+08:00Wentai Zhangopencl : fix a bug in ggml_cl_pool_malloc() for ggml_cl_mul_mat_f32() (#2955)
98721f3d1be867b4d7be07c26f5da6e4bc69bcf4d27571083f508266c4eb5cb5457d836df5dd3c173ce2023-09-02T20:23:45+08:00Jhen-Jie Hongk-quants : fix build on armv7 (android only) (#2920)
98818705a30ef3d6a89e1d7c6cb8cfe8633f760cb53e8d91589258f9204397a7ac5f9b3c857835c98f82023-09-01T05:03:49-04:00Cebtenzzrellama2c : fix segfault and alloc-dealloc-mismatch (#2913)
989e8422de39e4aa2f7e50574124b060a80607e654a92d0b751a77a089e650983e9f1564ef4d31b32b92023-08-31T04:21:45-07:00DannyDaemonic@vxiiduu's fix for PrefetchVirtualMemory (#2930)
9908afe2280009ecbfc9de2c93b8f41283dc810609a71d6975559acfd6c8407a4ef8275a9979c7377652023-08-30T21:46:19+02:00Johannes GäßlerCUDA: mul_mat_q=true llama_context_params default (#2912)
99106abf8eebabe086ca4003dee2754ab45032cd3fdc03a243abf9f30889f31fefdfa94fe9d7034820c2023-08-29T23:24:42+02:00slarenggml : add view_src and view_offs to ggml_tensor for views (#2874)
99295b6e5212f5e4e1419de1d833d7f8d788f9f222744c117f41ee01c5ac8fb86bba041f08d8b87b46d2023-08-28T23:33:27-07:00Marcus Dunnadded `struct` to llama_dump_timing_info_yaml's `llama_context` (#2857)
99344c117f41ee01c5ac8fb86bba041f08d8b87b46d43033b7bb4858da4f591715b3babdf906c9b7cbc2023-08-28T21:51:47+02:00xaedestrain : mem usage and other improvements (#2439)
99435feac6560387cf0484371af3d9b12bff678e0b992b1bbd2ec43c82ec0530ba3c8758846c5790c752023-08-28T14:24:53+03:00Georgi Gerganovggml : sync (mem align to header + conv_transpose_2d fixes + ggml_alloc) (#2852)
995f55538c3ccba9b926846ef862fa830cea08c433eebcee207b6058b7f695bb5c203ad87b1066a97902023-08-28T10:59:08+03:00Georgi Gerganovmetal : fix memory leak (#2762)
99650526f37eba0b28336700890242ff282b949cd8304f4b1eb10f3e25750ca3e530265ce2841730e6b2023-08-26T12:53:52-04:00Cebtenzzrellama : use std::abs in llama_sample_tail_free (#2800)
997b91ad7f46134d0d051dc516eb59a76f402de55c22e5f70a25fc4576e9ed78603fe493eb7702c37a32023-08-25T01:58:00-04:00Shouzheng Liuggml-alloc : enlarge size of parse_seq (#2776)
998d0f77b1353fc820d1ff1e6b87bc6bedde315938d0d3094f0c742ce61f84feb6e4f0b59beee6194d72023-08-24T21:10:39+02:00slarenconvert.py : try to determine n_ctx automatically for CodeLlama (#2770)
99938b16dfca6e5032e6cfb90c1653bf1ba4cf647b48f8c28e89cb9531211783da697d6e7c445e2af1d2023-08-24T12:27:25-04:00Shouzheng Liumetal : bug-fix when enable ggml-alloc (#2757)
1000cf658adc832badaaa2ca119fe86070e5a830f8f6a192860cfec89a38d59a943623bf595b1fe4495b2023-08-23T23:08:04+03:00Georgi Gerganovllm : add Falcon support (#2717)
1001bac66994cf356cf488078c056831396eb4ce31d5519c981f8b65ee6c87c2965539685ced0a17223b2023-08-22T19:14:09+03:00KawrakowQuantization imrovements for k_quants (#2707)
1002519c981f8b65ee6c87c2965539685ced0a17223b1123f7fbdfb8012e46f05e903e6f6759229163782023-08-22T16:03:12+02:00slarenembedding : evaluate prompt in batches (#2713)
10031123f7fbdfb8012e46f05e903e6f675922916378ef3f333d3775600d1646a9fa249aca532d15fb892023-08-22T15:25:19+02:00slarenggml-cuda : use graph allocator (#2684)
10046381d4e110bd0ec02843a60bbeb8b6fc37a9ace9dadbed99e65252d79f81101a392d0d6497b86caa2023-08-21T23:07:43+03:00Georgi Gerganovgguf : new file format with flexible meta data (beta) (#2398)
1005cb1c0727bd59803b439b6a3af121c99e6393ff3d9e232f0234073358e7031c1b8d7aa45020469a3b2023-08-21T11:11:31+03:00KawrakowHellaSwag: split token evaluation into batches if needed (#2681)
1006097e121e2f17ed3541cf02c55ff7e9febc091b19eaf98c2649d7da705de255712f0038ac7e47c6102023-08-18T12:44:58+02:00slarenllama : add benchmark example (#2626)
1007a872a2b28eaefc8d464eaa535c94deeb501666f90919a0f73d95cfb93a1646a1d1741a0615fe2c5e2023-08-17T03:35:53-04:00Shouzheng Liuggml-alloc : fix discrepency between measure&eval (#2639)
1008fc8ef549e50087762a0b4f901cd74b2defcc6ae3bf83bff6742c0f1795b4c18695a13a34ac7adf622023-08-16T16:08:28-04:00Shouzheng Liumetal : enable ggml-alloc (#2627)
1009bf83bff6742c0f1795b4c18695a13a34ac7adf62b5ffb2849d23afe73647f68eec7b68187af09be62023-08-16T16:07:04-04:00Shouzheng Liumetal : matrix-matrix multiplication kernel (#2615)
1010e59fcb2bc129881f4a269fee748fb38bce0a64de1638757767072a4957f52b9e3594f0b67610631b2023-08-10T10:28:27-04:00Christian DemsarAdd --n-predict -2 for stopping generation on full context (#2565)
1011916a9acdd0a411426690400ebe2bb7ce840a6bbaea04a4ca1940d92becc0ee26523aa2c4a18cf9382023-08-09T23:47:42+03:00Sam Spilsburyggml-alloc: Don't try to re-use buffers of external tensors (#2562)
1012ea04a4ca1940d92becc0ee26523aa2c4a18cf93825d43e0eb578b6e73046d9d6644a3a14d460600d2023-08-09T22:46:40+02:00grahamethadd log_callback to llama_context_params for custom logging. (#2234)
1013415e99fec27be5a2e4283f1937afd17eb33fbd66ff966e7ca6af127c9405523cdb07ef8fa01bf6d62023-08-04T19:29:52+08:00l3utterflyStream save llama context data to file instead of allocating entire buffer upfront (#2488)
10142dbf518911926ef5a30f43aa83a0b1b1cdeaaa119d2382b3e45b5815fc6a054045a2f2c2b18c22a22023-07-31T13:18:51+02:00Johannes GäßlerCUDA: fewer memory bank conflicts for mul_mat_q (#2458)
10159d2382b3e45b5815fc6a054045a2f2c2b18c22a2a113689571420fb4d6540f1a324d12965781356a2023-07-31T11:02:53+02:00slarenFix Metal backend broken from the allocator changes (#2455)
1016a113689571420fb4d6540f1a324d12965781356a11f3ca06b8c66b0427aab0a472479da22553b4722023-07-30T15:58:01+02:00slarenggml : add graph tensor allocator (#2411)
1017d91f3f0c55663719ea03b76311e8c36ed55eb0e265cdf34bdc469fa86248e667a5880992684ef1142023-07-28T10:44:43+02:00Weird Constructorreadme : fix the description of the Tail free sampling (TFS) method (#2431)
10185488fb789ea5692268309baa76f67598155060beeb542d39324574a6778fad9ba9e34ba7a14a82a32023-07-26T15:56:53+02:00slarenggml : allocate graphs in a context (#2392)
10190c06204fb39aa5560e883e0ae74be9518c57d88e1fed755b1fb9babb6dbc1b4023e492950cd5a5be2023-07-25T07:19:11-05:00Xiao-Yong Jinmain : add `--in-prefix-bos` to prefix BOS to user inputs; keep EOS (#2304)
10201aa18ef994a6a2b531434eb13251ef48e56d345b9a08eaf3c4010962d0126e9e5bfbe9af64b2ac902023-07-25T08:00:19-04:00Shouzheng Liumetal : concurrently dispatch commands (#2358)
102191171b8072f6f0c8ae3a61e23451acb538bb9ece355c80f49e32b0b15c0a457f3bad380e57f5b9ac2023-07-23T07:52:08-04:00Jose Maldonadomake : fix CLBLAST compile support in FreeBSD (#2331)
1022b47b8a9cfeb439d271bf997fb985fd6d82b3af5eb5fe67f8c69113bd9354bc1adcfe2df6be3237402023-07-22T21:17:57+03:00Georgi Gerganovllama : optimize memory buffers (#2325)
1023d924522a46c5ef097af4a88087d91673e8e87e4d4d76a5f49b9b5382dba5d13d92edb9159536c2252023-07-21T17:27:51+03:00KawrakowCustom RoPE + bettter memory management for CUDA (#2295)
102473643f5fb1136dc2b65ae910bdc5a431520d70a2a814d04f81121e0429b39a61fe4afd946cd420462023-07-21T06:53:27-04:00Jose Maldonadogitignore : changes for Poetry users + chat examples (#2284)
1025417a85a0010519224cf154eb85d383ffeafeeead294f424554c1599784ac9962462fc39ace92d8a52023-07-20T06:32:22-04:00Shouzheng Liumetal: minor q4 optimization and reduce code size (#2248)
1026d01bccde9f759b24449fdaa16306b406a50eb3676cbf9dfb32f0e23ed3afd02d30ab066ed53e2c4d2023-07-18T14:24:43+03:00Georgi Gerganovci : integrate with ggml-org/ci (#2250)
10276e7cca404748dd4b1a3affd0d1296e37f4ac0a6fa6803cab946c817fb7aaf2a40b317f5d3e373bd12023-07-15T06:34:16-04:00Xiao-Yong Jinllama : add custom RoPE (#2054)
10287cdd30bf1f84339c55a5e3de29384f6bbdebb61ce8035f141e1f71d739fa5cfc9c01531cdee6fc162023-07-15T03:00:58+08:00Bach Lecuda : allocate all temporary ggml_tensor_extra_gpu from a fixed-size buffer (#2220)
10297513b7b0a1c11faa00ad5a34d22681e5f07d32e4de8342423d9600cf6e15455c1a27bae441262b452023-07-15T02:55:24+08:00Bach Lellama : add functions that work directly on model (#2197)
1030975221e9548ef6d9f4af8d39cdffc4811c050beb4523d10d0cf8c088f1b26c76d38d73290eb3b4442023-07-12T20:51:29+03:00Georgi Gerganovggml : broadcast mul_mat + conv batch support (#2199)
1031c9c74b4e3f9dcfab8b0032749ff8a579ab4e4d8d3ec7e596b2ba3f43c22f441254ca2bcfa91102ba2023-07-12T00:18:43+08:00Bach Lellama : add classifier-free guidance (#2135)
10321d656d6360359cfdaaf5d64ed9690047b600dbcb72421402834141df6cbdcf595fe46dbd11874dce2023-07-08T00:24:01+08:00Qingyou Mengggml : change ggml_graph_compute() API to not require context (#1999)
103336680f6e40e4440c3ec3385d0b7e5ca8bb6c37f7a17a2683d8fdb899ba497d0c28ccafb28c62efb62023-07-07T00:23:49+08:00Juddconvert : update for baichuan (#2081)
1034befb3a35627432473f143c90994557d78ff5bc67b2132270678c473f7cd9ba871b03d694126bc33a2023-07-01T21:47:26+02:00Johannes GäßlerTest-based VRAM scratch size + context adjustment (#2056)
10352f8cd979ecd1fa582852e7136e92ff8990b98fd8471aab6e4cb89d8ef6d043f1bc93acb6eb78ab672023-07-01T11:14:59-07:00Aaron Millermetal : release buffers when freeing metal context (#2062)
1036d3494bb86bf7ad5b0b60aae0220ea576f273b5c05b351e94d041742cd50ffcf2d44718d63bab398a2023-06-28T20:39:08+02:00m3ndaxllama : replacing auto &kv with const auto &kv (#2041)
1037cfa0750bc9dbc2d957a91b8ed09ab0035d8f3d4e9d23589d638dc74577d5ff880e6d4248b795f12e2023-06-28T23:53:37+08:00ningshanwutuobangllama : support input embeddings directly (#1910)
1038b853d456018b10820686362af41b2f2f75f1eec69225baef71407d799a6f7f563b77fd7f827914162023-06-26T13:57:59-04:00zrmggml : add NUMA support (#1556)
1039cbebf61ca7584e9709265395f0127ae7fc0f1882447ccbe8c39332fcdd0d98a041b6e2ff6f06219d2023-06-26T23:15:47+08:00Howard SuFix assert when free invalid cuda pointer (#2005)
1040527b6fba1d237befb324fd846bda7418c0fa394dd7b7484f74d486f77feb4c0b7af7e1718ed916512023-06-24T11:47:58+03:00Didzis Goskollama : make model stateless and context stateful (llama_state) (#1797)
1041ba4e85a8339b9dd7cdffad31838235f2fe45a8ea23fc5c219a9aebd57c8af3fac454062cc46229802023-06-19T23:20:06+08:00l3utterflyllama : use aligned memory during ggml_init call from loading saved sessions (#1934)
1042ca7c3f4da5d144d4cd1dd44903552e6ba49b8ec8b97ca431db35ec96a339a721acb1219c1dd78bed2023-06-19T18:14:09+03:00Kawrakowcuda : faster k-quants on older GPUs (#1930)
1043ce2c7d72e2d06988b5ddec6811ab92325454207757cd69460f736031a3fc54af1e97c03f801284782023-06-18T09:09:47+03:00Georgi Gerganovmetal : handle buffers larger than device's maxBufferLength (#1826)
1044051e1b0e6a6e3aee7d989b47760980e6fda5861c86c7571864ff331f8cdb9e092f3abeb123729a562023-06-17T19:30:22+03:00Georgi Gerganovllama : fix kv_cache `n` init (close #1903)
1045794db3e7b982fee37e3995db9c3a216a57ff65e35ddf7ea1fb42bac21026de2f77e0f9c069b922342023-06-17T07:53:04-04:00Randall FitzgeraldServer Example Refactor and Improvements (#1570)
10464bfcc855abdb2c9fcc3c5a84747974521909fa416b8312e7979b852f6b6ac9d29cd51fda16c179482023-06-15T20:29:48+03:00Georgi Gerganovmetal : parallel command buffer encoding (#1860)
1047254a7a7a5ff4c874ff8488f1f5cbdd7e9c89d68292549202659fc23ba9fec5e688227d0da9b06b402023-06-14T19:47:19+02:00Johannes GäßlerCUDA full GPU acceleration, KV cache in VRAM (#1827)
1048e32089b2c20b1b87b22912f4a8b93fe01647d5b92347e45e7bdb09c9a7d74b2c0bc86c2b65f0c3432023-06-13T21:04:40+02:00xaedestrain : improved training-from-scratch example (#1652)
104974a6d922f12ccfe16b0c265f43be8978c6f25e98e4caa8da59c1c97dc23fa336f4d726984a20560f2023-06-12T22:39:21+03:00KawrakowMetal implementation for all k_quants (#1807)
105058970a4c39124a647ac2a640d9e178ea6c961e658c0a10e64dbf60fd9946c0cd5e6f59690800b1232023-06-12T20:44:16+08:00Howard SuLeverage mmap for offloading tensors to GPU (#1597)
10518c0a10e64dbf60fd9946c0cd5e6f59690800b123fa84c4b3e80199a5683438f062009c031a06c4fa2023-06-12T14:31:36+03:00Kawrakowmetal : fix failure to load model (#1817)
1052fa84c4b3e80199a5683438f062009c031a06c4fa12b063f0ecf280e98028e444fc492ee6222cdcdc2023-06-11T08:19:17-06:00KerfuffleFix issue where interactive mode crashes when input exceeds ctx size (#1789)
105317c10acfb44ecb7af25e37fb67b9501cbc0034d2e9b66ee9829039d4ab54550d6222e42a0b31e52a2023-06-10T12:06:45+03:00Georgi Gerganovggml : force no_alloc == false when creating opt tensors (close #1699)
1054e9b66ee9829039d4ab54550d6222e42a0b31e52a4f0154b0bad775ac4651bf73b5c216eb43c45cdc2023-06-10T11:28:11+03:00Kawrakowmetal : add Q4_1 implementation (#1785)
105598ed16557432d7a5179c57eddcc3a08a7ae6d54dae9663f1887513e152839e91f61c513075a194222023-06-10T01:24:40+09:00Robert Sung-wook ShinOpenCL: Add release memory (#1741)
105672ff5282bf0388c60821f504c4c8cc2b1f491aa60bf7cf1b296fc9fca05411b37afdf08a531487d22023-06-08T22:28:21+03:00Kawrakowmetal : add Q2_K implementation (#1762)
105735a84916fb029905c44746127026079268216e7a2d7bf110edd8c49209401a16132052cba706ffd02023-06-07T04:10:17+02:00Willy Tarreaumain: add the possibility to open the prompt cache read-only (#1640)
105817366df842e358768c0df7024484fffecfc7865b44f906e8537fcec965e312d621c80556d6aa9bec2023-06-06T21:33:23+02:00Johannes GäßlerMulti GPU support, CUDA refactor, CUDA scratch buffer (#1703)
1059d5b111f53d14972669eb52055f9df2567663ad8b2d43387dafe9c60f15f57aa23ee0b37864b98b322023-06-07T01:00:01+08:00LostRuinsClblast fixes + enhancements to save VRAM and offload more layers (#1675)
1060590250f7a9847bc9c83aa063dbaac8fa0fea27c8f4c55d3bd7e124b101bc974cbbf0e0dbbc32d5a32023-06-05T23:28:17-04:00Spencer Suttonmetal : add checks for buffer size (#1706)
10619d0693bce38013364b1042568d9083353bfff48fefe05076323f5c6bafece109e21cce046f5e4b072023-06-05T13:24:04-07:00kiltyjmetal : use shared buffers between CPU and GPU (#1696)
106299009e72f8072fa552eb02efee436be596c71cdd5220a991a5e92bddad9542267ab445a2c033681c2023-06-05T22:56:18+03:00Kawrakowggml : add SOTA 2,3,4,5,6 bit k-quantizations (#1684)
10635220a991a5e92bddad9542267ab445a2c033681cd1f563a743a83dabc11e125d4a7d64189c16498c2023-06-05T13:43:08+03:00Henri VassermanIncrease 3B scratch buffers. (#1698)
1064d1f563a743a83dabc11e125d4a7d64189c16498c827f5eda91e5b7299848ee2c7179d873bdee0f7b2023-06-05T10:19:03+03:00Georgi Gerganovllama : fix Metal KV cache sync (close #1695)
1065dcb2ed48268e421baf25adc00d602dad0f415564d8bd0013e8768aaa3dc9cfc1ff01499419d5348e2023-06-04T08:12:05+02:000cc4mOpenCL: Fix duplication of layers in VRAM and RAM, add GPU mul kernel (#1653)
1066136476e898fb96c302b0829ee3e79267ae12660fffb06a345e3a9e30d39aaa5b46a23201a74be6de2023-06-03T07:28:45-04:00Evan JonesFix prompt cache saving and chat-persistent rollover (#1678)
1067248367605ead6fb7c36d2bfb1ebd8f00a23f7c710e730dd23b0fb5f93dba574e0a48d9a69dc5dbae2023-05-29T05:13:40-07:00DannyDaemonicWork around for recalculating logits in cached prompts (Fixes #1585) (#1609)
10681b78ed20818b72306edc7208b9bfb69a1a0d3297337aea11390221bc925e4acb1f603f1649af27352023-05-28T11:48:57-06:00KerfuffleOnly show -ngl option when relevant + other doc/arg handling updates (#1625)
1069bb051d9723d628414b9e929e5264e23262a2f1b2ca74884f6625b15ef69832f07fc60fe00db5f90c2023-05-29T01:13:36+08:00Howard Suopencl : no need to allocate cl_mem on heap (#1612)
1070ca74884f6625b15ef69832f07fc60fe00db5f90ca6704643b62243bc4b6bbcd727d63d44e01a10022023-05-29T01:09:56+08:00Howard Suopencl : use strstr to check if fp16 supported (#1611)
107166874d4fbcc7866377246efbcee938e8cc9c7d761fcdcc28b119a6608774d52de905931bd5f8a43d2023-05-25T20:18:01-06:00KerfuffleSome improvements to loading the session with --prompt-cache (#1550)
1072affc76edfdefa7b326f526e463cc65ff13fcfb92ea600071cb005267e9e8f2629c1e406dd5fde0832023-05-20T14:19:28+02:00Johannes Gäßlercuda : loading models directly into VRAM, norm calculation on GPU, broadcasting for ggml_mul (#1483)
10735ea43392731040b454c293123839b90e159cbb99ee9654138ab0ae5f138f4abddf56ca234ea3c3522023-05-18T19:31:01+02:00Erik Scholzmake kv_f16 the default for api users (#1517)
107413c351ad7292c5b5ab35db25c7a4f993e75d9cfd60f8c361ca26328ef8523dfb08077fe2f10344902023-05-14T18:22:50+03:00Georgi Gerganovggml : various fixes (#1450)
1075f954edda935a70a14cf0cc45ecc7fe7d60cf3e4bf048af0230ad5381bb20b9e3c1467ec6fc7debdf2023-05-13T14:56:40+02:00xaedesggml : implement backward pass for llama + small training-llama-from-scratch example (#1360)
1076738ace394a6f8cf0174e90a97185d9e512c0e200699b1ad7fe6f7b9e41d3cb41e61a8cc3ea5fc6b52023-05-13T09:08:52+03:00Georgi Gerganovllama : free ggml context in set / copy state data (close #1425)
1077cf348a60e0af3905acd1d297cb064b918265b7ace6a46b0ed1884c77267dc70693183e3b7164e0e02023-05-10T11:37:14-04:00Evan Jonesmain : add option to save full output to session (#1338)
1078e216aa04633892b972d013719e38b59fd49173412485d7a4d39406cd0f468e35551b472cceb5bd612023-05-02T22:26:13-04:00Evan Jonesllama : only copy used KV cache in get / set state (#1272)
1079b925f1f1b082319ee69943f8d1a83ac9b6ff09ca90b19bd6eee943832584f9cac0b6f9ea29cc42a42023-05-01T13:32:22+02:00slarencuBLAS: fall back to pageable memory if pinned alloc fails (#1233)
108090b19bd6eee943832584f9cac0b6f9ea29cc42a47ff0dcd32091c703a12adb0c57c32c565ce176642023-05-01T00:24:20-07:00Alex Klinkhamerllama : let context be const when accessing const data (#1261)
108176a884920aa1d2fc0dc7a7ac12dfc5ec5816377c6bc4400e67e6bc4faad3ad3d5e9d8a6576a9752d2023-04-30T20:34:52+02:000cc4mggml : add CLBlast q5_0, q5_1, q8_0 dequant kernels (#1225)
1082214b6a35702a489e3738acd81fad6d46182d3036305eb5afd51325e3142c01c17431febb7c67de872023-04-29T18:43:28+03:00Georgi Gerganovggml : adjust mul_mat_f16 work memory (#1226)
1083dd7eff57d8491792010b1002b8de6a4b54912e5c7fc50c051ae8a78e9643fdf172d12e20f2dd9b6c2023-04-29T08:34:41+03:00Ivan Stepanovllama : new sampling algorithms (#1126)
10847fc50c051ae8a78e9643fdf172d12e20f2dd9b6cb1ee8f59b4101b46999a0995d9a34506f72854662023-04-29T02:04:18+02:00slarencuBLAS: use host pinned memory and dequantize while copying (#1207)
10857296c961d9303010a2b98379f738da2a8a55aa1b78ec543733d10a1629f984fd0302fdaa4e87fe662023-04-28T16:57:16+02:000cc4mggml : add CLBlast support (#1164)
1086dd0eabc049fb1efc631cab8eb0a646808d704e1854bb60e26858be251a0eb3cb70f80322aff804a02023-04-25T19:20:46+02:00unboundedggml : use full range for Q4_0 and Q4_2 quantization (#729)
1087957c8ae21d1e7052ea45a40ee8c0407b909e90cc9b0a4d421459f4e5e1af735c9784c3247b3790252023-04-24T18:47:03+03:00Georgi Gerganovllama : increase scratch buffer size for 65B (ref #1152)
1088c4fe84fb0d28851a5c10e5a633f82ae2ba3b7fae1d78fecdab4087028a38517e86ed129f077174d82023-04-24T07:40:02+03:00Georgi Gerganovllama : refactor get / set state + remove redundant kv cache API (#1143)
108936b4f7e06406eed8a605cc9f2921d9244ef6a8e510f19c1121068ce3dab9bece03a8b9caaea2db362023-04-22T16:56:35+08:00wbpxre150llama : print timings on ctrl+c exit (#1021)
109010f19c1121068ce3dab9bece03a8b9caaea2db367e312f165c5047d6e16680d1eebc83055e95c3132023-04-22T04:27:05-04:00eieryllama : have n_batch default to 512 (#1091)
1091b6e7f9b09e9c340ec97a2fae61c1eb8db861f2f950cb666b8a2e35a49b08c0f6bc81138c8f6f2ac12023-04-22T08:21:32+02:00xaedesllama : add api for getting/setting the complete state: rng, logits, embedding and kv_cache (#1105)
109250cb666b8a2e35a49b08c0f6bc81138c8f6f2ac125d7abbd1f73582b7e0fdc422a936e8541c0780b2023-04-21T21:59:17+02:00slarenImprove cuBLAS performance by using a memory pool (#1094)
10939411288271ab548216902a029f42a0a38ebcedb78687c1f2581d059cd5b6a9502f89bd343566062a2023-04-21T11:18:09-07:00Alex Klinkhamermain : evaluate tokens in batches after swapping context (#1014)
10948687c1f2581d059cd5b6a9502f89bd343566062a1bfc153e2f35ddd9d64b084e8d1a5e6fa57ad1c92023-04-21T17:25:21+02:00xaedesllama : remember and restore kv cache data pointers (#1104)
109574f5899df4a6083fc467b620baa1cf821e37799d2f7c8e014e3c0ceaf39688845c2ff6f919fb03b72023-04-15T14:53:21-07:00comexconvert.py: Fix loading safetensors and ggml format on Windows (#991)
1096c14e0d2f23e6d1e785255f4da8c253c1b4723659723dac55fa2ba7adc6e3fc8609781d1ad03789062023-04-14T13:31:15+03:00Georgi Gerganovggml : always allocate buffers with size multiple of GGML_MEM_ALIGN
1097723dac55fa2ba7adc6e3fc8609781d1ad03789060f07cacb05f49704d35a39aa27cfd4b419eb6f8d2023-04-14T00:03:03-07:00comexpy : new conversion script (#545)
1098be87b6ed20a5f7528bf491a83e759a9fc6a24fea0e07e6a8399fd993739a3ba3c6f95f92bfab6f582023-04-13T14:50:42-07:00Gary Linscottperplexity : add support for batch size to `--perplexity` (#407)
10996c248707f51c8a50f7792e7f7787ec481881db888cda5c981d0bf4dcb7664194b2cb9a06e2dbdd542023-04-13T16:08:32+02:00Pavol Rusnakggml : introduce GGML_ALIGNED_MALLOC/GGML_ALIGNED_FREE macros (#884)
11002663d2c6784ad7b77998c6874df25648d597f74ba0caa34b162449b5c13b8d604573053300ff54a12023-04-11T06:19:54-07:00comexWindows fixes (#890)
1101f963b63afa0e057cfb9eba4d88407c6a0850a0d8aaf3b23debc1fe1a06733c8c6468fb84233cc44f2023-04-08T12:24:37-07:00comexRewrite loading code to try to satisfy everyone:
1102cc9cee8e9e7598bd280295f6264f36d3a9224006d2beca95dcfcd6f1145886e914b879ffc3604b7a2023-04-06T17:59:11+02:00Sergey AlirzaevDo not crash when it has nothing to say. (#796)
1103986b6ce9f99503c51ec5afd8a10baa32359434c634162989297fdfe3ab7305451ce55bc87e3f4c9c2023-04-05T22:07:33+03:00Georgi Gerganovggml, llama : avoid heavy V transpose + improvements (#775)
1104e986f94829bae0b9e66b326acbbba179931c84f1c0bb1d3ce21005ab21d686626ba87261a6e3a6602023-04-02T12:23:04+02:00Christian FalchAdded api for getting/setting the kv_cache (#685)
110578ca9838ee36660a776e97e3391b6fb5dcaacf7fa017390358cdb23fffb30988dc84bb190d0403ca2023-03-29T13:51:37-07:00Justine TunneyMake loading weights 10-100x faster
1106276e5b781155e3bbe6834472c58f03dfe62efabed68c5dc4356c8f49e933df210f2ceca5002a81182023-03-29T08:31:26+02:00SlarenUnmap the file in llama_free
1107692ce3164ef1201ecb9cfad315cc0a08b965adb896f9c0506fa81cada6f96f45768c34f45406c4bb2023-03-29T02:02:34+09:00DooWoong Lee (David)py : removed unused `model` variable and verified that the code functions correctly with `vocab_only` setting. Also confirmed that the code works as expected after running with reduced memory usage due to deletion of no-longer-needed variable. (#547)
1108d502bc7c9d9d6dfb3a09aea404395b666d7b374d436e56193199a1625f8c561069f702e8840a9e082023-03-28T19:51:55+03:00Georgi Gerganovtests : free llama context at the end of the test
1109e2d490dafd860eaaaf9aa8008ab790527d556daf03f7e335604b3d68f74995aa2ccb4955833ee4232023-03-25T21:36:22+02:00Georgi GerganovInifinite generation via context swapping (#71)
1110ab77d7631211b299cb734bea6ad1f7432415415029b7baab670ae8b76ac0da21c2ded69ff18971ee2023-03-25T16:47:59+02:00Georgi GerganovAdd longer DAN prompt for testing big batch numbers
111158e6c9f36f97d0a3e287b97256dc5f6b0e9fb5ae36d07532ef7ccf0bdc12e050472f359a6794957f2023-03-25T01:26:28-04:00Jed FoxAdd support for file load progress reporting callbacks (#434)
11126f1ee4b640912211a4b07965c585d327e32e734d8520fc310eab87f2c4612f2a00d4adbd44a20d0d2023-03-24T23:38:14-05:00Chris KuehlFix crash for 65B model with pre-allocated memory (#485)
11137a9b6c3a8bdc1cb75fefc826dfaa7331eb63695d31572d966531f7d768eb773322016ab78eb6e8352023-03-24T23:17:37+02:00Georgi GerganovReduce memory usage and allocate enough memory for largest context (#473)
111431572d966531f7d768eb773322016ab78eb6e835f4f5362edb01b05c383b23f36d7b3489c77061b52023-03-24T18:23:56+02:00Georgi GerganovTemporary bump the memory buffer size - hopefully fix issues from 483bab2e
1115afd220d9c665e4c19107120ace2f0cb742e28aa1481044d50cfe8eaa6cd0c1a1b445680e4b0b3ebc2023-03-24T17:21:01+02:00Georgi GerganovProperly free llama_context on failure
1116563cdc391dde140f1084d1012234e8e6f57f881f8d4a855c241ecb0f3ddc03447fe56002ebf27a372023-03-24T08:19:05-07:00comexSupport calling mlock() on loaded model data on Linux and macOS (#453)
11174870e455b3653f7d7769fa5772b2c90ffad088df483bab2e3d4a868fe679d8bb32827d2a4df214dc2023-03-24T00:11:53+02:00Georgi GerganovFix memory allocation issues and seg faults
1118483bab2e3d4a868fe679d8bb32827d2a4df214dc404e1da38ec8025707031a8027da14dc1590f9522023-03-23T23:22:01+02:00Georgi GerganovAvoid the transposed X branch in the Z = X * Y matrix multiplication (#439)
11192e17dfd80a473099dacc0f41c9146d233c6a597220a1a4e09c522a80e2a0db51643d25fa383260652023-03-23T15:22:47-05:00rabidcopyReplace EOS with newline to prevent context/memory being flushed by EOS in interactive mode (#333)
112056e659a0b271436e24813a801640d015e7b0532840ea807a972ec7b5a426f034ebfa593b5e7a06ed2023-03-22T17:09:38+01:00Erik Scholzfix perplexity after c-api refactor (#390)
1121928480ef5b7b03d7a07e98286aebe3d8b24457d956817b1f882b1894daa4051d0de0bf9a0926d3152023-03-22T07:45:00+02:00Georgi GerganovInit llama_context_params properly from CLI (#370)
112216ffc013c62f22bdaa3cdc022d7a13fd952d73fc486ae645fd3eda8b9d7413d5ff34fb65a3e337fb2023-03-21T09:42:25-07:00comexImporter for GPTQ quantized LLaMA models (#301)
1123486ae645fd3eda8b9d7413d5ff34fb65a3e337fb3ab3e6582f7320c2b6568c892fdfc8215caf7e6c2023-03-21T09:27:42-07:00Gary LinscottCompute perplexity over prompt (#270)
11243ab3e6582f7320c2b6568c892fdfc8215caf7e6cf157088cb75f23208abc92b473a132ef3f7a7f152023-03-21T18:23:15+02:00Jean-Christophe HoeltAdd chatLLaMa script (#198)
1125eb34620aeceaf9d9df7fcb19acc17ad41b9f60f82e664f1ff413995506c9a54f3a8d5b8c64e37a912023-03-21T17:29:41+02:00Georgi GerganovAdd tokenizer test + revert to C++11 (#355)
11265c19c70ba631a8f5d54feb6634e0eea178911a8424568371ae0d7caf85164abe4753f36a7dba02882023-03-19T13:44:30-06:00Rickey Bowers Jrfix coloring of last `n_batch` of prompt, and refactor line input (#221)
11270b366e735729327476ec31da02de3c9c9771ddfb160bfb217da5038ccbd74438f9f16a16012d78662023-03-19T18:57:00+01:00Erik ScholzCommand line switch to use F16 for memory_k and memory_v (refactor of #154) (#294)
1128d7def1a7524f712e5ebb7cd02bab0f13aa56a7f96f61c18ec9a30416e21ed5abfb1321bdb14979be2023-03-18T17:10:47-07:00RonsorWarn user if a context size greater than 2048 tokens is specified (#274)
1129554b54152145c30618bac171efb712cf4a7d1e96d3f202d57b694376cef6f381a6b6901825c3f6d92023-03-18T21:58:46+01:00Pavol RusnakAdd memory/disk requirements to readme
113063fd76fbb06f9b723ca11505352387a3148b18142a20f48efad692a8c2744f10c673bbdbe0c751b72023-03-14T01:33:43+09:00uint256_tReduce model loading time (#43)
1131d1f224712d78ab2cbb78777acfeb6739f660eb961808ee0500ea674b4bc2911acd0489ee5cbcef872023-03-13T17:15:20+01:00Pavol RusnakAdd quantize script for batch quantization (#92)
1132eb062bb012c4e131818dd757a6d3a757fdee39617027a97837c351e0a7bc48db2027af368de382db2023-03-12T17:15:00-03:00Sebastián AWindows fixes (#31)
11337d9ed7b25fe17db3fc8848b5116d14682864ce8e0c6803321c818f3f2da4a0693d20128b0f79ad282023-03-11T12:44:21+02:00Georgi GerganovBump memory buffer
Reference in New Issue View Git Blame Copy Permalink