fd9bd632f346085920d523dd307a3e4c11cc0a05 e08e473cf292ba73789769b993bb35bcdfc31689 wangbomeng wangbomeng@calculet.tech 2026-06-01T14:15:35+08:00 wangbomeng wangbomeng@calculet.tech 2026-06-01T14:15:35+08:00 HEAD -> dev, origin/dev llama-bench: fix device-info e08e473cf292ba73789769b993bb35bcdfc31689 f86112d6758298241ab2fa84ad4f0f7b1ace35c6 wangbomeng wangbomeng@calculet.tech 2026-06-01T11:47:01+08:00 wangbomeng wangbomeng@calculet.tech 2026-06-01T11:47:01+08:00 llama-bench:add device-info f86112d6758298241ab2fa84ad4f0f7b1ace35c6 d1bc743c10080fed1253686ce01749b42611ded4 wangbomeng wangbomeng@calculet.tech 2026-05-29T16:40:38+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-29T16:40:38+08:00 add dump_pos,fix pos_tensor of pre_by_embeds d1bc743c10080fed1253686ce01749b42611ded4 2f201160b1960fd7be18b70d5770599d7082dd2f wangbomeng wangbomeng@calculet.tech 2026-05-28T14:23:06+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-28T14:23:06+08:00 set u_batch = max_seq_len 2f201160b1960fd7be18b70d5770599d7082dd2f f131bf1908b8c5dd1f49d7ae7f616506fc471666 wangbomeng wangbomeng@calculet.tech 2026-05-27T09:58:31+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-27T09:58:31+08:00 delete useless code f131bf1908b8c5dd1f49d7ae7f616506fc471666 33d96dae28e17208ef61a71dba40cda3c04f135a wangbomeng wangbomeng@calculet.tech 2026-05-27T09:38:51+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-27T09:38:51+08:00 fix patch of qwen2.5vl,n_patch of minicpm = 64,fix n_ctx_slot,comment out M-ROPE ebce9e1a548c2329aa97f53b8b2ad50d968088dc e850ff4de25ad1221abc9bf3bb30df252ccd9c36 Yunzhe Jia yunzhe@calculet.tech 2026-05-27T09:38:00+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-27T09:38:00+08:00 origin/runtime_replace Refactor cal-llm integration by removing runtime capacity loading and adjusting request handling e850ff4de25ad1221abc9bf3bb30df252ccd9c36 9985688f7fa96f755882f96a2cd8a18114fdc42d Yunzhe Jia yunzhe@calculet.tech 2026-05-26T10:59:45+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-26T10:59:45+08:00 Enhance cal-llm integration by adding runtime checks for CALRT_LIBRARY and updating link settings for Linux and Windows 9985688f7fa96f755882f96a2cd8a18114fdc42d 38ca6895815671a87ceec86b4aa4eb976580ce76 Yunzhe Jia yunzhe@calculet.tech 2026-05-25T09:05:49+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-25T09:05:49+08:00 Refactor CalrtEngineConfig initialization by removing redundant parameters 33d96dae28e17208ef61a71dba40cda3c04f135a 37f068d87cc2d0a0b40ce978031fc9932fc89077 wangbomeng wangbomeng@calculet.tech 2026-05-22T14:33:45+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-22T14:33:45+08:00 add llama-build.sh 1921024604db640f09ade83c8a437ebf15bde360 d028722a12c5a463cc97207787cfd03d7a2590b0 5af229fe518ee2d761ab8ae2a69f7a9aaa9057f4 Bomeng Wang wangbomeng@calculet.tech 2026-05-21T14:08:14+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-05-21T14:08:14+08:00 refs/stash WIP on dev: d028722a replace cparam.n_ubatch with n_ubatch() 5af229fe518ee2d761ab8ae2a69f7a9aaa9057f4 d028722a12c5a463cc97207787cfd03d7a2590b0 Bomeng Wang wangbomeng@calculet.tech 2026-05-21T14:08:14+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-05-21T14:08:14+08:00 index on dev: d028722a replace cparam.n_ubatch with n_ubatch() 37f068d87cc2d0a0b40ce978031fc9932fc89077 13508e5e18bbb1202ae60c2ee5ecbdda1cca817b wangbomeng wangbomeng@calculet.tech 2026-05-21T14:07:00+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-21T14:07:00+08:00 release 13508e5e18bbb1202ae60c2ee5ecbdda1cca817b b6b7919468126d5ba30a10941825154789e9082f wangbomeng wangbomeng@calculet.tech 2026-05-21T11:43:59+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-21T11:43:59+08:00 reduce cmake log b6b7919468126d5ba30a10941825154789e9082f 46847c24fc5c52dd57bb4fb17fc4d684149bb4fa wangbomeng wangbomeng@calculet.tech 2026-05-20T16:40:33+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-20T16:40:33+08:00 reduce cmake log 38ca6895815671a87ceec86b4aa4eb976580ce76 ddd60c928224b52eb4e1e60e01e8a1ecbb80ee45 Yunzhe Jia yunzhe@calculet.tech 2026-05-20T16:37:13+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-20T16:37:13+08:00 Add support for cal-llm profile dumping to JSONL file 46847c24fc5c52dd57bb4fb17fc4d684149bb4fa d028722a12c5a463cc97207787cfd03d7a2590b0 wangbomeng wangbomeng@calculet.tech 2026-05-20T16:02:12+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-20T16:02:12+08:00 Simplify CMakeLists.txt d028722a12c5a463cc97207787cfd03d7a2590b0 63442fde8dc285817f725bd6c5fae80f478a3813 wangbomeng wangbomeng@calculet.tech 2026-05-20T14:41:05+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-20T14:41:05+08:00 replace cparam.n_ubatch with n_ubatch() 63442fde8dc285817f725bd6c5fae80f478a3813 fc3f4a9fab88e98eff8eeca8cbc6617333edba2c wangbomeng wangbomeng@calculet.tech 2026-05-20T11:47:42+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-20T11:47:42+08:00 fix ubatch and cmakelist ddd60c928224b52eb4e1e60e01e8a1ecbb80ee45 2716130010c58f1cdd9d94f0e2f8e047bc92dc66 Yunzhe Jia yunzhe@calculet.tech 2026-05-20T09:22:40+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-20T09:27:06+08:00 Add cal-llm backend profiling support with command-line option fc3f4a9fab88e98eff8eeca8cbc6617333edba2c c931ef3163940227c9b6291e04216245cce21429 wangbomeng wangbomeng@calculet.tech 2026-05-19T16:54:16+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-19T16:54:16+08:00 add dump and rt case generation c931ef3163940227c9b6291e04216245cce21429 a43741244a646d3f8fa3ff01bb9b7d059223d0a5 wangbomeng wangbomeng@calculet.tech 2026-05-18T17:34:37+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-18T17:34:37+08:00 calrt: fix prefill_by_ids.fix create_buf 2716130010c58f1cdd9d94f0e2f8e047bc92dc66 b35bda124ccd4ed581dd6088d3ffa4931c2809b6 Yunzhe Jia yunzhe@calculet.tech 2026-05-18T08:40:54+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-18T08:40:54+08:00 fix n_ctx_slot error by adding runtime capacity loading for cal-llm a43741244a646d3f8fa3ff01bb9b7d059223d0a5 b2003d6c90ef0fa26ab9e930ba3e22b355eb3d39 wangbomeng wangbomeng@calculet.tech 2026-05-15T14:39:53+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-15T14:39:53+08:00 run llama-passkey successfully, fix llama-bench, copy 3 times input to infer qwen2.5vl b35bda124ccd4ed581dd6088d3ffa4931c2809b6 81a2cbd4cef5d28334ff513d6b69e593d9d1cb49 Yunzhe Jia yunzhe@calculet.tech 2026-05-15T09:19:31+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-15T09:19:31+08:00 Implement vocab-only model initialization and enhance cal-llm backend error handling 81a2cbd4cef5d28334ff513d6b69e593d9d1cb49 3b14ed556c15529fe1f94b649f49db7156dfaf67 Yunzhe Jia yunzhe@calculet.tech 2026-05-14T16:06:37+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-14T16:06:37+08:00 Enhance CMake configuration for cal-llm integration by updating paths and adding library properties 3b14ed556c15529fe1f94b649f49db7156dfaf67 dde8b8228081769c8d8f1e0751d47fa7875f8423 Yunzhe Jia yunzhe@calculet.tech 2026-05-14T14:34:25+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-05-14T14:34:25+08:00 - Removed references to CALRT from clip.h, mtmd-helper.cpp, mtmd-helper.h, mtmd.cpp, and mtmd.h. - Updated server.cpp to conditionally include CAL-LLM support. - Introduced option LLAMA_SERVER_USE_CAL_LLM in CMakeLists.txt to enable CAL-LLM integration. - Refactored server context to handle CAL-LLM requests and responses. - Updated image processing logic to remove CALRT-specific implementations. - Ensured compatibility with existing llama functionality while integrating new CAL-LLM features. b2003d6c90ef0fa26ab9e930ba3e22b355eb3d39 7800c772d44a7415640ec1669a691232939927cf wangbomeng wangbomeng@calculet.tech 2026-05-13T15:24:24+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-13T15:24:24+08:00 try passkey and embedding 7800c772d44a7415640ec1669a691232939927cf dde8b8228081769c8d8f1e0751d47fa7875f8423 wangbomeng wangbomeng@calculet.tech 2026-05-13T15:23:54+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-13T15:23:54+08:00 try passkey and embedding dde8b8228081769c8d8f1e0751d47fa7875f8423 fabcc7dac4b34d83fe430980bb364b81087c7ee9 wangbomeng wangbomeng@calculet.tech 2026-05-12T10:46:08+08:00 wangbomeng wangbomeng@calculet.tech 2026-05-12T10:46:08+08:00 clip:add minicpm patch; fabcc7dac4b34d83fe430980bb364b81087c7ee9 1e9787962f20a7e82c71589ac1dd0585454e4bfe wangbomeng wangbomeng@calculet.tech 2026-04-30T16:17:17+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-30T16:17:17+08:00 calrt: fix encode dump 1e9787962f20a7e82c71589ac1dd0585454e4bfe f584311c716dd078c4b737eebbfda97fe12b7274 1bec0db5a675ddc60fb793be2a746e8f3c8855dc wangbomeng wangbomeng@calculet.tech 2026-04-30T16:06:23+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-30T16:06:23+08:00 Merge remote-tracking branch 'origin/dev-ben' into dev f584311c716dd078c4b737eebbfda97fe12b7274 a339954cc2a145a80c5ea349e7896211916cbed9 wangbomeng wangbomeng@calculet.tech 2026-04-30T16:05:15+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-30T16:05:15+08:00 calrt: fix encode dump 1bec0db5a675ddc60fb793be2a746e8f3c8855dc 465df20c3e1f3f58d0f5531c796e0941a49c32b0 wangbomeng wangbomeng@calculet.tech 2026-04-30T16:02:59+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-30T16:02:59+08:00 origin/dev-ben bench: support 2 calbins 465df20c3e1f3f58d0f5531c796e0941a49c32b0 a339954cc2a145a80c5ea349e7896211916cbed9 wangbomeng wangbomeng@calculet.tech 2026-04-30T09:08:12+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-30T09:08:12+08:00 support one calbin a339954cc2a145a80c5ea349e7896211916cbed9 8e76fc330ad624d1768b412d7894e272dbe696f4 eeeef3f6d6a5aad9296323ca15d9d929745b8932 wangbomeng wangbomeng@calculet.tech 2026-04-29T14:57:55+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-29T14:57:55+08:00 tag: v0.3.0 Merge branch 'dev' into dev-cli 8e76fc330ad624d1768b412d7894e272dbe696f4 431ad6f1c787f1b47e9e7be14b2e1a7d8565485c wangbomeng wangbomeng@calculet.tech 2026-04-29T14:56:59+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-29T14:56:59+08:00 finish dev of llama-cli eeeef3f6d6a5aad9296323ca15d9d929745b8932 431ad6f1c787f1b47e9e7be14b2e1a7d8565485c wangbomeng wangbomeng@calculet.tech 2026-04-28T16:16:19+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-28T16:16:19+08:00 add unknown time info: prefill and decode 431ad6f1c787f1b47e9e7be14b2e1a7d8565485c efa1876bb8b2a449a55054a8c3745892f2c0f262 wangbomeng wangbomeng@calculet.tech 2026-04-27T16:52:15+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-27T16:52:15+08:00 update calrt_infer efa1876bb8b2a449a55054a8c3745892f2c0f262 4a2a3181f1cea170105c771e6ba69d851b82b937 wangbomeng wangbomeng@calculet.tech 2026-04-27T11:35:58+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-27T11:35:58+08:00 tag: v0.2.2 update version print 4a2a3181f1cea170105c771e6ba69d851b82b937 e3ea5541259f35871c3e995331770aaca297e9ec wangbomeng wangbomeng@calculet.tech 2026-04-26T10:56:02+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-26T10:56:02+08:00 add calrt-version e3ea5541259f35871c3e995331770aaca297e9ec 0b8e43943846a1cd05c26c9186b9ef778ebae6e2 wangbomeng wangbomeng@calculet.tech 2026-04-26T10:55:48+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-26T10:55:48+08:00 add calrt-version 0b8e43943846a1cd05c26c9186b9ef778ebae6e2 5eb46fffbd2e260c540fb112314ad67bbb8f56b6 wangbomeng wangbomeng@calculet.tech 2026-04-26T08:43:43+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-26T08:43:43+08:00 to debug 5eb46fffbd2e260c540fb112314ad67bbb8f56b6 005112c51e00d998b457337ca3dad53fc46e2d0b wangbomeng wangbomeng@calculet.tech 2026-04-24T16:24:51+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-24T16:24:51+08:00 clip: add image_to_patches; calrt: vision copy data from bf16 to fp32 005112c51e00d998b457337ca3dad53fc46e2d0b 509670642e8090a1713f5d73590b549be827aaef wangbomeng wangbomeng@calculet.tech 2026-04-24T16:24:31+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-24T16:24:31+08:00 clip: add image_to_patches; calrt: vision copy data from bf16 to fp32 509670642e8090a1713f5d73590b549be827aaef 5d219bfcef239c1e3ddd1baecc0514a9462ff6a8 wangbomeng wangbomeng@calculet.tech 2026-04-24T09:40:34+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-24T09:40:34+08:00 rm debug code 5d219bfcef239c1e3ddd1baecc0514a9462ff6a8 6787a53ab9b4897c5a7b51a4062d7b90e6697d35 wangbomeng wangbomeng@calculet.tech 2026-04-24T09:31:32+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-24T09:31:32+08:00 calrt: recover base = batch_index * dict_len 6787a53ab9b4897c5a7b51a4062d7b90e6697d35 e1a76abf66178ea85656479a5e19aea7785c09f4 wangbomeng wangbomeng@calculet.tech 2026-04-24T09:22:02+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-24T09:22:02+08:00 calrt:input of multimodel from fp32 to bf16 e1a76abf66178ea85656479a5e19aea7785c09f4 99dd308adb8488fa869bb669e3335e646a938eff wangbomeng wangbomeng@calculet.tech 2026-04-22T16:29:22+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-22T16:29:22+08:00 calrt:add CalcoreRT version print 99dd308adb8488fa869bb669e3335e646a938eff 06c7852e99e34c71e24e3ef6001cb54c72970e4d wangbomeng wangbomeng@calculet.tech 2026-04-21T15:50:01+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-21T15:50:01+08:00 tag: v0.2.1 server: fix max_seq_len 06c7852e99e34c71e24e3ef6001cb54c72970e4d 96b4e267e562f44e76ec6c965ee0fa361f0439bf ed62eb33447f993d578e156a3c7e38c91b420ece wangbomeng wangbomeng@calculet.tech 2026-04-21T15:06:01+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-21T15:06:01+08:00 Merge branch 'dev-ot' into dev ed62eb33447f993d578e156a3c7e38c91b420ece 015e988a271573a7c11e2a4b45084e1219d69d50 wangbomeng wangbomeng@calculet.tech 2026-04-21T15:05:22+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-21T15:05:22+08:00 try server-test:not success 015e988a271573a7c11e2a4b45084e1219d69d50 99962021f2dd1994dbbf90d5712f0ca2633f20f8 wangbomeng wangbomeng@calculet.tech 2026-04-17T16:56:08+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-17T16:56:08+08:00 little change 96b4e267e562f44e76ec6c965ee0fa361f0439bf 99962021f2dd1994dbbf90d5712f0ca2633f20f8 wangbomeng wangbomeng@calculet.tech 2026-04-17T15:33:01+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-17T15:33:01+08:00 merge dev-ot 99962021f2dd1994dbbf90d5712f0ca2633f20f8 9af6682ef101c6d006c743c703cf150ac25f466f wangbomeng wangbomeng@calculet.tech 2026-04-17T14:48:33+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-17T14:48:33+08:00 origin/dev-ot calrt: add onetoken slice and untile_prefill 9af6682ef101c6d006c743c703cf150ac25f466f e24f775ee26ea5249d5ff6cd23b8a0a17fe1438d wangbomeng wangbomeng@calculet.tech 2026-04-16T10:09:44+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-16T10:09:44+08:00 args: add -ca e24f775ee26ea5249d5ff6cd23b8a0a17fe1438d a206460e610aef995e36e50b33e8bd41fe477541 wangbomeng wangbomeng@calculet.tech 2026-04-15T10:01:09+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-15T10:01:09+08:00 calrt: add copy_data_to_ibuf for vision model 795179beb5bba728e0ccf0a7e80a5ec5ef36e92a 6d55eae7e76b768acfcec045b8e84cded8ae3b55 Yunzhe Jia yunzhe@calculet.tech 2026-04-14T16:03:22+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-04-14T16:03:22+08:00 origin/test-cicd add ci yaml a206460e610aef995e36e50b33e8bd41fe477541 d8b2aaa00049978cebac15041f960d75b552f97b wangbomeng wangbomeng@calculet.tech 2026-04-14T10:23:43+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-14T10:23:43+08:00 calrt: support mixture of prefill_by_ids and prefill_by_embeds d8b2aaa00049978cebac15041f960d75b552f97b b6f29ec8140028619efaa50aa6e73146cecf631d wangbomeng wangbomeng@calculet.tech 2026-04-13T11:23:44+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-13T11:23:44+08:00 server: fix context-shift b6f29ec8140028619efaa50aa6e73146cecf631d 6d55eae7e76b768acfcec045b8e84cded8ae3b55 wangbomeng wangbomeng@calculet.tech 2026-04-13T11:23:31+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-13T11:23:31+08:00 server: fix context-shift 6d55eae7e76b768acfcec045b8e84cded8ae3b55 af8055f5f033a730e7e1fe80185b18b7e9473966 wangbomeng wangbomeng@calculet.tech 2026-04-10T14:58:56+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-10T14:58:56+08:00 tag: v0.2.0 add annotations af8055f5f033a730e7e1fe80185b18b7e9473966 6f54682df62b8ec6bb4154b99c47b5becda3291f wangbomeng wangbomeng@calculet.tech 2026-04-10T14:58:19+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-10T14:58:19+08:00 add annotations 6f54682df62b8ec6bb4154b99c47b5becda3291f 42a181674565411efa5c8b318bc77d6fd084df8a a2d5bf362d6c9a546f9deadf4f8975605a915d33 Yunzhe Jia yunzhe@calculet.tech 2026-04-10T09:07:20+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-04-10T09:07:20+08:00 Merge branch 'dev' of http://192.168.10.26:1000/yunzhe/llama.cpp into dev 42a181674565411efa5c8b318bc77d6fd084df8a bd3a57a11aa008e0b5dc2a30f82fcd3a02587208 Yunzhe Jia yunzhe@calculet.tech 2026-04-10T09:06:55+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-04-10T09:06:55+08:00 fix prompt_cache issue a2d5bf362d6c9a546f9deadf4f8975605a915d33 bd3a57a11aa008e0b5dc2a30f82fcd3a02587208 Bomeng Wang wangbomeng@calculet.tech 2026-04-09T21:01:46+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-04-09T21:01:46+08:00 server: comment fixed time bd3a57a11aa008e0b5dc2a30f82fcd3a02587208 fcfcdf80b01664a08fb12df879a05df3cc0eba70 74d437ab1456831573def7ca385a74d0ca460c37 Yunzhe Jia yunzhe@calculet.tech 2026-04-09T19:42:57+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-04-09T19:42:57+08:00 Merge branch 'dev' of http://192.168.10.26:1000/yunzhe/llama.cpp into dev fcfcdf80b01664a08fb12df879a05df3cc0eba70 0cd44929b442860dc0e5f88976108be82350f9dc Yunzhe Jia yunzhe@calculet.tech 2026-04-09T19:41:52+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-04-09T19:41:52+08:00 tmp fix one run >4K problem 74d437ab1456831573def7ca385a74d0ca460c37 5813c943005d1ede551e67ee98e35aefd210ff22 0cd44929b442860dc0e5f88976108be82350f9dc wangbomeng wangbomeng@calculet.tech 2026-04-09T17:44:26+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-09T17:44:26+08:00 Merge branch 'dev' of http://192.168.10.26:1000/yunzhe/llama.cpp into dev 5813c943005d1ede551e67ee98e35aefd210ff22 1f0d5bb0453f93ffa51faf68237c89251c2e0c40 wangbomeng wangbomeng@calculet.tech 2026-04-09T17:38:57+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-09T17:38:57+08:00 add --device-info 1f0d5bb0453f93ffa51faf68237c89251c2e0c40 b8153b6b8f244b223c9f4c2940ae1f212634519e wangbomeng wangbomeng@calculet.tech 2026-04-09T17:38:37+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-09T17:38:37+08:00 add --device-info 0cd44929b442860dc0e5f88976108be82350f9dc b8153b6b8f244b223c9f4c2940ae1f212634519e Bomeng Wang wangbomeng@calculet.tech 2026-04-09T15:08:50+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-04-09T15:08:50+08:00 rm calculet0.txt b8153b6b8f244b223c9f4c2940ae1f212634519e 82842cfc3fa3d5c004c4540aeff8a81f38509bee wangbomeng wangbomeng@calculet.tech 2026-04-09T15:07:02+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-09T15:07:02+08:00 server: fix limit tokens to max_seq_len 82842cfc3fa3d5c004c4540aeff8a81f38509bee 622a22991089c6debd5f4b57738f3df3a0bda8b5 wangbomeng wangbomeng@calculet.tech 2026-04-09T15:06:26+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-09T15:06:26+08:00 server: fix limit tokens to max_seq_len 622a22991089c6debd5f4b57738f3df3a0bda8b5 5f47a738ba56a37d0ff281a16212f41979568e35 wangbomeng wangbomeng@calculet.tech 2026-04-09T10:36:31+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-09T10:36:31+08:00 server: comment circle print 5f47a738ba56a37d0ff281a16212f41979568e35 ecf01a17651cd7b1e8b85fd6075e83831463ee23 207ca38c751f7e6c251833eb11e89a2a0bb0fd07 Yunzhe Jia yunzhe@calculet.tech 2026-04-09T09:48:20+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-04-09T09:48:20+08:00 Merge branch 'dev' of http://192.168.10.26:1000/yunzhe/llama.cpp into dev ecf01a17651cd7b1e8b85fd6075e83831463ee23 99d2078e89305035d87d966cee7987c7d50b3925 Yunzhe Jia yunzhe@calculet.tech 2026-04-09T09:47:55+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-04-09T09:47:55+08:00 fix n_parallel problem 207ca38c751f7e6c251833eb11e89a2a0bb0fd07 99d2078e89305035d87d966cee7987c7d50b3925 wangbomeng wangbomeng@calculet.tech 2026-04-08T19:14:37+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-08T19:14:37+08:00 delet extra printf 99d2078e89305035d87d966cee7987c7d50b3925 9626239f5a9e568c9975d67dd6745fef90279a87 Yunzhe Jia yunzhe@calculet.tech 2026-04-08T19:01:16+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-04-08T19:01:16+08:00 fix buffer resize problem 9626239f5a9e568c9975d67dd6745fef90279a87 da724f3e2a8af2c2537e3edcff41e9415ac66fef wangbomeng wangbomeng@calculet.tech 2026-04-08T17:47:04+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-08T17:47:04+08:00 calrt: fix MapBuf da724f3e2a8af2c2537e3edcff41e9415ac66fef 18814f9bf1a423ebe8f32d6043c9cbc5b21b0218 Bomeng Wang wangbomeng@calculet.tech 2026-04-08T16:58:32+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-04-08T16:58:32+08:00 calrt: add MapBuf 18814f9bf1a423ebe8f32d6043c9cbc5b21b0218 bbee06d6d851e3f84f75b578047967d7e0e4a8dc wangbomeng wangbomeng@calculet.tech 2026-04-08T13:55:52+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-08T13:55:52+08:00 server : reset inference time bbee06d6d851e3f84f75b578047967d7e0e4a8dc e126c21ed7b793b648d63533ea7ee30885f5a82a wangbomeng wangbomeng@calculet.tech 2026-04-08T11:29:36+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-08T11:29:36+08:00 calrt: add test time print 39edc6ec2e9560ccb20a2b8e547f58102bb268c1 2006831fd6ea8a8e16518b81bcee29fb674676ea Bomeng Wang wangbomeng@calculet.tech 2026-04-01T14:06:50+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-04-01T14:06:50+08:00 tag: v0.1.1 add t_incopy t_outcopy 2006831fd6ea8a8e16518b81bcee29fb674676ea 41e6636ea1fb3884e41fb2023f92b0e7262ef09b wangbomeng wangbomeng@calculet.tech 2026-04-01T10:42:27+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-01T10:42:27+08:00 tag: v0.1.0 v0.1.0 e126c21ed7b793b648d63533ea7ee30885f5a82a 5603e050af07173589f3b8850121263d86da01fd wangbomeng wangbomeng@calculet.tech 2026-04-01T10:07:12+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-01T10:07:12+08:00 sever: fix commit id 5603e050af07173589f3b8850121263d86da01fd 1e8a16cd0e3ad3c9a0f8746852a0a38c6e52322e wangbomeng wangbomeng@calculet.tech 2026-04-01T05:57:55+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-01T05:57:55+08:00 calrt: add t_incopy and t_outcopy 1e8a16cd0e3ad3c9a0f8746852a0a38c6e52322e 893e52bda0fee99bbda1521ea733a760bc4201f1 wangbomeng wangbomeng@calculet.tech 2026-04-01T10:14:22+08:00 wangbomeng wangbomeng@calculet.tech 2026-04-01T10:14:22+08:00 calrt: fix slice and add infer/copy time 893e52bda0fee99bbda1521ea733a760bc4201f1 398ecf71f5f574037ce33f84dd1576fa164909ad Bomeng Wang wangbomeng@calculet.tech 2026-03-30T11:35:40+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-03-30T11:35:40+08:00 delete unnecessary commit id 398ecf71f5f574037ce33f84dd1576fa164909ad 6d5e83902bdd3142a8ff0d3db19f02a8619e95a0 wangbomeng wangbomeng@calculet.tech 2026-03-27T02:01:23+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-27T02:01:23+08:00 rm llama-server_old.cpp 6d5e83902bdd3142a8ff0d3db19f02a8619e95a0 e4eaab700333490e80ebefc8d023f6c1adfcc312 wangbomeng wangbomeng@calculet.tech 2026-03-27T00:57:11+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-27T00:57:11+08:00 server: printf calrt commit ID e4eaab700333490e80ebefc8d023f6c1adfcc312 aeacc23a883400db0ebffb50e23355ada054da66 wangbomeng wangbomeng@calculet.tech 2026-03-25T09:03:55+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-25T09:03:55+08:00 calrt: prefil to prefill aeacc23a883400db0ebffb50e23355ada054da66 cbaa7492663630132adeddccd9fec5e44ffa3336 wangbomeng wangbomeng@calculet.tech 2026-03-25T08:49:22+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-25T08:49:22+08:00 calrt: add slice only on decode cbaa7492663630132adeddccd9fec5e44ffa3336 3c08ebf0e442cd730ddffd959ec676f9caf31c82 wangbomeng wangbomeng@calculet.tech 2026-03-25T03:21:02+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-25T03:21:02+08:00 reduce warning 3c08ebf0e442cd730ddffd959ec676f9caf31c82 613bc2b9a9d7af92efbe42d876ffcdb244f7a00e wangbomeng wangbomeng@calculet.tech 2026-03-24T02:43:03+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-24T02:43:03+08:00 calrt: comment LOG_ERROR in untile_decode_cpy 613bc2b9a9d7af92efbe42d876ffcdb244f7a00e 4e194ecc6c4b8ebe4c72d93f6055e3605dbfcefa wangbomeng wangbomeng@calculet.tech 2026-03-24T02:30:12+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-24T02:30:12+08:00 calrt: fix max_seq_len judgment 4e194ecc6c4b8ebe4c72d93f6055e3605dbfcefa 16cbf81a731f5839a2f6b0eb9d8539826353748b wangbomeng wangbomeng@calculet.tech 2026-03-24T01:59:03+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-24T01:59:03+08:00 calrt: fix ubatch.embd cpy 16cbf81a731f5839a2f6b0eb9d8539826353748b 8b4e17d990c23025148c4abadefe1010a0dd0734 wangbomeng wangbomeng@calculet.tech 2026-03-23T09:38:21+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-23T09:38:21+08:00 calrt: fix model_name of untile_cpy 8b4e17d990c23025148c4abadefe1010a0dd0734 6dacdd2cf9574214ca272a86674e7bd7ba21c4f0 41e6636ea1fb3884e41fb2023f92b0e7262ef09b wangbomeng wangbomeng@calculet.tech 2026-03-23T14:50:27+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-23T14:50:27+08:00 merge origin dev 6dacdd2cf9574214ca272a86674e7bd7ba21c4f0 3e6ad64467771224f072e5bad90e270951ba4cdf wangbomeng wangbomeng@calculet.tech 2026-03-23T14:42:28+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-23T14:42:28+08:00 calrt: add multimodal 3e6ad64467771224f072e5bad90e270951ba4cdf b37b7d9756feb3dac2db526eeab38b572a095d47 wangbomeng wangbomeng@calculet.tech 2026-03-23T14:36:41+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-23T14:36:41+08:00 calrt: add multimodal 41e6636ea1fb3884e41fb2023f92b0e7262ef09b d0a1b2c758440720d42fa108b3444f54593daa73 Bomeng Wang wangbomeng@calculet.tech 2026-03-12T16:36:07+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-03-12T16:36:07+08:00 tag: v0.0.1 server: correct the max_seq_len judgment d0a1b2c758440720d42fa108b3444f54593daa73 b37b7d9756feb3dac2db526eeab38b572a095d47 Bomeng Wang wangbomeng@calculet.tech 2026-03-12T16:10:28+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-03-12T16:10:28+08:00 calrt: comment out dump b37b7d9756feb3dac2db526eeab38b572a095d47 07817cefc14fa359dcecad0630defd3610f8f5c8 7ddafbdcb9426ae3af016925b8b596f11e8742d0 Yunzhe Jia yunzhe@calculet.tech 2026-03-10T16:32:31+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-03-10T16:32:31+08:00 Merge branch 'dev' of http://192.168.10.26:1000/yunzhe/llama.cpp into dev 07817cefc14fa359dcecad0630defd3610f8f5c8 b2c45d8ddefb7ec09435231c8ec5b736fabf9c91 Yunzhe Jia yunzhe@calculet.tech 2026-03-10T16:32:07+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-03-10T16:32:07+08:00 fix kv issue 7ddafbdcb9426ae3af016925b8b596f11e8742d0 059009eeaf20a569abfbac5eb37dad3bb9376428 wangbomeng wangbomeng@calculet.tech 2026-03-10T07:13:04+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-10T07:13:04+08:00 calrt: commented out cal_ctx->encode(batch) 059009eeaf20a569abfbac5eb37dad3bb9376428 1b073661d1311b5300173993b9f31ee7241d8606 wangbomeng wangbomeng@calculet.tech 2026-03-10T07:04:22+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-10T07:04:22+08:00 calrt: add multimodel 1b073661d1311b5300173993b9f31ee7241d8606 6295273c5866b9249bacdedbfc3a31877e6e6a85 b2c45d8ddefb7ec09435231c8ec5b736fabf9c91 wangbomeng wangbomeng@calculet.tech 2026-03-10T06:58:35+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-10T06:58:35+08:00 Merge branch 'dev' of http://192.168.20.229:1000/yunzhe/llama.cpp into dev 6295273c5866b9249bacdedbfc3a31877e6e6a85 c6ea3f0d2468e3b8eab5de80bacee7fe4c9dd3d7 wangbomeng wangbomeng@calculet.tech 2026-03-10T06:54:41+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-10T06:54:41+08:00 arg: update hf_repo url to https://download.calculet.tech:9443 b2c45d8ddefb7ec09435231c8ec5b736fabf9c91 c6ea3f0d2468e3b8eab5de80bacee7fe4c9dd3d7 Bomeng Wang wangbomeng@calculet.tech 2026-03-09T15:03:26+08:00 Bomeng Wang wangbomeng@calculet.tech 2026-03-09T15:03:26+08:00 clart: past_kv_len = n_tokens c6ea3f0d2468e3b8eab5de80bacee7fe4c9dd3d7 f8fba66b657c51a1df1efc7267bfea9f6140cebf wangbomeng wangbomeng@calculet.tech 2026-03-09T03:26:47+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-09T03:26:47+08:00 CMakeLists: STATIC to SHARED f8fba66b657c51a1df1efc7267bfea9f6140cebf db4a61d2234c2f457b42ecc3f7219a1e1a35508a Yunzhe Jia yunzhe@calculet.tech 2026-03-06T15:06:36+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-03-06T07:05:39+08:00 fix calrt_install include path db4a61d2234c2f457b42ecc3f7219a1e1a35508a e672e02ca1a582f9e3a9d011d9e1b4fea4942a25 Yunzhe Jia yunzhe@calculet.tech 2026-03-06T06:59:58+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-03-06T06:59:58+08:00 adapt to host-rt install structure e672e02ca1a582f9e3a9d011d9e1b4fea4942a25 27b82f64252779ad160a3fa76bd378c39421de36 7a3a9a0e76bb5f530beafcfe52e87e388e93117a Yunzhe Jia yunzhe@calculet.tech 2026-03-06T11:47:09+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-03-06T11:47:09+08:00 Merge branch 'dev' of http://192.168.10.26:1000/yunzhe/llama.cpp into dev 27b82f64252779ad160a3fa76bd378c39421de36 0bd95719fed2f0247c8d0199dd897d24157dfae6 Yunzhe Jia yunzhe@calculet.tech 2026-03-06T11:42:45+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-03-06T11:45:03+08:00 adapt to calrt_objs target 7a3a9a0e76bb5f530beafcfe52e87e388e93117a 583c387efaf7bc030574e554088de79d09a27fbd wangbomeng wangbomeng@calculet.tech 2026-03-05T07:24:59+08:00 wangbomeng wangbomeng@calculet.tech 2026-03-05T07:24:59+08:00 calrt_infer: fix past_kv_len 583c387efaf7bc030574e554088de79d09a27fbd 0edc14a60a2d5596f71d728c655e3523da924d89 wangbomeng wangbomeng@calculet.tech 2026-02-27T02:24:56+08:00 wangbomeng wangbomeng@calculet.tech 2026-02-27T02:24:56+08:00 caserver: lim generated tokens to n_ctx_per_seq 0edc14a60a2d5596f71d728c655e3523da924d89 aa0a17d719326a63e0d6fea60aa0ced44e7abc33 wangbomeng wangbomeng@calculet.tech 2026-02-27T01:57:39+08:00 wangbomeng wangbomeng@calculet.tech 2026-02-27T01:57:39+08:00 caserver: limit generated tokens to n_ctx_per_seq aa0a17d719326a63e0d6fea60aa0ced44e7abc33 fa332e773da681f3e77d6ffe83a87edb557f758b wangbomeng wangbomeng@calculet.tech 2026-02-27T01:40:11+08:00 wangbomeng wangbomeng@calculet.tech 2026-02-27T01:40:11+08:00 calrt: add bf16_to_fp32 fa332e773da681f3e77d6ffe83a87edb557f758b aa54a7c637f2c92d924a5f35386975b1c27de898 wangbomeng wangbomeng@calculet.tech 2026-02-27T01:31:42+08:00 wangbomeng wangbomeng@calculet.tech 2026-02-27T01:31:42+08:00 Add bf16_to_fp32 aa54a7c637f2c92d924a5f35386975b1c27de898 84f2c0b5f334a7ab77347c6779b6027a951d0ff0 wangbomeng wangbomeng@calculet.tech 2026-02-27T01:30:36+08:00 wangbomeng wangbomeng@calculet.tech 2026-02-27T01:30:36+08:00 Add bf16_to_fp32 84f2c0b5f334a7ab77347c6779b6027a951d0ff0 0bd95719fed2f0247c8d0199dd897d24157dfae6 wangbomeng wangbomeng@calculet.tech 2026-02-09T01:46:36+08:00 wangbomeng wangbomeng@calculet.tech 2026-02-09T01:46:36+08:00 add bf16_to_fp32 in untile_decode/prefill_cpy 0bd95719fed2f0247c8d0199dd897d24157dfae6 365eb0b719e30b0f9e348d854f11c9c3f954a96e Yunzhe Jia yunzhe@calculet.tech 2026-02-07T10:25:06+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-02-07T10:25:06+08:00 fix get_model_by_name 365eb0b719e30b0f9e348d854f11c9c3f954a96e 886cd1fb3b217efcf51c46209fcb694022346797 Yunzhe Jia yunzhe@calculet.tech 2026-02-07T09:05:17+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-02-07T09:05:17+08:00 comment out pos buffer file 886cd1fb3b217efcf51c46209fcb694022346797 9328f21378275f2354a19c3af907332b216ab492 Yunzhe Jia yunzhe@calculet.tech 2026-02-06T17:36:54+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-02-06T17:36:54+08:00 fix compile problem 9328f21378275f2354a19c3af907332b216ab492 49fea70a20f550bfdb59d0ffe041b54919c22447 Yunzhe Jia yunzhe@calculet.tech 2026-01-28T15:32:36+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-01-28T15:32:36+08:00 add prefill-only mode 49fea70a20f550bfdb59d0ffe041b54919c22447 74a118df31b6a4acc8876b2a2db90d2395cc6ec3 Yunzhe Jia yunzhe@calculet.tech 2026-01-28T09:11:16+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-01-28T09:11:16+08:00 adapt to new calbin test_case 74a118df31b6a4acc8876b2a2db90d2395cc6ec3 98e8f9acfe87cd93646482cb2d942b8a132f0852 Yunzhe Jia yunzhe@calculet.tech 2026-01-13T14:32:47+08:00 Yunzhe Jia yunzhe@calculet.tech 2026-01-13T14:32:47+08:00 sync with calrt update 98e8f9acfe87cd93646482cb2d942b8a132f0852 8c8152a04036bb0d4756a4e08e54f514dae4e64d Yunzhe Jia yunzhe@calculet.tech 2025-12-10T00:20:57+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-12-10T00:20:57+08:00 add timestamp 8c8152a04036bb0d4756a4e08e54f514dae4e64d 5d2387931528fbe04aa8d66de935147efb65ca4d Yunzhe Jia yunzhe@calculet.tech 2025-12-09T14:40:04+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-12-09T14:40:28+08:00 adapt to calrt csr 5d2387931528fbe04aa8d66de935147efb65ca4d e6285a748657add8d974b78ca920c5b41753ee12 Yunzhe Jia yunzhe@calculet.tech 2025-12-08T16:40:34+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-12-08T16:40:54+08:00 add timestamp for one decode process e6285a748657add8d974b78ca920c5b41753ee12 6b1ecdbbe3cc72ed8888be38f9e934cfd79b9fbb Yunzhe Jia yunzhe@calculet.tech 2025-11-27T16:58:35+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-27T16:58:35+08:00 adapt to calrt 6b1ecdbbe3cc72ed8888be38f9e934cfd79b9fbb 240d9bb75241c91896afe125f2fc74698a7395f3 Yunzhe Jia yunzhe@calculet.tech 2025-11-25T17:06:14+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-25T17:06:14+08:00 add kv_tensor for pld test 240d9bb75241c91896afe125f2fc74698a7395f3 7eeb241aad429a68fb8cfa3ba85d000fb2348cd2 Yunzhe Jia yunzhe@calculet.tech 2025-11-24T12:03:00+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-24T12:03:00+08:00 adapt to calrt modification 7eeb241aad429a68fb8cfa3ba85d000fb2348cd2 dfb6250031a25058dbc3757e8a7ec75d90dcb48a Yunzhe Jia yunzhe@calculet.tech 2025-11-21T09:20:01+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-21T09:20:01+08:00 fix byte_size error dfb6250031a25058dbc3757e8a7ec75d90dcb48a e31bce25e85dc7fdf0a0e69fd0e58ba4a2ffaf3e Yunzhe Jia yunzhe@calculet.tech 2025-11-18T10:28:00+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-18T10:43:01+08:00 add elf in CMakelist e31bce25e85dc7fdf0a0e69fd0e58ba4a2ffaf3e 6444e227c36df16199c40d8cead2244fe014f377 Yunzhe Jia yunzhe@calculet.tech 2025-11-07T11:52:17+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-07T11:52:17+08:00 dump src tensor 6444e227c36df16199c40d8cead2244fe014f377 d81d7b190ff5f21325167936496dfe5ab48b922e Yunzhe Jia yunzhe@calculet.tech 2025-11-07T09:19:11+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-07T09:19:11+08:00 test golden case d81d7b190ff5f21325167936496dfe5ab48b922e e492f5dc8cec17529f53ef2bbdf7b502107f8148 1f5accb8d0056e6099cd5b772b1cb787dd590a13 Yunzhe Jia yunzhe@calculet.tech 2025-11-04T14:43:32+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-04T15:04:07+08:00 Merge branch 'master' into dev e492f5dc8cec17529f53ef2bbdf7b502107f8148 2e05afd22b3d77c7013b11eee88e88b17188f21a Yunzhe Jia yunzhe@calculet.tech 2025-11-04T14:09:59+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-11-04T14:09:59+08:00 redo untile logic 1f5accb8d0056e6099cd5b772b1cb787dd590a13 2759ccdb4adc8568add4316780d5e675519b0775 Noah 99681487+NoahOksuz@users.noreply.github.com 2025-11-04T05:04:59Z GitHub noreply@github.com 2025-11-03T21:04:59-08:00 Fix garbled output with REPACK at high thread counts (#16956) 2759ccdb4adc8568add4316780d5e675519b0775 c5023daf607c578d6344c628eb7da18ac3d92d32 Aman Gupta amangupta052@gmail.com 2025-11-04T10:53:48+08:00 GitHub noreply@github.com 2025-11-04T10:53:48+08:00 CUDA: avoid mul + bias fusion when doing fusion (#16935) c5023daf607c578d6344c628eb7da18ac3d92d32 e7da30b584dc1f2ee0414c4a1298ce64eef97e8d lhez lih@qti.qualcomm.com 2025-11-03T11:47:57-08:00 GitHub noreply@github.com 2025-11-03T11:47:57-08:00 opencl: support imrope (#16914) e7da30b584dc1f2ee0414c4a1298ce64eef97e8d ed8aa63320393512bdcfe4b05b5ae01ba91888e1 Aleksander Grygier aleksander.grygier@gmail.com 2025-11-03T18:53:26+01:00 GitHub noreply@github.com 2025-11-03T18:53:26+01:00 fix: Viewing multiple PDF attachments (#16974) ed8aa63320393512bdcfe4b05b5ae01ba91888e1 48bd26501b08a3f0bff1249db47f313641f7bebb Daniel Bevenius daniel.bevenius@gmail.com 2025-11-03T18:01:59+01:00 GitHub noreply@github.com 2025-11-03T18:01:59+01:00 model-conversion : pass config to from_pretrained (#16963) 48bd26501b08a3f0bff1249db47f313641f7bebb 622cd010ff4a65bef67edbf2f9bf4707c01f98f7 Georgi Gerganov ggerganov@gmail.com 2025-11-03T15:38:23+02:00 GitHub noreply@github.com 2025-11-03T14:38:23+01:00 server : add props.model_alias (#16943) 622cd010ff4a65bef67edbf2f9bf4707c01f98f7 070ff4d5356083d60b807bb34d36b31c3653a29e theo77186 theo77186@users.noreply.github.com 2025-11-03T14:29:11+01:00 GitHub noreply@github.com 2025-11-03T14:29:11+01:00 ggml: CUDA: add head size 72 for flash-attn (#16962) 070ff4d5356083d60b807bb34d36b31c3653a29e bf7b0c9725ed0c406c5debaea022ec7258430634 Xuan-Son Nguyen son@huggingface.co 2025-11-03T11:11:18+01:00 GitHub noreply@github.com 2025-11-03T11:11:18+01:00 mtmd: add --image-min/max-tokens (#16921) bf7b0c9725ed0c406c5debaea022ec7258430634 fcfce040e816c542f374ad51461b3561d73c4bc9 Xuan-Son Nguyen son@huggingface.co 2025-11-03T10:25:55+01:00 GitHub noreply@github.com 2025-11-03T10:25:55+01:00 mtmd: pad mask for qwen2.5vl (#16954) fcfce040e816c542f374ad51461b3561d73c4bc9 ee3a5a10adf9e83722d1914dddc56a0623ececaf Jinyang He hejinyang@loongson.cn 2025-11-03T14:40:02+08:00 GitHub noreply@github.com 2025-11-03T08:40:02+02:00 ggml : LoongArch fixes (#16958) ee3a5a10adf9e83722d1914dddc56a0623ececaf 7e994168b1ccc12337ba8de939c4fd466107c1fb Olivier Chafik olivier.chafik@gmail.com 2025-11-03T05:33:56Z GitHub noreply@github.com 2025-11-03T07:33:56+02:00 sync: minja (glm 4.6 & minmax m2 templates) (#16949) 7e994168b1ccc12337ba8de939c4fd466107c1fb bcfa87622ae46be6345a8e3dfdbdc5ba5414042b shani-f s0556787439@gmail.com 2025-11-03T03:35:33+02:00 GitHub noreply@github.com 2025-11-03T09:35:33+08:00 SYCL: optimized repeat_back kernel (3× fewer asm instructions, 2× faster)Feature/sycl repeat back opt (#16869) bcfa87622ae46be6345a8e3dfdbdc5ba5414042b a2054e3a8ff0da3978a4acc18c349ff58554d336 Sascha Rogmann 59577610+srogmann@users.noreply.github.com 2025-11-03T00:41:08+01:00 GitHub noreply@github.com 2025-11-03T00:41:08+01:00 feat(webui): improve LaTeX rendering with currency detection (#16508) a2054e3a8ff0da3978a4acc18c349ff58554d336 dd5286805004db1f9ac3176a1cbbfe373bdda0f8 Shagun Bera 141054835+notV3NOM@users.noreply.github.com 2025-11-03T04:40:30+05:30 GitHub noreply@github.com 2025-11-03T00:10:30+01:00 test-backend-ops : fix segfault in moe-expert-reduce test in support mode and coverage (#16936) dd5286805004db1f9ac3176a1cbbfe373bdda0f8 6b9a52422bac0f50dd8f1f8386744fa3ce9783bf Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-11-02T23:11:21+01:00 GitHub noreply@github.com 2025-11-02T23:11:21+01:00 ci : disable failing riscv cross build (#16952) 6b9a52422bac0f50dd8f1f8386744fa3ce9783bf 2f966b8ed87514e74bb96592217226cb6a6974dd Zhiyong Wang 85110830+ravenouse@users.noreply.github.com 2025-11-02T13:08:04-08:00 GitHub noreply@github.com 2025-11-02T22:08:04+01:00 model: add Janus Pro for image understanding (#16906) 2f966b8ed87514e74bb96592217226cb6a6974dd cd5e3b57541ecc52421130742f4d89acbcf77cd4 Georgi Gerganov ggerganov@gmail.com 2025-11-02T22:21:48+02:00 GitHub noreply@github.com 2025-11-02T21:21:48+01:00 clip : use FA (#16837) cd5e3b57541ecc52421130742f4d89acbcf77cd4 87c9efc3b297b8a498716b1db3d061842e6fc85b Georgi Gerganov ggerganov@gmail.com 2025-11-02T18:14:04+02:00 GitHub noreply@github.com 2025-11-02T18:14:04+02:00 server : support unified cache across slots (#16736) 87c9efc3b297b8a498716b1db3d061842e6fc85b 76af40aaaad78c42faecd8016a88362c788b84b0 Aldehir Rojas hello@alde.dev 2025-11-02T08:56:28-06:00 GitHub noreply@github.com 2025-11-02T16:56:28+02:00 common : move gpt-oss reasoning processing to init params (#16937) 76af40aaaad78c42faecd8016a88362c788b84b0 7db35a7958a943be1693879f42d166f152613979 Adrian Lundberg 47256989+alundb@users.noreply.github.com 2025-11-02T10:28:37+01:00 GitHub noreply@github.com 2025-11-02T11:28:37+02:00 docs: remove llama_sampler_accept reference in sampling sample usage (#16920) 7db35a7958a943be1693879f42d166f152613979 a864132ba546b6385922226480ee4e392aaa065c mnehete32 33429707+mnehete32@users.noreply.github.com 2025-11-02T08:42:57+05:30 GitHub noreply@github.com 2025-11-02T11:12:57+08:00 CUDA: add FLOOR, CEIL, ROUND, TRUNC unary ops (#16917) a864132ba546b6385922226480ee4e392aaa065c d38d9f0877a5872daa3c5f06fb9a86376bf15d50 Aaron Teo aaron.teo1@ibm.com 2025-11-02T08:48:46+08:00 GitHub noreply@github.com 2025-11-02T08:48:46+08:00 devops: fix failing s390x docker build (#16918) d38d9f0877a5872daa3c5f06fb9a86376bf15d50 7fd205a8e8832ead273f62c5fd81d2b8aa910535 Aaron Teo aaron.teo1@ibm.com 2025-11-02T08:48:23+08:00 GitHub noreply@github.com 2025-11-02T08:48:23+08:00 ggml: add s390x cpu-feats (#16774) 7fd205a8e8832ead273f62c5fd81d2b8aa910535 2f68ce7cfd20e9e7098514bf730e5389b7bba908 Georgi Gerganov ggerganov@gmail.com 2025-11-02T00:15:31+02:00 GitHub noreply@github.com 2025-11-02T00:15:31+02:00 scripts : add script to bench models (#16894) 2f68ce7cfd20e9e7098514bf730e5389b7bba908 e4a71599e5846110159955dec0008eb4aa24222b Pascal admin@serveurperso.com 2025-11-01T19:49:51+01:00 GitHub noreply@github.com 2025-11-01T19:49:51+01:00 webui: auto-refresh /props on inference start to resync model metadata (#16784) e4a71599e5846110159955dec0008eb4aa24222b dd5e8cab512c7752392b6e51a6f118f348fb3f16 Pascal admin@serveurperso.com 2025-11-01T17:14:54+01:00 GitHub noreply@github.com 2025-11-01T17:14:54+01:00 webui: add HTML/JS preview support to MarkdownContent with sandboxed iframe (#16757) dd5e8cab512c7752392b6e51a6f118f348fb3f16 cf659bbb8ef9eb048e5153a27cf787fd83c05560 Adrien Gallouët angt@huggingface.co 2025-11-01T16:52:17+01:00 GitHub noreply@github.com 2025-11-01T16:52:17+01:00 vendor : update cpp-httplib to 0.27.0 (#16846) cf659bbb8ef9eb048e5153a27cf787fd83c05560 d8b860a219c2415faac8cc0e50b48b4aa11e3b64 Xuan-Son Nguyen son@huggingface.co 2025-11-01T15:51:36+01:00 GitHub noreply@github.com 2025-11-01T15:51:36+01:00 mtmd: refactor preprocessing + support max/min pixels (#16878) d8b860a219c2415faac8cc0e50b48b4aa11e3b64 1ae74882f8f6755e44dff8f23f3abdc5b53ab7c1 Aleksander Grygier aleksander.grygier@gmail.com 2025-11-01T15:35:57+01:00 GitHub noreply@github.com 2025-11-01T15:35:57+01:00 Add a setting to display message generation statistics (#16901) 1ae74882f8f6755e44dff8f23f3abdc5b53ab7c1 961660b8c395afb8903f61ace69396a158ea0cb4 Jaromír Hradílek jhradilek@gmail.com 2025-11-01T15:02:57+01:00 GitHub noreply@github.com 2025-11-01T15:02:57+01:00 webui: recognize AsciiDoc files as valid text files (#16850) 961660b8c395afb8903f61ace69396a158ea0cb4 74fef4129fa58f6277c243777a4894371479dbad Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-11-01T11:01:42+01:00 GitHub noreply@github.com 2025-11-01T11:01:42+01:00 common : allow --system-prompt-file for diffusion-cli (#16903) 74fef4129fa58f6277c243777a4894371479dbad 5d8bb900bc7daa84bfa7bb1d25ab7e32394919f3 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-11-01T08:55:25+01:00 GitHub noreply@github.com 2025-11-01T09:55:25+02:00 codeowners : update after refactor (#16905) 5d8bb900bc7daa84bfa7bb1d25ab7e32394919f3 2e76e013600cb0d51ccf158571ca1d0502952a07 Jeff Bolz jbolz@nvidia.com 2025-11-01T00:52:14-05:00 GitHub noreply@github.com 2025-11-01T06:52:14+01:00 vulkan: Fix multi_add invalid descriptor usage (#16899) 2e76e013600cb0d51ccf158571ca1d0502952a07 d3dc9dd898be805c23a408cc36daed5b3bf29221 Jeff Bolz jbolz@nvidia.com 2025-11-01T00:45:28-05:00 GitHub noreply@github.com 2025-11-01T06:45:28+01:00 vulkan: fuse mul_mat+add and mul_mat_id+add_id (#16868) d3dc9dd898be805c23a408cc36daed5b3bf29221 bea04522ff1a0d8559ccfd353aa018dcfbb608cc Oliver Simons osimons@nvidia.com 2025-11-01T06:13:26+01:00 GitHub noreply@github.com 2025-11-01T13:13:26+08:00 CUDA: Remove unneded bias/gate dims in fused mmvq (#16858) bea04522ff1a0d8559ccfd353aa018dcfbb608cc 0de0a01576772032008a689afc4d7c80685074c4 Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-10-31T23:40:23+01:00 GitHub noreply@github.com 2025-10-31T23:40:23+01:00 refactor : llama-model.cpp (#16252) 0de0a01576772032008a689afc4d7c80685074c4 e58d585604bbbbbc24f8149effe444621b5191c6 Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-10-31T21:20:47+01:00 GitHub noreply@github.com 2025-10-31T21:20:47+01:00 model : Minimax M2 (#16831) e58d585604bbbbbc24f8149effe444621b5191c6 31c511a968348281e11d590446bb815048a1e912 Giuseppe Scrivano gscrivan@redhat.com 2025-10-31T21:20:07+01:00 GitHub noreply@github.com 2025-10-31T21:20:07+01:00 model : add Granite Hybrid nano types (#16896) 31c511a968348281e11d590446bb815048a1e912 6d39015a744b47bb7643ea73f412e6d64677f305 Johannes Gäßler johannesg@5d6.de 2025-10-31T15:57:19+01:00 GitHub noreply@github.com 2025-10-31T15:57:19+01:00 CUDA: Volta tensor core support for MMF (#16843) 6d39015a744b47bb7643ea73f412e6d64677f305 4146d6a1a6228711a487a1e3e9ddd120f8d027d7 Georgi Gerganov ggerganov@gmail.com 2025-10-31T16:25:50+02:00 Georgi Gerganov ggerganov@gmail.com 2025-10-31T16:26:28+02:00 sync : ggml 4146d6a1a6228711a487a1e3e9ddd120f8d027d7 8da3c0e200a586f768ada6f38745acb01380174c Aman Gupta amangupta052@gmail.com 2025-10-31T20:05:07+08:00 GitHub noreply@github.com 2025-10-31T20:05:07+08:00 CUDA: add expert reduce kernel (#16857) 8da3c0e200a586f768ada6f38745acb01380174c c22473b580807929fd9e3a3344a48e8cfbe6c88f Georgi Gerganov ggerganov@gmail.com 2025-10-31T13:50:33+02:00 GitHub noreply@github.com 2025-10-31T13:50:33+02:00 batch : fix consistency checks for the input positions (#16890) c22473b580807929fd9e3a3344a48e8cfbe6c88f 0f715b4e759acceccb9f437cfd2a988fff85514a Georgi Gerganov ggerganov@gmail.com 2025-10-31T10:54:19+02:00 GitHub noreply@github.com 2025-10-31T10:54:19+02:00 server : don't print user inputs to console (#16871) 0f715b4e759acceccb9f437cfd2a988fff85514a d2d931f173b8a736b08999436e9259aafddec718 Daniel Bevenius daniel.bevenius@gmail.com 2025-10-31T09:51:26+01:00 GitHub noreply@github.com 2025-10-31T09:51:26+01:00 server : fix typos in server.cpp comments [no ci] (#16883) d2d931f173b8a736b08999436e9259aafddec718 2976b0374d36609b0429dd6ce48807e2ad39a7c2 Jeff Bolz jbolz@nvidia.com 2025-10-31T02:34:47-05:00 GitHub noreply@github.com 2025-10-31T08:34:47+01:00 vulkan: disable spirv-opt for rope shaders (#16872) 2976b0374d36609b0429dd6ce48807e2ad39a7c2 d2a2673dd1c86a01ab010e18b13b8bb959968c48 Masato Nakasaka masato.nakasaka@intel.com 2025-10-31T16:18:59+09:00 GitHub noreply@github.com 2025-10-31T08:18:59+01:00 vulkan: Fix crash when FP16 mul_mat accumulation is not supported (#16796) d2a2673dd1c86a01ab010e18b13b8bb959968c48 13002a08960e51a76c4d696165b5d7638d2f2b99 Ruben Ortlam picard12@live.de 2025-10-31T08:14:49+01:00 GitHub noreply@github.com 2025-10-31T08:14:49+01:00 vulkan: fix shmem overrun in mmq id shader (#16873) 2e05afd22b3d77c7013b11eee88e88b17188f21a c20c0a5c0c44179f5267a262546624854b1203d1 Yunzhe Jia yunzhe@calculet.tech 2025-10-31T14:06:06+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-10-31T14:06:06+08:00 add pcie support 13002a08960e51a76c4d696165b5d7638d2f2b99 6eb208d17ea29bb60295d9a2b5e7122dfb8f4b55 l3utterfly gc.pthzfoldr@gmail.com 2025-10-31T12:46:31+08:00 GitHub noreply@github.com 2025-10-30T21:46:31-07:00 ggml-hexagon: respect input size when getting/setting tensor data (#16836) 6eb208d17ea29bb60295d9a2b5e7122dfb8f4b55 9984cbb61d12491d604484fb38b678fa15064061 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-31T00:34:27+01:00 GitHub noreply@github.com 2025-10-31T00:34:27+01:00 ci : enable free-disk-space on cuda docker build (#16877) 9984cbb61d12491d604484fb38b678fa15064061 ce18efeaf1234bd08f25bc08f88ef95cf5d9e51f lhez lih@qti.qualcomm.com 2025-10-30T16:00:20-07:00 GitHub noreply@github.com 2025-10-30T16:00:20-07:00 opencl: fix boundary handling for mul_mm (#16875) ce18efeaf1234bd08f25bc08f88ef95cf5d9e51f 16724b5b6836a2d4b8936a5824d2ff27c52b4517 RodriMora bullerwins@gmail.com 2025-10-30T23:15:03+01:00 GitHub noreply@github.com 2025-10-30T23:15:03+01:00 convert : update transformers requirements (#16866) 16724b5b6836a2d4b8936a5824d2ff27c52b4517 b52edd25586fabb70f0c21b274473b307cf14499 chansikpark chansik.park@gmail.com 2025-10-30T14:22:23-04:00 GitHub noreply@github.com 2025-10-30T20:22:23+02:00 server : bump request URI max length to 32768 (#16862) b52edd25586fabb70f0c21b274473b307cf14499 517b7170e1a4d733583c4b07c5b7a49acc05911c Georgi Gerganov ggerganov@gmail.com 2025-10-30T18:42:57+02:00 GitHub noreply@github.com 2025-10-30T18:42:57+02:00 server : remove n_past (#16818) 517b7170e1a4d733583c4b07c5b7a49acc05911c 835e918d8428f5119927d7150bf5a26176dedda0 Max Krasnyansky maxk@qti.qualcomm.com 2025-10-30T09:06:13-07:00 GitHub noreply@github.com 2025-10-30T09:06:13-07:00 cpu: introduce chunking for repack matmuls and enable matmul-id chunking on ARM64 (#16833) 835e918d8428f5119927d7150bf5a26176dedda0 d261223d24e97f2df50220e4a5b7f0adb69bba81 Shagun Bera 141054835+notV3NOM@users.noreply.github.com 2025-10-30T21:17:31+05:30 GitHub noreply@github.com 2025-10-30T17:47:31+02:00 common: fix typo in cli help text (#16864) d261223d24e97f2df50220e4a5b7f0adb69bba81 dcca0d3ab840ebe9b2ccd4719033d408eeb758d7 JJJYmmm 92386084+JJJYmmm@users.noreply.github.com 2025-10-30T23:19:14+08:00 GitHub noreply@github.com 2025-10-30T16:19:14+01:00 model: add support for qwen3vl series (#16780) dcca0d3ab840ebe9b2ccd4719033d408eeb758d7 bacddc049a00786df44e682262f6e298742bfbc3 Max Krasnyansky maxk@qti.qualcomm.com 2025-10-30T05:26:05-07:00 GitHub noreply@github.com 2025-10-30T14:26:05+02:00 cpu: introduce chunking for flash attention (#16829) bacddc049a00786df44e682262f6e298742bfbc3 229bf686287d18f82c44e89888cc662145ecfdb4 Tianyue-Zhao zhaotianyue@outlook.com 2025-10-30T07:18:50-04:00 GitHub noreply@github.com 2025-10-30T12:18:50+01:00 model: Add support for CogVLM model (#15002) 229bf686287d18f82c44e89888cc662145ecfdb4 d7395115baf395b75a73a17b0b796e746e468da9 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-30T08:56:28+01:00 GitHub noreply@github.com 2025-10-30T08:56:28+01:00 cuda : fix argsort with 64k+ rows (#16849) d7395115baf395b75a73a17b0b796e746e468da9 052df28b0e3ca0398e5928c61c7de40254317894 Jan Boon jan.boon@kaetemi.be 2025-10-30T14:30:58+08:00 GitHub noreply@github.com 2025-10-30T08:30:58+02:00 llama : use std::abs instead of abs (#16853) 052df28b0e3ca0398e5928c61c7de40254317894 8b11deea4663f29d3e042ce1056ba643264cd5f1 Jeff Bolz jbolz@nvidia.com 2025-10-30T01:27:41-05:00 GitHub noreply@github.com 2025-10-30T07:27:41+01:00 vulkan: Handle argsort with a large number of rows (#16851) 8b11deea4663f29d3e042ce1056ba643264cd5f1 b9ce94017729465895402cbcfffb51fa926c15e3 Oliver Simons osimons@nvidia.com 2025-10-30T04:34:15+01:00 GitHub noreply@github.com 2025-10-30T11:34:15+08:00 Hide latency of bias and gate-loading (#16847) b9ce94017729465895402cbcfffb51fa926c15e3 3464bdac37027c5e9661621fc75ffcef3c19c6ef Jeff Bolz jbolz@nvidia.com 2025-10-29T15:13:10-05:00 GitHub noreply@github.com 2025-10-29T15:13:10-05:00 vulkan: Fuse rope+set_rows (#16769) 3464bdac37027c5e9661621fc75ffcef3c19c6ef e3af5563bd049141e036b50f843196db33d23e97 Xuan-Son Nguyen son@huggingface.co 2025-10-29T20:11:39+01:00 GitHub noreply@github.com 2025-10-29T20:11:39+01:00 llama: fix ASAN error with M-RoPE (#16848) e3af5563bd049141e036b50f843196db33d23e97 10fcc41290e233788f5a4215314156e8e023eb92 Xuan-Son Nguyen son@huggingface.co 2025-10-29T18:09:18+01:00 GitHub noreply@github.com 2025-10-29T18:09:18+01:00 llama: store mrope data in KV cell (#16825) 10fcc41290e233788f5a4215314156e8e023eb92 bcf5bda6f5df559565d11d7c8e8295c1159a85ec Jeff Bolz jbolz@nvidia.com 2025-10-29T08:44:29-05:00 GitHub noreply@github.com 2025-10-29T14:44:29+01:00 vulkan: Update topk_moe fusion to handle gpt's late softmax (#16656) bcf5bda6f5df559565d11d7c8e8295c1159a85ec 3eb2be1ca5f37480aeb16102970d9e65f43347fe Ruben Ortlam picard12@live.de 2025-10-29T14:39:03+01:00 GitHub noreply@github.com 2025-10-29T14:39:03+01:00 Vulkan MMQ Integer Dot Refactor and K-Quant support (#16536) 3eb2be1ca5f37480aeb16102970d9e65f43347fe e41bcce8f0b53032a1fed275cd253e931c041cf6 Max Krasnyansky maxk@qti.qualcomm.com 2025-10-29T06:29:12-07:00 GitHub noreply@github.com 2025-10-29T06:29:12-07:00 Hexagon Op queue & dispatch optimizations (#16820) e41bcce8f0b53032a1fed275cd253e931c041cf6 144a4ce824b6bd0e48d62009d10cae1daf5308db Aman Gupta amangupta052@gmail.com 2025-10-29T21:11:53+08:00 GitHub noreply@github.com 2025-10-29T21:11:53+08:00 CUDA: use fastdiv in set-rows (#16834) 144a4ce824b6bd0e48d62009d10cae1daf5308db f549b0007dbdd683215820f7229ce180a12b191d Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-29T14:09:50+01:00 GitHub noreply@github.com 2025-10-29T14:09:50+01:00 vendor : sync minja (#16500) f549b0007dbdd683215820f7229ce180a12b191d 9a3ea685b937c0f0cbfda2e50004ea54bf187512 Jeff Bolz jbolz@nvidia.com 2025-10-29T03:53:04-05:00 GitHub noreply@github.com 2025-10-29T09:53:04+01:00 vulkan: Call ggml_vk_buffer_write_2d from ggml_vk_buffer_copy (#16793) 9a3ea685b937c0f0cbfda2e50004ea54bf187512 338074c383c81366320d176d83b94b0a567ee0c2 Aman Gupta amangupta052@gmail.com 2025-10-29T15:55:06+08:00 GitHub noreply@github.com 2025-10-29T15:55:06+08:00 CUDA: Fix bug in topk-moe for gpt-oss (#16821) 338074c383c81366320d176d83b94b0a567ee0c2 851553ea6b24cb39fd5fd188b437d777cb411de8 YaelLogic y0548591250@gmail.com 2025-10-29T08:14:39+02:00 GitHub noreply@github.com 2025-10-29T14:14:39+08:00 sycl: add RMS_NORM_BACK operation support (#16808) 851553ea6b24cb39fd5fd188b437d777cb411de8 85a7d8677bf2200981e52f744a21d5267964ffcf YaelGitAccount 38328157276@mby.co.il 2025-10-28T21:10:28+02:00 GitHub noreply@github.com 2025-10-28T20:10:28+01:00 cuda: add SET operation support (#16804) 85a7d8677bf2200981e52f744a21d5267964ffcf a8ca18b4b815a2abdbecb958ee5f4c542d69aac7 Georgi Gerganov ggerganov@gmail.com 2025-10-28T20:19:44+02:00 GitHub noreply@github.com 2025-10-28T20:19:44+02:00 memory : remove KV cache size padding (#16812) a8ca18b4b815a2abdbecb958ee5f4c542d69aac7 8284efc35c909217ee1b9a06903245d808ac2283 Georgi Gerganov ggerganov@gmail.com 2025-10-28T19:41:43+02:00 GitHub noreply@github.com 2025-10-28T19:41:43+02:00 llama-bench : clarify benchmarked parts of the computation (#16823) 8284efc35c909217ee1b9a06903245d808ac2283 1c1409e13169d0ced4b1b4d39fb5b268b7525091 l3utterfly gc.pthzfoldr@gmail.com 2025-10-28T23:16:20+08:00 GitHub noreply@github.com 2025-10-28T08:16:20-07:00 initialise buffer.device in ggml_hexagon_session (#16816) 1c1409e13169d0ced4b1b4d39fb5b268b7525091 7a0e900e3615fa46c074a7fdf900b47d3c0a1c7e Sam Malayek 12037535+SamMalayek@users.noreply.github.com 2025-10-28T03:51:41-07:00 GitHub noreply@github.com 2025-10-28T12:51:41+02:00 embedding: add raw option for --embd-output-format (#16541) 7a0e900e3615fa46c074a7fdf900b47d3c0a1c7e 280d97be9660e7a5feaa28a6e7a299bc73dd83fc Johannes Gäßler johannesg@5d6.de 2025-10-28T11:23:54+01:00 GitHub noreply@github.com 2025-10-28T11:23:54+01:00 llama: consistent ctx <-> buf order for KV cache (#16746) 280d97be9660e7a5feaa28a6e7a299bc73dd83fc 3479efd112b3910d2d008ad08fe4cb4895605903 Aldehir Rojas hello@alde.dev 2025-10-28T03:37:52-05:00 GitHub noreply@github.com 2025-10-28T09:37:52+01:00 grammar : support array references in json schema (#16792) 3479efd112b3910d2d008ad08fe4cb4895605903 463bbf20bfbe0da10ac58984d4d62bed5a49362a Chenguang Li 757486878@qq.com 2025-10-28T10:54:53+08:00 GitHub noreply@github.com 2025-10-28T10:54:53+08:00 CANN: Improve device ID handling and aclnnArange checks (#16752) 463bbf20bfbe0da10ac58984d4d62bed5a49362a ad8d36beffd791db10c94eb9e964afb891e3ca55 Aman Gupta amangupta052@gmail.com 2025-10-28T10:31:21+08:00 GitHub noreply@github.com 2025-10-28T10:31:21+08:00 CUDA: add unused vars to mmvf and mmvq (#16807) ad8d36beffd791db10c94eb9e964afb891e3ca55 c053e18a66dd95dc340aa61317877c2a41d4e3cf tamarPal tamarp3385@gmail.com 2025-10-28T03:50:33+02:00 GitHub noreply@github.com 2025-10-28T09:50:33+08:00 sycl: add SSM_CONV operation support (#16800) c053e18a66dd95dc340aa61317877c2a41d4e3cf e1ab0848037c9f9bfe68b3e1cee9ee375e1018a3 Yuri Khrustalev ykhrustalev@users.noreply.github.com 2025-10-27T18:54:01-04:00 GitHub noreply@github.com 2025-10-27T23:54:01+01:00 chat: Add LFM2 tool handling (#16763) e1ab0848037c9f9bfe68b3e1cee9ee375e1018a3 5a4ff43e7dd049e35942bc3d12361dab2f155544 Xuan-Son Nguyen son@huggingface.co 2025-10-27T23:12:16+01:00 GitHub noreply@github.com 2025-10-27T23:12:16+01:00 mtmd : fix idefics3 preprocessing (#16806) 5a4ff43e7dd049e35942bc3d12361dab2f155544 10640e31aab0819f31c1e1f2d008b019ee737232 Diego Devesa slarengh@gmail.com 2025-10-27T13:51:28-07:00 GitHub noreply@github.com 2025-10-27T21:51:28+01:00 llama : disable pipeline parallelism if compute buffer allocation fails (#16748) 10640e31aab0819f31c1e1f2d008b019ee737232 80d28f104c0c3e61c11d6af073642b246a9fc19c Acly aclysia@gmail.com 2025-10-27T21:50:22+01:00 GitHub noreply@github.com 2025-10-27T21:50:22+01:00 ggml : fix interpolate with align-corners and ne=1 (#16700) 80d28f104c0c3e61c11d6af073642b246a9fc19c c55d53acec864f64afa1ba92972203dce1bf88f5 Johannes Gäßler johannesg@5d6.de 2025-10-27T21:39:49+01:00 GitHub noreply@github.com 2025-10-27T21:39:49+01:00 HIP: fix AMDGPU_TARGETS, update documentation (#16803) c55d53acec864f64afa1ba92972203dce1bf88f5 945501f5ea4b8ca56b181ecb035e9ee3fb31f432 Xuan-Son Nguyen son@huggingface.co 2025-10-27T16:02:58+01:00 GitHub noreply@github.com 2025-10-27T16:02:58+01:00 model : add LightOnOCR-1B model (#16764) 945501f5ea4b8ca56b181ecb035e9ee3fb31f432 75cbdd3fce38ea12d50cd19e73a069aa5dbbd5fa Johannes Gäßler johannesg@5d6.de 2025-10-27T09:17:31+01:00 GitHub noreply@github.com 2025-10-27T09:17:31+01:00 llama: fix leaked buffers for mmap + split files (#16765) 75cbdd3fce38ea12d50cd19e73a069aa5dbbd5fa 2b9bd9bf4e759c05db629ec1c391dc8aeaa71887 Aman Gupta amangupta052@gmail.com 2025-10-27T09:25:10+08:00 GitHub noreply@github.com 2025-10-27T09:25:10+08:00 test-backend-ops: print failed tests at the end (#16785) 2b9bd9bf4e759c05db629ec1c391dc8aeaa71887 59fc1ec8e83b14354c1a3a8acf8c5c2cbf9af42f tamarPal tamarp3385@gmail.com 2025-10-27T03:20:24+02:00 GitHub noreply@github.com 2025-10-27T09:20:24+08:00 sycl: add ROLL operation support (#16665) 59fc1ec8e83b14354c1a3a8acf8c5c2cbf9af42f 75d33b9302f84a5b89f82205d2bcd8def5a64e0a shani-f s0556787439@gmail.com 2025-10-27T03:19:50+02:00 GitHub noreply@github.com 2025-10-27T09:19:50+08:00 sycl: add REPEAT_BACK operation support (#16734) 75d33b9302f84a5b89f82205d2bcd8def5a64e0a 3470a5c891dcc94363e492a3760af92b6b07241c Aman Gupta amangupta052@gmail.com 2025-10-27T09:06:16+08:00 GitHub noreply@github.com 2025-10-27T09:06:16+08:00 CUDA: support for weight clamp in top-k norm (#16702) 3470a5c891dcc94363e492a3760af92b6b07241c bd562fe4f7bd55625511d5f9d639c4fb1db1d440 Acly aclysia@gmail.com 2025-10-26T23:19:03+01:00 GitHub noreply@github.com 2025-10-26T23:19:03+01:00 ggml-alloc : make gallocr prefer chunks that allow memory reuse (#16788) bd562fe4f7bd55625511d5f9d639c4fb1db1d440 bbac6a26b2bd7f7c1f0831cb1e7b52734c66673b Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-26T21:31:41+01:00 GitHub noreply@github.com 2025-10-26T21:31:41+01:00 cuda : use fast copy when src and dst are of different type and contiguous (#16789) bbac6a26b2bd7f7c1f0831cb1e7b52734c66673b 73a48c9790d320476b3e5ef75bda09f2f8269e6e leejet leejet714@gmail.com 2025-10-27T02:13:31+08:00 GitHub noreply@github.com 2025-10-26T19:13:31+01:00 ggml: fix cuda kernel launch configuration for k_compute_batched_ptrs to support large batch (#16744) 73a48c9790d320476b3e5ef75bda09f2f8269e6e f696428ce8e4d16c17acbffeaa7feac3b0fb9061 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-26T17:21:23+01:00 GitHub noreply@github.com 2025-10-26T17:21:23+01:00 convert : enable expert group selection for all models with it (#16691) f696428ce8e4d16c17acbffeaa7feac3b0fb9061 7cce4f8158f0c4c88d8dadd4c23d33938127b897 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-26T17:20:32+01:00 GitHub noreply@github.com 2025-10-26T17:20:32+01:00 graph : add clamping to ffn_moe_weights_sum to avoid div-by-zero (#16655) 7cce4f8158f0c4c88d8dadd4c23d33938127b897 8d8862829cd770fc9c0c7a726ed162de824ff5ea Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-26T16:08:52+01:00 GitHub noreply@github.com 2025-10-26T16:08:52+01:00 model : set res->t_embd in SmallThinker models (#16782) 8d8862829cd770fc9c0c7a726ed162de824ff5ea f77c13b91f4d25754b6a0b857f98a6bc922a0aa7 amirai21 89905406+amirai21@users.noreply.github.com 2025-10-26T14:01:20+02:00 GitHub noreply@github.com 2025-10-26T13:01:20+01:00 docs : add Jamba to Text-only models list (#16778) f77c13b91f4d25754b6a0b857f98a6bc922a0aa7 3cfa9c3f125763305b4226bc032f1954f08990dc Aman Gupta amangupta052@gmail.com 2025-10-26T19:28:04+08:00 GitHub noreply@github.com 2025-10-26T19:28:04+08:00 CUDA: General GEMV fusion (#16715) 3cfa9c3f125763305b4226bc032f1954f08990dc 5d195f17bc60eacc15cfb929f9403cf29ccdf419 Gilad S. 7817232+giladgd@users.noreply.github.com 2025-10-26T06:37:38+02:00 GitHub noreply@github.com 2025-10-26T05:37:38+01:00 vulkan: deduplicate Microsoft Direct3D12 devices (#16689) 5d195f17bc60eacc15cfb929f9403cf29ccdf419 226f295f4dd92ad714533adc5497afed5fa88bb8 Galunid karolek1231456@gmail.com 2025-10-25T20:41:36+02:00 GitHub noreply@github.com 2025-10-25T20:41:36+02:00 convert : handle mmproj filename/path properly (#16760) 226f295f4dd92ad714533adc5497afed5fa88bb8 f90b4a8efe4466215c51155a5c22c1ae6207da23 Shunta Saito shunta.saito@gmail.com 2025-10-25T19:26:27+09:00 GitHub noreply@github.com 2025-10-25T12:26:27+02:00 model : set res->t_embd in PLaMo2 models (#16766) f90b4a8efe4466215c51155a5c22c1ae6207da23 8423d019318b446640bb620e4ce80066d8530f05 Giuseppe Scrivano gscrivan@redhat.com 2025-10-25T10:59:54+02:00 GitHub noreply@github.com 2025-10-25T10:59:54+02:00 vulkan: delete dead code (#16732) 8423d019318b446640bb620e4ce80066d8530f05 5cca2542ac3f3f86831d32bce744d08fc2b353b0 Jeff Bolz jbolz@nvidia.com 2025-10-25T00:04:12-05:00 GitHub noreply@github.com 2025-10-25T07:04:12+02:00 vulkan: Optimize SSM_SCAN (#16645) 5cca2542ac3f3f86831d32bce744d08fc2b353b0 55945d2ef51b93821d4b6f4a9b994393344a90db compilade git@compilade.net 2025-10-24T20:52:00-04:00 GitHub noreply@github.com 2025-10-24T20:52:00-04:00 convert : avoid dequantizing mxfp4 for GPT-OSS (#16756) 55945d2ef51b93821d4b6f4a9b994393344a90db 0bcb40b48c6fc6f17ba9672625e526ab2574344b leejet leejet714@gmail.com 2025-10-25T03:39:37+08:00 GitHub noreply@github.com 2025-10-24T21:39:37+02:00 ggml: fix CUDA grid launch condition for large block_nums.y in binbcast (#16742) 0bcb40b48c6fc6f17ba9672625e526ab2574344b 69e9ff010309a1155d704cf9320bdb3aaf4160ca Aman Gupta amangupta052@gmail.com 2025-10-24T20:46:19+08:00 GitHub noreply@github.com 2025-10-24T20:46:19+08:00 CUDA: use CUB for arbitary size argsort (#16754) 69e9ff010309a1155d704cf9320bdb3aaf4160ca 5a91109a5d7dab5d7adc40bedb397ede99a705b1 Florian Badie florianbadie@odrling.xyz 2025-10-24T14:10:29+02:00 GitHub noreply@github.com 2025-10-24T14:10:29+02:00 webui: support q URL parameter (#16728) 5a91109a5d7dab5d7adc40bedb397ede99a705b1 f8f071faddf32ea09f4234edb6e809b380a9ee26 Daniel Bevenius daniel.bevenius@gmail.com 2025-10-24T12:02:02+02:00 GitHub noreply@github.com 2025-10-24T12:02:02+02:00 model-conversion : add trust_remote_code for orig model run [no ci] (#16751) f8f071faddf32ea09f4234edb6e809b380a9ee26 0bf47a1dbba4d36f2aff4e8c34b06210ba34e688 compilade git@compilade.net 2025-10-23T16:31:41-04:00 GitHub noreply@github.com 2025-10-23T16:31:41-04:00 convert : handle pre-quantized models (#14810) 0bf47a1dbba4d36f2aff4e8c34b06210ba34e688 dd62dcfab97e420949519fd0eac9fca7bf97e635 Johannes Gäßler johannesg@5d6.de 2025-10-23T21:30:17+02:00 GitHub noreply@github.com 2025-10-23T21:30:17+02:00 server: add memory breakdown print (#16740) dd62dcfab97e420949519fd0eac9fca7bf97e635 d0660f237a5c31771a3d6d1030ebe3e0c409ba92 Julien Denize 40604584+juliendenize@users.noreply.github.com 2025-10-23T15:54:46+02:00 GitHub noreply@github.com 2025-10-23T15:54:46+02:00 convert : Make mistral-common dependency optional (#16738) d0660f237a5c31771a3d6d1030ebe3e0c409ba92 fe6a9882acf5c02f96624ed8f80144100d7006cb Xuan-Son Nguyen son@huggingface.co 2025-10-23T15:00:49+02:00 GitHub noreply@github.com 2025-10-23T15:00:49+02:00 mtmd-cli : allow using --jinja (#16718) fe6a9882acf5c02f96624ed8f80144100d7006cb 061f0eff02dc9a82f7bd850db3bd70b8a0b5e87a Prajwal B Mehendarkar prajwal.b.mehendarkar@ibm.com 2025-10-23T17:07:31+05:30 GitHub noreply@github.com 2025-10-23T19:37:31+08:00 Manually link -lbsd to resolve flock symbol on AIX (#16610) 061f0eff02dc9a82f7bd850db3bd70b8a0b5e87a 8cf6b42d467d05fa7d9776d2bcc69974ecce6900 Aman Gupta amangupta052@gmail.com 2025-10-23T19:14:06+08:00 GitHub noreply@github.com 2025-10-23T19:14:06+08:00 ggml-cuda: use passed ops instead of hardcoded ops (#16712) 8cf6b42d467d05fa7d9776d2bcc69974ecce6900 9de9672adb0f4ca4e39483ac3ffed52b3f70a55d matteo matteo.serva@gmail.com 2025-10-23T11:32:24+02:00 GitHub noreply@github.com 2025-10-23T12:32:24+03:00 server : send partial stop string when is reached (#15007) 9de9672adb0f4ca4e39483ac3ffed52b3f70a55d 63d2fc46e17a06be5b4b5823a5ada088317f1f0a Matthew Michel matthew.michel@intel.com 2025-10-22T20:05:15-05:00 GitHub noreply@github.com 2025-10-23T09:05:15+08:00 sycl: use async memory allocation to fix crashes during graph recording (#16644) 63d2fc46e17a06be5b4b5823a5ada088317f1f0a a2e0088d9242bd9e57f8b852b05a6e47843b5a45 Max Krasnyansky maxk@qti.qualcomm.com 2025-10-22T13:47:09-07:00 GitHub noreply@github.com 2025-10-22T13:47:09-07:00 Add experimental ggml-hexagon backend for the Hexagon NPU (#16547) a2e0088d9242bd9e57f8b852b05a6e47843b5a45 9b9201f65a22c02cee8e300f58f480a588591227 Diego Devesa slarengh@gmail.com 2025-10-22T11:20:55-07:00 GitHub noreply@github.com 2025-10-22T20:20:55+02:00 Revert "ggml : Leverage the existing GGML_F32_VEC helpers to vectorize ggml_v…" (#16723) 9b9201f65a22c02cee8e300f58f480a588591227 19a5a3edfd306516cc419679d69d6435943b6816 Pascal admin@serveurperso.com 2025-10-22T16:58:23+02:00 GitHub noreply@github.com 2025-10-22T16:58:23+02:00 webui: introduce OpenAI-compatible model selector in JSON payload (#16562) 19a5a3edfd306516cc419679d69d6435943b6816 d8eaa26e4d9228df3aa46a930db60c8eaab67c1b sirus20x6 sirus20x6@users.noreply.github.com 2025-10-22T05:14:14-05:00 GitHub noreply@github.com 2025-10-22T12:14:14+02:00 ggml : Leverage the existing GGML_F32_VEC helpers to vectorize ggml_vec_set_f32 for faster fills (#16522) d8eaa26e4d9228df3aa46a930db60c8eaab67c1b 9285325ce0174631c3cd6121d56084adc4ef2d8f Acly aclysia@gmail.com 2025-10-22T12:01:22+02:00 GitHub noreply@github.com 2025-10-22T12:01:22+02:00 tests : fix test-thread-safety when compiling with multiple backends (#16699) 9285325ce0174631c3cd6121d56084adc4ef2d8f 03792ad93609fc67e41041c6347d9aa14e5e0d74 Aman Gupta amangupta052@gmail.com 2025-10-22T12:33:08+08:00 GitHub noreply@github.com 2025-10-22T12:33:08+08:00 CUDA: fix bug in topk-moe softmax (#16711) 03792ad93609fc67e41041c6347d9aa14e5e0d74 51d1a8c997bd2629ef211a30208058ea87a30982 Aman Gupta amangupta052@gmail.com 2025-10-21T22:40:38+08:00 GitHub noreply@github.com 2025-10-21T22:40:38+08:00 CUDA: topk-moe: add optional parameter for gpt-oss (#16649) 51d1a8c997bd2629ef211a30208058ea87a30982 4926419c4d74a1cf724e7163d937eb72f36e7b26 Johannes Gäßler johannesg@5d6.de 2025-10-21T15:27:53+02:00 GitHub noreply@github.com 2025-10-21T15:27:53+02:00 CUDA: better error for FA kernel with 0 occupancy (#16643) 4926419c4d74a1cf724e7163d937eb72f36e7b26 6ea37f57391d27736c35cd3c20c1f990b7952b74 Aman Gupta amangupta052@gmail.com 2025-10-21T16:43:14+08:00 GitHub noreply@github.com 2025-10-21T16:43:14+08:00 ggml: add ggml_can_fuse_subgraph (#16662) 6ea37f57391d27736c35cd3c20c1f990b7952b74 fb349848f387f355450c3187556e71e6d32c145f lhez lih@qti.qualcomm.com 2025-10-20T22:26:17-07:00 GitHub noreply@github.com 2025-10-20T22:26:17-07:00 opencl: fix warnings and clean up profiling (#16688) fb349848f387f355450c3187556e71e6d32c145f 6de8ed75196c7cd98c1f34bbf3a7452451ba8ac2 Jeff Bolz jbolz@nvidia.com 2025-10-20T22:16:08-05:00 GitHub noreply@github.com 2025-10-20T22:16:08-05:00 vulkan: Handle FA with all -inf mask values (#16447) 6de8ed75196c7cd98c1f34bbf3a7452451ba8ac2 84bf3c677857279037adf67cdcfd89eaa4ca9281 YehuditE y8703470@gmail.com 2025-10-21T01:21:12+03:00 GitHub noreply@github.com 2025-10-21T00:21:12+02:00 sycl : add PAD_REFLECT_D1 operator support (#16145) 84bf3c677857279037adf67cdcfd89eaa4ca9281 c9c1972e2c2cc6a771fcc145bfa138700179f961 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-20T21:38:20+02:00 GitHub noreply@github.com 2025-10-20T21:38:20+02:00 model : add BailingMoeV2 support (#16063) c9c1972e2c2cc6a771fcc145bfa138700179f961 b617cfd2896edd592a36ebbc041817eb030a1005 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-20T19:49:02+02:00 GitHub noreply@github.com 2025-10-20T19:49:02+02:00 Handle legacy 'context' attachments (#16687) b617cfd2896edd592a36ebbc041817eb030a1005 79068501fac9a74cca7129a8e5a8281b410a853e Diego Devesa slarengh@gmail.com 2025-10-20T05:53:50-07:00 GitHub noreply@github.com 2025-10-20T14:53:50+02:00 ggml-alloc : fix leak when reusing a tensor with a larger size (#16679) 79068501fac9a74cca7129a8e5a8281b410a853e 0e4a0cf2fae667d3efcf52f2f52398779d986b1d Aleksander Grygier aleksander.grygier@gmail.com 2025-10-20T14:21:12+02:00 GitHub noreply@github.com 2025-10-20T14:21:12+02:00 Prevent premature submission on IME input (#16673) 0e4a0cf2fae667d3efcf52f2f52398779d986b1d 13f2cfad4170c096c51a02c24a6a158cb47f1480 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-20T13:29:14+02:00 GitHub noreply@github.com 2025-10-20T13:29:14+02:00 Import/Export UX improvements (#16619) 13f2cfad4170c096c51a02c24a6a158cb47f1480 06332e28672356b964d6dfc2ba4657e20581cd43 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-20T12:41:13+02:00 GitHub noreply@github.com 2025-10-20T12:41:13+02:00 Enable per-conversation loading states to allow having parallel conversations (#16327) 06332e28672356b964d6dfc2ba4657e20581cd43 72d53e6c4decee8b339e49aed8cc0e234b9639dc takuya kodama otegami@clear-code.com 2025-10-20T16:27:09+08:00 GitHub noreply@github.com 2025-10-20T11:27:09+03:00 llama-batch: fix build fails with `-Werror=missing-braces` (#16614) 72d53e6c4decee8b339e49aed8cc0e234b9639dc 2330de7b847ca84eac766df372c604c26db72747 Ron Evans ron@hybridgroup.com 2025-10-20T10:20:04+02:00 GitHub noreply@github.com 2025-10-20T11:20:04+03:00 readme: update bindings (#16651) 2330de7b847ca84eac766df372c604c26db72747 7062dd8460685d6700ed7621e50a22c6f3400ca3 safranowith bsh155762@gmail.com 2025-10-20T11:08:32+03:00 GitHub noreply@github.com 2025-10-20T11:08:32+03:00 SYCL: Add support for FLOOR,CEIL,ROUND and TRUNC unary operators (#16613) 7062dd8460685d6700ed7621e50a22c6f3400ca3 0398752dd450dfabdd1b9e289f6364c2600f6ab5 takuya kodama otegami@clear-code.com 2025-10-20T15:44:21+08:00 GitHub noreply@github.com 2025-10-20T10:44:21+03:00 llama-context: only warn on pooling_type when user specified (#16674) c20c0a5c0c44179f5267a262546624854b1203d1 62425b3a7e8b169a3bbe6ce8c7324513a7fcee8c Yunzhe Jia yunzhe@calculet.tech 2025-10-20T14:41:08+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-10-20T14:41:08+08:00 kv: turn on common-prefix mtmd: fix output size bug 0398752dd450dfabdd1b9e289f6364c2600f6ab5 4f73d0a95120687e2c527739f771330a5271259a Giuseppe Scrivano gscrivan@redhat.com 2025-10-19T23:54:31+02:00 GitHub noreply@github.com 2025-10-19T23:54:31+02:00 model : add Granite Hybrid types (#16635) 4f73d0a95120687e2c527739f771330a5271259a cec5edbcaec69bbf6d5851cabce4ac148be41701 Aaron Teo aaron.teo1@ibm.com 2025-10-20T05:06:39+08:00 GitHub noreply@github.com 2025-10-19T23:06:39+02:00 ci : fix binaries release failure for s390x (binaries may not work yet) (#16664) cec5edbcaec69bbf6d5851cabce4ac148be41701 fcb235b46618921cbd826acd49b553b5302233aa Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-19T14:03:25+02:00 GitHub noreply@github.com 2025-10-19T14:03:25+02:00 ci : avoid manual updates of docs/ops.md (#16663) fcb235b46618921cbd826acd49b553b5302233aa 55754bebd5d570960cde9c0ba991a0b2991f6b1e Aaron Teo aaron.teo1@ibm.com 2025-10-19T18:37:47+08:00 GitHub noreply@github.com 2025-10-19T18:37:47+08:00 ci: include s390x release binaries (#16648) 55754bebd5d570960cde9c0ba991a0b2991f6b1e ee09828cb057460b369576410601a3a09279e23c Aman Gupta amangupta052@gmail.com 2025-10-19T15:37:12+08:00 GitHub noreply@github.com 2025-10-19T10:37:12+03:00 CODEOWNERS: update for ggml-cuda/mmf (#16660) ee09828cb057460b369576410601a3a09279e23c e56abd2098dd2e2b0804691b93c13b48ae421627 Johannes Gäßler johannesg@5d6.de 2025-10-18T14:47:32+02:00 GitHub noreply@github.com 2025-10-18T14:47:32+02:00 HIP: fix GPU_TARGETS (#16642) e56abd2098dd2e2b0804691b93c13b48ae421627 38355c6c8e43204e11a22daa7483082c0ff01e71 Jeff Bolz jbolz@nvidia.com 2025-10-18T05:22:57-05:00 GitHub noreply@github.com 2025-10-18T12:22:57+02:00 vulkan: Implement topk_moe fused shader, ported from CUDA (#16641) 38355c6c8e43204e11a22daa7483082c0ff01e71 81387858f1fbcc1acedbd308486e1016618ca8f8 Aman Gupta amangupta052@gmail.com 2025-10-18T17:52:53+08:00 GitHub noreply@github.com 2025-10-18T11:52:53+02:00 CUDA: use registers instead of smem in topk-moe (#16647) 81387858f1fbcc1acedbd308486e1016618ca8f8 66b0dbcb2d462e7b70ba5a69ee8c3899ac2efb1c Shawn Gu shawngu@qti.qualcomm.com 2025-10-17T17:55:32-07:00 GitHub noreply@github.com 2025-10-17T17:55:32-07:00 opencl: transposed gemm/gemv moe kernel with mxfp4,f32 (#16602) 66b0dbcb2d462e7b70ba5a69ee8c3899ac2efb1c 41386cf365d894134ee0813d15e2f5d76f6a4d8e Johannes Gäßler johannesg@5d6.de 2025-10-17T17:41:09+02:00 GitHub noreply@github.com 2025-10-17T17:41:09+02:00 llama-model: fix insonsistent ctxs <-> bufs order (#16581) 41386cf365d894134ee0813d15e2f5d76f6a4d8e 3d4e86bbeb15f487d6da6174ba6191b7c212cc25 Radoslav Gerganov rgerganov@gmail.com 2025-10-17T18:02:52+03:00 GitHub noreply@github.com 2025-10-17T18:02:52+03:00 rpc : report actual free memory (#16616) 3d4e86bbeb15f487d6da6174ba6191b7c212cc25 342c728d031d50673feded797520a44127d73379 Giuseppe Scrivano gscrivan@redhat.com 2025-10-17T14:23:47+02:00 GitHub noreply@github.com 2025-10-17T14:23:47+02:00 vulkan: Add State Space Model (SSM) Operations Support (#16463) 342c728d031d50673feded797520a44127d73379 ababae7e1ec3e9cfdab0322ee55ea3389e82a4d5 muggle-stack promuggle@qq.com 2025-10-17T18:01:23+08:00 GitHub noreply@github.com 2025-10-17T13:01:23+03:00 ggml : fix SpaceMit IME array out-of-bounds in task assignment (#16629) ababae7e1ec3e9cfdab0322ee55ea3389e82a4d5 b19491599d4d42c606601d75e95c1d1de3291f8e Pascal admin@serveurperso.com 2025-10-17T10:35:03+02:00 GitHub noreply@github.com 2025-10-17T10:35:03+02:00 webui: reorganize settings layout (#16607) b19491599d4d42c606601d75e95c1d1de3291f8e 9ad4f1931ee0f3b41d9355245ef744786aaae0aa Jeff Bolz jbolz@nvidia.com 2025-10-17T02:31:04-05:00 GitHub noreply@github.com 2025-10-17T09:31:04+02:00 vulkan: fix debug build (add_rms_len/data not found) (#16624) 9ad4f1931ee0f3b41d9355245ef744786aaae0aa 79967ec596c0dacfd2251b085a57e79df292b1cc Ilia Ilmer iliailmer@users.noreply.github.com 2025-10-17T02:33:58-04:00 GitHub noreply@github.com 2025-10-17T09:33:58+03:00 metal : add `CONV_TRANSPOSE_2D` (#16542) 79967ec596c0dacfd2251b085a57e79df292b1cc ceff6bb253dd306f5404d7ccb3f11fadafe71b52 Olivier Chafik olivier.chafik@gmail.com 2025-10-17T06:59:31+01:00 GitHub noreply@github.com 2025-10-17T08:59:31+03:00 grammar : use int64_t to avoid int overflows in int schema to grammar conversion logic (#16626) ceff6bb253dd306f5404d7ccb3f11fadafe71b52 1bb4f43380944e94c9a86e305789ba103f5e62bd GittyBurstein g0534163997@gmail.com 2025-10-17T05:36:40+03:00 GitHub noreply@github.com 2025-10-17T10:36:40+08:00 SYCL SET operator optimized for F32 tensors (#16350) 1bb4f43380944e94c9a86e305789ba103f5e62bd 683fa6ba4ed3a23b939d3e11e6dc860bd47a0ccf Xuan-Son Nguyen son@huggingface.co 2025-10-16T19:00:31+02:00 GitHub noreply@github.com 2025-10-16T19:00:31+02:00 mtmd : support home-cooked Mistral Small Omni (#14928) 683fa6ba4ed3a23b939d3e11e6dc860bd47a0ccf b22572e97dc51757d3ebe917a5a283385010ec68 Pascal admin@serveurperso.com 2025-10-16T16:28:41+02:00 GitHub noreply@github.com 2025-10-16T16:28:41+02:00 fix: added a normalization step for MathJax-style \[\] and \(\) delimiters (#16599) b22572e97dc51757d3ebe917a5a283385010ec68 7a50cf388a530127cbbdd2a507ef81a451c9d819 GittyBurstein g0534163997@gmail.com 2025-10-16T16:26:21+03:00 GitHub noreply@github.com 2025-10-16T15:26:21+02:00 sycl : add ARANGE operator (#16362) 62425b3a7e8b169a3bbe6ce8c7324513a7fcee8c 76a63a2998286c1649de2496bccb7d8a46549895 Yunzhe Jia yunzhe@calculet.tech 2025-10-16T17:34:39+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-10-16T17:34:39+08:00 kv: let seq_rm free whole seq since current hw-op cannot support common-prefix 7a50cf388a530127cbbdd2a507ef81a451c9d819 6f5d924637a15abedb111cbbffd7da5f31c81855 Chenguang Li 757486878@qq.com 2025-10-16T16:41:11+08:00 GitHub noreply@github.com 2025-10-16T16:41:11+08:00 CANN: format code using .clang-format (#15863) 76a63a2998286c1649de2496bccb7d8a46549895 b67217078aa748b06e1b2269d88dd18c0aca2c26 Yunzhe Jia yunzhe@calculet.tech 2025-10-16T16:31:15+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-10-16T16:31:15+08:00 kv: fix common-prefix bug by implementing llama_memory_seq_rm 6f5d924637a15abedb111cbbffd7da5f31c81855 adc9b60f190c1016a09f439862fa1cbb302262ac takasurazeem takasurazeem@gmail.com 2025-10-16T01:11:33-04:00 GitHub noreply@github.com 2025-10-16T08:11:33+03:00 common : Update the docs on -t --threads (#16236) adc9b60f190c1016a09f439862fa1cbb302262ac ee50ee1eadff58777ae746827b04de7ba0befc55 takuya kodama a.s.takuya1026@gmail.com 2025-10-16T13:10:32+08:00 GitHub noreply@github.com 2025-10-16T08:10:32+03:00 ggml-cpu: replace putenv with setenv for const-correctness (#16573) ee50ee1eadff58777ae746827b04de7ba0befc55 7adc79c03234de9a20661fd6dbf2d02c32ca7acb yael-works 106673277+yael-works@users.noreply.github.com 2025-10-16T07:21:28+03:00 GitHub noreply@github.com 2025-10-16T12:21:28+08:00 SYCL: Add GGML_OP_MEAN operator support (#16009) 7adc79c03234de9a20661fd6dbf2d02c32ca7acb 466c1911ab736f0b7366127edee99f8ee5687417 Aleksei Nikiforov 103434461+AlekseiNikiforovIBM@users.noreply.github.com 2025-10-15T22:43:08+02:00 GitHub noreply@github.com 2025-10-15T22:43:08+02:00 gguf-py : add support for endian conversion of BF16 data (#16594) 466c1911ab736f0b7366127edee99f8ee5687417 0cb7a0683b0529172472d74d21f05470a607f297 safranowith bsh155762@gmail.com 2025-10-15T22:24:51+03:00 GitHub noreply@github.com 2025-10-15T21:24:51+02:00 cpu : add FLOOR, CEIL, ROUND and TRUNC unary operators (#16083) 0cb7a0683b0529172472d74d21f05470a607f297 d93f8439b08c4f35e13a41a7366901fdbe770fc8 lhez lih@qti.qualcomm.com 2025-10-15T10:51:04-07:00 GitHub noreply@github.com 2025-10-15T10:51:04-07:00 opencl: add q8_0 mm support (#16469) d93f8439b08c4f35e13a41a7366901fdbe770fc8 f9fb33f2630b4b4ba9081ce9c0c921f8cd8ba4eb lhez lih@qti.qualcomm.com 2025-10-15T10:48:28-07:00 GitHub noreply@github.com 2025-10-15T10:48:28-07:00 opencl: fix FA for f32 (#16584) f9fb33f2630b4b4ba9081ce9c0c921f8cd8ba4eb f4ce81c45e7bd910e36bf44c253fc5255c49b1e4 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-15T16:22:20+02:00 GitHub noreply@github.com 2025-10-15T16:22:20+02:00 Add server-driven parameter defaults and syncing (#16515) f4ce81c45e7bd910e36bf44c253fc5255c49b1e4 17304cbcc1dd24de7741cbe57925d58e90a98ac1 Sam/Samuel 57896620+cern1710@users.noreply.github.com 2025-10-15T23:05:56+09:00 GitHub noreply@github.com 2025-10-15T17:05:56+03:00 metal: optimise `GGML_OP_SUM` (#16559) 17304cbcc1dd24de7741cbe57925d58e90a98ac1 3e3cb19f6449f9168a128eaeae01f8f41b049acc Georgi Gerganov ggerganov@gmail.com 2025-10-15T16:53:12+03:00 GitHub noreply@github.com 2025-10-15T16:53:12+03:00 server : fix img token logs (#16595) 3e3cb19f6449f9168a128eaeae01f8f41b049acc 5acd455460f457942d8dd02e3dd9b1eebfce99fe Xuan-Son Nguyen son@huggingface.co 2025-10-15T14:48:08+02:00 GitHub noreply@github.com 2025-10-15T14:48:08+02:00 llama-quant: add support for mmproj (#16592) 5acd455460f457942d8dd02e3dd9b1eebfce99fe 554fd578a5ed78a10f371f5850c6a69aa83df15a Julius Tischbein ju.tischbein@gmail.com 2025-10-15T13:54:15+02:00 GitHub noreply@github.com 2025-10-15T14:54:15+03:00 CUDA: Changing the CUDA scheduling strategy to spin (#16585) 554fd578a5ed78a10f371f5850c6a69aa83df15a fa882fd2b1bcb663de23af06fdc391489d05b007 Georgi Gerganov ggerganov@gmail.com 2025-10-15T12:51:27+03:00 GitHub noreply@github.com 2025-10-15T11:51:27+02:00 server : fix mtmd checkpoints (#16591) fa882fd2b1bcb663de23af06fdc391489d05b007 ffa059034c1e41f3e58363ef3c46d2fcf854bc4d Georgi Gerganov ggerganov@gmail.com 2025-10-14T20:33:05+03:00 GitHub noreply@github.com 2025-10-14T20:33:05+03:00 metal : avoid using Metal's gpuAddress property (#16576) ffa059034c1e41f3e58363ef3c46d2fcf854bc4d 120bf7046d85a893f064c12abe58bfeebd735f84 SavicStefan 50296686+SavicStefan@users.noreply.github.com 2025-10-14T19:18:05+02:00 GitHub noreply@github.com 2025-10-14T19:18:05+02:00 vulkan: Add ACC_TYPE_VEC2 implementation (#16203) 120bf7046d85a893f064c12abe58bfeebd735f84 4258e0cfe72c72ad2787be1e9f93253e89219455 Aman Gupta amangupta052@gmail.com 2025-10-14T22:48:08+08:00 GitHub noreply@github.com 2025-10-14T07:48:08-07:00 CUDA + openCL: fix bug in accessing rms_norm->src while doing fusion (#16577) 4258e0cfe72c72ad2787be1e9f93253e89219455 7ea15bb64c81e3813eb0babf9a57e1bc5697f569 Jeff Bolz jbolz@nvidia.com 2025-10-14T08:53:37-05:00 GitHub noreply@github.com 2025-10-14T15:53:37+02:00 vulkan: Support FA with K/V in F32 (#16543) 7ea15bb64c81e3813eb0babf9a57e1bc5697f569 9c7185dd28416cf67f5e3b268381f311b5e3da56 Jeff Bolz jbolz@nvidia.com 2025-10-14T07:51:36-05:00 GitHub noreply@github.com 2025-10-14T14:51:36+02:00 vulkan: Improve build time for MSVC (#16545) 9c7185dd28416cf67f5e3b268381f311b5e3da56 1ee9d0b415cdf5240418c110a18b419f4002b154 Johannes Gäßler johannesg@5d6.de 2025-10-14T14:22:47+02:00 GitHub noreply@github.com 2025-10-14T14:22:47+02:00 CUDA: enable FA for FP32 KV cache (#16546) 1ee9d0b415cdf5240418c110a18b419f4002b154 48e2fa9fb7c2de1e53808fdb65ec33f916020fc4 Aman Gupta amangupta052@gmail.com 2025-10-14T19:16:21+08:00 GitHub noreply@github.com 2025-10-14T13:16:21+02:00 CUDA: use fastdiv + ggml_cuda_mad for mmvf (#16557) 48e2fa9fb7c2de1e53808fdb65ec33f916020fc4 5b6913c47b6bc71a6f927805a45387d5657d8b89 Aman Gupta amangupta052@gmail.com 2025-10-14T19:15:15+08:00 GitHub noreply@github.com 2025-10-14T13:15:15+02:00 CUDA: add fp kernel for larger batch size MoE (#16512) 5b6913c47b6bc71a6f927805a45387d5657d8b89 bc07349a7f87ba6eb31ed4b0ea9d9a7352185213 Anav Prasad anavp@nvidia.com 2025-10-14T09:53:49Z GitHub noreply@github.com 2025-10-14T11:53:49+02:00 cuda : remove legacy copy-op pointer indirection code (#16485) bc07349a7f87ba6eb31ed4b0ea9d9a7352185213 e60f241eacec42d3bd7c9edd37d236ebf35132a8 Georgi Gerganov ggerganov@gmail.com 2025-10-14T08:48:50+03:00 GitHub noreply@github.com 2025-10-14T08:48:50+03:00 server : dynamic token limit for prompt cache (#16560) e60f241eacec42d3bd7c9edd37d236ebf35132a8 e38b7c6e9e4453e3b3e96d76e38bc2ccb6bce458 Georgi Gerganov ggerganov@gmail.com 2025-10-13T23:07:57+03:00 GitHub noreply@github.com 2025-10-13T23:07:57+03:00 metal : FA support F32 K and V and head size = 32 (#16531) e38b7c6e9e4453e3b3e96d76e38bc2ccb6bce458 5016b7286240d29f8f640039989b84ea3a854344 Georgi Gerganov ggerganov@gmail.com 2025-10-13T22:42:37+03:00 GitHub noreply@github.com 2025-10-13T22:42:37+03:00 graph : support cacheless embeddings with FA and iSWA (#16528) 5016b7286240d29f8f640039989b84ea3a854344 7049736b2dd9011bf819e298b844ebbc4b5afdc9 lhez lih@qti.qualcomm.com 2025-10-13T11:50:37-07:00 GitHub noreply@github.com 2025-10-13T11:50:37-07:00 opencl: fix build targeting CL 2 (#16554) 7049736b2dd9011bf819e298b844ebbc4b5afdc9 01d2bdc2bc61b676706830305b286b08b9885a41 Johannes Gäßler johannesg@5d6.de 2025-10-13T16:29:45+02:00 GitHub noreply@github.com 2025-10-13T17:29:45+03:00 CUDA: fix numerical issues in tile FA kernel (#16540) 01d2bdc2bc61b676706830305b286b08b9885a41 56fc38b9655fbe1869d8bd6cfb269418196cea69 Jie Fu (傅杰) jiefu@tencent.com 2025-10-13T20:48:47+08:00 GitHub noreply@github.com 2025-10-13T15:48:47+03:00 ggml : fix build broken with -march=armv9-a on MacOS (#16520) 56fc38b9655fbe1869d8bd6cfb269418196cea69 1fb9504eb744969a990bfe4cfcf1d3d7a479541c Chenguang Li 757486878@qq.com 2025-10-13T17:01:24+08:00 GitHub noreply@github.com 2025-10-13T17:01:24+08:00 CANN: fix CPU memory leak in CANN backend (#16549) 1fb9504eb744969a990bfe4cfcf1d3d7a479541c 3f750f8d760ab5a61491e6a9409072dfeee4b4d7 Pascal admin@serveurperso.com 2025-10-13T10:55:32+02:00 GitHub noreply@github.com 2025-10-13T10:55:32+02:00 fix: add remark plugin to render raw HTML as literal text (#16505) b67217078aa748b06e1b2269d88dd18c0aca2c26 98c8269abefdfa8fde47dfb2769ceebc8b704e86 Yunzhe Jia yunzhe@calculet.tech 2025-10-13T11:48:14+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-10-13T16:52:48+08:00 add different path for unified and seperate compilation 3f750f8d760ab5a61491e6a9409072dfeee4b4d7 c515fc577166042234241c6bd0da9b08dcbe2bb9 Sam/Samuel 57896620+cern1710@users.noreply.github.com 2025-10-13T16:25:02+08:00 GitHub noreply@github.com 2025-10-13T11:25:02+03:00 metal: add support for opt_step_sgd (#16539) c515fc577166042234241c6bd0da9b08dcbe2bb9 f9bc66c3ebcfddb5f09e4b21253623caeb8e414a Georgi Gerganov ggerganov@gmail.com 2025-10-13T11:22:27+03:00 GitHub noreply@github.com 2025-10-13T11:22:27+03:00 ggml : fix scalar path for computing norm (#16558) f9bc66c3ebcfddb5f09e4b21253623caeb8e414a a31cf36ad946a13b3a646bf0dadf2a481e89f944 hipudding huafengchun@gmail.com 2025-10-13T08:52:22+08:00 GitHub noreply@github.com 2025-10-13T08:52:22+08:00 CANN: Update several operators to support FP16 data format (#16251) a31cf36ad946a13b3a646bf0dadf2a481e89f944 81d54bbfd599811b354c39f04550888168be7780 Sam/Samuel 57896620+cern1710@users.noreply.github.com 2025-10-13T02:43:14+08:00 GitHub noreply@github.com 2025-10-12T21:43:14+03:00 metal : add opt_step_adamw and op_sum (#16529) 81d54bbfd599811b354c39f04550888168be7780 c7be9febcbafa9af7d1b9443f86475c59c9c5f87 Pascal admin@serveurperso.com 2025-10-12T18:06:41+02:00 GitHub noreply@github.com 2025-10-12T18:06:41+02:00 webui: remove client-side context pre-check and rely on backend for limits (#16506) c7be9febcbafa9af7d1b9443f86475c59c9c5f87 8415f61e23d04427cd0d912fbb9d33b85f849456 Neo Zhang Jianyu jianyu.zhang@intel.com 2025-10-12T21:53:35+08:00 GitHub noreply@github.com 2025-10-12T21:53:35+08:00 [SYCL] fix UT fault cases: count-equal, argsort, pad OPs (#16521) 8415f61e23d04427cd0d912fbb9d33b85f849456 2c301e91abb92d03c1a682b4b540ba835562a74b Mathieu Baudier mbaudier@argeo.org 2025-10-12T15:48:03+02:00 GitHub noreply@github.com 2025-10-12T15:48:03+02:00 ci : add Vulkan on Ubuntu with default packages build (#16532) 2c301e91abb92d03c1a682b4b540ba835562a74b 4b2dae383df708e2afc49c4859a81cd074f5ac10 Aldehir Rojas hello@alde.dev 2025-10-12T08:18:47-05:00 GitHub noreply@github.com 2025-10-12T16:18:47+03:00 common : handle unicode during partial json parsing (#16526) 4b2dae383df708e2afc49c4859a81cd074f5ac10 41aac5c69b5fb281bc1f486afb053f78101bb39e Georgi Gerganov ggerganov@gmail.com 2025-10-12T09:29:13+03:00 GitHub noreply@github.com 2025-10-12T09:29:13+03:00 common : update presets (#16504) 41aac5c69b5fb281bc1f486afb053f78101bb39e a2fba89a426ff8005d303c73f0436e7e67368b70 sirus20x6 sirus20x6@users.noreply.github.com 2025-10-12T00:25:37-05:00 GitHub noreply@github.com 2025-10-12T08:25:37+03:00 ggml : Fix FP16 ELU positive branch (#16519) a2fba89a426ff8005d303c73f0436e7e67368b70 20cc625edc2264aae2779e71bef1593e6a4e8c43 Daniel Bevenius daniel.bevenius@gmail.com 2025-10-12T07:19:06+02:00 GitHub noreply@github.com 2025-10-12T07:19:06+02:00 hparams : add check for layer index in is_recurrent (#16511) 20cc625edc2264aae2779e71bef1593e6a4e8c43 11f0af5504252e453d57406a935480c909e3ff37 sirus20x6 sirus20x6@users.noreply.github.com 2025-10-12T00:15:00-05:00 GitHub noreply@github.com 2025-10-12T08:15:00+03:00 ggml: Correct SVE implementation in ggml_vec_dot_f16_unroll (#16518) 11f0af5504252e453d57406a935480c909e3ff37 a3cb04744fb5c591985f53b749fef5407d07a145 Johannes Gäßler johannesg@5d6.de 2025-10-11T20:54:32+02:00 GitHub noreply@github.com 2025-10-11T20:54:32+02:00 CUDA: faster tile FA, add oob checks, more HSs (#16492) a3cb04744fb5c591985f53b749fef5407d07a145 4a8fbe0a5eb75f339782ebcf29c17848122184d3 Georgi Gerganov ggerganov@gmail.com 2025-10-11T16:54:10+03:00 GitHub noreply@github.com 2025-10-11T16:54:10+03:00 metal : fix mul-mm condition + fix mul-mv permuted kernels (#16494) 4a8fbe0a5eb75f339782ebcf29c17848122184d3 31d0ff1869aa2ea31f4e96d5877d0343e9a2171b Pascal admin@serveurperso.com 2025-10-11T15:50:49+02:00 GitHub noreply@github.com 2025-10-11T15:50:49+02:00 feat: render user content as markdown option (#16358) 31d0ff1869aa2ea31f4e96d5877d0343e9a2171b 97870e64975b26c5e06a3540a8dc0ff601351e86 Yann Follet 131855179+YannFollet@users.noreply.github.com 2025-10-11T21:39:04+08:00 GitHub noreply@github.com 2025-10-11T16:39:04+03:00 server / ranking : add sorting and management of top_n (#16403) 97870e64975b26c5e06a3540a8dc0ff601351e86 477a66b03501cf3bd067f8968b77ca4d053ff1bd Diego Devesa slarengh@gmail.com 2025-10-11T04:02:26-07:00 GitHub noreply@github.com 2025-10-11T13:02:26+02:00 cuda : avoid initializing unused devices (#16510) 477a66b03501cf3bd067f8968b77ca4d053ff1bd e60f01d941bc5b7fae62dd57fee4cec76ec0ea6e amirai21 89905406+amirai21@users.noreply.github.com 2025-10-11T11:33:41+03:00 GitHub noreply@github.com 2025-10-11T10:33:41+02:00 convert : correctly handle LLaMA tokenizer for Jamba (#16470) e60f01d941bc5b7fae62dd57fee4cec76ec0ea6e 81086cd6a3ca1252f0dc0f938171648399179c53 Georgi Gerganov ggerganov@gmail.com 2025-10-10T22:15:05+03:00 GitHub noreply@github.com 2025-10-10T22:15:05+03:00 server : fix division by zero when reporting stats (#16501) 81086cd6a3ca1252f0dc0f938171648399179c53 68ee98ae181a5c83a5cc6261daeee69a1f588c15 Georgi Gerganov ggerganov@gmail.com 2025-10-10T17:17:31+03:00 GitHub noreply@github.com 2025-10-10T17:17:31+03:00 vocab : mark EOT token for Granite models (#16499) 68ee98ae181a5c83a5cc6261daeee69a1f588c15 cdb6da468cc33323955a523738d2e1675aeb5e9a Radoslav Gerganov rgerganov@gmail.com 2025-10-10T17:11:07+03:00 GitHub noreply@github.com 2025-10-10T16:11:07+02:00 server : return HTTP 400 if prompt exceeds context length (#16486) cdb6da468cc33323955a523738d2e1675aeb5e9a 6d69ab3f262eca3f0c6ad0ce075d2c80b9924d0e Radoslav Gerganov rgerganov@gmail.com 2025-10-10T13:22:27+03:00 GitHub noreply@github.com 2025-10-10T13:22:27+03:00 server : log requests to /v1/completions (#16495) 6d69ab3f262eca3f0c6ad0ce075d2c80b9924d0e 1faa13a1187051af66b0fd9f0d6effe4c77f0b3e Prajwal B Mehendarkar prajwal.b.mehendarkar@ibm.com 2025-10-10T13:45:46+05:30 GitHub noreply@github.com 2025-10-10T11:15:46+03:00 cmake : Dont define XOPENSOURCE on AIX (#16481) 1faa13a1187051af66b0fd9f0d6effe4c77f0b3e 1deee0f8d494981c32597dca8b5f8696d399b0f2 Pascal admin@serveurperso.com 2025-10-09T22:54:57+02:00 GitHub noreply@github.com 2025-10-09T22:54:57+02:00 webui: updated the chat service to only include max_tokens in the req… (#16489) 1deee0f8d494981c32597dca8b5f8696d399b0f2 d00cbea63c671cd85a57adaa50abf60b3b87d86f duduta simona.gherman@gmail.com 2025-10-09T22:11:15+03:00 GitHub noreply@github.com 2025-10-09T21:11:15+02:00 cpu : optimize the ggml NORM operation (#15953) d00cbea63c671cd85a57adaa50abf60b3b87d86f 8328fd4bae76fc027f8ca0e9a05acd3788dabe3f Georgi Gerganov ggerganov@gmail.com 2025-10-09T18:54:51+03:00 GitHub noreply@github.com 2025-10-09T18:54:51+03:00 server : host-memory prompt caching (#16391) 8328fd4bae76fc027f8ca0e9a05acd3788dabe3f 56b4795842d852152222ca7a2d4304008facf1b9 Pascal admin@serveurperso.com 2025-10-09T17:36:29+02:00 GitHub noreply@github.com 2025-10-09T17:36:29+02:00 No markdown in cot (#16483) 56b4795842d852152222ca7a2d4304008facf1b9 2c0d875ae6c6043fbbafd2831e160955e9ca6af1 Daniel Bevenius daniel.bevenius@gmail.com 2025-10-09T14:35:22+02:00 GitHub noreply@github.com 2025-10-09T14:35:22+02:00 model-conversion : add support for SentenceTransformers (#16387) 2c0d875ae6c6043fbbafd2831e160955e9ca6af1 aa4711d369b0d4adb6802b98ad6ea362f838710e sudhiarm sudhi.sathyavathy@arm.com 2025-10-09T09:13:18+01:00 GitHub noreply@github.com 2025-10-09T10:13:18+02:00 ci: add ARM64 Kleidiai build and test support (#16462) aa4711d369b0d4adb6802b98ad6ea362f838710e d80d6d2400b8faa80654c723cb5bdf9fc8f4db06 Chenguang Li 757486878@qq.com 2025-10-09T15:50:25+08:00 GitHub noreply@github.com 2025-10-09T15:50:25+08:00 CANN: Improve ACL graph matching (#16166) d80d6d2400b8faa80654c723cb5bdf9fc8f4db06 b2602137557b2b28a39e03612717d85ead9a6f5a Charles Xu charles.xu@arm.com 2025-10-09T09:29:17+02:00 GitHub noreply@github.com 2025-10-09T10:29:17+03:00 kleidiai: kernel interface refactoring (#16460) b2602137557b2b28a39e03612717d85ead9a6f5a e08db4259521de493b7aeb49dadf29ebd1ee966a Neo Zhang Jianyu jianyu.zhang@intel.com 2025-10-09T15:25:11+08:00 GitHub noreply@github.com 2025-10-09T10:25:11+03:00 [SYCL] refactor soft_max, add soft_max_back (#16472) e08db4259521de493b7aeb49dadf29ebd1ee966a 12bbc3fa50b6df03318a4451c9a2210200a0b28d Saba Fallah 10401143+sfallah@users.noreply.github.com 2025-10-09T08:39:18+02:00 GitHub noreply@github.com 2025-10-09T09:39:18+03:00 model: EmbeddingGemma Adding Support for SentenceTransformers Dense Modules (#16367) 12bbc3fa50b6df03318a4451c9a2210200a0b28d 9d0882840e6c3fb62965d03af0e22880ea90e012 Pascal admin@serveurperso.com 2025-10-08T22:18:41+02:00 GitHub noreply@github.com 2025-10-08T23:18:41+03:00 refactor: centralize CoT parsing in backend for streaming mode (#16394) 9d0882840e6c3fb62965d03af0e22880ea90e012 d2ee056e1df0fdf646270fb5621e9f92084b59a7 ai-fonsi length-amiss-7k@icloud.com 2025-10-08T20:21:46+02:00 GitHub noreply@github.com 2025-10-08T20:21:46+02:00 Disable CUDA host buffers on integrated GPUs (#16308) d2ee056e1df0fdf646270fb5621e9f92084b59a7 b2c08c9ec4cd59ed88c1ebdd94f109af5ccc978e issixx 46835150+issixx@users.noreply.github.com 2025-10-08T17:20:18+09:00 GitHub noreply@github.com 2025-10-08T11:20:18+03:00 server : fix cancel pending task (#16467) b2c08c9ec4cd59ed88c1ebdd94f109af5ccc978e 7fdd16b432d247121fe5fe7b21f8805f85266c85 Georgi Gerganov ggerganov@gmail.com 2025-10-08T10:57:53+03:00 GitHub noreply@github.com 2025-10-08T10:57:53+03:00 metal : mark FA blocks (#16372) 7fdd16b432d247121fe5fe7b21f8805f85266c85 74b8fc17f92ada295a648e3c5eb28f46bca7d892 Georgi Gerganov ggerganov@gmail.com 2025-10-08T10:57:29+03:00 GitHub noreply@github.com 2025-10-08T10:57:29+03:00 server : improve context checkpoint logic (#16440) 74b8fc17f92ada295a648e3c5eb28f46bca7d892 aeaf8a36f06b5810f5ae4bbefe26edb33925cf5e Reese Levine reeselevine1@gmail.com 2025-10-07T13:48:56-07:00 GitHub noreply@github.com 2025-10-07T13:48:56-07:00 ggml webgpu: profiling, CI updates, reworking of command submission (#16452) aeaf8a36f06b5810f5ae4bbefe26edb33925cf5e df1b612e29ba97a2e67db339b1e8c7465702b7e8 Tarek Dakhran tarek@liquid.ai 2025-10-07T20:03:35+02:00 GitHub noreply@github.com 2025-10-07T20:03:35+02:00 llama : support LiquidAI LFM2-MoE hybrid model (#16464) df1b612e29ba97a2e67db339b1e8c7465702b7e8 4e0388aa8a1f4ef1065701dc9c3947aea0a1b9a5 Georgi Gerganov ggerganov@gmail.com 2025-10-07T15:57:14+03:00 GitHub noreply@github.com 2025-10-07T15:57:14+03:00 server : add `/v1/health` endpoint (#16461) 4e0388aa8a1f4ef1065701dc9c3947aea0a1b9a5 ef4c5b87ea2556ff8ca99cca3abdf48bdbca22f2 Sascha Rogmann 59577610+srogmann@users.noreply.github.com 2025-10-07T11:11:08+02:00 GitHub noreply@github.com 2025-10-07T11:11:08+02:00 webui : added download action (#13552) (#16282) ef4c5b87ea2556ff8ca99cca3abdf48bdbca22f2 c61ae20d05bd4fdd8551311325a2845336449426 Georgi Gerganov ggerganov@gmail.com 2025-10-07T10:32:32+03:00 GitHub noreply@github.com 2025-10-07T10:32:32+03:00 presets : fix pooling param for embedding models (#16455) c61ae20d05bd4fdd8551311325a2845336449426 0123ff38f53d34752f29239a29d0e40a6dc4110f Radoslav Gerganov rgerganov@gmail.com 2025-10-07T09:59:13+03:00 GitHub noreply@github.com 2025-10-07T06:59:13Z rpc : update documentation (#16441) 0123ff38f53d34752f29239a29d0e40a6dc4110f 0a319bb75ed29d968e2a9b544011b09ccb932915 Georgi Gerganov ggerganov@gmail.com 2025-10-07T08:24:17+03:00 GitHub noreply@github.com 2025-10-07T08:24:17+03:00 memory : use sequential equal splits for recurrent modules (#16442) 0a319bb75ed29d968e2a9b544011b09ccb932915 1d6092fc72f4d10f4486ac95edfd414bc08b62b8 Georgi Gerganov ggerganov@gmail.com 2025-10-07T08:23:30+03:00 GitHub noreply@github.com 2025-10-07T08:23:30+03:00 metal : add support for non-padded FA KV (#16148) 1d6092fc72f4d10f4486ac95edfd414bc08b62b8 8ae32dc9ecd7aeebf3a5b43557e0552a0a04cd4f Georgi Gerganov ggerganov@gmail.com 2025-10-07T08:22:35+03:00 GitHub noreply@github.com 2025-10-07T08:22:35+03:00 tests : add -INF blocks to the KQ mask in the FA tests (#16380) 8ae32dc9ecd7aeebf3a5b43557e0552a0a04cd4f 3df2244df40c67dfd6ad548b40ccc507a066af2b Georgi Gerganov ggerganov@gmail.com 2025-10-07T08:21:40+03:00 GitHub noreply@github.com 2025-10-07T08:21:40+03:00 metal : various optimizations + refactoring (#16446) 3df2244df40c67dfd6ad548b40ccc507a066af2b c08002a1988348403a5fd59b1fa3de3a10a6f92f Gadflyii 34758915+Gadflyii@users.noreply.github.com 2025-10-06T12:55:53-05:00 GitHub noreply@github.com 2025-10-06T19:55:53+02:00 llama : add --no-host to disable host buffers (#16310) c08002a1988348403a5fd59b1fa3de3a10a6f92f 3a002afafa8e06555f11aed88b02d055d8a166f3 Gabe Goodhart ghart@us.ibm.com 2025-10-06T10:59:40-06:00 GitHub noreply@github.com 2025-10-06T18:59:40+02:00 chat : Granite Docling stopping (#16438) 3a002afafa8e06555f11aed88b02d055d8a166f3 a23b9bdbd3b64ce172f9962249f432d01aea7437 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-06T17:40:21+02:00 GitHub noreply@github.com 2025-10-06T17:40:21+02:00 ci : refactor sdk caching to minimize storage (#16414) a23b9bdbd3b64ce172f9962249f432d01aea7437 04e632a4aab8e6bfff0f8bc216b36ceb1e199ff9 Georgi Gerganov ggerganov@gmail.com 2025-10-06T16:05:27+03:00 GitHub noreply@github.com 2025-10-06T16:05:27+03:00 ggml : fix unaligned access in AMX code (#16315) 04e632a4aab8e6bfff0f8bc216b36ceb1e199ff9 a80ff183abe4e5a76316257ffa597da41b3b6fa0 Daniel Bevenius daniel.bevenius@gmail.com 2025-10-06T14:56:59+02:00 GitHub noreply@github.com 2025-10-06T14:56:59+02:00 ci : remove missing reranker model files (#16444) a80ff183abe4e5a76316257ffa597da41b3b6fa0 1d49ca37594fb49db6aa9518ba7c512e5ccd0108 Daniel Bevenius daniel.bevenius@gmail.com 2025-10-06T14:17:12+02:00 GitHub noreply@github.com 2025-10-06T14:17:12+02:00 ggml-cpu : fix leftover handling in ggml_vec_scale_f32 for SVE (#16443) 1d49ca37594fb49db6aa9518ba7c512e5ccd0108 c5fef0fcea3b978a2318b1af170209ecec7c37b4 Yuannan yuannan@users.noreply.github.com 2025-10-06T09:29:56Z GitHub noreply@github.com 2025-10-06T12:29:56+03:00 nix : removed metal for nix (#16118) c5fef0fcea3b978a2318b1af170209ecec7c37b4 ca71fb9b368e3db96e028f80c4c9df6b6b370edd Oleksandr Kuvshynov 661042+okuvshynov@users.noreply.github.com 2025-10-06T03:53:31-04:00 GitHub noreply@github.com 2025-10-06T10:53:31+03:00 server: update readme to mention n_past_max metric (#16436) ca71fb9b368e3db96e028f80c4c9df6b6b370edd 35266573b968e1c947b367782fb4b3eddbb4f3c0 Gabe Goodhart ghart@us.ibm.com 2025-10-05T06:57:47-06:00 GitHub noreply@github.com 2025-10-05T14:57:47+02:00 model : Granite docling + Idefics3 preprocessing (SmolVLM) (#16206) 35266573b968e1c947b367782fb4b3eddbb4f3c0 86df2c9ae4f2f1ee63d2558a9dc797b98524639b Reese Levine reeselevine1@gmail.com 2025-10-04T20:59:31-07:00 GitHub noreply@github.com 2025-10-04T20:59:31-07:00 ggml webgpu: actually add softmax, fix rms_norm offset (#16400) 86df2c9ae4f2f1ee63d2558a9dc797b98524639b f39283960b58a92ecc0c72567711318b20e22b55 Eve 139727413+netrunnereve@users.noreply.github.com 2025-10-04T20:04:27Z GitHub noreply@github.com 2025-10-04T22:04:27+02:00 vulkan: use a more appropriate amount of threads when generating shaders (#16418) f39283960b58a92ecc0c72567711318b20e22b55 898acba6816ad23b6a9491347d30e7570bffadfd Radoslav Gerganov rgerganov@gmail.com 2025-10-04T16:22:45+03:00 GitHub noreply@github.com 2025-10-04T16:22:45+03:00 rpc : check src buffer when copying tensor (#16421) 898acba6816ad23b6a9491347d30e7570bffadfd e29acf74fea996014380d59d31aa504ae8964258 Radoslav Gerganov rgerganov@gmail.com 2025-10-04T12:49:16+03:00 GitHub noreply@github.com 2025-10-04T12:49:16+03:00 rpc : add support for multiple devices (#16276) e29acf74fea996014380d59d31aa504ae8964258 128d522c04286e019666bd6ee4d18e3fbf8772e2 Acly aclysia@gmail.com 2025-10-04T11:42:56+02:00 GitHub noreply@github.com 2025-10-04T11:42:56+02:00 vulkan : incremental shader builds (#16341) 128d522c04286e019666bd6ee4d18e3fbf8772e2 f6dcda390004b627ef30af378d0c01ad2519289e Pascal admin@serveurperso.com 2025-10-03T20:51:48+02:00 GitHub noreply@github.com 2025-10-03T21:51:48+03:00 chat : support Magistral thinking (#16413) f6dcda390004b627ef30af378d0c01ad2519289e 606a73f53175077429484b23dcf799f69a31d0bd ddh0 dylanhalladay02@icloud.com 2025-10-03T13:34:51-05:00 GitHub noreply@github.com 2025-10-03T21:34:51+03:00 server : context checkpointing for hybrid and recurrent models (#16382) 606a73f53175077429484b23dcf799f69a31d0bd 946f71ed9ade07e319859b5ce656144140e066fb Georgi Gerganov ggerganov@gmail.com 2025-10-03T19:18:56+03:00 GitHub noreply@github.com 2025-10-03T19:18:56+03:00 metal : fix loop bound in ggml_mem_ranges (#16412) 946f71ed9ade07e319859b5ce656144140e066fb 638d330246954e88dffc22ce01fec15e6894e544 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-03T14:40:25+02:00 GitHub noreply@github.com 2025-10-03T14:40:25+02:00 llama : fix shapes for bert/mpt q/k norm (#16409) 638d330246954e88dffc22ce01fec15e6894e544 84c8e305e8010a1a3d43bdd0a25f737ac67809a4 Acly aclysia@gmail.com 2025-10-03T13:49:08+02:00 GitHub noreply@github.com 2025-10-03T13:49:08+02:00 ggml : fix graph reallocation with multiple chunks (#16396) 84c8e305e8010a1a3d43bdd0a25f737ac67809a4 2aaf0a2a2056d75d0dd53ab8a181473760e6ab22 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-03T12:51:40+02:00 GitHub noreply@github.com 2025-10-03T12:51:40+02:00 Fix missing messages on sibling navigation (#16408) 2aaf0a2a2056d75d0dd53ab8a181473760e6ab22 0e1f83855609d73beaf05d818640b6cfd39d287b Jeff Bolz jbolz@nvidia.com 2025-10-03T05:50:46-05:00 GitHub noreply@github.com 2025-10-03T12:50:46+02:00 vulkan: Replace uses of maxMemoryAllocationSize and VK_WHOLE_SIZE (#16354) 0e1f83855609d73beaf05d818640b6cfd39d287b ad126479c25cf983a0f994a08ba0911cf49ed62b Jeff Bolz jbolz@nvidia.com 2025-10-03T04:52:46-05:00 GitHub noreply@github.com 2025-10-03T11:52:46+02:00 vulkan: Fix FA coopmat1 invalid array indexing (#16365) ad126479c25cf983a0f994a08ba0911cf49ed62b 77233277c912d495d1069543ca540c21a2f9e5b4 Daniel Bevenius daniel.bevenius@gmail.com 2025-10-03T11:45:16+02:00 GitHub noreply@github.com 2025-10-03T11:45:16+02:00 ci : change macos-13 to macos-15-intel (#16401) 77233277c912d495d1069543ca540c21a2f9e5b4 e308efda8e54e3f07578937a45a5d83624eb7970 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-03T11:30:39+02:00 GitHub noreply@github.com 2025-10-03T11:30:39+02:00 Capture model name only after first token (streaming) or completed request (#16405) e308efda8e54e3f07578937a45a5d83624eb7970 136bda78c5679b266362b39c58338fff46fd2592 Jeff Bolz jbolz@nvidia.com 2025-10-03T03:33:08-05:00 GitHub noreply@github.com 2025-10-03T10:33:08+02:00 vulkan: in flash attention, bounds check against nem1 (don't rely on GGML_KQ_MASK_PAD) (#16316) 136bda78c5679b266362b39c58338fff46fd2592 5113efd34ceda709292de26c72716c49e024fb32 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-03T09:11:34+02:00 GitHub noreply@github.com 2025-10-03T10:11:34+03:00 webui : Fix messages payload sent to chat completions (#16402) 5113efd34ceda709292de26c72716c49e024fb32 d64c8104f090b27b1f99e8da5995ffcfa6b726e2 Pascal admin@serveurperso.com 2025-10-03T08:01:31+02:00 GitHub noreply@github.com 2025-10-03T08:01:31+02:00 fix: track viewportHeight via window.innerHeight to avoid unwanted scrolling (#16356) d64c8104f090b27b1f99e8da5995ffcfa6b726e2 ef07a4090672a3438d7f64f197795d7dc1c18957 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-02T20:10:12+02:00 GitHub noreply@github.com 2025-10-02T20:10:12+02:00 test-barrier : do not use more threads than physically available (#16389) ef07a4090672a3438d7f64f197795d7dc1c18957 34fcc5a4ace8c69476ef2ea3857f39a60334acc4 Reese Levine reeselevine1@gmail.com 2025-10-02T11:00:31-07:00 GitHub noreply@github.com 2025-10-02T11:00:31-07:00 ggml webgpu: add support for soft_max, optimize rms_norm (#16357) 34fcc5a4ace8c69476ef2ea3857f39a60334acc4 91a2a5655658bb9ab77894716b82fae7ecb4b4d1 Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-10-02T19:43:22+02:00 GitHub noreply@github.com 2025-10-02T20:43:22+03:00 model : Apertus model implementation (#15852) 91a2a5655658bb9ab77894716b82fae7ecb4b4d1 72ee736c447be4ededb51ab97904b4d33c90c261 R0CKSTAR yeahdongcn@gmail.com 2025-10-02T21:29:56+08:00 GitHub noreply@github.com 2025-10-02T16:29:56+03:00 musa: update compile flags (#16265) 72ee736c447be4ededb51ab97904b4d33c90c261 f09aefaa84d7f4d5df3f400f67944b94fef5b795 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-02T13:51:36+02:00 GitHub noreply@github.com 2025-10-02T13:51:36+02:00 ci : fix ubuntu-latest-cmake-rpc (disable ccache) (#16388) f09aefaa84d7f4d5df3f400f67944b94fef5b795 bbd32bc0384fbfcf07369617de58856b1e0e95a3 Eve 139727413+netrunnereve@users.noreply.github.com 2025-10-02T08:10:07Z GitHub noreply@github.com 2025-10-02T10:10:07+02:00 ci: update vulkan ci (#16294) bbd32bc0384fbfcf07369617de58856b1e0e95a3 2be72c2b121ee99f33927149265ce6073ade9e59 Georgi Gerganov ggerganov@gmail.com 2025-10-02T10:35:43+03:00 GitHub noreply@github.com 2025-10-02T10:35:43+03:00 ci : fix clean-up of old logs (#16381) 2be72c2b121ee99f33927149265ce6073ade9e59 95ce0985449c52fb9d6849b95563f7cf933ea0e3 Neo Zhang Jianyu jianyu.zhang@intel.com 2025-10-02T15:16:25+08:00 GitHub noreply@github.com 2025-10-02T10:16:25+03:00 SYCL: Update to oneAPI 2025.2 (#16371) 95ce0985449c52fb9d6849b95563f7cf933ea0e3 c8dedc9999eccf7821a9fe5b29f10e8d075e2217 uvos carl@uvos.xyz 2025-10-02T05:52:59+02:00 GitHub noreply@github.com 2025-10-02T05:52:59+02:00 HIP: add IMbackK to codeowner (#16375) c8dedc9999eccf7821a9fe5b29f10e8d075e2217 e95fec640f43623911a2cd5bda8b19b1898c530c uvos carl@uvos.xyz 2025-10-01T23:32:39+02:00 GitHub noreply@github.com 2025-10-01T23:32:39+02:00 CI: reenable cdna in rocm docker builds (#16376) e95fec640f43623911a2cd5bda8b19b1898c530c ded67b94446ef4f7fd988dbde7a12deef9870c13 uvos carl@uvos.xyz 2025-10-01T23:09:25+02:00 GitHub noreply@github.com 2025-10-01T23:09:25+02:00 HIP: Disable ROCWMMA fattn on CDNA when compiled against ROCWMMA 2.0.0 (#16221) ded67b94446ef4f7fd988dbde7a12deef9870c13 1fe4e38cc20af058ed320bd46cac934991190056 Shunta Saito shunta.saito@gmail.com 2025-10-02T06:08:15+09:00 GitHub noreply@github.com 2025-10-01T23:08:15+02:00 llama : parameter conversion and loading fixes for PLaMo2 variants (#16075) 1fe4e38cc20af058ed320bd46cac934991190056 4201deae9c2ae4732db8957b6ce0808d02ec597c uvos carl@uvos.xyz 2025-10-01T20:18:03+02:00 GitHub noreply@github.com 2025-10-01T20:18:03+02:00 ci: Properly install rocwmma for hip builds (#16305) 4201deae9c2ae4732db8957b6ce0808d02ec597c 764799279f801c696503bacca490b4358587d94c Adrien Gallouët adrien@gallouet.fr 2025-10-01T19:22:18+02:00 GitHub noreply@github.com 2025-10-01T20:22:18+03:00 common: introduce http.h for httplib-based client (#16373) 764799279f801c696503bacca490b4358587d94c 2a9b63383a448ec18c754dfdc6e95cb853940a52 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-01T18:18:10+02:00 GitHub noreply@github.com 2025-10-01T18:18:10+02:00 Conversation action dialogs as singletons from Chat Sidebar + apply conditional rendering for Actions Dropdown for Chat Conversation Items (#16369) 2a9b63383a448ec18c754dfdc6e95cb853940a52 1104ca1a1c926b832274694a2773a17066982ad0 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-01T15:54:42+02:00 GitHub noreply@github.com 2025-10-01T15:54:42+02:00 Improve code block color theming (#16325) 1104ca1a1c926b832274694a2773a17066982ad0 4f1575921cac9b489fe8f8bbf20aff985e7da45e Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-10-01T14:09:52+02:00 GitHub noreply@github.com 2025-10-01T14:09:52+02:00 ci : use registry cache for docker builds (#16366) 4f1575921cac9b489fe8f8bbf20aff985e7da45e 132d673554e65282e92505f4f77401ab971528bb Aleksander Grygier aleksander.grygier@gmail.com 2025-10-01T12:08:16+02:00 GitHub noreply@github.com 2025-10-01T12:08:16+02:00 Add optional setting for showing "Model used:" information (#16337) 132d673554e65282e92505f4f77401ab971528bb aa9538a63aef35f0224b2502f13b19d650a8ce04 Eve 139727413+netrunnereve@users.noreply.github.com 2025-10-01T07:56:36Z GitHub noreply@github.com 2025-10-01T09:56:36+02:00 vulkan: make ggml_vk_default_dispatcher support older vulkan headers (#16345) aa9538a63aef35f0224b2502f13b19d650a8ce04 e74c92e84236b2bab3f3c77bee4ead94928be360 Aleksander Grygier aleksander.grygier@gmail.com 2025-10-01T07:40:26+02:00 GitHub noreply@github.com 2025-10-01T08:40:26+03:00 webui: Remove running `llama-server` within WebUI `dev.sh` script (#16363) e74c92e84236b2bab3f3c77bee4ead94928be360 b2ba81dbe07b6dbea9c96b13346c66973dede32c Bartowski 3266127+bartowski1182@users.noreply.github.com 2025-09-30T16:24:36-04:00 GitHub noreply@github.com 2025-09-30T22:24:36+02:00 model : support GLM 4.6 (make a few NextN/MTP tensors not required) (#16359) b2ba81dbe07b6dbea9c96b13346c66973dede32c bf6f3b3a1965d70e07ca94aab7b01268fe483e96 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-30T21:41:42+02:00 GitHub noreply@github.com 2025-09-30T21:41:42+02:00 ci : fix ccache key for ubuntu-cpu-cmake (#16355) bf6f3b3a1965d70e07ca94aab7b01268fe483e96 7c156df4148eb524b22f7f363bfd2df00f264e0b Adrien Gallouët angt@huggingface.co 2025-09-30T19:52:41+02:00 GitHub noreply@github.com 2025-09-30T20:52:41+03:00 common : disable progress bar without a tty (#16352) 7c156df4148eb524b22f7f363bfd2df00f264e0b 16b0ca0d2e63fc1a0a43795f8552f9cb61d9f7a5 lhez lih@qti.qualcomm.com 2025-09-30T10:45:45-07:00 GitHub noreply@github.com 2025-09-30T10:45:45-07:00 opencl: support pad_ext (#15888) 16b0ca0d2e63fc1a0a43795f8552f9cb61d9f7a5 8d78cd2613ccdeb3cc86f59bc8f9ddd31cfbd3ed Pascal admin@serveurperso.com 2025-09-30T19:18:54+02:00 GitHub noreply@github.com 2025-09-30T19:18:54+02:00 Chatapi ignore empty sampling (#16330) 8d78cd2613ccdeb3cc86f59bc8f9ddd31cfbd3ed d1c84a662daa91be975863913a59975b11458141 Reese Levine reeselevine1@gmail.com 2025-09-30T09:57:51-07:00 GitHub noreply@github.com 2025-09-30T09:57:51-07:00 ggml webgpu: support for rope,div,sub,glu,scale,cont operators (#16187) d1c84a662daa91be975863913a59975b11458141 364a7a6d4a786e98947c8a90430ea581213c0ba9 lhez lih@qti.qualcomm.com 2025-09-30T09:55:13-07:00 GitHub noreply@github.com 2025-09-30T09:55:13-07:00 opencl: support ne3 in get_rows (#15866) 364a7a6d4a786e98947c8a90430ea581213c0ba9 2df5bcf357dba0c49b4df1d684b2ffc9e88b7054 Adrien Gallouët angt@huggingface.co 2025-09-30T16:39:44+02:00 GitHub noreply@github.com 2025-09-30T17:39:44+03:00 common : remove common_has_curl() (#16351) 2df5bcf357dba0c49b4df1d684b2ffc9e88b7054 075c01567bb7e93965ae60b7e119182c3e68f3e9 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-30T15:38:01+02:00 GitHub noreply@github.com 2025-09-30T15:38:01+02:00 ci : disable ccache for android (#16348) 075c01567bb7e93965ae60b7e119182c3e68f3e9 a014310374a16f9204f2bcc1b458fc1eda67e469 Georgi Gerganov ggerganov@gmail.com 2025-09-30T13:42:39+03:00 Georgi Gerganov ggerganov@gmail.com 2025-09-30T13:53:55+03:00 ggml : bump version to 0.9.4 (ggml/1363) 98c8269abefdfa8fde47dfb2769ceebc8b704e86 2790ecc304e384d9075c55d0628d8bf440025dc4 Yunzhe Jia yunzhe@calculet.tech 2025-09-30T17:42:57+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-09-30T17:44:27+08:00 server: adapt to calrt 2790ecc304e384d9075c55d0628d8bf440025dc4 f528e97d894b4d93934a0711a00253eea980e799 Yunzhe Jia yunzhe@calculet.tech 2025-09-30T17:37:58+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-09-30T17:44:27+08:00 adapt to calrt f528e97d894b4d93934a0711a00253eea980e799 dcbe998a6d8f13b6c9cb36bbe7b355ced2ac3842 Yunzhe Jia yunzhe@calculet.tech 2025-09-11T14:13:35+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-09-30T17:44:27+08:00 add calrt mtmd support a014310374a16f9204f2bcc1b458fc1eda67e469 35fb82497ec6c5904b0adb7e1c881a76c1c692db anavp-nvidia anavp@nvidia.com 2025-09-30T08:13:22Z GitHub noreply@github.com 2025-09-30T11:13:22+03:00 cuda : Enable CUDA Graph usage for Nemotron Nano v2 (NemotronH) (#16328) 35fb82497ec6c5904b0adb7e1c881a76c1c692db 3c62aed89fbe3e15f6f82035e805a0a1add26306 Georgi Gerganov ggerganov@gmail.com 2025-09-30T11:03:23+03:00 GitHub noreply@github.com 2025-09-30T11:03:23+03:00 metal : dynamic simdgroups for MV kernels (#16340) 3c62aed89fbe3e15f6f82035e805a0a1add26306 f1eb1cb1eba042b2d583234e80f65e2e225d8995 Adrien Gallouët angt@huggingface.co 2025-09-30T09:36:33+02:00 GitHub noreply@github.com 2025-09-30T10:36:33+03:00 common : simplify etag tracking by removing json (#16342) f1eb1cb1eba042b2d583234e80f65e2e225d8995 de41f2b7bffab7582944b213ce12b605f8cf6e0f Charles Xu charles.xu@arm.com 2025-09-30T09:07:20+02:00 GitHub noreply@github.com 2025-09-30T10:07:20+03:00 kleidiai : fix work size and threads sync for fp16 (#16246) de41f2b7bffab7582944b213ce12b605f8cf6e0f a74a0d69f34f52fa10d4f0a7ce749fb3490d0774 lhez lih@qti.qualcomm.com 2025-09-29T22:30:16-07:00 GitHub noreply@github.com 2025-09-30T08:30:16+03:00 codeowners: add codeowners for opencl backend (#16344) a74a0d69f34f52fa10d4f0a7ce749fb3490d0774 5f7e166cbf7b9ca928c7fad990098ef32358ac75 Jeff Bolz jbolz@nvidia.com 2025-09-29T19:26:34-05:00 GitHub noreply@github.com 2025-09-29T19:26:34-05:00 tests: override test_set_rows::max_nmse_err to allow for occasional rounding differences (#16295) 5f7e166cbf7b9ca928c7fad990098ef32358ac75 d72f5f7ba260b546190338b0b76f2f152581424f Pascal admin@serveurperso.com 2025-09-29T18:49:47+02:00 GitHub noreply@github.com 2025-09-29T18:49:47+02:00 Fix thinking blocks with quotes + add handling `[THINK]...[/THINK]` blocks (#16326) d72f5f7ba260b546190338b0b76f2f152581424f b77e6c18e1a6fac5705ed95f03af5436d67484c1 Georgi Gerganov ggerganov@gmail.com 2025-09-29T17:51:48+03:00 GitHub noreply@github.com 2025-09-29T17:51:48+03:00 ci : add AMD runners and workflows (#16249) b77e6c18e1a6fac5705ed95f03af5436d67484c1 2ddd3f2356c313385fafc19a9f0ce678c8fe03ee alex-spacemit jinghui.huang@spacemit.com 2025-09-29T22:50:44+08:00 GitHub noreply@github.com 2025-09-29T17:50:44+03:00 ggml: riscv: add riscv spacemit backend (#15288) 2ddd3f2356c313385fafc19a9f0ce678c8fe03ee 4d3d455d3c8719eb8f206de328d1b9ba191efd7c Georgi Gerganov ggerganov@gmail.com 2025-09-29T16:50:52+03:00 Georgi Gerganov ggerganov@gmail.com 2025-09-29T17:43:58+03:00 sync : ggml 4d3d455d3c8719eb8f206de328d1b9ba191efd7c c9b1c06467aca8d30fafcab51292bcb285df5b0d Georgi Gerganov ggerganov@gmail.com 2025-09-29T16:49:11+03:00 Georgi Gerganov ggerganov@gmail.com 2025-09-29T17:43:58+03:00 sync : whisper.cpp (ggml/1359) c9b1c06467aca8d30fafcab51292bcb285df5b0d b6ae75afb49b23db3dfbc2b01e8aabfb047e6880 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-26T17:34:42+02:00 Georgi Gerganov ggerganov@gmail.com 2025-09-29T17:43:58+03:00 ggml : remove -dev suffix from release version (ggml/1355) b6ae75afb49b23db3dfbc2b01e8aabfb047e6880 b6dff20e2fe352093039e2359370c6bb2b505e0b Daniel Bevenius daniel.bevenius@gmail.com 2025-09-25T14:39:05+02:00 Georgi Gerganov ggerganov@gmail.com 2025-09-29T17:43:58+03:00 ggml : bump version to 0.9.3 (ggml/1353) b6dff20e2fe352093039e2359370c6bb2b505e0b 2db78c75e4ef7497313fda6dbbb3fcaf9ca0752d Georgi Gerganov ggerganov@gmail.com 2025-09-20T16:44:23+03:00 Georgi Gerganov ggerganov@gmail.com 2025-09-29T17:43:58+03:00 ggml : prepare for development of 0.9.2-dev 2db78c75e4ef7497313fda6dbbb3fcaf9ca0752d 02463ab27b1379ff5e3d936ce8b3bfd356872ea6 Georgi Gerganov ggerganov@gmail.com 2025-09-20T16:44:23+03:00 Georgi Gerganov ggerganov@gmail.com 2025-09-29T17:43:58+03:00 ggml : bump version to 0.9.1 02463ab27b1379ff5e3d936ce8b3bfd356872ea6 adc76347d73b3d915e946efa5de8d0ad9f3904c2 Rafal Lewczuk rafal.lewczuk@gmail.com 2025-09-29T13:17:09+02:00 GitHub noreply@github.com 2025-09-29T13:17:09+02:00 ggml-backend : add root cause in error message if loading backend library fails (#16172) adc76347d73b3d915e946efa5de8d0ad9f3904c2 3a2bdcda0b31e51db954551e0cd49424133a84f3 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-29T11:09:00+02:00 GitHub noreply@github.com 2025-09-29T11:09:00+02:00 ggml : check cuda and metal argsort limits and add test (#16323) 3a2bdcda0b31e51db954551e0cd49424133a84f3 66bb7985c3fcefda7a74aae9f8321f70eb1646c4 Aleksander Grygier aleksander.grygier@gmail.com 2025-09-29T10:37:20+02:00 GitHub noreply@github.com 2025-09-29T10:37:20+02:00 Improve Mobile UI for dialogs and action dropdowns (#16222) 66bb7985c3fcefda7a74aae9f8321f70eb1646c4 2f61c0f5bf8a620ca4c3872408803ab38cfb9613 Pascal admin@serveurperso.com 2025-09-29T09:08:41+02:00 GitHub noreply@github.com 2025-09-29T09:08:41+02:00 fix: preserved zero values in chat settings inputs and textareas by switching to nullish coalescing for field values and default placeholders (#16312) 2f61c0f5bf8a620ca4c3872408803ab38cfb9613 3ffd0fae473c954bb3e67526b31262048fb508d4 Vinkal vinkal-chudgar@users.noreply.github.com 2025-09-29T12:33:12+05:30 GitHub noreply@github.com 2025-09-29T10:03:12+03:00 llama-cli: prevent spurious assistant token (#16202) 3ffd0fae473c954bb3e67526b31262048fb508d4 a4a0aa5ea2a88e4858996199fefd5439f17b481c ddh0 chemist-mulches-39@icloud.com 2025-09-29T01:30:45-05:00 GitHub noreply@github.com 2025-09-29T09:30:45+03:00 perplexity : show more kl-divergence data (#16321) a4a0aa5ea2a88e4858996199fefd5439f17b481c 92cd103f627015a95f2107f1c27dc218c7b8ec64 Georgi Gerganov ggerganov@gmail.com 2025-09-29T08:41:28+03:00 GitHub noreply@github.com 2025-09-29T08:41:28+03:00 ggml : fix dependencies for ggml_set_rows (#16318) 92cd103f627015a95f2107f1c27dc218c7b8ec64 b887d2f3413ac231e3cb5925260c39902af4a70c Jeff Bolz jbolz@nvidia.com 2025-09-28T23:50:37-05:00 GitHub noreply@github.com 2025-09-29T06:50:37+02:00 vulkan: Fix validation failure in quantized flash attention (#16292) b887d2f3413ac231e3cb5925260c39902af4a70c bd0af02fc96c2057726f33c0f0daf7bb8f3e462a Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-28T23:15:03+02:00 GitHub noreply@github.com 2025-09-28T23:15:03+02:00 ggml : fix GGML_F32_VEC_FMA argument order in ggml_vec_mad1_f32 (#16307) bd0af02fc96c2057726f33c0f0daf7bb8f3e462a d9e0e7c8194dfd7d23bf3a86608c9ece68d77c93 crat0z 11581854+crat0z@users.noreply.github.com 2025-09-28T14:13:50-04:00 GitHub noreply@github.com 2025-09-28T21:13:50+03:00 common : fix reasoning before forced tool call via tool_choice = required (#16264) d9e0e7c8194dfd7d23bf3a86608c9ece68d77c93 0124ac989f7e7bf08803788f66dbe4106bdcdd58 R0CKSTAR yeahdongcn@gmail.com 2025-09-28T22:38:15+08:00 GitHub noreply@github.com 2025-09-28T16:38:15+02:00 ci : fix musa docker build (#16306) 0124ac989f7e7bf08803788f66dbe4106bdcdd58 2811c65286ae954bec87049f75b86dc022006dcc Aaron Teo aaron.teo1@ibm.com 2025-09-28T19:25:58+08:00 GitHub noreply@github.com 2025-09-28T19:25:58+08:00 devops: switch to using ubuntu-22.04-s390x image (#16302) 2811c65286ae954bec87049f75b86dc022006dcc d8359f5fde480da030bf75c7711573c7c4d993ba Imad Saddik 79410781+ImadSaddik@users.noreply.github.com 2025-09-28T12:04:46+01:00 GitHub noreply@github.com 2025-09-28T13:04:46+02:00 Fixed a few typos in the README of the LLaMA.cpp HTTP Server [no ci] (#16297) d8359f5fde480da030bf75c7711573c7c4d993ba 6a2c6145a0b91b40eb3c3dba7b20ccc4b270490f Jeff Bolz jbolz@nvidia.com 2025-09-28T01:38:37-05:00 GitHub noreply@github.com 2025-09-28T08:38:37+02:00 vulkan: 64-bit im2col (#16135) 6a2c6145a0b91b40eb3c3dba7b20ccc4b270490f 3b53634fe35771e2e318227aa81585726bae7234 Georgi Gerganov ggerganov@gmail.com 2025-09-28T09:34:44+03:00 GitHub noreply@github.com 2025-09-28T09:34:44+03:00 metal : extend mat-mat multiplication support (#16225) 3b53634fe35771e2e318227aa81585726bae7234 1384abf8b8d5894d32fada453ccf4d196ffba7de Georgi Gerganov ggerganov@gmail.com 2025-09-28T09:34:05+03:00 GitHub noreply@github.com 2025-09-28T09:34:05+03:00 metal : fuse non-sequential nodes (#16102) 1384abf8b8d5894d32fada453ccf4d196ffba7de e6d65fb02d553bd79cad94e517cdca18b687788d Jeff Bolz jbolz@nvidia.com 2025-09-27T20:36:34-05:00 GitHub noreply@github.com 2025-09-27T20:36:34-05:00 vulkan: handle mat_mul with A matrix > 4GB (#16176) e6d65fb02d553bd79cad94e517cdca18b687788d 8656f5de688cddcaea1d6174535eb60ee23ef6a0 Jeff Bolz jbolz@nvidia.com 2025-09-27T16:43:39-04:00 GitHub noreply@github.com 2025-09-27T22:43:39+02:00 vulkan: support arbitrary KV dimension in flash attention (#16160) 8656f5de688cddcaea1d6174535eb60ee23ef6a0 4807e8f96a61b2adccebd5e57444c94d18de7264 Acly aclysia@gmail.com 2025-09-27T22:41:03+02:00 GitHub noreply@github.com 2025-09-27T22:41:03+02:00 vulkan : make the vulkan.hpp dynamic dispatcher instance private (#16224) 4807e8f96a61b2adccebd5e57444c94d18de7264 c0bfc57af421f8fd63c946c13b7666aed82560e2 Aleksander Grygier aleksander.grygier@gmail.com 2025-09-27T19:56:40+02:00 GitHub noreply@github.com 2025-09-27T19:56:40+02:00 Show message actions by default (#16289) c0bfc57af421f8fd63c946c13b7666aed82560e2 75a3a6c2cd0002ba40e2dcc92007bc9fdbc69f1a Aman Gupta amangupta052@gmail.com 2025-09-28T00:49:32+08:00 GitHub noreply@github.com 2025-09-27T18:49:32+02:00 CUDA: mul_mat_id for mmf for bs <= 64 for f16 and bs <= 32 for f32 (#16277) 75a3a6c2cd0002ba40e2dcc92007bc9fdbc69f1a 0499b29c6f64c705faaf5860dc4600fca23671f4 Johannes Gäßler johannesg@5d6.de 2025-09-27T18:45:07+02:00 GitHub noreply@github.com 2025-09-27T18:45:07+02:00 CUDA: refactor and deduplicate vector FA kernels (#16208) 0499b29c6f64c705faaf5860dc4600fca23671f4 234e2ff8ed09716fb553437596779399bee31b11 Dmytro Minochkin dmytro.minochkin@gmail.com 2025-09-27T19:26:46+03:00 GitHub noreply@github.com 2025-09-27T18:26:46+02:00 vulkan: throw system error instead of SIGABRT during init on older devices (#16156) 234e2ff8ed09716fb553437596779399bee31b11 3f81b4e91c1d5f098148af117e3f13cf4b077f52 Adrien Gallouët angt@huggingface.co 2025-09-27T18:17:08+02:00 GitHub noreply@github.com 2025-09-27T19:17:08+03:00 server : remove old LLAMA_SERVER_SSL (#16290) 3f81b4e91c1d5f098148af117e3f13cf4b077f52 ace6a54565444b6377bee8e7ac693238e7766279 Jeff Bolz jbolz@nvidia.com 2025-09-27T06:36:11-04:00 GitHub noreply@github.com 2025-09-27T12:36:11+02:00 vulkan: support GET_ROWS for k-quants (#16235) ace6a54565444b6377bee8e7ac693238e7766279 72b24d96c6888c609d562779a23787304ae4609c Adrien Gallouët angt@huggingface.co 2025-09-27T11:12:46+02:00 GitHub noreply@github.com 2025-09-27T12:12:46+03:00 build : add LLAMA_OPENSSL option (#16287) 72b24d96c6888c609d562779a23787304ae4609c 624207e676ab5eb3ce7af631902bb45fb73a8359 Vinkal vinkal-chudgar@users.noreply.github.com 2025-09-27T02:58:29+05:30 GitHub noreply@github.com 2025-09-26T23:28:29+02:00 model : make minicpm embedding_scale, residual_scale and logit_scale optional with legacy defaults (#16273) 624207e676ab5eb3ce7af631902bb45fb73a8359 807e8c6d310952f2f5656afc63dab9d7083dcb5c Aaron Teo aaron.teo1@ibm.com 2025-09-27T02:03:33+08:00 GitHub noreply@github.com 2025-09-27T02:03:33+08:00 devops: add s390x & ppc64le CI (#15925) 807e8c6d310952f2f5656afc63dab9d7083dcb5c 1a189278944d030211f336e103e96b65f976c361 Aleksander Grygier aleksander.grygier@gmail.com 2025-09-26T19:25:29+02:00 GitHub noreply@github.com 2025-09-26T19:25:29+02:00 Enhance text file detection logic for file attachments (#16199) 1a189278944d030211f336e103e96b65f976c361 e0539eb6aed346d4b25a6ea019044e88771e7690 Aleksander Grygier aleksander.grygier@gmail.com 2025-09-26T18:35:42+02:00 GitHub noreply@github.com 2025-09-26T18:35:42+02:00 Allow viewing conversations even when llama server is down (#16255) e0539eb6aed346d4b25a6ea019044e88771e7690 5d0a40f390732cbf85d8a3b7b0fc3cbebffe780a Isaac McFadyen isaac@imcf.me 2025-09-26T11:36:48-04:00 GitHub noreply@github.com 2025-09-26T18:36:48+03:00 webui: switch to hash-based routing (alternative of #16079) (#16157) 5d0a40f390732cbf85d8a3b7b0fc3cbebffe780a d12a9836597b1ae4440d64d5a6ed727d27ac0702 Aleksander Grygier aleksander.grygier@gmail.com 2025-09-26T15:59:07+02:00 GitHub noreply@github.com 2025-09-26T15:59:07+02:00 Always show message actions for mobile UI + improvements for user message sizing (#16076) d12a9836597b1ae4440d64d5a6ed727d27ac0702 cc1cfa277b3aae1f6cc9180472072336597a78d4 Radoslav Gerganov rgerganov@gmail.com 2025-09-26T16:09:34+03:00 GitHub noreply@github.com 2025-09-26T16:09:34+03:00 codeowners : add rgerganov as owner of RPC [no ci] (#16279) cc1cfa277b3aae1f6cc9180472072336597a78d4 54dbc37053f7d75ea5b0631d5ee88f19391e4314 Aleksei Nikiforov 103434461+AlekseiNikiforovIBM@users.noreply.github.com 2025-09-26T15:00:44+02:00 GitHub noreply@github.com 2025-09-26T15:00:44+02:00 mtmd : fix uninitialized variable in bicubic_resize (#16275) 54dbc37053f7d75ea5b0631d5ee88f19391e4314 b995a10760cb93d23d617d76ecb82a5f95b5e0d3 Georgi Gerganov ggerganov@gmail.com 2025-09-26T14:14:28+03:00 GitHub noreply@github.com 2025-09-26T14:14:28+03:00 metal : report OOM errors (#16274) b995a10760cb93d23d617d76ecb82a5f95b5e0d3 4710dd31bbcef79d04f85a3a6a8c7d9439c5c79a Adrien Gallouët angt@huggingface.co 2025-09-26T13:12:19+02:00 GitHub noreply@github.com 2025-09-26T14:12:19+03:00 common : use cpp-httplib as a cURL alternative for downloads (#16185) 4710dd31bbcef79d04f85a3a6a8c7d9439c5c79a 9b26511857ac09ae69ab485168fe2d3ee5fb1d6e Adrien Gallouët angt@huggingface.co 2025-09-26T12:39:35+02:00 GitHub noreply@github.com 2025-09-26T13:39:35+03:00 build : fix build-ios-device (#16257) 9b26511857ac09ae69ab485168fe2d3ee5fb1d6e 00217cd41388328c93e9e9644921c4319bb03bcc Aaron Teo aaron.teo1@ibm.com 2025-09-26T18:27:25+08:00 GitHub noreply@github.com 2025-09-26T13:27:25+03:00 ggml-cpu: implement MXFP4 SIMD for s390x (#16193) 00217cd41388328c93e9e9644921c4319bb03bcc 3b337b01a1a85c2f5b49376e8c0bbdb3f521528e Radoslav Gerganov rgerganov@gmail.com 2025-09-26T13:19:23+03:00 GitHub noreply@github.com 2025-09-26T10:19:23Z ci : create git tags for released docker images (#16008) 3b337b01a1a85c2f5b49376e8c0bbdb3f521528e a86a580a6692edd1da076d5422f944c344b4fe5e Daniel Bevenius daniel.bevenius@gmail.com 2025-09-26T07:53:36+02:00 GitHub noreply@github.com 2025-09-26T08:53:36+03:00 codeowners : add danbev as owner of build-xcframework.sh [no ci] (#16268) a86a580a6692edd1da076d5422f944c344b4fe5e 0f7c69689f4e681948a6005adea9bee9c08ff903 R0CKSTAR yeahdongcn@gmail.com 2025-09-26T08:56:38+08:00 GitHub noreply@github.com 2025-09-26T02:56:38+02:00 musa: upgrade musa sdk to 4.3.0 (#16240) 0f7c69689f4e681948a6005adea9bee9c08ff903 835b2b915c52bcabcd688d025eacff9a07b65f52 R0CKSTAR yeahdongcn@gmail.com 2025-09-26T08:56:10+08:00 GitHub noreply@github.com 2025-09-26T02:56:10+02:00 musa: fix build warnings (#15611) 835b2b915c52bcabcd688d025eacff9a07b65f52 b05a9d650f4da1e70e7e2cbe8fe44e61b39656db Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-25T19:50:28+02:00 GitHub noreply@github.com 2025-09-25T19:50:28+02:00 model : add GroveMoE support (#15510) b05a9d650f4da1e70e7e2cbe8fe44e61b39656db 27052978e487957b3d39e3bd8fc8ee4aa304bff9 Aaron Teo aaron.teo1@ibm.com 2025-09-25T23:38:10+08:00 GitHub noreply@github.com 2025-09-25T23:38:10+08:00 vendors: update miniaudio version (#16212) 27052978e487957b3d39e3bd8fc8ee4aa304bff9 077c94d0caf87fbd3cf3288dbb5c0fd9670294cf rtaluyev taluyev@gmail.com 2025-09-25T18:20:34+03:00 GitHub noreply@github.com 2025-09-25T18:20:34+03:00 readme : update bindings (#16144) 077c94d0caf87fbd3cf3288dbb5c0fd9670294cf aa3ee0eb0b80efca126cedf9bcb4fb5864b46ce3 Aman Gupta amangupta052@gmail.com 2025-09-25T22:35:05+08:00 GitHub noreply@github.com 2025-09-25T16:35:05+02:00 CUDA: add a fused top-K MoE kernel (#16130) aa3ee0eb0b80efca126cedf9bcb4fb5864b46ce3 d0991da39d3c39b3980c24cdfccb9a4c95a46870 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-25T12:02:36+02:00 GitHub noreply@github.com 2025-09-25T12:02:36+02:00 model-conversion : add embedding prompt file support (#15871) d0991da39d3c39b3980c24cdfccb9a4c95a46870 aa719c2f886c445b78113bf1e5849f1fce4124a7 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-25T11:36:47+02:00 GitHub noreply@github.com 2025-09-25T11:36:47+02:00 server : add support for external server for tests (#16243) aa719c2f886c445b78113bf1e5849f1fce4124a7 4cdd0bb4537f9617e9efcfef6b9454fcefe2ff08 junchao-zhao 68935141+junchao-loongson@users.noreply.github.com 2025-09-25T17:22:55+08:00 GitHub noreply@github.com 2025-09-25T12:22:55+03:00 ggml : fix loongarch lsx compilation error (#15864) 4cdd0bb4537f9617e9efcfef6b9454fcefe2ff08 b5bd037832bcb8ed3086dfe26ce9090bea989af1 Johannes Gäßler johannesg@5d6.de 2025-09-25T11:12:27+02:00 GitHub noreply@github.com 2025-09-25T12:12:27+03:00 docs: fix typo [no ci] (#16244) b5bd037832bcb8ed3086dfe26ce9090bea989af1 dfcd53f7ecb9bb897a9d752d09c59d10be47237a Douglas Hanley thesecretaryofwar@gmail.com 2025-09-25T03:53:09-05:00 GitHub noreply@github.com 2025-09-25T11:53:09+03:00 llama : add support for qwen3 reranker (#15824) dfcd53f7ecb9bb897a9d752d09c59d10be47237a 4ea00794b8c995b6deaf4bac159c1778dc27419a Georgi Gerganov ggerganov@gmail.com 2025-09-25T11:30:16+03:00 GitHub noreply@github.com 2025-09-25T11:30:16+03:00 metal : fuse NORM + MUL + ADD, support non-multiples of 4 (#16220) 4ea00794b8c995b6deaf4bac159c1778dc27419a 02a6a82ae7c7ddd1819ae26b0cc36675879aecee Georgi Gerganov ggerganov@gmail.com 2025-09-25T11:29:42+03:00 GitHub noreply@github.com 2025-09-25T11:29:42+03:00 metal : relax reorder conditions (#16216) 02a6a82ae7c7ddd1819ae26b0cc36675879aecee c498fc82fe5b83fc8c6e1627286bdc1f93caddbf Georgi Gerganov ggerganov@gmail.com 2025-09-25T11:29:08+03:00 GitHub noreply@github.com 2025-09-25T11:29:08+03:00 metal : restore im2col perf (#16219) c498fc82fe5b83fc8c6e1627286bdc1f93caddbf e7a5130a20cf45a3358308356c249734ca982143 Radoslav Gerganov rgerganov@gmail.com 2025-09-25T10:20:02+03:00 GitHub noreply@github.com 2025-09-25T07:20:02Z rpc : use ggml logging facilities e7a5130a20cf45a3358308356c249734ca982143 bee378e0988b44bbe93b0768208080951b312363 Aaron Teo aaron.teo1@ibm.com 2025-09-25T13:06:30+08:00 GitHub noreply@github.com 2025-09-25T08:06:30+03:00 codeowners: add ownership of zdnn backend [no ci] (#16232) bee378e0988b44bbe93b0768208080951b312363 5fb557653b8756ac2b6c6a102b1e44abcadf552f Eve 139727413+netrunnereve@users.noreply.github.com 2025-09-25T05:06:06Z GitHub noreply@github.com 2025-09-25T08:06:06+03:00 ci: run the x64 and arm ci on the github machines instead (#16183) 5fb557653b8756ac2b6c6a102b1e44abcadf552f 4ae88d07d026e66b41e85afece74e88af54f4e66 Aaron Teo aaron.teo1@ibm.com 2025-09-25T11:36:30+08:00 GitHub noreply@github.com 2025-09-25T11:36:30+08:00 devops: fix s390x docker release failure (#16231) 4ae88d07d026e66b41e85afece74e88af54f4e66 e789095502b337690c7616db32d7c679a5bd2533 Aaron Teo aaron.teo1@ibm.com 2025-09-25T00:25:04+08:00 GitHub noreply@github.com 2025-09-25T00:25:04+08:00 codeowners: add ownership of zdnn backend [no ci] (#16229) e789095502b337690c7616db32d7c679a5bd2533 f2a789e33490deb483a2694b066b37e45524bb79 Johannes Gäßler johannesg@5d6.de 2025-09-24T16:53:48+02:00 GitHub noreply@github.com 2025-09-24T16:53:48+02:00 llama: print memory breakdown on exit (#15860) f2a789e33490deb483a2694b066b37e45524bb79 3a599719673c850647e3bb911ed6d91109bb91d2 Acly aclysia@gmail.com 2025-09-24T16:17:49+02:00 GitHub noreply@github.com 2025-09-24T16:17:49+02:00 ggml : split graph allocations according to backend max buffer size (#15815) 3a599719673c850647e3bb911ed6d91109bb91d2 63b54c81a620981be020184ab99e63a8e50e47cb Tarek Dakhran t.dakhran@gmail.com 2025-09-24T13:42:26+02:00 GitHub noreply@github.com 2025-09-24T13:42:26+02:00 model : add label for LiquidAI LFM2-2.6B model (#16204) 63b54c81a620981be020184ab99e63a8e50e47cb 152729f8848b692e1ae53c197ccca92ef10a523b Jie Fu (傅杰) jiefu@tencent.com 2025-09-24T16:25:26+08:00 GitHub noreply@github.com 2025-09-24T10:25:26+02:00 model-conversion : make causal-verify-logits fails with model names containing "." (#16215) 152729f8848b692e1ae53c197ccca92ef10a523b c0c59c1157b5aeded285a3b25d2463c866f583d3 Uilian Ries uilianries@gmail.com 2025-09-24T08:53:47+02:00 GitHub noreply@github.com 2025-09-24T09:53:47+03:00 common : add missing chrono header for common.cpp (#16211) c0c59c1157b5aeded285a3b25d2463c866f583d3 7735706b939847df2c6c00b44c36f95a20137c61 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-24T08:53:20+02:00 GitHub noreply@github.com 2025-09-24T08:53:20+02:00 codeowners : match all requirements files (#16214) 7735706b939847df2c6c00b44c36f95a20137c61 4d9ea03d170cf504a40bf3a85788af84cccaee80 Jie Fu (傅杰) jiefu@tencent.com 2025-09-24T14:46:52+08:00 GitHub noreply@github.com 2025-09-24T08:46:52+02:00 model-conversion : run-org-model.py fails to run on mac m1 (#16213) 4d9ea03d170cf504a40bf3a85788af84cccaee80 8ba548dae251f8f2e3834ed5226b76b6e4f4a485 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-24T08:10:09+02:00 GitHub noreply@github.com 2025-09-24T08:10:09+02:00 codeowners : use slash prefix for root files [no ci] (#16210) 8ba548dae251f8f2e3834ed5226b76b6e4f4a485 f505bd83ca7a43c4585ff3d59135e77eae9c793b Jie Fu (傅杰) jiefu@tencent.com 2025-09-24T12:19:23+08:00 GitHub noreply@github.com 2025-09-24T06:19:23+02:00 model-conversion : fix the make targets in the README.md (#16209) f505bd83ca7a43c4585ff3d59135e77eae9c793b 0889589dbe8c67dd518c58a57afbb02dde2dccbe Georgi Gerganov ggerganov@gmail.com 2025-09-23T20:41:40+03:00 GitHub noreply@github.com 2025-09-23T20:41:40+03:00 ci : disable AMD workflows + update NVIDIA workflows (#16200) 0889589dbe8c67dd518c58a57afbb02dde2dccbe 4e29084ba4104c4ea529fd3163bb6e76f64383df Georgi Gerganov ggerganov@gmail.com 2025-09-23T13:44:25+03:00 GitHub noreply@github.com 2025-09-23T13:44:25+03:00 ci : enable Vulkan workflow on Mac (#16194) 4e29084ba4104c4ea529fd3163bb6e76f64383df f6b4af3d04763b1e0130f5b5fce19c4bc6f83f1c Xiangyan Sun wishstudio@gmail.com 2025-09-23T01:58:12-07:00 GitHub noreply@github.com 2025-09-23T11:58:12+03:00 ggml-cpu: Respect cpumask settings (#16164) f6b4af3d04763b1e0130f5b5fce19c4bc6f83f1c 264f1b51872c125e23fa0ac1da5e2a1170de9a08 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-23T10:25:20+02:00 GitHub noreply@github.com 2025-09-23T10:25:20+02:00 ggml : fix uninitialized is_on_grid in quantize_row_iq3_xxs_impl (#15928) 264f1b51872c125e23fa0ac1da5e2a1170de9a08 0bc7cc715472fe28822e57f036f0746592ed2c04 Aaron Teo aaron.teo1@ibm.com 2025-09-23T14:53:05+08:00 GitHub noreply@github.com 2025-09-23T14:53:05+08:00 zdnn: refactor codebase + add docs (#16178) 0bc7cc715472fe28822e57f036f0746592ed2c04 4b9f4cb0f89a88de4bdf97727d0457b0c648804c Daniel Bevenius daniel.bevenius@gmail.com 2025-09-23T08:13:22+02:00 GitHub noreply@github.com 2025-09-23T09:13:22+03:00 codeowners : add @danbev to model-conversion example [no ci] (#16190) 4b9f4cb0f89a88de4bdf97727d0457b0c648804c 85e72271ba1ce78adf34fd8997803c991e617ca6 Aaron Teo aaron.teo1@ibm.com 2025-09-23T13:59:34+08:00 GitHub noreply@github.com 2025-09-23T13:59:34+08:00 devops: add s390x containers (#15915) 85e72271ba1ce78adf34fd8997803c991e617ca6 1d0125bcf1cbd7195ad0faf826a20bc7cec7d3f4 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-23T05:59:03+02:00 GitHub noreply@github.com 2025-09-23T05:59:03+02:00 ggml-cpu : fix typo in gemm comments [no ci] (#16189) 1d0125bcf1cbd7195ad0faf826a20bc7cec7d3f4 351f3da39c85f59d581fc184f09283da7f099a3b Gabe Goodhart ghart@us.ibm.com 2025-09-22T12:40:10-06:00 GitHub noreply@github.com 2025-09-22T20:40:10+02:00 feat: Add conversion support in GraniteHybrid for non-hybrid (all attn) (#16177) 351f3da39c85f59d581fc184f09283da7f099a3b 3ecb2f671a2f49d56357f99d135a94e841759178 Haiyue Wang haiyuewa@163.com 2025-09-23T01:57:46+08:00 GitHub noreply@github.com 2025-09-22T19:57:46+02:00 clang-tidy : disable warning about performance enum size (#16127) 3ecb2f671a2f49d56357f99d135a94e841759178 432cf4304c5379bfbd1f3feed65ec205955b2f4b Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-22T19:13:00+02:00 GitHub noreply@github.com 2025-09-22T19:13:00+02:00 ggml : implement set_rows with i32 index (#16159) 432cf4304c5379bfbd1f3feed65ec205955b2f4b 37a23c17bdc9c99c9c6ad41168e4ced3724b72cd Georgi Gerganov ggerganov@gmail.com 2025-09-22T18:20:21+03:00 GitHub noreply@github.com 2025-09-22T18:20:21+03:00 codeowners : update + cleanup (#16174) 37a23c17bdc9c99c9c6ad41168e4ced3724b72cd 138c87ce8bd558b2cc134ada7316a3dad8eb67ac Adrien Gallouët angt@huggingface.co 2025-09-22T14:13:51+02:00 GitHub noreply@github.com 2025-09-22T15:13:51+03:00 common : enable `--offline` mode without curl support (#16137) 138c87ce8bd558b2cc134ada7316a3dad8eb67ac c6db9a10278f40b4b8d3f6931728f2cce2356daa Quentin Bramas quentin.bramas@gmail.com 2025-09-22T10:53:13+02:00 GitHub noreply@github.com 2025-09-22T11:53:13+03:00 webui : fix handling incomplete chunks (#16107) c6db9a10278f40b4b8d3f6931728f2cce2356daa d05affbab7d1364fa7ac06c03cd33f03235ae840 GideonSerf gdserf.gs@gmail.com 2025-09-22T10:49:58+02:00 GitHub noreply@github.com 2025-09-22T11:49:58+03:00 embedding : fix typos in README (#16171) d05affbab7d1364fa7ac06c03cd33f03235ae840 4f324a556caa5be0be2261604ef4aab0faf36eb8 Haiyue Wang haiyuewa@163.com 2025-09-22T16:48:42+08:00 GitHub noreply@github.com 2025-09-22T11:48:42+03:00 common : remove unused local variables (#16140) 4f324a556caa5be0be2261604ef4aab0faf36eb8 a71ae3ba7a857394103f12e30b387c48242c84f2 Georgi Gerganov ggerganov@gmail.com 2025-09-22T11:12:37+03:00 GitHub noreply@github.com 2025-09-22T11:12:37+03:00 ggml : extend ggml_can_fuse to work with non-sequential nodes (#16123) a71ae3ba7a857394103f12e30b387c48242c84f2 05a2458121f9cdcc98ea6ffeeab6f23023136266 Georgi Gerganov ggerganov@gmail.com 2025-09-22T11:12:09+03:00 GitHub noreply@github.com 2025-09-22T11:12:09+03:00 ggml : add ggml_op_is_empty (#16122) 05a2458121f9cdcc98ea6ffeeab6f23023136266 96fdca043be8fa733bbd59868a75517f92633376 Xuan-Son Nguyen son@huggingface.co 2025-09-22T15:10:58+07:00 GitHub noreply@github.com 2025-09-22T11:10:58+03:00 codeowners : update ownership for @ngxson and @allozuar (#16128) 96fdca043be8fa733bbd59868a75517f92633376 b2d980fce063fa3591c59ad50b946c8da66c294f Shin-myoung-serp relent95@naver.com 2025-09-22T17:04:01+09:00 GitHub noreply@github.com 2025-09-22T10:04:01+02:00 Vulkan: add conv_transpose_2d operation (#16022) b2d980fce063fa3591c59ad50b946c8da66c294f 5c6106a696af2b00ce01d14ba525979d5a32b199 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-22T09:59:05+02:00 GitHub noreply@github.com 2025-09-22T10:59:05+03:00 codeowners : claim responsibility for ci, models, gguf-py and convert (#16124) 5c6106a696af2b00ce01d14ba525979d5a32b199 ec65fb52f0cc890e5085a6c5995ab08a156265fb Georgi Gerganov ggerganov@gmail.com 2025-09-22T10:58:02+03:00 GitHub noreply@github.com 2025-09-22T10:58:02+03:00 contrib : update roles (#16113) ec65fb52f0cc890e5085a6c5995ab08a156265fb 1d660d2fae42ea2e1d3569638e722bf7a37b6b19 Georgi Gerganov ggerganov@gmail.com 2025-09-22T10:16:05+03:00 GitHub noreply@github.com 2025-09-22T10:16:05+03:00 ci : remove vulkaninfo calls (#16169) 1d660d2fae42ea2e1d3569638e722bf7a37b6b19 a20d810d79adfbf82e0eb703af6f27a0e2d1a539 Georgi Gerganov ggerganov@gmail.com 2025-09-22T09:11:39+03:00 GitHub noreply@github.com 2025-09-22T09:11:39+03:00 ci : use smaller model (#16168) a20d810d79adfbf82e0eb703af6f27a0e2d1a539 4d0a7cbc617e384fc355077a304c883b5c7d4fb6 Jeff Bolz jbolz@nvidia.com 2025-09-22T00:37:17-05:00 GitHub noreply@github.com 2025-09-22T07:37:17+02:00 vulkan: add RTE variants of exp shader (#16165) 4d0a7cbc617e384fc355077a304c883b5c7d4fb6 9073a73d82a916cea0809de225ef5175c3a86e91 Georgi Gerganov ggerganov@gmail.com 2025-09-22T08:31:40+03:00 GitHub noreply@github.com 2025-09-22T08:31:40+03:00 ci : adjust params for less runtime (#16167) 9073a73d82a916cea0809de225ef5175c3a86e91 51f5a45fbe575dcd54bdd2a339ef8e8424d1c12a Ruben Ortlam picard12@live.de 2025-09-22T07:22:43+02:00 GitHub noreply@github.com 2025-09-22T07:22:43+02:00 vulkan: vec dot matrix multiplication fix (#16151) 51f5a45fbe575dcd54bdd2a339ef8e8424d1c12a c4510dc9374e17dcb8726902ab5216067a92b3d3 lhez lih@qti.qualcomm.com 2025-09-21T16:42:10-07:00 GitHub noreply@github.com 2025-09-21T16:42:10-07:00 opencl: fix concat crash on win arm64 with Adreno (#15944) c4510dc9374e17dcb8726902ab5216067a92b3d3 da30ab5f8696cabb2d4620cdc0aa41a298c54fd6 lhez lih@qti.qualcomm.com 2025-09-21T14:48:44-07:00 GitHub noreply@github.com 2025-09-21T14:48:44-07:00 opencl: initial `q8_0` mv support (#15732) da30ab5f8696cabb2d4620cdc0aa41a298c54fd6 28baac9c9f491c872e2c37762d3bd90446b005e9 Georgi Gerganov ggerganov@gmail.com 2025-09-21T19:00:27+03:00 GitHub noreply@github.com 2025-09-21T19:00:27+03:00 ci : add label for the RISC-V runner (#16150) 28baac9c9f491c872e2c37762d3bd90446b005e9 1eeb523c3e0c7ffbd59469f5463dcbdecba3535e Georgi Gerganov ggerganov@gmail.com 2025-09-21T16:50:45+03:00 GitHub noreply@github.com 2025-09-21T16:50:45+03:00 ci : migrate ggml ci to self-hosted runners (#16116) 1eeb523c3e0c7ffbd59469f5463dcbdecba3535e 5bb4a3edec297e74b0f7bd4ed5d0fdd12e28d858 Giuseppe Scrivano gscrivan@redhat.com 2025-09-21T08:31:55+02:00 GitHub noreply@github.com 2025-09-21T08:31:55+02:00 vulkan: optimize UMA buffer operations and fix driver hangs (#16059) 5bb4a3edec297e74b0f7bd4ed5d0fdd12e28d858 7f766929ca8e8e01dcceb1c526ee584f7e5e1408 Jeff Bolz jbolz@nvidia.com 2025-09-21T01:23:37-05:00 GitHub noreply@github.com 2025-09-21T08:23:37+02:00 vulkan: fix validation error about VK_PIPELINE_CREATE_CAPTURE_STATISTICS_BIT_KHR (#16086) 7f766929ca8e8e01dcceb1c526ee584f7e5e1408 405921dcefdd4e90ed948a4bf179007c2fa92b2d Georgi Gerganov ggerganov@gmail.com 2025-09-20T12:55:47+03:00 Georgi Gerganov ggerganov@gmail.com 2025-09-20T13:02:14+03:00 sync : ggml 405921dcefdd4e90ed948a4bf179007c2fa92b2d fa6383ca7e7ccb8ca3bdfeb37e348ddc4113aa26 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-16T06:16:52+02:00 Georgi Gerganov ggerganov@gmail.com 2025-09-20T13:02:14+03:00 ggml : introduce semantic versioning (ggml/1336) fa6383ca7e7ccb8ca3bdfeb37e348ddc4113aa26 803dac2e48ef3ba26a504eb27c4e77ec2d21f7d0 Gregor Jasny gjasny@googlemail.com 2025-09-10T17:21:11+02:00 Georgi Gerganov ggerganov@gmail.com 2025-09-20T13:02:14+03:00 CUDA : conditionally add cuda architectures (ggml/1341) 803dac2e48ef3ba26a504eb27c4e77ec2d21f7d0 459c0c2c1a400f960d7b8e8d94d31a8426f80986 Ruben Ortlam picard12@live.de 2025-09-20T10:42:56+02:00 GitHub noreply@github.com 2025-09-20T10:42:56+02:00 vulkan: use vec dot for matrix matrix multiplications (#16056) 459c0c2c1a400f960d7b8e8d94d31a8426f80986 be79d9fdd95ab8955527c4aaa67b90e8b9516718 Benni 73313922+BenjaminBruenau@users.noreply.github.com 2025-09-20T07:56:30+02:00 GitHub noreply@github.com 2025-09-20T07:56:30+02:00 server: fix SSE and OpenAI compatibility for error messages when streaming (#16109) be79d9fdd95ab8955527c4aaa67b90e8b9516718 f432d8d83e7407073634c5e4fd81a3d23a10827f ssweens 1149151+ssweens@users.noreply.github.com 2025-09-19T15:15:21-07:00 GitHub noreply@github.com 2025-09-20T00:15:21+02:00 llama-bench: add --devices and --list-devices support (#16039) f432d8d83e7407073634c5e4fd81a3d23a10827f 4067f07fc5aa5722b87db303b561597004696f6c shun095 8069181+shun095@users.noreply.github.com 2025-09-20T00:57:30+09:00 GitHub noreply@github.com 2025-09-19T09:57:30-06:00 chat: Fix streaming parser for granite models (#15682) 4067f07fc5aa5722b87db303b561597004696f6c 4b8560ab56fdd9819358b47c338bbc8ec357c57e Aleksander Grygier aleksander.grygier@gmail.com 2025-09-19T09:52:27+02:00 GitHub noreply@github.com 2025-09-19T09:52:27+02:00 feat: Improve mobile UI for Settings Dialog (#16084) 4b8560ab56fdd9819358b47c338bbc8ec357c57e 0dd58b6877f3dc106593d6bc68d98305f553c566 Xuan-Son Nguyen son@huggingface.co 2025-09-19T13:02:51+07:00 GitHub noreply@github.com 2025-09-19T13:02:51+07:00 chat : fix build on arm64 (#16101) 0dd58b6877f3dc106593d6bc68d98305f553c566 69ffd891631befa9e6b485fd646a16dab4f2c007 Xuan-Son Nguyen son@huggingface.co 2025-09-19T11:31:56+07:00 GitHub noreply@github.com 2025-09-19T06:31:56+02:00 ggml : refactor forward_dup for cpu backend (#16062) 69ffd891631befa9e6b485fd646a16dab4f2c007 246c0d9c795fb25da8268625d597580155a3672f Adrien Gallouët angt@huggingface.co 2025-09-18T23:07:26+02:00 GitHub noreply@github.com 2025-09-18T23:07:26+02:00 ggml-amx : fix ggml_amx_init() on generic Linux (#16049) 246c0d9c795fb25da8268625d597580155a3672f 3edd87cd055a45d885fa914d879d36d33ecfc3e1 Adrien Gallouët angt@huggingface.co 2025-09-18T23:07:18+02:00 GitHub noreply@github.com 2025-09-18T23:07:18+02:00 cmake : fix static linking for OpenMP on Unix-like systems (#16031) 3edd87cd055a45d885fa914d879d36d33ecfc3e1 c0b45097c33e2667a94444f08cc9e36bec0a5e2e Shawn Gu shawngu@qti.qualcomm.com 2025-09-18T12:03:34-07:00 GitHub noreply@github.com 2025-09-18T12:03:34-07:00 opencl: optimize mxfp4 kernels (#16037) c0b45097c33e2667a94444f08cc9e36bec0a5e2e 38dbdf4c057515ccea9bec0ca2518f86d5e4d28e Jeff Bolz jbolz@nvidia.com 2025-09-18T13:46:17-05:00 GitHub noreply@github.com 2025-09-18T13:46:17-05:00 rename optimize_graph to graph_optimize (#16082) 38dbdf4c057515ccea9bec0ca2518f86d5e4d28e 368560a1e3b9a3bc83af741b0b2bc9e46fb420d2 Bowen Han fancycode@gmail.com 2025-09-18T11:26:03-07:00 GitHub noreply@github.com 2025-09-18T20:26:03+02:00 CUDA: Optimize PAD_REFLECT_1D (#15957) 368560a1e3b9a3bc83af741b0b2bc9e46fb420d2 4ca088b036313c2d8e682f4cfeb7c29edd85d0b9 Johannes Gäßler johannesg@5d6.de 2025-09-18T19:28:32+02:00 GitHub noreply@github.com 2025-09-18T19:28:32+02:00 CUDA: fix compilation on CC 6.0 (#16091) 4ca088b036313c2d8e682f4cfeb7c29edd85d0b9 703f9e32c4eb3166f8d63007c26e31a1466c21af Eric Curtin eric.curtin@docker.com 2025-09-18T16:22:50+01:00 GitHub noreply@github.com 2025-09-18T16:22:50+01:00 Add resumable downloads for llama-server model loading (#15963) 703f9e32c4eb3166f8d63007c26e31a1466c21af ad6bd9083bc346afd4ecd2fbb83e0306c0141c99 Georgi Gerganov ggerganov@gmail.com 2025-09-18T16:28:41+03:00 GitHub noreply@github.com 2025-09-18T16:28:41+03:00 metal : use function constants for mul_mv_ext kernels (#16074) ad6bd9083bc346afd4ecd2fbb83e0306c0141c99 2b6b55a59f590a1e1eb2dcd09d5b8b6feb9cc748 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-18T13:28:22+02:00 GitHub noreply@github.com 2025-09-18T13:28:22+02:00 cuda : add missing F32<->I32 entries in ggml_cuda_cpy_fn (#16060) 2b6b55a59f590a1e1eb2dcd09d5b8b6feb9cc748 e58174cecbc45bf79bf653cd2c984395940c6ef4 Radoslav Gerganov rgerganov@gmail.com 2025-09-18T13:36:57+03:00 GitHub noreply@github.com 2025-09-18T10:36:57Z server : include usage statistics only when user request them (#16052) e58174cecbc45bf79bf653cd2c984395940c6ef4 b213fce89bee8cb56b587b91e15a4278f8ed0180 Georgi Gerganov ggerganov@gmail.com 2025-09-18T12:47:56+03:00 GitHub noreply@github.com 2025-09-18T12:47:56+03:00 llama : bump max seq limit from 64 to 256 (#15916) b213fce89bee8cb56b587b91e15a4278f8ed0180 e00f3fd8fff2cf5a8c8c9f475034bd089c8bcce4 Georgi Gerganov ggerganov@gmail.com 2025-09-18T12:33:45+03:00 GitHub noreply@github.com 2025-09-18T12:33:45+03:00 metal : improve F32, F16 and BF16 mat-vec multiplication (#16057) e00f3fd8fff2cf5a8c8c9f475034bd089c8bcce4 f2f28380ea68939c0481df3e4216b698824a59df Jhen-Jie Hong iainst0409@gmail.com 2025-09-18T15:06:48+08:00 GitHub noreply@github.com 2025-09-18T10:06:48+03:00 metal : avoid call free for non-owned buffer (#16067) f2f28380ea68939c0481df3e4216b698824a59df 62c3b645c56ad5288aa401901f043b050edac5a2 Georgi Gerganov ggerganov@gmail.com 2025-09-18T10:03:24+03:00 GitHub noreply@github.com 2025-09-18T10:03:24+03:00 metal : handle nil cv during pipeline creation (#16065) 62c3b645c56ad5288aa401901f043b050edac5a2 d304f459d8619399eb232b1e3689379306f439d3 Chenguang Li 757486878@qq.com 2025-09-18T09:26:33+08:00 GitHub noreply@github.com 2025-09-18T09:26:33+08:00 CANN: Remove print (#16044) d304f459d8619399eb232b1e3689379306f439d3 0320ac5264279d74f8ee91bafa6c90e9ab9bbb91 Reese Levine reeselevine1@gmail.com 2025-09-17T13:09:40-07:00 GitHub noreply@github.com 2025-09-17T13:09:40-07:00 GGML WebGPU: Support for ADD, MUL, RMS_NORM, GET_ROWS operators (#16018) 0320ac5264279d74f8ee91bafa6c90e9ab9bbb91 a7a98e0fffed794396b3fbad4dcdbbc184963645 Georgi Gerganov ggerganov@gmail.com 2025-09-17T20:38:12+03:00 GitHub noreply@github.com 2025-09-17T20:38:12+03:00 metal : refactor + optimize v2 (#15995) a7a98e0fffed794396b3fbad4dcdbbc184963645 8f8f2274ee3601fecf6e2d57b52f701c81bede21 Aleksander Grygier aleksander.grygier@gmail.com 2025-09-17T19:29:13+02:00 GitHub noreply@github.com 2025-09-17T19:29:13+02:00 SvelteKit-based WebUI (#14839) 8f8f2274ee3601fecf6e2d57b52f701c81bede21 c959b676be29e93f8dbc3bd6056ceba812a9eb72 Xuan-Son Nguyen son@huggingface.co 2025-09-18T00:18:21+07:00 GitHub noreply@github.com 2025-09-17T19:18:21+02:00 convert : add Llama4ForCausalLM (#16042) c959b676be29e93f8dbc3bd6056ceba812a9eb72 cd08fc3ecc0264b4414b68af3874a6c689ed60c1 Johannes Gäßler johannesg@5d6.de 2025-09-17T15:32:42+02:00 GitHub noreply@github.com 2025-09-17T15:32:42+02:00 CUDA: fix FA occupancy, optimize tile kernel (#15982) cd08fc3ecc0264b4414b68af3874a6c689ed60c1 cb5bb6cc05119c24e7711ca2956cd0e6d409d396 David Ribeiro Alves davidralves@gmail.com 2025-09-17T01:08:02-07:00 GitHub noreply@github.com 2025-09-17T11:08:02+03:00 common : Fix corrupted memory error on json grammar initialization (#16038) cb5bb6cc05119c24e7711ca2956cd0e6d409d396 a91d035b901e8a9edf810f63d130ee49adf27be2 Eve 139727413+netrunnereve@users.noreply.github.com 2025-09-17T07:35:37Z GitHub noreply@github.com 2025-09-17T09:35:37+02:00 vulkan: automatically remove unsupported devices (#15976) a91d035b901e8a9edf810f63d130ee49adf27be2 745cbcf2fe1eb88f8db615ac622f0b944d924ad6 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-17T09:34:09+02:00 GitHub noreply@github.com 2025-09-17T09:34:09+02:00 ci : revert back to macos-13 for macOS-latest-cmake-x64 (#16040) 745cbcf2fe1eb88f8db615ac622f0b944d924ad6 1cbd80f8cf80a817715b1ccc5680fe2a3c5172c8 Jie Fu (傅杰) jiefu@tencent.com 2025-09-17T15:30:55+08:00 GitHub noreply@github.com 2025-09-17T09:30:55+02:00 llama-quant : fix the verification of attention layers for encoder-decoder models (#16023) 1cbd80f8cf80a817715b1ccc5680fe2a3c5172c8 85286f354813056f6c835046c0acfa3bf6ba9432 Jie Fu (傅杰) jiefu@tencent.com 2025-09-17T15:29:00+08:00 GitHub noreply@github.com 2025-09-17T10:29:00+03:00 examples : support encoder-decoder models in the simple example (#16002) 85286f354813056f6c835046c0acfa3bf6ba9432 d5fabe3682de515fd09d6c981f7a0d1b75614455 Shane A shanea@allenai.org 2025-09-17T00:01:58-07:00 GitHub noreply@github.com 2025-09-17T09:01:58+02:00 model : add OLMo3 support (#16015) d5fabe3682de515fd09d6c981f7a0d1b75614455 8ff206097c2bf3ca1c7aa95f9d6db779fc7bdd68 Chenguang Li 757486878@qq.com 2025-09-17T14:33:08+08:00 GitHub noreply@github.com 2025-09-17T14:33:08+08:00 CANN: Optimize ggml_cann_set_device (#15935) 8ff206097c2bf3ca1c7aa95f9d6db779fc7bdd68 77475530b8bbea3bf578632507e1284cdfe2c8c0 jacekpoplawski 67507230+jacekpoplawski@users.noreply.github.com 2025-09-16T16:17:08+02:00 GitHub noreply@github.com 2025-09-16T16:17:08+02:00 llama-bench: add --n-cpu-moe support (#15952) 77475530b8bbea3bf578632507e1284cdfe2c8c0 3913f8730ec6d6245480affc30ae3049107956f4 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-16T15:27:52+02:00 GitHub noreply@github.com 2025-09-16T15:27:52+02:00 ci : use macos-latest for arm64 webgpu build (#16029) 3913f8730ec6d6245480affc30ae3049107956f4 76888d202ed2b835ae19ea9f9db6baf39e419297 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-16T15:25:57+02:00 GitHub noreply@github.com 2025-09-16T15:25:57+02:00 ggml : fix padding in timestep embedding kernels (#15932) 76888d202ed2b835ae19ea9f9db6baf39e419297 f1fbffb5c0b34b2a68febb7da3fd0f8333f1ed4c Daniel Bevenius daniel.bevenius@gmail.com 2025-09-16T13:41:38+02:00 GitHub noreply@github.com 2025-09-16T13:41:38+02:00 ci : upload xcframework artifact from ios-xcode-build job (#16010) f1fbffb5c0b34b2a68febb7da3fd0f8333f1ed4c 51abc96bdc52ba8cd6ad78dcf12ed9a041d7b442 Bowen Han fancycode@gmail.com 2025-09-15T23:59:19-07:00 GitHub noreply@github.com 2025-09-16T08:59:19+02:00 fix: apply clang-format to CUDA macros (#16017) 51abc96bdc52ba8cd6ad78dcf12ed9a041d7b442 07808ebb07e2b1aa19032705e332679ddf967614 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-16T05:57:16+02:00 GitHub noreply@github.com 2025-09-16T05:57:16+02:00 ci : update macos-latest* jobs to use macos-latest (#15938) 07808ebb07e2b1aa19032705e332679ddf967614 6d758839ff741d4966ca92b7f801b7a8b5b96364 Yuri Khrustalev ykhrustalev@users.noreply.github.com 2025-09-15T22:54:44-04:00 GitHub noreply@github.com 2025-09-16T09:54:44+07:00 cmake : Do not install tools on iOS targets (#15903) 6d758839ff741d4966ca92b7f801b7a8b5b96364 3d4053f77f0f78ee2b791088c02af653ebee42dd Aman Gupta amangupta052@gmail.com 2025-09-16T10:38:28+08:00 GitHub noreply@github.com 2025-09-16T10:38:28+08:00 Add LLaDA-7b-MoE diffusion model (#16003) 3d4053f77f0f78ee2b791088c02af653ebee42dd dc381aa9a6dc45f00673471d34b8bddd30e77570 Jake Karnes jake.karnes@gmail.com 2025-09-15T16:28:31-06:00 GitHub noreply@github.com 2025-09-16T00:28:31+02:00 CUDA: fix im2col_3d to respect non-contiguous inputs (views) (#15956) dc381aa9a6dc45f00673471d34b8bddd30e77570 10d197409bd9537ff302ad09966fe406882fef9d Diego Devesa slarengh@gmail.com 2025-09-15T14:38:52-07:00 GitHub noreply@github.com 2025-09-15T23:38:52+02:00 docker : enable rocWMMA in ROCm images, add gfx1151 (#15997) 10d197409bd9537ff302ad09966fe406882fef9d b907255f4bd169b0dc7dca9553b4c54af5170865 Diego Devesa slarengh@gmail.com 2025-09-15T14:38:42-07:00 GitHub noreply@github.com 2025-09-15T23:38:42+02:00 releases : switch to rocWMMA develop branch, add gfx1151 (#15992) b907255f4bd169b0dc7dca9553b4c54af5170865 28c39da7c645185ade5436767929d7ec33006033 yael-works 106673277+yael-works@users.noreply.github.com 2025-09-15T19:51:35+03:00 GitHub noreply@github.com 2025-09-15T18:51:35+02:00 SYCL: Add COUNT_EQUAL operator support (#15991) 28c39da7c645185ade5436767929d7ec33006033 106220562aca42b6738b8f51acfce0db1b8a2fb6 Nikolay Popov 131475237+npopov-vst@users.noreply.github.com 2025-09-15T13:08:30+03:00 GitHub noreply@github.com 2025-09-15T11:08:30+01:00 llama-run: Fix model download on Windows (#15988) 106220562aca42b6738b8f51acfce0db1b8a2fb6 a68f31edd71cc39141113f05f7133a3e9ece8c61 Aman Gupta amangupta052@gmail.com 2025-09-15T17:35:11+08:00 GitHub noreply@github.com 2025-09-15T17:35:11+08:00 CUDA: some micro-optimizations in mmf.cuh for mul_mat_id (#15926) a68f31edd71cc39141113f05f7133a3e9ece8c61 b8e09f08b9a91c0401bc67d17a17c90756420346 ddh0 chemist-mulches-39@icloud.com 2025-09-15T02:54:57-05:00 GitHub noreply@github.com 2025-09-15T09:54:57+02:00 fix KLD percentile output (#15999) b8e09f08b9a91c0401bc67d17a17c90756420346 6c019cb04e86e2dacfe62ce7666c64e9717dde1f Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-14T23:00:59+02:00 GitHub noreply@github.com 2025-09-14T23:00:59+02:00 model : add grok-2 support (#15539) 6c019cb04e86e2dacfe62ce7666c64e9717dde1f 9dcd200d57bc6f05a59a6a8df361d5d183af4124 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-14T21:17:04+02:00 GitHub noreply@github.com 2025-09-14T21:17:04+02:00 server : only attempt to enable thinking if using jinja (#15967) 9dcd200d57bc6f05a59a6a8df361d5d183af4124 0fa154e3502e940df914f03b41475a2b80b985b0 Georgi Gerganov ggerganov@gmail.com 2025-09-14T22:02:32+03:00 GitHub noreply@github.com 2025-09-14T22:02:32+03:00 metal : remove memory pools (#15966) 0fa154e3502e940df914f03b41475a2b80b985b0 261e6a20ffdb79c4875e674b4f6b514bc73cff8f Adam channeladam@users.noreply.github.com 2025-09-15T04:43:54+10:00 GitHub noreply@github.com 2025-09-14T20:43:54+02:00 rocm.Dockerfile: added gfx1200,gfx1201 architectures to support AMD Radeon RX 9000 series (#15994) 261e6a20ffdb79c4875e674b4f6b514bc73cff8f a0e13dcbe5bae7025660349ef3e4ead060e507f2 Ruben Ortlam picard12@live.de 2025-09-14T16:56:28+02:00 GitHub noreply@github.com 2025-09-14T16:56:28+02:00 Vulkan: Clean up mul_mm shader (#15987) a0e13dcbe5bae7025660349ef3e4ead060e507f2 a14bd350141fb42b8bf2dd2342cebc27bfdce399 lcy lcy0321@users.noreply.github.com 2025-09-14T22:20:35+08:00 GitHub noreply@github.com 2025-09-14T07:20:35-07:00 build: fix the build failures of Windows HIP release job (#15984) a14bd350141fb42b8bf2dd2342cebc27bfdce399 918b26f197f55d5d562446dfc876d0e637929d07 Georgi Gerganov ggerganov@gmail.com 2025-09-14T15:33:22+03:00 GitHub noreply@github.com 2025-09-14T15:33:22+03:00 metal : fix kernel requirements (#15983) 918b26f197f55d5d562446dfc876d0e637929d07 9ecb88434644c865232bb665d2f6f05049fc6456 Radoslav Gerganov rgerganov@gmail.com 2025-09-14T12:28:18+03:00 GitHub noreply@github.com 2025-09-14T12:28:18+03:00 rpc : fix regression when --device is used (#15981) 9ecb88434644c865232bb665d2f6f05049fc6456 d1c6f11f47bae063a68a6be8e4830c060a11b7bd Diego Devesa slarengh@gmail.com 2025-09-14T02:21:59-07:00 GitHub noreply@github.com 2025-09-14T02:21:59-07:00 releases : update ROCM, add gfx1200, gfx1201, gfx1151 (#15972) d1c6f11f47bae063a68a6be8e4830c060a11b7bd 6380d6a3e709cb02a8695afdd96b40e674477332 Radoslav Gerganov rgerganov@gmail.com 2025-09-14T12:10:07+03:00 GitHub noreply@github.com 2025-09-14T12:10:07+03:00 doc : update documentation for --tensor-split (#15980) 6380d6a3e709cb02a8695afdd96b40e674477332 aa0c461efe3603639af1a1defed2438d9c16ca0f Aaron Teo aaron.teo1@ibm.com 2025-09-14T13:37:03+08:00 GitHub noreply@github.com 2025-09-14T13:37:03+08:00 ggml-zdnn: rm user mapped buffers (#15965) aa0c461efe3603639af1a1defed2438d9c16ca0f b9c9c9f789cd57fb0b28b25223305613cd90fa10 Jeff Bolz jbolz@nvidia.com 2025-09-13T16:29:43+01:00 GitHub noreply@github.com 2025-09-13T17:29:43+02:00 vulkan: fix failing dequant shaders (#15862) b9c9c9f789cd57fb0b28b25223305613cd90fa10 50f4281a6f5c3a5d68bdeb12f904fa01e0e2ba91 Jeff Bolz jbolz@nvidia.com 2025-09-13T16:23:30+01:00 GitHub noreply@github.com 2025-09-13T17:23:30+02:00 vulkan: initialize vulkan-hpp to allow using extension function pointers (#15705) 50f4281a6f5c3a5d68bdeb12f904fa01e0e2ba91 55758b00cae1451a0d789c50a5eb8c9bee42b9a0 Diego Devesa slarengh@gmail.com 2025-09-13T07:49:49-07:00 GitHub noreply@github.com 2025-09-13T16:49:49+02:00 llama : allow using iGPUs with --device (#15951) 55758b00cae1451a0d789c50a5eb8c9bee42b9a0 f161463a54d9f93d41246286aa4a9569a91d804d Georgi Gerganov ggerganov@gmail.com 2025-09-13T16:24:22+03:00 GitHub noreply@github.com 2025-09-13T16:24:22+03:00 metal : refactor kernel loading (#15964) f161463a54d9f93d41246286aa4a9569a91d804d 84d7b2fca11d1be118ce776f6d72a486c4883b74 Georgi Gerganov ggerganov@gmail.com 2025-09-13T13:54:28+03:00 GitHub noreply@github.com 2025-09-13T13:54:28+03:00 metal : allow ops to run concurrently (#15929) 84d7b2fca11d1be118ce776f6d72a486c4883b74 40be51152d4dc2d47444a4ed378285139859895b Georgi Gerganov ggerganov@gmail.com 2025-09-13T12:45:04+03:00 GitHub noreply@github.com 2025-09-13T12:45:04+03:00 metal : fix memory leaks (#15962) 40be51152d4dc2d47444a4ed378285139859895b 4bf5549269d99bac936a65892da2312cc71b2421 Aaron Teo aaron.teo1@ibm.com 2025-09-13T02:39:52+08:00 GitHub noreply@github.com 2025-09-13T02:39:52+08:00 ggml-zdnn: fix #15414, activate FP16 and BF16 acceleration and incorrect zTensor free (#15839) 4bf5549269d99bac936a65892da2312cc71b2421 f4e664f838cc65d89fa845c48c372e43852112e4 Eric Curtin eric.curtin@docker.com 2025-09-12T16:31:50+01:00 GitHub noreply@github.com 2025-09-12T16:31:50+01:00 Add docker protocol support for llama-server model loading (#15790) f4e664f838cc65d89fa845c48c372e43852112e4 f088b6a84f6aed0abb619dbb9d375e726fc888a6 Haiyue Wang haiyuewa@163.com 2025-09-12T23:16:32+08:00 GitHub noreply@github.com 2025-09-12T18:16:32+03:00 context : remove redundant explicit casting to the same type (#15948) f088b6a84f6aed0abb619dbb9d375e726fc888a6 304ac5693d1e2124f83e0584bc5eea6311d3d3b4 Georgi Gerganov ggerganov@gmail.com 2025-09-12T17:02:55+03:00 GitHub noreply@github.com 2025-09-12T17:02:55+03:00 server : adjust prompt similarity thold + add logs (#15913) 304ac5693d1e2124f83e0584bc5eea6311d3d3b4 6c88ad8fa741a2182a2dcf8c5353ae4b5c66e656 Ruben Ortlam picard12@live.de 2025-09-12T13:24:21+02:00 GitHub noreply@github.com 2025-09-12T13:24:21+02:00 Vulkan iGPU device selection overhaul and PCI ID API support (#15947) 6c88ad8fa741a2182a2dcf8c5353ae4b5c66e656 704d90c987cdf00751567b2088c4e54742aa2d3f Mathieu Baudier mbaudier@argeo.org 2025-09-12T09:06:20+02:00 GitHub noreply@github.com 2025-09-12T09:06:20+02:00 vulkan: Make device memory check more portable (#15939) 704d90c987cdf00751567b2088c4e54742aa2d3f 360d6533db39e11577afe9b0aece20c6b5ddaf1f Neo Zhang Jianyu jianyu.zhang@intel.com 2025-09-12T09:15:12+08:00 GitHub noreply@github.com 2025-09-12T09:15:12+08:00 Revert "sycl: add usage of enqueue_functions extension (#14244)" (#15910) 360d6533db39e11577afe9b0aece20c6b5ddaf1f 0e6ff0046f4a2983b2c77950aa75960fe4b4f0e2 Diego Devesa slarengh@gmail.com 2025-09-11T13:47:38-07:00 GitHub noreply@github.com 2025-09-11T22:47:38+02:00 ggml-backend : add GGML_BACKEND_DEVICE_TYPE_IGPU device type (#15797) 0e6ff0046f4a2983b2c77950aa75960fe4b4f0e2 df082f56309073ecf885eceaa21b86e8a487e61b Johannes Gäßler johannesg@5d6.de 2025-09-11T21:19:58+02:00 GitHub noreply@github.com 2025-09-11T21:19:58+02:00 CUDA: larger SRAM reads for tile FA, AMD FP16 dot (#15927) df082f56309073ecf885eceaa21b86e8a487e61b 24a6734daf6932ff29ba8c1ff0245c51d76f783e ddh0 chemist-mulches-39@icloud.com 2025-09-11T12:12:34-05:00 GitHub noreply@github.com 2025-09-11T19:12:34+02:00 nitpick : correct MB to MiB (#15934) 24a6734daf6932ff29ba8c1ff0245c51d76f783e 2b3efea9a4d91216850856fbb77075db26f6a6eb Daniel Bevenius daniel.bevenius@gmail.com 2025-09-11T15:39:12+02:00 GitHub noreply@github.com 2025-09-11T14:39:12+01:00 ggml-cpu : add check for ARM MATMUL_INT8/i8mm support (#15922) 2b3efea9a4d91216850856fbb77075db26f6a6eb c0389dba43d50695f9d3f57dd1f1a14cbefc100c Charles Xu charles.xu@arm.com 2025-09-11T12:45:40+02:00 GitHub noreply@github.com 2025-09-11T12:45:40+02:00 kleidiai: fix GGML_ASSERT(*cur_backend_id != -1) failed (#15614) c0389dba43d50695f9d3f57dd1f1a14cbefc100c 00681dfc16ba4cebb9c7fbd2cf2656e06a0692a4 hipudding huafengchun@gmail.com 2025-09-11T15:59:37+08:00 GitHub noreply@github.com 2025-09-11T15:59:37+08:00 CANN: Disable acl_graph for prefill stage (#15933) 00681dfc16ba4cebb9c7fbd2cf2656e06a0692a4 4f658855fa8f2e42b7ed9a5b298fa39a2e39b096 Oliver Simons osimons@nvidia.com 2025-09-10T22:04:03+02:00 GitHub noreply@github.com 2025-09-10T22:04:03+02:00 CUDA: Add `fastdiv` to `k_bin_bcast*`, giving 1-3% E2E performance (#15872) 4f658855fa8f2e42b7ed9a5b298fa39a2e39b096 6ab397e12ba8e9f776341cdae68f7ffb2f8d2cde Jie Fu (傅杰) jiefu@tencent.com 2025-09-11T02:51:51+08:00 GitHub noreply@github.com 2025-09-10T20:51:51+02:00 llama : support T5 models with unequal number of encoder-decoder layers (#15909) 6ab397e12ba8e9f776341cdae68f7ffb2f8d2cde 9de447d94e1ae9d1a36e5a2e5bf47483352c0d9c Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-10T19:08:59+02:00 GitHub noreply@github.com 2025-09-10T19:08:59+02:00 graph : support non-contiguous Q in build_attn_mha (#15908) 9de447d94e1ae9d1a36e5a2e5bf47483352c0d9c 0f0a3c2851134d49955f3c85afbb0b1bb47c3e07 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-10T17:31:40+02:00 GitHub noreply@github.com 2025-09-10T17:31:40+02:00 ggml-cpu : fix padding in ggml_timestep_embedding (#15917) 0f0a3c2851134d49955f3c85afbb0b1bb47c3e07 33daece86b65607451d0d4378d2d04ba6a20ad55 Georgi Gerganov ggerganov@gmail.com 2025-09-10T17:52:35+03:00 GitHub noreply@github.com 2025-09-10T17:52:35+03:00 metal : make the backend async (#15906) 33daece86b65607451d0d4378d2d04ba6a20ad55 e7b6d83b524bbc24a1343d862de6dba8e8eddbd6 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-10T15:39:57+02:00 GitHub noreply@github.com 2025-09-10T15:39:57+02:00 ci : add caching for ROCm installation in release workflow (#15924) e7b6d83b524bbc24a1343d862de6dba8e8eddbd6 2cfef4d117d67ab1dec002915b48a15d11ee1973 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-10T14:17:09+02:00 GitHub noreply@github.com 2025-09-10T14:17:09+02:00 tests : filter out no-ops from coverage report (#15900) 2cfef4d117d67ab1dec002915b48a15d11ee1973 09e72a037c77b6823fde9e1e4cf28e815b9c41ab j-k dev@j-k.io 2025-09-10T12:51:28+01:00 GitHub noreply@github.com 2025-09-10T14:51:28+03:00 media : add transparent icon svg and png [no ci] (#15891) 09e72a037c77b6823fde9e1e4cf28e815b9c41ab 10d8b2b6b0ac2cae252a80b4daea5da55ab63c2f Jesse jesse@createthis.com 2025-09-10T07:28:47-04:00 GitHub noreply@github.com 2025-09-10T14:28:47+03:00 gitignore : Ignore vim swap files in tests (#15901) 10d8b2b6b0ac2cae252a80b4daea5da55ab63c2f 28b5f190ef1dbea5edf82dbc8b4407b721fadd13 Chenguang Li 757486878@qq.com 2025-09-10T18:42:00+08:00 GitHub noreply@github.com 2025-09-10T18:42:00+08:00 CANN: Add ROPE sin/cos cache for reuse (#15912) 28b5f190ef1dbea5edf82dbc8b4407b721fadd13 86587da03bd78df8f4e7d8b111a0c1d2494d6ed0 Chenguang Li 757486878@qq.com 2025-09-10T15:29:12+08:00 GitHub noreply@github.com 2025-09-10T15:29:12+08:00 CANN: implement LRU cache for ACL graphs (#15814) 86587da03bd78df8f4e7d8b111a0c1d2494d6ed0 ff02caf9eed261423289d1531a56536fbf57bfc2 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-10T05:33:58+02:00 GitHub noreply@github.com 2025-09-10T05:33:58+02:00 llama : check returned fn ptrs from ggml_backend_reg_get_proc_address (#15893) ff02caf9eed261423289d1531a56536fbf57bfc2 ae355f6f7108540297d8b7f7ae71d20fe610a0b7 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-10T05:23:19+02:00 GitHub noreply@github.com 2025-09-10T05:23:19+02:00 ci : cache ROCm installation in windows-latest-cmake-hip (#15887) ae355f6f7108540297d8b7f7ae71d20fe610a0b7 4f63cd705c7b6f457f36c63fdc053e07f6f3cc6b Ruben Ortlam picard12@live.de 2025-09-09T22:26:03+02:00 GitHub noreply@github.com 2025-09-09T22:26:03+02:00 vulkan: throw the oom error instead of no memory type found (#15905) 4f63cd705c7b6f457f36c63fdc053e07f6f3cc6b 17bc5a815f0bebc1844c99796760cd7df5e9b8d9 Jeff Bolz jbolz@nvidia.com 2025-09-09T07:41:15-05:00 GitHub noreply@github.com 2025-09-09T14:41:15+02:00 vulkan: Fix OOB accesses in soft_max_back (#15861) 17bc5a815f0bebc1844c99796760cd7df5e9b8d9 ed54e32558ffe0f2c3b1d31f3227211fae05bd49 Johannes Gäßler johannesg@5d6.de 2025-09-09T14:04:43+02:00 GitHub noreply@github.com 2025-09-09T14:04:43+02:00 HIP: use v_dot2_f32_f16 instruction for FA (#15884) ed54e32558ffe0f2c3b1d31f3227211fae05bd49 a972faebed5fdc4a3d2a844d92d476058c02e02d lksj92hs 134250687+lksj92hs@users.noreply.github.com 2025-09-09T15:01:15+03:00 GitHub noreply@github.com 2025-09-09T14:01:15+02:00 Workaround for subgroup arithmetic failing on MoltenVK with AMD GPUs (issue 15846) (#15886) a972faebed5fdc4a3d2a844d92d476058c02e02d 550cf726e133fd0a069d991287fd3a2a3e3e1cbd Aman Gupta amangupta052@gmail.com 2025-09-09T14:38:02+08:00 GitHub noreply@github.com 2025-09-09T14:38:02+08:00 CUDA: Add mul_mat_id support for the mmf kernel (#15767) 550cf726e133fd0a069d991287fd3a2a3e3e1cbd c252ce67c4b99f056eeb18d42a289e28fee03475 Johannes Gäßler johannesg@5d6.de 2025-09-09T08:11:01+02:00 GitHub noreply@github.com 2025-09-09T08:11:01+02:00 CUDA: fix GET_ROWS for large tensors (#15882) c252ce67c4b99f056eeb18d42a289e28fee03475 70cd37dbbebdb7a2f84f08207f18eabb0b291a55 Georgi Gerganov ggerganov@gmail.com 2025-09-09T08:42:10+03:00 GitHub noreply@github.com 2025-09-09T08:42:10+03:00 contrib : add notes about merging PRs (#15881) 70cd37dbbebdb7a2f84f08207f18eabb0b291a55 acc1b008cfd95e63d5f99a370dbffb98e5a99d2c Daniel Bevenius daniel.bevenius@gmail.com 2025-09-09T06:06:52+02:00 GitHub noreply@github.com 2025-09-09T06:06:52+02:00 requirements : update transformers/torch for Embedding Gemma (#15828) acc1b008cfd95e63d5f99a370dbffb98e5a99d2c 7057faf64b514e991e2f70147f82bb13d544b1c0 Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-09-09T06:05:55+02:00 GitHub noreply@github.com 2025-09-09T06:05:55+02:00 model-conversion : add extra debugging support for model conversion (#15877) 7057faf64b514e991e2f70147f82bb13d544b1c0 fe1c92cd7bb491e9c5767c9de413f2b7dc1e832b Aldehir Rojas hello@alde.dev 2025-09-08T16:14:32-05:00 GitHub noreply@github.com 2025-09-08T16:14:32-05:00 json : support `enum` values within `allOf` (#15830) fe1c92cd7bb491e9c5767c9de413f2b7dc1e832b e68aa10d8f3d26fdad5b912540362d79de5460e3 j-k dev@j-k.io 2025-09-08T19:57:01+01:00 GitHub noreply@github.com 2025-09-08T21:57:01+03:00 media : add llama1 icon (#15878) e68aa10d8f3d26fdad5b912540362d79de5460e3 0a16bf52e6874369cb4fd2bdc4863ef984b688e7 Jeff Bolz jbolz@nvidia.com 2025-09-08T13:10:07-05:00 GitHub noreply@github.com 2025-09-09T02:10:07+08:00 vulkan: sort graph to allow more parallel execution (#15850) 0a16bf52e6874369cb4fd2bdc4863ef984b688e7 88021565f08e0b7c4e07ac089a15ec16fae9166c Aman Gupta amangupta052@gmail.com 2025-09-09T01:23:46+08:00 GitHub noreply@github.com 2025-09-09T01:23:46+08:00 CUDA: generate_cu_files.py - add missing mxfp4 (#15880) 88021565f08e0b7c4e07ac089a15ec16fae9166c 56920f56651908d5cce7a310dabf54ac4f6fbb7f Jesse jesse@createthis.com 2025-09-08T10:59:48-04:00 GitHub noreply@github.com 2025-09-08T16:59:48+02:00 chat : Deepseek V3.1 reasoning and tool calling support (OpenAI Style) (#15533) 56920f56651908d5cce7a310dabf54ac4f6fbb7f b0d52998b962bd2681c34bf52af993af79f178b8 Xuan-Son Nguyen son@huggingface.co 2025-09-08T21:50:05+07:00 GitHub noreply@github.com 2025-09-08T16:50:05+02:00 server : bring back timings_per_token (#15879) b0d52998b962bd2681c34bf52af993af79f178b8 f28d4f4ac963f182ea9d0fe9e269f3f5f3782aaf Georgi Gerganov ggerganov@gmail.com 2025-09-08T13:56:51+03:00 GitHub noreply@github.com 2025-09-08T13:56:51+03:00 cuda : fix supports_op condition for get_rows when number of blocks is too large (#15868) f28d4f4ac963f182ea9d0fe9e269f3f5f3782aaf 9fcb29f22f5c33c04c7f0daebb24057899d67a1a Georgi Gerganov ggerganov@gmail.com 2025-09-08T13:34:56+03:00 GitHub noreply@github.com 2025-09-08T13:34:56+03:00 metal : refactor + optimize (#15857) 9fcb29f22f5c33c04c7f0daebb24057899d67a1a 5ef22d281de9c5eaaf616874bc490b89241128cb Xuan-Son Nguyen son@huggingface.co 2025-09-08T17:33:01+07:00 GitHub noreply@github.com 2025-09-08T12:33:01+02:00 ggml: allow casting between f32 and i32 (#15783) 5ef22d281de9c5eaaf616874bc490b89241128cb 233d773d02c37982badddf3994e9953a175f34da Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-08T11:55:44+02:00 GitHub noreply@github.com 2025-09-08T12:55:44+03:00 CUDA: non-contiguous src0 not supported for PAD (#15869) 233d773d02c37982badddf3994e9953a175f34da a885dcff11a7b73f9377812d6151f6b15d307de0 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-08T09:44:34+02:00 GitHub noreply@github.com 2025-09-08T09:44:34+02:00 convert : force setting sliding_window from original config (#15867) a885dcff11a7b73f9377812d6151f6b15d307de0 663027fd5490438ce9c27ea866a560e1e268d11f Georgi Gerganov ggerganov@gmail.com 2025-09-08T10:27:07+03:00 GitHub noreply@github.com 2025-09-08T10:27:07+03:00 batched-bench : fix llama_synchronize usage during prompt processing (#15835) 663027fd5490438ce9c27ea866a560e1e268d11f cf0e3ba1500bd23635b444f64ba23cbdb56c92ef Georgi Gerganov ggerganov@gmail.com 2025-09-08T10:26:36+03:00 GitHub noreply@github.com 2025-09-08T10:26:36+03:00 context : fix n_outputs during reserve (#15858) cf0e3ba1500bd23635b444f64ba23cbdb56c92ef d413dca00360a7e4cb71441dacecfa32556fcc31 Georgi Gerganov ggerganov@gmail.com 2025-09-08T10:25:33+03:00 GitHub noreply@github.com 2025-09-08T10:25:33+03:00 model : avoid ggml_cont_3d for fused QKV weights (#15662) d413dca00360a7e4cb71441dacecfa32556fcc31 85ca66a74676e6d5df4433016488e039a4b464ae Jeff Bolz jbolz@nvidia.com 2025-09-07T23:23:41-05:00 GitHub noreply@github.com 2025-09-07T23:23:41-05:00 tests: large sizes for get_rows (#15687) 85ca66a74676e6d5df4433016488e039a4b464ae 3976dfbe00f02a62c0deca32c46138e4f0ca81d8 Chenguang Li 757486878@qq.com 2025-09-08T10:03:29+08:00 GitHub noreply@github.com 2025-09-08T10:03:29+08:00 CANN: Stream sync between devices for acl_graph (#15809) 3976dfbe00f02a62c0deca32c46138e4f0ca81d8 d36e61c580bf7fc7879c443c542312a42b718e11 Jeff Bolz jbolz@nvidia.com 2025-09-07T13:50:26-05:00 GitHub noreply@github.com 2025-09-07T13:50:26-05:00 vulkan: support im2col_3d (#15795) d36e61c580bf7fc7879c443c542312a42b718e11 c97b5e5854b47b18a248d77edb693c63018a0865 Aaron Teo aaron.teo1@ibm.com 2025-09-08T02:18:28+08:00 GitHub noreply@github.com 2025-09-08T02:18:28+08:00 ggml-cpu: clean up s390x SIMD (#15855) c97b5e5854b47b18a248d77edb693c63018a0865 267e99867f09bec8bcc2e424ad9bcddd6cccf9d0 Jeff Bolz jbolz@nvidia.com 2025-09-07T12:00:49-05:00 GitHub noreply@github.com 2025-09-07T19:00:49+02:00 vulkan: Support pad_ext (#15794) 267e99867f09bec8bcc2e424ad9bcddd6cccf9d0 3b15924d71237a43bb5ad71f5b885ee66a821342 Jeff Bolz jbolz@nvidia.com 2025-09-07T11:53:07-05:00 GitHub noreply@github.com 2025-09-07T18:53:07+02:00 vulkan: Use larger loads in scalar/coopmat1 matmul (#15729) 3b15924d71237a43bb5ad71f5b885ee66a821342 79bc429262268ad2ac8a364cfe6c2d6b9c5f008a Daniel Bevenius daniel.bevenius@gmail.com 2025-09-07T10:19:45+02:00 GitHub noreply@github.com 2025-09-07T11:19:45+03:00 ggml WebGPU: remove userdata from request adapter callback (#15527) 79bc429262268ad2ac8a364cfe6c2d6b9c5f008a c4df49a42d396bdf7344501813e7de53bc9e7bb3 Johannes Gäßler johannesg@5d6.de 2025-09-07T00:26:28+02:00 GitHub noreply@github.com 2025-09-07T00:26:28+02:00 CUDA: faster tile FA (Pascal/AMD), headsize 256 (#15769) c4df49a42d396bdf7344501813e7de53bc9e7bb3 3c3635d2f20424d557b5b0605a2a356214ffe048 Charles Xu charles.xu@arm.com 2025-09-06T16:08:43+02:00 GitHub noreply@github.com 2025-09-06T22:08:43+08:00 kleidiai: generalize compute_forward_kv_cache to compute_forward_fp16 (#15817) 3c3635d2f20424d557b5b0605a2a356214ffe048 61bdfd5298a78593be649a1035ee2a120b13c4f0 Xuan-Son Nguyen son@huggingface.co 2025-09-06T19:45:24+07:00 GitHub noreply@github.com 2025-09-06T14:45:24+02:00 server : speed up tests (#15836) 61bdfd5298a78593be649a1035ee2a120b13c4f0 01806e77714ae8a78130d432945b959a0956c56f Xuan-Son Nguyen son@huggingface.co 2025-09-06T18:35:04+07:00 GitHub noreply@github.com 2025-09-06T13:35:04+02:00 server : implement prompt processing progress report in stream mode (#15827) 01806e77714ae8a78130d432945b959a0956c56f 186415d59552bd5c90549cd8e9be3cc287621fd9 Johannes Gäßler johannesg@5d6.de 2025-09-06T13:28:44+02:00 GitHub noreply@github.com 2025-09-06T13:28:44+02:00 ggml-cpu: document use of "free" memory [no ci] (#15834) 186415d59552bd5c90549cd8e9be3cc287621fd9 fd621880f3fd908424e675a41715a2dc760247a2 Aaron Teo aaron.teo1@ibm.com 2025-09-06T11:27:28+08:00 GitHub noreply@github.com 2025-09-06T11:27:28+08:00 ggml-cpu: drop support for nnpa intrinsics (#15821) fd621880f3fd908424e675a41715a2dc760247a2 4281c7b315f8a904549fe527039b976e08098d1a Gabe Goodhart ghart@us.ibm.com 2025-09-05T17:32:39-06:00 GitHub noreply@github.com 2025-09-05T17:32:39-06:00 aLoRA Support (#15327) 4281c7b315f8a904549fe527039b976e08098d1a 5fac79cbc77b6d12c9feb5f34fc63586b35fd561 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-09-06T01:21:15+02:00 GitHub noreply@github.com 2025-09-06T01:21:15+02:00 ci : exempt correct research label (#15825) 5fac79cbc77b6d12c9feb5f34fc63586b35fd561 408ff524b40baf4f51a81d42a9828200dd4fcb6b Gabe Goodhart ghart@us.ibm.com 2025-09-05T14:31:24-06:00 GitHub noreply@github.com 2025-09-05T14:31:24-06:00 Thinking model disabled assistant prefill (#15404) 408ff524b40baf4f51a81d42a9828200dd4fcb6b 5143fa895e7725c5bd2135daf7d8f793d98fa91c Eric Curtin ericcurtin17@gmail.com 2025-09-05T19:43:59+01:00 GitHub noreply@github.com 2025-09-05T19:43:59+01:00 Implement --log-colors with always/never/auto (#15792) 5143fa895e7725c5bd2135daf7d8f793d98fa91c 3a550b5ca4565c9e28f63880d47840feb27d0ff6 Johannes Gäßler johannesg@5d6.de 2025-09-05T16:07:02+02:00 GitHub noreply@github.com 2025-09-05T16:07:02+02:00 CUDA: fastdiv, launch bounds for mmvq + q8_1 quant (#15802) 3a550b5ca4565c9e28f63880d47840feb27d0ff6 a81283820a466f2ace06ce4d4bc9512761f9365f Daniel Bevenius daniel.bevenius@gmail.com 2025-09-05T14:49:21+02:00 GitHub noreply@github.com 2025-09-05T13:49:21+01:00 tests : add --list-ops and --show-coverage options (#15745) a81283820a466f2ace06ce4d4bc9512761f9365f c610b6c11b1ef7d678671dcf15acd7187a7ad8f3 Erik Scholz Green-Sky@users.noreply.github.com 2025-09-05T11:34:28+02:00 GitHub noreply@github.com 2025-09-05T11:34:28+02:00 gguf: gguf_writer refactor (#15691) c610b6c11b1ef7d678671dcf15acd7187a7ad8f3 5d6688de08e73acc2532d668380801ed79d704eb Georgi Gerganov ggerganov@gmail.com 2025-09-05T10:39:22+03:00 GitHub noreply@github.com 2025-09-05T10:39:22+03:00 kv-cache : fix SWA checks + disable cacheless iSWA (#15811) dcbe998a6d8f13b6c9cb36bbe7b355ced2ac3842 87932ec016f0ec81e98a13ce5212851f8c12458a Yunzhe Jia yunzhe@calculet.tech 2025-09-05T14:34:07+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-09-05T14:34:07+08:00 add calrt API used in server side 5d6688de08e73acc2532d668380801ed79d704eb 4fd1242bef6cb2325b4ff1c1a80f3b54b64508a6 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-05T04:36:23+02:00 GitHub noreply@github.com 2025-09-05T04:36:23+02:00 model-conversion : add --embeddings flag to modelcard.template [no ci] (#15801) 4fd1242bef6cb2325b4ff1c1a80f3b54b64508a6 b2426e469e2fdb6c44216d56baa4cfff4f39ae00 ExtReMLapin 3909752+ExtReMLapin@users.noreply.github.com 2025-09-05T01:24:08+02:00 GitHub noreply@github.com 2025-09-05T01:24:08+02:00 chat : fixed crash when Hermes 2 had a newline before it (#15639) b2426e469e2fdb6c44216d56baa4cfff4f39ae00 9e2b1e83c68a38ea0c64f726dd979439bd02189b Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-09-05T01:22:22+02:00 GitHub noreply@github.com 2025-09-05T01:22:22+02:00 chat : nemotron thinking & toolcalling support (#15676) 9e2b1e83c68a38ea0c64f726dd979439bd02189b fb15d649ed14ab447eeab911e0c9d21e35fb243e Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-09-05T01:05:12+02:00 GitHub noreply@github.com 2025-09-05T01:05:12+02:00 scripts : add Jinja tester PySide6 simple app (#15756) fb15d649ed14ab447eeab911e0c9d21e35fb243e 856ed0947f27b4ec3ad269fceda0402fbab263d3 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-04T18:10:29+02:00 GitHub noreply@github.com 2025-09-04T18:10:29+02:00 llama : add support for EmbeddingGemma 300m (#15798) 856ed0947f27b4ec3ad269fceda0402fbab263d3 d1e2adba65208d6de3d7f6f23b00c562ff5bb777 Gabe Goodhart ghart@us.ibm.com 2025-09-04T09:53:22-06:00 GitHub noreply@github.com 2025-09-04T18:53:22+03:00 metal : Add template specialization for mul_mm_id w/ ne20 == 10 (#15799) d1e2adba65208d6de3d7f6f23b00c562ff5bb777 c1c354e44c06d259679bb5bb7a8fa9f0b28480e4 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-04T15:40:44+02:00 GitHub noreply@github.com 2025-09-04T15:40:44+02:00 llama : set n_outputs to 1 to avoid 0 outputs mean-pooling (#15791) c1c354e44c06d259679bb5bb7a8fa9f0b28480e4 a68d9144262f1d0ef4f6ba7ad4a7e73e977ba78c Chenguang Li 757486878@qq.com 2025-09-04T20:20:14+08:00 GitHub noreply@github.com 2025-09-04T20:20:14+08:00 CANN: Refactor ND to NZ workspace to be per-device (#15763) a68d9144262f1d0ef4f6ba7ad4a7e73e977ba78c badb80cadbc40e047b30c43611aba575fc8d6845 Xuan-Son Nguyen son@huggingface.co 2025-09-04T11:50:23+02:00 GitHub noreply@github.com 2025-09-04T11:50:23+02:00 server: add exceed_context_size_error type (#15780) badb80cadbc40e047b30c43611aba575fc8d6845 0a1b3982cd0bd18730d50a693053b88c13fd04a6 Eric Curtin ecurtin@redhat.com 2025-09-04T10:49:44+01:00 GitHub noreply@github.com 2025-09-04T10:49:44+01:00 Document the new max GPU layers default in help (#15771) 0a1b3982cd0bd18730d50a693053b88c13fd04a6 5421f63ab08ca1e1a093662a5ccd0117e461185f leejet leejet714@gmail.com 2025-09-04T16:38:49+08:00 GitHub noreply@github.com 2025-09-04T10:38:49+02:00 ggml: add ops for WAN video model (cuda && cpu) (#15669) 5421f63ab08ca1e1a093662a5ccd0117e461185f 820bc9853100708011036e10f40b923968dd9b66 hipudding huafengchun@gmail.com 2025-09-04T15:12:30+08:00 GitHub noreply@github.com 2025-09-04T15:12:30+08:00 CANN: Fix precision issue on 310I DUO multi-devices (#15784) 820bc9853100708011036e10f40b923968dd9b66 239b60e8986bbcb944f57311a02ac994431bf652 rmatif rmatif@proton.me 2025-09-04T08:30:28+02:00 GitHub noreply@github.com 2025-09-03T23:30:28-07:00 opencl: add hs=40 to FA (#15758) 239b60e8986bbcb944f57311a02ac994431bf652 dff7551bfdbdd6e57c13e523d7dcca317640e907 Chenguang Li 757486878@qq.com 2025-09-04T11:03:02+08:00 GitHub noreply@github.com 2025-09-04T11:03:02+08:00 CANN: fix acl_rstd allocation size in ggml_cann_rms_norm (#15760) dff7551bfdbdd6e57c13e523d7dcca317640e907 0fce7a1248b74148c1eb0d368b7e18e8bcb96809 Ruben Ortlam picard12@live.de 2025-09-03T22:55:10+02:00 GitHub noreply@github.com 2025-09-03T21:55:10+01:00 vulkan: fix mmv subgroup16 selection (#15775) 0fce7a1248b74148c1eb0d368b7e18e8bcb96809 8227695d7a3e5b357cb37fad263f00a8ca6db710 Jeff Bolz jbolz@nvidia.com 2025-09-03T13:33:15-05:00 GitHub noreply@github.com 2025-09-03T20:33:15+02:00 vulkan: don't use std::string in load_shaders, to improve compile time (#15724) 8227695d7a3e5b357cb37fad263f00a8ca6db710 0014fb4add3a7d000c14b763538045c6170f57d9 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-03T20:24:50+02:00 GitHub noreply@github.com 2025-09-03T20:24:50+02:00 vulkan : update ggml_vk_instance_validation_ext_available (#15666) 0014fb4add3a7d000c14b763538045c6170f57d9 661ae31c9c68201577e70278285b349a5a662caf Shin-myoung-serp relent95@naver.com 2025-09-04T03:22:55+09:00 GitHub noreply@github.com 2025-09-03T20:22:55+02:00 ggml vulkan: add hardsigmoid and hardswish operations (#15762) 661ae31c9c68201577e70278285b349a5a662caf 407c23786dd0d3a503e9429eead96e611d3950c9 Oliver Simons osimons@nvidia.com 2025-09-03T19:59:16+02:00 GitHub noreply@github.com 2025-09-03T19:59:16+02:00 CUDA: Optimize `rms_norm_f32` kernel and its fused variants, giving 1-6% perf E2E (#15715) 407c23786dd0d3a503e9429eead96e611d3950c9 cdedb70a998cea7052560fe0b0615a839443564d Daniel Bevenius daniel.bevenius@gmail.com 2025-09-03T18:28:36+02:00 GitHub noreply@github.com 2025-09-03T18:28:36+02:00 model-conversion : fix pyright errors (#15770) cdedb70a998cea7052560fe0b0615a839443564d 2c8dac72eb6acd4e20c0da251535dfc46d35178b Georgi Gerganov ggerganov@gmail.com 2025-09-03T18:16:26+03:00 GitHub noreply@github.com 2025-09-03T18:16:26+03:00 sampling : optimize dist sampler (#15704) 2c8dac72eb6acd4e20c0da251535dfc46d35178b 40a751ea9a94364da73537b86502a808ebe1fc3a Daniel Bevenius daniel.bevenius@gmail.com 2025-09-03T13:35:49+02:00 GitHub noreply@github.com 2025-09-03T13:35:49+02:00 llama : fix incorrect model type for Gemma 270M (#15764) 40a751ea9a94364da73537b86502a808ebe1fc3a 5eae9348835037046f33fafd852df7c952c95209 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-03T12:50:47+02:00 GitHub noreply@github.com 2025-09-03T12:50:47+02:00 model-conversion : remove hardcoded /bin/bash shebangs [no ci] (#15765) 5eae9348835037046f33fafd852df7c952c95209 05c0380f2ae5c7193445e03ebc7b6de9ce49f1a6 hipudding huafengchun@gmail.com 2025-09-03T16:46:01+08:00 GitHub noreply@github.com 2025-09-03T16:46:01+08:00 CANN: Add RoPE contiguous check for 310I DUP device (#15735) 05c0380f2ae5c7193445e03ebc7b6de9ce49f1a6 8c3fdf44ecf08335942f2ba558955f55e88c7991 xctan xc-tan@outlook.com 2025-09-03T16:16:21+08:00 GitHub noreply@github.com 2025-09-03T16:16:21+08:00 ggml-cpu : optimize RVV kernels (#15720) 8c3fdf44ecf08335942f2ba558955f55e88c7991 f6da8cb86a28f0319b40d9d2a957a26a7d875f8c Daniel Bevenius daniel.bevenius@gmail.com 2025-09-03T09:48:35+02:00 GitHub noreply@github.com 2025-09-03T09:48:35+02:00 model-conversion : add missing curl script [no ci] (#15761) f6da8cb86a28f0319b40d9d2a957a26a7d875f8c 8a2234ea0c89f212190c176d741b7742f0082582 hipudding huafengchun@gmail.com 2025-09-03T14:08:22+08:00 GitHub noreply@github.com 2025-09-03T14:08:22+08:00 CANN: Mask unsupported TRANSPOSE_1D operator (#15733) 8a2234ea0c89f212190c176d741b7742f0082582 3de008208b9b8a33f49f979097a99b4d59e6e521 Chenguang Li 757486878@qq.com 2025-09-03T10:43:53+08:00 GitHub noreply@github.com 2025-09-03T10:43:53+08:00 CANN: Fix type float_t to float (#15736) 3de008208b9b8a33f49f979097a99b4d59e6e521 69db8a52e6dd0db81ec05c581150cc1e43b8ac46 SnA1lGo 44647694+skrandy@users.noreply.github.com 2025-09-03T03:27:30+08:00 GitHub noreply@github.com 2025-09-02T21:27:30+02:00 fix: resolve unsigned int initialization warning for n_dims/size in gguf.cpp (#15754) 69db8a52e6dd0db81ec05c581150cc1e43b8ac46 c466abe1587bde458c806e2134e37750f2edf8fb Oliver Simons osimons@nvidia.com 2025-09-02T19:40:37+02:00 GitHub noreply@github.com 2025-09-03T01:40:37+08:00 chore: Update `.clang-format` to use `BinPackArguments=true` (#15744) c466abe1587bde458c806e2134e37750f2edf8fb 0a2a3841e8ebc570da343e8cf58a21c1010b41e7 Johannes Gäßler johannesg@5d6.de 2025-09-02T18:17:26+02:00 GitHub noreply@github.com 2025-09-02T18:17:26+02:00 llama: -fa 1/0/-1 aliases for -fa on/off/auto (#15746) 0a2a3841e8ebc570da343e8cf58a21c1010b41e7 9961d244f2df6baf40af2f1ddc0927f8d91578c8 Ruben Ortlam picard12@live.de 2025-09-02T16:02:26+02:00 GitHub noreply@github.com 2025-09-02T16:02:26+02:00 vulkan: fix shaders gen when no integer dot is available (#15740) 9961d244f2df6baf40af2f1ddc0927f8d91578c8 25f1045f07cf0daf667d63e35618842e3174a8c7 hipudding huafengchun@gmail.com 2025-09-02T17:12:37+08:00 GitHub noreply@github.com 2025-09-02T17:12:37+08:00 CANN: Resolve soft_max precision issue (#15730) 25f1045f07cf0daf667d63e35618842e3174a8c7 97669e40735d08db65ef094a8faae8e8411a97db Jeff Bolz jbolz@nvidia.com 2025-09-02T01:37:01-05:00 GitHub noreply@github.com 2025-09-02T14:37:01+08:00 vulkan: Fix macro parameter order for f32 matmul shaders (#15716) 97669e40735d08db65ef094a8faae8e8411a97db 2f853687b3bce15e143a22f678d1715060fd606c rmatif rmatif@proton.me 2025-09-02T08:26:53+02:00 GitHub noreply@github.com 2025-09-01T23:26:53-07:00 opencl: add attn sinks support for FA kernels (#15706) 2f853687b3bce15e143a22f678d1715060fd606c ef2af57ddf04ab367f6ba6e8ed100fc56785e4b7 Chenguang Li 757486878@qq.com 2025-09-02T14:07:48+08:00 GitHub noreply@github.com 2025-09-02T14:07:48+08:00 CANN: Support eager execution mode under ACL graph compilation (#15712) ef2af57ddf04ab367f6ba6e8ed100fc56785e4b7 5d804a4938d896f9089687a8b99911cdb43b0b94 hipudding huafengchun@gmail.com 2025-09-02T14:05:23+08:00 GitHub noreply@github.com 2025-09-02T14:05:23+08:00 CANN: Support ext_factor in rope (#15710) 5d804a4938d896f9089687a8b99911cdb43b0b94 d4d8dbe383e8b9600cbe8b42016e3a4529b51219 Johannes Gäßler johannesg@5d6.de 2025-09-02T01:14:55+02:00 GitHub noreply@github.com 2025-09-01T16:14:55-07:00 ggml-backend: raise GGML_MAX_SPLIT_INPUTS (#15722) d4d8dbe383e8b9600cbe8b42016e3a4529b51219 35a42edac84690990a75726898d975cd08d220c0 Gilad S. 7817232+giladgd@users.noreply.github.com 2025-09-01T22:17:42+03:00 GitHub noreply@github.com 2025-09-01T21:17:42+02:00 vulkan: use memory budget extension to read memory usage (#15545) 35a42edac84690990a75726898d975cd08d220c0 fec7911f8febb22c27ce1acb12441800da24cc06 Jeff Bolz jbolz@nvidia.com 2025-09-01T14:01:10-05:00 GitHub noreply@github.com 2025-09-01T21:01:10+02:00 vulkan: add missing clamps in new mul_mat_id paths (#15702) fec7911f8febb22c27ce1acb12441800da24cc06 078ce23ea77988f2fe1a42afb18d58bc084d55fa Ruben Ortlam picard12@live.de 2025-09-01T20:58:35+02:00 GitHub noreply@github.com 2025-09-01T20:58:35+02:00 vulkan: disable large mmv subgroups on older Nvidia GPUs (#15717) 078ce23ea77988f2fe1a42afb18d58bc084d55fa a0c2b207c596d1092a08615de61ab56a7d63515f s-goto-11 206795233+s-goto-11@users.noreply.github.com 2025-09-02T03:13:49+09:00 GitHub noreply@github.com 2025-09-01T20:13:49+02:00 ggml: SVE support for exponential functions (#15145) a0c2b207c596d1092a08615de61ab56a7d63515f 4b20d8b7e31718d3fe8b4f12d220a9d19e4f1997 Prashant Vithule 119530321+Vithulep@users.noreply.github.com 2025-09-01T23:43:16+05:30 GitHub noreply@github.com 2025-09-01T20:13:16+02:00 ggml: aarch64: Implement SVE F16 kernels for vector functions (#15115) 4b20d8b7e31718d3fe8b4f12d220a9d19e4f1997 02c1813517412f3e00aa6ca7c0273fea64edb492 Jie Fu (傅杰) jiefu@tencent.com 2025-09-01T23:53:31+08:00 GitHub noreply@github.com 2025-09-01T23:53:31+08:00 convert : remove redundant code (#15708) 02c1813517412f3e00aa6ca7c0273fea64edb492 77dee9de97be75b7143a213bc48893e0c0b29af7 Ruben Ortlam picard12@live.de 2025-09-01T16:19:07+02:00 GitHub noreply@github.com 2025-09-01T16:19:07+02:00 Vulkan: Add Integer Dot Product mul_mat_vec shader for legacy quants (#14903) 77dee9de97be75b7143a213bc48893e0c0b29af7 4795c91c32fec7165a1364763d4d4f0c93abf933 Daniel Bevenius daniel.bevenius@gmail.com 2025-09-01T14:28:49+02:00 GitHub noreply@github.com 2025-09-01T14:28:49+02:00 ggml : WebGPU add TRANSPOSE and RESHAPE to supported ops (#15695) 4795c91c32fec7165a1364763d4d4f0c93abf933 b66df9d9c942254d03209186ef24ed7c994a576e Jie Fu (傅杰) jiefu@tencent.com 2025-09-01T15:34:59+08:00 GitHub noreply@github.com 2025-09-01T10:34:59+03:00 docs : add Hunyuan to models section (#15707) b66df9d9c942254d03209186ef24ed7c994a576e b9382c3877c6067feccf182efe9449a2d1cb24c7 Akarshan Biswas akarshan@menlo.ai 2025-09-01T06:55:06+05:30 GitHub noreply@github.com 2025-09-01T06:55:06+05:30 CUDA: fix build error from ambiguous __half conversions in conv2d (#15690) b9382c3877c6067feccf182efe9449a2d1cb24c7 3dc7397a2799bdc07bccf637ab7ae5a1e786d1a4 hipudding huafengchun@gmail.com 2025-09-01T08:57:23+08:00 GitHub noreply@github.com 2025-09-01T08:57:23+08:00 CANN: Optimize MUL_MAT_ID (#15658) 3dc7397a2799bdc07bccf637ab7ae5a1e786d1a4 e92d53b29e393fc4c0f9f1f7c3fe651be8d36faa hipudding huafengchun@gmail.com 2025-09-01T08:57:00+08:00 GitHub noreply@github.com 2025-09-01T08:57:00+08:00 CANN: fix RoPE cache issue on multi-device (#15629) e92d53b29e393fc4c0f9f1f7c3fe651be8d36faa 0d161f021aa33ec0e90cce96f5d1a88925557327 Georgi Gerganov ggerganov@gmail.com 2025-08-31T20:41:02+03:00 GitHub noreply@github.com 2025-08-31T20:41:02+03:00 sampling : optimize samplers by reusing bucket sort (#15665) 0d161f021aa33ec0e90cce96f5d1a88925557327 4efd5a83163ff383285b3a4c2106feabf5c69557 Georgi Gerganov ggerganov@gmail.com 2025-08-31T20:11:58+03:00 GitHub noreply@github.com 2025-08-31T20:11:58+03:00 server : enable /slots by default and make it secure (#15630) 4efd5a83163ff383285b3a4c2106feabf5c69557 274966226f87f301ac132da898280ca3142b60e5 Georgi Gerganov ggerganov@gmail.com 2025-08-31T19:43:30+03:00 GitHub noreply@github.com 2025-08-31T19:43:30+03:00 metal : fix checks for available FA kernels (#15700) 274966226f87f301ac132da898280ca3142b60e5 9777032dccd67bdc7785aeab7497014a8be8dacc Diego Devesa slarengh@gmail.com 2025-08-31T08:47:05-07:00 GitHub noreply@github.com 2025-08-31T18:47:05+03:00 llama : fix fattn reserve call n_seqs parameter (#15699) 9777032dccd67bdc7785aeab7497014a8be8dacc 7d3c9f2b217acf0ce5db81ae83d3f375f49ab2c7 Diego Devesa slarengh@gmail.com 2025-08-31T06:49:03-07:00 GitHub noreply@github.com 2025-08-31T15:49:03+02:00 llama : separate compute buffer reserve from fattn check (#15696) 7d3c9f2b217acf0ce5db81ae83d3f375f49ab2c7 bbbf5ecccb35286521f735239d499eec4279a840 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-31T15:30:20+02:00 GitHub noreply@github.com 2025-08-31T15:30:20+02:00 ci : explicitly set fa off or on (#15692) bbbf5ecccb35286521f735239d499eec4279a840 c37052ab4d6d1ae73c0e90bc6e560cc6409e1311 Jeff Bolz jbolz@nvidia.com 2025-08-31T03:13:27-05:00 GitHub noreply@github.com 2025-08-31T10:13:27+02:00 vulkan: handle large sizes for get_rows (#15686) c37052ab4d6d1ae73c0e90bc6e560cc6409e1311 5c16b9c87d840e4d5d55fa83c732c6b693346f40 Jeff Bolz jbolz@nvidia.com 2025-08-31T02:06:43-05:00 GitHub noreply@github.com 2025-08-31T09:06:43+02:00 vulkan: mul_mat_id coopmat2 optimizations (#15546) 5c16b9c87d840e4d5d55fa83c732c6b693346f40 b97c9edc59d4a1b4069991aa670411190f4f3a3e Daniel Bevenius daniel.bevenius@gmail.com 2025-08-31T08:46:42+02:00 GitHub noreply@github.com 2025-08-31T08:46:42+02:00 vulkan : remove unused portability_enumeration_ext variable (#15679) b97c9edc59d4a1b4069991aa670411190f4f3a3e 94e82c7eadeb8fff0db4bfd1ab6d8cf65fa6f2e0 Jeff Bolz jbolz@nvidia.com 2025-08-31T01:30:54-05:00 GitHub noreply@github.com 2025-08-31T08:30:54+02:00 vulkan: Allow fallback to sysmem memory when vidmem is full (#15649) 94e82c7eadeb8fff0db4bfd1ab6d8cf65fa6f2e0 4d74393bcc956ccd7df68a6a06d1a0575cfa712c Jeff Bolz jbolz@nvidia.com 2025-08-31T01:27:57-05:00 GitHub noreply@github.com 2025-08-31T08:27:57+02:00 vulkan: clamp matmul and FA results to the max finite value (#15652) 4d74393bcc956ccd7df68a6a06d1a0575cfa712c dd892555b0681b7f56d38780f6fdfe00a195160f Charles Xu charles.xu@arm.com 2025-08-30T18:03:42+02:00 GitHub noreply@github.com 2025-08-31T00:03:42+08:00 ggml: update kleidiai to v1.13.0 (#15663) dd892555b0681b7f56d38780f6fdfe00a195160f e81b8e4b7f5ab870836fad26d154a7507b341b36 Diego Devesa slarengh@gmail.com 2025-08-30T08:51:28-07:00 GitHub noreply@github.com 2025-08-30T23:51:28+08:00 Update build.md to remove MSVC arm64 notes (#15684) e81b8e4b7f5ab870836fad26d154a7507b341b36 38ad381f9f5d4dd368a96d844fb19cf501ed9d22 Johannes Gäßler johannesg@5d6.de 2025-08-30T16:32:10+02:00 GitHub noreply@github.com 2025-08-30T16:32:10+02:00 llama: use FA + max. GPU layers by default (#15434) 38ad381f9f5d4dd368a96d844fb19cf501ed9d22 696fccf354e9dbdfbce135bc40b44c9dcc64dda9 Johannes Gäßler johannesg@5d6.de 2025-08-30T16:20:32+02:00 GitHub noreply@github.com 2025-08-30T16:20:32+02:00 CUDA: use FP32 arithmetic for conv2d (#15683) 696fccf354e9dbdfbce135bc40b44c9dcc64dda9 ef476916bba4b44f44be0c98babc1cb025968e75 Jeff Bolz jbolz@nvidia.com 2025-08-30T04:11:22-05:00 GitHub noreply@github.com 2025-08-30T11:11:22+02:00 vulkan: Skip syncing for prealloc_y when it is reused (#15544) ef476916bba4b44f44be0c98babc1cb025968e75 d82f6aa34a216f5df1945cdfe121ba5e6cd80be0 Chenguang Li 757486878@qq.com 2025-08-30T10:18:35+08:00 GitHub noreply@github.com 2025-08-30T10:18:35+08:00 CANN: FIx compiler warnings (#15661) d82f6aa34a216f5df1945cdfe121ba5e6cd80be0 3d16b29c3bb1ec816ac0e782f20d169097063919 Sergey Alirzaev l29ah@riseup.net 2025-08-30T00:12:53+02:00 GitHub noreply@github.com 2025-08-30T00:12:53+02:00 server : removed obsolete doc (#15670) 3d16b29c3bb1ec816ac0e782f20d169097063919 792b44f2ed9668cce7f267ff0ae4950ed9b4a5de Johannes Gäßler johannesg@5d6.de 2025-08-29T22:04:08+02:00 GitHub noreply@github.com 2025-08-29T22:04:08+02:00 scripts: strip "AMD Instinct" from GPU name (#15668) 792b44f2ed9668cce7f267ff0ae4950ed9b4a5de 81017865ee444cf49ce0136f2be1e41a0270ff91 ExtReMLapin 3909752+ExtReMLapin@users.noreply.github.com 2025-08-29T19:25:40+02:00 GitHub noreply@github.com 2025-08-29T20:25:40+03:00 server : add documentation for `parallel_tool_calls` param (#15647) 81017865ee444cf49ce0136f2be1e41a0270ff91 60e5eee31f1af9bb579ac45380e3857d610020b9 Aman Gupta amangupta052@gmail.com 2025-08-29T21:30:06+08:00 GitHub noreply@github.com 2025-08-29T21:30:06+08:00 CUDA: fix bug in rms_norm fusion (#15660) 60e5eee31f1af9bb579ac45380e3857d610020b9 009b709d6efd24820ac67765ed339a72dc797814 Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-08-29T14:53:41+02:00 GitHub noreply@github.com 2025-08-29T14:53:41+02:00 chat : Seed OSS thinking + tool call support (#15552) 009b709d6efd24820ac67765ed339a72dc797814 e8d99dd0b67f2ecc1e45fca8074a3a18c3e036d2 Aman Gupta amangupta052@gmail.com 2025-08-29T11:35:58+08:00 GitHub noreply@github.com 2025-08-29T11:35:58+08:00 CUDA: fuse adds, fuse add with rms norm (#15631) e8d99dd0b67f2ecc1e45fca8074a3a18c3e036d2 a8bca68f727844e7dcf24a956003b3c2039ea563 Gabe Goodhart ghart@us.ibm.com 2025-08-28T18:39:31-06:00 GitHub noreply@github.com 2025-08-28T18:39:31-06:00 nvidia nemotron nano v2 (nemotronh) (#15507) a8bca68f727844e7dcf24a956003b3c2039ea563 c97dc093912ad014f6d22743ede0d4d7fd82365a Gabe Goodhart ghart@us.ibm.com 2025-08-28T15:27:36-05:00 GitHub noreply@github.com 2025-08-28T15:27:36-05:00 fix: Compute the full sum in llama-eval-callback, not just the sum of printed values (#15637) c97dc093912ad014f6d22743ede0d4d7fd82365a 6c442f42ff25564a0cd6b1435d9abc1b0178eac5 mnehete32 33429707+mnehete32@users.noreply.github.com 2025-08-29T00:03:03+05:30 GitHub noreply@github.com 2025-08-28T20:33:03+02:00 CUDA: add conv2d (#15635) 6c442f42ff25564a0cd6b1435d9abc1b0178eac5 73804145ab6c06888e314046abf08f66a0133679 Aaron Teo aaron.teo1@ibm.com 2025-08-28T22:39:27+08:00 GitHub noreply@github.com 2025-08-28T22:39:27+08:00 ggml-cpu: fix invalid hsum build in debug s390x (#15634) 73804145ab6c06888e314046abf08f66a0133679 c8d0d14e77c3c45df5cbbddde9c2b4c377ec4e7a compilade git@compilade.net 2025-08-28T10:11:36-04:00 GitHub noreply@github.com 2025-08-28T10:11:36-04:00 ggml : fix SSM_SCAN for n_groups > 1 (#15625) c8d0d14e77c3c45df5cbbddde9c2b4c377ec4e7a 84ab83cc0b4b7e769451ee48e4c7d1acef91ef25 Georgi Gerganov ggerganov@gmail.com 2025-08-28T17:09:05+03:00 GitHub noreply@github.com 2025-08-28T17:09:05+03:00 kv-cache : fix find_slot to not search for continuous slot (#15638) 84ab83cc0b4b7e769451ee48e4c7d1acef91ef25 55042b3692cb1467c9ee15c62c4a9fbf180f89e3 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-28T15:49:50+02:00 GitHub noreply@github.com 2025-08-28T15:49:50+02:00 model : jina-embeddings-v3 support (#13693) 55042b3692cb1467c9ee15c62c4a9fbf180f89e3 8a4280ce431da6b33e5a95ae1fd61472c8c3f8cc Aman Gupta amangupta052@gmail.com 2025-08-28T19:23:22+08:00 GitHub noreply@github.com 2025-08-28T19:23:22+08:00 scripts: add sqlite3 check for compare-commits.sh (#15633) 8a4280ce431da6b33e5a95ae1fd61472c8c3f8cc 64387f6e95434b393ac3df285864692b7fd9c4d2 Georgi Gerganov ggerganov@gmail.com 2025-08-28T12:27:02+03:00 GitHub noreply@github.com 2025-08-28T12:27:02+03:00 kv-cache : remove LLAMA_SET_ROWS checks (#15505) 64387f6e95434b393ac3df285864692b7fd9c4d2 d35a1e8c41f747548775225973a99507896a8c61 Aleksei Nikiforov 103434461+AlekseiNikiforovIBM@users.noreply.github.com 2025-08-28T10:56:41+02:00 GitHub noreply@github.com 2025-08-28T16:56:41+08:00 gguf-py: byteswapping improvements (#12851) d35a1e8c41f747548775225973a99507896a8c61 46d9caa27a0281150e8cf082308c0f9e7576ebe5 Joshua Cogliati jrincayc@users.noreply.github.com 2025-08-28T01:48:20-06:00 GitHub noreply@github.com 2025-08-28T10:48:20+03:00 cli : change log to warning to explain reason for stopping (#15604) 46d9caa27a0281150e8cf082308c0f9e7576ebe5 5a0e3ef6f00c658fbae53797f02d5a360ebf8fec Daniel Bevenius daniel.bevenius@gmail.com 2025-08-28T09:26:48+02:00 GitHub noreply@github.com 2025-08-28T09:26:48+02:00 model-conversion : add mmproj conversion target (#15628) 5a0e3ef6f00c658fbae53797f02d5a360ebf8fec fbef0fad7a7c765939f6c9e322fa05cd52cf0c15 matiaslin 45382001+matiaslin@users.noreply.github.com 2025-08-27T17:32:36-07:00 GitHub noreply@github.com 2025-08-28T02:32:36+02:00 cuda: Add cublasLt_static linking when GGML_STATIC is enabled (#15622) fbef0fad7a7c765939f6c9e322fa05cd52cf0c15 da54f9f1a2db07aaae390024ac466e7867685d94 Johannes Gäßler johannesg@5d6.de 2025-08-27T20:58:09+02:00 GitHub noreply@github.com 2025-08-27T20:58:09+02:00 server: higher timeout for tests (#15621) da54f9f1a2db07aaae390024ac466e7867685d94 47373271f971aa5a0a6462b286f4a7b5bd4ba644 Georgi Gerganov ggerganov@gmail.com 2025-08-27T15:48:07+03:00 GitHub noreply@github.com 2025-08-27T15:48:07+03:00 presets : add qwen3-30B-a3b FIM (#15616) 47373271f971aa5a0a6462b286f4a7b5bd4ba644 1bded5a3b3376b2aa3ba2f11ade5910c550f354b uvos carl@uvos.xyz 2025-08-27T13:58:54+02:00 GitHub noreply@github.com 2025-08-27T13:58:54+02:00 HIP: Enable support for ggml_backend_cuda_register_host_buffer (#15615) 1bded5a3b3376b2aa3ba2f11ade5910c550f354b 1e7489745a74996fc36e8fd05b73aa16bc184e0c Georgi Gerganov ggerganov@gmail.com 2025-08-27T13:55:12+03:00 GitHub noreply@github.com 2025-08-27T13:55:12+03:00 kv-cache : better estimate of n_kv for multi-sequence batches (#15610) 1e7489745a74996fc36e8fd05b73aa16bc184e0c 1cf123a343ab7ca5586aacb9e0a1d2de7fe33be4 Chenguang Li 757486878@qq.com 2025-08-27T17:21:41+08:00 GitHub noreply@github.com 2025-08-27T17:21:41+08:00 CANN: refactor mask handling and improve performance in FA (#15561) 1cf123a343ab7ca5586aacb9e0a1d2de7fe33be4 fcca2182a18c786c57d02d1d1927204b133f1fdc xctan xc-tan@outlook.com 2025-08-27T16:44:22+08:00 GitHub noreply@github.com 2025-08-27T16:44:22+08:00 ggml-cpu : add basic RVV support for vector f32 ops (#15057) fcca2182a18c786c57d02d1d1927204b133f1fdc 86076f92de1a547ef87c304facca3b4b9fae6c21 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-27T10:28:53+02:00 GitHub noreply@github.com 2025-08-27T10:28:53+02:00 common : add -m to bash completion for --model [no ci] (#15591) 86076f92de1a547ef87c304facca3b4b9fae6c21 bcbddcd54f0d5c22eab180831fdea6484107112f rmatif rmatif@proton.me 2025-08-27T08:36:05+02:00 GitHub noreply@github.com 2025-08-26T23:36:05-07:00 OpenCL: add fused group_norm/norm, mul, add (#15314) 87932ec016f0ec81e98a13ce5212851f8c12458a 0115cfb7c1dec0753aecb34f3af4338bf9de8eeb Yunzhe Jia yunzhe@calculet.tech 2025-08-27T14:29:18+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-08-27T14:29:18+08:00 fix merge problem bcbddcd54f0d5c22eab180831fdea6484107112f 8b696861364360770e9f61a3422d32941a477824 Diego Devesa slarengh@gmail.com 2025-08-26T13:14:38-07:00 GitHub noreply@github.com 2025-08-26T22:14:38+02:00 tests : fix test-opt with GGML_BACKEND_DL (#15599) 8b696861364360770e9f61a3422d32941a477824 8ce3ff1d91245e158d98d8062cd64b0dd98dcfe3 Akarshan Biswas akarshan@menlo.ai 2025-08-27T00:27:49+05:30 GitHub noreply@github.com 2025-08-27T00:27:49+05:30 SYCL: fix rms_norm_mul_add for tensor dim not a multiple of sg_size (#15592) 8ce3ff1d91245e158d98d8062cd64b0dd98dcfe3 44b1efa41acd4df3f56ee0e46f898135ecd1a054 fidoriel 49869342+fidoriel@users.noreply.github.com 2025-08-26T20:05:50+02:00 GitHub noreply@github.com 2025-08-26T20:05:50+02:00 mtmd : fix mtmd ios build (#15579) 44b1efa41acd4df3f56ee0e46f898135ecd1a054 a6a58d64785cb458ed9de52f391aa38142d38d64 Eve 139727413+netrunnereve@users.noreply.github.com 2025-08-26T15:42:49Z GitHub noreply@github.com 2025-08-26T15:42:49Z tests: add performance test for mul mat id (#15543) a6a58d64785cb458ed9de52f391aa38142d38d64 0373486dbc0dccbdcb3b5fdd65759d88cec06196 shalinib-ibm Shalini.Salomi.Bodapati@ibm.com 2025-08-26T21:05:25+05:30 GitHub noreply@github.com 2025-08-26T23:35:25+08:00 llamafile: PowerPC Sgemm Optimization (#15558) 0373486dbc0dccbdcb3b5fdd65759d88cec06196 62cef26ac5b6b7acb635d3dc963813b43952dc2b Georgi Gerganov ggerganov@gmail.com 2025-08-26T17:45:17+03:00 GitHub noreply@github.com 2025-08-26T17:45:17+03:00 graph : fix assert in memory-less build_attn (#15590) 62cef26ac5b6b7acb635d3dc963813b43952dc2b 8f5afa94c4f929da71f560db7c9f38ef6a783d95 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-26T16:12:29+02:00 GitHub noreply@github.com 2025-08-26T16:12:29+02:00 model-conversion : add qat-q4 quantization targets (#15588) 8f5afa94c4f929da71f560db7c9f38ef6a783d95 b3964c1e890ef8c947afb36a5124ce6fcb2136d4 Johannes Gäßler johannesg@5d6.de 2025-08-26T16:01:20+02:00 GitHub noreply@github.com 2025-08-26T16:01:20+02:00 CUDA: return -1 for nonexistent compiled arch (#15587) b3964c1e890ef8c947afb36a5124ce6fcb2136d4 79a546220c719e6a70627b243a478ab8d84dc9e1 Georgi Gerganov ggerganov@gmail.com 2025-08-26T14:22:14+03:00 GitHub noreply@github.com 2025-08-26T14:22:14+03:00 metal : optimize FA vec for large sequences and BS <= 8 (#15566) 79a546220c719e6a70627b243a478ab8d84dc9e1 85cc1ae998e4898d9fa992cb9b8620338cee97bf Xuan-Son Nguyen son@huggingface.co 2025-08-26T12:54:19+02:00 GitHub noreply@github.com 2025-08-26T12:54:19+02:00 mtmd : support Kimi VL model (#15458) 85cc1ae998e4898d9fa992cb9b8620338cee97bf 1d8d83deaa48d4a5491820d58ff1c0d8cf9d196c Georgi Gerganov ggerganov@gmail.com 2025-08-26T12:47:00+03:00 GitHub noreply@github.com 2025-08-26T12:47:00+03:00 context : print graph stats for memory-less contexts (#15586) 1d8d83deaa48d4a5491820d58ff1c0d8cf9d196c c4e9239064a564de7b94ee2b401ae907235a8fca Georgi Gerganov ggerganov@gmail.com 2025-08-26T12:46:15+03:00 GitHub noreply@github.com 2025-08-26T12:46:15+03:00 metal : improve `MUL_MAT_ID` (#15541) c4e9239064a564de7b94ee2b401ae907235a8fca 39842a7f73012eb42816ca4f26411782bd3da7c5 tc-mb 157115220+tc-mb@users.noreply.github.com 2025-08-26T16:05:55+08:00 GitHub noreply@github.com 2025-08-26T10:05:55+02:00 model : support MiniCPM-V 4.5 (#15575) 0115cfb7c1dec0753aecb34f3af4338bf9de8eeb 3a617cb18121a85eed17ac5585788810c5d9c1e0 0d5a470223fc90b6b6807921d68011ff06ae7f9e Yunzhe Jia yunzhe@calculet.tech 2025-08-26T15:19:56+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-08-26T15:19:56+08:00 Merge branch 'master' into dev 39842a7f73012eb42816ca4f26411782bd3da7c5 0fd90db5858e325358a5fbdaa4e327ee2a79d0d4 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-26T09:08:08+02:00 GitHub noreply@github.com 2025-08-26T09:08:08+02:00 gguf-py : remove erroneous FFN_GATE entry (#15583) 0fd90db5858e325358a5fbdaa4e327ee2a79d0d4 4c37636b3ea96f2574eeb7668b93fcc0e64b05dd Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-26T08:51:43+02:00 GitHub noreply@github.com 2025-08-26T09:51:43+03:00 metal : remove contiguous assertion for src0 in IM2COL (#15577) 4c37636b3ea96f2574eeb7668b93fcc0e64b05dd 34bdbbd7c2b70b848718e95bc48010f6aecd2816 Yoshi_likes_e4 104140648+pt13762104@users.noreply.github.com 2025-08-26T13:15:33+07:00 GitHub noreply@github.com 2025-08-26T08:15:33+02:00 Add a warning for special devices (#15563) 34bdbbd7c2b70b848718e95bc48010f6aecd2816 74f52f77f28a5ad6d6075231afcb8d1ad763ca32 Jeff Bolz jbolz@nvidia.com 2025-08-25T23:42:44-05:00 GitHub noreply@github.com 2025-08-26T06:42:44+02:00 vulkan: Remove splitting for mul_mat_id (#15568) 74f52f77f28a5ad6d6075231afcb8d1ad763ca32 f7207b0415986dd7f48447149da7de3a82338276 Qeeweew 68716978+Qeeweew@users.noreply.github.com 2025-08-26T05:21:22+08:00 GitHub noreply@github.com 2025-08-25T23:21:22+02:00 CUDA: Accelerate MXFP4 table lookup using `__byte_perm` (#15451) f7207b0415986dd7f48447149da7de3a82338276 4d917cd4f64cc37744e76d084659475819fb0728 lhez lih@qti.qualcomm.com 2025-08-25T14:18:09-07:00 GitHub noreply@github.com 2025-08-25T14:18:09-07:00 opencl: fix support ops condition for `rms_norm` (#15560) 4d917cd4f64cc37744e76d084659475819fb0728 886b97a5d693550c2da470c091d9d27bf38398f8 Ruben Ortlam picard12@live.de 2025-08-25T17:56:59+02:00 GitHub noreply@github.com 2025-08-25T17:56:59+02:00 vulkan: fix min subgroup 16 condition for mmid subgroup optimization (#15565) 886b97a5d693550c2da470c091d9d27bf38398f8 111f8d06f0b9169059779a35f655b343242ccbb6 Jeff Bolz jbolz@nvidia.com 2025-08-25T10:47:16-05:00 GitHub noreply@github.com 2025-08-25T10:47:16-05:00 tests: Generate unique input values for count_equal (#15487) 111f8d06f0b9169059779a35f655b343242ccbb6 5eff6ec9b1220b599a43b594b1110487ab6aca08 Ihar Hrachyshka ihar.hrachyshka@gmail.com 2025-08-25T11:27:34-04:00 GitHub noreply@github.com 2025-08-25T18:27:34+03:00 metal: fix regression when no metal devices are present (#15531) 5eff6ec9b1220b599a43b594b1110487ab6aca08 dfd9b5f6c7586c88588f06a644c131bec071a0a1 Johannes Gäßler johannesg@5d6.de 2025-08-25T17:23:40+02:00 GitHub noreply@github.com 2025-08-25T17:23:40+02:00 CUDA: MoE helper in device code, better tile sizes (#15525) dfd9b5f6c7586c88588f06a644c131bec071a0a1 5a6bc6b1a6cb665a944426c2055794950e524bf5 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-25T15:00:43+02:00 GitHub noreply@github.com 2025-08-25T15:00:43+02:00 model-conversion : set pooling type to none in logits.cpp (#15564) 5a6bc6b1a6cb665a944426c2055794950e524bf5 6b64f74b55628e4193f4fb00313f07dbd8556528 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-25T14:25:25+02:00 GitHub noreply@github.com 2025-08-25T14:25:25+02:00 model-conversion : add model card template for embeddings [no ci] (#15557) 6b64f74b55628e4193f4fb00313f07dbd8556528 0d5a470223fc90b6b6807921d68011ff06ae7f9e Georgi Gerganov ggerganov@gmail.com 2025-08-25T13:56:43+03:00 GitHub noreply@github.com 2025-08-25T13:56:43+03:00 batched-bench : fix unified KV cache handling + pp timing (#15562) 0d5a470223fc90b6b6807921d68011ff06ae7f9e b0ba31f525a2ad7fb539660be0c8f088c9869f6a Weizhao Ouyang o451686892@gmail.com 2025-08-25T17:15:06+08:00 GitHub noreply@github.com 2025-08-25T11:15:06+02:00 origin/master, origin/HEAD, master convert : update Ernie 4.5 dense architecture name (#15555) b0ba31f525a2ad7fb539660be0c8f088c9869f6a 7da9fed0d6f1750a8783436f6f313b87e76e6378 Georgi Gerganov ggerganov@gmail.com 2025-08-25T10:14:48+03:00 GitHub noreply@github.com 2025-08-25T10:14:48+03:00 metal : add FA kernels for HS=40 (#15559) 7da9fed0d6f1750a8783436f6f313b87e76e6378 c247d06f38fc09059c9607a28aa44f5ff6be208d RunningLeon mnsheng@yeah.net 2025-08-25T14:32:16+08:00 GitHub noreply@github.com 2025-08-25T08:32:16+02:00 convert : support interns1-mini (#15412) c247d06f38fc09059c9607a28aa44f5ff6be208d 043fb27d3808766d8ea8195bbd12359727264402 Chenguang Li 757486878@qq.com 2025-08-25T10:32:21+08:00 GitHub noreply@github.com 2025-08-25T10:32:21+08:00 CANN: ROPE cache sin/cos repeat (#15501) 043fb27d3808766d8ea8195bbd12359727264402 b730706a49e576fb882dc34d9966345778b3ab0b Ruben Ortlam picard12@live.de 2025-08-24T19:36:36+02:00 GitHub noreply@github.com 2025-08-24T19:36:36+02:00 vulkan: apply MUL_MAT_ID subgroup optimization to non-coopmat devices (#15524) b730706a49e576fb882dc34d9966345778b3ab0b c9a24fb93208fbbd3da6d903eb75431bfa97e59e Georgi Gerganov ggerganov@gmail.com 2025-08-24T13:07:07+03:00 GitHub noreply@github.com 2025-08-24T13:07:07+03:00 kv-cache : support layer reuse (#15504) c9a24fb93208fbbd3da6d903eb75431bfa97e59e a9c6ffcbfacee092bfaaa400306fceda18199737 Jeff Bolz jbolz@nvidia.com 2025-08-24T04:24:25-05:00 GitHub noreply@github.com 2025-08-24T11:24:25+02:00 vulkan: Support FA with any multiple of 8 head sizes (#15537) a9c6ffcbfacee092bfaaa400306fceda18199737 e78cf0d4b1bdbbc2479f11d58ce0c8f51f755875 Ruben Ortlam picard12@live.de 2025-08-24T10:48:53+02:00 GitHub noreply@github.com 2025-08-24T10:48:53+02:00 vulkan: enable Conv2D for Apple after MoltenVK fixed the bug (#15526) e78cf0d4b1bdbbc2479f11d58ce0c8f51f755875 710dfc465a68f7443b87d9f792cffba00ed739fe Jeff Bolz jbolz@nvidia.com 2025-08-24T03:48:21-05:00 GitHub noreply@github.com 2025-08-24T10:48:21+02:00 vulkan: workaround MoltenVK compile failure in multi_add (#15506) 710dfc465a68f7443b87d9f792cffba00ed739fe 611f419cff11e4952228162a1c44cb35dff2274a Johannes Gäßler johannesg@5d6.de 2025-08-23T21:37:06+02:00 GitHub noreply@github.com 2025-08-23T21:37:06+02:00 CUDA: fix half2 -> half conversion for HIP (#15529) 611f419cff11e4952228162a1c44cb35dff2274a b1afcab804e3281867a5471fbd701e32eb32e512 Jeff Bolz jbolz@nvidia.com 2025-08-23T13:16:17-05:00 GitHub noreply@github.com 2025-08-23T13:16:17-05:00 vulkan: optimize rms_norm, and allow the work to spread across multiple SMs (#15281) b1afcab804e3281867a5471fbd701e32eb32e512 9ef536907de1b50c30e0369284898d30472a755a Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-08-23T15:21:52+02:00 GitHub noreply@github.com 2025-08-23T15:21:52+02:00 model : add support for Seed-OSS (#15490) 9ef536907de1b50c30e0369284898d30472a755a 21dc4ddaf21b8ed551d717e7606abd2cffbacdbf Johannes Gäßler johannesg@5d6.de 2025-08-23T12:58:58+02:00 GitHub noreply@github.com 2025-08-23T13:58:58+03:00 scripts: fix compare-llama-bench.py (#15521) 21dc4ddaf21b8ed551d717e7606abd2cffbacdbf 289bf4113ef5c02d8f5eb0cf2d86683d8b8bc4d9 LaffeyNyaa 112215776+LaffeyNyaa@users.noreply.github.com 2025-08-23T16:38:30+08:00 GitHub noreply@github.com 2025-08-23T10:38:30+02:00 chat : fix debug build assertion in trim function (#15520) 289bf4113ef5c02d8f5eb0cf2d86683d8b8bc4d9 b55f06e1aa67fb10e89f53e31bbccf37eb2678ea Jeff Bolz jbolz@nvidia.com 2025-08-23T02:33:36-05:00 GitHub noreply@github.com 2025-08-23T09:33:36+02:00 vulkan: Rewrite synchronization to allow some overlap between nodes (#15489) b55f06e1aa67fb10e89f53e31bbccf37eb2678ea 0a9b43e507a359ca392c037cf341f55137ad0b69 R0CKSTAR yeahdongcn@gmail.com 2025-08-23T14:58:57+08:00 GitHub noreply@github.com 2025-08-23T08:58:57+02:00 vulkan.Dockerfile: install vulkan SDK using tarball (#15282) 0a9b43e507a359ca392c037cf341f55137ad0b69 330c3d2d21b55bca5517db7d2eea2ea8f131df4a Acly aclysia@gmail.com 2025-08-23T08:35:21+02:00 GitHub noreply@github.com 2025-08-23T08:35:21+02:00 vulkan : support ggml_mean (#15393) 330c3d2d21b55bca5517db7d2eea2ea8f131df4a e92734d51bcb82cc35f0a6b5a14928f0036b2c90 Jeff Bolz jbolz@nvidia.com 2025-08-23T01:31:54-05:00 GitHub noreply@github.com 2025-08-23T08:31:54+02:00 vulkan: optimize mul_mat_id loading row ids into shared memory (#15427) e92734d51bcb82cc35f0a6b5a14928f0036b2c90 45363632cbd593537d541e81b600242e0b3d47fc Johannes Gäßler johannesg@5d6.de 2025-08-22T23:47:01+02:00 GitHub noreply@github.com 2025-08-22T23:47:01+02:00 test-opt: allow slight inprecision (#15503) 45363632cbd593537d541e81b600242e0b3d47fc 32732f2459a598606055f0403f0e4ec148d06d68 Reese Levine reeselevine1@gmail.com 2025-08-22T11:28:03-07:00 GitHub noreply@github.com 2025-08-22T11:28:03-07:00 ggml WebGPU: add support for quantization types (#15440) 32732f2459a598606055f0403f0e4ec148d06d68 92f7f0a53cf6484e16a8084ed90807c35a164809 Aldehir Rojas hello@alde.dev 2025-08-22T11:04:08-05:00 GitHub noreply@github.com 2025-08-22T11:04:08-05:00 model : gpt-oss add response_format support (#15494) 92f7f0a53cf6484e16a8084ed90807c35a164809 b1ab91821f980f8993423c3f2a82a0a0f60c09d2 rmatif rmatif@proton.me 2025-08-22T15:33:15+02:00 GitHub noreply@github.com 2025-08-22T15:33:15+02:00 ggml: add `conv3d` op (#15182) b1ab91821f980f8993423c3f2a82a0a0f60c09d2 9ebebef62fd0adf8685874f154e227ea87b7c6f4 Yavor Ivanov yavorgenadiev@gmail.com 2025-08-22T14:06:29+03:00 GitHub noreply@github.com 2025-08-22T13:06:29+02:00 cuda : add Pad Reflect 1D support (#14659) 9ebebef62fd0adf8685874f154e227ea87b7c6f4 ad5c975c2d0297124fad210776ef8eed6b90d578 Georgi Gerganov ggerganov@gmail.com 2025-08-22T12:22:13+03:00 GitHub noreply@github.com 2025-08-22T12:22:13+03:00 llama : remove KV cache defragmentation logic (#15473) ad5c975c2d0297124fad210776ef8eed6b90d578 4afb0a746f22abaa545b3ebdb76a400d7da3a713 Aaron Teo aaron.teo1@ibm.com 2025-08-22T16:11:04+08:00 GitHub noreply@github.com 2025-08-22T16:11:04+08:00 ggml-cpu: Support Q5_0 and Q5_1 on s390x (#15486) 4afb0a746f22abaa545b3ebdb76a400d7da3a713 e288693669cf9d0a71e2f2b8bd57305f06340257 65a 10104049+65a@users.noreply.github.com 2025-08-22T08:10:14Z GitHub noreply@github.com 2025-08-22T10:10:14+02:00 server : Support multimodal completion and embeddings prompts in JSON format (#15108) e288693669cf9d0a71e2f2b8bd57305f06340257 a0f98dd604d34826eb5ea2560d1e23fe726921df Tarek Dakhran tarek@liquid.ai 2025-08-22T09:29:08+02:00 GitHub noreply@github.com 2025-08-22T09:29:08+02:00 readme : model : mtdm : lfm2 improvements (#15476) a0f98dd604d34826eb5ea2560d1e23fe726921df 54a241f505d515d625767b993bfd573ecee306b9 Chenguang Li 757486878@qq.com 2025-08-22T14:12:07+08:00 GitHub noreply@github.com 2025-08-22T14:12:07+08:00 CANN: Optimize RMS_NORM using cache (#15419) 54a241f505d515d625767b993bfd573ecee306b9 cd36b5e5c7fed2a3ac671dd542d579ca40b48b54 Diego Devesa slarengh@gmail.com 2025-08-21T14:09:32-07:00 GitHub noreply@github.com 2025-08-21T23:09:32+02:00 sched : fix possible use of wrong ids tensor when offloading moe prompt processing (#15488) cd36b5e5c7fed2a3ac671dd542d579ca40b48b54 3f196be84b1376945737163e35a91e29e3e24d2f Georgi Gerganov ggerganov@gmail.com 2025-08-21T19:13:45+03:00 GitHub noreply@github.com 2025-08-21T19:13:45+03:00 llama : remove deprecated llama_kv_self API (#15472) 3f196be84b1376945737163e35a91e29e3e24d2f 97ae5961a4d9ff9c60f51bac304b97de18e75eaf Georgi Gerganov ggerganov@gmail.com 2025-08-21T18:44:45+03:00 GitHub noreply@github.com 2025-08-21T18:44:45+03:00 graph : remove build_attn_with_sinks overload (#15469) 97ae5961a4d9ff9c60f51bac304b97de18e75eaf 20c2dac8c6e05f2ad5295ddef1aebaf2d266090e Acly aclysia@gmail.com 2025-08-21T17:01:51+02:00 GitHub noreply@github.com 2025-08-21T17:01:51+02:00 vulkan : support conv_2d_dw with f16 weights (#15392) 20c2dac8c6e05f2ad5295ddef1aebaf2d266090e 96452a3fa426de83494fb0268a636aa1bfe557fe Dong Won Kim 63934649+ddwkim@users.noreply.github.com 2025-08-22T00:00:16+09:00 GitHub noreply@github.com 2025-08-21T17:00:16+02:00 vulkan: add exp operation (#15456) 96452a3fa426de83494fb0268a636aa1bfe557fe 9ad5e60dba38a6718366b7ac43e7d8e8abdc36c9 Jeff Bolz jbolz@nvidia.com 2025-08-21T09:55:00-05:00 GitHub noreply@github.com 2025-08-21T16:55:00+02:00 vulkan: Reuse conversion results in prealloc_y (#15410) 9ad5e60dba38a6718366b7ac43e7d8e8abdc36c9 715a6db02ccb16284837885f2c6fab05d8f7a6ee Jie Fu (傅杰) jiefu@tencent.com 2025-08-21T22:53:13+08:00 GitHub noreply@github.com 2025-08-21T16:53:13+02:00 examples : fix some typos in examples/model-conversion/README.md (#15477) 715a6db02ccb16284837885f2c6fab05d8f7a6ee ad294df03ff2dccd227c3fee653166f3d78b23a4 Georgi Gerganov ggerganov@gmail.com 2025-08-21T17:00:33+03:00 GitHub noreply@github.com 2025-08-21T17:00:33+03:00 kv-cache : drop the "unified" prefix (#15467) ad294df03ff2dccd227c3fee653166f3d78b23a4 029bb39eb1e7ae0e5df817ce97931abcd5fa52a9 Jie Fu (傅杰) jiefu@tencent.com 2025-08-21T21:42:34+08:00 GitHub noreply@github.com 2025-08-21T15:42:34+02:00 examples : install torch-cpu for model conversion tool/example (#15475) 029bb39eb1e7ae0e5df817ce97931abcd5fa52a9 30649cab657d87ac46692332a76e1b75d5d22e00 Ali Tariq alitariq4589@gmail.com 2025-08-21T17:52:16+05:00 GitHub noreply@github.com 2025-08-21T14:52:16+02:00 ci : enable RVV1.0 native build (#15386) 30649cab657d87ac46692332a76e1b75d5d22e00 2758fa10dab0556e6c3f130e664750fd6773dc7c Georgi Gerganov ggerganov@gmail.com 2025-08-21T13:42:55+03:00 GitHub noreply@github.com 2025-08-21T13:42:55+03:00 ci : continue file download with wget (#15471) 2758fa10dab0556e6c3f130e664750fd6773dc7c b108e429043ee5c9fc8fa4957a0a52c3e490d5c9 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-21T12:16:54+02:00 GitHub noreply@github.com 2025-08-21T12:16:54+02:00 examples : add model conversion tool/example (#15455) b108e429043ee5c9fc8fa4957a0a52c3e490d5c9 245be739df942861ddc52331b095b40f18e2a3f1 Michael Giba michaelgiba@gmail.com 2025-08-21T05:06:46-05:00 GitHub noreply@github.com 2025-08-21T12:06:46+02:00 ci : fix -Werror=return-type in clip.cpp so ci/run.sh can run without issue (#15221) 245be739df942861ddc52331b095b40f18e2a3f1 b2caf67db1208fd38a0570785c39f7370d906d8a Copilot 198982749+Copilot@users.noreply.github.com 2025-08-21T11:47:52+02:00 GitHub noreply@github.com 2025-08-21T11:47:52+02:00 ci : add copilot-instructions.md (#15286) b2caf67db1208fd38a0570785c39f7370d906d8a 2f3dbffb17ef782edfd50e5a130cec6e8a7e47f8 Julien Denize 40604584+juliendenize@users.noreply.github.com 2025-08-21T11:19:50+02:00 GitHub noreply@github.com 2025-08-21T11:19:50+02:00 convert : make Mistral community chat templates optional via parameter (#15420) 2f3dbffb17ef782edfd50e5a130cec6e8a7e47f8 945e1f12a6b586ebf82fa4fd7f347225e58174c5 Jie Fu (傅杰) jiefu@tencent.com 2025-08-21T16:54:34+08:00 GitHub noreply@github.com 2025-08-21T11:54:34+03:00 common : fix incorrect print of non-ascii characters in the logging (#15466) 945e1f12a6b586ebf82fa4fd7f347225e58174c5 1b0db8f6e08951969e2447c2d18bf638effb8f75 Xuan-Son Nguyen son@huggingface.co 2025-08-21T07:32:26+02:00 GitHub noreply@github.com 2025-08-21T08:32:26+03:00 ggml : fix condition of im2col on Metal backend (#15460) 1b0db8f6e08951969e2447c2d18bf638effb8f75 29f538ac630d6544406a0702476e36808a6bd1b3 stduhpf stephduh@live.fr 2025-08-21T07:19:22+02:00 GitHub noreply@github.com 2025-08-21T08:19:22+03:00 server : fix webui (#15462) 29f538ac630d6544406a0702476e36808a6bd1b3 8ad038c0fdc719ced9fbf921a02cbae9ad79287f Daniel Bevenius daniel.bevenius@gmail.com 2025-08-21T06:12:28+02:00 GitHub noreply@github.com 2025-08-21T06:12:28+02:00 examples : remove references to `make` in examples [no ci] (#15457) 8ad038c0fdc719ced9fbf921a02cbae9ad79287f 5682a3745f2b653dcb855d5766d8edc318fb3336 R0CKSTAR yeahdongcn@gmail.com 2025-08-21T11:06:05+08:00 GitHub noreply@github.com 2025-08-21T11:06:05+08:00 musa: add GGML_UNUSED_VARS (#15446) 5682a3745f2b653dcb855d5766d8edc318fb3336 1bc664a26a1d93a48baf2483a7ff95291ca8bcb8 Diego Devesa slarengh@gmail.com 2025-08-20T16:35:28-07:00 GitHub noreply@github.com 2025-08-21T01:35:28+02:00 sched : copy only the used experts when offloading prompt processing (#15346) 1bc664a26a1d93a48baf2483a7ff95291ca8bcb8 13aeb7aef284daed4ad07dc14b4b3e2a42b5ea97 teo TeoZosa@users.noreply.github.com 2025-08-21T07:10:08+09:00 GitHub noreply@github.com 2025-08-21T00:10:08+02:00 server: fix OpenAI API compatibility for usage statistics in chat streams (#15444) 13aeb7aef284daed4ad07dc14b4b3e2a42b5ea97 7a6e91ad26160dd6dfb33d29ac441617422f28e7 Johannes Gäßler johannesg@5d6.de 2025-08-20T23:14:14+02:00 GitHub noreply@github.com 2025-08-20T23:14:14+02:00 CUDA: refactor FA support/selection code (#15454) 7a6e91ad26160dd6dfb33d29ac441617422f28e7 fec9519802ae1567048abb126cdd5ea160a22d0f Johannes Gäßler johannesg@5d6.de 2025-08-20T16:58:49+02:00 GitHub noreply@github.com 2025-08-20T16:58:49+02:00 CUDA: replace GGML_CUDA_F16 with CUDA arch checks (#15433) fec9519802ae1567048abb126cdd5ea160a22d0f 657b8a77bd01854f99d37a47318fa24f2e7e298f Jeff Bolz jbolz@nvidia.com 2025-08-20T09:33:14-05:00 GitHub noreply@github.com 2025-08-20T16:33:14+02:00 vulkan: shorten pipeline name strings (#15431) 657b8a77bd01854f99d37a47318fa24f2e7e298f ec5ab1a36c11dd3efcf4ec8d1ac89a13a8117bc3 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-20T14:26:01+02:00 GitHub noreply@github.com 2025-08-20T14:26:01+02:00 chat: handle gpt-oss return/end token inconsistency (#15421) ec5ab1a36c11dd3efcf4ec8d1ac89a13a8117bc3 1a99c2d948209d9ea5eac8b6dc0a297107244540 Jie Fu (傅杰) fujie_email@sina.com 2025-08-20T18:33:30+08:00 GitHub noreply@github.com 2025-08-20T13:33:30+03:00 common : fix context shift help message (#15448) 1a99c2d948209d9ea5eac8b6dc0a297107244540 37f10f955f70e0158d50343d0b9a3f92d194daae xiaobing318 71554036+xiaobing318@users.noreply.github.com 2025-08-20T18:32:05+08:00 GitHub noreply@github.com 2025-08-20T13:32:05+03:00 cmake : fix target include directories (#15450) 37f10f955f70e0158d50343d0b9a3f92d194daae 2f37014073f4c6ddc8f241c927db87337c71aa52 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-20T12:31:16+02:00 GitHub noreply@github.com 2025-08-20T13:31:16+03:00 make : remove make in favor of CMake (#15449) 2f37014073f4c6ddc8f241c927db87337c71aa52 a094f381432d92c4bf92d2d6167284316ba73a62 Georgi Gerganov ggerganov@gmail.com 2025-08-20T13:30:46+03:00 GitHub noreply@github.com 2025-08-20T13:30:46+03:00 lookahead : add sample command to readme (#15447) a094f381432d92c4bf92d2d6167284316ba73a62 fb22dd07a639e81c7415e30b146f545f1a2f2caf R0CKSTAR yeahdongcn@gmail.com 2025-08-20T10:17:37+08:00 GitHub noreply@github.com 2025-08-20T10:17:37+08:00 musa: fix build warnings (#15258) fb22dd07a639e81c7415e30b146f545f1a2f2caf 9ef6b0b835450ee10cd7be934ad8aef681dc1f43 lhez lih@qti.qualcomm.com 2025-08-20T02:25:51+08:00 GitHub noreply@github.com 2025-08-19T11:25:51-07:00 opencl: mark `argsort` unsupported if cols exceed workgroup limit (#15375) 9ef6b0b835450ee10cd7be934ad8aef681dc1f43 1e19f5d462b6df490a13103ca555d851f47e5fa5 Georgi Gerganov ggerganov@gmail.com 2025-08-19T19:58:28+03:00 GitHub noreply@github.com 2025-08-19T19:58:28+03:00 model : add gpt-oss type strings (#15424) 1e19f5d462b6df490a13103ca555d851f47e5fa5 d2fcd91cf96b46f4485ce46b4e3a32bf0df37715 Gian-Carlo Pascutto gcp@sjeng.org 2025-08-19T18:58:14+02:00 GitHub noreply@github.com 2025-08-19T19:58:14+03:00 common : Add top-nsigma sampler to help globally (#15428) d2fcd91cf96b46f4485ce46b4e3a32bf0df37715 a6d3cfe7fa6ea1fb0e1ba8243b731db22ddc0b49 Georgi Gerganov ggerganov@gmail.com 2025-08-19T16:46:37+03:00 GitHub noreply@github.com 2025-08-19T16:46:37+03:00 server : disable context shift by default (#15416) a6d3cfe7fa6ea1fb0e1ba8243b731db22ddc0b49 67f09a3a27db443f9870aac87e163dba0d08131e SHUAI YANG shuaiyang047@163.com 2025-08-19T21:28:22+08:00 GitHub noreply@github.com 2025-08-19T21:28:22+08:00 CANN: optimize rope operator (#15335) 67f09a3a27db443f9870aac87e163dba0d08131e 6424594c56f4dbd0573455d89a0d89a0ac093d13 R0CKSTAR yeahdongcn@gmail.com 2025-08-19T18:33:47+08:00 GitHub noreply@github.com 2025-08-19T12:33:47+02:00 musa: handle __hgt2_mask, available starting from MUSA SDK rc4.3.0 (#15413) 6424594c56f4dbd0573455d89a0d89a0ac093d13 e9288e886970884f288533cd597b3798995b4099 Marvin Gießing marvin.giessing@gmail.com 2025-08-19T10:54:31+02:00 GitHub noreply@github.com 2025-08-19T11:54:31+03:00 ggml-cpu: add mxfp4 VSX intrinsics for Power9+ (ppc64le) hardware (#15385) e9288e886970884f288533cd597b3798995b4099 9d262f4bad0d37838100133537aaf0a83835ed12 Xuan-Son Nguyen son@huggingface.co 2025-08-19T10:29:36+02:00 GitHub noreply@github.com 2025-08-19T10:29:36+02:00 chat : clarify the meaning of reasoning_format (#15408) 9d262f4bad0d37838100133537aaf0a83835ed12 f0d3c7405c323784a60f14ddddfbac3f7404d417 Georgi Gerganov ggerganov@gmail.com 2025-08-19T08:45:26+03:00 GitHub noreply@github.com 2025-08-19T08:45:26+03:00 server : remove swa_full warning (#15399) f0d3c7405c323784a60f14ddddfbac3f7404d417 f08c4c0d8d0cb6caaf8b7ad316039232b9fa059c Georgi Gerganov ggerganov@gmail.com 2025-08-19T08:45:12+03:00 GitHub noreply@github.com 2025-08-19T08:45:12+03:00 batched-bench : use rand tokens (#15398) f08c4c0d8d0cb6caaf8b7ad316039232b9fa059c 6d7f1117e3e3285d0c5c11b5ebb0439e27920082 Xuan-Son Nguyen son@huggingface.co 2025-08-18T22:53:52+02:00 GitHub noreply@github.com 2025-08-18T22:53:52+02:00 mtmd : clean up clip_n_output_tokens (#15391) 6d7f1117e3e3285d0c5c11b5ebb0439e27920082 60212f1ead2dce9bf1ac69633a7069258ae604d8 Georgi Gerganov ggerganov@gmail.com 2025-08-18T22:02:50+03:00 Georgi Gerganov ggerganov@gmail.com 2025-08-18T22:06:44+03:00 codeowners : remove mmv.* 60212f1ead2dce9bf1ac69633a7069258ae604d8 f0c541d315e97b297b3421c52ebde53340ee66b3 Georgi Gerganov ggerganov@gmail.com 2025-08-18T22:02:11+03:00 Georgi Gerganov ggerganov@gmail.com 2025-08-18T22:06:44+03:00 sync : ggml f0c541d315e97b297b3421c52ebde53340ee66b3 baa9255a45105d2d3b4ec432af13b7a6eda3ff35 Georgi Gerganov ggerganov@gmail.com 2025-08-18T20:35:47+03:00 Georgi Gerganov ggerganov@gmail.com 2025-08-18T22:06:44+03:00 scripts : update sync scripts baa9255a45105d2d3b4ec432af13b7a6eda3ff35 3007baf201e7ffcda17dbdb0335997fa50a6595b Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-18T19:30:17+02:00 GitHub noreply@github.com 2025-08-18T19:30:17+02:00 llama : merge conts and reshapes and remove unnecessary cont (#15380) 3007baf201e7ffcda17dbdb0335997fa50a6595b d1d82416006e7ff41780cb0e9b5f28d30a267497 Georgi Gerganov ggerganov@gmail.com 2025-08-18T18:11:44+03:00 GitHub noreply@github.com 2025-08-18T18:11:44+03:00 readme : update hot topics (#15397) d1d82416006e7ff41780cb0e9b5f28d30a267497 618575c5825d7d4f170e686e772178d2aae148ae davidef davidef1986@gmail.com 2025-08-18T16:51:42+02:00 GitHub noreply@github.com 2025-08-18T17:51:42+03:00 server : fix incoming tasks not process in order (#15395) 618575c5825d7d4f170e686e772178d2aae148ae f44f7931729022c57319a0124931120a169e0da9 Dobri Danchev 12420863+danchev@users.noreply.github.com 2025-08-18T05:50:48-05:00 GitHub noreply@github.com 2025-08-18T12:50:48+02:00 Fix broken build: require updated pip to support --break-system-packages (#15357) f44f7931729022c57319a0124931120a169e0da9 ae532eac2c1df1d8edc3d2719145895b966de1bf compilade git@compilade.net 2025-08-18T03:23:56-04:00 GitHub noreply@github.com 2025-08-18T09:23:56+02:00 ggml-quants : fix make_qp_quants NANs and IQ1 assertion errors (#15379) ae532eac2c1df1d8edc3d2719145895b966de1bf e5155e698645242d4f019267ecc40ea9bad81b09 Jeff Bolz jbolz@nvidia.com 2025-08-18T00:56:29-05:00 GitHub noreply@github.com 2025-08-18T07:56:29+02:00 vulkan: disable spirv-opt for bfloat16 shaders (#15352) e5155e698645242d4f019267ecc40ea9bad81b09 21c17b5befc5f6be5992bc87fc1ba99d388561df Oleksandr Kuvshynov 661042+okuvshynov@users.noreply.github.com 2025-08-17T18:28:58-04:00 GitHub noreply@github.com 2025-08-18T00:28:58+02:00 server : export max observed n_past value (#15361) 21c17b5befc5f6be5992bc87fc1ba99d388561df 19f4decae0ead52debe56095ba8d693b4f14e4df Jeff Bolz jbolz@nvidia.com 2025-08-17T11:08:57-05:00 GitHub noreply@github.com 2025-08-17T18:08:57+02:00 vulkan: Use larger workgroups for mul_mat_vec when M is small (#15355) 19f4decae0ead52debe56095ba8d693b4f14e4df 4d196981d4db79e0105b939eaa7ecd40385b721c Dong Won Kim 63934649+ddwkim@users.noreply.github.com 2025-08-17T23:03:09+09:00 GitHub noreply@github.com 2025-08-17T16:03:09+02:00 vulkan: support sqrt (#15370) 4d196981d4db79e0105b939eaa7ecd40385b721c b143fbc87af0324aa49e16cd91faf3ba8bb22231 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-17T14:47:42+02:00 GitHub noreply@github.com 2025-08-17T14:47:42+02:00 convert : force patch_embd weights to F16 or F32 to avoid broken GGUFs (#15367) b143fbc87af0324aa49e16cd91faf3ba8bb22231 de5627910df74298c998e6bb36ee3217375a5719 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-17T13:30:23+02:00 GitHub noreply@github.com 2025-08-17T13:30:23+02:00 ci : fix hang in windows-hip build/release (#15365) de5627910df74298c998e6bb36ee3217375a5719 65349f26f2299e06477ec8e85e46243046801358 Jeff Bolz jbolz@nvidia.com 2025-08-17T03:41:45-05:00 GitHub noreply@github.com 2025-08-17T10:41:45+02:00 vulkan: Optimize argsort (#15354) 65349f26f2299e06477ec8e85e46243046801358 1fe00296f587dfca0957e006d146f5875b61e43d Tarek Dakhran t.dakhran@gmail.com 2025-08-16T23:33:54+02:00 GitHub noreply@github.com 2025-08-16T23:33:54+02:00 model : support vision LiquidAI LFM2-VL family (#15347) 1fe00296f587dfca0957e006d146f5875b61e43d de2192794f4e8e04f2e8167ef2424905145e88fc Jeff Bolz jbolz@nvidia.com 2025-08-16T11:48:22-05:00 GitHub noreply@github.com 2025-08-16T11:48:22-05:00 vulkan: fuse adds (#15252) de2192794f4e8e04f2e8167ef2424905145e88fc 2e2b22ba6607414a5d619ac6d2f034b5b02214e5 Jeff Bolz jbolz@nvidia.com 2025-08-16T04:18:31-05:00 GitHub noreply@github.com 2025-08-16T11:18:31+02:00 vulkan: Support mul_mat_id with f32 accumulators (#15337) 2e2b22ba6607414a5d619ac6d2f034b5b02214e5 912ff8c119f01ae029543c7fdf7a84f91a0437a3 Jeff Bolz jbolz@nvidia.com 2025-08-16T03:58:38-05:00 GitHub noreply@github.com 2025-08-16T10:58:38+02:00 vulkan: Add missing bounds checking to scalar/coopmat1 mul_mat_id (#15334) 912ff8c119f01ae029543c7fdf7a84f91a0437a3 5e6229a8409ac786e62cb133d09f1679a9aec13e rmatif kingrealriadh@gmail.com 2025-08-16T10:05:55+02:00 GitHub noreply@github.com 2025-08-16T01:05:55-07:00 OpenCL: add initial FA support (#14987) 5e6229a8409ac786e62cb133d09f1679a9aec13e e2c1bfff5305c661ac53e9d57cb732ff626a2242 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-15T19:50:52+02:00 GitHub noreply@github.com 2025-08-15T19:50:52+02:00 common : fix double bos, use common_chat_templates for add_bos and add_eos (#15326) e2c1bfff5305c661ac53e9d57cb732ff626a2242 5edf1592fdb9131d01321aeef4241c6a34969e27 lhez lih@qti.qualcomm.com 2025-08-16T00:52:14+08:00 GitHub noreply@github.com 2025-08-15T09:52:14-07:00 opencl: add initial mxfp4 support via mv (#15270) 5edf1592fdb9131d01321aeef4241c6a34969e27 db3010bd23980bad5f5d93ec3e6757ec531a413b Georgi Gerganov ggerganov@gmail.com 2025-08-15T17:16:36+03:00 GitHub noreply@github.com 2025-08-15T16:16:36+02:00 vulkan : fix out-of-bounds access in argmax kernel (#15342) db3010bd23980bad5f5d93ec3e6757ec531a413b ff27f80a74bbe5303acd511a6781a1de6d619b3c Georgi Gerganov ggerganov@gmail.com 2025-08-15T16:28:28+03:00 GitHub noreply@github.com 2025-08-15T15:28:28+02:00 vulkan : fix compile warnings on macos (#15340) ff27f80a74bbe5303acd511a6781a1de6d619b3c d3248d9b6557c75d59954c594bb53cf517591e91 Aaron Teo aaron.teo1@ibm.com 2025-08-15T21:11:22+08:00 GitHub noreply@github.com 2025-08-15T21:11:22+08:00 ggml: initial IBM zDNN backend (#14975) d3248d9b6557c75d59954c594bb53cf517591e91 7aeee88cfe9c0434eac722b0f7c21404f48758a5 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-15T14:02:39+02:00 GitHub noreply@github.com 2025-08-15T14:02:39+02:00 ci : fix ios-xcode-build (#15324) 7aeee88cfe9c0434eac722b0f7c21404f48758a5 b07791aa1d4831a08ad54ca19aa206c0c5c0f34a Diego Devesa slarengh@gmail.com 2025-08-15T03:27:02-07:00 GitHub noreply@github.com 2025-08-15T12:27:02+02:00 ci : move ccache action to ggml-org fork (#15328) b07791aa1d4831a08ad54ca19aa206c0c5c0f34a 4227c9be4268ac844921b90f31595f81236bd317 Johannes Gäßler johannesg@5d6.de 2025-08-15T11:23:17+02:00 GitHub noreply@github.com 2025-08-15T11:23:17+02:00 test-opt: fix backend support check (#15317) 4227c9be4268ac844921b90f31595f81236bd317 df36bce667bf14f8e538645547754386f9516326 Johannes Gäßler johannesg@5d6.de 2025-08-14T23:21:24+02:00 GitHub noreply@github.com 2025-08-14T23:21:24+02:00 CUDA: fix negative KV_max values in FA (#15321) df36bce667bf14f8e538645547754386f9516326 f75b8306472811ecc4e4457d5573a09201f02183 Georgi Gerganov ggerganov@gmail.com 2025-08-14T22:10:51+03:00 GitHub noreply@github.com 2025-08-14T22:10:51+03:00 eval-callback : stop on first NaN (#15320) f75b8306472811ecc4e4457d5573a09201f02183 7a0de960452f9a57de7f1d167e57b6f3ee5ac1b6 Diego Devesa slarengh@gmail.com 2025-08-14T10:28:29-07:00 GitHub noreply@github.com 2025-08-14T10:28:29-07:00 chat : include kwargs in template example (#15309) 7a0de960452f9a57de7f1d167e57b6f3ee5ac1b6 e4e915912cfd2ee15c5a4a0074813232134892f6 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-14T17:56:26+02:00 GitHub noreply@github.com 2025-08-14T17:56:26+02:00 llama : add 18-layer model type for Gemma 3-270m (#15319) e4e915912cfd2ee15c5a4a0074813232134892f6 5ba36f61033d2819be27e2abb70e9a3bd20c0fde simevo github@simevo.com 2025-08-14T17:45:27+02:00 GitHub noreply@github.com 2025-08-14T18:45:27+03:00 devops : fix compile bug when the BASE_CUDA_DEV_CONTAINER is based on Ubuntu 24.04 (#15005) 5ba36f61033d2819be27e2abb70e9a3bd20c0fde b204a5a234f4cf0b3f449476da639a5510c6d157 uvos carl@uvos.xyz 2025-08-14T16:23:56+02:00 GitHub noreply@github.com 2025-08-14T16:23:56+02:00 HIP: Cleanup hipification header (#15285) b204a5a234f4cf0b3f449476da639a5510c6d157 646944cfa8961afd914dd6637739b3cda9a72e11 Aldehir Rojas hello@alde.dev 2025-08-14T09:23:11-05:00 GitHub noreply@github.com 2025-08-14T17:23:11+03:00 gpt-oss: implement harmony parsing (#15181) 646944cfa8961afd914dd6637739b3cda9a72e11 1a01899b612ae8f99a174ad076207090e08d4d7b Christian Kastner ckk@kvr.at 2025-08-14T16:22:58+02:00 GitHub noreply@github.com 2025-08-14T16:22:58+02:00 docker : Enable GGML_CPU_ALL_VARIANTS for ARM (#15267) 1a01899b612ae8f99a174ad076207090e08d4d7b 863d341eeb81db104902b14b6f9413daa515e957 Georgi Gerganov ggerganov@gmail.com 2025-08-14T17:16:03+03:00 GitHub noreply@github.com 2025-08-14T17:16:03+03:00 readme : update hot topics (#15315) 863d341eeb81db104902b14b6f9413daa515e957 d32e03f4495d3efa1c5126f53b449f1d429c5664 Jeff Bolz jbolz@nvidia.com 2025-08-14T08:38:10-05:00 GitHub noreply@github.com 2025-08-14T08:38:10-05:00 vulkan: perf_logger improvements (#15246) d32e03f4495d3efa1c5126f53b449f1d429c5664 3973163bff40f7f5161b0f08a0011729e2b0406a Georgi Gerganov ggerganov@gmail.com 2025-08-14T14:59:50+03:00 GitHub noreply@github.com 2025-08-14T14:59:50+03:00 server : add SWA checkpoints (#15293) 3973163bff40f7f5161b0f08a0011729e2b0406a 5ade3000bd6040bf6878eafd6c77168552f73c47 Georgi Gerganov ggerganov@gmail.com 2025-08-14T14:19:23+03:00 Georgi Gerganov ggerganov@gmail.com 2025-08-14T14:59:27+03:00 sync : ggml 5ade3000bd6040bf6878eafd6c77168552f73c47 8b2483730f90d6f23e2b188a54a210498bba937f Jason Ni jason.ni.py@gmail.com 2025-08-14T19:17:51+08:00 Georgi Gerganov ggerganov@gmail.com 2025-08-14T14:59:27+03:00 ggml: fix ggml_conv_1d_dw bug (ggml/1323) 8b2483730f90d6f23e2b188a54a210498bba937f 810b9fc8b99dd55517a3e94c6dee29748d482559 Georgi Gerganov ggerganov@gmail.com 2025-08-14T13:41:03+03:00 Georgi Gerganov ggerganov@gmail.com 2025-08-14T14:59:27+03:00 tests : remove unused includes (ggml/0) 810b9fc8b99dd55517a3e94c6dee29748d482559 4ebd0c125b24a0d7a78b0ffc1d9567530ed8f0c4 kallewoof karljohan-alm@garage.co.jp 2025-08-14T20:03:30+09:00 GitHub noreply@github.com 2025-08-14T14:03:30+03:00 perplexity : provide a helpful hint for has_cpl case in split_equal error. (#15304) 4ebd0c125b24a0d7a78b0ffc1d9567530ed8f0c4 5cdb27e0917479d2d742cea7beee089574bb09fa Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-14T12:22:07+02:00 GitHub noreply@github.com 2025-08-14T13:22:07+03:00 cuda : fix GGML_CUDA_GRAPHS=OFF (#15300) 5cdb27e0917479d2d742cea7beee089574bb09fa 3ea913f1ce9567289aedd866a569dbab8fb8e419 Jonathan Graehl 99024+graehl@users.noreply.github.com 2025-08-14T03:03:57-07:00 GitHub noreply@github.com 2025-08-14T12:03:57+02:00 finetune: SGD optimizer, more CLI args (#13873) 3ea913f1ce9567289aedd866a569dbab8fb8e419 29c8fbe4e05fd23c44950d0958299e25fbeabc5c kallewoof karljohan-alm@garage.co.jp 2025-08-14T15:16:32+09:00 GitHub noreply@github.com 2025-08-14T09:16:32+03:00 perplexity: give more information about constraints on failure (#15303) 29c8fbe4e05fd23c44950d0958299e25fbeabc5c 1adc9812bd33dc85489bf093528d61c22917d54f uvos carl@uvos.xyz 2025-08-13T20:44:30+02:00 GitHub noreply@github.com 2025-08-13T20:44:30+02:00 HIP: bump requirement to rocm 6.1 (#15296) 1adc9812bd33dc85489bf093528d61c22917d54f b3e16665e14032d1276f9d0263acef8321b6f518 Bas Nijholt basnijholt@gmail.com 2025-08-13T11:21:31-07:00 GitHub noreply@github.com 2025-08-13T11:21:31-07:00 fix(nix): remove non-functional llama-cpp cachix cache from flake.nix (#15295) b3e16665e14032d1276f9d0263acef8321b6f518 c24f4e26883a91219af5acfaf1471ca7ef582683 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-13T15:43:00+02:00 GitHub noreply@github.com 2025-08-13T15:43:00+02:00 server : enable -td and -tbd parameters (#15172) c24f4e26883a91219af5acfaf1471ca7ef582683 d8914fc47e8a69b28c670325cb1c8ce33e3a2960 Judd 4046440+foldl@users.noreply.github.com 2025-08-13T18:45:15+08:00 GitHub noreply@github.com 2025-08-13T13:45:15+03:00 ggml : update `ggml_rope_multi` (#12665) d8914fc47e8a69b28c670325cb1c8ce33e3a2960 e885445bc1719d2e289a9b2e11714e0f994e68be Copilot 198982749+Copilot@users.noreply.github.com 2025-08-13T12:44:40+02:00 GitHub noreply@github.com 2025-08-13T12:44:40+02:00 common : add --override-tensor-draft, --cpu-moe-draft and --n-cpu-moe-draft parameters (#15191) e885445bc1719d2e289a9b2e11714e0f994e68be 648ebcdb739abfb5a9be14f4c5c0fd19ff268ef0 Aldehir Rojas hello@alde.dev 2025-08-13T05:28:21-05:00 GitHub noreply@github.com 2025-08-13T12:28:21+02:00 server : filter out harmony thought messages (#15278) 648ebcdb739abfb5a9be14f4c5c0fd19ff268ef0 07aa869a91837d95fcb5612c65a188763ac38647 Ali Tariq alitariq4589@gmail.com 2025-08-13T15:14:44+05:00 GitHub noreply@github.com 2025-08-13T13:14:44+03:00 ci : Added CI with RISC-V RVV1.0 Hardware (#14439) 07aa869a91837d95fcb5612c65a188763ac38647 00f35d509e5367bf19ccaf4b4adddc38db4811c6 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-13T11:30:45+02:00 GitHub noreply@github.com 2025-08-13T11:30:45+02:00 ci : add more python requirements to copilot-setup-steps (#15289) 00f35d509e5367bf19ccaf4b4adddc38db4811c6 6028bf74351d35a06bd98498624f8c2f029f7d1a Georgi Gerganov ggerganov@gmail.com 2025-08-13T11:09:39+03:00 GitHub noreply@github.com 2025-08-13T11:09:39+03:00 ggml : repack block_iq4_nlx8 (#14904) 6028bf74351d35a06bd98498624f8c2f029f7d1a bc5182272c373267352bc689e5fca276934bea2d Oliver Simons osimons@nvidia.com 2025-08-13T10:04:46+02:00 GitHub noreply@github.com 2025-08-13T10:04:46+02:00 CUDA: Optimize `reduce_rows_f32` kernel, leading up to 25x perf improvement on kernel-level and 10% perf increase for Gemma3n (#15132) bc5182272c373267352bc689e5fca276934bea2d e71d48e3265027351e44a8e198f933c98f242c2e Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-13T09:07:13+02:00 GitHub noreply@github.com 2025-08-13T09:07:13+02:00 ci : add copilot-setup-steps.yml (#15214) e71d48e3265027351e44a8e198f933c98f242c2e b0493156fa8622694f21f42460a84da3eded0bc0 Tak-RS snosk.t@gmail.com 2025-08-13T14:54:30+09:00 GitHub noreply@github.com 2025-08-13T08:54:30+03:00 ggml-rpc: chunk send()/recv() to avoid EINVAL for very large tensors over RPC (macOS & others) (#15188) 3a617cb18121a85eed17ac5585788810c5d9c1e0 c42293848dd9f03eba9a6c1b85a600b398f06541 Yunzhe Jia yunzhe@calculet.tech 2025-08-13T10:58:59+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-08-13T10:58:59+08:00 add calrt_decode b0493156fa8622694f21f42460a84da3eded0bc0 f4586ee5986d6f965becb37876d6f3666478a961 uvos carl@uvos.xyz 2025-08-12T22:15:12+02:00 GitHub noreply@github.com 2025-08-12T22:15:12+02:00 HIP: disable sync warp shuffel operators from clr amd_warp_sync_functions.h (#15273) f4586ee5986d6f965becb37876d6f3666478a961 60a76588106d22ccacd6b1a0d15d8545861d0a0d Romain Biessy romain.biessy@codeplay.com 2025-08-12T13:58:22+02:00 GitHub noreply@github.com 2025-08-12T13:58:22+02:00 sycl: Fix and disable more configurations of mul_mat (#15151) 60a76588106d22ccacd6b1a0d15d8545861d0a0d efe3a90996ca6e67ca90f337fc03ad551082c408 rmatif kingrealriadh@gmail.com 2025-08-12T11:42:41+02:00 GitHub noreply@github.com 2025-08-12T02:42:41-07:00 opencl: allow mixed f16/f32 `add` (#15140) efe3a90996ca6e67ca90f337fc03ad551082c408 bbd57b7eafb3b32e3f7f3a2175bce0b35abc7de8 Aman Gupta amangupta052@gmail.com 2025-08-12T17:21:45+08:00 GitHub noreply@github.com 2025-08-12T17:21:45+08:00 CUDA cmake: add `-lineinfo` for easier debug (#15260) bbd57b7eafb3b32e3f7f3a2175bce0b35abc7de8 25ff6f7659f6a5c47d6a73eada5813f0495331f0 Chenguang Li 757486878@qq.com 2025-08-12T16:12:13+08:00 GitHub noreply@github.com 2025-08-12T16:12:13+08:00 CANN: GGML_OP_CPY optimization (#15070) 25ff6f7659f6a5c47d6a73eada5813f0495331f0 be48528b068111304e4a0bb82c028558b5705f05 R0CKSTAR yeahdongcn@gmail.com 2025-08-12T10:02:51+08:00 GitHub noreply@github.com 2025-08-12T10:02:51+08:00 musa: fix failures in test-backend-ops for mul_mat_id op (#15236) be48528b068111304e4a0bb82c028558b5705f05 cf9e5648a7ae02f7728fefcbcfd2979d83c96e8f hipudding huafengchun@gmail.com 2025-08-11T22:50:31+08:00 GitHub noreply@github.com 2025-08-11T22:50:31+08:00 CANN: Add broadcast for softmax and FA (#15208) cf9e5648a7ae02f7728fefcbcfd2979d83c96e8f fba5c0d680b555cbac563fef57b33bc5c9621e08 rainred 107027757+gryffindor-rr@users.noreply.github.com 2025-08-11T22:12:12+08:00 GitHub noreply@github.com 2025-08-11T16:12:12+02:00 mtmd : Fix MinicpmV model converter and clip to avoid using hardcode. (#14750) fba5c0d680b555cbac563fef57b33bc5c9621e08 53d0a1265826f40c5dbc01d06aeab9e14fcbd69b Xuan-Son Nguyen son@huggingface.co 2025-08-11T15:31:35+02:00 GitHub noreply@github.com 2025-08-11T15:31:35+02:00 chat : hotfix gpt-oss jinja raising an exception (#15243) 53d0a1265826f40c5dbc01d06aeab9e14fcbd69b 27093afe78912494073eb043fec93a007e49653c Xuan-Son Nguyen son@huggingface.co 2025-08-11T14:48:41+02:00 GitHub noreply@github.com 2025-08-11T14:48:41+02:00 server : allow specifying reasoning_format in HTTP request (#15238) 27093afe78912494073eb043fec93a007e49653c 228f724d9ce6c56e8cec75bfdabab4dd013def7f Zagaj m.zagajewska@gmail.com 2025-08-11T14:27:54+02:00 GitHub noreply@github.com 2025-08-11T15:27:54+03:00 readme : update infra list (#15234) 228f724d9ce6c56e8cec75bfdabab4dd013def7f cd3069dfcbeee8e0e96cffb93b0ebb9e595e273a Georgi Gerganov ggerganov@gmail.com 2025-08-11T13:58:24+03:00 GitHub noreply@github.com 2025-08-11T13:58:24+03:00 kv-cache : fix seq_rm with seq_id == -1 (#15226) cd3069dfcbeee8e0e96cffb93b0ebb9e595e273a 50e81bdf5db563ab57a9a722b08f96fa8a76c927 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-11T11:21:19+02:00 GitHub noreply@github.com 2025-08-11T11:21:19+02:00 kv-cache : log (debug) all streams in find_slot (#15176) 50e81bdf5db563ab57a9a722b08f96fa8a76c927 1ebbaddff2a44b0599df659175d3274bd5bbeb81 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-11T11:15:44+02:00 GitHub noreply@github.com 2025-08-11T11:15:44+02:00 convert : fix merge conflicts (#15229) 1ebbaddff2a44b0599df659175d3274bd5bbeb81 a3a7874272e5a060079658eb5cca4617b7f99062 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-11T10:21:24+02:00 GitHub noreply@github.com 2025-08-11T11:21:24+03:00 perplexity : update comments/error msg to use decode [no ci] (#15227) a3a7874272e5a060079658eb5cca4617b7f99062 002cb1bb3345967fbe0fa7766c2d94c2da31ef45 Julien Denize 40604584+juliendenize@users.noreply.github.com 2025-08-11T10:07:49+02:00 GitHub noreply@github.com 2025-08-11T10:07:49+02:00 convert : improve Mistral models integration (#14737) 002cb1bb3345967fbe0fa7766c2d94c2da31ef45 79c1160b073b8148a404f3dd2584be1606dccc66 Charles Xu charles.xu@arm.com 2025-08-11T09:59:26+02:00 GitHub noreply@github.com 2025-08-11T09:59:26+02:00 kleidiai: fix unsigned overflow bug (#15150) 79c1160b073b8148a404f3dd2584be1606dccc66 34c9d765bf173c551398f1e7fa4595019bc53bab David Zhao 90013954+Your-Cheese@users.noreply.github.com 2025-08-09T13:29:43-05:00 GitHub noreply@github.com 2025-08-09T20:29:43+02:00 cuda: refactored ssm_scan and use CUB (#13291) 34c9d765bf173c551398f1e7fa4595019bc53bab e54d41befcc1575f4c898c5ff4ef43970cead75f Aman Gupta amangupta052@gmail.com 2025-08-09T20:00:24+08:00 GitHub noreply@github.com 2025-08-09T20:00:24+08:00 CUDA: add attention sinks for tile and wmma (#15178) e54d41befcc1575f4c898c5ff4ef43970cead75f 4850b52aedceeb70bb4fe49f2d7cd1df6ee98682 compilade git@compilade.net 2025-08-08T17:48:26-04:00 GitHub noreply@github.com 2025-08-08T17:48:26-04:00 gguf-py : add Numpy MXFP4 de/quantization support (#15111) 4850b52aedceeb70bb4fe49f2d7cd1df6ee98682 cd6983d56d2cce94ecb86bb114ae8379a609073c Johannes Gäßler johannesg@5d6.de 2025-08-08T23:04:36+02:00 GitHub noreply@github.com 2025-08-08T23:04:36+02:00 server-bench: external OAI servers, sqlite (#15179) cd6983d56d2cce94ecb86bb114ae8379a609073c 6c7e9a54406dbba5e53754a8f70a285414717b06 AN Long aisk@users.noreply.github.com 2025-08-08T21:37:22+09:00 GitHub noreply@github.com 2025-08-08T14:37:22+02:00 ggml : fix field name when new ggml_backend (#14944) 6c7e9a54406dbba5e53754a8f70a285414717b06 1425f587a82bc303469b5c32759a2746ba4e1e20 Olivier Chafik olivier.chafik@gmail.com 2025-08-08T10:45:18+01:00 GitHub noreply@github.com 2025-08-08T10:45:18+01:00 vendor: sync minja (#15161) 1425f587a82bc303469b5c32759a2746ba4e1e20 aaa3d07ae749b781d6135eaff23c7fa8a4ab404a Johannes Gäßler johannesg@5d6.de 2025-08-08T08:19:58+02:00 GitHub noreply@github.com 2025-08-08T08:19:58+02:00 CUDA: attention sinks for mma FlashAttention (#15157) aaa3d07ae749b781d6135eaff23c7fa8a4ab404a 50aa9389014bba2dd12234132aa6b8ca3601a17f lhez lih@qti.qualcomm.com 2025-08-08T13:47:03+09:00 GitHub noreply@github.com 2025-08-07T21:47:03-07:00 opencl: support sink in `soft_max` (attn sinks) (#15152) 50aa9389014bba2dd12234132aa6b8ca3601a17f c4f53563df4575196ea13f5ed669ea8ea659a6be Xuan-Son Nguyen son@huggingface.co 2025-08-07T23:26:03+02:00 GitHub noreply@github.com 2025-08-07T23:26:03+02:00 convert : support non-mxfp4 HF model (#15153) c4f53563df4575196ea13f5ed669ea8ea659a6be a0552c8beef74e843bb085c8ef0c63f9ed7a2b27 Jeff Bolz jbolz@nvidia.com 2025-08-07T15:44:20-05:00 GitHub noreply@github.com 2025-08-07T22:44:20+02:00 vulkan: support fattn sinks (#15126) a0552c8beef74e843bb085c8ef0c63f9ed7a2b27 99acbc9921b119aa7ed929eb5780a66a8f06e6d9 Jeff Bolz jbolz@nvidia.com 2025-08-07T15:07:11-05:00 GitHub noreply@github.com 2025-08-07T22:07:11+02:00 vulkan: Add env var to disable host visible vidmem (#15109) 99acbc9921b119aa7ed929eb5780a66a8f06e6d9 7ad67ba9fe2b909e271dd31b99c5fce3aba35899 RunningLeon mnsheng@yeah.net 2025-08-08T00:20:40+08:00 GitHub noreply@github.com 2025-08-07T18:20:40+02:00 llama : Support intern-s1 (#14875) 7ad67ba9fe2b909e271dd31b99c5fce3aba35899 9a96389544a08fd829fccda28142ce2066017fde uvos carl@uvos.xyz 2025-08-07T16:44:14+02:00 GitHub noreply@github.com 2025-08-07T16:44:14+02:00 HIP: add cmake option to enable compiler output of kernel resource usage metrics (#15103) 9a96389544a08fd829fccda28142ce2066017fde 1d72c841888b9450916bdd5a9b3274da380f5b36 Christian Kastner ckk@kvr.at 2025-08-07T13:45:41+02:00 GitHub noreply@github.com 2025-08-07T13:45:41+02:00 ggml: Skip backend library linking code when GGML_BACKEND_DL=ON (#15094) 1d72c841888b9450916bdd5a9b3274da380f5b36 20638e4f16fcc21f836c7556e83bbf532bb5a0f0 Johannes Gäßler johannesg@5d6.de 2025-08-07T10:53:21+02:00 GitHub noreply@github.com 2025-08-07T10:53:21+02:00 CUDA: GEMM for FP32/FP16/BF16 and ne11 <= 16 (#15131) 20638e4f16fcc21f836c7556e83bbf532bb5a0f0 36d3f00e142696f708ab297b9f8f1c825594712d Johannes Gäßler johannesg@5d6.de 2025-08-07T08:50:30+02:00 GitHub noreply@github.com 2025-08-07T08:50:30+02:00 scripts: fix crash when --tool is not set (#15133) 36d3f00e142696f708ab297b9f8f1c825594712d 5fd160bbd9d70b94b5b11b0001fd7f477005e4a0 Daniel Bevenius daniel.bevenius@gmail.com 2025-08-07T05:31:48+02:00 GitHub noreply@github.com 2025-08-07T05:31:48+02:00 requirements : fix PyTorch uint64 compatibility (#15134) 5fd160bbd9d70b94b5b11b0001fd7f477005e4a0 756cfea82608911bbfcbf45164b8fdaddbafaa31 Reese Levine reeselevine1@gmail.com 2025-08-06T15:14:40-07:00 GitHub noreply@github.com 2025-08-06T15:14:40-07:00 ggml: Add basic SET_ROWS support in WebGPU (#15137) 756cfea82608911bbfcbf45164b8fdaddbafaa31 e725a1a982ca870404a9c4935df52466327bbd02 rmatif kingrealriadh@gmail.com 2025-08-06T23:17:51+02:00 GitHub noreply@github.com 2025-08-06T14:17:51-07:00 fix profiling crash (#15072) e725a1a982ca870404a9c4935df52466327bbd02 3db4da56a5a00c4b47bbb3b18f85fb4473662adb lhez lih@qti.qualcomm.com 2025-08-07T04:12:17+09:00 GitHub noreply@github.com 2025-08-06T12:12:17-07:00 opencl: add `swiglu_oai` and `add_id` (#15121) 3db4da56a5a00c4b47bbb3b18f85fb4473662adb 476aa3fd5779b32d06cd84338f777da82341195c Sachin Desai smdesai@gmail.com 2025-08-06T11:27:30-07:00 GitHub noreply@github.com 2025-08-06T20:27:30+02:00 chat : support Granite model reasoning and tool call (#14864) 476aa3fd5779b32d06cd84338f777da82341195c 0d8831543cdc368fb248bae6f1b4aa5516684edc Juk Armstrong 69222624+jukofyork@users.noreply.github.com 2025-08-06T17:28:48+01:00 GitHub noreply@github.com 2025-08-06T17:28:48+01:00 Fixed name `-override-tensors` to `-override-tensor` (#15129) 0d8831543cdc368fb248bae6f1b4aa5516684edc 65c797c4fad4d9966695ac4b4a1560be44109267 Diego Devesa slarengh@gmail.com 2025-08-06T05:37:35-07:00 GitHub noreply@github.com 2025-08-06T14:37:35+02:00 ggml : fix fallback to CPU for ununsupported ops (#15118) 65c797c4fad4d9966695ac4b4a1560be44109267 25726898e855ec6dffba227f2233a63c57184036 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-06T13:26:49+02:00 GitHub noreply@github.com 2025-08-06T13:26:49+02:00 chat : fix yandex chat template (#15116) 25726898e855ec6dffba227f2233a63c57184036 2241453252147bb7362a286977ee9f9a92130062 stevenkuang stevenkuang@tencent.com 2025-08-06T17:48:30+08:00 GitHub noreply@github.com 2025-08-06T11:48:30+02:00 chat : fix hunyuan auto-detection (#15114) c42293848dd9f03eba9a6c1b85a600b398f06541 1cbf442f50a154aac0bf35d7b8b3436a2cdfd6f5 Yunzhe Jia yunzhe@calculet.tech 2025-08-06T15:46:06+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-08-06T15:46:06+08:00 sync with calrt API 1cbf442f50a154aac0bf35d7b8b3436a2cdfd6f5 ba67d6c03fed3193987a4ef13050e6e2a4451df4 Yunzhe Jia yunzhe@calculet.tech 2025-07-24T15:50:19+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-08-06T15:45:10+08:00 1.sync with calrt prefill,decode,kv_update op 2.add hw pattern untile function 2241453252147bb7362a286977ee9f9a92130062 9515c6131aecaccc955fdedcfe16c3e030aaefcb Chenguang Li 757486878@qq.com 2025-08-06T14:12:42+08:00 GitHub noreply@github.com 2025-08-06T14:12:42+08:00 CANN: add support for ACL Graph (#15065) 9515c6131aecaccc955fdedcfe16c3e030aaefcb fd1234cb468935ea087d6929b2487926c3afff4b Reese Levine reeselevine1@gmail.com 2025-08-05T16:26:38-07:00 GitHub noreply@github.com 2025-08-05T16:26:38-07:00 ggml: WebGPU disable SET_ROWS for now (#15078) fd1234cb468935ea087d6929b2487926c3afff4b f324a3b715d5c1081c110ce459f8a8486fb1ee89 Georgi Gerganov ggerganov@gmail.com 2025-08-05T22:10:36+03:00 GitHub noreply@github.com 2025-08-05T22:10:36+03:00 llama : add gpt-oss (#15091) f324a3b715d5c1081c110ce459f8a8486fb1ee89 be426425817bc3e6a2d91dae476dba6fa85894be Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-05T20:43:36+02:00 GitHub noreply@github.com 2025-08-05T20:43:36+02:00 chat : only remove double bos/eos if added (#15086) be426425817bc3e6a2d91dae476dba6fa85894be 3306ceabf02e3df66666e5851800e843c7ca207e Georgi Gerganov ggerganov@gmail.com 2025-08-05T20:19:33+03:00 GitHub noreply@github.com 2025-08-05T20:19:33+03:00 readme : update hot topics (#15097) 3306ceabf02e3df66666e5851800e843c7ca207e c81de6e107ef51ef76aadcb8a6f008711c462517 Romain Biessy romain.biessy@codeplay.com 2025-08-05T18:39:55+02:00 GitHub noreply@github.com 2025-08-05T18:39:55+02:00 sycl: fix mul_mat selection (#15092) c81de6e107ef51ef76aadcb8a6f008711c462517 22f060c9c4b5ef49a83a20eda25fcb792419580b Juk Armstrong 69222624+jukofyork@users.noreply.github.com 2025-08-05T13:56:44+01:00 GitHub noreply@github.com 2025-08-05T13:56:44+01:00 Fix `glm4moe` bug (#15088) 22f060c9c4b5ef49a83a20eda25fcb792419580b ee3a9fcf88fe5b5e1213711e05861b83cd4fdfe6 Alex Wu dindinw@users.noreply.github.com 2025-08-05T19:56:44+08:00 GitHub noreply@github.com 2025-08-05T13:56:44+02:00 webui: fix markdown table (#15081) ee3a9fcf88fe5b5e1213711e05861b83cd4fdfe6 ec428b02c347767f24c78111309e3f30d2ada289 compilade git@compilade.net 2025-08-05T05:27:45-04:00 GitHub noreply@github.com 2025-08-05T11:27:45+02:00 context : fix index overflow on huge outputs (#15080) ec428b02c347767f24c78111309e3f30d2ada289 19f68fa5a4c3bf796de52a6db9008e77d29f423a Diego Devesa slarengh@gmail.com 2025-08-04T16:05:36-07:00 GitHub noreply@github.com 2025-08-05T01:05:36+02:00 llama : add --n-cpu-moe option (#15077) 19f68fa5a4c3bf796de52a6db9008e77d29f423a 41613437ffee0dbccad684fc744788bc504ec213 compilade git@compilade.net 2025-08-04T17:26:52-04:00 GitHub noreply@github.com 2025-08-04T23:26:52+02:00 imatrix : warn when GGUF imatrix is saved without .gguf suffix (#15076) 41613437ffee0dbccad684fc744788bc504ec213 e5bebe5251cee2678e8531aa1598ca21b3c6ce1d Christian Kastner ckk@kvr.at 2025-08-04T21:29:14+02:00 GitHub noreply@github.com 2025-08-04T21:29:14+02:00 cmake: Add GGML_BACKEND_DIR option (#15074) e5bebe5251cee2678e8531aa1598ca21b3c6ce1d ef0144c087b33e5b8da42d529ac71aaf05cb49df Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-04T21:01:48+02:00 GitHub noreply@github.com 2025-08-04T21:01:48+02:00 gguf-py : add --chat-template-file to gguf_new_metadata (#15075) ef0144c087b33e5b8da42d529ac71aaf05cb49df 2721257e3e2c4c944ac8a08221113ee7cb503f1b Sam sammcj@users.noreply.github.com 2025-08-05T04:29:25+10:00 GitHub noreply@github.com 2025-08-04T20:29:25+02:00 model: support GLM 4.5 family of models (#14939) 2721257e3e2c4c944ac8a08221113ee7cb503f1b 587d0118f50b7e8f4bafbcdd218aefd9da0272e1 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-04T18:11:02+02:00 GitHub noreply@github.com 2025-08-04T18:11:02+02:00 quantize : fix confusing error message if ftype is invalid (#15071) 587d0118f50b7e8f4bafbcdd218aefd9da0272e1 5aa1105da24a8dd1661cea3db0582c9b2c2f54d3 Reese Levine reeselevine1@gmail.com 2025-08-04T08:52:43-07:00 GitHub noreply@github.com 2025-08-04T08:52:43-07:00 ggml: WebGPU backend host improvements and style fixing (#14978) 5aa1105da24a8dd1661cea3db0582c9b2c2f54d3 d31192b4ee1441bbbecd3cbf9e02633368bdc4f5 Jeff Bolz jbolz@nvidia.com 2025-08-04T00:09:19-05:00 GitHub noreply@github.com 2025-08-04T07:09:19+02:00 vulkan: fix build when using glslang that does not support coopmat2 (#15062) d31192b4ee1441bbbecd3cbf9e02633368bdc4f5 0a2f5496bef9e54e5f42d6c2c3ad9eb7b379aed0 compilade git@compilade.net 2025-08-03T16:00:05-04:00 GitHub noreply@github.com 2025-08-03T22:00:05+02:00 imatrix : use GGUF by default (#14842) 0a2f5496bef9e54e5f42d6c2c3ad9eb7b379aed0 11a3811164ef2d75393c6b0a632f4c608e3e3dd2 compilade git@compilade.net 2025-08-03T15:49:13-04:00 GitHub noreply@github.com 2025-08-03T21:49:13+02:00 imatrix : fix 3d activation handling for hybrid and recurrent models (#14994) 11a3811164ef2d75393c6b0a632f4c608e3e3dd2 97366dc6abdd0bdc74260bd3c42bd06f0feb7428 compilade git@compilade.net 2025-08-03T15:43:07-04:00 GitHub noreply@github.com 2025-08-03T21:43:07+02:00 memory : handle kv_unified for hybrid models (#15050) 97366dc6abdd0bdc74260bd3c42bd06f0feb7428 83bc2f288c0e08e676d9beca9c4669197e920593 Csaba Kecskemeti csaba.kecskemeti@gmail.com 2025-08-03T12:38:18-07:00 GitHub noreply@github.com 2025-08-03T21:38:18+02:00 vocab : JetBrains Mellum pre-tokenizer (#15045) 83bc2f288c0e08e676d9beca9c4669197e920593 6c7a441161080551ce8a52ba32563b6295067192 Gabriel Larson 55459720+gabriellarson@users.noreply.github.com 2025-08-03T09:56:25-05:00 GitHub noreply@github.com 2025-08-03T16:56:25+02:00 model : add text-only support for Kimi-VL (and find special tokens in text_config) (#15051) 6c7a441161080551ce8a52ba32563b6295067192 5c0eb5ef544aeefd81c303e03208f768e158d93c Jeff Bolz jbolz@nvidia.com 2025-08-03T07:23:57-05:00 GitHub noreply@github.com 2025-08-03T14:23:57+02:00 vulkan: Use coopmat2 for conv2d (#14982) 5c0eb5ef544aeefd81c303e03208f768e158d93c 03d46982180c2fb624bd2a233e46426ab22be5d1 lhez lih@qti.qualcomm.com 2025-08-02T10:51:18-07:00 GitHub noreply@github.com 2025-08-02T19:51:18+02:00 opencl: fix adreno compiler detection logic (#15029) 03d46982180c2fb624bd2a233e46426ab22be5d1 3303c19b1691088275ee864a823697177c94a15d Johannes Gäßler johannesg@5d6.de 2025-08-02T16:37:08+02:00 GitHub noreply@github.com 2025-08-02T16:37:08+02:00 CUDA: use mma FA kernel for gqa > 4 on RTX 4000 (#15035) 3303c19b1691088275ee864a823697177c94a15d 4fdea540bda4648f98b85e8ee9dc66db4bfb5945 leejet leejet714@gmail.com 2025-08-02T22:15:36+08:00 GitHub noreply@github.com 2025-08-02T17:15:36+03:00 cuda: make im2col a little faster (#15025) 4fdea540bda4648f98b85e8ee9dc66db4bfb5945 a4569c41fd2253c89ef52fc2378687bdbf42f61a Daniel Bevenius daniel.bevenius@gmail.com 2025-08-02T16:14:57+02:00 GitHub noreply@github.com 2025-08-02T17:14:57+03:00 kv-cache : skip alignment of n_stream in kv-cache log msg [no ci] (#15040) a4569c41fd2253c89ef52fc2378687bdbf42f61a 15e92fd33791e60a4ddb5970b47242a855c27117 Georgi Gerganov ggerganov@gmail.com 2025-08-02T17:14:21+03:00 GitHub noreply@github.com 2025-08-02T17:14:21+03:00 llama : enable LLAMA_SET_ROWS=1 by default (#14959) 15e92fd33791e60a4ddb5970b47242a855c27117 2bf3fbf0b54f97aef2b388b76d222789e1c170f1 Georgi Gerganov ggerganov@gmail.com 2025-08-02T17:13:05+03:00 GitHub noreply@github.com 2025-08-02T17:13:05+03:00 cuda, sycl : fix batched gemm when ne02 == 1 && ne03 > 1 (#15038) 2bf3fbf0b54f97aef2b388b76d222789e1c170f1 711d5e6fe66eb6cd7a10d71cec4567321848be08 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-08-02T14:39:01+02:00 GitHub noreply@github.com 2025-08-02T14:39:01+02:00 ci : check that pre-tokenizer hashes are up-to-date (#15032) 711d5e6fe66eb6cd7a10d71cec4567321848be08 f738989dcb9ccbe468c945553eafbeef7b869675 Douglas Hanley thesecretaryofwar@gmail.com 2025-08-02T05:51:02-05:00 GitHub noreply@github.com 2025-08-02T12:51:02+02:00 convert : fix Qwen3-Embedding pre-tokenizer hash (#15030) f738989dcb9ccbe468c945553eafbeef7b869675 4cb208c93c1c938591a5b40354e2a6f9b94489bc Jhen-Jie Hong iainst0409@gmail.com 2025-08-02T18:04:48+08:00 GitHub noreply@github.com 2025-08-02T18:04:48+08:00 chat : fix multiple tool_calls on hermes-2-pro (#14962) 4cb208c93c1c938591a5b40354e2a6f9b94489bc 3025b621d12a6931ff5e9775d4f644719980ad91 Jeff Bolz jbolz@nvidia.com 2025-08-02T04:21:37-05:00 GitHub noreply@github.com 2025-08-02T11:21:37+02:00 vulkan: coopmat2 mul_mat optimizations (#14934) 3025b621d12a6931ff5e9775d4f644719980ad91 ec0b18802c91badd3ff1388ffd09ee163251bd72 R0CKSTAR yeahdongcn@gmail.com 2025-08-02T17:20:40+08:00 GitHub noreply@github.com 2025-08-02T17:20:40+08:00 llama-bench: rename DB table name from test to llama_bench (#15003) ec0b18802c91badd3ff1388ffd09ee163251bd72 339bd0268c498c89529cd0e90c44883c211e3745 Jeff Bolz jbolz@nvidia.com 2025-08-02T03:48:30-05:00 GitHub noreply@github.com 2025-08-02T10:48:30+02:00 vulkan: Support ne[3]>1 in noncontig matrix-vector multiply (#15015) 339bd0268c498c89529cd0e90c44883c211e3745 f906275537d14c8fc7c6976d944233771fd6672c Douglas Hanley thesecretaryofwar@gmail.com 2025-08-02T03:44:50-05:00 GitHub noreply@github.com 2025-08-02T10:44:50+02:00 model : support Qwen3-Embedding (#15023) f906275537d14c8fc7c6976d944233771fd6672c a9f7541ec25c4c8547daf5ff48700ad2836e2b7d Johannes Gäßler johannesg@5d6.de 2025-08-02T10:12:41+02:00 GitHub noreply@github.com 2025-08-02T10:12:41+02:00 server: enable token array inputs for OAI API (#15001) a9f7541ec25c4c8547daf5ff48700ad2836e2b7d 9c35706b98ea271858acef4194f526a71b24cdc9 Jeff Bolz jbolz@nvidia.com 2025-08-02T02:57:04-05:00 GitHub noreply@github.com 2025-08-02T09:57:04+02:00 vulkan: optimizations for direct convolution (#14933) 9c35706b98ea271858acef4194f526a71b24cdc9 c76b420e4ce06f7b7cdfbb0b85d02c90e5cc5a3a Johannes Gäßler johannesg@5d6.de 2025-08-01T20:47:32+02:00 GitHub noreply@github.com 2025-08-01T20:47:32+02:00 CUDA: fix MMQ nwarps for AMD with warp_size==32 (#15014) c76b420e4ce06f7b7cdfbb0b85d02c90e5cc5a3a 0f5ccd6fd1a1f709010312933db0316867cc30b6 l-austenfeld 53152202+l-austenfeld@users.noreply.github.com 2025-08-01T16:59:06+02:00 GitHub noreply@github.com 2025-08-01T16:59:06+02:00 vendor : update vendored copy of google/minja (#15011) 0f5ccd6fd1a1f709010312933db0316867cc30b6 1c872f71fb8a25589efa3ee9b6bf8b517cb8caa4 stevenkuang stevenkuang@tencent.com 2025-08-01T21:31:12+08:00 GitHub noreply@github.com 2025-08-01T15:31:12+02:00 model : add hunyuan dense (#14878) 1c872f71fb8a25589efa3ee9b6bf8b517cb8caa4 baad94885df512bb24ab01e2b22d1998fce4d00e lhez quic_lih@quicinc.com 2025-08-01T04:15:44-07:00 GitHub noreply@github.com 2025-08-01T13:15:44+02:00 opencl: add f16 for `add`, `sub`, `mul`, `div` (#14984) baad94885df512bb24ab01e2b22d1998fce4d00e ba42794c9ead96ad52311ba1b23eefcbf3d6f63d Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2025-08-01T11:50:33+05:30 GitHub noreply@github.com 2025-08-01T09:20:33+03:00 ggml : Q2k interleaving implementation - x86/x64 SIMD (#14373) ba42794c9ead96ad52311ba1b23eefcbf3d6f63d 2860d479b456e1caa026b40b829d5b13c42a8ed7 Georgi Gerganov ggerganov@gmail.com 2025-08-01T06:38:12+03:00 GitHub noreply@github.com 2025-08-01T06:38:12+03:00 graph : fix equal_seq() check (#14986) 2860d479b456e1caa026b40b829d5b13c42a8ed7 484b2091ce5017901483b5204c07878f171d1441 diannao 55k@outlook.com 2025-08-01T10:02:34+08:00 GitHub noreply@github.com 2025-08-01T10:02:34+08:00 docker : add cann build pipline (#14591) 484b2091ce5017901483b5204c07878f171d1441 daf2dd788066b8b239cb7f68210e090c2124c199 R0CKSTAR yeahdongcn@gmail.com 2025-08-01T08:47:27+08:00 GitHub noreply@github.com 2025-08-01T08:47:27+08:00 compare-commits.sh: support both llama-bench and test-backend-ops (#14392) daf2dd788066b8b239cb7f68210e090c2124c199 a06ed5feaec9f935fbf662035b2673167bc88460 Ed Addario 29247825+EAddario@users.noreply.github.com 2025-07-31T20:32:18+01:00 GitHub noreply@github.com 2025-07-31T21:32:18+02:00 quantize : skip tensor override when in fallback mode (#14995) a06ed5feaec9f935fbf662035b2673167bc88460 784524053d986255e383dae45235e4f76c6792a7 Diego Devesa slarengh@gmail.com 2025-07-31T11:15:41-07:00 GitHub noreply@github.com 2025-07-31T20:15:41+02:00 llama : add simple option to enable CPU for MoE weights (--cpu-moe) (#14992) 784524053d986255e383dae45235e4f76c6792a7 d6818d06a6237631523bc0f45d42e79482667948 Aman Gupta amangupta052@gmail.com 2025-08-01T01:22:58+08:00 GitHub noreply@github.com 2025-08-01T01:22:58+08:00 Fix params bug in diffusion example (#14993) d6818d06a6237631523bc0f45d42e79482667948 e08a98826bcce70a3377592c51d0efed99eafe07 Diego Devesa slarengh@gmail.com 2025-07-31T09:11:34-07:00 GitHub noreply@github.com 2025-07-31T18:11:34+02:00 llama : allow other bufts when overriding to CPU, add --no-repack option (#14990) e08a98826bcce70a3377592c51d0efed99eafe07 952a47f455fbd92e2659b98b9b6317a2dafeb532 Ruben Ortlam picard12@live.de 2025-07-31T17:46:54+02:00 GitHub noreply@github.com 2025-07-31T17:46:54+02:00 Vulkan: Fix minor debug mode issues (#14899) 952a47f455fbd92e2659b98b9b6317a2dafeb532 36e5fe7bcd6640f4ff974d4c5cb04a13b5b29bce tc-mb 157115220+tc-mb@users.noreply.github.com 2025-07-31T23:22:17+08:00 GitHub noreply@github.com 2025-07-31T17:22:17+02:00 mtmd : support MiniCPM-V 4.0 (#14983) 36e5fe7bcd6640f4ff974d4c5cb04a13b5b29bce 94933c8c2eeaa9a7983e3f6c08af76bd86724094 Csaba Kecskemeti csaba.kecskemeti@gmail.com 2025-07-31T07:59:49-07:00 GitHub noreply@github.com 2025-07-31T10:59:49-04:00 MODEL_TENSOR.SSM_DT_NORM has defined twice (#14991) 94933c8c2eeaa9a7983e3f6c08af76bd86724094 c1dacaa99b4ead6edbac928cd2da59436573f6b0 g2mt 166577174+g2mt@users.noreply.github.com 2025-07-31T05:25:23-07:00 GitHub noreply@github.com 2025-07-31T14:25:23+02:00 server : implement universal assisted decoding (#12635) c1dacaa99b4ead6edbac928cd2da59436573f6b0 a9f77a8be348deeb11fb3d54d412bf583003c90d Dongliang Wei 121270393+wdl339@users.noreply.github.com 2025-07-31T20:12:20+08:00 GitHub noreply@github.com 2025-07-31T14:12:20+02:00 llama : merge build_moe_ffn_from_probs function into build_moe_ffn (#14968) a9f77a8be348deeb11fb3d54d412bf583003c90d 8a4a85627702b569d7d2810f2de06a4321656e9d Lukas Straub lukasstraub2@web.de 2025-07-31T14:08:23+02:00 GitHub noreply@github.com 2025-07-31T14:08:23+02:00 server : add openai-style logit_bias support (#14946) 8a4a85627702b569d7d2810f2de06a4321656e9d 11490b36723d511d75fb601995c79b5c363ba3a2 Aman Gupta amangupta052@gmail.com 2025-07-31T19:49:09+08:00 GitHub noreply@github.com 2025-07-31T19:49:09+08:00 Add LLaDA 8b Diffusion model (#14771) 11490b36723d511d75fb601995c79b5c363ba3a2 66625a59a54d0a7504eda4c4e83abfcd83ba1cf8 hipudding huafengchun@gmail.com 2025-07-31T19:47:20+08:00 GitHub noreply@github.com 2025-07-31T19:47:20+08:00 CANN: Improve loading efficiency after converting weights to NZ format. (#14985) 66625a59a54d0a7504eda4c4e83abfcd83ba1cf8 6e6725459a892b49602b596339de4916c7c7965a compilade git@compilade.net 2025-07-31T01:02:46-04:00 GitHub noreply@github.com 2025-07-31T08:02:46+03:00 graph : reduce splits for recurrent and hybrid models (#14825) 6e6725459a892b49602b596339de4916c7c7965a e9192bec564780bd4313ad6524d20a0ab92797db lhez lih@qti.qualcomm.com 2025-07-30T14:56:55-07:00 GitHub noreply@github.com 2025-07-30T14:56:55-07:00 opencl: add `mul_mat_f32_f32_l4_lm` and `mul_mat_f16_f32_l4_lm` (#14809) e9192bec564780bd4313ad6524d20a0ab92797db 41e78c567e9a8c652e405f4f909deb598deecd31 Ed Addario 29247825+EAddario@users.noreply.github.com 2025-07-30T20:11:56+01:00 GitHub noreply@github.com 2025-07-30T21:11:56+02:00 quantize : fix using combined imatrix GGUFs (multiple datasets) (#14973) 41e78c567e9a8c652e405f4f909deb598deecd31 ad4a700117d1746799d2d6599e526e2c3a7938d2 Daniel Bevenius daniel.bevenius@gmail.com 2025-07-30T18:07:11+02:00 GitHub noreply@github.com 2025-07-30T18:07:11+02:00 server : add support for `embd_normalize` parameter (#14964) ad4a700117d1746799d2d6599e526e2c3a7938d2 e32a4ec60ee472acf3fe82b5966976fd6965ac6b uvos carl@uvos.xyz 2025-07-30T17:38:06+02:00 GitHub noreply@github.com 2025-07-30T17:38:06+02:00 HIP: enable mfma mmq on gfx908 and gfx90a for select datatypes and shapes (#14949) e32a4ec60ee472acf3fe82b5966976fd6965ac6b e228de94496636681b36690247d96f97d5f76c0d Georgi Gerganov ggerganov@gmail.com 2025-07-30T16:03:13+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-30T17:33:11+03:00 sync : ggml e228de94496636681b36690247d96f97d5f76c0d 73a8e5ca0372f4dcaf1aed4e42261723da0914aa Kai Pastor dg0yt@darc.de 2025-07-30T14:53:16+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-30T17:33:11+03:00 cmake : Fix BLAS link interface (ggml/1316) 73a8e5ca0372f4dcaf1aed4e42261723da0914aa 92b8810ec7aa6d778bc287cc918443cf67b962e2 Kai Pastor dg0yt@darc.de 2025-07-30T14:52:26+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-30T17:33:11+03:00 vulkan : fix 32-bit builds (ggml/1313) 92b8810ec7aa6d778bc287cc918443cf67b962e2 00131d6eaf4df029e1ec84de868c2c5957503007 Johannes Gäßler johannesg@5d6.de 2025-07-30T15:46:13+02:00 GitHub noreply@github.com 2025-07-30T15:46:13+02:00 CUDA: skip masked KV slices for all FA kernels (#14924) 00131d6eaf4df029e1ec84de868c2c5957503007 1e15bfd42c3938506e0da5939bf7f42780965f01 Georgi Gerganov ggerganov@gmail.com 2025-07-30T15:12:02+03:00 GitHub noreply@github.com 2025-07-30T15:12:02+03:00 tests : update for LLAMA_SET_ROWS=1 (#14961) 1e15bfd42c3938506e0da5939bf7f42780965f01 a118d80233d3bf92569c051346fd2638f87bf202 Georgi Gerganov ggerganov@gmail.com 2025-07-30T13:52:11+03:00 GitHub noreply@github.com 2025-07-30T13:52:11+03:00 graph : fix stack-use-after-return (#14960) a118d80233d3bf92569c051346fd2638f87bf202 61550f8231dc0aa478e2f537c5009ede6878ce22 Douglas Hanley thesecretaryofwar@gmail.com 2025-07-30T00:25:05-05:00 GitHub noreply@github.com 2025-07-30T08:25:05+03:00 embeddings: fix extraction of CLS pooling results (#14927) 61550f8231dc0aa478e2f537c5009ede6878ce22 aa79524c51fb014f8df17069d31d7c44b9ea6cb8 Xinpeng Dou 15529241576@163.com 2025-07-30T08:39:24+08:00 GitHub noreply@github.com 2025-07-30T08:39:24+08:00 CANN: update ops docs (#14935) aa79524c51fb014f8df17069d31d7c44b9ea6cb8 b77d11179d7efe68ce5c913006f2ef9ee17cc8f7 uvos carl@uvos.xyz 2025-07-29T20:23:04+02:00 GitHub noreply@github.com 2025-07-29T20:23:04+02:00 HIP: remove the use of __HIP_PLATFORM_AMD__, explicitly support only AMD targets (#14945) b77d11179d7efe68ce5c913006f2ef9ee17cc8f7 c7aa1364fd59b2ac06fd9e0a719253d968472dc3 uvos carl@uvos.xyz 2025-07-29T17:44:30+02:00 GitHub noreply@github.com 2025-07-29T17:44:30+02:00 HIP: add GGML_HIP_MMQ_MFMA option to allow disableing the MFMA path. (#14930) c7aa1364fd59b2ac06fd9e0a719253d968472dc3 1a67fcc30677e96dda76bb1b290788e7d8852b51 uvos carl@uvos.xyz 2025-07-29T17:43:43+02:00 GitHub noreply@github.com 2025-07-29T17:43:43+02:00 HIP: Ignore unsupported unroll transformation in fattn-vec (#14931) 1a67fcc30677e96dda76bb1b290788e7d8852b51 204f2cf168dc01ca7b200b1510e0ff585ca9a92e kallewoof karljohan-alm@garage.co.jp 2025-07-30T00:05:38+09:00 GitHub noreply@github.com 2025-07-29T17:05:38+02:00 common : avoid logging partial messages (which can contain broken UTF-8 sequences) (#14937) 204f2cf168dc01ca7b200b1510e0ff585ca9a92e 138b288b594eeb19d58e0ab0d74eae32009c9c20 hipudding huafengchun@gmail.com 2025-07-29T22:36:43+08:00 GitHub noreply@github.com 2025-07-29T22:36:43+08:00 CANN: Add ggml_set_rows (#14943) 138b288b594eeb19d58e0ab0d74eae32009c9c20 bbd0f917797e9d524680f1b30d34a46eb06d7651 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-29T14:22:03+02:00 GitHub noreply@github.com 2025-07-29T14:22:03+02:00 cuda : add softcap fusion (#14907) bbd0f917797e9d524680f1b30d34a46eb06d7651 0a5036bee9cfb946870689db4400e9e0d17844c9 Johannes Gäßler johannesg@5d6.de 2025-07-29T10:40:50+02:00 GitHub noreply@github.com 2025-07-29T10:40:50+02:00 server-bench: make seed choice configurable (#14929) 0a5036bee9cfb946870689db4400e9e0d17844c9 8ad7b3e65b5834e5574c2f5640056c9047b5d93b Aman Gupta amangupta052@gmail.com 2025-07-29T14:45:18+08:00 GitHub noreply@github.com 2025-07-29T14:45:18+08:00 CUDA: add roll (#14919) 8ad7b3e65b5834e5574c2f5640056c9047b5d93b bda62193b2a6bebbf515c3c389303094a44458c1 lhez lih@qti.qualcomm.com 2025-07-28T09:50:17-07:00 GitHub noreply@github.com 2025-07-28T18:50:17+02:00 opencl : add ops docs (#14910) bda62193b2a6bebbf515c3c389303094a44458c1 c556418b600ad5792440942079d93e393595688b Leonard Mosescu tlemo@users.noreply.github.com 2025-07-28T09:04:27-07:00 GitHub noreply@github.com 2025-07-28T18:04:27+02:00 test-backend-ops : extend test case filtering (#14865) c556418b600ad5792440942079d93e393595688b db16e2831c0f344f041af3d067db81c42b16eb22 Radoslav Gerganov rgerganov@gmail.com 2025-07-28T18:59:04+03:00 GitHub noreply@github.com 2025-07-28T18:59:04+03:00 llama-bench : use local GPUs along with RPC servers (#14917) db16e2831c0f344f041af3d067db81c42b16eb22 cd1fce6d4f9c191f1c7429cc96f61281c3b63ffc xctan xc-tan@outlook.com 2025-07-28T23:40:24+08:00 GitHub noreply@github.com 2025-07-28T17:40:24+02:00 ggml-cpu : deduplicate scalar implementations (#14897) cd1fce6d4f9c191f1c7429cc96f61281c3b63ffc 00fa15fedc79263fa0285e6a3bbb0cfb3e3878a2 Akarshan Biswas akarshan@menlo.ai 2025-07-28T20:32:15+05:30 GitHub noreply@github.com 2025-07-28T20:32:15+05:30 SYCL: Add set_rows support for quantized types (#14883) 00fa15fedc79263fa0285e6a3bbb0cfb3e3878a2 946b1f685909c8c9c044f145bce819c02f327eaa Xuan-Son Nguyen son@huggingface.co 2025-07-28T15:01:48+02:00 GitHub noreply@github.com 2025-07-28T15:01:48+02:00 mtmd : add support for Voxtral (#14862) 946b1f685909c8c9c044f145bce819c02f327eaa 6c6e397affc4fac717e718364fb4b635cec6433a Johannes Gäßler johannesg@5d6.de 2025-07-28T14:30:22+02:00 GitHub noreply@github.com 2025-07-28T14:30:22+02:00 CUDA: fix pointer incrementation in FA (#14916) 6c6e397affc4fac717e718364fb4b635cec6433a afc0e8969896ada62238da07b98731e5a4b12ba4 Dongliang Wei 121270393+wdl339@users.noreply.github.com 2025-07-28T19:47:00+08:00 GitHub noreply@github.com 2025-07-28T13:47:00+02:00 model : add support for SmallThinker series (#14898) afc0e8969896ada62238da07b98731e5a4b12ba4 a5771c9eea801f573dc7375416e4c3306543563d Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2025-07-28T11:05:53+01:00 GitHub noreply@github.com 2025-07-28T11:05:53+01:00 sycl: refactor quantization to q8_1 (#14815) a5771c9eea801f573dc7375416e4c3306543563d c35f9eaf095e0db3aa9b77b56846bab5d3bf5661 Georgi Gerganov ggerganov@gmail.com 2025-07-28T11:01:03+03:00 GitHub noreply@github.com 2025-07-28T10:01:03+02:00 ops : update BLAS (#14914) c35f9eaf095e0db3aa9b77b56846bab5d3bf5661 1f45f2890ef7f365ba0a45e08a8d1f46b8bc6b9e Georgi Gerganov ggerganov@gmail.com 2025-07-28T08:22:56+03:00 GitHub noreply@github.com 2025-07-28T08:22:56+03:00 ops : update Metal (#14912) 1f45f2890ef7f365ba0a45e08a8d1f46b8bc6b9e 613c5095c33de9b7bbf1097d7c32510f51d58b01 Georgi Gerganov ggerganov@gmail.com 2025-07-28T08:14:20+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-28T08:15:01+03:00 sync : ggml 613c5095c33de9b7bbf1097d7c32510f51d58b01 7f97599581fcf0c37432dd3b1f503b91bed97695 Kai Pastor dg0yt@darc.de 2025-07-24T19:58:02+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-28T08:15:01+03:00 cmake : Indent ggml-config.cmake (ggml/1310) 7f97599581fcf0c37432dd3b1f503b91bed97695 bf78f5439ee8e82e367674043303ebf8e92b4805 Ed Addario 29247825+EAddario@users.noreply.github.com 2025-07-27T22:31:11+01:00 GitHub noreply@github.com 2025-07-27T23:31:11+02:00 quantize : update README.md (#14905) bf78f5439ee8e82e367674043303ebf8e92b4805 bbfc84927481d1e59d8af6939b93b76850f0ab53 Ruben Ortlam picard12@live.de 2025-07-27T15:33:08+02:00 GitHub noreply@github.com 2025-07-27T15:33:08+02:00 vulkan: add ops docs (#14900) bbfc84927481d1e59d8af6939b93b76850f0ab53 ca0ef2dddb022cb1337d775cd05cd27d7808aff4 Akarshan Biswas akarshan@menlo.ai 2025-07-27T17:52:58+05:30 GitHub noreply@github.com 2025-07-27T17:52:58+05:30 SYCL: add ops doc (#14901) ca0ef2dddb022cb1337d775cd05cd27d7808aff4 89d1029559bd2968f76db854f9f113d73e34527c Daniel Bevenius daniel.bevenius@gmail.com 2025-07-27T12:10:51+02:00 GitHub noreply@github.com 2025-07-27T12:10:51+02:00 llama : clarify comment about pp and tg graphs [no ci] (#14895) 89d1029559bd2968f76db854f9f113d73e34527c f1a4e72de5950ab7136aeadddd675caf30dd6b3f Erik Scholz Green-Sky@users.noreply.github.com 2025-07-27T12:04:33+02:00 GitHub noreply@github.com 2025-07-27T12:04:33+02:00 vulkan : add fp16 support for the conv_2d kernel (#14872) f1a4e72de5950ab7136aeadddd675caf30dd6b3f 4762ad7316dcdec20016ab5985fb46a27902204d Jeff Bolz jbolz@nvidia.com 2025-07-27T04:05:34-05:00 GitHub noreply@github.com 2025-07-27T11:05:34+02:00 vulkan: skip empty set_rows to avoid invalid API usage (#14860) 4762ad7316dcdec20016ab5985fb46a27902204d 1dc9614e0673e794d2e2bf88ba04f7d57b63a57b Gabriel Larson 55459720+gabriellarson@users.noreply.github.com 2025-07-27T03:18:37-05:00 GitHub noreply@github.com 2025-07-27T11:18:37+03:00 model : make rope_yarn_log_mul optional for deepseek2 (#14896) 1dc9614e0673e794d2e2bf88ba04f7d57b63a57b 446595b9b3a113d9ba10506922c3a156cca9d477 Shunta Saito shunta.saito@gmail.com 2025-07-27T16:38:44+09:00 GitHub noreply@github.com 2025-07-27T09:38:44+02:00 llama : fix kq_scale for the attention layers of PLaMo2 (#14892) 446595b9b3a113d9ba10506922c3a156cca9d477 66906cd82a4a1fd10151707cee3f66cb61fc4055 Aman Gupta amangupta052@gmail.com 2025-07-27T09:36:43+08:00 GitHub noreply@github.com 2025-07-27T09:36:43+08:00 Docs: add instructions for adding backends (#14889) 66906cd82a4a1fd10151707cee3f66cb61fc4055 11dd5a44eb180e1d69fac24d3852b5222d66fb7f deepsek 166548550+deepsek@users.noreply.github.com 2025-07-26T18:28:14-04:00 GitHub noreply@github.com 2025-07-27T00:28:14+02:00 HIP: Enable Matrix cores for MMQ Kernels, Enable stream-K for CDNA 3 (#14624) 11dd5a44eb180e1d69fac24d3852b5222d66fb7f 9b8f3c6c776d77045ee4f7f13fdf7863dfe59e5b hipudding huafengchun@gmail.com 2025-07-26T17:56:18+08:00 GitHub noreply@github.com 2025-07-26T17:56:18+08:00 CANN: Implement GLU ops (#14884) 9b8f3c6c776d77045ee4f7f13fdf7863dfe59e5b c7f3169cd523140a288095f2d79befb20a0b73f4 R0CKSTAR yeahdongcn@gmail.com 2025-07-26T10:36:02+08:00 GitHub noreply@github.com 2025-07-26T10:36:02+08:00 musa: fix build warnings (unused variable) (#14869) c7f3169cd523140a288095f2d79befb20a0b73f4 793c0d7f46384001738c337d7afa46b45ae32745 Aaron Teo aaron.teo1@ibm.com 2025-07-26T01:09:03+08:00 GitHub noreply@github.com 2025-07-25T19:09:03+02:00 ggml-cpu : disable GGML_NNPA by default due to instability (#14880) 793c0d7f46384001738c337d7afa46b45ae32745 ce111d39d666f4a3b6a561ad020f6feb8cc67790 Gabe Goodhart ghart@us.ibm.com 2025-07-25T10:47:39-06:00 GitHub noreply@github.com 2025-07-25T10:47:39-06:00 metal: SSM_SCAN performance (#14743) ce111d39d666f4a3b6a561ad020f6feb8cc67790 e7fecba93416c8aaf343932b5e36dd5e208a815e lhez lih@qti.qualcomm.com 2025-07-25T08:12:13-07:00 GitHub noreply@github.com 2025-07-25T17:12:13+02:00 opencl: add fused `rms_norm_mul` (#14841) e7fecba93416c8aaf343932b5e36dd5e208a815e e2b7621e7c265a6739225125cf9c534f471b3472 wooksong wook16.song@samsung.com 2025-07-25T23:25:05+09:00 GitHub noreply@github.com 2025-07-25T16:25:05+02:00 docs : update HOWTO‑add‑model.md for ModelBase and new model classes (#14874) e2b7621e7c265a6739225125cf9c534f471b3472 c1dbea752a630169c104118fd0c82f3ffcb19c91 Oliver Simons osimons@nvidia.com 2025-07-25T13:29:57+02:00 GitHub noreply@github.com 2025-07-25T14:29:57+03:00 ggml : remove invalid portPos specifiers from dot files (#14838) c1dbea752a630169c104118fd0c82f3ffcb19c91 749e0d27f0247337869f4698f59dd7fafba94326 Georgi Gerganov ggerganov@gmail.com 2025-07-25T14:28:06+03:00 GitHub noreply@github.com 2025-07-25T14:28:06+03:00 context : restore preemptive sched reset when LLAMA_SET_ROWS=0 (#14870) 749e0d27f0247337869f4698f59dd7fafba94326 64bf1c3744053cf7def10aeed21ff48883ee755b kiwi 122582483+kiwi142857@users.noreply.github.com 2025-07-25T19:08:04+08:00 GitHub noreply@github.com 2025-07-25T13:08:04+02:00 mtmd : fix 32-bit narrowing issue in export-lora and mtmd clip (#14503) 64bf1c3744053cf7def10aeed21ff48883ee755b c12bbde37258c6d053322d1cedd4a9672109ae58 Chris Rohlf chris.rohlf@gmail.com 2025-07-25T06:17:02-04:00 GitHub noreply@github.com 2025-07-25T12:17:02+02:00 rpc : check for null buffers in get/set/copy tensor endpoints (#14868) c12bbde37258c6d053322d1cedd4a9672109ae58 3f4fc97f1d745f1d5d3c853949503136d419e6de Diego Devesa slarengh@gmail.com 2025-07-25T01:07:26-07:00 GitHub noreply@github.com 2025-07-25T11:07:26+03:00 sched : fix multiple evaluations of the same graph with pipeline parallelism (#14855) 3f4fc97f1d745f1d5d3c853949503136d419e6de 2df255da3cea108de0ae9b302ffdd31674b6d88d R0CKSTAR yeahdongcn@gmail.com 2025-07-25T03:05:37+08:00 GitHub noreply@github.com 2025-07-24T20:05:37+01:00 musa: upgrade musa sdk to rc4.2.0 (#14498) 2df255da3cea108de0ae9b302ffdd31674b6d88d 60f816a79dd74007158745530e71738aa6caa67e Georgi Gerganov ggerganov@gmail.com 2025-07-24T18:30:33+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-24T20:27:23+03:00 sync : ggml 60f816a79dd74007158745530e71738aa6caa67e 5592f278b6ec6aa4a1793e89e8c61838a12ebc9d Kai Pastor dg0yt@darc.de 2025-07-22T20:13:21+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-24T20:27:23+03:00 cmake : fix usage issues (ggml/1257) 5592f278b6ec6aa4a1793e89e8c61838a12ebc9d e4868d16d24dec55e61bcaadaca28feed8f98b13 Daniel Bevenius daniel.bevenius@gmail.com 2025-07-21T15:53:12+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-24T20:27:23+03:00 ggml-cpu : remove stdlib include from repack.cpp (ggml/1276) e4868d16d24dec55e61bcaadaca28feed8f98b13 820de57d4faa427a3d0bfb14e48057247fae036e Georgi Gerganov ggerganov@gmail.com 2025-07-24T16:31:48+03:00 GitHub noreply@github.com 2025-07-24T16:31:48+03:00 context : perform output reorder lazily upon access after sync (#14853) 820de57d4faa427a3d0bfb14e48057247fae036e cb4a63aad6650c2b536a7578403935388cb2920e Xuan-Son Nguyen son@huggingface.co 2025-07-24T13:59:56+02:00 GitHub noreply@github.com 2025-07-24T13:59:56+02:00 chat : fix kimi-k2 chat template (#14852) cb4a63aad6650c2b536a7578403935388cb2920e 86f5623d904cfd392fdeb14a143097b4074660f6 Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2025-07-24T11:09:57+01:00 GitHub noreply@github.com 2025-07-24T11:09:57+01:00 sycl: fixed semantics of block offset calculation (#14814) 86f5623d904cfd392fdeb14a143097b4074660f6 39cffdf18855e0d2beba62572542251d87421e73 yummy 57988893+jk3456a@users.noreply.github.com 2025-07-24T17:50:51+08:00 GitHub noreply@github.com 2025-07-24T11:50:51+02:00 llama : fix MiniCPM inference after Granite Four changes (#14850) 39cffdf18855e0d2beba62572542251d87421e73 065908cb09adff8b2a5f1173f80050d5d9a6790b Pouya PooyaGhahramanian@Gmail.com 2025-07-24T12:26:44+03:00 GitHub noreply@github.com 2025-07-24T11:26:44+02:00 docs: add libcurl-dev install hint for Linux distros (#14801) 065908cb09adff8b2a5f1173f80050d5d9a6790b 4ec6291a2407404de52239c1f9ca66c07e7fb28b Georgi Gerganov ggerganov@gmail.com 2025-07-24T10:24:05+03:00 GitHub noreply@github.com 2025-07-24T10:24:05+03:00 metal : fix fusion across different encoders (#14849) 4ec6291a2407404de52239c1f9ca66c07e7fb28b a12363bbf0ca11f787dee9756861043136edc0df Donghyeon Jeong 54725479+djeong20@users.noreply.github.com 2025-07-24T13:50:41+09:00 GitHub noreply@github.com 2025-07-24T12:50:41+08:00 sycl: fix undefined variable in work group size check (#14843) a12363bbf0ca11f787dee9756861043136edc0df a86f52b2859dae4db5a7a0bbc0f1ad9de6b43ec6 jacekpoplawski 67507230+jacekpoplawski@users.noreply.github.com 2025-07-23T23:23:57+02:00 GitHub noreply@github.com 2025-07-23T23:23:57+02:00 convert : text-only support for GLM-4.1V-9B-Thinking (#14823) a86f52b2859dae4db5a7a0bbc0f1ad9de6b43ec6 b284197df426fb189cdcfe56a43c863a788ac756 Johannes Gäßler johannesg@5d6.de 2025-07-23T21:43:25+02:00 GitHub noreply@github.com 2025-07-23T21:43:25+02:00 CUDA: fix overflow in FA, tune performance (#14840) b284197df426fb189cdcfe56a43c863a788ac756 221c0e0c5841a814e95b9bfd549de9d6ae00ac6e Johannes Gäßler johannesg@5d6.de 2025-07-23T18:22:30+02:00 GitHub noreply@github.com 2025-07-23T18:22:30+02:00 CUDA: fix compilation with GGML_CUDA_F16 (#14837) 221c0e0c5841a814e95b9bfd549de9d6ae00ac6e 07a19e27a26f76d34be62da53807f93131fb3cab Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-23T14:27:54+02:00 GitHub noreply@github.com 2025-07-23T14:27:54+02:00 ci : correct label refactor->refactoring (#14832) 07a19e27a26f76d34be62da53807f93131fb3cab 18f3b5ff9e5eda4e7d04bceff8ffdccb0a696ed8 Johannes Gäßler johannesg@5d6.de 2025-07-23T12:35:53+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-23T14:08:09+03:00 CUDA: fix quantized KV cache + multiple sequences (#14822) 18f3b5ff9e5eda4e7d04bceff8ffdccb0a696ed8 7233358d29df3dfbf80693f2de7e5865401ad724 Georgi Gerganov ggerganov@gmail.com 2025-07-18T13:36:27+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-23T14:08:09+03:00 tests : add non-cont K,V FA tests 7233358d29df3dfbf80693f2de7e5865401ad724 6c88b3bb2509d980e6a64c50fdf8dd304929f770 l3utterfly gc.pthzfoldr@gmail.com 2025-07-23T16:16:41+08:00 GitHub noreply@github.com 2025-07-23T11:16:41+03:00 memory : handle saving/loading null layers in recurrent memory (#14675) 6c88b3bb2509d980e6a64c50fdf8dd304929f770 14c28dfc50a2e3915e697122cd3c5343224009be lixing-star 104126818+lixing-star@users.noreply.github.com 2025-07-23T14:39:51+08:00 GitHub noreply@github.com 2025-07-23T09:39:51+03:00 ggml: fix loongarch quantize_row_q8_1 error (#14827) 14c28dfc50a2e3915e697122cd3c5343224009be 8c988fa41db4d9dbc05abfd666c04def1259c645 chen fan 350211548@qq.com 2025-07-23T11:58:00+08:00 GitHub noreply@github.com 2025-07-23T11:58:00+08:00 CANN: weight format to NZ for Ascend310P3 (#14407) 8c988fa41db4d9dbc05abfd666c04def1259c645 acd6cb1c41676f6bbb25c2a76fa5abeb1719301e Aman Gupta amangupta052@gmail.com 2025-07-23T09:25:42+08:00 GitHub noreply@github.com 2025-07-23T09:25:42+08:00 CUDA: add fused rms norm (#14800) acd6cb1c41676f6bbb25c2a76fa5abeb1719301e 84712b60439453fb393c2ca753cee682a4ad41f5 Csaba Kecskemeti csaba.kecskemeti@gmail.com 2025-07-22T09:29:43-07:00 GitHub noreply@github.com 2025-07-22T19:29:43+03:00 ggml : model card yaml tab->2xspace (#14819) 84712b60439453fb393c2ca753cee682a4ad41f5 d4d1522b20809a350ffb094db20f40f17d3ab80f Jeff Bolz jbolz@nvidia.com 2025-07-22T10:35:21-05:00 GitHub noreply@github.com 2025-07-22T17:35:21+02:00 vulkan: fix rms_norm_mul to handle broadcasting dim0 (#14817) d4d1522b20809a350ffb094db20f40f17d3ab80f d1aa0cc5d13ec3fa553de5d298f9166679e58479 Molly Sophia mollysophia379@gmail.com 2025-07-22T23:01:29+08:00 GitHub noreply@github.com 2025-07-22T23:01:29+08:00 llama : add model type detection for rwkv7 7B&14B (#14816) d1aa0cc5d13ec3fa553de5d298f9166679e58479 c8ade30036139e32108fee53d8b7164dbfda4bee Ed Addario 29247825+EAddario@users.noreply.github.com 2025-07-22T13:33:37+01:00 GitHub noreply@github.com 2025-07-22T14:33:37+02:00 imatrix: add option to display importance score statistics for a given imatrix file (#12718) c8ade30036139e32108fee53d8b7164dbfda4bee e28c0b80c249b5618c3a0d5e960f63929f380ac9 stduhpf stephduh@live.fr 2025-07-22T12:51:03+02:00 GitHub noreply@github.com 2025-07-22T12:51:03+02:00 Mtmd: add a way to select device for vision encoder (#14236) e28c0b80c249b5618c3a0d5e960f63929f380ac9 8e6f8bc875358968b63e08f7bbbe0a288f29d856 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-22T12:33:10+02:00 GitHub noreply@github.com 2025-07-22T12:33:10+02:00 cuda : implement bf16 cpy ops and enable bf16 cont (#14763) 8e6f8bc875358968b63e08f7bbbe0a288f29d856 adef81781a15083f218eae6c488b95cdad781971 lhez lih@qti.qualcomm.com 2025-07-21T23:53:30-07:00 GitHub noreply@github.com 2025-07-22T08:53:30+02:00 opencl: remove unreachable `return` (#14806) adef81781a15083f218eae6c488b95cdad781971 48b86c4fdb1b1246d9fe17d2d53e3a1c2a0c3245 Molly Sophia mollysophia379@gmail.com 2025-07-22T09:24:22+08:00 GitHub noreply@github.com 2025-07-22T09:24:22+08:00 server : allow setting `--reverse-prompt` arg (#14799) 48b86c4fdb1b1246d9fe17d2d53e3a1c2a0c3245 38d3af1b73302377111e88ddd725257638339286 R0CKSTAR yeahdongcn@gmail.com 2025-07-22T07:45:26+08:00 GitHub noreply@github.com 2025-07-22T07:45:26+08:00 cuda: remove linking to cublasLt (#14790) 38d3af1b73302377111e88ddd725257638339286 6c9ee3b17e19dcc82ab93d52ae46fdd0226d4777 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-21T22:55:10+02:00 GitHub noreply@github.com 2025-07-21T13:55:10-07:00 opencl: fix `im2col` when `KW!=KH` (#14803) 6c9ee3b17e19dcc82ab93d52ae46fdd0226d4777 cd465d823c378853f1f6570eebfb77f69c7e1d39 rmatif kingrealriadh@gmail.com 2025-07-21T19:03:19+02:00 GitHub noreply@github.com 2025-07-21T10:03:19-07:00 opencl: add conv2d kernel (#14403) cd465d823c378853f1f6570eebfb77f69c7e1d39 922042601b8a16877ccb1c2afaa2071f76734f10 Romain Biessy romain.biessy@codeplay.com 2025-07-21T18:39:29+02:00 GitHub noreply@github.com 2025-07-21T18:39:29+02:00 sycl: Fix im2col (#14797) 922042601b8a16877ccb1c2afaa2071f76734f10 2ba1333b35e471b344974dde553db11bf1c2836f Charles Xu charles.xu@arm.com 2025-07-21T15:49:52+02:00 GitHub noreply@github.com 2025-07-21T16:49:52+03:00 kleidiai: add support for get_rows (#14676) 2ba1333b35e471b344974dde553db11bf1c2836f c2e058f1b4e799f1be085560c1bcef95b7b5ed02 Radoslav Gerganov rgerganov@gmail.com 2025-07-21T15:03:49+03:00 GitHub noreply@github.com 2025-07-21T14:03:49+02:00 docs : fix backends table in README.md (#14796) c2e058f1b4e799f1be085560c1bcef95b7b5ed02 c82d48ec23fb8749c341d0838f6891fd5f6b6da0 Jeff Bolz jbolz@nvidia.com 2025-07-21T06:35:40-05:00 GitHub noreply@github.com 2025-07-21T13:35:40+02:00 vulkan/cuda: Fix im2col when KW!=KH (#14789) c82d48ec23fb8749c341d0838f6891fd5f6b6da0 b4efd77f8ab407836ca73a5176f041650c5b2411 Molly Sophia mollysophia379@gmail.com 2025-07-21T17:38:36+08:00 GitHub noreply@github.com 2025-07-21T17:38:36+08:00 llama : fix `--reverse-prompt` crashing issue (#14794) b4efd77f8ab407836ca73a5176f041650c5b2411 2be60cbc2707359241c2784f9d2e30d8fc7cdabb IsaacDynamo 61521674+IsaacDynamo@users.noreply.github.com 2025-07-21T09:24:51+02:00 GitHub noreply@github.com 2025-07-21T10:24:51+03:00 server : add parse_special option to /tokenize endpoint (#14783) 2be60cbc2707359241c2784f9d2e30d8fc7cdabb b526ad2668944a7b2b1721f60679153646313831 Aman Gupta amangupta052@gmail.com 2025-07-21T02:13:47+08:00 GitHub noreply@github.com 2025-07-20T20:13:47+02:00 docs : fix link for tools/perplexity in README.md (#14780) b526ad2668944a7b2b1721f60679153646313831 938b785764683c298e7805e712f8728489cc2f18 rspOverflow 217881046+rspOverflow@users.noreply.github.com 2025-07-20T23:55:32+07:00 GitHub noreply@github.com 2025-07-20T18:55:32+02:00 Documentation: Further revisions to the Vulkan section in build.md (#14785) 938b785764683c298e7805e712f8728489cc2f18 36c153248faf969af1b62ab231348694b2047b8b Aman Gupta amangupta052@gmail.com 2025-07-20T19:42:34+08:00 GitHub noreply@github.com 2025-07-20T19:42:34+08:00 Clang-format: local files first + fix BinPacking (#14779) 36c153248faf969af1b62ab231348694b2047b8b a979ca22db0d737af1e548a73291193655c6be99 0cc4m picard12@live.de 2025-07-19T22:47:21+02:00 GitHub noreply@github.com 2025-07-19T23:47:21+03:00 Contrib: add 0cc4m as codeowner for Vulkan backend (#14775) a979ca22db0d737af1e548a73291193655c6be99 90083283ec254fa8d33897746dea229aee401b37 Ervin Áron Tasnádi etasnadi@protonmail.com 2025-07-19T21:59:08+02:00 GitHub noreply@github.com 2025-07-19T21:59:08+02:00 ggml: adds CONV_2D op and direct GEMM Vulkan implementation (#14316) 90083283ec254fa8d33897746dea229aee401b37 d4b91ea7b2da253e1355b503f0fcb7b428ce005d compilade git@compilade.net 2025-07-19T12:51:22-04:00 GitHub noreply@github.com 2025-07-19T12:51:22-04:00 imatrix : use GGUF to store importance matrices (#9400) d4b91ea7b2da253e1355b503f0fcb7b428ce005d 83f5872404baa39d826af2ef66351e63c64205a8 Peter0x44 peter0x44@disroot.org 2025-07-19T16:58:03+01:00 GitHub noreply@github.com 2025-07-19T17:58:03+02:00 vulkan: Add logging for bf16 features to ggml_vk_print_gpu_info (#13274) (#14707) 83f5872404baa39d826af2ef66351e63c64205a8 f0d4d176df72734a543c29eef9f942850c13311e 0cc4m picard12@live.de 2025-07-19T17:47:53+02:00 GitHub noreply@github.com 2025-07-19T17:47:53+02:00 Vulkan: Fix fprintf format-security warning (#14770) f0d4d176df72734a543c29eef9f942850c13311e b17230917c18a25af9cd143a941001466af845a2 rspOverflow 217881046+rspOverflow@users.noreply.github.com 2025-07-19T17:18:36+07:00 GitHub noreply@github.com 2025-07-19T12:18:36+02:00 Documentation: Update build.md's Vulkan section (#14736) b17230917c18a25af9cd143a941001466af845a2 bf9087f59aab940cf312b85a67067ce33d9e365a Georgi Gerganov ggerganov@gmail.com 2025-07-19T11:46:12+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-19T11:46:50+03:00 sync : ggml bf9087f59aab940cf312b85a67067ce33d9e365a 9fb1042ce6719bc46dbe88ab013148aabe3105f1 Georgi Gerganov ggerganov@gmail.com 2025-07-18T20:37:26+03:00 GitHub noreply@github.com 2025-07-18T20:37:26+03:00 metal : fuse add, mul + add tests (#14596) 9fb1042ce6719bc46dbe88ab013148aabe3105f1 2adf8d83acdb9b1bf58db6c9729ac9dc6847a58b Georgi Gerganov ggerganov@gmail.com 2025-07-18T20:08:33+03:00 GitHub noreply@github.com 2025-07-18T20:08:33+03:00 graph : fix graph reuse reset of params (#14760) 2adf8d83acdb9b1bf58db6c9729ac9dc6847a58b 021cc28bef4dd7d0bf9c91dbbd0803caa6cb15f2 Georgi Gerganov ggerganov@gmail.com 2025-07-18T17:33:41+03:00 GitHub noreply@github.com 2025-07-18T17:33:41+03:00 parallel : add option for different RNG seeds (#14757) 021cc28bef4dd7d0bf9c91dbbd0803caa6cb15f2 d498af3d5a00f96bdd37b534860f03a6d9e98d39 Oliver Simons oliver.simons@posteo.de 2025-07-18T13:35:32+02:00 GitHub noreply@github.com 2025-07-18T04:35:32-07:00 cuda : Fix Gemma3n not executed as CUDA_GRAPH on NVGPUs (#14741) d498af3d5a00f96bdd37b534860f03a6d9e98d39 eacdeb5bfcb6c6cd54461fd0e9f04cab78bf975b Georgi Gerganov ggerganov@gmail.com 2025-07-18T14:31:15+03:00 GitHub noreply@github.com 2025-07-18T14:31:15+03:00 graph : avoid huge warm-up graphs for MoE models (#14753) eacdeb5bfcb6c6cd54461fd0e9f04cab78bf975b e0cb5c5cb8a61ac232130cf6bf878035f93824d9 Georgi Gerganov ggerganov@gmail.com 2025-07-18T11:53:55+03:00 GitHub noreply@github.com 2025-07-18T11:53:55+03:00 model : fix build after merge conflict (#14754) e0cb5c5cb8a61ac232130cf6bf878035f93824d9 f9a31eea06a859e34cecb88b4d020c7f03d86cc4 lgai-exaone exaonemodels@lgresearch.ai 2025-07-18T17:45:49+09:00 GitHub noreply@github.com 2025-07-18T10:45:49+02:00 model : add EXAONE 4.0 support (#14630) f9a31eea06a859e34cecb88b4d020c7f03d86cc4 8f974bc1e980c06833504276021072e7a4088c81 Aman Gupta amangupta052@gmail.com 2025-07-18T14:54:18+08:00 GitHub noreply@github.com 2025-07-18T14:54:18+08:00 CUDA: set_rows + cpy.cu refactor (#14712) 8f974bc1e980c06833504276021072e7a4088c81 09651d09ffc1e941bd1be23163abf5495c416547 Georgi Gerganov ggerganov@gmail.com 2025-07-18T08:29:28+03:00 GitHub noreply@github.com 2025-07-18T08:29:28+03:00 graph : refactor context to not pass gf explicitly (#14629) 09651d09ffc1e941bd1be23163abf5495c416547 349ea79fcebc75b0c55bf61594a47736966d4f95 Nexes the Elder 124105151+Nexesenex@users.noreply.github.com 2025-07-18T06:25:54+02:00 GitHub noreply@github.com 2025-07-18T07:25:54+03:00 graph : Pass the graph placeholder message in debug mode (#14748) 349ea79fcebc75b0c55bf61594a47736966d4f95 670e1360cd40f242ae76ba0966542fae6cb59392 Neo Zhang Jianyu jianyu.zhang@intel.com 2025-07-18T10:23:14+08:00 GitHub noreply@github.com 2025-07-18T10:23:14+08:00 use max work group size for device to replace the magic number (#14732) 670e1360cd40f242ae76ba0966542fae6cb59392 760b4484e3c192a2649a6ffd0d90086c5558f849 Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-07-18T01:17:16+02:00 GitHub noreply@github.com 2025-07-18T01:17:16+02:00 convert : fix Ernie4.5 MoE without shared experts (#14746) 760b4484e3c192a2649a6ffd0d90086c5558f849 cb887f1bc1001c92f7b4a595b9014f3a454a07ab Wroclaw wroclaw223@outlook.com 2025-07-18T00:18:16+02:00 GitHub noreply@github.com 2025-07-17T15:18:16-07:00 nix : use optionalAttrs for env mkDerivation attrset argument (#14726) cb887f1bc1001c92f7b4a595b9014f3a454a07ab d6fb3f6b49b27ef1c0f4cf5128e041f7e7dc03af Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-07-17T23:15:32+02:00 GitHub noreply@github.com 2025-07-17T23:15:32+02:00 model: add Ernie 4.5 MoE support (#14658) d6fb3f6b49b27ef1c0f4cf5128e041f7e7dc03af 01612b74090df592663cfa01f661c9628f403b59 Georgi Gerganov ggerganov@gmail.com 2025-07-17T20:52:33+03:00 GitHub noreply@github.com 2025-07-17T20:52:33+03:00 kv-cache : fix k-shift for multiple streams (#14742) 01612b74090df592663cfa01f661c9628f403b59 086cf81e88fb75287b71ff19c08a206b7bc2e02f Georgi Gerganov ggerganov@gmail.com 2025-07-17T19:08:33+03:00 GitHub noreply@github.com 2025-07-17T19:08:33+03:00 llama : reuse compute graphs (#14482) 086cf81e88fb75287b71ff19c08a206b7bc2e02f d9b691081c04ec5fb0daa9d2b979f915c142963d Tarek Dakhran tarek@liquid.ai 2025-07-17T09:22:11+02:00 GitHub noreply@github.com 2025-07-17T09:22:11+02:00 llama : fix parallel processing for lfm2 (#14705) d9b691081c04ec5fb0daa9d2b979f915c142963d ad57d3edd2f48cf6dc41a98fd9b303435ecb4fb0 Georgi Gerganov ggerganov@gmail.com 2025-07-17T09:49:15+03:00 GitHub noreply@github.com 2025-07-17T09:49:15+03:00 kv-cache : opt mask set input (#14600) ad57d3edd2f48cf6dc41a98fd9b303435ecb4fb0 1ba45d49822c39ca9a552c7b75efe0495ff400c3 Georgi Gerganov ggerganov@gmail.com 2025-07-17T09:45:54+03:00 GitHub noreply@github.com 2025-07-17T09:45:54+03:00 batch : fix uninitialized has_cpl flag (#14733) 1ba45d49822c39ca9a552c7b75efe0495ff400c3 19e5943d9e976b59ccd5a0a87ae3d2ff3da44390 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-17T01:52:08+02:00 GitHub noreply@github.com 2025-07-16T20:52:08-03:00 ci : disable failing vulkan crossbuilds (#14723) 19e5943d9e976b59ccd5a0a87ae3d2ff3da44390 496957e1cbcb522abc63aa18521036e40efce985 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-16T23:17:43+02:00 GitHub noreply@github.com 2025-07-16T23:17:43+02:00 convert : make hf token optional (#14717) 496957e1cbcb522abc63aa18521036e40efce985 21c021745d781edf9c44b4972ef6cbbf53b0ecff Diner Burger burger@diner.name 2025-07-16T15:17:25-04:00 GitHub noreply@github.com 2025-07-16T21:17:25+02:00 llama : fix parameter order for hybrid memory initialization (#14725) 21c021745d781edf9c44b4972ef6cbbf53b0ecff b0f0ecc3dce806c68609d375a2b3edc430d8db18 Reese Levine reeselevine1@gmail.com 2025-07-16T08:18:51-07:00 GitHub noreply@github.com 2025-07-16T18:18:51+03:00 ggml: Add initial WebGPU backend (#14521) b0f0ecc3dce806c68609d375a2b3edc430d8db18 225e7a1438f4ea85eaa7b5ef3ab3b266ee4d9c06 tempstudio 49735574+tempstudio@users.noreply.github.com 2025-07-16T10:02:06-05:00 GitHub noreply@github.com 2025-07-16T18:02:06+03:00 model : support output bias for qwen2 (#14711) 225e7a1438f4ea85eaa7b5ef3ab3b266ee4d9c06 ab140198211385b85eeeb0abd549a4bbe259e10d Georgi Gerganov ggerganov@gmail.com 2025-07-16T16:35:42+03:00 GitHub noreply@github.com 2025-07-16T16:35:42+03:00 llama : add high-throughput mode (#14363) ab140198211385b85eeeb0abd549a4bbe259e10d 64978340b0b4a0a6e2fb74270c1509383d2eff32 Aman Gupta amangupta052@gmail.com 2025-07-16T20:03:51+08:00 GitHub noreply@github.com 2025-07-16T20:03:51+08:00 Support diffusion models: Add Dream 7B (#14644) 64978340b0b4a0a6e2fb74270c1509383d2eff32 6ffd4e9c442e99afac3d138543ebf86d5fc5ee03 Georgi Gerganov ggerganov@gmail.com 2025-07-16T14:43:32+03:00 GitHub noreply@github.com 2025-07-16T14:43:32+03:00 ggml : add asserts (#14720) 6ffd4e9c442e99afac3d138543ebf86d5fc5ee03 e4841d24d3485ea4af54f8b65a19ec3123f0ff3c Georgi Gerganov ggerganov@gmail.com 2025-07-16T14:04:12+03:00 GitHub noreply@github.com 2025-07-16T14:04:12+03:00 server : pre-calculate EOG logit biases (#14721) e4841d24d3485ea4af54f8b65a19ec3123f0ff3c 538cc77f7f44dfa047dba6a06d90c86dda69cf1d Shunta Saito shunta.saito@gmail.com 2025-07-16T19:12:22+09:00 GitHub noreply@github.com 2025-07-16T12:12:22+02:00 llama : fix parallel processing for plamo2 (#14716) 538cc77f7f44dfa047dba6a06d90c86dda69cf1d 5cae76654113160f691f581930b69fc5535e8159 Georgi Gerganov ggerganov@gmail.com 2025-07-16T12:13:57+03:00 GitHub noreply@github.com 2025-07-16T12:13:57+03:00 server : fix handling of the ignore_eos flag (#14710) 5cae76654113160f691f581930b69fc5535e8159 4b91d6f71f14040979bcdb7b6729b3bca93ec1c1 Johannes Gäßler johannesg@5d6.de 2025-07-16T09:33:28+02:00 GitHub noreply@github.com 2025-07-16T09:33:28+02:00 scripts: synthetic prompt mode for server-bench.py (#14695) 4b91d6f71f14040979bcdb7b6729b3bca93ec1c1 cf91f217f1c8b98b6db8e4ba6b480f017f81d206 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-16T08:52:04+02:00 GitHub noreply@github.com 2025-07-16T08:52:04+02:00 convert : only check for tokenizer folder if we need it (#14704) cf91f217f1c8b98b6db8e4ba6b480f017f81d206 79e0b68c178656bb0632cb8602d2940b755077f8 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-16T08:51:12+02:00 GitHub noreply@github.com 2025-07-16T08:51:12+02:00 convert : add pre-computed hashes first to prevent order mishaps (#14701) 79e0b68c178656bb0632cb8602d2940b755077f8 c81f4192f91a1e209c1eec7a84fe5371ef9175da Min-Hua 136287195+Min-Hua@users.noreply.github.com 2025-07-16T12:00:42+08:00 GitHub noreply@github.com 2025-07-16T07:00:42+03:00 llama: add LLAMA_API to deprecated llama_kv_self_seq_div (#14708) c81f4192f91a1e209c1eec7a84fe5371ef9175da 4a4f426944e79b79e389f9ed7b34831cb9b637ad Ed Addario 29247825+EAddario@users.noreply.github.com 2025-07-15T23:04:42+01:00 GitHub noreply@github.com 2025-07-16T00:04:42+02:00 gguf-py : dump bpw per layer and model in markdown mode (#14703) 4a4f426944e79b79e389f9ed7b34831cb9b637ad ba1ceb34566c889a1fc500efa79799ffed25d9b0 Gabriel Larson 55459720+gabriellarson@users.noreply.github.com 2025-07-15T14:54:22-05:00 GitHub noreply@github.com 2025-07-15T21:54:22+02:00 model : add Kimi-K2 support (#14654) ba1ceb34566c889a1fc500efa79799ffed25d9b0 10a0351a97c25471aea0bbde9cca54d32d163eec Jeff Bolz jbolz@nvidia.com 2025-07-15T14:51:09-05:00 GitHub noreply@github.com 2025-07-15T21:51:09+02:00 vulkan: fix noncontig check for mat_mul_id splitting (#14683) 10a0351a97c25471aea0bbde9cca54d32d163eec 68e37a61a7b24863f67541db65b4f6195962a268 Jeff Bolz jbolz@nvidia.com 2025-07-15T14:32:11-05:00 GitHub noreply@github.com 2025-07-15T21:32:11+02:00 vulkan: add RTE variants for glu/add/sub/mul/div (#14653) 68e37a61a7b24863f67541db65b4f6195962a268 cbc68be51d88b1d5531643b926a4b359c3cff131 Shunta Saito shunta.saito@gmail.com 2025-07-16T01:11:42+09:00 GitHub noreply@github.com 2025-07-15T18:11:42+02:00 model : add PLaMo-2 support (#14560) cbc68be51d88b1d5531643b926a4b359c3cff131 bdca38376f7e8dd928defe01ce6a16218a64b040 R0CKSTAR yeahdongcn@gmail.com 2025-07-15T15:28:53+08:00 GitHub noreply@github.com 2025-07-15T15:28:53+08:00 cuda: fix build warnings in set-rows.cu (unused variable) (#14687) bdca38376f7e8dd928defe01ce6a16218a64b040 55c509daf51d25bfaee9c8b8ce6abff103d4473b Anton Mitkov anton_b_mitkov@abv.bg 2025-07-14T18:12:42+01:00 GitHub noreply@github.com 2025-07-14T18:12:42+01:00 sycl: Hotfix for non dnnl codepath (#14677) 55c509daf51d25bfaee9c8b8ce6abff103d4473b 9c9e4fc6354fc811efa06a8eb7a86d3315cec9c8 shalinib-ibm Shalini.Salomi.Bodapati@ibm.com 2025-07-14T18:46:42+05:30 GitHub noreply@github.com 2025-07-14T16:16:42+03:00 ggml : refactor llamafile_sgemm PPC code (#14673) 9c9e4fc6354fc811efa06a8eb7a86d3315cec9c8 494c5899cb76859f32ddd913534f2685fd684a3d Aman Gupta amangupta052@gmail.com 2025-07-14T21:01:41+08:00 GitHub noreply@github.com 2025-07-14T21:01:41+08:00 llama-context: add ability to get logits (#14672) 494c5899cb76859f32ddd913534f2685fd684a3d 0f4c6ec0f1a9607ba67071f8a02c69b0afc2f91e Johannes Gäßler johannesg@5d6.de 2025-07-14T13:14:30+02:00 GitHub noreply@github.com 2025-07-14T13:14:30+02:00 scripts: benchmark for HTTP server throughput (#14668) 0f4c6ec0f1a9607ba67071f8a02c69b0afc2f91e 65a3ebb0aa56d6c501466e0f950ad15105fd32d8 Akarshan Biswas akarshan@menlo.ai 2025-07-14T15:07:55+05:30 GitHub noreply@github.com 2025-07-14T10:37:55+01:00 SYCL: use 1D kernel for set_rows (#14618) 65a3ebb0aa56d6c501466e0f950ad15105fd32d8 0d9226763c82562186122f3b827fa3862864a19c Anton Mitkov anton_b_mitkov@abv.bg 2025-07-14T10:37:35+01:00 GitHub noreply@github.com 2025-07-14T10:37:35+01:00 sycl: Batched mulmat rework for oneDNN dispatch (#14617) 0d9226763c82562186122f3b827fa3862864a19c 982e347255723fe6d02e60ee30cfdd0559c884c5 Molly Sophia mollysophia379@gmail.com 2025-07-14T07:43:43+08:00 GitHub noreply@github.com 2025-07-14T07:43:43+08:00 llama : add jinja template for rwkv-world (#14665) 982e347255723fe6d02e60ee30cfdd0559c884c5 923e3ea2e3c96a0b4c208f53bc3bc90cdcdf13c0 Ed Addario 29247825+EAddario@users.noreply.github.com 2025-07-13T17:02:17+01:00 GitHub noreply@github.com 2025-07-13T18:02:17+02:00 quantize : fix minor logic flaw in --tensor-type (#14572) 923e3ea2e3c96a0b4c208f53bc3bc90cdcdf13c0 e743cddb60dc3a8815b9de7dd7d5c491e61b2259 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-13T15:01:24+02:00 GitHub noreply@github.com 2025-07-13T15:01:24+02:00 cuda : add set rows for bf16 (#14664) e743cddb60dc3a8815b9de7dd7d5c491e61b2259 05fec5bd298d3c0243cbb9336e59b8b6aff75a81 Yavor Ivanov yavorgenadiev@gmail.com 2025-07-13T02:33:16-07:00 GitHub noreply@github.com 2025-07-13T11:33:16+02:00 cuda : add ELU support (#14657) 05fec5bd298d3c0243cbb9336e59b8b6aff75a81 dcf7f2ea3c4e3cf36dd9ab5a36785c00e6033267 Georgi Gerganov ggerganov@gmail.com 2025-07-13T10:36:33+03:00 GitHub noreply@github.com 2025-07-13T10:36:33+03:00 ggml : add build-time message to remind about ggml_set_rows (#14661) dcf7f2ea3c4e3cf36dd9ab5a36785c00e6033267 84b396e0510855a95d591afdf1f21c562cb3712a Yavor Ivanov yavorgenadiev@gmail.com 2025-07-12T22:38:13-07:00 GitHub noreply@github.com 2025-07-13T08:38:13+03:00 metal : Add missing unary ops Metal support (#14660) 84b396e0510855a95d591afdf1f21c562cb3712a c31e60647def83d671bac5ab5b35579bf25d9aa1 Yavor Ivanov yavorgenadiev@gmail.com 2025-07-12T22:12:36-07:00 GitHub noreply@github.com 2025-07-13T08:12:36+03:00 cmake : Add CMake presets for Linux and GCC (#14656) c31e60647def83d671bac5ab5b35579bf25d9aa1 67eade1bf93f40e7c4975e5bd0782f426c89cff4 Tarek Dakhran tarek@liquid.ai 2025-07-12T19:10:14+02:00 GitHub noreply@github.com 2025-07-12T19:10:14+02:00 tests : cover lfm2 cases in test_ssm_conv (#14651) 67eade1bf93f40e7c4975e5bd0782f426c89cff4 7de5c7cab61d4da4387ed9b216f88b96297bcc2d Tarek Dakhran tarek@liquid.ai 2025-07-12T19:07:08+02:00 GitHub noreply@github.com 2025-07-12T19:07:08+02:00 docs : add LFM2 to models section (#14650) 7de5c7cab61d4da4387ed9b216f88b96297bcc2d 8eff95544e817704d44bec20f9fc956ce76a33be Aman Gupta amangupta052@gmail.com 2025-07-12T21:31:38+08:00 GitHub noreply@github.com 2025-07-12T16:31:38+03:00 CUDA: add set rows for f32 and f16 (#14551) 8eff95544e817704d44bec20f9fc956ce76a33be 3120413ccd7797d4cbdee32ef89a641765d1f6c4 Georgi Gerganov ggerganov@gmail.com 2025-07-12T16:06:12+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-12T16:13:27+03:00 sync : ggml 3120413ccd7797d4cbdee32ef89a641765d1f6c4 215535701d659e873c52d2a9163d4616a42da4f7 Georgi Gerganov ggerganov@gmail.com 2025-07-12T12:39:32+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-12T14:25:44+03:00 vulkan : remove unused vars (#0) 215535701d659e873c52d2a9163d4616a42da4f7 74bb294591054a6bf0cc3f6e4637d87414cf2c46 Georgi Gerganov ggerganov@gmail.com 2025-07-12T12:39:27+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-12T14:25:44+03:00 sync : ggml 74bb294591054a6bf0cc3f6e4637d87414cf2c46 3e303b1107e4748cd50a675544d8c7fabb40ec54 Acly aclysia@gmail.com 2025-07-12T12:37:37+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-12T14:25:44+03:00 vulkan : implement bilinear interpolation (ggml/1291) 3e303b1107e4748cd50a675544d8c7fabb40ec54 0c1df14b5f8d992805cb22d0b77b44092a18aeab Acly aclysia@gmail.com 2025-07-12T12:32:32+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-12T14:25:44+03:00 vulkan : implement ggml_roll (ggml/1290) 0c1df14b5f8d992805cb22d0b77b44092a18aeab b3ad3a0191994d6c47b2bd389d5c7431526ecd2c Douglas Hanley thesecretaryofwar@gmail.com 2025-07-12T06:21:02-04:00 GitHub noreply@github.com 2025-07-12T13:21:02+03:00 server : fix pooled embedding output (#14645) b3ad3a0191994d6c47b2bd389d5c7431526ecd2c 98197e5c98388470030d908f355ec5937dcccaaa Jeff Bolz jbolz@nvidia.com 2025-07-12T05:12:26-05:00 GitHub noreply@github.com 2025-07-12T12:12:26+02:00 vulkan: support SET_ROWS (#14587) 98197e5c98388470030d908f355ec5937dcccaaa f5e96b368f1acc7f53c390001b936517c4d18999 Jeff Bolz jbolz@nvidia.com 2025-07-12T04:51:58-05:00 GitHub noreply@github.com 2025-07-12T11:51:58+02:00 vulkan: optimizations for deepseek prompt processing (#14555) f5e96b368f1acc7f53c390001b936517c4d18999 756aa1020aabc70b71b9a8efcd35fbf42b3bb9ab Tarek Dakhran t.dakhran@gmail.com 2025-07-11T20:27:01+02:00 GitHub noreply@github.com 2025-07-11T20:27:01+02:00 model : support LiquidAI LFM2 hybrid family (#14620) 756aa1020aabc70b71b9a8efcd35fbf42b3bb9ab aaa088d87f9006d56866085fe46e4b2755ef723f Slobodan Josic 127323561+slojosic-amd@users.noreply.github.com 2025-07-11T18:55:00+02:00 GitHub noreply@github.com 2025-07-11T18:55:00+02:00 HIP : Add HIP 7.0+ compatibility for hipBLAS compute types (#14634) aaa088d87f9006d56866085fe46e4b2755ef723f 0d5375d54b258ec63edd1fb5d58c37d58ce8be8b Georgi Gerganov ggerganov@gmail.com 2025-07-11T16:07:55+03:00 GitHub noreply@github.com 2025-07-11T16:07:55+03:00 readme : add hot PRs (#14636) 0d5375d54b258ec63edd1fb5d58c37d58ce8be8b 576c82eda210ca0111c04f5256bf77897a4d4cc4 Georgi Gerganov ggerganov@gmail.com 2025-07-11T13:46:07+03:00 GitHub noreply@github.com 2025-07-11T13:46:07+03:00 llama : move enum llama_vocab_pre_type to implementation (#14631) 576c82eda210ca0111c04f5256bf77897a4d4cc4 0aedae00e6fb48680324a5ac5da9cba0e35de6b5 Dowon ks2515@naver.com 2025-07-11T16:36:04+09:00 GitHub noreply@github.com 2025-07-11T09:36:04+02:00 vocab : add midm-2.0 model pre-tokenizer (#14626) 0aedae00e6fb48680324a5ac5da9cba0e35de6b5 6bdda13981d6c8189b7dc4f9fb8ecb91c21529f8 Gabe Goodhart ghart@us.ibm.com 2025-07-10T18:20:13-06:00 GitHub noreply@github.com 2025-07-11T02:20:13+02:00 model : Granite Four (#13550) 6bdda13981d6c8189b7dc4f9fb8ecb91c21529f8 0b8855775c6b873931d40b77a5e42558aacbde52 rmatif kingrealriadh@gmail.com 2025-07-10T23:58:12+02:00 GitHub noreply@github.com 2025-07-10T14:58:12-07:00 opencl: add tiled mul_mat_f16_f32 (#14535) 0b8855775c6b873931d40b77a5e42558aacbde52 4bb625b713fd9b294b4f7af87eaa752b710c7cc1 lhez quic_lih@quicinc.com 2025-07-10T11:48:52-07:00 GitHub noreply@github.com 2025-07-10T11:48:52-07:00 opencl: add `set_rows` for `f16` and `f32` (#14547) 4bb625b713fd9b294b4f7af87eaa752b710c7cc1 11ee0fea2a24da1d3206eeba8fc52b759d9dfb24 Ryan Mangeno 160974989+ryan-mangeno@users.noreply.github.com 2025-07-10T13:41:00-04:00 GitHub noreply@github.com 2025-07-10T19:41:00+02:00 Smoldocling support (#14597) 11ee0fea2a24da1d3206eeba8fc52b759d9dfb24 a457551332853ef19d0796fec12b62c538126ea5 Aman Gupta amangupta052@gmail.com 2025-07-10T23:29:01+08:00 GitHub noreply@github.com 2025-07-10T23:29:01+08:00 Docs: script to auto-generate ggml operations docs (#14598) a457551332853ef19d0796fec12b62c538126ea5 704bb7a71c01dc07c1478b85f6322bf5dfde1eaf Eric Zhang 34133756+EZForever@users.noreply.github.com 2025-07-10T20:29:05+08:00 GitHub noreply@github.com 2025-07-10T15:29:05+03:00 cmake : do not search for curl libraries by ourselves (#14613) 704bb7a71c01dc07c1478b85f6322bf5dfde1eaf 435a6d10d618a015060e45a38c7e9f27f4243316 Akarshan Biswas akarshan@menlo.ai 2025-07-10T13:59:38+05:30 GitHub noreply@github.com 2025-07-10T09:29:38+01:00 SYCL: Initial set_rows kernel implementation (#14562) 435a6d10d618a015060e45a38c7e9f27f4243316 f9a867f5921a85f3fa64d7b067f4c8ffc5f62eb4 Xuan-Son Nguyen son@huggingface.co 2025-07-10T09:00:20+02:00 GitHub noreply@github.com 2025-07-10T10:00:20+03:00 llama : minor coding style fix for smollm3 (#14605) f9a867f5921a85f3fa64d7b067f4c8ffc5f62eb4 ac44eb6c808bd5d677261ce86edd8c43ec54cf2c Eric Zhang 34133756+EZForever@users.noreply.github.com 2025-07-10T13:19:37+08:00 GitHub noreply@github.com 2025-07-10T08:19:37+03:00 cmake : bump llguidance version to v1.0.1 (#14609) ac44eb6c808bd5d677261ce86edd8c43ec54cf2c a57d1bcb3c0165ac87b1f0dbb429839b0da69689 Eric Zhang 34133756+EZForever@users.noreply.github.com 2025-07-10T13:19:13+08:00 GitHub noreply@github.com 2025-07-10T08:19:13+03:00 cmake : llguidance build parser library only (#14608) a57d1bcb3c0165ac87b1f0dbb429839b0da69689 cb9178f885d1986cc0b12feb26ff426bc8a3556c compilade git@compilade.net 2025-07-09T23:54:38-04:00 GitHub noreply@github.com 2025-07-09T23:54:38-04:00 cuda : support Falcon-H1 state size for SSM_SCAN (#14602) cb9178f885d1986cc0b12feb26ff426bc8a3556c 4a5686da22057867c23bd4a6be941ddc8c51e585 Xuan-Son Nguyen son@huggingface.co 2025-07-09T23:09:28+02:00 GitHub noreply@github.com 2025-07-09T23:09:28+02:00 llama : remove llm_graph_input_one (#14603) 4a5686da22057867c23bd4a6be941ddc8c51e585 98bab638fb28cf95a5a66dd2d51b40d6c8f6d69a compilade git@compilade.net 2025-07-09T14:59:57-04:00 GitHub noreply@github.com 2025-07-09T14:59:57-04:00 llama : support Jamba hybrid Transformer-Mamba models (#7531) 98bab638fb28cf95a5a66dd2d51b40d6c8f6d69a 26a48ad699d50b6268900062661bd22f3e792579 Xuan-Son Nguyen son@huggingface.co 2025-07-09T18:16:12+02:00 GitHub noreply@github.com 2025-07-09T18:16:12+02:00 ggml : add ggml_scale_bias (#14417) 26a48ad699d50b6268900062661bd22f3e792579 ffd59e7d18a76459d5c31ba97073c7c9d73cb752 Miaoqian Lin linmq006@gmail.com 2025-07-09T20:33:53+08:00 GitHub noreply@github.com 2025-07-09T14:33:53+02:00 ggml : prevent integer overflow in gguf tensor size calculation (#14595) ffd59e7d18a76459d5c31ba97073c7c9d73cb752 105554595f9a7bf3e02232ed7798201d47c2a4a2 Dowon ks2515@naver.com 2025-07-09T17:22:31+09:00 GitHub noreply@github.com 2025-07-09T11:22:31+03:00 model : add skt/A.X-4.0 model vocabulary (#14589) 105554595f9a7bf3e02232ed7798201d47c2a4a2 04655063c47af6cbced295c8c7ad369402b15300 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-09T10:19:50+02:00 GitHub noreply@github.com 2025-07-09T10:19:50+02:00 llama : remove unintended whitespace (#14592) 04655063c47af6cbced295c8c7ad369402b15300 20b7bf8a32259ad9189a4797fd3e3a859c537b99 ibrahim khadraoui 132432132+ibrahimkhadraoui@users.noreply.github.com 2025-07-09T12:03:49+04:00 GitHub noreply@github.com 2025-07-09T10:03:49+02:00 model : add support for Falcon-H1 family (#14534) 20b7bf8a32259ad9189a4797fd3e3a859c537b99 6efcd65945a98cf6883cdd9de4c8ccd8c79d219a Xuan-Son Nguyen son@huggingface.co 2025-07-09T08:26:13+02:00 GitHub noreply@github.com 2025-07-09T09:26:13+03:00 convert : fix smollm3 jinja template (#14586) 6efcd65945a98cf6883cdd9de4c8ccd8c79d219a 699f4392a33f57c3352cf8d60bdc53db7ca235e7 Jeff Bolz jbolz@nvidia.com 2025-07-08T13:11:42-05:00 GitHub noreply@github.com 2025-07-08T20:11:42+02:00 vulkan: optimize flash attention split_k_reduce (#14554) 699f4392a33f57c3352cf8d60bdc53db7ca235e7 08382869a2d6dca5d84710f2f82cc17a9696585a stevenkuang stevenkuang@tencent.com 2025-07-09T00:29:29+08:00 GitHub noreply@github.com 2025-07-08T18:29:29+02:00 model : fix hunyuan moe chat template (#14584) 08382869a2d6dca5d84710f2f82cc17a9696585a bb4f7a9e4eec171fecf0f640b1337a1c24485560 Xuan-Son Nguyen son@huggingface.co 2025-07-08T18:07:01+02:00 GitHub noreply@github.com 2025-07-08T18:07:01+02:00 model : add SmolLM3 (#14581) bb4f7a9e4eec171fecf0f640b1337a1c24485560 b8eeb8741d4483daf576498cf90537b4de71633c compilade git@compilade.net 2025-07-08T11:37:47-04:00 GitHub noreply@github.com 2025-07-08T18:37:47+03:00 memory : fix broken batch splits for recurrent cache (#14575) b8eeb8741d4483daf576498cf90537b4de71633c 17a1f0d2d407040ee242e18dd79be8bb212cfcef Jeff Bolz jbolz@nvidia.com 2025-07-08T08:21:21-05:00 GitHub noreply@github.com 2025-07-08T15:21:21+02:00 vulkan : fix rope with partial rotation and non-cont src (#14582) 17a1f0d2d407040ee242e18dd79be8bb212cfcef 8f22dc0a53338c629c1ef8fa878d8e39bfe627c9 Alawode Oluwandabira dabiraalawode@yahoo.com 2025-07-08T11:47:33+03:00 GitHub noreply@github.com 2025-07-08T11:47:33+03:00 server: Add ability to mount server at prefix (#14544) 8f22dc0a53338c629c1ef8fa878d8e39bfe627c9 53903ae6fa5f1caf187889c839cdd1ad25da4018 Xuan-Son Nguyen son@huggingface.co 2025-07-08T10:24:06+02:00 GitHub noreply@github.com 2025-07-08T11:24:06+03:00 model : add hunyuan moe (#14425) 53903ae6fa5f1caf187889c839cdd1ad25da4018 4d0dcd4a06080e796e6742a88f2ffa7fc41b28b8 Jeff Bolz jbolz@nvidia.com 2025-07-08T02:38:31-05:00 GitHub noreply@github.com 2025-07-08T09:38:31+02:00 vulkan: increase timeout for CI (#14574) 4d0dcd4a06080e796e6742a88f2ffa7fc41b28b8 75c91de6e955d5b8f3f28173f5040593e1964eb3 Georgi Gerganov ggerganov@gmail.com 2025-07-08T10:15:21+03:00 GitHub noreply@github.com 2025-07-08T10:15:21+03:00 cuda : fix rope with partial rotation and non-cont src (#14580) 75c91de6e955d5b8f3f28173f5040593e1964eb3 68155c66f0e76680f34442247e589a090add22d3 Aman Gupta amangupta052@gmail.com 2025-07-08T10:11:18+08:00 GitHub noreply@github.com 2025-07-08T10:11:18+08:00 CUDA: add bilinear interpolation for upscale (#14563) 68155c66f0e76680f34442247e589a090add22d3 e1a7059053a9b8958f2d57a21fc46dbc7fb24f8e R0CKSTAR yeahdongcn@gmail.com 2025-07-08T07:58:30+08:00 GitHub noreply@github.com 2025-07-08T07:58:30+08:00 musa: fix build warnings (unused variable) (#14561) e1a7059053a9b8958f2d57a21fc46dbc7fb24f8e 12f55c302b35cfe900b84c5fe67c262026af9c44 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-07T23:35:35+02:00 GitHub noreply@github.com 2025-07-07T23:35:35+02:00 llama : fix incorrect minicpm3 v_states shape (#14571) 12f55c302b35cfe900b84c5fe67c262026af9c44 b9c3eefde1b67104bd993485ff38dd62abe9d70c Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-07T21:35:08+02:00 GitHub noreply@github.com 2025-07-07T21:35:08+02:00 llama : remove ggml_cont where possible (#14568) b9c3eefde1b67104bd993485ff38dd62abe9d70c 6491d6e4f1caf0ad2221865b4249ae6938a6308c Aman Gupta amangupta052@gmail.com 2025-07-07T21:45:43+08:00 GitHub noreply@github.com 2025-07-07T21:45:43+08:00 CUDA: add bf16 and i32 to getrows (#14529) 6491d6e4f1caf0ad2221865b4249ae6938a6308c e592be15756ae546ba87bb5a5250b71248121971 Eve 139727413+netrunnereve@users.noreply.github.com 2025-07-06T10:29:36Z GitHub noreply@github.com 2025-07-06T12:29:36+02:00 vulkan: increase LOAD_VEC_A to 8 (IQ1/IQ2) or 4 (IQ3) (#14485) e592be15756ae546ba87bb5a5250b71248121971 a0374a67e2924f2e845cdc59dd67d9a44065a89c Jeff Bolz jbolz@nvidia.com 2025-07-06T03:08:16-05:00 GitHub noreply@github.com 2025-07-06T10:08:16+02:00 vulkan: fix rms_norm+mul fusion (#14545) a0374a67e2924f2e845cdc59dd67d9a44065a89c ddef99522d1ba74193b7394e803fab8db5c78bae Jeff Bolz jbolz@nvidia.com 2025-07-05T02:26:04-05:00 GitHub noreply@github.com 2025-07-05T09:26:04+02:00 vulkan: Handle updated FA dim2/3 definition (#14518) ddef99522d1ba74193b7394e803fab8db5c78bae 668168814601d2aef6161ce49ab186bc74177ae7 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-05T09:17:14+02:00 GitHub noreply@github.com 2025-07-05T09:17:14+02:00 server : fix assistant prefilling when content is an array (#14360) 668168814601d2aef6161ce49ab186bc74177ae7 bac8bed248d15419137c5bc7f834582397baaebc Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-05T08:24:56+02:00 GitHub noreply@github.com 2025-07-04T23:24:56-07:00 opencl: add GELU_ERF (#14476) bac8bed248d15419137c5bc7f834582397baaebc b81510a7b78f7c5a6f069c4f3d0e569b9d287913 Georgi Gerganov ggerganov@gmail.com 2025-07-05T07:18:09+03:00 GitHub noreply@github.com 2025-07-05T07:18:09+03:00 eval-callback : check for empty input (#14539) b81510a7b78f7c5a6f069c4f3d0e569b9d287913 ef797db357e44ecb7437fa9d22f4e1614104b342 R0CKSTAR yeahdongcn@gmail.com 2025-07-05T12:10:53+08:00 GitHub noreply@github.com 2025-07-05T12:10:53+08:00 test-backend-ops: add support for specifying output format (#14368) ef797db357e44ecb7437fa9d22f4e1614104b342 67d1ef23c68e1e09dc6d7423d1ef5fc0ea56ed5e Georgi Gerganov ggerganov@gmail.com 2025-07-04T19:19:09+03:00 GitHub noreply@github.com 2025-07-04T19:19:09+03:00 metal : disable fast math in all quantize kernels (#14528) 67d1ef23c68e1e09dc6d7423d1ef5fc0ea56ed5e 7b50f7c0257695d0f6918bffce4e68d5c13b0c9e Georgi Gerganov ggerganov@gmail.com 2025-07-04T09:08:59+03:00 GitHub noreply@github.com 2025-07-04T09:08:59+03:00 batch : add optional for sequential equal split (#14511) 7b50f7c0257695d0f6918bffce4e68d5c13b0c9e c79184d2d192489e3c918bab8ed717d22f8c02bd Georgi Gerganov ggerganov@gmail.com 2025-07-04T09:05:36+03:00 GitHub noreply@github.com 2025-07-04T09:05:36+03:00 graph : prepare for 4D mask (#14515) c79184d2d192489e3c918bab8ed717d22f8c02bd 499a8f5a787f5fdc7f3fdfb9d1ff01b6cb5e994e Georgi Gerganov ggerganov@gmail.com 2025-07-04T09:04:59+03:00 GitHub noreply@github.com 2025-07-04T09:04:59+03:00 batch : add n_used count (#14512) 499a8f5a787f5fdc7f3fdfb9d1ff01b6cb5e994e 28657a8229b5adc6028cf1c4ed62191792d2fdb0 luyhcsu 110711054+luyhcsu@users.noreply.github.com 2025-07-04T11:50:07+08:00 GitHub noreply@github.com 2025-07-04T11:50:07+08:00 CANN: Replace aclrtMemsetSync with aclnnInplaceZero operator (#14002) 28657a8229b5adc6028cf1c4ed62191792d2fdb0 bee28421be25fd447f61cb6db64d556cbfce32ec Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-03T23:07:22+02:00 GitHub noreply@github.com 2025-07-03T23:07:22+02:00 ggml : implement GEGLU_ERF and GEGLU_QUICK ops (#14445) bee28421be25fd447f61cb6db64d556cbfce32ec 2b72bedec198a90bb5b0cceaf1d0aff9e34ffbc2 lhez quic_lih@quicinc.com 2025-07-03T11:22:24-07:00 GitHub noreply@github.com 2025-07-03T20:22:24+02:00 opencl : broadcast for soft_max (#14510) 2b72bedec198a90bb5b0cceaf1d0aff9e34ffbc2 c8c4495b8d3a8799e2d46778f993965b0ac1ae43 Jeff Bolz jbolz@nvidia.com 2025-07-03T13:21:14-05:00 GitHub noreply@github.com 2025-07-03T20:21:14+02:00 vulkan: support mixed/deepseekR1 FA head sizes (#14509) c8c4495b8d3a8799e2d46778f993965b0ac1ae43 7b63a71a6b0f54effe9b94073d4d0519dcf53676 Johannes Gäßler johannesg@5d6.de 2025-07-03T17:05:18+02:00 GitHub noreply@github.com 2025-07-03T17:05:18+02:00 ggml: backward pass for split swiglu (#14483) 7b63a71a6b0f54effe9b94073d4d0519dcf53676 0c2ee38ab729f3f6a7c28dc4c5dcd8d5f0cbf8fc Nicolò Scipione nicolo.scipione@codeplay.com 2025-07-03T11:00:03+02:00 GitHub noreply@github.com 2025-07-03T11:00:03+02:00 Fix conditional enabling following arch checks for ggml-sycl (#14504) 0c2ee38ab729f3f6a7c28dc4c5dcd8d5f0cbf8fc a70c8a0c4b4c1606cd9a0ba889ce61aa88610095 Xuan-Son Nguyen son@huggingface.co 2025-07-03T10:03:06+02:00 GitHub noreply@github.com 2025-07-03T10:03:06+02:00 convert : correct gemma 3n conversion (#14450) a70c8a0c4b4c1606cd9a0ba889ce61aa88610095 9067487c4411efb20400103fcccfdd389c80d428 Georgi Gerganov ggerganov@gmail.com 2025-07-03T10:53:35+03:00 GitHub noreply@github.com 2025-07-03T10:53:35+03:00 kv-cache : use ggml_set_rows (#14285) 9067487c4411efb20400103fcccfdd389c80d428 d4cdd9c1c3cebdafca735958597de4ff7b7c0f54 Georgi Gerganov ggerganov@gmail.com 2025-07-03T10:46:57+03:00 GitHub noreply@github.com 2025-07-03T10:46:57+03:00 ggml : fix FA mask dim 2 and 3 (#14505) d4cdd9c1c3cebdafca735958597de4ff7b7c0f54 55c2646b452419be7552157b9b11373ccc1bf2f2 Georgi Gerganov ggerganov@gmail.com 2025-07-03T07:48:32+03:00 GitHub noreply@github.com 2025-07-03T07:48:32+03:00 ggml : remove kompute backend (#14501) 55c2646b452419be7552157b9b11373ccc1bf2f2 e75ba4c0434eb759eb7ff74e034ebe729053e575 Aman Gupta amangupta052@gmail.com 2025-07-03T07:45:11+08:00 GitHub noreply@github.com 2025-07-03T07:45:11+08:00 CUDA: add dynamic shared mem to softmax, refactor general usage (#14497) e75ba4c0434eb759eb7ff74e034ebe729053e575 5d46babdc2d4675d96ebcf23cac098a02f0d30cc Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-02T21:02:35+02:00 GitHub noreply@github.com 2025-07-02T21:02:35+02:00 gguf-py : add support for chat template jinja files (#14508) 5d46babdc2d4675d96ebcf23cac098a02f0d30cc e17991c466ac835b2c71bd813c1ca7ff8dd97b94 compilade git@compilade.net 2025-07-02T13:10:24-04:00 GitHub noreply@github.com 2025-07-02T13:10:24-04:00 llama : initial Mamba-2 support (#9126) e17991c466ac835b2c71bd813c1ca7ff8dd97b94 c46944aa25b5560e6195f8cbcbc8947e9a6395c3 Georgi Gerganov ggerganov@gmail.com 2025-07-02T19:35:47+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-02T20:08:45+03:00 sync : ggml c46944aa25b5560e6195f8cbcbc8947e9a6395c3 f3ed38d793fab9f97987dc52d7a41622f2056701 Daniel Bevenius daniel.bevenius@gmail.com 2025-07-02T13:55:32+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-02T20:08:45+03:00 ggml : add version function to get lib version (ggml/1286) f3ed38d793fab9f97987dc52d7a41622f2056701 55a1c5a5fdefef808e95aabd3d5563af1068cc80 Rotem Dan rotemdan@gmail.com 2025-07-02T19:37:16+03:00 GitHub noreply@github.com 2025-07-02T18:37:16+02:00 Set RPATH to "@loader_path" / "$ORIGIN" to ensure executables and dynamic libraries search for dependencies in their origin directory. (#14309) 55a1c5a5fdefef808e95aabd3d5563af1068cc80 12a81af45f0dbbab24bd819a15f57c03ceb1be90 Aman Gupta amangupta052@gmail.com 2025-07-02T20:34:24+08:00 Georgi Gerganov ggerganov@gmail.com 2025-07-02T15:48:33+03:00 CUDA: add softmax broadcast (#14475) 12a81af45f0dbbab24bd819a15f57c03ceb1be90 8875523eb311cac832bfda0c581e852292185ae9 Johannes Gäßler johannesg@5d6.de 2025-07-02T13:42:12+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-02T15:48:33+03:00 CUDA: broadcasting for FlashAttention mask (#14500) 8875523eb311cac832bfda0c581e852292185ae9 ec68e84c32325a3417fbcd2e60d4bda6adb4e4bc Jeff Bolz jbolz@nvidia.com 2025-07-01T03:32:56-05:00 Georgi Gerganov ggerganov@gmail.com 2025-07-02T15:48:33+03:00 vulkan: support softmax/FA batch and broadcast (#14449) ec68e84c32325a3417fbcd2e60d4bda6adb4e4bc 307e79d33d4cdd9f1d6c42fc861724e6ba12b98f Georgi Gerganov ggerganov@gmail.com 2025-06-27T21:50:57+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-02T15:48:33+03:00 ggml : support bcast ggml_soft_max_ext, ggml_flash_attn_ext (#14435) 307e79d33d4cdd9f1d6c42fc861724e6ba12b98f d7f5f4e578d1f60b0835d1734a50438c309b3e5c zhouwg zhouwg2000@gmail.com 2025-07-02T20:38:10+08:00 GitHub noreply@github.com 2025-07-02T14:38:10+02:00 opencl : fix possible buffer overflow in dump_tensor (#14490) d7f5f4e578d1f60b0835d1734a50438c309b3e5c c8a4e470f65c3a932e18eddc5fba1844876d7463 Georgi Gerganov ggerganov@gmail.com 2025-07-02T14:12:07+03:00 GitHub noreply@github.com 2025-07-02T14:12:07+03:00 simple-chat : fix context-exceeded condition (#14494) c8a4e470f65c3a932e18eddc5fba1844876d7463 603e43dc913f90d9c3291624728b731687eddc35 Eric Zhang 34133756+EZForever@users.noreply.github.com 2025-07-02T19:00:04+08:00 GitHub noreply@github.com 2025-07-02T13:00:04+02:00 opencl : skip empty nodes on cgraph compute (#14491) 603e43dc913f90d9c3291624728b731687eddc35 611ba4b264c8fbb9d2d978bd6a5c17aeab1700fb lhez quic_lih@quicinc.com 2025-07-02T00:07:42-07:00 GitHub noreply@github.com 2025-07-02T09:07:42+02:00 opencl : update upscale to support align corners (#14488) 611ba4b264c8fbb9d2d978bd6a5c17aeab1700fb 85841e121db5b96ae4d277a3cd3995eef66b98ec Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-02T09:02:51+02:00 GitHub noreply@github.com 2025-07-02T09:02:51+02:00 ci : add OpenCL to labeler workflow (#14496) 85841e121db5b96ae4d277a3cd3995eef66b98ec 68b3cd65141586ef13a698baa9029627f038f102 Eric Zhang 34133756+EZForever@users.noreply.github.com 2025-07-02T13:41:35+08:00 GitHub noreply@github.com 2025-07-02T08:41:35+03:00 github : add OpenCL backend to issue templates (#14492) 68b3cd65141586ef13a698baa9029627f038f102 de569441470332ff922c23fb0413cc957be75b25 Björn Ganster mail@bjoern-ganster.de 2025-07-02T07:19:31+02:00 GitHub noreply@github.com 2025-07-02T08:19:31+03:00 ggml : Callback before abort (#14481) de569441470332ff922c23fb0413cc957be75b25 1b2aaf28acd632b79bc3b07acb5dad877dc7dbb1 Georgi Gerganov ggerganov@gmail.com 2025-07-01T18:04:08+03:00 GitHub noreply@github.com 2025-07-01T18:04:08+03:00 ci : disable fast-math for Metal GHA CI (#14478) 1b2aaf28acd632b79bc3b07acb5dad877dc7dbb1 343b6e94b61674ac94e64ede7b7ea3365793f7ed Grzegorz Grasza xek@redhat.com 2025-07-01T15:44:11+02:00 GitHub noreply@github.com 2025-07-01T15:44:11+02:00 Add Vulkan images to docker.md (#14472) 343b6e94b61674ac94e64ede7b7ea3365793f7ed 6a746cf9c40f8d93d13eb8a6c2b989a15e7fa61a Chenguang Li 757486878@qq.com 2025-07-01T16:47:30+08:00 GitHub noreply@github.com 2025-07-01T16:47:30+08:00 CANN: update aclnnGroupedMatmulV2 to aclnnGroupedMatmulV3 (#14411) 6a746cf9c40f8d93d13eb8a6c2b989a15e7fa61a eff5e45443a248f89cc61e64786f38b68b4da489 Jeff Bolz jbolz@nvidia.com 2025-07-01T03:43:08-05:00 GitHub noreply@github.com 2025-07-01T10:43:08+02:00 vulkan: Split large mul_mat_id to fit in shared memory (#14451) eff5e45443a248f89cc61e64786f38b68b4da489 a6a47958a1d9edc628cedc2f9320c84b59fe1f4f Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-07-01T10:14:21+02:00 GitHub noreply@github.com 2025-07-01T10:14:21+02:00 add GELU_ERF (#14455) a6a47958a1d9edc628cedc2f9320c84b59fe1f4f f61c05d4b1f8ad498a5cf7f0f57a40383aad563c Georgi Gerganov ggerganov@gmail.com 2025-07-01T11:05:48+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-01T11:06:39+03:00 ggml : remove trailing whitespace (#0) f61c05d4b1f8ad498a5cf7f0f57a40383aad563c 431b2c24f31bf5411786c378b409d0202df8d53c Georgi Gerganov ggerganov@gmail.com 2025-07-01T10:27:52+03:00 Georgi Gerganov ggerganov@gmail.com 2025-07-01T11:06:39+03:00 sync : ggml 431b2c24f31bf5411786c378b409d0202df8d53c 497be7c01dab243100f9c42e0a763a746e42d038 Acly aclysia@gmail.com 2025-07-01T09:11:00+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-01T11:06:39+03:00 ggml-cpu : "align corners" for bilinear upscale/downscale (ggml/1285) 497be7c01dab243100f9c42e0a763a746e42d038 79b33b231774d5c39c8df018e9a276becae6d41a Daniel Bevenius daniel.bevenius@gmail.com 2025-06-24T06:10:16+02:00 Georgi Gerganov ggerganov@gmail.com 2025-07-01T11:06:39+03:00 ggml-quants : rename best_mad to best_error (ggml/1283) 79b33b231774d5c39c8df018e9a276becae6d41a 0a5a3b5cdfd887cf0f8e09d9ff89dee130cfcdde lhez quic_lih@quicinc.com 2025-07-01T00:19:16-07:00 GitHub noreply@github.com 2025-07-01T09:19:16+02:00 opencl : add GEGLU, REGLU, SWIGLU (#14456) 0a5a3b5cdfd887cf0f8e09d9ff89dee130cfcdde 745f11fed09ba2303f3f494efb12286d382608e5 Aman Gupta amangupta052@gmail.com 2025-06-30T23:57:04+08:00 GitHub noreply@github.com 2025-06-30T23:57:04+08:00 Add Conv2d for CPU (#14388) 745f11fed09ba2303f3f494efb12286d382608e5 5dd942de5922a22ec8446a4ad2203738dbcb9389 Georgi Gerganov ggerganov@gmail.com 2025-06-30T18:03:03+03:00 GitHub noreply@github.com 2025-06-30T18:03:03+03:00 memory : correctly handle failure in apply() (#14438) 5dd942de5922a22ec8446a4ad2203738dbcb9389 a7417f55945eb7c7a5ea6807e66564b8066a4e50 Georgi Gerganov ggerganov@gmail.com 2025-06-30T17:04:05+03:00 GitHub noreply@github.com 2025-06-30T17:04:05+03:00 metal : disable fast-math for some cpy kernels (#14460) a7417f55945eb7c7a5ea6807e66564b8066a4e50 eb3fa2913e0ace38912011fcb331624c679656f5 Romain Biessy romain.biessy@codeplay.com 2025-06-30T14:52:02+02:00 GitHub noreply@github.com 2025-06-30T14:52:02+02:00 ggml-cpu: sycl: Re-enable exp f16 (#14462) eb3fa2913e0ace38912011fcb331624c679656f5 c839a2da1a0d4275c3a98f70c3a7627487573a46 Diego Devesa slarengh@gmail.com 2025-06-30T03:43:15-07:00 GitHub noreply@github.com 2025-06-30T12:43:15+02:00 test-backend-ops : disable llama test (#14461) c839a2da1a0d4275c3a98f70c3a7627487573a46 e9b6350e61d592634263a14b3d77ecbf6c1fb096 xiaobing318 71554036+xiaobing318@users.noreply.github.com 2025-06-30T17:48:24+08:00 GitHub noreply@github.com 2025-06-30T12:48:24+03:00 cmake : Remove redundant include path in CMakeLists.txt (#14452) e9b6350e61d592634263a14b3d77ecbf6c1fb096 caf5681fcb47dfe9bafee94ef9aa8f669ac986c7 Vedran Miletić vedran@miletic.net 2025-06-30T10:17:18+02:00 GitHub noreply@github.com 2025-06-30T10:17:18+02:00 scripts : make the shell scripts cross-platform (#14341) caf5681fcb47dfe9bafee94ef9aa8f669ac986c7 83790b0e7e09ab17238b16452a33053a71dbdfad matteo matteo.serva@gmail.com 2025-06-29T20:02:53+02:00 GitHub noreply@github.com 2025-06-29T20:02:53+02:00 server : support jinja extra template kwargs (Qwen3 enable_thinking feature), from command line and from client (#13196) 83790b0e7e09ab17238b16452a33053a71dbdfad f47c1d7106e49062279bcc57fc1077c0db61e278 Renat rntk@users.noreply.github.com 2025-06-29T19:29:57+02:00 GitHub noreply@github.com 2025-06-29T19:29:57+02:00 server : fix appearance of the chats list context menu for Safari (#14322) f47c1d7106e49062279bcc57fc1077c0db61e278 a5d1fb6212298db1be1639db4c03adb2c522ee13 Akarshan Biswas akarshan@menlo.ai 2025-06-29T21:07:58+05:30 GitHub noreply@github.com 2025-06-29T21:07:58+05:30 SYCL: disable faulty fp16 exp kernel (#14395) a5d1fb6212298db1be1639db4c03adb2c522ee13 a0535ffa0d35fccfec3e1a0a3bfc9dbb6054d7c0 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-29T14:38:10+02:00 GitHub noreply@github.com 2025-06-29T14:38:10+02:00 ggml : fix unmerged GGML_FPxx_TO_FPxx refactoring (#14443) a0535ffa0d35fccfec3e1a0a3bfc9dbb6054d7c0 bd9c981d7226107f18deb8344c3301450311bb8b Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-29T11:04:10+02:00 GitHub noreply@github.com 2025-06-29T11:04:10+02:00 ggml : implement REGLU/GEGLU/SWIGLU ops (#14158) bd9c981d7226107f18deb8344c3301450311bb8b 27208bf657cfe7262791df473927225e48efe482 Jeff Bolz jbolz@nvidia.com 2025-06-29T02:43:36-05:00 GitHub noreply@github.com 2025-06-29T09:43:36+02:00 vulkan: Add fusion support for RMS_NORM+MUL (#14366) 27208bf657cfe7262791df473927225e48efe482 63a7bb3c7e1c6b0a92d03b0a594d3cd501d6ed3e Aman Gupta amangupta052@gmail.com 2025-06-29T01:30:53+08:00 GitHub noreply@github.com 2025-06-29T01:30:53+08:00 CUDA: add bf16 and f32 support to cublas_mul_mat_batched (#14361) 63a7bb3c7e1c6b0a92d03b0a594d3cd501d6ed3e 00d5282c7f2a0bb05bb315fb81ca3b0f42cf9f07 Jeff Bolz jbolz@nvidia.com 2025-06-28T10:36:40-05:00 GitHub noreply@github.com 2025-06-28T17:36:40+02:00 vulkan: handle noncontig in the final case of ggml_vk_get_cpy_pipeline (#14378) 00d5282c7f2a0bb05bb315fb81ca3b0f42cf9f07 566c16fcce44876a167c37f159085afe6f84b28c Jeff Bolz jbolz@nvidia.com 2025-06-28T10:17:09-05:00 GitHub noreply@github.com 2025-06-28T17:17:09+02:00 vulkan: lock accesses of pinned_memory vector (#14333) 566c16fcce44876a167c37f159085afe6f84b28c b25e92774e2fa4ee3820e458d5cf43f40190f8d2 Weizhao Ouyang weizhao.ouyang@arm.com 2025-06-28T22:08:21+08:00 GitHub noreply@github.com 2025-06-28T16:08:21+02:00 model : add support for ERNIE 4.5 0.3B model (#14408) b25e92774e2fa4ee3820e458d5cf43f40190f8d2 6609507a910aa7437aaa53fd999447de3947d998 Xinpeng Dou 15529241576@163.com 2025-06-28T17:35:41+08:00 GitHub noreply@github.com 2025-06-28T17:35:41+08:00 fix async_mode bug (#14432) 6609507a910aa7437aaa53fd999447de3947d998 ceb1bf5a34d5e66e28b23dcc7a3cd83fe1e27481 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-28T09:57:07+02:00 GitHub noreply@github.com 2025-06-28T09:57:07+02:00 ci : fix windows build and release (#14431) ceb1bf5a34d5e66e28b23dcc7a3cd83fe1e27481 72babea5dea56c8a8e8420ccf731b12a5cf37854 Jeff Bolz jbolz@nvidia.com 2025-06-27T22:35:30-05:00 GitHub noreply@github.com 2025-06-27T22:35:30-05:00 vulkan: Fix GGML_VULKAN_SHADER_DEBUG_INFO (#14427) 72babea5dea56c8a8e8420ccf731b12a5cf37854 43678060c1f4cfab2f899466fd615e358677f807 Georgi Gerganov ggerganov@gmail.com 2025-06-27T21:42:02+03:00 GitHub noreply@github.com 2025-06-27T21:42:02+03:00 graph : make llm_graph_context destructor virtual (#14410) 43678060c1f4cfab2f899466fd615e358677f807 8d94219a4a7f2da72ee542019ca01f36af93d1d6 Georgi Gerganov ggerganov@gmail.com 2025-06-27T17:55:45+03:00 GitHub noreply@github.com 2025-06-27T17:55:45+03:00 recurrent : call balloc split_reset() in init_batch() (#14414) 8d94219a4a7f2da72ee542019ca01f36af93d1d6 f667f1e6244e1f420512fa66692b7096ff17f366 Radoslav Gerganov rgerganov@gmail.com 2025-06-27T16:41:40+03:00 GitHub noreply@github.com 2025-06-27T16:41:40+03:00 ggml : add ggml_set_rows (#14274) f667f1e6244e1f420512fa66692b7096ff17f366 8846aace4934ad29651ea61b8c7e3f6b0556e3d2 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-27T10:42:19+02:00 GitHub noreply@github.com 2025-06-27T10:42:19+02:00 convert : fix broken sentencepiece vocab (#14416) 8846aace4934ad29651ea61b8c7e3f6b0556e3d2 a01047b041aa04aeea351933658433ed004516ab Xuan-Son Nguyen son@huggingface.co 2025-06-26T19:34:02+02:00 GitHub noreply@github.com 2025-06-26T20:34:02+03:00 model : gemma3n text-only (#14400) a01047b041aa04aeea351933658433ed004516ab b25346221dadb9101aa9dda55431dde4d3596943 bandoti 141645996+bandoti@users.noreply.github.com 2025-06-26T13:46:53-03:00 GitHub noreply@github.com 2025-06-26T13:46:53-03:00 cmake: regen vulkan shaders when shaders-gen sources change (#14398) b25346221dadb9101aa9dda55431dde4d3596943 e8215dbb96b8fb94a24c29cdd228166fb972dbfc Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-26T15:01:14+02:00 GitHub noreply@github.com 2025-06-26T15:01:14+02:00 llama : return mistral-v7-tekken as default template only (#14390) e8215dbb96b8fb94a24c29cdd228166fb972dbfc 5783ae43599400b723b5da0569c1f848419ff3c7 Georgi Gerganov ggerganov@gmail.com 2025-06-26T15:51:19+03:00 GitHub noreply@github.com 2025-06-26T15:51:19+03:00 metal : add special-case mat-vec mul for ne00 == 4 (#14385) 5783ae43599400b723b5da0569c1f848419ff3c7 bf5bcd0b857db420235e03639f0a5f218a7f8cf8 Georgi Gerganov ggerganov@gmail.com 2025-06-26T15:50:15+03:00 GitHub noreply@github.com 2025-06-26T15:50:15+03:00 metal : batch rows copy in a single threadgroup (#14384) bf5bcd0b857db420235e03639f0a5f218a7f8cf8 716301d1b03c31875ec3b24526c48c8b1bd0fd8c Aaron Teo aaron.teo1@ibm.com 2025-06-26T18:41:41+08:00 GitHub noreply@github.com 2025-06-26T12:41:41+02:00 docs: update s390x documentation + add faq (#14389) ba67d6c03fed3193987a4ef13050e6e2a4451df4 8f52c6f8abb2ed9ad967cc0bfb90acb073011193 Yunzhe Jia yunzhe@calculet.tech 2025-06-26T17:42:32+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-06-26T17:42:32+08:00 llama-calrt: infer-op: copy data from output buffer to dst-tensor.data 8f52c6f8abb2ed9ad967cc0bfb90acb073011193 d2f325130c4ed77a3dcff2c23a926587fb08e2cc Yunzhe Jia yunzhe@calculet.tech 2025-06-26T15:10:45+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-06-26T15:10:45+08:00 Squashed commit of the following: 716301d1b03c31875ec3b24526c48c8b1bd0fd8c 60ef23d6c14d325d83eae5752e5de39ad268e9b0 R0CKSTAR yeahdongcn@gmail.com 2025-06-26T12:11:59+08:00 GitHub noreply@github.com 2025-06-26T12:11:59+08:00 musa: enable fp16 mma (all) and cublas on qy2 (#13842) 60ef23d6c14d325d83eae5752e5de39ad268e9b0 b193d5306912a2adae0fde7481819f6ee0941bc6 Aaron Teo aaron.teo1@ibm.com 2025-06-26T05:49:04+08:00 GitHub noreply@github.com 2025-06-25T23:49:04+02:00 ggml-cpu: enable IBM NNPA Vector Intrinsics (#14317) b193d5306912a2adae0fde7481819f6ee0941bc6 2bf9d539dd158345e3a3b096e16474af535265b4 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-25T23:26:51+02:00 GitHub noreply@github.com 2025-06-25T23:26:51+02:00 ggml : do not output unprintable characters on GGUF load failure (#14381) 2bf9d539dd158345e3a3b096e16474af535265b4 73e53dc834c0a2336cd104473af6897197b96277 Anton Mitkov anton_b_mitkov@abv.bg 2025-06-25T17:09:55+01:00 GitHub noreply@github.com 2025-06-25T18:09:55+02:00 sycl: GGML_SYCL_DISABLE_OPT on by default for all Intel Devices (#13973) 73e53dc834c0a2336cd104473af6897197b96277 62af464227dafa1c55e0535bcb24346326748f46 lhez quic_lih@quicinc.com 2025-06-24T11:46:25-07:00 GitHub noreply@github.com 2025-06-24T11:46:25-07:00 opencl: ref count `ggml_backend_opencl_context` and refactor profiling (#14254) 62af464227dafa1c55e0535bcb24346326748f46 c148cf1946275952a79ad50b6199725f12a70411 Georgi Gerganov ggerganov@gmail.com 2025-06-24T18:26:30+03:00 GitHub noreply@github.com 2025-06-24T18:26:30+03:00 batch : fix check for empty sequences in memory (#14364) c148cf1946275952a79ad50b6199725f12a70411 1b809cee225222094a0ff5be8467240487ce4ae4 Mathieu Baudier mbaudier@argeo.org 2025-06-24T15:05:31+02:00 GitHub noreply@github.com 2025-06-24T15:05:31+02:00 cmake : use LLAMA_BUILD_NUMBER when defining LLAMA_INSTALL_VERSION (#14362) 1b809cee225222094a0ff5be8467240487ce4ae4 abf241045d09cad70dc797b0fba393ad09ee2cbe Nigel Bosch pnigelb@gmail.com 2025-06-24T08:59:11Z GitHub noreply@github.com 2025-06-24T10:59:11+02:00 server : move no API key doc to /health (#14352) abf241045d09cad70dc797b0fba393ad09ee2cbe 901e20bbe571fbde48d13eb188f4e7cdc7562fb6 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-24T09:31:00+02:00 GitHub noreply@github.com 2025-06-24T09:31:00+02:00 main : honor --verbose-prompt on interactive prompts (#14350) 901e20bbe571fbde48d13eb188f4e7cdc7562fb6 0142961a2e67909e33cdf410274b56c08c5dce7a Bartowski 3266127+bartowski1182@users.noreply.github.com 2025-06-24T02:17:58-04:00 GitHub noreply@github.com 2025-06-24T09:17:58+03:00 jinja : Add Mistral-Small-3.2-24B-Instruct-2506.jinja (#14349) 0142961a2e67909e33cdf410274b56c08c5dce7a ce82bd0117bd3598300b3a089d13d401b90279c7 uvos philipp@uvos.xyz 2025-06-24T01:12:56+02:00 GitHub noreply@github.com 2025-06-24T01:12:56+02:00 CUDA/HIP: optimize mmv paths taken for HIP devices (#14324) ce82bd0117bd3598300b3a089d13d401b90279c7 bf2a99e3cb06a14dce2a586450d012dc31f922ae bandoti 141645996+bandoti@users.noreply.github.com 2025-06-23T15:30:51-03:00 GitHub noreply@github.com 2025-06-23T15:30:51-03:00 ci: add workflow for relocatable cmake package (#14346) bf2a99e3cb06a14dce2a586450d012dc31f922ae 72c6bc3f3d0cf3bf160dddf1b803fed52bcbb0a3 Jeff Bolz jbolz@nvidia.com 2025-06-23T08:44:48-05:00 GitHub noreply@github.com 2025-06-23T15:44:48+02:00 vulkan: update windows SDK in release.yml (#14344) 72c6bc3f3d0cf3bf160dddf1b803fed52bcbb0a3 defe2158dd5e250b4ef53994057a4ec03131a263 Molly Sophia mollysophia379@gmail.com 2025-06-23T19:56:19+08:00 GitHub noreply@github.com 2025-06-23T19:56:19+08:00 llama : better rwkv chat template and add missing `inputs.use_jinja` setting (#14336) defe2158dd5e250b4ef53994057a4ec03131a263 7b50d589a863c7631135c1226f6eab65cb406212 Johannes Gäßler johannesg@5d6.de 2025-06-23T13:11:31+02:00 GitHub noreply@github.com 2025-06-23T13:11:31+02:00 CUDA: mul_mat_v support for batch sizes > 1 (#14262) 7b50d589a863c7631135c1226f6eab65cb406212 3a9457df962b5883f2773f1c295e8c19df60d89f Georgi Gerganov ggerganov@gmail.com 2025-06-23T12:27:35+03:00 GitHub noreply@github.com 2025-06-23T12:27:35+03:00 kv-cells : fix tracking of seq_pos (#14339) 3a9457df962b5883f2773f1c295e8c19df60d89f fa4a9f2a1ccda2573189a9d4995bdf0bceb41156 Jeff Bolz jbolz@nvidia.com 2025-06-23T03:19:24-05:00 GitHub noreply@github.com 2025-06-23T10:19:24+02:00 vulkan: update windows SDK in CI (#14334) fa4a9f2a1ccda2573189a9d4995bdf0bceb41156 238005c2dc67426cf678baa2d54c881701693288 Ed Addario 29247825+EAddario@users.noreply.github.com 2025-06-22T22:16:26+01:00 GitHub noreply@github.com 2025-06-22T23:16:26+02:00 quantize : handle user-defined pruning of whole layers (blocks) (#13037) 238005c2dc67426cf678baa2d54c881701693288 66aba7aca9a245d71f1ddf02c7a97223529752a8 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-22T19:46:17+02:00 GitHub noreply@github.com 2025-06-22T19:46:17+02:00 gguf-py : fix SpecialVocab parsing when post_processor is null (#14330) 66aba7aca9a245d71f1ddf02c7a97223529752a8 f1f5e82df6222dcbaca6396c0de44df259a1694f Ruikai Peng retr0@retr0.blog 2025-06-23T01:28:06+08:00 GitHub noreply@github.com 2025-06-23T01:28:06+08:00 run : avoid double tokenization (#14327) f1f5e82df6222dcbaca6396c0de44df259a1694f af3373f1adfca56119f3e4de0e6a0a8df8edf3d9 Georgi Gerganov ggerganov@gmail.com 2025-06-22T20:10:07+03:00 GitHub noreply@github.com 2025-06-22T20:10:07+03:00 examples : fix is_first logic for tokenization (#14329) af3373f1adfca56119f3e4de0e6a0a8df8edf3d9 5d5c066de8a3d2cb32f04c4d5ad1560945f30bf3 uvos philipp@uvos.xyz 2025-06-22T16:51:23+02:00 GitHub noreply@github.com 2025-06-22T16:51:23+02:00 HIP: enable vec fattn on RDNA4 (#14323) 5d5c066de8a3d2cb32f04c4d5ad1560945f30bf3 40bfa04c95c19fb42bafd4e21b5c2a7771846801 yuiseki yuiseki@gmail.com 2025-06-22T21:44:57+09:00 GitHub noreply@github.com 2025-06-22T14:44:57+02:00 mtmd : fix Pixtral OOM with large images by capping image_size to 1024 (#14326) 40bfa04c95c19fb42bafd4e21b5c2a7771846801 aa064b2eb7f7779db5e094a9d8d66d5033557de2 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-22T07:37:43+02:00 GitHub noreply@github.com 2025-06-22T08:37:43+03:00 common : use std::string_view now that we target c++17 (#14319) aa064b2eb7f7779db5e094a9d8d66d5033557de2 aa0ef5c578eef4c2adc7be1282f21bab5f3e8d26 Aman Gupta amangupta052@gmail.com 2025-06-22T12:39:54+08:00 GitHub noreply@github.com 2025-06-22T12:39:54+08:00 CUDA: add mean operation (#14313) aa0ef5c578eef4c2adc7be1282f21bab5f3e8d26 bb16041caef45cd4348cd6f84906b5dfec7a1f6a Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-21T18:12:05+02:00 GitHub noreply@github.com 2025-06-21T18:12:05+02:00 gguf-py : fix Qwen3-Embedding eos token (#14314) bb16041caef45cd4348cd6f84906b5dfec7a1f6a 58cba76a9aab728717509d62b67c13afd8dc227a Markus Tavenrath mtavenrath@users.noreply.github.com 2025-06-21T08:17:12+02:00 GitHub noreply@github.com 2025-06-21T08:17:12+02:00 Add support for VK_EXT_debug_utils to add labels to Vulkan objects. (#13792) 58cba76a9aab728717509d62b67c13afd8dc227a 67ae5312e255ae97852a4a216e2245580bfafd72 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-21T07:33:21+02:00 GitHub noreply@github.com 2025-06-21T07:33:21+02:00 gguf-py : fix TemplateProcessing pair when bos/eos is missing (#14312) 67ae5312e255ae97852a4a216e2245580bfafd72 692e3cdd0a069ab56411b64506a67537d767683e Georgi Gerganov ggerganov@gmail.com 2025-06-21T08:04:18+03:00 GitHub noreply@github.com 2025-06-21T08:04:18+03:00 metal : fix thread-safety (#14300) 692e3cdd0a069ab56411b64506a67537d767683e b23fa0b3f40165ca3aae8ad4ee756e72f9a130dd Georgi Gerganov ggerganov@gmail.com 2025-06-21T08:03:46+03:00 GitHub noreply@github.com 2025-06-21T08:03:46+03:00 memory : rename interface to llama_memory_context_i (#14296) b23fa0b3f40165ca3aae8ad4ee756e72f9a130dd 06cbedfca1587473df9b537f1dd4d6bfa2e3de13 Daniel Han danielhanchen@gmail.com 2025-06-20T21:32:01-07:00 GitHub noreply@github.com 2025-06-21T06:32:01+02:00 convert : fix Llama 4 conversion (#14311) 06cbedfca1587473df9b537f1dd4d6bfa2e3de13 b7147673f26c7ceb926a43c4734002bba291bcb8 Georgi Gerganov ggerganov@gmail.com 2025-06-20T20:50:24+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-20T21:02:47+03:00 sync : ggml b7147673f26c7ceb926a43c4734002bba291bcb8 d860dd99a4178f58d1d1fa64eebc2aabc95392a7 Acly aclysia@gmail.com 2025-06-18T13:34:50+02:00 Georgi Gerganov ggerganov@gmail.com 2025-06-20T21:02:47+03:00 Add `ggml_roll` (ggml/1274) d860dd99a4178f58d1d1fa64eebc2aabc95392a7 c959f462a0b4d42eaf930ffd72df0e435c97d5d5 David Chiu david20571015@gmail.com 2025-06-21T01:43:35+08:00 GitHub noreply@github.com 2025-06-20T19:43:35+02:00 docs : fix the link to llama.h (#14293) c959f462a0b4d42eaf930ffd72df0e435c97d5d5 22015b2092e291022ea3cfedc6aeb8f2643807da Aman Gupta amangupta052@gmail.com 2025-06-20T22:48:24+08:00 GitHub noreply@github.com 2025-06-20T22:48:24+08:00 CUDA: add conv_2d_transpose (#14287) 22015b2092e291022ea3cfedc6aeb8f2643807da dd6e6d0b6a4bbe3ebfc931d1eb14db2f2b1d70af Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-20T16:37:44+02:00 GitHub noreply@github.com 2025-06-20T16:37:44+02:00 lint : remove trailing whitepace (#14304) dd6e6d0b6a4bbe3ebfc931d1eb14db2f2b1d70af 8308f98c7fb778e54bf75538f5234d8bd20915e9 Ruikai Peng retr0@retr0.blog 2025-06-20T22:13:06+08:00 GitHub noreply@github.com 2025-06-20T07:13:06-07:00 vocab : prevent tokenizer overflow (#14301) 8308f98c7fb778e54bf75538f5234d8bd20915e9 6369be07359d03723f38c0a4a014ff0f698a0738 Nicolò Scipione nicolo.scipione@codeplay.com 2025-06-20T15:07:21+02:00 GitHub noreply@github.com 2025-06-20T15:07:21+02:00 sycl: add usage of enqueue_functions extension (#14244) 6369be07359d03723f38c0a4a014ff0f698a0738 88fc854b4bd2e3caf10e705e6afcbbca136f0a3c Christian Kastner ckk@kvr.at 2025-06-20T12:17:32Z GitHub noreply@github.com 2025-06-20T14:17:32+02:00 Implement GGML_CPU_ALL_VARIANTS for PowerPC (#14286) 88fc854b4bd2e3caf10e705e6afcbbca136f0a3c e28c1b93fd7d3f8faf9551d962e8a65fe2122e38 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-20T14:04:09+02:00 GitHub noreply@github.com 2025-06-20T14:04:09+02:00 llama : improve sep token handling (#14272) e28c1b93fd7d3f8faf9551d962e8a65fe2122e38 d27b3ca1758dfb1718e333d497ef4b68ad109bc2 Diego Devesa slarengh@gmail.com 2025-06-20T04:57:36-07:00 GitHub noreply@github.com 2025-06-20T13:57:36+02:00 cuda : synchronize graph capture and cublas handle destruction (#14288) d27b3ca1758dfb1718e333d497ef4b68ad109bc2 9230dbe2c757e2d5071329095727d0fa9d4b85c4 Georgi Gerganov ggerganov@gmail.com 2025-06-20T11:19:15+03:00 GitHub noreply@github.com 2025-06-20T11:19:15+03:00 ggml : fix repack work size for mul_mat_id (#14292) 9230dbe2c757e2d5071329095727d0fa9d4b85c4 812939a9e90f99d1bd5bb1bc6b99d12600671d50 Charles Xu charles.xu@arm.com 2025-06-20T09:51:01+02:00 GitHub noreply@github.com 2025-06-20T10:51:01+03:00 ggml: Update KleidiAI to v1.9.0 (#14277) 812939a9e90f99d1bd5bb1bc6b99d12600671d50 4c9fdfbe1580a66fd7d77c77418ce2c606a29fdd Georgi Gerganov ggerganov@gmail.com 2025-06-20T10:50:27+03:00 GitHub noreply@github.com 2025-06-20T10:50:27+03:00 model : more uniform output id handling (#14275) 4c9fdfbe1580a66fd7d77c77418ce2c606a29fdd 9eaa51e7f08593f123f00136591179a8f5956ecd Georgi Gerganov ggerganov@gmail.com 2025-06-20T10:14:14+03:00 GitHub noreply@github.com 2025-06-20T10:14:14+03:00 ubatch : new splitting logic (#14217) d2f325130c4ed77a3dcff2c23a926587fb08e2cc 6391ee7ffd97e723dfbdcc32f001e1b95a63cba6 Yunzhe Jia yunzhe@calculet.tech 2025-06-17T11:53:33+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-06-20T09:57:18+08:00 calrt: add input_token in graph; input_token will be copy to calrt_in_buffer 9eaa51e7f08593f123f00136591179a8f5956ecd 8f71d0f3e86ccbba059350058af8758cafed73e6 Aman Gupta amangupta052@gmail.com 2025-06-20T09:50:24+08:00 GitHub noreply@github.com 2025-06-20T09:50:24+08:00 CUDA: add conv_2d_dw (#14265) 8f71d0f3e86ccbba059350058af8758cafed73e6 381174bbdaf10d6a80dc2099f284b20544d86962 Diego Devesa slarengh@gmail.com 2025-06-19T12:24:14-07:00 GitHub noreply@github.com 2025-06-19T21:24:14+02:00 ggml-cpu : remove unnecesary arm feature detection (#14281) 381174bbdaf10d6a80dc2099f284b20544d86962 d67341dc18fc5cc63362880ab2f8f9ecfc7932e7 Alex Trotta 44127594+Ahajha@users.noreply.github.com 2025-06-19T09:56:12-04:00 GitHub noreply@github.com 2025-06-19T15:56:12+02:00 gguf-py : make sentencepiece optional (#14200) d67341dc18fc5cc63362880ab2f8f9ecfc7932e7 456af35eb70177b8dd5779b6d4c21bb020f9cebd aa956 aa956@users.noreply.github.com 2025-06-19T16:01:03+03:00 GitHub noreply@github.com 2025-06-19T16:01:03+03:00 server : add server parameters for draft model cache type (#13782) 456af35eb70177b8dd5779b6d4c21bb020f9cebd 600e3e9b50c1f0c9fc4a70356241fd87f00e8e14 fanyang fanyang89@outlook.com 2025-06-19T20:49:48+08:00 GitHub noreply@github.com 2025-06-19T14:49:48+02:00 build : suppress gcc15 compile warnings (#14261) 600e3e9b50c1f0c9fc4a70356241fd87f00e8e14 fffcce535ebbdc25f81966a15b758658788e7466 Anton Mitkov anton_b_mitkov@abv.bg 2025-06-19T11:40:21+01:00 GitHub noreply@github.com 2025-06-19T11:40:21+01:00 sycl: Cleanup codepaths in Get Rows in sycl backend (#14215) fffcce535ebbdc25f81966a15b758658788e7466 5fc7856815920f828f9e90cb759ac82c7f0c1ea5 bashayer hijji bashayer.hijji@gmail.com 2025-06-19T13:24:12+03:00 GitHub noreply@github.com 2025-06-19T12:24:12+02:00 llama-bench : add --no-warmup flag (#14224) (#14270) 5fc7856815920f828f9e90cb759ac82c7f0c1ea5 faed5a5f5dde4d816adecdccb4f4682a04126f92 pqnet 119850+pqnet@users.noreply.github.com 2025-06-19T12:21:40+02:00 GitHub noreply@github.com 2025-06-19T12:21:40+02:00 convert : fix remote option in Windows (#14100) faed5a5f5dde4d816adecdccb4f4682a04126f92 10bb545c5b54175ed9874ad8d187effa2bcb4b5f Aaron Teo aaron.teo1@ibm.com 2025-06-19T17:48:54+08:00 GitHub noreply@github.com 2025-06-19T11:48:54+02:00 llamafile : support s390x SIMD instruction set (#14273) 10bb545c5b54175ed9874ad8d187effa2bcb4b5f edc4a29effe716956fdfd2bc5b9cba2a6d8492f8 0cc4m picard12@live.de 2025-06-19T09:15:42+02:00 GitHub noreply@github.com 2025-06-19T09:15:42+02:00 Vulkan: Set device max size for host memory to avoid OOM warning and fallback to CPU buffer (#14249) edc4a29effe716956fdfd2bc5b9cba2a6d8492f8 ed3290ab34493fd3d2e0f925f816a401da4f2dfd Gabe Goodhart gabe.l.hart@gmail.com 2025-06-19T00:08:14-05:00 GitHub noreply@github.com 2025-06-19T08:08:14+03:00 memory : Hybrid recurrent cache (#13979) ed3290ab34493fd3d2e0f925f816a401da4f2dfd 8d947136546773f6410756f37fcc5d3e65b8135d Georgi Gerganov ggerganov@gmail.com 2025-06-19T08:05:21+03:00 GitHub noreply@github.com 2025-06-19T08:05:21+03:00 metal : add mean kernel (#14267) 8d947136546773f6410756f37fcc5d3e65b8135d 50d2227953ca9024f04255b4f116d06fcc0db74c Aaron Teo aaron.teo1@ibm.com 2025-06-19T01:10:26+08:00 GitHub noreply@github.com 2025-06-18T18:10:26+01:00 docs: add s390x build documentation (#14264) 50d2227953ca9024f04255b4f116d06fcc0db74c 6231c5cd6d49d61511f328b5f43322407df90a91 Aaron Teo aaron.teo1@ibm.com 2025-06-19T01:10:08+08:00 GitHub noreply@github.com 2025-06-18T18:10:08+01:00 ggml-cpu: reduce asm calls for hsum (#14037) 6231c5cd6d49d61511f328b5f43322407df90a91 ef035803eb9dbc306ea9a8ff82e30af12b567cf7 Aaron Teo aaron.teo1@ibm.com 2025-06-19T01:06:49+08:00 GitHub noreply@github.com 2025-06-18T18:06:49+01:00 ggml-cpu: fix uncaught underscore terminators (#14023) ef035803eb9dbc306ea9a8ff82e30af12b567cf7 413977de32e90712ecec84d0b9c738847da8dc02 Charles Xu charles.xu@arm.com 2025-06-18T13:40:07+02:00 GitHub noreply@github.com 2025-06-18T12:40:07+01:00 ggml: Add Apple support for GGML_CPU_ALL_VARIANTS (#14258) 413977de32e90712ecec84d0b9c738847da8dc02 95402553a5effc61ddc9e29c7bcb56f71311dd4a Xuan-Son Nguyen son@huggingface.co 2025-06-18T10:43:57+02:00 GitHub noreply@github.com 2025-06-18T10:43:57+02:00 mtmd : refactor llava-uhd preprocessing logic (#14247) 95402553a5effc61ddc9e29c7bcb56f71311dd4a 3865cff4f5b84c16119590efd6ce537789a27715 Xuan-Son Nguyen son@huggingface.co 2025-06-18T09:58:43+02:00 GitHub noreply@github.com 2025-06-18T09:58:43+02:00 llama-chat : fix multiple system message for gemma, orion (#14246) 3865cff4f5b84c16119590efd6ce537789a27715 d03172cc797b6adbbc00bfc09caf614fb0f895a0 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-18T09:52:07+02:00 GitHub noreply@github.com 2025-06-18T09:52:07+02:00 convert : fix null head_dim AutoConfig regression (#14248) d03172cc797b6adbbc00bfc09caf614fb0f895a0 dd8e59f4435342eda93c5b0cf4109e21c9c7d0eb Georgi Gerganov ggerganov@gmail.com 2025-06-18T09:58:23+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-18T09:59:21+03:00 sync : ggml dd8e59f4435342eda93c5b0cf4109e21c9c7d0eb bbe98d27840453c8787d18470963530fdc27d89f Daniel Bevenius daniel.bevenius@gmail.com 2025-06-13T15:06:42+02:00 Georgi Gerganov ggerganov@gmail.com 2025-06-18T09:59:21+03:00 ggml : disable warnings for tests when using MSVC (ggml/1273) bbe98d27840453c8787d18470963530fdc27d89f c2056ed6d461e6d5432f04f221e221ab795dc652 Daniel Bevenius daniel.bevenius@gmail.com 2025-06-13T09:05:44+02:00 Georgi Gerganov ggerganov@gmail.com 2025-06-18T09:59:21+03:00 ggml : remove unused ggml_context_container (ggml/1272) c2056ed6d461e6d5432f04f221e221ab795dc652 c46503014db0d63fa7b1b28c58adfb51054e2dec Daniel Bevenius daniel.bevenius@gmail.com 2025-06-12T12:27:09+02:00 Georgi Gerganov ggerganov@gmail.com 2025-06-18T09:59:21+03:00 examples : include examples in msvc disable warn (ggml/1270) c46503014db0d63fa7b1b28c58adfb51054e2dec 860a9e4eeff3eb2e7bd1cc38f65787cc6c8177af bandoti 141645996+bandoti@users.noreply.github.com 2025-06-17T17:33:25-03:00 GitHub noreply@github.com 2025-06-17T22:33:25+02:00 cmake: remove shader-gen step-targets from ggml-vulkan (#14226) 860a9e4eeff3eb2e7bd1cc38f65787cc6c8177af fe9d60e74a6cb71bcaed2029377bfa2872b4abb0 xctan xc-tan@outlook.com 2025-06-17T17:58:32+08:00 GitHub noreply@github.com 2025-06-17T12:58:32+03:00 ggml-cpu : remove the weak alias trick (#14221) fe9d60e74a6cb71bcaed2029377bfa2872b4abb0 e434e69183fd9e1031f4445002083178c331a28b R0CKSTAR yeahdongcn@gmail.com 2025-06-17T17:48:08+08:00 GitHub noreply@github.com 2025-06-17T17:48:08+08:00 musa: fix build warning (unused variable) (#14231) e434e69183fd9e1031f4445002083178c331a28b 89fea80d298184d1cd93564f48e060d9f541f4b4 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-16T21:58:42+02:00 GitHub noreply@github.com 2025-06-16T21:58:42+02:00 common : suggest --jinja when autodetection fails (#14222) 89fea80d298184d1cd93564f48e060d9f541f4b4 6adc3c3ebc029af058ac950a8e2a825fdf18ecc6 Georgi Gerganov ggerganov@gmail.com 2025-06-16T22:33:27+03:00 GitHub noreply@github.com 2025-06-16T22:33:27+03:00 server : fix incorrect usage of llama_get_embeddings() (#14225) 6adc3c3ebc029af058ac950a8e2a825fdf18ecc6 0dbcabde8c006d5cf781ca0fe071c41559572a72 Diego Devesa slarengh@gmail.com 2025-06-16T08:11:43-07:00 GitHub noreply@github.com 2025-06-16T08:11:43-07:00 llama : add thread safety test (#14035) 0dbcabde8c006d5cf781ca0fe071c41559572a72 ad590be98c83217fcf1a101d78d9ab389fd5dc0b bandoti 141645996+bandoti@users.noreply.github.com 2025-06-16T10:32:13-03:00 GitHub noreply@github.com 2025-06-16T10:32:13-03:00 cmake: clean up external project logic for vulkan-shaders-gen (#14179) ad590be98c83217fcf1a101d78d9ab389fd5dc0b 7d6d91babfa129906b39c9099eca4234c44f4f1e Đinh Trọng Huy 77562200+huydt84@users.noreply.github.com 2025-06-16T21:53:41+09:00 GitHub noreply@github.com 2025-06-16T14:53:41+02:00 model : add NeoBERT (#14164) 7d6d91babfa129906b39c9099eca4234c44f4f1e d3e64b9f490cee41b7b9aa275dae2f6568ae3054 uvos philipp@uvos.xyz 2025-06-16T13:47:38+02:00 GitHub noreply@github.com 2025-06-16T13:47:38+02:00 HIP: disable rocwmma on gfx12 by default until rocm 7.0 (#14202) d3e64b9f490cee41b7b9aa275dae2f6568ae3054 3ba0d843c6bd3faea5cf5e53dc7f3c82be20bffb Georgi Gerganov ggerganov@gmail.com 2025-06-16T14:14:00+03:00 GitHub noreply@github.com 2025-06-16T14:14:00+03:00 llama : rework embeddings logic (#14208) 3ba0d843c6bd3faea5cf5e53dc7f3c82be20bffb 0bf49eb668bb95b50e41583e22aaf60ddade1fbe Charles Xu charles.xu@arm.com 2025-06-16T11:47:57+02:00 GitHub noreply@github.com 2025-06-16T11:47:57+02:00 ggml: Add Android support for GGML_CPU_ALL_VARIANTS (#14206) 0bf49eb668bb95b50e41583e22aaf60ddade1fbe 4ad243677bca6c97f14dbc187b2116b51fcb7ffd Bartowski 3266127+bartowski1182@users.noreply.github.com 2025-06-16T09:16:06+01:00 GitHub noreply@github.com 2025-06-16T10:16:06+02:00 convert : remove arcee change in convert_hf_to_gguf_update.py (#14207) 4ad243677bca6c97f14dbc187b2116b51fcb7ffd c89c2d1ab94b11845240b7d3313c87691ea18d88 Đinh Trọng Huy 77562200+huydt84@users.noreply.github.com 2025-06-16T16:20:59+09:00 GitHub noreply@github.com 2025-06-16T09:20:59+02:00 gguf-py : allow key override when adding value to GGUFWriter (#14194) c89c2d1ab94b11845240b7d3313c87691ea18d88 3555b3004ba7687be3d734acade52a3345758aa4 Jeff Bolz jbolz@nvidia.com 2025-06-16T00:21:08-06:00 GitHub noreply@github.com 2025-06-16T08:21:08+02:00 vulkan: mutex around vkQueueSubmit (#14127) 3555b3004ba7687be3d734acade52a3345758aa4 d7da8dc83a03b30e1ec10317080082ea76840c38 xctan xc-tan@outlook.com 2025-06-16T13:54:15+08:00 GitHub noreply@github.com 2025-06-16T13:54:15+08:00 ggml-cpu : rework weak alias on apple targets (#14146) d7da8dc83a03b30e1ec10317080082ea76840c38 cd355eda7df1898d25d433b4bdaa4b4b479e0bad Bartowski 3266127+bartowski1182@users.noreply.github.com 2025-06-16T00:04:06+01:00 GitHub noreply@github.com 2025-06-16T01:04:06+02:00 model : Add support for Arcee AI's upcoming AFM model (#14185) cd355eda7df1898d25d433b4bdaa4b4b479e0bad 30e5b01de2a0bcddc7c063c8ef0802703a958417 Eric Curtin ecurtin@redhat.com 2025-06-15T23:36:22+02:00 GitHub noreply@github.com 2025-06-15T23:36:22+02:00 server : When listening on a unix domain socket don't print http:// and port (#14180) 30e5b01de2a0bcddc7c063c8ef0802703a958417 e54b394082de242be4ee2e692b11fcc8d4eba371 Ed Addario 29247825+EAddario@users.noreply.github.com 2025-06-15T17:53:45+01:00 GitHub noreply@github.com 2025-06-15T18:53:45+02:00 quantize : change int to unsigned int for KV overrides (#14197) e54b394082de242be4ee2e692b11fcc8d4eba371 2c2caa444341d99c87ff153f142c2d4762a776a2 uvos philipp@uvos.xyz 2025-06-15T17:30:13+02:00 GitHub noreply@github.com 2025-06-15T17:30:13+02:00 CUDA/HIP: fix ssm_scan on devices where warp size is not 32 (#14196) 2c2caa444341d99c87ff153f142c2d4762a776a2 5fce5f948df8f189a5401a8ecaa9753106e75abb uvos philipp@uvos.xyz 2025-06-15T15:45:27+02:00 GitHub noreply@github.com 2025-06-15T15:45:27+02:00 HIP: Replace usage of depricated preprocessor macro __AMDGCN_WAVEFRONT_SIZE__ (#14183) 5fce5f948df8f189a5401a8ecaa9753106e75abb 9ae4143bc6ecb4c2f0f0301578f619f6c201b857 Georgi Gerganov ggerganov@gmail.com 2025-06-15T10:52:11+03:00 GitHub noreply@github.com 2025-06-15T10:52:11+03:00 kv-cache : fix use-after-move of defrag info (#14189) 9ae4143bc6ecb4c2f0f0301578f619f6c201b857 c311ac664d68d10781a3e7b9f02d9d9520837d80 Mikko Juola mikjuo@gmail.com 2025-06-15T00:52:06-07:00 GitHub noreply@github.com 2025-06-15T09:52:06+02:00 model : add dots.llm1 architecture support (#14044) (#14118) c311ac664d68d10781a3e7b9f02d9d9520837d80 b9912ac570de8945ae9383c9ca8291027bf287dd Georgi Gerganov ggerganov@gmail.com 2025-06-15T10:08:58+03:00 GitHub noreply@github.com 2025-06-15T10:08:58+03:00 cparams : rename LLAMA_MAX_PARALLEL_SEQUENCES to LLAMA_MAX_SEQ (#14188) b9912ac570de8945ae9383c9ca8291027bf287dd 00ba7726100d7e1941d9f5a06f56a7559945b33c Georgi Gerganov ggerganov@gmail.com 2025-06-15T09:18:37+03:00 GitHub noreply@github.com 2025-06-15T09:18:37+03:00 batch : auto-gen positions + verify multi-sequence input (#14177) 00ba7726100d7e1941d9f5a06f56a7559945b33c 3cb203c89f60483e349f841684173446ed23c28f Pepijn de Vos me@pepijndevos.nl 2025-06-15T08:06:37+02:00 GitHub noreply@github.com 2025-06-15T08:06:37+02:00 docs : remove WIP since PR has been merged (#13912) 3cb203c89f60483e349f841684173446ed23c28f 2e42be42bd6bf1dcc643d6ac4e77419bfe5dd24f Piotr piotr.stankiewicz@docker.com 2025-06-14T18:25:15+02:00 GitHub noreply@github.com 2025-06-14T17:25:15+01:00 llama-chat : Do not throw when tool parsing fails (#14012) 2e42be42bd6bf1dcc643d6ac4e77419bfe5dd24f fb85a288d72abbd5e5daa8de96e6f8bfa7b5ab46 Aman Gupta amangupta052@gmail.com 2025-06-14T16:34:20+08:00 GitHub noreply@github.com 2025-06-14T10:34:20+02:00 compare-llama-bench: add option to plot (#14169) fb85a288d72abbd5e5daa8de96e6f8bfa7b5ab46 40643edb86eb10b471b0f57d4f3f7eb0e06a0df7 Georgi Gerganov ggerganov@gmail.com 2025-06-13T20:03:05+03:00 GitHub noreply@github.com 2025-06-13T20:03:05+03:00 vocab : fix build (#14175) 40643edb86eb10b471b0f57d4f3f7eb0e06a0df7 3cfbbdb44e08fd19429fed6cc85b982a91f0efd5 Svetlozar Georgiev 55534064+sgeor255@users.noreply.github.com 2025-06-13T17:32:56+01:00 GitHub noreply@github.com 2025-06-13T18:32:56+02:00 sycl: fix docker image (#14144) 3cfbbdb44e08fd19429fed6cc85b982a91f0efd5 80709b70a2f87c13ccaf1480b799393109996789 Guy Goldenberg guy110698@gmail.com 2025-06-13T19:20:25+03:00 GitHub noreply@github.com 2025-06-13T19:20:25+03:00 Merge commit from fork 80709b70a2f87c13ccaf1480b799393109996789 26ff3685bfbaa4c8838d7afd988b17dd5eb99f92 Georgi Gerganov ggerganov@gmail.com 2025-06-13T18:35:00+03:00 GitHub noreply@github.com 2025-06-13T18:35:00+03:00 batch : add LLAMA_BATCH_DEBUG environment variable (#14172) 26ff3685bfbaa4c8838d7afd988b17dd5eb99f92 60c666347becacff81cd4bc9a52038ba71038e41 ddpasa 112642920+ddpasa@users.noreply.github.com 2025-06-13T15:17:53+02:00 GitHub noreply@github.com 2025-06-13T15:17:53+02:00 docs : Update multimodal.md (#14122) 60c666347becacff81cd4bc9a52038ba71038e41 b7cc7745e38141060845145af0a1fd489d8e3e33 Georgi Gerganov ggerganov@gmail.com 2025-06-13T13:47:55+03:00 GitHub noreply@github.com 2025-06-13T13:47:55+03:00 batch : rework llama_batch_allocr (#14153) b7cc7745e38141060845145af0a1fd489d8e3e33 cc8d08187918c6f643c3ffabb7b1ac21aa19f3d1 Georgi Gerganov ggerganov@gmail.com 2025-06-13T11:55:44+03:00 GitHub noreply@github.com 2025-06-13T11:55:44+03:00 readme : remove survey link (#14168) cc8d08187918c6f643c3ffabb7b1ac21aa19f3d1 d714dadb57d8feaa03d13b79345a4c3382172d61 Christian Kastner ckk@kvr.at 2025-06-13T08:38:52Z GitHub noreply@github.com 2025-06-13T10:38:52+02:00 cmake: Add ability to pass in LLAMA_BUILD_NUMBER/COMMIT (#14167) d714dadb57d8feaa03d13b79345a4c3382172d61 ffad04397399ea1650fda6560c7c753059804876 Đinh Trọng Huy 77562200+huydt84@users.noreply.github.com 2025-06-13T17:34:08+09:00 GitHub noreply@github.com 2025-06-13T11:34:08+03:00 pooling : make cls_b and cls_out_b optional (#14165) ffad04397399ea1650fda6560c7c753059804876 0889eba570126f8a2f5a0e88fde776bbc91cca66 Georgi Gerganov ggerganov@gmail.com 2025-06-13T11:18:25+03:00 GitHub noreply@github.com 2025-06-13T11:18:25+03:00 server : fix SWA condition for full context reprocess (#14163) 0889eba570126f8a2f5a0e88fde776bbc91cca66 c61285e7396c8e526fe7794c19e8d4f1c99bfc51 Anton Mitkov anton.mitkov@codeplay.com 2025-06-13T08:51:39+01:00 GitHub noreply@github.com 2025-06-13T08:51:39+01:00 sycl: Adding additional cpy dbg print output (#14034) c61285e7396c8e526fe7794c19e8d4f1c99bfc51 09cf2c7c655c90e53e100f29b830a788bab0653d Ewan Crawford ewan@codeplay.com 2025-06-13T08:45:37+01:00 GitHub noreply@github.com 2025-06-13T08:45:37+01:00 SYCL: Bump oneMath commit (#14152) 09cf2c7c655c90e53e100f29b830a788bab0653d c33fe8b8c4427202706b1434e9fc8ab5752c9cac Christian Kastner ckk@kvr.at 2025-06-13T06:51:34Z GitHub noreply@github.com 2025-06-13T09:51:34+03:00 cmake : Improve build-info.cpp generation (#14156) c33fe8b8c4427202706b1434e9fc8ab5752c9cac ed52f3668e633423054a4eab61bb7efee47025ab Georgi Gerganov ggerganov@gmail.com 2025-06-13T08:03:54+03:00 GitHub noreply@github.com 2025-06-13T08:03:54+03:00 vocab : prevent heap overflow when vocab is too small (#14145) 6391ee7ffd97e723dfbdcc32f001e1b95a63cba6 b9850050a5b5147cb2e6720b98ed8c1e8e02eff9 Yunzhe Jia yunzhe@calculet.tech 2025-06-13T10:58:06+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-06-13T10:58:06+08:00 update .gitignore to update cal_test.gguf b9850050a5b5147cb2e6720b98ed8c1e8e02eff9 933b205c32551848589b8339437d778cb4f2a46d Yunzhe Jia yunzhe@calculet.tech 2025-06-06T17:25:52+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-06-13T09:20:23+08:00 calrt: add call chain for using calrt::infer ed52f3668e633423054a4eab61bb7efee47025ab a681b4ba83a61dce71a4f24e558efe7278d8b1a9 Anton Mitkov anton.mitkov@codeplay.com 2025-06-12T14:15:11+01:00 GitHub noreply@github.com 2025-06-12T15:15:11+02:00 sycl: Remove not needed copy f16->f32 for dnnl mul mat (#14125) a681b4ba83a61dce71a4f24e558efe7278d8b1a9 7d516443dd7766569110b38e6374649bee6eb1c4 Georgi Gerganov ggerganov@gmail.com 2025-06-12T14:43:09+03:00 GitHub noreply@github.com 2025-06-12T14:43:09+03:00 readme : remove project status link (#14149) 933b205c32551848589b8339437d778cb4f2a46d 33fea9a8592fd338abe279b89350a3af2e0ff1f7 Yunzhe Jia yunzhe@calculet.tech 2025-06-12T17:47:19+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-06-12T17:50:26+08:00 gguf-py: add cal_test.py 7d516443dd7766569110b38e6374649bee6eb1c4 f6e1a7aa8787b5c00acba6370cb70a0beff48b1e Georgi Gerganov ggerganov@gmail.com 2025-06-12T11:51:38+03:00 GitHub noreply@github.com 2025-06-12T11:51:38+03:00 server : re-enable SWA speculative decoding (#14131) f6e1a7aa8787b5c00acba6370cb70a0beff48b1e c3ee46fab49a765d2e32e171e9ed7a5fa121dd9c Georgi Gerganov ggerganov@gmail.com 2025-06-12T11:50:01+03:00 GitHub noreply@github.com 2025-06-12T11:50:01+03:00 context : simplify output counting logic during decode (#14142) c3ee46fab49a765d2e32e171e9ed7a5fa121dd9c e2c0b6e46a5596665569ae765f0993cea2619af6 Georgi Gerganov ggerganov@gmail.com 2025-06-12T11:49:26+03:00 GitHub noreply@github.com 2025-06-12T11:49:26+03:00 batch : remove logits_all flag (#14141) e2c0b6e46a5596665569ae765f0993cea2619af6 9596506965f65be5d802ecef6a315fe43d2391a8 Georgi Gerganov ggerganov@gmail.com 2025-06-12T10:14:24+03:00 GitHub noreply@github.com 2025-06-12T10:14:24+03:00 cmake : handle whitepsaces in path during metal build (#14126) 9596506965f65be5d802ecef6a315fe43d2391a8 a20b2b05bce6622c585459ebf46f142f113d021c Georgi Gerganov ggerganov@gmail.com 2025-06-12T10:02:15+03:00 GitHub noreply@github.com 2025-06-12T10:02:15+03:00 kv-cache : fix split_equal handling in unified implementation (#14130) a20b2b05bce6622c585459ebf46f142f113d021c 2e89f76b7af2c0b827be785e445f2e2b3e52e1ca compilade git@compilade.net 2025-06-12T02:56:04-04:00 GitHub noreply@github.com 2025-06-12T02:56:04-04:00 context : round n_tokens to next multiple of n_seqs when reserving (#14140) 2e89f76b7af2c0b827be785e445f2e2b3e52e1ca 532802f938c6a18cc6a704057ab571f253fd77ed bandoti 141645996+bandoti@users.noreply.github.com 2025-06-11T17:19:44-03:00 GitHub noreply@github.com 2025-06-11T17:19:44-03:00 common: fix issue with regex_escape routine on windows (#14133) 532802f938c6a18cc6a704057ab571f253fd77ed d4e0d95cf581f50c9a21d06eaecae2dd580076bd Christian Kastner ckk@kvr.at 2025-06-11T19:07:44Z GitHub noreply@github.com 2025-06-11T21:07:44+02:00 Implement GGML_CPU_ALL_VARIANTS for ARM (#14080) d4e0d95cf581f50c9a21d06eaecae2dd580076bd cc66a7f78f95dcb5208420f4dd1abc3fc6aec0cc Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-11T19:04:23+02:00 GitHub noreply@github.com 2025-06-11T19:04:23+02:00 chore : clean up relative source dir paths (#14128) cc66a7f78f95dcb5208420f4dd1abc3fc6aec0cc bd248d4dc7265437e1918dc53ae3f49a0c592e5f Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-11T17:16:32+02:00 GitHub noreply@github.com 2025-06-11T17:16:32+02:00 tests : add test-tokenizers-repo (#14017) bd248d4dc7265437e1918dc53ae3f49a0c592e5f 7781e5fe99f2a4fc1c8af0a8488eedac4644cb72 Jeff Bolz jbolz@nvidia.com 2025-06-11T09:48:52-05:00 GitHub noreply@github.com 2025-06-11T09:48:52-05:00 vulkan: Better thread-safety for command pools/buffers (#14116) 7781e5fe99f2a4fc1c8af0a8488eedac4644cb72 89a184fa7125b7c3e2fb567337cc2bd7bc2d376c Aman amangupta052@gmail.com 2025-06-11T22:42:25+08:00 GitHub noreply@github.com 2025-06-11T16:42:25+02:00 webui: Wrap long numbers instead of infinite horizontal scroll (#14062) 89a184fa7125b7c3e2fb567337cc2bd7bc2d376c 2baf07727f921d9a4a1b63a2eff941e95d0488ed Georgi Gerganov ggerganov@gmail.com 2025-06-11T16:48:45+03:00 GitHub noreply@github.com 2025-06-11T16:48:45+03:00 kv-cache : relax SWA masking condition (#14119) 2baf07727f921d9a4a1b63a2eff941e95d0488ed 7ae2932116a4de3141cbc8c488f7f6e4e06d8171 Taylor quantumtraveling@gmail.com 2025-06-11T06:43:43-04:00 GitHub noreply@github.com 2025-06-11T13:43:43+03:00 server : pass default --keep argument (#14120) 7ae2932116a4de3141cbc8c488f7f6e4e06d8171 1f7d50b2936023b26eb218e944e62834b80a2ce0 Georgi Gerganov ggerganov@gmail.com 2025-06-11T12:52:45+03:00 GitHub noreply@github.com 2025-06-11T12:52:45+03:00 kv-cache : add LLAMA_KV_CACHE_DEBUG environment variable (#14121) 1f7d50b2936023b26eb218e944e62834b80a2ce0 4c763c8d1b4d4de20bf364ec1837430783cba984 Jeff Bolz jbolz@nvidia.com 2025-06-11T00:19:25-05:00 GitHub noreply@github.com 2025-06-11T07:19:25+02:00 vulkan: Track descriptor pools/sets per-context (#14109) 4c763c8d1b4d4de20bf364ec1837430783cba984 dad5c44398b78467943ed0a603ea427fe9f6fc62 lhez quic_lih@quicinc.com 2025-06-10T16:55:58-07:00 GitHub noreply@github.com 2025-06-10T16:55:58-07:00 opencl: add `mul_mv_id_q4_0_f32_8x_flat` (#14003) dad5c44398b78467943ed0a603ea427fe9f6fc62 55f6b9fa6563f6ae49113f9abdc980c12348cc1c compilade git@compilade.net 2025-06-10T18:20:14-04:00 GitHub noreply@github.com 2025-06-10T18:20:14-04:00 kv-cache : avoid modifying recurrent cells when setting inputs (#13834) 55f6b9fa6563f6ae49113f9abdc980c12348cc1c 3678b838bb71eaccbaeb479ff38c2e12bfd2f960 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-10T23:29:52+02:00 GitHub noreply@github.com 2025-06-10T23:29:52+02:00 convert : fix duplicate key DeepSeek-R1 conversion error (#14103) 3678b838bb71eaccbaeb479ff38c2e12bfd2f960 652b70e6678d503626f3c9d33831ba604b473401 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-10T18:02:08+02:00 GitHub noreply@github.com 2025-06-10T18:02:08+02:00 llama : support GEGLU for jina-bert-v2 (#14090) 652b70e6678d503626f3c9d33831ba604b473401 3a12db23b6918682ccb70ab15d897f11fe5fc320 Jeff Bolz jbolz@nvidia.com 2025-06-10T10:53:47-05:00 GitHub noreply@github.com 2025-06-10T10:53:47-05:00 vulkan: force device 0 in CI (#14106) 3a12db23b6918682ccb70ab15d897f11fe5fc320 ae92c1855b1a5b604fa1bfcced19a556ab3e78c5 Juk Armstrong 69222624+jukofyork@users.noreply.github.com 2025-06-10T16:48:07+01:00 GitHub noreply@github.com 2025-06-10T16:48:07+01:00 Fixed spec timings to: accepted/tested instead of accepted/drafted (#14104) ae92c1855b1a5b604fa1bfcced19a556ab3e78c5 b7ce1ad1e332da5909772d173a7e6748fbd1887a Georgi Gerganov ggerganov@gmail.com 2025-06-10T17:37:45+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-10T18:39:33+03:00 sync : ggml b7ce1ad1e332da5909772d173a7e6748fbd1887a 97340b4c9924be86704dbf155e97c8319849ee19 Georgi Gerganov ggerganov@gmail.com 2025-06-10T11:34:10+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-10T18:39:33+03:00 ggml : fix weak alias win32 (whisper/0) 97340b4c9924be86704dbf155e97c8319849ee19 2bb0467043258bdc58dbaefb33786f1731b38937 0cc4m picard12@live.de 2025-06-10T14:01:33+02:00 GitHub noreply@github.com 2025-06-10T13:01:33+01:00 Vulkan: Don't default to CPU device (like llvmpipe), even if no other device is available, to allow fallback to CPU backend (#14099) 2bb0467043258bdc58dbaefb33786f1731b38937 b8e2194efc529378be45ab9b27d6648a5b81458a Isaac McFadyen isaac@imcf.me 2025-06-10T02:41:01-04:00 GitHub noreply@github.com 2025-06-10T09:41:01+03:00 rpc : nicer error messages for RPC server crash (#14076) b8e2194efc529378be45ab9b27d6648a5b81458a 1a3b5e80f77c51e6d9fb7762f19b9e61fceca3de Georgi Gerganov ggerganov@gmail.com 2025-06-10T09:20:51+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-10T09:21:56+03:00 sync : ggml 1a3b5e80f77c51e6d9fb7762f19b9e61fceca3de 1f63e75f3b5dc7f44dbe63c8a41d23958fe95bc0 Kai Pastor dg0yt@darc.de 2025-06-03T12:33:28+02:00 Georgi Gerganov ggerganov@gmail.com 2025-06-10T09:21:56+03:00 Add in-build ggml::ggml ALIAS library (ggml/1260) 1f63e75f3b5dc7f44dbe63c8a41d23958fe95bc0 40cbf571c9210031340f022d104317e89a1a6bb2 Georgi Gerganov ggerganov@gmail.com 2025-06-09T23:05:02+03:00 GitHub noreply@github.com 2025-06-09T23:05:02+03:00 metal : use less stack memory in FA kernel (#14088) 40cbf571c9210031340f022d104317e89a1a6bb2 7f4fbe5183b23b6b2e25fd1ccc5d1fa8bb010cb7 Georgi Gerganov ggerganov@gmail.com 2025-06-09T23:04:35+03:00 GitHub noreply@github.com 2025-06-09T23:04:35+03:00 kv-cache : fix shift and defrag logic (#14081) 7f4fbe5183b23b6b2e25fd1ccc5d1fa8bb010cb7 f470bc36bed4d836b9de5a483fa0dfaee176d6f5 Diego Devesa slarengh@gmail.com 2025-06-09T11:03:09-07:00 GitHub noreply@github.com 2025-06-09T20:03:09+02:00 llama : allow building all tests on windows when not using shared libs (#13980) f470bc36bed4d836b9de5a483fa0dfaee176d6f5 8f47e25f56e9792093b7497c68e9f80bab82ed19 xctan axunlei@gmail.com 2025-06-09T22:47:13+08:00 GitHub noreply@github.com 2025-06-09T16:47:13+02:00 ggml-cpu : split arch-specific implementations (#13892) 8f47e25f56e9792093b7497c68e9f80bab82ed19 201b31dc2e6fef3de598280b53fd3b69420a4e49 Diego Devesa slarengh@gmail.com 2025-06-09T07:36:26-07:00 GitHub noreply@github.com 2025-06-09T16:36:26+02:00 cuda : fix device sync on buffer clear (#14033) 201b31dc2e6fef3de598280b53fd3b69420a4e49 e21d2d4ae26d6c5daf1602a1061aa4f8e722ca57 Georgi Gerganov ggerganov@gmail.com 2025-06-09T17:17:31+03:00 GitHub noreply@github.com 2025-06-09T17:17:31+03:00 graph : fix geglu (#14077) e21d2d4ae26d6c5daf1602a1061aa4f8e722ca57 dc0623fddb661926e0998538895155e4f081ff09 Xinpeng Dou 15529241576@163.com 2025-06-09T19:47:39+08:00 GitHub noreply@github.com 2025-06-09T19:47:39+08:00 CANN: Simplify the environment variable setting(#13104) dc0623fddb661926e0998538895155e4f081ff09 87d34b381d5868e75586210ec17b5ef5deddc276 R0CKSTAR yeahdongcn@gmail.com 2025-06-09T18:01:17+08:00 GitHub noreply@github.com 2025-06-09T12:01:17+02:00 webui: fix sidebar being covered by main content (#14082) 87d34b381d5868e75586210ec17b5ef5deddc276 b460d16ae858c6624fd37aec316622a4060ca325 Georgi Gerganov ggerganov@gmail.com 2025-06-09T12:57:58+03:00 GitHub noreply@github.com 2025-06-09T12:57:58+03:00 server : fix LRU check (#14079) b460d16ae858c6624fd37aec316622a4060ca325 91a8ee6a6f1f4c8547ff7b745ef95c6edc1d2af6 Nicolò Scipione nicolo.scipione@codeplay.com 2025-06-09T11:47:07+02:00 GitHub noreply@github.com 2025-06-09T11:47:07+02:00 sycl: Add reorder to Q6_K mmvq implementation (#13885) 91a8ee6a6f1f4c8547ff7b745ef95c6edc1d2af6 056eb74534ffd3efc50a9b854156eb6876a52a44 Đinh Trọng Huy 77562200+huydt84@users.noreply.github.com 2025-06-09T13:15:31+09:00 GitHub noreply@github.com 2025-06-09T05:15:31+01:00 add geglu activation function (#14074) 056eb74534ffd3efc50a9b854156eb6876a52a44 247e5c6e447707bb4539bdf1913d206088a8fc69 Yuanhao Ji jiyuanhao@apache.org 2025-06-09T11:20:06+08:00 GitHub noreply@github.com 2025-06-09T11:20:06+08:00 CANN: Enable labeler for Ascend NPU (#13914) 247e5c6e447707bb4539bdf1913d206088a8fc69 5787b5da57e54dba760c2deeac1edf892e8fc450 Diego Devesa slarengh@gmail.com 2025-06-08T11:39:56-07:00 GitHub noreply@github.com 2025-06-08T11:39:56-07:00 cuda : fix buffer type check with integrated GPUs (#14069) 5787b5da57e54dba760c2deeac1edf892e8fc450 228f34c9ceefa3ea4f4d6933edd858121e8106cb 吴小白 296015668@qq.com 2025-06-07T21:39:11+08:00 GitHub noreply@github.com 2025-06-07T10:39:11-03:00 ci: add LoongArch cross-compile build (#13944) 228f34c9ceefa3ea4f4d6933edd858121e8106cb 0974ad7a7cd4bca846b15c484ff3be890135a52c Akarshan Biswas akarshan@menlo.ai 2025-06-07T18:58:20+05:30 GitHub noreply@github.com 2025-06-07T18:58:20+05:30 SYCL: Implement few same quantized type copy kernels (#13739) 0974ad7a7cd4bca846b15c484ff3be890135a52c 745aa5319b9930068aff5e87cf5e9eef7227339b Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-07T14:13:12+02:00 GitHub noreply@github.com 2025-06-07T14:13:12+02:00 llama : fix llama_model_chat_template with template name (LLM_KV with suffix) (#14050) 745aa5319b9930068aff5e87cf5e9eef7227339b 487a5e0401423bba02cd6e97e4d45131bb20b22b Georgi Gerganov ggerganov@gmail.com 2025-06-06T14:11:15+03:00 GitHub noreply@github.com 2025-06-06T14:11:15+03:00 llama : deprecate llama_kv_self_ API (#14030) 487a5e0401423bba02cd6e97e4d45131bb20b22b d17a809ef0af09b16625e991a76f6fe80d9c332e Georgi Gerganov ggerganov@gmail.com 2025-06-06T13:29:18+03:00 GitHub noreply@github.com 2025-06-06T13:29:18+03:00 context : fix SWA-related warning for multiple sequences (#14045) 33fea9a8592fd338abe279b89350a3af2e0ff1f7 806f05a353d322e97d97e02f910de2b408a8b4ab Yunzhe Jia yunzhe@calculet.tech 2025-06-06T17:25:52+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-06-06T17:35:48+08:00 calrt: add load calbin d17a809ef0af09b16625e991a76f6fe80d9c332e 1caae7fc6c77551cb1066515e0f414713eebb367 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-06T09:03:25+02:00 GitHub noreply@github.com 2025-06-06T09:03:25+02:00 llama : support multiple classifier outputs and labels (#13940) 806f05a353d322e97d97e02f910de2b408a8b4ab ec9e0301fef6476df83e94842c3b625501c95566 Yunzhe Jia yunzhe@calculet.tech 2025-05-30T09:45:37+08:00 Yunzhe Jia yunzhe@calculet.tech 2025-06-06T09:24:26+08:00 add calrt demo 1caae7fc6c77551cb1066515e0f414713eebb367 669c13e0f67edfba891c5bd7105eb4376bcf8626 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-05T17:42:31+02:00 GitHub noreply@github.com 2025-06-05T17:42:31+02:00 gguf-py : add add_classifier_output_labels method to writer (#14031) 669c13e0f67edfba891c5bd7105eb4376bcf8626 146b88e8b3e16d22cea22f8be982a4d45e913b1e Masato Nakasaka rillomas@gmail.com 2025-06-05T23:00:29+09:00 GitHub noreply@github.com 2025-06-05T16:00:29+02:00 vulkan: Enable VK_KHR_cooperative_matrix extension for Intel Xe2 GPUs (#14001) 146b88e8b3e16d22cea22f8be982a4d45e913b1e 7f37b6cf1e2c1b90bf0d9c8d91904b4b6c512748 pockers21 134406831+pockers21@users.noreply.github.com 2025-06-05T06:25:29-07:00 GitHub noreply@github.com 2025-06-05T16:25:29+03:00 ci: fix CUDA build failure on autodl cloud machines (#14005) 7f37b6cf1e2c1b90bf0d9c8d91904b4b6c512748 3a077146a4761fdbd24bdd8eb098f46b8adc4dda Georgi Gerganov ggerganov@gmail.com 2025-06-05T15:29:22+03:00 GitHub noreply@github.com 2025-06-05T15:29:22+03:00 memory : migrate from llama_kv_cache to more generic llama_memory (#14006) 3a077146a4761fdbd24bdd8eb098f46b8adc4dda d01d112abb055549d33bb8aac1755eb0c40918ad Diego Devesa slarengh@gmail.com 2025-06-05T02:57:42-07:00 GitHub noreply@github.com 2025-06-05T11:57:42+02:00 llama : allow using mmap without PrefetchVirtualMemory, apply GGML_WIN_VER to llama.cpp sources (#14013) d01d112abb055549d33bb8aac1755eb0c40918ad 9f47fa5792bae5312615439e68dbf1826913f7ac Olexandr88 radole1203@gmail.com 2025-06-05T10:50:55+03:00 GitHub noreply@github.com 2025-06-05T10:50:55+03:00 readme : add badge (#13938) 9f47fa5792bae5312615439e68dbf1826913f7ac 9e31bec4fd53634c9e5b04650488a09a055f5dab Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-05T09:29:18+02:00 GitHub noreply@github.com 2025-06-05T09:29:18+02:00 vocab : warn about missing mask token (#14022) 9e31bec4fd53634c9e5b04650488a09a055f5dab 5a8ae3053ced350ed300ba91600519fcad1c6ba7 Georgi Gerganov ggerganov@gmail.com 2025-06-05T09:06:29+03:00 GitHub noreply@github.com 2025-06-05T09:06:29+03:00 context : fix pos_min initialization upon error decode (#14008) 5a8ae3053ced350ed300ba91600519fcad1c6ba7 0d3984424f2973c49c4bcabe4cc0153b4f90c601 Jeff Bolz jbolz@nvidia.com 2025-06-05T00:17:58-05:00 GitHub noreply@github.com 2025-06-05T07:17:58+02:00 vulkan: automatically deduce size of push constants (#13936) 0d3984424f2973c49c4bcabe4cc0153b4f90c601 3e63a58ef7addec35408e2eb67850d7cdc935dc3 Ervin Áron Tasnádi etasnadi@protonmail.com 2025-06-04T22:02:00+02:00 GitHub noreply@github.com 2025-06-04T22:02:00+02:00 ggml-vulkan: adds support for op CONV_TRANSPOSE_1D (#13813) 3e63a58ef7addec35408e2eb67850d7cdc935dc3 2589ad3704559f4dd860f5f303b19349c688a28a Georgi Gerganov ggerganov@gmail.com 2025-06-04T18:58:20+03:00 GitHub noreply@github.com 2025-06-04T18:58:20+03:00 kv-cache : refactor the update/defrag mechanism (#13988) 2589ad3704559f4dd860f5f303b19349c688a28a 482548716f664f76e325ded58c9e8b7563e5e23a Diego Devesa slarengh@gmail.com 2025-06-04T06:37:40-07:00 GitHub noreply@github.com 2025-06-04T15:37:40+02:00 ci : remove cuda 11.7 releases, switch runner to windows 2022 (#13997) 482548716f664f76e325ded58c9e8b7563e5e23a 3ac67535c86e2fc43e4eddf594412acc370bbb04 Diego Devesa slarengh@gmail.com 2025-06-04T04:15:54-07:00 GitHub noreply@github.com 2025-06-04T13:15:54+02:00 releases : use dl backend for linux release, remove arm64 linux release (#13996) 3ac67535c86e2fc43e4eddf594412acc370bbb04 0b4be4c435849b00dbd98b109cf7a22298d27b69 Xuan-Son Nguyen son@huggingface.co 2025-06-04T10:11:26+02:00 GitHub noreply@github.com 2025-06-04T10:11:26+02:00 llama-graph : use ggml_repeat_4d (#13998) 0b4be4c435849b00dbd98b109cf7a22298d27b69 e0e806f52ebcd0ee285c994fe8fd8b8787d2cb0a Johannes Gäßler johannesg@5d6.de 2025-06-04T08:57:05+02:00 GitHub noreply@github.com 2025-06-04T08:57:05+02:00 CUDA: fix FTZ in FA for Gemma 3 (#13991) e0e806f52ebcd0ee285c994fe8fd8b8787d2cb0a 7e00e60ef86645a01fda738fef85b74afa016a34 Georgi Gerganov ggerganov@gmail.com 2025-06-04T09:50:32+03:00 GitHub noreply@github.com 2025-06-04T09:50:32+03:00 kv-cache : fix unified::seq_rm to work with seq_id < 0 (#13985) 7e00e60ef86645a01fda738fef85b74afa016a34 ea1431b0fa3a8108aac1e0a94a13ccc4a749963e Jeff Bolz jbolz@nvidia.com 2025-06-03T13:30:22-05:00 GitHub noreply@github.com 2025-06-03T20:30:22+02:00 vulkan: fix warnings in perf logger querypool code (#13937) ea1431b0fa3a8108aac1e0a94a13ccc4a749963e 71e74a3ac929b8af91f16f73f3c2b9b2f796d207 Xuan-Son Nguyen son@huggingface.co 2025-06-03T13:09:36+02:00 GitHub noreply@github.com 2025-06-03T13:09:36+02:00 docs : add "Quick start" section for new users (#13862) 71e74a3ac929b8af91f16f73f3c2b9b2f796d207 bfb1e012a0b7658e8f00ed4333d059943ea9d648 lhez quic_lih@quicinc.com 2025-06-02T16:54:58-07:00 GitHub noreply@github.com 2025-06-02T16:54:58-07:00 opencl: add `backend_synchronize` (#13939) bfb1e012a0b7658e8f00ed4333d059943ea9d648 363757628848a27a435bbf22ff9476e9aeda5f40 rmatif 66360289+rmatif@users.noreply.github.com 2025-06-02T23:53:36Z GitHub noreply@github.com 2025-06-02T16:53:36-07:00 OpenCL: Add concat, tsembd, upscale, tanh, pad and repeat (#13840) 363757628848a27a435bbf22ff9476e9aeda5f40 ea394d7ab1f8101716d48ce9421c94c71b93a00f Georgi Gerganov ggerganov@gmail.com 2025-06-02T21:34:40+03:00 GitHub noreply@github.com 2025-06-02T21:34:40+03:00 server : disable speculative decoding for SWA models (#13970) ea394d7ab1f8101716d48ce9421c94c71b93a00f 5582c49c3961269eca96822abfb87528e942dd07 Georgi Gerganov ggerganov@gmail.com 2025-06-02T21:33:40+03:00 GitHub noreply@github.com 2025-06-02T21:33:40+03:00 metal : use F32 accumulators in FA kernels (#13975) 5582c49c3961269eca96822abfb87528e942dd07 c9bbc77931d223ed7e7cbcf1cb057bc02fd0db19 Georgi Gerganov ggerganov@gmail.com 2025-06-02T20:54:26+03:00 GitHub noreply@github.com 2025-06-02T20:54:26+03:00 gemma : more consistent attention scaling for v2 and v3 (#13951) c9bbc77931d223ed7e7cbcf1cb057bc02fd0db19 bfd322796cd838f906535ff3352624fc46338894 Olivier Chafik olivier.chafik@gmail.com 2025-06-02T10:15:44-07:00 GitHub noreply@github.com 2025-06-02T10:15:44-07:00 `server`: update deepseek reasoning format (pass reasoning_content as diffs) (#13933) bfd322796cd838f906535ff3352624fc46338894 093e3f1feb16e25e58f7d61e01266c830dd424b8 Xuan-Son Nguyen son@huggingface.co 2025-06-02T16:29:28+02:00 GitHub noreply@github.com 2025-06-02T16:29:28+02:00 mtmd : fix memory leak in mtmd_helper_eval_chunk_single (#13961) 093e3f1feb16e25e58f7d61e01266c830dd424b8 663445b0deb21fb602176da030d4154197a4fca6 shalinib-ibm Shalini.Salomi.Bodapati@ibm.com 2025-06-02T17:48:36+05:30 GitHub noreply@github.com 2025-06-02T15:18:36+03:00 cmake : Handle mixed-case 'Power' strings in POWER CPU detection (#13966) 663445b0deb21fb602176da030d4154197a4fca6 7675c555a13c9f473249e59a54db35032ce8e0fc Atharva Dubey atharva.dubey@codeplay.com 2025-06-02T10:12:20+01:00 GitHub noreply@github.com 2025-06-02T10:12:20+01:00 sycl: quantize and reorder the input to q8_1 when reorder is enabled (#13826) 7675c555a13c9f473249e59a54db35032ce8e0fc 5e1c3aed4074480f63e914d6c44c93536ed1452a Johannes Gäßler johannesg@5d6.de 2025-06-01T18:08:05+02:00 GitHub noreply@github.com 2025-06-01T18:08:05+02:00 gguf: fix failure on version == 0 (#13956) 5e1c3aed4074480f63e914d6c44c93536ed1452a c496fe0b1da28618dd17bda8b0a6bf5004554080 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-01T18:07:21+02:00 GitHub noreply@github.com 2025-06-01T18:07:21+02:00 convert : fix nomic-bert-moe mask token (#13757) c496fe0b1da28618dd17bda8b0a6bf5004554080 e57bb87cede38341963a7a884630dbfb09c7dc00 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-06-01T17:23:11+02:00 GitHub noreply@github.com 2025-06-01T17:23:11+02:00 convert : fix vocab padding code for bert models (#13954) e57bb87cede38341963a7a884630dbfb09c7dc00 f3a4b1659ccc94ebdf3590d472b518377bfecc7a Aaron Teo aaron.teo1@ibm.com 2025-06-01T22:53:57+08:00 GitHub noreply@github.com 2025-06-01T16:53:57+02:00 ggml: check if non-native endian model is being loaded (#13943) f3a4b1659ccc94ebdf3590d472b518377bfecc7a 108009f5c7267b77292c11f788f602d6d7ae8fcd Georgi Gerganov ggerganov@gmail.com 2025-06-01T12:23:14+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-01T13:43:57+03:00 sync : ggml 108009f5c7267b77292c11f788f602d6d7ae8fcd d337252acf14a91a685c355fa4f3f599a8068207 Kai Pastor dg0yt@darc.de 2025-05-31T12:49:55+02:00 Georgi Gerganov ggerganov@gmail.com 2025-06-01T13:43:57+03:00 vulkan : Remove unexpected ; (ggml/1253) d337252acf14a91a685c355fa4f3f599a8068207 af6f91db470da543dc32bc00c057aad8f060dfdb Kai Pastor dg0yt@darc.de 2025-05-31T12:39:19+02:00 Georgi Gerganov ggerganov@gmail.com 2025-06-01T13:43:57+03:00 cmake : Fix broken CMake error messages (ggml/1252) af6f91db470da543dc32bc00c057aad8f060dfdb a7b8d35f780ab3e7639ae5345442fb1ad10f0163 Radoslav Gerganov rgerganov@gmail.com 2025-05-30T09:11:09+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-01T13:43:57+03:00 ggml : remove ggml_graph_import and ggml_graph_export declarations (ggml/1247) a7b8d35f780ab3e7639ae5345442fb1ad10f0163 6eba72b71c677ce9aae33c422a6e67008137987a Georgi Gerganov ggerganov@gmail.com 2025-05-29T13:29:50+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-01T13:43:57+03:00 sync : whisper.cpp (ggml/1250) 6eba72b71c677ce9aae33c422a6e67008137987a fedf034a98378059274e4243d2a370a243335e73 Radoslav Gerganov rgerganov@gmail.com 2025-05-29T08:34:46+03:00 Georgi Gerganov ggerganov@gmail.com 2025-06-01T13:43:57+03:00 ggml : install dynamic backends (ggml/1240) fedf034a98378059274e4243d2a370a243335e73 8726392d3d927fe3e504868d3548d694496ee37e Daniel Tang danielzgtg.opensource@gmail.com 2025-05-27T20:58:46-04:00 Georgi Gerganov ggerganov@gmail.com 2025-06-01T13:43:57+03:00 ggml : Print backtrace on uncaught C++ exceptions (ggml/1232) 8726392d3d927fe3e504868d3548d694496ee37e c04621711a893cbd09cff6c927cb005bc6749e36 ddh0 chemist-mulches-39@icloud.com 2025-06-01T03:44:30-05:00 GitHub noreply@github.com 2025-06-01T11:44:30+03:00 readme : update bindings (#13950) c04621711a893cbd09cff6c927cb005bc6749e36 0fc16b42e8793364918e830c234c1f3caec29dae Georgi Gerganov ggerganov@gmail.com 2025-06-01T11:42:16+03:00 GitHub noreply@github.com 2025-06-01T11:42:16+03:00 parallel : fix n_junk == 0 (#13952) 0fc16b42e8793364918e830c234c1f3caec29dae 053b1539c02617eff744f89525ee57497c3c1fbe Georgi Gerganov ggerganov@gmail.com 2025-06-01T11:39:27+03:00 GitHub noreply@github.com 2025-06-01T11:39:27+03:00 kv-cache : split implementation in separate sources (#13920) 053b1539c02617eff744f89525ee57497c3c1fbe b3a89c3d9e34c28c5be70d8b687a84775746d4a0 Max Krasnyansky quic_maxk@quicinc.com 2025-05-31T15:39:19-07:00 GitHub noreply@github.com 2025-05-31T15:39:19-07:00 threading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling (#12995) b3a89c3d9e34c28c5be70d8b687a84775746d4a0 e15898d1c7e87f1b3d14e0a3c385eeb424b085fe Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2025-05-31T18:58:35+02:00 GitHub noreply@github.com 2025-05-31T18:58:35+02:00 docs : Note about necessity of having libcurl installed for standard build. (#13945) e15898d1c7e87f1b3d14e0a3c385eeb424b085fe 803f8baf4f741d2f0465c46c33f285886b97a071 Olivier Chafik olivier.chafik@gmail.com 2025-05-31T08:26:10-07:00 GitHub noreply@github.com 2025-05-31T08:26:10-07:00 server: allow unclosed thinking tags (#13931) 803f8baf4f741d2f0465c46c33f285886b97a071 3600cc2886956fc0a07ef6ad2f4128ccfdbc8c6f Georgi Gerganov ggerganov@gmail.com 2025-05-31T15:58:33+03:00 GitHub noreply@github.com 2025-05-31T15:58:33+03:00 llama : deprecate explicit kv_self defrag/update calls (#13921) 3600cc2886956fc0a07ef6ad2f4128ccfdbc8c6f c7e0a2054b908c28bf93bb18d4b63ccbff2c4127 Georgi Gerganov ggerganov@gmail.com 2025-05-31T15:57:44+03:00 GitHub noreply@github.com 2025-05-31T15:57:44+03:00 llama : use n_swa + n_ubatch cells for SWA cache (#13833) c7e0a2054b908c28bf93bb18d4b63ccbff2c4127 3f55f781f1da9241f209648636fa0426fe62a495 igardev 49397134+igardev@users.noreply.github.com 2025-05-31T12:56:08+03:00 GitHub noreply@github.com 2025-05-31T11:56:08+02:00 webui : Replace alert and confirm with custom modals. (#13711) 3f55f781f1da9241f209648636fa0426fe62a495 51fa76f172957c9916e43aeb581f82c533e12394 Georgi Gerganov ggerganov@gmail.com 2025-05-31T12:55:57+03:00 GitHub noreply@github.com 2025-05-31T12:55:57+03:00 llama : auto-batch preparation (#13845) 51fa76f172957c9916e43aeb581f82c533e12394 12d0188c0dc6146ffde6d277a93f232ccbe699f8 Xuan-Son Nguyen son@huggingface.co 2025-05-31T10:14:29+02:00 GitHub noreply@github.com 2025-05-31T10:14:29+02:00 mtmd : drop `_shared` from `libmtmd` name, merge helpers into libmtmd (⚠️ breaking change) (#13917) 12d0188c0dc6146ffde6d277a93f232ccbe699f8 eb3949938e82a128855bc0676220bb2ce6e4228d Georgi Gerganov ggerganov@gmail.com 2025-05-31T10:24:04+03:00 GitHub noreply@github.com 2025-05-31T10:24:04+03:00 kv-cache : refactor + add llama_memory_state_i (#13746) eb3949938e82a128855bc0676220bb2ce6e4228d e562eece7cb476276bfc4cbb18deb7c0369b2233 Shawn yang 137684499+Yangxiaoz@users.noreply.github.com 2025-05-31T14:48:04+08:00 GitHub noreply@github.com 2025-05-31T08:48:04+02:00 CUDA: add a prop in ggml_cuda_device_infor for distinguish iGPU or dGPU in cuda (#13856) (#13895) e562eece7cb476276bfc4cbb18deb7c0369b2233 b47ab7b8e9c4f6154eb6abb6234866ab117d64c7 Johannes Gäßler johannesg@5d6.de 2025-05-30T21:22:03+02:00 GitHub noreply@github.com 2025-05-30T21:22:03+02:00 CUDA: fix typo in FlashAttention code (#13926) b47ab7b8e9c4f6154eb6abb6234866ab117d64c7 dd665cc9d4b378626cde11ea0c4c91f514fdc994 Diego Devesa slarengh@gmail.com 2025-05-30T09:56:19-07:00 GitHub noreply@github.com 2025-05-30T18:56:19+02:00 sched : avoid changing cur_copy when a graph is already allocated (#13922) dd665cc9d4b378626cde11ea0c4c91f514fdc994 df0c0c7d02f951dc639d48fd536f79200460ac83 Georgi Gerganov ggerganov@gmail.com 2025-05-30T19:38:07+03:00 GitHub noreply@github.com 2025-05-30T19:38:07+03:00 parallel : increase the variability of the prompt lengths (#13927) df0c0c7d02f951dc639d48fd536f79200460ac83 b49a8ff96b769b8a4c36d89fb783ec0135be582b Diego Devesa slarengh@gmail.com 2025-05-30T07:37:18-07:00 GitHub noreply@github.com 2025-05-30T16:37:18+02:00 cuda : prevent using split buffers with 3d/4d matrices (#13919) b49a8ff96b769b8a4c36d89fb783ec0135be582b 53f925074de02c5304b00c14b4d6d8910c58667d Akarshan Biswas akarshan@menlo.ai 2025-05-30T19:40:57+05:30 GitHub noreply@github.com 2025-05-30T19:40:57+05:30 SYCL: Add mrope kernel (#13755) 53f925074de02c5304b00c14b4d6d8910c58667d db38704f0133be7832123495fa8fc2601ea999d4 Georgi Gerganov ggerganov@gmail.com 2025-05-30T16:25:45+03:00 GitHub noreply@github.com 2025-05-30T16:25:45+03:00 sync : vendor (#13901) db38704f0133be7832123495fa8fc2601ea999d4 07e4351ce663a7802b31f92fd7bc0e555c2044b6 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-30T14:50:43+02:00 GitHub noreply@github.com 2025-05-30T14:50:43+02:00 convert : fix rwkv bos/eos token (#13844) 07e4351ce663a7802b31f92fd7bc0e555c2044b6 291f2b6913c7ef8350dbf0e77da38f7af131a08e Xuan-Son Nguyen son@huggingface.co 2025-05-30T12:24:37+02:00 GitHub noreply@github.com 2025-05-30T12:24:37+02:00 convert : allow partial update to the chkhsh pre-tokenizer list (#13847) 291f2b6913c7ef8350dbf0e77da38f7af131a08e 2c90da4c7ec694797f524042aaafbb047a7e65ff Đinh Trọng Huy 77562200+huydt84@users.noreply.github.com 2025-05-30T18:56:02+09:00 GitHub noreply@github.com 2025-05-30T11:56:02+02:00 llama : add support for DistilBert (#13907) 2c90da4c7ec694797f524042aaafbb047a7e65ff ec9e0301fef6476df83e94842c3b625501c95566 zhangkaihuo zhangkaihuo@gmail.com 2025-05-30T16:31:48+08:00 GitHub noreply@github.com 2025-05-30T10:31:48+02:00 llama : use llm_build_granite for minicpm (#13911) ec9e0301fef6476df83e94842c3b625501c95566 e83ba3e460651b20a594e9f2f0f0bffb998d3ce1 Christian Kastner ckk@kvr.at 2025-05-30T01:28:54+02:00 GitHub noreply@github.com 2025-05-30T01:28:54+02:00 cmake: Guard GGML_CPU_ALL_VARIANTS by architecture (#13890) e83ba3e460651b20a594e9f2f0f0bffb998d3ce1 2b131621e60d8ec2cc961201beb6773ab37b6b69 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-29T21:42:31+02:00 GitHub noreply@github.com 2025-05-29T21:42:31+02:00 llama : add support for jina-reranker-v2 (#13900) 2b131621e60d8ec2cc961201beb6773ab37b6b69 54a2c7a8cd8a32b44e3a98c2999b0f5c9114be5c Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-29T15:36:05+02:00 GitHub noreply@github.com 2025-05-29T15:36:05+02:00 gguf-py : add support for sub_type (in arrays) in GGUFWriter add_key_value method (#13561) 54a2c7a8cd8a32b44e3a98c2999b0f5c9114be5c 21fcc21ad5e5de2daa5da8d08dbbcc86b8d815d7 Yibo Cai yibo.cai@arm.com 2025-05-29T19:39:20+08:00 GitHub noreply@github.com 2025-05-29T14:39:20+03:00 arm64: optimize q4_k_q8_k kernel with i8mm (#13886) 21fcc21ad5e5de2daa5da8d08dbbcc86b8d815d7 dd8ba93416f492c168f7e20f0b1434c08ebeaeb5 Christian Kastner ckk@kvr.at 2025-05-29T12:50:25+02:00 GitHub noreply@github.com 2025-05-29T12:50:25+02:00 cmake: Factor out CPU architecture detection (#13883) dd8ba93416f492c168f7e20f0b1434c08ebeaeb5 66c92061f5c34d2f9a630b7b50cca5ce3ebb4b16 Vineel Abhinav 131174187+vineelabhinav@users.noreply.github.com 2025-05-29T14:48:43+05:30 GitHub noreply@github.com 2025-05-29T12:18:43+03:00 ggml: aarch64: Implement SVE F32 kernels for Mamba Sequential Scan Algorithm (#13882) 66c92061f5c34d2f9a630b7b50cca5ce3ebb4b16 5ca82fc1d7f8cb27f3cbb3019e944a80d5219828 Georgi Gerganov ggerganov@gmail.com 2025-05-29T12:17:16+03:00 GitHub noreply@github.com 2025-05-29T12:17:16+03:00 tests : remove json.hpp from a test (#13880) 5ca82fc1d7f8cb27f3cbb3019e944a80d5219828 6385b843a8dc8e15b8362196039720c58dd79fa2 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-29T10:00:57+02:00 GitHub noreply@github.com 2025-05-29T10:00:57+02:00 convert : workaround for AutoConfig dummy labels (#13881) 6385b843a8dc8e15b8362196039720c58dd79fa2 1b8fb8152d0f3357a9c1d9b4c02f6cc5b9cb7232 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-29T08:15:01+02:00 GitHub noreply@github.com 2025-05-29T08:15:01+02:00 llama : add RobertaForSequenceClassification reranker support (#13875) 1b8fb8152d0f3357a9c1d9b4c02f6cc5b9cb7232 53ae30640e131082d8d19bd80485b47c4553d551 Vineel Abhinav 131174187+vineelabhinav@users.noreply.github.com 2025-05-29T11:31:33+05:30 GitHub noreply@github.com 2025-05-29T09:01:33+03:00 ggml: aarch64: Implement SVE F32 kernels for vector functions (#13843) 53ae30640e131082d8d19bd80485b47c4553d551 763d06edb7dd5094ea58bad1d81e2e8d35033e34 Beinsezii 39478211+Beinsezii@users.noreply.github.com 2025-05-28T14:50:20-07:00 GitHub noreply@github.com 2025-05-28T23:50:20+02:00 gguf-py : fix SafetensorRemote return on undefined size (< 0) (#13841) 763d06edb7dd5094ea58bad1d81e2e8d35033e34 10961339b26bd2eff01d5479e8879f435da261b7 Xuan-Son Nguyen son@huggingface.co 2025-05-28T22:35:31+02:00 GitHub noreply@github.com 2025-05-28T22:35:31+02:00 llama : fix KV shift for qwen2vl (#13870) 10961339b26bd2eff01d5479e8879f435da261b7 d98f2a35fcf4a8d3e660ad48cd19e2a1f3d5b2ef Xuan-Son Nguyen son@huggingface.co 2025-05-28T22:35:22+02:00 GitHub noreply@github.com 2025-05-28T22:35:22+02:00 mtmd : move helpers to dedicated library (⚠️ breaking change) (#13866) d98f2a35fcf4a8d3e660ad48cd19e2a1f3d5b2ef e0e3aa231d899720c2864d09cdb89d4c400eeb55 bandoti 141645996+bandoti@users.noreply.github.com 2025-05-28T15:46:47-03:00 GitHub noreply@github.com 2025-05-28T15:46:47-03:00 ci: disable LLAMA_CURL for Linux cross-builds (#13871) e0e3aa231d899720c2864d09cdb89d4c400eeb55 aa6dff05be25709bb218bf648951d690029c4b19 Đinh Trọng Huy 77562200+huydt84@users.noreply.github.com 2025-05-29T02:01:58+09:00 GitHub noreply@github.com 2025-05-28T19:01:58+02:00 llama : add support for BertForSequenceClassification reranker (#13858) aa6dff05be25709bb218bf648951d690029c4b19 c962ae3382a1e759c8517a229549ee53685313a1 Đinh Trọng Huy 77562200+huydt84@users.noreply.github.com 2025-05-28T23:34:18+09:00 GitHub noreply@github.com 2025-05-28T16:34:18+02:00 convert: small addition to support LlamaModel (#13838) c962ae3382a1e759c8517a229549ee53685313a1 a3938fb53d0b5183db4f5a6db21fd9122a0e4780 Sky Iflyinskyin2013@gmail.com 2025-05-28T22:33:54+08:00 GitHub noreply@github.com 2025-05-28T16:33:54+02:00 server: fix remove 'image_url'/'input_audio' json-object effectlly for 'llama_params' in multimodal-model-mode (#13853) a3938fb53d0b5183db4f5a6db21fd9122a0e4780 f7873fc698c09047e2873630ab7e7730a0bfb224 Xuan-Son Nguyen son@huggingface.co 2025-05-28T16:12:35+02:00 GitHub noreply@github.com 2025-05-28T16:12:35+02:00 convert : fix qwen omni conversion (#13859) f7873fc698c09047e2873630ab7e7730a0bfb224 a68247439bd6fb756cc93ad2817e55a02aa0b100 Alex Fanthome xfanth@gmail.com 2025-05-28T14:49:28+01:00 GitHub noreply@github.com 2025-05-28T15:49:28+02:00 tests : change umlaut test (#11600) a68247439bd6fb756cc93ad2817e55a02aa0b100 26b79b6cb3e7840ff15729350e95907e19f9f480 Johannes Gäßler johannesg@5d6.de 2025-05-28T13:33:37+02:00 GitHub noreply@github.com 2025-05-28T13:33:37+02:00 CUDA: fix FA tg at long context for CC >= 8.9 (#13852) 26b79b6cb3e7840ff15729350e95907e19f9f480 1e8659e65ad0f640674d2785d02b556ab938728c Xuan-Son Nguyen son@huggingface.co 2025-05-28T10:05:54+02:00 GitHub noreply@github.com 2025-05-28T10:05:54+02:00 convert : fix tensor naming conflict for llama 4 vision (#13836) 1e8659e65ad0f640674d2785d02b556ab938728c a3c30846e410c91c11d7bf80978795a03bb03dee leo-pony nengjunma@outlook.com 2025-05-28T11:54:20+08:00 GitHub noreply@github.com 2025-05-28T11:54:20+08:00 CANN: Add SOC TYPE printing in cmake configuration (#13837) a3c30846e410c91c11d7bf80978795a03bb03dee 1701d4c54f93c0d203bf92ea7202a94b037c2338 lhez quic_lih@quicinc.com 2025-05-27T12:56:08-07:00 GitHub noreply@github.com 2025-05-27T12:56:08-07:00 opencl: add new ops - `argsort`, `div`, `sub`, `addrows`, `sigmoid`, `group_norm` (#13787) 1701d4c54f93c0d203bf92ea7202a94b037c2338 bef817638780dcbd8c0c80c97b7a4f8e92c8fe74 lhez quic_lih@quicinc.com 2025-05-27T12:53:14-07:00 GitHub noreply@github.com 2025-05-27T12:53:14-07:00 opencl: mark `mul_mat` `f32f32` as supporting non-contiguous tensors (#13790) bef817638780dcbd8c0c80c97b7a4f8e92c8fe74 34b7c0439ed0f98575cc4689dfecd98991dee8be Jeff Bolz jbolz@nvidia.com 2025-05-27T11:39:07-05:00 GitHub noreply@github.com 2025-05-27T18:39:07+02:00 vulkan: use timestamp queries for GGML_VULKAN_PERF (#13817) 34b7c0439ed0f98575cc4689dfecd98991dee8be f3101a8cc665f73217c752a10a7042889275cfbc Georgi Gerganov ggerganov@gmail.com 2025-05-27T19:08:44+03:00 GitHub noreply@github.com 2025-05-27T19:08:44+03:00 cmake : add llama-cparams.cpp to build (#13832) f3101a8cc665f73217c752a10a7042889275cfbc 1c49c70d07ef87635daa5e8fdd0b5bfd88493dd3 Akarshan Biswas akarshan@menlo.ai 2025-05-27T20:52:59+05:30 GitHub noreply@github.com 2025-05-27T20:52:59+05:30 SYCL: add gelu_erf kernel (#13749) 1c49c70d07ef87635daa5e8fdd0b5bfd88493dd3 a8ea03d8ad9c984fe6cfafead183ab188f8cbeb0 Georgi Gerganov ggerganov@gmail.com 2025-05-27T18:04:38+03:00 Georgi Gerganov ggerganov@gmail.com 2025-05-27T18:05:33+03:00 sync : ggml a8ea03d8ad9c984fe6cfafead183ab188f8cbeb0 05f6ac6283a7859a03cd59405504262c85c4bf06 Xuan-Son Nguyen son@huggingface.co 2025-05-27T15:53:55+02:00 GitHub noreply@github.com 2025-05-27T15:53:55+02:00 ggml : add ggml_repeat_4d (#13824) 05f6ac6283a7859a03cd59405504262c85c4bf06 bc583e3c63c04a11d287c108ea9e6a515ead0423 xctan xc-tan@outlook.com 2025-05-27T21:21:36+08:00 GitHub noreply@github.com 2025-05-27T16:21:36+03:00 ggml : riscv: add xtheadvector support (#13720) bc583e3c63c04a11d287c108ea9e6a515ead0423 72b090da2c50e540143fd312a2f9aa5f151e6136 Xuan-Son Nguyen son@huggingface.co 2025-05-27T14:06:10+02:00 GitHub noreply@github.com 2025-05-27T14:06:10+02:00 mtmd : support Qwen 2.5 Omni (input audio+vision, no audio output) (#13784) 72b090da2c50e540143fd312a2f9aa5f151e6136 7fe03e7446ed4388539d3bcc013ae4e851b8d1e2 bandoti 141645996+bandoti@users.noreply.github.com 2025-05-27T08:52:40-03:00 GitHub noreply@github.com 2025-05-27T08:52:40-03:00 docs: remove link for llama-cli function calling (#13810) 7fe03e7446ed4388539d3bcc013ae4e851b8d1e2 952f3953c1b61cc70e79e536c42ddce6a5ea5ea7 Christian Kastner ckk@kvr.at 2025-05-27T13:18:39+02:00 GitHub noreply@github.com 2025-05-27T13:18:39+02:00 ggml-cpu: x86 feature detection is specific to x86 (#13811) 952f3953c1b61cc70e79e536c42ddce6a5ea5ea7 81713121ee56755ba4a147d1a1e3fa248ec31a66 Diego Devesa slarengh@gmail.com 2025-05-27T04:05:18-07:00 GitHub noreply@github.com 2025-05-27T13:05:18+02:00 ggml : allow CUDA graphs when using pipeline parallelism (#13814) 81713121ee56755ba4a147d1a1e3fa248ec31a66 f9cd68398baf2ba8af4725ca9ed00bef205e6706 Georgi Gerganov ggerganov@gmail.com 2025-05-27T13:49:41+03:00 GitHub noreply@github.com 2025-05-27T13:49:41+03:00 kv-cells : track min/max used cells and per-sequence positions (#13808) f9cd68398baf2ba8af4725ca9ed00bef205e6706 4f81b33e324b1669b282eb81104e4a1131be7dce Georgi Gerganov ggerganov@gmail.com 2025-05-27T12:07:52+03:00 GitHub noreply@github.com 2025-05-27T12:07:52+03:00 sampling : make sure samplers return at least 1 token (#13822) 4f81b33e324b1669b282eb81104e4a1131be7dce cdf94a18023c92f41808ec874ba577d914674717 Georgi Gerganov ggerganov@gmail.com 2025-05-27T09:40:59+03:00 GitHub noreply@github.com 2025-05-27T09:40:59+03:00 llama : validate seq id batch input (#13809) cdf94a18023c92f41808ec874ba577d914674717 a26c4cc11ec7c6574e3691e90ecdbd67deeea35b Olivier Chafik olivier.chafik@gmail.com 2025-05-26T14:34:27-07:00 GitHub noreply@github.com 2025-05-26T22:34:27+01:00 server: --offline mode (#13804) a26c4cc11ec7c6574e3691e90ecdbd67deeea35b 4265a87b59ebfc25f35adbf4db3b608995b0a78a Georgi Gerganov ggerganov@gmail.com 2025-05-26T22:24:01+03:00 GitHub noreply@github.com 2025-05-26T22:24:01+03:00 scripts : add option to compare commits in Debug (#13806) 4265a87b59ebfc25f35adbf4db3b608995b0a78a 6f180b915c9ed9ec0c240b5dcd64644988fb5e82 Georgi Gerganov ggerganov@gmail.com 2025-05-26T22:14:52+03:00 GitHub noreply@github.com 2025-05-26T22:14:52+03:00 cuda : avoid cuGetErrorString (#13791) 6f180b915c9ed9ec0c240b5dcd64644988fb5e82 03f582ae8fccecff225c30a2802461b44761e822 Akarshan Biswas akarshan@menlo.ai 2025-05-26T21:10:36+05:30 GitHub noreply@github.com 2025-05-26T21:10:36+05:30 SYCL: Add non contiguous support in RMS_NORM and NORM kernels (#13611) 03f582ae8fccecff225c30a2802461b44761e822 88c125f2acce0e25e5fc8481ab0681415fc64a10 Olivier Chafik olivier.chafik@gmail.com 2025-05-26T08:03:57-07:00 GitHub noreply@github.com 2025-05-26T16:03:57+01:00 server: fix streaming crashes (#13786) 88c125f2acce0e25e5fc8481ab0681415fc64a10 d74e94c1b3ac02db01e47008e1f8d371029d81ac standby24x7 standby24x7@gmail.com 2025-05-26T23:55:24+09:00 GitHub noreply@github.com 2025-05-26T16:55:24+02:00 examples/training: Fix file name in README (#13803) d74e94c1b3ac02db01e47008e1f8d371029d81ac f13847cfb560d92492b480cca2c5d6aa9473cde3 Olivier Chafik olivier.chafik@gmail.com 2025-05-26T06:56:49-07:00 GitHub noreply@github.com 2025-05-26T14:56:49+01:00 `server`: fix format of streamed tool call deltas (diff name, fix id location) (#13800) f13847cfb560d92492b480cca2c5d6aa9473cde3 79c137f77677b3c8ee3c60a7da033721b938399a Olivier Chafik olivier.chafik@gmail.com 2025-05-26T06:16:37-07:00 GitHub noreply@github.com 2025-05-26T14:16:37+01:00 server: fix regression on streamed non-chat completion w/ stops (#13785) 79c137f77677b3c8ee3c60a7da033721b938399a 22229314fc46b2f741bb21b12cde71f6c6a60b52 Georgi Gerganov ggerganov@gmail.com 2025-05-26T14:03:54+03:00 GitHub noreply@github.com 2025-05-26T14:03:54+03:00 examples : allow extracting embeddings from decoder contexts (#13797) 22229314fc46b2f741bb21b12cde71f6c6a60b52 9012eb9b454a82eaa4cd77ae904c0ea391e4db42 Georgi Gerganov ggerganov@gmail.com 2025-05-26T12:57:50+03:00 GitHub noreply@github.com 2025-05-26T12:57:50+03:00 llama : clarify deprecation message (#13794) 9012eb9b454a82eaa4cd77ae904c0ea391e4db42 fef693dc6b959a8e8ba11558fbeaad0b264dd457 Romain Biessy romain.biessy@codeplay.com 2025-05-26T10:28:53+02:00 GitHub noreply@github.com 2025-05-26T10:28:53+02:00 sycl: Add more debug prints (#13640) fef693dc6b959a8e8ba11558fbeaad0b264dd457 2d38b6e4004fb1c341723e657fb1e71a4d3fb473 Jeff Bolz jbolz@nvidia.com 2025-05-25T23:02:07-05:00 GitHub noreply@github.com 2025-05-26T06:02:07+02:00 vulkan: mark IM2COL as supporting non-contig (#13783) 2d38b6e4004fb1c341723e657fb1e71a4d3fb473 e121edc4324a640be11b7e567edd39b721b0f8e4 Bizhao Shi 37729561+shibizhao@users.noreply.github.com 2025-05-26T10:20:18+08:00 GitHub noreply@github.com 2025-05-26T10:20:18+08:00 CANN: Add the basic supports of Flash Attention kernel (#13627) e121edc4324a640be11b7e567edd39b721b0f8e4 2f099b510f460374acd52742b494595e3e3442d3 Olivier Chafik olivier.chafik@gmail.com 2025-05-26T00:30:51+01:00 GitHub noreply@github.com 2025-05-26T00:30:51+01:00 `server`: add `--reasoning-budget 0` to disable thinking (incl. qwen3 w/ enable_thinking:false) (#13771) 2f099b510f460374acd52742b494595e3e3442d3 aa50ba462f63759e1731227f20f5009cfc2a0f16 Xuan-Son Nguyen son@huggingface.co 2025-05-25T19:02:18+02:00 GitHub noreply@github.com 2025-05-25T18:02:18+01:00 webui : bump max upload file size to 500MB (#13779) aa50ba462f63759e1731227f20f5009cfc2a0f16 de2ef53a4b2c0d703749a309d19fe68fd8f1b9ac Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-25T16:22:29+02:00 GitHub noreply@github.com 2025-05-25T16:22:29+02:00 tests : improve UGM tokenizer test coverage (#13773) de2ef53a4b2c0d703749a309d19fe68fd8f1b9ac c508256db2de2b032e19c8ed833f4683c827c9a1 Georgi Gerganov ggerganov@gmail.com 2025-05-25T16:34:36+03:00 GitHub noreply@github.com 2025-05-25T16:34:36+03:00 kv-cache : rework kv_cell (#13706) c508256db2de2b032e19c8ed833f4683c827c9a1 40aaa8a403df5dcfb515eb2fd7a817fd99779526 Percy Piper piper.percy@googlemail.com 2025-05-25T13:35:53+01:00 GitHub noreply@github.com 2025-05-25T15:35:53+03:00 rpc : Fix build on OpenBSD (#13541) 40aaa8a403df5dcfb515eb2fd7a817fd99779526 a08c1d2845dc279d58d826ef3c3ecad97cbbcef7 Xuan-Son Nguyen son@huggingface.co 2025-05-25T14:06:32+02:00 GitHub noreply@github.com 2025-05-25T14:06:32+02:00 mtmd : add support for Qwen2-Audio and SeaLLM-Audio (#13760) a08c1d2845dc279d58d826ef3c3ecad97cbbcef7 d785f9c1fd1a1929c6d0e2a0b12cae5db867908b ddpasa 112642920+ddpasa@users.noreply.github.com 2025-05-25T14:04:49+02:00 GitHub noreply@github.com 2025-05-25T14:04:49+02:00 docs : add Moondream2 pre-quantized link (#13745) d785f9c1fd1a1929c6d0e2a0b12cae5db867908b 4032ca406632212b075e3c61c2a4476128321410 Olivier Chafik olivier.chafik@gmail.com 2025-05-25T10:45:49+01:00 GitHub noreply@github.com 2025-05-25T10:45:49+01:00 server: fix/test add_generation_prompt (#13770) 4032ca406632212b075e3c61c2a4476128321410 515fdbf7ed839dfe6a24aeb6225936609a7f6d6d Piotr Jasiukajtis estibi@me.com 2025-05-25T10:29:43+02:00 GitHub noreply@github.com 2025-05-25T10:29:43+02:00 llama : add support for Qwen3 MoE tied word embeddings (#13768) 515fdbf7ed839dfe6a24aeb6225936609a7f6d6d f5cd27b71da3ac375a04a41643d14fc779a8057b Akarshan Biswas akarshan@menlo.ai 2025-05-25T12:38:37+05:30 GitHub noreply@github.com 2025-05-25T10:08:37+03:00 SYCL: revert "sycl: simplify bin_bcast_kernel (#13383)" (#13752) f5cd27b71da3ac375a04a41643d14fc779a8057b a2d02d5793fd9af7a7224773456501691b95fd02 Olivier Chafik olivier.chafik@gmail.com 2025-05-25T01:48:08+01:00 GitHub noreply@github.com 2025-05-25T01:48:08+01:00 `server`: streaming of tool calls and thoughts when `--jinja` is on (#12379) a2d02d5793fd9af7a7224773456501691b95fd02 17fc817b58942569c43aa63299cedde217b61f68 Diego Devesa slarengh@gmail.com 2025-05-24T15:55:16-07:00 GitHub noreply@github.com 2025-05-25T00:55:16+02:00 releases : bundle llvm omp library in windows release (#13763) 17fc817b58942569c43aa63299cedde217b61f68 2bd1b30f6979235ec67b95c183b9b77baa7ab9ce Diego Devesa slarengh@gmail.com 2025-05-24T13:27:03-07:00 GitHub noreply@github.com 2025-05-24T22:27:03+02:00 releases : enable openmp in windows cpu backend build (#13756) 2bd1b30f6979235ec67b95c183b9b77baa7ab9ce 259469c4b57c1a32606353bcac52ba683424a990 Diego Devesa slarengh@gmail.com 2025-05-24T13:26:47-07:00 GitHub noreply@github.com 2025-05-24T22:26:47+02:00 ggml-cpu : set openmp wait time if not set (#13758) 259469c4b57c1a32606353bcac52ba683424a990 4c32832c59e97d96c19349fdc92763e8949fda83 0cc4m picard12@live.de 2025-05-24T16:49:12+02:00 GitHub noreply@github.com 2025-05-24T16:49:12+02:00 Move GLM4 f32 attention fix to the correct function (#13750) 4c32832c59e97d96c19349fdc92763e8949fda83 c3a2624339187e89c4f65fd72a5fe7103968b5ad Xuan-Son Nguyen son@huggingface.co 2025-05-24T13:06:47+02:00 GitHub noreply@github.com 2025-05-24T13:06:47+02:00 ggml : add ggml_gelu_erf() CUDA kernel (#13719) c3a2624339187e89c4f65fd72a5fe7103968b5ad ffd0eae60b7642e942c8245b89642527e36099e6 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-24T12:29:09+02:00 GitHub noreply@github.com 2025-05-24T12:29:09+02:00 vocab : fix ugm tokenizer precision (#13743) ffd0eae60b7642e942c8245b89642527e36099e6 b775345d788ac16260e7eef49e11fe57ee5677f7 Johannes Gäßler johannesg@5d6.de 2025-05-24T11:46:19+02:00 GitHub noreply@github.com 2025-05-24T11:46:19+02:00 CUDA: fix race condition in FA vector kernels (#13742) b775345d788ac16260e7eef49e11fe57ee5677f7 a70a8a69c2ad3de8d5525ad2b185b098011be2e8 Diego Devesa slarengh@gmail.com 2025-05-23T13:14:00-07:00 GitHub noreply@github.com 2025-05-23T23:14:00+03:00 ci : enable winget package updates (#13734) a70a8a69c2ad3de8d5525ad2b185b098011be2e8 d13d0f6135803822ec1cd7e3efb49360b88a1bdf Diego Devesa slarengh@gmail.com 2025-05-23T13:09:38-07:00 GitHub noreply@github.com 2025-05-23T22:09:38+02:00 ci : add winget package updater (#13732) d13d0f6135803822ec1cd7e3efb49360b88a1bdf 8a2afb7520bbc8f9fa1bbe314d5f2807eb0116b2 Georgi Gerganov ggerganov@gmail.com 2025-05-23T20:16:13+03:00 GitHub noreply@github.com 2025-05-23T20:16:13+03:00 hparams : initialize arrays (#13728) 8a2afb7520bbc8f9fa1bbe314d5f2807eb0116b2 9ecf3e66a38f20e41d221f62f05b17f70d040839 Xuan-Son Nguyen son@huggingface.co 2025-05-23T17:07:04+02:00 GitHub noreply@github.com 2025-05-23T17:07:04+02:00 llama : allow custom list of swa_layers (#13726) 9ecf3e66a38f20e41d221f62f05b17f70d040839 faaaff5f947064d13ef8b98659d81a1384c3e57b Xuan-Son Nguyen son@huggingface.co 2025-05-23T11:03:47+02:00 GitHub noreply@github.com 2025-05-23T11:03:47+02:00 server : support audio input (#13714) faaaff5f947064d13ef8b98659d81a1384c3e57b e16c4731c7a6bfa8f61b5b39c77245a37e7fc232 Chenguang Li 757486878@qq.com 2025-05-23T16:47:53+08:00 GitHub noreply@github.com 2025-05-23T16:47:53+08:00 CANN: Support MUL_MAT_ID for q8_0 and q4_0 (#13705) e16c4731c7a6bfa8f61b5b39c77245a37e7fc232 1dcd01960c33f52afb782b3d850fc2149a08cc6b Xuan-Son Nguyen son@huggingface.co 2025-05-23T08:12:48+02:00 GitHub noreply@github.com 2025-05-23T08:12:48+02:00 ggml : fix the order of ggml_unary_op (#13718) 1dcd01960c33f52afb782b3d850fc2149a08cc6b c10ed6cbcc3904b7d6bbcd137589eebd899aa38f Jeff Bolz jbolz@nvidia.com 2025-05-23T00:45:02-04:00 GitHub noreply@github.com 2025-05-23T06:45:02+02:00 vulkan: support CPY from any type to itself (#13695) c10ed6cbcc3904b7d6bbcd137589eebd899aa38f a127ff17800452075bb323277e3b9f8db8612ced Jeff Bolz jbolz@nvidia.com 2025-05-23T00:33:45-04:00 GitHub noreply@github.com 2025-05-23T06:33:45+02:00 vulkan: Disable coopmat/coopmat2/bfloat extensions if glslc doesn't support it (#13696) a127ff17800452075bb323277e3b9f8db8612ced 3079e9ac8e04ef6eddeb0c164d72edb6b6fd2df5 Judd 4046440+foldl@users.noreply.github.com 2025-05-23T12:33:08+08:00 GitHub noreply@github.com 2025-05-23T06:33:08+02:00 use LOG_WARN to replace `std::cerr` (#13657) 3079e9ac8e04ef6eddeb0c164d72edb6b6fd2df5 8a1d206f1d2b4e45918b589f3165b4be232f7ba8 Diego Devesa slarengh@gmail.com 2025-05-22T15:21:37-07:00 GitHub noreply@github.com 2025-05-23T00:21:37+02:00 release : fix windows hip release (#13707) 8a1d206f1d2b4e45918b589f3165b4be232f7ba8 797990c4bca0dca5be295c63e3fb2800dc0a69c2 Georgi Gerganov ggerganov@gmail.com 2025-05-22T22:21:07+03:00 GitHub noreply@github.com 2025-05-22T22:21:07+03:00 tts : fix n_ubatch + make WavTokenizer cache-less (#13713) 797990c4bca0dca5be295c63e3fb2800dc0a69c2 ab86335760ebb441574eb47f886fa1ee302e2131 Xuan-Son Nguyen son@huggingface.co 2025-05-22T20:42:48+02:00 GitHub noreply@github.com 2025-05-22T20:42:48+02:00 mtmd : add ultravox audio input (#13623) ab86335760ebb441574eb47f886fa1ee302e2131 cc74d5be990e37f201591fd868a92e64abdbf902 Aaron Teo aaron.teo1@ibm.com 2025-05-23T02:31:29+08:00 GitHub noreply@github.com 2025-05-22T21:31:29+03:00 common: Include torch package for s390x (#13699) cc74d5be990e37f201591fd868a92e64abdbf902 5be24af73d77ea23d399726f1d2a01a70ee86331 Georgi Gerganov ggerganov@gmail.com 2025-05-22T16:33:39+03:00 GitHub noreply@github.com 2025-05-22T16:33:39+03:00 server : pad small embedding batches (#13692) 5be24af73d77ea23d399726f1d2a01a70ee86331 d394a9aedc50a13b7f6373416f7c1ccabfe79c32 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-22T14:25:05+02:00 GitHub noreply@github.com 2025-05-22T14:25:05+02:00 gguf-py : correct charsmap parameter typing (#13701) d394a9aedc50a13b7f6373416f7c1ccabfe79c32 6b56a64690a318fcabcd7739ac7e314d44785ea8 Nicolò Scipione nicolo.scipione@codeplay.com 2025-05-22T13:54:43+02:00 GitHub noreply@github.com 2025-05-22T12:54:43+01:00 sycl : Remove waits from function calls (#13702) 6b56a64690a318fcabcd7739ac7e314d44785ea8 a4e8912dfd4604be1e39bc86ba4c0b02969967ef Ewan Crawford ewan@codeplay.com 2025-05-22T09:24:09+01:00 GitHub noreply@github.com 2025-05-22T16:24:09+08:00 SYCL: Avoid using with SYCL-Graph for unsupported nodes (#13587) a4e8912dfd4604be1e39bc86ba4c0b02969967ef edbf42edfdabb9cea72ae12137570cf48f5d8076 Henry Linjamäki henry.mikael.linjamaki@intel.com 2025-05-22T02:21:45+03:00 GitHub noreply@github.com 2025-05-21T16:21:45-07:00 opencl: Add support for multiple devices (#12622) edbf42edfdabb9cea72ae12137570cf48f5d8076 d643bb2c798df9c2cd61067d2692b1cd417df402 Henry Linjamäki henry.mikael.linjamaki@intel.com 2025-05-21T23:21:17+03:00 GitHub noreply@github.com 2025-05-21T13:21:17-07:00 opencl: fix couple crashes (#12795) d643bb2c798df9c2cd61067d2692b1cd417df402 8e186ef0e764c7a620e402d1f76ebad60bf31c49 Diego Devesa slarengh@gmail.com 2025-05-21T13:09:57-07:00 GitHub noreply@github.com 2025-05-21T22:09:57+02:00 releases : build CPU backend separately (windows) (#13642) 8e186ef0e764c7a620e402d1f76ebad60bf31c49 5fbfe384d4659f81c47a477eb8ee97692c7ffef9 Georgi Gerganov ggerganov@gmail.com 2025-05-21T20:00:49+03:00 GitHub noreply@github.com 2025-05-21T20:00:49+03:00 hparams : support models for which all layers use SWA (#13682) 5fbfe384d4659f81c47a477eb8ee97692c7ffef9 c76532e7ba128bb097bf6836bf0f5592e1b56b76 Georgi Gerganov ggerganov@gmail.com 2025-05-21T19:46:56+03:00 GitHub noreply@github.com 2025-05-21T19:46:56+03:00 server : improve error reporting (#13680) c76532e7ba128bb097bf6836bf0f5592e1b56b76 2aa777d86d3f7bb80b93e226f1c25e47825f6a83 antichristHater 142441588+antichristHater@users.noreply.github.com 2025-05-21T19:40:35+03:00 GitHub noreply@github.com 2025-05-21T18:40:35+02:00 convert : add qwen2vl support for unsloth merges (#13686) 2aa777d86d3f7bb80b93e226f1c25e47825f6a83 eb0f5c28d37126baa756117d5bdaadc62e03344e Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-21T16:57:38+02:00 GitHub noreply@github.com 2025-05-21T16:57:38+02:00 examples : switch retrieval to llama_encode (#13685) eb0f5c28d37126baa756117d5bdaadc62e03344e cf4cb59e64d72a1b4c781f71a74de5756a4e2376 Emmanuel Ferdman emmanuelferdman@gmail.com 2025-05-21T17:33:54+03:00 GitHub noreply@github.com 2025-05-21T16:33:54+02:00 gguf-py : display the invalid gguf type (#13687) cf4cb59e64d72a1b4c781f71a74de5756a4e2376 0d5c74216170ef97e5e7511563837263f2d1a496 Xuan-Son Nguyen son@huggingface.co 2025-05-21T16:26:33+02:00 GitHub noreply@github.com 2025-05-21T16:26:33+02:00 ggml : add ggml_gelu_erf() (#13667) 0d5c74216170ef97e5e7511563837263f2d1a496 42158ae2e8ead667a83f07247321ce85f32ace66 Robin Davidsson 40024429+R-Dson@users.noreply.github.com 2025-05-21T15:15:27+02:00 GitHub noreply@github.com 2025-05-21T15:15:27+02:00 server : Add the endpoints /api/tags and /api/chat (#13659) 42158ae2e8ead667a83f07247321ce85f32ace66 797f2ac0625b22edeff03cc30e0f988da6b6b068 Dorin-Andrei Geman doringeman@gmail.com 2025-05-21T16:07:57+03:00 GitHub noreply@github.com 2025-05-21T15:07:57+02:00 server : fix first message identification (#13634) 797f2ac0625b22edeff03cc30e0f988da6b6b068 b44890df2e4fad0ece1d5366dcbc8bedae23b658 Georgi Gerganov ggerganov@gmail.com 2025-05-21T15:11:13+03:00 GitHub noreply@github.com 2025-05-21T15:11:13+03:00 kv-cache : simplify the interface (#13660) b44890df2e4fad0ece1d5366dcbc8bedae23b658 33983057d0f578aca74ba15eccc3de9c267a5ff6 Georgi Gerganov ggerganov@gmail.com 2025-05-21T13:09:21+03:00 GitHub noreply@github.com 2025-05-21T13:09:21+03:00 model : disable SWA for Phi models (#13676) 33983057d0f578aca74ba15eccc3de9c267a5ff6 fb1cab201c6c4cd36731f100df74eece2f4706fa R0CKSTAR yeahdongcn@gmail.com 2025-05-21T09:58:49+08:00 GitHub noreply@github.com 2025-05-21T09:58:49+08:00 musa: Upgrade MUSA SDK version to rc4.0.1 and use mudnn::Unary::IDENTITY op to accelerate D2D memory copy (#13647) fb1cab201c6c4cd36731f100df74eece2f4706fa b7a17463ec190aeee7b9077c606c910fb4688b84 Eve 139727413+netrunnereve@users.noreply.github.com 2025-05-20T21:35:16Z GitHub noreply@github.com 2025-05-20T21:35:16Z vulkan: fix warnings (#13626) b7a17463ec190aeee7b9077c606c910fb4688b84 be0239693c1530a18496086331fc18d8a9adbad1 l3utterfly gc.pthzfoldr@gmail.com 2025-05-21T00:55:30+08:00 GitHub noreply@github.com 2025-05-20T18:55:30+02:00 mtmd-helper : bug fix to token batching in mtmd (#13650) be0239693c1530a18496086331fc18d8a9adbad1 a4090d1174aed22dde5cacce2a4c27656b987a2f Georgi Gerganov ggerganov@gmail.com 2025-05-20T19:21:04+03:00 GitHub noreply@github.com 2025-05-20T19:21:04+03:00 model : fix llama4 graph (#13663) a4090d1174aed22dde5cacce2a4c27656b987a2f b69f1647f9953cb3773266d2c83a92fd0e7e6d66 Georgi Gerganov ggerganov@gmail.com 2025-05-20T16:13:16+03:00 GitHub noreply@github.com 2025-05-20T16:13:16+03:00 llama : remove llama_kv_cache_view API + remove deprecated (#13653) b69f1647f9953cb3773266d2c83a92fd0e7e6d66 759e37b0d89bc4bd1bce860dc5f3c3052e08575c Johannes Gäßler johannesg@5d6.de 2025-05-20T14:45:07+02:00 GitHub noreply@github.com 2025-05-20T14:45:07+02:00 CUDA: skip fully masked-out KV in FA vec kernel (#13584) 759e37b0d89bc4bd1bce860dc5f3c3052e08575c 4245e622e0cc3af1ca3056104e465dc4d303bd7d Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-20T12:03:17+02:00 GitHub noreply@github.com 2025-05-20T12:03:17+02:00 tests : avoid github urls due to throttling (#13654) 4245e622e0cc3af1ca3056104e465dc4d303bd7d c9c64dee572c5eedf073a6c6eb5d92d8283f7639 Svetlozar Georgiev 55534064+sgeor255@users.noreply.github.com 2025-05-20T10:34:15+01:00 GitHub noreply@github.com 2025-05-20T11:34:15+02:00 sycl: disable reorder for sycl mulmat (#13536) c9c64dee572c5eedf073a6c6eb5d92d8283f7639 c00a2634bec49805c6b31438b1a6006a2bd793cb 0cc4m picard12@live.de 2025-05-20T10:11:56+02:00 GitHub noreply@github.com 2025-05-20T10:11:56+02:00 Set GLM4 blk.*.attn_output.weight, kqv_out-* matmul to GGML_PREC_F32 to fix infinity values in output (#13639) c00a2634bec49805c6b31438b1a6006a2bd793cb e298d2fbd082a52c0f6ed02729f94e9bf630cf17 Georgi Gerganov ggerganov@gmail.com 2025-05-20T10:41:40+03:00 GitHub noreply@github.com 2025-05-20T10:41:40+03:00 metal : fix typo in FA kernel comments (#13651) e298d2fbd082a52c0f6ed02729f94e9bf630cf17 f0adb80bf7c2c0d80abb04f4533b5513622d9964 Georgi Gerganov ggerganov@gmail.com 2025-05-20T08:05:46+03:00 GitHub noreply@github.com 2025-05-20T08:05:46+03:00 kv-cache : add SWA support (#13194) f0adb80bf7c2c0d80abb04f4533b5513622d9964 f7c9429c85748cde9599499601ba48d0057722e6 Xinpeng Dou 15529241576@163.com 2025-05-20T11:43:43+08:00 GitHub noreply@github.com 2025-05-20T11:43:43+08:00 CANN: Update CANN model support (#13162) f7c9429c85748cde9599499601ba48d0057722e6 1dfbf2cf3a9f15193dd893396d07762bbd2c4785 Nicolò Scipione nicolo.scipione@codeplay.com 2025-05-20T02:54:43+02:00 GitHub noreply@github.com 2025-05-20T08:54:43+08:00 sycl : Overcoming workaround for mmap() allocation on Windows (#13482) 1dfbf2cf3a9f15193dd893396d07762bbd2c4785 8960efd0a65e5a4830a14f6937c10703534f2569 psocolovsky 50770545+psocolovsky@users.noreply.github.com 2025-05-19T21:17:36+02:00 GitHub noreply@github.com 2025-05-19T21:17:36+02:00 common : add load_progress_callback (#13617) 8960efd0a65e5a4830a14f6937c10703534f2569 725f23f1f3f0d3adf49f95d8dfa6e7c74adff149 0cc4m picard12@live.de 2025-05-19T17:54:08+02:00 GitHub noreply@github.com 2025-05-19T17:54:08+02:00 Vulkan: Add f32 accumulator support to quantized mul mat to fix GLM4 32B incoherence (#13607) 725f23f1f3f0d3adf49f95d8dfa6e7c74adff149 92ecdcc06a4c405a415bcaa0cb772bc560aa23b1 Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2025-05-19T14:38:20+01:00 GitHub noreply@github.com 2025-05-19T14:38:20+01:00 sycl : backend documentation review (#13544) 92ecdcc06a4c405a415bcaa0cb772bc560aa23b1 f71f40a2847d4c9f57b86cd206e0a27b2bfb6d1c Xuan-Son Nguyen son@huggingface.co 2025-05-19T13:04:14+02:00 GitHub noreply@github.com 2025-05-19T13:04:14+02:00 mtmd : add vision support for llama 4 (#13282) f71f40a2847d4c9f57b86cd206e0a27b2bfb6d1c d30cb5a7fa17362c47e94a023276f169916e0d03 Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2025-05-19T11:46:09+01:00 GitHub noreply@github.com 2025-05-19T11:46:09+01:00 ci : upgraded oneAPI version in SYCL workflows and dockerfile (#13532) d30cb5a7fa17362c47e94a023276f169916e0d03 6c35981a643d4f74a286268b62f65bfa156af2e6 Georgi Gerganov ggerganov@gmail.com 2025-05-19T12:50:29+03:00 Georgi Gerganov ggerganov@gmail.com 2025-05-19T13:29:56+03:00 sync : ggml 6c35981a643d4f74a286268b62f65bfa156af2e6 8b5e19aea6ce9fe4452598663924373234041440 Johannes Gäßler johannesg@5d6.de 2025-05-19T09:33:35+02:00 Georgi Gerganov ggerganov@gmail.com 2025-05-19T13:29:56+03:00 mnist: fix segmentation fault (ggml/1227) 8b5e19aea6ce9fe4452598663924373234041440 60aea028b575ab54e0dfbb31db641bb93d2ae8c4 Diego Devesa slarengh@gmail.com 2025-05-18T18:30:13-07:00 Georgi Gerganov ggerganov@gmail.com 2025-05-19T13:29:56+03:00 ggml : fix apple OS check in ggml_print_backtrace (ggml/1229) 60aea028b575ab54e0dfbb31db641bb93d2ae8c4 9c55e5c5c24990dd17fd6c8f4f2159052d2b06f1 Daniel Tang danielzgtg.opensource@gmail.com 2025-05-17T19:06:26-04:00 Georgi Gerganov ggerganov@gmail.com 2025-05-19T13:29:56+03:00 ggml : Fix missing backtrace on Linux (ggml/1228) 9c55e5c5c24990dd17fd6c8f4f2159052d2b06f1 33d7aed4a83e7bbecac4af535208d4ed3e1ca8fd Nick 0x0b4ac@gmail.com 2025-05-19T18:25:41+08:00 GitHub noreply@github.com 2025-05-19T13:25:41+03:00 fix: check model pointer validity before use (#13631) 33d7aed4a83e7bbecac4af535208d4ed3e1ca8fd 6a2bc8bfb7cd502e5ebc72e36c97a6f848c21c2c Chenguang Li 757486878@qq.com 2025-05-19T14:21:17+08:00 GitHub noreply@github.com 2025-05-19T14:21:17+08:00 CANN: Support MOE Model MUL_MAT_ID (#13042) 6a2bc8bfb7cd502e5ebc72e36c97a6f848c21c2c e3a7cf6c5bf6a0a24217f88607b06e4405a2b5d9 Isaac McFadyen isaac@imcf.me 2025-05-17T17:59:48-04:00 GitHub noreply@github.com 2025-05-17T23:59:48+02:00 server : added --no-prefill-assistant flag (#13608) e3a7cf6c5bf6a0a24217f88607b06e4405a2b5d9 518329b2d4434d0683c30b741b4f4cf5734b5f99 Gilad S. 7817232+giladgd@users.noreply.github.com 2025-05-17T21:26:43+03:00 GitHub noreply@github.com 2025-05-17T15:26:43-03:00 cmake: use the current build config for vulkan-shaders-gen (#13595) 518329b2d4434d0683c30b741b4f4cf5734b5f99 2f5a4e1e09567e09be8b84a5f976334de9539d17 Georgi Gerganov ggerganov@gmail.com 2025-05-17T12:58:55+03:00 GitHub noreply@github.com 2025-05-17T12:58:55+03:00 parallel : add option for non-shared and larger prompts (#13598) 2f5a4e1e09567e09be8b84a5f976334de9539d17 4f41ee11d6a4ddb02e4644922bf0b56880ee6f55 Jeff Bolz jbolz@nvidia.com 2025-05-17T16:14:55+09:00 GitHub noreply@github.com 2025-05-17T09:14:55+02:00 vulkan: move common FA code to flash_attn_base.comp (#13556) 4f41ee11d6a4ddb02e4644922bf0b56880ee6f55 3e0be1cacef290c99cbb99ceaa433b4344f87355 Jeff Bolz jbolz@nvidia.com 2025-05-17T15:35:47+09:00 GitHub noreply@github.com 2025-05-17T08:35:47+02:00 vulkan: use scalar FA rather than coopmat2 when N==1 (#13554) 3e0be1cacef290c99cbb99ceaa433b4344f87355 6aa892ec2aa7fe0c93e87c4b970d83a942fb9454 Z coffeevampirebusiness@gmail.com 2025-05-16T14:56:28-06:00 GitHub noreply@github.com 2025-05-16T22:56:28+02:00 llguidance : official v0.7.20 release (no actual changes) [noci] (#13594) 6aa892ec2aa7fe0c93e87c4b970d83a942fb9454 aea9f8b4e73876739bf88fb705502f291294e469 Xuan-Son Nguyen son@huggingface.co 2025-05-16T21:50:00+02:00 GitHub noreply@github.com 2025-05-16T21:50:00+02:00 server : do not return error out of context (with ctx shift disabled) (#13577) aea9f8b4e73876739bf88fb705502f291294e469 06c1e4abc1e50ef6dcf6369f9685ca8fe82d21fe Xuan-Son Nguyen son@huggingface.co 2025-05-16T21:49:01+02:00 GitHub noreply@github.com 2025-05-16T21:49:01+02:00 webui : improve accessibility for visually impaired people (#13551) 06c1e4abc1e50ef6dcf6369f9685ca8fe82d21fe 415e40a357002d5d9b7a97ef20ccea9f4ae04c80 Xuan-Son Nguyen son@huggingface.co 2025-05-16T20:04:18+02:00 GitHub noreply@github.com 2025-05-16T20:04:18+02:00 readme : add list of dependencies and their license (#13591) 415e40a357002d5d9b7a97ef20ccea9f4ae04c80 654a67794f7a420c6931de2f4c145eaaade5dd16 Diego Devesa slarengh@gmail.com 2025-05-16T10:36:51-07:00 GitHub noreply@github.com 2025-05-16T19:36:51+02:00 releases : use arm version of curl for arm releases (#13592) 654a67794f7a420c6931de2f4c145eaaade5dd16 5364ae4ba53cc6367b8c8bf78876839122ca4e57 Georgi Gerganov ggerganov@gmail.com 2025-05-16T20:32:58+03:00 GitHub noreply@github.com 2025-05-16T20:32:58+03:00 metal : add FA-vec kernel for head size 64 (#13583) 5364ae4ba53cc6367b8c8bf78876839122ca4e57 7c07ac244d59c833bf209582f6df019a77cdda59 Diego Devesa slarengh@gmail.com 2025-05-16T07:38:07-07:00 GitHub noreply@github.com 2025-05-16T16:38:07+02:00 llama : print hint when loading a model when no backends are loaded (#13589) 7c07ac244d59c833bf209582f6df019a77cdda59 0a338ed013c23aecdce6449af736a35a465fa60f Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-16T14:54:23+02:00 GitHub noreply@github.com 2025-05-16T14:54:23+02:00 ci : add ppc64el to build-linux-cross (#13575) 0a338ed013c23aecdce6449af736a35a465fa60f bc098c3cf0aac93e57e9bda9d95e2456acf88894 Łukasz Ślusarczyk 112692748+lslusarczyk@users.noreply.github.com 2025-05-16T12:15:29+02:00 GitHub noreply@github.com 2025-05-16T18:15:29+08:00 sycl : fixed compilation warnings (#13582) bc098c3cf0aac93e57e9bda9d95e2456acf88894 c6a2c9e7411f54b0770b319740561bbd6a2ebd27 Olivier Chafik olivier.chafik@gmail.com 2025-05-15T23:29:10+01:00 GitHub noreply@github.com 2025-05-15T23:29:10+01:00 minja: sync (qwen3) (#13573) c6a2c9e7411f54b0770b319740561bbd6a2ebd27 07ad2b6db3c117868728e2cdfe3b711473d66e14 Diego Devesa slarengh@gmail.com 2025-05-15T10:13:11-07:00 GitHub noreply@github.com 2025-05-15T19:13:11+02:00 gguf : use ggml log system (#13571) 07ad2b6db3c117868728e2cdfe3b711473d66e14 c531edfa34fec8074d0b424396cdd450df23b612 Daniel Tang danielzgtg.opensource@gmail.com 2025-05-15T12:47:10-04:00 GitHub noreply@github.com 2025-05-15T18:47:10+02:00 gguf-py : fix disconnect-before-connect in editor-gui (#13569) c531edfa34fec8074d0b424396cdd450df23b612 02cdd2d8b092b5a4bb18e013c6887ce49ba20ac5 Xuan-Son Nguyen son@huggingface.co 2025-05-15T17:40:07+02:00 GitHub noreply@github.com 2025-05-15T17:40:07+02:00 convert : fix conversion for llama 4 (#13567) 02cdd2d8b092b5a4bb18e013c6887ce49ba20ac5 64bb51cf90d3eede8c150a23d59a0c718b78065b Atharva Dubey atharva.dubey@codeplay.com 2025-05-15T16:39:52+01:00 GitHub noreply@github.com 2025-05-15T17:39:52+02:00 sycl: simplify bin_bcast_kernel (#13383) 64bb51cf90d3eede8c150a23d59a0c718b78065b 9c404ed54c3c8d8d2aa3153313766c8286739387 Svetlozar Georgiev 55534064+sgeor255@users.noreply.github.com 2025-05-15T16:35:44+01:00 GitHub noreply@github.com 2025-05-15T17:35:44+02:00 sycl: reordered Q4_K MMVQ (#13109) 9c404ed54c3c8d8d2aa3153313766c8286739387 6c8b91500e75df6664278d1e9af3e39e8a2fb0d0 Łukasz Ślusarczyk 112692748+lslusarczyk@users.noreply.github.com 2025-05-15T16:53:41+02:00 GitHub noreply@github.com 2025-05-15T16:53:41+02:00 sycl: use oneDNN for matrices multiplication (#12972) 6c8b91500e75df6664278d1e9af3e39e8a2fb0d0 3cc1f1f1d24472a6558c942b1c78989ff4b0e569 Diego Devesa slarengh@gmail.com 2025-05-15T06:46:55-07:00 GitHub noreply@github.com 2025-05-15T15:46:55+02:00 llama-bench : fix -ot with dl backends (#13563) 3cc1f1f1d24472a6558c942b1c78989ff4b0e569 c753d7bed0dc2cc2d5c42dfa9806cba91748392e Xuan-Son Nguyen son@huggingface.co 2025-05-15T14:24:50+02:00 GitHub noreply@github.com 2025-05-15T14:24:50+02:00 webui : handle PDF input (as text or image) + convert pasted long content to file (#13562) c753d7bed0dc2cc2d5c42dfa9806cba91748392e b2838049ccf50859cea4c390e81405c2fb01e820 Piotr Wilkin (ilintar) piotr.wilkin@syndatis.com 2025-05-15T08:40:58+02:00 GitHub noreply@github.com 2025-05-15T08:40:58+02:00 server : proper error handling for missing elements in messages array (OpenAI compatible backend) (#13540) b2838049ccf50859cea4c390e81405c2fb01e820 aa48e373f256df9608395ee6881e7010840d6202 Georgi Gerganov ggerganov@gmail.com 2025-05-15T05:57:02+03:00 GitHub noreply@github.com 2025-05-15T05:57:02+03:00 bench : handle decode errors (#13548) aa48e373f256df9608395ee6881e7010840d6202 e3a9421b78da5c810d812e6348a405f5acc39f34 Olivier Chafik ochafik@users.noreply.github.com 2025-05-15T02:39:51+01:00 GitHub noreply@github.com 2025-05-15T02:39:51+01:00 `server`: inject date_string in llama 3.x template + fix date for firefunction v2 (#12802) e3a9421b78da5c810d812e6348a405f5acc39f34 5ab5d5fb256aa23f8ab4de8463515ea627f43006 Georgi Gerganov ggerganov@gmail.com 2025-05-14T23:15:15+03:00 GitHub noreply@github.com 2025-05-14T23:15:15+03:00 kv-cache : fix out-of-bounds view during reserve graph (#13547) 5ab5d5fb256aa23f8ab4de8463515ea627f43006 3198405e98530c683d12fd123e3920f2bd2aafa5 Yibo Cai cyb70289@gmail.com 2025-05-15T03:53:52+08:00 GitHub noreply@github.com 2025-05-14T21:53:52+02:00 arm64: optimize q6_k_q8_k kernel with i8mm (#13519) 3198405e98530c683d12fd123e3920f2bd2aafa5 f5170c1d7a66222ca7c75d2022fec3ed87257e0b Olivier Chafik ochafik@users.noreply.github.com 2025-05-14T19:50:57+01:00 GitHub noreply@github.com 2025-05-14T19:50:57+01:00 `common`: add partial regex support (#12808) f5170c1d7a66222ca7c75d2022fec3ed87257e0b 017f10b5fa630a013ec4f9936e410a60d4f460d5 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-14T20:22:49+02:00 GitHub noreply@github.com 2025-05-14T21:22:49+03:00 editorconfig : fix trailing whitespace from #13542 (#13546) 017f10b5fa630a013ec4f9936e410a60d4f460d5 4696d5674999dc10a7fb8c27b33406a929f7463a Gilad S. 7817232+giladgd@users.noreply.github.com 2025-05-14T19:18:18+03:00 GitHub noreply@github.com 2025-05-14T19:18:18+03:00 fix: crash when calling `llama_state_get_size` on a context without a KV cache (#13542) 4696d5674999dc10a7fb8c27b33406a929f7463a b7d26720821823e23e2273a99e38398d511242e9 Johannes Gäßler johannesg@5d6.de 2025-05-14T16:41:02+02:00 GitHub noreply@github.com 2025-05-14T16:41:02+02:00 CUDA: fix crash on large batch size for quant. MoE (#13537) b7d26720821823e23e2273a99e38398d511242e9 6da34fa27620fa56e3334172e023f4f2533df51f Diego Devesa slarengh@gmail.com 2025-05-14T07:12:36-07:00 GitHub noreply@github.com 2025-05-14T16:12:36+02:00 llama : fix quantize with dl backends (#13539) 6da34fa27620fa56e3334172e023f4f2533df51f 5e7d95e22e386d316f7f659b74c9c34b65507912 Johannes Gäßler johannesg@5d6.de 2025-05-14T16:08:20+02:00 GitHub noreply@github.com 2025-05-14T16:08:20+02:00 CUDA: faster Deepseek FA, add Turing support (#13435) 5e7d95e22e386d316f7f659b74c9c34b65507912 053174436f3b3cbb01077f26ff17809537b832c7 Gabe Goodhart ghart@us.ibm.com 2025-05-14T06:53:59-06:00 GitHub noreply@github.com 2025-05-14T15:53:59+03:00 fix: Move build_inp_pos to the top of the graph section for build_granite (#13538) 053174436f3b3cbb01077f26ff17809537b832c7 360a9c98e13d35f322b4c5b1309aab0cc90ed82b Georgi Gerganov ggerganov@gmail.com 2025-05-14T15:42:10+03:00 GitHub noreply@github.com 2025-05-14T15:42:10+03:00 server : passthrough the /models endpoint during loading (#13535) 360a9c98e13d35f322b4c5b1309aab0cc90ed82b 09d13d94fb169093095416ac6323551c37b75339 Xuan-Son Nguyen son@huggingface.co 2025-05-14T13:35:07+02:00 GitHub noreply@github.com 2025-05-14T13:35:07+02:00 server : fix cache_tokens bug with no cache_prompt (#13533) 09d13d94fb169093095416ac6323551c37b75339 24e86cae7219b0f3ede1d5abdf5bf3ad515cccb8 bandoti 141645996+bandoti@users.noreply.github.com 2025-05-14T07:53:57-03:00 GitHub noreply@github.com 2025-05-14T07:53:57-03:00 cmake: simplify vulkan shader test logic (#13263) 24e86cae7219b0f3ede1d5abdf5bf3ad515cccb8 bb1681fbd532eba26ae4c14cd8be884c8afeb31c Jeff Bolz jbolz@nvidia.com 2025-05-14T18:55:26+09:00 GitHub noreply@github.com 2025-05-14T11:55:26+02:00 vulkan: KHR_coopmat flash attention (#13506) bb1681fbd532eba26ae4c14cd8be884c8afeb31c d486dd3e8ecfba0170f8632a1530540de560f4a3 Xuan-Son Nguyen son@huggingface.co 2025-05-14T10:26:12+02:00 GitHub noreply@github.com 2025-05-14T10:26:12+02:00 webui : use fflate for more deterministic gzip compress (#13525) d486dd3e8ecfba0170f8632a1530540de560f4a3 21ca987fba504d273ea28ebc4d3e5b3736a11c8e Luca Stefani luca.stefani.ge1@gmail.com 2025-05-14T10:07:31+02:00 GitHub noreply@github.com 2025-05-14T10:07:31+02:00 webui: Allow pasting file from clipboard (#13526) 21ca987fba504d273ea28ebc4d3e5b3736a11c8e be1d4a13db26750fac702ceb3af88ae4f39dc9f4 ddpasa 112642920+ddpasa@users.noreply.github.com 2025-05-14T09:59:12+02:00 GitHub noreply@github.com 2025-05-14T09:59:12+02:00 docs: Update link to ggml-org in multimodal.md (#13513) be1d4a13db26750fac702ceb3af88ae4f39dc9f4 ab3971f2a0a526564bfde0e4cc8a5b90f9d33ad2 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-14T08:41:01+02:00 GitHub noreply@github.com 2025-05-14T08:41:01+02:00 scripts : fix compare-llama-bench.py show parameter (#13514) ab3971f2a0a526564bfde0e4cc8a5b90f9d33ad2 e5c834f718a32b7584f142799bbf508fddb9021c Jeff Bolz jbolz@nvidia.com 2025-05-14T13:15:50+09:00 GitHub noreply@github.com 2025-05-14T06:15:50+02:00 vulkan: workaround FA compile failures on macos (#13517) e5c834f718a32b7584f142799bbf508fddb9021c 71bdbdb58757d508557e6d8b387f666cdfb25c5e Ed Addario 29247825+EAddario@users.noreply.github.com 2025-05-13T18:12:31+01:00 GitHub noreply@github.com 2025-05-13T19:12:31+02:00 quantize : improve tensor-type pattern matching (#13033) 71bdbdb58757d508557e6d8b387f666cdfb25c5e f0995d28ce3d15095b6845d94ce4465e46575873 Xuan-Son Nguyen son@huggingface.co 2025-05-13T17:07:21+02:00 GitHub noreply@github.com 2025-05-13T17:07:21+02:00 clip : clip.h become private API (⚠️ breaking change) (#13510) f0995d28ce3d15095b6845d94ce4465e46575873 c252e0c4097b34666e5a81db9d0450d71fa3098f Georgi Gerganov ggerganov@gmail.com 2025-05-13T18:04:39+03:00 GitHub noreply@github.com 2025-05-13T18:04:39+03:00 metal : use FA-vec kernel up to batch size 20 (#13496) c252e0c4097b34666e5a81db9d0450d71fa3098f 4f711afed5e7ef4304b567c8888ee1aa60e868eb Georgi Gerganov ggerganov@gmail.com 2025-05-13T18:04:00+03:00 GitHub noreply@github.com 2025-05-13T18:04:00+03:00 metal : optimize multi-sequence FA vec kernel (#13493) 4f711afed5e7ef4304b567c8888ee1aa60e868eb b89d605a91dee0a518ecd582f8991a07d523e2fa Dan Johansson dan.johansson@arm.com 2025-05-13T17:02:28+02:00 GitHub noreply@github.com 2025-05-13T18:02:28+03:00 ggml-cpu: Update KleidiAI to v1.6 and fix include directives (#13509) b89d605a91dee0a518ecd582f8991a07d523e2fa b4726345aca49e2ad62d615e6e370b3dbad6434f Georgi Gerganov ggerganov@gmail.com 2025-05-13T18:01:53+03:00 GitHub noreply@github.com 2025-05-13T18:01:53+03:00 batched-bench : fix pp batch contents (#13492) b4726345aca49e2ad62d615e6e370b3dbad6434f bf7937112058f2815fc3825a9ff7b536ecafa3bb Xuan-Son Nguyen son@huggingface.co 2025-05-13T15:33:58+02:00 GitHub noreply@github.com 2025-05-13T15:33:58+02:00 mtmd : remove libllava, remove clip-quantize-cli (⚠️ breaking change) (#13460) bf7937112058f2815fc3825a9ff7b536ecafa3bb d590cd4c244e5f260c42c290b83a358b9d86d763 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-13T15:31:12+02:00 GitHub noreply@github.com 2025-05-13T15:31:12+02:00 scripts : support arbitrary input file formats in compare-llama-bench.py (#13455) d590cd4c244e5f260c42c290b83a358b9d86d763 1e2809bc4b5d8db2a9ed12ac872eca832c53f5fd Gabe Goodhart ghart@us.ibm.com 2025-05-13T07:12:01-06:00 GitHub noreply@github.com 2025-05-13T15:12:01+02:00 model : Granite MoE shared (#13269) 1e2809bc4b5d8db2a9ed12ac872eca832c53f5fd cf0a43bb6490bd49344775abb22ba26f8047cb54 Georgi Gerganov ggerganov@gmail.com 2025-05-13T14:01:45+03:00 Georgi Gerganov ggerganov@gmail.com 2025-05-13T14:02:28+03:00 sync : ggml cf0a43bb6490bd49344775abb22ba26f8047cb54 f0d46ef15717cd609a7b69cf6190edde64d466c8 Diego Devesa slarengh@gmail.com 2025-05-12T15:31:37-07:00 GitHub noreply@github.com 2025-05-13T00:31:37+02:00 llama-bench : add defrag-thold, check for invalid ranges (#13487) f0d46ef15717cd609a7b69cf6190edde64d466c8 de4c07f93783a1a96456a44dc16b9db538ee1618 lhez quic_lih@quicinc.com 2025-05-12T13:13:49-07:00 GitHub noreply@github.com 2025-05-12T13:13:49-07:00 opencl: remove unnecessary assert for `add` (#13257) de4c07f93783a1a96456a44dc16b9db538ee1618 10d2af0eaa0aafd7c6577b279dfa5221ff44a63f Xuan-Son Nguyen son@huggingface.co 2025-05-12T15:06:51+02:00 GitHub noreply@github.com 2025-05-12T15:06:51+02:00 clip : cap max image size 1024 for qwen vl model (#13478) 10d2af0eaa0aafd7c6577b279dfa5221ff44a63f 064cc596ac44308dc326a17c9e3163c34a6f29d1 Johannes Gäßler johannesg@5d6.de 2025-05-12T14:44:49+02:00 GitHub noreply@github.com 2025-05-12T14:44:49+02:00 llama/ggml: add LLM training support (#10544) 064cc596ac44308dc326a17c9e3163c34a6f29d1 91159ee9df84edb72ccf874e353d2414f3a8c60d Georgi Gerganov ggerganov@gmail.com 2025-05-12T15:12:27+03:00 GitHub noreply@github.com 2025-05-12T15:12:27+03:00 context : fix state io for memory-less contexts (#13470) 91159ee9df84edb72ccf874e353d2414f3a8c60d 22cdab343b63edd0906f1c132616746085f81983 Anudit Nagar nagaranudit@gmail.com 2025-05-12T18:56:42+07:00 GitHub noreply@github.com 2025-05-12T13:56:42+02:00 server : allow content to be null in oaicompat_completion_params_parse (#13477) 22cdab343b63edd0906f1c132616746085f81983 a71a4075cdb8e81eaaa834e48ffda88b038286bc Diego Devesa slarengh@gmail.com 2025-05-12T13:08:22+02:00 GitHub noreply@github.com 2025-05-12T13:08:22+02:00 llama-bench : accept ranges for integer parameters (#13410) a71a4075cdb8e81eaaa834e48ffda88b038286bc 95e18884fc7ea4031f70f1a518d5d1df616e5717 Dan Johansson dan.johansson@arm.com 2025-05-12T13:06:19+02:00 GitHub noreply@github.com 2025-05-12T13:06:19+02:00 ggml-cpu: Integrate fp32=bf16xbf16 SME KleidiAI kernel (#13053) 95e18884fc7ea4031f70f1a518d5d1df616e5717 df8491922f0ea4e032338186350dff2fb6b2b4e3 Johannes Gäßler johannesg@5d6.de 2025-05-12T10:51:21+02:00 GitHub noreply@github.com 2025-05-12T10:51:21+02:00 CUDA: fix misaligned synchronization in FA (#13469) df8491922f0ea4e032338186350dff2fb6b2b4e3 14492144c286bbf38bb1903128403d9e2ebad54c Xuan-Son Nguyen son@huggingface.co 2025-05-12T10:29:13+02:00 GitHub noreply@github.com 2025-05-12T10:29:13+02:00 ggml : add mrope kernel for metal (#13457) 14492144c286bbf38bb1903128403d9e2ebad54c c104023994d36a8e791fc6a43789b84fd552cefc Atharva Dubey atharva.dubey@codeplay.com 2025-05-12T06:15:32+01:00 GitHub noreply@github.com 2025-05-12T13:15:32+08:00 enable dpcpp nightly builds with libraries (#13406) c104023994d36a8e791fc6a43789b84fd552cefc 9a390c4829cd3058d26a2e2c09d16e3fd12bf1b1 City 125218114+city96@users.noreply.github.com 2025-05-12T00:39:06+02:00 GitHub noreply@github.com 2025-05-12T00:39:06+02:00 mtmd : Use RMS norm for InternVL 3 38B and 78B mmproj (#13459) 9a390c4829cd3058d26a2e2c09d16e3fd12bf1b1 09232370fc6426aa5dd9be01a8271b9c28f5af3a Anthony Umfer aumfer@gmail.com 2025-05-11T11:08:26-04:00 GitHub noreply@github.com 2025-05-11T17:08:26+02:00 tools : fix uninitialized llama_batch in server (#13436) 09232370fc6426aa5dd9be01a8271b9c28f5af3a 7474e00b34629e9cd8b06bc87ad935584ea30f8e Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-11T16:20:39+02:00 GitHub noreply@github.com 2025-05-11T16:20:39+02:00 scripts : exit compare-llama-bench.py gracefully when there's nothing to compare (#13451) 7474e00b34629e9cd8b06bc87ad935584ea30f8e 7f323a589f8684c0eb722e7309074cb5eac0c8b5 Johannes Gäßler johannesg@5d6.de 2025-05-11T16:09:33+02:00 GitHub noreply@github.com 2025-05-11T16:09:33+02:00 CUDA: fix crash with partial offloading of MoE (#13439) 7f323a589f8684c0eb722e7309074cb5eac0c8b5 3eac209319a6726fd9687c6188fc6b916b65953d David Huang 1969802+hjc4869@users.noreply.github.com 2025-05-11T20:18:39+08:00 GitHub noreply@github.com 2025-05-11T14:18:39+02:00 Add `--no-op-offload` to improve `-ot` pp perf in MoE models like llama4 400B (#13386) 3eac209319a6726fd9687c6188fc6b916b65953d a634d75d1bb4034b8c945042ceea268b5b895730 City 125218114+city96@users.noreply.github.com 2025-05-11T11:35:52+02:00 GitHub noreply@github.com 2025-05-11T11:35:52+02:00 mtmd : support InternVL 3 38B and 78B mmproj (#13443) a634d75d1bb4034b8c945042ceea268b5b895730 62d4250e52917dc4d634f054b1e2183ed7dee944 Xuan-Son Nguyen son@huggingface.co 2025-05-11T11:34:23+02:00 GitHub noreply@github.com 2025-05-11T11:34:23+02:00 mtmd : move helpers to dedicated file (#13442) 62d4250e52917dc4d634f054b1e2183ed7dee944 0208355f42bdab88a08507ead4a6302790a08323 Thomas Germer 99991@users.noreply.github.com 2025-05-10T22:26:46+02:00 GitHub noreply@github.com 2025-05-10T22:26:46+02:00 docs : Fix typo in InternVL3 model name (#13440) 0208355f42bdab88a08507ead4a6302790a08323 d2a4ef05c60506ee48e7375eb36f2257de7ab0d2 Johannes Gäßler johannesg@5d6.de 2025-05-10T22:22:48+02:00 GitHub noreply@github.com 2025-05-10T22:22:48+02:00 CUDA: fix race conditions FlashAttention kernels (#13438) d2a4ef05c60506ee48e7375eb36f2257de7ab0d2 15e6125a397f6086c1dfdf7584acdb7c730313dc Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-10T22:08:07+02:00 GitHub noreply@github.com 2025-05-10T22:08:07+02:00 vocab : add ByteDance-Seed/Seed-Coder (#13423) 15e6125a397f6086c1dfdf7584acdb7c730313dc 3b24d26c22b9d92b5aa39930988700c84e524cf4 Xuan-Son Nguyen son@huggingface.co 2025-05-10T19:57:54+02:00 GitHub noreply@github.com 2025-05-10T19:57:54+02:00 mtmd : add hard limit on image resolution for qwen2vl / qwen2.5vl (#13434) 3b24d26c22b9d92b5aa39930988700c84e524cf4 43dfd741a5da77982e0a94d1e522a2481dfb914d Xuan-Son Nguyen son@huggingface.co 2025-05-10T18:44:49+02:00 GitHub noreply@github.com 2025-05-10T18:44:49+02:00 server : update docs (#13432) 43dfd741a5da77982e0a94d1e522a2481dfb914d b064a51a4e7246fa43a3307f58e59f65e6be170a Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-10T17:19:52+02:00 GitHub noreply@github.com 2025-05-10T17:19:52+02:00 llguidance : set tokenizer slices to default (#13424) b064a51a4e7246fa43a3307f58e59f65e6be170a 053367d149f778cdabc356ee3024494e0dd53223 Thammachart Chinvarapon 1731496+Thammachart@users.noreply.github.com 2025-05-10T21:34:48+07:00 GitHub noreply@github.com 2025-05-10T16:34:48+02:00 ci: free_disk_space flag enabled for intel variant (#13426) 053367d149f778cdabc356ee3024494e0dd53223 d8919424f1dee7dc1638349c616f2ef5d2ee16fb Xuan-Son Nguyen son@huggingface.co 2025-05-10T16:26:42+02:00 GitHub noreply@github.com 2025-05-10T16:26:42+02:00 mtmd : support InternVL 2.5 and 3 (#13422) d8919424f1dee7dc1638349c616f2ef5d2ee16fb 7fef11766cdeb9fa7bbbe3db13580616b7d3d599 Johannes Gäßler johannesg@5d6.de 2025-05-10T09:16:52+02:00 GitHub noreply@github.com 2025-05-10T09:16:52+02:00 CUDA: fix FlashAttention on Turing (#13415) 7fef11766cdeb9fa7bbbe3db13580616b7d3d599 dc1d2adfc0f4de84da7923866d00781ec5c4e666 Xuan-Son Nguyen son@huggingface.co 2025-05-10T08:16:29+02:00 GitHub noreply@github.com 2025-05-10T08:16:29+02:00 arg : add env var to control mmproj (#13416) dc1d2adfc0f4de84da7923866d00781ec5c4e666 7c28a74e0783f4bb74a246fb9f19bf212139e365 Jeff Bolz jbolz@nvidia.com 2025-05-09T23:07:07-07:00 GitHub noreply@github.com 2025-05-10T08:07:07+02:00 vulkan: scalar flash attention implementation (#13324) 7c28a74e0783f4bb74a246fb9f19bf212139e365 33eff4024084d1f0c8441b79f7208a52fad79858 Helton Reis 47722840+HRKings@users.noreply.github.com 2025-05-09T17:15:39-03:00 GitHub noreply@github.com 2025-05-09T23:15:39+03:00 chore(llguidance): use tagged version that does not break the build (#13413) 33eff4024084d1f0c8441b79f7208a52fad79858 17512a94d636c4b6c1332370acb3e5af3ca70918 Xuan-Son Nguyen son@huggingface.co 2025-05-09T19:29:37+02:00 GitHub noreply@github.com 2025-05-09T19:29:37+02:00 server : vision support via libmtmd (#12898) 17512a94d636c4b6c1332370acb3e5af3ca70918 611aa914ef4231fab5d1ad04773c42e119ae2d2e Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2025-05-09T16:34:08+01:00 GitHub noreply@github.com 2025-05-09T16:34:08+01:00 sycl : implementation of reordered Q4_0 MMVQ for Intel GPUs (#12858) 611aa914ef4231fab5d1ad04773c42e119ae2d2e 0cf6725e9f9a164c39f7a87214d60342f7f946d8 Georgi Gerganov ggerganov@gmail.com 2025-05-09T15:14:56+03:00 GitHub noreply@github.com 2025-05-09T15:14:56+03:00 metal : optimize MoE for large batches (#13388) 0cf6725e9f9a164c39f7a87214d60342f7f946d8 27ebfcacbaadc6104e2b18acd8f13515cbf63dce Johannes Gäßler johannesg@5d6.de 2025-05-09T13:34:58+02:00 GitHub noreply@github.com 2025-05-09T13:34:58+02:00 CUDA: FA support for Deepseek (Ampere or newer) (#13306) 27ebfcacbaadc6104e2b18acd8f13515cbf63dce 5c86c9ed3ef1cc7307fdce05f0f0e2e45253cf90 Diego Devesa slarengh@gmail.com 2025-05-09T13:02:07+02:00 GitHub noreply@github.com 2025-05-09T13:02:07+02:00 llama : do not crash if there is no CPU backend (#13395) 5c86c9ed3ef1cc7307fdce05f0f0e2e45253cf90 efb8b47eda78ea8ae570d4fece3953aae499289e Johannes Gäßler johannesg@5d6.de 2025-05-09T12:14:04+02:00 GitHub noreply@github.com 2025-05-09T12:14:04+02:00 CUDA: fix crash on large batch size for MoE models (#13384) efb8b47eda78ea8ae570d4fece3953aae499289e 0527771dd80bd18479dfaaa0a98be297fc3592bf Bartowski 3266127+bartowski1182@users.noreply.github.com 2025-05-09T05:53:58-04:00 GitHub noreply@github.com 2025-05-09T11:53:58+02:00 imatrix : Add --parse-special for enabling parsing of special tokens in imatrix calculation (#13389) 0527771dd80bd18479dfaaa0a98be297fc3592bf 2189fd3b6327a1d17893694125da8edcf74a6468 R0CKSTAR xiaodong.ye@mthreads.com 2025-05-09T17:25:50+08:00 GitHub noreply@github.com 2025-05-09T10:25:50+01:00 llama-run: add support for downloading models from ModelScope (#13370) 2189fd3b6327a1d17893694125da8edcf74a6468 3f96aeff394e9b72bbd2fa665c3e023a70ed8648 Xuan-Son Nguyen son@huggingface.co 2025-05-09T11:18:02+02:00 GitHub noreply@github.com 2025-05-09T11:18:02+02:00 mtmd : fix batch_view for m-rope (#13397) 3f96aeff394e9b72bbd2fa665c3e023a70ed8648 b486ba05bf973aae3652b3fd593e0b257d3a41d4 Xuan-Son Nguyen son@huggingface.co 2025-05-09T11:17:51+02:00 GitHub noreply@github.com 2025-05-09T11:17:51+02:00 llama : one-off chat template fix for Mistral-Small-2503 (#13398) b486ba05bf973aae3652b3fd593e0b257d3a41d4 02115dcd9a6d0c03b527d5bee8c1493ab819ddbd Radoslav Gerganov rgerganov@gmail.com 2025-05-09T10:31:07+03:00 GitHub noreply@github.com 2025-05-09T10:31:07+03:00 rpc : add rpc_msg_set_tensor_hash_req (#13353) 02115dcd9a6d0c03b527d5bee8c1493ab819ddbd d9c4accaff30926e8a5fd8a2429e549158a845e0 Jeff Bolz jbolz@nvidia.com 2025-05-09T02:23:41-05:00 GitHub noreply@github.com 2025-05-09T09:23:41+02:00 vulkan: Allow up to 4096 elements for mul_mat_id row_ids (#13326) d9c4accaff30926e8a5fd8a2429e549158a845e0 15e03282bb432631193464100c2237a3b6bcfe4c Xuan-Son Nguyen son@huggingface.co 2025-05-09T09:06:37+02:00 GitHub noreply@github.com 2025-05-09T09:06:37+02:00 server : (webui) rename has_multimodal --> modalities (#13393) 15e03282bb432631193464100c2237a3b6bcfe4c f05a6d71a0f3dbf0730b56a1abbad41c0f42e63d Diego Devesa slarengh@gmail.com 2025-05-08T23:45:22+02:00 GitHub noreply@github.com 2025-05-08T23:45:22+02:00 ci : limit write permission to only the release step + fixes (#13392) f05a6d71a0f3dbf0730b56a1abbad41c0f42e63d ee01d71e585f4a17ae83ec55d77acfdcf0bfa798 Matt Clayton 156335168+mattjcly@users.noreply.github.com 2025-05-08T14:25:39-04:00 GitHub noreply@github.com 2025-05-08T20:25:39+02:00 mtmd : Expose helper_decode_image_chunk (#13366) ee01d71e585f4a17ae83ec55d77acfdcf0bfa798 8c83449cb780c201839653812681c3a4cf17feed Xuan-Son Nguyen son@huggingface.co 2025-05-08T18:51:45+02:00 GitHub noreply@github.com 2025-05-08T18:51:45+02:00 server : (webui) fix a very small misalignment (#13387) 8c83449cb780c201839653812681c3a4cf17feed 1a844be132dbe865358e1de81136920c1d35ac73 Xuan-Son Nguyen son@huggingface.co 2025-05-08T15:37:29+02:00 GitHub noreply@github.com 2025-05-08T15:37:29+02:00 server : (webui) revamp the input area, plus many small UI improvements (#13365) 1a844be132dbe865358e1de81136920c1d35ac73 0ccc1213549e39ef4c1affb1bf5f49651ef4ce48 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-08T15:34:29+02:00 GitHub noreply@github.com 2025-05-08T15:34:29+02:00 convert : support rope_scaling type and rope_type (#13349) 0ccc1213549e39ef4c1affb1bf5f49651ef4ce48 6562e5a4d6c58326dcd79002ea396d4141f1b18e welix taichitary@gmail.com 2025-05-08T22:03:53+09:00 GitHub noreply@github.com 2025-05-08T15:03:53+02:00 mtmd : fix the calculation of n_tokens for smolvlm (#13381) 6562e5a4d6c58326dcd79002ea396d4141f1b18e 51fb96b1ff2e1cc98b2492a012b7d93531a6a9a8 Georgi Gerganov ggerganov@gmail.com 2025-05-08T14:28:33+03:00 GitHub noreply@github.com 2025-05-08T14:28:33+03:00 context : allow cache-less context for embeddings (#13108) 51fb96b1ff2e1cc98b2492a012b7d93531a6a9a8 70a6991edf1b60e7afa8962f830320583f3babb0 Georgi Gerganov ggerganov@gmail.com 2025-05-08T14:26:50+03:00 GitHub noreply@github.com 2025-05-08T14:26:50+03:00 context : remove logits_all flag (#13284) 70a6991edf1b60e7afa8962f830320583f3babb0 f0610212069929f92d428ae3b5596bfbe69c020b Diego Devesa slarengh@gmail.com 2025-05-08T13:15:28+02:00 GitHub noreply@github.com 2025-05-08T13:15:28+02:00 ci : move release workflow to a separate file (#13362) f0610212069929f92d428ae3b5596bfbe69c020b 8733e0cf6eefc7c7752297cc22d0836706f4222c Diego Devesa slarengh@gmail.com 2025-05-08T13:15:15+02:00 GitHub noreply@github.com 2025-05-08T13:15:15+02:00 llama : print size and type of overridden tensors (#13364) 8733e0cf6eefc7c7752297cc22d0836706f4222c 814f795e063c257f33b921eab4073484238a151a Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2025-05-08T10:08:01+01:00 GitHub noreply@github.com 2025-05-08T10:08:01+01:00 sycl: addressing non-contiguous src1 mul_mats (nc and batched) (#13343) 814f795e063c257f33b921eab4073484238a151a d879433824ac3c16b3b5f00075895d0ee9688e34 Diego Devesa slarengh@gmail.com 2025-05-07T16:36:33+02:00 GitHub noreply@github.com 2025-05-07T16:36:33+02:00 docker : disable arm64 and intel images (#13356) d879433824ac3c16b3b5f00075895d0ee9688e34 13b0a04597a4581cad4d9027a848f450c623801d Georgi Gerganov ggerganov@gmail.com 2025-05-07T16:39:36+03:00 Georgi Gerganov ggerganov@gmail.com 2025-05-07T17:28:36+03:00 sync : ggml 13b0a04597a4581cad4d9027a848f450c623801d bba9d945c14b93c5264b7956e00736601ca6f89a Daniel Bevenius daniel.bevenius@gmail.com 2025-05-05T13:09:35+02:00 Georgi Gerganov ggerganov@gmail.com 2025-05-07T17:28:36+03:00 whisper: remove MSVC warnings pragmas (whisper/3090) bba9d945c14b93c5264b7956e00736601ca6f89a bc4e1128f78be0fbb4e2fa630adb6a04b969ac68 Jared Tweed jaredtwe@gmail.com 2025-05-02T02:41:35-07:00 Georgi Gerganov ggerganov@gmail.com 2025-05-07T17:28:36+03:00 cmake : removed stdc++fs (whisper/3097) bc4e1128f78be0fbb4e2fa630adb6a04b969ac68 39e73ae0d69f882d7e29cecc6dd8f5052fca6731 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-07T12:49:27+02:00 GitHub noreply@github.com 2025-05-07T12:49:27+02:00 llama : deci : support ffn-free with attention (#13296) 39e73ae0d69f882d7e29cecc6dd8f5052fca6731 1f73301b63668d61b5f3109489050a27dc3f65be Ycros 18012+ycros@users.noreply.github.com 2025-05-07T18:23:28+10:00 GitHub noreply@github.com 2025-05-07T11:23:28+03:00 common : Add a warning when we can't match samplers from a string or char. (#13330) 1f73301b63668d61b5f3109489050a27dc3f65be 4773d7a02ffdb05ba9e673ff21ce95351836e33a R0CKSTAR xiaodong.ye@mthreads.com 2025-05-07T15:48:23+08:00 GitHub noreply@github.com 2025-05-07T09:48:23+02:00 cuda : remove nrows_x in mul_mat_q_process_tile (#13325) 4773d7a02ffdb05ba9e673ff21ce95351836e33a 6c7fd67b647a76846d1691cd181011dff4549d02 Georgi Gerganov ggerganov@gmail.com 2025-05-07T10:28:02+03:00 GitHub noreply@github.com 2025-05-07T10:28:02+03:00 examples : remove infill (#13283) 6c7fd67b647a76846d1691cd181011dff4549d02 141a908a59bbc68ceae3bf090b850e33322a2ca9 piDack 104877312+piDack@users.noreply.github.com 2025-05-07T15:23:11+08:00 GitHub noreply@github.com 2025-05-07T09:23:11+02:00 llama : support tie embedding for chatglm models (#13328) 141a908a59bbc68ceae3bf090b850e33322a2ca9 32916a49072f01c43e20df374af5f8a1f70d6963 Johannes Gäßler johannesg@5d6.de 2025-05-06T23:35:51+02:00 GitHub noreply@github.com 2025-05-06T23:35:51+02:00 CUDA: mix virt/real CUDA archs for GGML_NATIVE=OFF (#13135) 32916a49072f01c43e20df374af5f8a1f70d6963 ffc727203af1061fdeb49efef30f76171722e403 Xuan-Son Nguyen son@huggingface.co 2025-05-06T22:40:24+02:00 GitHub noreply@github.com 2025-05-06T22:40:24+02:00 clip : refactor graph builder (#13321) ffc727203af1061fdeb49efef30f76171722e403 91a86a6f354aa73a7aab7bc3d283be410fdc93a5 DocShotgun 126566557+DocShotgun@users.noreply.github.com 2025-05-06T13:36:24-07:00 GitHub noreply@github.com 2025-05-06T22:36:24+02:00 sampling : make top_n_sigma no-op at <=0 or a single candidate (#13345) 91a86a6f354aa73a7aab7bc3d283be410fdc93a5 f4ed10b69cc38c54070a47f841827de5e8984cdf oobabooga oobabooga4@gmail.com 2025-05-06T15:24:15-03:00 GitHub noreply@github.com 2025-05-06T20:24:15+02:00 sampling : don't consider -infinity values in top_n_sigma (#13344) f4ed10b69cc38c54070a47f841827de5e8984cdf 1e333d5bba18e99bc328bb87ac1ee6a4e6260e0e Diego Devesa slarengh@gmail.com 2025-05-06T20:15:31+02:00 GitHub noreply@github.com 2025-05-06T20:15:31+02:00 cmake : remove arm64 msvc presets (#13342) 1e333d5bba18e99bc328bb87ac1ee6a4e6260e0e 2f54e348ad2999c4e31b8777592247622b20420f Akarshan Biswas akarshan@menlo.ai 2025-05-06T20:27:06+05:30 GitHub noreply@github.com 2025-05-06T20:27:06+05:30 SYCL: Disable reorder optimize by default and stop setting tensor extras when optimize is disabled (#13254) 2f54e348ad2999c4e31b8777592247622b20420f 2356fb1d53c86d838756211010bbabfafda7cb94 Xuan-Son Nguyen son@huggingface.co 2025-05-06T14:25:40+02:00 GitHub noreply@github.com 2025-05-06T14:25:40+02:00 llama : fix build_ffn without gate (#13336) 2356fb1d53c86d838756211010bbabfafda7cb94 764b85627b46f43d7ea801867cd1b6abef484574 Johannes Gäßler johannesg@5d6.de 2025-05-06T13:58:51+02:00 GitHub noreply@github.com 2025-05-06T13:58:51+02:00 CUDA: fix bad asserts for partial offload (#13337) 764b85627b46f43d7ea801867cd1b6abef484574 15a28ec8c705b188ebe178170966d1dcc36fe151 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-06T11:12:06+02:00 GitHub noreply@github.com 2025-05-06T11:12:06+02:00 convert : qwen2/3moe : set yarn metadata if present (#13331) 15a28ec8c705b188ebe178170966d1dcc36fe151 a7366faa5bb2fff97b9fb43340d853709f52d8c9 Johannes Gäßler johannesg@5d6.de 2025-05-06T08:36:46+02:00 GitHub noreply@github.com 2025-05-06T08:36:46+02:00 CUDA: fix --split-mode row for MMQ (#13323) a7366faa5bb2fff97b9fb43340d853709f52d8c9 907036502070ba608bdb2aaebf802092d4cfba07 compilade git@compilade.net 2025-05-05T22:27:31-04:00 GitHub noreply@github.com 2025-05-05T22:27:31-04:00 gguf-py : avoid requiring pyside6 for other scripts (#13036) 907036502070ba608bdb2aaebf802092d4cfba07 233461f8121455f957a47e6a22a77b3bc88277b0 Johannes Gäßler johannesg@5d6.de 2025-05-05T22:32:13+02:00 GitHub noreply@github.com 2025-05-05T22:32:13+02:00 CUDA: fix logic for clearing padding with -ngl 0 (#13320) 233461f8121455f957a47e6a22a77b3bc88277b0 b34c859146630dff136943abc9852ca173a7c9d6 oobabooga oobabooga4@gmail.com 2025-05-05T17:12:19-03:00 GitHub noreply@github.com 2025-05-05T22:12:19+02:00 sampling : Integrate Top-nσ into main sampling chain (and add it to the server) (#13264) b34c859146630dff136943abc9852ca173a7c9d6 9b61acf06041dcbaff6afa5f28940e93297f8520 igardev 49397134+igardev@users.noreply.github.com 2025-05-05T17:03:31+03:00 GitHub noreply@github.com 2025-05-05T16:03:31+02:00 server : Webui - change setText command from parent window to also send the message. (#13309) 9b61acf06041dcbaff6afa5f28940e93297f8520 5215b91e9377ce23e4ccc92ec3156bf5c7f892a3 Xuan-Son Nguyen son@huggingface.co 2025-05-05T16:02:55+02:00 GitHub noreply@github.com 2025-05-05T16:02:55+02:00 mtmd : rename llava directory to mtmd (#13311) 5215b91e9377ce23e4ccc92ec3156bf5c7f892a3 ae803bfc3d0fc2d0d3e1cce22ee103a30939e104 Xuan-Son Nguyen son@huggingface.co 2025-05-05T12:54:44+02:00 GitHub noreply@github.com 2025-05-05T12:54:44+02:00 clip : fix confused naming ffn_up and ffn_down (#13290) ae803bfc3d0fc2d0d3e1cce22ee103a30939e104 66645a5285d8c4c5f9a3b3f360d042baac2d820a Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-05T12:34:26+02:00 GitHub noreply@github.com 2025-05-05T12:34:26+02:00 convert : bailingmoe : set yarn metadata if present (#13312) 66645a5285d8c4c5f9a3b3f360d042baac2d820a 27aa2595321c4d9cc4086a8e67bdea204b8309b0 Akarshan Biswas akarshan@menlo.ai 2025-05-05T13:39:10+05:30 GitHub noreply@github.com 2025-05-05T13:39:10+05:30 SYCL: Disable mul_mat kernels for noncontiguous tensor b (#13308) 27aa2595321c4d9cc4086a8e67bdea204b8309b0 9fdfcdaeddd1ef57c6d041b89cd8fb7048a0f028 Xuan-Son Nguyen son@huggingface.co 2025-05-04T23:43:42+02:00 GitHub noreply@github.com 2025-05-04T23:43:42+02:00 mtmd : add C public API (#13184) 9fdfcdaeddd1ef57c6d041b89cd8fb7048a0f028 6eb7d25c700e29d9272064db408855178d01741b Diego Devesa slarengh@gmail.com 2025-05-04T21:25:43+02:00 GitHub noreply@github.com 2025-05-04T21:25:43+02:00 rpc : use backend registry, support dl backends (#13304) 6eb7d25c700e29d9272064db408855178d01741b 86bd60d3fe4a08b8c9d920e6defbc2412d803569 Aaron Teo aaron.teo1@ibm.com 2025-05-05T01:49:12+08:00 GitHub noreply@github.com 2025-05-04T19:49:12+02:00 ggml : activate s390x simd for Q3_K (#13301) 86bd60d3fe4a08b8c9d920e6defbc2412d803569 9f2da5871f4bbd205b8a3b952cdc76283218d595 Diego Devesa slarengh@gmail.com 2025-05-04T17:05:20+02:00 GitHub noreply@github.com 2025-05-04T17:05:20+02:00 llava/mtmd : fixes to fully support dl backends (#13303) 9f2da5871f4bbd205b8a3b952cdc76283218d595 93c4e23905987949b714b21ae918ff6bfb55fe36 Diego Devesa slarengh@gmail.com 2025-05-04T14:20:49+02:00 GitHub noreply@github.com 2025-05-04T14:20:49+02:00 llama : build windows releases with dl backends (#13220) 93c4e23905987949b714b21ae918ff6bfb55fe36 8afbd968182909cf93fb15959fc867b6dd3adb53 Johannes Gäßler johannesg@5d6.de 2025-05-04T14:16:39+02:00 GitHub noreply@github.com 2025-05-04T14:16:39+02:00 CUDA: fix race condition in MMQ stream-k fixup (#13299) 8afbd968182909cf93fb15959fc867b6dd3adb53 8ae5ebcf859b05a2ea3bbd930133a2fe4a89ed3c Johannes Gäßler johannesg@5d6.de 2025-05-04T13:58:38+02:00 GitHub noreply@github.com 2025-05-04T13:58:38+02:00 CUDA: fix race condition in MMQ ids_dst (#13294) 8ae5ebcf859b05a2ea3bbd930133a2fe4a89ed3c 3e959f09764a2bb0e64af594eab83f7fb3e08eb2 Jeff Bolz jbolz@nvidia.com 2025-05-04T00:17:16-05:00 GitHub noreply@github.com 2025-05-04T07:17:16+02:00 vulkan: Additional type support for unary, binary, and copy (#13266) 3e959f09764a2bb0e64af594eab83f7fb3e08eb2 36667c8edcded08063ed51c7d57e9e086bbfc903 Johannes Gäßler johannesg@5d6.de 2025-05-04T00:50:37+02:00 GitHub noreply@github.com 2025-05-04T00:50:37+02:00 imatrix: fix oob writes if src1 is not contiguous (#13286) 36667c8edcded08063ed51c7d57e9e086bbfc903 3bf785f3efa89ed28294fbf73054558a2b034bfb Xuan-Son Nguyen son@huggingface.co 2025-05-03T20:07:54+02:00 GitHub noreply@github.com 2025-05-03T20:07:54+02:00 clip : revert the change of BOI/EOI token for GLM-edge (⚠️ breaking change) (#13259) 3bf785f3efa89ed28294fbf73054558a2b034bfb 1d36b3670b285e69e58b9d687c770a2a0a192194 ymcki 84055651+ymcki@users.noreply.github.com 2025-05-03T23:39:51+08:00 GitHub noreply@github.com 2025-05-03T17:39:51+02:00 llama : Llama-3_1-Nemotron-Ultra-253B-v1 support (#12843) 1d36b3670b285e69e58b9d687c770a2a0a192194 b34443923cad751483cc53af2e680d595daadce7 Diego Devesa slarengh@gmail.com 2025-05-02T20:27:13+02:00 GitHub noreply@github.com 2025-05-02T20:27:13+02:00 llama : move end-user examples to tools directory (#13249) b34443923cad751483cc53af2e680d595daadce7 a75cb30dc9e63488c3614e2d5a9fe2306eaf47cd Georgi Gerganov ggerganov@gmail.com 2025-05-02T20:54:30+03:00 GitHub noreply@github.com 2025-05-02T20:54:30+03:00 sync : ggml (#13268) a75cb30dc9e63488c3614e2d5a9fe2306eaf47cd 3f3769ba76061a511f02f2a48da2ad2d93fce511 Georgi Gerganov ggerganov@gmail.com 2025-05-02T20:54:13+03:00 GitHub noreply@github.com 2025-05-02T20:54:13+03:00 context : fix reorder logic (#13267) 3f3769ba76061a511f02f2a48da2ad2d93fce511 2f567611c0234bbca0a4009762acb47b56866095 shalinib-ibm Shalini.Salomi.Bodapati@ibm.com 2025-05-02T22:23:12+05:30 GitHub noreply@github.com 2025-05-02T19:53:12+03:00 ggml : Enable MMA for BF16 in llamafile_sgemm (#13148) 2f567611c0234bbca0a4009762acb47b56866095 7d2123484e6ba8fcd90fff8c01661b9dbbefdc9d Jared Van Bortel jared@nomic.ai 2025-05-02T11:42:30-04:00 GitHub noreply@github.com 2025-05-02T11:42:30-04:00 llama-model : support Qwen2 embedding models and pooling_mode_lasttoken (#13245) 7d2123484e6ba8fcd90fff8c01661b9dbbefdc9d 074e42ab31de4c99aa7d9d2d239660f64b2380d6 Jared Van Bortel jared@nomic.ai 2025-05-02T11:41:54-04:00 GitHub noreply@github.com 2025-05-02T11:41:54-04:00 convert : use correct context length for nomic-embed-text-v2 (#13216) 074e42ab31de4c99aa7d9d2d239660f64b2380d6 c642bc014c105728ce45015813351dc5a37f60a2 Xuan-Son Nguyen son@huggingface.co 2025-05-02T17:17:15+02:00 GitHub noreply@github.com 2025-05-02T17:17:15+02:00 convert : converting mmproj for Qwen2/2.5VL from convert_hf_to_gguf (#13209) c642bc014c105728ce45015813351dc5a37f60a2 cb06a3c363f50cd35113984fe8fb164aea419077 Georgi Gerganov ggerganov@gmail.com 2025-05-02T17:48:36+03:00 GitHub noreply@github.com 2025-05-02T17:48:36+03:00 kv-cache : separate recurrent vs non-recurrent impl (#12799) cb06a3c363f50cd35113984fe8fb164aea419077 626083faf73faa54440f934bb1741bf443be91b1 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-02T12:44:24+02:00 GitHub noreply@github.com 2025-05-02T12:44:24+02:00 llama : orion rope type is neox (#13261) 626083faf73faa54440f934bb1741bf443be91b1 2af6880178b4bc2c0eced726bab68b4bf333042b Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-05-02T12:40:56+02:00 GitHub noreply@github.com 2025-05-02T12:40:56+02:00 llama : plamo rope type is neox (#13260) 2af6880178b4bc2c0eced726bab68b4bf333042b e84773ab604fe0d935d03741df003716398dc57b piDack 104877312+piDack@users.noreply.github.com 2025-05-02T17:06:09+08:00 GitHub noreply@github.com 2025-05-02T11:06:09+02:00 llama-chat : reset glmedge chat template (#13253) e84773ab604fe0d935d03741df003716398dc57b fab647e8842c5f80da7e8f2c625dab6a0e19e5d4 Shakil Ahmed 44522075+ahmedshakill@users.noreply.github.com 2025-05-02T14:20:27+06:00 GitHub noreply@github.com 2025-05-02T10:20:27+02:00 mtmd-cli : fix out_of_range when input image path is empty (#13244) fab647e8842c5f80da7e8f2c625dab6a0e19e5d4 dcf886007de4b8e5200f461a13233315f897fb9d Georgi Gerganov ggerganov@gmail.com 2025-05-02T09:48:31+03:00 GitHub noreply@github.com 2025-05-02T09:48:31+03:00 server : add cache reuse card link to help (#13230) dcf886007de4b8e5200f461a13233315f897fb9d d24d5928086471063fa9d9fd45aca710fd1336ae Xuan-Son Nguyen son@huggingface.co 2025-05-02T08:45:10+02:00 GitHub noreply@github.com 2025-05-02T08:45:10+02:00 convert : explicitly disable trust_remote_code for AutoConfig (#13246) d24d5928086471063fa9d9fd45aca710fd1336ae 8efbdadc616fa717c369059b9b388160958d886c bandoti 141645996+bandoti@users.noreply.github.com 2025-05-01T19:06:39-03:00 GitHub noreply@github.com 2025-05-01T19:06:39-03:00 ci: fix cross-compile sync issues (#12804) 8efbdadc616fa717c369059b9b388160958d886c f057808ffadce213f54c1884ab5096e60140f358 Justin Santa Barbara justinsb@google.com 2025-05-01T17:32:11-04:00 GitHub noreply@github.com 2025-05-01T23:32:11+02:00 rpc : avoid uninitialized memory in serialize_tensor (#13210) f057808ffadce213f54c1884ab5096e60140f358 d7a14c42a1883a34a6553cbfe30da1e1b84dfd6a Jesse Gross jesse@kernel.org 2025-05-01T13:46:10-07:00 GitHub noreply@github.com 2025-05-01T22:46:10+02:00 ggml: Don't assert fail when tensor data changes (#13222) d7a14c42a1883a34a6553cbfe30da1e1b84dfd6a b6e4ff69b8abd509647b531bd5b4e86950204f66 Diego Devesa slarengh@gmail.com 2025-05-01T21:48:08+02:00 GitHub noreply@github.com 2025-05-01T21:48:08+02:00 build : fix build info on windows (#13239) b6e4ff69b8abd509647b531bd5b4e86950204f66 e0f572c8466e70d35cbd70ee536ad8fc83b2acac Loïc Carrère loic.carrere@gmail.com 2025-05-01T21:32:21+02:00 GitHub noreply@github.com 2025-05-01T21:32:21+02:00 clip : (minicpmv) Re-enable upscaling of images smaller than the CLIP image size (#13237) e0f572c8466e70d35cbd70ee536ad8fc83b2acac 79f26e9e125b21760aeb016f34bfd42a93f48351 matteo matteo.serva@gmail.com 2025-05-01T21:16:38+02:00 GitHub noreply@github.com 2025-05-01T21:16:38+02:00 llama-chat : update GLM4 chat template (#13238) 79f26e9e125b21760aeb016f34bfd42a93f48351 fc727bcdd5a311c7c69a76dbf87f4784e828c7b4 Jeff Bolz jbolz@nvidia.com 2025-05-01T13:49:39-05:00 GitHub noreply@github.com 2025-05-01T20:49:39+02:00 vulkan: Add bfloat16 support (#12554) fc727bcdd5a311c7c69a76dbf87f4784e828c7b4 b0ecbd434be024c06bf547491be444ed92e1123e Jeff Bolz jbolz@nvidia.com 2025-05-01T13:19:31-05:00 GitHub noreply@github.com 2025-05-01T20:19:31+02:00 vulkan: Handle src1 batch dimension in non-contiguous mat-vec-mul shader (#13191) b0ecbd434be024c06bf547491be444ed92e1123e b1dd4d08e8fe40c9484422c0c5ec9bbd13b067a9 Johannes Gäßler johannesg@5d6.de 2025-05-01T20:18:56+02:00 GitHub noreply@github.com 2025-05-01T20:18:56+02:00 test: non-cont. b in test-backend-ops -o MUL_MAT (#13187) b1dd4d08e8fe40c9484422c0c5ec9bbd13b067a9 99881f77d82efda80b21057d84f1cc2df2f1e0f6 Georgi Gerganov ggerganov@gmail.com 2025-05-01T17:07:13+03:00 Georgi Gerganov ggerganov@gmail.com 2025-05-01T20:15:34+03:00 sync : ggml 99881f77d82efda80b21057d84f1cc2df2f1e0f6 b5769d92b4510c77691ad9e3f8b643c2ba202e53 Daniel Bevenius daniel.bevenius@gmail.com 2025-05-01T10:05:24+02:00 Georgi Gerganov ggerganov@gmail.com 2025-05-01T20:15:34+03:00 whisper : add check that target name exists (whisper/3103) b5769d92b4510c77691ad9e3f8b643c2ba202e53 8936784f7a1ec4f91637d04b77fdc90ec36ebac9 Daniel Bevenius daniel.bevenius@gmail.com 2025-04-29T15:47:55+02:00 Georgi Gerganov ggerganov@gmail.com 2025-05-01T20:15:34+03:00 ggml : suppress Windows compiler warnings (whisper/3075) 8936784f7a1ec4f91637d04b77fdc90ec36ebac9 13c9a3319b65469e883c49dd1c478abedc410157 Xuan-Son Nguyen son@huggingface.co 2025-05-01T17:05:42+02:00 GitHub noreply@github.com 2025-05-01T17:05:42+02:00 mtmd : add **vision** support for Mistral Small 3.1 (#13231) 13c9a3319b65469e883c49dd1c478abedc410157 a70183eb0079fdf6ebeaed12966338a039461b5d Xuan-Son Nguyen son@huggingface.co 2025-05-01T10:23:25+02:00 GitHub noreply@github.com 2025-05-01T10:23:25+02:00 arg : remove CURLINFO_EFFECTIVE_METHOD (#13228) a70183eb0079fdf6ebeaed12966338a039461b5d 8d33d740c308fe0049515668aac0e561269afe9e Jared Van Bortel jared@nomic.ai 2025-05-01T03:09:41-04:00 GitHub noreply@github.com 2025-05-01T10:09:41+03:00 llama-model : fix the reported size class for nomic-embed-text-v2-moe (#13223) 8d33d740c308fe0049515668aac0e561269afe9e 4254bb49518e0f920f14c9aadd96eefdfd38b429 Georgi Gerganov ggerganov@gmail.com 2025-05-01T09:59:02+03:00 Georgi Gerganov ggerganov@gmail.com 2025-05-01T10:00:39+03:00 sync : ggml 4254bb49518e0f920f14c9aadd96eefdfd38b429 9998540149a490200894acfe595e9a1546bab723 Diego Devesa slarengh@gmail.com 2025-04-30T15:20:40+02:00 Georgi Gerganov ggerganov@gmail.com 2025-05-01T09:58:44+03:00 ggml : fix ggml_gallocr_ptr type (ggml/1205) 9998540149a490200894acfe595e9a1546bab723 e1e8e0991ffd9e99a445c6812bb519d5bac9f4b5 Georgi Gerganov ggerganov@gmail.com 2025-04-24T18:59:06+03:00 Georgi Gerganov ggerganov@gmail.com 2025-05-01T09:58:44+03:00 cuda : fix unused variable compile warning (whisper/0) e1e8e0991ffd9e99a445c6812bb519d5bac9f4b5 6f67cf1f480926391ad75ff746e0a021214bf70c Johannes Gäßler johannesg@5d6.de 2025-04-30T23:12:59+02:00 GitHub noreply@github.com 2025-04-30T23:12:59+02:00 CUDA: batched+noncont MMQ, refactor bs>1 MoE code (#13199) 6f67cf1f480926391ad75ff746e0a021214bf70c 16a457facd996915652f6274384c87602b27d21a Xuan-Son Nguyen son@huggingface.co 2025-04-30T22:29:15+02:00 GitHub noreply@github.com 2025-04-30T21:29:15+01:00 arg : -hf do not fail if url mismatch (#13219) 16a457facd996915652f6274384c87602b27d21a 3e168bede4d27b35656ab8026015b87659ecbec2 ddh0 dylanhalladay02@icloud.com 2025-04-30T15:28:43-05:00 GitHub noreply@github.com 2025-04-30T21:28:43+01:00 fix typo: `n_ctx_pre_seq` -> `n_ctx_per_seq` (#13221) 3e168bede4d27b35656ab8026015b87659ecbec2 ceda28ef8e310a8dee60bf275077a3eedae8e36c Xuan-Son Nguyen son@huggingface.co 2025-04-30T16:56:24+02:00 GitHub noreply@github.com 2025-04-30T16:56:24+02:00 convert : improve model arch handling (#13122) ceda28ef8e310a8dee60bf275077a3eedae8e36c 3b127c738535d95e06abd0d43da147bc13516ad0 Tatsuya Tanaka tanakasan2525@gmail.com 2025-04-30T22:25:20+09:00 GitHub noreply@github.com 2025-04-30T15:25:20+02:00 llava : remove duplicate include (#13207) 3b127c738535d95e06abd0d43da147bc13516ad0 e5007a5edf2692ef7151a81a61ce2716b83374e5 Olivier Chafik ochafik@users.noreply.github.com 2025-04-30T13:52:35+01:00 GitHub noreply@github.com 2025-04-30T14:52:35+02:00 common : add -jf / --json-schema-file flag (#12011) e5007a5edf2692ef7151a81a61ce2716b83374e5 416313773b53585fddcafbcb914cbbfbaeb94b1f Jeff Bolz jbolz@nvidia.com 2025-04-30T07:38:37-05:00 GitHub noreply@github.com 2025-04-30T14:38:37+02:00 vulkan: use uint array index to avoid glslang bug (#13193) 416313773b53585fddcafbcb914cbbfbaeb94b1f 07c2e2f76cce9a61c110b6995fbb90ccea2c3aaa shalinib-ibm Shalini.Salomi.Bodapati@ibm.com 2025-04-30T16:47:08+05:30 GitHub noreply@github.com 2025-04-30T13:17:08+02:00 ggml : fix ppc64le build (#13176) 07c2e2f76cce9a61c110b6995fbb90ccea2c3aaa 44cd8d91ff2c9e4a0f2e3151f8d6f04c928e2571 Xuan-Son Nguyen son@huggingface.co 2025-04-30T13:06:15+02:00 GitHub noreply@github.com 2025-04-30T13:06:15+02:00 convert : correct typo image_mean --> image_std (#13208) 44cd8d91ff2c9e4a0f2e3151f8d6f04c928e2571 5933e6fdc9c051eea6c83b5a7608de12f9f15670 Aaron Teo 57927438+taronaeo@users.noreply.github.com 2025-04-30T17:47:35+08:00 GitHub noreply@github.com 2025-04-30T10:47:35+01:00 feat(ggml-cpu): enable z17 compile (#13182) 5933e6fdc9c051eea6c83b5a7608de12f9f15670 da84c04d8fa43ff92b172feb8130c74d062f956a Xuan-Son Nguyen son@huggingface.co 2025-04-30T10:46:32+02:00 GitHub noreply@github.com 2025-04-30T10:46:32+02:00 arg : allow using -hf offline (#13202) da84c04d8fa43ff92b172feb8130c74d062f956a a0f7016d170ca4bfe24d9a9f26c024d034af69f2 Xuan-Son Nguyen son@huggingface.co 2025-04-30T10:44:07+02:00 GitHub noreply@github.com 2025-04-30T10:44:07+02:00 docker : do not build tests (#13204) a0f7016d170ca4bfe24d9a9f26c024d034af69f2 19e899ce21a7c9ffcf8bb2b22269a75f6e078f8f xiaofei hbuxiaofei@gmail.com 2025-04-30T14:29:22+08:00 GitHub noreply@github.com 2025-04-30T09:29:22+03:00 rpc : fix cache directory initialization (#13188) 19e899ce21a7c9ffcf8bb2b22269a75f6e078f8f e2e1ddb93a01ce282e304431b37e60b3cddb6114 Johannes Gäßler johannesg@5d6.de 2025-04-29T23:32:04+02:00 GitHub noreply@github.com 2025-04-29T23:32:04+02:00 scripts: n_depth for compare-llama-bench [no ci] (#13201) e2e1ddb93a01ce282e304431b37e60b3cddb6114 d9d398f84f96d16c308d4976f5e90222ecc2a492 matteo matteo.serva@gmail.com 2025-04-29T20:33:10+02:00 GitHub noreply@github.com 2025-04-29T20:33:10+02:00 server : Prefilling assistant message in openai compatible API (#13174) d9d398f84f96d16c308d4976f5e90222ecc2a492 5a6398011704c31178d7b774be67856ba57647c8 Georgi Gerganov ggerganov@gmail.com 2025-04-29T20:22:57+03:00 GitHub noreply@github.com 2025-04-29T20:22:57+03:00 sampling : when top-k <= 0 -> noop (#13173) 5a6398011704c31178d7b774be67856ba57647c8 cdf76586b23c67abd3ca064ee2c084c57ae240bd Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2025-04-29T16:24:36+01:00 GitHub noreply@github.com 2025-04-29T17:24:36+02:00 llama-bench: fixed size of fields to correctly map to values (#13183) cdf76586b23c67abd3ca064ee2c084c57ae240bd 7d3af70b089bb349b5d17eb01839224c99ec1952 Johannes Gäßler johannesg@5d6.de 2025-04-29T16:00:27+02:00 GitHub noreply@github.com 2025-04-29T16:00:27+02:00 CUDA: fix non-cont. inputs for batched mat mul (#13155) 7d3af70b089bb349b5d17eb01839224c99ec1952 00e3e5a194e88e604e7c91391b9e90332888fd72 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-04-29T13:25:53+02:00 GitHub noreply@github.com 2025-04-29T13:25:53+02:00 llama : llm_type order by size (#13177) 00e3e5a194e88e604e7c91391b9e90332888fd72 e98b3692be4cd8fbbd9a56fbacc2f2bf0bf26a68 Xuan-Son Nguyen son@huggingface.co 2025-04-29T11:47:04+02:00 GitHub noreply@github.com 2025-04-29T11:47:04+02:00 mtmd : add qwen2vl and qwen2.5vl (#13141) e98b3692be4cd8fbbd9a56fbacc2f2bf0bf26a68 b6ce7430b7eb51f032152316880204e0a9c0470e Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-04-29T11:00:31+02:00 GitHub noreply@github.com 2025-04-29T11:00:31+02:00 llama : set qwen3 model type sizes (#13175) b6ce7430b7eb51f032152316880204e0a9c0470e 5f5e39e1ba5dbea814e41f2a15e035d749a520bc Xuan-Son Nguyen son@huggingface.co 2025-04-29T08:45:49+02:00 GitHub noreply@github.com 2025-04-29T09:45:49+03:00 llama-graph : fix text position for mrope (#13159) 5f5e39e1ba5dbea814e41f2a15e035d749a520bc eaea3253244dc4bbe07f6cd81325847ccc6cf93e AT manyoso@users.noreply.github.com 2025-04-28T15:52:15-04:00 GitHub noreply@github.com 2025-04-28T22:52:15+03:00 model : Nomic Embed Text V2 with Mixture-of-Experts (MoE) architecture (#12466) eaea3253244dc4bbe07f6cd81325847ccc6cf93e 43ddab6eeeaab5a04fe5a364af0bafb0e4d35065 Xuan-Son Nguyen son@huggingface.co 2025-04-28T21:23:19+02:00 GitHub noreply@github.com 2025-04-28T21:23:19+02:00 clip : fix model size display (#13153) 43ddab6eeeaab5a04fe5a364af0bafb0e4d35065 1831f538f720d1d99fba146f24f0a8e970838cc4 Ville Vesilehto ville@vesilehto.fi 2025-04-28T21:00:20+03:00 GitHub noreply@github.com 2025-04-28T21:00:20+03:00 fix(rpc): Improve input validation and error handling (#13069) 1831f538f720d1d99fba146f24f0a8e970838cc4 4e87962e34a4b257ec374c4baf6b1568554b81a9 Vishal Agarwal vishalagarwal.jss@gmail.com 2025-04-28T20:20:39+05:30 GitHub noreply@github.com 2025-04-28T16:50:39+02:00 llama-bench: add `-d` depth arg (#13096) 4e87962e34a4b257ec374c4baf6b1568554b81a9 fb0471d1753824e75474c24f82fbdd54c94dceda Xuan-Son Nguyen son@huggingface.co 2025-04-28T16:12:56+02:00 GitHub noreply@github.com 2025-04-28T16:12:56+02:00 mtmd : fix glm-edge redundant token count (#13139) fb0471d1753824e75474c24f82fbdd54c94dceda d2b2031e5f11b826dcc718138642f147a2009665 pockers21 134406831+pockers21@users.noreply.github.com 2025-04-28T06:45:40-07:00 GitHub noreply@github.com 2025-04-28T16:45:40+03:00 context : do not clear output buffer on reserve (#13152) d2b2031e5f11b826dcc718138642f147a2009665 5fa9e63be82225fb3249c76f39ddda3e5bdec0a3 Xuan-Son Nguyen son@huggingface.co 2025-04-28T14:20:56+02:00 GitHub noreply@github.com 2025-04-28T14:20:56+02:00 llama : (mrope) allow using normal 1D position for text token (#13138) 5fa9e63be82225fb3249c76f39ddda3e5bdec0a3 a4c340f974f9b7ac0c1aae897aabaa54549a97e5 Xuan-Son Nguyen son@huggingface.co 2025-04-28T12:18:59+02:00 GitHub noreply@github.com 2025-04-28T12:18:59+02:00 clip : refactor set input for cgraph + fix qwen2.5vl input (#13136) a4c340f974f9b7ac0c1aae897aabaa54549a97e5 d0a417f3c7a5a22ef05b3b76d91dbe1d3362bf0c Akarshan Biswas akarshan@menlo.ai 2025-04-28T15:03:25+05:30 GitHub noreply@github.com 2025-04-28T11:33:25+02:00 SYCL: Add all missing unary kernels (#13074) d0a417f3c7a5a22ef05b3b76d91dbe1d3362bf0c 43f2b07193cbcccd266734320ea9b948f5a01926 Georgi Gerganov ggerganov@gmail.com 2025-04-28T12:10:18+03:00 GitHub noreply@github.com 2025-04-28T12:10:18+03:00 readme : update hot topics (#13150) 43f2b07193cbcccd266734320ea9b948f5a01926 e5d6c2554e7597665e26991a93fa2f3d16c79ad5 Georgi Gerganov ggerganov@gmail.com 2025-04-28T11:57:19+03:00 GitHub noreply@github.com 2025-04-28T11:57:19+03:00 common : fix noreturn compile warning (#13151) e5d6c2554e7597665e26991a93fa2f3d16c79ad5 f0dd6a1926cdb2f4183a937deee40db26ef8f1da Xuan-Son Nguyen son@huggingface.co 2025-04-28T10:11:58+02:00 GitHub noreply@github.com 2025-04-28T10:11:58+02:00 llama-chat : fix typo GML --> GLM (#13143) f0dd6a1926cdb2f4183a937deee40db26ef8f1da 69699be48a6b94570773532850667f1591dc5bbe R0CKSTAR xiaodong.ye@mthreads.com 2025-04-28T15:33:28+08:00 GitHub noreply@github.com 2025-04-28T09:33:28+02:00 musa: fix typo in cc control (#13144) 69699be48a6b94570773532850667f1591dc5bbe 85f36e5e7173eef7c671c778db44c034e1d0ab19 Johannes Gäßler johannesg@5d6.de 2025-04-28T09:29:26+02:00 GitHub noreply@github.com 2025-04-28T09:29:26+02:00 CUDA: fix q_nope_absorbed prec for DS 2 Lite f16 (#13137) 85f36e5e7173eef7c671c778db44c034e1d0ab19 c0a97b762e5ec767dc414f0dc4979befd4c09a52 Xuan-Son Nguyen son@huggingface.co 2025-04-28T07:16:59+02:00 GitHub noreply@github.com 2025-04-28T08:16:59+03:00 arg : fix unused variable (#13142) c0a97b762e5ec767dc414f0dc4979befd4c09a52 ced44be34290fab450f8344efa047d8a08e723b4 4onen 11580688+4onen@users.noreply.github.com 2025-04-27T14:48:26-07:00 GitHub noreply@github.com 2025-04-27T23:48:26+02:00 llama-bench : Add `--override-tensors` arg (#12922) ced44be34290fab450f8344efa047d8a08e723b4 e291450b7602d7a36239e4ceeece37625f838373 matteo matteo.serva@gmail.com 2025-04-27T21:57:32+02:00 GitHub noreply@github.com 2025-04-27T21:57:32+02:00 llama-chat : fix wrong template in GLM4-0414 (#13140) e291450b7602d7a36239e4ceeece37625f838373 59e991c23cc44cb5fb657e7b9358cac21fb79828 R0CKSTAR xiaodong.ye@mthreads.com 2025-04-27T19:22:49+08:00 GitHub noreply@github.com 2025-04-27T13:22:49+02:00 musa: fix build warning (#13129) 59e991c23cc44cb5fb657e7b9358cac21fb79828 ca2bb89eac2097ab4620448737e58af8452e444b LostRuins Concedo 39025047+LostRuins@users.noreply.github.com 2025-04-27T18:43:37+08:00 GitHub noreply@github.com 2025-04-27T12:43:37+02:00 Fixes Qwen2.5VL segfault during inference with https://github.com/ggml-org/llama.cpp/pull/12402 as has_qwen2vl_merger migration was incomplete (#13133) ca2bb89eac2097ab4620448737e58af8452e444b 2d451c80590b9ac250322769ac13d3b4870dbcf7 HimariO dsfhe49854@gmail.com 2025-04-27T16:10:34+08:00 GitHub noreply@github.com 2025-04-27T10:10:34+02:00 clip : Add Qwen2.5VL support (#12402) 2d451c80590b9ac250322769ac13d3b4870dbcf7 4753791e70acd4d4e02f2098f14a03df26c992bd Xuan-Son Nguyen son@huggingface.co 2025-04-26T22:58:12+02:00 GitHub noreply@github.com 2025-04-26T22:58:12+02:00 common : add common_remote_get_content (#13123) 4753791e70acd4d4e02f2098f14a03df26c992bd 77d5e9a76a7b4a8a7c5bf9cf6ebef91860123cba Xuan-Son Nguyen son@huggingface.co 2025-04-26T22:39:47+02:00 GitHub noreply@github.com 2025-04-26T22:39:47+02:00 clip : improve projector naming (#13118) 77d5e9a76a7b4a8a7c5bf9cf6ebef91860123cba d5fe4e81bd447124836ecfb47d794f8768665b9f SXX sxx1136965276@gmail.com 2025-04-26T22:05:31+08:00 GitHub noreply@github.com 2025-04-26T16:05:31+02:00 ggml: move fp16/bf16 conversion optimizations to CPU backend + export conversion APIs (#13107) d5fe4e81bd447124836ecfb47d794f8768665b9f 295354ea6848a77bdee204ee1c971d9b92ffcca9 frob rick+github@frob.com.au 2025-04-26T10:10:20+02:00 GitHub noreply@github.com 2025-04-26T10:10:20+02:00 grammar : handle maxItems == 0 in JSON schema (#13117) 295354ea6848a77bdee204ee1c971d9b92ffcca9 558a764713468f26f5a163d25a22100c9a04a48f Diego Devesa slarengh@gmail.com 2025-04-25T19:40:11+02:00 GitHub noreply@github.com 2025-04-25T19:40:11+02:00 llama : fix K-shift with quantized K and BLAS backend (#13113) 558a764713468f26f5a163d25a22100c9a04a48f edb18b6e8f5ea6509ad43057f8bb98fc557dbc4e City 125218114+city96@users.noreply.github.com 2025-04-25T14:38:34+02:00 GitHub noreply@github.com 2025-04-25T14:38:34+02:00 Force FP32 compute in GLM4 FFN Down (#13101) edb18b6e8f5ea6509ad43057f8bb98fc557dbc4e 514c45608f93f66106a712dee1abe062099ce790 Xuan-Son Nguyen son@huggingface.co 2025-04-25T14:31:42+02:00 GitHub noreply@github.com 2025-04-25T14:31:42+02:00 clip : fix pixtral on some GPU backends (#13097) 514c45608f93f66106a712dee1abe062099ce790 553a5c3a9fdf771be2101bc3529937963f817457 Neo Zhang Jianyu jianyu.zhang@intel.com 2025-04-25T17:37:51+08:00 GitHub noreply@github.com 2025-04-25T17:37:51+08:00 change the reorder tensor from init to execute OP (#13003) 553a5c3a9fdf771be2101bc3529937963f817457 13be08daf992c89d5169518229b3740041c0f419 Radoslav Gerganov rgerganov@gmail.com 2025-04-25T10:08:08+03:00 GitHub noreply@github.com 2025-04-25T10:08:08+03:00 rpc : do not wait for response when sending RPC_CMD_SET_TENSOR (#12943) 13be08daf992c89d5169518229b3740041c0f419 226251ed56b85190e18a1cca963c45b888f4953c Xuan-Son Nguyen son@huggingface.co 2025-04-24T22:17:04+02:00 GitHub noreply@github.com 2025-04-24T22:17:04+02:00 clip : remove boi/eoi embeddings for GLM-edge model (#13081) 226251ed56b85190e18a1cca963c45b888f4953c 87616f0680947800ecba3e9f6bc6e101943bf8e6 Georgi Gerganov ggerganov@gmail.com 2025-04-24T22:29:22+03:00 GitHub noreply@github.com 2025-04-24T22:29:22+03:00 embeddings : fix batch sizes (#13076) 87616f0680947800ecba3e9f6bc6e101943bf8e6 63b4911494afe04778c61b9c19019341d71c99fc Georgi Gerganov ggerganov@gmail.com 2025-04-24T17:22:27+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-24T17:32:47+03:00 ggml : fix trailing whitespaces (#0) 63b4911494afe04778c61b9c19019341d71c99fc c6e8cc28c15166dba15629dba6a7366d4d5955ca Georgi Gerganov ggerganov@gmail.com 2025-04-24T16:47:43+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-24T17:32:47+03:00 sync : ggml c6e8cc28c15166dba15629dba6a7366d4d5955ca b10d8bfdb1dac40cce34e8860ca5ec7d950c3a44 Acly aclysia@gmail.com 2025-04-17T14:16:45+02:00 Georgi Gerganov ggerganov@gmail.com 2025-04-24T17:32:47+03:00 ggml : Depthwise 2D convolution (ggml/1152) b10d8bfdb1dac40cce34e8860ca5ec7d950c3a44 13b4548877326fdabee3e831b8cfd65d9844383c Johannes Gäßler johannesg@5d6.de 2025-04-24T15:57:10+02:00 GitHub noreply@github.com 2025-04-24T15:57:10+02:00 CUDA: use switch statements in constexpr functions (#13095) 13b4548877326fdabee3e831b8cfd65d9844383c 572b3141d343d7f947bf53b57513016e90db5680 Georgi Gerganov ggerganov@gmail.com 2025-04-24T16:00:10+03:00 GitHub noreply@github.com 2025-04-24T16:00:10+03:00 cmake : do not include ./src as public for libllama (#13062) 572b3141d343d7f947bf53b57513016e90db5680 7c727fbe39150fbe8381f4fa43fed08719ebebe6 Georgi Gerganov ggerganov@gmail.com 2025-04-24T15:44:05+03:00 GitHub noreply@github.com 2025-04-24T15:44:05+03:00 clang-tidy : disable warning about missing math parenthesis (#13091) 7c727fbe39150fbe8381f4fa43fed08719ebebe6 80982e815e67bae2442237f4e11466f44c9a2988 Xuan-Son Nguyen son@huggingface.co 2025-04-24T14:04:14+02:00 GitHub noreply@github.com 2025-04-24T14:04:14+02:00 arg : add --no-mmproj-offload (#13093) 80982e815e67bae2442237f4e11466f44c9a2988 7604a7d6b80e78eef8f275fc700d0f64820d672f Xuan-Son Nguyen son@huggingface.co 2025-04-24T12:14:13+02:00 GitHub noreply@github.com 2025-04-24T12:14:13+02:00 arg : clean up handling --mmproj with -hf (#13082) 7604a7d6b80e78eef8f275fc700d0f64820d672f b3b6d862cfdf190e1b9ad961639a25f5ebc0c7e3 Georgi Gerganov ggerganov@gmail.com 2025-04-24T10:38:30+03:00 GitHub noreply@github.com 2025-04-24T10:38:30+03:00 metal : fix floating-point range of attention scores in FA kernels (#13090) b3b6d862cfdf190e1b9ad961639a25f5ebc0c7e3 56304069599f4dd9749d94b9bca2c2c65bb27c02 Eve 139727413+netrunnereve@users.noreply.github.com 2025-04-24T07:18:33Z GitHub noreply@github.com 2025-04-24T09:18:33+02:00 vulkan: matmul gcn tuning (#13016) 56304069599f4dd9749d94b9bca2c2c65bb27c02 ecda2ec4b347031a9b8a89ee2efc664ce63f599c pl752 pl752@mail.ru 2025-04-24T02:32:35+05:00 GitHub noreply@github.com 2025-04-23T23:32:35+02:00 llama-mtmd-cli: Sigint rework in mtmd vision example (#13080) ecda2ec4b347031a9b8a89ee2efc664ce63f599c eb1776b15a32d832f1266deeeab75b9d255c5849 Xuan-Son Nguyen son@huggingface.co 2025-04-23T20:21:59+02:00 GitHub noreply@github.com 2025-04-23T20:21:59+02:00 mtmd : Support Pixtral 12B (#13065) eb1776b15a32d832f1266deeeab75b9d255c5849 2cca6c01e46d2fc1124d15730273ed2acdad1016 piDack 104877312+piDack@users.noreply.github.com 2025-04-23T22:59:14+08:00 GitHub noreply@github.com 2025-04-23T16:59:14+02:00 convert : Append mult-eos,half-rope,bos to GLM4-0414 and Z (#13021) 2cca6c01e46d2fc1124d15730273ed2acdad1016 658987cfc9d752dca7758987390d5fb1a7a0a54a Radoslav Gerganov rgerganov@gmail.com 2025-04-23T10:32:49+03:00 GitHub noreply@github.com 2025-04-23T10:32:49+03:00 rpc : add command line option for number of threads for the CPU backend (#13060) 658987cfc9d752dca7758987390d5fb1a7a0a54a dc39a5e7a84815a90fa0c515ed8927870cf858c9 Johannes Gäßler johannesg@5d6.de 2025-04-22T21:27:40+02:00 GitHub noreply@github.com 2025-04-22T21:27:40+02:00 CUDA: noncont MMVQ + batched bs1 MUL_MAT_ID (#13014) dc39a5e7a84815a90fa0c515ed8927870cf858c9 ab47dec3d37aa1927c2ec590e166b76141374ed3 Xuan-Son Nguyen son@huggingface.co 2025-04-22T16:24:54+02:00 GitHub noreply@github.com 2025-04-22T16:24:54+02:00 mtmd : support SmolVLM (version 1 and 2) (#13050) ab47dec3d37aa1927c2ec590e166b76141374ed3 7b53389c24a507564be39e1ea82746a39749059b Georgi Gerganov ggerganov@gmail.com 2025-04-22T16:16:10+03:00 GitHub noreply@github.com 2025-04-22T16:16:10+03:00 security : add note about RPC and server functionality (#13061) 7b53389c24a507564be39e1ea82746a39749059b 243453533e029334181dda50d911d5fc5a2b2486 Georgi Gerganov ggerganov@gmail.com 2025-04-22T16:15:51+03:00 GitHub noreply@github.com 2025-04-22T16:15:51+03:00 metal : add memory pool for temp allocs (#12850) 243453533e029334181dda50d911d5fc5a2b2486 1d735c0b4fa0551c51c2f4ac888dd9a01f447985 Xuan-Son Nguyen son@huggingface.co 2025-04-22T10:37:00+02:00 GitHub noreply@github.com 2025-04-22T10:37:00+02:00 llava : update documentations (#13055) 1d735c0b4fa0551c51c2f4ac888dd9a01f447985 5368ddda7a262d195b54687a31009dcc1f8b1602 Diego Devesa slarengh@gmail.com 2025-04-21T18:13:51+02:00 GitHub noreply@github.com 2025-04-21T18:13:51+02:00 ggml : add SSE 4.2 and x64 base variant for CPUs without AVX (#12871) 5368ddda7a262d195b54687a31009dcc1f8b1602 84a9bf2fc2875205f0806fbbfbb66dc67204094c Akarshan Biswas akarshan@menlo.ai 2025-04-21T19:13:30+05:30 GitHub noreply@github.com 2025-04-21T19:13:30+05:30 SYCL: Add non-contiguous support in ROPE (#12993) 84a9bf2fc2875205f0806fbbfbb66dc67204094c 2016f07bd106c73699ecbaace80f55db5ed95dac Xuan-Son Nguyen son@huggingface.co 2025-04-21T15:32:58+02:00 GitHub noreply@github.com 2025-04-21T15:32:58+02:00 mtmd : merge llava, gemma3 and minicpmv CLI into single `llama-mtmd-cli` (#13012) 2016f07bd106c73699ecbaace80f55db5ed95dac 6602304814e679cc8c162bb760a034aceb4f8965 Xuan-Son Nguyen son@huggingface.co 2025-04-20T23:29:36+02:00 GitHub noreply@github.com 2025-04-20T23:29:36+02:00 convert : experimental support for `--mmproj` flag (#13023) 6602304814e679cc8c162bb760a034aceb4f8965 66168204be9559dc841f06f0025f3da01d9a8546 Jeffrey Morgan jmorganca@gmail.com 2025-04-20T03:15:41-07:00 GitHub noreply@github.com 2025-04-20T12:15:41+02:00 llava: fix errors in clip.h on certain compilers (#13030) 66168204be9559dc841f06f0025f3da01d9a8546 4ba9d711ba0a5bf0be00936d3258d4a5e4164d4f Jeff Bolz jbolz@nvidia.com 2025-04-20T03:50:02-05:00 GitHub noreply@github.com 2025-04-20T10:50:02+02:00 vulkan: support noncontiguous rms_norm (#13031) 4ba9d711ba0a5bf0be00936d3258d4a5e4164d4f 00137157fca3d17b90380762b4d7cc158d385bd3 Jeffrey Morgan jmorganca@gmail.com 2025-04-19T22:28:40-07:00 GitHub noreply@github.com 2025-04-20T08:28:40+03:00 metal: add neg operator (#13029) 00137157fca3d17b90380762b4d7cc158d385bd3 fb28f4f80efb5a2990a6a240433b02e259b0dbb1 bandoti 141645996+bandoti@users.noreply.github.com 2025-04-19T13:05:03-03:00 GitHub noreply@github.com 2025-04-19T18:05:03+02:00 Disable CI cross-compile builds (#13022) fb28f4f80efb5a2990a6a240433b02e259b0dbb1 37b9f0d29d7301dd0ec5dfae8f357ccee96325d7 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-04-19T16:26:38+02:00 GitHub noreply@github.com 2025-04-19T16:26:38+02:00 gguf-py : fix upload python package workflow (#13020) 37b9f0d29d7301dd0ec5dfae8f357ccee96325d7 6408210082cc0a61b992b487be7e2ff2efbb9e36 Xuan-Son Nguyen son@huggingface.co 2025-04-19T09:15:45+02:00 GitHub noreply@github.com 2025-04-19T09:15:45+02:00 clip : refactor, add `image_manipulation` and `llava_uhd` classes (#13011) 6408210082cc0a61b992b487be7e2ff2efbb9e36 aff9d107b066c9d464f7ab1324280acfdcebf569 Daniel Tang danielzgtg.opensource@gmail.com 2025-04-18T16:02:55-04:00 GitHub noreply@github.com 2025-04-18T22:02:55+02:00 main : Fix Ctrl+D/newline handling (#12951) aff9d107b066c9d464f7ab1324280acfdcebf569 35370ba94593c3abc9550328377d461fc4f822b7 Chris Thompson christopherthompson81@gmail.com 2025-04-18T12:30:41-06:00 GitHub noreply@github.com 2025-04-18T20:30:41+02:00 gguf-py : GGUF Editor GUI - Python + Qt6 (#12930) 35370ba94593c3abc9550328377d461fc4f822b7 8d6600576318dfc6b091fca744b0fd36a5e5255f Xuan-Son Nguyen son@huggingface.co 2025-04-18T19:58:12+02:00 GitHub noreply@github.com 2025-04-18T19:58:12+02:00 server : use std::move whenever possible (#12936) 8d6600576318dfc6b091fca744b0fd36a5e5255f b9154ecff93ff54dc554411eb844a2a654be49f2 Akarshan Biswas akarshan@menlo.ai 2025-04-18T19:27:56+05:30 GitHub noreply@github.com 2025-04-18T15:57:56+02:00 SYCL: Refactor and enable FP16 in binary broadcast OPs (#12975) b9154ecff93ff54dc554411eb844a2a654be49f2 2db9ba1464f3de0aceb2b5289963e69fc369cb66 Xuan-Son Nguyen son@huggingface.co 2025-04-18T10:04:51+02:00 GitHub noreply@github.com 2025-04-18T10:04:51+02:00 mtmd : add methods to access `mtmd_image_tokens` (#12906) 2db9ba1464f3de0aceb2b5289963e69fc369cb66 2f74c354c0f752ed9aabf7d3a350e6edebd7e744 Radoslav Gerganov rgerganov@gmail.com 2025-04-18T10:13:42+03:00 GitHub noreply@github.com 2025-04-18T10:13:42+03:00 rpc : add RPC_CMD_HELLO (#12955) 2f74c354c0f752ed9aabf7d3a350e6edebd7e744 207c22ec2d6d793fc70830138617d1e016c5151c Georgi Gerganov ggerganov@gmail.com 2025-04-17T18:16:36+03:00 GitHub noreply@github.com 2025-04-17T18:16:36+03:00 graph : make FA compatible with MLA + add initial Metal kernels (#12953) 207c22ec2d6d793fc70830138617d1e016c5151c 7a395f67a7a02bb361d944b816d6e933889e28e1 Alan Gray agray3@users.noreply.github.com 2025-04-17T14:19:42+01:00 GitHub noreply@github.com 2025-04-17T15:19:42+02:00 ggml: Re-enable CUDA graphs in presence of CONT and DUP nodes (#12970) 7a395f67a7a02bb361d944b816d6e933889e28e1 971f245b3b5f3f55991bb779cb541b00f82eea1d hipudding huafengchun@gmail.com 2025-04-17T20:34:16+08:00 GitHub noreply@github.com 2025-04-17T20:34:16+08:00 CANN: Add support for async operator submission (#12864) 971f245b3b5f3f55991bb779cb541b00f82eea1d 12b17501e6015ffe568ac54fdf08e6580833bf1b Mikko Juola mikjuo@gmail.com 2025-04-17T01:37:05-07:00 GitHub noreply@github.com 2025-04-17T11:37:05+03:00 llama : recognize IBM Granite 3.3 FIM tokens (#12988) 12b17501e6015ffe568ac54fdf08e6580833bf1b 015022bb53387baa8b23817ac03743705c7d472b kimminsu 80271594+kimminsu38oo@users.noreply.github.com 2025-04-17T06:25:57+09:00 GitHub noreply@github.com 2025-04-16T14:25:57-07:00 opencl: fix incorrect local_size index in profiling log (#12868) 015022bb53387baa8b23817ac03743705c7d472b b43d89e311c5e7fbf62e5ec3c0401eb536677267 Jeff Bolz jbolz@nvidia.com 2025-04-16T13:37:25-05:00 GitHub noreply@github.com 2025-04-16T20:37:25+02:00 vulkan: enable coopmat2 FA gqa and split_k optimizations more often (#12931) b43d89e311c5e7fbf62e5ec3c0401eb536677267 80f19b41869728eeb6a26569957b92a773a2b2c6 Chenguang Li 757486878@qq.com 2025-04-16T16:21:05+08:00 GitHub noreply@github.com 2025-04-16T16:21:05+08:00 CANN: Add 310P operator support check (#12962) 80f19b41869728eeb6a26569957b92a773a2b2c6 f8f820cc4dc37032d5375972ba904ce53043445d lhez quic_lih@quicinc.com 2025-04-15T12:26:00-07:00 GitHub noreply@github.com 2025-04-15T12:26:00-07:00 opencl: split `ggml-opencl.cl` into multiple files and cleanup (#12886) f8f820cc4dc37032d5375972ba904ce53043445d 54a72720432810c89c2693f34908b60b88da1e46 Georgi Gerganov ggerganov@gmail.com 2025-04-15T14:45:05+03:00 GitHub noreply@github.com 2025-04-15T14:45:05+03:00 metal : add FA-vec kernels for head size 96 (#12952) 54a72720432810c89c2693f34908b60b88da1e46 84778e97703740d8ac5fb64e14d83b80eafa0f3c hipudding huafengchun@gmail.com 2025-04-15T19:08:55+08:00 GitHub noreply@github.com 2025-04-15T12:08:55+01:00 CANN: Add x86 build ci (#12950) 84778e97703740d8ac5fb64e14d83b80eafa0f3c 510676475f885ec064ff147af9f20ee7a9b12a50 David Huang 1969802+hjc4869@users.noreply.github.com 2025-04-15T17:20:38+08:00 GitHub noreply@github.com 2025-04-15T11:20:38+02:00 CUDA/HIP: Share the same unified memory allocation logic. (#12934) 510676475f885ec064ff147af9f20ee7a9b12a50 daa422881a0ec7944771bcc8ff8de34d11f5bd3b Akarshan Biswas akarshan@menlo.ai 2025-04-15T14:07:42+05:30 GitHub noreply@github.com 2025-04-15T10:37:42+02:00 SYCL: Add ROPE vision kernel (#12887) daa422881a0ec7944771bcc8ff8de34d11f5bd3b eccc7a1602f0752507de4aaad1008b9618a282c8 Juk Armstrong 69222624+jukofyork@users.noreply.github.com 2025-04-15T07:49:57+01:00 GitHub noreply@github.com 2025-04-15T09:49:57+03:00 llama : DeepSeek V2/V3 MLA implementation (#12801) eccc7a1602f0752507de4aaad1008b9618a282c8 0019279bb56f028e4eee19b59b19750736c719f7 Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2025-04-15T11:52:36+05:30 GitHub noreply@github.com 2025-04-15T09:22:36+03:00 ggml : Add AVX512 implementation of GEMM - Q4_Kx8 (#12829) 0019279bb56f028e4eee19b59b19750736c719f7 b0c75ac9f93322b45e3766e149417334b4fd1ed9 Chenguang Li 757486878@qq.com 2025-04-15T10:09:35+08:00 GitHub noreply@github.com 2025-04-15T10:09:35+08:00 CANN: Opt ROPE optimization (#12865) b0c75ac9f93322b45e3766e149417334b4fd1ed9 d6d2c2ab8c8865784ba9fef37f2b2de3f2134d33 Xinpeng Dou 15529241576@163.com 2025-04-15T10:04:24+08:00 GitHub noreply@github.com 2025-04-15T10:04:24+08:00 CANN: Optimize CANN buffer pool memory management (#12875) d6d2c2ab8c8865784ba9fef37f2b2de3f2134d33 75afa0ae31f0a51aaadcc5ff146eb7a32a7f9088 Russyyds 161207317+Russyyds@users.noreply.github.com 2025-04-15T01:18:20+08:00 GitHub noreply@github.com 2025-04-14T19:18:20+02:00 Add performance print for gemma3 in example (#12929) 75afa0ae31f0a51aaadcc5ff146eb7a32a7f9088 c772d549264c1be058411312a54049e0dc86a037 Akarshan Biswas akarshan@menlo.ai 2025-04-14T17:53:53+05:30 GitHub noreply@github.com 2025-04-14T14:23:53+02:00 SYCL: Fix im2col (#12910) c772d549264c1be058411312a54049e0dc86a037 81c7e64fc239288e91a58adad9145110e0353822 Radoslav Gerganov rgerganov@gmail.com 2025-04-14T13:59:34+03:00 GitHub noreply@github.com 2025-04-14T13:59:34+03:00 rpc : use ggml_context_ptr (#12938) 81c7e64fc239288e91a58adad9145110e0353822 526739b879c9d6a702ed681138611197fa4d3f18 Neo Zhang Jianyu jianyu.zhang@intel.com 2025-04-14T18:19:07+08:00 GitHub noreply@github.com 2025-04-14T18:19:07+08:00 dsiable curl lib check, this action is missed by commit bd3f59f81289b920bcc597a208c14f55e39ed37e (#12761) (#12937) 526739b879c9d6a702ed681138611197fa4d3f18 a25355e2643b1feb2fcbbc4c00312aa86370d858 Georgi Gerganov ggerganov@gmail.com 2025-04-14T08:52:10+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-14T09:26:15+03:00 sync : ggml a25355e2643b1feb2fcbbc4c00312aa86370d858 e959d32b1c16c02c0bc0bcdc7bc3f7077bc32f99 cmdr2 secondary.cmdr2@gmail.com 2025-04-11T12:14:19+05:30 Georgi Gerganov ggerganov@gmail.com 2025-04-14T09:26:15+03:00 cpu: fix cpu backend's supports-op for GET_ROWS_BACK. fixes a fatal when running test-backend-ops with only the CPU backend (ggml/1190) e959d32b1c16c02c0bc0bcdc7bc3f7077bc32f99 307bfa253dea07c9270e78fa53b133504e9c3c9d SXX sxx1136965276@gmail.com 2025-04-14T13:47:55+08:00 GitHub noreply@github.com 2025-04-14T08:47:55+03:00 ggml: use _mm[512/256]_dpbusd[_avx]_epi32 to directly accumulate into the result register (#12773) 307bfa253dea07c9270e78fa53b133504e9c3c9d 71e90e8813f90097701e62f7fce137d96ddf41e2 Alan Gray agray3@users.noreply.github.com 2025-04-13T22:12:21+01:00 GitHub noreply@github.com 2025-04-13T23:12:21+02:00 ggml: disable CUDA graphs for unsupported DUP and CONT node types (#12891) 71e90e8813f90097701e62f7fce137d96ddf41e2 bc091a4dc585af25c438c8473285a8cfec5c7695 Ed Addario 29247825+EAddario@users.noreply.github.com 2025-04-13T19:29:28+01:00 GitHub noreply@github.com 2025-04-13T21:29:28+03:00 quantize: Handle user-defined quantization levels for additional tensors (#12511) bc091a4dc585af25c438c8473285a8cfec5c7695 a4837577aae59c0ae640f3810094724bcac6bb28 Prajwal B Mehendarkar prajwal.b.mehendarkar@ibm.com 2025-04-12T21:03:39+05:30 GitHub noreply@github.com 2025-04-12T17:33:39+02:00 common : Define cache directory on AIX (#12915) a4837577aae59c0ae640f3810094724bcac6bb28 e59ea539b83d2c7947c99bd350549364dbba450c Jeff Bolz jbolz@nvidia.com 2025-04-12T03:44:48-05:00 GitHub noreply@github.com 2025-04-12T10:44:48+02:00 vulkan: use aligned loads for flash attention mask (#12853) e59ea539b83d2c7947c99bd350549364dbba450c c94085df2871d2cad11f6411304d7798d7dd4cdd Matt Clayton 156335168+mattjcly@users.noreply.github.com 2025-04-12T01:29:03-04:00 GitHub noreply@github.com 2025-04-12T07:29:03+02:00 llava: Fix cpu-only clip image encoding sefault (#12907) c94085df2871d2cad11f6411304d7798d7dd4cdd e8a62631b3b05bcf2ad0e0c686881a9f3e3f03ca Georgi Gerganov ggerganov@gmail.com 2025-04-11T23:37:41+03:00 GitHub noreply@github.com 2025-04-11T23:37:41+03:00 server : add VSCode's Github Copilot Chat support (#12896) e8a62631b3b05bcf2ad0e0c686881a9f3e3f03ca b6930ebc421e20399663bbd3bc7b7266d5236d06 yuri@FreeBSD yurivict@users.noreply.github.com 2025-04-11T13:04:14-07:00 GitHub noreply@github.com 2025-04-11T22:04:14+02:00 rpc : Set cache directory in rpc-server.cpp on FreeBSD (#12903) b6930ebc421e20399663bbd3bc7b7266d5236d06 68b08f36d047bfa048ebd7d16262c53bf9ece701 Olivier Chafik ochafik@users.noreply.github.com 2025-04-11T12:47:52-07:00 GitHub noreply@github.com 2025-04-11T21:47:52+02:00 `tool-call`: fix non-tool-calling grammar crashes w/ Qwen / Hermes 2 templates (#12900) 68b08f36d047bfa048ebd7d16262c53bf9ece701 578754b3157d662c2fdd51eaa62b6c1f43d3172c yuri@FreeBSD yurivict@users.noreply.github.com 2025-04-11T12:45:44-07:00 GitHub noreply@github.com 2025-04-11T21:45:44+02:00 common : Define cache directory on FreeBSD (#12892) 578754b3157d662c2fdd51eaa62b6c1f43d3172c b2034c2b55b36b2192bdefb3b295db2a911370f5 Ewan Crawford ewan.cr@gmail.com 2025-04-11T15:32:14+02:00 GitHub noreply@github.com 2025-04-11T15:32:14+02:00 sycl: Support sycl_ext_oneapi_limited_graph (#12873) b2034c2b55b36b2192bdefb3b295db2a911370f5 06bb53ad9b6e6d92b6ab6979927530080b1c990c tastelikefeet 58414341+tastelikefeet@users.noreply.github.com 2025-04-11T20:01:56+08:00 GitHub noreply@github.com 2025-04-11T14:01:56+02:00 contrib: support modelscope community (#12664) 06bb53ad9b6e6d92b6ab6979927530080b1c990c 0c509239445088f21265580b86733c5b184261d9 Yuxuan Zhang 2448370773@qq.com 2025-04-11T18:10:10+08:00 GitHub noreply@github.com 2025-04-11T12:10:10+02:00 llama-model : add Glm4Model implementation for GLM-4-0414 (#12867) 0c509239445088f21265580b86733c5b184261d9 fccf9cae83e6c6cd31a0ecb403d237638e427d0a Xuan-Son Nguyen son@huggingface.co 2025-04-11T12:09:39+02:00 GitHub noreply@github.com 2025-04-11T12:09:39+02:00 clip : use smart pointer (⚠️ breaking change) (#12869) fccf9cae83e6c6cd31a0ecb403d237638e427d0a ec6c09d0fac1f2699c3ea1994e10482cb4b95e0f Akarshan Biswas akarshan@menlo.ai 2025-04-11T13:33:50+05:30 GitHub noreply@github.com 2025-04-11T16:03:50+08:00 SYCL: Add fp16 type support to unary op kernels (#12788) ec6c09d0fac1f2699c3ea1994e10482cb4b95e0f 8ac9f5d765f2b1b7f821873618c0cff68ca59bc8 Daniel Han danielhanchen@gmail.com 2025-04-11T00:49:09-07:00 GitHub noreply@github.com 2025-04-11T09:49:09+02:00 convert : Llama4 RoPE fix (#12889) 8ac9f5d765f2b1b7f821873618c0cff68ca59bc8 12e9158f25cb47e97ca9b8083e1ec7415f262260 R0CKSTAR xiaodong.ye@mthreads.com 2025-04-11T15:26:17+08:00 GitHub noreply@github.com 2025-04-11T09:26:17+02:00 ci : Replace freediskspace to free_disk_space in docker.yml (#12861) 12e9158f25cb47e97ca9b8083e1ec7415f262260 5b1f13cb64dbb0de7e95dae86725928d350c188c Daniel Bevenius daniel.bevenius@gmail.com 2025-04-11T09:24:34+02:00 GitHub noreply@github.com 2025-04-11T09:24:34+02:00 xcf : add check for visionos build version (#12854) 5b1f13cb64dbb0de7e95dae86725928d350c188c 8b91d5355a84bbb5a73fd23fb658272473240efe Xuan-Son Nguyen son@huggingface.co 2025-04-11T09:23:37+02:00 GitHub noreply@github.com 2025-04-11T09:23:37+02:00 convert : proper tensor name mapping for llama4 (#12870) 8b91d5355a84bbb5a73fd23fb658272473240efe 0fed24c34787d2aa916ed204db88889591ad6e55 Xuan-Son Nguyen son@huggingface.co 2025-04-11T08:49:50+02:00 GitHub noreply@github.com 2025-04-11T08:49:50+02:00 llama : correct rms norm for llama 4 (#12882) 0fed24c34787d2aa916ed204db88889591ad6e55 47ba87d0a4f91eb181ea60f8a7fc2539b123aeaf Aaron Teo 57927438+taronaeo@users.noreply.github.com 2025-04-11T13:20:07+08:00 GitHub noreply@github.com 2025-04-11T08:20:07+03:00 ggml: fix compilation error s390x (#12848) 47ba87d0a4f91eb181ea60f8a7fc2539b123aeaf 1d2b613445bab6b179296aa63c8ee346fb947cc4 Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:08:23+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:17:47+03:00 sync : ggml 1d2b613445bab6b179296aa63c8ee346fb947cc4 eb420e11484ef32cc1abedb2a26ecef1bbf1e31b Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:04:25+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:17:47+03:00 tests : fix init order (#0) eb420e11484ef32cc1abedb2a26ecef1bbf1e31b cb79c2e7fa28e874694c14598c1fd2fde82263e1 Georgi Gerganov ggerganov@gmail.com 2025-04-10T23:59:16+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:17:47+03:00 sync : ggml cb79c2e7fa28e874694c14598c1fd2fde82263e1 fe92821ea9ae53f3088cf2699a9e102448295fa0 cmdr2 secondary.cmdr2@gmail.com 2025-04-10T17:53:08+05:30 Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:17:47+03:00 ggml: don't include arm_neon.h when using CUDA 12 with ARM Neon (ggml/1187) fe92821ea9ae53f3088cf2699a9e102448295fa0 459895c32629e36cc3ca53bf3596d3e1322cfe09 Diego Devesa slarengh@gmail.com 2025-04-09T12:32:13+02:00 Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:17:47+03:00 ggml : add bilinear upscale support (ggml/1185) 459895c32629e36cc3ca53bf3596d3e1322cfe09 e4bf72d631434b38656cb97e2411826d900433e8 Diego Devesa slarengh@gmail.com 2025-04-09T12:31:34+02:00 Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:17:47+03:00 ggml : add more generic custom op, remove deprecated custom ops (ggml/1183) e4bf72d631434b38656cb97e2411826d900433e8 8b9cc7cdd8a0dcf0176c60c755322c95b5965299 Georgi Gerganov ggerganov@gmail.com 2025-04-10T23:59:01+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-11T00:17:47+03:00 scripts : fix sync-ggml-am.sh 8b9cc7cdd8a0dcf0176c60c755322c95b5965299 64eda5deb9859e87a020e56bab5d2f9ca956f1de Xuan-Son Nguyen son@huggingface.co 2025-04-10T22:57:16+02:00 GitHub noreply@github.com 2025-04-10T22:57:16+02:00 llava : introduce libmtmd (#12849) 64eda5deb9859e87a020e56bab5d2f9ca956f1de fe5b78c89670b2f37ecb216306bed3e677b49d9f Xuan-Son Nguyen son@huggingface.co 2025-04-10T17:24:44+02:00 GitHub noreply@github.com 2025-04-10T17:24:44+02:00 convert : ability to lazy-load safetensors remotely without downloading to disk (#12820) fe5b78c89670b2f37ecb216306bed3e677b49d9f 11d07e1e69138b46375e9267b31acd58e3813577 Chenguang Li 757486878@qq.com 2025-04-10T08:51:52+08:00 GitHub noreply@github.com 2025-04-10T08:51:52+08:00 CANN: Support more ops (#12841) 11d07e1e69138b46375e9267b31acd58e3813577 b0091ecc1e5c0f689be856fade3803a534f35c9f Prajwal B Mehendarkar prajwal.b.mehendarkar@ibm.com 2025-04-10T04:48:01+05:30 GitHub noreply@github.com 2025-04-10T01:18:01+02:00 Fixes #12823 (#12830) b0091ecc1e5c0f689be856fade3803a534f35c9f 31f7803bc4e7c0dcc279ee04c2ecfb76b2afdd3e Rudi Servo rudiservo@gmail.com 2025-04-09T23:17:12Z GitHub noreply@github.com 2025-04-10T01:17:12+02:00 docker : added all CPU to GPU images (#12749) 31f7803bc4e7c0dcc279ee04c2ecfb76b2afdd3e 2391506ace6abb56186def40c7107fdfa694ed55 Piotr Kubaj pkubaj@anongoth.pl 2025-04-09T23:00:34Z GitHub noreply@github.com 2025-04-10T01:00:34+02:00 ggml-cpu-impl.h: do not redefine bool on POWER9 (#12856) 2391506ace6abb56186def40c7107fdfa694ed55 d3bd7193ba66c15963fd1c59448f22019a8caf6e Piotr Kubaj pkubaj@anongoth.pl 2025-04-09T23:00:25Z GitHub noreply@github.com 2025-04-10T01:00:25+02:00 ggml-impl.h: fix build on POWER9 (#12855) d3bd7193ba66c15963fd1c59448f22019a8caf6e d9a63b2f2e91cdcb0eda211b7f49fadcdea0f664 Bo Zheng 368586905@qq.com 2025-04-09T17:47:36+08:00 GitHub noreply@github.com 2025-04-09T11:47:36+02:00 llama : Support Qwen3 and Qwen3MoE (#12828) d9a63b2f2e91cdcb0eda211b7f49fadcdea0f664 8ed71242f464dc0a3fb3cffcfe064e55bdec72f9 R0CKSTAR xiaodong.ye@mthreads.com 2025-04-09T17:22:30+08:00 GitHub noreply@github.com 2025-04-09T11:22:30+02:00 musa: enable freediskspace for docker image build (#12839) 8ed71242f464dc0a3fb3cffcfe064e55bdec72f9 381603a77504ad1788965f694094540c1bed9ea2 Romain Biessy romain.biessy@codeplay.com 2025-04-09T11:22:04+02:00 GitHub noreply@github.com 2025-04-09T11:22:04+02:00 sycl: update documentation to use -no-cnv (#12845) 381603a77504ad1788965f694094540c1bed9ea2 65a69e6e1b6d55cd5f78f8bcdfaba8a8c59a8d96 Plamen Minev pacominev@gmail.com 2025-04-09T11:11:11+03:00 GitHub noreply@github.com 2025-04-09T10:11:11+02:00 ci: detach common from the library (#12827) 65a69e6e1b6d55cd5f78f8bcdfaba8a8c59a8d96 47277d6d1d0d515cff34292a1a78a0d1b7252350 Xuan-Son Nguyen son@huggingface.co 2025-04-09T10:09:53+02:00 GitHub noreply@github.com 2025-04-09T10:09:53+02:00 clip : do not print ftype (#12832) 47277d6d1d0d515cff34292a1a78a0d1b7252350 6e1c4cebdb697f925c523d3a969128d945161bdd Georgi Gerganov ggerganov@gmail.com 2025-04-09T10:54:42+03:00 GitHub noreply@github.com 2025-04-09T10:54:42+03:00 readme : add rpc backend (#12842) 6e1c4cebdb697f925c523d3a969128d945161bdd 0090950f679475c5ecaac2f7bca5049cca96492b Chenguang Li 757486878@qq.com 2025-04-09T14:04:14+08:00 GitHub noreply@github.com 2025-04-09T14:04:14+08:00 CANN: Support Opt CONV_TRANSPOSE_1D and ELU (#12786) 0090950f679475c5ecaac2f7bca5049cca96492b 7ecd780b1a1d5214b8d04c25ebfc194d310816ed Jeff Bolz jbolz@nvidia.com 2025-04-09T00:25:08-05:00 GitHub noreply@github.com 2025-04-09T07:25:08+02:00 vulkan: In coopmat2 mmq, load q4_k/q5_k scales through shared memory (#12833) 7ecd780b1a1d5214b8d04c25ebfc194d310816ed 7538246e7ce0606694c38055cc2fc9f60535be6c Jeff Bolz jbolz@nvidia.com 2025-04-09T00:12:57-05:00 GitHub noreply@github.com 2025-04-09T07:12:57+02:00 vulkan: Use fp16 for the flash attention P*V multiplication (#12783) 7538246e7ce0606694c38055cc2fc9f60535be6c b32efad2bc42460637c3a364c9554ea8217b3d7f Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-04-08T23:21:31+02:00 GitHub noreply@github.com 2025-04-08T23:21:31+02:00 cuda : add f32 to bf16 copy op (#12806) b32efad2bc42460637c3a364c9554ea8217b3d7f a19b5cef16d885c44c635da4a5c97113c1577de8 Matt Clayton 156335168+mattjcly@users.noreply.github.com 2025-04-08T16:01:58-04:00 GitHub noreply@github.com 2025-04-08T22:01:58+02:00 llava: improve clip_ctx destructor to not memleak load_image_size (#12834) a19b5cef16d885c44c635da4a5c97113c1577de8 78a1ba0a4f2bfed5b8b8e312592143d22e531698 Georgi Gerganov ggerganov@gmail.com 2025-04-08T19:54:51+03:00 GitHub noreply@github.com 2025-04-08T19:54:51+03:00 llama : fix FA when KV cache is not used (i.e. embeddings) (#12825) 78a1ba0a4f2bfed5b8b8e312592143d22e531698 2dabf759e7c8c827d38cdd18d0792070e6a4f4e1 Xuan-Son Nguyen son@huggingface.co 2025-04-08T18:37:06+02:00 GitHub noreply@github.com 2025-04-08T18:37:06+02:00 server : fix thread.join() on exit (#12831) 2dabf759e7c8c827d38cdd18d0792070e6a4f4e1 1d343b4069c74b2c7b19ae84260cd98aa2320a9a dm4 sunrisedm4@gmail.com 2025-04-08T21:49:13+08:00 GitHub noreply@github.com 2025-04-08T15:49:13+02:00 llava: add more helper functions to check projector types in clip context (#12824) 1d343b4069c74b2c7b19ae84260cd98aa2320a9a 8ca6e1c3a4deb6bb27ee294bfd5706098d94ae88 Prajwal B Mehendarkar prajwal.b.mehendarkar@ibm.com 2025-04-08T18:00:59+05:30 GitHub noreply@github.com 2025-04-08T14:30:59+02:00 arg : Including limits file on AIX (#12822) 8ca6e1c3a4deb6bb27ee294bfd5706098d94ae88 656babd6c21a3b9b3622324cfcc80a2ab78da25b characharm 123120856+characharm@users.noreply.github.com 2025-04-08T14:14:59+05:00 GitHub noreply@github.com 2025-04-08T11:14:59+02:00 server : webui : Improve Chat Input with Auto-Sizing Textarea (#12785) 656babd6c21a3b9b3622324cfcc80a2ab78da25b a226bc7a9ac50551f9f113808de0f0046837f188 Neo Zhang Jianyu jianyu.zhang@intel.com 2025-04-08T15:03:21+08:00 GitHub noreply@github.com 2025-04-08T15:03:21+08:00 Revert "sycl:remove redundant memcopy in function ggml_backend_sycl_buffer_set_tensor" (#12812) a226bc7a9ac50551f9f113808de0f0046837f188 1466621e738779eefe1bb672e17dc55d63d166bb compilade git@compilade.net 2025-04-08T03:03:07-04:00 GitHub noreply@github.com 2025-04-08T09:03:07+02:00 gguf-py : support lazy tensor splitting (#12809) 1466621e738779eefe1bb672e17dc55d63d166bb 82974011f312057b446c27267105bd7ad3810599 Xuan-Son Nguyen son@huggingface.co 2025-04-07T23:06:44+02:00 GitHub noreply@github.com 2025-04-07T23:06:44+02:00 llama : Support llama 4 text-only (#12791) 82974011f312057b446c27267105bd7ad3810599 4ccea213bc629c4eef7b520f7f6c59ce9bbdaca0 lhez quic_lih@quicinc.com 2025-04-07T13:22:54-07:00 GitHub noreply@github.com 2025-04-07T13:22:54-07:00 opencl: better identify Adreno GPU (#12760) 4ccea213bc629c4eef7b520f7f6c59ce9bbdaca0 1a1ab7e7a4a9b6e6440c1d9965d2b9d1b7e7dafb stduhpf stephduh@live.fr 2025-04-07T17:47:08+02:00 GitHub noreply@github.com 2025-04-07T18:47:08+03:00 hellaswag: display estimated score confidence interval (#12797) 1a1ab7e7a4a9b6e6440c1d9965d2b9d1b7e7dafb a4e46e28f99d7f64f38d8328cdb6bee5a3a1cd03 Georgi Gerganov ggerganov@gmail.com 2025-04-07T13:18:07+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-07T18:44:17+03:00 cuda : fix HIP and MUSA BF16 (#0) a4e46e28f99d7f64f38d8328cdb6bee5a3a1cd03 ff067dbcb9e6ca4ed464d3db999ff8e9c503498b Georgi Gerganov ggerganov@gmail.com 2025-04-07T12:32:39+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-07T18:44:17+03:00 sync : ggml ff067dbcb9e6ca4ed464d3db999ff8e9c503498b 36ca8b362885e4b4984d72e348ba403568864a95 Georgi Gerganov ggerganov@gmail.com 2025-04-07T12:25:15+03:00 Georgi Gerganov ggerganov@gmail.com 2025-04-07T18:44:17+03:00 ggml : simplify Arm fp16 CPU logic (ggml/1177) 36ca8b362885e4b4984d72e348ba403568864a95 995083e4ed24933e6a289472f9de0f0b53ca5eca Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-04-04T21:05:12+02:00 Georgi Gerganov ggerganov@gmail.com 2025-04-07T18:44:17+03:00 CUDA: don't convert BF16 weights to FP32 (ggml/1174) 995083e4ed24933e6a289472f9de0f0b53ca5eca 518a01480eb3a7c80a4951b430db9dee55428310 cmdr2 secondary.cmdr2@gmail.com 2025-04-02T17:46:16+05:30 Georgi Gerganov ggerganov@gmail.com 2025-04-07T18:44:17+03:00 cpu: move all the operators into a separate c++ file (except mul_mat) (ggml/1167) 518a01480eb3a7c80a4951b430db9dee55428310 e391d3ee8ddae86be70c034de1082ad51c55e211 zhouwg zhouwg2000@gmail.com 2025-04-07T23:22:57+08:00 GitHub noreply@github.com 2025-04-07T17:22:57+02:00 sycl: remove redundant memcopy in function ggml_backend_sycl_buffer_set_tensor (#12734) e391d3ee8ddae86be70c034de1082ad51c55e211 bd3f59f81289b920bcc597a208c14f55e39ed37e Xuan-Son Nguyen son@huggingface.co 2025-04-07T14:37:28+02:00 GitHub noreply@github.com 2025-04-07T15:37:28+03:00 ci : no curl on ggml-ci (#12796) bd3f59f81289b920bcc597a208c14f55e39ed37e 52b3d71f128c1eeab39b918adf1f7a8f5e256619 Xuan-Son Nguyen son@huggingface.co 2025-04-07T13:35:19+02:00 GitHub noreply@github.com 2025-04-07T13:35:19+02:00 cmake : enable curl by default (#12761) 52b3d71f128c1eeab39b918adf1f7a8f5e256619 d0d5b2232b3826cac2c6e62d0244a474ba79860f zhouwg zhouwg2000@gmail.com 2025-04-07T19:34:14+08:00 GitHub noreply@github.com 2025-04-07T19:34:14+08:00 CANN: fix typo in ggml-cann (#12733) d0d5b2232b3826cac2c6e62d0244a474ba79860f 916c83bfe7f8b08ada609c3b8e583cf5301e594b hipudding huafengchun@gmail.com 2025-04-07T17:10:36+08:00 GitHub noreply@github.com 2025-04-07T17:10:36+08:00 CANN: Refactor to reduce duplicate code (#12731) 916c83bfe7f8b08ada609c3b8e583cf5301e594b 0c74b04376b0b9efc096480fe10f866afc8d7c1c R0CKSTAR xiaodong.ye@mthreads.com 2025-04-06T21:23:54+08:00 GitHub noreply@github.com 2025-04-06T15:23:54+02:00 musa: fix compilation warnings in mp_22/31 (#12780) 0c74b04376b0b9efc096480fe10f866afc8d7c1c 80b717d493a9a5bae7167ad2384c12c60bb2ef20 Jeff Bolz jbolz@nvidia.com 2025-04-06T04:03:47-05:00 GitHub noreply@github.com 2025-04-06T11:03:47+02:00 vulkan: fix NaN issue in flash attention shader (#12776) 80b717d493a9a5bae7167ad2384c12c60bb2ef20 6bf28f0111ff9f21b3c1b1eace20c590281e7ba6 Jeff Bolz jbolz@nvidia.com 2025-04-06T03:47:13-05:00 GitHub noreply@github.com 2025-04-06T10:47:13+02:00 vulkan: Use unclamped loads for flash attention mask (#12720) 6bf28f0111ff9f21b3c1b1eace20c590281e7ba6 f1e3eb4249db68d97be352c0adf16eef7ae53795 0cc4m picard12@live.de 2025-04-05T18:04:03+02:00 GitHub noreply@github.com 2025-04-05T18:04:03+02:00 Vulkan: Tune Vulkan mmq int dot shader for performance (#12767) f1e3eb4249db68d97be352c0adf16eef7ae53795 0364178ca2452bd241a4cdd4350d0d7a6ff8c4b4 Sergey Fedorov vital.had@gmail.com 2025-04-05T23:46:00+08:00 GitHub noreply@github.com 2025-04-05T17:46:00+02:00 common : fix includes in arg.cpp and gemma3-cli.cpp (#12766) 0364178ca2452bd241a4cdd4350d0d7a6ff8c4b4 c6ff5d2a8da2587cc78c9ede9171dfc3f076c757 Xuan-Son Nguyen son@huggingface.co 2025-04-05T17:17:40+02:00 GitHub noreply@github.com 2025-04-05T17:17:40+02:00 clip : refactor clip_init, add tests (#12757) c6ff5d2a8da2587cc78c9ede9171dfc3f076c757 7a84777f42a9b3ba47db5d20b7662f8ddf92f652 エシュナヴァリシア 148695646+eternaphia@users.noreply.github.com 2025-04-05T21:31:42+08:00 GitHub noreply@github.com 2025-04-05T15:31:42+02:00 common: custom hf endpoint support (#12769) 7a84777f42a9b3ba47db5d20b7662f8ddf92f652 3e1d29348b5d77269f6931500dd1c1a729d429c8 Olivier Chafik ochafik@users.noreply.github.com 2025-04-04T13:16:39-07:00 GitHub noreply@github.com 2025-04-04T21:16:39+01:00 sync: minja (#12739) 3e1d29348b5d77269f6931500dd1c1a729d429c8 1be76e4620ddc99b3cbff0f206436117ae561047 Georgi Gerganov ggerganov@gmail.com 2025-04-04T21:48:10+03:00 GitHub noreply@github.com 2025-04-04T21:48:10+03:00 kv-cache : simplify + fix warning for recurrent models (#12756) 1be76e4620ddc99b3cbff0f206436117ae561047 b7723942970b70412793f1926a35cfb93d5c157c bandoti 141645996+bandoti@users.noreply.github.com 2025-04-04T14:05:12-03:00 GitHub noreply@github.com 2025-04-04T14:05:12-03:00 ci: add Linux cross-compile build (#12428) b7723942970b70412793f1926a35cfb93d5c157c 23106f94ea2bc3da929afb7330655fd5515d08dc Nauful Shaikh nauful@gmail.com 2025-04-04T09:09:52-05:00 GitHub noreply@github.com 2025-04-04T16:09:52+02:00 server : webui : Upgrade daisyui, tailwindcss. (#12735) 23106f94ea2bc3da929afb7330655fd5515d08dc 94148ba330968bbfb8d9ecc67751bdc2218486cd nick huang nickhuang99@hotmail.com 2025-04-04T22:09:12+08:00 GitHub noreply@github.com 2025-04-04T16:09:12+02:00 gguf-split : --merge now respects --dry-run option (#12681) 94148ba330968bbfb8d9ecc67751bdc2218486cd 9ac4d611d05b03f961e627f4f399e4377bdb4ad3 Nicolò Scipione nicolo.scipione@codeplay.com 2025-04-04T16:00:46+02:00 GitHub noreply@github.com 2025-04-04T16:00:46+02:00 sycl: allow ggml-sycl configuration and compilation using Visual Studio project/solution (#12625) 9ac4d611d05b03f961e627f4f399e4377bdb4ad3 348888e0dc469a476f9e9eccb394893c513daab8 Ronny Brendel ronnyb@nvidia.com 2025-04-04T15:12:40+02:00 GitHub noreply@github.com 2025-04-04T10:12:40-03:00 cmake: fix ggml-shaders-gen compiler paths containing spaces (#12747) 348888e0dc469a476f9e9eccb394893c513daab8 74d4f5b041ad837153b0e90fc864b8290e01d8d5 Daniel Bevenius daniel.bevenius@gmail.com 2025-04-04T10:24:12+02:00 GitHub noreply@github.com 2025-04-04T10:24:12+02:00 docs : add XCFramework section to README.md [no ci] (#12746) 74d4f5b041ad837153b0e90fc864b8290e01d8d5 35e592eb30832187412360912ab8f2f5b7984df1 Jeff Bolz jbolz@nvidia.com 2025-04-04T00:54:35-05:00 GitHub noreply@github.com 2025-04-04T07:54:35+02:00 vulkan: Hybrid waitForFences/getFenceStatus to reduce fence latency (#12630) 35e592eb30832187412360912ab8f2f5b7984df1 7d7b1bafa7980603a61cb102879fe38a33f66c08 Jeff Bolz jbolz@nvidia.com 2025-04-04T00:53:20-05:00 GitHub noreply@github.com 2025-04-04T07:53:20+02:00 vulkan: set cmake minimum and project name in vulkan-shaders (#12744) 7d7b1bafa7980603a61cb102879fe38a33f66c08 c262beddf29f3f3be5bbbf167b56029a19876956 lhez quic_lih@quicinc.com 2025-04-03T22:18:17-07:00 GitHub noreply@github.com 2025-04-03T22:18:17-07:00 opencl: update doc for OpenCL (#12702) c262beddf29f3f3be5bbbf167b56029a19876956 5dd5d1ab00d074e3b7c02ca3ae12f6bf3e86336a Gaurav Garg 52341457+gaugarg-nv@users.noreply.github.com 2025-04-03T21:50:29+05:30 GitHub noreply@github.com 2025-04-03T18:20:29+02:00 CUDA: Prefer vector flash decoding kernel for Gemma models (#12738) 5dd5d1ab00d074e3b7c02ca3ae12f6bf3e86336a 1c059995e080e37aeaacaae35850fc4c044b9dbe yumeyao yumeyao@gmail.com 2025-04-03T23:32:54+08:00 GitHub noreply@github.com 2025-04-03T18:32:54+03:00 vocab : use string_view::find() to avoid unnecessary looking up beyond the fragment range (#12706) 1c059995e080e37aeaacaae35850fc4c044b9dbe 2004644b7a5da6fe080e51861ab583480280f1d3 Jeff Bolz jbolz@nvidia.com 2025-04-03T10:08:26-05:00 GitHub noreply@github.com 2025-04-03T10:08:26-05:00 vulkan: Fix missing cmake logic for dot product extension (#12721) 2004644b7a5da6fe080e51861ab583480280f1d3 5f696e88e0eb490c8028421d3c5419df9dcf5385 Atharva Dubey atharva.dubey@codeplay.com 2025-04-03T13:12:39+01:00 GitHub noreply@github.com 2025-04-03T15:12:39+03:00 ci : add env variable in ggml-ci and document the same in SYCL.md (#12736) 5f696e88e0eb490c8028421d3c5419df9dcf5385 193c3e03a63ccda3ac3d6a2999e41e6d1414fe23 R0CKSTAR xiaodong.ye@mthreads.com 2025-04-03T19:51:35+08:00 GitHub noreply@github.com 2025-04-03T13:51:35+02:00 sync : minja (inclusionAI/Ling) and update tests (#12699) 193c3e03a63ccda3ac3d6a2999e41e6d1414fe23 65cfe136a0793b2fdf5af8bdb1ab2cf10c3e1d2e a3sh 38979186+A3shTnT@users.noreply.github.com 2025-04-03T15:32:55+08:00 GitHub noreply@github.com 2025-04-03T09:32:55+02:00 fix MUSA compiler warning (#12704) 65cfe136a0793b2fdf5af8bdb1ab2cf10c3e1d2e 3f9da22c2b21a2cef216de50006436ef1cab8764 Chenguang Li 757486878@qq.com 2025-04-03T15:18:08+08:00 GitHub noreply@github.com 2025-04-03T15:18:08+08:00 CANN: Support operator SIN COS ARGMAX (#12709) 3f9da22c2b21a2cef216de50006436ef1cab8764 2a0dc97e56eac6db0a4016f0b45da6d0a0055ef2 Alan Gray agray3@users.noreply.github.com 2025-04-03T02:31:15+01:00 GitHub noreply@github.com 2025-04-03T03:31:15+02:00 Simplify and improve CUDA graphs through use of indirect copy pointers (#9017) 2a0dc97e56eac6db0a4016f0b45da6d0a0055ef2 97a20c012be2f9bfbeee0209405a31001f93ccf9 hipudding huafengchun@gmail.com 2025-04-03T08:49:51+08:00 GitHub noreply@github.com 2025-04-03T08:49:51+08:00 CANN: Fix failed test cases (#12708) 97a20c012be2f9bfbeee0209405a31001f93ccf9 f01bd02376f919b05ee635f438311be8dfc91d7c lhez quic_lih@quicinc.com 2025-04-02T17:01:42-07:00 GitHub noreply@github.com 2025-04-02T17:01:42-07:00 opencl: use `max_alloc_size` in backend ctx instead of querying again (#12705) f01bd02376f919b05ee635f438311be8dfc91d7c 6f3bd38640f07e4dec7f145d2fbf093ce48c9544 Jeff Bolz jbolz@nvidia.com 2025-04-02T14:25:08-05:00 GitHub noreply@github.com 2025-04-02T14:25:08-05:00 vulkan: Implement split_k for coopmat2 flash attention. (#12627) 6f3bd38640f07e4dec7f145d2fbf093ce48c9544 be0a0f8cae039e2286f757612accebfb8f21b36e bandoti 141645996+bandoti@users.noreply.github.com 2025-04-02T14:56:26-03:00 GitHub noreply@github.com 2025-04-02T14:56:26-03:00 cmake: remove caching from vulkan coopmat checks (#12719) be0a0f8cae039e2286f757612accebfb8f21b36e 92e3006bb69dfeb656ccf5c7c1c1efadb03c88c2 Jeff Bolz jbolz@nvidia.com 2025-04-02T12:40:32-05:00 GitHub noreply@github.com 2025-04-02T19:40:32+02:00 vulkan: Implement grouped query attention in the coopmat2 FA shader (#12559) 92e3006bb69dfeb656ccf5c7c1c1efadb03c88c2 833e2b7409211a07df97716998c5002526642652 0cc4m picard12@live.de 2025-04-02T19:12:30+02:00 GitHub noreply@github.com 2025-04-02T19:12:30+02:00 Vulkan: Fix mmq int dot float cache size (#12722) 833e2b7409211a07df97716998c5002526642652 e0e912f49b3195ef9d0c51378629ba03c9b972da Georgi Gerganov ggerganov@gmail.com 2025-04-02T16:38:54+03:00 GitHub noreply@github.com 2025-04-02T16:38:54+03:00 model : print tensor size during load (#12711) e0e912f49b3195ef9d0c51378629ba03c9b972da a10b36c91a091f4606710fba4e9327fd71e0e738 Diego Devesa slarengh@gmail.com 2025-04-02T14:52:01+02:00 GitHub noreply@github.com 2025-04-02T14:52:01+02:00 llama : add option to override model tensor buffers (#11397) a10b36c91a091f4606710fba4e9327fd71e0e738 83a88bd6affbe148a622ac730952ac5b8b585979 Georgi Gerganov ggerganov@gmail.com 2025-04-02T14:32:59+03:00 GitHub noreply@github.com 2025-04-02T14:32:59+03:00 llama : refactor kv cache guard (#12695) 83a88bd6affbe148a622ac730952ac5b8b585979 42eb248f46e1175349e553b6eda6cb63027d74d1 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-04-02T11:21:48+02:00 GitHub noreply@github.com 2025-04-02T11:21:48+02:00 vocab : BailingMoE : change possessive quantifiers to greedy (#12677) 42eb248f46e1175349e553b6eda6cb63027d74d1 9bacd6b37461608385360fd64326c13247ccf18e Xuan-Son Nguyen son@huggingface.co 2025-04-02T09:58:34+02:00 GitHub noreply@github.com 2025-04-02T09:58:34+02:00 common : remove json.hpp from common.cpp (#12697) 9bacd6b37461608385360fd64326c13247ccf18e 267c1399f15a278ec8c3cdcf9c90dc94151fbc38 Chenguang Li 757486878@qq.com 2025-04-02T15:22:13+08:00 GitHub noreply@github.com 2025-04-02T15:22:13+08:00 [CANN] get_rows and dup optimization (#12671) 267c1399f15a278ec8c3cdcf9c90dc94151fbc38 f423981ac806bf031d83784bcb47d2721bc70f97 Xuan-Son Nguyen son@huggingface.co 2025-04-01T23:44:05+02:00 GitHub noreply@github.com 2025-04-01T23:44:05+02:00 common : refactor downloading system, handle mmproj with -hf option (#12694) f423981ac806bf031d83784bcb47d2721bc70f97 e39e727e9a3daec7082b9c59540a429ad85914af Junil Kim logyourself@gmail.com 2025-04-02T01:54:34+09:00 GitHub noreply@github.com 2025-04-01T09:54:34-07:00 opencl : fix memory allocation size (#12649) e39e727e9a3daec7082b9c59540a429ad85914af 5936a616e46cffad4579ccaff8f8fa315809da4c jklincn jklincn@outlook.com 2025-04-01T20:54:28+08:00 GitHub noreply@github.com 2025-04-01T14:54:28+02:00 llama : use LLM_KV_GENERAL_FILE_TYPE instead of gguf_find_key (#12672) 5936a616e46cffad4579ccaff8f8fa315809da4c 3fd072a54001a908c54e81fd2e82b682ecfdd475 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-04-01T14:37:13+02:00 GitHub noreply@github.com 2025-04-01T14:37:13+02:00 convert : BailingMoE : fix qkv split when head_dim is 0 (#12687) 3fd072a54001a908c54e81fd2e82b682ecfdd475 a6f32f0b3437ac79827ffb55521cb839343324ab Georgi Gerganov ggerganov@gmail.com 2025-04-01T14:57:19+03:00 GitHub noreply@github.com 2025-04-01T14:57:19+03:00 metal : use F32 prec in FA kernels (#12688) a6f32f0b3437ac79827ffb55521cb839343324ab 2bb3597e426ce092c3e10ae0bdd876963765e6ed R0CKSTAR xiaodong.ye@mthreads.com 2025-04-01T19:12:53+08:00 GitHub noreply@github.com 2025-04-01T13:12:53+02:00 Fix clang warning in gguf_check_reserved_keys (#12686) 2bb3597e426ce092c3e10ae0bdd876963765e6ed 82939705421f4ef27e924879fdeed6d7b5f6d769 Wagner Bruna wbruna@users.noreply.github.com 2025-04-01T06:38:07-03:00 GitHub noreply@github.com 2025-04-01T11:38:07+02:00 vulkan: fix build when glslc doesn't support coopmat (#12683) 82939705421f4ef27e924879fdeed6d7b5f6d769 8bbf26083d93274240a20d16cda324441c57fcc6 Romain Biessy romain.biessy@codeplay.com 2025-04-01T10:24:29+02:00 GitHub noreply@github.com 2025-04-01T16:24:29+08:00 SYCL: Rename oneMKL to oneMath (#12192) 8bbf26083d93274240a20d16cda324441c57fcc6 35782aeedb4f65140c0d522070901843213ff157 Akarshan Biswas akarshan@menlo.ai 2025-04-01T13:41:39+05:30 GitHub noreply@github.com 2025-04-01T10:11:39+02:00 SYCL: switch to SYCL namespace (#12674) 35782aeedb4f65140c0d522070901843213ff157 c80a7759dab10657b9b6c3e87eef988a133b9b6a Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-31T23:09:48+02:00 GitHub noreply@github.com 2025-03-31T23:09:48+02:00 convert : BailingMoE : avoid setting rope_dim to 0 (#12678) c80a7759dab10657b9b6c3e87eef988a133b9b6a 250d7953e829ff0e16074510fef0e7cec696461b Daniel Bevenius daniel.bevenius@gmail.com 2025-03-31T18:40:56+02:00 GitHub noreply@github.com 2025-03-31T18:40:56+02:00 vocab : add special infill tokens for CodeLlama (#11850) 250d7953e829ff0e16074510fef0e7cec696461b 403fbacbbcd3650abcc53e84f5e6bf88e51dd6f5 a3sh 38979186+A3shTnT@users.noreply.github.com 2025-04-01T00:05:13+08:00 GitHub noreply@github.com 2025-03-31T18:05:13+02:00 ggml : faster ssm scan (#10558) 403fbacbbcd3650abcc53e84f5e6bf88e51dd6f5 a8a1f3356786cbf8bcc3422e3c8737fc33b453e7 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-31T16:36:25+02:00 GitHub noreply@github.com 2025-03-31T16:36:25+02:00 convert : Qwerky : use lora_rank_tokenshift and lora_rank_decay if present (#12667) a8a1f3356786cbf8bcc3422e3c8737fc33b453e7 1790e7315726ceda256ea91eaa66fc36b63f6067 0cc4m picard12@live.de 2025-03-31T14:37:01+02:00 GitHub noreply@github.com 2025-03-31T14:37:01+02:00 Vulkan: Add DP4A MMQ and Q8_1 quantization shader (#12135) 1790e7315726ceda256ea91eaa66fc36b63f6067 0114a32da0adc21dc04f05278cd8e3f67ba5f8c7 Georgi Gerganov ggerganov@gmail.com 2025-03-31T15:05:30+03:00 Georgi Gerganov ggerganov@gmail.com 2025-03-31T15:07:32+03:00 cmake : fix whitespace (#0) 0114a32da0adc21dc04f05278cd8e3f67ba5f8c7 a7724480fd7eb5451981ba7458c1b3829f4897d1 Georgi Gerganov ggerganov@gmail.com 2025-03-31T14:59:21+03:00 Georgi Gerganov ggerganov@gmail.com 2025-03-31T15:07:32+03:00 sync : ggml a7724480fd7eb5451981ba7458c1b3829f4897d1 1a859490670c271e8e992c83cb11459ea0c3580c Sandro Hanea 40202887+sandrohanea@users.noreply.github.com 2025-03-31T12:44:36+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-31T15:07:32+03:00 cmake: improve Vulkan cooperative matrix support checks (whisper/2966) 1a859490670c271e8e992c83cb11459ea0c3580c 6c02a032fa21d69e881ef9a5c94ba28ebaf1d749 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-31T11:28:30+02:00 GitHub noreply@github.com 2025-03-31T11:28:30+02:00 llava : proper description fix (#12668) 6c02a032fa21d69e881ef9a5c94ba28ebaf1d749 f52d59d771dc231fc2ac39adacf157ddefc97730 Akarshan Biswas akarshan@menlo.ai 2025-03-31T14:55:24+05:30 GitHub noreply@github.com 2025-03-31T11:25:24+02:00 SYCL: Remove misleading ggml_sycl_op_flatten function (#12387) f52d59d771dc231fc2ac39adacf157ddefc97730 52de2e594979be52f0da92601747aa44a13e50e4 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-31T11:07:07+02:00 GitHub noreply@github.com 2025-03-31T11:07:07+02:00 llava : fix clip loading GGUFs with missing description (#12660) 52de2e594979be52f0da92601747aa44a13e50e4 2c3f8b850a4a6cff0f5dda2135c03fc81d33ed8b marcoStocchi marcostocchi77@gmail.com 2025-03-31T10:20:30+02:00 GitHub noreply@github.com 2025-03-31T11:20:30+03:00 tts : remove printfs (#12640) 2c3f8b850a4a6cff0f5dda2135c03fc81d33ed8b 4663bd353c61c1136cd8a97b9908755e4ab30cec Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-30T22:21:03+02:00 GitHub noreply@github.com 2025-03-30T22:21:03+02:00 llama : support BailingMoE (Ling) (#12634) 4663bd353c61c1136cd8a97b9908755e4ab30cec b3de7cac732e7dc0e10e1bb07502a500b0ee9022 Georgi Gerganov ggerganov@gmail.com 2025-03-30T22:04:04+03:00 GitHub noreply@github.com 2025-03-30T22:04:04+03:00 metal : use constexpr in FA kernels + fix typedef (#12659) b3de7cac732e7dc0e10e1bb07502a500b0ee9022 7242dd96756472f90ba41db4e5ebb3969dfc7e7c Juyoung Suk juyoung.suk@trillionlabs.co 2025-03-31T03:38:33+09:00 GitHub noreply@github.com 2025-03-30T20:38:33+02:00 llama : add Trillion 7B model support (#12556) 7242dd96756472f90ba41db4e5ebb3969dfc7e7c 492d7f1ff77e116e44962b832cc3db24cfc46d24 Sergei Vorobyov sergei.vorobyov01@gmail.com 2025-03-30T21:12:03+03:00 GitHub noreply@github.com 2025-03-30T20:12:03+02:00 llama-chat : Add Yandex instruct model template support (#12621) 492d7f1ff77e116e44962b832cc3db24cfc46d24 d3f1f0acfbf8aaafd2ce494309a10a7cc92042c8 R0CKSTAR xiaodong.ye@mthreads.com 2025-03-30T16:59:38+08:00 GitHub noreply@github.com 2025-03-30T10:59:38+02:00 musa: fix all warnings, re-enable `-DLLAMA_FATAL_WARNINGS=ON` in ci and update doc (#12611) d3f1f0acfbf8aaafd2ce494309a10a7cc92042c8 360dc22c007c7b926d0721930e279d87317d8466 Georgi Gerganov ggerganov@gmail.com 2025-03-29T15:37:54+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-30T08:33:31+03:00 sync : ggml 360dc22c007c7b926d0721930e279d87317d8466 a62d7fa7a979d21bcd760f1d7c19a66e55611f44 Xuan-Son Nguyen son@huggingface.co 2025-03-29T11:59:56+01:00 Georgi Gerganov ggerganov@gmail.com 2025-03-30T08:33:31+03:00 cpu : rm unused variable (ggml/1166) a62d7fa7a979d21bcd760f1d7c19a66e55611f44 e408d4351a4ad143656672479d8ae1479306ce31 cmdr2 secondary.cmdr2@gmail.com 2025-03-29T11:37:13+05:30 Georgi Gerganov ggerganov@gmail.com 2025-03-30T08:33:31+03:00 cpu: de-duplicate some of the operators and refactor (ggml/1144) e408d4351a4ad143656672479d8ae1479306ce31 3891e183c61c1e25c2c61afe68d7b95019ea3f89 Daniel Bevenius daniel.bevenius@gmail.com 2025-03-24T09:53:38+01:00 Georgi Gerganov ggerganov@gmail.com 2025-03-30T08:33:31+03:00 ggml : add logging for native build options/vars (whisper/2935) 3891e183c61c1e25c2c61afe68d7b95019ea3f89 af6ae1efb27a9a7c3f7f7f84639d2243f7303ac1 Daniel Bevenius daniel.bevenius@gmail.com 2025-03-20T07:02:18+01:00 Georgi Gerganov ggerganov@gmail.com 2025-03-30T08:33:31+03:00 examples : command.wasm updates (whisper/2904) af6ae1efb27a9a7c3f7f7f84639d2243f7303ac1 0bb2919335d00ff0bc79d5015da95c422de51f03 Xuan-Son Nguyen son@huggingface.co 2025-03-30T00:07:37+01:00 GitHub noreply@github.com 2025-03-30T00:07:37+01:00 llama : fix non-causal mask for gemma 3 (#12615) 0bb2919335d00ff0bc79d5015da95c422de51f03 a69f8463510a70fe0c85701b8c7ede283a0d1a7d Djip007 3705339+Djip007@users.noreply.github.com 2025-03-29T14:07:37+01:00 GitHub noreply@github.com 2025-03-29T14:07:37+01:00 llama : change cpu_buft_list order: ACCEL -> GPU host -> CPU extra -> CPU (#12632) a69f8463510a70fe0c85701b8c7ede283a0d1a7d d07a0d7a79bafb01fb3b6c8992121e75c57c0c2f Jay BusyJay@users.noreply.github.com 2025-03-29T18:04:58+08:00 GitHub noreply@github.com 2025-03-29T11:04:58+01:00 cmake : fix ccache conflict (#12522) d07a0d7a79bafb01fb3b6c8992121e75c57c0c2f 3714c3ee1a62ed64ac328ec7d699410ad1219150 hipudding huafengchun@gmail.com 2025-03-29T18:03:28+08:00 GitHub noreply@github.com 2025-03-29T11:03:28+01:00 CANN : remove clang-format in ggml-cann (#12607) 3714c3ee1a62ed64ac328ec7d699410ad1219150 b4ae50810e4304d052e630784c14bde7e79e4132 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-28T22:13:02+01:00 GitHub noreply@github.com 2025-03-28T22:13:02+01:00 llama : fix incorrect Qwen2Moe ffn_moe_out graph callback (#12631) b4ae50810e4304d052e630784c14bde7e79e4132 b86f6007234da4bff51a3ebef2bdb952b52059c6 Georgi Gerganov ggerganov@gmail.com 2025-03-28T20:21:59+02:00 GitHub noreply@github.com 2025-03-28T20:21:59+02:00 metal : improve FA + improve MoE (#12612) b86f6007234da4bff51a3ebef2bdb952b52059c6 dd373dd3bf81eced3e711fb7cb49123a6105933e Icenowy Zheng uwu@icenowy.me 2025-03-29T01:51:06+08:00 GitHub noreply@github.com 2025-03-28T14:51:06-03:00 vulkan: fix coopmat shader generation when cross-compiling (#12272) dd373dd3bf81eced3e711fb7cb49123a6105933e 5d01670266859444366e4f333ade5e0e5e2ae63d Johannes Gäßler johannesg@5d6.de 2025-03-28T18:08:52+01:00 GitHub noreply@github.com 2025-03-28T18:08:52+01:00 llama: fix error on bad grammar (#12628) 5d01670266859444366e4f333ade5e0e5e2ae63d ef03229ff423dd1991f4f44ef1352f03334d86eb Benson Wong mostlygeek@gmail.com 2025-03-28T01:05:44-07:00 GitHub noreply@github.com 2025-03-28T10:05:44+02:00 server : include speculative decoding stats when timings_per_token is enabled (#12603) ef03229ff423dd1991f4f44ef1352f03334d86eb 13731766db91ec927c1b61bf502ac7a9be2b11b9 Radoslav Gerganov rgerganov@gmail.com 2025-03-28T09:44:13+02:00 GitHub noreply@github.com 2025-03-28T09:44:13+02:00 rpc : update README for cache usage (#12620) 13731766db91ec927c1b61bf502ac7a9be2b11b9 ab6ab8f809bd514d88e02a63869b4d619f13fa86 amritahs-ibm amritahs@linux.vnet.ibm.com 2025-03-28T13:13:22+05:30 GitHub noreply@github.com 2025-03-28T09:43:22+02:00 llamafile : ppc64le GEMV forwarding for FP32. (#12594) ab6ab8f809bd514d88e02a63869b4d619f13fa86 2099a9d5dbdcd2841d02dd396a71d0de70bf4490 Radoslav Gerganov rgerganov@gmail.com 2025-03-28T08:18:04+02:00 GitHub noreply@github.com 2025-03-28T08:18:04+02:00 rpc : send hash when tensor data is above some fixed threshold (#12496) 2099a9d5dbdcd2841d02dd396a71d0de70bf4490 296901983700f3c37449bcb555d85d27150a679d Piotr piotr.stankiewicz@docker.com 2025-03-27T23:41:04+01:00 GitHub noreply@github.com 2025-03-27T23:41:04+01:00 server : Support listening on a unix socket (#12613) 296901983700f3c37449bcb555d85d27150a679d 5dec47dcd411fdf815a3708fd6194e2b13d19006 Georgi Gerganov ggerganov@gmail.com 2025-03-27T23:09:05+02:00 GitHub noreply@github.com 2025-03-27T23:09:05+02:00 media : add SVG logo [no ci] (#12616) 5dec47dcd411fdf815a3708fd6194e2b13d19006 f125b8dccff34439a26bf750c9edef358c48c1f8 lhez quic_lih@quicinc.com 2025-03-27T08:08:08-07:00 GitHub noreply@github.com 2025-03-27T08:08:08-07:00 opencl: add multi and vision rope, `gelu_quick` and `im2col` (#12600) f125b8dccff34439a26bf750c9edef358c48c1f8 953c2a62cf487e618140f3ea18d94e3b0257af93 Si1w 139008732+Si1w@users.noreply.github.com 2025-03-27T10:49:15Z GitHub noreply@github.com 2025-03-27T12:49:15+02:00 llama : add PLM GGUF Conversion & Inference Support (#12457) 953c2a62cf487e618140f3ea18d94e3b0257af93 d5c6309d91cb22ebc947920f92eb686d92f84eae HighDoping highdoping@gmail.com 2025-03-27T18:43:33+08:00 GitHub noreply@github.com 2025-03-27T11:43:33+01:00 model : restore support for T5Encoder (#12590) d5c6309d91cb22ebc947920f92eb686d92f84eae 029c693fdcdc80e8508553df61a4337bb5fe49a9 Csaba Kecskemeti csaba.kecskemeti@gmail.com 2025-03-27T03:11:23-07:00 GitHub noreply@github.com 2025-03-27T11:11:23+01:00 convert : Support Qwen2_5_VLForConditionalGeneration (#12595) 029c693fdcdc80e8508553df61a4337bb5fe49a9 771d84371c0785c2554aef9a47cdd551b8c7ceaf Georgi Gerganov ggerganov@gmail.com 2025-03-27T09:36:13+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-27T10:09:29+02:00 sync : ggml 771d84371c0785c2554aef9a47cdd551b8c7ceaf df0665a483b08da1c9b55534b624ae4e1fe89767 Georgi Gerganov ggerganov@gmail.com 2025-03-27T09:22:30+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-27T10:09:29+02:00 scripts : update sync + fix cmake merge df0665a483b08da1c9b55534b624ae4e1fe89767 0306aad1ca89a92fdb33450b35547b90b92f5dbe Georgi Gerganov ggerganov@gmail.com 2025-03-27T09:01:21+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-27T09:04:38+02:00 sync : ggml 0306aad1ca89a92fdb33450b35547b90b92f5dbe c7b43ab60855f752ae79937fb93d561bc30b69a4 Georgi Gerganov ggerganov@gmail.com 2025-03-27T09:00:57+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-27T09:04:38+02:00 cmake : sync/merge PowerPC build commands (#0) c7b43ab60855f752ae79937fb93d561bc30b69a4 24feaec05792b972d5ff3e2b12d9237ebd50d1ac amritahs-ibm amritahs@linux.vnet.ibm.com 2025-03-27T12:21:47+05:30 GitHub noreply@github.com 2025-03-27T08:51:47+02:00 llamafile : ppc64le MMA implementation for Q4_0. (#12489) 24feaec05792b972d5ff3e2b12d9237ebd50d1ac f28bc4c286c453cd8385388eea057a404fdb6402 xctan xc-tan@outlook.com 2025-03-27T14:38:34+08:00 GitHub noreply@github.com 2025-03-27T08:38:34+02:00 ggml : riscv: add 128-bit RVV support (#12530) f28bc4c286c453cd8385388eea057a404fdb6402 f17a3bb4e8b0aa24c0f86636d234aca7dc2cfa01 Georgi Gerganov ggerganov@gmail.com 2025-03-27T08:24:10+02:00 GitHub noreply@github.com 2025-03-27T08:24:10+02:00 llama : make loras compatible with repacking (#12593) f17a3bb4e8b0aa24c0f86636d234aca7dc2cfa01 bd40678df768ab189b26b8b03e3de4062e3b71a3 Akarshan Biswas akarshan@menlo.ai 2025-03-27T07:16:00+05:30 GitHub noreply@github.com 2025-03-27T09:46:00+08:00 SYCL: implement memset ggml backend buffer interface (#12580) bd40678df768ab189b26b8b03e3de4062e3b71a3 b3298fa47a2d56ae892127ea038942ab1cada190 Slobodan Josic 127323561+slojosic-amd@users.noreply.github.com 2025-03-26T23:46:30+01:00 GitHub noreply@github.com 2025-03-26T23:46:30+01:00 HIP: Add support for RDNA4 targets (#12372) b3298fa47a2d56ae892127ea038942ab1cada190 2447ad8a981253a2b8e9f4b31cc8e7fdff83423e Georgi Gerganov ggerganov@gmail.com 2025-03-26T21:38:38+02:00 GitHub noreply@github.com 2025-03-26T21:38:38+02:00 metal : refactor mat-vec code (#12569) 2447ad8a981253a2b8e9f4b31cc8e7fdff83423e 02082f1519565fc7b49de211b28bc5404a69209b Michał Moskal michal@moskal.me 2025-03-26T11:06:09-07:00 GitHub noreply@github.com 2025-03-26T11:06:09-07:00 upgrade to llguidance 0.7.10 (#12576) 02082f1519565fc7b49de211b28bc5404a69209b df4d20cd53d5bb6fc21c1dc65f026d53b566d097 Ivy233 952254420@qq.com 2025-03-26T22:06:04+08:00 GitHub noreply@github.com 2025-03-26T15:06:04+01:00 clip: Fix llama-llava-clip-quantize-cli quantization error under CUDA backend (#12566) df4d20cd53d5bb6fc21c1dc65f026d53b566d097 5ed38b6852bd509d56acfdae54bceec8ab3cc396 Georgi Gerganov ggerganov@gmail.com 2025-03-26T14:21:05+02:00 GitHub noreply@github.com 2025-03-26T08:21:05-04:00 convert : fix squeeze for ssm_conv tensors (#12573) 5ed38b6852bd509d56acfdae54bceec8ab3cc396 fd7855f8f522ab26681b25ae506ff99c654e2dd5 Georgi Gerganov ggerganov@gmail.com 2025-03-26T13:02:00+02:00 GitHub noreply@github.com 2025-03-26T13:02:00+02:00 ggml : fix MUL_MAT_ID repack with Q8_K (#12544) fd7855f8f522ab26681b25ae506ff99c654e2dd5 53af4dba425768fd507ce6b45873cfbb3df2295f R0CKSTAR xiaodong.ye@mthreads.com 2025-03-26T15:09:48+08:00 GitHub noreply@github.com 2025-03-26T09:09:48+02:00 doc: [MUSA] minor changes (#12583) 53af4dba425768fd507ce6b45873cfbb3df2295f ef19c71769681a0b3dde6bc90911728376e5d236 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-25T23:03:10+01:00 GitHub noreply@github.com 2025-03-25T23:03:10+01:00 convert: fix Mistral3/Gemma3 model hparams init (#12571) ef19c71769681a0b3dde6bc90911728376e5d236 053b3f9aae63151732eccf6b7408c6418ba8746e Eric Curtin ecurtin@redhat.com 2025-03-25T17:46:11Z GitHub noreply@github.com 2025-03-25T18:46:11+01:00 run: de-duplicate fmt and format functions and optimize (#11596) 053b3f9aae63151732eccf6b7408c6418ba8746e e2f560175a195f63c3276972a3d1caec0bd13e05 Dan Johansson dan.johansson@arm.com 2025-03-25T12:10:18+01:00 GitHub noreply@github.com 2025-03-25T13:10:18+02:00 ggml-cpu : update KleidiAI to v1.5.0 (#12568) e2f560175a195f63c3276972a3d1caec0bd13e05 36ee06dd2dcf8d3d1157ebe36366d7670770e75f Akarshan Biswas akarshan@menlo.ai 2025-03-25T16:10:18+05:30 GitHub noreply@github.com 2025-03-25T18:40:18+08:00 SYCL: disable Q4_0 reorder optimization (#12560) 36ee06dd2dcf8d3d1157ebe36366d7670770e75f 3cd3a395323fa9cdf6ecfa1fea290bf228d4e856 Dan Johansson dan.johansson@arm.com 2025-03-25T10:35:20+01:00 GitHub noreply@github.com 2025-03-25T11:35:20+02:00 docs : add build instructions for KleidiAI (#12563) 3cd3a395323fa9cdf6ecfa1fea290bf228d4e856 2d77d88e70d017cd82c3f1a4517e3102e2028ac4 R0CKSTAR xiaodong.ye@mthreads.com 2025-03-25T15:45:08+08:00 GitHub noreply@github.com 2025-03-25T09:45:08+02:00 ci: [MUSA] add CI and update doc (#12562) 2d77d88e70d017cd82c3f1a4517e3102e2028ac4 c95fa362b3587d1822558f7e28414521075f254f Georgi Gerganov ggerganov@gmail.com 2025-03-25T09:19:23+02:00 GitHub noreply@github.com 2025-03-25T09:19:23+02:00 context : fix worst-case reserve outputs (#12545) c95fa362b3587d1822558f7e28414521075f254f 2b65ae30299b9c67e25c51ee567e9a2ef22279ab Akarshan Biswas akarshan@menlo.ai 2025-03-24T23:05:38+05:30 GitHub noreply@github.com 2025-03-24T19:35:38+02:00 ci: [SYCL] ggml-ci Use main GPU and enable sysman (#12547) 2b65ae30299b9c67e25c51ee567e9a2ef22279ab 48d7021c61ceda6fcf1a7294d2115b8e1a53ae95 lhez quic_lih@quicinc.com 2025-03-24T09:20:47-07:00 GitHub noreply@github.com 2025-03-24T09:20:47-07:00 opencl: simplify kernel embedding logic in cmakefile (#12503) 48d7021c61ceda6fcf1a7294d2115b8e1a53ae95 3361e2deba0f24942877559b35c430dec68528c3 Akarshan Biswas akarshan@menlo.ai 2025-03-24T18:28:32+05:30 GitHub noreply@github.com 2025-03-24T14:58:32+02:00 CI: fix SYCL build (#12546) 3361e2deba0f24942877559b35c430dec68528c3 00d53800e00bb22a26bf710fa6bd1150e412cc1d Tei Home taiteitonghome@proton.me 2025-03-24T19:02:26+08:00 GitHub noreply@github.com 2025-03-24T11:02:26Z docs: update: improve the Fedoa CUDA guide (#12536) 00d53800e00bb22a26bf710fa6bd1150e412cc1d 7ea75035b67f44c22ed7039967f718011fd35ce5 compilade git@compilade.net 2025-03-24T06:47:24-04:00 GitHub noreply@github.com 2025-03-24T11:47:24+01:00 llama-vocab : add SuperBPE pre-tokenizer (#12532) 7ea75035b67f44c22ed7039967f718011fd35ce5 c54f6b7988b63714b87b0390e01a1e69aee79a12 R0CKSTAR xiaodong.ye@mthreads.com 2025-03-24T18:28:34+08:00 GitHub noreply@github.com 2025-03-24T11:28:34+01:00 CUDA: Fix clang warnings (#12540) c54f6b7988b63714b87b0390e01a1e69aee79a12 9b169a4d4e01af7bc07a6981b53b27c18c9470d8 Prajwal B Mehendarkar prajwal.b.mehendarkar@ibm.com 2025-03-24T15:47:10+05:30 GitHub noreply@github.com 2025-03-24T12:17:10+02:00 mmap : skip resource limit checks on AIX (#12541) 9b169a4d4e01af7bc07a6981b53b27c18c9470d8 77f9c6bbe55fccd9ea567794024cb80943947901 Jeff Bolz jbolz@nvidia.com 2025-03-24T01:56:17-05:00 GitHub noreply@github.com 2025-03-24T07:56:17+01:00 vulkan: fix mul_mat_vec failure in backend tests (#12529) 77f9c6bbe55fccd9ea567794024cb80943947901 18b663d8e4ef352a9a15ff15d695fc3258801d60 Marius Gerdes 141485318+mglambda@users.noreply.github.com 2025-03-23T19:30:26+01:00 GitHub noreply@github.com 2025-03-23T19:30:26+01:00 server : Add verbose output to OAI compatible chat endpoint. (#12246) 18b663d8e4ef352a9a15ff15d695fc3258801d60 fbdfefe74e736f1a3687283c25ac21b11ba07b2e Lars Sonchocky-Helldorf lars.sonchocky-helldorf@hamburg.de 2025-03-23T09:21:48+01:00 GitHub noreply@github.com 2025-03-23T10:21:48+02:00 install : add macports (#12518) fbdfefe74e736f1a3687283c25ac21b11ba07b2e ba932dfb50cc694645b1a148c72f8c06ee080b17 Xuan-Son Nguyen son@huggingface.co 2025-03-22T23:28:19+01:00 GitHub noreply@github.com 2025-03-22T23:28:19+01:00 llama : gemma3 : use output tensor if it exists in model weight (#12506) ba932dfb50cc694645b1a148c72f8c06ee080b17 fac63a3d786b2a0f97876c30add02cb525a9648e Georgi Gerganov ggerganov@gmail.com 2025-03-22T16:23:26+02:00 GitHub noreply@github.com 2025-03-22T16:23:26+02:00 ggml : fix quantized cpy op (#12310) fac63a3d786b2a0f97876c30add02cb525a9648e eddfb438502bd5d1014d63a812e9b6d03d326f8c R0CKSTAR xiaodong.ye@mthreads.com 2025-03-22T17:11:37+08:00 GitHub noreply@github.com 2025-03-22T10:11:37+01:00 musa: refine compute capability (#12493) eddfb438502bd5d1014d63a812e9b6d03d326f8c 4375415b4abf94fb36a5fd15f233ac0ee23c0bd1 Jeff Bolz jbolz@nvidia.com 2025-03-22T03:40:11-05:00 GitHub noreply@github.com 2025-03-22T09:40:11+01:00 vulkan: Optimize mul_mat_vec p021 and nc shaders (#12505) 4375415b4abf94fb36a5fd15f233ac0ee23c0bd1 30c42ef5cbb2db756eff9aadc6b6c528ba68388d stduhpf stephduh@live.fr 2025-03-21T20:34:50+01:00 GitHub noreply@github.com 2025-03-21T20:34:50+01:00 Vulkan: RTE rounding for cpy to quant (#12480) 30c42ef5cbb2db756eff9aadc6b6c528ba68388d af04481e6b3b8dcdf5c1cc7d84bc7ece5658e9ab Eve 139727413+netrunnereve@users.noreply.github.com 2025-03-21T19:27:47Z GitHub noreply@github.com 2025-03-21T20:27:47+01:00 vulkan: workaround for AMD Windows driver 16 bit unpack8 bug (#12472) af04481e6b3b8dcdf5c1cc7d84bc7ece5658e9ab 960e72607761eb2dd170b33f02a5a2840ec412fe Georgi Gerganov ggerganov@gmail.com 2025-03-21T16:14:29+02:00 GitHub noreply@github.com 2025-03-21T16:14:29+02:00 model : do not repack if a GPU device is present (#12498) 960e72607761eb2dd170b33f02a5a2840ec412fe ea1518e839abe668c20f7e0074c0721f803da898 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-21T10:21:36+01:00 GitHub noreply@github.com 2025-03-21T10:21:36+01:00 chore : cleanup llama_model_loader::TENSOR_ usage (#12492) ea1518e839abe668c20f7e0074c0721f803da898 1aa87ee53d05505247c54612e40f6a38c680b433 marcoStocchi marcostocchi77@gmail.com 2025-03-21T10:12:45+01:00 GitHub noreply@github.com 2025-03-21T11:12:45+02:00 llama-tts : avoid crashes related to bad model file paths (#12482) 1aa87ee53d05505247c54612e40f6a38c680b433 9ffcc9e374080f73b16b7a351e6d76e7a8a19ce3 蕭澧邦 45505768+shou692199@users.noreply.github.com 2025-03-21T14:58:47+08:00 GitHub noreply@github.com 2025-03-21T14:58:47+08:00 [SYCL] Fix build on Windows when ccache enabled (#9954) (#9976) 9ffcc9e374080f73b16b7a351e6d76e7a8a19ce3 e04643063b3d240b8c0fdba98677dff6ba346784 Svetlozar Georgiev 55534064+sgeor255@users.noreply.github.com 2025-03-21T02:15:56Z GitHub noreply@github.com 2025-03-21T10:15:56+08:00 sycl: cleanup oneDNN related code (#12097) e04643063b3d240b8c0fdba98677dff6ba346784 dbb3a4739e53226346c772dd72666e68b78dc583 Woof Dog 197125663+woof-dog@users.noreply.github.com 2025-03-20T14:57:43Z GitHub noreply@github.com 2025-03-20T15:57:43+01:00 webui : Prevent rerendering on textarea input (#12299) dbb3a4739e53226346c772dd72666e68b78dc583 3d82dbcbce2c677fc35fbf99574ccd107d95a1f8 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-20T12:49:59+01:00 GitHub noreply@github.com 2025-03-20T12:49:59+01:00 llama : make Qwen2MoE QKV bias optional (#12477) 3d82dbcbce2c677fc35fbf99574ccd107d95a1f8 732b5fbf5e7f9cf069942f0c5850ee959ef321ba Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2025-03-20T17:05:34+05:30 GitHub noreply@github.com 2025-03-20T13:35:34+02:00 ggml : block interleaving support for Q4_K quantization for x86 AVX2 architecture (#12332) 732b5fbf5e7f9cf069942f0c5850ee959ef321ba 568013d0cd3d5add37c376b3d5e959809b711fc7 Bartowski ckealty1182@gmail.com 2025-03-20T02:36:37-04:00 GitHub noreply@github.com 2025-03-20T08:36:37+02:00 convert : avoid calls to tokenizer.added_tokens_decoder (#12473) 568013d0cd3d5add37c376b3d5e959809b711fc7 517b5ddbf002b91fd6d6daf5d8db8c88a0173039 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2025-03-19T21:01:57+01:00 GitHub noreply@github.com 2025-03-19T21:01:57+01:00 context : clear sets containing encoder output sequence ids before storing new values (#12470) 517b5ddbf002b91fd6d6daf5d8db8c88a0173039 a9b59288e222f39fc0311dc66944ed5a86c815fa Gaurav Garg 52341457+gaugarg-nv@users.noreply.github.com 2025-03-20T01:22:06+05:30 GitHub noreply@github.com 2025-03-19T20:52:06+01:00 CUDA: Improve flash decoding kernel GPU occupancy for BS=1 case (#12183) a9b59288e222f39fc0311dc66944ed5a86c815fa 0fd8487b142b2b92565bc95b39ddc440955a237c Jeff Bolz jbolz@nvidia.com 2025-03-19T13:56:23-05:00 GitHub noreply@github.com 2025-03-19T19:56:23+01:00 vulkan: optimize iq1 coopmat2 dequant functions (#12427) 0fd8487b142b2b92565bc95b39ddc440955a237c 108e53c2f1b57b79c8bcf9773fba5a82e2f1eeb5 Guus Waals _@guusw.nl 2025-03-19T10:15:23Z GitHub noreply@github.com 2025-03-19T11:15:23+01:00 Fix visionOS build and add CI (#12415) 108e53c2f1b57b79c8bcf9773fba5a82e2f1eeb5 a686171ea71ed8cb8a324850d146cb65a001e141 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-19T09:08:49+01:00 GitHub noreply@github.com 2025-03-19T09:08:49+01:00 llama : add support for GPT2, Bloom and CodeShell tied word embeddings (#12456) a686171ea71ed8cb8a324850d146cb65a001e141 c446b2edd2a9fe2772a1a18923c3e54a6749c364 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-19T08:58:13+01:00 GitHub noreply@github.com 2025-03-19T08:58:13+01:00 convert : Support chat_template.json (#12460) c446b2edd2a9fe2772a1a18923c3e54a6749c364 d84635b1b085d54d6a21924e6171688d6e3dfb46 Jeff Bolz jbolz@nvidia.com 2025-03-19T02:26:26-05:00 GitHub noreply@github.com 2025-03-19T08:26:26+01:00 vulkan: Submit once enough matmul work has been recorded (#12406) d84635b1b085d54d6a21924e6171688d6e3dfb46 75422e8bc42646005be0754f7aa438b97a5e777e lhez quic_lih@quicinc.com 2025-03-18T12:54:55-07:00 GitHub noreply@github.com 2025-03-18T12:54:55-07:00 opencl: improve profiling (#12442) 75422e8bc42646005be0754f7aa438b97a5e777e bb115d2bf7ed2cdd7dccd7ae74cc9cfe4b0adb71 Georgi Gerganov ggerganov@gmail.com 2025-03-18T21:35:19+02:00 GitHub noreply@github.com 2025-03-18T21:35:19+02:00 graph : normalize Q, K, V shapes + sync cross attention (#12449) bb115d2bf7ed2cdd7dccd7ae74cc9cfe4b0adb71 29fff308c704c1c752cdb5153361e545e2bac09d R0CKSTAR xiaodong.ye@mthreads.com 2025-03-19T02:28:26+08:00 GitHub noreply@github.com 2025-03-18T19:28:26+01:00 musa: override warp_size of musa device to 32 (#12445) 29fff308c704c1c752cdb5153361e545e2bac09d c6af2161b200284d55633cf184a07406ca89908e Xuan-Son Nguyen son@huggingface.co 2025-03-18T19:16:19+01:00 GitHub noreply@github.com 2025-03-18T19:16:19+01:00 llama : support converting Mistral Small text-only (#12450) c6af2161b200284d55633cf184a07406ca89908e 99aa304fb900654ec338749f64e62895b9a88afd Georgi Gerganov ggerganov@gmail.com 2025-03-18T19:35:11+02:00 GitHub noreply@github.com 2025-03-18T19:35:11+02:00 speculative : fix seg fault in certain cases (#12454) 99aa304fb900654ec338749f64e62895b9a88afd 8551c44d840a7db50adb958ccaf464dc3ded82e7 Xuan-Son Nguyen son@huggingface.co 2025-03-18T17:24:33+01:00 GitHub noreply@github.com 2025-03-18T17:24:33+01:00 llama : add support for EXAONE tied word embeddings (#12451) 8551c44d840a7db50adb958ccaf464dc3ded82e7 35cae5ba05a5292dc3108636a71ec59fa2f80ab7 Georgi Gerganov ggerganov@gmail.com 2025-03-18T13:05:49+02:00 GitHub noreply@github.com 2025-03-18T13:05:49+02:00 context : always use non-causal attention for encoder graphs (#12447) 35cae5ba05a5292dc3108636a71ec59fa2f80ab7 810e0af3f50379682dd46b7967c4aadf3f8286f6 Łukasz Ślusarczyk 112692748+lslusarczyk@users.noreply.github.com 2025-03-18T11:16:31+01:00 GitHub noreply@github.com 2025-03-18T11:16:31+01:00 SYCL: using graphs is configurable by environment variable and compile option (#12371) 810e0af3f50379682dd46b7967c4aadf3f8286f6 eba92d64c3f6d86de2e6b4dd3a540d2805a22b0c Georgi Gerganov ggerganov@gmail.com 2025-03-18T12:05:42+02:00 GitHub noreply@github.com 2025-03-18T12:05:42+02:00 server : fix warmup draft cache type (#12446) eba92d64c3f6d86de2e6b4dd3a540d2805a22b0c d9a14523bb9074eef42d1b43ae4a1e149f81b7e2 Prajwal B Mehendarkar prajwal.b.mehendarkar@ibm.com 2025-03-18T15:07:33+05:30 GitHub noreply@github.com 2025-03-18T11:37:33+02:00 cmake : fix PowerPC build (#12241) d9a14523bb9074eef42d1b43ae4a1e149f81b7e2 fd123cfead49eb32e386e26b8ef7a6d41554dda5 fj-y-saito 85871716+fj-y-saito@users.noreply.github.com 2025-03-18T17:14:39+09:00 GitHub noreply@github.com 2025-03-18T10:14:39+02:00 ggml : add SVE support for q6_K_q8_K (#12361) fd123cfead49eb32e386e26b8ef7a6d41554dda5 a53f7f7b8859f3e634415ab03e1e295b9861d7e6 0cc4m picard12@live.de 2025-03-18T07:21:40+01:00 GitHub noreply@github.com 2025-03-18T07:21:40+01:00 Vulkan: Default to 1GB allocations instead of 4GB to avoid fragmentation and driver issues (#12434) a53f7f7b8859f3e634415ab03e1e295b9861d7e6 7dfad387e3f6ac98d383ded2d175eb59736a3993 Łukasz Ślusarczyk 112692748+lslusarczyk@users.noreply.github.com 2025-03-18T01:51:25+01:00 GitHub noreply@github.com 2025-03-18T08:51:25+08:00 fixed compilation warnings in ggml-sycl (#12424) 7dfad387e3f6ac98d383ded2d175eb59736a3993 60c902926c928f9c2cd6390ce411876f92feeaf3 Molly Sophia mollysophia379@gmail.com 2025-03-18T07:27:50+08:00 GitHub noreply@github.com 2025-03-18T07:27:50+08:00 llama: Add support for RWKV v7 architecture (#12412) 60c902926c928f9c2cd6390ce411876f92feeaf3 b1b132efcba216c873715c483809730bb253f4a1 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-17T21:14:32+01:00 GitHub noreply@github.com 2025-03-17T21:14:32+01:00 docs : bring llama-cli conversation/template docs up-to-date (#12426) b1b132efcba216c873715c483809730bb253f4a1 01e8f2138b2e40902afe2983ecbf503a08d74b1d Gaurav Garg 52341457+gaugarg-nv@users.noreply.github.com 2025-03-17T23:55:13+05:30 GitHub noreply@github.com 2025-03-17T20:25:13+02:00 cuda : enable CUDA Graph on CUDA Toolkit < 12.x (#12394) 01e8f2138b2e40902afe2983ecbf503a08d74b1d 484a8ab513bbd740cc49f30280c1acf52cb4e7e9 Guus Waals _@guusw.nl 2025-03-18T00:35:43+08:00 GitHub noreply@github.com 2025-03-17T13:35:43-03:00 ggml-vulkan: remove unused find_program(glslc) (#12416) 484a8ab513bbd740cc49f30280c1acf52cb4e7e9 cf2270e4d3685ac46f4a166d8718997ba7cbc45a Jeff Bolz jbolz@nvidia.com 2025-03-17T09:26:18-05:00 GitHub noreply@github.com 2025-03-17T09:26:18-05:00 vulkan: Add N/2 and N/4 optimized paths in coopmat2 shader (#12312) cf2270e4d3685ac46f4a166d8718997ba7cbc45a f07690c930f74d82d4f108e567c7092544847f77 Daniele daniele.dilotorres@gmail.com 2025-03-17T12:42:33+01:00 GitHub noreply@github.com 2025-03-17T12:42:33+01:00 vulkan: subgroup size tuning (#12087) f07690c930f74d82d4f108e567c7092544847f77 891c63956dbfbdf7ed2ecd0b5882cff49dbfe90f Jeff Bolz jbolz@nvidia.com 2025-03-17T04:43:35-05:00 GitHub noreply@github.com 2025-03-17T10:43:35+01:00 vulkan: use fp32 in coopmat2 q4_k dequant function (#12309) 891c63956dbfbdf7ed2ecd0b5882cff49dbfe90f 2f21123c1deb3ce1be3c0578c5f6980fe19ed077 Jeff Bolz jbolz@nvidia.com 2025-03-17T04:41:59-05:00 GitHub noreply@github.com 2025-03-17T10:41:59+01:00 vulkan: Pad N dimension of B matrix for coopmat2 perf, to avoid bounds checking (#12273) 2f21123c1deb3ce1be3c0578c5f6980fe19ed077 374101fd742bb35cb9bf46c86836e54d51191ffd Jeff Bolz jbolz@nvidia.com 2025-03-17T04:35:00-05:00 GitHub noreply@github.com 2025-03-17T10:35:00+01:00 vulkan: Adjust coopmat2 tile sizes and selection heuristic (#12258) 374101fd742bb35cb9bf46c86836e54d51191ffd b3c9a65673a63a6c9a75da24ee00d13499494e0c Christian Kastner ckk@kvr.at 2025-03-17T10:05:23+01:00 GitHub noreply@github.com 2025-03-17T11:05:23+02:00 cmake : enable building llama.cpp using system libggml (#12321) b3c9a65673a63a6c9a75da24ee00d13499494e0c 8ba95dca2065c0073698afdfcda4c8a8f08bf0d9 Akarshan Biswas akarshan@menlo.ai 2025-03-17T07:15:12+05:30 GitHub noreply@github.com 2025-03-17T09:45:12+08:00 SYCL: set extras only on GGML_TYPE_Q4_0 (#12366) 8ba95dca2065c0073698afdfcda4c8a8f08bf0d9 dc079cfdffa1141a6caf5d41a33d73a1ef03da55 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-16T18:46:36+01:00 GitHub noreply@github.com 2025-03-16T19:46:36+02:00 llama : fix OLMo-2-0325-32B-Instruct K-norm size (#12400) dc079cfdffa1141a6caf5d41a33d73a1ef03da55 7b61bcc87cfdeab88350e82df1c4b7be64331ea6 Georgi Gerganov ggerganov@gmail.com 2025-03-16T19:29:36+02:00 GitHub noreply@github.com 2025-03-16T19:29:36+02:00 context : fix init of n_outputs (#12397) 7b61bcc87cfdeab88350e82df1c4b7be64331ea6 f4c3dd5daa3a79f713813cf1aabdc5886071061d Daniel Bevenius daniel.bevenius@gmail.com 2025-03-16T18:22:05+01:00 GitHub noreply@github.com 2025-03-16T18:22:05+01:00 ci : add --symlinks to xcframework zip command (#12409) f4c3dd5daa3a79f713813cf1aabdc5886071061d 3d35d87b4113648e224b837bb88e6b2c4c7f29e5 marcoStocchi marcostocchi77@gmail.com 2025-03-15T17:23:11+01:00 GitHub noreply@github.com 2025-03-15T17:23:11+01:00 llama-tts : add '-o' option (#12398) 3d35d87b4113648e224b837bb88e6b2c4c7f29e5 b19bd064c09822cb81efe4a38abafab3e979c9ce aubreyli aubreylee@gmail.com 2025-03-15T22:49:03+08:00 GitHub noreply@github.com 2025-03-15T15:49:03+01:00 SYCL: Delete redundant plus sign and space (#12391) b19bd064c09822cb81efe4a38abafab3e979c9ce 92a391327e9201b9b5b32fdd3afb452026c22d4c fairydreaming 166155368+fairydreaming@users.noreply.github.com 2025-03-15T15:19:30+01:00 GitHub noreply@github.com 2025-03-15T22:19:30+08:00 SYCL : support non-contiguous tensors in binary ops (add, sub, etc) (#12399) 92a391327e9201b9b5b32fdd3afb452026c22d4c 9f2250ba722738ec0e6ab684636268a79160c854 Chenguang Li 757486878@qq.com 2025-03-15T09:31:08+08:00 GitHub noreply@github.com 2025-03-15T09:31:08+08:00 [CANN]MUL_MAT optimization (#12382) 9f2250ba722738ec0e6ab684636268a79160c854 774973b8f3d5e375b0b74d58638eeb1817e950a8 Eric Curtin ecurtin@redhat.com 2025-03-14T16:41:20Z GitHub noreply@github.com 2025-03-14T16:41:20Z Add CLI arg to llama-run to adjust the number of threads used (#12370) 774973b8f3d5e375b0b74d58638eeb1817e950a8 8fcb563613e20a04dd9791f0a9b8a41086428c09 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-14T16:57:05+01:00 GitHub noreply@github.com 2025-03-14T16:57:05+01:00 main : add -sysf / --system-prompt-file (#12249) (#12250) 8fcb563613e20a04dd9791f0a9b8a41086428c09 add2a3aa5a1571211aa5c7303b8e80c8d1824b91 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2025-03-14T13:47:05+01:00 GitHub noreply@github.com 2025-03-14T13:47:05+01:00 Load all MoE experts during warmup (#11571) add2a3aa5a1571211aa5c7303b8e80c8d1824b91 c522ce4143a2b5c277f1e5f65cd570dbd0626466 Victor 194116445+dodekapod@users.noreply.github.com 2025-03-14T11:21:17+01:00 GitHub noreply@github.com 2025-03-14T11:21:17+01:00 server: fix "--grammar-file" parameter (#12285) c522ce4143a2b5c277f1e5f65cd570dbd0626466 081bee8c643b1f6302e9edfe789ce2d5f0be6c77 Georgi Gerganov ggerganov@gmail.com 2025-03-14T10:47:44+02:00 GitHub noreply@github.com 2025-03-14T10:47:44+02:00 graph : simplify attn input build for unified KV cache (#12381) 081bee8c643b1f6302e9edfe789ce2d5f0be6c77 84d547554123a62e9ac77107cb20e4f6cc503af4 Georgi Gerganov ggerganov@gmail.com 2025-03-14T09:03:24+02:00 GitHub noreply@github.com 2025-03-14T09:03:24+02:00 hparams : add SWA rope parameters (#12374) 84d547554123a62e9ac77107cb20e4f6cc503af4 be7c3034108473beda214fd1d7c98fd6a7a3bdf5 Georgi Gerganov ggerganov@gmail.com 2025-03-13T19:08:07+02:00 GitHub noreply@github.com 2025-03-13T19:08:07+02:00 llama : fix Gemma3 SWA KV cache shift (#12373) be7c3034108473beda214fd1d7c98fd6a7a3bdf5 e0dbec0bc6cd4b6230cda7a6ed1e9dac08d1600b Xuan-Son Nguyen son@huggingface.co 2025-03-13T12:34:54+01:00 GitHub noreply@github.com 2025-03-13T12:34:54+01:00 arg : no n_predict = -2 for examples except for main and infill (#12364) e0dbec0bc6cd4b6230cda7a6ed1e9dac08d1600b 2048b5913d51beab82dfe29955f9008130b936c0 Georgi Gerganov ggerganov@gmail.com 2025-03-13T12:35:44+02:00 GitHub noreply@github.com 2025-03-13T12:35:44+02:00 llama : refactor llama_context, llama_kv_cache, llm_build_context (#12181) 2048b5913d51beab82dfe29955f9008130b936c0 f08f4b3187b691bb08a8884ed39ebaa94e956707 Ishaan Gandhi Ishaangandhi@gmail.com 2025-03-13T06:10:05-04:00 GitHub noreply@github.com 2025-03-13T11:10:05+01:00 server : fix crash when using verbose output with input tokens that are not in printable range (#12178) (#12338) f08f4b3187b691bb08a8884ed39ebaa94e956707 80a02aa8588ef167d616f76f1781b104c245ace0 Oscar Barenys rtfss1@gmail.com 2025-03-12T20:06:58+01:00 GitHub noreply@github.com 2025-03-12T20:06:58+01:00 Update build.yml for Windows Vulkan builder to use Vulkan 1.4.304 SDK for VK_NV_cooperative_matrix2 support (#12301) 80a02aa8588ef167d616f76f1781b104c245ace0 363f8c5d67dcf80e00c39580dfa86dc2774d74c2 Daniel Bevenius daniel.bevenius@gmail.com 2025-03-12T13:45:32+01:00 GitHub noreply@github.com 2025-03-12T13:45:32+01:00 llama.swiftui : fix xcframework dir in README [no ci] (#12353) 363f8c5d67dcf80e00c39580dfa86dc2774d74c2 34c961b181836a4f06ab4c56d5ce61ce03fc478b Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2025-03-12T09:57:32Z GitHub noreply@github.com 2025-03-12T09:57:32Z sycl : variable sg_size support for mmvq kernels (#12336) 34c961b181836a4f06ab4c56d5ce61ce03fc478b 7841fc723e059d1fd9640e5c0ef19050fcc7c698 uvos philipp@uvos.xyz 2025-03-12T10:14:11+01:00 GitHub noreply@github.com 2025-03-12T10:14:11+01:00 CUDA/HIP: Fix fattn-vec-* when device warp size is not 32 (#12315) 7841fc723e059d1fd9640e5c0ef19050fcc7c698 bf69cfe62f9ccc01112c0232a55820b95a8c1fda Xuan-Son Nguyen son@huggingface.co 2025-03-12T09:30:24+01:00 GitHub noreply@github.com 2025-03-12T09:30:24+01:00 llama : Add Gemma 3 support (+ experimental vision capability) (#12343) bf69cfe62f9ccc01112c0232a55820b95a8c1fda 10f2e81809bbb69ecfe64fc8b4686285f84b0c07 Jeff Bolz jbolz@nvidia.com 2025-03-12T00:59:19-05:00 GitHub noreply@github.com 2025-03-12T06:59:19+01:00 vulkan: fix bug in coopmat1 mul_mat_id (#12316) 10f2e81809bbb69ecfe64fc8b4686285f84b0c07 ba7654380a3c7c1b5ae154bea19134a3a9417a1e uvos philipp@uvos.xyz 2025-03-11T20:16:03+01:00 GitHub noreply@github.com 2025-03-11T20:16:03+01:00 CUDA/HIP: refractor mmqv to unify the calculation of nwarps and rows per block between host and device code. (#12177) ba7654380a3c7c1b5ae154bea19134a3a9417a1e 6ab2e4765a673abcd162258a2671560a76106d69 jklincn 985765408@qq.com 2025-03-11T21:25:17+08:00 GitHub noreply@github.com 2025-03-11T14:25:17+01:00 ggml-backend : fix backend search path (#12330) 6ab2e4765a673abcd162258a2671560a76106d69 96e1280839561aaabb73851f94972a2cd37b2d96 BB-fat 45072480+BB-fat@users.noreply.github.com 2025-03-11T19:45:02+08:00 GitHub noreply@github.com 2025-03-11T13:45:02+02:00 metal : Cache the Metal library at the device context level (#12265) 96e1280839561aaabb73851f94972a2cd37b2d96 2c9f833d17bb5b8ea89dec663b072b5420fc5438 Xuan-Son Nguyen thichthat@gmail.com 2025-03-11T09:20:16+01:00 GitHub noreply@github.com 2025-03-11T09:20:16+01:00 clip : bring back GPU support (#12322) 2c9f833d17bb5b8ea89dec663b072b5420fc5438 251364549fe4a78d4e2e66f1adfaf2bd53041d2c Eve 139727413+netrunnereve@users.noreply.github.com 2025-03-10T19:28:11Z GitHub noreply@github.com 2025-03-10T19:28:11Z mat vec double buffer (#12188) 251364549fe4a78d4e2e66f1adfaf2bd53041d2c 8acdacb3ea00697477eb019efbb6fc183371055c R0CKSTAR xiaodong.ye@mthreads.com 2025-03-11T01:18:25+08:00 GitHub noreply@github.com 2025-03-10T18:18:25+01:00 musa: support new arch mp_31 and update doc (#12296) 8acdacb3ea00697477eb019efbb6fc183371055c 89b2b56e8658800375a8314200870b1ad4208a0b Henry Linjamäki henry.mikael.linjamaki@intel.com 2025-03-10T18:57:00+02:00 GitHub noreply@github.com 2025-03-10T09:57:00-07:00 opencl: use OpenCL C standard supported by the device (#12221) 89b2b56e8658800375a8314200870b1ad4208a0b e128a1bf5b65741b485dd094a4201264d0580d68 John Bean 113509988+johnbean393@users.noreply.github.com 2025-03-10T22:13:09+08:00 GitHub noreply@github.com 2025-03-10T16:13:09+02:00 readme: added Sidekick to available UIs (#12311) e128a1bf5b65741b485dd094a4201264d0580d68 6ef79a67caf1159b0150b44ee80888c7ec98b83f Georgi Gerganov ggerganov@gmail.com 2025-03-10T14:07:15+02:00 GitHub noreply@github.com 2025-03-10T14:07:15+02:00 tests : fix test-quantize-fns to init the CPU backend (#12306) 6ef79a67caf1159b0150b44ee80888c7ec98b83f 4e39a3c332f84b890e91353ec448502cb8373a6f marcoStocchi marcostocchi77@gmail.com 2025-03-10T12:34:13+01:00 GitHub noreply@github.com 2025-03-10T13:34:13+02:00 common : refactor '-o' option (#12278) 4e39a3c332f84b890e91353ec448502cb8373a6f be421fc429795d135786f5a0e489709220a9c43a Olivier Chafik ochafik@users.noreply.github.com 2025-03-10T10:59:03Z GitHub noreply@github.com 2025-03-10T10:59:03Z `server`: extract tags from qwq outputs (#12297) be421fc429795d135786f5a0e489709220a9c43a 87c2630546cd8ccc836ae4c7d87d03fa597d2267 Olivier Chafik ochafik@users.noreply.github.com 2025-03-10T09:45:29Z GitHub noreply@github.com 2025-03-10T09:45:29Z `tool-call`: ensure there's always a non-empty tool call id (#12292) 87c2630546cd8ccc836ae4c7d87d03fa597d2267 2b3a25c212f8c4d8a49cec23e03343a7719d51c9 Olivier Chafik ochafik@users.noreply.github.com 2025-03-10T09:45:07Z GitHub noreply@github.com 2025-03-10T09:45:07Z allow missing content in message if tool_calls provided (#12293) 2b3a25c212f8c4d8a49cec23e03343a7719d51c9 8352cdc87b207a735d34c431a36c425728cb4586 Olivier Chafik ochafik@users.noreply.github.com 2025-03-10T09:44:42Z GitHub noreply@github.com 2025-03-10T09:44:42Z `sampler`: fixes trigger tokens + lazy grammars (fix typo cast from token to string) (#12291) 8352cdc87b207a735d34c431a36c425728cb4586 1e2f78a00450593e2dfa458796fcdd9987300dfc tc-mb 157115220+tc-mb@users.noreply.github.com 2025-03-10T16:33:24+08:00 GitHub noreply@github.com 2025-03-10T10:33:24+02:00 llava : fix bug in minicpm-v code (#11513) 1e2f78a00450593e2dfa458796fcdd9987300dfc 0fd7ca7a210bd4abc995cd728491043491dbdef7 Georgi Gerganov ggerganov@gmail.com 2025-03-09T19:08:20+02:00 GitHub noreply@github.com 2025-03-09T19:08:20+02:00 server : add speculative decoding presets for FIM (#12287) 0fd7ca7a210bd4abc995cd728491043491dbdef7 6fefc05a7a4e676780ae10b0a4d0728e5281f367 Georgi Gerganov ggerganov@gmail.com 2025-03-08T18:26:00+02:00 GitHub noreply@github.com 2025-03-08T18:26:00+02:00 authors : update (#12271) 6fefc05a7a4e676780ae10b0a4d0728e5281f367 7ab364390f92b0b8d83f69821a536b424838f3f8 Jason C.H ctrysbita@outlook.com 2025-03-09T00:02:39+08:00 GitHub noreply@github.com 2025-03-08T17:02:39+01:00 ggml-backend : make path_str compatible with C++20 (#12269) 7ab364390f92b0b8d83f69821a536b424838f3f8 7c7f3b7f435f41f2508e0e3010f0013cd8335156 Georgi Gerganov ggerganov@gmail.com 2025-03-07T20:54:30+02:00 GitHub noreply@github.com 2025-03-07T20:54:30+02:00 server : infill gen ends on new line (#12254) 7c7f3b7f435f41f2508e0e3010f0013cd8335156 102ac1891db32c346a7b6b96145a2a23c1e4c352 Daniel Bevenius daniel.bevenius@gmail.com 2025-03-07T14:15:27+01:00 GitHub noreply@github.com 2025-03-07T14:15:27+01:00 ggml : skip intermediate .air file when compiling .metallib (#12247) 102ac1891db32c346a7b6b96145a2a23c1e4c352 d6ae2fa06139e496880cbf65197c84341e9d98e7 Georgi Gerganov ggerganov@gmail.com 2025-03-07T14:00:27+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-07T14:49:44+02:00 sync : ggml d6ae2fa06139e496880cbf65197c84341e9d98e7 68d0027f3d19eb579c1863814c91e37ffa699014 vmobilis 75476228+vmobilis@users.noreply.github.com 2025-03-07T11:11:40+03:00 Georgi Gerganov ggerganov@gmail.com 2025-03-07T14:49:44+02:00 ggml : ggml_compute_forward_concat() for arbitrary tensor type (ggml/1118) 68d0027f3d19eb579c1863814c91e37ffa699014 ea002810a209246d034d1b6ddac387f778751588 Rémy O remyoudompheng@gmail.com 2025-03-07T12:54:22+01:00 GitHub noreply@github.com 2025-03-07T13:54:22+02:00 ggml-cpu: faster AVX2 variant for IQ1_M (#12216) ea002810a209246d034d1b6ddac387f778751588 8fad3c7a7c54a25a1ca38dfb08244df55288e675 Georgi Gerganov ggerganov@gmail.com 2025-03-07T12:19:31+02:00 GitHub noreply@github.com 2025-03-07T12:19:31+02:00 ci : fix save-load test invocations (#12245) 8fad3c7a7c54a25a1ca38dfb08244df55288e675 7cf64f6beecf54c6ac71503181f154667fd4228a Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-07T11:15:33+01:00 GitHub noreply@github.com 2025-03-07T11:15:33+01:00 server : Log original chat template parsing error (#12233) 7cf64f6beecf54c6ac71503181f154667fd4228a 5e2d57b2b2e43eadbe6d66ba3e873a824b95e725 Olivier Chafik ochafik@users.noreply.github.com 2025-03-07T09:33:37Z GitHub noreply@github.com 2025-03-07T09:33:37Z sync: minja - support QwQ-32B (#12235) 5e2d57b2b2e43eadbe6d66ba3e873a824b95e725 f1648e91cf6c52e9593810aa70857e412d474c09 BB-fat 45072480+BB-fat@users.noreply.github.com 2025-03-07T15:35:57+08:00 GitHub noreply@github.com 2025-03-07T08:35:57+01:00 metal : simplify kernel arguments using a struct (#3229) (#12194) f1648e91cf6c52e9593810aa70857e412d474c09 d6c95b0740510231b3797b80d6d3440d8fe188b6 David Huang 1969802+hjc4869@users.noreply.github.com 2025-03-07T15:06:08+08:00 GitHub noreply@github.com 2025-03-07T08:06:08+01:00 HIP: fix rocWMMA build flags under Windows (#12230) d6c95b0740510231b3797b80d6d3440d8fe188b6 d76a86d967ef491d530400b08bc8ef8a14807936 Daniel Bevenius daniel.bevenius@gmail.com 2025-03-07T06:23:16+01:00 GitHub noreply@github.com 2025-03-07T06:23:16+01:00 metal : fix default.metallib build (#12224) d76a86d967ef491d530400b08bc8ef8a14807936 776f9e59cc8a85e840d1d4af8540d199c77190ac lhez quic_lih@quicinc.com 2025-03-06T16:20:35-08:00 GitHub noreply@github.com 2025-03-07T00:20:35Z opencl: Noncontiguous `norm`, `rms_norm`, disable `fp16` for some ops (#12217) 776f9e59cc8a85e840d1d4af8540d199c77190ac 3d652bfddfba09022525067e672c3c145c074649 xiaofei hbuxiaofei@gmail.com 2025-03-07T06:58:25+08:00 GitHub noreply@github.com 2025-03-06T22:58:25Z cmake : fix undefined reference errors for std::filesystem in ggml (#12092) (#12094) 3d652bfddfba09022525067e672c3c145c074649 5220a16d18563d3ffc509002f0514415fdda4036 Lucas Moura Belo lucas.belo@live.com 2025-03-06T16:15:13-03:00 GitHub noreply@github.com 2025-03-06T21:15:13+02:00 readme : update bindings (#12229) 5220a16d18563d3ffc509002f0514415fdda4036 3ffbbd5ce130859be91909e9b77d4c1962a6be2c Johannes Gäßler johannesg@5d6.de 2025-03-06T18:45:09+01:00 GitHub noreply@github.com 2025-03-06T18:45:09+01:00 CUDA: fix FA logic for PTX 7.0 and CC >= 7.5 (#12222) 3ffbbd5ce130859be91909e9b77d4c1962a6be2c 42994048a34b0bddd72b57c26f8ae2c7d417946d David Huang 1969802+hjc4869@users.noreply.github.com 2025-03-06T21:14:11+08:00 GitHub noreply@github.com 2025-03-06T14:14:11+01:00 HIP: rocWMMA documentation and enabling in workflow builds (#12179) 42994048a34b0bddd72b57c26f8ae2c7d417946d e9b2f84f145fbd458dcb98f227bd09370918be6e Olivier Chafik ochafik@users.noreply.github.com 2025-03-06T09:03:31Z GitHub noreply@github.com 2025-03-06T09:03:31Z update function-calling.md w/ template override for functionary-small-v3.2 (#12214) e9b2f84f145fbd458dcb98f227bd09370918be6e e721c05c9336a72fbb59d5c75967360bc67036c6 Aaron Teo 57927438+taronaeo@users.noreply.github.com 2025-03-06T16:33:21+08:00 GitHub noreply@github.com 2025-03-06T09:33:21+01:00 llava: add big-endian conversion for image encoder (#12218) e721c05c9336a72fbb59d5c75967360bc67036c6 57b6abf85acb1f697913a44e5e105e333d894b78 uvos philipp@uvos.xyz 2025-03-06T08:20:52+01:00 GitHub noreply@github.com 2025-03-06T08:20:52+01:00 HIP/CUDA: set the paramerter value in maintain_cuda_graph instead of replaceing it. (#12209) 57b6abf85acb1f697913a44e5e105e333d894b78 94bb63e4f0f6135579b88e5700ed40cefa374e09 Han Yin han.yin@arm.com 2025-03-05T22:22:49-08:00 GitHub noreply@github.com 2025-03-06T08:22:49+02:00 android : fix KV cache log message condition (#12212) 94bb63e4f0f6135579b88e5700ed40cefa374e09 f79243992cd31e4e4844d5158d2f3325b1d2d811 Henry Linjamäki henry.linjamaki@gmail.com 2025-03-06T03:33:40+02:00 GitHub noreply@github.com 2025-03-06T02:33:40+01:00 opencl : fix buffer alignment (#12197) f79243992cd31e4e4844d5158d2f3325b1d2d811 ed4ce0dda24986c80d58e2ce112049af00f632f4 Henry Linjamäki henry.linjamaki@gmail.com 2025-03-06T03:31:14+02:00 GitHub noreply@github.com 2025-03-06T02:31:14+01:00 opencl : fix `ulong` kernel args were set from `int` variables (#12174) ed4ce0dda24986c80d58e2ce112049af00f632f4 07d15723470a0a5b15d8ccad1aff5b20354ffbe1 simon886212 37953122+simon886212@users.noreply.github.com 2025-03-06T09:30:05+08:00 GitHub noreply@github.com 2025-03-06T02:30:05+01:00 opencl : fix profile-related errors (#12095) 07d15723470a0a5b15d8ccad1aff5b20354ffbe1 5e43f104cca1a14874e980326a506b44fde022b8 Rémy O remyoudompheng@gmail.com 2025-03-06T02:26:10+01:00 GitHub noreply@github.com 2025-03-06T02:26:10+01:00 ggml-cpu: Faster IQ1 mul_mat_vec on AVX2 using BMI2 instructions (#12154) 5e43f104cca1a14874e980326a506b44fde022b8 16e4b22c5e58ee200ede913fd7b7b8ba92132ce8 Akarshan Biswas akarshan@menlo.ai 2025-03-05T21:28:23+05:30 GitHub noreply@github.com 2025-03-05T16:58:23+01:00 SYCL: Disable f16 Unary OPs as not supported by the kernels (#12201) 16e4b22c5e58ee200ede913fd7b7b8ba92132ce8 074c4fd39df3af974e10fbee6cc6db5d9304655b Plamen Minev pacominev@gmail.com 2025-03-05T17:16:01+02:00 GitHub noreply@github.com 2025-03-05T17:16:01+02:00 ggml : fix GGMLMetalClass ODR (#12200) 074c4fd39df3af974e10fbee6cc6db5d9304655b 669912d9a5bf927312c553332ff997f0a99da8fb Daniel Bevenius daniel.bevenius@gmail.com 2025-03-05T14:16:40+01:00 GitHub noreply@github.com 2025-03-05T14:16:40+01:00 ci : add fetch-depth to xcframework upload (#12195) 669912d9a5bf927312c553332ff997f0a99da8fb fa31c438e0e709242ab7334d26fc3be3dcda07a0 Olivier Chafik ochafik@users.noreply.github.com 2025-03-05T13:05:13Z GitHub noreply@github.com 2025-03-05T13:05:13Z `tool-call`: fix Qwen 2.5 Coder support, add micro benchmarks, support trigger patterns for lazy grammars (#12034) fa31c438e0e709242ab7334d26fc3be3dcda07a0 3ccbfe5a71c74ac574b00607067d0aa0a49df04c Daniel Bevenius daniel.bevenius@gmail.com 2025-03-05T10:22:29+01:00 GitHub noreply@github.com 2025-03-05T10:22:29+01:00 ci : fix xcframework artifact tag (#12191) 3ccbfe5a71c74ac574b00607067d0aa0a49df04c 06a92a193a07afe445929607be9d5e4d033956fb Daniel Bevenius daniel.bevenius@gmail.com 2025-03-05T08:34:02+01:00 GitHub noreply@github.com 2025-03-05T08:34:02+01:00 ci : remove xframework upload (#12190) 06a92a193a07afe445929607be9d5e4d033956fb a057897ad4e48e3df0354256e0cc80dadcb57595 Clauszy zhangyub@uniontech.com 2025-03-05T15:25:45+08:00 GitHub noreply@github.com 2025-03-05T09:25:45+02:00 server : fix cache reuse logic (#12161) a057897ad4e48e3df0354256e0cc80dadcb57595 5bbe6a9fe9a8796a9389c85accec89dbc4d91e39 Daniel Bevenius daniel.bevenius@gmail.com 2025-03-05T06:30:31+01:00 GitHub noreply@github.com 2025-03-05T06:30:31+01:00 llama : add xcframework build script (#11996) 5bbe6a9fe9a8796a9389c85accec89dbc4d91e39 20a9b8f5e1380243ed03aeb50ae1bf94b8d68501 mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2025-03-04T17:53:26+01:00 GitHub noreply@github.com 2025-03-04T18:53:26+02:00 ggml : portability fixes for VS 2017 (#12150) 20a9b8f5e1380243ed03aeb50ae1bf94b8d68501 56d7a9f81274717fe035db192f315a049261c7fa Georgi Gerganov ggerganov@gmail.com 2025-03-04T18:42:44+02:00 GitHub noreply@github.com 2025-03-04T18:42:44+02:00 readme : fix roadmap link (#12185) 56d7a9f81274717fe035db192f315a049261c7fa 1a24c4621f0280306b0d53a4fa474fc65d3f1b2e Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-04T17:19:39+01:00 GitHub noreply@github.com 2025-03-04T12:19:39-04:00 main: allow preloading conversation with -p and add -st / --single-turn (#12145) 1a24c4621f0280306b0d53a4fa474fc65d3f1b2e becade5de77674696539163dfbaf5c041a1a8e97 Olivier Chafik ochafik@users.noreply.github.com 2025-03-04T06:24:07Z GitHub noreply@github.com 2025-03-04T08:24:07+02:00 `server`: fix deadly typo in response_format.json_schema.schema handling (#12168) becade5de77674696539163dfbaf5c041a1a8e97 dfd6b2c0be191b3abe2fd9c1b25deff01c6249d8 David Huang 1969802+hjc4869@users.noreply.github.com 2025-03-04T05:10:54+08:00 GitHub noreply@github.com 2025-03-03T22:10:54+01:00 HIP: implement FlashAttention via rocWMMA for CDNA and RDNA3+ (#12032) dfd6b2c0be191b3abe2fd9c1b25deff01c6249d8 b64d7cc2720a90c03480d9408fae40d936c110e2 Georgi Gerganov ggerganov@gmail.com 2025-03-03T17:57:38+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 sync : ggml b64d7cc2720a90c03480d9408fae40d936c110e2 3d1cf3cf3394f2288dca3e1fb1fbb467d366e4bb cmdr2 secondary.cmdr2@gmail.com 2025-03-03T20:51:31+05:30 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 cuda: unary ops as float + de-duplicate (ggml/1130) 3d1cf3cf3394f2288dca3e1fb1fbb467d366e4bb 0cbee131ad332a8dab5bc72315f085a544682c26 Georgi Gerganov ggerganov@gmail.com 2025-02-28T12:37:35+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 sync : ggml 0cbee131ad332a8dab5bc72315f085a544682c26 8371d445958195f9ecdff39f67c0b357f1347e3a cmdr2 secondary.cmdr2@gmail.com 2025-02-28T12:36:46+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 cuda/vulkan: specify fp32-only support for some operations in supports_op (ggml/1129) 8371d445958195f9ecdff39f67c0b357f1347e3a 87abb7e903635a2660e89f9336122f374d683e0a Georgi Gerganov ggerganov@gmail.com 2025-02-28T09:09:58+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 sync : ggml 87abb7e903635a2660e89f9336122f374d683e0a 6d4c23b81b03c7b089a5f7a21ca73d1385d37191 cmdr2 secondary.cmdr2@gmail.com 2025-02-28T12:34:39+05:30 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 cuda/cpu: Increase support for fp16 unary operations (ggml/1125) 6d4c23b81b03c7b089a5f7a21ca73d1385d37191 6512a9003741bd3fe2e11bf3bb3608ec497d368d Diego Devesa slarengh@gmail.com 2025-02-27T13:35:07+01:00 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 whisper : support GGML_BACKEND_DL (whisper/2843) 6512a9003741bd3fe2e11bf3bb3608ec497d368d 4512055792cff7ea107f2d2231f79aa1af073c62 midnight midnightmagic@users.noreply.github.com 2025-02-05T04:41:10-08:00 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 cmake : fix compile assumptions for power9/etc (whisper/2777) 4512055792cff7ea107f2d2231f79aa1af073c62 f54a4ba11ed8ab59da778d5e280f7dc73c775a7a petterreinholdtsen pere-github@hungry.com 2025-02-26T21:44:00+01:00 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 Told cmake to install ggml-cpp.h as a public header file. (ggml/1126) f54a4ba11ed8ab59da778d5e280f7dc73c775a7a aede2074f608d7bf6614cdd047bce687a2b3d908 cmdr2 shashank.shekhar.global@gmail.com 2025-02-25T18:06:34+05:30 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 Support pure float16 add/sub/mul/div operations in the CUDA (and CPU) backend (ggml/1121) aede2074f608d7bf6614cdd047bce687a2b3d908 2679c3b55d1c9c3fed56dea00fea713622e42594 Georgi Gerganov ggerganov@gmail.com 2025-02-28T09:09:38+02:00 Georgi Gerganov ggerganov@gmail.com 2025-03-03T18:18:11+02:00 scripts : sync-ggml-am.sh fix 2679c3b55d1c9c3fed56dea00fea713622e42594 c43af9276b119dae7436b7878d59671d0f6b1a97 Daniel Bevenius daniel.bevenius@gmail.com 2025-03-03T16:17:36+01:00 GitHub noreply@github.com 2025-03-03T16:17:36+01:00 ci : set GITHUB_ACTION env var for server tests (#12162) c43af9276b119dae7436b7878d59671d0f6b1a97 d5c63cd7f932663a20a860ef11a238e627abac53 dm4 dm4@secondstate.io 2025-03-03T21:09:29+08:00 GitHub noreply@github.com 2025-03-03T15:09:29+02:00 tts: add speaker file support (#12048) d5c63cd7f932663a20a860ef11a238e627abac53 9660ffef582ca275092b621c87085a6eea136a90 Diego Devesa slarengh@gmail.com 2025-03-03T14:00:46+01:00 GitHub noreply@github.com 2025-03-03T14:00:46+01:00 test-backend-ops : add option -p to filter by op params (#12155) 9660ffef582ca275092b621c87085a6eea136a90 c950a1f69269bff416d74349a82d78181ce6f0f8 ag2s20150909 19373730+ag2s20150909@users.noreply.github.com 2025-03-03T20:54:08+08:00 GitHub noreply@github.com 2025-03-03T13:54:08+01:00 ggml : fix kleidiai build (#12159) c950a1f69269bff416d74349a82d78181ce6f0f8 7b69003af74924ac04d97313c2e57c45ba56b616 Eric Curtin ecurtin@redhat.com 2025-03-03T12:44:56Z GitHub noreply@github.com 2025-03-03T12:44:56Z Adding UTF-8 support to llama.cpp (#12111) 7b69003af74924ac04d97313c2e57c45ba56b616 ece9745bb8079b9f4af45df29b67ad0c6e50584d Xuan-Son Nguyen thichthat@gmail.com 2025-03-03T11:42:45+01:00 GitHub noreply@github.com 2025-03-03T11:42:45+01:00 webui : add ?m=... and ?q=... params (#12148) ece9745bb8079b9f4af45df29b67ad0c6e50584d cc473cac7cea1484c1f870231073b0bf0352c6f9 Akarshan Biswas akarshan@menlo.ai 2025-03-03T15:37:22+05:30 GitHub noreply@github.com 2025-03-03T11:07:22+01:00 SYCL: Move CPY kernels to a separate file and add few missing kernels (#12133) cc473cac7cea1484c1f870231073b0bf0352c6f9 14dec0c2f29ae56917907dbf2eed6b19438d0a0e Diego Devesa slarengh@gmail.com 2025-03-02T22:11:00+01:00 GitHub noreply@github.com 2025-03-02T22:11:00+01:00 ggml-backend : keep paths in native string type when possible (#12144) 14dec0c2f29ae56917907dbf2eed6b19438d0a0e 1782cdfed60952f9ff333fc2ab5245f2be702453 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-02T14:53:48+01:00 GitHub noreply@github.com 2025-03-02T14:53:48+01:00 main: use jinja chat template system prompt by default (#12118) 1782cdfed60952f9ff333fc2ab5245f2be702453 45a8e7674548fca9b4ff6d8de97b73bf60f83d72 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-01T15:22:27+01:00 GitHub noreply@github.com 2025-03-01T15:22:27+01:00 main: update outdated system prompt message (followup to #12131) (#12132) 45a8e7674548fca9b4ff6d8de97b73bf60f83d72 80c41ddd8f11b8094018296d9820c2b6c119ac12 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-03-01T13:56:45+01:00 GitHub noreply@github.com 2025-03-01T13:56:45+01:00 common : add --system-prompt parameter, replace behavior of -p in conversation mode (#12131) 80c41ddd8f11b8094018296d9820c2b6c119ac12 2cc4a5e44a3a9ab94426816770d611b33fae8f77 Erik Scholz Green-Sky@users.noreply.github.com 2025-03-01T12:57:22+01:00 GitHub noreply@github.com 2025-03-01T12:57:22+01:00 CUDA: compress mode option and default to size (#12029) 2cc4a5e44a3a9ab94426816770d611b33fae8f77 06c2b1561d8b882bc018554591f8c35eb04ad30e Vivian vynride@gmail.com 2025-03-01T15:45:09+05:30 GitHub noreply@github.com 2025-03-01T11:15:09+01:00 webui : minor typo fixes (#12116) 06c2b1561d8b882bc018554591f8c35eb04ad30e 70680c48e5f77d2d3138712a6582bd8c1e548922 Xuan-Son Nguyen thichthat@gmail.com 2025-02-28T17:44:46+01:00 GitHub noreply@github.com 2025-02-28T17:44:46+01:00 convert : fix Norway problem when parsing YAML (#12114) 70680c48e5f77d2d3138712a6582bd8c1e548922 c43a3e7996e585e2addde1e44057a4f3cdbadef8 William Tambellini wtambellini@sdl.com 2025-02-28T05:41:47-08:00 GitHub noreply@github.com 2025-02-28T14:41:47+01:00 ggml : upgrade init_tensor API to return a ggml_status (#11854) c43a3e7996e585e2addde1e44057a4f3cdbadef8 84d5f4bc195b9540fcb902d869015fba7ef6baa4 Xuan-Son Nguyen thichthat@gmail.com 2025-02-28T12:44:11+01:00 GitHub noreply@github.com 2025-02-28T12:44:11+01:00 llama : add Phi-4-mini support (supersede #12099) (#12108) 84d5f4bc195b9540fcb902d869015fba7ef6baa4 438a83926afcff3643ffef5543db67545ceffe39 Alex Brooks alex.brooks@ibm.com 2025-02-28T04:31:47-07:00 GitHub noreply@github.com 2025-02-28T11:31:47Z Update granite vision docs for 3.2 model (#12105) 438a83926afcff3643ffef5543db67545ceffe39 9c42b1718ca8299f9afeabdc122badeab64c9690 Rémy O remyoudompheng@gmail.com 2025-02-28T09:42:52+01:00 GitHub noreply@github.com 2025-02-28T09:42:52+01:00 vulkan: add specific MMV kernels for IQ2 and IQ3 quants + optimizations (#11595) 9c42b1718ca8299f9afeabdc122badeab64c9690 05e6f5aad0a4bb48fb2775f2a78505540fdbc47d Johannes Gäßler johannesg@5d6.de 2025-02-28T09:26:43+01:00 GitHub noreply@github.com 2025-02-28T09:26:43+01:00 CUDA: fix logic for V100 + GGML_CUDA_FORCE_MMQ (#12098) 05e6f5aad0a4bb48fb2775f2a78505540fdbc47d 673cfef9aa8daad09966208909f346eb996628a4 Prashant Vithule 119530321+Vithulep@users.noreply.github.com 2025-02-28T13:06:12+05:30 GitHub noreply@github.com 2025-02-28T09:36:12+02:00 ggml: aarch64: implement SVE kernels for q2_k_q8_k vector dot (#12064) 673cfef9aa8daad09966208909f346eb996628a4 fbeda9002d4b8b79a4f9288a7ff0d34ef4fb23d5 hipudding huafengchun@gmail.com 2025-02-28T15:23:47+08:00 GitHub noreply@github.com 2025-02-28T15:23:47+08:00 CANN: Fix build error with GCC 13 (#11990) fbeda9002d4b8b79a4f9288a7ff0d34ef4fb23d5 581650b7cacec2872982fde381bd3bcda0f78699 Eve 139727413+netrunnereve@users.noreply.github.com 2025-02-28T07:20:08Z GitHub noreply@github.com 2025-02-28T08:20:08+01:00 vulkan: matmul dequantization improvements (#12015) 581650b7cacec2872982fde381bd3bcda0f78699 b95c8af37ccf169b0a3216b7ed691af0534e5091 Daniele 57776841+daniandtheweb@users.noreply.github.com 2025-02-28T06:52:51Z GitHub noreply@github.com 2025-02-28T07:52:51+01:00 vulkan: improve im2col (#11826) b95c8af37ccf169b0a3216b7ed691af0534e5091 a800ae46da2ed7dac236aa6bf2b595da6b6294b5 Vladimir Vuksanovic 109677816+vvuksanovic@users.noreply.github.com 2025-02-27T08:42:48+01:00 GitHub noreply@github.com 2025-02-27T09:42:48+02:00 cmake: Fix ggml backend dependencies and installation (#11818) a800ae46da2ed7dac236aa6bf2b595da6b6294b5 69050a11be0ae3e01329f11371ecb6850bdaded5 Ting Lou ting.lou@gmail.com 2025-02-26T22:26:52+08:00 GitHub noreply@github.com 2025-02-26T15:26:52+01:00 llava : add struct for FFI bindgen (#12079) 69050a11be0ae3e01329f11371ecb6850bdaded5 3567ee3a94d57ba72b6d023ca507d11e75767edb Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2025-02-26T14:04:48+01:00 GitHub noreply@github.com 2025-02-26T08:04:48-05:00 Refactor gguf scripts to improve metadata handling (#11909) 3567ee3a94d57ba72b6d023ca507d11e75767edb 53e4db10126e277486eccb94c6728f4146c75741 Aleksei Nikiforov 103434461+AlekseiNikiforovIBM@users.noreply.github.com 2025-02-26T12:39:27+01:00 GitHub noreply@github.com 2025-02-26T11:39:27Z gguf-py: enable reading non-native endian files (#12081) 53e4db10126e277486eccb94c6728f4146c75741 d7cfe1ffe0f435d0048a6058d529daf76e072d9c Kante Yin kerthcet@gmail.com 2025-02-26T15:49:36+08:00 GitHub noreply@github.com 2025-02-26T09:49:36+02:00 readme : update infra list (#9096) d7cfe1ffe0f435d0048a6058d529daf76e072d9c a82c9e7c23ef6db48cebfa194dc9cebbc4ac3552 Olivier Chafik ochafik@users.noreply.github.com 2025-02-25T18:52:56Z GitHub noreply@github.com 2025-02-25T18:52:56Z docs: add docs/function-calling.md to lighten server/README.md's plight (#12069) a82c9e7c23ef6db48cebfa194dc9cebbc4ac3552 401af80b546005a06854827b732e3b46979ae028 Jeff Bolz jbolz@nvidia.com 2025-02-25T09:30:21-06:00 GitHub noreply@github.com 2025-02-25T16:30:21+01:00 vulkan: fix assertion when qy_needs_dequant (#12068) 401af80b546005a06854827b732e3b46979ae028 c132239bfbc1aae3a96dfee3350afcb491f64ade rhjdvsgsgks 26178113+rhjdvsgsgks@users.noreply.github.com 2025-02-25T11:52:52Z GitHub noreply@github.com 2025-02-25T12:52:52+01:00 server: handle echo=false on /v1/completions (#12060) c132239bfbc1aae3a96dfee3350afcb491f64ade 393fca629e6ec391b76edf778cb3f7a8a3bc56f9 Judd foldl@users.noreply.github.com 2025-02-25T19:32:20+08:00 GitHub noreply@github.com 2025-02-25T12:32:20+01:00 add OP sigmoid (#12056) 393fca629e6ec391b76edf778cb3f7a8a3bc56f9 61d4f39dfeaf3bbcf4e1535ac03953a5d766680b Molly Sophia mollysophia379@gmail.com 2025-02-25T19:28:22+08:00 GitHub noreply@github.com 2025-02-25T19:28:22+08:00 ggml-cpu: Fix build with sve (#12059) 61d4f39dfeaf3bbcf4e1535ac03953a5d766680b 0b52745649062c04b546aab662cfdb220f4f0621 Rémy O remyoudompheng@gmail.com 2025-02-25T12:04:45+01:00 GitHub noreply@github.com 2025-02-25T12:04:45+01:00 vulkan: implement more backpropagation operators (#11914) 0b52745649062c04b546aab662cfdb220f4f0621 4d1051a40ffc2fdff80bc532eea5b9568b85c156 Olivier Chafik ochafik@users.noreply.github.com 2025-02-25T10:40:22Z GitHub noreply@github.com 2025-02-25T10:40:22Z server: support add_generation_prompt query param (#12062) 4d1051a40ffc2fdff80bc532eea5b9568b85c156 3e9a2860e996657fc10db8393cf65adc40703082 Alex Brooks alex.brooks@ibm.com 2025-02-25T02:46:05-07:00 GitHub noreply@github.com 2025-02-25T10:46:05+01:00 Add Doc for Converting Granite Vision -> GGUF (#12006) 3e9a2860e996657fc10db8393cf65adc40703082 58d07a8043a1395177cf77b3e4f388e34182ae64 Vitali Lovich vlovich+github@gmail.com 2025-02-25T01:29:33-08:00 GitHub noreply@github.com 2025-02-25T11:29:33+02:00 llama : expose llama_model_n_head_kv in the API (#11997) 58d07a8043a1395177cf77b3e4f388e34182ae64 34a846b5847a18d133b360074f1fb485b2632b2d Gian-Carlo Pascutto gcp@sjeng.org 2025-02-25T10:27:58+01:00 GitHub noreply@github.com 2025-02-25T11:27:58+02:00 metal : copy kernels for quant to F32/F16 conversions (#12017) 34a846b5847a18d133b360074f1fb485b2632b2d 7a2c913e66353362d7f28d612fd3c9d51a831eda lhez quic_lih@quicinc.com 2025-02-24T13:47:07-08:00 GitHub noreply@github.com 2025-02-24T14:47:07-07:00 opencl: fix for small models (#11950) 7a2c913e66353362d7f28d612fd3c9d51a831eda 08d5986290cc42d2c52739e046642b8252f97e4b Alex Brooks alex.brooks@ibm.com 2025-02-24T09:09:51-07:00 GitHub noreply@github.com 2025-02-24T17:09:51+01:00 llava : Add Granite Vision Support (#11794) 08d5986290cc42d2c52739e046642b8252f97e4b 651adf4b6675339465179b81b194d98cc14704d6 Neo Zhang Jianyu jianyu.zhang@intel.com 2025-02-24T22:33:23+08:00 GitHub noreply@github.com 2025-02-24T22:33:23+08:00 [SYCL] Optimize mul_mat for Q4_0 on Intel GPU (#12035) 651adf4b6675339465179b81b194d98cc14704d6 8303e8b0fb2c1e26a8edd58071f9120a5bd6930a Aleksei Nikiforov 103434461+AlekseiNikiforovIBM@users.noreply.github.com 2025-02-24T12:27:01+01:00 GitHub noreply@github.com 2025-02-24T11:27:01Z gguf_convert_endian.py: implement byteswapping for q4_k and q6_k (#11349) 8303e8b0fb2c1e26a8edd58071f9120a5bd6930a 7ad0779f5de84a68143b2c00ab5dc94a948925d3 Akarshan Biswas akarshan.biswas@gmail.com 2025-02-24T15:48:25+05:30 GitHub noreply@github.com 2025-02-24T10:18:25Z SYCL: Fix GGML_SYCL_DEBUG macro (#11995) 7ad0779f5de84a68143b2c00ab5dc94a948925d3 f777a73e18c218848bca0748581c043987348a5d Florent BENOIT fbenoit@redhat.com 2025-02-23T18:15:51+01:00 GitHub noreply@github.com 2025-02-23T17:15:51Z run: allow to customize prompt by env var LLAMA_PROMPT_PREFIX (#12041) f777a73e18c218848bca0748581c043987348a5d af7747c95ae71a5db4184947f837179e82c70b77 Eric Curtin ecurtin@redhat.com 2025-02-23T13:14:32Z GitHub noreply@github.com 2025-02-23T13:14:32Z Some llama-run cleanups (#11973) af7747c95ae71a5db4184947f837179e82c70b77 a28e0d5eb18c18e6a4598286158f427269b1444e Aaron Teo 57927438+taronaeo@users.noreply.github.com 2025-02-23T05:39:24+08:00 GitHub noreply@github.com 2025-02-22T21:39:24Z ggml-cpu: Support s390x SIMD Instruction Set (#12019) a28e0d5eb18c18e6a4598286158f427269b1444e 36c258ee921dbb5c96bdc57c0872e4a9a129bef6 Johannes Gäßler johannesg@5d6.de 2025-02-22T20:44:34+01:00 GitHub noreply@github.com 2025-02-22T20:44:34+01:00 CUDA: app option to compile without FlashAttention (#12025) 36c258ee921dbb5c96bdc57c0872e4a9a129bef6 f3e64859edb0d55d4223ead78672597cd1a218df Ting Lou ting.lou@gmail.com 2025-02-22T22:28:28+08:00 GitHub noreply@github.com 2025-02-22T15:28:28+01:00 llava: build clip image from pixels (#11999) f3e64859edb0d55d4223ead78672597cd1a218df 5fa07c2f93c73161bf09ef0b23b5d2686f9a073e Georgi Gerganov ggerganov@gmail.com 2025-02-22T15:03:00+02:00 GitHub noreply@github.com 2025-02-22T15:03:00+02:00 ci : fix arm upload artifacts (#12024) 5fa07c2f93c73161bf09ef0b23b5d2686f9a073e 335eb04a91f481f37c0c9b302ee31b449b04c3e9 Johannes Gäßler johannesg@5d6.de 2025-02-22T12:20:17+01:00 GitHub noreply@github.com 2025-02-22T12:20:17+01:00 CUDA: optimize FA for GQA + large batches (#12014) 335eb04a91f481f37c0c9b302ee31b449b04c3e9 cf756d6e0a314e0fe25ff944341d79ea8c94cc96 Rohanjames1997 rohan.james4@gmail.com 2025-02-22T04:48:57-06:00 GitHub noreply@github.com 2025-02-22T11:48:57+01:00 ci : Build on Github-hosted arm64 runners (#12009) cf756d6e0a314e0fe25ff944341d79ea8c94cc96 d70908421ff22b013e8209f2d12e5c750663c620 Georgi Gerganov ggerganov@gmail.com 2025-02-22T12:46:31+02:00 GitHub noreply@github.com 2025-02-22T11:46:31+01:00 server : disable Nagle's algorithm (#12020) d70908421ff22b013e8209f2d12e5c750663c620 de8b5a3624499bdb9fa6e99840259998638f093f Gian-Carlo Pascutto gcp@sjeng.org 2025-02-22T09:43:24+01:00 GitHub noreply@github.com 2025-02-22T09:43:24+01:00 cuda: Add Q5_1, Q5_0, Q4_1 and Q4_0 to F32 conversion support. (#12000) de8b5a3624499bdb9fa6e99840259998638f093f 51f311e057723b7454d0ebe20f545a1a2c4db6b2 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-22T06:33:29+01:00 GitHub noreply@github.com 2025-02-22T06:33:29+01:00 llama.swiftui : add "Done" dismiss button to help view (#11998) 51f311e057723b7454d0ebe20f545a1a2c4db6b2 586d5fe6ebb8f92e9dd53420e04cec2697046073 Georgi Gerganov ggerganov@gmail.com 2025-02-21T18:33:18+02:00 GitHub noreply@github.com 2025-02-21T18:33:18+02:00 llama : skip loading unused tensors (#12004) 586d5fe6ebb8f92e9dd53420e04cec2697046073 ecc8e3aeff269037aa786dfb393e8b531f96832f Johannes Gäßler johannesg@5d6.de 2025-02-21T12:51:25+01:00 GitHub noreply@github.com 2025-02-21T12:51:25+01:00 doc: update contributing guidelines [no ci] (#11969) ecc8e3aeff269037aa786dfb393e8b531f96832f 0b3863ff9576872855b0265da2cab2ce7e25c4d9 PureJourney edward.pong@qq.com 2025-02-21T19:21:05+08:00 GitHub noreply@github.com 2025-02-21T12:21:05+01:00 CUDA: correct the lowest Maxwell supported by CUDA 12 (#11984) 0b3863ff9576872855b0265da2cab2ce7e25c4d9 ee02ad02c56ff36a5edd22d8617ab3f9546ce7fe Bodhi 3882561+BodhiHu@users.noreply.github.com 2025-02-21T15:46:23+08:00 GitHub noreply@github.com 2025-02-21T09:46:23+02:00 MUSA: support ARM64 and enable dp4a .etc (#11843) ee02ad02c56ff36a5edd22d8617ab3f9546ce7fe c392e5094deaf2d1a7c18683214f007fad3fe42b Alex Brooks alex.brooks@ibm.com 2025-02-20T23:11:03-07:00 GitHub noreply@github.com 2025-02-21T08:11:03+02:00 clip : fix visual encoders with no CLS (#11982) c392e5094deaf2d1a7c18683214f007fad3fe42b c5d91a74006c49376590ef2b67420bc7ce206397 momonga 146910567+mmngays@users.noreply.github.com 2025-02-21T03:43:22+09:00 GitHub noreply@github.com 2025-02-20T19:43:22+01:00 server (webui): Fix Premature Submission During IME Conversion (#11971) c5d91a74006c49376590ef2b67420bc7ce206397 4806498bf15ef767de3776b00856e56c373dd1b1 Charles Xu charles.xu@arm.com 2025-02-20T14:06:51+01:00 GitHub noreply@github.com 2025-02-20T15:06:51+02:00 ggml-cpu: Add CPU backend support for KleidiAI library (#11390) 4806498bf15ef767de3776b00856e56c373dd1b1 0d559580a0a74c842c3a876035ba96a338aabfb2 Prashant Vithule 119530321+Vithulep@users.noreply.github.com 2025-02-20T15:38:32+05:30 GitHub noreply@github.com 2025-02-20T12:08:32+02:00 ggml: aarch64: implement SVE kernels for q3_K_q8_K vector dot (#11917) 0d559580a0a74c842c3a876035ba96a338aabfb2 d04e7163c85a847bc61d58c22f2c503596db7aa8 Michael Engel mengel@redhat.com 2025-02-20T09:35:11+01:00 GitHub noreply@github.com 2025-02-20T10:35:11+02:00 run : add --chat-template-file (#11961) d04e7163c85a847bc61d58c22f2c503596db7aa8 d07c621393fab6cf32f3add2aa65e4d5331d4a67 Johannes Gäßler johannesg@5d6.de 2025-02-19T20:45:17+01:00 GitHub noreply@github.com 2025-02-19T20:45:17+01:00 doc: add links to ggml examples [no ci] (#11958) d07c621393fab6cf32f3add2aa65e4d5331d4a67 abd4d0bc4f1a9a0e429bc8ee0d5ece2a394a0a39 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-19T12:29:52+01:00 GitHub noreply@github.com 2025-02-19T12:29:52+01:00 common : add llama.vim preset for Qwen2.5 Coder (#11945) abd4d0bc4f1a9a0e429bc8ee0d5ece2a394a0a39 9626d9351a6dfb665400d9fccbda876a0a96ef67 Georgi Gerganov ggerganov@gmail.com 2025-02-19T13:29:42+02:00 GitHub noreply@github.com 2025-02-19T13:29:42+02:00 speculative : update default params (#11954) 9626d9351a6dfb665400d9fccbda876a0a96ef67 b58934c1836b5ea51dbacbe899eee125775e77c9 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-19T06:16:23+01:00 GitHub noreply@github.com 2025-02-19T06:16:23+01:00 llama : fix indentation in llama-grammar [no ci] (#11943) b58934c1836b5ea51dbacbe899eee125775e77c9 63e489c025d61c7ca5ec06c5d10f36e2b76aaa1d igardev 49397134+igardev@users.noreply.github.com 2025-02-19T00:01:44+02:00 GitHub noreply@github.com 2025-02-18T23:01:44+01:00 server : (webui) Enable communication with parent html (if webui is in iframe) (#11940) 63e489c025d61c7ca5ec06c5d10f36e2b76aaa1d 63ac12856303108ee46635e6c9e751f81415ee64 Olivier Chafik ochafik@users.noreply.github.com 2025-02-18T18:03:23Z GitHub noreply@github.com 2025-02-18T18:03:23Z tool-call: refactor common chat / tool-call api (+ tests / fixes) (#11900) 63ac12856303108ee46635e6c9e751f81415ee64 5137da7b8c3eaa090476a632888ca178ba109f8a Xuan-Son Nguyen thichthat@gmail.com 2025-02-18T14:21:41+01:00 GitHub noreply@github.com 2025-02-18T14:21:41+01:00 server : add TEI API format for /rerank endpoint (#11942) 5137da7b8c3eaa090476a632888ca178ba109f8a 09aaf4f1f5b69b5173b7fcd24eab96729f6b242a MoonRide303 130458190+MoonRide303@users.noreply.github.com 2025-02-18T10:30:16+01:00 GitHub noreply@github.com 2025-02-18T10:30:16+01:00 scripts: corrected encoding when getting chat template (#11866) (#11907) 09aaf4f1f5b69b5173b7fcd24eab96729f6b242a 73e2ed3ce3492d3ed70193dd09ae8aa44779651d xiaobing318 71554036+xiaobing318@users.noreply.github.com 2025-02-18T17:12:49+08:00 GitHub noreply@github.com 2025-02-18T10:12:49+01:00 docs : Fix duplicated file extension in test command (#11935) 73e2ed3ce3492d3ed70193dd09ae8aa44779651d f7b1116af102bcac450c1a522e9c59db241c6767 Johannes Gäßler johannesg@5d6.de 2025-02-17T14:03:24+01:00 GitHub noreply@github.com 2025-02-17T14:03:24+01:00 CUDA: use async data loading for FlashAttention (#11894) f7b1116af102bcac450c1a522e9c59db241c6767 c4d29baf3236b011730b6ccaae6852e781fb1b91 Eve 139727413+netrunnereve@users.noreply.github.com 2025-02-17T11:20:23Z GitHub noreply@github.com 2025-02-17T12:20:23+01:00 update release requirements (#11897) c4d29baf3236b011730b6ccaae6852e781fb1b91 2eea03d86a2d132c8245468c26290ce07a27a8e8 Antoine Viallon antoine@lesviallon.fr 2025-02-17T11:25:12+01:00 GitHub noreply@github.com 2025-02-17T11:25:12+01:00 server : fix divide-by-zero in metrics reporting (#11915) 2eea03d86a2d132c8245468c26290ce07a27a8e8 0f2bbe656473177538956d22b6842bcaa0449fab Rémy O remyoudompheng@gmail.com 2025-02-17T07:55:57+01:00 GitHub noreply@github.com 2025-02-17T07:55:57+01:00 vulkan: implement several ops relevant for ggml_opt (#11769) 0f2bbe656473177538956d22b6842bcaa0449fab fe163d5bf3fb8e99c96ff6d21775fa3d1daee3cf Xuan-Son Nguyen thichthat@gmail.com 2025-02-16T18:11:22+01:00 GitHub noreply@github.com 2025-02-16T17:11:22Z server : bump httplib to 0.19.0 (#11908) fe163d5bf3fb8e99c96ff6d21775fa3d1daee3cf 818a340ea8be55b3706e1772527cb8738e90a8c7 standby24x7 standby24x7@gmail.com 2025-02-16T18:51:13+09:00 GitHub noreply@github.com 2025-02-16T10:51:13+01:00 common : Fix a typo in help (#11899) 818a340ea8be55b3706e1772527cb8738e90a8c7 bf42a23d0a515a508aecf10fde1d52c5ed5104c6 Xuan-Son Nguyen thichthat@gmail.com 2025-02-16T10:36:39+01:00 GitHub noreply@github.com 2025-02-16T10:36:39+01:00 ci : fix (again) arm64 build fails (#11895) bf42a23d0a515a508aecf10fde1d52c5ed5104c6 c2ea16f260445d1f2bd9fee78fb8acb2b3aaaad4 Jeff Bolz jbolz@nvidia.com 2025-02-16T01:52:23-06:00 GitHub noreply@github.com 2025-02-16T08:52:23+01:00 vulkan: support multi/vision rope, and noncontiguous rope (#11902) c2ea16f260445d1f2bd9fee78fb8acb2b3aaaad4 6dde1782483d6b0a1d59f5a5fbcb3119b9d34c27 Hale Chan halechan@qq.com 2025-02-16T14:50:26+08:00 GitHub noreply@github.com 2025-02-16T08:50:26+02:00 metal : fix the crash caused by the lack of residency set support on Intel Macs. (#11904) 6dde1782483d6b0a1d59f5a5fbcb3119b9d34c27 fc10c38ded84670955d4c44770f8acfb64617e15 Johannes Gäßler johannesg@5d6.de 2025-02-15T20:23:22+01:00 GitHub noreply@github.com 2025-02-15T20:23:22+01:00 scripts: fix compare-llama-bench commit hash logic (#11891) fc10c38ded84670955d4c44770f8acfb64617e15 22885105a6b034abaf1c1471ad90fb2ae96146bb 708-145 40387547+708-145@users.noreply.github.com 2025-02-15T20:03:30+01:00 GitHub noreply@github.com 2025-02-15T21:03:30+02:00 examples: fix typo in imatrix/README.md (#11884) 22885105a6b034abaf1c1471ad90fb2ae96146bb c2cd24fbfdfeacc2fc6ad03878379de104264114 Adrian Kretz me@akretz.com 2025-02-15T19:39:20+01:00 GitHub noreply@github.com 2025-02-15T20:39:20+02:00 metal : optimize dequant q6_K kernel (#11892) c2cd24fbfdfeacc2fc6ad03878379de104264114 68ff663a04ed92044a9937bcae353e9d9733f9cd Georgi Gerganov ggerganov@gmail.com 2025-02-15T20:29:56+02:00 GitHub noreply@github.com 2025-02-15T20:29:56+02:00 readme : add notice about new package registry (#11890) 68ff663a04ed92044a9937bcae353e9d9733f9cd f3552296924e704c11ca936907b9cad7873db8e2 Georgi Gerganov ggerganov@gmail.com 2025-02-15T16:40:57+02:00 GitHub noreply@github.com 2025-02-15T16:40:57+02:00 repo : update links to new url (#11886) f3552296924e704c11ca936907b9cad7873db8e2 fc1b0d0936e4dfc52a81f38e7420c7d23f6caa88 Olivier Chafik ochafik@users.noreply.github.com 2025-02-15T10:11:36Z GitHub noreply@github.com 2025-02-15T10:11:36Z server: fix type promotion typo causing crashes w/ --jinja w/o tools (#11880) fc1b0d0936e4dfc52a81f38e7420c7d23f6caa88 89daa2564f6eab33be53c6a1b39273af536d6bb3 Rémy O remyoudompheng@gmail.com 2025-02-15T09:01:40+01:00 GitHub noreply@github.com 2025-02-15T09:01:40+01:00 vulkan: initial support for IQ1_S and IQ1_M quantizations (#11528) 89daa2564f6eab33be53c6a1b39273af536d6bb3 300907b2110cc17b4337334dc397e05de2d8f5e0 Michał Moskal michal@moskal.me 2025-02-14T12:46:08-08:00 GitHub noreply@github.com 2025-02-14T12:46:08-08:00 llguidance build fixes for Windows (#11664) 300907b2110cc17b4337334dc397e05de2d8f5e0 94b87f87b502d2ab6c8b28d2b5935cf008ca1505 lhez quic_lih@quicinc.com 2025-02-14T11:12:23-08:00 GitHub noreply@github.com 2025-02-14T12:12:23-07:00 opencl: Fix rope and softmax (#11833) 94b87f87b502d2ab6c8b28d2b5935cf008ca1505 dbc2ec59b5603fbd25f3833b2407b4f3612b9f02 Diego Devesa slarengh@gmail.com 2025-02-14T15:33:52+01:00 GitHub noreply@github.com 2025-02-14T15:33:52+01:00 cuda : add ampere to the list of default architectures (#11870) dbc2ec59b5603fbd25f3833b2407b4f3612b9f02 3d68f034dad53f0f27ad626b2732ef48fbcea4ee Georgi Gerganov ggerganov@gmail.com 2025-02-14T14:48:40+02:00 GitHub noreply@github.com 2025-02-14T14:48:40+02:00 docker : drop to CUDA 12.4 (#11869) 3d68f034dad53f0f27ad626b2732ef48fbcea4ee 38e32eb6a0cec32130b699ce9fefad6c74571953 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-14T11:16:56+01:00 GitHub noreply@github.com 2025-02-14T11:16:56+01:00 llama : add completion for --chat-template-file (#11860) 38e32eb6a0cec32130b699ce9fefad6c74571953 a4f011e8d02179f032627130f961eb77ee30401c Jinyang He hejinyang@loongson.cn 2025-02-14T16:54:27+08:00 GitHub noreply@github.com 2025-02-14T10:54:27+02:00 ggml: optimize some vec dot functions for LoongArch ASX (#11842) a4f011e8d02179f032627130f961eb77ee30401c a7b8ce226071b2b0faaad0d36cc5ebd7fb074730 Eve 139727413+netrunnereve@users.noreply.github.com 2025-02-14T02:59:40Z GitHub noreply@github.com 2025-02-14T02:59:40Z vulkan: linux builds + small subgroup size fixes (#11767) a7b8ce226071b2b0faaad0d36cc5ebd7fb074730 04045bb84288dc7e9e424d4e2bf7f40d259b4c6a theraininsky 76763719+theraininsky@users.noreply.github.com 2025-02-14T09:13:43+08:00 GitHub noreply@github.com 2025-02-14T02:13:43+01:00 llama-bench : fix unexpected global variable initialize sequence issue (#11832) 04045bb84288dc7e9e424d4e2bf7f40d259b4c6a 8a8c4ceb6050bd9392609114ca56ae6d26f5b8f5 Georgi Gerganov ggerganov@gmail.com 2025-02-14T00:16:56+02:00 GitHub noreply@github.com 2025-02-14T00:16:56+02:00 readme : minor 8a8c4ceb6050bd9392609114ca56ae6d26f5b8f5 c1f958c0381e797135b7d42a677542133264193c Jeffrey Morgan jmorganca@gmail.com 2025-02-13T09:05:04-08:00 GitHub noreply@github.com 2025-02-13T18:05:04+01:00 llamafile: use member variable instead of constant for iq4nlt (#11780) c1f958c0381e797135b7d42a677542133264193c c48f630d1c1942fce08aa7cb18a53ace443cd611 Reza Rahemtola 49811529+RezaRahemtola@users.noreply.github.com 2025-02-13T17:22:44+01:00 GitHub noreply@github.com 2025-02-13T17:22:44+01:00 server : (docs) Update wrong tool calling example (#11809) c48f630d1c1942fce08aa7cb18a53ace443cd611 bd6e55bfd3f3af3e5705cf87a10e5178cef7c3c1 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-13T14:46:59+01:00 GitHub noreply@github.com 2025-02-13T14:46:59+01:00 llama : add --completion-bash option (#11846) bd6e55bfd3f3af3e5705cf87a10e5178cef7c3c1 c7f460ab882065ec5696e3d51e24dbf67b539287 R0CKSTAR xiaodong.ye@mthreads.com 2025-02-13T20:28:18+08:00 GitHub noreply@github.com 2025-02-13T13:28:18+01:00 musa: bump MUSA SDK version to rc3.1.1 (#11822) c7f460ab882065ec5696e3d51e24dbf67b539287 27e8a23300e30cd6ff6107ce262acf832ca60597 Olivier Chafik ochafik@users.noreply.github.com 2025-02-13T10:05:16Z GitHub noreply@github.com 2025-02-13T10:05:16Z `server`: fix tool-call of DeepSeek R1 Qwen, return reasoning_content (Command 7RB & DeepSeek R1) unless `--reasoning-format none` (#11607) 27e8a23300e30cd6ff6107ce262acf832ca60597 e4376270d971cff7992bdb6c5412a739195b1459 Vinesh Janarthanan 36610342+VJHack@users.noreply.github.com 2025-02-13T00:45:57-06:00 GitHub noreply@github.com 2025-02-13T08:45:57+02:00 sampling: add Top-nσ sampler (#11223) e4376270d971cff7992bdb6c5412a739195b1459 3e693197724c31d53a9b69018c2f1bd0b93ebab2 Oleksandr Kuvshynov 661042+okuvshynov@users.noreply.github.com 2025-02-13T01:25:34-05:00 GitHub noreply@github.com 2025-02-13T08:25:34+02:00 llama.cpp: fix warning message (#11839) 3e693197724c31d53a9b69018c2f1bd0b93ebab2 a394039db004c6ee00098250d160b5aa018c2314 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-13T07:07:51+01:00 GitHub noreply@github.com 2025-02-13T08:07:51+02:00 llama : update llama_decode_internal ref [no ci] (#11840) a394039db004c6ee00098250d160b5aa018c2314 be3bbd62153820ff6d358c817360927f429105c4 Diego Devesa slarengh@gmail.com 2025-02-13T01:02:38+01:00 GitHub noreply@github.com 2025-02-13T01:02:38+01:00 ggml-cpu : add chunking support to mul_mat_id (#11666) be3bbd62153820ff6d358c817360927f429105c4 31afcbee0eebbc998ab311aa314e389d60aa2127 Xuan-Son Nguyen thichthat@gmail.com 2025-02-13T00:33:45+01:00 GitHub noreply@github.com 2025-02-13T00:33:45+01:00 ggml : x2 speed for WASM by optimizing SIMD (#11453) 31afcbee0eebbc998ab311aa314e389d60aa2127 5c4284d57bbc613121e90de5271f1cbc95d55872 Woof Dog 197125663+woof-dog@users.noreply.github.com 2025-02-12T22:47:11Z GitHub noreply@github.com 2025-02-12T23:47:11+01:00 server : (webui) Give copy button back to all message bubbles (#11814) 5c4284d57bbc613121e90de5271f1cbc95d55872 bfd11a2344ce6005bfbd5432a06fc7325bc0ecae uvos philipp@uvos.xyz 2025-02-12T22:25:28+01:00 GitHub noreply@github.com 2025-02-12T22:25:28+01:00 HIP: Remove GCN from list of devices that avoid MMQ (#11831) bfd11a2344ce6005bfbd5432a06fc7325bc0ecae 0fb77f821f6e70ad8b8247a97d1022f0fef78991 JC 43374599+MrSMlT@users.noreply.github.com 2025-02-12T20:36:11Z GitHub noreply@github.com 2025-02-12T21:36:11+01:00 Fix: Compile failure due to Microsoft STL breaking change (#11836) 0fb77f821f6e70ad8b8247a97d1022f0fef78991 e598697d63d062b4af939f1b0383d6adc6292d1a Georgi Gerganov ggerganov@gmail.com 2025-02-12T21:46:02+02:00 Georgi Gerganov ggerganov@gmail.com 2025-02-12T21:46:02+02:00 sync : ggml e598697d63d062b4af939f1b0383d6adc6292d1a fef0cbeadf5c8e221f832158cb4006ade39ef786 uvos philipp@uvos.xyz 2025-02-12T17:25:03+01:00 GitHub noreply@github.com 2025-02-12T17:25:03+01:00 HIP: Switch to std::vector in rocblas version check (#11820) fef0cbeadf5c8e221f832158cb4006ade39ef786 748ee9fe9312acb8755ee4bf46fd9de2e6a45f29 bandoti 141645996+bandoti@users.noreply.github.com 2025-02-12T10:06:53-04:00 GitHub noreply@github.com 2025-02-12T10:06:53-04:00 cleanup: fix compile warnings associated with gnu_printf (#11811) 748ee9fe9312acb8755ee4bf46fd9de2e6a45f29 198b1ec611a2c551ea40e5b9c0b862f37555a4cc Richard r-burton@hotmail.co.uk 2025-02-12T13:57:33Z GitHub noreply@github.com 2025-02-12T15:57:33+02:00 ggml : fix multi-threaded clamp_f32 (#11824) 198b1ec611a2c551ea40e5b9c0b862f37555a4cc c3d6af7cd2d79dc89da086b2d08c777d0319d829 Weizhao Ouyang o451686892@gmail.com 2025-02-12T20:22:58+08:00 GitHub noreply@github.com 2025-02-12T13:22:58+01:00 ggml-cpu: Fix duplicate MATMUL_INT8 (#11817) c3d6af7cd2d79dc89da086b2d08c777d0319d829 369be5598ac5f71f6aa6d6606e9aec769a23dafa Johannes Gäßler johannesg@5d6.de 2025-02-12T13:16:39+01:00 GitHub noreply@github.com 2025-02-12T13:16:39+01:00 CUDA: fix CUDART_VERSION checks (#11821) 369be5598ac5f71f6aa6d6606e9aec769a23dafa 4078c77f9891831f29ffc7c315c8ec6695ba5ce7 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-12T08:40:01+01:00 GitHub noreply@github.com 2025-02-12T09:40:01+02:00 llama : fix typo in llama-grammar.h [no ci] (#11816) 4078c77f9891831f29ffc7c315c8ec6695ba5ce7 90e4dba461b07e635fd1daf3b491c978c7dd0013 lhez quic_lih@quicinc.com 2025-02-11T14:04:13-08:00 GitHub noreply@github.com 2025-02-11T15:04:13-07:00 docs: add OpenCL (#11697) 90e4dba461b07e635fd1daf3b491c978c7dd0013 a18f481f99962638092d6f1c98b1d34d3e3256de Sheldon Robinson sheldon.robinson@live.com 2025-02-11T10:55:45-05:00 GitHub noreply@github.com 2025-02-11T16:55:45+01:00 Fix #11802: Compile bug - RegQueryValueExA changed to RegQueryValueEx (#11803) a18f481f99962638092d6f1c98b1d34d3e3256de b9ab0a4d0b2ed19effec130921d05fb5c30b68c5 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-11T14:06:45+01:00 GitHub noreply@github.com 2025-02-11T14:06:45+01:00 server : use common_token_to_piece instead of common_detokenize (#11740) b9ab0a4d0b2ed19effec130921d05fb5c30b68c5 7b891bdc863663b4dc1155aa9429b58c721524b2 Johannes Gäßler johannesg@5d6.de 2025-02-11T00:17:22+01:00 GitHub noreply@github.com 2025-02-11T00:17:22+01:00 CUDA: use arch list for compatibility check (#11775) 7b891bdc863663b4dc1155aa9429b58c721524b2 81732619fd2d570712dc78db5965cee9224b38bd Maxim Evtush 154841002+maximevtush@users.noreply.github.com 2025-02-10T23:21:31+01:00 GitHub noreply@github.com 2025-02-10T23:21:31+01:00 fix: typos in documentation files (#11791) 81732619fd2d570712dc78db5965cee9224b38bd 507f9174fe856772b6c7c17e81be442de7ee6d1e jason_w jason.wang@126.com 2025-02-11T06:17:48+08:00 GitHub noreply@github.com 2025-02-10T23:17:48+01:00 docs: utilize the forward slash (/) as the path separator for Unix-like systems (#11770) 507f9174fe856772b6c7c17e81be442de7ee6d1e 19b392d58dc08c366d0b29bd3b9c6991fa4e1662 Xuan-Son Nguyen thichthat@gmail.com 2025-02-10T21:23:17+01:00 GitHub noreply@github.com 2025-02-10T21:23:17+01:00 server : (webui) introduce conversation branching + idb storage (#11792) 19b392d58dc08c366d0b29bd3b9c6991fa4e1662 0893e0114e934bdd0eba0ff69d9ef8c59343cbc3 Wilken Gottwalt 12194808+wgottwalt@users.noreply.github.com 2025-02-10T19:58:18+01:00 GitHub noreply@github.com 2025-02-10T20:58:18+02:00 llama-mmap: fix missing include (#11796) 0893e0114e934bdd0eba0ff69d9ef8c59343cbc3 d7b31a9d84297b493a61c9a8ee3a458e7ccc64a7 Xuan-Son Nguyen thichthat@gmail.com 2025-02-10T18:03:28+01:00 GitHub noreply@github.com 2025-02-10T18:03:28+01:00 server : correct signal handler (#11795) d7b31a9d84297b493a61c9a8ee3a458e7ccc64a7 9ac3457b39c78c5eb103280e08fb6163642484ff Olivier Chafik ochafik@users.noreply.github.com 2025-02-10T09:34:09Z GitHub noreply@github.com 2025-02-10T09:34:09Z sync: minja (https://github.com/google/minja/commit/a72057e5190de2c612d4598bb10b4bfd0f53011f) (#11774) 9ac3457b39c78c5eb103280e08fb6163642484ff c2a67efe38e6782c0217e1de3edc68de6951612b pascal-lc 49066376+pascal-lc@users.noreply.github.com 2025-02-10T16:05:57+08:00 GitHub noreply@github.com 2025-02-10T09:05:57+01:00 Update README.md [no ci] (#11781) c2a67efe38e6782c0217e1de3edc68de6951612b b044a0fe3ca0cbef9dd041edce3ebda8c501fae4 Danny Milosavljevic dannym@friendly-machines.com 2025-02-10T07:17:21+01:00 GitHub noreply@github.com 2025-02-10T07:17:21+01:00 vulkan: Make Vulkan optional at runtime (#11493). (#11494) b044a0fe3ca0cbef9dd041edce3ebda8c501fae4 19d3c8293b1f61acbe2dab1d49a17950fd788a4a Wagner Bruna wbruna@users.noreply.github.com 2025-02-10T03:08:22-03:00 GitHub noreply@github.com 2025-02-10T07:08:22+01:00 vulkan: add environment variable GGML_VK_PREFER_HOST_MEMORY to avoid VRAM allocation (#11592) 19d3c8293b1f61acbe2dab1d49a17950fd788a4a 98f6b0fd1ea88ce33f4fcd1775d9a463067156ac Eric Curtin ecurtin@redhat.com 2025-02-09T10:34:49Z GitHub noreply@github.com 2025-02-09T10:34:49Z There's a better way of clearing lines (#11756) 98f6b0fd1ea88ce33f4fcd1775d9a463067156ac 55ac8c7791ff44aeb82bd7fe3ca2041844fc77f1 Jeff Bolz jbolz@nvidia.com 2025-02-09T01:43:51-06:00 GitHub noreply@github.com 2025-02-09T08:43:51+01:00 vulkan: account for lookup tables when checking shared memory size (#11502) 55ac8c7791ff44aeb82bd7fe3ca2041844fc77f1 e6e658319952f7ad269dc11275b9edddc721fc6d Xuan-Son Nguyen thichthat@gmail.com 2025-02-08T21:54:50+01:00 GitHub noreply@github.com 2025-02-08T21:54:50+01:00 server : (webui) revamp Settings dialog, add Pyodide interpreter (#11759) e6e658319952f7ad269dc11275b9edddc721fc6d aaa55053076f3d5d7da4d9a877cb77e112dabed4 Woof Dog 197125663+woof-dog@users.noreply.github.com 2025-02-08T19:09:55Z GitHub noreply@github.com 2025-02-08T20:09:55+01:00 server : (webui) increase edit textarea size (#11763) aaa55053076f3d5d7da4d9a877cb77e112dabed4 bdcf8b6a56f4d49d3420ae5b21cdf9a116040551 Georgi Gerganov ggerganov@gmail.com 2025-02-08T18:08:43+02:00 GitHub noreply@github.com 2025-02-08T18:08:43+02:00 server : minor log updates (#11760) bdcf8b6a56f4d49d3420ae5b21cdf9a116040551 4d3465c5aeca8be29cac77f1535c35f4fb274eca Georgi Gerganov ggerganov@gmail.com 2025-02-08T16:49:38+02:00 Georgi Gerganov ggerganov@gmail.com 2025-02-08T16:49:38+02:00 cont : fix mmap flag print (#11699) 4d3465c5aeca8be29cac77f1535c35f4fb274eca d80be897acdca45f8f7ea2e52c930b1d0e738a14 Karol Kontny 82021046+kkontny@users.noreply.github.com 2025-02-08T15:30:53+01:00 GitHub noreply@github.com 2025-02-08T15:30:53+01:00 ggml: Fix data race in ggml threadpool (#11736) d80be897acdca45f8f7ea2e52c930b1d0e738a14 3ab410f55f26038de45d6cc569aaa5cb29acb918 Johannes Gäßler johannesg@5d6.de 2025-02-08T10:46:07+01:00 GitHub noreply@github.com 2025-02-08T10:46:07+01:00 CUDA: fix min. version for movmatrix (#11751) 3ab410f55f26038de45d6cc569aaa5cb29acb918 0cf867160ca9d492e06c0bc688b337cffd1eb187 Nikolaos Pothitos pothitos@di.uoa.gr 2025-02-08T11:43:04+02:00 GitHub noreply@github.com 2025-02-08T10:43:04+01:00 readme : update front-end framework (#11753) 0cf867160ca9d492e06c0bc688b337cffd1eb187 d2fe216fb2fb7ca8627618c9ea3a2e7886325780 Xuan-Son Nguyen thichthat@gmail.com 2025-02-08T10:42:34+01:00 GitHub noreply@github.com 2025-02-08T10:42:34+01:00 server : (webui) fix numeric settings being saved as string (#11739) d2fe216fb2fb7ca8627618c9ea3a2e7886325780 ed926d8833df3c29797edbc98dafd9b575aa0729 Eric Curtin ecurtin@redhat.com 2025-02-07T14:42:46Z GitHub noreply@github.com 2025-02-07T14:42:46Z Make logging more verbose (#11714) ed926d8833df3c29797edbc98dafd9b575aa0729 2d219b389e8c8c40bce547b08c8aa7add60fde1f Georgi Gerganov ggerganov@gmail.com 2025-02-07T16:05:34+02:00 GitHub noreply@github.com 2025-02-07T16:05:34+02:00 llama : fix defrag logic (#11707) 2d219b389e8c8c40bce547b08c8aa7add60fde1f 333820d7491cd31c707a340ff23b984a84e40154 Christian Fillion cfillion@users.noreply.github.com 2025-02-07T08:55:47-05:00 GitHub noreply@github.com 2025-02-07T15:55:47+02:00 vocab : ignore invalid UTF-8 input in the BPE tokenizer (#11729) 333820d7491cd31c707a340ff23b984a84e40154 c026ba3c23765a648ca27c7a15ecf179f8e27f26 magicse magicse@users.noreply.github.com 2025-02-07T15:48:47+02:00 GitHub noreply@github.com 2025-02-07T15:48:47+02:00 llama : fix progress dots (#11730) c026ba3c23765a648ca27c7a15ecf179f8e27f26 7ee953a64a40c09438b2064539becdbc577cefd0 Jeff Bolz jbolz@nvidia.com 2025-02-07T04:26:03-06:00 GitHub noreply@github.com 2025-02-07T11:26:03+01:00 vulkan: print shared memory size (#11719) 7ee953a64a40c09438b2064539becdbc577cefd0 ec3bc8270bc67b58955748d40a3e558a05b2d8f2 Christian Fillion cfillion@users.noreply.github.com 2025-02-07T04:33:27-05:00 GitHub noreply@github.com 2025-02-07T11:33:27+02:00 llama : add llama_sampler_init for safe usage of llama_sampler_free (#11727) ec3bc8270bc67b58955748d40a3e558a05b2d8f2 b7552cfcbc7defccd8bdefd0a7b9c47d145ed3d7 Akarshan Biswas akarshan.biswas@gmail.com 2025-02-07T14:57:53+05:30 GitHub noreply@github.com 2025-02-07T09:27:53Z SYCL: remove XMX info from print devices (#11712) b7552cfcbc7defccd8bdefd0a7b9c47d145ed3d7 225bbbfa39930cda38a2e5d1f3e5b38226732009 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-07T09:15:22+01:00 GitHub noreply@github.com 2025-02-07T09:15:22+01:00 common : add default embeddings presets (#11677) 225bbbfa39930cda38a2e5d1f3e5b38226732009 855cd0734aca26c86cc23d94aefd34f934464ac9 Jinyang He hejinyang@loongson.cn 2025-02-07T15:38:31+08:00 GitHub noreply@github.com 2025-02-07T09:38:31+02:00 ggml : optimize and build warning fix for LoongArch (#11709) 855cd0734aca26c86cc23d94aefd34f934464ac9 8a59053f63fffc24e730cd3ea067760abfe4a919 tv1wnd 55383215+tv1wnd@users.noreply.github.com 2025-02-06T22:48:51+01:00 GitHub noreply@github.com 2025-02-06T22:48:51+01:00 llama : fix old glm4 models (#11670) 8a59053f63fffc24e730cd3ea067760abfe4a919 1d20e53c40c3cc848ba2b95f5bf7c075eeec8b19 Georgi Gerganov ggerganov@gmail.com 2025-02-06T21:23:03+02:00 Georgi Gerganov ggerganov@gmail.com 2025-02-06T21:23:03+02:00 sync : ggml 1d20e53c40c3cc848ba2b95f5bf7c075eeec8b19 2fb3c32a1634488a5265d1304ab37628eeb5480d Patrick Peng retr0@retr0.blog 2025-02-06T09:29:13-05:00 Georgi Gerganov ggerganov@gmail.com 2025-02-06T21:22:54+02:00 rpc: fix known RCE in rpc-server (ggml/1103) 2fb3c32a1634488a5265d1304ab37628eeb5480d 9ab42dc722ad19a12af80f38a06474d498f96da3 Xuan-Son Nguyen thichthat@gmail.com 2025-02-06T17:32:29+01:00 GitHub noreply@github.com 2025-02-06T17:32:29+01:00 server : (webui) migrate project to ReactJS with typescript (#11688) 9ab42dc722ad19a12af80f38a06474d498f96da3 194b2e69f8da3a22395c74fd9acd6d5835437b96 Tei Home taiteitonghome@proton.me 2025-02-06T20:16:15+08:00 GitHub noreply@github.com 2025-02-06T12:16:15Z docs: update fedora cuda guide for 12.8 release (#11393) 194b2e69f8da3a22395c74fd9acd6d5835437b96 9dd7a0390feffcc1f4b17eb7692a6e43030d85af Akarshan Biswas akarshan.biswas@gmail.com 2025-02-06T17:12:35+05:30 GitHub noreply@github.com 2025-02-06T11:42:35Z SYCL: Adjust support condition for norm operators (#11674) 9dd7a0390feffcc1f4b17eb7692a6e43030d85af c0d4843225eed38903ea71ef302a02fa0b27f048 Georgi Gerganov ggerganov@gmail.com 2025-02-06T13:41:37+02:00 GitHub noreply@github.com 2025-02-06T13:41:37+02:00 llama : add log about loading model tensors (#11699) c0d4843225eed38903ea71ef302a02fa0b27f048 8d4d2be143a3919c3d194b9bf7a221e50abed893 Adrien Gallouët adrien@gallouet.fr 2025-02-06T12:08:13+01:00 GitHub noreply@github.com 2025-02-06T13:08:13+02:00 build : fix llama.pc (#11658) 8d4d2be143a3919c3d194b9bf7a221e50abed893 2c6c8df56d8a3edd657b9a295e95d469a37f0044 junchao-zhao 68935141+junchao-loongson@users.noreply.github.com 2025-02-06T17:20:00+08:00 GitHub noreply@github.com 2025-02-06T11:20:00+02:00 ggml : fix LoongArch compile error with 128-bit SIMD (#11701) 2c6c8df56d8a3edd657b9a295e95d469a37f0044 8a7e3bf17aa5a8412854787746c92a28623a8925 Jeff Bolz jbolz@nvidia.com 2025-02-06T00:15:30-06:00 GitHub noreply@github.com 2025-02-06T07:15:30+01:00 vulkan: optimize coopmat2 iq2/iq3 callbacks (#11521) 8a7e3bf17aa5a8412854787746c92a28623a8925 1b598b30581bad59e5af86c94362f9a30f261fac Rémy O remyoudompheng@gmail.com 2025-02-06T07:09:59+01:00 GitHub noreply@github.com 2025-02-06T07:09:59+01:00 vulkan: initial support for IQ4_XS quantization (#11501) 1b598b30581bad59e5af86c94362f9a30f261fac 902368a06b915b860236cfc97ff885b2aceae256 Jeff Bolz jbolz@nvidia.com 2025-02-06T00:02:18-06:00 GitHub noreply@github.com 2025-02-06T07:02:18+01:00 vulkan: use smaller combined allocations to avoid fragmentation (#11551) 902368a06b915b860236cfc97ff885b2aceae256 c3db0480bb820e120249014b380e1f4badf2a1c1 Charles Duffy charles@dyfis.net 2025-02-05T19:52:31-06:00 GitHub noreply@github.com 2025-02-06T09:52:31+08:00 metal : avoid breaking build when metal API predates TARGET_OS_VISION (#11690) c3db0480bb820e120249014b380e1f4badf2a1c1 d774ab3acc4fee41fbed6dbfc192b57d5f79f34b Matvey Soloviev blackhole89@gmail.com 2025-02-06T01:55:25+01:00 GitHub noreply@github.com 2025-02-06T01:55:25+01:00 readme : add link to Autopen under UIs (#11684) d774ab3acc4fee41fbed6dbfc192b57d5f79f34b fa62da9b2dc03539a30f1306f59c4c6ffbe4f50a Georgi Gerganov ggerganov@gmail.com 2025-02-05T10:57:42+02:00 GitHub noreply@github.com 2025-02-05T10:57:42+02:00 metal : adjust support conditions for norm operators (#11671) fa62da9b2dc03539a30f1306f59c4c6ffbe4f50a 1ec208083cb896dd8772a2f962151fa3d4a74e36 Johannes Gäßler johannesg@5d6.de 2025-02-05T08:58:31+01:00 GitHub noreply@github.com 2025-02-05T08:58:31+01:00 CUDA: support for mat. mul. with ne03 != ne13 (#11656) 1ec208083cb896dd8772a2f962151fa3d4a74e36 9f4cc8f8d310b13ab3fc93a9be81b1d1376a7fa6 SAMI samuel.koesnadi@stud.uni-due.de 2025-02-05T14:45:40+07:00 GitHub noreply@github.com 2025-02-05T10:45:40+03:00 llava: add quantization for the visual projector LLAVA, Qwen2VL (#11644) 9f4cc8f8d310b13ab3fc93a9be81b1d1376a7fa6 fd08255d0dea6625596c0367ee0a11d195f36762 Olivier Chafik ochafik@users.noreply.github.com 2025-02-05T01:00:12Z GitHub noreply@github.com 2025-02-05T01:00:12Z `sync`: minja (#11641) fd08255d0dea6625596c0367ee0a11d195f36762 3ec9fd4b77b6aca03a3c2bf678eae3f9517d6904 Johannes Gäßler johannesg@5d6.de 2025-02-04T22:21:42+01:00 GitHub noreply@github.com 2025-02-04T22:21:42+01:00 CUDA: non-contiguous (RMS) norm support (#11659) 3ec9fd4b77b6aca03a3c2bf678eae3f9517d6904 3962fc1a7956dd0afacfbce10fd1a3ffd3ad857e fxzjshm 11426482+fxzjshm@users.noreply.github.com 2025-02-05T02:18:38+08:00 GitHub noreply@github.com 2025-02-04T19:18:38+01:00 HIP: force max threads per block to be 1024 (#11621) 3962fc1a7956dd0afacfbce10fd1a3ffd3ad857e 1bef571f6a23c36a26dabacba631763f9a893b83 Xuan-Son Nguyen thichthat@gmail.com 2025-02-04T18:25:42+01:00 GitHub noreply@github.com 2025-02-04T18:25:42+01:00 server : add try..catch to places not covered by set_exception_handler (#11620) 1bef571f6a23c36a26dabacba631763f9a893b83 db288b60cb49eab69703f995379b8d75c18bf5b3 Radoslav Gerganov rgerganov@gmail.com 2025-02-04T18:16:20+02:00 GitHub noreply@github.com 2025-02-04T18:16:20+02:00 arg : list RPC devices first when using --list-devices (#11655) db288b60cb49eab69703f995379b8d75c18bf5b3 106045e7bb8db481bb2ebbc60e3b53cb27ada117 Olivier Chafik ochafik@users.noreply.github.com 2025-02-04T15:48:53Z GitHub noreply@github.com 2025-02-04T15:48:53Z `tool-call`: command r7b fix for normal responses (#11608) 106045e7bb8db481bb2ebbc60e3b53cb27ada117 f117d84b48104992ba16961b35a96fa93efbb355 Shelby Jenkins 47464908+ShelbyJenkins@users.noreply.github.com 2025-02-04T05:20:55-06:00 GitHub noreply@github.com 2025-02-04T13:20:55+02:00 readme : add llm_client Rust crate to readme bindings (#11628) f117d84b48104992ba16961b35a96fa93efbb355 534c46b53c23613628d72e93c63ea01ea4d1e2ac Jhen-Jie Hong iainst0409@gmail.com 2025-02-04T19:15:24+08:00 GitHub noreply@github.com 2025-02-04T13:15:24+02:00 swift : fix llama-vocab api usage (#11645) 534c46b53c23613628d72e93c63ea01ea4d1e2ac 387a1598ca094a4755303ec964c3b09b4c5c300e Jhen-Jie Hong iainst0409@gmail.com 2025-02-04T19:07:18+08:00 GitHub noreply@github.com 2025-02-04T13:07:18+02:00 metal : use residency set for other platforms (#11648) 387a1598ca094a4755303ec964c3b09b4c5c300e 7c9e0ca52039530b6ddf16015cf1263491e92d75 Georgi Gerganov ggerganov@gmail.com 2025-02-04T13:04:10+02:00 Georgi Gerganov ggerganov@gmail.com 2025-02-04T13:04:10+02:00 authors : update 7c9e0ca52039530b6ddf16015cf1263491e92d75 8f8290ada96194138bb3d4dc6958323bc728ef83 Georgi Gerganov ggerganov@gmail.com 2025-02-04T12:59:21+02:00 Georgi Gerganov ggerganov@gmail.com 2025-02-04T12:59:21+02:00 sync : ggml 8f8290ada96194138bb3d4dc6958323bc728ef83 b34aedd558dcf67f7e96b0260d6b554877bd3499 Christian Kastner ckk@kvr.at 2025-02-04T00:17:15+01:00 Georgi Gerganov ggerganov@gmail.com 2025-02-04T12:59:15+02:00 cmake: Add ability to pass in GGML_BUILD_NUMBER (ggml/1096) b34aedd558dcf67f7e96b0260d6b554877bd3499 cde383323959544abe10a4d79e1d3e1ee479933c Georgi Gerganov ggerganov@gmail.com 2025-02-04T09:30:42+02:00 Georgi Gerganov ggerganov@gmail.com 2025-02-04T09:31:01+02:00 ci : do not stale-close roadmap issues cde383323959544abe10a4d79e1d3e1ee479933c b3451785aca16fbf4214eeba7e4612676cdf8ccb Olivier Chafik ochafik@users.noreply.github.com 2025-02-03T23:49:27Z GitHub noreply@github.com 2025-02-03T23:49:27Z `tool-call`: allow `--chat-template chatml` w/ `--jinja`, default to chatml upon parsing issue, avoid double bos (#11616) b3451785aca16fbf4214eeba7e4612676cdf8ccb 1d1e6a90bcf485ad2dee309c31cf19bd802465e5 Xuan-Son Nguyen thichthat@gmail.com 2025-02-04T00:10:52+01:00 GitHub noreply@github.com 2025-02-04T00:10:52+01:00 server : (webui) revert hacky solution from #11626 (#11634) 1d1e6a90bcf485ad2dee309c31cf19bd802465e5 5598f475be3e31430fbe17ebb85654ec90dc201e Woof Dog 197125663+woof-dog@users.noreply.github.com 2025-02-03T22:16:27Z GitHub noreply@github.com 2025-02-03T23:16:27+01:00 server : (webui) allow typing and submitting during llm response (#11626) 5598f475be3e31430fbe17ebb85654ec90dc201e 8ec05832fa8409c49b3bbd13f957c6ae8486e618 Daniel Bevenius daniel.bevenius@gmail.com 2025-02-03T16:45:38+01:00 GitHub noreply@github.com 2025-02-03T16:45:38+01:00 server : remove CPPHTTPLIB_NO_EXCEPTIONS define (#11622) 8ec05832fa8409c49b3bbd13f957c6ae8486e618 21c84b5d2dc04050714567501bf78762bfa17846 Georgi Gerganov ggerganov@gmail.com 2025-02-03T14:57:08+02:00 Georgi Gerganov ggerganov@gmail.com 2025-02-03T14:57:08+02:00 sync : ggml 21c84b5d2dc04050714567501bf78762bfa17846 d92cb67e37abc23b1c6f7b0ef27a9889da8537e3 Johannes Gäßler johannesg@5d6.de 2025-02-03T13:25:56+01:00 GitHub noreply@github.com 2025-02-03T14:25:56+02:00 CUDA: fix Volta FlashAttention logic (#11615) d92cb67e37abc23b1c6f7b0ef27a9889da8537e3 6eecde3cc8fda44da7794042e3668de4af3c32c6 mashdragon 122402293+mashdragon@users.noreply.github.com 2025-02-03T09:42:55Z GitHub noreply@github.com 2025-02-03T10:42:55+01:00 server : (webui) Fix Shift+Enter handling (#11609) 6eecde3cc8fda44da7794042e3668de4af3c32c6 396856b40029dd6747d2fbdb179e828683418045 Johannes Gäßler johannesg@5d6.de 2025-02-02T23:48:29+01:00 GitHub noreply@github.com 2025-02-02T23:48:29+01:00 HIP: fix flash_attn_stream_k_fixup warning (#11604) 396856b40029dd6747d2fbdb179e828683418045 4d0598e1445a64c99cf2faac72f8d5d023f1e6a1 uvos philipp@uvos.xyz 2025-02-02T22:40:09+01:00 GitHub noreply@github.com 2025-02-02T22:40:09+01:00 CUDA/HIP: add support for selectable warp size to mmv (#11519) 4d0598e1445a64c99cf2faac72f8d5d023f1e6a1 90f9b88afb6447d3929843a2aa98c0f11074762d uvos philipp@uvos.xyz 2025-02-02T22:08:05+01:00 GitHub noreply@github.com 2025-02-02T22:08:05+01:00 HIP: add GGML_CUDA_CC_IS_* for amd familys as increasing cc archtectures for amd gpus are not supersets of eatch other (#11601) 90f9b88afb6447d3929843a2aa98c0f11074762d 864a0b67a6c8f648c43ce8271f9cb2e12dd5df6e Olivier Chafik ochafik@users.noreply.github.com 2025-02-02T19:58:34Z GitHub noreply@github.com 2025-02-02T19:58:34Z nit: more informative crash when grammar sampler fails (#11593) 864a0b67a6c8f648c43ce8271f9cb2e12dd5df6e 84ec8a58f7b6aad6887bbfbd1321f3ff417341a5 Johannes Gäßler johannesg@5d6.de 2025-02-02T19:31:09+01:00 GitHub noreply@github.com 2025-02-02T19:31:09+01:00 CUDA: use mma PTX instructions for FlashAttention (#11583) 84ec8a58f7b6aad6887bbfbd1321f3ff417341a5 bfcce4d693617ec843d0b2510f6ee16e6bc6720d Eric Curtin ecurtin@redhat.com 2025-02-02T16:14:48+01:00 GitHub noreply@github.com 2025-02-02T15:14:48Z Name colors (#11573) bfcce4d693617ec843d0b2510f6ee16e6bc6720d 69804487e0b10f2c5c06316f0ac0eb6ada68433f Olivier Chafik ochafik@users.noreply.github.com 2025-02-02T09:25:38Z GitHub noreply@github.com 2025-02-02T09:25:38Z `tool-call`: support Command R7B (+ return tool_plan "thoughts" in API) (#11585) 69804487e0b10f2c5c06316f0ac0eb6ada68433f ff227703d6d6e1888bdc7af6138514092ffcdb96 Olivier Chafik ochafik@users.noreply.github.com 2025-02-02T09:10:15Z GitHub noreply@github.com 2025-02-02T09:10:15Z Fix exotic ci env that lacks ostringstream::str (#11581) ff227703d6d6e1888bdc7af6138514092ffcdb96 0cec062a638700495673f5494d200b74340538be Michał Moskal michal@moskal.me 2025-02-01T23:55:32-08:00 GitHub noreply@github.com 2025-02-02T09:55:32+02:00 sampling : support for llguidance grammars (#10224) 0cec062a638700495673f5494d200b74340538be 53debe6f3c9cca87e9520a83ee8c14d88977afa4 piDack 104877312+piDack@users.noreply.github.com 2025-02-02T15:48:46+08:00 GitHub noreply@github.com 2025-02-02T09:48:46+02:00 llama : add support for GLM-Edge and GLM-Edge-V series models (#10573) 53debe6f3c9cca87e9520a83ee8c14d88977afa4 cfd74c86dbaa95ed30aa6b30e14d8801eb975d63 Olivier Chafik ochafik@users.noreply.github.com 2025-02-01T18:22:38Z GitHub noreply@github.com 2025-02-01T18:22:38Z ci: use sccache on windows HIP jobs (#11553) cfd74c86dbaa95ed30aa6b30e14d8801eb975d63 ecef206ccb186a1cde8dd2523b1da3e12f593f9e Olivier Chafik ochafik@users.noreply.github.com 2025-02-01T12:24:51Z GitHub noreply@github.com 2025-02-01T12:24:51Z `sync`: minja (https://github.com/google/minja/commit/418a2364b56dc9be4ed9a1a2b0fb16fb53a7a22e) (#11574) ecef206ccb186a1cde8dd2523b1da3e12f593f9e 5bbc7362cb93265f4c853fd89800a6255cc26985 Eric Curtin ecurtin@redhat.com 2025-02-01T11:30:54+01:00 GitHub noreply@github.com 2025-02-01T10:30:54Z Implement s3:// protocol (#11511) 5bbc7362cb93265f4c853fd89800a6255cc26985 aa6fb1321333fae8853d0cdc26bcb5d438e650a1 Olivier Chafik ochafik@users.noreply.github.com 2025-02-01T00:01:20Z GitHub noreply@github.com 2025-02-01T00:01:20Z ci: simplify cmake build commands (#11548) aa6fb1321333fae8853d0cdc26bcb5d438e650a1 a83f528688324a21484a97af1d1be5e1bc8d4c8e Olivier Chafik ochafik@users.noreply.github.com 2025-01-31T17:12:40Z GitHub noreply@github.com 2025-01-31T17:12:40Z `ci`: use sccache on windows instead of ccache (#11545) a83f528688324a21484a97af1d1be5e1bc8d4c8e b1bcd309fc8ac929cbd4a6207b3a19886bda031f Olivier Chafik ochafik@users.noreply.github.com 2025-01-31T14:15:25Z GitHub noreply@github.com 2025-01-31T14:15:25Z `tool-call`: fix llama 3.x and functionary 3.2, play nice w/ pydantic_ai package, update readme (#11539) b1bcd309fc8ac929cbd4a6207b3a19886bda031f 5783575c9d99c4d9370495800663aa5397ceb0be Olivier Chafik ochafik@users.noreply.github.com 2025-01-31T13:48:31Z GitHub noreply@github.com 2025-01-31T13:48:31Z fix stop regression (#11543) 5783575c9d99c4d9370495800663aa5397ceb0be 4a2b196d03d52da31236390e9f5694a88d43d11d Olivier Chafik ochafik@users.noreply.github.com 2025-01-31T08:24:29Z GitHub noreply@github.com 2025-01-31T08:24:29Z Fix chatml fallback for unsupported builtin templates (when --jinja not enabled) (#11533) 4a2b196d03d52da31236390e9f5694a88d43d11d 1bd3047a939e561adfb3c7dd2e17c4cc7a4e4e6f Olivier Chafik ochafik@users.noreply.github.com 2025-01-31T08:12:40Z GitHub noreply@github.com 2025-01-31T10:12:40+02:00 server : fix --jinja when there's no tools or schema (typo was forcing JSON) (#11531) 1bd3047a939e561adfb3c7dd2e17c4cc7a4e4e6f a2df2787b32e0846205f7151dfad88ceab592beb Steve Grubb ausearch.1@gmail.com 2025-01-31T00:58:55-05:00 GitHub noreply@github.com 2025-01-31T07:58:55+02:00 common: Add missing va_end (#11529) a2df2787b32e0846205f7151dfad88ceab592beb 553f1e46e9e864514bbd6bf4009146db66be0541 Daniel Bevenius daniel.bevenius@gmail.com 2025-01-31T06:04:53+01:00 GitHub noreply@github.com 2025-01-31T06:04:53+01:00 server : update help metrics processing/deferred (#11512) 553f1e46e9e864514bbd6bf4009146db66be0541 8b576b6c55bc4e6be898b47522f0ef402b93ef62 Olivier Chafik ochafik@users.noreply.github.com 2025-01-30T22:01:06Z GitHub noreply@github.com 2025-01-30T22:01:06Z `ci`: ccache for all github worfklows (#11516) 8b576b6c55bc4e6be898b47522f0ef402b93ef62 27d135c970c00f655d486f870edacded792bef5c Olivier Chafik ochafik@users.noreply.github.com 2025-01-30T19:13:58Z GitHub noreply@github.com 2025-01-30T19:13:58Z Tool call support (generic + native for Llama, Functionary, Hermes, Mistral, Firefunction, DeepSeek) w/ lazy grammars (#9639) 27d135c970c00f655d486f870edacded792bef5c 6af1ca48cbdf9a438438afd0a9a549a272bc95bf uvos devnull@uvos.xyz 2025-01-29T19:36:00+01:00 uvos devnull@uvos.xyz 2025-01-30T16:25:44+01:00 HIP: require at least HIP 5.5 6af1ca48cbdf9a438438afd0a9a549a272bc95bf c300e68ef490e6cf6c04ed96fd27a6a53ab8a422 uvos devnull@uvos.xyz 2025-01-29T19:12:42+01:00 uvos devnull@uvos.xyz 2025-01-30T16:25:44+01:00 HIP: Prepare reduction operators for wave 64 c300e68ef490e6cf6c04ed96fd27a6a53ab8a422 3d804dec7661fbb7de9b7f93267e2fc3ca0193c1 uvos devnull@uvos.xyz 2025-01-29T17:46:23+01:00 uvos devnull@uvos.xyz 2025-01-30T16:25:44+01:00 CUDA/HIP: add warp_size to cuda_device_info 3d804dec7661fbb7de9b7f93267e2fc3ca0193c1 ffd0821c57edc7e5d04338ab0c6b1461198df15f Olivier Chafik ochafik@users.noreply.github.com 2025-01-30T10:30:27Z GitHub noreply@github.com 2025-01-30T10:30:27Z sync: minja (#11499) ffd0821c57edc7e5d04338ab0c6b1461198df15f 4314e56c4f8c5091f45732f39bd94c0c6c323798 mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2025-01-30T11:10:59+01:00 GitHub noreply@github.com 2025-01-30T12:10:59+02:00 vocab : correctly identify LF token for GPT-2 style BPE tokenizer (#11496) 4314e56c4f8c5091f45732f39bd94c0c6c323798 496e5bf46bab757d05e18227cb4e17055ae42f42 Daniel Bevenius daniel.bevenius@gmail.com 2025-01-30T11:05:00+01:00 GitHub noreply@github.com 2025-01-30T11:05:00+01:00 server : use lambda instead of std::bind (#11507) 496e5bf46bab757d05e18227cb4e17055ae42f42 7919256c57f05c09b0b50ec9abb37ff62dab7251 Isaac McFadyen isaac@imcf.me 2025-01-30T04:11:53-05:00 GitHub noreply@github.com 2025-01-30T10:11:53+01:00 server : (docs) added response format for /apply-template [no ci] (#11503) 7919256c57f05c09b0b50ec9abb37ff62dab7251 e0449763a4f335cca374254a72892141f41eaa59 Guspan Tanadi 36249910+guspan-tanadi@users.noreply.github.com 2025-01-30T12:58:02+07:00 GitHub noreply@github.com 2025-01-30T06:58:02+01:00 readme : reference examples relative links (#11505) e0449763a4f335cca374254a72892141f41eaa59 eb7cf15a808d4d7a71eef89cc6a9b96fe82989dc Daniel Bevenius daniel.bevenius@gmail.com 2025-01-30T05:48:14+01:00 GitHub noreply@github.com 2025-01-30T05:48:14+01:00 server : update json snippets in README.md [no ci] (#11492) eb7cf15a808d4d7a71eef89cc6a9b96fe82989dc 66ee4f297cff3c7ce98b31dbc0ce909d41b9e408 Nigel Bosch pnigelb@gmail.com 2025-01-29T12:45:44-06:00 GitHub noreply@github.com 2025-01-29T19:45:44+01:00 server : add /apply-template endpoint for additional use cases of Minja functionality (#11489) 66ee4f297cff3c7ce98b31dbc0ce909d41b9e408 e51c47b401f8cb5f21630a05171e2529cde4d186 Rémy Oudompheng oudomphe@phare.normalesup.org 2025-01-29T18:29:39+01:00 GitHub noreply@github.com 2025-01-29T18:29:39+01:00 vulkan: implement initial support for IQ2 and IQ3 quantizations (#11360) e51c47b401f8cb5f21630a05171e2529cde4d186 2711d0215ff6a1ecb2202ac8c1f135bceed7057b Daniel Bevenius daniel.bevenius@gmail.com 2025-01-29T16:34:18+01:00 GitHub noreply@github.com 2025-01-29T16:34:18+01:00 server : update auto gen files comments [no ci] (#11484) 2711d0215ff6a1ecb2202ac8c1f135bceed7057b f0d4b29edfc633ec3ba6442482a6b43a5cd80a01 Jeff Bolz jbolz@nvidia.com 2025-01-29T09:26:50-06:00 GitHub noreply@github.com 2025-01-29T09:26:50-06:00 vulkan: Catch pipeline creation failure and print an error message (#11436) f0d4b29edfc633ec3ba6442482a6b43a5cd80a01 815857791d3639a4d544d0a8cf25a49b0325c08c Eric Curtin ecurtin@redhat.com 2025-01-29T12:23:10+01:00 GitHub noreply@github.com 2025-01-29T11:23:10Z Parse https://ollama.com/library/ syntax (#11480) 815857791d3639a4d544d0a8cf25a49b0325c08c 1a0e87d29152cb9d4ce13b4ad64b0382c9ba1ab6 Georgi Gerganov ggerganov@gmail.com 2025-01-29T11:25:29+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-29T11:25:29+02:00 sync : ggml 1a0e87d29152cb9d4ce13b4ad64b0382c9ba1ab6 d2e518e9b4231077013ffb0e5c4cc24a0c4c1b7d William Tambellini wtambellini@sdl.com 2025-01-23T11:59:08-08:00 Georgi Gerganov ggerganov@gmail.com 2025-01-29T11:24:53+02:00 ggml : add option to not print stack on abort (ggml/1081) d2e518e9b4231077013ffb0e5c4cc24a0c4c1b7d b636228c0ad0db95bf73008094c6145a05615cf6 issixx 46835150+issixx@users.noreply.github.com 2025-01-17T21:29:08+09:00 Georgi Gerganov ggerganov@gmail.com 2025-01-29T11:24:51+02:00 ggml-cpu : fix ggml_graph_compute_thread did not terminate on abort. (ggml/1065) b636228c0ad0db95bf73008094c6145a05615cf6 325afb370a1a7b32b5fe46a749bc840c66db9765 Daniel Bevenius daniel.bevenius@gmail.com 2025-01-29T09:38:54+01:00 GitHub noreply@github.com 2025-01-29T10:38:54+02:00 embedding : enable --no-warmup option (#11475) 325afb370a1a7b32b5fe46a749bc840c66db9765 794fe23f29fb40104975c91fe19f23798f7c726e Molly Sophia mollysophia379@gmail.com 2025-01-29T12:07:21+08:00 GitHub noreply@github.com 2025-01-29T12:07:21+08:00 llama: fix missing k_cache store for rwkv6qwen2 (#11445) 794fe23f29fb40104975c91fe19f23798f7c726e cf8cc856d7d02165bd08593b4757e1256a62d501 Emreerdog 34742675+Emreerdog@users.noreply.github.com 2025-01-29T02:22:06+03:00 GitHub noreply@github.com 2025-01-28T19:22:06-04:00 cmake: add hints for locating ggml on Windows using Llama find-package (#11466) cf8cc856d7d02165bd08593b4757e1256a62d501 d0c08040b6c8bebeade7b8d5764df6cf901678d5 peidaqi peidaqi@gmail.com 2025-01-28T16:03:42-07:00 GitHub noreply@github.com 2025-01-29T00:03:42+01:00 server : Fixed wrong function name in llamacpp server unit test (#11473) d0c08040b6c8bebeade7b8d5764df6cf901678d5 be5ef7963fcf14a9c77c963fdd3f7b606eacb498 Xuan-Son Nguyen thichthat@gmail.com 2025-01-29T00:02:56+01:00 GitHub noreply@github.com 2025-01-29T00:02:56+01:00 ci : fix build CPU arm64 (#11472) be5ef7963fcf14a9c77c963fdd3f7b606eacb498 cae9fb4361138b937464524eed907328731b81f6 uvos philipp@uvos.xyz 2025-01-28T23:06:32+01:00 GitHub noreply@github.com 2025-01-28T23:06:32+01:00 HIP: Supress transformation warning in softmax.cu cae9fb4361138b937464524eed907328731b81f6 7fee2889e6565830631fbe76d47ef85cf8fd946a Nikita Sarychev 42014488+sARY77@users.noreply.github.com 2025-01-28T07:42:20-08:00 GitHub noreply@github.com 2025-01-28T16:42:20+01:00 HIP: Only call rocblas_initialize on rocblas versions with the multiple instantation bug (#11080) 7fee2889e6565830631fbe76d47ef85cf8fd946a d7d1eccacccaa698c9232014b96a82b359595d6e Eric Curtin ecurtin@redhat.com 2025-01-28T15:45:41+01:00 GitHub noreply@github.com 2025-01-28T14:45:41Z Add github protocol pulling and http:// (#11465) d7d1eccacccaa698c9232014b96a82b359595d6e 4bf3119d61c1de5660025fd5a611effe503e3d2b Nuno rare-magma@posteo.eu 2025-01-28T15:17:25+01:00 GitHub noreply@github.com 2025-01-28T14:17:25Z docker: allow installing pip packages system-wide (#11437) 4bf3119d61c1de5660025fd5a611effe503e3d2b f643120bad8ab3a753daa64aaac8288ee5800e06 someone13574 81528246+someone13574@users.noreply.github.com 2025-01-28T09:15:34-05:00 GitHub noreply@github.com 2025-01-28T15:15:34+01:00 cmake : don't fail on `GGML_CPU=OFF` (#11457) f643120bad8ab3a753daa64aaac8288ee5800e06 6e84b0ab8e10b8f6f99a32855f976ebcd35b0353 Nuno rare-magma@posteo.eu 2025-01-28T11:42:32+01:00 GitHub noreply@github.com 2025-01-28T10:42:32Z docker: add perplexity and bench commands to full image (#11438) 6e84b0ab8e10b8f6f99a32855f976ebcd35b0353 2b8525d5c89b124c4578a2621cbeb64354ff3d9c Akarshan Biswas akarshan.biswas@gmail.com 2025-01-28T15:26:58+05:30 GitHub noreply@github.com 2025-01-28T09:56:58Z SYCL : SOFTMAX F16 mask support and other fixes (#11261) 2b8525d5c89b124c4578a2621cbeb64354ff3d9c a4417ddda98fd0558fb4d802253e68a933704b59 Michael Engel mengel@redhat.com 2025-01-28T09:32:40+01:00 GitHub noreply@github.com 2025-01-28T08:32:40Z Handle missing model in CLI parameters for llama-run (#11399) a4417ddda98fd0558fb4d802253e68a933704b59 d6d24cd9ed6d0b9558643dcc28f2124bef488c52 Eric Curtin ecurtin@redhat.com 2025-01-27T19:36:10+01:00 GitHub noreply@github.com 2025-01-27T19:36:10+01:00 Add new hf protocol for ollama (#11449) d6d24cd9ed6d0b9558643dcc28f2124bef488c52 a5203b4465c5c87813936bde98170e25bb09024f Haus1 haus.xda@gmail.com 2025-01-27T08:58:17-05:00 GitHub noreply@github.com 2025-01-27T14:58:17+01:00 AMD: parse the architecture as supplied by gcnArchName (#11244) a5203b4465c5c87813936bde98170e25bb09024f df984e014714cba4c99ef894b20b51cbcef31b16 lexasub lexakopp2212@gmail.com 2025-01-27T17:42:09+04:00 GitHub noreply@github.com 2025-01-27T14:42:09+01:00 llama : minor fixes for up llama load model speed (#11448) df984e014714cba4c99ef894b20b51cbcef31b16 acd38efee316f3a5ed2e6afcbc5814807c347053 Johannes Gäßler johannesg@5d6.de 2025-01-27T12:07:12+01:00 GitHub noreply@github.com 2025-01-27T12:07:12+01:00 llama: refactor llama_decode_impl (#11381) acd38efee316f3a5ed2e6afcbc5814807c347053 caf773f249aa267c78d3da5567b8ab156080ea59 Ihar Hrachyshka ihrachys@redhat.com 2025-01-27T02:41:59-05:00 GitHub noreply@github.com 2025-01-27T09:41:59+02:00 metal: Handle null returned from MTLCreateSystemDefaultDevice() (#11441) caf773f249aa267c78d3da5567b8ab156080ea59 178a7eb952d211b8d4232d5e50ae1b64519172a9 Xuan Son Nguyen thichthat@gmail.com 2025-01-26T22:45:32+01:00 GitHub noreply@github.com 2025-01-26T22:45:32+01:00 docker : fix ARM build and Vulkan build (#11434) 178a7eb952d211b8d4232d5e50ae1b64519172a9 6f53d8a6b41e48c73b345fc6c712c3b00ea4fb93 Georgi Gerganov ggerganov@gmail.com 2025-01-26T20:06:16+02:00 GitHub noreply@github.com 2025-01-26T20:06:16+02:00 metal : use residency sets (#11427) 6f53d8a6b41e48c73b345fc6c712c3b00ea4fb93 19f65187cbf009801288861133267ee5573ceead Nuno rare-magma@posteo.eu 2025-01-26T18:22:43+01:00 GitHub noreply@github.com 2025-01-26T18:22:43+01:00 docker: add missing vulkan library to base layer and update to 24.04 (#11422) 19f65187cbf009801288861133267ee5573ceead 1d8ee06000ecdd274e7f0a0465d6bf26ad2b3491 bandoti 141645996+bandoti@users.noreply.github.com 2025-01-26T12:07:48-04:00 GitHub noreply@github.com 2025-01-26T12:07:48-04:00 cmake: add ggml find package (#11369) 1d8ee06000ecdd274e7f0a0465d6bf26ad2b3491 2cc9b8c32c78d09cd1b4df0aaa605ab2d0176243 Frank Mai thxcode0824@gmail.com 2025-01-26T23:20:34+08:00 GitHub noreply@github.com 2025-01-26T16:20:34+01:00 rpc: fix register position (#11424) 2cc9b8c32c78d09cd1b4df0aaa605ab2d0176243 f35726c2fb0a824246e004ab4bedcde37f3f0dd0 Georgi Gerganov ggerganov@gmail.com 2025-01-26T14:30:15+02:00 GitHub noreply@github.com 2025-01-26T14:30:15+02:00 readme : update hot topics f35726c2fb0a824246e004ab4bedcde37f3f0dd0 4a75d19376f2f00dbae6c266eb9c4f3001872b52 Jeff Bolz jbolz@nvidia.com 2025-01-25T20:10:03-06:00 GitHub noreply@github.com 2025-01-26T03:10:03+01:00 build: apply MSVC /bigobj option to c/cpp files only (#11423) 4a75d19376f2f00dbae6c266eb9c4f3001872b52 26771a1491f3a4c3d5b99c4c267b81aca9a7dfa0 Jeff Bolz jbolz@nvidia.com 2025-01-25T15:29:57-06:00 GitHub noreply@github.com 2025-01-25T22:29:57+01:00 vulkan: compile shaders on-demand (#11406) 26771a1491f3a4c3d5b99c4c267b81aca9a7dfa0 ca6baf76c1a7adb9134b08d2bc4c65557297ff87 uvos devnull@uvos.xyz 2025-01-25T21:01:12+01:00 GitHub noreply@github.com 2025-01-25T21:01:12+01:00 Hip: disable VMM on hip as it seams that it dosent work in some configurations (#11420) ca6baf76c1a7adb9134b08d2bc4c65557297ff87 6e264a905bec9e4c0111eb4c91379c88accef7c6 Jeff Bolz jbolz@nvidia.com 2025-01-25T11:26:37-06:00 GitHub noreply@github.com 2025-01-25T11:26:37-06:00 build: add /bigobj to MSVC build (#11407) 6e264a905bec9e4c0111eb4c91379c88accef7c6 49b0e3cec4b67dc9f4debe3a16acd4c819f751d6 Diego Devesa slarengh@gmail.com 2025-01-25T17:22:41+01:00 GitHub noreply@github.com 2025-01-25T17:22:41+01:00 docker : add GGML_CPU_ARM_ARCH arg to select ARM architecture to build for (#11419) 49b0e3cec4b67dc9f4debe3a16acd4c819f751d6 20a758155bc5f37290b20ea44d76ba99c4e7f2cb Xuan Son Nguyen thichthat@gmail.com 2025-01-25T16:36:44+01:00 GitHub noreply@github.com 2025-01-25T16:36:44+01:00 server : fix cleaning up stream task (#11418) 20a758155bc5f37290b20ea44d76ba99c4e7f2cb 00c24acb2ac49d9f8318e808b6ada2f5649f253f Diego Devesa slarengh@gmail.com 2025-01-25T15:22:29+01:00 GitHub noreply@github.com 2025-01-25T15:22:29+01:00 docker : fix CPU ARM build (#11403) 00c24acb2ac49d9f8318e808b6ada2f5649f253f 466ea66f338d63109540dae1df97ccfdbf4cd08f Georgi Gerganov ggerganov@gmail.com 2025-01-25T13:36:48+02:00 GitHub noreply@github.com 2025-01-25T13:36:48+02:00 ci : fix line breaks on windows builds (#11409) 466ea66f338d63109540dae1df97ccfdbf4cd08f 5f0db9522f347b095f84c3033d6c1c1895402e25 jiahao su damow890@gmail.com 2025-01-25T07:26:01+08:00 GitHub noreply@github.com 2025-01-25T00:26:01+01:00 CANN: Add Ascend CANN build ci (#10217) 5f0db9522f347b095f84c3033d6c1c1895402e25 c5d9effb49649db80a52caf5c0626de6f342f526 uvos devnull@uvos.xyz 2025-01-25T00:02:23+01:00 GitHub noreply@github.com 2025-01-25T00:02:23+01:00 hip : Add hipGraph and VMM support to ROCM (#11362) c5d9effb49649db80a52caf5c0626de6f342f526 9fbadaef4f0903c64895ba9c70f02ac6e6a4b41c Johannes Gäßler johannesg@5d6.de 2025-01-24T21:02:43+01:00 GitHub noreply@github.com 2025-01-24T21:02:43+01:00 CUDA: fix FP16 cuBLAS GEMM (#11396) 9fbadaef4f0903c64895ba9c70f02ac6e6a4b41c 9755129c27da76d768bd7e47e206bac61b40cf18 uvos devnull@uvos.xyz 2025-01-24T17:50:49+01:00 GitHub noreply@github.com 2025-01-24T17:50:49+01:00 rocBLAS: Avoid fp32->fp16->fp32 conversion on cdna (#11356) 9755129c27da76d768bd7e47e206bac61b40cf18 a07c2c8a52464646ce13040e62c1ea04459f721e Georgi Gerganov ggerganov@gmail.com 2025-01-24T18:41:30+02:00 GitHub noreply@github.com 2025-01-24T18:41:30+02:00 release : pack /lib in the packages (#11392) a07c2c8a52464646ce13040e62c1ea04459f721e 8137b4bb2b5fd4cb4a752bfe69ccfd915a313d58 Jafar Uruç jafar.uruc@gmail.com 2025-01-24T13:30:13Z GitHub noreply@github.com 2025-01-24T14:30:13+01:00 docs : Update readme to build targets for local docker build (#11368) 8137b4bb2b5fd4cb4a752bfe69ccfd915a313d58 1af6945eb0d0e97525dc0ec18167abf05c28f482 Johannes Gäßler johannesg@5d6.de 2025-01-24T12:38:31+01:00 GitHub noreply@github.com 2025-01-24T12:38:31+01:00 CPU/CUDA: fix (GQA) mul mat back, add CUDA support (#11380) 1af6945eb0d0e97525dc0ec18167abf05c28f482 01f37edf1a6fae76fd9e2e02109aae6995a914f0 Bernhard M. Wiedemann githubbmwprimary@lsmod.de 2025-01-24T12:21:35+01:00 GitHub noreply@github.com 2025-01-24T13:21:35+02:00 cmake : avoid -march=native when reproducible build is wanted (#11366) 01f37edf1a6fae76fd9e2e02109aae6995a914f0 c07e87f38bd0c22ec6dbc852ae50aaa1c64632d4 Eric Curtin ecurtin@redhat.com 2025-01-24T09:39:24Z GitHub noreply@github.com 2025-01-24T09:39:24Z Update llama-run README.md (#11386) c07e87f38bd0c22ec6dbc852ae50aaa1c64632d4 564804b79b78df1469ec8646869972de5e885ec4 stduhpf stephduh@live.fr 2025-01-24T09:02:38+01:00 GitHub noreply@github.com 2025-01-24T09:02:38+01:00 server : (webui) put DeepSeek R1 CoT in a collapsible
element (#11364) 564804b79b78df1469ec8646869972de5e885ec4 05f63cc9ee859de07f585f7b12939345f39ada8b Jeff Bolz jbolz@nvidia.com 2025-01-23T14:51:24-06:00 GitHub noreply@github.com 2025-01-23T14:51:24-06:00 tests: fix some mul_mat test gaps (#11375) 05f63cc9ee859de07f585f7b12939345f39ada8b f7fb43cd0b84280c261f440dc8e85eafad4a0ca6 Eric Curtin ecurtin@redhat.com 2025-01-23T20:04:31Z GitHub noreply@github.com 2025-01-23T20:04:31Z Update documentation (#11373) f7fb43cd0b84280c261f440dc8e85eafad4a0ca6 58456616408c828a1ce6d2481940fd1c97bce3b5 Eric Curtin ecurtin@redhat.com 2025-01-23T16:16:18Z GitHub noreply@github.com 2025-01-23T16:16:18Z Add -ngl (#11372) 58456616408c828a1ce6d2481940fd1c97bce3b5 f211d1dc10332b7e89a4abd1041903fa63bfed27 Xuan Son Nguyen thichthat@gmail.com 2025-01-23T13:56:05+01:00 GitHub noreply@github.com 2025-01-23T13:56:05+01:00 server : add more clean up when cancel_tasks is called (#11340) f211d1dc10332b7e89a4abd1041903fa63bfed27 955a6c2d91480954e90469517c4b91c4052c6a59 Eric Curtin ecurtin@redhat.com 2025-01-23T10:38:20Z GitHub noreply@github.com 2025-01-23T10:38:20Z Treat hf.co/ prefix the same as hf:// (#11350) 955a6c2d91480954e90469517c4b91c4052c6a59 1971adf55e3ebbfab83771d6174d37b77c352c71 amd-dwang dong.wang@amd.com 2025-01-23T15:14:28+08:00 GitHub noreply@github.com 2025-01-23T08:14:28+01:00 Vulkan-run-test: fix mmq_wg_denoms (#11343) 1971adf55e3ebbfab83771d6174d37b77c352c71 5245729e3317064959faefc5e5cbc63f4e9cfbea Jeff Bolz jbolz@nvidia.com 2025-01-23T01:07:50-06:00 GitHub noreply@github.com 2025-01-23T08:07:50+01:00 vulkan: sort shaders for more deterministic binary (#11315) 5245729e3317064959faefc5e5cbc63f4e9cfbea 6152129d05870cb38162c422c6ba80434e021e9f Jeff Bolz jbolz@nvidia.com 2025-01-23T01:01:17-06:00 GitHub noreply@github.com 2025-01-23T08:01:17+01:00 vulkan: fix diag_mask_inf (#11323) 6152129d05870cb38162c422c6ba80434e021e9f 16d3df7ab0bb9def78047eab4d9b15393997f495 Diego Devesa slarengh@gmail.com 2025-01-22T19:22:20+01:00 GitHub noreply@github.com 2025-01-22T19:22:20+01:00 main : update README documentation for batch size (#11353) 16d3df7ab0bb9def78047eab4d9b15393997f495 12c2bdf2de34f747d13b270fc9d3b52490bf194f Georgi Gerganov ggerganov@gmail.com 2025-01-22T19:44:26+02:00 GitHub noreply@github.com 2025-01-22T19:44:26+02:00 readme : add plugin links (#11355) 12c2bdf2de34f747d13b270fc9d3b52490bf194f c64d2becb13f2a7c0145b516a05f028d949a046b Diego Devesa slarengh@gmail.com 2025-01-22T17:44:40+01:00 GitHub noreply@github.com 2025-01-22T17:44:40+01:00 server : fix draft context not being released (#11354) c64d2becb13f2a7c0145b516a05f028d949a046b 96f405393461062450692430e4916809bf71c3c4 Olivier Chafik ochafik@users.noreply.github.com 2025-01-22T16:16:27Z GitHub noreply@github.com 2025-01-22T16:16:27Z `minja`: sync at https://github.com/google/minja/commit/0f5f7f2b3770eb682fbc11763266d45204173686 (#11352) 96f405393461062450692430e4916809bf71c3c4 a94f3b2727e97eb6c904006eb786960c069282bc Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2025-01-22T12:51:32+01:00 GitHub noreply@github.com 2025-01-22T12:51:32+01:00 Adding logprobs to /v1/completions (#11344) a94f3b2727e97eb6c904006eb786960c069282bc 3e3357fd77bcf5bd8cfcc53ca53d4a5532e67e1b Olivier Chafik ochafik@users.noreply.github.com 2025-01-22T09:51:44Z GitHub noreply@github.com 2025-01-22T09:51:44Z `common`: utils to split / join / repeat strings (from json converter) (#11342) 3e3357fd77bcf5bd8cfcc53ca53d4a5532e67e1b 6171c9d25820ccf676b243c172868819d882848f tc-mb 157115220+tc-mb@users.noreply.github.com 2025-01-22T15:35:48+08:00 GitHub noreply@github.com 2025-01-22T09:35:48+02:00 llava : support Minicpm-omni (#11289) 6171c9d25820ccf676b243c172868819d882848f e28245f35f2faaf249dd352998b3693a8cc28c51 Olivier Chafik ochafik@users.noreply.github.com 2025-01-21T13:18:51Z GitHub noreply@github.com 2025-01-21T13:18:51Z Add Jinja template support (#11016) e28245f35f2faaf249dd352998b3693a8cc28c51 6da5bec81c34f3b8a8f1b367cf23ad016e83d332 Xuan Son Nguyen thichthat@gmail.com 2025-01-21T14:07:12+01:00 GitHub noreply@github.com 2025-01-21T14:07:12+01:00 export-lora : fix tok_embd tensor (#11330) 6da5bec81c34f3b8a8f1b367cf23ad016e83d332 2e2f8f093cd4fb6bbb87ba84f6b9684fa082f3fa Radoslav Gerganov rgerganov@gmail.com 2025-01-21T15:06:41+02:00 GitHub noreply@github.com 2025-01-21T15:06:41+02:00 rpc : better caching of the base buffer pointer (#11331) 2e2f8f093cd4fb6bbb87ba84f6b9684fa082f3fa 2139667ec419cdd953987f4df2ace9da87bbda28 Eric Curtin ecurtin@redhat.com 2025-01-21T09:32:35Z GitHub noreply@github.com 2025-01-21T09:32:35Z linenoise.cpp refactoring (#11301) 2139667ec419cdd953987f4df2ace9da87bbda28 80d0d6b4b7abca342fc990399e78af8d213eabaf Georgi Gerganov ggerganov@gmail.com 2025-01-21T08:48:13+02:00 GitHub noreply@github.com 2025-01-21T08:48:13+02:00 metal : fix out-of-bounds write (#11314) 80d0d6b4b7abca342fc990399e78af8d213eabaf aea8ddd5165d525a449e2fc3839db77a71f4a318 Georgi Gerganov ggerganov@gmail.com 2025-01-20T22:29:43+02:00 GitHub noreply@github.com 2025-01-20T22:29:43+02:00 common : add -hfd option for the draft model (#11318) aea8ddd5165d525a449e2fc3839db77a71f4a318 9f7add1cde670ff744b791f5ed6649e86a0c586c Jeff Bolz jbolz@nvidia.com 2025-01-20T10:38:32-06:00 GitHub noreply@github.com 2025-01-20T10:38:32-06:00 vulkan: fix coopmat2 validation failures (#11284) 9f7add1cde670ff744b791f5ed6649e86a0c586c 90d987b105db6016cb395f673b4d21e02129721e Georgi Gerganov ggerganov@gmail.com 2025-01-20T16:36:08+02:00 GitHub noreply@github.com 2025-01-20T16:36:08+02:00 examples : fix add_special conditions (#11311) 90d987b105db6016cb395f673b4d21e02129721e a4251edd6fc16d168f517a7e4b0936212b296603 Christopher Nielsen 62156882+mascguy@users.noreply.github.com 2025-01-20T09:02:43-05:00 GitHub noreply@github.com 2025-01-20T16:02:43+02:00 mmap: add include for cerrno (#11296) a4251edd6fc16d168f517a7e4b0936212b296603 ec7f3ac9ab33e46b136eb5ab6a76c4d81f57c7f1 Michael Podvitskiy podvitskiymichael@gmail.com 2025-01-20T15:02:15+01:00 GitHub noreply@github.com 2025-01-20T16:02:15+02:00 cmake: fix shell command quoting in build-info script (#11309) ec7f3ac9ab33e46b136eb5ab6a76c4d81f57c7f1 ef6dada60ca710f4edfbb6fd9e1258685d8ea49d Xuan Son Nguyen thichthat@gmail.com 2025-01-20T14:35:07+01:00 GitHub noreply@github.com 2025-01-20T14:35:07+01:00 llama : add support for Deepseek-R1-Qwen distill model (#11310) ef6dada60ca710f4edfbb6fd9e1258685d8ea49d ae3c1db2f9c4beec0737c6a82d1f3791fd6fcdb2 Georgi Gerganov ggerganov@gmail.com 2025-01-20T09:29:32+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-20T09:29:32+02:00 cont : fix whitespaces (#11305) ae3c1db2f9c4beec0737c6a82d1f3791fd6fcdb2 92bc493917d43b83e592349e138b54c90b1c3ea7 Kyle Bruene KyleBruene@users.noreply.github.com 2025-01-20T01:21:01-06:00 GitHub noreply@github.com 2025-01-20T09:21:01+02:00 llama : re-add LLM_ARCH_PHIMOE (#11305) 92bc493917d43b83e592349e138b54c90b1c3ea7 b9daaffe02d6a77d85f0420bce5dfe0e00daeff6 Georgi Gerganov ggerganov@gmail.com 2025-01-19T20:22:30+02:00 GitHub noreply@github.com 2025-01-19T20:22:30+02:00 tests : increase timeout when sanitizers are enabled (#11300) b9daaffe02d6a77d85f0420bce5dfe0e00daeff6 99487b57d47e14dc342b7b89d238ca11c0345241 Georgi Gerganov ggerganov@gmail.com 2025-01-19T18:12:09+02:00 GitHub noreply@github.com 2025-01-19T18:12:09+02:00 simple-chat : fix BOS being added to each message (#11278) 99487b57d47e14dc342b7b89d238ca11c0345241 a1649cc13f89946322358f92ea268ae1b7b5096c Nicolò Scipione nicolo.scipione@codeplay.com 2025-01-19T14:33:34+01:00 GitHub noreply@github.com 2025-01-19T21:33:34+08:00 SYCL: Introducing memory host pool (#11251) a1649cc13f89946322358f92ea268ae1b7b5096c 4dd34ff83165a483ebff7bd43621b28490fa1fd6 Eric Curtin ecurtin@redhat.com 2025-01-18T14:42:31Z GitHub noreply@github.com 2025-01-18T14:42:31Z Adding linenoise.cpp to llama-run (#11252) 4dd34ff83165a483ebff7bd43621b28490fa1fd6 f30f099228f774209aa3010b78dfbe5d262e69aa Georgi Gerganov ggerganov@gmail.com 2025-01-18T16:18:15+02:00 GitHub noreply@github.com 2025-01-18T16:18:15+02:00 cmake : add sanitizer flags for llama.cpp (#11279) f30f099228f774209aa3010b78dfbe5d262e69aa f26c87417999209e7f4576b4f3ecf7a5b9c66a29 Xuan Son Nguyen thichthat@gmail.com 2025-01-18T14:12:05+01:00 GitHub noreply@github.com 2025-01-18T14:12:05+01:00 server : implement cancellable request (#11285) f26c87417999209e7f4576b4f3ecf7a5b9c66a29 6390a998bfc63241fde8509022b0768a71bf20bb Georgi Gerganov ggerganov@gmail.com 2025-01-18T13:18:32+02:00 GitHub noreply@github.com 2025-01-18T13:18:32+02:00 scripts : restore hf.sh (#11288) 6390a998bfc63241fde8509022b0768a71bf20bb 44e18ef93995f3040660750b527e5becf85899d0 LostRuins Concedo 39025047+LostRuins@users.noreply.github.com 2025-01-18T18:20:57+08:00 GitHub noreply@github.com 2025-01-18T12:20:57+02:00 tts : add guide tokens support (#11186) 44e18ef93995f3040660750b527e5becf85899d0 3edfa7d3753c29e44b964c0ff424d2ea8d5fdee6 Jeff Bolz jbolz@nvidia.com 2025-01-18T02:26:50-06:00 GitHub noreply@github.com 2025-01-18T09:26:50+01:00 vulkan: fix coopmat2 flash attention for non-contiguous inputs (#11281) 3edfa7d3753c29e44b964c0ff424d2ea8d5fdee6 667d72846c06b2cf4f7c8a4265e210991a49706b codezjx code.zjx@gmail.com 2025-01-17T20:57:56+08:00 GitHub noreply@github.com 2025-01-17T14:57:56+02:00 llama.android: add field formatChat to control whether to parse special tokens when send message (#11270) 667d72846c06b2cf4f7c8a4265e210991a49706b a133566d34a1dd3693c504786963bf1b7b7d8c0e Radoslav Gerganov rgerganov@gmail.com 2025-01-17T10:57:09+02:00 GitHub noreply@github.com 2025-01-17T10:57:09+02:00 rpc : early register backend devices (#11262) a133566d34a1dd3693c504786963bf1b7b7d8c0e 960ec65273a9554ff2510ba285a970289256ebfb Georgi Gerganov ggerganov@gmail.com 2025-01-17T09:28:00+02:00 GitHub noreply@github.com 2025-01-17T09:28:00+02:00 vocab : fix double-eos check (#11273) 960ec65273a9554ff2510ba285a970289256ebfb 7a689c415e2aecea1a5ae438542afeaf69815d52 David Renshaw dwrenshaw@gmail.com 2025-01-17T02:12:01-05:00 GitHub noreply@github.com 2025-01-17T08:12:01+01:00 llama : fix deprecation message: vocabable -> vocab (#11269) 7a689c415e2aecea1a5ae438542afeaf69815d52 bd38ddea0181bc717de7cae66fd4323975c85656 musoles 135031143+musoles@users.noreply.github.com 2025-01-17T00:10:49Z GitHub noreply@github.com 2025-01-17T01:10:49+01:00 README : added kalavai to infrastructure list (#11216) bd38ddea0181bc717de7cae66fd4323975c85656 466300fe1416de2802b710215817db28d4496f41 Jeff Bolz jbolz@nvidia.com 2025-01-16T15:47:10-06:00 GitHub noreply@github.com 2025-01-16T22:47:10+01:00 vulkan: support copy from f32 to q4_0/q4_1/q5_0/q5_1/q8_0/iq4_nl (#11166) 466300fe1416de2802b710215817db28d4496f41 206bc53422521a67de5e2caba661e21d590d2bae Jeff Bolz jbolz@nvidia.com 2025-01-16T15:23:49-06:00 GitHub noreply@github.com 2025-01-16T22:23:49+01:00 vulkan: optimize coopmat2 q4_k/q5_k dequant functions. (#11206) 206bc53422521a67de5e2caba661e21d590d2bae 4dbc8b9cb71876e005724f4e8f73a3544646bcf5 Jeff Bolz jbolz@nvidia.com 2025-01-16T15:16:39-06:00 GitHub noreply@github.com 2025-01-16T22:16:39+01:00 vulkan: optimize coopmat2 q2_k dequant function (#11130) 4dbc8b9cb71876e005724f4e8f73a3544646bcf5 9c8dcefe171ba70ed14f2a64d1433da12d0dd1c1 RunningLeon mnsheng@yeah.net 2025-01-17T02:10:38+08:00 GitHub noreply@github.com 2025-01-16T20:10:38+02:00 llama : add internlm3 support (#11233) 9c8dcefe171ba70ed14f2a64d1433da12d0dd1c1 681149ced2582f48c24792403df1307fed5eb951 Johannes Gäßler johannesg@5d6.de 2025-01-16T16:43:38+01:00 GitHub noreply@github.com 2025-01-16T16:43:38+01:00 CUDA: backwards pass for misc. ops, add tests (#11257) 681149ced2582f48c24792403df1307fed5eb951 c67cc9837d48ea7f612b5666b90d189e63dfd7d3 Xuan Son Nguyen thichthat@gmail.com 2025-01-16T13:54:08+01:00 GitHub noreply@github.com 2025-01-16T13:54:08+01:00 llama : add `llama_model_load_from_splits` (#11255) c67cc9837d48ea7f612b5666b90d189e63dfd7d3 adc5dd92e8aea98f5e7ac84f6e1bc15de35130b5 fj-y-saito 85871716+fj-y-saito@users.noreply.github.com 2025-01-16T18:11:49+09:00 GitHub noreply@github.com 2025-01-16T11:11:49+02:00 ggml: aarch64: implement SVE kernels for q4_K_q8_K vector dot (#11227) adc5dd92e8aea98f5e7ac84f6e1bc15de35130b5 f11cfdfd7fe29436fce512d934c2ff6b94bd89d2 Eve 139727413+netrunnereve@users.noreply.github.com 2025-01-15T19:50:13Z GitHub noreply@github.com 2025-01-15T19:50:13Z vulkan: scale caching for k quants + misc fixes (#11081) f11cfdfd7fe29436fce512d934c2ff6b94bd89d2 1d8504338ea27c7331998c11afdbd47ce5a9daac Georgi Gerganov ggerganov@gmail.com 2025-01-15T18:28:35+02:00 GitHub noreply@github.com 2025-01-15T18:28:35+02:00 ci : use -no-cnv in gguf-split tests (#11254) 1d8504338ea27c7331998c11afdbd47ce5a9daac 432df2d5f901a8ac93d1befad916144a0478bd9e Junil Kim logyourself@gmail.com 2025-01-15T22:17:42+09:00 GitHub noreply@github.com 2025-01-15T14:17:42+01:00 fix: ggml: fix vulkan-shaders-gen build (#10448) 432df2d5f901a8ac93d1befad916144a0478bd9e 0ccd7f3eb2debe477ffe3c44d5353cc388c9418d Johannes Gäßler johannesg@5d6.de 2025-01-15T12:51:37+01:00 GitHub noreply@github.com 2025-01-15T12:51:37+01:00 RoPE: fix back, CUDA support for back + noncont. (#11240) 0ccd7f3eb2debe477ffe3c44d5353cc388c9418d f446c2cf6a56a750b67c967505e717a996d2f2fd Daniel Bevenius daniel.bevenius@gmail.com 2025-01-15T05:44:38+01:00 GitHub noreply@github.com 2025-01-15T05:44:38+01:00 examples : add embd_to_audio to tts-outetts.py [no ci] (#11235) f446c2cf6a56a750b67c967505e717a996d2f2fd b4d92a59a20eea400d8dd30844a339b76210daa0 Akarshan Biswas akarshan.biswas@gmail.com 2025-01-15T08:50:17+05:30 GitHub noreply@github.com 2025-01-15T11:20:17+08:00 SYCL: Add gated linear attention kernel (#11175) b4d92a59a20eea400d8dd30844a339b76210daa0 bbf3e55e352d309573bdafee01a014b0a2492155 Xuan Son Nguyen thichthat@gmail.com 2025-01-14T15:42:23+01:00 GitHub noreply@github.com 2025-01-14T16:42:23+02:00 ci : add -no-cnv for tests (#11238) bbf3e55e352d309573bdafee01a014b0a2492155 c5bf0d1bd7eb8762edca47e0a95f64e6fbfaf5b1 Georgi Gerganov ggerganov@gmail.com 2025-01-14T12:54:58+02:00 GitHub noreply@github.com 2025-01-14T11:54:58+01:00 vocab : add dummy tokens for "no_vocab" type (#11231) c5bf0d1bd7eb8762edca47e0a95f64e6fbfaf5b1 091592d758cb55af7bfadd6c397f61db387aa8f3 ebraminio ebrahim@gnu.org 2025-01-14T14:09:33+03:30 GitHub noreply@github.com 2025-01-14T11:39:33+01:00 server : Improve code snippets direction between RTL text (#11221) 091592d758cb55af7bfadd6c397f61db387aa8f3 44d1e796d08641e7083fcbf37b33c79842a2f01e Olivier Chafik ochafik@users.noreply.github.com 2025-01-14T10:16:41Z GitHub noreply@github.com 2025-01-14T10:16:41Z Refactor test-chat-template.cpp (#11224) 44d1e796d08641e7083fcbf37b33c79842a2f01e a4f3f5d8e64b10fcf59913d487c1782dd0bc23e0 Georgi Gerganov ggerganov@gmail.com 2025-01-14T10:39:42+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-14T10:39:42+02:00 sync : ggml a4f3f5d8e64b10fcf59913d487c1782dd0bc23e0 48e1ae0e61b04c6a458ac207e84e86f5ce0abccd Georgi Gerganov ggerganov@gmail.com 2025-01-14T09:40:15+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-14T09:40:52+02:00 scripts : sync gguf (cont) 48e1ae0e61b04c6a458ac207e84e86f5ce0abccd d00a80e89dc6632dde7391e1e1ecbbbac088dd25 Georgi Gerganov ggerganov@gmail.com 2025-01-14T09:36:58+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-14T09:36:58+02:00 scripts : sync gguf d00a80e89dc6632dde7391e1e1ecbbbac088dd25 504af20ee4eae72080a56d59d744f6774f7901ce Georgi Gerganov ggerganov@gmail.com 2025-01-14T09:19:58+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-14T09:19:58+02:00 scripts : sync opencl 504af20ee4eae72080a56d59d744f6774f7901ce 84a44815f704aaed8e8edec7a39e846a975c7ba9 ebraminio ebrahim@gnu.org 2025-01-13T22:53:31+03:30 GitHub noreply@github.com 2025-01-13T20:23:31+01:00 server : (UI) Improve messages bubble shape in RTL (#11220) 84a44815f704aaed8e8edec7a39e846a975c7ba9 39509fb082895d1eae2486f8ad2cbf0e905346c4 Xuan Son Nguyen thichthat@gmail.com 2025-01-13T20:18:12+01:00 GitHub noreply@github.com 2025-01-13T20:18:12+01:00 cli : auto activate conversation mode if chat template is available (#11214) 39509fb082895d1eae2486f8ad2cbf0e905346c4 a29f0870d4846f52eda14ae28cea612ab66d903c Andreas Kieslinger 47689530+aendk@users.noreply.github.com 2025-01-13T16:45:53+01:00 GitHub noreply@github.com 2025-01-13T16:45:53+01:00 cuda : CUDA Graph Compute Function Refactor (precursor for performance improvements) (#11042) a29f0870d4846f52eda14ae28cea612ab66d903c 437e05f714cdd67757405221578d3f95f8228b63 Georgi Gerganov ggerganov@gmail.com 2025-01-13T15:59:26+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-13T15:59:26+02:00 contrib : add naming guidelines (cont) (#11177) 437e05f714cdd67757405221578d3f95f8228b63 ca001f6656c1c3d29ef479b3aa5d669453e63be5 ebraminio ebrahim@gnu.org 2025-01-13T17:16:39+03:30 GitHub noreply@github.com 2025-01-13T14:46:39+01:00 server : (UI) Support for RTL text as models input or output (#11208) ca001f6656c1c3d29ef479b3aa5d669453e63be5 00b4c3da6202e855087a4986bf19bb41b959e333 Georgi Gerganov ggerganov@gmail.com 2025-01-13T15:08:44+02:00 GitHub noreply@github.com 2025-01-13T15:08:44+02:00 contrib : add naming guidelines (cont) (#11177) 00b4c3da6202e855087a4986bf19bb41b959e333 7426a26b2492fc546a4db6991e871ee605714093 Xuan Son Nguyen thichthat@gmail.com 2025-01-13T13:56:23+01:00 GitHub noreply@github.com 2025-01-13T13:56:23+01:00 common : support tag-based --hf-repo like on ollama (#11195) 7426a26b2492fc546a4db6991e871ee605714093 8f70fc3d1b1d3c17b61842330dd106d391cc1227 Georgi Gerganov ggerganov@gmail.com 2025-01-13T14:46:36+02:00 GitHub noreply@github.com 2025-01-13T14:46:36+02:00 contrib : add naming guidelines (#11177) 8f70fc3d1b1d3c17b61842330dd106d391cc1227 1244cdcf14900dd199907b13f25d9c91a507f578 Daniel Bevenius daniel.bevenius@gmail.com 2025-01-13T13:38:20+01:00 GitHub noreply@github.com 2025-01-13T13:38:20+01:00 llama : remove 'd' from bad special token log (#11212) 1244cdcf14900dd199907b13f25d9c91a507f578 924518e2e5726e81f3aeb2518fb85963a500e93a Radoslav Gerganov rgerganov@gmail.com 2025-01-13T13:31:41+02:00 GitHub noreply@github.com 2025-01-13T13:31:41+02:00 ggml : do not define GGML_USE_CUDA when building with GGML_BACKEND_DL (#11211) 924518e2e5726e81f3aeb2518fb85963a500e93a 9a483999a6fda350772aaf7bc541f1cb246f8a29 Eric Curtin ecurtin@redhat.com 2025-01-12T18:23:10Z GitHub noreply@github.com 2025-01-12T18:23:10Z Reset color before we exit (#11205) 9a483999a6fda350772aaf7bc541f1cb246f8a29 08f10f69c38288e9e8bb1f933af63a3fc9013d40 Xuan Son Nguyen thichthat@gmail.com 2025-01-12T13:45:14+01:00 GitHub noreply@github.com 2025-01-12T13:45:14+01:00 llama : fix chat template gguf key (#11201) 08f10f69c38288e9e8bb1f933af63a3fc9013d40 afa8a9ec9b520137bbd1ca6838cda93ee39baf20 Georgi Gerganov ggerganov@gmail.com 2025-01-12T12:15:53+02:00 GitHub noreply@github.com 2025-01-12T12:15:53+02:00 llama : remove notion of CLS token (#11064) afa8a9ec9b520137bbd1ca6838cda93ee39baf20 c05e8c9934f94fde49bc1bc9dc51eed282605150 Georgi Gerganov ggerganov@gmail.com 2025-01-12T11:32:42+02:00 GitHub noreply@github.com 2025-01-12T11:32:42+02:00 llama : add `llama_vocab`, functions -> methods, naming (#11110) c05e8c9934f94fde49bc1bc9dc51eed282605150 2739a71e4b88474833b64aa974ca4515574fd3c4 Vinesh Janarthanan 36610342+VJHack@users.noreply.github.com 2025-01-11T03:42:31-06:00 GitHub noreply@github.com 2025-01-11T11:42:31+02:00 gguf-py: fixed local detection of gguf package (#11180) 2739a71e4b88474833b64aa974ca4515574fd3c4 ba8a1f9c5b675459c55a83e3f97f10df3a66c788 Daniel Bevenius daniel.bevenius@gmail.com 2025-01-11T05:50:33+01:00 GitHub noreply@github.com 2025-01-11T05:50:33+01:00 convert : sort print supported models [no ci] (#11179) ba8a1f9c5b675459c55a83e3f97f10df3a66c788 ff3fcabc727b2dd0c477d23a258217b27cc639fb Daniel Bevenius daniel.bevenius@gmail.com 2025-01-10T13:16:16+01:00 GitHub noreply@github.com 2025-01-10T13:16:16+01:00 examples : add README.md to tts example [no ci] (#11155) ff3fcabc727b2dd0c477d23a258217b27cc639fb c3f9d25706ac84297067aeaa662c1f1af42ed443 Daniel Bevenius daniel.bevenius@gmail.com 2025-01-10T11:30:53+01:00 GitHub noreply@github.com 2025-01-10T11:30:53+01:00 convert : add --print-supported-models option (#11172) c3f9d25706ac84297067aeaa662c1f1af42ed443 ee7136c6d1e0ba7633294dad137b1573048031ec 0cc4m picard12@live.de 2025-01-10T06:39:33+01:00 GitHub noreply@github.com 2025-01-10T06:39:33+01:00 Vulkan: Fix float16 use on devices without float16 support + fix subgroup_size_control validation error (#11161) ee7136c6d1e0ba7633294dad137b1573048031ec c6860cc7346c90219475e4467bb8a288e0df975c Molly Sophia mollysophia379@gmail.com 2025-01-10T09:58:08+08:00 GitHub noreply@github.com 2025-01-10T09:58:08+08:00 llama: add support for QRWKV6 model architecture (#11001) c6860cc7346c90219475e4467bb8a288e0df975c 1204f9727005974587d6fc1dcd4d4f0ead87c856 Akarshan Biswas akarshan.biswas@gmail.com 2025-01-10T05:43:03+05:30 GitHub noreply@github.com 2025-01-10T08:13:03+08:00 SYCL: Refactor ggml_sycl_compute_forward (#11121) 1204f9727005974587d6fc1dcd4d4f0ead87c856 8eceb888d7b7f5e93d20a4f85ca6511022b87040 Tei Home taiteitonghome@proton.me 2025-01-09T19:32:06+08:00 GitHub noreply@github.com 2025-01-09T11:32:06Z doc: add cuda guide for fedora (#11135) 8eceb888d7b7f5e93d20a4f85ca6511022b87040 f8feb4b01af374ad2fce302fd5790529c615710b Daniel Bevenius daniel.bevenius@gmail.com 2025-01-09T11:28:29+01:00 GitHub noreply@github.com 2025-01-09T11:28:29+01:00 server : add tooltips to settings and themes btn (#11154) f8feb4b01af374ad2fce302fd5790529c615710b be0e950c91cde2d8488ae32162b549d7023482f0 Pierrick Hymbert pierrick.hymbert@gmail.com 2025-01-09T11:21:41+01:00 GitHub noreply@github.com 2025-01-09T11:21:41+01:00 model: Add support for PhiMoE arch (#11003) be0e950c91cde2d8488ae32162b549d7023482f0 d9feae1c06321aac9662fd4b4249452dccaec553 Georgi Gerganov ggerganov@gmail.com 2025-01-09T11:15:15+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-09T11:15:15+02:00 media : remove old img [no ci] d9feae1c06321aac9662fd4b4249452dccaec553 8d59d911711b8f1ba9ec57c4b192ccd2628af033 Xuan Son Nguyen thichthat@gmail.com 2025-01-09T10:07:33+01:00 GitHub noreply@github.com 2025-01-09T10:07:33+01:00 llama-chat : add phi 4 template (#11148) 8d59d911711b8f1ba9ec57c4b192ccd2628af033 8a1d9c25fafbaf4182dd0b785dd6303ee40d55bc hydai z54981220@gmail.com 2025-01-09T04:03:28+08:00 GitHub noreply@github.com 2025-01-08T20:03:28Z fix: add missing msg in static_assert (#11143) 8a1d9c25fafbaf4182dd0b785dd6303ee40d55bc 1bf839b1e8b9d043306c65eddd9021fe4337733e Vinesh Janarthanan 36610342+VJHack@users.noreply.github.com 2025-01-08T12:54:58-06:00 GitHub noreply@github.com 2025-01-08T20:54:58+02:00 gguf-py : move scripts directory (#11116) 1bf839b1e8b9d043306c65eddd9021fe4337733e f7cd13301c2a88f97073fd119072b4cc92c08df1 Eric Curtin ecurtin@redhat.com 2025-01-08T18:47:05Z GitHub noreply@github.com 2025-01-08T18:47:05Z Enhance user input handling for llama-run (#11138) f7cd13301c2a88f97073fd119072b4cc92c08df1 4d2b3d88041705b20c30b3219838aa435e7ffbde Xuan Son Nguyen thichthat@gmail.com 2025-01-08T16:09:20+01:00 GitHub noreply@github.com 2025-01-08T16:09:20+01:00 ci : use actions from ggml-org (#11140) 4d2b3d88041705b20c30b3219838aa435e7ffbde c07d437bbd417f42b122e767ad42b3298767dca0 Xuan Son Nguyen thichthat@gmail.com 2025-01-08T15:59:53+01:00 GitHub noreply@github.com 2025-01-08T15:59:53+01:00 lora : improve compat with `mergekit-extract-lora` (#11131) c07d437bbd417f42b122e767ad42b3298767dca0 99a3755a3c518119d0156766122f7b4b796ea576 Georgi Gerganov ggerganov@gmail.com 2025-01-08T16:19:36+02:00 GitHub noreply@github.com 2025-01-08T16:19:36+02:00 llama : avoid hardcoded QK_K (#11061) 99a3755a3c518119d0156766122f7b4b796ea576 c792dcf4880461c2b5f3960584db241ac71a893a Georgi Gerganov ggerganov@gmail.com 2025-01-08T13:40:30+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-08T13:40:30+02:00 sync : ggml c792dcf4880461c2b5f3960584db241ac71a893a 80ccf5d725571035b454659e3c1b4b2b07b65e71 Radoslav Gerganov rgerganov@gmail.com 2025-01-05T09:50:37+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-08T13:40:18+02:00 ggml : allow loading backend with env variable (ggml/1059) 80ccf5d725571035b454659e3c1b4b2b07b65e71 a3c1232c3f475f0a77b9cc5225516ac31c567a06 Xuan Son Nguyen thichthat@gmail.com 2025-01-08T12:07:20+01:00 GitHub noreply@github.com 2025-01-08T12:07:20+01:00 ci : pin dependency to specific version (#11137) a3c1232c3f475f0a77b9cc5225516ac31c567a06 8cef75c743ba13ebbd6d380c531200c768a8b8aa Georgi Gerganov ggerganov@gmail.com 2025-01-08T12:55:36+02:00 GitHub noreply@github.com 2025-01-08T12:55:36+02:00 arg : option to exclude arguments from specific examples (#11136) 8cef75c743ba13ebbd6d380c531200c768a8b8aa 0d52a69e4bf0d6181beec7853307bdcdeec9905b amritahs-ibm amritahs@linux.vnet.ibm.com 2025-01-08T16:24:19+05:30 GitHub noreply@github.com 2025-01-08T12:54:19+02:00 llamafile : ppc64le MMA INT8 implementation (#10912) 0d52a69e4bf0d6181beec7853307bdcdeec9905b 02f04301417e7fb44fa1025bc1b0aef866e2ca89 Georgi Gerganov ggerganov@gmail.com 2025-01-08T11:29:34+02:00 GitHub noreply@github.com 2025-01-08T11:29:34+02:00 ci : fix cmake option (#11125) 02f04301417e7fb44fa1025bc1b0aef866e2ca89 bec2183f2c8d37cf1278c11d1adb9311e9eaa242 Mathieu Baudier mbaudier@argeo.org 2025-01-08T09:18:13+01:00 GitHub noreply@github.com 2025-01-08T09:18:13+01:00 Disable GL_KHR_cooperative_matrix Vulkan extension if not available. (#11117) bec2183f2c8d37cf1278c11d1adb9311e9eaa242 53ff6b9b9fb25ed0ec0a213e05534fe7c3d0040f ag2s20150909 19373730+ag2s20150909@users.noreply.github.com 2025-01-08T16:17:29+08:00 GitHub noreply@github.com 2025-01-08T09:17:29+01:00 fix: Vulkan shader gen binary path when Cross-compiling (#11096) 53ff6b9b9fb25ed0ec0a213e05534fe7c3d0040f 017cc5f446863316d05522a87f25ec48713a9492 Johannes Gäßler johannesg@5d6.de 2025-01-07T18:01:58+01:00 GitHub noreply@github.com 2025-01-07T18:01:58+01:00 GGUF: C++ refactor, backend support, misc fixes (#11030) 017cc5f446863316d05522a87f25ec48713a9492 a3d50bc022bedd6c7754c24749a1fef4d2d60c7c Diego Devesa slarengh@gmail.com 2025-01-07T16:11:57+01:00 GitHub noreply@github.com 2025-01-07T16:11:57+01:00 ggml-backend : only offload from host buffers (fix) (#11124) a3d50bc022bedd6c7754c24749a1fef4d2d60c7c a4dd490069a66ae56b42127048f06757fc4de4f7 Diego Devesa slarengh@gmail.com 2025-01-07T12:38:05+01:00 GitHub noreply@github.com 2025-01-07T12:38:05+01:00 ggml-backend : only offload from host buffers (#11120) a4dd490069a66ae56b42127048f06757fc4de4f7 c0d6f790d07aa78be15584ec394ac20739ade93b Radoslav Gerganov rgerganov@gmail.com 2025-01-07T08:37:02+02:00 GitHub noreply@github.com 2025-01-07T08:37:02+02:00 rpc : code cleanup (#11107) c0d6f790d07aa78be15584ec394ac20739ade93b dc7cef9f373f2a24b851f0df7a618c5209e593fa Akarshan Biswas akarshan.biswas@gmail.com 2025-01-07T11:56:07+05:30 GitHub noreply@github.com 2025-01-07T14:26:07+08:00 SYCL: Use get_multi_ptr instead of deprecated get_pointer in wkv6 (#11087) dc7cef9f373f2a24b851f0df7a618c5209e593fa ecebbd292d741ac084cf248146b2cfb17002aa1d Eric Curtin ecurtin@redhat.com 2025-01-06T22:45:28Z GitHub noreply@github.com 2025-01-06T23:45:28+01:00 llama-run : fix context size (#11094) ecebbd292d741ac084cf248146b2cfb17002aa1d 96be8c32649378a23031630a48c440f3a5d0839b Georgi Gerganov ggerganov@gmail.com 2025-01-06T17:52:35+02:00 GitHub noreply@github.com 2025-01-06T17:52:35+02:00 llama : remove unused headers (#11109) 96be8c32649378a23031630a48c440f3a5d0839b e6e7c75d94adf4d39e846d30807c531ff22865e7 Xuan Son Nguyen thichthat@gmail.com 2025-01-06T16:34:49+01:00 GitHub noreply@github.com 2025-01-06T16:34:49+01:00 github : add cmd line field to bug report (#11090) e6e7c75d94adf4d39e846d30807c531ff22865e7 09186fabbe05236f2b9446ba6c643cb737540d10 Georgi Gerganov ggerganov@gmail.com 2025-01-06T15:36:08+02:00 GitHub noreply@github.com 2025-01-06T15:36:08+02:00 server : fix extra BOS in infill endpoint (#11106) 09186fabbe05236f2b9446ba6c643cb737540d10 96a1dc27c3f09bf1ed83a26292d571795bcf27fa Xuan Son Nguyen thichthat@gmail.com 2025-01-06T13:41:12+01:00 GitHub noreply@github.com 2025-01-06T13:41:12+01:00 llama : remove check flash_attn with lora (#11104) 96a1dc27c3f09bf1ed83a26292d571795bcf27fa 6369f867a410416239d9f20ec27c2b1d6a9fee52 Asghar Ghorbani a-ghorbani@users.noreply.github.com 2025-01-06T12:21:46+01:00 GitHub noreply@github.com 2025-01-06T13:21:46+02:00 llama : prevent system info string accumulation across calls (#11101) 6369f867a410416239d9f20ec27c2b1d6a9fee52 47182dd03fe04a4ffda5d7f4c8a109ae0056cf56 Daniel Bevenius daniel.bevenius@gmail.com 2025-01-06T10:28:17+01:00 GitHub noreply@github.com 2025-01-06T11:28:17+02:00 llama : rename missed batch params/vars to ubatch (#10059) 47182dd03fe04a4ffda5d7f4c8a109ae0056cf56 3e6e7a6bc2c4b980a0cf0fcb5cb3b79a965b5f14 Georgi Gerganov ggerganov@gmail.com 2025-01-06T10:55:18+02:00 GitHub noreply@github.com 2025-01-06T10:55:18+02:00 llama : update llama_model API names (#11063) 3e6e7a6bc2c4b980a0cf0fcb5cb3b79a965b5f14 ae2f606bb598b287f5fb69c9fdfc98b86598c6cc Georgi Gerganov ggerganov@gmail.com 2025-01-06T10:54:25+02:00 GitHub noreply@github.com 2025-01-06T10:54:25+02:00 tokenize : escape the prompt (#11058) ae2f606bb598b287f5fb69c9fdfc98b86598c6cc 727368c60f2ebf2d6a7473a4a9f80957ab063a8e Georgi Gerganov ggerganov@gmail.com 2025-01-06T10:52:38+02:00 GitHub noreply@github.com 2025-01-06T10:52:38+02:00 mmap : fix fileno macro clash (#11076) 727368c60f2ebf2d6a7473a4a9f80957ab063a8e 5047dd3546951dea3d65c02257d06c46c8662338 Georgi Gerganov ggerganov@gmail.com 2025-01-06T10:52:15+02:00 GitHub noreply@github.com 2025-01-06T10:52:15+02:00 llama : use LLAMA_TOKEN_NULL (#11062) 5047dd3546951dea3d65c02257d06c46c8662338 46e3556e01b824e52395fb050b29804b6cff2a7c Georgi Gerganov ggerganov@gmail.com 2025-01-06T10:52:01+02:00 GitHub noreply@github.com 2025-01-06T10:52:01+02:00 llama : use _impl suffix instead of _internal (#11060) 46e3556e01b824e52395fb050b29804b6cff2a7c b56f079e28fda692f11a8b59200ceb815b05d419 Johannes Gäßler johannesg@5d6.de 2025-01-06T02:33:52+01:00 GitHub noreply@github.com 2025-01-06T02:33:52+01:00 CUDA: add BF16 support (#11093) b56f079e28fda692f11a8b59200ceb815b05d419 9394bbd484f802ce80d2858033583af3ef700d25 0cc4m picard12@live.de 2025-01-04T21:09:59+01:00 GitHub noreply@github.com 2025-01-04T21:09:59+01:00 Vulkan: Add device-specific blacklist for coopmat for the AMD proprietary driver (#11074) 9394bbd484f802ce80d2858033583af3ef700d25 f922a9c542ee117550a168395c63ea79261f5c99 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2025-01-04T21:06:11+01:00 GitHub noreply@github.com 2025-01-04T21:06:11+01:00 llama : Add support for DeepSeek V3 (#11049) f922a9c542ee117550a168395c63ea79261f5c99 46be942214e295cd34660bbbd6b846155d1c36a0 matt23654 matthew.webber@protonmail.com 2025-01-04T16:10:30Z GitHub noreply@github.com 2025-01-04T17:10:30+01:00 [GGML][RPC] Support for models with non-512-aligned tensors over RPC. (#11047) 46be942214e295cd34660bbbd6b846155d1c36a0 78c678517530d411b4263341cdb4dc28c9d117c8 DAN™ dranger003@gmail.com 2025-01-04T09:33:31-05:00 GitHub noreply@github.com 2025-01-04T16:33:31+02:00 llama : add support for the cohere2 model architecture (#10900) 78c678517530d411b4263341cdb4dc28c9d117c8 5e3b08d606b5b0caaea16541b504c3bba8f3ec1d Georgi Gerganov ggerganov@gmail.com 2025-01-04T10:54:01+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-04T16:09:53+02:00 sync : ggml 5e3b08d606b5b0caaea16541b504c3bba8f3ec1d db68c93b57bfdf6da1fbdae81080382d6998cbc9 Georgi Gerganov ggerganov@gmail.com 2025-01-04T10:53:54+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-04T16:09:53+02:00 ggml : do not install metal source when embed library (ggml/1054) db68c93b57bfdf6da1fbdae81080382d6998cbc9 c31fc8b966817b2f0b277fd28e04a189e388972a Daniel Bevenius daniel.bevenius@gmail.com 2024-12-19T03:50:12+01:00 Georgi Gerganov ggerganov@gmail.com 2025-01-04T16:09:53+02:00 ggml : improve inputs log sched_print_assignments (ggml/1053) c31fc8b966817b2f0b277fd28e04a189e388972a 4b0c638b9a68f577cb2066b638c9f622d91ee661 Gilad S. 7817232+giladgd@users.noreply.github.com 2025-01-04T10:17:31+02:00 GitHub noreply@github.com 2025-01-04T09:17:31+01:00 fix: Vulkan shader gen binary path (#11037) 4b0c638b9a68f577cb2066b638c9f622d91ee661 e7da954eccdf39ee795a6135bdb86f0978902681 Molly Sophia mollysophia379@gmail.com 2025-01-03T20:13:18+08:00 GitHub noreply@github.com 2025-01-03T14:13:18+02:00 common : disable KV cache shifting automatically for unsupported models (#11053) e7da954eccdf39ee795a6135bdb86f0978902681 f66f5829276650cd83a087ab2cfed1a760183ea1 Georgi Gerganov ggerganov@gmail.com 2025-01-03T11:26:14+02:00 Georgi Gerganov ggerganov@gmail.com 2025-01-03T11:26:14+02:00 metal : avoid uint (#11019) f66f5829276650cd83a087ab2cfed1a760183ea1 2f0ee84b9b02d2a98742308026f060ebdc2423f1 Georgi Gerganov ggerganov@gmail.com 2025-01-03T10:18:53+02:00 GitHub noreply@github.com 2025-01-03T10:18:53+02:00 llama : refactor `src/llama.cpp` (#10902) 2f0ee84b9b02d2a98742308026f060ebdc2423f1 0da5d860266c6928b8c9408efbd264ae59fedda6 Pierrick Hymbert pierrick.hymbert@gmail.com 2025-01-02T18:06:12+01:00 GitHub noreply@github.com 2025-01-02T18:06:12+01:00 server: bench: minor fixes (#10765) 0da5d860266c6928b8c9408efbd264ae59fedda6 a45433ba209ee0b33d02c7dc4c31f29894ad83a6 Xuan Son Nguyen thichthat@gmail.com 2025-01-02T15:05:18+01:00 GitHub noreply@github.com 2025-01-02T15:05:18+01:00 server : allow using LoRA adapters per-request (#10994) a45433ba209ee0b33d02c7dc4c31f29894ad83a6 0827b2c1da299805288abbd556d869318f2b121e Benson Wong mostlygeek@gmail.com 2025-01-01T23:14:54-08:00 GitHub noreply@github.com 2025-01-02T09:14:54+02:00 readme : add llama-swap to infrastructure section (#11032) 0827b2c1da299805288abbd556d869318f2b121e 45095a61bfd164e87563a0dc0fbd7b0e9891590b Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-12-31T19:53:33+05:30 GitHub noreply@github.com 2024-12-31T15:23:33+01:00 ggml : fixes for AVXVNNI instruction set with MSVC and Clang (#11027) 45095a61bfd164e87563a0dc0fbd7b0e9891590b 5896c65232c7dc87d78426956b16f63fbf58dcf6 Xuan Son Nguyen thichthat@gmail.com 2024-12-31T15:22:01+01:00 GitHub noreply@github.com 2024-12-31T15:22:01+01:00 server : clean up built-in template detection (#11026) 5896c65232c7dc87d78426956b16f63fbf58dcf6 bc7b1f86324279a3dabb705c04ad754a2b27df16 Xuan Son Nguyen thichthat@gmail.com 2024-12-31T12:34:13+01:00 GitHub noreply@github.com 2024-12-31T12:34:13+01:00 server : add OAI compat for /v1/completions (#10974) bc7b1f86324279a3dabb705c04ad754a2b27df16 6e1531aca5ed17f078973b4700fcdadbda4a34a5 ymcki 84055651+ymcki@users.noreply.github.com 2024-12-31T19:04:48+08:00 GitHub noreply@github.com 2024-12-31T13:04:48+02:00 convert : fix Llama-3_1-Nemotron-51B rope settings (#11008) 6e1531aca5ed17f078973b4700fcdadbda4a34a5 716bd6dec3e044e5c325386b5b0483392b24cefe Peter peter277@users.noreply.github.com 2024-12-31T11:46:06+11:00 GitHub noreply@github.com 2024-12-31T01:46:06+01:00 common, examples, ggml : fix MSYS2 GCC compiler errors and warnings when building with LLAMA_CURL=ON and GGML_OPENCL=ON (#11013) 716bd6dec3e044e5c325386b5b0483392b24cefe c250ecb3157f3bae0a45f44c3c953b5414d4c2f7 Jeff Bolz jbolz@nvidia.com 2024-12-30T11:27:11-06:00 GitHub noreply@github.com 2024-12-30T18:27:11+01:00 vulkan: optimize mul_mat for small values of N (#10991) c250ecb3157f3bae0a45f44c3c953b5414d4c2f7 a813badbbdf0d38705f249df7a0c99af5cdee678 ag2s20150909 19373730+ag2s20150909@users.noreply.github.com 2024-12-30T20:35:13+08:00 GitHub noreply@github.com 2024-12-30T14:35:13+02:00 android : fix llama_batch free (#11014) a813badbbdf0d38705f249df7a0c99af5cdee678 fdd21889123bec62b1db3b2fc22b5a4abab32174 Jeff Bolz jbolz@nvidia.com 2024-12-29T03:16:34-06:00 GitHub noreply@github.com 2024-12-29T10:16:34+01:00 vulkan: im2col and matmul optimizations for stable diffusion (#10942) fdd21889123bec62b1db3b2fc22b5a4abab32174 f865ea149d71ef883e3780fced8a20a1464eccf4 Jeff Bolz jbolz@nvidia.com 2024-12-29T02:35:11-06:00 GitHub noreply@github.com 2024-12-29T09:35:11+01:00 vulkan: Use push constant offset to handle misaligned descriptors (#10987) f865ea149d71ef883e3780fced8a20a1464eccf4 16cdce7b68218959e0658e2f95b4572573d5008e Isaac McFadyen isaac@imcf.me 2024-12-28T10:09:19-05:00 GitHub noreply@github.com 2024-12-28T16:09:19+01:00 server: added more docs for response_fields field (#10995) 16cdce7b68218959e0658e2f95b4572573d5008e d79d8f39b4da6deca4aea8bf130c6034c482b320 Alexey Parfenov zxed@alkatrazstudio.net 2024-12-28T15:08:54Z GitHub noreply@github.com 2024-12-28T16:08:54+01:00 server : fix token duplication when streaming with stop strings (#10997) d79d8f39b4da6deca4aea8bf130c6034c482b320 d283d02bf254a7f2991e1502066330cc0d4321a6 Eve 139727413+netrunnereve@users.noreply.github.com 2024-12-26T10:54:44-05:00 GitHub noreply@github.com 2024-12-26T16:54:44+01:00 vulkan: multi-row k quants (#10846) d283d02bf254a7f2991e1502066330cc0d4321a6 9ba399dfa7f115effc63d48e6860a94c9faa31b2 Peter peter277@users.noreply.github.com 2024-12-27T00:59:11+11:00 GitHub noreply@github.com 2024-12-26T14:59:11+01:00 examples, ggml : fix GCC compiler warnings (#10983) 9ba399dfa7f115effc63d48e6860a94c9faa31b2 2cd43f4900ba0e34124fdcbf02a7f9df25a10a3d Reza Kakhki rezakakhki.de@gmail.com 2024-12-24T21:33:04+01:00 GitHub noreply@github.com 2024-12-24T21:33:04+01:00 server : add support for "encoding_format": "base64" to the */embeddings endpoints (#10967) 2cd43f4900ba0e34124fdcbf02a7f9df25a10a3d 09fe2e76137dde850b13313f720e7ffa17efdefa Djip007 3705339+Djip007@users.noreply.github.com 2024-12-24T18:54:49+01:00 GitHub noreply@github.com 2024-12-24T18:54:49+01:00 ggml : more perfo with llamafile tinyblas on x86_64 (#10714) 09fe2e76137dde850b13313f720e7ffa17efdefa 30caac3a68a54de8396b21e20ba972554c587230 NeverLucky 92274250+nvrxq@users.noreply.github.com 2024-12-24T19:39:49+03:00 GitHub noreply@github.com 2024-12-24T17:39:49+01:00 server: allow filtering llama server response fields (#10940) 30caac3a68a54de8396b21e20ba972554c587230 60cfa728e27c28537657d4e627ed432508eb9537 Georgi Gerganov ggerganov@gmail.com 2024-12-24T09:44:20+02:00 GitHub noreply@github.com 2024-12-24T09:44:20+02:00 llama : the WPM vocabs use the CLS token as BOS (#10930) 60cfa728e27c28537657d4e627ed432508eb9537 3327bb0f8dea381118f8e66c18ea14db56d3b942 Diego Devesa slarengh@gmail.com 2024-12-24T04:05:27+01:00 GitHub noreply@github.com 2024-12-24T04:05:27+01:00 ggml : use wstring for backend search paths (#10960) 3327bb0f8dea381118f8e66c18ea14db56d3b942 32d6ee6385b3fc908b283f509b845f757a6e7206 Diego Devesa slarengh@gmail.com 2024-12-24T04:05:17+01:00 GitHub noreply@github.com 2024-12-24T04:05:17+01:00 ggml : fix arm enabled features check (#10961) 32d6ee6385b3fc908b283f509b845f757a6e7206 14b699ecde8f1e9e251ebff9eca39ebc5603b83b Diego Devesa slarengh@gmail.com 2024-12-23T20:25:52+01:00 GitHub noreply@github.com 2024-12-23T20:25:52+01:00 ggml : fix const usage in SSE path (#10962) 14b699ecde8f1e9e251ebff9eca39ebc5603b83b 485dc01214f266afff7004bc702498b491abc404 Xuan Son Nguyen thichthat@gmail.com 2024-12-23T12:52:25+01:00 GitHub noreply@github.com 2024-12-23T12:52:25+01:00 server : fix missing model id in /model endpoint (#10957) 485dc01214f266afff7004bc702498b491abc404 86bf31cfe684849157f0875b4f0ebccac7034547 Xuan Son Nguyen thichthat@gmail.com 2024-12-23T12:02:44+01:00 GitHub noreply@github.com 2024-12-23T12:02:44+01:00 server : add system_fingerprint to chat/completion (#10917) 86bf31cfe684849157f0875b4f0ebccac7034547 b92a14a841fb4dfaf27b29d982ec8ba5289a3bff Radoslav Gerganov rgerganov@gmail.com 2024-12-23T10:39:30+02:00 GitHub noreply@github.com 2024-12-23T10:39:30+02:00 rpc-server : add support for the SYCL backend (#10934) b92a14a841fb4dfaf27b29d982ec8ba5289a3bff 6f0c9e034bb398915a6617ee4acc62adb87d387d Yun Dou dixyes@gmail.com 2024-12-23T08:35:44+08:00 GitHub noreply@github.com 2024-12-23T01:35:44+01:00 llama : support InfiniAI Megrez 3b (#10893) 6f0c9e034bb398915a6617ee4acc62adb87d387d dab76c92cc63072d9495ba87f2f3f3a4872d4f57 ymcki 84055651+ymcki@users.noreply.github.com 2024-12-23T08:22:33+08:00 GitHub noreply@github.com 2024-12-23T01:22:33+01:00 llama : support for Llama-3_1-Nemotron-51B (#10669) dab76c92cc63072d9495ba87f2f3f3a4872d4f57 7024d59e6a572730626cb11896829d115043a1b1 Eric Curtin ecurtin@redhat.com 2024-12-23T00:21:40Z GitHub noreply@github.com 2024-12-23T01:21:40+01:00 llama-run : include temperature option (#10899) 7024d59e6a572730626cb11896829d115043a1b1 7c0e28585843b366864b43b48f92425e2ea17df6 yuri@FreeBSD yurivict@users.noreply.github.com 2024-12-22T16:20:11-08:00 GitHub noreply@github.com 2024-12-23T01:20:11+01:00 ggml : fix run-time on FreeBSD in get_executable_path() (#10948) 7c0e28585843b366864b43b48f92425e2ea17df6 7ae33a616f44ecc081f3dcb589be20962d1d4a92 Rudi Servo rudiservo@gmail.com 2024-12-22T21:22:58-01:00 GitHub noreply@github.com 2024-12-22T23:22:58+01:00 devops : add docker-multi-stage builds (#10832) 7ae33a616f44ecc081f3dcb589be20962d1d4a92 ebdee9478ca7ba65497b9b96f7457698c6ee5115 Billel Mokeddem billel.mokeddem.ml@gmail.com 2024-12-23T01:09:58+03:00 GitHub noreply@github.com 2024-12-23T00:09:58+02:00 llama : add Falcon3 support (#10883) ebdee9478ca7ba65497b9b96f7457698c6ee5115 5cd85b5e008de2ec398d6596e240187d627561e3 Jeff Bolz jbolz@nvidia.com 2024-12-22T03:44:01-06:00 GitHub noreply@github.com 2024-12-22T10:44:01+01:00 vulkan: build fixes for 32b (#10927) 5cd85b5e008de2ec398d6596e240187d627561e3 a91a41364b25705dbb81ae996bc35c3440c63b35 Georgi Gerganov ggerganov@gmail.com 2024-12-21T10:10:18+02:00 GitHub noreply@github.com 2024-12-21T10:10:18+02:00 convert : add BertForMaskedLM (#10919) a91a41364b25705dbb81ae996bc35c3440c63b35 e34c5af43f941f0ddb92466776339897295aca11 Jeff Bolz jbolz@nvidia.com 2024-12-21T01:04:45-06:00 GitHub noreply@github.com 2024-12-21T08:04:45+01:00 vulkan: optimize coopmat2 dequant functions (#10855) e34c5af43f941f0ddb92466776339897295aca11 eb5c3dc64bd967f2e23c87d9dec195f45468de60 Adrien Gallouët angt@huggingface.co 2024-12-21T00:33:37+01:00 GitHub noreply@github.com 2024-12-21T00:33:37+01:00 ggml-cpu: replace NEON asm with intrinsics in ggml_gemv_q4_0_4x8_q8_0() (#10874) eb5c3dc64bd967f2e23c87d9dec195f45468de60 0ca416c91aea4549d9c77e3efeca403e15aa6c75 Akarshan Biswas akarshan.biswas@gmail.com 2024-12-20T21:01:28+05:30 GitHub noreply@github.com 2024-12-20T23:31:28+08:00 SYCL: Migrate away from deprecated ggml_tensor->backend (#10840) 0ca416c91aea4549d9c77e3efeca403e15aa6c75 21ae3b9be83820565d1a720999b7f63ce95b4920 Xuan Son Nguyen thichthat@gmail.com 2024-12-20T14:12:06+01:00 GitHub noreply@github.com 2024-12-20T14:12:06+01:00 server : (UI) fix copy to clipboard function (#10916) 21ae3b9be83820565d1a720999b7f63ce95b4920 0a11f8b7b5c39fdf6e91ef9674bc68ff08681af7 Diego Devesa slarengh@gmail.com 2024-12-20T13:31:28+01:00 GitHub noreply@github.com 2024-12-20T13:31:28+01:00 ggml : add test for SVE and disable when it fails (#10906) 0a11f8b7b5c39fdf6e91ef9674bc68ff08681af7 d408bb9268a988c5a60a5746d3a6430386e7604d Molly Sophia mollysophia379@gmail.com 2024-12-20T17:44:58+08:00 GitHub noreply@github.com 2024-12-20T11:44:58+02:00 convert : fix RWKV v6 model conversion (#10913) d408bb9268a988c5a60a5746d3a6430386e7604d 5cab3e4aaa892df4620b720f20a503a1bbbe7a52 Georgi Gerganov ggerganov@gmail.com 2024-12-19T18:47:15+02:00 GitHub noreply@github.com 2024-12-19T18:47:15+02:00 clip : disable GPU support (#10896) 5cab3e4aaa892df4620b720f20a503a1bbbe7a52 36319dec5d75a7dfe3e3de37b9ca2a76cd52b7b2 Georgi Gerganov ggerganov@gmail.com 2024-12-19T17:42:13+02:00 GitHub noreply@github.com 2024-12-19T17:42:13+02:00 llama : minor grammar refactor (#10897) 36319dec5d75a7dfe3e3de37b9ca2a76cd52b7b2 57bb2c40cd94c5a09f5210ed8264cc93b21c4b7e Georgi Gerganov ggerganov@gmail.com 2024-12-19T17:35:15+02:00 GitHub noreply@github.com 2024-12-19T17:35:15+02:00 tts : small QoL for easy model fetch (#10903) 57bb2c40cd94c5a09f5210ed8264cc93b21c4b7e a3c33b1dce2d4f25040b75f66629104bd1e40128 Xuan Son Nguyen thichthat@gmail.com 2024-12-19T15:40:08+01:00 GitHub noreply@github.com 2024-12-19T15:40:08+01:00 server : fix logprobs, make it OAI-compatible (#10783) a3c33b1dce2d4f25040b75f66629104bd1e40128 2fffc52b50992ac5bc64db19d33c39cbc06f52cf Adrien Gallouët angt@huggingface.co 2024-12-19T14:20:41+01:00 GitHub noreply@github.com 2024-12-19T14:20:41+01:00 ggml: fix arm build with gcc (#10895) 2fffc52b50992ac5bc64db19d33c39cbc06f52cf 7585edbdebd02861e0994dae67c9338731fb3fc5 Sukriti Sharma Ssukriti@users.noreply.github.com 2024-12-19T06:04:51-07:00 GitHub noreply@github.com 2024-12-19T15:04:51+02:00 llama : fix Roberta embeddings (#10856) 7585edbdebd02861e0994dae67c9338731fb3fc5 cd920d0ac38ec243605a5a57c50941140a193f9e fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-12-19T10:37:12+01:00 GitHub noreply@github.com 2024-12-19T10:37:12+01:00 convert : Add support for Microsoft Phi-4 model (#10817) cd920d0ac38ec243605a5a57c50941140a193f9e 7909e8588ddf70820adf1f325490eb3f67b32875 Johannes Gäßler johannesg@5d6.de 2024-12-19T08:53:58+01:00 GitHub noreply@github.com 2024-12-19T08:53:58+01:00 tests: disable GGUF test for bad value size (#10886) 7909e8588ddf70820adf1f325490eb3f67b32875 9177484f589d770ffc4e655b9819124d6a22c1d9 Eric Curtin ecurtin@redhat.com 2024-12-19T02:58:00Z GitHub noreply@github.com 2024-12-19T03:58:00+01:00 llama-run : improve progress bar (#10821) 9177484f589d770ffc4e655b9819124d6a22c1d9 0bf2d10c5514ff61b99897a4a5054f846e384e1e Diego Devesa slarengh@gmail.com 2024-12-18T23:21:42+01:00 GitHub noreply@github.com 2024-12-18T23:21:42+01:00 ggml : fix arm build (#10890) 0bf2d10c5514ff61b99897a4a5054f846e384e1e 7bbb5acf125d1d2840cac7d31b9aaa72210dd5ec Georgi Gerganov ggerganov@gmail.com 2024-12-18T19:27:21+02:00 GitHub noreply@github.com 2024-12-18T19:27:21+02:00 tts : add OuteTTS support (#10784) 7bbb5acf125d1d2840cac7d31b9aaa72210dd5ec 152610eda91217ac409342cd976d05f5114ad39f Gaetan Bisson gaetan@fenua.org 2024-12-18T04:00:07-10:00 GitHub noreply@github.com 2024-12-18T15:00:07+01:00 server: avoid overwriting Authorization header (#10878) 152610eda91217ac409342cd976d05f5114ad39f 0e70ba686e6c717a0aa41d88284e2a392c2bd0cd Georgi Gerganov ggerganov@gmail.com 2024-12-18T13:01:41+02:00 GitHub noreply@github.com 2024-12-18T13:01:41+02:00 server : output embeddings for all tokens when pooling = none (#10861) 0e70ba686e6c717a0aa41d88284e2a392c2bd0cd 46828872c31b25df16169cbbf5c2225fa9cb0675 Georgi Gerganov ggerganov@gmail.com 2024-12-18T11:05:29+02:00 GitHub noreply@github.com 2024-12-18T11:05:29+02:00 server : add "tokens" output (#10853) 46828872c31b25df16169cbbf5c2225fa9cb0675 6b064c92b4c0228e333193c167f2c98d2ec8d9bc Xuan Son Nguyen thichthat@gmail.com 2024-12-18T09:55:09+01:00 GitHub noreply@github.com 2024-12-18T10:55:09+02:00 server : (embeddings) using same format for "input" and "content" (#10872) 6b064c92b4c0228e333193c167f2c98d2ec8d9bc 4da69d1abd15263061aff94c10f205836a96a4bc redbeard bharrington@alticon.net 2024-12-18T00:35:00-08:00 GitHub noreply@github.com 2024-12-18T10:35:00+02:00 docs: Fix HIP (née hipBLAS) in README (#10880) 4da69d1abd15263061aff94c10f205836a96a4bc d62b532c52e0118323277eaa5f442e11ce6505ed Diego Devesa slarengh@gmail.com 2024-12-18T01:36:46+01:00 GitHub noreply@github.com 2024-12-18T01:36:46+01:00 Revert "llama : add Falcon3 support (#10864)" (#10876) d62b532c52e0118323277eaa5f442e11ce6505ed 081b29bd2a3d91e7772e3910ce223dd63b8d7d26 DAN™ dranger003@gmail.com 2024-12-17T17:24:22-05:00 GitHub noreply@github.com 2024-12-17T23:24:22+01:00 Use model->gguf_kv for loading the template instead of using the C API. (#10868) 081b29bd2a3d91e7772e3910ce223dd63b8d7d26 5437d4aaf5132c879acda0bb67f2f8f71da4c9fe Johannes Gäßler johannesg@5d6.de 2024-12-17T19:09:35+01:00 GitHub noreply@github.com 2024-12-17T19:09:35+01:00 tests: add tests for GGUF (#10830) 5437d4aaf5132c879acda0bb67f2f8f71da4c9fe 78f766768d94e9c8b30ce10ca76f568e710d84f7 Georgi Gerganov ggerganov@gmail.com 2024-12-17T18:36:02+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-17T18:36:02+02:00 sync : ggml 78f766768d94e9c8b30ce10ca76f568e710d84f7 8dd19a48129d578972ea3d98896edbbf492891a9 Georgi Gerganov ggerganov@gmail.com 2024-12-17T18:34:32+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-17T18:35:49+02:00 cmake : fix "amd64" processor string (whisper/2638) 8dd19a48129d578972ea3d98896edbbf492891a9 130d0c90bd26b25e3a713b17a61a5d4eb0a66405 gn64 yukikaze.jp@gmail.com 2024-12-16T19:34:38+09:00 Georgi Gerganov ggerganov@gmail.com 2024-12-17T18:35:49+02:00 vulkan : fix soft_max.comp division by zero (whisper/2633) 130d0c90bd26b25e3a713b17a61a5d4eb0a66405 3919da8e335dbfd0741d239932a9b9e72061bf27 Daniel Bevenius daniel.bevenius@gmail.com 2024-12-14T03:23:08+01:00 Georgi Gerganov ggerganov@gmail.com 2024-12-17T18:35:49+02:00 ggml : remove return from ggml_gallocr_allocate_node (ggml/1048) 3919da8e335dbfd0741d239932a9b9e72061bf27 0006f5a74a59945f22ff966e723c3d566b3ca8d0 Daniel Bevenius daniel.bevenius@gmail.com 2024-12-13T08:19:38+01:00 Georgi Gerganov ggerganov@gmail.com 2024-12-17T18:35:48+02:00 ggml : add check for grad_accs (ggml/1046) 0006f5a74a59945f22ff966e723c3d566b3ca8d0 05c3a444b8b017b01b366b725ba22c740aae6b38 Georgi Gerganov ggerganov@gmail.com 2024-12-17T18:35:42+02:00 GitHub noreply@github.com 2024-12-17T18:35:42+02:00 ggml : update ggml_backend_cpu_device_supports_op (#10867) 05c3a444b8b017b01b366b725ba22c740aae6b38 382bc7f2e8ffd0b89f23e840d097e21f301197ba krystiancha krystian@krystianch.com 2024-12-17T16:00:24Z GitHub noreply@github.com 2024-12-17T18:00:24+02:00 server : fill usage info in embeddings and rerank responses (#10852) 382bc7f2e8ffd0b89f23e840d097e21f301197ba 4f51968aca049080dc77e26603aa0681ea77fe45 Billel Mokeddem billel.mokeddem.ml@gmail.com 2024-12-17T19:24:56+04:00 GitHub noreply@github.com 2024-12-17T17:24:56+02:00 llama : add Falcon3 support (#10864) 4f51968aca049080dc77e26603aa0681ea77fe45 227d7c5a7f4cc7734fde8a4ef4382d613486d3c8 Ruan 47767371+ruanych@users.noreply.github.com 2024-12-17T17:47:20+08:00 GitHub noreply@github.com 2024-12-17T11:47:20+02:00 readme : update typos (#10863) 227d7c5a7f4cc7734fde8a4ef4382d613486d3c8 7b1ec53f56bb72c49e4c8434157895f94d3709c2 Xuan Son Nguyen thichthat@gmail.com 2024-12-17T09:52:09+01:00 GitHub noreply@github.com 2024-12-17T09:52:09+01:00 server : (UI) fix missing async generator on safari (#10857) 7b1ec53f56bb72c49e4c8434157895f94d3709c2 160bc039c862c10b9938269a90ddb09d794a7b0d Eve 139727413+netrunnereve@users.noreply.github.com 2024-12-17T05:52:55Z GitHub noreply@github.com 2024-12-17T06:52:55+01:00 vulkan: bugfixes for small subgroup size systems + llvmpipe test (#10809) 160bc039c862c10b9938269a90ddb09d794a7b0d 08ea539df211e46bb4d0dd275e541cb591d5ebc8 Zhiyuan Li uniartisan2017@gmail.com 2024-12-17T05:00:46+08:00 GitHub noreply@github.com 2024-12-16T22:00:46+01:00 rwkv6: add wkv6 support for Vulkan backend (#10829) 08ea539df211e46bb4d0dd275e541cb591d5ebc8 644fd71b44c4cdbfc6482fbf0353d289c3bc29e6 Georgi Gerganov ggerganov@gmail.com 2024-12-16T12:31:45+02:00 GitHub noreply@github.com 2024-12-16T12:31:45+02:00 unicode : improve naming style (#10838) 644fd71b44c4cdbfc6482fbf0353d289c3bc29e6 4ddd199f6f6b980e0a7ed9f9b44efeae2fbdf5c4 Georgi Gerganov ggerganov@gmail.com 2024-12-16T12:31:14+02:00 GitHub noreply@github.com 2024-12-16T12:31:14+02:00 sampling : refactor + optimize penalties sampler (#10803) 4ddd199f6f6b980e0a7ed9f9b44efeae2fbdf5c4 a0974156f334acf8af5858d7ede5ab7d7490d415 Bartowski ckealty1182@gmail.com 2024-12-15T15:43:25-05:00 GitHub noreply@github.com 2024-12-15T21:43:25+01:00 llava : Allow locally downloaded models for QwenVL (#10833) a0974156f334acf8af5858d7ede5ab7d7490d415 87cf323cef80f6aa530f047ab05b539ebc6b7e3c Valentin Mamedov 45292985+Inf1delis@users.noreply.github.com 2024-12-16T00:02:46+07:00 GitHub noreply@github.com 2024-12-15T19:02:46+02:00 llama : add Deepseek MoE v1 & GigaChat models (#10827) 87cf323cef80f6aa530f047ab05b539ebc6b7e3c 5478bbcd173e7027af7689493c7421719f5c43df Georgi Gerganov ggerganov@gmail.com 2024-12-15T18:44:47+02:00 GitHub noreply@github.com 2024-12-15T18:44:47+02:00 scripts : change build path to "build-bench" for compare-commits.sh (#10836) 5478bbcd173e7027af7689493c7421719f5c43df b5ae1ddff93403300fc79c7ab5ee73b8cfbb3457 Vinesh Janarthanan 36610342+VJHack@users.noreply.github.com 2024-12-15T05:55:54-06:00 GitHub noreply@github.com 2024-12-15T12:55:54+01:00 server: (UI) add syntax highlighting and latex math rendering (#10808) b5ae1ddff93403300fc79c7ab5ee73b8cfbb3457 89d604f2c87af9db657d8a27a1528bc4b7579c29 Georgi Gerganov ggerganov@gmail.com 2024-12-15T13:16:42+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-15T13:16:42+02:00 gguf-py : bump to v0.13.0 89d604f2c87af9db657d8a27a1528bc4b7579c29 e52aba537a34d51a65cddec6bc6dafc9031edc63 Michelle Tan 41475767+MichelleTanPY@users.noreply.github.com 2024-12-14T22:29:45Z GitHub noreply@github.com 2024-12-14T23:29:45+01:00 server: Fix `has_next_line` in JSON response (#10818) e52aba537a34d51a65cddec6bc6dafc9031edc63 ba1cb19cdd0d92e012e0f6e009e0620f854b6afd Evgeny Kurnevsky kurnevsky@gmail.com 2024-12-14T18:17:36Z GitHub noreply@github.com 2024-12-14T10:17:36-08:00 nix: allow to override rocm gpu targets (#10794) ba1cb19cdd0d92e012e0f6e009e0620f854b6afd 56eea0781cbd2608ed8ff524955495569b9264be HimariO dsfhe49854@gmail.com 2024-12-14T20:43:46+08:00 GitHub noreply@github.com 2024-12-14T14:43:46+02:00 llama : add Qwen2VL support + multimodal RoPE (#10361) 56eea0781cbd2608ed8ff524955495569b9264be a76c56fa1a3d27467eb97468d8c3b2fe1243b61a cduk 19917266+cduk@users.noreply.github.com 2024-12-13T23:21:49+01:00 GitHub noreply@github.com 2024-12-13T23:21:49+01:00 Removes spurious \r in output that causes logging in journalctl to treat lines as binary and therefore hidden by default (#10771) a76c56fa1a3d27467eb97468d8c3b2fe1243b61a c27ac678dd393af0da9b8acf10266e760c8a0912 lhez quic_lih@quicinc.com 2024-12-13T12:23:52-08:00 GitHub noreply@github.com 2024-12-13T12:23:52-08:00 Introducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (#10693) c27ac678dd393af0da9b8acf10266e760c8a0912 11e07fd63bac1cb642380a7a3eac03fd8703e948 Eric Curtin ecurtin@redhat.com 2024-12-13T18:34:25Z GitHub noreply@github.com 2024-12-13T19:34:25+01:00 Opt class for positional argument handling (#10508) 11e07fd63bac1cb642380a7a3eac03fd8703e948 4601a8bb6784d2ab8b4b605354b51979fbeea1d3 Corentin REGAL corentin.regal@gmail.com 2024-12-13T18:23:50+01:00 GitHub noreply@github.com 2024-12-13T18:23:50+01:00 fix: graceful shutdown for Docker images (#10815) 4601a8bb6784d2ab8b4b605354b51979fbeea1d3 9f35e44592a7646a5803620eb6a3f0ed5ac90553 Jett Janiak jettjaniak@gmail.com 2024-12-13T15:48:44+01:00 GitHub noreply@github.com 2024-12-13T16:48:44+02:00 gguf-py : numpy 2 newbyteorder fix (#9772) 9f35e44592a7646a5803620eb6a3f0ed5ac90553 64ae0655114f84f11a724bc6878c6f8f4a55560b 谢乃闻 sienaiwun@users.noreply.github.com 2024-12-13T12:56:07Z GitHub noreply@github.com 2024-12-13T13:56:07+01:00 Fix crash caused by ggml_backend_load_all when launching on Android Activity (#10812) 64ae0655114f84f11a724bc6878c6f8f4a55560b 83ed24a97b500ccdb32b90b94e6f9621ad8db79e Eve 139727413+netrunnereve@users.noreply.github.com 2024-12-13T08:42:04Z GitHub noreply@github.com 2024-12-13T09:42:04+01:00 vulkan: small mul_mat_vec optimizations (#10665) 83ed24a97b500ccdb32b90b94e6f9621ad8db79e d583cd03f663701858ba152a334cbb467fabe342 Akarshan Biswas akarshan.biswas@gmail.com 2024-12-13T12:12:15+05:30 GitHub noreply@github.com 2024-12-13T12:12:15+05:30 SYCL: Reduce most of the compiler warnings (#10748) d583cd03f663701858ba152a334cbb467fabe342 adffa6ffd59997da59f62b72d2b79fc37e085e84 Karol Kontny 82021046+kkontny@users.noreply.github.com 2024-12-13T01:04:19+01:00 GitHub noreply@github.com 2024-12-13T01:04:19+01:00 ggml : Fix compilation issues on ARM platform when building without fp16 (#10811) adffa6ffd59997da59f62b72d2b79fc37e085e84 274ec65af6e54039eb95cb44904af5c945dca1fa Xuan Son Nguyen thichthat@gmail.com 2024-12-12T22:53:05+01:00 GitHub noreply@github.com 2024-12-12T22:53:05+01:00 common : improve -ctv -ctk CLI arguments (#10806) 274ec65af6e54039eb95cb44904af5c945dca1fa 8faa1d4dd42f6cb26088ce7f5bbca5996b921685 Xuan Son Nguyen thichthat@gmail.com 2024-12-12T20:52:28+01:00 GitHub noreply@github.com 2024-12-12T20:52:28+01:00 contrib : add ngxson as codeowner (#10804) 8faa1d4dd42f6cb26088ce7f5bbca5996b921685 cb13ef85a444eb52a3f1b82dce198ceb25606583 a3sh 38979186+A3shTnT@users.noreply.github.com 2024-12-13T02:09:50+08:00 GitHub noreply@github.com 2024-12-12T19:09:50+01:00 CUDA: faster non-contiguous concat (#10760) cb13ef85a444eb52a3f1b82dce198ceb25606583 4064c0e3b6c27440f5d12b7caaf90b4088c28c61 Diego Devesa slarengh@gmail.com 2024-12-12T19:02:49+01:00 GitHub noreply@github.com 2024-12-12T19:02:49+01:00 remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (#10797) 4064c0e3b6c27440f5d12b7caaf90b4088c28c61 dc5301d565b7d0b2c691f7df13099aab3997479c 0cc4m picard12@live.de 2024-12-12T18:36:00+01:00 GitHub noreply@github.com 2024-12-12T18:36:00+01:00 Vulkan: Use improved q4_k and q5_k dequant code in dequant shaders (#10798) dc5301d565b7d0b2c691f7df13099aab3997479c 9fdb1243049aa7e8211693f116daf2052d47507d 0cc4m picard12@live.de 2024-12-12T18:35:37+01:00 GitHub noreply@github.com 2024-12-12T18:35:37+01:00 Vulkan: Add VK_EXT_subgroup_size_control support to ensure full subgroups for coopmats (#10721) 9fdb1243049aa7e8211693f116daf2052d47507d 5555c0c1f66d766c544c30699190dec0285bcbfc Xuan Son Nguyen thichthat@gmail.com 2024-12-12T16:57:32+01:00 GitHub noreply@github.com 2024-12-12T16:57:32+01:00 common : add missing env var for speculative (#10801) 5555c0c1f66d766c544c30699190dec0285bcbfc 973f328b1e92a6406030442dfd15b29449e89747 CentricStorm CentricStorm@users.noreply.github.com 2024-12-11T22:40:40Z GitHub noreply@github.com 2024-12-11T23:40:40+01:00 docs: update server streaming mode documentation (#9519) 973f328b1e92a6406030442dfd15b29449e89747 235f6e14bf0ed0211c51aeff14139038ae1000aa fb18934a97425c42c8b32a1baaa94f0080eb051d Georgi Gerganov ggerganov@gmail.com 2024-12-11T23:14:46+02:00 GitHub noreply@github.com 2024-12-11T23:14:46+02:00 Merge pull request #10788 from ggerganov/gg/gguf-py-0.11.0 fb18934a97425c42c8b32a1baaa94f0080eb051d 235f6e14bf0ed0211c51aeff14139038ae1000aa Georgi Gerganov ggerganov@gmail.com 2024-12-11T23:13:31+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-11T23:13:31+02:00 gguf-py : bump version to 0.11.0 235f6e14bf0ed0211c51aeff14139038ae1000aa 1a31d0dc00ba946d448e16ecc915ce5e8355994e Xuan Son Nguyen thichthat@gmail.com 2024-12-11T20:52:14+01:00 GitHub noreply@github.com 2024-12-11T20:52:14+01:00 server : (UI) add tok/s, get rid of completion.js (#10786) 1a31d0dc00ba946d448e16ecc915ce5e8355994e 92f77a640f763c0af73554fb810a85a7d4c85e5e qingy1337 qxli2@students.everettcc.edu 2024-12-11T07:16:32-08:00 GitHub noreply@github.com 2024-12-11T16:16:32+01:00 Update README.md (#10772) 92f77a640f763c0af73554fb810a85a7d4c85e5e 484d2f31aed34ff9f096e3961125762e81d9b7d6 Xuan Son Nguyen thichthat@gmail.com 2024-12-11T14:59:41+01:00 GitHub noreply@github.com 2024-12-11T14:59:41+01:00 ci : pin nodejs to 22.11.0 (#10779) 484d2f31aed34ff9f096e3961125762e81d9b7d6 4b4d92b0986e3b627b9a9ef4782973108bf47691 kallewoof kalle.alm@gmail.com 2024-12-11T22:48:04+09:00 GitHub noreply@github.com 2024-12-11T14:48:04+01:00 bug-fix: snprintf prints NULL in place of the last character (#10419) 4b4d92b0986e3b627b9a9ef4782973108bf47691 43041d2eb32623d5b6ca734313327abe96f73146 CentricStorm CentricStorm@users.noreply.github.com 2024-12-11T10:47:43Z GitHub noreply@github.com 2024-12-11T11:47:43+01:00 docs: fix server documentation formatting (#10776) 43041d2eb32623d5b6ca734313327abe96f73146 b685daf3867c54e42a9db484d7b92619021d4510 Gilad S. 7817232+giladgd@users.noreply.github.com 2024-12-11T02:47:21+02:00 GitHub noreply@github.com 2024-12-11T01:47:21+01:00 ggml: load all backends from a user-provided search path (#10699) b685daf3867c54e42a9db484d7b92619021d4510 dafae66cc242eb766797194d3c85c5e502625623 Jeff Bolz jbolz@nvidia.com 2024-12-10T14:23:17-06:00 GitHub noreply@github.com 2024-12-10T21:23:17+01:00 vulkan: request round-to-even for fp16 in im2col/rope_head (#10767) dafae66cc242eb766797194d3c85c5e502625623 ae4b922614d452477cf5d2fb8cad247c9c12596c Eve 139727413+netrunnereve@users.noreply.github.com 2024-12-10T19:33:23Z GitHub noreply@github.com 2024-12-10T20:33:23+01:00 vulkan: dynamic subgroup size for the remaining k quants (#10745) ae4b922614d452477cf5d2fb8cad247c9c12596c 750cb3e246f7e544124cf18cb0c5e0c8b7e38738 Bartowski ckealty1182@gmail.com 2024-12-10T12:23:50-05:00 GitHub noreply@github.com 2024-12-10T18:23:50+01:00 imatrix : Add imatrix to --no-context-shift (#10766) 750cb3e246f7e544124cf18cb0c5e0c8b7e38738 a86ad841f103e471ac0fd7ee8852d1eb5015ce89 Andreas Kieslinger 47689530+aendk@users.noreply.github.com 2024-12-10T18:23:24+01:00 GitHub noreply@github.com 2024-12-10T18:23:24+01:00 CUDA: rename macros to avoid conflicts with WinAPI (#10736) a86ad841f103e471ac0fd7ee8852d1eb5015ce89 a05e2afcc241c1ecd38ec5cb4c579d90cdf3f918 Yüg eugeniosegalaweb@gmail.com 2024-12-10T17:22:34Z GitHub noreply@github.com 2024-12-10T18:22:34+01:00 server : add flag to disable the web-ui (#10762) (#10751) a05e2afcc241c1ecd38ec5cb4c579d90cdf3f918 26a8406ba9198eb6fdd8329fa717555b4f77f05f Jeff Bolz jbolz@nvidia.com 2024-12-10T11:22:20-06:00 GitHub noreply@github.com 2024-12-10T18:22:20+01:00 vulkan: disable spirv-opt for coopmat shaders (#10763) 26a8406ba9198eb6fdd8329fa717555b4f77f05f c37fb4cf62ddf0d33562c4c4a4d6fb45e32ad3b6 Johannes Gäßler johannesg@5d6.de 2024-12-09T20:07:12+01:00 GitHub noreply@github.com 2024-12-09T20:07:12+01:00 CUDA: fix shared memory access condition for mmv (#10740) c37fb4cf62ddf0d33562c4c4a4d6fb45e32ad3b6 3d98b4cb226c3140bd1ae6c65ed126b7d90332fa Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-12-09T23:10:19+05:30 GitHub noreply@github.com 2024-12-09T09:40:19-08:00 Changes to CMakePresets.json to add ninja clang target on windows (#10668) 3d98b4cb226c3140bd1ae6c65ed126b7d90332fa 1a05004743e00aca400833be625f0ec8cce176a7 Jeff Bolz jbolz@nvidia.com 2024-12-09T01:24:01-06:00 GitHub noreply@github.com 2024-12-09T08:24:01+01:00 vulkan: fix compile warnings (#10731) 1a05004743e00aca400833be625f0ec8cce176a7 ce8784bdb153ff7794dde5a50b0ebfa51baa6171 Borislav Stanimirov b@ibob.bg 2024-12-09T09:15:13+02:00 GitHub noreply@github.com 2024-12-09T09:15:13+02:00 cmake : simplify msvc charsets (#10672) ce8784bdb153ff7794dde5a50b0ebfa51baa6171 e52522b8694ae73abf12feb18d29168674aa1c1b Xuan Son Nguyen thichthat@gmail.com 2024-12-08T23:04:29+01:00 GitHub noreply@github.com 2024-12-08T23:04:29+01:00 server : fix format_infill (#10724) e52522b8694ae73abf12feb18d29168674aa1c1b 06d70147e6480c021e493c442ae0f0d83ae366de Xuan Son Nguyen thichthat@gmail.com 2024-12-08T20:38:51+01:00 GitHub noreply@github.com 2024-12-08T20:38:51+01:00 server : bring back info of final chunk in stream mode (#10722) 06d70147e6480c021e493c442ae0f0d83ae366de 43ed389a3f102517e6f7d5620d8e451e88afbf27 stduhpf stephduh@live.fr 2024-12-08T19:19:19+01:00 GitHub noreply@github.com 2024-12-08T19:19:19+01:00 Vulkan: fix NaN in tanh.comp with AMD proprietary driver on Windows (#10723) 43ed389a3f102517e6f7d5620d8e451e88afbf27 ecc93d0558fc3ecb8a5af69d2ece02fae4710ade Diego Devesa slarengh@gmail.com 2024-12-08T12:14:54+01:00 GitHub noreply@github.com 2024-12-08T13:14:54+02:00 llama : use cmake for swift build (#10525) ecc93d0558fc3ecb8a5af69d2ece02fae4710ade 62e84d984875372f4b0fb89a67658e012ff0cc9f Jeff Bolz jbolz@nvidia.com 2024-12-08T02:05:55-06:00 GitHub noreply@github.com 2024-12-08T09:05:55+01:00 vulkan: compile a test shader in cmake to check for coopmat2 support (#10713) 62e84d984875372f4b0fb89a67658e012ff0cc9f 3573fa8e7b7f0865638b52b4e9b4d2006f0558a2 Robert Collins roberto.tomas.cuentas@gmail.com 2024-12-07T16:12:27-05:00 GitHub noreply@github.com 2024-12-07T23:12:27+02:00 llama : add 128k yarn context for Qwen (#10698) 3573fa8e7b7f0865638b52b4e9b4d2006f0558a2 d9c3ba2b7749c00df477599aa141a98b4521aa2c Xuan Son Nguyen thichthat@gmail.com 2024-12-07T20:21:09+01:00 GitHub noreply@github.com 2024-12-07T20:21:09+01:00 server : (refactor) no more json in server_task input (#10691) d9c3ba2b7749c00df477599aa141a98b4521aa2c ce4a7b849388b67d45ad420bdd82d5efcd55647a Georgi Gerganov ggerganov@gmail.com 2024-12-07T18:38:15+02:00 GitHub noreply@github.com 2024-12-07T18:38:15+02:00 ggml : disable iq4_nl interleave size 8 (#10709) ce4a7b849388b67d45ad420bdd82d5efcd55647a 19d8762ab61df8286367588a80b9c7db4cb568db Georgi Gerganov ggerganov@gmail.com 2024-12-07T18:02:05+02:00 GitHub noreply@github.com 2024-12-07T18:02:05+02:00 server : various fixes (#10704) 19d8762ab61df8286367588a80b9c7db4cb568db c2a16c0bdbe2e51adf318918bad82f0c3e3d6f3b Djip007 3705339+Djip007@users.noreply.github.com 2024-12-07T13:37:50+01:00 GitHub noreply@github.com 2024-12-07T14:37:50+02:00 ggml : refactor online repacking (#10446) c2a16c0bdbe2e51adf318918bad82f0c3e3d6f3b 3df784b3050f657ea681f804187ce5bddb433e88 Georgi Gerganov ggerganov@gmail.com 2024-12-07T11:52:44+02:00 GitHub noreply@github.com 2024-12-07T11:52:44+02:00 server : fix free of spec context and batch (#10651) 3df784b3050f657ea681f804187ce5bddb433e88 86a1934978ce5daffc7e5b4251f6540ee5e7b47b 0cc4m picard12@live.de 2024-12-07T10:24:15+01:00 GitHub noreply@github.com 2024-12-07T10:24:15+01:00 Vulkan: VK_KHR_cooperative_matrix support to speed up prompt processing (#10597) 86a1934978ce5daffc7e5b4251f6540ee5e7b47b 784a14aa496a66cc43e76014a1bf4333b4a2a55a Robert Ormandi 52251610+ormandi@users.noreply.github.com 2024-12-07T01:55:01-06:00 GitHub noreply@github.com 2024-12-07T09:55:01+02:00 metal : Extend how Llama.cpp locates metal resources (#10676) 784a14aa496a66cc43e76014a1bf4333b4a2a55a c5ede3849fc021174862f9c0bf8273808d8f0d39 Sukriti Sharma Ssukriti@users.noreply.github.com 2024-12-07T00:02:14-07:00 GitHub noreply@github.com 2024-12-07T09:02:14+02:00 convert : add support for Roberta embeddings (#10695) c5ede3849fc021174862f9c0bf8273808d8f0d39 f162d45a21b27f7613f14539f9a4932d6ff3ca48 Georgi Gerganov ggerganov@gmail.com 2024-12-06T21:33:15+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-06T21:33:49+02:00 convert : add custom attention mapping f162d45a21b27f7613f14539f9a4932d6ff3ca48 6c5bc0625fae6909cb40def15bc4bb45db6f7f4d Xuan Son Nguyen thichthat@gmail.com 2024-12-06T13:29:05+01:00 GitHub noreply@github.com 2024-12-06T13:29:05+01:00 common : bring back --no-warmup to server (#10686) 6c5bc0625fae6909cb40def15bc4bb45db6f7f4d 7736837d62efed1dbebfe579472fca041eda12d6 Xuan Son Nguyen thichthat@gmail.com 2024-12-06T11:14:32+01:00 GitHub noreply@github.com 2024-12-06T11:14:32+01:00 server : (refactoring) do not rely on JSON internally (#10643) 7736837d62efed1dbebfe579472fca041eda12d6 c9c6e01daedac542b174c235872569fce5385982 Plamen Minev pacominev@gmail.com 2024-12-05T23:36:41+02:00 GitHub noreply@github.com 2024-12-05T22:36:41+01:00 fix(server) : not show alert when DONE is received (#10674) c9c6e01daedac542b174c235872569fce5385982 6fe624783166e7355cec915de0094e63cd3558eb Jeff Bolz jbolz@nvidia.com 2024-12-05T13:15:05-06:00 GitHub noreply@github.com 2024-12-05T20:15:05+01:00 vulkan: Add VK_NV_cooperative_matrix2 support for mul_mat and flash attention (#10206) 6fe624783166e7355cec915de0094e63cd3558eb 0cd182ebcc2c45907240e727e80e8fbf0f5fdee9 Riccardo Orlando Riccorl@users.noreply.github.com 2024-12-05T19:30:59+01:00 GitHub noreply@github.com 2024-12-05T20:30:59+02:00 llama : add Minerva 7B model support (#10673) 0cd182ebcc2c45907240e727e80e8fbf0f5fdee9 a8cbab201dcf4c76c30b8028427494d71f02bb57 Georgi Gerganov ggerganov@gmail.com 2024-12-05T13:27:42+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-05T13:27:42+02:00 sync : ggml a8cbab201dcf4c76c30b8028427494d71f02bb57 c2082d93a82d66dca112098c63775d7a97e6d47b PAB pierreantoine.bannier@gmail.com 2024-12-04T09:19:30+01:00 Georgi Gerganov ggerganov@gmail.com 2024-12-05T13:27:33+02:00 ggml: add `GGML_SET` Metal kernel + i32 CPU kernel (ggml/1037) c2082d93a82d66dca112098c63775d7a97e6d47b d405804be84cfdac936f28b3446ce8cb988408d8 PAB pierreantoine.bannier@gmail.com 2024-12-03T20:20:04+01:00 Georgi Gerganov ggerganov@gmail.com 2024-12-05T13:27:31+02:00 ggml : add `GGML_PAD_REFLECT_1D` operation (ggml/1034) d405804be84cfdac936f28b3446ce8cb988408d8 f112d198cd9953b633ac662c2705d6e423438717 Daniel Bevenius daniel.bevenius@gmail.com 2024-12-05T08:47:55+01:00 GitHub noreply@github.com 2024-12-05T09:47:55+02:00 py : update outdated copy-paste instructions [no ci] (#10667) f112d198cd9953b633ac662c2705d6e423438717 1da7b765692764a8b33b08da61cbee63812a7bd9 aryantandon01 80969509+aryantandon01@users.noreply.github.com 2024-12-05T03:49:20+05:30 GitHub noreply@github.com 2024-12-04T23:19:20+01:00 Update deprecation-warning.cpp (#10619) 1da7b765692764a8b33b08da61cbee63812a7bd9 59f4db10883a4f3e855cffbf2c3ab68430e95272 Georgi Gerganov ggerganov@gmail.com 2024-12-04T22:38:20+02:00 GitHub noreply@github.com 2024-12-04T22:38:20+02:00 server : fix speculative decoding with context shift (#10641) 59f4db10883a4f3e855cffbf2c3ab68430e95272 2803540814bf0a4e44d0960ff6afda6bac971c17 Diego Devesa slarengh@gmail.com 2024-12-04T14:45:40+01:00 GitHub noreply@github.com 2024-12-04T14:45:40+01:00 ggml : add predefined list of CPU backend variants to build (#10626) 2803540814bf0a4e44d0960ff6afda6bac971c17 253b7fde910731104670724391bfbcb94d97d0c3 Diego Devesa slarengh@gmail.com 2024-12-04T14:40:44+01:00 GitHub noreply@github.com 2024-12-04T14:40:44+01:00 ggml-cpu : fix HWCAP2_I8MM value (#10646) 253b7fde910731104670724391bfbcb94d97d0c3 8d0cfd554a9ae545ff94d27e04458f537b4e8c0e ltoniazzi 61414566+ltoniazzi@users.noreply.github.com 2024-12-04T09:45:48Z GitHub noreply@github.com 2024-12-04T10:45:48+01:00 Fix HF repo commit to clone lora test models (#10649) 8d0cfd554a9ae545ff94d27e04458f537b4e8c0e 2759916d86b70e7aceaed4d0b4e7ed126f0f9e51 JFLFY2255 JFLFY2255@163.com 2024-12-04T17:42:50+08:00 GitHub noreply@github.com 2024-12-04T11:42:50+02:00 llama: Support MiniCPM-1B (with & w/o longrope) (#10559) 2759916d86b70e7aceaed4d0b4e7ed126f0f9e51 40c6d79fb52f995f47507fedfeaae2ac05d9b35c Jeff Bolz jbolz@nvidia.com 2024-12-04T01:28:59-06:00 GitHub noreply@github.com 2024-12-04T08:28:59+01:00 vulkan: Implement "fast divide" (mul+shift) for unary ops like copy (#10642) 40c6d79fb52f995f47507fedfeaae2ac05d9b35c 98036d5670f21e9b9a99d5e3dbb3bf7589f5c4e3 Nicolò Scipione nicolo.scipione@codeplay.com 2024-12-04T02:29:20+01:00 GitHub noreply@github.com 2024-12-04T09:29:20+08:00 SYCL : Move to compile time oneMKL interface backend selection for NVIDIA backend (#10584) 98036d5670f21e9b9a99d5e3dbb3bf7589f5c4e3 cd2f37b304f8e88b9de8424b31078b97f9cf7c60 Wang Ran (汪然) wangr@smail.nju.edu.cn 2024-12-04T09:22:50+08:00 GitHub noreply@github.com 2024-12-04T02:22:50+01:00 fix typo of README.md (#10605) cd2f37b304f8e88b9de8424b31078b97f9cf7c60 da6aac91f150a3b0bcc26d3fd50288accb15f179 Frankie Robertson frankier@users.noreply.github.com 2024-12-04T02:41:37+02:00 GitHub noreply@github.com 2024-12-04T01:41:37+01:00 Avoid using __fp16 on ARM with old nvcc (#10616) da6aac91f150a3b0bcc26d3fd50288accb15f179 01e6d9bb71eb71fe1f811f2fdef15753232cd0f2 Benson Wong mostlygeek@gmail.com 2024-12-03T16:40:36-08:00 GitHub noreply@github.com 2024-12-04T01:40:36+01:00 Add docs for creating a static build (#10268) (#10630) 01e6d9bb71eb71fe1f811f2fdef15753232cd0f2 cc98896db858df7aa40d0e16a505883ef196a482 piDack 104877312+piDack@users.noreply.github.com 2024-12-04T08:26:37+08:00 GitHub noreply@github.com 2024-12-04T01:26:37+01:00 clip : add sycl support (#10574) cc98896db858df7aa40d0e16a505883ef196a482 91c36c269bca75b2d08119c653512cd20b4ea2ba Jeff Bolz jbolz@nvidia.com 2024-12-03T13:29:54-06:00 GitHub noreply@github.com 2024-12-03T20:29:54+01:00 vulkan: optimize and reenable split_k (#10637) 91c36c269bca75b2d08119c653512cd20b4ea2ba 1cd3df46bd49a0c1c78da8b68c956448a73b7476 Xuan Son Nguyen thichthat@gmail.com 2024-12-03T19:38:44+01:00 GitHub noreply@github.com 2024-12-03T19:38:44+01:00 server : (web ui) Various improvements, now use vite as bundler (#10599) 1cd3df46bd49a0c1c78da8b68c956448a73b7476 c5054718575d37f43cc4e6f61ea7c4014ad2c0cf Georgi Gerganov ggerganov@gmail.com 2024-12-03T19:42:30+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-03T20:04:49+02:00 scripts : remove amx sync c5054718575d37f43cc4e6f61ea7c4014ad2c0cf e9e661bd59364e5d4fce035834b6cadcadf8c2ef Georgi Gerganov ggerganov@gmail.com 2024-12-03T19:40:25+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-03T20:04:49+02:00 sync : ggml e9e661bd59364e5d4fce035834b6cadcadf8c2ef efb6ae963031709fc331e6e48cc4606ac8f9c3a7 mahorozte 41834471+mahorozte@users.noreply.github.com 2024-12-03T21:11:43+08:00 Georgi Gerganov ggerganov@gmail.com 2024-12-03T20:04:49+02:00 CUDA: remove unnecessary warp reduce in FA (ggml/1032) efb6ae963031709fc331e6e48cc4606ac8f9c3a7 667d70d1704dfa6977505f5d01d4638669b90dce PAB pierreantoine.bannier@gmail.com 2024-12-02T19:27:24+01:00 Georgi Gerganov ggerganov@gmail.com 2024-12-03T20:04:49+02:00 feat: add `GGML_UNARY_OP_ARGMAX` Metal kernel (ggml/1019) 667d70d1704dfa6977505f5d01d4638669b90dce 3b4f2e33e2cbfca621e623c4b92b88da57a8c2f4 PAB pierreantoine.bannier@gmail.com 2024-11-28T09:25:06+01:00 Georgi Gerganov ggerganov@gmail.com 2024-12-03T20:04:49+02:00 metal : add `GGML_OP_CONV_TRANSPOSE_1D` kernels (ggml/1026) 3b4f2e33e2cbfca621e623c4b92b88da57a8c2f4 82bca2257b3cec72676abb26011f1b99fcdab29d Xuan Son Nguyen thichthat@gmail.com 2024-12-03T12:54:30+01:00 GitHub noreply@github.com 2024-12-03T12:54:30+01:00 llama : add missing LLAMA_API for llama_chat_builtin_templates (#10636) 82bca2257b3cec72676abb26011f1b99fcdab29d 0115df2f65ac7c64dd0e5915c72ecc4a9343a130 Nikolaos Pothitos pothitos@di.uoa.gr 2024-12-03T12:50:08+02:00 GitHub noreply@github.com 2024-12-03T12:50:08+02:00 readme : add option, update default value, fix formatting (#10271) 0115df2f65ac7c64dd0e5915c72ecc4a9343a130 515d4e53727924e48774f45ecb15bdacbf851e13 Georgi Gerganov ggerganov@gmail.com 2024-12-03T11:52:33+02:00 GitHub noreply@github.com 2024-12-03T11:52:33+02:00 metal : small-batch mat-mul kernels (#10581) 515d4e53727924e48774f45ecb15bdacbf851e13 844e2e1feec887c803845a3b8762f3b15979b095 Georgi Gerganov ggerganov@gmail.com 2024-12-03T11:21:43+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-03T11:21:43+02:00 github : minify link [no ci] (revert) 844e2e1feec887c803845a3b8762f3b15979b095 70b98fadbc8c07a0144f3b50a4d7ab7e2aeff878 Georgi Gerganov ggerganov@gmail.com 2024-12-03T11:20:35+02:00 Georgi Gerganov ggerganov@gmail.com 2024-12-03T11:20:35+02:00 github : minify link [no ci] 70b98fadbc8c07a0144f3b50a4d7ab7e2aeff878 642330ac7cea11dc1f9d3df2b8f3dbd10b5e3f3e Georgi Gerganov ggerganov@gmail.com 2024-12-03T11:20:00+02:00 GitHub noreply@github.com 2024-12-03T11:20:00+02:00 server : fix default draft model parameters (#10586) 642330ac7cea11dc1f9d3df2b8f3dbd10b5e3f3e 8648c521010620c2daccfa1d26015c668ba2c717 Xuan Son Nguyen thichthat@gmail.com 2024-12-02T22:10:19+01:00 GitHub noreply@github.com 2024-12-02T22:10:19+01:00 llama : add enum for built-in chat templates (#10623) 8648c521010620c2daccfa1d26015c668ba2c717 64ed2091b24b2f9747148fdf49a34ed5938762c3 Georgi Gerganov ggerganov@gmail.com 2024-12-02T21:22:53+02:00 GitHub noreply@github.com 2024-12-02T21:22:53+02:00 make : deprecate (#10514) 64ed2091b24b2f9747148fdf49a34ed5938762c3 991f8aabeec89d801300bb179e52013fb0eb0584 haopeng 657407891@qq.com 2024-12-02T21:45:54+08:00 GitHub noreply@github.com 2024-12-02T14:45:54+01:00 server: Add "tokens per second" information in the backend (#10548) 991f8aabeec89d801300bb179e52013fb0eb0584 4cb003dd8d1f37523120a21e4b1a50a2adcb8c84 Akarshan Biswas akarshan.biswas@gmail.com 2024-12-02T12:34:11+05:30 GitHub noreply@github.com 2024-12-02T15:04:11+08:00 SYCL: Fix and switch to GGML_LOG system instead of fprintf (#10579) 4cb003dd8d1f37523120a21e4b1a50a2adcb8c84 917786f43d0f29b7c77a0c56767c0fa4df68b1c5 Georgi Gerganov ggerganov@gmail.com 2024-12-02T08:53:27+02:00 GitHub noreply@github.com 2024-12-02T08:53:27+02:00 contrib : refresh (#10593) 917786f43d0f29b7c77a0c56767c0fa4df68b1c5 5e1ed95583ca552a98d8528b73e1ff81249c2bf9 Juk Armstrong 69222624+jukofyork@users.noreply.github.com 2024-12-01T22:09:49Z GitHub noreply@github.com 2024-12-01T23:09:49+01:00 Add `mistral-v1`, `mistral-v3`, `mistral-v3-tekken` and `mistral-v7` chat template types (#10572) 5e1ed95583ca552a98d8528b73e1ff81249c2bf9 5c7a5aa0c32eb19ce03e178560797db5875d7692 Georgi Gerganov ggerganov@gmail.com 2024-12-01T21:37:54+02:00 GitHub noreply@github.com 2024-12-01T21:37:54+02:00 grammars : add English-only grammar (#10612) 5c7a5aa0c32eb19ce03e178560797db5875d7692 3420909dffa50e70660524797a1e715a717684d2 Wang Qin 37098874+wangqin0@users.noreply.github.com 2024-12-01T10:11:42-08:00 GitHub noreply@github.com 2024-12-01T20:11:42+02:00 ci: add error handling for Python venv creation in run.sh (#10608) 3420909dffa50e70660524797a1e715a717684d2 86dc11c5bcf34db2749d8bd8d4fa07a542c94f84 Diego Devesa slarengh@gmail.com 2024-12-01T16:12:41+01:00 GitHub noreply@github.com 2024-12-01T16:12:41+01:00 ggml : automatic selection of best CPU backend (#10606) 86dc11c5bcf34db2749d8bd8d4fa07a542c94f84 6acce3971098772a8aacb10fe8550b4119110581 alek3y 44779186+alek3y@users.noreply.github.com 2024-12-01T12:33:12+01:00 GitHub noreply@github.com 2024-12-01T13:33:12+02:00 server : bind to any port when specified (#10590) 6acce3971098772a8aacb10fe8550b4119110581 43957ef203b4c9ceaee42c176b3ef44ea4359c85 Georgi Gerganov ggerganov@gmail.com 2024-12-01T11:25:17+02:00 GitHub noreply@github.com 2024-12-01T11:25:17+02:00 readme : update the usage section with examples (#10596) 43957ef203b4c9ceaee42c176b3ef44ea4359c85 0c39f44d70d058940fe2afe50cfc789e3e44d756 Wang Qin 37098874+wangqin0@users.noreply.github.com 2024-11-30T19:19:44-08:00 GitHub noreply@github.com 2024-12-01T04:19:44+01:00 build: update Makefile comments for C++ version change (#10598) 0c39f44d70d058940fe2afe50cfc789e3e44d756 3e0ba0e604b1ac5b2cbca9a3f38f91f2be4ef1cd Adrien Gallouët adrien@gallouet.fr 2024-11-30T18:13:18+01:00 GitHub noreply@github.com 2024-11-30T09:13:18-08:00 ggml-cpu: replace AArch64 NEON assembly with intrinsics in ggml_gemv_q4_0_4x4_q8_0() (#10567) 3e0ba0e604b1ac5b2cbca9a3f38f91f2be4ef1cd abadba05be52cccf6c0da49534e37f6062ce8ded Georgi Gerganov ggerganov@gmail.com 2024-11-30T10:09:21+02:00 GitHub noreply@github.com 2024-11-30T10:09:21+02:00 readme : remove old badge abadba05be52cccf6c0da49534e37f6062ce8ded 0533e7fb3842a523f64dc533bd7bd7147ec2c63a Georgi Gerganov ggerganov@gmail.com 2024-11-30T09:47:07+02:00 GitHub noreply@github.com 2024-11-30T09:47:07+02:00 readme : refresh (#10587) 0533e7fb3842a523f64dc533bd7bd7147ec2c63a 7cc2d2c88908fc92b97b28acafb82f7d6e425b85 Eve 139727413+netrunnereve@users.noreply.github.com 2024-11-30T07:00:02Z GitHub noreply@github.com 2024-11-30T08:00:02+01:00 vulkan: Dynamic subgroup size support for Q6_K mat_vec (#10536) 7cc2d2c88908fc92b97b28acafb82f7d6e425b85 b782e5c7d453b3f1fa8dc6c34cde7e2fa946af93 Diego Devesa slarengh@gmail.com 2024-11-29T21:54:58+01:00 GitHub noreply@github.com 2024-11-29T21:54:58+01:00 ggml : move AMX to the CPU backend (#10570) b782e5c7d453b3f1fa8dc6c34cde7e2fa946af93 3a8e9af402f7893423bdab444aa16c5d9a2d429a Xuan Son Nguyen thichthat@gmail.com 2024-11-29T21:48:56+01:00 GitHub noreply@github.com 2024-11-29T21:48:56+01:00 server : add more test cases (#10569) 3a8e9af402f7893423bdab444aa16c5d9a2d429a a3a3048e7a0f9464d0d625a29257d8bce5da5090 Robert Collins roberto.tomas.cuentas@gmail.com 2024-11-29T12:21:37-05:00 GitHub noreply@github.com 2024-11-29T19:21:37+02:00 imatrix : support combine-only (#10492) a3a3048e7a0f9464d0d625a29257d8bce5da5090 f0678c5ff4cb8873d6ff48801475ff270db656fa Diego Devesa slarengh@gmail.com 2024-11-29T17:45:08+01:00 GitHub noreply@github.com 2024-11-29T17:45:08+01:00 cleanup UI link list (#10577) f0678c5ff4cb8873d6ff48801475ff270db656fa 4b3242bbea172ac0980378496fbc676d44c4f459 Georgi Gerganov ggerganov@gmail.com 2024-11-29T16:25:39+02:00 GitHub noreply@github.com 2024-11-29T16:25:39+02:00 ggml : fix I8MM Q4_1 scaling factor conversion (#10562) 4b3242bbea172ac0980378496fbc676d44c4f459 0f77aae5608f16780a49926b67be6d56ec4b09bd Shupei Fan dymarkfan@outlook.com 2024-11-29T21:49:02+08:00 GitHub noreply@github.com 2024-11-29T14:49:02+01:00 ggml-cpu: fix typo in gemv/gemm iq4_nl_4_4 (#10580) 0f77aae5608f16780a49926b67be6d56ec4b09bd 266b8519ee6d21e7ba2bf56f5629e20a181fee8b Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2024-11-29T12:38:45Z GitHub noreply@github.com 2024-11-29T20:38:45+08:00 sycl : offload of get_rows set to 0 (#10432) 266b8519ee6d21e7ba2bf56f5629e20a181fee8b 938f6087421889a3af7d0786c64406ced2be81b8 Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2024-11-29T09:49:43Z GitHub noreply@github.com 2024-11-29T09:49:43Z sycl : Reroute permuted mul_mats through oneMKL (#10408) 938f6087421889a3af7d0786c64406ced2be81b8 f095a649ec390e04dfab1b04e646ae8549dafaef Chenguang Li 87689256+noemotiovon@users.noreply.github.com 2024-11-29T14:46:55+08:00 GitHub noreply@github.com 2024-11-29T14:46:55+08:00 CANN: RoPE operator optimization (#10563) f095a649ec390e04dfab1b04e646ae8549dafaef 678d7994f4da0af3d29046be99950ac999ee9762 Jeff Bolz jbolz@nvidia.com 2024-11-29T00:18:02-06:00 GitHub noreply@github.com 2024-11-29T07:18:02+01:00 vulkan: get the first command buffer submitted sooner (#10499) 678d7994f4da0af3d29046be99950ac999ee9762 dc22344088a7ee81a1e4f096459b03a72f24ccdc Ting Lou louting@189.cn 2024-11-29T08:09:46+08:00 GitHub noreply@github.com 2024-11-29T01:09:46+01:00 llava: return false instead of exit (#10546) dc22344088a7ee81a1e4f096459b03a72f24ccdc 4c0a95b1074907ce7efe6f5bb6ae3351c01429ab Georgi Gerganov ggerganov@gmail.com 2024-11-28T20:46:40+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-28T20:46:40+02:00 ggml : remove redundant copyright notice + update authors 4c0a95b1074907ce7efe6f5bb6ae3351c01429ab 6c595676899013102fdb0aa4b06a49954300c94a Georgi Gerganov ggerganov@gmail.com 2024-11-28T20:45:07+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-28T20:45:07+02:00 llama : add missing model types 6c595676899013102fdb0aa4b06a49954300c94a 890719311b6535e572f15965c6d7ec4ac2537f60 Xuan Son Nguyen thichthat@gmail.com 2024-11-28T19:17:49+01:00 GitHub noreply@github.com 2024-11-28T19:17:49+01:00 server : (tests) don't use thread for capturing stdout/stderr, bump openai client library (#10568) 890719311b6535e572f15965c6d7ec4ac2537f60 7281cf13addfae9b64bb2be87e3b5b1914505d63 Johannes Gäßler johannesg@5d6.de 2024-11-28T18:15:25+01:00 GitHub noreply@github.com 2024-11-28T18:15:25+01:00 common: fix warning message when no GPU found (#10564) 7281cf13addfae9b64bb2be87e3b5b1914505d63 e90688edd004fdb7063f463bd18408ba9ae008dd Random Fly renfei8@live.cn 2024-11-28T23:03:11+08:00 GitHub noreply@github.com 2024-11-28T16:03:11+01:00 docs: fix outdated usage of llama-simple (#10565) e90688edd004fdb7063f463bd18408ba9ae008dd 76b27d29c22af03172cf211a8a31025c7c828a57 Diego Devesa slarengh@gmail.com 2024-11-28T15:58:54+01:00 GitHub noreply@github.com 2024-11-28T15:58:54+01:00 ci : fix tag name in cuda and hip releases (#10566) 76b27d29c22af03172cf211a8a31025c7c828a57 eea986f215e1dc490654d012ccf2ab62fe8f606d Georgi Gerganov ggerganov@gmail.com 2024-11-28T14:56:37+02:00 GitHub noreply@github.com 2024-11-28T14:56:37+02:00 ggml : fix row condition for i8mm kernels (#10561) eea986f215e1dc490654d012ccf2ab62fe8f606d c202cef1686182a78f8f4e253ab8d0c0ffe2fcc8 Georgi Gerganov ggerganov@gmail.com 2024-11-28T14:56:23+02:00 GitHub noreply@github.com 2024-11-28T14:56:23+02:00 cmake : fix ARM feature detection (#10543) c202cef1686182a78f8f4e253ab8d0c0ffe2fcc8 2025fa67e94358deda4740a74fe9803916cb2f60 Shupei Fan dymarkfan@outlook.com 2024-11-28T20:52:03+08:00 GitHub noreply@github.com 2024-11-28T13:52:03+01:00 ggml-cpu: support IQ4_NL_4_4 by runtime repack (#10541) 2025fa67e94358deda4740a74fe9803916cb2f60 c6bc73951ed52466392b1abda98c28ecbe522c7f Sergio López slp@redhat.com 2024-11-28T12:51:38+01:00 GitHub noreply@github.com 2024-11-28T12:51:38+01:00 kompute : improve backend to pass test_backend_ops (#10542) c6bc73951ed52466392b1abda98c28ecbe522c7f 605fa66c509f9f117bd654cf0b9b3ea08bb86e80 Ruixin Huang 18860020911@163.com 2024-11-28T15:27:11+08:00 GitHub noreply@github.com 2024-11-28T15:27:11+08:00 CANN: Update cann.md to display correctly in CLion (#10538) 605fa66c509f9f117bd654cf0b9b3ea08bb86e80 b7420131bf8ab3e067bc660439ab1ab18be7edbd leo-pony nengjunma@outlook.com 2024-11-28T15:25:24+08:00 GitHub noreply@github.com 2024-11-28T15:25:24+08:00 CANN: Fix SOC_TYPE compile bug (#10519) b7420131bf8ab3e067bc660439ab1ab18be7edbd 9f912511bc9414fa7a3c521378b6388cd932b58d Chenguang Li 87689256+noemotiovon@users.noreply.github.com 2024-11-28T14:24:46+08:00 GitHub noreply@github.com 2024-11-28T14:24:46+08:00 CANN: ROPE operator optimization (#10540) 9f912511bc9414fa7a3c521378b6388cd932b58d 3ad5451f3b75809e3033e4e577b9f60bcaf6676a Xuan Son Nguyen thichthat@gmail.com 2024-11-27T22:30:52+01:00 GitHub noreply@github.com 2024-11-27T22:30:52+01:00 common : fix duplicated file name with hf_repo and hf_file (#10550) 3ad5451f3b75809e3033e4e577b9f60bcaf6676a 46c69e0e752ff16206347bb12f96ed69f4a01abf uvos devnull@uvos.xyz 2024-11-27T17:10:08+01:00 GitHub noreply@github.com 2024-11-27T17:10:08+01:00 Add some minimal optimizations for CDNA (#10498) 46c69e0e752ff16206347bb12f96ed69f4a01abf 9e2301f4a4ef1690bd99360c11de43fe830b1c8d Diego Devesa slarengh@gmail.com 2024-11-27T11:03:25+01:00 GitHub noreply@github.com 2024-11-27T11:03:25+01:00 ci : faster CUDA toolkit installation method and use ccache (#10537) 9e2301f4a4ef1690bd99360c11de43fe830b1c8d fee824a1a1e35b5c49d482f654613addade61764 Georgi Gerganov ggerganov@gmail.com 2024-11-27T11:22:14+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-27T11:22:14+02:00 metal : fix group_norm support condition (#0) fee824a1a1e35b5c49d482f654613addade61764 9150f8fef95327474d39ccd6c6e30787e85f3529 Georgi Gerganov ggerganov@gmail.com 2024-11-27T11:10:42+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-27T11:10:42+02:00 sync : ggml 9150f8fef95327474d39ccd6c6e30787e85f3529 c31ed2abfce05c38a2a5189586bfae45a139a547 Frankie Robertson frankier@users.noreply.github.com 2024-11-26T15:50:26+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-27T11:10:27+02:00 Do not include arm_neon.h when compiling CUDA code (ggml/1028) c31ed2abfce05c38a2a5189586bfae45a139a547 5b3466bedfa84aa29c6871c7254467550186ecc6 Jeff Bolz jbolz@nvidia.com 2024-11-27T01:32:54-06:00 GitHub noreply@github.com 2024-11-27T08:32:54+01:00 vulkan: define all quant data structures in types.comp (#10440) 5b3466bedfa84aa29c6871c7254467550186ecc6 249a7902ec710c8d027b9cc0ed10219d2b4184f8 Jeff Bolz jbolz@nvidia.com 2024-11-27T01:30:27-06:00 GitHub noreply@github.com 2024-11-27T08:30:27+01:00 vulkan: Handle GPUs with less shared memory (#10468) 249a7902ec710c8d027b9cc0ed10219d2b4184f8 71a64989a5d2e25c13507efada145f12cf358914 Jeff Bolz jbolz@nvidia.com 2024-11-27T01:21:59-06:00 GitHub noreply@github.com 2024-11-27T08:21:59+01:00 vulkan: further optimize q5_k mul_mat_vec (#10479) 71a64989a5d2e25c13507efada145f12cf358914 4a57d362e1948ada50af997a92c3cbff9711e78b Jeff Bolz jbolz@nvidia.com 2024-11-27T01:08:54-06:00 GitHub noreply@github.com 2024-11-27T08:08:54+01:00 vulkan: skip integer div/mod in get_offsets for batch_idx==0 (#10506) 4a57d362e1948ada50af997a92c3cbff9711e78b c9b00a70b080d5c0668608024afc3e0e2fed822f Jeff Bolz jbolz@nvidia.com 2024-11-27T01:00:50-06:00 GitHub noreply@github.com 2024-11-27T08:00:50+01:00 vulkan: optimize Q2_K and Q3_K mul_mat_vec (#10459) c9b00a70b080d5c0668608024afc3e0e2fed822f de5097351caffb3deaea3393633609df49ef41d0 Diego Devesa slarengh@gmail.com 2024-11-26T22:12:10+01:00 GitHub noreply@github.com 2024-11-26T22:12:10+01:00 ci : fix cuda releases (#10532) de5097351caffb3deaea3393633609df49ef41d0 5a349f2809dc825960dfcfdf8f76b19cd0345be7 Shane A shanea@allenai.org 2024-11-26T12:55:29-08:00 GitHub noreply@github.com 2024-11-26T21:55:29+01:00 Add OLMo 2 model in docs (#10530) 5a349f2809dc825960dfcfdf8f76b19cd0345be7 30ec39832165627dd6ed98938df63adfc6e6a21a Diego Devesa slarengh@gmail.com 2024-11-26T21:13:54+01:00 GitHub noreply@github.com 2024-11-26T21:13:54+01:00 ci : remove nix workflows (#10526) 30ec39832165627dd6ed98938df63adfc6e6a21a be0e350c8b69632b27d5fb41fa064fa256dd7fbf Diego Devesa slarengh@gmail.com 2024-11-26T21:01:47+01:00 GitHub noreply@github.com 2024-11-26T21:01:47+01:00 llama : disable warnings for 3rd party sha1 dependency (#10527) be0e350c8b69632b27d5fb41fa064fa256dd7fbf 249cd93da3df9c8fa78869b0522526d1625aca91 Tristan Druyen tristan@vault81.mozmail.com 2024-11-26T19:27:28+01:00 GitHub noreply@github.com 2024-11-26T19:27:28+01:00 Fix HIP flag inconsistency & build docs (#10524) 249cd93da3df9c8fa78869b0522526d1625aca91 904109ed0d97c9b656a5e8bf612925f739bb8166 R0CKSTAR xiaodong.ye@mthreads.com 2024-11-27T00:00:41+08:00 GitHub noreply@github.com 2024-11-26T17:00:41+01:00 mtgpu: Add MUSA_DOCKER_ARCH in Dockerfiles && update cmake and make (#10516) 904109ed0d97c9b656a5e8bf612925f739bb8166 45abe0f74ee281aea6e5283c1e738061256cfcae Jeff Bolz jbolz@nvidia.com 2024-11-26T09:45:05-06:00 GitHub noreply@github.com 2024-11-26T16:45:05+01:00 vulkan: fix group_norm (#10496) 45abe0f74ee281aea6e5283c1e738061256cfcae 0bbd2262a3263f37385297b30de37941836e57f7 Xuan Son Nguyen thichthat@gmail.com 2024-11-26T16:20:18+01:00 GitHub noreply@github.com 2024-11-26T16:20:18+01:00 server : replace behave with pytest (#10416) 0bbd2262a3263f37385297b30de37941836e57f7 ab96610b1e58684bc5e8b810130c4cf6d8252e21 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-11-26T21:43:47+08:00 GitHub noreply@github.com 2024-11-26T21:43:47+08:00 restore the condistion to build & update pacakge when merge (#10507) ab96610b1e58684bc5e8b810130c4cf6d8252e21 7db3846a94ce7683b3e8120abe427457edf840c9 Georgi Gerganov ggerganov@gmail.com 2024-11-26T14:18:08+02:00 GitHub noreply@github.com 2024-11-26T14:18:08+02:00 cmake : enable warnings in llama (#10474) 7db3846a94ce7683b3e8120abe427457edf840c9 c6807b3f28cc3dbfda3ec390bcb87e69fb5634e2 Diego Devesa slarengh@gmail.com 2024-11-26T13:05:20+01:00 GitHub noreply@github.com 2024-11-26T13:05:20+01:00 ci : publish the docker images created during scheduled runs (#10515) c6807b3f28cc3dbfda3ec390bcb87e69fb5634e2 25669aa92caaddff09f39b54a5173e5cb2680fa3 Diego Devesa slarengh@gmail.com 2024-11-26T13:05:07+01:00 GitHub noreply@github.com 2024-11-26T13:05:07+01:00 ci : add ubuntu cuda build, build with one arch on windows (#10456) 25669aa92caaddff09f39b54a5173e5cb2680fa3 84e1c33cde9e0a7aafcda2d4f21ba51c300482d7 Charles Xu charles.xu@arm.com 2024-11-26T12:37:05+01:00 GitHub noreply@github.com 2024-11-26T13:37:05+02:00 ggml-cpu: cmake add arm64 cpu feature check for macos (#10487) 84e1c33cde9e0a7aafcda2d4f21ba51c300482d7 811872a59daefb25fc0c4326bcb6d8ae893c2f7c Georgi Gerganov ggerganov@gmail.com 2024-11-26T13:36:40+02:00 GitHub noreply@github.com 2024-11-26T13:36:40+02:00 server : fix parallel speculative decoding (#10513) 811872a59daefb25fc0c4326bcb6d8ae893c2f7c 9a4b79bcfa4338b922fa8cf903bd5ac058aaf46f Georgi Gerganov ggerganov@gmail.com 2024-11-26T12:29:38+02:00 GitHub noreply@github.com 2024-11-26T12:29:38+02:00 speculative : simplify the implementation (#10504) 9a4b79bcfa4338b922fa8cf903bd5ac058aaf46f 7066b4cce2898993e943ad6af5d8f1de5840c8e9 Shanshan Shen 467638484@qq.com 2024-11-26T18:08:37+08:00 GitHub noreply@github.com 2024-11-26T18:08:37+08:00 CANN: Improve the Inferencing Performance for Ascend NPU Device (#10454) 7066b4cce2898993e943ad6af5d8f1de5840c8e9 0eb4e12beebabae46d37b78742f4c5d4dbe52dc1 Chenguang Li 87689256+noemotiovon@users.noreply.github.com 2024-11-26T17:31:05+08:00 GitHub noreply@github.com 2024-11-26T17:31:05+08:00 CANN: RoPE and CANCAT operator optimization (#10488) 0eb4e12beebabae46d37b78742f4c5d4dbe52dc1 0cc63754b831d3a6c37bc5d721d12ce9540ffe76 Junil Kim logyourself@gmail.com 2024-11-26T10:47:20+09:00 GitHub noreply@github.com 2024-11-26T01:47:20Z vulkan: Fix a vulkan-shaders-gen arugment parsing error (#10484) 0cc63754b831d3a6c37bc5d721d12ce9540ffe76 50d5cecbda3b0d03344eed326287adc1f6c7f3ef Eric Curtin ecurtin@redhat.com 2024-11-25T16:56:24-05:00 GitHub noreply@github.com 2024-11-25T22:56:24+01:00 Introduce llama-run (#10291) 50d5cecbda3b0d03344eed326287adc1f6c7f3ef 9fd8c2687f5aa2f095ac6e12a376e1c0583888e8 Diego Devesa slarengh@gmail.com 2024-11-25T22:05:39+01:00 GitHub noreply@github.com 2024-11-25T22:05:39+01:00 ci : build docker images only once daily (#10503) 9fd8c2687f5aa2f095ac6e12a376e1c0583888e8 47f931c8f9a26c072d71224bc8013cc66ea9e445 Georgi Gerganov ggerganov@gmail.com 2024-11-25T22:28:27+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-25T22:28:59+02:00 server : add more information about error (#10455) 47f931c8f9a26c072d71224bc8013cc66ea9e445 106964e3d266740f571b5aad7b57545b4a901ac9 Georgi Gerganov ggerganov@gmail.com 2024-11-25T21:50:07+02:00 GitHub noreply@github.com 2024-11-25T21:50:07+02:00 server : enable cache_prompt by default (#10501) 106964e3d266740f571b5aad7b57545b4a901ac9 80acb7b430d826a7685326603a07342e9abc1b45 Georgi Gerganov ggerganov@gmail.com 2024-11-25T21:49:31+02:00 GitHub noreply@github.com 2024-11-25T21:49:31+02:00 metal : enable mat-vec kernels for bs <= 4 (#10491) 80acb7b430d826a7685326603a07342e9abc1b45 10bce0450f0c4d80087e06312b9dbbab3e87f16b Shane A shanea@allenai.org 2024-11-25T10:36:09-08:00 GitHub noreply@github.com 2024-11-25T19:36:09+01:00 Rename Olmo1124 to Olmo2 (#10500) 10bce0450f0c4d80087e06312b9dbbab3e87f16b 1f922254f0c984a8fb9fbaa0c390d7ffae49aedb Diego Devesa slarengh@gmail.com 2024-11-25T19:30:06+01:00 GitHub noreply@github.com 2024-11-25T19:30:06+01:00 llama : accept a list of devices to use to offload a model (#10497) 1f922254f0c984a8fb9fbaa0c390d7ffae49aedb a9a678a6b2264e5895533217b0cf8f250534cc58 Johannes Gäßler johannesg@5d6.de 2024-11-25T19:18:37+01:00 GitHub noreply@github.com 2024-11-25T19:18:37+01:00 Github: update issue templates [no ci] (#10489) a9a678a6b2264e5895533217b0cf8f250534cc58 9ca2e677626fce759d5d95c407c03677b9c87a26 brucepro git@brucepro.net 2024-11-25T08:11:55-08:00 GitHub noreply@github.com 2024-11-25T17:11:55+01:00 Add download chat feature to server chat (#10481) 9ca2e677626fce759d5d95c407c03677b9c87a26 5931c1f233c616083d64e41a228249d58e039aa5 Georgi Gerganov ggerganov@gmail.com 2024-11-25T16:31:38+02:00 GitHub noreply@github.com 2024-11-25T16:31:38+02:00 server : add speculative decoding support (#10455) 5931c1f233c616083d64e41a228249d58e039aa5 f6d12e7df8fe64384f1939976871252e6422a01e Diego Devesa slarengh@gmail.com 2024-11-25T15:13:39+01:00 GitHub noreply@github.com 2024-11-25T15:13:39+01:00 ggml : add support for dynamic loading of backends (#10469) f6d12e7df8fe64384f1939976871252e6422a01e b756441104ca8384640d6df22ba4ea6dab7ad799 Georgi Gerganov ggerganov@gmail.com 2024-11-25T15:17:32+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-25T15:17:32+02:00 tests : fix compile warning b756441104ca8384640d6df22ba4ea6dab7ad799 5a8987793f3e7c1fbfa6806bfcd17d578071b6c9 Georgi Gerganov ggerganov@gmail.com 2024-11-25T15:08:04+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-25T15:08:04+02:00 metal : minor code formatting 5a8987793f3e7c1fbfa6806bfcd17d578071b6c9 d9d54e498d38ec99bbc0031022f9c92711e97bbc Neo Zhang Jianyu jianyu.zhang@intel.com 2024-11-25T17:31:10+08:00 GitHub noreply@github.com 2024-11-25T17:31:10+08:00 [SYCL] Fix building Win package for oneAPI 2025.0 update (#10483) d9d54e498d38ec99bbc0031022f9c92711e97bbc cce5a9007572c6e9fa522296b77571d2e5071357 Georgi Gerganov ggerganov@gmail.com 2024-11-25T09:58:41+02:00 GitHub noreply@github.com 2024-11-25T09:58:41+02:00 speculative : refactor and add a simpler example (#10362) cce5a9007572c6e9fa522296b77571d2e5071357 dc39012cbaf8752fabecaeb60af78ccdd1dfb73b Georgi Gerganov ggerganov@gmail.com 2024-11-24T18:03:25+02:00 GitHub noreply@github.com 2024-11-24T08:03:25-08:00 flake.lock: Update (#10470) dc39012cbaf8752fabecaeb60af78ccdd1dfb73b 9336db462c0c34bbe2055413fe4e16442626c38b Diego Devesa slarengh@gmail.com 2024-11-24T16:10:26+01:00 GitHub noreply@github.com 2024-11-24T16:10:26+01:00 llama : fix op mul check with command-r-plus (#10476) 9336db462c0c34bbe2055413fe4e16442626c38b 96fa2c5e2d6cb5319f34c3a7fb0cec05694b22f1 Gabe Goodhart ghart@us.ibm.com 2024-11-24T02:02:34-07:00 GitHub noreply@github.com 2024-11-24T11:02:34+02:00 convert : XLMRoberta Type Vocab Size (#10458) 96fa2c5e2d6cb5319f34c3a7fb0cec05694b22f1 55ed008b2de01592659b9eba068ea01bb2f72160 momonga 146910567+mmngays@users.noreply.github.com 2024-11-24T09:09:22+09:00 GitHub noreply@github.com 2024-11-24T01:09:22+01:00 fix gguf-py: Conversion error when multiple licenses are configured (#9807) 55ed008b2de01592659b9eba068ea01bb2f72160 6dfcfef0787e9902df29f510b63621f60a09a50b Diego Devesa slarengh@gmail.com 2024-11-23T14:41:12+01:00 GitHub noreply@github.com 2024-11-23T14:41:12+01:00 ggml : do not use ARM features not included in the build (#10457) 6dfcfef0787e9902df29f510b63621f60a09a50b 599b3e0cd40432cd1975a8906f3db70bbe53b627 蕭澧邦 45505768+shou692199@users.noreply.github.com 2024-11-22T17:44:08+08:00 GitHub noreply@github.com 2024-11-22T10:44:08+01:00 ci: Update oneAPI runtime dll packaging (#10428) 599b3e0cd40432cd1975a8906f3db70bbe53b627 c18610b4ee29ca056bb4f2d375a4ad1b16f44ef7 Johannes Gäßler johannesg@5d6.de 2024-11-22T08:32:40+01:00 GitHub noreply@github.com 2024-11-22T08:32:40+01:00 GitHub: ask for more info in issue templates (#10426) c18610b4ee29ca056bb4f2d375a4ad1b16f44ef7 a5e47592b6171ae21f3eaa1aba6fb2b707875063 leo-pony nengjunma@outlook.com 2024-11-22T14:07:20+08:00 GitHub noreply@github.com 2024-11-22T14:07:20+08:00 CANN: Support Ascend310P to accelerate F32 and F16 Model (#10216) a5e47592b6171ae21f3eaa1aba6fb2b707875063 1bb30bf28cb5a7adf111bc41c935bdaf128397e7 Diego Devesa slarengh@gmail.com 2024-11-21T18:18:50+01:00 GitHub noreply@github.com 2024-11-21T18:18:50+01:00 cuda : optimize argmax (#10441) 1bb30bf28cb5a7adf111bc41c935bdaf128397e7 87a533be57e602f8ca469d14ad15ee851265b655 Georgi Gerganov ggerganov@gmail.com 2024-11-21T10:22:47+02:00 GitHub noreply@github.com 2024-11-21T10:22:47+02:00 llama : handle KV shift for recurrent models (#10402) 87a533be57e602f8ca469d14ad15ee851265b655 59b917282236eadfb82bf1f46a31eb119941da08 Georgi Gerganov ggerganov@gmail.com 2024-11-21T09:22:11+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-21T09:22:11+02:00 sync : ggml 59b917282236eadfb82bf1f46a31eb119941da08 02e4eaf22f229a114054b053a9eff61483653670 slaren slarengh@gmail.com 2024-11-20T13:25:08+01:00 Georgi Gerganov ggerganov@gmail.com 2024-11-21T09:22:05+02:00 ggml/sched : do not skip views in pre-assignments 02e4eaf22f229a114054b053a9eff61483653670 9abe9eeae98b11fa93b82632b264126a010225ff Johannes Gäßler johannesg@5d6.de 2024-11-20T14:56:04+01:00 Georgi Gerganov ggerganov@gmail.com 2024-11-21T09:22:02+02:00 ggml-opt: fix data corruption (ggml/1022) 9abe9eeae98b11fa93b82632b264126a010225ff f95caa79546271722ada703da20ffb1cfcd21fed Jeff Bolz jbolz@nvidia.com 2024-11-20T13:47:36-06:00 GitHub noreply@github.com 2024-11-20T20:47:36+01:00 vulkan: predicate max operation in soft_max shaders/soft_max (#10437) f95caa79546271722ada703da20ffb1cfcd21fed fab5d30ff6729ff6ff615c41e8c0215d6bc30393 bandoti 141645996+bandoti@users.noreply.github.com 2024-11-20T12:22:19-04:00 GitHub noreply@github.com 2024-11-20T17:22:19+01:00 cmake: add link dependencies to cmake find pkg (#10433) fab5d30ff6729ff6ff615c41e8c0215d6bc30393 8fd4b7fa29c3061b2e02e897d818dfcbc593430a Diego Devesa slarengh@gmail.com 2024-11-20T12:57:53+01:00 GitHub noreply@github.com 2024-11-20T12:57:53+01:00 llama : add .clang-format file (#10415) 8fd4b7fa29c3061b2e02e897d818dfcbc593430a 1bacb9f62514b520bdf74ed6feb46c80508dad38 Jeff Bolz jbolz@nvidia.com 2024-11-20T01:40:18-06:00 GitHub noreply@github.com 2024-11-20T08:40:18+01:00 vulkan: copy iq4_nl LUT into shared memory (#10409) 1bacb9f62514b520bdf74ed6feb46c80508dad38 ad21c9e1f14d82b8c15ae369a8839019e3d498b4 Jeff Bolz jbolz@nvidia.com 2024-11-20T01:11:00-06:00 GitHub noreply@github.com 2024-11-20T08:11:00+01:00 vulkan: further optimize mul_mat_vec using larger loads (#10387) ad21c9e1f14d82b8c15ae369a8839019e3d498b4 3952a221af54b8a6549bc2bd4a7363ef7ad3081e Neo Zhang Jianyu jianyu.zhang@intel.com 2024-11-20T13:54:25+08:00 GitHub noreply@github.com 2024-11-20T13:54:25+08:00 update rel to 4040 (#10395) 3952a221af54b8a6549bc2bd4a7363ef7ad3081e 42ae10bbcd7b56f29a302c86796542a6dadf46c9 Anthony Van de Gejuchte anthonyvdgent@gmail.com 2024-11-19T23:18:17+01:00 GitHub noreply@github.com 2024-11-19T23:18:17+01:00 Fix missing file renames in Makefile due to changes in commit ae8de6d50a (#10413) 42ae10bbcd7b56f29a302c86796542a6dadf46c9 9fe0fb062630728e3c21b5839e3bce87bff2440a haopeng 657407891@qq.com 2024-11-20T04:10:31+08:00 GitHub noreply@github.com 2024-11-19T21:10:31+01:00 add cmake rvv support (#10411) 9fe0fb062630728e3c21b5839e3bce87bff2440a 611fabd7922050e1e99bd276d3544527cd46047b Georgi Gerganov ggerganov@gmail.com 2024-11-19T19:15:50+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-19T20:03:21+02:00 sync : ggml 611fabd7922050e1e99bd276d3544527cd46047b 12b0ad953a59563ea8d973708760d747321d8432 Plamen Minev pacominev@gmail.com 2024-11-18T15:02:27+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-19T20:03:21+02:00 metal : fox offset integer overflows in im2col (ggml/1015) 12b0ad953a59563ea8d973708760d747321d8432 342397dc7edb311e0373205134d0d3a928b891b3 PAB pierreantoine.bannier@gmail.com 2024-11-18T10:02:49+01:00 Georgi Gerganov ggerganov@gmail.com 2024-11-19T20:03:21+02:00 metal : add `GGML_UNARY_OP_ELU` kernel (ggml/1018) 342397dc7edb311e0373205134d0d3a928b891b3 2a11b6b0946c1abab2ab150725610e5ee736b3af 蕭澧邦 45505768+shou692199@users.noreply.github.com 2024-11-20T01:42:00+08:00 GitHub noreply@github.com 2024-11-19T18:42:00+01:00 cmake: force MSVC compiler charset to utf-8 (#9989) 2a11b6b0946c1abab2ab150725610e5ee736b3af 3ee6382d48b07b31e64983969c16019490e19740 bandoti 141645996+bandoti@users.noreply.github.com 2024-11-19T12:10:30-04:00 GitHub noreply@github.com 2024-11-19T17:10:30+01:00 Add required ggml-base and backend libs to cmake pkg (#10407) 3ee6382d48b07b31e64983969c16019490e19740 8e752a777b272606f22cb741b03e062de4ddb8fe Diego Devesa slarengh@gmail.com 2024-11-19T14:29:38+01:00 GitHub noreply@github.com 2024-11-19T14:29:38+01:00 cuda : fix CUDA_FLAGS not being applied (#10403) 8e752a777b272606f22cb741b03e062de4ddb8fe a88ad007de3eb5d92cb538fd269ff94c4bf0c8d2 Georgi Gerganov ggerganov@gmail.com 2024-11-19T13:29:26+02:00 GitHub noreply@github.com 2024-11-19T13:29:26+02:00 llama : add check for KV cache shifts (#10401) a88ad007de3eb5d92cb538fd269ff94c4bf0c8d2 2a1507c1629975d9d20a503d6a14f44eff292c25 Shane A shanea@allenai.org 2024-11-19T01:04:08-08:00 GitHub noreply@github.com 2024-11-19T11:04:08+02:00 llama : add OLMo November 2024 support (#10394) 2a1507c1629975d9d20a503d6a14f44eff292c25 b3e585988fc65d3a8083c6d94dfc0629f9ce226d Romain Biessy romain.biessy@codeplay.com 2024-11-19T09:02:23+01:00 GitHub noreply@github.com 2024-11-19T08:02:23Z sycl : Add option to set the SYCL architecture for all targets (#10266) b3e585988fc65d3a8083c6d94dfc0629f9ce226d 557924f22237c76387a39c4db5abae154d57e754 Jeff Bolz jbolz@nvidia.com 2024-11-19T01:25:17-06:00 GitHub noreply@github.com 2024-11-19T08:25:17+01:00 vulkan: Optimize soft_max (#10301) 557924f22237c76387a39c4db5abae154d57e754 d3481e631661b5e9517f78908cdd58cee63c4903 Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2024-11-19T00:50:04Z GitHub noreply@github.com 2024-11-19T08:50:04+08:00 sycl: Revert MUL_MAT_OP support changes (#10385) d3481e631661b5e9517f78908cdd58cee63c4903 531cb1c233800e6acb021dc56d69595e314db072 Diego Devesa slarengh@gmail.com 2024-11-18T18:43:40+01:00 GitHub noreply@github.com 2024-11-18T18:43:40+01:00 cuda : only use native when supported by cmake (#10389) 531cb1c233800e6acb021dc56d69595e314db072 f139d2ea611c5604395c95160d3c53f7c4eaf220 bandoti 141645996+bandoti@users.noreply.github.com 2024-11-18T11:23:58-04:00 GitHub noreply@github.com 2024-11-18T16:23:58+01:00 Skip searching root path for cross-compile builds (#10383) f139d2ea611c5604395c95160d3c53f7c4eaf220 2eb76b2a5e4ea395b971a419c95b473ab6f253e4 Jeff Bolz jbolz@nvidia.com 2024-11-18T08:28:42-06:00 GitHub noreply@github.com 2024-11-18T08:28:42-06:00 vulkan: remove use of null initializer (#10372) 2eb76b2a5e4ea395b971a419c95b473ab6f253e4 9b75f03cd2ec9cc482084049d87a0f08f9f01517 Georgi Gerganov ggerganov@gmail.com 2024-11-18T16:08:20+02:00 GitHub noreply@github.com 2024-11-18T06:08:20-08:00 flake.lock: Update (#10346) 9b75f03cd2ec9cc482084049d87a0f08f9f01517 75207b3a887f91f813de1eb6e9fd135d3cb2b8c6 0cc4m picard12@live.de 2024-11-18T11:02:43+01:00 GitHub noreply@github.com 2024-11-18T11:02:43+01:00 Vulkan: Fix device info output format specifiers (#10366) 75207b3a887f91f813de1eb6e9fd135d3cb2b8c6 76e9e58b7847112848aa1f40b65d1cbcd6d5f5a3 Johannes Gäßler johannesg@5d6.de 2024-11-18T00:21:53+01:00 GitHub noreply@github.com 2024-11-18T00:21:53+01:00 docker: use GGML_NATIVE=OFF (#10368) 76e9e58b7847112848aa1f40b65d1cbcd6d5f5a3 ce2e59ba107cf71ed566040ff20a15d1c58e09c2 Johannes Gäßler johannesg@5d6.de 2024-11-17T23:20:42+01:00 GitHub noreply@github.com 2024-11-17T23:20:42+01:00 CUDA: fix MMV kernel being used for FP16 src1 (#10357) ce2e59ba107cf71ed566040ff20a15d1c58e09c2 be5caccef945546ee9fd25a151330a88d785faf9 Johannes Gäßler johannesg@5d6.de 2024-11-17T12:59:38+01:00 GitHub noreply@github.com 2024-11-17T12:59:38+01:00 CMake: fix typo in comment [no ci] (#10360) be5caccef945546ee9fd25a151330a88d785faf9 20a780c7b64c38071fe70ad23e16e80c23c0147b Diego Devesa slarengh@gmail.com 2024-11-17T12:25:45+01:00 GitHub noreply@github.com 2024-11-17T12:25:45+01:00 llama : only use default buffer types for the KV cache (#10358) 20a780c7b64c38071fe70ad23e16e80c23c0147b cf32a9b93ad859ea592c31785b6bd3b4b2121463 Georgi Gerganov ggerganov@gmail.com 2024-11-17T13:12:22+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-17T13:12:22+02:00 gitignore : ignore local run scripts [no ci] cf32a9b93ad859ea592c31785b6bd3b4b2121463 a43178299c2f74f14bcfc659bfd9fd32d931d1f4 Georgi Gerganov ggerganov@gmail.com 2024-11-17T11:23:01+02:00 GitHub noreply@github.com 2024-11-17T11:23:01+02:00 metal : refactor kernel args into structs (#10238) a43178299c2f74f14bcfc659bfd9fd32d931d1f4 c3ea58aca406911eb4d409cdbfc76683393442b6 FirstTimeEZ 179362031+FirstTimeEZ@users.noreply.github.com 2024-11-17T21:39:22+13:00 GitHub noreply@github.com 2024-11-17T10:39:22+02:00 ggml : fix undefined reference to 'getcpu' (#10354) c3ea58aca406911eb4d409cdbfc76683393442b6 467576b6cc7d2b9220f55bc635aa51469cf26fb5 Johannes Gäßler johannesg@5d6.de 2024-11-17T09:09:55+01:00 GitHub noreply@github.com 2024-11-17T09:09:55+01:00 CUDA: remove DMMV, consolidate F16 mult mat vec (#10318) 467576b6cc7d2b9220f55bc635aa51469cf26fb5 eda7e1d4f54711de1c9b40502d6c88bbc217da60 Johannes Gäßler johannesg@5d6.de 2024-11-17T09:06:34+01:00 GitHub noreply@github.com 2024-11-17T09:06:34+01:00 CMake: default to -arch=native for CUDA build (#10320) eda7e1d4f54711de1c9b40502d6c88bbc217da60 24203e9dd7355a4a10bc32d959fd0148d37bf666 Diego Devesa slarengh@gmail.com 2024-11-17T07:31:17+01:00 GitHub noreply@github.com 2024-11-17T08:31:17+02:00 ggml : fix possible buffer use after free in sched reserve (#9930) 24203e9dd7355a4a10bc32d959fd0148d37bf666 5d9e59979c2fd91c99d03c23e8df9c50c9d55a85 Georgi Gerganov ggerganov@gmail.com 2024-11-16T23:40:39+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-17T08:30:29+02:00 ggml : inttypes.h -> cinttypes (#0) 5d9e59979c2fd91c99d03c23e8df9c50c9d55a85 a4200cafadebb7576a9a3905039858f5e73ce4cc Georgi Gerganov ggerganov@gmail.com 2024-11-16T21:38:01+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-17T08:30:29+02:00 ggml : adapt AMX to tensor->grad removal (#0) a4200cafadebb7576a9a3905039858f5e73ce4cc 84274a10c399d843cff39feb2b86dc8224f613d8 Georgi Gerganov ggerganov@gmail.com 2024-11-16T21:35:31+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-17T08:30:29+02:00 make : add ggml-opt (#0) 84274a10c399d843cff39feb2b86dc8224f613d8 68fcb4759c0ca2874b59d9c1e6a35ceca1cc04ce Georgi Gerganov ggerganov@gmail.com 2024-11-16T21:34:03+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-17T08:30:29+02:00 tests : remove test-grad0 68fcb4759c0ca2874b59d9c1e6a35ceca1cc04ce 8a43e940ab0daaff198809bf9277289994ec62f5 Georgi Gerganov ggerganov@gmail.com 2024-11-16T21:32:41+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-17T08:30:29+02:00 ggml : fix compile warnings (#0) 8a43e940ab0daaff198809bf9277289994ec62f5 5c9a8b22b10132db620529435e3cfa49304b65cc Johannes Gäßler johannesg@5d6.de 2024-11-16T22:17:59+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-17T08:30:29+02:00 ggml: new optimization interface (ggml/988) 5c9a8b22b10132db620529435e3cfa49304b65cc 0fff7fd79818980763a601660f25b01a0cf4b87a Georgi Gerganov ggerganov@gmail.com 2024-11-16T22:16:04+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-17T08:30:29+02:00 scripts : update sync 0fff7fd79818980763a601660f25b01a0cf4b87a 4e54be0ec6cb5cd6ed56e52e927b80b2796ec844 FirstTimeEZ 179362031+FirstTimeEZ@users.noreply.github.com 2024-11-17T12:29:18+13:00 GitHub noreply@github.com 2024-11-17T00:29:18+01:00 docs : vulkan build instructions to use git bash mingw64 (#10303) 4e54be0ec6cb5cd6ed56e52e927b80b2796ec844 db4cfd5dbc31c90f0d5c413a2e182d068b8ee308 Johannes Gäßler johannesg@5d6.de 2024-11-16T23:00:41+01:00 GitHub noreply@github.com 2024-11-16T23:00:41+01:00 llama/ex: remove --logdir argument (#10339) db4cfd5dbc31c90f0d5c413a2e182d068b8ee308 8ee0d09ae6928d0501765cfc4e430b9236730caf Georgi Gerganov ggerganov@gmail.com 2024-11-16T17:58:56+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-16T20:36:26+02:00 llamafile : fix include path (#0) 8ee0d09ae6928d0501765cfc4e430b9236730caf bcdb7a23862b61aa307fc462fadfe1e2e653d010 Georgi Gerganov ggerganov@gmail.com 2024-11-16T17:58:32+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-16T20:36:26+02:00 make : auto-determine dependencies (#0) bcdb7a23862b61aa307fc462fadfe1e2e653d010 f245cc28d4eb900efad0bc740145f58d713c6e4f MaggotHATE clay1326@gmail.com 2024-11-16T18:26:54+05:00 GitHub noreply@github.com 2024-11-16T14:26:54+01:00 server: (web UI) Add samplers sequence customization (#10255) f245cc28d4eb900efad0bc740145f58d713c6e4f 772703c8fffdd83d2e28f60119e83525f1189412 Georgi Gerganov ggerganov@gmail.com 2024-11-16T10:32:50+02:00 GitHub noreply@github.com 2024-11-16T10:32:50+02:00 scripts : fix missing key in compare-llama-bench.py (#10332) 772703c8fffdd83d2e28f60119e83525f1189412 dd3a6ce9f84d21ba05fe98af9f983bdea0398e6c Jeff Bolz jbolz@nvidia.com 2024-11-16T00:26:57-06:00 GitHub noreply@github.com 2024-11-16T07:26:57+01:00 vulkan: Optimize some mat-vec mul quant shaders (#10296) dd3a6ce9f84d21ba05fe98af9f983bdea0398e6c 1e58ee1318429a3e97aa66f3034cdfd65ffc6c34 FirstTimeEZ 179362031+FirstTimeEZ@users.noreply.github.com 2024-11-16T14:59:33+13:00 GitHub noreply@github.com 2024-11-16T02:59:33+01:00 vulkan : add cmake preset debug/release (#10306) 1e58ee1318429a3e97aa66f3034cdfd65ffc6c34 89e4caaaf081f4712af61a3e08cb67b406c02b80 Dan Johansson dan.johansson@arm.com 2024-11-16T01:53:37+01:00 GitHub noreply@github.com 2024-11-16T01:53:37+01:00 ggml : optimize Q4_0 into Q4_0_X_Y repack (#10324) 89e4caaaf081f4712af61a3e08cb67b406c02b80 74d73dc85cc2057446bf63cc37ff649ae7cebd80 FirstTimeEZ 179362031+FirstTimeEZ@users.noreply.github.com 2024-11-16T13:42:13+13:00 GitHub noreply@github.com 2024-11-16T01:42:13+01:00 llama : save number of parameters and the size in llama_model (#10286) 74d73dc85cc2057446bf63cc37ff649ae7cebd80 4047be74da398acb8717a4d21b77b929ad7ed4f7 Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-11-16T02:57:00+05:30 GitHub noreply@github.com 2024-11-15T22:27:00+01:00 Make updates to fix issues with clang-cl builds while using AVX512 flags (#10314) 4047be74da398acb8717a4d21b77b929ad7ed4f7 883d206fbd2c5b2b9b589a9328503b9005e146c9 Johannes Gäßler johannesg@5d6.de 2024-11-15T21:19:03+01:00 GitHub noreply@github.com 2024-11-15T21:19:03+01:00 scripts: update compare-llama-bench.py (#10319) 883d206fbd2c5b2b9b589a9328503b9005e146c9 09ecbcb596ed8fa97d503d7440f0b3eff872e8f1 slaren slarengh@gmail.com 2024-11-15T20:20:54+01:00 Georgi Gerganov ggerganov@gmail.com 2024-11-15T21:45:32+02:00 ggml : fix some build issues 09ecbcb596ed8fa97d503d7440f0b3eff872e8f1 3225008973579cc6a784890c237e1bfc9de41819 Georgi Gerganov ggerganov@gmail.com 2024-11-15T15:35:22+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-15T15:44:06+02:00 cmake : fix ppc64 check (whisper/0) 3225008973579cc6a784890c237e1bfc9de41819 cbf5541a82952bcd7c4fceb55f5e332cafbf1720 thewh1teagle 61390950+thewh1teagle@users.noreply.github.com 2024-11-15T15:33:53+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-15T15:44:06+02:00 ggml : vulkan logs (whisper/2547) cbf5541a82952bcd7c4fceb55f5e332cafbf1720 18429220bdb344da1bc7df9bc580c7b41b3cd57b Georgi Gerganov ggerganov@gmail.com 2024-11-15T15:31:16+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-15T15:44:06+02:00 sync : ggml 18429220bdb344da1bc7df9bc580c7b41b3cd57b f0204a0ec70d50ca60e07bc0096ec1d6508ab0c7 Eve 139727413+netrunnereve@users.noreply.github.com 2024-11-15T11:47:58Z GitHub noreply@github.com 2024-11-15T12:47:58+01:00 AVX BF16 and single scale quant optimizations (#10212) f0204a0ec70d50ca60e07bc0096ec1d6508ab0c7 57f8355b29a8c7dfcd1fb6094758ad85644f8535 R0CKSTAR xiaodong.ye@mthreads.com 2024-11-15T19:47:25+08:00 GitHub noreply@github.com 2024-11-15T12:47:25+01:00 ci: build test musa with cmake (#10298) 57f8355b29a8c7dfcd1fb6094758ad85644f8535 9901068ac78838745e604fffb4601d315a610456 Romain Biessy romain.biessy@codeplay.com 2024-11-15T12:10:45+01:00 GitHub noreply@github.com 2024-11-15T13:10:45+02:00 sycl: Update Intel docker images to use DPC++ 2025.0 (#10305) 9901068ac78838745e604fffb4601d315a610456 231f9360d94446cd083b6b116f63991b1328c484 Xuan Son Nguyen thichthat@gmail.com 2024-11-15T05:48:49-04:00 GitHub noreply@github.com 2024-11-15T10:48:49+01:00 server : (web UI) add copy button for code block, fix api key (#10242) 231f9360d94446cd083b6b116f63991b1328c484 4802ad350b8e19cbc7a77269b4494c896f6e0896 Chenguang Li 87689256+noemotiovon@users.noreply.github.com 2024-11-15T15:09:35+08:00 GitHub noreply@github.com 2024-11-15T15:09:35+08:00 cann: dockerfile and doc adjustment (#10302) 4802ad350b8e19cbc7a77269b4494c896f6e0896 5a54af4d4f588f109f31e456483fdf77096399d9 Georgi Gerganov ggerganov@gmail.com 2024-11-15T08:38:43+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-15T08:38:43+02:00 scripts : fix regex in sync [no ci] 5a54af4d4f588f109f31e456483fdf77096399d9 1607a5e5b08f4e55f118af3d7de325949d8f1835 Romain Biessy romain.biessy@codeplay.com 2024-11-15T04:09:12+01:00 GitHub noreply@github.com 2024-11-15T11:09:12+08:00 sycl: Use syclcompat::dp4a (#10267) 1607a5e5b08f4e55f118af3d7de325949d8f1835 ae8de6d50a09d49545e0afab2e50cc4acfb280e2 Charles Xu charles.xu@arm.com 2024-11-15T01:28:50+01:00 GitHub noreply@github.com 2024-11-15T01:28:50+01:00 backend cpu: add online flow for aarch64 Q4_0 GEMV/GEMM kernels (#9921) ae8de6d50a09d49545e0afab2e50cc4acfb280e2 4a8ccb37ad9c9027cbcfd5548c19cdffe48d5197 Diego Devesa slarengh@gmail.com 2024-11-14T18:04:35+01:00 GitHub noreply@github.com 2024-11-14T18:04:35+01:00 ggml : build backends as libraries (#10256) 4a8ccb37ad9c9027cbcfd5548c19cdffe48d5197 2a82891a853db908679f7b24b04586e6f6393fe0 Johannes Gäßler johannesg@5d6.de 2024-11-14T13:00:15+01:00 GitHub noreply@github.com 2024-11-14T13:00:15+01:00 CUDA: no -sm row for very small matrices (#10185) 2a82891a853db908679f7b24b04586e6f6393fe0 af148c9386da825a60c7038549c121c35ca56b50 Georgi Gerganov ggerganov@gmail.com 2024-11-14T11:44:15+02:00 GitHub noreply@github.com 2024-11-14T11:44:15+02:00 speculative : fix out-of-bounds access (#10289) af148c9386da825a60c7038549c121c35ca56b50 66798e42fbe636f1cb6236e4bc30939d23ef7c25 Jeff Bolz jbolz@nvidia.com 2024-11-13T23:22:55-06:00 GitHub noreply@github.com 2024-11-14T06:22:55+01:00 vulkan: Optimize binary ops (#10270) 66798e42fbe636f1cb6236e4bc30939d23ef7c25 fb4a0ec0833c71cff5a1a367ba375447ce6106eb Jeff Bolz jbolz@nvidia.com 2024-11-13T14:59:47-06:00 GitHub noreply@github.com 2024-11-13T21:59:47+01:00 vulkan: Use macros to make the mat mul pipeline creation more concise (#10259) fb4a0ec0833c71cff5a1a367ba375447ce6106eb 5ea926dad7f62ebccff7b24784bd1e01a06d13ae Michael Podvitskiy podvitskiymichael@gmail.com 2024-11-13T20:00:35+02:00 GitHub noreply@github.com 2024-11-13T20:00:35+02:00 llama : propagate the results of `graph_compute` (#9525) 5ea926dad7f62ebccff7b24784bd1e01a06d13ae 1ee9eea094fe5846c7d8d770aa7caa749d246b23 Georgi Gerganov ggerganov@gmail.com 2024-11-13T18:11:54+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-13T18:11:54+02:00 sync : ggml 1ee9eea094fe5846c7d8d770aa7caa749d246b23 ff7fb670d0a62897c5662536aeb53422c549fbe8 Small Grass Forest zixuanxcl@gmail.com 2024-11-13T19:17:10+08:00 GitHub noreply@github.com 2024-11-13T13:17:10+02:00 docs : update bindings list (#10261) ff7fb670d0a62897c5662536aeb53422c549fbe8 0e712a5acbbdd1593e5aeb86d4f6b896a11b438c Alexey Parfenov zxed@alkatrazstudio.net 2024-11-13T11:16:30Z GitHub noreply@github.com 2024-11-13T13:16:30+02:00 server : add missing docs (#10269) 0e712a5acbbdd1593e5aeb86d4f6b896a11b438c a0ec17b32ec6077f5ca22fe833ebdc9b86795a4d Jhen-Jie Hong iainst0409@gmail.com 2024-11-13T19:15:23+08:00 GitHub noreply@github.com 2024-11-13T13:15:23+02:00 server : fix incorrect res in validate_model_chat_template (#10272) a0ec17b32ec6077f5ca22fe833ebdc9b86795a4d 2e82ffa4af29f87e7d3d6dff8060a2a79613b72f Brian mofosyne@gmail.com 2024-11-13T21:10:38+11:00 GitHub noreply@github.com 2024-11-13T21:10:38+11:00 metadata: Detailed Dataset Authorship Metadata (#8875) 2e82ffa4af29f87e7d3d6dff8060a2a79613b72f 80dd7ff22fd050fed58b552cc8001aaf968b7ebf Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2024-11-13T09:40:57Z GitHub noreply@github.com 2024-11-13T09:40:57Z sycl : Fixes to broken builds and test-backend-ops (#10257) 80dd7ff22fd050fed58b552cc8001aaf968b7ebf 54ef9cfc726a799e6f454ac22c4815d037716eda Jeff Bolz jbolz@nvidia.com 2024-11-13T00:58:57-06:00 GitHub noreply@github.com 2024-11-13T07:58:57+01:00 vulkan: Optimize contiguous copies (#10254) 54ef9cfc726a799e6f454ac22c4815d037716eda b0cefea58a20020754b7431e3c725625274372a5 Jeff Bolz jbolz@nvidia.com 2024-11-11T11:13:51-06:00 GitHub noreply@github.com 2024-11-11T18:13:51+01:00 vulkan: Throttle the number of shader compiles during the build step. (#10222) b0cefea58a20020754b7431e3c725625274372a5 b141e5f6efbab3a00633df88c4f9425bfe8b78ab Georgi Gerganov ggerganov@gmail.com 2024-11-11T08:39:13+02:00 GitHub noreply@github.com 2024-11-11T08:39:13+02:00 metal : more precise Q*K in FA vec kernel (#10247) b141e5f6efbab3a00633df88c4f9425bfe8b78ab 4b3a9212b602be3d4e2e3ca26efd796cef13c55e Georgi Gerganov ggerganov@gmail.com 2024-11-11T08:38:43+02:00 GitHub noreply@github.com 2024-11-11T08:38:43+02:00 server : enable KV cache defrag by default (#10233) 4b3a9212b602be3d4e2e3ca26efd796cef13c55e 505f33274d60676320216b662a97672a76ec600e Georgi Gerganov ggerganov@gmail.com 2024-11-10T21:45:25+02:00 GitHub noreply@github.com 2024-11-10T11:45:25-08:00 flake.lock: Update (#10243) 505f33274d60676320216b662a97672a76ec600e 160687b3ed002eee83a04de83a9cd752f928ced1 MaggotHATE clay1326@gmail.com 2024-11-11T00:42:25+05:00 GitHub noreply@github.com 2024-11-10T15:42:25-04:00 server : (web UI) Add back sampler settings (#10239) 160687b3ed002eee83a04de83a9cd752f928ced1 6423c65aa8be1b98f990cf207422505ac5a441a1 Jeff Bolz jbolz@nvidia.com 2024-11-10T05:37:56-06:00 GitHub noreply@github.com 2024-11-10T12:37:56+01:00 vulkan: Fix newly added tests for permuted mul_mat and 1D im2col (#10226) 6423c65aa8be1b98f990cf207422505ac5a441a1 39a334a9aaf2000f93a899d9f43d889e460640ee Georgi Gerganov ggerganov@gmail.com 2024-11-09T11:53:13+02:00 GitHub noreply@github.com 2024-11-09T11:53:13+02:00 metal : reorder write loop in mul mat kernel + style (#10231) 39a334a9aaf2000f93a899d9f43d889e460640ee bb38cdd8baf37de1fadab3e867c6ba4ae452eff6 Georgi Gerganov ggerganov@gmail.com 2024-11-09T11:53:02+02:00 GitHub noreply@github.com 2024-11-09T11:53:02+02:00 metal : fix build and some more comments (#10229) bb38cdd8baf37de1fadab3e867c6ba4ae452eff6 f018acba22095b8995bf6c5ef815b16a3ce4cf1b Georgi Gerganov ggerganov@gmail.com 2024-11-09T11:52:45+02:00 GitHub noreply@github.com 2024-11-09T11:52:45+02:00 metal : fix F32 accumulation in FA vec kernel (#10232) f018acba22095b8995bf6c5ef815b16a3ce4cf1b 46323fa9efd5e6c8aeef8d6eb6c332ee0d95eb13 Georgi Gerganov ggerganov@gmail.com 2024-11-09T11:26:34+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-09T11:26:34+02:00 llama : fix Qwen model type strings 46323fa9efd5e6c8aeef8d6eb6c332ee0d95eb13 5b359bb1e3585de45bec79fd6c18934897662cdf Georgi Gerganov ggerganov@gmail.com 2024-11-09T11:21:49+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-09T11:21:49+02:00 metal : hide debug messages from normal log 5b359bb1e3585de45bec79fd6c18934897662cdf e89213492d3e01705739789733f0f2d250b4c449 SXX sxx1136965276@gmail.com 2024-11-09T15:35:46+08:00 GitHub noreply@github.com 2024-11-09T08:35:46+01:00 ggml: fix zero division in ‘dne’ calculation in CUDA COUNT_EQUAL operator when ‘ne’ is small (#10213) e89213492d3e01705739789733f0f2d250b4c449 8fc393f246c550d2481e53323a47644a94e8d01f amritahs-ibm amritahs@linux.vnet.ibm.com 2024-11-09T12:47:50+05:30 GitHub noreply@github.com 2024-11-09T09:17:50+02:00 ggml : optimize llamafile cpu matrix multiplication for ppc64le (#10156) 8fc393f246c550d2481e53323a47644a94e8d01f ec450d3bbf9fdb3cd06b27c00c684fd1861cb0cf haopeng 657407891@qq.com 2024-11-09T15:06:54+08:00 GitHub noreply@github.com 2024-11-09T09:06:54+02:00 scripts : fix pattern and get n_tokens in one go (#10221) ec450d3bbf9fdb3cd06b27c00c684fd1861cb0cf 695ad752b2631af84ba321177656705b30c6e401 Georgi Gerganov ggerganov@gmail.com 2024-11-08T21:59:46+02:00 GitHub noreply@github.com 2024-11-08T21:59:46+02:00 metal : opt-in compile flag for BF16 (#10218) 695ad752b2631af84ba321177656705b30c6e401 841f27abdbbcecc9daac14dc540ba6202e4ffe40 Georgi Gerganov ggerganov@gmail.com 2024-11-08T18:37:41+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-08T18:37:41+02:00 metal : improve clarity (minor) (#10171) 841f27abdbbcecc9daac14dc540ba6202e4ffe40 d05b3127bd30515955aa4ee2bacdb68ebafe88f4 Georgi Gerganov ggerganov@gmail.com 2024-11-08T13:47:22+02:00 GitHub noreply@github.com 2024-11-08T13:47:22+02:00 metal : optimize FA kernels (#10171) d05b3127bd30515955aa4ee2bacdb68ebafe88f4 76c6e7f10551960e4ec9e14e0535b72081f1c7ad Jhen-Jie Hong iainst0409@gmail.com 2024-11-08T17:34:06+08:00 GitHub noreply@github.com 2024-11-08T11:34:06+02:00 swift : exclude ggml-metal-embed.metal (#10211) 76c6e7f10551960e4ec9e14e0535b72081f1c7ad a71d81cf8c1afb26b166f897c94ee1581f9fac7d Xuan Son Nguyen thichthat@gmail.com 2024-11-07T18:44:38-04:00 GitHub noreply@github.com 2024-11-07T18:44:38-04:00 server : minor UI fix (#10207) a71d81cf8c1afb26b166f897c94ee1581f9fac7d eec4d71737b32f312e0082b671629a0368e1a20d Xuan Son Nguyen thichthat@gmail.com 2024-11-07T17:31:10-04:00 GitHub noreply@github.com 2024-11-07T17:31:10-04:00 server : revamp chat UI with vuejs and daisyui (#10175) eec4d71737b32f312e0082b671629a0368e1a20d 3b08828674f561c78af182d47fc0636fc3ccd1e9 Georgi Gerganov ggerganov@gmail.com 2024-11-07T23:11:36+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-07T23:11:36+02:00 scripts : add amx to sync-ggml.sh [no ci] 3b08828674f561c78af182d47fc0636fc3ccd1e9 a2c6fd747c77fe183e2f556a4a2f1fb0a0be4c7b Georgi Gerganov ggerganov@gmail.com 2024-11-07T23:08:24+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-07T23:08:24+02:00 sync : ggml a2c6fd747c77fe183e2f556a4a2f1fb0a0be4c7b 97404c4a0374cac45c8c34a32d13819de1dd023d Georgi Gerganov ggerganov@gmail.com 2024-11-07T23:07:55+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-07T23:07:55+02:00 scripts : sync update 97404c4a0374cac45c8c34a32d13819de1dd023d 60e17ce23c2740369af6304113a2dfa0454eaf26 Diego Devesa slarengh@gmail.com 2024-11-07T18:16:08+01:00 GitHub noreply@github.com 2024-11-07T18:16:08+01:00 ggml : add ggml-cpu.h to the public headers (#10204) 60e17ce23c2740369af6304113a2dfa0454eaf26 5107e8cea35be46a27cfc940e6841c0cf81c0525 Faisal Zaghloul quic_fzaghlou@quicinc.com 2024-11-07T11:46:12-05:00 GitHub noreply@github.com 2024-11-07T08:46:12-08:00 Remove identical wte/etw logic for jais (#10203) 5107e8cea35be46a27cfc940e6841c0cf81c0525 2319126a70b541f8670225a04a38202bbdccbedb wwoodsTM 104587230+wwoodsTM@users.noreply.github.com 2024-11-07T08:20:25-07:00 GitHub noreply@github.com 2024-11-07T16:20:25+01:00 DRY: Fixes clone functionality (#10192) 2319126a70b541f8670225a04a38202bbdccbedb 3bcd40b3c593d14261fb2abfabad3c0fb5b9e318 snadampal 87143774+snadampal@users.noreply.github.com 2024-11-07T02:02:08-06:00 GitHub noreply@github.com 2024-11-07T09:02:08+01:00 fix q4_0_8_8 format for corrupted tokens issue (#10198) 3bcd40b3c593d14261fb2abfabad3c0fb5b9e318 5c333e014059122245c318e7ed4ec27d1085573c Zhiyuan Li lizhiyuan@uniartisan.com 2024-11-07T18:19:10+11:00 GitHub noreply@github.com 2024-11-07T15:19:10+08:00 Optimize RWKV6 Operator Naming and Implement Multi-core CPU/ SYCL Acceleration (#10133) 5c333e014059122245c318e7ed4ec27d1085573c b11f9ba9b8ce319f04b88afe40d264e6b7f4ba46 Georgi Gerganov ggerganov@gmail.com 2024-11-06T19:53:51+02:00 GitHub noreply@github.com 2024-11-06T19:53:51+02:00 metal : add BF16 support (#8439) b11f9ba9b8ce319f04b88afe40d264e6b7f4ba46 94d8cb8be13b7c4d04eeca5a2b956b9148e6f222 Georgi Gerganov ggerganov@gmail.com 2024-11-06T13:29:01+02:00 GitHub noreply@github.com 2024-11-06T13:29:01+02:00 server : remove hack for extra parallel slot (#10187) 94d8cb8be13b7c4d04eeca5a2b956b9148e6f222 1dc04b2deed2d2f2ae3aff9b14ae29674dee1fb8 Diego Devesa slarengh@gmail.com 2024-11-06T12:10:07+01:00 GitHub noreply@github.com 2024-11-06T12:10:07+01:00 metal : fix from ptr buffer name (#10189) 1dc04b2deed2d2f2ae3aff9b14ae29674dee1fb8 a1eaf6a9600bb1608753420ba886a3b0a208ffc0 Georgi Gerganov ggerganov@gmail.com 2024-11-06T11:20:10+02:00 GitHub noreply@github.com 2024-11-06T11:20:10+02:00 ggml : adjust is_first_call init value (#10193) a1eaf6a9600bb1608753420ba886a3b0a208ffc0 b8deef0ec0af5febac1d2cfd9119ff330ed0b762 Georgi Gerganov ggerganov@gmail.com 2024-11-06T10:24:23+02:00 GitHub noreply@github.com 2024-11-06T10:24:23+02:00 metal : add quantized FA support (#10149) b8deef0ec0af5febac1d2cfd9119ff330ed0b762 a9e8a9a0306a8093eef93b0022d9f45510490072 Gabe Goodhart ghart@us.ibm.com 2024-11-05T05:23:04-07:00 GitHub noreply@github.com 2024-11-05T14:23:04+02:00 llama : add <|tool_call|> formatting to Granite template (#10177) a9e8a9a0306a8093eef93b0022d9f45510490072 340736477651095a98a3b10e19b038ec62593a1d Diego Devesa slarengh@gmail.com 2024-11-04T23:17:01+01:00 GitHub noreply@github.com 2024-11-04T23:17:01+01:00 ggml : fix arch check in bf16_to_fp32 (#10164) 340736477651095a98a3b10e19b038ec62593a1d d5a409e57fe8bd24fef597ab8a31110d390a6392 Eve 139727413+netrunnereve@users.noreply.github.com 2024-11-04T22:06:31Z GitHub noreply@github.com 2024-11-04T23:06:31+01:00 Q6_K AVX improvements (#10118) d5a409e57fe8bd24fef597ab8a31110d390a6392 401558b7ba7a08175c153cd3607230f63c8a528e Diego Devesa slarengh@gmail.com 2024-11-04T20:06:58+01:00 GitHub noreply@github.com 2024-11-04T20:06:58+01:00 ggml : fix gelu tables initialization (#10172) 401558b7ba7a08175c153cd3607230f63c8a528e 9e0ecfb697d297355e43c20559d29bcc71beb0c3 Diego Devesa slarengh@gmail.com 2024-11-04T17:34:08+01:00 GitHub noreply@github.com 2024-11-04T17:34:08+01:00 ggml : fix q4xx mat mul, increase ggml_aligned_malloc alignment (#10167) 9e0ecfb697d297355e43c20559d29bcc71beb0c3 6a066b9978533e2ab9890b7f4f8c0262d91798b3 Xuan Son Nguyen thichthat@gmail.com 2024-11-04T16:33:29+01:00 GitHub noreply@github.com 2024-11-04T16:33:29+01:00 server : clarify /slots endpoint, add is_processing (#10162) 6a066b9978533e2ab9890b7f4f8c0262d91798b3 ea02c753ebf9342114cb173f10b3ffc2af1e7d04 snadampal 87143774+snadampal@users.noreply.github.com 2024-11-04T09:08:33-06:00 GitHub noreply@github.com 2024-11-04T16:08:33+01:00 fix build break on arm64 linux (#10166) ea02c753ebf9342114cb173f10b3ffc2af1e7d04 05697f670b1ea28b80c39854832ea53527f75c55 Diego Devesa slarengh@gmail.com 2024-11-04T13:10:23+01:00 GitHub noreply@github.com 2024-11-04T13:10:23+01:00 cuda : clear error after changing peer access (#10153) 05697f670b1ea28b80c39854832ea53527f75c55 f8e58135cff1c373df2934306f9c9da99673c2ed Georgi Gerganov ggerganov@gmail.com 2024-11-04T13:49:34+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-04T13:49:34+02:00 metal : simplify f16 and f32 dequant kernels (#0) f8e58135cff1c373df2934306f9c9da99673c2ed 329ed914c959c510d076fb06b43eeb3f7b804d6f Georgi Gerganov ggerganov@gmail.com 2024-11-04T13:43:32+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-04T13:44:06+02:00 metal : move dequantize templates to beginning of MSL source (#0) 329ed914c959c510d076fb06b43eeb3f7b804d6f ce027adfb3b131f0d2368294fc276bb0e342b3f6 leo-pony nengjunma@outlook.com 2024-11-04T19:08:22+08:00 GitHub noreply@github.com 2024-11-04T19:08:22+08:00 CANN: adjust backend registry refactor. (#10158) ce027adfb3b131f0d2368294fc276bb0e342b3f6 284e5b0275cc1292096e72e808e41d17e8cdf019 Georgi Gerganov ggerganov@gmail.com 2024-11-04T10:33:37+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-04T10:33:37+02:00 sync : ggml 284e5b0275cc1292096e72e808e41d17e8cdf019 e2292aaa17cf8530b0d0d899909588c3a095799d Yuri Khrustalev ykhrustalev@users.noreply.github.com 2024-11-02T05:09:12-04:00 Georgi Gerganov ggerganov@gmail.com 2024-11-04T10:33:11+02:00 cmake : make it possible linking ggml as external lib (ggml/1003) e2292aaa17cf8530b0d0d899909588c3a095799d 9f409893519b4a6def46ef80cd6f5d05ac0fb157 Plamen Minev pacominev@gmail.com 2024-11-01T16:55:10+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-04T10:33:10+02:00 metal : fix minor string leaks (ggml/1004) 9f409893519b4a6def46ef80cd6f5d05ac0fb157 08828a6d7d0006a487c9655ba8ace0ebe35ecad1 Diego Devesa slarengh@gmail.com 2024-11-03T19:34:08+01:00 GitHub noreply@github.com 2024-11-03T19:34:08+01:00 ggml : move CPU backend to a separate file (#10144) 08828a6d7d0006a487c9655ba8ace0ebe35ecad1 1839f69130151ceeac4d01c0ef8964e1fb43bba6 Georgi Gerganov ggerganov@gmail.com 2024-11-03T15:18:40+02:00 GitHub noreply@github.com 2024-11-03T15:18:40+02:00 metal : minor fixup in FA kernel (#10143) 1839f69130151ceeac4d01c0ef8964e1fb43bba6 9830b6923b61f1e652a35afeac77aa5f886dad09 Georgi Gerganov ggerganov@gmail.com 2024-11-03T15:14:15+02:00 GitHub noreply@github.com 2024-11-03T05:14:15-08:00 flake.lock: Update (#10146) 9830b6923b61f1e652a35afeac77aa5f886dad09 42cadc74bda60afafb45b71b1a39d150ede0ed4d Christian Köhnenkamp cvk5@me.com 2024-11-02T23:35:31+01:00 GitHub noreply@github.com 2024-11-02T15:35:31-07:00 Add apple arm to presets (#10134) 42cadc74bda60afafb45b71b1a39d150ede0ed4d 45950415ed985830c59bf42cf9c9216b20cf08ef sasha0552 admin@sasha0552.org 2024-11-02T16:34:56Z GitHub noreply@github.com 2024-11-02T18:34:56+02:00 server : fix slot selection by lru (#10126) 45950415ed985830c59bf42cf9c9216b20cf08ef 1926d6e39d6f6358bc1a4c52316a560178be7233 Georgi Gerganov ggerganov@gmail.com 2024-11-02T18:34:00+02:00 GitHub noreply@github.com 2024-11-02T18:34:00+02:00 server : fix endpoint checks (#10135) 1926d6e39d6f6358bc1a4c52316a560178be7233 b634f8a26fef65210fd9fb2f87e83a2809535e89 Georgi Gerganov ggerganov@gmail.com 2024-11-02T15:18:56+02:00 GitHub noreply@github.com 2024-11-02T15:18:56+02:00 llama : adjust default context size + print warnings (#10136) b634f8a26fef65210fd9fb2f87e83a2809535e89 7554aa4655f44b33a29068f2b18c5976fae45f9d Diego Devesa slarengh@gmail.com 2024-11-02T13:08:53+01:00 GitHub noreply@github.com 2024-11-02T13:08:53+01:00 simple-chat : only add bos on first prompt (#10129) 7554aa4655f44b33a29068f2b18c5976fae45f9d a6744e43e80f4be6398fc7733a01642c846dce1d Xuan Son Nguyen thichthat@gmail.com 2024-11-02T12:53:17+01:00 GitHub noreply@github.com 2024-11-02T12:53:17+01:00 convert-lora : make `--base` optional (#10110) a6744e43e80f4be6398fc7733a01642c846dce1d e991e3127ff71a29e61fe1de5dd1cbd2e1df1858 Diego Devesa slarengh@gmail.com 2024-11-01T23:50:59+01:00 GitHub noreply@github.com 2024-11-01T23:50:59+01:00 llama : add simple-chat example (#10124) e991e3127ff71a29e61fe1de5dd1cbd2e1df1858 418f5eef262cea07c2af4f45ee6a88d882221fcb Diego Devesa slarengh@gmail.com 2024-11-01T23:48:26+01:00 GitHub noreply@github.com 2024-11-01T23:48:26+01:00 llama : use smart pointers for ggml resources (#10117) 418f5eef262cea07c2af4f45ee6a88d882221fcb ba6f62eb793d6617892d252f5c04d7685d908a38 Shupei Fan dymarkfan@outlook.com 2024-11-02T02:33:14+08:00 GitHub noreply@github.com 2024-11-01T19:33:14+01:00 vulkan : improve ggml_vk_create_buffer error handling (#9898) ba6f62eb793d6617892d252f5c04d7685d908a38 d865d1478cd4e403f82d793c2afcd0f943412f05 Georgi Gerganov ggerganov@gmail.com 2024-11-01T17:31:51+02:00 GitHub noreply@github.com 2024-11-01T17:31:51+02:00 readme : update hot topics d865d1478cd4e403f82d793c2afcd0f943412f05 1804adb0cfee4811eaf633741503d683a46e4c77 sasha0552 admin@sasha0552.org 2024-11-01T13:33:14Z GitHub noreply@github.com 2024-11-01T14:33:14+01:00 server : fix smart selection of available slot (#10120) 1804adb0cfee4811eaf633741503d683a46e4c77 815fe72adcea5ec79d358db6a4c479191f396b3c Georgi Gerganov ggerganov@gmail.com 2024-11-01T12:58:45+02:00 GitHub noreply@github.com 2024-11-01T12:58:45+02:00 ggml : remove ggml_scratch (#10121) 815fe72adcea5ec79d358db6a4c479191f396b3c f221d56220899f38f0126e683b2432bc79d1e3f6 Georgi Gerganov ggerganov@gmail.com 2024-11-01T10:28:24+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-01T10:28:24+02:00 sync : ggml f221d56220899f38f0126e683b2432bc79d1e3f6 e597e50794f07ec8dc24b9efb18f94ec6386fda0 Georgi Gerganov ggerganov@gmail.com 2024-11-01T10:23:05+02:00 Georgi Gerganov ggerganov@gmail.com 2024-11-01T10:24:50+02:00 ggml : alloc ggml_contexts on the heap (whisper/2525) e597e50794f07ec8dc24b9efb18f94ec6386fda0 85679d37f34f66783cc04664a06c405b28e8e035 Zhenwei Jin 109658203+kylo5aby@users.noreply.github.com 2024-11-01T11:09:59+08:00 GitHub noreply@github.com 2024-11-01T11:09:59+08:00 build: fix build error in Windows env with OneAPI setup (#10107) 85679d37f34f66783cc04664a06c405b28e8e035 1e9f94994ef908d964cf81069f03d9d3668beb7d Diego Devesa slarengh@gmail.com 2024-11-01T00:49:53+01:00 GitHub noreply@github.com 2024-11-01T00:49:53+01:00 llama : improve output buffer type selection (#10098) 1e9f94994ef908d964cf81069f03d9d3668beb7d c02e5ab2a675c8bc1abc8b1e4cb6a93b26bdcce7 Diego Devesa slarengh@gmail.com 2024-11-01T00:45:34+01:00 GitHub noreply@github.com 2024-11-01T00:45:34+01:00 quantize : fix --keep-split (#10114) c02e5ab2a675c8bc1abc8b1e4cb6a93b26bdcce7 ab3d71f97f5b2915a229099777af00d3eada1d24 Diego Devesa slarengh@gmail.com 2024-10-31T22:54:23+01:00 GitHub noreply@github.com 2024-10-31T22:54:23+01:00 llama : fix buffer checks for mamba and rwk (#10111) ab3d71f97f5b2915a229099777af00d3eada1d24 0a683e8088d849626e7471f9e2ed381f7dbdf2e9 Zhenwei Jin 109658203+kylo5aby@users.noreply.github.com 2024-11-01T02:50:39+08:00 GitHub noreply@github.com 2024-10-31T19:50:39+01:00 loader: refactor tensor weights storage (#9935) 0a683e8088d849626e7471f9e2ed381f7dbdf2e9 dea5e86051aadcdf42f7db7a8855a78d8f5ff3d6 Kevin Gibbons bakkot@gmail.com 2024-10-31T06:02:35-07:00 GitHub noreply@github.com 2024-10-31T14:02:35+01:00 server : include scheme when printing URL (#10106) dea5e86051aadcdf42f7db7a8855a78d8f5ff3d6 1329c0a75e6a7defc5c380eaf80d8e0f66d7da78 Diego Devesa slarengh@gmail.com 2024-10-31T11:40:59+01:00 GitHub noreply@github.com 2024-10-31T11:40:59+01:00 ggml : check tensor name lengths in gguf files (#10100) 1329c0a75e6a7defc5c380eaf80d8e0f66d7da78 61408e7fad082dc44a11c8a9f1398da4837aad44 Sergio López slp@redhat.com 2024-10-31T10:09:52+01:00 GitHub noreply@github.com 2024-10-31T11:09:52+02:00 kompute: add mul_mat_q4_k shader (#10097) 61408e7fad082dc44a11c8a9f1398da4837aad44 b9e02e8184f5e6094a9e87eaf040becd404bfc90 Sergio López slp@redhat.com 2024-10-30T17:01:52+01:00 GitHub noreply@github.com 2024-10-30T17:01:52+01:00 kompute: add backend registry / device interfaces (#10045) b9e02e8184f5e6094a9e87eaf040becd404bfc90 6763f713bb692910e9b2d9d1a82d6959cee2dcf3 Diego Devesa slarengh@gmail.com 2024-10-30T14:51:21+01:00 GitHub noreply@github.com 2024-10-30T14:51:21+01:00 ggml : fix memory leaks when loading invalid gguf files (#10094) 6763f713bb692910e9b2d9d1a82d6959cee2dcf3 79a2bc042dcacaad59306865208a8c8c3149e3ea Rich Dougherty rich@rd.nz 2024-10-31T01:22:39+13:00 GitHub noreply@github.com 2024-10-30T13:22:39+01:00 readme : more lora detail in main example readme (#10064) 79a2bc042dcacaad59306865208a8c8c3149e3ea fc83a9e58479e4dd70054daa7afe5184c1bbe545 Rich Dougherty rich@rd.nz 2024-10-31T01:22:21+13:00 GitHub noreply@github.com 2024-10-30T13:22:21+01:00 convert : more detailed convert lora usage docs (#10065) fc83a9e58479e4dd70054daa7afe5184c1bbe545 c5b0f4b5d90297f3e729fca7f78ddb25fcab5ddc xctan axunlei@gmail.com 2024-10-30T15:00:40+08:00 GitHub noreply@github.com 2024-10-30T09:00:40+02:00 ggml : add Q4_0_8_8 RISC-V GEMV and GEMM kernels (#10029) c5b0f4b5d90297f3e729fca7f78ddb25fcab5ddc 8f275a7c4593aa34147595a90282cf950a853690 Diego Devesa slarengh@gmail.com 2024-10-30T02:01:23+01:00 GitHub noreply@github.com 2024-10-30T02:01:23+01:00 llama : refactor model loader with backend registry (#10026) 8f275a7c4593aa34147595a90282cf950a853690 8d8ff715367480b856ad86ac3888e9742b13a6fa Changyeon Kim cyzero.kim@samsung.com 2024-10-29T17:52:56+09:00 GitHub noreply@github.com 2024-10-29T09:52:56+01:00 ggml: Add POOL2D OP for GPU acceleration to the Vulkan backend in the MobileVLM model. (#9763) 8d8ff715367480b856ad86ac3888e9742b13a6fa 61715d5cc83a28181df6a641846e4f6a740f3c74 Georgi Gerganov ggerganov@gmail.com 2024-10-29T10:42:05+02:00 GitHub noreply@github.com 2024-10-29T10:42:05+02:00 llama : remove Tail-Free sampling (#10071) 61715d5cc83a28181df6a641846e4f6a740f3c74 07028f9d74d895da2ca4a1956624e3f07e04e620 arch-btw 57669023+arch-btw@users.noreply.github.com 2024-10-28T10:45:33-07:00 GitHub noreply@github.com 2024-10-28T18:45:33+01:00 llama : Add IBM granite template (#10013) 07028f9d74d895da2ca4a1956624e3f07e04e620 524afeec9dad7d765ce91f5cf30c73703867cb47 Georgi Gerganov ggerganov@gmail.com 2024-10-28T17:41:24+02:00 GitHub noreply@github.com 2024-10-28T08:41:24-07:00 flake.lock: Update (#10063) 524afeec9dad7d765ce91f5cf30c73703867cb47 8125e6cbfcf2b3b9066e4d923aca9295526730f5 R0CKSTAR xiaodong.ye@mthreads.com 2024-10-28T17:02:48+08:00 GitHub noreply@github.com 2024-10-28T10:02:48+01:00 musa: workaround for Guilty Lockup in cleaning src0 (#10042) 8125e6cbfcf2b3b9066e4d923aca9295526730f5 8841ce3f439de6e770f70319b7e08b6613197ea7 Georgi Gerganov ggerganov@gmail.com 2024-10-28T08:49:32+02:00 GitHub noreply@github.com 2024-10-28T08:49:32+02:00 server : don't overfill the batch during infill (#10018) 8841ce3f439de6e770f70319b7e08b6613197ea7 cc2983d3753c94a630ca7257723914d4c4f6122b Georgi Gerganov ggerganov@gmail.com 2024-10-27T20:59:58+02:00 GitHub noreply@github.com 2024-10-27T20:59:58+02:00 llama : switch KQ multiplication to F32 precision by default (#10015) cc2983d3753c94a630ca7257723914d4c4f6122b 8c60a8a46261ffb92b6d23a78acfac2fcb6fe233 Georgi Gerganov ggerganov@gmail.com 2024-10-26T10:34:08+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-26T10:34:08+03:00 sync : ggml 8c60a8a46261ffb92b6d23a78acfac2fcb6fe233 9e4a2563eadf34e9432d248224d4f43e8495e8fe bssrdf merlintiger@hotmail.com 2024-10-23T14:34:00-04:00 Georgi Gerganov ggerganov@gmail.com 2024-10-26T10:33:56+03:00 increase cuda_cpy block size (ggml/996) 9e4a2563eadf34e9432d248224d4f43e8495e8fe 668750357e66bfa3d1504b65699f5a0dfe3cb7cb Georgi Gerganov ggerganov@gmail.com 2024-10-26T10:33:31+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-26T10:33:31+03:00 scripts : fix amx sync [no ci] 668750357e66bfa3d1504b65699f5a0dfe3cb7cb ff252ea48e90e6552010fd74584334fb41bdd387 Georgi Gerganov ggerganov@gmail.com 2024-10-25T22:26:15+03:00 GitHub noreply@github.com 2024-10-25T22:26:15+03:00 metal : support permuted matrix multiplicaions (#10033) ff252ea48e90e6552010fd74584334fb41bdd387 d80fb71f8b8bf69ec095ba281f8248d136d21c76 wwoodsTM 104587230+wwoodsTM@users.noreply.github.com 2024-10-25T10:07:34-06:00 GitHub noreply@github.com 2024-10-25T19:07:34+03:00 llama : add DRY sampler (#9702) d80fb71f8b8bf69ec095ba281f8248d136d21c76 2f8bd2b90133cf37ae752015e1bfd738cc6d0112 Michael Podvitskiy podvitskiymichael@gmail.com 2024-10-25T17:57:54+02:00 GitHub noreply@github.com 2024-10-25T17:57:54+02:00 llama: string_split fix (#10022) 2f8bd2b90133cf37ae752015e1bfd738cc6d0112 bc5ba007b2c83ac95875e68724dabfc12159fc61 Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-10-25T12:57:41+05:30 GitHub noreply@github.com 2024-10-25T10:27:41+03:00 llamafile : extend sgemm.cpp support for Q5_0 models (#10010) bc5ba007b2c83ac95875e68724dabfc12159fc61 958367bf530d943a902afa1ce1c342476098576b Georgi Gerganov ggerganov@gmail.com 2024-10-25T10:13:46+03:00 GitHub noreply@github.com 2024-10-25T10:13:46+03:00 server : check that the prompt fits in the slot's context (#10030) 958367bf530d943a902afa1ce1c342476098576b 40f2555797f97314de749873cdc29dc102be66e2 Xuan Son Nguyen thichthat@gmail.com 2024-10-24T21:51:22+02:00 GitHub noreply@github.com 2024-10-24T21:51:22+02:00 server : refactor slot input data, move tokenizer to HTTP thread (#10023) 40f2555797f97314de749873cdc29dc102be66e2 167a515651a4b065a16225ffc69564c5674f3d0f Georgi Gerganov ggerganov@gmail.com 2024-10-24T21:23:33+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-24T21:23:33+03:00 ci : fix cmake flags for SYCL 167a515651a4b065a16225ffc69564c5674f3d0f c39665f589091903396a442a6ee56613303e0350 Johannes Gäßler johannesg@5d6.de 2024-10-24T14:40:23+02:00 GitHub noreply@github.com 2024-10-24T14:40:23+02:00 CUDA: fix insufficient buffer clearing for MMQ (#10032) c39665f589091903396a442a6ee56613303e0350 0a1c750c80147687df267114c81956757cc14382 Johannes Gäßler johannesg@5d6.de 2024-10-24T11:09:36+02:00 GitHub noreply@github.com 2024-10-24T11:09:36+02:00 CUDA: fix MMQ for non-contiguous src0, add tests (#10021) 0a1c750c80147687df267114c81956757cc14382 190a37d7977eb5bd6a729299bd1e371208c87149 wwoodsTM 104587230+wwoodsTM@users.noreply.github.com 2024-10-23T13:27:51-06:00 GitHub noreply@github.com 2024-10-23T22:27:51+03:00 server : samplers accept the prompt correctly (#10019) 190a37d7977eb5bd6a729299bd1e371208c87149 2d3aba9ee8da9c026d54e8a912a1d64f56809be3 Georgi Gerganov ggerganov@gmail.com 2024-10-23T17:23:55+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-23T17:23:55+03:00 sync : ggml 2d3aba9ee8da9c026d54e8a912a1d64f56809be3 80273a306d07ed95059d6130389deacb3b2d7196 Georgi Gerganov ggerganov@gmail.com 2024-10-23T17:16:56+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-23T17:16:56+03:00 llama.vim : bump generation time limit to 3s [no ci] 80273a306d07ed95059d6130389deacb3b2d7196 c19af0acb1fe6d0fdbecadd8483c1fbe5d68d095 Johannes Gäßler johannesg@5d6.de 2024-10-18T09:24:44+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-23T16:50:02+03:00 CUDA: fix 1D im2col, add tests (ggml/993) c19af0acb1fe6d0fdbecadd8483c1fbe5d68d095 ac113a0feee0935b2018312f7bc8d7a646b117ed Daniel Bevenius daniel.bevenius@gmail.com 2024-10-16T20:10:01+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-23T16:50:02+03:00 ggml : remove redundant set of contexts used field (ggml/978) ac113a0feee0935b2018312f7bc8d7a646b117ed 4c9388fb96ac2415fbb1239b7ba8346616606e2e Michael Coppola m18coppola@gmail.com 2024-10-23T07:09:26-04:00 GitHub noreply@github.com 2024-10-23T14:09:26+03:00 llama.vim : add classic vim support (#9995) 4c9388fb96ac2415fbb1239b7ba8346616606e2e 873279b1592e433c4d9eb5065091cc98473c7bee Jun Hee Yoo contact.jhyoo@gmail.com 2024-10-23T19:33:45+09:00 GitHub noreply@github.com 2024-10-23T13:33:45+03:00 metal : add POOL2D and fix IM2COL (#9943) 873279b1592e433c4d9eb5065091cc98473c7bee c8c07d658a6cefc5a50cfdf6be7d726503612303 github-actions[bot] github-actions[bot]@users.noreply.github.com 2024-10-20T00:22:59Z Someone else@someonex.net 2024-10-23T01:28:07Z flake.lock: Update c8c07d658a6cefc5a50cfdf6be7d726503612303 19d900a7565b8f6b0a708836a57d26966cb9efe2 Xuan Son Nguyen thichthat@gmail.com 2024-10-22T16:59:02+02:00 GitHub noreply@github.com 2024-10-22T16:59:02+02:00 llama : fix empty batch causing llama_batch_allocr to crash (#9966) 19d900a7565b8f6b0a708836a57d26966cb9efe2 11d47057a51f3d9b9231e6b57d0ca36020c0ee99 Daniel Bevenius daniel.bevenius@gmail.com 2024-10-22T15:31:06+02:00 GitHub noreply@github.com 2024-10-22T16:31:06+03:00 llama : rename batch to ubatch (#9950) 11d47057a51f3d9b9231e6b57d0ca36020c0ee99 c421ac072d46172ab18924e1e8be53680b54ed3b Molly Sophia mollysophia379@gmail.com 2024-10-22T21:22:26+08:00 GitHub noreply@github.com 2024-10-22T15:22:26+02:00 Rwkv chat template fix (#10001) c421ac072d46172ab18924e1e8be53680b54ed3b 4ff7fe1fb36b04ddd158b2de881c348c5f0ff5e4 Xuan Son Nguyen thichthat@gmail.com 2024-10-22T13:08:41+02:00 GitHub noreply@github.com 2024-10-22T13:08:41+02:00 lora : warn user if new token is added in the adapter (#9948) 4ff7fe1fb36b04ddd158b2de881c348c5f0ff5e4 6b8447352df3d662b56280c8fc38d7f092885787 Molly Sophia mollysophia379@gmail.com 2024-10-22T18:33:37+08:00 GitHub noreply@github.com 2024-10-22T13:33:37+03:00 llama : add chat template for RWKV-World + fix EOT (#9968) 6b8447352df3d662b56280c8fc38d7f092885787 674804a99617b4f90292b4080ecab450ea3d30ba leo-pony nengjunma@outlook.com 2024-10-22T16:16:01+08:00 GitHub noreply@github.com 2024-10-22T16:16:01+08:00 [CANN] Adapt to dynamically loadable backends mechanism (#9970) 674804a99617b4f90292b4080ecab450ea3d30ba e94a138d644a9b34da61805f7aeb8af595c61b53 Daniel Bevenius daniel.bevenius@gmail.com 2024-10-22T09:40:02+02:00 GitHub noreply@github.com 2024-10-22T10:40:02+03:00 arg : fix typo in embeddings argument help [no ci] (#9994) e94a138d644a9b34da61805f7aeb8af595c61b53 e01c67affe450638162a1a457e2e57859ef6ebf0 Georgi Gerganov ggerganov@gmail.com 2024-10-22T00:35:25+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-22T00:37:55+03:00 llama.vim : fix info text display [no ci] (#9787) e01c67affe450638162a1a457e2e57859ef6ebf0 994cfb1acb9144bc95be0ab319175f30737cc92b Georgi Gerganov ggerganov@gmail.com 2024-10-21T22:52:22+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-21T22:53:18+03:00 llama.vim : move info to the right of screen [no ci] (#9787) 994cfb1acb9144bc95be0ab319175f30737cc92b 94008cc76075fb4a29ee371e7ac255378d1bce6c Asghar Ghorbani a-ghorbani@users.noreply.github.com 2024-10-21T20:20:59+02:00 GitHub noreply@github.com 2024-10-21T21:20:59+03:00 readme : update UI list (#9972) 94008cc76075fb4a29ee371e7ac255378d1bce6c dbd5f2f5736aec6ff8fd63df3b351dae23c43e2f Daniel Bevenius daniel.bevenius@gmail.com 2024-10-21T20:12:52+02:00 GitHub noreply@github.com 2024-10-21T21:12:52+03:00 arg : fix attention non-causal arg value hint (#9985) dbd5f2f5736aec6ff8fd63df3b351dae23c43e2f f594bc80baf683818f29d8f5d6fb52daab99e572 Georgi Gerganov ggerganov@gmail.com 2024-10-21T20:25:02+03:00 GitHub noreply@github.com 2024-10-21T20:25:02+03:00 llama.vim : plugin for Neovim (#9787) f594bc80baf683818f29d8f5d6fb52daab99e572 d5ebd79c76abd4887f0283cd6f6f9689122094d0 Georgi Gerganov ggerganov@gmail.com 2024-10-21T16:20:46+03:00 GitHub noreply@github.com 2024-10-21T16:20:46+03:00 ggml : add asserts for type conversion in fattn kernels (#9971) d5ebd79c76abd4887f0283cd6f6f9689122094d0 55e47786e373c90fc7803e718e3e1dd6d53c3db6 Radoslav Gerganov rgerganov@gmail.com 2024-10-21T13:35:40+03:00 GitHub noreply@github.com 2024-10-21T13:35:40+03:00 rpc : pack only RPC structs (#9959) 55e47786e373c90fc7803e718e3e1dd6d53c3db6 bc219750845a59166d79f0d4ee3da1993b369b8a Georgi Gerganov ggerganov@gmail.com 2024-10-21T09:46:40+03:00 GitHub noreply@github.com 2024-10-21T09:46:40+03:00 llama : default sampling changes + greedy update (#9897) bc219750845a59166d79f0d4ee3da1993b369b8a 1db8c84fc62857e1e45c1c7ea93bcd5344cb3d31 Georgi Gerganov ggerganov@gmail.com 2024-10-21T09:37:12+03:00 GitHub noreply@github.com 2024-10-21T09:37:12+03:00 speculative : fix handling of some input params (#9963) 1db8c84fc62857e1e45c1c7ea93bcd5344cb3d31 45f097645efb11b6d09a5b4adbbfd7c312ac0126 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-10-21T14:26:09+08:00 GitHub noreply@github.com 2024-10-21T14:26:09+08:00 fix mul_mat_vec_q and *_vec_q error (#9939) 45f097645efb11b6d09a5b4adbbfd7c312ac0126 7cab2083c768dd92c20b105556c4165b59cd8a41 Loïc Carrère loic.carrere@gmail.com 2024-10-20T18:25:41+02:00 GitHub noreply@github.com 2024-10-20T19:25:41+03:00 readme : update bindings list (#9951) 7cab2083c768dd92c20b105556c4165b59cd8a41 cda0e4b648dde8fac162b3430b14a99597d3d74f icppWorld 124377669+icppWorld@users.noreply.github.com 2024-10-20T12:01:34-04:00 GitHub noreply@github.com 2024-10-20T19:01:34+03:00 readme : update infra list (#9942) cda0e4b648dde8fac162b3430b14a99597d3d74f afd9909a6481402844aecefa8a8908afdd7f52f1 Xuan Son Nguyen thichthat@gmail.com 2024-10-18T23:18:01+02:00 GitHub noreply@github.com 2024-10-18T23:18:01+02:00 llama : remove all_pos_0, all_pos_1, all_seq_id from llama_batch (#9745) afd9909a6481402844aecefa8a8908afdd7f52f1 87421a23e8c60e00a7b227d501e8aab2a1aff7ce Radoslav Gerganov rgerganov@gmail.com 2024-10-18T14:33:58+03:00 GitHub noreply@github.com 2024-10-18T14:33:58+03:00 rpc : backend refactoring (#9912) 87421a23e8c60e00a7b227d501e8aab2a1aff7ce 60ce97c9d809f4b040e90b597468b839df5728d0 Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-10-18T06:46:16+01:00 GitHub noreply@github.com 2024-10-18T06:46:16+01:00 [SYCL] Add SYCL Backend registry, device and Event Interfaces (#9705) 60ce97c9d809f4b040e90b597468b839df5728d0 8901755ba328643c9ab071c20e1939ea52951a0e Ma Mingfei mingfei.ma@intel.com 2024-10-18T13:34:36+08:00 GitHub noreply@github.com 2024-10-18T13:34:36+08:00 add amx kernel for gemm (#8998) 8901755ba328643c9ab071c20e1939ea52951a0e 6f55bccbb8835d42147add4ee48807450f5ff535 Georgi Gerganov ggerganov@gmail.com 2024-10-18T07:32:19+03:00 GitHub noreply@github.com 2024-10-18T07:32:19+03:00 server : add n_indent parameter for line indentation requirement (#9929) 6f55bccbb8835d42147add4ee48807450f5ff535 17bb9280807cfbb6611b853aa1ef05114bd9efe9 Daniel Bevenius daniel.bevenius@gmail.com 2024-10-18T01:41:51+02:00 GitHub noreply@github.com 2024-10-18T01:41:51+02:00 llama : rename batch_all to batch (#8881) 17bb9280807cfbb6611b853aa1ef05114bd9efe9 9f45fc1e9950a496febc575cdd196cd5cad000cc Georgi Gerganov ggerganov@gmail.com 2024-10-17T23:43:05+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-17T23:43:05+03:00 readme : remove --memory-f32 references (#9925) 9f45fc1e9950a496febc575cdd196cd5cad000cc 99bd4ac28c32cd17c0e337ff5601393b033dc5fc Georgi Gerganov ggerganov@gmail.com 2024-10-17T23:26:32+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-17T23:27:42+03:00 llama : change warning to debug log 99bd4ac28c32cd17c0e337ff5601393b033dc5fc 3752217ed5a6a11864682fbf009bcb36afffd6bc Georgi Gerganov ggerganov@gmail.com 2024-10-17T22:32:47+03:00 GitHub noreply@github.com 2024-10-17T22:32:47+03:00 llama : infill sampling handle very long tokens (#9924) 3752217ed5a6a11864682fbf009bcb36afffd6bc f010b77a372ffcfaf4338c670d6d3ecd89aa4eb6 Tim Wang overocean@gmail.com 2024-10-17T17:57:14+11:00 GitHub noreply@github.com 2024-10-17T09:57:14+03:00 readme : update bindings list (#9918) f010b77a372ffcfaf4338c670d6d3ecd89aa4eb6 21942002780352b4a54f4bd3e5eefa3bc7f14fe6 Diego Devesa slarengh@gmail.com 2024-10-17T02:46:58+02:00 GitHub noreply@github.com 2024-10-17T02:46:58+02:00 vulkan : add backend registry / device interfaces (#9721) 21942002780352b4a54f4bd3e5eefa3bc7f14fe6 73afe681aa76e818733fc1f30de082c1d6910bcd Gilad S. 7817232+giladgd@users.noreply.github.com 2024-10-17T02:34:22+03:00 GitHub noreply@github.com 2024-10-17T01:34:22+02:00 fix: allocating CPU buffer with size `0` (#9917) 73afe681aa76e818733fc1f30de082c1d6910bcd 9e041024481f6b249ab8918e18b9477f873b5a5e Gilad S. 7817232+giladgd@users.noreply.github.com 2024-10-17T01:36:51+03:00 GitHub noreply@github.com 2024-10-17T00:36:51+02:00 fix: use `vm_allocate` to allocate CPU backend buffer on macOS (#9875) 9e041024481f6b249ab8918e18b9477f873b5a5e dbf18e4de9e7aa496871f1555f9f0c8d84567108 Daniel Bevenius daniel.bevenius@gmail.com 2024-10-16T19:34:28+02:00 GitHub noreply@github.com 2024-10-16T20:34:28+03:00 llama : suppress conversion from 'size_t' to 'int' (#9046) dbf18e4de9e7aa496871f1555f9f0c8d84567108 66c2c93082289325199ae1f773f3b0ab2e399a47 Daniel Bevenius daniel.bevenius@gmail.com 2024-10-16T19:24:05+02:00 GitHub noreply@github.com 2024-10-16T20:24:05+03:00 llava : fix typo in error message [no ci] (#9884) 66c2c93082289325199ae1f773f3b0ab2e399a47 10433e8b457c4cfd759cbb41fc55fc398db4a5da Joe Eli McIlvain joe.eli.mac@gmail.com 2024-10-16T09:03:24-07:00 GitHub noreply@github.com 2024-10-16T19:03:24+03:00 grammar : fix JSON Schema for string regex with top-level alt. (#9903) 10433e8b457c4cfd759cbb41fc55fc398db4a5da 1f66b699c48cb5ab3265ed72c48e8549b1674291 Molly Sophia mollysophia379@gmail.com 2024-10-16T18:10:21+08:00 GitHub noreply@github.com 2024-10-16T13:10:21+03:00 llama : add tensor name for "result_norm" (#9907) 1f66b699c48cb5ab3265ed72c48e8549b1674291 0e41b300ed28f7fe185d938b2e3d56a0bf7411ed Alexey Parfenov zxed@alkatrazstudio.net 2024-10-16T08:35:53Z GitHub noreply@github.com 2024-10-16T11:35:53+03:00 server : fix the disappearance of the end of the text (#9867) 0e41b300ed28f7fe185d938b2e3d56a0bf7411ed cd60b88bf7ad7785fb6ac9864e360cf10e42faad Georgi Gerganov ggerganov@gmail.com 2024-10-16T11:28:14+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-16T11:28:14+03:00 sync : ggml cd60b88bf7ad7785fb6ac9864e360cf10e42faad becfd387f6919d99ec34b76c2522f90ac250c489 Daniel Bevenius daniel.bevenius@gmail.com 2024-10-09T16:40:35+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-16T11:28:01+03:00 ggml-alloc : remove buffer_id from leaf_alloc (ggml/987) becfd387f6919d99ec34b76c2522f90ac250c489 755a9b2bf00fbae988e03a47e852b66eaddd113a leo-pony nengjunma@outlook.com 2024-10-16T08:51:46+08:00 GitHub noreply@github.com 2024-10-16T08:51:46+08:00 [CANN] Fix cann compilation error (#9891) 755a9b2bf00fbae988e03a47e852b66eaddd113a 223c25a72fcc3f65cdfd7f5d57edd5b44b550e18 Georgi Gerganov ggerganov@gmail.com 2024-10-15T16:35:33+03:00 GitHub noreply@github.com 2024-10-15T16:35:33+03:00 llama : add infill sampler (#9896) 223c25a72fcc3f65cdfd7f5d57edd5b44b550e18 fbc98b748e7b075e327bcf13237057f647678049 Georgi Gerganov ggerganov@gmail.com 2024-10-15T16:28:55+03:00 GitHub noreply@github.com 2024-10-15T16:28:55+03:00 server : improve infill context reuse (#9894) fbc98b748e7b075e327bcf13237057f647678049 dcdd535302fc9702a4709be25f56540d65163a44 MaggotHATE clay1326@gmail.com 2024-10-15T15:54:55+05:00 GitHub noreply@github.com 2024-10-15T12:54:55+02:00 sampling : add XTC sampler (#9742) dcdd535302fc9702a4709be25f56540d65163a44 4c42f93b22146c83b763d8cbee5fafc512746649 Georgi Gerganov ggerganov@gmail.com 2024-10-15T12:48:44+03:00 GitHub noreply@github.com 2024-10-15T12:48:44+03:00 server : update preact (#9895) 4c42f93b22146c83b763d8cbee5fafc512746649 a89f75e1b7b90cb2d4d4c52ca53ef9e9b466aa45 Michał Tuszyński srgtuszy@gmail.com 2024-10-15T10:20:34+02:00 GitHub noreply@github.com 2024-10-15T11:20:34+03:00 readme : update bindings list (#9889) a89f75e1b7b90cb2d4d4c52ca53ef9e9b466aa45 13dca2a54a394757d56fdd652b9f0df08f44ea22 VoidIsVoid 343750470@qq.com 2024-10-14T15:04:36+08:00 GitHub noreply@github.com 2024-10-14T10:04:36+03:00 server : handle "logprobs" field with false value (#9871) 13dca2a54a394757d56fdd652b9f0df08f44ea22 d4c19c0f5cdb1e512573e8c86c79e8d0238c73c4 agray3 agray3@users.noreply.github.com 2024-10-14T01:49:08+01:00 GitHub noreply@github.com 2024-10-14T02:49:08+02:00 Vectorize load instructions in dmmv f16 CUDA kernel (#9816) d4c19c0f5cdb1e512573e8c86c79e8d0238c73c4 c7181bd294757dd80a7904e3dd0fea2d0be914e7 Georgi Gerganov ggerganov@gmail.com 2024-10-13T21:31:35+03:00 GitHub noreply@github.com 2024-10-13T21:31:35+03:00 server : accept extra_context for the infill endpoint (#9874) c7181bd294757dd80a7904e3dd0fea2d0be914e7 92be9f12164f18ce845a5bab60cefa5f7fec6836 Georgi Gerganov ggerganov@gmail.com 2024-10-13T18:52:48+03:00 GitHub noreply@github.com 2024-10-13T18:52:48+03:00 server : reuse cached context chunks (#9866) 92be9f12164f18ce845a5bab60cefa5f7fec6836 edc265661cd707327297b6ec4d83423c43cb50a5 Georgi Gerganov ggerganov@gmail.com 2024-10-13T06:11:26+03:00 GitHub noreply@github.com 2024-10-12T20:11:26-07:00 flake.lock: Update (#9870) edc265661cd707327297b6ec4d83423c43cb50a5 1bde94dd024b632f98428f4bf2ce483295130779 Georgi Gerganov ggerganov@gmail.com 2024-10-12T16:14:27+03:00 GitHub noreply@github.com 2024-10-12T16:14:27+03:00 server : add option to time limit the generation phase (#9865) 1bde94dd024b632f98428f4bf2ce483295130779 95c76e8e92ecc93f784b185eafae36a0e7ad2fa3 Georgi Gerganov ggerganov@gmail.com 2024-10-12T16:06:31+03:00 GitHub noreply@github.com 2024-10-12T16:06:31+03:00 server : remove self-extend features (#9860) 95c76e8e92ecc93f784b185eafae36a0e7ad2fa3 11ac9800aff532715a5bc7991062c68ba3472e6e Georgi Gerganov ggerganov@gmail.com 2024-10-12T14:51:54+03:00 GitHub noreply@github.com 2024-10-12T14:51:54+03:00 server : remove legacy system_prompt feature (#9857) 11ac9800aff532715a5bc7991062c68ba3472e6e 943d20b4111c746bcd9dbc7e4771de313b08b50c Georgi Gerganov ggerganov@gmail.com 2024-10-12T08:21:51+03:00 GitHub noreply@github.com 2024-10-12T08:21:51+03:00 llama : improve infill support and special token detection (#9798) 943d20b4111c746bcd9dbc7e4771de313b08b50c 96776405a17034dcfd53d3ddf5d142d34bdbb657 R0CKSTAR xiaodong.ye@mthreads.com 2024-10-12T13:09:53+08:00 GitHub noreply@github.com 2024-10-12T08:09:53+03:00 musa : update doc (#9856) 96776405a17034dcfd53d3ddf5d142d34bdbb657 7eee341bee09957139789c2d828995953f0fc7ff Diego Devesa slarengh@gmail.com 2024-10-11T15:34:45+02:00 GitHub noreply@github.com 2024-10-11T15:34:45+02:00 ggml : move more prints to the ggml log system (#9839) 7eee341bee09957139789c2d828995953f0fc7ff 0e9f760eb12546704ef8fa72577bc1a3ffe1bc04 Diego Devesa slarengh@gmail.com 2024-10-10T22:57:42+02:00 GitHub noreply@github.com 2024-10-10T22:57:42+02:00 common : use common_ prefix for common library functions (#9805) 0e9f760eb12546704ef8fa72577bc1a3ffe1bc04 cf8e0a3bb9c0e93e371773b282054cdbbb231038 Diego Devesa slarengh@gmail.com 2024-10-10T20:14:55+02:00 GitHub noreply@github.com 2024-10-10T20:14:55+02:00 rpc : add backend registry / device interfaces (#9812) cf8e0a3bb9c0e93e371773b282054cdbbb231038 c7499c557cc1efafaf0a6bc12963c39826299703 R0CKSTAR xiaodong.ye@mthreads.com 2024-10-11T02:10:37+08:00 GitHub noreply@github.com 2024-10-10T20:10:37+02:00 musa: add docker image support (#9685) c7499c557cc1efafaf0a6bc12963c39826299703 c81f3bbb051f8b736e117dfc78c99d7c4e0450f6 Diego Devesa slarengh@gmail.com 2024-10-10T19:50:49+02:00 GitHub noreply@github.com 2024-10-10T19:50:49+02:00 examples : do not use common library in simple example (#9803) c81f3bbb051f8b736e117dfc78c99d7c4e0450f6 e7022064ab637ccb5f37867196f1802c4a453c91 Diego Devesa slarengh@gmail.com 2024-10-09T18:49:52+02:00 GitHub noreply@github.com 2024-10-09T18:49:52+02:00 cmake : do not build common library by default when standalone (#9804) e7022064ab637ccb5f37867196f1802c4a453c91 3dc48fe75ad48f8856118520a267c96f74df8e90 Georgi Gerganov ggerganov@gmail.com 2024-10-09T17:00:18+03:00 GitHub noreply@github.com 2024-10-09T17:00:18+03:00 perplexity : fix integer overflow (#9783) 3dc48fe75ad48f8856118520a267c96f74df8e90 dca1d4b58a7f1acf1bd253be84e50d6367f492fd Georgi Gerganov ggerganov@gmail.com 2024-10-09T10:55:42+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-09T10:55:42+03:00 examples : remove llama.vim dca1d4b58a7f1acf1bd253be84e50d6367f492fd 458367a90606448a9c0262b276947c9e536086e0 Diego Devesa slarengh@gmail.com 2024-10-08T14:21:43+02:00 GitHub noreply@github.com 2024-10-08T14:21:43+02:00 ggml : fix BLAS with unsupported types (#9775) 458367a90606448a9c0262b276947c9e536086e0 fa42aa6d8902cc4eaf31866b3b3b7b61b69da930 Xuan Son Nguyen thichthat@gmail.com 2024-10-08T13:27:04+02:00 GitHub noreply@github.com 2024-10-08T13:27:04+02:00 server : better security control for public deployments (#9776) fa42aa6d8902cc4eaf31866b3b3b7b61b69da930 6374743747b14db4eb73ce82ae449a2978bc3b47 standby24x7 standby24x7@gmail.com 2024-10-08T15:19:53+09:00 GitHub noreply@github.com 2024-10-08T09:19:53+03:00 scripts : fix spelling typo in messages and comments (#9782) 6374743747b14db4eb73ce82ae449a2978bc3b47 f1af42fa8c925096407c61ff0a3d5d5d669cc535 Diego Devesa slarengh@gmail.com 2024-10-07T21:55:08+02:00 GitHub noreply@github.com 2024-10-07T21:55:08+02:00 ggml : add backend registry / device interfaces to BLAS backend (#9752) f1af42fa8c925096407c61ff0a3d5d5d669cc535 6279dac039ddeb6d5ebd125a6274fd3c37a77ba8 Andrew Minh Nguyen 40281306+amqdn@users.noreply.github.com 2024-10-07T09:37:31-07:00 GitHub noreply@github.com 2024-10-07T09:37:31-07:00 Update building for Android (#9672) 6279dac039ddeb6d5ebd125a6274fd3c37a77ba8 d5ac8cf2f2e30459489e6721a17d15b92a0c42a6 Georgi Gerganov ggerganov@gmail.com 2024-10-07T19:35:42+03:00 GitHub noreply@github.com 2024-10-07T09:35:42-07:00 flake.lock: Update (#9753) d5ac8cf2f2e30459489e6721a17d15b92a0c42a6 96b69121033d2b6b951d1b6b1b43f8b4f97dac99 Georgi Gerganov ggerganov@gmail.com 2024-10-07T18:27:51+03:00 GitHub noreply@github.com 2024-10-07T18:27:51+03:00 ggml : add metal backend registry / device (#9713) 96b69121033d2b6b951d1b6b1b43f8b4f97dac99 d5cb86844f26f600c48bf3643738ea68138f961d Paul Tsochantaris ptsochantaris@icloud.com 2024-10-07T13:26:31+01:00 GitHub noreply@github.com 2024-10-07T15:26:31+03:00 metal : single allocation of encode_async block (#9747) d5cb86844f26f600c48bf3643738ea68138f961d f4b2dcdf4992ef11a854abc9b662624490e37b4c Georgi Gerganov ggerganov@gmail.com 2024-10-06T14:15:27+03:00 GitHub noreply@github.com 2024-10-06T14:15:27+03:00 contrib : simplify + minor edits [no ci] f4b2dcdf4992ef11a854abc9b662624490e37b4c b6d6c5289f1c9c677657c380591201ddb210b649 Georgi Gerganov ggerganov@gmail.com 2024-10-06T13:49:41+03:00 GitHub noreply@github.com 2024-10-06T13:49:41+03:00 readme : fix typo [no ci] b6d6c5289f1c9c677657c380591201ddb210b649 b0915d5b51cbaa982ce9bbb9ce302bb9abdca0eb Georgi Gerganov ggerganov@gmail.com 2024-10-06T12:53:28+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-06T12:53:28+03:00 sync : llama.cpp b0915d5b51cbaa982ce9bbb9ce302bb9abdca0eb 8c475b97b8ba7d678d4c9904b1161bd8811a9b44 SRHMorris 69468379+SRHMorris@users.noreply.github.com 2024-10-06T08:34:20+01:00 Georgi Gerganov ggerganov@gmail.com 2024-10-06T12:52:11+03:00 vulkan : retry allocation with fallback flags (whisper/2451) 8c475b97b8ba7d678d4c9904b1161bd8811a9b44 58b16695e146628481c6b9b8a3b101c0c9bac00f Georgi Gerganov ggerganov@gmail.com 2024-10-05T15:55:04+03:00 GitHub noreply@github.com 2024-10-05T15:55:04+03:00 rerank : use [SEP] token instead of [BOS] (#9737) 58b16695e146628481c6b9b8a3b101c0c9bac00f 905f5485b279518d30b402565c23fb153f822c0d Georgi Gerganov ggerganov@gmail.com 2024-10-05T15:53:49+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-05T15:53:49+03:00 sync : ggml 905f5485b279518d30b402565c23fb153f822c0d 71967c2a6d30da9f61580d3e2d4cb00e0223b6fa Georgi Gerganov ggerganov@gmail.com 2024-10-05T14:33:54+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-05T15:53:00+03:00 metal : zero-init buffer contexts (whisper/0) 71967c2a6d30da9f61580d3e2d4cb00e0223b6fa 17880771ad7dca16cdc969062f2a56f779662835 Viet-Anh NGUYEN (Andrew) vietanh.dev@gmail.com 2024-10-05T01:29:35+07:00 GitHub noreply@github.com 2024-10-04T20:29:35+02:00 Add Llama Assistant (#9744) 17880771ad7dca16cdc969062f2a56f779662835 55951c018d7c107b6ef0a4c8561a6e68183d19d9 Georgi Gerganov ggerganov@gmail.com 2024-10-04T18:50:25+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-04T18:50:25+03:00 sync : ggml 55951c018d7c107b6ef0a4c8561a6e68183d19d9 ff565769f289c6adcc91ed1b8fdabaf9a0d4f6ee Daniel Bevenius daniel.bevenius@gmail.com 2024-10-04T15:46:18+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-04T18:50:05+03:00 ggml : fix typo in example usage ggml_gallocr_new (ggml/984) ff565769f289c6adcc91ed1b8fdabaf9a0d4f6ee f3fdcfaa79afa12047def3a8793d4a566e0532d4 Diego Devesa slarengh@gmail.com 2024-10-04T08:41:40+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-04T18:50:04+03:00 ggml : fixes after sync (ggml/983) f3fdcfaa79afa12047def3a8793d4a566e0532d4 133c7b46b3482f7c126c0c4ba14265f684138306 Xuan Son Nguyen thichthat@gmail.com 2024-10-04T11:47:19+02:00 GitHub noreply@github.com 2024-10-04T11:47:19+02:00 ci : fine-grant permission (#9710) 133c7b46b3482f7c126c0c4ba14265f684138306 d5ed2b929d85bbd7dbeecb690880f07d9d7a6077 Daniel Kleine 53251018+d-kleine@users.noreply.github.com 2024-10-04T10:54:44+02:00 GitHub noreply@github.com 2024-10-04T10:54:44+02:00 Fixed RNG seed docs (#9723) d5ed2b929d85bbd7dbeecb690880f07d9d7a6077 1bb8a64ebfcbe599dacb4fc8069731b6cba0b5d6 Georgi Gerganov ggerganov@gmail.com 2024-10-03T21:18:19+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-03T21:18:19+03:00 metal : remove abort (skip) (ggml/0) 1bb8a64ebfcbe599dacb4fc8069731b6cba0b5d6 fabdc3bda396307565c4f3f4ecbc3a751a2eb6d3 Georgi Gerganov ggerganov@gmail.com 2024-10-03T21:17:49+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-03T21:17:49+03:00 sync : ggml fabdc3bda396307565c4f3f4ecbc3a751a2eb6d3 eee39bdc96065b69242877fe8f1be05c885fc2aa Johannes Gäßler johannesg@5d6.de 2024-10-03T17:29:59+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-03T21:17:26+03:00 ggml/ex: calculate accuracy in graph, adapt MNIST (ggml/980) eee39bdc96065b69242877fe8f1be05c885fc2aa 5d5ab1e5cca0d6d63701a1cb85dbe26cb57d2c4e Johannes Gäßler johannesg@5d6.de 2024-10-02T15:32:39+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-03T21:17:26+03:00 ggml: refactor cross entropy loss CPU impl. (ggml/976) 5d5ab1e5cca0d6d63701a1cb85dbe26cb57d2c4e a7ad553513a5d70b4ceacd36f64705cf3654dc97 Jack Mousseau jack@software.inc 2024-10-03T11:01:46-07:00 GitHub noreply@github.com 2024-10-03T21:01:46+03:00 metal : fix compute pass descriptor autorelease crash (#9718) a7ad553513a5d70b4ceacd36f64705cf3654dc97 d6fe7abf04e8ec5240dead6e2773ed1b7e7495d3 Diego Devesa slarengh@gmail.com 2024-10-03T17:39:18+02:00 GitHub noreply@github.com 2024-10-03T17:39:18+02:00 ggml-backend : add device description to CPU backend (#9720) d6fe7abf04e8ec5240dead6e2773ed1b7e7495d3 e3c355ba654d4164c1c09e5d0dcacecb8b214af8 bandoti 141645996+bandoti@users.noreply.github.com 2024-10-03T12:39:03-03:00 GitHub noreply@github.com 2024-10-03T17:39:03+02:00 ggml: unify backend logging mechanism (#9709) e3c355ba654d4164c1c09e5d0dcacecb8b214af8 841713e1e487bdb82fd106a52ad998c5f87b59e9 compilade git@compilade.net 2024-10-03T10:22:15-04:00 GitHub noreply@github.com 2024-10-03T17:22:15+03:00 convert : handle tokenizer merges format from transformers 4.45 (#9696) 841713e1e487bdb82fd106a52ad998c5f87b59e9 5639971466ed74386a1811938022f0c333007b55 Radoslav Gerganov rgerganov@gmail.com 2024-10-03T13:00:52+03:00 GitHub noreply@github.com 2024-10-03T13:00:52+03:00 rpc : enable vulkan (#9714) 5639971466ed74386a1811938022f0c333007b55 c83ad6d01e7b89ec71080d97c7e5db7ac1f4fda6 Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-10-03T07:50:44+01:00 GitHub noreply@github.com 2024-10-03T07:50:44+01:00 Fixed dequant precision issues in Q4_1 and Q5_1 (#9711) c83ad6d01e7b89ec71080d97c7e5db7ac1f4fda6 a39ab216aa624308fda7fa84439c6b61dc98b87a Diego Devesa slarengh@gmail.com 2024-10-03T01:49:47+02:00 GitHub noreply@github.com 2024-10-03T01:49:47+02:00 ggml-backend : add device and backend reg interfaces (#9707) a39ab216aa624308fda7fa84439c6b61dc98b87a f536f4c4391bec74c432a924625c04e8c484d3ee Xuan Son Nguyen thichthat@gmail.com 2024-10-02T15:49:55+02:00 GitHub noreply@github.com 2024-10-02T15:49:55+02:00 llama : reduce compile time and binary size (#9712) f536f4c4391bec74c432a924625c04e8c484d3ee 00b7317e636ffdc7dae59cb51dbd1cda357a3168 Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2024-10-02T13:57:18+01:00 GitHub noreply@github.com 2024-10-02T13:57:18+01:00 [SYCL] Initial cmake support of SYCL for AMD GPUs (#9658) 00b7317e636ffdc7dae59cb51dbd1cda357a3168 76b37d1541a880b9645557c8715a343fd074cc5c Radoslav Gerganov rgerganov@gmail.com 2024-10-02T13:49:16+03:00 GitHub noreply@github.com 2024-10-02T13:49:16+03:00 vulkan : do not use tensor->extra (#9407) 76b37d1541a880b9645557c8715a343fd074cc5c 148844fe97fff4c1563a3111bf238ba4dd22ef56 Zhenwei Jin 109658203+kylo5aby@users.noreply.github.com 2024-10-02T15:21:57+08:00 GitHub noreply@github.com 2024-10-02T10:21:57+03:00 gguf-split : improve --split and --merge logic (#9619) 148844fe97fff4c1563a3111bf238ba4dd22ef56 3f1ae2e32cde00c39b96be6d01c2997c29bae555 Georgi Gerganov ggerganov@gmail.com 2024-10-02T10:14:44+03:00 GitHub noreply@github.com 2024-10-02T10:14:44+03:00 examples : remove benchmark (#9704) 3f1ae2e32cde00c39b96be6d01c2997c29bae555 f1b8c4271125c2bfb9cfebd72a8b9e9f99061a30 Paweł Wodnicki 151604+32bitmicro@users.noreply.github.com 2024-10-01T12:18:46-05:00 GitHub noreply@github.com 2024-10-01T19:18:46+02:00 Update README.md (#9591) f1b8c4271125c2bfb9cfebd72a8b9e9f99061a30 e98c1c188ebbeb55d0cd6389ad03f0cbf995b451 Georgi Gerganov ggerganov@gmail.com 2024-10-01T16:09:42+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-01T16:09:42+03:00 sync : ggml e98c1c188ebbeb55d0cd6389ad03f0cbf995b451 cb00020504416606601f8cb35f55ee07b710b4b7 Johannes Gäßler johannesg@5d6.de 2024-09-30T09:55:23+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-01T16:07:40+03:00 test: fix OPT_STEP_ADAMW for test-backend-ops (ggml/974) cb00020504416606601f8cb35f55ee07b710b4b7 6c5322481a75f1be54e58a000c3f78484d07f948 Salvatore Mesoraca s.mesoraca16@gmail.com 2024-09-30T09:14:09+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-01T16:07:39+03:00 vulkan : mul_mat: fix UB with small warps (ggml/952) 6c5322481a75f1be54e58a000c3f78484d07f948 7254cdf7e8d6312840509ca8d766358c687c6266 Borislav Stanimirov b.stanimirov@abv.bg 2024-09-30T10:11:41+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-01T16:07:39+03:00 ggml : fix ggml_cast (ggml/973) 7254cdf7e8d6312840509ca8d766358c687c6266 cad341d88924788b940997c55e7bef000c99e784 Johannes Gäßler johannesg@5d6.de 2024-09-29T23:18:02+02:00 Georgi Gerganov ggerganov@gmail.com 2024-10-01T16:07:38+03:00 ggml: fix gradient allocation logic (ggml/966) cad341d88924788b940997c55e7bef000c99e784 a90484c6d9db699bf739d0f33daf1c50cbdd45c9 Georgi Gerganov ggerganov@gmail.com 2024-10-01T16:00:25+03:00 GitHub noreply@github.com 2024-10-01T16:00:25+03:00 metal : reduce command encoding overhead (#9698) a90484c6d9db699bf739d0f33daf1c50cbdd45c9 1927378bcce20ba72b6c89d5977b854a4bcaeb5d Georgi Gerganov ggerganov@gmail.com 2024-10-01T11:42:01+03:00 Georgi Gerganov ggerganov@gmail.com 2024-10-01T11:42:01+03:00 llama : print correct model type for Llama 3.2 1B and 3B 1927378bcce20ba72b6c89d5977b854a4bcaeb5d 6f1d9d71f4c568778a7637ff6582e6f6ba5fb9d3 compilade git@compilade.net 2024-10-01T02:31:36-04:00 GitHub noreply@github.com 2024-10-01T09:31:36+03:00 convert : refactor rope_freqs generation (#9396) 6f1d9d71f4c568778a7637ff6582e6f6ba5fb9d3 511636df0c90826b4dd1fc21ff260c19d69a3b5d serhii-nakon 57632032+serhii-nakon@users.noreply.github.com 2024-09-30T21:57:12+03:00 GitHub noreply@github.com 2024-09-30T20:57:12+02:00 Fix Docker ROCM builds, use AMDGPU_TARGETS instead of GPU_TARGETS (#9641) 511636df0c90826b4dd1fc21ff260c19d69a3b5d 08a43d05b6ba74de97610ae519450ad9996475e0 compilade git@compilade.net 2024-09-30T14:13:16-04:00 GitHub noreply@github.com 2024-09-30T14:13:16-04:00 ci : reduce severity of unused Pyright ignore comments (#9697) 08a43d05b6ba74de97610ae519450ad9996475e0 ace4f4be37abed4801fbd54a94cf38a7ae462416 vb vaibhavs10@gmail.com 2024-09-30T17:03:47+02:00 GitHub noreply@github.com 2024-09-30T18:03:47+03:00 py : update transfomers version (#9694) ace4f4be37abed4801fbd54a94cf38a7ae462416 8277a817f18967581b02b2248989d773e8e99998 Georgi Gerganov ggerganov@gmail.com 2024-09-30T17:48:49+03:00 GitHub noreply@github.com 2024-09-30T07:48:49-07:00 flake.lock: Update (#9680) 8277a817f18967581b02b2248989d773e8e99998 c919d5db39c8a7fcb64737f008e4b105ee0acd20 Ruchira Hasaranga ruchira66@gmail.com 2024-09-30T13:53:42+05:30 GitHub noreply@github.com 2024-09-30T11:23:42+03:00 console : utf-8 fix for windows stdin (#9690) c919d5db39c8a7fcb64737f008e4b105ee0acd20 d0b1d663e430354ab35853a6e1bce51cc8819376 Georgi Gerganov ggerganov@gmail.com 2024-09-29T21:18:23+03:00 GitHub noreply@github.com 2024-09-29T21:18:23+03:00 ggml : define missing HWCAP flags (#9684) d0b1d663e430354ab35853a6e1bce51cc8819376 aaa40999251f5d18309b3fddc5a7d576f5fdb4e1 Georgi Gerganov ggerganov@gmail.com 2024-09-29T21:16:07+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-29T21:16:07+03:00 sync : ggml aaa40999251f5d18309b3fddc5a7d576f5fdb4e1 641002fba8d2a0c0269027e23d2ef58e90546028 Johannes Gäßler johannesg@5d6.de 2024-09-29T19:56:17+02:00 Georgi Gerganov ggerganov@gmail.com 2024-09-29T21:15:37+03:00 CUDA: remove bad assert (ggml/972) 641002fba8d2a0c0269027e23d2ef58e90546028 0de8b203f1d31cf5ee0d2a3560a0ad78d44f4d4c Jeff Bolz jbolz@nvidia.com 2024-09-29T11:50:17-05:00 Georgi Gerganov ggerganov@gmail.com 2024-09-29T21:15:37+03:00 vulkan : multithread pipeline creation (ggml/963) 0de8b203f1d31cf5ee0d2a3560a0ad78d44f4d4c 544f409b4bd8fc98a3e87820f0ac934e00402de7 Jeff Bolz jbolz@nvidia.com 2024-09-27T02:58:01-05:00 Georgi Gerganov ggerganov@gmail.com 2024-09-29T21:15:37+03:00 vulkan : fix build for GGML_VULKAN_RUN_TESTS, add TFLOPS to log (ggml/961) 544f409b4bd8fc98a3e87820f0ac934e00402de7 6084bfb261b03f812de2255b05b6b5bb8d1c7171 Salvatore Mesoraca s.mesoraca16@gmail.com 2024-09-26T08:59:42+02:00 Georgi Gerganov ggerganov@gmail.com 2024-09-29T21:15:37+03:00 vulkan : argsort barriers must be under uniform control flow (ggml/951) 6084bfb261b03f812de2255b05b6b5bb8d1c7171 faac0bae265449fd988c57bf894018edc36fbe1e Georgi Gerganov ggerganov@gmail.com 2024-09-24T13:23:59+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-29T21:15:35+03:00 ggml : fix GGML_MAX_N_THREADS + improve formatting (ggml/969) faac0bae265449fd988c57bf894018edc36fbe1e f99d3f8367174f7aba73c07fd87de687d4a0ece1 matiaslin 45382001+matiaslin@users.noreply.github.com 2024-09-29T05:25:00-07:00 GitHub noreply@github.com 2024-09-29T15:25:00+03:00 common : ensure llama_batch size does not exceed max size (#9668) f99d3f8367174f7aba73c07fd87de687d4a0ece1 589b48d41efb0e95133b77c335f4fb9779af9bfb nopperl 54780682+nopperl@users.noreply.github.com 2024-09-29T12:02:06Z GitHub noreply@github.com 2024-09-29T15:02:06+03:00 py : add model class for Chameleon conversion (#9683) 589b48d41efb0e95133b77c335f4fb9779af9bfb f4d2b8846a6b34419ff9e9491aee6cd95e444bfc Georgi Gerganov ggerganov@gmail.com 2024-09-29T14:38:18+03:00 GitHub noreply@github.com 2024-09-29T14:38:18+03:00 contrib : add Resources section (#9675) f4d2b8846a6b34419ff9e9491aee6cd95e444bfc 1b2f992cd2cff0b69e5abe78bb8888d51ed19d67 Georgi Gerganov ggerganov@gmail.com 2024-09-28T17:42:03+03:00 GitHub noreply@github.com 2024-09-28T17:42:03+03:00 llama : add reranking support (#9510) 1b2f992cd2cff0b69e5abe78bb8888d51ed19d67 739842703e32cd43443c45e0b4f6647cc4e6b3d6 slaren slarengh@gmail.com 2024-09-28T14:32:46+02:00 GitHub noreply@github.com 2024-09-28T14:32:46+02:00 test-backend-ops : use flops for some performance tests (#9657) 739842703e32cd43443c45e0b4f6647cc4e6b3d6 6102037bbb55521880ae78a6ee6c2a0c00c901df Georgi Gerganov ggerganov@gmail.com 2024-09-28T15:13:21+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-28T15:13:42+03:00 llama : add comment about thread-safety [no ci] (#9449) 6102037bbb55521880ae78a6ee6c2a0c00c901df 9a913110cf471a8287ac06c43cbe307d3cf6df99 Zhenwei Jin 109658203+kylo5aby@users.noreply.github.com 2024-09-28T20:10:58+08:00 GitHub noreply@github.com 2024-09-28T15:10:58+03:00 vocab : refactor tokenizer to reduce init overhead (#9449) 9a913110cf471a8287ac06c43cbe307d3cf6df99 43bcdd9703ec19af7d2a519640b5ed6f4aac3d53 nopperl 54780682+nopperl@users.noreply.github.com 2024-09-28T12:08:43Z GitHub noreply@github.com 2024-09-28T15:08:43+03:00 llama : add support for Chameleon (#8543) 43bcdd9703ec19af7d2a519640b5ed6f4aac3d53 6a0f7794847244fb3b99a983e03137d7e832b585 Aarni Koskela akx@iki.fi 2024-09-28T15:07:14+03:00 GitHub noreply@github.com 2024-09-28T15:07:14+03:00 readme : add tool (#9655) 6a0f7794847244fb3b99a983e03137d7e832b585 89f9944981010d195e411a9fbfbb19959412f710 Dan Johansson 164997844+eddnjjn@users.noreply.github.com 2024-09-28T14:06:16+02:00 GitHub noreply@github.com 2024-09-28T15:06:16+03:00 ggml : add run-time detection of neon, i8mm and sve (#9331) 89f9944981010d195e411a9fbfbb19959412f710 b5de3b74a595cbfefab7eeb5a567425c6a9690cf Markus Tavenrath mtavenrath@users.noreply.github.com 2024-09-28T12:05:05+02:00 GitHub noreply@github.com 2024-09-28T12:05:05+02:00 Enable use to the rebar feature to upload buffers to the device. (#9251) b5de3b74a595cbfefab7eeb5a567425c6a9690cf 44f59b4301c51f071daa2e951301bb17c14acc9b Georgi Gerganov ggerganov@gmail.com 2024-09-27T20:57:51+03:00 GitHub noreply@github.com 2024-09-27T20:57:51+03:00 readme : update hot topics 44f59b4301c51f071daa2e951301bb17c14acc9b 95bc82fbc0df6d48cf66c857a4dda3d044f45ca2 Borislav Stanimirov b.stanimirov@abv.bg 2024-09-27T10:42:06+03:00 GitHub noreply@github.com 2024-09-27T10:42:06+03:00 cmake : add option for common library (#9661) 95bc82fbc0df6d48cf66c857a4dda3d044f45ca2 7691654c68aa5316108f881136c59f815ccb6809 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-09-26T17:38:31+08:00 GitHub noreply@github.com 2024-09-26T17:38:31+08:00 [SYCL] add missed dll file in package (#9577) 7691654c68aa5316108f881136c59f815ccb6809 ea9c32be71b91b42ecc538bd902e93cbb5fb36cb R0CKSTAR xiaodong.ye@mthreads.com 2024-09-26T09:27:40+08:00 GitHub noreply@github.com 2024-09-26T03:27:40+02:00 mtgpu: enable VMM (#9597) ea9c32be71b91b42ecc538bd902e93cbb5fb36cb 1e436302188a704ac9ea044af03193648806f19c Xuan Son Nguyen thichthat@gmail.com 2024-09-25T17:26:01+02:00 GitHub noreply@github.com 2024-09-25T17:26:01+02:00 ci : fix docker build number and tag name (#9638) 1e436302188a704ac9ea044af03193648806f19c afbbfaa537a96f562c34df4542930fa951b40d9e Charles Xu 63788048+chaxu01@users.noreply.github.com 2024-09-25T15:12:20+02:00 GitHub noreply@github.com 2024-09-25T16:12:20+03:00 ggml : remove assert for AArch64 GEMV and GEMM Q4 kernels (#9217) afbbfaa537a96f562c34df4542930fa951b40d9e 3d6bf6919f7b10726421779cd344f2da05421c68 Xuan Son Nguyen thichthat@gmail.com 2024-09-25T14:05:13+02:00 GitHub noreply@github.com 2024-09-25T14:05:13+02:00 server : add more env vars, improve gen-docs (#9635) 3d6bf6919f7b10726421779cd344f2da05421c68 904837e0cb2f5f01bf5d5901b7aa57a026860ae4 Gabe Goodhart ghart@us.ibm.com 2024-09-25T01:06:52-06:00 GitHub noreply@github.com 2024-09-25T10:06:52+03:00 llama : add IBM Granite MoE architecture (#9438) 904837e0cb2f5f01bf5d5901b7aa57a026860ae4 70392f1f81470607ba3afef04aa56c9f65587664 Dou Xinpeng 15529241576@163.com 2024-09-25T11:30:38+08:00 GitHub noreply@github.com 2024-09-25T11:30:38+08:00 cann: fix crash when llama-bench is running on multiple cann devices (#9627) 70392f1f81470607ba3afef04aa56c9f65587664 bb5f8199754b5f055cf436711d14c58e7be28e12 Eric Zhang 34133756+EZForever@users.noreply.github.com 2024-09-24T16:03:21+08:00 GitHub noreply@github.com 2024-09-24T11:03:21+03:00 ggml : add AVX512DQ requirement for AVX512 builds (#9622) bb5f8199754b5f055cf436711d14c58e7be28e12 c038931615d2525d732943fa8661e29aa361b192 Georgi Gerganov ggerganov@gmail.com 2024-09-24T11:01:18+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-24T11:01:18+03:00 sync : ggml c038931615d2525d732943fa8661e29aa361b192 31ac5834fe75c296476658a124c06c84772aa641 Georgi Gerganov ggerganov@gmail.com 2024-09-20T21:50:16+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-24T11:00:52+03:00 examples : adapt to ggml.h changes (ggml/0) 31ac5834fe75c296476658a124c06c84772aa641 cea1486ecf34a1c7e014a9e290eb458f5a11f876 Georgi Gerganov ggerganov@gmail.com 2024-09-24T10:16:06+03:00 GitHub noreply@github.com 2024-09-24T10:16:06+03:00 llama : keep track of all EOG tokens in the vocab (#9609) cea1486ecf34a1c7e014a9e290eb458f5a11f876 0aa15011e315659640504731d1d05663837130fa Georgi Gerganov ggerganov@gmail.com 2024-09-24T10:15:35+03:00 GitHub noreply@github.com 2024-09-24T10:15:35+03:00 log : add CONT level for continuing previous log entry (#9610) 0aa15011e315659640504731d1d05663837130fa b0f27361f3539a81d983a8b045f3c61e682d9fc0 StrangeBytesDev 141275258+StrangeBytesDev@users.noreply.github.com 2024-09-23T23:04:39-07:00 GitHub noreply@github.com 2024-09-24T09:04:39+03:00 server : add newline after chat example (#9616) b0f27361f3539a81d983a8b045f3c61e682d9fc0 c087b6f11d3385f4293b6841ebfb755063479490 Georgi Gerganov ggerganov@gmail.com 2024-09-24T09:03:17+03:00 GitHub noreply@github.com 2024-09-24T09:03:17+03:00 sampling : avoid expensive softmax during greedy sampling (#9605) c087b6f11d3385f4293b6841ebfb755063479490 116efee0eef09d8c3c4c60b52fa01b56ddeb432c Max Krasnyansky quic_maxk@quicinc.com 2024-09-23T21:18:48-07:00 GitHub noreply@github.com 2024-09-23T21:18:48-07:00 threads: fix msvc build without openmp (#9615) 116efee0eef09d8c3c4c60b52fa01b56ddeb432c 0b3bf966f47bf2ba88e5d4e3ed429602008c7e63 Ivan nekotekina@gmail.com 2024-09-24T03:14:24+03:00 GitHub noreply@github.com 2024-09-24T02:14:24+02:00 cuda: add q8_0->f32 cpy operation (#9571) 0b3bf966f47bf2ba88e5d4e3ed429602008c7e63 f0c7b5edf82aa200656fd88c11ae3a805d7130bf Xuan Son Nguyen thichthat@gmail.com 2024-09-23T22:23:54+02:00 GitHub noreply@github.com 2024-09-23T22:23:54+02:00 server : add --no-context-shift option (#9607) f0c7b5edf82aa200656fd88c11ae3a805d7130bf 1d48e98e4f3316bd2f6b187d288c7b6cb88d5cb3 Max Krasnyansky quic_maxk@quicinc.com 2024-09-23T11:42:43-07:00 GitHub noreply@github.com 2024-09-23T11:42:43-07:00 threads: improve ggml_barrier scaling with large number of threads (#9598) 1d48e98e4f3316bd2f6b187d288c7b6cb88d5cb3 f3979df762b75a2b1c0f622a2cd15d1bc60f037f Riceball LEE snowyu.lee@gmail.com 2024-09-23T23:58:17+08:00 GitHub noreply@github.com 2024-09-23T18:58:17+03:00 readme : add programmable prompt engine language CLI (#9599) f3979df762b75a2b1c0f622a2cd15d1bc60f037f 1e7b9299c6ccb5bbc55d3db7cfa9b51f3ab09b59 Georgi Gerganov ggerganov@gmail.com 2024-09-23T18:43:40+03:00 GitHub noreply@github.com 2024-09-23T08:43:40-07:00 flake.lock: Update (#9586) 1e7b9299c6ccb5bbc55d3db7cfa9b51f3ab09b59 37f8c7b4c97784496cfd91040d55fa22f50b1d57 Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-09-23T19:36:38+05:30 GitHub noreply@github.com 2024-09-23T17:06:38+03:00 ggml : AVX512 gemm for Q4_0_8_8 (#9532) 37f8c7b4c97784496cfd91040d55fa22f50b1d57 bf9c1013ac40e5f1bd8e60b6d8bf16e0e8401445 Georgi Gerganov ggerganov@gmail.com 2024-09-23T11:28:02+03:00 GitHub noreply@github.com 2024-09-23T11:28:02+03:00 perplexity : remove extra new lines after chunks (#9596) bf9c1013ac40e5f1bd8e60b6d8bf16e0e8401445 e62e9789cda3bf5573a747e55ec2a7ee32908f56 Georgi Gerganov ggerganov@gmail.com 2024-09-23T11:27:47+03:00 GitHub noreply@github.com 2024-09-23T11:27:47+03:00 metal : use F32 prec for K*Q in vec FA (#9595) e62e9789cda3bf5573a747e55ec2a7ee32908f56 c35e586ea57221844442c65a1172498c54971cb0 Akarshan Biswas akarshanbiswas@fedoraproject.org 2024-09-23T08:58:06+05:30 GitHub noreply@github.com 2024-09-23T11:28:06+08:00 Revert "[SYCL] fallback mmvq (#9088)" (#9579) c35e586ea57221844442c65a1172498c54971cb0 912c331d3dba3a079815844208dc36164baa8cc7 R0CKSTAR xiaodong.ye@mthreads.com 2024-09-22T22:55:49+08:00 GitHub noreply@github.com 2024-09-22T16:55:49+02:00 musa: enable building fat binaries, enable unified memory, and disable Flash Attention on QY1 (MTT S80) (#9526) 912c331d3dba3a079815844208dc36164baa8cc7 a5b57b08ce1998f7046df75324e86b9e2561c7af Molly Sophia mollysophia379@gmail.com 2024-09-22T21:26:50+08:00 GitHub noreply@github.com 2024-09-22T15:26:50+02:00 Fix merge error in #9454 (#9589) a5b57b08ce1998f7046df75324e86b9e2561c7af ecd5d6b65be08927e62de1587d5fd22778cdc250 Johannes Gäßler johannesg@5d6.de 2024-09-22T09:34:52+02:00 GitHub noreply@github.com 2024-09-22T09:34:52+02:00 CUDA: enable Gemma FA for HIP/Pascal (#9581) ecd5d6b65be08927e62de1587d5fd22778cdc250 2a63caaa69fad6c2afddc47bae052cf2afb01529 Shankar gshankar.87@gmail.com 2024-09-21T19:30:34-07:00 GitHub noreply@github.com 2024-09-22T04:30:34+02:00 llama: remove redundant loop when constructing ubatch (#9574) 2a63caaa69fad6c2afddc47bae052cf2afb01529 d09770cae71b416c032ec143dda530f7413c4038 Molly Sophia mollysophia379@gmail.com 2024-09-22T10:29:12+08:00 GitHub noreply@github.com 2024-09-22T04:29:12+02:00 RWKV v6: RWKV_WKV op CUDA implementation (#9454) d09770cae71b416c032ec143dda530f7413c4038 41f477879fd5ccc31211634292e8e293ec700e85 slaren slarengh@gmail.com 2024-09-21T14:24:23+02:00 GitHub noreply@github.com 2024-09-21T14:24:23+02:00 ggml-alloc : fix list of allocated tensors with GGML_ALLOCATOR_DEBUG (#9573) 41f477879fd5ccc31211634292e8e293ec700e85 e948a7da7af7f2dbfdcd695b3ba903ab12575f78 agray3 agray3@users.noreply.github.com 2024-09-21T01:41:07+01:00 GitHub noreply@github.com 2024-09-21T02:41:07+02:00 Update CUDA graph on scale change plus clear nodes/params (#9550) e948a7da7af7f2dbfdcd695b3ba903ab12575f78 63351143b2ea5efe9f8b9c61f553af8a51f1deff Huang Qi huangqi3@xiaomi.com 2024-09-21T08:39:41+08:00 GitHub noreply@github.com 2024-09-21T02:39:41+02:00 CI: Provide prebuilt windows binary for hip (#9467) 63351143b2ea5efe9f8b9c61f553af8a51f1deff d13edb17ed1ce3b961016cbdb616b1c8d161c026 slaren slarengh@gmail.com 2024-09-20T20:55:36+02:00 GitHub noreply@github.com 2024-09-20T20:55:36+02:00 quantize : improve type name parsing (#9570) d13edb17ed1ce3b961016cbdb616b1c8d161c026 27609c49b94766a136764ce7919c8a082bf71548 Georgi Gerganov ggerganov@gmail.com 2024-09-20T20:12:52+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-20T21:15:05+03:00 ggml : fix builds (#0) 27609c49b94766a136764ce7919c8a082bf71548 43015353262de835215103475055b74045cb9f49 Georgi Gerganov ggerganov@gmail.com 2024-09-20T19:13:02+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-20T21:15:05+03:00 ggml : fix trailing whitespace (#0) 43015353262de835215103475055b74045cb9f49 424c5d00a9b97dd5559635872db9b57f87c23b02 Georgi Gerganov ggerganov@gmail.com 2024-09-20T19:06:59+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-20T21:15:05+03:00 sync : ggml 424c5d00a9b97dd5559635872db9b57f87c23b02 a6809c6a2e8163cba296d4cc0c5cd4bbc8073638 Johannes Gäßler johannesg@5d6.de 2024-09-20T19:04:44+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-20T21:15:05+03:00 ggml/examples: add backend support for numerical optimization (ggml/949) a6809c6a2e8163cba296d4cc0c5cd4bbc8073638 5cb12f68395a5ec00b357e408883ce124a11dcdb Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:10:43+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-20T21:15:05+03:00 examples : add null threadpool args where needed (ggml/0) 5cb12f68395a5ec00b357e408883ce124a11dcdb d39e26741f9f02340651dbc640c9776e1a1128ef Johannes Gäßler johannesg@5d6.de 2024-09-20T18:35:35+02:00 GitHub noreply@github.com 2024-09-20T18:35:35+02:00 CUDA: fix sum.cu compilation for CUDA < 11.7 (#9562) d39e26741f9f02340651dbc640c9776e1a1128ef 722ec1eb51ed53be2ab1ef31c6a1da8261803c71 Georgi Gerganov ggerganov@gmail.com 2024-09-20T11:46:56+03:00 GitHub noreply@github.com 2024-09-20T11:46:56+03:00 examples : flush log upon ctrl+c (#9559) 722ec1eb51ed53be2ab1ef31c6a1da8261803c71 6026da52d6942b253df835070619775d849d0258 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-09-20T08:38:10+02:00 GitHub noreply@github.com 2024-09-20T09:38:10+03:00 perplexity : do not escape input data by default (#9548) 6026da52d6942b253df835070619775d849d0258 eca0fab44eb449ed34e17a9b651ae7398b494117 Georgi Gerganov ggerganov@gmail.com 2024-09-19T12:44:53+03:00 GitHub noreply@github.com 2024-09-19T12:44:53+03:00 server : clean-up completed tasks from waiting list (#9531) eca0fab44eb449ed34e17a9b651ae7398b494117 64c6af3195c3cd4aa3328a1282d29cd2635c34c9 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-09-19T09:58:14+02:00 GitHub noreply@github.com 2024-09-19T10:58:14+03:00 imatrix : disable prompt escape by default (#9543) 64c6af3195c3cd4aa3328a1282d29cd2635c34c9 0d2f22e45c3c3b6f8222acb6284d0c8c93443ba1 slaren slarengh@gmail.com 2024-09-18T19:13:08+02:00 GitHub noreply@github.com 2024-09-18T10:13:08-07:00 ggml : fix n_threads_cur initialization with one thread (#9538) 0d2f22e45c3c3b6f8222acb6284d0c8c93443ba1 6443ddd98576a9da904ef9f07df4e4398bb6a01a Georgi Gerganov ggerganov@gmail.com 2024-09-18T18:34:32+03:00 GitHub noreply@github.com 2024-09-18T18:34:32+03:00 scripts : verify py deps at the start of compare (#9520) 6443ddd98576a9da904ef9f07df4e4398bb6a01a 8a308354f6520df6bea851b435bd8054ee5617b4 Daniel Bevenius daniel.bevenius@gmail.com 2024-09-18T13:42:36+02:00 GitHub noreply@github.com 2024-09-18T14:42:36+03:00 llama : use reserve/emplace_back in sampler_sample (#9534) 8a308354f6520df6bea851b435bd8054ee5617b4 f799155ab8279cfa668086ce584aa52ecc05f5e5 Vinesh Janarthanan 36610342+VJHack@users.noreply.github.com 2024-09-18T01:50:34-05:00 GitHub noreply@github.com 2024-09-18T09:50:34+03:00 server : match OAI structured output response (#9527) f799155ab8279cfa668086ce584aa52ecc05f5e5 faf67b3de4688f47c3b1019c89df255df2fd59b4 Eric Zhang 34133756+EZForever@users.noreply.github.com 2024-09-18T14:28:20+08:00 GitHub noreply@github.com 2024-09-18T09:28:20+03:00 server : fix OpenSSL build (remove obsolete `LOG_INFO`) (#9529) faf67b3de4688f47c3b1019c89df255df2fd59b4 7be099fa817e9c53ffb4c3ed7d063e1cffcd675a Neo Zhang Jianyu jianyu.zhang@intel.com 2024-09-18T08:30:31+08:00 GitHub noreply@github.com 2024-09-18T08:30:31+08:00 [SYCL]set context default value to avoid memory issue, update guide (#9476) 7be099fa817e9c53ffb4c3ed7d063e1cffcd675a 8b836ae731bbb2c5640bc47df5b0a78ffcb129cb Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-17T22:41:38+02:00 GitHub noreply@github.com 2024-09-17T22:41:38+02:00 llama-bench: correct argument parsing error message (#9524) 8b836ae731bbb2c5640bc47df5b0a78ffcb129cb 8344ef58f8cdb8ebd6faf4463ca32ae91c374c81 Bert Wagner github@bertwagner.com 2024-09-17T09:35:38-04:00 GitHub noreply@github.com 2024-09-17T16:35:38+03:00 arg : add env variable for parallel (#9513) 8344ef58f8cdb8ebd6faf4463ca32ae91c374c81 0226613853133c081b55bb892a41bb5eacc0bc94 Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-17T12:18:22+02:00 GitHub noreply@github.com 2024-09-17T13:18:22+03:00 llama : fix n_vocab init for 'no_vocab' case (#9511) 0226613853133c081b55bb892a41bb5eacc0bc94 503147a9f9d195d6a14e7c998df23b6eb61f2bae Max Krasnyansky quic_maxk@quicinc.com 2024-09-17T01:19:46-07:00 GitHub noreply@github.com 2024-09-17T11:19:46+03:00 threadpool : skip polling for unused threads (#9461) 503147a9f9d195d6a14e7c998df23b6eb61f2bae 0d2ec438330271d201c2e9224aca23d0d5c908bf Yuri Khrustalev ykhrustalev@users.noreply.github.com 2024-09-17T02:51:15-04:00 GitHub noreply@github.com 2024-09-17T09:51:15+03:00 unicode : add (#9508) 0d2ec438330271d201c2e9224aca23d0d5c908bf 37f3a3810e545be6ac835c726f97956c1403ea02 Gabe Goodhart ghart@us.ibm.com 2024-09-17T00:44:58-06:00 GitHub noreply@github.com 2024-09-17T09:44:58+03:00 llama : support IBM Granite architecture (#9412) 37f3a3810e545be6ac835c726f97956c1403ea02 23e0d70bacaaca1429d365a44aa9e7434f17823b Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-17T08:23:30+02:00 GitHub noreply@github.com 2024-09-17T09:23:30+03:00 llama : add llama_n_head() (#9512) 23e0d70bacaaca1429d365a44aa9e7434f17823b acb2c32c336ce60d765bb189563cc216e57e9fc2 slaren slarengh@gmail.com 2024-09-16T16:22:07+02:00 GitHub noreply@github.com 2024-09-16T16:22:07+02:00 ggml : move common CPU backend impl to new header (#9509) acb2c32c336ce60d765bb189563cc216e57e9fc2 a6a3a5c531c73aef85750a847d21e7d4671e723d Daniel Bevenius daniel.bevenius@gmail.com 2024-09-16T13:07:13+02:00 GitHub noreply@github.com 2024-09-16T14:07:13+03:00 llama : rename n_embed to n_embd in rwkv6_time_mix (#9504) a6a3a5c531c73aef85750a847d21e7d4671e723d d54c21df7e2669c6cd7492713479d1aeb5846883 Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-16T13:06:50+02:00 GitHub noreply@github.com 2024-09-16T14:06:50+03:00 ggml : link MATH_LIBRARY not by its full path (#9339) d54c21df7e2669c6cd7492713479d1aeb5846883 19514d632e5274bc0c27c2269e8f2ad88b526d62 compilade git@compilade.net 2024-09-16T03:30:22-04:00 GitHub noreply@github.com 2024-09-16T10:30:22+03:00 convert : identify missing model files (#9397) 19514d632e5274bc0c27c2269e8f2ad88b526d62 5c3d0f1824714e9a97fc9b06e046eefcb6ecc721 Georgi Gerganov ggerganov@gmail.com 2024-09-16T10:27:50+03:00 GitHub noreply@github.com 2024-09-16T10:27:50+03:00 cmake : do not hide GGML options + rename option (#9465) 5c3d0f1824714e9a97fc9b06e046eefcb6ecc721 0aadac10c7dd704f8285ddf5a63d6f764cb340aa Eve 139727413+netrunnereve@users.noreply.github.com 2024-09-16T06:48:24Z GitHub noreply@github.com 2024-09-16T09:48:24+03:00 ggml : IQ4_NL sgemm + Q4_0 AVX optimization (#9422) 0aadac10c7dd704f8285ddf5a63d6f764cb340aa 95ca85168b5b089b80a811b59528ce0a2f1bd1dd Shane A shanea@allenai.org 2024-09-15T23:47:37-07:00 GitHub noreply@github.com 2024-09-16T09:47:37+03:00 llama : support OLMoE (#9462) 95ca85168b5b089b80a811b59528ce0a2f1bd1dd 441b72b91f818fe69497e5816f87969e90c73c43 CarryFun 76023481+CarryFun@users.noreply.github.com 2024-09-16T14:45:20+08:00 GitHub noreply@github.com 2024-09-16T09:45:20+03:00 llama : support MiniCPM3 (#9322) 441b72b91f818fe69497e5816f87969e90c73c43 c4965a64f72ac9434c21cf0e1c3421d13e889155 Vinesh Janarthanan 36610342+VJHack@users.noreply.github.com 2024-09-16T01:20:01-05:00 GitHub noreply@github.com 2024-09-16T09:20:01+03:00 main : option to disable context shift (#9484) c4965a64f72ac9434c21cf0e1c3421d13e889155 90a2fff0e7f80c6fea3fc6cf9a7b482744f3f164 Georgi Gerganov ggerganov@gmail.com 2024-09-16T09:05:56+03:00 GitHub noreply@github.com 2024-09-16T09:05:56+03:00 metal : handle zero-sized allocs (#9466) 90a2fff0e7f80c6fea3fc6cf9a7b482744f3f164 6262d13e0b2da91f230129a93a996609a2f5a2f2 Georgi Gerganov ggerganov@gmail.com 2024-09-16T05:14:23+03:00 GitHub noreply@github.com 2024-09-15T19:14:23-07:00 flake.lock: Update (#9488) 6262d13e0b2da91f230129a93a996609a2f5a2f2 e6deac31f7e62db43b6afbc3be814f764fd5a187 Georgi Gerganov ggerganov@gmail.com 2024-09-15T20:46:12+03:00 GitHub noreply@github.com 2024-09-15T20:46:12+03:00 common : reimplement logging (#9418) e6deac31f7e62db43b6afbc3be814f764fd5a187 6988da94a261444859f78595899212eeedc5ff9d slaren slarengh@gmail.com 2024-09-15T19:02:27+02:00 GitHub noreply@github.com 2024-09-15T19:02:27+02:00 gguf-split : add basic checks (#9499) 6988da94a261444859f78595899212eeedc5ff9d 3c7989fd29a2db2b75e28fd708cc441febe99a82 Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-15T18:55:52+02:00 GitHub noreply@github.com 2024-09-15T19:55:52+03:00 cmake : correct order of sycl flags (#9497) 3c7989fd29a2db2b75e28fd708cc441febe99a82 d6b37c881f056bd32b681dcd7658a37ea6ec3a1e Csaba Kecskemeti csaba.kecskemeti@gmail.com 2024-09-15T00:48:25-07:00 GitHub noreply@github.com 2024-09-15T10:48:25+03:00 py : add "LLaMAForCausalLM" conversion support (#9485) d6b37c881f056bd32b681dcd7658a37ea6ec3a1e 7596487bebd58eade3cd0133d42a9008aaaf9d09 OSecret 135510162+OLSecret@users.noreply.github.com 2024-09-15T10:36:53+03:00 GitHub noreply@github.com 2024-09-15T10:36:53+03:00 readme : update tools list (#9475) 7596487bebd58eade3cd0133d42a9008aaaf9d09 822b6322dea704110797a5671fc80ae39ee6ac97 Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-15T09:06:38+02:00 GitHub noreply@github.com 2024-09-15T10:06:38+03:00 cmake : try to fix sycl+intel build (#9487) 822b6322dea704110797a5671fc80ae39ee6ac97 dcdcee3a744f39714503ee2b19c49b7c7b6209c9 Yuri Khrustalev ykhrustalev@users.noreply.github.com 2024-09-14T05:54:37-04:00 GitHub noreply@github.com 2024-09-14T12:54:37+03:00 ggml : ggml_type_name return "NONE" for invalid values (#9458) dcdcee3a744f39714503ee2b19c49b7c7b6209c9 1f4111e540bacec8d00ca9fd96417bf4c1339394 VoidIsVoid 343750470@qq.com 2024-09-14T17:36:44+08:00 GitHub noreply@github.com 2024-09-14T11:36:44+02:00 server: add data: [DONE] to /chat/completions stream response (#9459) 1f4111e540bacec8d00ca9fd96417bf4c1339394 befaf1197fa447f61714de041828852a270659d2 Georgi Gerganov ggerganov@gmail.com 2024-09-14T10:55:05+03:00 GitHub noreply@github.com 2024-09-14T10:55:05+03:00 cmake : use list(APPEND ...) instead of set() + dedup linker (#9463) befaf1197fa447f61714de041828852a270659d2 feff4aa8461da7c432d144c11da4802e41fef3cf Daniel Bevenius daniel.bevenius@gmail.com 2024-09-14T09:50:12+02:00 GitHub noreply@github.com 2024-09-14T10:50:12+03:00 llama : make cell_id const in inp_s_mask block (#9470) feff4aa8461da7c432d144c11da4802e41fef3cf 0abc6a2c25272d5cf01384dda8ee8bfec4ba8745 Xuan Son Nguyen thichthat@gmail.com 2024-09-13T14:23:11+02:00 GitHub noreply@github.com 2024-09-13T14:23:11+02:00 server : add loading html page while model is loading (#9468) 0abc6a2c25272d5cf01384dda8ee8bfec4ba8745 bd35cb0ae357185c173345f10dc89a4ff925fc25 Georgi Gerganov ggerganov@gmail.com 2024-09-13T09:53:38+03:00 GitHub noreply@github.com 2024-09-13T09:53:38+03:00 llama : llama_perf + option to disable timings during decode (#9355) bd35cb0ae357185c173345f10dc89a4ff925fc25 78203641fee3b1f82abaff0c7f667e1b4a286390 Gilad S. 7817232+giladgd@users.noreply.github.com 2024-09-13T04:54:49+03:00 GitHub noreply@github.com 2024-09-13T03:54:49+02:00 feat: remove a sampler from a chain (#9445) 78203641fee3b1f82abaff0c7f667e1b4a286390 e6b7801bd189d102d901d3e72035611a25456ef1 Mathijs Henquet mathijs.henquet@gmail.com 2024-09-12T22:30:11+02:00 GitHub noreply@github.com 2024-09-12T22:30:11+02:00 server : Add option to return token pieces in /tokenize endpoint (#9108) e6b7801bd189d102d901d3e72035611a25456ef1 e665744317c77fc3483fc5224fe6d586b5166b33 Dou Xinpeng 81913537+Dou-Git@users.noreply.github.com 2024-09-12T19:46:43+08:00 GitHub noreply@github.com 2024-09-12T19:46:43+08:00 cann: Add host buffer type for Ascend NPU (#9406) e665744317c77fc3483fc5224fe6d586b5166b33 d4c3c10fad1bd6adec72d2f1f236761a8d6a07f8 fengerhu1 2748250768@qq.com 2024-09-12T19:34:22+08:00 GitHub noreply@github.com 2024-09-12T14:34:22+03:00 llava : fix the script error in MobileVLM README (#9054) d4c3c10fad1bd6adec72d2f1f236761a8d6a07f8 2a825116b6f7f3a9b1726e5e0c3eb22f7768bd33 Xuan Son Nguyen thichthat@gmail.com 2024-09-12T13:33:57+02:00 GitHub noreply@github.com 2024-09-12T14:33:57+03:00 lora : raise error if lm_head is ignored (#9103) 2a825116b6f7f3a9b1726e5e0c3eb22f7768bd33 4dc4f5f14ae522494649d82ad06b031cf9501038 Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-12T13:30:01+02:00 GitHub noreply@github.com 2024-09-12T14:30:01+03:00 cmake : fix for builds without `GGML_CDEF_PUBLIC` (#9338) 4dc4f5f14ae522494649d82ad06b031cf9501038 c837981bba7cf6839b69d32b25552ce685936b14 Huang Qi huangqi3@xiaomi.com 2024-09-12T19:28:43+08:00 GitHub noreply@github.com 2024-09-12T14:28:43+03:00 ci : update HIP SDK to 24.Q3 (ROCm 6.1) (#9329) c837981bba7cf6839b69d32b25552ce685936b14 3c26a1644dacfa6b5d58af550210524efd7b93fc daminho 37615795+daminho@users.noreply.github.com 2024-09-12T20:28:20+09:00 GitHub noreply@github.com 2024-09-12T14:28:20+03:00 py : add Phi-1.5/Phi-2 tokenizer (#9361) 3c26a1644dacfa6b5d58af550210524efd7b93fc ff76e18516dbe269b35ba1bb500524ed5e39225c Trivikram Kamat 16024985+trivikr@users.noreply.github.com 2024-09-12T04:27:45-07:00 GitHub noreply@github.com 2024-09-12T14:27:45+03:00 ci : bump actions/checkout to v4 (#9377) ff76e18516dbe269b35ba1bb500524ed5e39225c 39f852f44039b058fdd0611ee127c6efa7ba4a04 Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-12T13:27:14+02:00 GitHub noreply@github.com 2024-09-12T14:27:14+03:00 cmake : fixed the order of linking libraries for llama-quantize (#9450) 39f852f44039b058fdd0611ee127c6efa7ba4a04 2b00fa799773cc75d53b841c03d21d7468a1e3a1 Molly Sophia mollysophia379@gmail.com 2024-09-12T19:25:16+08:00 GitHub noreply@github.com 2024-09-12T14:25:16+03:00 py : add special tokens in hf_converter for RWKV v6 (#9428) 2b00fa799773cc75d53b841c03d21d7468a1e3a1 d6a04f872dea8ade92527bb1488d4b0b90cc49f0 Ahmad Tameem 113388789+Tameem-10xE@users.noreply.github.com 2024-09-12T16:24:31+05:00 GitHub noreply@github.com 2024-09-12T14:24:31+03:00 riscv : modify Makefile and add a RISCV_VECT to print log info (#9442) d6a04f872dea8ade92527bb1488d4b0b90cc49f0 c9c8575a1a8a170329afca4c4df4c005806efb1d Georgi Gerganov ggerganov@gmail.com 2024-09-12T14:23:49+03:00 GitHub noreply@github.com 2024-09-12T14:23:49+03:00 ggml : hide ggml_object, ggml_cgraph, ggml_hash_set (#9408) c9c8575a1a8a170329afca4c4df4c005806efb1d df4b7945aeccae2a71348e5a9c1eab5241e3e0ef Neo Zhang Jianyu jianyu.zhang@intel.com 2024-09-12T17:44:17+08:00 GitHub noreply@github.com 2024-09-12T17:44:17+08:00 enhance run script to be easy to change the parameters (#9448) df4b7945aeccae2a71348e5a9c1eab5241e3e0ef 449ccfb6f5f1bbd70e04f75a330d9d7c1af82187 Xinpeng Dou 81913537+Dou-Git@users.noreply.github.com 2024-09-12T09:02:35+08:00 GitHub noreply@github.com 2024-09-12T09:02:35+08:00 cann: Fix error when running a non-exist op (#9424) 449ccfb6f5f1bbd70e04f75a330d9d7c1af82187 1b28061400eb9832603c9f1dfbec4d339a8490a2 Faisal Zaghloul quic_fzaghlou@quicinc.com 2024-09-11T20:29:53-04:00 GitHub noreply@github.com 2024-09-12T02:29:53+02:00 Add Jais to list of supported models (#9439) 1b28061400eb9832603c9f1dfbec4d339a8490a2 8db003a19d7055b5bd248ce2afff9324e5b8da95 slaren slarengh@gmail.com 2024-09-11T17:52:13+02:00 GitHub noreply@github.com 2024-09-11T17:52:13+02:00 llama : skip token bounds check when evaluating embeddings (#9437) 8db003a19d7055b5bd248ce2afff9324e5b8da95 0996c5597f680effacc046832bb807c14900e22d Pavel Zloi github.com@drteam.rocks 2024-09-11T15:29:51+03:00 GitHub noreply@github.com 2024-09-11T15:29:51+03:00 py : support converting local models (#7547) 0996c5597f680effacc046832bb807c14900e22d 5bb2c5dbd26b246d334f0087b3cbd800f2e65c54 Xuan Son Nguyen thichthat@gmail.com 2024-09-11T12:59:13+02:00 GitHub noreply@github.com 2024-09-11T12:59:13+02:00 llava : correct args for minicpmv-cli (#9429) 5bb2c5dbd26b246d334f0087b3cbd800f2e65c54 67155ab7f5e47c01b62aa989eab30f517bf6dc67 Xuan Son Nguyen thichthat@gmail.com 2024-09-11T12:02:09+02:00 GitHub noreply@github.com 2024-09-11T13:02:09+03:00 files : remove accidentally added `lora_test` submodule (#9430) 67155ab7f5e47c01b62aa989eab30f517bf6dc67 5af118efdaf1098798a06b24fd8a557760e99631 Farbod Bijary 110523279+farbodbj@users.noreply.github.com 2024-09-11T12:52:37+03:30 GitHub noreply@github.com 2024-09-11T11:22:37+02:00 feat: Implements retrying logic for downloading models using --model-url flag (#9255) 5af118efdaf1098798a06b24fd8a557760e99631 d2b496bff4f353a6429f8e833448f071bd237ba7 Johannes Gäßler johannesg@5d6.de 2024-09-11T10:22:40+02:00 GitHub noreply@github.com 2024-09-11T10:22:40+02:00 CUDA: fix --split-mode row race condition (#9413) d2b496bff4f353a6429f8e833448f071bd237ba7 b34e02348064c2f0cef1f89b44d9bee4eb15b9e7 Georgi Gerganov ggerganov@gmail.com 2024-09-11T10:03:54+03:00 GitHub noreply@github.com 2024-09-11T10:03:54+03:00 batched-bench : remove unused code (#9305) b34e02348064c2f0cef1f89b44d9bee4eb15b9e7 51b603863627c4074e77b7e556e18ece86bdf9a3 R0CKSTAR xiaodong.ye@mthreads.com 2024-09-11T09:46:55+08:00 GitHub noreply@github.com 2024-09-11T03:46:55+02:00 musa: remove Clang builtins mapping (#9421) 51b603863627c4074e77b7e556e18ece86bdf9a3 cb9c933eb2a0d2b514556bdcb934b56dfe5d6771 Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2024-09-11T01:53:42+01:00 GitHub noreply@github.com 2024-09-11T08:53:42+08:00 sycl : update support conditions (#9394) cb9c933eb2a0d2b514556bdcb934b56dfe5d6771 6cd4e034442f71718563e600070c2b6fc389e100 Georgi Gerganov ggerganov@gmail.com 2024-09-11T01:46:59+03:00 GitHub noreply@github.com 2024-09-10T15:46:59-07:00 flake.lock: Update (#9360) 6cd4e034442f71718563e600070c2b6fc389e100 8d300bd35fbe23b35a4e1ece0cf0fe8f43331029 Xuan Son Nguyen thichthat@gmail.com 2024-09-10T22:41:29+02:00 GitHub noreply@github.com 2024-09-10T22:41:29+02:00 arg : bring back missing ifdef (#9411) 8d300bd35fbe23b35a4e1ece0cf0fe8f43331029 49006c67b4c6cc2e7c75a875b4d6e161ebae287c matteo matteogeniaccio@yahoo.it 2024-09-10T22:40:59+02:00 GitHub noreply@github.com 2024-09-10T22:40:59+02:00 enable --special arg for llama-server (#9419) 49006c67b4c6cc2e7c75a875b4d6e161ebae287c 00ba2ff78100e187ae17987bacd1c916211718b2 slaren slarengh@gmail.com 2024-09-10T18:04:25+02:00 GitHub noreply@github.com 2024-09-10T18:04:25+02:00 llama : move random seed generation to the samplers (#9398) 00ba2ff78100e187ae17987bacd1c916211718b2 83008b7cfe90ad89d0c0ed2c2424fd75edc25ac1 Georgi Gerganov ggerganov@gmail.com 2024-09-10T10:17:03+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-10T10:17:43+03:00 metal : fix compile warning with GGML_METAL_NDEBUG (#0) 83008b7cfe90ad89d0c0ed2c2424fd75edc25ac1 0b4ac75772b744bb0a0d674927587621d1057884 Daniel Bevenius daniel.bevenius@gmail.com 2024-09-10T09:03:21+02:00 GitHub noreply@github.com 2024-09-10T10:03:21+03:00 llama : update llm_build_copy_mask_state comment [no ci] (#9385) 0b4ac75772b744bb0a0d674927587621d1057884 fb3f2498156b3140e2050ec9c7bf61372f63ff56 Molly Sophia mollysophia379@gmail.com 2024-09-10T15:02:30+08:00 GitHub noreply@github.com 2024-09-10T10:02:30+03:00 RWKV v6: Add time_mix_decay_w1/w2 in quant exclusion list (#9387) fb3f2498156b3140e2050ec9c7bf61372f63ff56 bfe76d4a17228bfd1565761f203123bc4914771b slaren slarengh@gmail.com 2024-09-10T08:23:33+02:00 GitHub noreply@github.com 2024-09-10T09:23:33+03:00 make : do not run llama-gen-docs when building (#9399) bfe76d4a17228bfd1565761f203123bc4914771b 293bebe0773c907c0c866213856eeba41b035df1 Xuan Son Nguyen thichthat@gmail.com 2024-09-09T23:36:09+02:00 GitHub noreply@github.com 2024-09-09T23:36:09+02:00 common : move arg parser code to `arg.cpp` (#9388) 293bebe0773c907c0c866213856eeba41b035df1 5fac4d57643b1de8e9ab746f14d2fc4e319ae0c2 Radoslav Gerganov rgerganov@gmail.com 2024-09-09T18:40:10+03:00 GitHub noreply@github.com 2024-09-09T18:40:10+03:00 rpc : fix segfault with nkvo (#9389) 5fac4d57643b1de8e9ab746f14d2fc4e319ae0c2 5fb5e24811cb01d48b482c15a974bfbd9f433e1d Prashant Vithule 119530321+Vithulep@users.noreply.github.com 2024-09-09T21:07:18+05:30 GitHub noreply@github.com 2024-09-09T18:37:18+03:00 ggml : vector length agnostic SVE support (#9290) 5fb5e24811cb01d48b482c15a974bfbd9f433e1d 38ca6f644bd48301e9caa80f9913c22e70a8fd1b slaren slarengh@gmail.com 2024-09-09T17:10:46+02:00 GitHub noreply@github.com 2024-09-09T17:10:46+02:00 llama : minor sampling refactor (2) (#9386) 38ca6f644bd48301e9caa80f9913c22e70a8fd1b 8e6e2fbe1458ac91387266241262294a964d6b95 Georgi Gerganov ggerganov@gmail.com 2024-09-09T15:51:37+03:00 GitHub noreply@github.com 2024-09-09T15:51:37+03:00 readme : update hot topics 8e6e2fbe1458ac91387266241262294a964d6b95 5ed087573e1f326cfa70e29c1895d074a7a1a00c Johannes Gäßler johannesg@5d6.de 2024-09-09T14:22:53+02:00 GitHub noreply@github.com 2024-09-09T14:22:53+02:00 CUDA: fix variable name conflict for Windows build (#9382) 5ed087573e1f326cfa70e29c1895d074a7a1a00c 54f376d0b92c6ff6feb1fa2ef8ed2022348100ba Antonis Makropoulos benuix@gmail.com 2024-09-09T14:21:38+03:00 GitHub noreply@github.com 2024-09-09T14:21:38+03:00 readme : add LLMUnity to UI projects (#9381) 54f376d0b92c6ff6feb1fa2ef8ed2022348100ba b2e89a327457179a34eae4d7de0d412ed945679c Radoslav Gerganov rgerganov@gmail.com 2024-09-09T11:04:39+03:00 GitHub noreply@github.com 2024-09-09T11:04:39+03:00 rpc : update README [no ci] (#9320) b2e89a327457179a34eae4d7de0d412ed945679c daa9623ab051a8162ae750b150b9522571b55f21 Dan Johansson 164997844+eddnjjn@users.noreply.github.com 2024-09-09T09:02:45+02:00 GitHub noreply@github.com 2024-09-09T10:02:45+03:00 Arm AArch64: Documentation updates (#9321) daa9623ab051a8162ae750b150b9522571b55f21 e079bffb6678e1b5ded21c719600bedd7175e726 Markus Tavenrath mtavenrath@users.noreply.github.com 2024-09-08T21:43:48+02:00 GitHub noreply@github.com 2024-09-08T21:43:48+02:00 Overlap cmdbuffer creation and cmdbuffer execution in Vulkan backend by submitting smaller cmdbuffers early. (#9118) e079bffb6678e1b5ded21c719600bedd7175e726 3f7ccfd649abc83d059b5462221ac14de4ede6b7 Georgi Gerganov ggerganov@gmail.com 2024-09-08T22:01:02+03:00 GitHub noreply@github.com 2024-09-08T22:01:02+03:00 cuda : fix FA Q src index (1 -> 0) (#9374) 3f7ccfd649abc83d059b5462221ac14de4ede6b7 a249843d89bd6373772eede26d7f2aa708b26600 Xuan Son Nguyen thichthat@gmail.com 2024-09-08T18:08:55+02:00 GitHub noreply@github.com 2024-09-08T18:08:55+02:00 common : bring back missing args, add env var duplication check (#9375) a249843d89bd6373772eede26d7f2aa708b26600 19f4a7b296efda7c13a6b21d428b2286b5d1aa06 slaren slarengh@gmail.com 2024-09-08T16:44:42+02:00 GitHub noreply@github.com 2024-09-08T16:44:42+02:00 common : restore --n-gpu-layers (#9371) 19f4a7b296efda7c13a6b21d428b2286b5d1aa06 2a358fb0c4b6e917ac852aa17444cc94dd28a2a6 slaren slarengh@gmail.com 2024-09-08T15:52:07+02:00 GitHub noreply@github.com 2024-09-08T15:52:07+02:00 llama : refactor samplers internal implementation (#9370) 2a358fb0c4b6e917ac852aa17444cc94dd28a2a6 eae597182cb61bbde0b26e7cec5999d28b9327fe Neo Zhang Jianyu jianyu.zhang@intel.com 2024-09-08T19:05:29+08:00 GitHub noreply@github.com 2024-09-08T19:05:29+08:00 [SYCL] add check malloc result on device (#9346) eae597182cb61bbde0b26e7cec5999d28b9327fe 00b02bb249406ec0123757a67a5b714c3f33a699 slaren slarengh@gmail.com 2024-09-08T12:41:51+02:00 GitHub noreply@github.com 2024-09-08T12:41:51+02:00 llama : sanitize tokens in the upper bound (#9359) 00b02bb249406ec0123757a67a5b714c3f33a699 a8768614558262b6762fc0feeddcc6f7ce4bc5fc Xuan Son Nguyen thichthat@gmail.com 2024-09-08T12:12:17+02:00 GitHub noreply@github.com 2024-09-08T12:12:17+02:00 imatrix : fix arg parser for imatrix (#9366) a8768614558262b6762fc0feeddcc6f7ce4bc5fc 385decbd632f70db9f1fbd93dbb2b908853e135c Georgi Gerganov ggerganov@gmail.com 2024-09-08T09:57:57+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 metal : update support condition for im2col + fix warning (#0) 385decbd632f70db9f1fbd93dbb2b908853e135c 60a3107ccd791d858e12a87e6024ce918d3bf595 Georgi Gerganov ggerganov@gmail.com 2024-09-08T09:38:56+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 sync : ggml 60a3107ccd791d858e12a87e6024ce918d3bf595 406c1a32a18807b9b5711aa2fa1859527106bc1d Georgi Gerganov ggerganov@gmail.com 2024-09-08T09:38:42+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 scripts : option to increase git patch context 406c1a32a18807b9b5711aa2fa1859527106bc1d 9cb9260861e464e40d38764cfb021856786c7202 Salvatore Mesoraca s.mesoraca16@gmail.com 2024-09-06T14:34:25+02:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 vulkan: add dryrun support to sin and cos ops (ggml/947) 9cb9260861e464e40d38764cfb021856786c7202 202084d31d4247764fc6d6d40d2e2bda0c89a73a Salvatore Mesoraca s.mesoraca16@gmail.com 2024-09-06T14:34:07+02:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 vulkan: correctly report support for OP_CONT (ggml/946) 202084d31d4247764fc6d6d40d2e2bda0c89a73a dbbebcab339c7d63d3806e8f32574bb9aad9a694 Johannes Gäßler johannesg@5d6.de 2024-09-03T17:21:46+02:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 tests: add gradient tests for all backends (ggml/932) dbbebcab339c7d63d3806e8f32574bb9aad9a694 ba1cf846edeb94fc567a9ab0f7ef705282e6d7d3 Johannes Gäßler johannesg@5d6.de 2024-08-31T14:35:42+02:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 ggml: fix ggml_graph_cpy undefined behavior (ggml/943) ba1cf846edeb94fc567a9ab0f7ef705282e6d7d3 d2d3200b385970cb2a4658fd9342a3b1212bdb46 Georgi Gerganov ggerganov@gmail.com 2024-08-28T18:45:01+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 cann : fix doxy (ggml/0) d2d3200b385970cb2a4658fd9342a3b1212bdb46 51d964a4efdb0e8d43f5b85a775951185f6b641e Mengqing Cao cmq0113@163.com 2024-08-09T20:21:56+08:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 cann : add Ascend NPU support (whisper/2336) 51d964a4efdb0e8d43f5b85a775951185f6b641e efe6a83e3046a94cda38c8be5a7801eadc21d78d Georgi Gerganov ggerganov@gmail.com 2024-08-28T17:08:03+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 cuda : mark BF16 CONT as unsupported efe6a83e3046a94cda38c8be5a7801eadc21d78d fbb7fcffbcfc50009c275e75982b1603a6cb6b80 Salvatore Mesoraca s.mesoraca16@gmail.com 2024-08-28T10:23:02+02:00 Georgi Gerganov ggerganov@gmail.com 2024-09-08T11:05:55+03:00 ggml : fix cont with transposed tensors when one dimension is 1 (ggml/934) fbb7fcffbcfc50009c275e75982b1603a6cb6b80 a5b5d9a1014248f385939e6739e9db9de7147e55 Kevin Gibbons bakkot@gmail.com 2024-09-07T22:51:00-07:00 GitHub noreply@github.com 2024-09-08T08:51:00+03:00 llama : set attrs of mislabelled EOT/EOM tokens (#9348) a5b5d9a1014248f385939e6739e9db9de7147e55 f12295b8a9336962bf02a359beb1be0d322b4be7 Georgi Gerganov ggerganov@gmail.com 2024-09-08T00:33:50+03:00 GitHub noreply@github.com 2024-09-08T00:33:50+03:00 llama.android : fix build (#9350) f12295b8a9336962bf02a359beb1be0d322b4be7 faf69d4237c9ae4d7f572b4674d1002463e8acd3 Georgi Gerganov ggerganov@gmail.com 2024-09-08T00:33:33+03:00 GitHub noreply@github.com 2024-09-08T00:33:33+03:00 llama : fix empty ring buffer push (#9358) faf69d4237c9ae4d7f572b4674d1002463e8acd3 e536426ded3fb4a8cd13626e53508cd92928d6c2 Georgi Gerganov ggerganov@gmail.com 2024-09-08T00:33:13+03:00 GitHub noreply@github.com 2024-09-08T00:33:13+03:00 llama : sanitize invalid tokens (#9357) e536426ded3fb4a8cd13626e53508cd92928d6c2 1b9ae5189cd279c6b45e36d43e4f9ccae628d02f Eve 139727413+netrunnereve@users.noreply.github.com 2024-09-07T19:02:26Z GitHub noreply@github.com 2024-09-07T22:02:26+03:00 llamafile : disable sgemm for batch-size 1 (#9330) 1b9ae5189cd279c6b45e36d43e4f9ccae628d02f e32d0816edfd247784f6780b0bb9ae0bceef5e47 Xuan Son Nguyen thichthat@gmail.com 2024-09-07T20:43:51+02:00 GitHub noreply@github.com 2024-09-07T20:43:51+02:00 common : refactor arg parser (#9308) e32d0816edfd247784f6780b0bb9ae0bceef5e47 df270ef74596da8f1178f08991f4c51f18c9ee82 slaren slarengh@gmail.com 2024-09-07T20:23:07+02:00 GitHub noreply@github.com 2024-09-07T20:23:07+02:00 ggml : always check bounds on get_rows operations (#9354) df270ef74596da8f1178f08991f4c51f18c9ee82 947538acb8617756a092042ff7e58db18dde05ec Georgi Gerganov ggerganov@gmail.com 2024-09-07T15:16:19+03:00 GitHub noreply@github.com 2024-09-07T15:16:19+03:00 llama : refactor sampling v2 (#9294) 947538acb8617756a092042ff7e58db18dde05ec 6c89eb0b4749184f4d19e96ada5dcb8847129b9c Xuan Son Nguyen thichthat@gmail.com 2024-09-07T12:01:34+02:00 GitHub noreply@github.com 2024-09-07T12:01:34+02:00 ggml : fix missing `cpu_set_t` on emscripten (#9336) 6c89eb0b4749184f4d19e96ada5dcb8847129b9c 9b2c24c0993487d3b34a873980e292da571481f3 slaren slarengh@gmail.com 2024-09-07T09:48:54+02:00 GitHub noreply@github.com 2024-09-07T10:48:54+03:00 ci : disable rocm image creation (#9340) 9b2c24c0993487d3b34a873980e292da571481f3 134bc38ecf3e2c5460581badce289a1ffa680453 Xuan Son Nguyen thichthat@gmail.com 2024-09-06T23:21:29+02:00 GitHub noreply@github.com 2024-09-06T23:21:29+02:00 server : simplify state machine for slot (#9283) 134bc38ecf3e2c5460581badce289a1ffa680453 815b1fb20a53e439882171757825bacb1350de04 Aarni Koskela akx@iki.fi 2024-09-07T00:03:01+03:00 GitHub noreply@github.com 2024-09-06T23:03:01+02:00 llama-bench : log benchmark progress (#9287) 815b1fb20a53e439882171757825bacb1350de04 409dc4f8bb5185786087f52259ee4626be93f54d Aarni Koskela akx@iki.fi 2024-09-06T18:59:58+03:00 GitHub noreply@github.com 2024-09-06T17:59:58+02:00 batched-bench : add `--output-format jsonl` option (#9293) 409dc4f8bb5185786087f52259ee4626be93f54d 4a1411b4f17b6569dc0a067e5914dcc0f738b370 Changyeon Kim cyzero.kim@samsung.com 2024-09-06T21:54:50+09:00 GitHub noreply@github.com 2024-09-06T15:54:50+03:00 ggml : fix build break for the vulkan-debug (#9265) 4a1411b4f17b6569dc0a067e5914dcc0f738b370 8ebe8ddebd68526757c631cd019de009697c63c2 Xuan Son Nguyen thichthat@gmail.com 2024-09-06T14:06:04+02:00 GitHub noreply@github.com 2024-09-06T14:06:04+02:00 server : fix missing lock (#9334) 8ebe8ddebd68526757c631cd019de009697c63c2 9bc6db28d011d47a5f318dc4aebbe7927fac4629 Markus Tavenrath mtavenrath@users.noreply.github.com 2024-09-06T08:56:17+02:00 GitHub noreply@github.com 2024-09-06T08:56:17+02:00 Improve Vulkan shader build system (#9239) 9bc6db28d011d47a5f318dc4aebbe7927fac4629 32b2ec88bc44b086f3807c739daf28a1613abde1 compilade git@compilade.net 2024-09-05T21:48:47-04:00 GitHub noreply@github.com 2024-09-05T21:48:47-04:00 ggml-quants : ternary packing for TriLMs and BitNet b1.58 (#8151) 32b2ec88bc44b086f3807c739daf28a1613abde1 1031771faaba28d07b4ec6a812cb21532efc8c31 awatuna 23447591+awatuna@users.noreply.github.com 2024-09-06T06:34:36+08:00 GitHub noreply@github.com 2024-09-06T00:34:36+02:00 Update build.yml (#9184) 1031771faaba28d07b4ec6a812cb21532efc8c31 4db04784f96757d74f74c8c110c2a00d55e33514 Michael Podvitskiy podvitskiymichael@gmail.com 2024-09-06T00:14:12+02:00 GitHub noreply@github.com 2024-09-06T00:14:12+02:00 CMake fix: host for msvc compiler can only be x86 or x64 (#8624) 4db04784f96757d74f74c8c110c2a00d55e33514 bdf314f38a2c90e18285f7d7067e8d736a14000a slaren slarengh@gmail.com 2024-09-05T11:13:11+02:00 GitHub noreply@github.com 2024-09-05T11:13:11+02:00 cuda : fix defrag with quantized KV (#9319) bdf314f38a2c90e18285f7d7067e8d736a14000a 581c305186a0ff93f360346c57e21fe16e967bb7 slaren slarengh@gmail.com 2024-09-05T02:19:39+02:00 GitHub noreply@github.com 2024-09-05T02:19:39+02:00 llama-bench : fix NUL terminators in CPU name (#9313) 581c305186a0ff93f360346c57e21fe16e967bb7 5910ea942772ab6cbc21d0ad2d1208750ba39e1d Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-09-04T22:21:22+05:30 GitHub noreply@github.com 2024-09-04T19:51:22+03:00 ggml : AVX2 support for Q4_0_8_8 (#8713) 5910ea942772ab6cbc21d0ad2d1208750ba39e1d c8671ae28214b29920b86c66a5d36fd6dd0db870 Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-09-04T16:26:33+01:00 GitHub noreply@github.com 2024-09-04T16:26:33+01:00 [SYCL] Fix DMMV dequantization (#9279) c8671ae28214b29920b86c66a5d36fd6dd0db870 82e3b03c11826d20a24ab66d60f4de58f48ddcdb 杨朱 · Kiki baofa.fan@daocloud.io 2024-09-04T19:45:28+08:00 GitHub noreply@github.com 2024-09-04T13:45:28+02:00 Fix broken links in docker.md (#9306) 82e3b03c11826d20a24ab66d60f4de58f48ddcdb 9379d3cc1718e9ec6ab5ffcb60a06bbe048a61ee Radoslav Gerganov rgerganov@gmail.com 2024-09-04T11:08:32+03:00 GitHub noreply@github.com 2024-09-04T11:08:32+03:00 rpc : make RPC servers come first in the device list (#9296) 9379d3cc1718e9ec6ab5ffcb60a06bbe048a61ee 7605ae7daf31c02211bcfec2f46635ef6ec4b98a Pascal Patry ppatry@mtacitlabs.com 2024-09-04T02:45:40-04:00 GitHub noreply@github.com 2024-09-04T09:45:40+03:00 readme : rename result_format to response_format (#9300) 7605ae7daf31c02211bcfec2f46635ef6ec4b98a 8962422b1c6f9b8b15f5aeaea42600bcc2d44177 Georgi Gerganov ggerganov@gmail.com 2024-09-04T02:36:43+03:00 GitHub noreply@github.com 2024-09-03T16:36:43-07:00 flake.lock: Update (#9261) 8962422b1c6f9b8b15f5aeaea42600bcc2d44177 b69a480af4db8ffd6cbb2efe7ed8132bc7d39073 Aarni Koskela akx@iki.fi 2024-09-03T20:58:54+03:00 GitHub noreply@github.com 2024-09-03T19:58:54+02:00 llama-bench : add JSONL (NDJSON) output mode (#9288) b69a480af4db8ffd6cbb2efe7ed8132bc7d39073 48baa61eccdca9205daf8d620ba28055c2347b64 Georgi Gerganov ggerganov@gmail.com 2024-09-03T10:00:36+03:00 GitHub noreply@github.com 2024-09-03T10:00:36+03:00 readme : refactor API section + remove old hot topics 48baa61eccdca9205daf8d620ba28055c2347b64 f1485161e58d562099dd050f8ac3a9ea9f4cd765 Xuan Son Nguyen thichthat@gmail.com 2024-09-02T22:08:38+02:00 GitHub noreply@github.com 2024-09-02T22:08:38+02:00 server : test script : add timeout for all requests (#9282) f1485161e58d562099dd050f8ac3a9ea9f4cd765 048de848ee4baad4531fcd6239438f5d55be365c Zhenwei Jin 109658203+kylo5aby@users.noreply.github.com 2024-09-03T01:53:23+08:00 GitHub noreply@github.com 2024-09-02T13:53:23-04:00 src: make tail invalid when kv cell is intersection for mamba (#9249) 048de848ee4baad4531fcd6239438f5d55be365c f771d064a95d212ddadea452ad0cc1e42b2c3db3 slaren slarengh@gmail.com 2024-09-02T18:11:13+02:00 GitHub noreply@github.com 2024-09-02T18:11:13+02:00 docker : fix missing binaries in full-cuda image (#9278) f771d064a95d212ddadea452ad0cc1e42b2c3db3 6e7d133a5f9409dd257fad90d7f320721b07a1b2 yuri@FreeBSD yurivict@users.noreply.github.com 2024-09-02T08:25:30-07:00 GitHub noreply@github.com 2024-09-02T18:25:30+03:00 ggml : add pthread includes on FreeBSD (#9258) 6e7d133a5f9409dd257fad90d7f320721b07a1b2 b60074f1c26d8354053a952844ef3f7ebefd8803 Xuan Son Nguyen thichthat@gmail.com 2024-09-02T17:11:51+02:00 GitHub noreply@github.com 2024-09-02T17:11:51+02:00 server : refactor multitask handling (#9274) b60074f1c26d8354053a952844ef3f7ebefd8803 9c1ba557335c3cab46807e6478eb7d17a63361f1 Guoliang Hua 32868157+nbcsm@users.noreply.github.com 2024-09-02T20:36:43+08:00 GitHub noreply@github.com 2024-09-02T15:36:43+03:00 llama-cli : remove duplicated log message (#9275) 9c1ba557335c3cab46807e6478eb7d17a63361f1 c6d4cb46559b359d2682cf2a002e7fe01bb7a767 Tushar ditsuke@protonmail.com 2024-09-02T16:51:01+05:30 GitHub noreply@github.com 2024-09-02T14:21:01+03:00 build(nix): Package gguf-py (#5664) c6d4cb46559b359d2682cf2a002e7fe01bb7a767 8f1d81a0b6f50b9bad72db0b6fcd299ad9ecd48c Georgi Gerganov ggerganov@gmail.com 2024-09-02T11:52:04+03:00 Georgi Gerganov ggerganov@gmail.com 2024-09-02T11:52:37+03:00 llama : minor style 8f1d81a0b6f50b9bad72db0b6fcd299ad9ecd48c a47667cff41f5a198eb791974e0afcc1cddd3229 Molly Sophia mollysophia379@gmail.com 2024-09-01T22:38:17+08:00 GitHub noreply@github.com 2024-09-01T17:38:17+03:00 llama : support RWKV v6 models (#8980) a47667cff41f5a198eb791974e0afcc1cddd3229 ea5d7478b1edcfc83f8ea8f9f0934585cc0b92e3 Echo Nolan echo@echonolan.net 2024-08-22T17:19:14-04:00 Someone else@someonex.net 2024-08-31T08:44:21Z nix: fix CUDA build - replace deprecated autoAddOpenGLRunpathHook ea5d7478b1edcfc83f8ea8f9f0934585cc0b92e3 49271efbaf3f7a4ae52c4ca299b6d4e82598a97d Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-08-31T13:50:35+05:30 GitHub noreply@github.com 2024-08-31T11:20:35+03:00 sgemm : improved Q4_0 and Q8_0 performance via 4xN and Mx4 gemm (#8908) 49271efbaf3f7a4ae52c4ca299b6d4e82598a97d 0ab30f8d82fc7156b750c194d64a887e80cbfb82 Daniel Bevenius daniel.bevenius@gmail.com 2024-08-31T09:50:22+02:00 GitHub noreply@github.com 2024-08-31T10:50:22+03:00 llama : fix typo in xcda_array_view comment [no ci] (#9132) 0ab30f8d82fc7156b750c194d64a887e80cbfb82 cddae4884c853b1a7ab420458236d666e2e34423 Sutou Kouhei kou@cozmixng.org 2024-08-31T03:08:10+09:00 GitHub noreply@github.com 2024-08-30T20:08:10+02:00 llama : fix llama_split_mode enum values in main_gpu document (#9057) cddae4884c853b1a7ab420458236d666e2e34423 7ea8d80d53364f2cdcd7bbc0f031106f529b1fe3 蕭澧邦 45505768+shou692199@users.noreply.github.com 2024-08-30T20:10:01+08:00 GitHub noreply@github.com 2024-08-30T22:10:01+10:00 Correct typo run_llama2.sh > run-llama2.sh (#9149) 7ea8d80d53364f2cdcd7bbc0f031106f529b1fe3 42c76d1358021ccdbe8ba89109c143dd7ae166df tc-mb 157115220+tc-mb@users.noreply.github.com 2024-08-30T13:21:57+08:00 GitHub noreply@github.com 2024-08-30T07:21:57+02:00 llava : the function "clip" should be int (#9237) 42c76d1358021ccdbe8ba89109c143dd7ae166df 9f7d4bcf5c27d37b0c7da82eeaf9c1499510554b Faisal Zaghloul quic_fzaghlou@quicinc.com 2024-08-29T19:20:53-04:00 GitHub noreply@github.com 2024-08-30T01:20:53+02:00 Threadpool: take 2 (#8672) 9f7d4bcf5c27d37b0c7da82eeaf9c1499510554b 1d1ccce67613674c75c9c7e3fa4c1e24e428ba48 Jan Boon jan.boon@kaetemi.be 2024-08-27T18:28:06+08:00 Jan Boon kaetemi@gmail.com 2024-08-30T07:15:26+08:00 server : fix crash when error handler dumps invalid utf-8 json (#9195) 1d1ccce67613674c75c9c7e3fa4c1e24e428ba48 9fe94ccac92693d4ae1bc283ff0574e8b3f4e765 Georgi Gerganov ggerganov@gmail.com 2024-08-29T07:28:14+03:00 GitHub noreply@github.com 2024-08-28T21:28:14-07:00 flake.lock: Update (#9162) 9fe94ccac92693d4ae1bc283ff0574e8b3f4e765 66b039a5011522b6a61495eea2a9862601e169f7 slaren slarengh@gmail.com 2024-08-28T17:28:00+02:00 GitHub noreply@github.com 2024-08-28T17:28:00+02:00 docker : build images only once (#9225) 66b039a5011522b6a61495eea2a9862601e169f7 20f1789dfb4e535d64ba2f523c64929e7891f428 slaren slarengh@gmail.com 2024-08-28T13:20:36+02:00 GitHub noreply@github.com 2024-08-28T13:20:36+02:00 docker : update CUDA images (#9213) 20f1789dfb4e535d64ba2f523c64929e7891f428 231cff5f6f1c050bcb448a8ac5857533b4c05dc7 Georgi Gerganov ggerganov@gmail.com 2024-08-27T22:10:58+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-27T22:41:27+03:00 vulkan : fix build (#0) 231cff5f6f1c050bcb448a8ac5857533b4c05dc7 3246fe84d78c8ccccd4291132809236ef477e9ea Georgi Gerganov ggerganov@gmail.com 2024-08-27T22:01:45+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-27T22:41:27+03:00 sync : ggml 3246fe84d78c8ccccd4291132809236ef477e9ea 78eb487bb0038eae95506d3d832b94c979185b09 Xie Yanbo xieyanbo@gmail.com 2024-08-27T20:33:08+08:00 GitHub noreply@github.com 2024-08-27T14:33:08+02:00 Fix minicpm example directory (#9111) 78eb487bb0038eae95506d3d832b94c979185b09 a77feb5d71831c61e455541e8a655b9f0337ea8c compilade git@compilade.net 2024-08-27T06:09:23-04:00 GitHub noreply@github.com 2024-08-27T13:09:23+03:00 llama : fix qs.n_attention_wv for DeepSeek-V2 (#9156) a77feb5d71831c61e455541e8a655b9f0337ea8c 2e59d61c1b321431c597c1f12249273427d5640d Xuan Son Nguyen thichthat@gmail.com 2024-08-27T11:07:01+02:00 GitHub noreply@github.com 2024-08-27T11:07:01+02:00 server : add some missing env variables (#9116) 2e59d61c1b321431c597c1f12249273427d5640d 75e1dbbaaba5d762223370f3dab9db24a7f53465 CausalLM 148736309+CausalLM@users.noreply.github.com 2024-08-27T14:58:22+08:00 GitHub noreply@github.com 2024-08-27T09:58:22+03:00 llama : fix ChatGLM4 wrong shape (#9194) 75e1dbbaaba5d762223370f3dab9db24a7f53465 ad76569f8e78ab6ca921bda25cef25a157361719 Carsten Kragelund Jørgensen carsten@kragelund.me 2024-08-27T08:53:40+02:00 GitHub noreply@github.com 2024-08-27T09:53:40+03:00 llama : fix llama3.1 rope_freqs not respecting custom head_dim (#9141) ad76569f8e78ab6ca921bda25cef25a157361719 7d787ed96c32be18603c158ab0276992cf0dc346 arch-btw 57669023+arch-btw@users.noreply.github.com 2024-08-26T22:58:50-07:00 GitHub noreply@github.com 2024-08-27T08:58:50+03:00 common : Update stb_image.h to latest version (#9161) 7d787ed96c32be18603c158ab0276992cf0dc346 06658ad7c37f440502de2b9486ce43c47b4ec710 slaren slarengh@gmail.com 2024-08-26T19:44:43+02:00 GitHub noreply@github.com 2024-08-26T19:44:43+02:00 ggml : do not crash when quantizing q4_x_x with an imatrix (#9192) 06658ad7c37f440502de2b9486ce43c47b4ec710 fc18425b6a8ad03847383ce2b69d52edfd49b0ff Georgi Gerganov ggerganov@gmail.com 2024-08-26T18:31:02+03:00 GitHub noreply@github.com 2024-08-26T18:31:02+03:00 metal : separate scale and mask from QKT in FA kernel (#9189) fc18425b6a8ad03847383ce2b69d52edfd49b0ff 879275ac984235373dd44ed780d5e3a3883cb558 Georgi Gerganov ggerganov@gmail.com 2024-08-26T17:55:36+03:00 GitHub noreply@github.com 2024-08-26T17:55:36+03:00 ggml : add SSM Metal kernels (#8546) 879275ac984235373dd44ed780d5e3a3883cb558 7a3df798fc43e1b366146b1ad99137a9e95c87dd Georgi Gerganov ggerganov@gmail.com 2024-08-26T16:30:25+03:00 GitHub noreply@github.com 2024-08-26T16:30:25+03:00 tests : fix compile warnings for unreachable code (#9185) 7a3df798fc43e1b366146b1ad99137a9e95c87dd e5edb210cd361689da41f032f1b36c45ff1bf9be Georgi Gerganov ggerganov@gmail.com 2024-08-26T12:19:39+03:00 GitHub noreply@github.com 2024-08-26T12:19:39+03:00 ci : add VULKAN support to ggml-ci (#9055) e5edb210cd361689da41f032f1b36c45ff1bf9be 0c41e03ceba1aafaf469476a56347419d5785376 Georgi Gerganov ggerganov@gmail.com 2024-08-26T12:16:57+03:00 GitHub noreply@github.com 2024-08-26T12:16:57+03:00 server : update deps (#9183) 0c41e03ceba1aafaf469476a56347419d5785376 f12ceaca0c7b59def30b7a832a6904df7ed3f4f7 slaren slarengh@gmail.com 2024-08-26T11:08:59+02:00 GitHub noreply@github.com 2024-08-26T11:08:59+02:00 metal : gemma2 flash attention support (#9159) f12ceaca0c7b59def30b7a832a6904df7ed3f4f7 436787f170329b3f549e6c2c46593d2af8482e7c slaren slarengh@gmail.com 2024-08-26T11:03:30+02:00 GitHub noreply@github.com 2024-08-26T11:03:30+02:00 ggml-ci : try to improve build time (#9160) 436787f170329b3f549e6c2c46593d2af8482e7c 93bc3839f980ff14be86efe408b4cd7e89b26835 Justine Tunney jtunney@mozilla.com 2024-08-25T23:09:53-07:00 GitHub noreply@github.com 2024-08-26T09:09:53+03:00 llama : fix time complexity of string replacement (#9163) 93bc3839f980ff14be86efe408b4cd7e89b26835 f91fc5639be1e272194303ea26e1d4c226ec981b Herman Semenov GermanAizek@yandex.ru 2024-08-25T22:54:37Z GitHub noreply@github.com 2024-08-26T00:54:37+02:00 common: fixed not working find argument --n-gpu-layers-draft (#9175) f91fc5639be1e272194303ea26e1d4c226ec981b e11bd856d538e44d24d8cad4b0381fba0984d162 Johannes Gäßler johannesg@5d6.de 2024-08-25T22:11:48+02:00 GitHub noreply@github.com 2024-08-25T22:11:48+02:00 CUDA: fix Gemma 2 numerical issues for FA (#9166) e11bd856d538e44d24d8cad4b0381fba0984d162 8f824ffe8ee1feadd14428f1dda1283fa3b933be Johannes Gäßler johannesg@5d6.de 2024-08-24T21:34:59+02:00 GitHub noreply@github.com 2024-08-24T21:34:59+02:00 CPU/CUDA: Gemma 2 FlashAttention support (#8542) 8f824ffe8ee1feadd14428f1dda1283fa3b933be 3ba780e2a8f0ffe13f571b27f0bbf2ca5a199efc João Dinis Ferreira hello@joaof.eu 2024-08-24T07:22:45+01:00 GitHub noreply@github.com 2024-08-24T09:22:45+03:00 quantize : fix typo in usage help of `quantize.cpp` (#9145) 3ba780e2a8f0ffe13f571b27f0bbf2ca5a199efc a07c32ea54850c989f0ef6989da5b955b77b7172 Xuan Son Nguyen thichthat@gmail.com 2024-08-23T12:58:53+02:00 GitHub noreply@github.com 2024-08-23T12:58:53+02:00 lora : fix llama conversion script with ROPE_FREQS (#9117) a07c32ea54850c989f0ef6989da5b955b77b7172 11b84eb4578864827afcf956db5b571003f18180 piDack 104877312+piDack@users.noreply.github.com 2024-08-23T15:27:17+08:00 GitHub noreply@github.com 2024-08-23T10:27:17+03:00 llama : use F32 precision in GLM4 attention and no FA (#9130) 11b84eb4578864827afcf956db5b571003f18180 1731d4238f9e4f925a750810e7f5480827c66dcf Akarshan Biswas akarshan.biswas@gmail.com 2024-08-22T19:39:47+05:30 GitHub noreply@github.com 2024-08-22T22:09:47+08:00 [SYCL] Add a space to supress a cmake warning (#9133) 1731d4238f9e4f925a750810e7f5480827c66dcf a1631e53f6763e17da522ba219b030d8932900bd luoyu-intel yu.luo@intel.com 2024-08-22T12:50:10+08:00 GitHub noreply@github.com 2024-08-22T12:50:10+08:00 [SYCL] Add oneDNN primitive support (#9091) a1631e53f6763e17da522ba219b030d8932900bd fc54ef0d1c138133a01933296d50a36a1ab64735 compilade git@compilade.net 2024-08-21T17:58:11-04:00 GitHub noreply@github.com 2024-08-21T17:58:11-04:00 llama : simplify Mamba with advanced batch splits (#8526) fc54ef0d1c138133a01933296d50a36a1ab64735 b40eb84895bf723c7b327a1e3bf6e0e2c41877f8 Xuan Son Nguyen thichthat@gmail.com 2024-08-21T11:04:34+02:00 GitHub noreply@github.com 2024-08-21T11:04:34+02:00 server : support reading arguments from environment variables (#9105) b40eb84895bf723c7b327a1e3bf6e0e2c41877f8 f63f603c879c2232eaeded8c0aeba4244471d720 Younes Belkada 49240599+younesbelkada@users.noreply.github.com 2024-08-21T12:06:36+04:00 GitHub noreply@github.com 2024-08-21T11:06:36+03:00 llama : support for `falcon-mamba` architecture (#9074) f63f603c879c2232eaeded8c0aeba4244471d720 8455340b874aa90d5f70104c99ed2778d9e31c36 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-08-21T09:45:49+02:00 GitHub noreply@github.com 2024-08-21T09:45:49+02:00 llava : zero-initialize clip_ctx structure fields with aggregate initialization 908) 8455340b874aa90d5f70104c99ed2778d9e31c36 2f3c1466ff46a2413b0e363a5005c46538186ee6 Daniel Bevenius daniel.bevenius@gmail.com 2024-08-21T09:32:58+02:00 GitHub noreply@github.com 2024-08-21T10:32:58+03:00 llama : std::move llm_bigram_bpe from work_queue (#9062) 2f3c1466ff46a2413b0e363a5005c46538186ee6 50addec9a532a6518146ab837a85504850627316 Changyeon Kim cyzero.kim@samsung.com 2024-08-21T04:00:00+09:00 GitHub noreply@github.com 2024-08-20T21:00:00+02:00 llava: Add ACC OP for GPU acceleration to the Vulkan backend in the LLAVA CLIP model. (#8984) 50addec9a532a6518146ab837a85504850627316 4f8d19ff17faa601f456ee1db7d8b8a15fa3f90b Meng, Hengyu hengyu.meng@intel.com 2024-08-20T23:50:17+08:00 GitHub noreply@github.com 2024-08-20T23:50:17+08:00 [SYCL] fallback mmvq (#9088) 4f8d19ff17faa601f456ee1db7d8b8a15fa3f90b 90db8146d56d83a605f2c475eca39bcda29cf16d zhentaoyu zhentao.yu@intel.com 2024-08-20T23:06:51+08:00 GitHub noreply@github.com 2024-08-20T23:06:51+08:00 [SYCL] Fix SYCL `im2col` and `convert` Overflow with Large Dims (#9052) 90db8146d56d83a605f2c475eca39bcda29cf16d cfac111e2b3953cdb6b0126e67a2487687646971 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-08-20T11:09:55+02:00 GitHub noreply@github.com 2024-08-20T12:09:55+03:00 tests : add missing comma in grammar integration tests (#9099) cfac111e2b3953cdb6b0126e67a2487687646971 1b6ff90ff8301d9fe2027be2bb9fea26177d775e wangshuai09 391746016@qq.com 2024-08-19T16:46:38+08:00 GitHub noreply@github.com 2024-08-19T16:46:38+08:00 cann: add doc for cann backend (#8867) 1b6ff90ff8301d9fe2027be2bb9fea26177d775e 18eaf29f4c5f7c8e89412f3a9a0392a9e4e01d75 Radoslav Gerganov rgerganov@gmail.com 2024-08-19T10:11:45+03:00 GitHub noreply@github.com 2024-08-19T10:11:45+03:00 rpc : print error message when failed to connect endpoint (#9042) 18eaf29f4c5f7c8e89412f3a9a0392a9e4e01d75 554b049068de24201d19dde2fa83e35389d4585d Radoslav Gerganov rgerganov@gmail.com 2024-08-19T10:10:21+03:00 GitHub noreply@github.com 2024-08-19T10:10:21+03:00 rpc : prevent crashes on invalid input (#9040) 554b049068de24201d19dde2fa83e35389d4585d 2339a0be1c8e31fcf4531427183b94f2ef019e56 Georgi Gerganov ggerganov@gmail.com 2024-08-18T17:43:32+03:00 GitHub noreply@github.com 2024-08-18T07:43:32-07:00 flake.lock: Update (#9068) 2339a0be1c8e31fcf4531427183b94f2ef019e56 2fb9267887d24a431892ce4dccc75c7095b0d54d ltoniazzi 61414566+ltoniazzi@users.noreply.github.com 2024-08-18T10:58:04+01:00 GitHub noreply@github.com 2024-08-18T11:58:04+02:00 tests : add integration test for lora adapters (#8957) 2fb9267887d24a431892ce4dccc75c7095b0d54d 8b3befc0e2ed8fb18b903735831496b8b0c80949 Yoshi Suhara ysuhara@nvidia.com 2024-08-17T06:34:21-07:00 GitHub noreply@github.com 2024-08-17T15:34:21+02:00 Fix incorrect use of ctx_split for bias tensors (#9063) 8b3befc0e2ed8fb18b903735831496b8b0c80949 d565bb2fd5a2a58b9924a7a34e77a87c78c52137 Xuan Son Nguyen thichthat@gmail.com 2024-08-16T17:19:05+02:00 GitHub noreply@github.com 2024-08-16T17:19:05+02:00 server : refactor middleware and /health endpoint (#9056) d565bb2fd5a2a58b9924a7a34e77a87c78c52137 ee2984bdaf10c14d440ad873a049bcc09b786d9b tc-mb 157115220+tc-mb@users.noreply.github.com 2024-08-16T21:34:41+08:00 GitHub noreply@github.com 2024-08-16T16:34:41+03:00 llava : support MiniCPM-V-2.6 (#8967) ee2984bdaf10c14d440ad873a049bcc09b786d9b c8ddce85606d9fb6e30745b6e4fe103eecadc73f Farbod Bijary 110523279+farbodbj@users.noreply.github.com 2024-08-16T14:06:30+03:30 GitHub noreply@github.com 2024-08-16T13:36:30+03:00 py : fix wrong input type for raw_dtype in ggml to gguf scripts (#8928) c8ddce85606d9fb6e30745b6e4fe103eecadc73f 23fd4535445e3e859ded2f02d6142b6e93b43890 Aisuko urakiny@gmail.com 2024-08-16T19:08:59+10:00 GitHub noreply@github.com 2024-08-16T11:08:59+02:00 Fix inference example lacks required parameters (#9035) 23fd4535445e3e859ded2f02d6142b6e93b43890 c679e0cb5c378bfb63643c44a453345ba8b90126 compilade git@compilade.net 2024-08-16T02:36:11-04:00 GitHub noreply@github.com 2024-08-16T09:36:11+03:00 gguf-py : bump version from 0.9.1 to 0.10.0 (#9051) c679e0cb5c378bfb63643c44a453345ba8b90126 fb487bb5671b37267f35ff43fe8849ddccd925cd Minsoo Cheong icycle0409@snu.ac.kr 2024-08-16T15:35:18+09:00 GitHub noreply@github.com 2024-08-16T09:35:18+03:00 llama : add EXAONE model support (#9025) fb487bb5671b37267f35ff43fe8849ddccd925cd 2a24c8caa6d10a7263ca317fa7cb64f0edc72aae Liu Jia jia3.liu@intel.com 2024-08-16T14:23:12+08:00 GitHub noreply@github.com 2024-08-16T09:23:12+03:00 common : add support for cpu_get_num_physical_cores() on Windows (#8771) 2a24c8caa6d10a7263ca317fa7cb64f0edc72aae e3f6fd56b1ab3c426596217d786910d641ae6ce0 Yoshi Suhara y.suhara@gmail.com 2024-08-15T19:23:33-07:00 GitHub noreply@github.com 2024-08-16T04:23:33+02:00 Add Nemotron/Minitron GGUF Conversion & Inference Support (#8922) e3f6fd56b1ab3c426596217d786910d641ae6ce0 4b9afbbe9037f8a2d659097c0c7d9fce32c6494c Nico Bosshard nico@bosshome.ch 2024-08-16T04:22:55+02:00 GitHub noreply@github.com 2024-08-16T04:22:55+02:00 ggml : dynamic ggml_sched_max_splits based on graph_size (#9047) 4b9afbbe9037f8a2d659097c0c7d9fce32c6494c 37501d9c79ed5897db4b73ea7502211d25b3f763 gtygo gtydoit@gmail.com 2024-08-15T15:40:12+08:00 GitHub noreply@github.com 2024-08-15T10:40:12+03:00 retrieval : fix memory leak in retrieval query handling (#8955) 37501d9c79ed5897db4b73ea7502211d25b3f763 4af8420afba39de4968adf2695ce12fd42422a13 Riceball LEE snowyu.lee@gmail.com 2024-08-15T15:28:05+08:00 GitHub noreply@github.com 2024-08-15T10:28:05+03:00 server : fix duplicated n_predict key in the generation_settings (#8994) 4af8420afba39de4968adf2695ce12fd42422a13 6bda7ce6c3a9284fcbb70c1ace4107db8eb63e5c Zhenwei Jin 109658203+kylo5aby@users.noreply.github.com 2024-08-15T15:23:23+08:00 GitHub noreply@github.com 2024-08-15T10:23:23+03:00 common : remove duplicate function llama_should_add_bos_token (#8778) 6bda7ce6c3a9284fcbb70c1ace4107db8eb63e5c d5492f0525fa533817a67e93a4bde9d71d81cf58 Esko Toivonen eskot98@gmail.com 2024-08-15T10:17:12+03:00 GitHub noreply@github.com 2024-08-15T10:17:12+03:00 llama : add pre-tokenizer regexes for BLOOM and gpt3-finnish (#8850) d5492f0525fa533817a67e93a4bde9d71d81cf58 234b30676a97ce227b604c38beb9dcaca406dea9 Georgi Gerganov ggerganov@gmail.com 2024-08-15T10:11:11+03:00 GitHub noreply@github.com 2024-08-15T10:11:11+03:00 ci : disable bench workflow (#9010) 234b30676a97ce227b604c38beb9dcaca406dea9 5fd89a70ead34d1a17015ddecad05aaa2490ca46 Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2024-08-15T08:21:57+02:00 GitHub noreply@github.com 2024-08-15T09:21:57+03:00 server : init stop and error fields of the result struct (#9026) 5fd89a70ead34d1a17015ddecad05aaa2490ca46 98a532d474c73d3494a5353024cb6a4fbbabbb35 0cc4m picard12@live.de 2024-08-14T18:32:53+02:00 GitHub noreply@github.com 2024-08-14T18:32:53+02:00 Vulkan Optimizations and Fixes (#8959) 98a532d474c73d3494a5353024cb6a4fbbabbb35 43bdd3ce188cd247bda7c1bd1ad01fa64e566690 compilade git@compilade.net 2024-08-14T02:51:02-04:00 GitHub noreply@github.com 2024-08-14T09:51:02+03:00 server : fix segfault on long system prompt (#8987) 43bdd3ce188cd247bda7c1bd1ad01fa64e566690 06943a69f678fb32829ff06d9c18367b17d4b361 Georgi Gerganov ggerganov@gmail.com 2024-08-14T09:14:49+03:00 GitHub noreply@github.com 2024-08-14T09:14:49+03:00 cmake : remove unused option GGML_CURL (#9011) 06943a69f678fb32829ff06d9c18367b17d4b361 828d6ff7d796f48b2c345f6be2805a3c531a089c Daniel Bevenius daniel.bevenius@gmail.com 2024-08-13T21:13:15+02:00 GitHub noreply@github.com 2024-08-13T21:13:15+02:00 ggml : move rope type enum to ggml.h (#8949) 828d6ff7d796f48b2c345f6be2805a3c531a089c fc4ca27b25464a11b3b86c9dbb5b6ed6065965c2 Xuan Son Nguyen thichthat@gmail.com 2024-08-13T11:41:14+02:00 GitHub noreply@github.com 2024-08-13T11:41:14+02:00 export-lora : throw error if lora is quantized (#9002) fc4ca27b25464a11b3b86c9dbb5b6ed6065965c2 1f67436c5ee6f4c99e71a8518bdfc214c27ce934 Diogo Teles Sant'Anna diogoteles@google.com 2024-08-12T13:28:23-03:00 GitHub noreply@github.com 2024-08-12T19:28:23+03:00 ci : fix github workflow vulnerable to script injection (#9008) 1f67436c5ee6f4c99e71a8518bdfc214c27ce934 0fd93cdef5e583aa980b3c0d693c0d207f0787a7 Radoslav Gerganov rgerganov@gmail.com 2024-08-12T19:17:03+03:00 GitHub noreply@github.com 2024-08-12T19:17:03+03:00 ci : enable RPC in all of the released builds (#9006) 0fd93cdef5e583aa980b3c0d693c0d207f0787a7 84eb2f4fad28ceadd415a4e775320c983f4d9a7d Nico Bosshard nico@bosshome.ch 2024-08-12T17:13:59+02:00 GitHub noreply@github.com 2024-08-12T17:13:59+02:00 llama : model-based max number of graph nodes calculation (#8970) 84eb2f4fad28ceadd415a4e775320c983f4d9a7d 1262e7ed13ac197c944f15e1ddb083cb4f36cf65 Frank Mai thxcode0824@gmail.com 2024-08-12T20:45:50+08:00 GitHub noreply@github.com 2024-08-12T14:45:50+02:00 docs: introduce gpustack and gguf-parser (#8873) 1262e7ed13ac197c944f15e1ddb083cb4f36cf65 df5478fbea7e652cfad4ee7974ac3b624fd6c7f6 DavidKorczynski david@adalogics.com 2024-08-12T13:36:41+01:00 GitHub noreply@github.com 2024-08-12T15:36:41+03:00 grammar-parser : fix possible null-deref (#9004) df5478fbea7e652cfad4ee7974ac3b624fd6c7f6 2589292cde038ba876c041bcd7b3f0c81f3f11fe DavidKorczynski david@adalogics.com 2024-08-12T13:21:41+01:00 GitHub noreply@github.com 2024-08-12T14:21:41+02:00 ggml: fix div-by-zero (#9003) 2589292cde038ba876c041bcd7b3f0c81f3f11fe d3ae0ee8d75033921a076131d4d0fa1c6ec579a7 Liu Jia jia3.liu@intel.com 2024-08-12T17:46:03+08:00 GitHub noreply@github.com 2024-08-12T11:46:03+02:00 Fix a spelling mistake (#9001) d3ae0ee8d75033921a076131d4d0fa1c6ec579a7 5ef07e25ac39e62297a67208c5bcced50835a2dd Georgi Gerganov ggerganov@gmail.com 2024-08-12T11:02:01+03:00 GitHub noreply@github.com 2024-08-12T11:02:01+03:00 py : fix requirements check '==' -> '~=' (#8982) 5ef07e25ac39e62297a67208c5bcced50835a2dd 4134999e01f31256b15342b41c4de9e2477c4a6c Georgi Gerganov ggerganov@gmail.com 2024-08-12T10:21:50+03:00 GitHub noreply@github.com 2024-08-12T10:21:50+03:00 server : handle models with missing EOS token (#8997) 4134999e01f31256b15342b41c4de9e2477c4a6c 8cd1bcfd3fc9f2b5cbafd7fb7581b3278acec25f compilade git@compilade.net 2024-08-11T14:45:41-04:00 GitHub noreply@github.com 2024-08-11T14:45:41-04:00 gguf-py : Numpy dequantization for most types (#8939) 8cd1bcfd3fc9f2b5cbafd7fb7581b3278acec25f a21c6fd45032a20180e026773582d21294c85619 Georgi Gerganov ggerganov@gmail.com 2024-08-11T16:58:58+03:00 GitHub noreply@github.com 2024-08-11T06:58:58-07:00 flake.lock: Update (#8979) a21c6fd45032a20180e026773582d21294c85619 33309f661a93c9c0ab65a79e5e7e30fa6162992e Neo Zhang zhang.jianyu@outlook.com 2024-08-11T16:37:43+08:00 GitHub noreply@github.com 2024-08-11T14:07:43+05:30 update guide (#8909) 33309f661a93c9c0ab65a79e5e7e30fa6162992e 7c5bfd57f83fd3630934cfa70892aa4022d3faf7 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-08-11T10:35:26+02:00 GitHub noreply@github.com 2024-08-11T10:35:26+02:00 llama : check all graph nodes when searching for result_embd_pooled (#8956) 7c5bfd57f83fd3630934cfa70892aa4022d3faf7 6e02327e8b7837358e0406bf90a4632e18e27846 Markus Tavenrath mtavenrath@users.noreply.github.com 2024-08-11T10:09:09+02:00 GitHub noreply@github.com 2024-08-11T10:09:09+02:00 Optimize Vulkan backend for better CPU performance and less GPU synchronization overhead. (#8943) 6e02327e8b7837358e0406bf90a4632e18e27846 7eb23840ed0f388e10c4bbc4d65802fdfb977b40 slaren slarengh@gmail.com 2024-08-10T15:42:10+02:00 GitHub noreply@github.com 2024-08-10T15:42:10+02:00 metal : fix uninitialized abort_callback (#8968) 7eb23840ed0f388e10c4bbc4d65802fdfb977b40 7c3f55c10051c634546247387c5c359c9d499360 Xuan Son Nguyen thichthat@gmail.com 2024-08-10T13:04:40+02:00 GitHub noreply@github.com 2024-08-10T13:04:40+02:00 llama : default n_swa for phi-3 (#8931) 7c3f55c10051c634546247387c5c359c9d499360 911b437f228e75aa3d235acec21bfddd23ecce2f fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-08-10T11:43:26+02:00 GitHub noreply@github.com 2024-08-10T11:43:26+02:00 Add support for encoder-only T5 models (#8900) 911b437f228e75aa3d235acec21bfddd23ecce2f b72942fac998672a79a1ae3c03b340f7e629980b Matteo Mortari matteo.mortari@gmail.com 2024-08-10T07:58:49+02:00 GitHub noreply@github.com 2024-08-10T08:58:49+03:00 gguf-py : fix double call to add_architecture() (#8952) b72942fac998672a79a1ae3c03b340f7e629980b 6afd1a99dc9792096d4567ab9fa1ad530c81c6cd Georgi Gerganov ggerganov@gmail.com 2024-08-09T23:03:21+03:00 GitHub noreply@github.com 2024-08-09T23:03:21+03:00 Merge commit from fork 6afd1a99dc9792096d4567ab9fa1ad530c81c6cd 272e3bd95e620d285b2fb9faaa7b6b1b8edbbf3a fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-08-09T18:53:09+02:00 GitHub noreply@github.com 2024-08-09T18:53:09+02:00 llama : add support for lora adapters in T5 model (#8938) 272e3bd95e620d285b2fb9faaa7b6b1b8edbbf3a 45a55b91aa87958a75d691be64b070266d4fbb94 Georgi Gerganov ggerganov@gmail.com 2024-08-09T18:24:30+03:00 GitHub noreply@github.com 2024-08-09T18:24:30+03:00 make : fix llava obj file race (#8946) 45a55b91aa87958a75d691be64b070266d4fbb94 3071c0a5f218f107dabd13b73f6090af683ef5ec Georgi Gerganov ggerganov@gmail.com 2024-08-09T18:23:52+03:00 GitHub noreply@github.com 2024-08-09T18:23:52+03:00 llama : better replace_all (cont) (#8926) 3071c0a5f218f107dabd13b73f6090af683ef5ec 4305b57c80eff4f0df5f6acb60b292f03b8f0dd0 tc-mb 157115220+tc-mb@users.noreply.github.com 2024-08-09T18:33:53+08:00 GitHub noreply@github.com 2024-08-09T13:33:53+03:00 llava : support MiniCPM-V-2.5 (#7599) 4305b57c80eff4f0df5f6acb60b292f03b8f0dd0 70c0ea35609a2ab87a358e25f4ffad1aad408992 Georgi Gerganov ggerganov@gmail.com 2024-08-09T10:03:48+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-09T10:03:48+03:00 sync : ggml 70c0ea35609a2ab87a358e25f4ffad1aad408992 5b2c04f4925e152c362b824f4b41eb2a081fa623 Matt Stephenson mstephenson6@users.noreply.github.com 2024-07-16T03:21:09-04:00 Georgi Gerganov ggerganov@gmail.com 2024-08-09T10:03:44+03:00 whisper : use vulkan as gpu backend when available (whisper/2302) 5b2c04f4925e152c362b824f4b41eb2a081fa623 6f6496bb0999d1bce5daff0cfc55ceb0dd13c888 Daniel Bevenius daniel.bevenius@gmail.com 2024-08-09T08:33:30+02:00 GitHub noreply@github.com 2024-08-09T09:33:30+03:00 embedding : add --pooling option to README.md [no ci] (#8934) 6f6496bb0999d1bce5daff0cfc55ceb0dd13c888 daef3ab233fc02e4c53afd9b07366379876f00d1 Daniel Bevenius daniel.bevenius@gmail.com 2024-08-09T08:32:23+02:00 GitHub noreply@github.com 2024-08-09T09:32:23+03:00 llama : fix typo in llama_tensor_get_type comment [no ci] (#8937) daef3ab233fc02e4c53afd9b07366379876f00d1 345a686d8271a24db20d31789c0d4b9ed51dcb0c Mathieu Geli mathieu.geli@gmail.com 2024-08-09T08:32:02+02:00 GitHub noreply@github.com 2024-08-09T09:32:02+03:00 server : add one level list nesting for embeddings (#8936) 345a686d8271a24db20d31789c0d4b9ed51dcb0c 3a14e00366399040a139c67dd5951177a8cb5695 compilade git@compilade.net 2024-08-08T23:54:00-04:00 GitHub noreply@github.com 2024-08-08T23:54:00-04:00 llama : reduce useless copies when saving session (#8916) 3a14e00366399040a139c67dd5951177a8cb5695 afd27f01fe832ece3d07ef03b7d34a9e80c4a895 compilade git@compilade.net 2024-08-08T13:33:09-04:00 GitHub noreply@github.com 2024-08-08T13:33:09-04:00 gguf-py : simplify support for quant types (#8838) afd27f01fe832ece3d07ef03b7d34a9e80c4a895 366d486c163ea883442313cff1a3b154ab93e168 Georgi Gerganov ggerganov@gmail.com 2024-08-08T14:56:52+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-08T14:56:52+03:00 scripts : sync cann files (#0) 366d486c163ea883442313cff1a3b154ab93e168 e44a561ab090b11d3a6fe539b768404663e4c3d2 Georgi Gerganov ggerganov@gmail.com 2024-08-08T14:40:12+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-08T14:40:12+03:00 scripts : fix sync filenames (#0) e44a561ab090b11d3a6fe539b768404663e4c3d2 f93d49ab1e2d1d698af8f17ead6d635405d4b289 Georgi Gerganov ggerganov@gmail.com 2024-08-08T13:19:47+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-08T13:19:47+03:00 sync : ggml f93d49ab1e2d1d698af8f17ead6d635405d4b289 5b33ea1ee72fadfa4f96d86dc614631739758cce Borislav Stanimirov b.stanimirov@abv.bg 2024-08-07T10:00:56+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-08T13:19:31+03:00 ggml : ignore more msvc warnings (ggml/906) 5b33ea1ee72fadfa4f96d86dc614631739758cce 85fca8deb6273b956bc9184bc9d70a07e1d50d07 Georgi Gerganov ggerganov@gmail.com 2024-08-07T09:57:00+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-08T13:19:31+03:00 metal : fix struct name (ggml/912) 85fca8deb6273b956bc9184bc9d70a07e1d50d07 ebd541a5705b6f7a4ce67824d1c2d4fc790f1770 Conrad Kramer conrad@conradkramer.com 2024-08-07T02:55:49-04:00 Georgi Gerganov ggerganov@gmail.com 2024-08-08T13:19:30+03:00 metal : add abort callback (ggml/905) ebd541a5705b6f7a4ce67824d1c2d4fc790f1770 15fa07a5c564d3ed7e7eb64b73272cedb27e73ec Pablo Duboue pablo.duboue@gmail.com 2024-08-08T04:44:51-04:00 GitHub noreply@github.com 2024-08-08T11:44:51+03:00 make : clean llamafile objects (#8923) 15fa07a5c564d3ed7e7eb64b73272cedb27e73ec be55695eff44784a141a863f273661a6bce63dfc slaren slarengh@gmail.com 2024-08-07T18:24:05+02:00 GitHub noreply@github.com 2024-08-07T18:24:05+02:00 make : use C compiler to build metal embed object (#8899) be55695eff44784a141a863f273661a6bce63dfc 0478174d5959b66096ae6609fcb0df14cab66b51 slaren slarengh@gmail.com 2024-08-07T13:29:02+02:00 GitHub noreply@github.com 2024-08-07T13:29:02+02:00 ggml-backend : fix async copy from CPU (#8897) 0478174d5959b66096ae6609fcb0df14cab66b51 a8dbc6f753f296108121d50f78f67463403dd6fc Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-08-07T11:25:36+01:00 GitHub noreply@github.com 2024-08-07T11:25:36+01:00 [SYCL] Updated SYCL device filtering (#8901) a8dbc6f753f296108121d50f78f67463403dd6fc 506122d854c5d05b4a3d45a294f14bd4c02d9868 Johannes Gäßler johannesg@5d6.de 2024-08-07T09:07:52+02:00 GitHub noreply@github.com 2024-08-07T09:07:52+02:00 CUDA/HIP: fix tests/test-backend-ops (#8896) 506122d854c5d05b4a3d45a294f14bd4c02d9868 725e3d94379d5b619c027347308bccf2e0ead89f Zhenwei Jin 109658203+kylo5aby@users.noreply.github.com 2024-08-07T09:01:06+08:00 GitHub noreply@github.com 2024-08-07T03:01:06+02:00 llama-bench : add support for getting cpu info on Windows (#8824) 725e3d94379d5b619c027347308bccf2e0ead89f 31958546c3e4695a8a24bb7ba3b79a8f76d05afe Daniel Bevenius daniel.bevenius@gmail.com 2024-08-07T01:43:00+02:00 GitHub noreply@github.com 2024-08-07T01:43:00+02:00 quantize : update usage comment in quantize.cpp (#8889) 31958546c3e4695a8a24bb7ba3b79a8f76d05afe 1e6f6554aa11fa10160a5fda689e736c3c34169f Nexes the Old 124105151+Nexesenex@users.noreply.github.com 2024-08-07T01:41:54+02:00 GitHub noreply@github.com 2024-08-07T01:41:54+02:00 typo correction (#8891) 1e6f6554aa11fa10160a5fda689e736c3c34169f 641f5dd2a6422941faa9f32ab5cb50fc1b24c1f5 Xuan Son Nguyen thichthat@gmail.com 2024-08-06T17:33:39+02:00 GitHub noreply@github.com 2024-08-06T17:33:39+02:00 server : add lora hotswap endpoint (WIP) (#8857) 641f5dd2a6422941faa9f32ab5cb50fc1b24c1f5 5f4dcb1e60bbfe936b45778dad177a5b9a09b066 Johannes Gäßler johannesg@5d6.de 2024-08-06T17:13:55+02:00 GitHub noreply@github.com 2024-08-06T17:13:55+02:00 CUDA: fix padding logic for FP16/FP32 (#8884) 5f4dcb1e60bbfe936b45778dad177a5b9a09b066 db20f50cf4710c46e3a996919a26858cae8c80ed Daniel Bevenius daniel.bevenius@gmail.com 2024-08-06T16:44:35+02:00 GitHub noreply@github.com 2024-08-06T16:44:35+02:00 simple : update name of executable to llama-simple (#8885) db20f50cf4710c46e3a996919a26858cae8c80ed efda90c93a62274ec0b0bfa80c4eee4bdb6966d0 Jaeden Amero jaeden@patater.com 2024-08-06T17:21:47+04:00 GitHub noreply@github.com 2024-08-06T15:21:47+02:00 cmake : Link vulkan-shaders-gen with pthreads (#8835) efda90c93a62274ec0b0bfa80c4eee4bdb6966d0 0bf16de07b0692e7df26b9a633e232bbd66e0360 MaggotHATE clay1326@gmail.com 2024-08-06T16:32:03+05:00 GitHub noreply@github.com 2024-08-06T13:32:03+02:00 [Vulkan] Fix compilation of `vulkan-shaders-gen` on w64devkit after `e31a4f6` (#8880) 0bf16de07b0692e7df26b9a633e232bbd66e0360 2d5dd7bb3fa382806cd3e0bfc7a1d92349bc0ccf Georgi Gerganov ggerganov@gmail.com 2024-08-06T11:48:01+03:00 GitHub noreply@github.com 2024-08-06T11:48:01+03:00 contributing : add note about write access 2d5dd7bb3fa382806cd3e0bfc7a1d92349bc0ccf cdd1889de62a7140c8c016405ec917464bde8bd3 Molly Sophia mollysophia379@gmail.com 2024-08-06T15:26:46+08:00 GitHub noreply@github.com 2024-08-06T10:26:46+03:00 ggml : add epsilon as a parameter for group_norm (#8818) cdd1889de62a7140c8c016405ec917464bde8bd3 c21a896405de4cdf4207eb8130555ceaac0ab110 Douglas Hanley thesecretaryofwar@gmail.com 2024-08-06T02:20:54-05:00 GitHub noreply@github.com 2024-08-06T10:20:54+03:00 convert : add support for XLMRoberta embedding models (#8658) c21a896405de4cdf4207eb8130555ceaac0ab110 d4ff847153e9cf7220d1b39aa21172069e6e8cea Mengqing Cao cmq0113@163.com 2024-08-06T12:42:42+08:00 GitHub noreply@github.com 2024-08-06T12:42:42+08:00 [CANN]: Fix ggml_backend_cann_buffer_get_tensor (#8871) d4ff847153e9cf7220d1b39aa21172069e6e8cea 0a4ce786814b123096d18aadca89cd352b9e590b Neo Zhang zhang.jianyu@outlook.com 2024-08-06T09:09:12+08:00 GitHub noreply@github.com 2024-08-06T09:09:12+08:00 [SYCL] correct cmd name (#8877) 0a4ce786814b123096d18aadca89cd352b9e590b bc0f887e159c0d78c28121e2c8b5c58094170875 Liu Jia 109258120+Septa2112@users.noreply.github.com 2024-08-06T00:14:10+08:00 GitHub noreply@github.com 2024-08-05T18:14:10+02:00 common : Changed tuple to struct (TODO fix) (#8823) bc0f887e159c0d78c28121e2c8b5c58094170875 b42978e7e4d56eaaa93588414e804d9fbbc3cae2 wangshuai09 391746016@qq.com 2024-08-05T21:10:37+08:00 GitHub noreply@github.com 2024-08-05T21:10:37+08:00 cann: fix buffer_num and runtime speed slowly error (#8865) b42978e7e4d56eaaa93588414e804d9fbbc3cae2 b9dfc25ca385a83bde9e9456c4d4fae15377bc7b Eric Curtin ericcurtin17@gmail.com 2024-08-05T13:45:01+01:00 GitHub noreply@github.com 2024-08-05T15:45:01+03:00 readme : add ramalama to the availables UI (#8811) b9dfc25ca385a83bde9e9456c4d4fae15377bc7b 1ef14b30075da594cb24f0ab858a14bf1d8d1797 Justine Tunney jtunney@mozilla.com 2024-08-05T05:43:40-07:00 GitHub noreply@github.com 2024-08-05T15:43:40+03:00 ggml : fix overflows in elu function (#8866) 1ef14b30075da594cb24f0ab858a14bf1d8d1797 d3f0c7166adfa952237e0f437a5344362d8256d4 Brian mofosyne@gmail.com 2024-08-05T21:15:28+10:00 GitHub noreply@github.com 2024-08-05T21:15:28+10:00 py: Add more authorship metadata from model card (#8810) d3f0c7166adfa952237e0f437a5344362d8256d4 e31a4f679779220312c165b0f5994c680a610e38 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-08-05T09:38:01+02:00 GitHub noreply@github.com 2024-08-05T09:38:01+02:00 Stop the generation when <|eom_id|> token is encountered - needed for Llama 3.1 tool call support (#8858) e31a4f679779220312c165b0f5994c680a610e38 400ae6f65f0b55babd48d1e3ec7fd663a97fc8d0 stduhpf stephduh@live.fr 2024-08-05T08:18:27+02:00 GitHub noreply@github.com 2024-08-05T08:18:27+02:00 cmake: fix paths for vulkan shaders compilation on Windows (#8573) 400ae6f65f0b55babd48d1e3ec7fd663a97fc8d0 f1ea5146d741a0c9be6d8fbfab9323fea6c4a3f0 BarfingLemurs 128182951+BarfingLemurs@users.noreply.github.com 2024-08-05T01:54:10-04:00 GitHub noreply@github.com 2024-08-05T08:54:10+03:00 readme : update model list (#8851) f1ea5146d741a0c9be6d8fbfab9323fea6c4a3f0 064cdc265fb63590c7c8f04a609d36ef200d55a7 Georgi Gerganov ggerganov@gmail.com 2024-08-05T08:53:39+03:00 GitHub noreply@github.com 2024-08-05T08:53:39+03:00 llama : better replace_all (#8852) 064cdc265fb63590c7c8f04a609d36ef200d55a7 5587e57a76630651752031223cc7024cb32cf308 0cc4m picard12@live.de 2024-08-05T07:52:55+02:00 GitHub noreply@github.com 2024-08-05T08:52:55+03:00 vulkan : fix Qantized Mat-Vec Mul on AMD GPUs for ncols < 64 (#8855) 5587e57a76630651752031223cc7024cb32cf308 a3738b2fa7c60ef2c4592435d1aa7fb8f1f69c3e Georgi Gerganov ggerganov@gmail.com 2024-08-04T19:13:25+03:00 Georgi Gerganov ggerganov@gmail.com 2024-08-05T08:50:57+03:00 sync : ggml a3738b2fa7c60ef2c4592435d1aa7fb8f1f69c3e 655858ace0cf2720e56eb01f84ad05e0c94ada3c 0cc4m picard12@live.de 2024-08-04T17:28:08+02:00 Georgi Gerganov ggerganov@gmail.com 2024-08-05T08:50:57+03:00 vulkan : implement Stable Diffusion operators (ggml/904) 655858ace0cf2720e56eb01f84ad05e0c94ada3c c02b0a8a4dee489b29073f25a27ed6e5628e86e1 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-29T15:06:06+02:00 Georgi Gerganov ggerganov@gmail.com 2024-08-05T08:50:57+03:00 ggml : move c parameter comment to ggml_rope_ext (ggml/901) c02b0a8a4dee489b29073f25a27ed6e5628e86e1 0d6fb52be0c1b7e77eb855f3adc4952771c8ce4c wangshuai09 391746016@qq.com 2024-08-05T12:22:30+08:00 GitHub noreply@github.com 2024-08-05T12:22:30+08:00 cann: support q4_0 model (#8822) 0d6fb52be0c1b7e77eb855f3adc4952771c8ce4c 978ba3d83d17b10fdf9807006048432b5b3769fc Brandon Squizzato 35474886+bsquizz@users.noreply.github.com 2024-08-04T14:17:16-04:00 GitHub noreply@github.com 2024-08-04T20:17:16+02:00 Install curl in runtime layer (#8693) 978ba3d83d17b10fdf9807006048432b5b3769fc ecf6b7f23e664afd7ff856ec39034240ce438daa ardfork 134447697+ardfork@users.noreply.github.com 2024-08-04T18:16:23Z GitHub noreply@github.com 2024-08-04T20:16:23+02:00 Server: Don't ignore llama.cpp params (#8754) ecf6b7f23e664afd7ff856ec39034240ce438daa 01aae2b4975b57a265ce8194928fd87f2d71027e Brian Cunnie brian.cunnie@gmail.com 2024-08-04T03:55:03-07:00 GitHub noreply@github.com 2024-08-04T13:55:03+03:00 batched-bench : handle empty `-npl` (#8839) 01aae2b4975b57a265ce8194928fd87f2d71027e 4b77ea95f56a4c49bc995f08eac62a6416875ccc Daniel Bevenius daniel.bevenius@gmail.com 2024-08-03T15:07:47+02:00 Georgi Gerganov ggerganov@gmail.com 2024-08-04T13:24:59+03:00 baby-llama : remove duplicate vector include 4b77ea95f56a4c49bc995f08eac62a6416875ccc 76614f352e94d25659306d9e97321f204e5de0d3 Georgi Gerganov ggerganov@gmail.com 2024-08-04T05:53:20+03:00 GitHub noreply@github.com 2024-08-03T19:53:20-07:00 flake.lock: Update (#8847) 76614f352e94d25659306d9e97321f204e5de0d3 b72c20b85c1029d135022d39e9a20d4807c11893 jdomke 28772296+jdomke@users.noreply.github.com 2024-08-04T01:34:41+09:00 GitHub noreply@github.com 2024-08-03T18:34:41+02:00 ggml : reading the runtime sve config of the cpu (#8709) b72c20b85c1029d135022d39e9a20d4807c11893 e09a800f9a9b19c73aa78e03b4c4be8ed988f3e6 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-08-02T21:11:39+02:00 GitHub noreply@github.com 2024-08-02T15:11:39-04:00 Fix conversion of unnormalized BF16->BF16 weights (#7843) e09a800f9a9b19c73aa78e03b4c4be8ed988f3e6 0fbbd884589d585c3b43cae8c16938ffffb863b9 Mengqing Cao cmq0113@163.com 2024-08-02T16:50:53+08:00 GitHub noreply@github.com 2024-08-02T16:50:53+08:00 cann: Fix ggml_cann_im2col for 1D im2col (#8819) 0fbbd884589d585c3b43cae8c16938ffffb863b9 afbb4c1322a747d2a7b4bf67c868148f8afcc6c8 Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-08-02T01:55:17+01:00 GitHub noreply@github.com 2024-08-02T08:55:17+08:00 [SYCL] Fixing wrong VDR iq4nl value (#8812) afbb4c1322a747d2a7b4bf67c868148f8afcc6c8 b7a08fd5e0e7c898c68d1743066ea495202d9608 matteo matteogeniaccio@yahoo.it 2024-08-01T23:28:28+02:00 GitHub noreply@github.com 2024-08-01T23:28:28+02:00 ggml-cuda: Adding support for unified memory (#8035) b7a08fd5e0e7c898c68d1743066ea495202d9608 7a11eb3a260915aee16101808f291a244e2facc7 Alex O'Connell 35843486+acon96@users.noreply.github.com 2024-08-01T12:53:46-04:00 GitHub noreply@github.com 2024-08-01T18:53:46+02:00 Build: Only include execinfo.h on linux systems that support it (#8783) 7a11eb3a260915aee16101808f291a244e2facc7 c8a0090922bad576623de4aae227717085249262 slaren slarengh@gmail.com 2024-08-01T15:26:22+02:00 GitHub noreply@github.com 2024-08-01T15:26:22+02:00 cuda : fix dmmv cols requirement to 2*GGML_CUDA_DMMV_X (#8800) c8a0090922bad576623de4aae227717085249262 afbbcf3c04e3c6420cad3d72571478cd62ac176c wangshuai09 391746016@qq.com 2024-08-01T10:39:05+08:00 GitHub noreply@github.com 2024-08-01T10:39:05+08:00 cann: support q8_0 for Ascend backend (#8805) afbbcf3c04e3c6420cad3d72571478cd62ac176c ed9d2854c9de4ae1f448334294e61167b04bec2a Igor Okulist okigan@gmail.com 2024-07-31T18:59:09-05:00 GitHub noreply@github.com 2024-07-31T19:59:09-04:00 server : update llama-server embedding flag documentation (#8779) ed9d2854c9de4ae1f448334294e61167b04bec2a 398ede5efeb07b9adf9fbda7ea63f630d476a792 Clint Herron hanclinto@gmail.com 2024-07-31T15:51:06-04:00 GitHub noreply@github.com 2024-07-31T15:51:06-04:00 Build: Fix potential race condition (#8781) 398ede5efeb07b9adf9fbda7ea63f630d476a792 44d28ddd5caaa5e9de573bdaaa5b5b2448a29ace pculliton phillipculliton@gmail.com 2024-07-31T11:12:10-04:00 GitHub noreply@github.com 2024-07-31T17:12:10+02:00 Adding Gemma 2 2B configs (#8784) 44d28ddd5caaa5e9de573bdaaa5b5b2448a29ace 268c5660062270a2c19a36fc655168aa287aaec2 Borislav Stanimirov b.stanimirov@abv.bg 2024-07-31T16:40:08+03:00 GitHub noreply@github.com 2024-07-31T15:40:08+02:00 cmake : fix use of external ggml (#8787) 268c5660062270a2c19a36fc655168aa287aaec2 7e72aa74fd676a093eb9970e761085ec22734c71 Someone sergei.kozlukov@aalto.fi 2024-07-30T23:35:30+03:00 GitHub noreply@github.com 2024-07-30T13:35:30-07:00 nix: cuda: rely on propagatedBuildInputs (#8772) 7e72aa74fd676a093eb9970e761085ec22734c71 7c27a19b2eb91bb0f43c7f7aec0386cec2dddc33 Brian mofosyne@gmail.com 2024-07-31T00:57:03+10:00 GitHub noreply@github.com 2024-07-31T00:57:03+10:00 py: add_array() will not add to kv store if value is an empty array (#8774) 7c27a19b2eb91bb0f43c7f7aec0386cec2dddc33 140074bb8647df41840d6f32f4409fa8959bcf9f l3utterfly gc.pthzfoldr@gmail.com 2024-07-30T23:40:18+09:00 GitHub noreply@github.com 2024-07-30T16:40:18+02:00 added android implementation of ggml_print_backtrace_symbols (#8751) 140074bb8647df41840d6f32f4409fa8959bcf9f 6e2b6000e5fe808954a7dcef8225b5b7f2c1b9e9 Georgi Gerganov ggerganov@gmail.com 2024-07-30T15:58:57+03:00 GitHub noreply@github.com 2024-07-30T05:58:57-07:00 flake.lock: Update (#8729) 6e2b6000e5fe808954a7dcef8225b5b7f2c1b9e9 c887d8b01726b11ea03dbcaa9d44fa74422d0076 wangshuai09 391746016@qq.com 2024-07-30T18:37:35+08:00 GitHub noreply@github.com 2024-07-30T12:37:35+02:00 cann: update cmake (#8765) c887d8b01726b11ea03dbcaa9d44fa74422d0076 75af08c475e285888f66556d0f459c533b7deb95 zhentaoyu zhentao.yu@intel.com 2024-07-30T14:56:51+08:00 GitHub noreply@github.com 2024-07-30T14:56:51+08:00 [SYCL] Add `TIMESTEP_EMBEDDING` OP (#8707) 75af08c475e285888f66556d0f459c533b7deb95 439b3fc75a8deb42899ac47a8f52aae75e0339fe CarterLi999 664681047@qq.com 2024-07-30T00:38:34+08:00 GitHub noreply@github.com 2024-07-29T18:38:34+02:00 ggml: bugfix: fix the inactive elements is agnostic for risc-v vector (#8748) 439b3fc75a8deb42899ac47a8f52aae75e0339fe 0832de723695ab400316a6c49b9f712380e3a731 R0CKSTAR xiaodong.ye@mthreads.com 2024-07-29T20:56:12+08:00 GitHub noreply@github.com 2024-07-29T14:56:12+02:00 cuda : organize vendor-specific headers into vendors directory (#8746) 0832de723695ab400316a6c49b9f712380e3a731 6eeaeba126ff701f3e8f79f246805b7023709972 Meng, Hengyu hengyu.meng@intel.com 2024-07-29T10:50:27+08:00 GitHub noreply@github.com 2024-07-29T10:50:27+08:00 [SYCL] add conv support (#8688) 6eeaeba126ff701f3e8f79f246805b7023709972 4730faca618ff9cee0780580145e3cbe86f24876 Johannes Gäßler johannesg@5d6.de 2024-07-28T22:32:44+02:00 GitHub noreply@github.com 2024-07-28T22:32:44+02:00 cmake: use 1 more thread for non-ggml in CI (#8740) 4730faca618ff9cee0780580145e3cbe86f24876 4c676c85e59ef8f771f3a129e6eb217552139231 Austin 77757836+teleprint-me@users.noreply.github.com 2024-07-28T03:52:42-04:00 GitHub noreply@github.com 2024-07-28T09:52:42+02:00 chore : Fix vulkan related compiler warnings, add help text, improve CLI options (#8477) 4c676c85e59ef8f771f3a129e6eb217552139231 e54c35e4fb5777c76316a50671640e6e144c9538 compilade git@compilade.net 2024-07-28T00:42:05-04:00 GitHub noreply@github.com 2024-07-28T00:42:05-04:00 llama : refactor session file management (#8699) e54c35e4fb5777c76316a50671640e6e144c9538 5e2727fe0321c38d1664d26173c654fa1801dc5f R0CKSTAR yeahdongcn@gmail.com 2024-07-28T07:41:25+08:00 GitHub noreply@github.com 2024-07-28T01:41:25+02:00 feat: Support Moore Threads GPU (#8383) 5e2727fe0321c38d1664d26173c654fa1801dc5f 56f20aa25d5f97248a204b473c99f4040900f0e5 Georgi Gerganov ggerganov@gmail.com 2024-07-27T18:08:31+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T18:08:47+03:00 scripts : sync vulkan-shaders (#0) 56f20aa25d5f97248a204b473c99f4040900f0e5 345c8c0c87a97c1595f9c8b14833d531c8c7d8df Georgi Gerganov ggerganov@gmail.com 2024-07-27T17:19:35+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T18:07:33+03:00 scripts : sync ggml-aarch64 sources 345c8c0c87a97c1595f9c8b14833d531c8c7d8df ae7985cd7beca3b849328d169a8d592469cd021f Georgi Gerganov ggerganov@gmail.com 2024-07-27T15:57:09+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T17:43:44+03:00 ggml : add missing semicolon (#0) ae7985cd7beca3b849328d169a8d592469cd021f a05ca9369716a8319014cd1fc365980d43f8aae9 Georgi Gerganov ggerganov@gmail.com 2024-07-27T15:53:48+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T17:43:44+03:00 sync : ggml a05ca9369716a8319014cd1fc365980d43f8aae9 9f77d899b7b0d56496f679e54b797da6199fed8e Mahesh Madhav 67384846+heshpdx@users.noreply.github.com 2024-07-25T00:54:08-07:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T17:43:44+03:00 ggml : loop tiling optimizations for scalar path (ggml/898) 9f77d899b7b0d56496f679e54b797da6199fed8e 203b7f1531303a060730ec1d1e01920e70302398 Ivan Filipov 159561759+vanaka11@users.noreply.github.com 2024-07-22T14:32:02+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T17:43:44+03:00 ggml: add support for float16 input tensors in pooling operations (ggml/895) 203b7f1531303a060730ec1d1e01920e70302398 d2b851bfa131478665315bc5c7c707506c14d703 Tony Wasserka 4840017+neobrain@users.noreply.github.com 2024-07-20T20:49:44+02:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T17:43:44+03:00 vulkan : initialize vk_buffer_struct members to VK_NULL_HANDLE (ggml/893) d2b851bfa131478665315bc5c7c707506c14d703 c12b6e8ee7d905e0f299caf311689189fb1b4ac5 Borislav Stanimirov b.stanimirov@abv.bg 2024-07-12T17:24:20+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T17:43:44+03:00 cmake : only enable GGML_NATIVE and x86 flags if not crosscompiling (ggml/885) c12b6e8ee7d905e0f299caf311689189fb1b4ac5 b5e95468b1676e1e5c9d80d1eeeb26f542a38f42 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-08T12:03:42+02:00 Georgi Gerganov ggerganov@gmail.com 2024-07-27T17:43:44+03:00 ggml : remove unnecessary UNUSED macro call (ggml/880) b5e95468b1676e1e5c9d80d1eeeb26f542a38f42 92090eca212650727e38b335c1d4accfbcc9b79c Jeffrey Morgan jmorganca@gmail.com 2024-07-27T05:03:45-07:00 GitHub noreply@github.com 2024-07-27T15:03:45+03:00 llama : add support for llama 3.1 rope scaling factors (#8676) 92090eca212650727e38b335c1d4accfbcc9b79c 9d03d085dd6cb275c078690bb64073b9b043e95f Georgi Gerganov ggerganov@gmail.com 2024-07-27T14:59:29+03:00 GitHub noreply@github.com 2024-07-27T14:59:29+03:00 llama : add function for model-based max number of graph nodes (#8622) 9d03d085dd6cb275c078690bb64073b9b043e95f bfb4c74981f0a40d757b450b596a9fe4ca983d26 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-27T12:45:02+02:00 GitHub noreply@github.com 2024-07-27T13:45:02+03:00 common : add --no-warmup option for main/llama-cli (#8712) bfb4c74981f0a40d757b450b596a9fe4ca983d26 2b1f616b208a4a21c4ee7a7eb85d822ff1d787af wangshuai09 391746016@qq.com 2024-07-27T16:36:44+08:00 GitHub noreply@github.com 2024-07-27T16:36:44+08:00 cann: Fix Multi-NPU execution error (#8710) 2b1f616b208a4a21c4ee7a7eb85d822ff1d787af 01245f5b1629075543bc4478418c7d72a0b4b3c7 slaren slarengh@gmail.com 2024-07-27T04:41:55+02:00 GitHub noreply@github.com 2024-07-27T04:41:55+02:00 ggml : reduce hash table reset cost (#8698) 01245f5b1629075543bc4478418c7d72a0b4b3c7 01aec4a6310ab0160483196db0e726d78d4c94b6 Judd foldl@users.noreply.github.com 2024-07-26T16:38:12+08:00 GitHub noreply@github.com 2024-07-26T11:38:12+03:00 llama : fix order of parameters (#8706) 01aec4a6310ab0160483196db0e726d78d4c94b6 41cd47caab88c442edc50e90c8d8d0ac3e82768d Yaiko elyaiko@hotmail.com 2024-07-25T18:10:16-04:00 GitHub noreply@github.com 2024-07-26T00:10:16+02:00 server : add Speech Recognition & Synthesis to UI (#8679) 41cd47caab88c442edc50e90c8d8d0ac3e82768d 49ce0ab6d45402e8bb622bf86f86529f2b0ba552 Xuan Son Nguyen thichthat@gmail.com 2024-07-25T23:49:39+02:00 GitHub noreply@github.com 2024-07-25T23:49:39+02:00 examples : export-lora : fix issue with quantized base models (#8687) 49ce0ab6d45402e8bb622bf86f86529f2b0ba552 4226a8d10e3904db3a1297919fe6c7f06beba6c0 DavidKorczynski david@adalogics.com 2024-07-25T22:23:05+01:00 GitHub noreply@github.com 2024-07-25T23:23:05+02:00 ggml: handle ggml_init failure to fix NULL pointer deref (#8692) 4226a8d10e3904db3a1297919fe6c7f06beba6c0 bf5a81df375f1c71e41462e1f48d57db359c9e80 Georgi Gerganov ggerganov@gmail.com 2024-07-25T19:57:31+03:00 GitHub noreply@github.com 2024-07-25T19:57:31+03:00 llama : fix build + fix fabs compile warnings (#8683) bf5a81df375f1c71e41462e1f48d57db359c9e80 88954f7fbd31aeb8c75140edee03e7a8ad5e2d9c Andreas (Andi) Kunar andreask@msn.com 2024-07-25T18:01:00+02:00 GitHub noreply@github.com 2024-07-25T19:01:00+03:00 ggml : fix build on Windows with Snapdragon X (#8531) 88954f7fbd31aeb8c75140edee03e7a8ad5e2d9c ed67bcb24f2d6ac0072cae72620b2bd971741b98 Georgi Gerganov ggerganov@gmail.com 2024-07-25T18:57:44+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-25T18:58:04+03:00 tests : fix printfs (#8068) ed67bcb24f2d6ac0072cae72620b2bd971741b98 eddcb5238b2e09a37798b87cde1244017a194bcc Chen Xi xi2.chen@intel.com 2024-07-25T11:45:18Z GitHub noreply@github.com 2024-07-25T19:45:18+08:00 [SYCL] fix multi-gpu issue on sycl (#8554) eddcb5238b2e09a37798b87cde1244017a194bcc be6d7c079173d941b4f784500f9148f46cec2724 Georgi Gerganov ggerganov@gmail.com 2024-07-25T12:37:42+03:00 GitHub noreply@github.com 2024-07-25T12:37:42+03:00 ggml : add and use ggml_cpu_has_llamafile() (#8664) be6d7c079173d941b4f784500f9148f46cec2724 4b0eff3df58d8d86e47348fb73d54da3194d416d Xuan Son Nguyen thichthat@gmail.com 2024-07-25T10:39:04+02:00 GitHub noreply@github.com 2024-07-25T10:39:04+02:00 examples : remove `finetune` and `train-text-from-scratch` (#8669) 4b0eff3df58d8d86e47348fb73d54da3194d416d 8a4bad50a8ed24ed1e9df003521468dcc37320e8 Ujjawal Panchal 31011628+Ujjawal-K-Panchal@users.noreply.github.com 2024-07-25T13:43:27+05:30 GitHub noreply@github.com 2024-07-25T11:13:27+03:00 docs : Quantum -> Quantized (#8666) 8a4bad50a8ed24ed1e9df003521468dcc37320e8 68504f0970db5a3602d176953690f503059906b1 Fan Shupei dymarkfan@outlook.com 2024-07-25T15:21:09+08:00 GitHub noreply@github.com 2024-07-25T10:21:09+03:00 llama: use sliding window for phi3 (#8627) 68504f0970db5a3602d176953690f503059906b1 f19bf99c015d3d745143e8bb4f056e0ea015ad40 MorganRO8 47795945+MorganRO8@users.noreply.github.com 2024-07-24T12:48:00-04:00 GitHub noreply@github.com 2024-07-24T19:48:00+03:00 readme : update games list (#8673) f19bf99c015d3d745143e8bb4f056e0ea015ad40 3a7ac5300a7e8ebbe4a3eb5aff9dba11ed76ea61 Joe Todd joe.todd@codeplay.com 2024-07-24T14:36:00+01:00 GitHub noreply@github.com 2024-07-24T14:36:00+01:00 Build Llama SYCL Intel with static libs (#8668) 3a7ac5300a7e8ebbe4a3eb5aff9dba11ed76ea61 96952e7181929c6001b2bc69a33f240de731cc3a Thorsten Sommer SommerEngineering@users.noreply.github.com 2024-07-24T14:52:30+02:00 GitHub noreply@github.com 2024-07-24T15:52:30+03:00 readme : update UI list [no ci] (#8505) 96952e7181929c6001b2bc69a33f240de731cc3a 79167d9e49aef9caa98e13ee7ca067ec9f88b4b5 Xuan Son Nguyen thichthat@gmail.com 2024-07-24T13:48:46+02:00 GitHub noreply@github.com 2024-07-24T13:48:46+02:00 llama : fix `llama_chat_format_single` for mistral (#8657) 79167d9e49aef9caa98e13ee7ca067ec9f88b4b5 b115105f05e3372bc75b2a486c1930c365fd2846 Joe Todd joe.todd@codeplay.com 2024-07-24T11:55:26+01:00 GitHub noreply@github.com 2024-07-24T11:55:26+01:00 Re-add erroneously removed -fsycl from GGML_EXTRA_LIBS (#8667) b115105f05e3372bc75b2a486c1930c365fd2846 de280085e7917dbb7f5753de5842ff4455f82a81 Xuan Son Nguyen thichthat@gmail.com 2024-07-24T11:25:19+02:00 GitHub noreply@github.com 2024-07-24T11:25:19+02:00 add llama_lora_adapter_clear (#8653) de280085e7917dbb7f5753de5842ff4455f82a81 b841d0740855c5af1344a81f261139a45a2b39ee Xuan Son Nguyen thichthat@gmail.com 2024-07-23T23:48:37+02:00 GitHub noreply@github.com 2024-07-23T23:48:37+02:00 examples : Fix `llama-export-lora` example (#8607) b841d0740855c5af1344a81f261139a45a2b39ee 64cf50a0ed62d41e4f6c13e08a9b6b0816f46c6e Vali Malinoiu 0x4139@gmail.com 2024-07-23T17:37:42+03:00 GitHub noreply@github.com 2024-07-23T17:37:42+03:00 server : fix URL.parse in the UI (#8646) 64cf50a0ed62d41e4f6c13e08a9b6b0816f46c6e 938943cdbf4dd79005a394d732bc226f9e34e0ff Joe Todd joe.todd@codeplay.com 2024-07-23T14:58:37+01:00 GitHub noreply@github.com 2024-07-23T14:58:37+01:00 sycl : Add support for non-release DPC++ & oneMKL (#8644) 938943cdbf4dd79005a394d732bc226f9e34e0ff 751fcfc6c33ea5f43cadd4d976f8fb176871df5e Georgi Gerganov ggerganov@gmail.com 2024-07-23T13:10:17+03:00 GitHub noreply@github.com 2024-07-23T13:10:17+03:00 llama : move vocab, grammar and sampling into separate files (#8508) 751fcfc6c33ea5f43cadd4d976f8fb176871df5e 46e47417aa4f18c08738afd4d9a3e838e97ca03f 0cc4m picard12@live.de 2024-07-23T10:56:49+02:00 GitHub noreply@github.com 2024-07-23T10:56:49+02:00 Vulkan IQ4_NL Support (#8613) 46e47417aa4f18c08738afd4d9a3e838e97ca03f e7e6487ba06634edf58dfdf9673bad9df41b445a Jeroen Mostert jeroen.mostert@cm.com 2024-07-23T10:50:40+02:00 GitHub noreply@github.com 2024-07-23T10:50:40+02:00 Allow all RDNA2 archs to use sdot4 intrinsic (#8629) e7e6487ba06634edf58dfdf9673bad9df41b445a 063d99ad11f1295046610ce5b97e105849a4b573 Georgi Gerganov ggerganov@gmail.com 2024-07-23T11:28:38+03:00 GitHub noreply@github.com 2024-07-23T11:28:38+03:00 contrib : clarify PR squashing + module names (#8630) 063d99ad11f1295046610ce5b97e105849a4b573 081fe431aa8fb6307145c4feb3eed4f48cab19f8 luoyu-intel yu.luo@intel.com 2024-07-23T07:43:28Z GitHub noreply@github.com 2024-07-23T15:43:28+08:00 [SYCL] fix scratch size of softmax (#8642) 081fe431aa8fb6307145c4feb3eed4f48cab19f8 d94c6e0ccbd29ee1ba4f44e9caa8682ad94df9fa Keke Han hankeke303@163.com 2024-07-23T00:43:43+08:00 GitHub noreply@github.com 2024-07-22T19:43:43+03:00 llama : fix codeshell support (#8599) d94c6e0ccbd29ee1ba4f44e9caa8682ad94df9fa 566daa5a5b38018b2727950bbd280239adb981b6 Jason Stillerman jason.t.stillerman@gmail.com 2024-07-22T10:43:01-04:00 GitHub noreply@github.com 2024-07-22T17:43:01+03:00 llama : add support for SmolLm pre-tokenizer (#8609) 566daa5a5b38018b2727950bbd280239adb981b6 6f11a83e4e7700fdf353ed4a29599cb662c792f6 Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2024-07-22T15:44:53+02:00 GitHub noreply@github.com 2024-07-22T23:44:53+10:00 *.py: Stylistic adjustments for python (#8233) 6f11a83e4e7700fdf353ed4a29599cb662c792f6 e093dd238282a980d248db0786063f8174400f70 Georgi Gerganov ggerganov@gmail.com 2024-07-22T13:33:22+03:00 GitHub noreply@github.com 2024-07-22T13:33:22+03:00 llama : allow overrides for tokenizer flags (#8614) e093dd238282a980d248db0786063f8174400f70 50e05353e88d50b644688caa91f5955e8bdb9eb9 Georgi Gerganov ggerganov@gmail.com 2024-07-22T13:32:49+03:00 GitHub noreply@github.com 2024-07-22T13:32:49+03:00 tests : re-enable tokenizer tests (#8611) 50e05353e88d50b644688caa91f5955e8bdb9eb9 628154492a0b2dcc954a3af99e5c267097568eae Douglas Hanley thesecretaryofwar@gmail.com 2024-07-22T03:06:17-05:00 GitHub noreply@github.com 2024-07-22T11:06:17+03:00 llama : add Mistral Nemo inference support (#8604) 628154492a0b2dcc954a3af99e5c267097568eae 04bab6b7da0ed2b0a57174a57784d602aabb6c10 Jan Boon jan.boon@kaetemi.be 2024-07-22T16:02:09+08:00 GitHub noreply@github.com 2024-07-22T11:02:09+03:00 server : update doc to clarify n_keep when there is bos token (#8619) 04bab6b7da0ed2b0a57174a57784d602aabb6c10 b7c11d36e605b35206901d0e21905f1b99508e33 Mark Zhuang zhuangqiubin@gmail.com 2024-07-22T15:56:45+08:00 GitHub noreply@github.com 2024-07-22T10:56:45+03:00 ggml: fix compile error for RISC-V (#8623) b7c11d36e605b35206901d0e21905f1b99508e33 45f2c19cc57286eead7b232ce8028273a817aa4d devojony 61173062+devojony@users.noreply.github.com 2024-07-22T14:54:42+08:00 GitHub noreply@github.com 2024-07-22T09:54:42+03:00 examples: fix android example cannot be generated continuously (#8621) 45f2c19cc57286eead7b232ce8028273a817aa4d 22f281aa16f44d8f6ec2c180a0685ff27e04e714 Georgi Gerganov ggerganov@gmail.com 2024-07-21T16:45:10+03:00 GitHub noreply@github.com 2024-07-21T06:45:10-07:00 flake.lock: Update (#8610) 22f281aa16f44d8f6ec2c180a0685ff27e04e714 328884f4219c0228673cf1870ac63987fb4f9fd0 M-A maruel@gmail.com 2024-07-20T22:09:17-04:00 GitHub noreply@github.com 2024-07-20T22:09:17-04:00 examples : Rewrite pydantic_models_to_grammar_examples.py (#8493) 328884f4219c0228673cf1870ac63987fb4f9fd0 c69c63039cd75f8f33f253ab7485d82a8b4cd403 compilade git@compilade.net 2024-07-20T21:58:49-04:00 GitHub noreply@github.com 2024-07-20T21:58:49-04:00 gguf-py : fix some metadata name extraction edge cases (#8591) c69c63039cd75f8f33f253ab7485d82a8b4cd403 69c487f4ed57bb4d4514a1b7ff12608d5a8e7ef0 compilade git@compilade.net 2024-07-20T21:53:01-04:00 GitHub noreply@github.com 2024-07-20T21:53:01-04:00 convert_hf : fix Gemma v1 conversion (#8597) 69c487f4ed57bb4d4514a1b7ff12608d5a8e7ef0 07283b1a90e1320aae4762c7e03c879043910252 Johannes Gäßler johannesg@5d6.de 2024-07-20T22:25:26+02:00 GitHub noreply@github.com 2024-07-20T22:25:26+02:00 CUDA: MMQ code deduplication + iquant support (#8495) 07283b1a90e1320aae4762c7e03c879043910252 940362224d20e35f13aa5fd34a0d937ae57bdf7d Georgi Gerganov ggerganov@gmail.com 2024-07-20T17:15:42+03:00 GitHub noreply@github.com 2024-07-20T17:15:42+03:00 gguf : handle null name during init (#8587) 940362224d20e35f13aa5fd34a0d937ae57bdf7d 69b9945b44c3057ec17cb556994cd36060455d44 Michael Coppola m18coppola@gmail.com 2024-07-20T09:43:51-04:00 GitHub noreply@github.com 2024-07-20T16:43:51+03:00 llama : add support for Tekken pre-tokenizer (#8579) 69b9945b44c3057ec17cb556994cd36060455d44 c3776cacabce2ee35f172fb72be7a519752125fa Huifeng Ou 79071290+ho2103@users.noreply.github.com 2024-07-20T09:09:37-04:00 GitHub noreply@github.com 2024-07-20T16:09:37+03:00 llama.swiftui: fix end of generation bug (#8268) c3776cacabce2ee35f172fb72be7a519752125fa 87e397d00bdcedd5cbf6dfda06a7b0f302462728 Brian mofosyne@gmail.com 2024-07-20T17:35:25+10:00 GitHub noreply@github.com 2024-07-20T17:35:25+10:00 gguf_dump.py: fix markddown kv array print (#8588) 87e397d00bdcedd5cbf6dfda06a7b0f302462728 57b1d4f9eb6f2c139b31ea79626d954b261e1051 slaren slarengh@gmail.com 2024-07-19T17:17:27+02:00 GitHub noreply@github.com 2024-07-19T17:17:27+02:00 ggml : fix quant dot product with odd number of blocks (#8549) 57b1d4f9eb6f2c139b31ea79626d954b261e1051 d19754553029296c46d40b2f7bd4e2c2f531a8c5 Brian mofosyne@gmail.com 2024-07-20T00:04:38+10:00 GitHub noreply@github.com 2024-07-20T00:04:38+10:00 convert-*.py: remove add_name from ChatGLMModel class (#8590) d19754553029296c46d40b2f7bd4e2c2f531a8c5 be0cfb41752551a4680ee7dfd29f2a05b50db442 Georgi Gerganov ggerganov@gmail.com 2024-07-19T16:50:47+03:00 GitHub noreply@github.com 2024-07-19T16:50:47+03:00 llama : bump max layers from 256 to 512 (#8530) be0cfb41752551a4680ee7dfd29f2a05b50db442 b57eb9ca4fb79f3163c6a69e154ff76157a4f716 Georgi Gerganov ggerganov@gmail.com 2024-07-19T14:34:55+03:00 GitHub noreply@github.com 2024-07-19T14:34:55+03:00 readme : fix server badge b57eb9ca4fb79f3163c6a69e154ff76157a4f716 f299aa98ecc19cbc574e9d698e03999e89de3d3d Clint Herron hanclinto@gmail.com 2024-07-19T07:05:45-04:00 GitHub noreply@github.com 2024-07-19T14:05:45+03:00 ggml : add friendlier error message to fopen errors (#8575) f299aa98ecc19cbc574e9d698e03999e89de3d3d 3d0e4367d99087892e355ddbeebd232a0b2f40de Frank Mai thxcode0824@gmail.com 2024-07-19T17:44:41+08:00 GitHub noreply@github.com 2024-07-19T11:44:41+02:00 fix: typo of chatglm4 chat tmpl (#8586) 3d0e4367d99087892e355ddbeebd232a0b2f40de a15ef8f8a08e12f9c5162c221e67779e71182073 Brian mofosyne@gmail.com 2024-07-19T17:51:51+10:00 GitHub noreply@github.com 2024-07-19T17:51:51+10:00 convert-*.py: add general.name kv override (#8571) a15ef8f8a08e12f9c5162c221e67779e71182073 705b7ecf60e667ced57c15d67aa86865e3cc7aa7 Johannes Gäßler johannesg@5d6.de 2024-07-18T23:48:47+02:00 GitHub noreply@github.com 2024-07-18T23:48:47+02:00 CUDA: fix partial offloading for ne0 % 256 != 0 (#8572) 705b7ecf60e667ced57c15d67aa86865e3cc7aa7 0d2c7321e9678e91b760ebe57f0d063856bb018b 65a 10104049+65a@users.noreply.github.com 2024-07-18T07:47:12-07:00 GitHub noreply@github.com 2024-07-18T17:47:12+03:00 cmake : install all ggml public headers (#8480) 0d2c7321e9678e91b760ebe57f0d063856bb018b 672a6f101839a150180fc28891ebed379c8f2353 Eric Zhang 34133756+EZForever@users.noreply.github.com 2024-07-18T18:43:49+08:00 GitHub noreply@github.com 2024-07-18T12:43:49+02:00 server: use relative routes for static files in new UI (#8552) 672a6f101839a150180fc28891ebed379c8f2353 3807c3de04cde853418033c95e96642876545f3e Brian mofosyne@gmail.com 2024-07-18T20:40:15+10:00 GitHub noreply@github.com 2024-07-18T20:40:15+10:00 convert-*.py: GGUF Naming Convention Refactor and Metadata Override Refactor (#7499) 3807c3de04cde853418033c95e96642876545f3e e02b597be3702174e7b47b44cd03e1da1553284b RunningLeon mnsheng@yeah.net 2024-07-18T16:06:22+08:00 GitHub noreply@github.com 2024-07-18T11:06:22+03:00 server : respect `--special` cli arg (#8553) e02b597be3702174e7b47b44cd03e1da1553284b b3283448ce9a5098226afe1d8648ccc578511fe4 Johannes Gäßler johannesg@5d6.de 2024-07-17T23:35:44+02:00 GitHub noreply@github.com 2024-07-17T23:35:44+02:00 lookup: fibonacci hashing, fix crashes (#8548) b3283448ce9a5098226afe1d8648ccc578511fe4 30f80ca0bcee58669ada7a94244eeccc8c4807cc Al Mochkin 14274697+amochkin@users.noreply.github.com 2024-07-17T20:21:55+02:00 GitHub noreply@github.com 2024-07-17T20:21:55+02:00 build : Fix docker build warnings (#8535) (#8537) 30f80ca0bcee58669ada7a94244eeccc8c4807cc 1bdd8ae19f50bc6f108fa247e90688e5c60559fc Brian mofosyne@gmail.com 2024-07-18T00:57:06+10:00 GitHub noreply@github.com 2024-07-17T17:57:06+03:00 CONTRIBUTING.md : remove mention of noci (#8541) 1bdd8ae19f50bc6f108fa247e90688e5c60559fc da3913d8f9475b0d4bcbeb4936c724af4eade092 hipudding huafengchun@gmail.com 2024-07-17T19:23:50+08:00 GitHub noreply@github.com 2024-07-17T14:23:50+03:00 [CANN] Add Ascend NPU backend (#6035) da3913d8f9475b0d4bcbeb4936c724af4eade092 d65a8361fed8be97389776fb94217e937ec6b03c Masaya, Kato 62578291+msy-kato@users.noreply.github.com 2024-07-17T16:34:28+09:00 GitHub noreply@github.com 2024-07-17T10:34:28+03:00 batched: fix n_predict parameter (#8527) d65a8361fed8be97389776fb94217e937ec6b03c 5e116e8dd51775f8f1c090570be148d5d7eea6c3 Georgi Gerganov ggerganov@gmail.com 2024-07-17T10:32:59+03:00 GitHub noreply@github.com 2024-07-17T10:32:59+03:00 llama : disable context-shift for DeepSeek v2 (#8501) 5e116e8dd51775f8f1c090570be148d5d7eea6c3 1666f92dcda14e002cbb08e1028f9fff341d73ad Johannes Gäßler johannesg@5d6.de 2024-07-16T21:20:59+02:00 GitHub noreply@github.com 2024-07-16T21:20:59+02:00 make/cmake: add missing force MMQ/cuBLAS for HIP (#8515) 1666f92dcda14e002cbb08e1028f9fff341d73ad 37b12f92ab696d70f9a65d7447ce721b094fb32e Brian mofosyne@gmail.com 2024-07-16T17:14:16+10:00 GitHub noreply@github.com 2024-07-16T10:14:16+03:00 gguf-hash : update clib.json to point to original xxhash repo (#8491) 37b12f92ab696d70f9a65d7447ce721b094fb32e 0efec57787cb8d8d76b17cd3765e6521e22c1f19 Steve Bonds sbonds@gmail.com 2024-07-16T00:04:45-07:00 GitHub noreply@github.com 2024-07-16T10:04:45+03:00 export-lora : handle help argument (#8497) 0efec57787cb8d8d76b17cd3765e6521e22c1f19 7acfd4e8d55082c1b597dfc3ffe04fb5d530c6dc Georgi Gerganov ggerganov@gmail.com 2024-07-16T10:00:30+03:00 GitHub noreply@github.com 2024-07-16T10:00:30+03:00 llama : valign + remove unused ftype (#8502) 7acfd4e8d55082c1b597dfc3ffe04fb5d530c6dc 97bdd26eee11fe109dec00de75690ceef61c03f2 compilade git@compilade.net 2024-07-15T23:13:10-04:00 GitHub noreply@github.com 2024-07-15T23:13:10-04:00 convert_hf : faster lazy safetensors (#8482) 97bdd26eee11fe109dec00de75690ceef61c03f2 4db8f60fe79a391e82b0464013ada123baced96a Xuan Son Nguyen thichthat@gmail.com 2024-07-15T20:50:47+02:00 GitHub noreply@github.com 2024-07-15T20:50:47+02:00 Refactor lora adapter support (#8332) 4db8f60fe79a391e82b0464013ada123baced96a 8fac431b0692e88cdc55250f29f8d4386be82c5d Xuan Son Nguyen thichthat@gmail.com 2024-07-15T19:23:10+02:00 GitHub noreply@github.com 2024-07-15T19:23:10+02:00 fix ci (#8494) 8fac431b0692e88cdc55250f29f8d4386be82c5d f17f39ff9cb71fa7b0eda0c649bb0ce9b2d8a6b8 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-15T14:48:17+02:00 GitHub noreply@github.com 2024-07-15T15:48:17+03:00 ggml : suppress unknown pragma 'GCC' on windows (#8460) f17f39ff9cb71fa7b0eda0c649bb0ce9b2d8a6b8 9104bc20edf47f74dc49379b7d61d0c6e85a4882 M-A maruel@gmail.com 2024-07-15T08:04:56-04:00 GitHub noreply@github.com 2024-07-15T15:04:56+03:00 server: update README.md with llama-server --help output [no ci] (#8472) 9104bc20edf47f74dc49379b7d61d0c6e85a4882 fc690b018e459012cd82c37f1343e9bb658987d1 Georgi Gerganov ggerganov@gmail.com 2024-07-15T14:54:58+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-15T14:54:58+03:00 common : add --no-cont-batching arg (#6358) fc690b018e459012cd82c37f1343e9bb658987d1 16bdfa42acb09175e88cf97e9d9e4e48f616d120 NikolaiLyssogor 59844691+NikolaiLyssogor@users.noreply.github.com 2024-07-15T04:46:39-07:00 GitHub noreply@github.com 2024-07-15T14:46:39+03:00 docs: fix links in development docs [no ci] (#8481) 16bdfa42acb09175e88cf97e9d9e4e48f616d120 3dfda05956befb350745c5c2f7134d06adfe8724 Meng, Hengyu hengyu.meng@intel.com 2024-07-15T19:32:15+08:00 GitHub noreply@github.com 2024-07-15T19:32:15+08:00 [SYCL] add concat through dim 1/2 (#8483) 3dfda05956befb350745c5c2f7134d06adfe8724 bda62d7999caa8c222b6c354ac1e7c7442508539 Georgi Gerganov ggerganov@gmail.com 2024-07-15T14:10:39+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-15T14:10:39+03:00 llama : de-duplicate deepseek2 norm bda62d7999caa8c222b6c354ac1e7c7442508539 090fca7a073efe9ad3dd2b9ac12491a4f20ac957 0cc4m picard12@live.de 2024-07-15T09:38:52+02:00 GitHub noreply@github.com 2024-07-15T09:38:52+02:00 Vulkan MMQ Fix (#8479) 090fca7a073efe9ad3dd2b9ac12491a4f20ac957 aaab2419eaa17ab3aa38f4ba49c7eea406999e99 compilade git@compilade.net 2024-07-14T19:51:21-04:00 GitHub noreply@github.com 2024-07-14T19:51:21-04:00 pydantic : replace uses of __annotations__ with get_type_hints (#8474) aaab2419eaa17ab3aa38f4ba49c7eea406999e99 73cf442e7bc9baca7b3e213b261551812f1676c9 Georgi Gerganov ggerganov@gmail.com 2024-07-14T18:54:02+03:00 GitHub noreply@github.com 2024-07-14T08:54:02-07:00 flake.lock: Update (#8475) 73cf442e7bc9baca7b3e213b261551812f1676c9 e236528e7628a0e59751eee9addf21fc3c33d376 Georgi Gerganov ggerganov@gmail.com 2024-07-14T14:05:09+03:00 GitHub noreply@github.com 2024-07-14T14:05:09+03:00 llama : fix Gemma-2 Query scaling factors (#8473) e236528e7628a0e59751eee9addf21fc3c33d376 fa79495bb4897953a75607addd9d2cdd2ec63222 Brian mofosyne@gmail.com 2024-07-14T16:47:14+10:00 GitHub noreply@github.com 2024-07-14T16:47:14+10:00 gguf_hash.py: Add sha256 (#8470) fa79495bb4897953a75607addd9d2cdd2ec63222 17eb6aa8a992cda37ee65cf848d9289bd6cad860 compilade git@compilade.net 2024-07-13T23:35:10-04:00 GitHub noreply@github.com 2024-07-13T23:35:10-04:00 llama : fix pre-tokenization of non-special added tokens (#8228) 17eb6aa8a992cda37ee65cf848d9289bd6cad860 c917b67f06c42d8ca8391b9bc73f5fe62c83bf70 bandoti 141645996+bandoti@users.noreply.github.com 2024-07-13T13:12:39-03:00 GitHub noreply@github.com 2024-07-13T18:12:39+02:00 vulkan : cmake integration (#8119) c917b67f06c42d8ca8391b9bc73f5fe62c83bf70 4e24cffd8cccd653634e24ee461c252bd77b1426 Georgi Gerganov ggerganov@gmail.com 2024-07-13T18:32:33+03:00 GitHub noreply@github.com 2024-07-13T18:32:33+03:00 metal : template-ify some of the kernels (#8447) 4e24cffd8cccd653634e24ee461c252bd77b1426 6af51c0d96e6268769fc05c98d5b1a5e832c0017 Georgi Gerganov ggerganov@gmail.com 2024-07-12T14:48:15+03:00 GitHub noreply@github.com 2024-07-12T14:48:15+03:00 server : handle content array in chat API (#8449) 6af51c0d96e6268769fc05c98d5b1a5e832c0017 f53226245f421bd01b47cce43a47e791de82c636 Georgi Gerganov ggerganov@gmail.com 2024-07-12T14:48:04+03:00 GitHub noreply@github.com 2024-07-12T14:48:04+03:00 main : print error on empty input (#8456) f53226245f421bd01b47cce43a47e791de82c636 c3ebcfa148e867a68e78fd5c4f0c23e8f84c788b Daniel Bevenius daniel.bevenius@gmail.com 2024-07-12T11:05:21+02:00 GitHub noreply@github.com 2024-07-12T12:05:21+03:00 llama : suppress unary minus operator warning (#8448) c3ebcfa148e867a68e78fd5c4f0c23e8f84c788b 8a4441ea1a2564578134404f31158c318e9c0bf3 Douglas Hanley thesecretaryofwar@gmail.com 2024-07-12T03:14:12-05:00 GitHub noreply@github.com 2024-07-12T11:14:12+03:00 server : ensure batches are either all embed or all completion (#8420) 8a4441ea1a2564578134404f31158c318e9c0bf3 5aefbce27a66473d4b1263ba3f7bdd3d14245975 Armen Kaleshian kriation@users.noreply.github.com 2024-07-12T04:08:19-04:00 GitHub noreply@github.com 2024-07-12T11:08:19+03:00 docker : fix filename for convert-hf-to-gguf.py in tools.sh (#8441) 5aefbce27a66473d4b1263ba3f7bdd3d14245975 71c1121d11f1437be9421fd0cbaa011b9ef49098 Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2024-07-12T10:06:33+02:00 GitHub noreply@github.com 2024-07-12T11:06:33+03:00 convert : remove fsep token from GPTRefactForCausalLM (#8237) 71c1121d11f1437be9421fd0cbaa011b9ef49098 370b1f7e7a7514d9a63daf15f7b0f00319b7f908 Georgi Gerganov ggerganov@gmail.com 2024-07-12T10:46:14+03:00 GitHub noreply@github.com 2024-07-12T10:46:14+03:00 examples : sprintf -> snprintf (#8434) 370b1f7e7a7514d9a63daf15f7b0f00319b7f908 b549a1bbefb2f1fbb8b558bac1f2ae7967e60964 Georgi Gerganov ggerganov@gmail.com 2024-07-12T10:46:02+03:00 GitHub noreply@github.com 2024-07-12T10:46:02+03:00 ggml : minor naming changes (#8433) b549a1bbefb2f1fbb8b558bac1f2ae7967e60964 368645698ab648e390dcd7c00a2bf60efa654f57 Chen Xi xixichen08@foxmail.com 2024-07-12T00:52:04Z GitHub noreply@github.com 2024-07-12T08:52:04+08:00 [SYCL] fix the mul_mat_id ut issues (#8427) 368645698ab648e390dcd7c00a2bf60efa654f57 b078c619aa4e97fe726d61b0b5499a2e19a418a4 Nicholai Tukanov nicholaitukanov@gmail.com 2024-07-11T11:49:15-05:00 GitHub noreply@github.com 2024-07-11T18:49:15+02:00 ggml : add NVPL BLAS support (#8329) (#8425) b078c619aa4e97fe726d61b0b5499a2e19a418a4 808aba39161e5d7ca2ff24110b5aa14d2e536988 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-11T17:53:42+02:00 GitHub noreply@github.com 2024-07-11T17:53:42+02:00 cuda : suppress 'noreturn' warn in no_device_code (#8414) 808aba39161e5d7ca2ff24110b5aa14d2e536988 a977c115448e40856fb9cbe3ceb6d8ce802553b0 Johannes Gäßler johannesg@5d6.de 2024-07-11T16:47:47+02:00 GitHub noreply@github.com 2024-07-11T16:47:47+02:00 CUDA: optimize and refactor MMQ (#8416) a977c115448e40856fb9cbe3ceb6d8ce802553b0 9a55ffe6fbf7f19c865f8f277ca7cd585cc0b094 Georgi Gerganov ggerganov@gmail.com 2024-07-11T11:20:40+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-11T11:20:40+03:00 gitignore : deprecated binaries 9a55ffe6fbf7f19c865f8f277ca7cd585cc0b094 7a221b672e49dfae459b1af27210ba3f2b5419b6 compilade git@compilade.net 2024-07-11T03:41:48-04:00 GitHub noreply@github.com 2024-07-11T10:41:48+03:00 tokenize : add --no-parse-special option (#8423) 7a221b672e49dfae459b1af27210ba3f2b5419b6 278d0e18469aacf505be18ce790a63c7cc31be26 Georgi Gerganov ggerganov@gmail.com 2024-07-11T10:21:30+03:00 GitHub noreply@github.com 2024-07-11T10:21:30+03:00 llama : use F32 precision in Qwen2 attention and no FA (#8412) 278d0e18469aacf505be18ce790a63c7cc31be26 dd07a123b79f9bd9e8a4ba0447427b3083e9347a Clint Herron hanclinto@gmail.com 2024-07-10T20:08:17-04:00 GitHub noreply@github.com 2024-07-10T20:08:17-04:00 Initialize default slot sampling parameters from the global context. (#8418) dd07a123b79f9bd9e8a4ba0447427b3083e9347a f4444d992c16b6b9442f4770c7c3a10b19a08343 Clint Herron hanclinto@gmail.com 2024-07-10T12:35:18-04:00 GitHub noreply@github.com 2024-07-10T12:35:18-04:00 Name Migration: Build the deprecation-warning 'main' binary every time (#8404) f4444d992c16b6b9442f4770c7c3a10b19a08343 6b2a849d1f43d46b82d2f9c08c3275137b528784 AidanBeltonS aidan.belton@codeplay.com 2024-07-10T16:10:49+01:00 GitHub noreply@github.com 2024-07-10T16:10:49+01:00 [SYCL] Use multi_ptr to clean up deprecated warnings (#8256) 6b2a849d1f43d46b82d2f9c08c3275137b528784 0f1a39f3439825acf7e3a1663566d410be152170 Georgi Gerganov ggerganov@gmail.com 2024-07-10T15:23:29+03:00 GitHub noreply@github.com 2024-07-10T15:23:29+03:00 ggml : move sgemm sources to llamafile subfolder (#8394) 0f1a39f3439825acf7e3a1663566d410be152170 83321c6958acf20747d288031174cc1bf8816e33 Dibakar Gope dibakar.gope@arm.com 2024-07-10T07:14:51-05:00 GitHub noreply@github.com 2024-07-10T15:14:51+03:00 ggml : add AArch64 optimized GEMV and GEMM Q4 kernels (#5780) 83321c6958acf20747d288031174cc1bf8816e33 cc61948b1f97165b46990f4c2d9699bf9bb64443 M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2024-07-10T15:12:35+03:00 GitHub noreply@github.com 2024-07-10T15:12:35+03:00 gguf-py rel pipeline (#8410) cc61948b1f97165b46990f4c2d9699bf9bb64443 7a80710d93ee0f4e0d335d65984ae747e62c0337 Borislav Stanimirov b.stanimirov@abv.bg 2024-07-10T14:45:44+03:00 GitHub noreply@github.com 2024-07-10T14:45:44+03:00 llama : C++20 compatibility for u8 strings (#8408) 7a80710d93ee0f4e0d335d65984ae747e62c0337 a8be1e6f5995bec3b1d8474d48ce3a139553a8e1 Borislav Stanimirov b.stanimirov@abv.bg 2024-07-10T14:40:53+03:00 GitHub noreply@github.com 2024-07-10T14:40:53+03:00 msvc : silence codecvt c++17 deprecation warnings (#8395) a8be1e6f5995bec3b1d8474d48ce3a139553a8e1 e4dd31ff892627665d3c53c5d1a03c0d282d9d45 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-07-10T13:38:58+02:00 GitHub noreply@github.com 2024-07-10T14:38:58+03:00 llama : add assert about missing llama_encode() call (#8400) e4dd31ff892627665d3c53c5d1a03c0d282d9d45 8f0fad42b9589f9b11362c74ea5338c51e4c7e61 RunningLeon maningsheng@sensetime.com 2024-07-10T19:26:40+08:00 GitHub noreply@github.com 2024-07-10T14:26:40+03:00 py : fix converter for internlm2 (#8321) 8f0fad42b9589f9b11362c74ea5338c51e4c7e61 a59f8fdc85e1119d470d8766e29617962549d993 laik laik.lj@me.com 2024-07-10T19:19:10+08:00 GitHub noreply@github.com 2024-07-10T14:19:10+03:00 py : fix extra space in convert_hf_to_gguf.py (#8407) a59f8fdc85e1119d470d8766e29617962549d993 fd560fe680c72fd0a0af2bc8881add20ad919071 Clint Herron hanclinto@gmail.com 2024-07-09T18:26:40-04:00 GitHub noreply@github.com 2024-07-09T18:26:40-04:00 Server: Enable setting default sampling parameters via command-line (#8402) fd560fe680c72fd0a0af2bc8881add20ad919071 e500d6135ac42c4a23baf6a9a1a934dc023e5c7d Andy Salerno andysalerno@gmail.com 2024-07-09T11:58:44-07:00 GitHub noreply@github.com 2024-07-09T14:58:44-04:00 Update README.md to fix broken link to docs (#8399) e500d6135ac42c4a23baf6a9a1a934dc023e5c7d a03e8dd99d3954e7547137936c5a2a3b348bdb7f Clint Herron hanclinto@gmail.com 2024-07-09T11:54:43-04:00 GitHub noreply@github.com 2024-07-09T11:54:43-04:00 Deprecation warning to assist with migration to new binary names (#8283) a03e8dd99d3954e7547137936c5a2a3b348bdb7f 5b0b8d8cfb5ddf2118f686ba6c30fab3f71b384b Johannes Gäßler johannesg@5d6.de 2024-07-09T17:11:07+02:00 GitHub noreply@github.com 2024-07-09T17:11:07+02:00 make/cmake: LLAMA_NO_CCACHE -> GGML_NO_CCACHE (#8392) 5b0b8d8cfb5ddf2118f686ba6c30fab3f71b384b 9925ca4087a34ab973b07bf06c0b770cb586830b Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2024-07-09T15:03:15+01:00 GitHub noreply@github.com 2024-07-09T22:03:15+08:00 sycl : Reenabled mmvq path for the SYCL Nvidia Backend (#8372) 9925ca4087a34ab973b07bf06c0b770cb586830b 9beb2dda038e2107a39a95def94a4cf971e048ea Borislav Stanimirov b.stanimirov@abv.bg 2024-07-09T11:38:00+03:00 GitHub noreply@github.com 2024-07-09T11:38:00+03:00 cmake : allow external ggml (#8370) 9beb2dda038e2107a39a95def94a4cf971e048ea 7d0e23d72ef4540d0d4409cb63ae682c17d53926 daghanerdonmez 44506702+daghanerdonmez@users.noreply.github.com 2024-07-09T09:16:00+03:00 GitHub noreply@github.com 2024-07-09T09:16:00+03:00 readme : fix typo [no ci] (#8389) 7d0e23d72ef4540d0d4409cb63ae682c17d53926 7fdb6f73e35605c8dbc39e9f19cd9ed84dbc87f2 compilade git@compilade.net 2024-07-09T01:04:49-04:00 GitHub noreply@github.com 2024-07-09T01:04:49-04:00 gguf-py : do not use internal numpy types (#7472) 7fdb6f73e35605c8dbc39e9f19cd9ed84dbc87f2 a130eccef42b75a84da270411cefeed45c153e30 Georgi Gerganov ggerganov@gmail.com 2024-07-09T01:36:38+03:00 GitHub noreply@github.com 2024-07-08T15:36:38-07:00 flake.lock: Update (#8342) a130eccef42b75a84da270411cefeed45c153e30 c4dd11d1d3903e1922c06242e189f6310fc4d8c3 Alberto Cabrera Pérez alberto.cabrera@codeplay.com 2024-07-08T21:35:17+01:00 GitHub noreply@github.com 2024-07-08T22:35:17+02:00 labeler : updated sycl to match docs and code refactor (#8373) c4dd11d1d3903e1922c06242e189f6310fc4d8c3 2ec846d558f6385ea647f7b8e665eb249c1ebce7 b4b4o zwbao@foxmail.com 2024-07-08T22:19:24+08:00 GitHub noreply@github.com 2024-07-08T17:19:24+03:00 readme : fix web link error [no ci] (#8347) 2ec846d558f6385ea647f7b8e665eb249c1ebce7 3f2d538b817112ad8429341c7e8657dcd660f4d3 Alberto Cabrera Pérez alberto.cabrera@intel.com 2024-07-08T14:22:41+01:00 GitHub noreply@github.com 2024-07-08T14:22:41+01:00 sycl : fix powf call in device code (#8368) 3f2d538b817112ad8429341c7e8657dcd660f4d3 2ee44c9a1865a928ccbbc16a2d7841d7513f31c1 Georgi Gerganov ggerganov@gmail.com 2024-07-08T13:51:31+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-08T13:51:31+03:00 scripts : fix sync for sycl 2ee44c9a1865a928ccbbc16a2d7841d7513f31c1 6847d54c4f72f8bf6767b6feae7a95394654335e Georgi Gerganov ggerganov@gmail.com 2024-07-08T10:39:50+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-08T12:23:00+03:00 sync : ggml 6847d54c4f72f8bf6767b6feae7a95394654335e fde13b3bb9f569f07fd8af74696ee48a43d05131 Georgi Gerganov ggerganov@gmail.com 2024-07-08T10:39:36+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-08T12:23:00+03:00 tests : fix whitespace (#0) fde13b3bb9f569f07fd8af74696ee48a43d05131 470939d483d1c89b7292f78bac1fd27c42c171ce John Balis phobossystems@gmail.com 2024-07-02T11:09:52-05:00 Georgi Gerganov ggerganov@gmail.com 2024-07-08T12:23:00+03:00 feat: cuda implementation for `ggml_conv_transpose_1d` (ggml/854) 470939d483d1c89b7292f78bac1fd27c42c171ce 6f0dbf6ab087bcd286fb78560099ca0458316735 Kevin Wang kevmo314@gmail.com 2024-07-08T03:26:53-04:00 GitHub noreply@github.com 2024-07-08T10:26:53+03:00 common : preallocate sampling token data vector (#8363) 6f0dbf6ab087bcd286fb78560099ca0458316735 ffd00797d81ef7db1528b9e10adbdc333ade6495 Georgi Gerganov ggerganov@gmail.com 2024-07-08T09:34:35+03:00 GitHub noreply@github.com 2024-07-08T09:34:35+03:00 infill : assert prefix/suffix tokens + remove old space logic (#8351) ffd00797d81ef7db1528b9e10adbdc333ade6495 04ce3a8b19256a155aea4d14eaa87edf274c93c3 Kevin Wang kevmo314@gmail.com 2024-07-08T02:31:55-04:00 GitHub noreply@github.com 2024-07-08T09:31:55+03:00 common : avoid unnecessary logits fetch (#8358) 04ce3a8b19256a155aea4d14eaa87edf274c93c3 3fd62a6b1c9ca7b7c0093e984cc9c133c6f2726d toyer 2042519524@qq.com 2024-07-08T13:57:19+08:00 GitHub noreply@github.com 2024-07-08T08:57:19+03:00 readme : add supported glm models (#8360) 3fd62a6b1c9ca7b7c0093e984cc9c133c6f2726d a8db2a9ce64cd4417f6a312ab61858f17f0f8584 compilade git@compilade.net 2024-07-07T15:04:39-04:00 GitHub noreply@github.com 2024-07-07T15:04:39-04:00 py : type-check all Python scripts with Pyright (#8341) a8db2a9ce64cd4417f6a312ab61858f17f0f8584 4090ea5501c702cd858095c394ba068919c56cc8 Denis Spasyuk 34203011+dspasyuk@users.noreply.github.com 2024-07-07T09:08:28-06:00 GitHub noreply@github.com 2024-07-07T17:08:28+02:00 Update llama-cli documentation (#8315) 4090ea5501c702cd858095c394ba068919c56cc8 f1948f1e108157d5e7eb60fc8f24a10412e5d4ff Alex Tuddenham 61622354+AlexsCode@users.noreply.github.com 2024-07-07T15:59:14+01:00 GitHub noreply@github.com 2024-07-07T17:59:14+03:00 ci : add checks for cmake,make and ctest in ci/run.sh (#8200) f1948f1e108157d5e7eb60fc8f24a10412e5d4ff f7cab35ef9e66c57b4a416d09eb6c814e0ba4e4c Andy Tai andy-tai@users.noreply.github.com 2024-07-07T06:21:37-07:00 GitHub noreply@github.com 2024-07-07T16:21:37+03:00 readme : update bindings list (#8222) f7cab35ef9e66c57b4a416d09eb6c814e0ba4e4c 905942abdba5ba0b28a1b0805e51e4f818c54bc9 Brian mofosyne@gmail.com 2024-07-07T22:58:43+10:00 GitHub noreply@github.com 2024-07-07T22:58:43+10:00 gguf-hash: model wide and per tensor hashing using xxhash and sha1 (#8048) 905942abdba5ba0b28a1b0805e51e4f818c54bc9 b5040086d436e7345e4fa33a5b9558060c75603f toyer 2042519524@qq.com 2024-07-07T20:52:10+08:00 GitHub noreply@github.com 2024-07-07T15:52:10+03:00 llama : support glm3 and glm4 (#8031) b5040086d436e7345e4fa33a5b9558060c75603f d39130a3985ce0747d7229a6b7ebebb6376b5abf Georgi Gerganov ggerganov@gmail.com 2024-07-07T14:59:02+03:00 GitHub noreply@github.com 2024-07-07T14:59:02+03:00 llama : fix n_rot default (#8348) d39130a3985ce0747d7229a6b7ebebb6376b5abf b81ba1f96b06c691020429a2d2dfcbada899ca86 compilade git@compilade.net 2024-07-07T07:23:38-04:00 GitHub noreply@github.com 2024-07-07T14:23:38+03:00 py : use cpu-only torch in requirements.txt (#8335) b81ba1f96b06c691020429a2d2dfcbada899ca86 210eb9ed0ac3979a93e37568d96447ec3e95ad05 standby24x7 standby24x7@gmail.com 2024-07-07T19:38:02+09:00 GitHub noreply@github.com 2024-07-07T13:38:02+03:00 finetune: Rename command name in README.md (#8343) 210eb9ed0ac3979a93e37568d96447ec3e95ad05 cb4d86c4d723af87d3d7e3177e9485f200391384 standby24x7 standby24x7@gmail.com 2024-07-07T19:37:47+09:00 GitHub noreply@github.com 2024-07-07T13:37:47+03:00 finetune: Rename an old command name in finetune.sh (#8344) cb4d86c4d723af87d3d7e3177e9485f200391384 86e7299ef5dff0f388922dc6fcbce009e99d8005 Bjarke Viksøe 164612031+bviksoe@users.noreply.github.com 2024-07-07T11:10:38+02:00 GitHub noreply@github.com 2024-07-07T11:10:38+02:00 server: Retrieve prompt template in /props (#8337) 86e7299ef5dff0f388922dc6fcbce009e99d8005 60d83a0149849e9217e4b8ae26e277a41aea906e Derrick T. Woolworth dwoolworth@gmail.com 2024-07-06T15:32:04-05:00 GitHub noreply@github.com 2024-07-06T22:32:04+02:00 added support for Authorization Bearer tokens when downloading model (#8307) 60d83a0149849e9217e4b8ae26e277a41aea906e 87e25a1d1bd26eb06d1cab9e2ee4e14a7a0be33c Xuan Son Nguyen thichthat@gmail.com 2024-07-06T19:01:23+02:00 GitHub noreply@github.com 2024-07-06T19:01:23+02:00 update main readme (#8333) 87e25a1d1bd26eb06d1cab9e2ee4e14a7a0be33c 213701b51a17175d0d326b566efc03f30ec7fbe6 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-06T09:22:16+02:00 GitHub noreply@github.com 2024-07-06T10:22:16+03:00 llama : add early return for empty range (#8327) 213701b51a17175d0d326b566efc03f30ec7fbe6 be20e7f49d9e5c6d9e8d9b4871eeba3df7a1639d jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-07-05T19:01:35+02:00 GitHub noreply@github.com 2024-07-05T19:01:35+02:00 Detokenizer fixes (#8039) be20e7f49d9e5c6d9e8d9b4871eeba3df7a1639d 7ed03b8974269b6c48e55c4245d12fb3264a6cf5 Xuan Son Nguyen thichthat@gmail.com 2024-07-05T18:08:32+02:00 GitHub noreply@github.com 2024-07-05T18:08:32+02:00 Reorganize documentation pages (#8325) 7ed03b8974269b6c48e55c4245d12fb3264a6cf5 1d894a790e62b10d066adcdd9c9feb559455b7d2 Georgi Gerganov ggerganov@gmail.com 2024-07-05T17:32:09+03:00 Georgi Gerganov ggerganov@gmail.com 2024-07-05T17:32:09+03:00 llama : fix compile warning (#8304) 1d894a790e62b10d066adcdd9c9feb559455b7d2 1f3e1b66e21310ed78b964f72f19766549633f0e Natsu chino@hotococoa.moe 2024-07-05T22:29:35+08:00 GitHub noreply@github.com 2024-07-05T17:29:35+03:00 cmake : add GGML_BUILD and GGML_SHARED macro definitions (#8281) 1f3e1b66e21310ed78b964f72f19766549633f0e 148ec970b62c3c5ae0a8bfdaad2fc237aaae350d Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-07-05T13:23:25+01:00 GitHub noreply@github.com 2024-07-05T13:23:25+01:00 Enabled more data types for oneMKL gemm_batch (#8236) 148ec970b62c3c5ae0a8bfdaad2fc237aaae350d 2cccbaa0086c62801f95405131a5b2faf3ba1de8 Georgi Gerganov ggerganov@gmail.com 2024-07-05T10:15:36+03:00 GitHub noreply@github.com 2024-07-05T10:15:36+03:00 convert : remove AWQ remnants (#8320) 2cccbaa0086c62801f95405131a5b2faf3ba1de8 8e558309dc149dc1f9fd159185b0b9071527ffb5 Georgi Gerganov ggerganov@gmail.com 2024-07-05T10:15:24+03:00 GitHub noreply@github.com 2024-07-05T10:15:24+03:00 llama : minor indentation during tensor loading (#8304) 8e558309dc149dc1f9fd159185b0b9071527ffb5 0a423800ffe4e5da3d83527ef3473da88cd78146 Johannes Gäßler johannesg@5d6.de 2024-07-05T09:06:31+02:00 GitHub noreply@github.com 2024-07-05T09:06:31+02:00 CUDA: MMQ support for iq4_nl, iq4_xs (#8278) 0a423800ffe4e5da3d83527ef3473da88cd78146 d12f781074b92589a72a36ffabb583933f7b9dc0 Daniele 57776841+daniandtheweb@users.noreply.github.com 2024-07-05T07:06:09Z GitHub noreply@github.com 2024-07-05T09:06:09+02:00 CUDA: revert part of the RDNA1 optimizations (#8309) d12f781074b92589a72a36ffabb583933f7b9dc0 bcefa03bc01a41aace2e200ee8e77827d6d39b4f Douglas Hanley thesecretaryofwar@gmail.com 2024-07-05T02:05:56-05:00 GitHub noreply@github.com 2024-07-05T10:05:56+03:00 llama : streamline embeddings from "non-embedding" models (#8087) bcefa03bc01a41aace2e200ee8e77827d6d39b4f 5a7447c5692c9e3dde1161e8e69edb76d3d34714 Johannes Gäßler johannesg@5d6.de 2024-07-05T09:05:34+02:00 GitHub noreply@github.com 2024-07-05T09:05:34+02:00 CUDA: fix MMQ stream-k rounding if ne00 % 128 != 0 (#8311) 5a7447c5692c9e3dde1161e8e69edb76d3d34714 61ecafa3905a02a299b7ae889b5578cfeb8d79df Pieter Ouwerkerk pieter.ouwerkerk@gmail.com 2024-07-05T02:58:41-04:00 GitHub noreply@github.com 2024-07-05T09:58:41+03:00 readme : fix minor typos [no ci] (#8314) 61ecafa3905a02a299b7ae889b5578cfeb8d79df aa5898dc53afcf77f16222cd719e10b29049f95a Daniel Bevenius daniel.bevenius@gmail.com 2024-07-05T08:14:24+02:00 GitHub noreply@github.com 2024-07-05T09:14:24+03:00 passkey : add short intro to README.md [no-ci] (#8317) aa5898dc53afcf77f16222cd719e10b29049f95a 6c05752c507f51b88348f16d9e2c8df49f66c028 Georgi Gerganov ggerganov@gmail.com 2024-07-05T09:10:03+03:00 GitHub noreply@github.com 2024-07-05T09:10:03+03:00 llama : prefer n_ over num_ prefix (#8308) 6c05752c507f51b88348f16d9e2c8df49f66c028 a9554e20b66546b0549aebe2e1034bc8afe9d809 Georgi Gerganov ggerganov@gmail.com 2024-07-05T09:09:47+03:00 GitHub noreply@github.com 2024-07-05T09:09:47+03:00 contributing : update guidelines (#8316) a9554e20b66546b0549aebe2e1034bc8afe9d809 e235b267a2539d043734ff340eff74107722eb57 luoyu-intel yu.luo@intel.com 2024-07-05T05:06:13Z GitHub noreply@github.com 2024-07-05T13:06:13+08:00 [SYCL] Fix WARP_SIZE=16 bug of Intel GPU (#8266) e235b267a2539d043734ff340eff74107722eb57 f09b7cb609d80b8031803f89255991dc8b35db69 Georgi Gerganov ggerganov@gmail.com 2024-07-05T07:53:33+03:00 GitHub noreply@github.com 2024-07-05T07:53:33+03:00 py : switch to snake_case (#8305) f09b7cb609d80b8031803f89255991dc8b35db69 a38b884c6c4b0c256583acfaaabdf556c62fabea Neo Zhang Jianyu jianyu.zhang@intel.com 2024-07-05T10:32:29+08:00 GitHub noreply@github.com 2024-07-05T10:32:29+08:00 rm get_work_group_size() by local cache for performance (#8286) a38b884c6c4b0c256583acfaaabdf556c62fabea d7fd29fff16456ce9c3a23fd2d09a66256b05aff Xuan Son Nguyen thichthat@gmail.com 2024-07-04T20:55:03+02:00 GitHub noreply@github.com 2024-07-04T20:55:03+02:00 cli: add EOT when user hit Ctrl+C (#8296) d7fd29fff16456ce9c3a23fd2d09a66256b05aff 6f63d646c1a06a6e09f721009a2676864ae04e31 Icecream95 the.real.icecream95@gmail.com 2024-07-05T05:14:21+12:00 GitHub noreply@github.com 2024-07-04T20:14:21+03:00 llama : add OpenELM support (#7359) 6f63d646c1a06a6e09f721009a2676864ae04e31 51d2ebadbbf365b894f3888361df42dbacb12b7a Daniel Bevenius daniel.bevenius@gmail.com 2024-07-04T18:38:58+02:00 GitHub noreply@github.com 2024-07-04T19:38:58+03:00 tokenize : add --show-count (token) option (#8299) 51d2ebadbbf365b894f3888361df42dbacb12b7a 1e920018d38aee270a044cc48eaefe7c64cb8750 ditsuke ditsuke@protonmail.com 2024-07-04T20:54:35+05:30 Someone else@someonex.net 2024-07-04T15:39:13Z build: Export hf-to-gguf as snakecase 1e920018d38aee270a044cc48eaefe7c64cb8750 01a5f06550a866bd63717635e5e7e1ff4203b873 ditsuke ditsuke@protonmail.com 2024-07-03T01:02:56+05:30 Someone else@someonex.net 2024-07-04T15:39:13Z doc: Add context for why we add an explicit pytorch source 01a5f06550a866bd63717635e5e7e1ff4203b873 07786a61a2097306ee496f565f78796f1aa205e6 ditsuke ditsuke@protonmail.com 2024-07-02T15:48:13+05:30 Someone else@someonex.net 2024-07-04T15:39:13Z chore: Remove rebase artifacts 07786a61a2097306ee496f565f78796f1aa205e6 de14e2ea2b542386dcb800226eb360be76f433fd ditsuke ditsuke@protonmail.com 2024-07-02T15:35:43+05:30 Someone else@someonex.net 2024-07-04T15:39:13Z chore: Fixup requirements and build de14e2ea2b542386dcb800226eb360be76f433fd 821922916f95cc3853fc7f58ea2f1e15a0ffa669 ditsuke ditsuke@protonmail.com 2024-07-02T15:18:13+05:30 Someone else@someonex.net 2024-07-04T15:39:13Z chore: ignore all __pychache__ 821922916f95cc3853fc7f58ea2f1e15a0ffa669 b1c3f26e5e882fa46d7a6704d893b31a025e9000 ditsuke ditsuke@protonmail.com 2024-03-10T23:21:46+05:30 Someone else@someonex.net 2024-07-04T15:39:13Z fix: Update script paths in CI scripts b1c3f26e5e882fa46d7a6704d893b31a025e9000 b0a46993dfbf8b8127598f319d4dcfdd83824ba8 ditsuke ditsuke@protonmail.com 2024-02-29T01:47:15+05:30 Someone else@someonex.net 2024-07-04T15:39:13Z fix: Actually include scripts in build b0a46993dfbf8b8127598f319d4dcfdd83824ba8 807b0c49ff7071094f97ebc3a0a8e2b9e274f503 ditsuke ditsuke@protonmail.com 2024-02-27T12:01:02+05:30 Someone else@someonex.net 2024-07-04T15:39:13Z build(python): Package scripts with pip-0517 compliance 807b0c49ff7071094f97ebc3a0a8e2b9e274f503 f8c4c0738d72d2162736edd72dd5db8b269adca1 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-07-04T15:46:11+02:00 GitHub noreply@github.com 2024-07-04T15:46:11+02:00 Inference support for T5 and FLAN-T5 model families (#5763) f8c4c0738d72d2162736edd72dd5db8b269adca1 402d6feffa0572d1c7a957901b6d1702bd188484 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-04T12:53:42+02:00 GitHub noreply@github.com 2024-07-04T13:53:42+03:00 tests : add _CRT_SECURE_NO_WARNINGS for WIN32 (#8231) 402d6feffa0572d1c7a957901b6d1702bd188484 20fc3804bfb727074bc270b6eacb60af8d0bf7d4 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-04T12:50:57+02:00 GitHub noreply@github.com 2024-07-04T13:50:57+03:00 llama : suppress unref var in Windows MSVC (#8150) 20fc3804bfb727074bc270b6eacb60af8d0bf7d4 f619024764e72261f14d7c31d892b8fb976603b4 Georgi Gerganov ggerganov@gmail.com 2024-07-04T10:41:03+03:00 GitHub noreply@github.com 2024-07-04T10:41:03+03:00 convert : fix gemma v1 tokenizer convert (#8248) f619024764e72261f14d7c31d892b8fb976603b4 d23287f122c34ebef368742116d53a0ccb2041ee AidanBeltonS aidan.belton@codeplay.com 2024-07-04T02:07:19+01:00 GitHub noreply@github.com 2024-07-04T09:07:19+08:00 [SYCL] Remove unneeded semicolons (#8280) d23287f122c34ebef368742116d53a0ccb2041ee 5f2d4e60e202aabee10051e6615bb821e51787be Daniele 57776841+daniandtheweb@users.noreply.github.com 2024-07-03T23:02:58Z GitHub noreply@github.com 2024-07-04T01:02:58+02:00 Define and optimize RDNA1 (#8085) 5f2d4e60e202aabee10051e6615bb821e51787be 916248af1f3c16abd7408de848e025da095c621c slaren slarengh@gmail.com 2024-07-03T19:33:31+02:00 GitHub noreply@github.com 2024-07-03T20:33:31+03:00 ppl : fix n_seq_max for perplexity (#8277) 916248af1f3c16abd7408de848e025da095c621c f8d6a23804f3798ff2869da68c1223b618df09ec Xuan Son Nguyen thichthat@gmail.com 2024-07-03T16:01:54+02:00 GitHub noreply@github.com 2024-07-03T16:01:54+02:00 fix phi 3 conversion (#8262) f8d6a23804f3798ff2869da68c1223b618df09ec fadde6713506d9e6c124f5680ab8c7abebe31837 Judd foldl@users.noreply.github.com 2024-07-03T20:40:16+08:00 GitHub noreply@github.com 2024-07-03T14:40:16+02:00 fix typo (#8267) fadde6713506d9e6c124f5680ab8c7abebe31837 a27152b602b369e76f85b7cb7b872a321b7218f7 AidanBeltonS aidan.belton@codeplay.com 2024-07-03T02:55:34+01:00 GitHub noreply@github.com 2024-07-03T09:55:34+08:00 Dequant improvements rebase (#8255) a27152b602b369e76f85b7cb7b872a321b7218f7 3e2618bc7bf9e9fbf58c32cc3c8dd7d5df1de27e MistApproach 98988043+MistApproach@users.noreply.github.com 2024-07-02T22:56:46+02:00 GitHub noreply@github.com 2024-07-02T22:56:46+02:00 fix: add missing short command line argument -mli for multiline-input (#8261) 3e2618bc7bf9e9fbf58c32cc3c8dd7d5df1de27e 07a3fc0608a68c0c93a5fbfa9c58f4c9ec64cb81 Clint Herron hanclinto@gmail.com 2024-07-02T13:19:56-04:00 GitHub noreply@github.com 2024-07-02T13:19:56-04:00 Adding step to `clean` target to remove legacy binary names to reduce upgrade / migration confusion arising from #7809. (#8257) 07a3fc0608a68c0c93a5fbfa9c58f4c9ec64cb81 968967376dc2c018d29f897c4883d335bbf384fb Clint Herron hanclinto@gmail.com 2024-07-02T12:18:10-04:00 GitHub noreply@github.com 2024-07-02T12:18:10-04:00 Removes multiple newlines at the end of files that is breaking the editorconfig step of CI. (#8258) 968967376dc2c018d29f897c4883d335bbf384fb 023b8807e10bc3ade24a255f01c1ad2a01bb4228 Faisal Zaghloul faisal.zaghloul@gmail.com 2024-07-02T10:36:00-04:00 GitHub noreply@github.com 2024-07-02T16:36:00+02:00 Add `JAIS` model(s) (#8118) 023b8807e10bc3ade24a255f01c1ad2a01bb4228 0e0590adab9f367b15ae2bf090a6d24f9df47ff1 Daniel Bevenius daniel.bevenius@gmail.com 2024-07-02T08:40:49+02:00 GitHub noreply@github.com 2024-07-02T09:40:49+03:00 convert-hf : print output file name when completed (#8181) 0e0590adab9f367b15ae2bf090a6d24f9df47ff1 a9f3b102157ba992cfe058909b7f6e1906d2d647 slaren slarengh@gmail.com 2024-07-02T08:39:38+02:00 GitHub noreply@github.com 2024-07-02T09:39:38+03:00 cuda : update supports_op for matrix multiplication (#8245) a9f3b102157ba992cfe058909b7f6e1906d2d647 d08c20eddedb24515a3212e2de66bdff41a26b8c luoyu-intel yu.luo@intel.com 2024-07-02T04:50:07Z GitHub noreply@github.com 2024-07-02T12:50:07+08:00 [SYCL] Fix win build conflict of math library (#8230) d08c20eddedb24515a3212e2de66bdff41a26b8c 5fac350b9cc49d0446fc291b9c4ad53666c77591 luoyu-intel yu.luo@intel.com 2024-07-02T02:16:00Z GitHub noreply@github.com 2024-07-02T10:16:00+08:00 [SYCL] Fix the sub group size of Intel (#8106) 5fac350b9cc49d0446fc291b9c4ad53666c77591 cb5fad4c6c2cbef92e9b8b63449e1cb7664e4846 Xuan Son Nguyen thichthat@gmail.com 2024-07-02T01:07:23+02:00 GitHub noreply@github.com 2024-07-02T01:07:23+02:00 Fix gemma2 tokenizer convert (#8244) cb5fad4c6c2cbef92e9b8b63449e1cb7664e4846 dae57a1ebc1c9bd5693ab999e19d77c5506ae559 Johannes Gäßler johannesg@5d6.de 2024-07-01T20:39:06+02:00 GitHub noreply@github.com 2024-07-01T20:39:06+02:00 CUDA: refactor and optimize IQ MMVQ (#8215) dae57a1ebc1c9bd5693ab999e19d77c5506ae559 49122a873f54615626d1b49a2a39013ed4be98d5 Mateusz Charytoniuk mateusz.charytoniuk@protonmail.com 2024-07-01T19:13:22+02:00 GitHub noreply@github.com 2024-07-01T20:13:22+03:00 readme: add Paddler to the list of projects (#8239) 49122a873f54615626d1b49a2a39013ed4be98d5 0ddeff10230b88f1fa9866bbe5fe0d71ba2323a0 Xuan Son Nguyen thichthat@gmail.com 2024-07-01T18:48:34+02:00 GitHub noreply@github.com 2024-07-01T18:48:34+02:00 gemma2: add sliding window mask (#8227) 0ddeff10230b88f1fa9866bbe5fe0d71ba2323a0 3840b6f593751a0ba636bfda73b630cd6c29d7b5 Roni sulpher@gmx.net 2024-07-01T14:48:16+02:00 GitHub noreply@github.com 2024-07-01T15:48:16+03:00 readme : update tool list (#8209) 3840b6f593751a0ba636bfda73b630cd6c29d7b5 257f8e41e24b5bbfc27d9e907189a3e0cdb650d4 Michael Francis edude03@gmail.com 2024-07-01T07:47:04-04:00 GitHub noreply@github.com 2024-07-01T14:47:04+03:00 nix : enable curl (#8043) 257f8e41e24b5bbfc27d9e907189a3e0cdb650d4 694c59cb42d1ebd6a7d912ca65d3d7363e0f14c9 Georgi Gerganov ggerganov@gmail.com 2024-07-01T14:46:18+03:00 GitHub noreply@github.com 2024-07-01T14:46:18+03:00 nix : remove OpenCL remnants (#8235) 694c59cb42d1ebd6a7d912ca65d3d7363e0f14c9 197fe6c1d7bec6718ce901f0141b2725240f298c iacore 74560659+iacore@users.noreply.github.com 2024-07-01T11:40:58Z GitHub noreply@github.com 2024-07-01T13:40:58+02:00 Document BERT support. (#8205) 197fe6c1d7bec6718ce901f0141b2725240f298c d0a7145ba99ed3a8bc3145aa785b5c86ffe65020 zhentaoyu zhentao.yu@intel.com 2024-07-01T19:39:06+08:00 GitHub noreply@github.com 2024-07-01T19:39:06+08:00 [SYCL] Update SYCL-Rope op and Refactor (#8157) d0a7145ba99ed3a8bc3145aa785b5c86ffe65020 9ef07800622e4c371605f9419864d15667c3558f Georgi Gerganov ggerganov@gmail.com 2024-07-01T02:09:34+03:00 GitHub noreply@github.com 2024-06-30T16:09:34-07:00 flake.lock: Update (#8218) 9ef07800622e4c371605f9419864d15667c3558f 1c5eba6f8e628fb0a98afb27d8aaeb3b0e136451 Xuan Son Nguyen thichthat@gmail.com 2024-06-30T20:27:13+02:00 GitHub noreply@github.com 2024-06-30T20:27:13+02:00 Fix new line issue with chat template, disable template when in-prefix/suffix is set (#8203) 1c5eba6f8e628fb0a98afb27d8aaeb3b0e136451 72272b83a3878e91251218c981b4c6ec16c33912 Andrei abetlen@gmail.com 2024-06-29T20:44:08-07:00 GitHub noreply@github.com 2024-06-29T23:44:08-04:00 llama: Add attention and final logit soft-capping, update scaling factor to Gemma2 (#8197) 72272b83a3878e91251218c981b4c6ec16c33912 8748d8ac6f172b99826ab18f01d9a3a165987d54 Xuan Son Nguyen thichthat@gmail.com 2024-06-29T00:14:20+02:00 GitHub noreply@github.com 2024-06-29T00:14:20+02:00 fix code typo in llama-cli (#8198) 8748d8ac6f172b99826ab18f01d9a3a165987d54 26a39bbd6b0bbd66118bb68569f0276d7fe7df6c Olivier Chafik ochafik@users.noreply.github.com 2024-06-28T18:02:05+01:00 GitHub noreply@github.com 2024-06-28T18:02:05+01:00 json: attempt to skip slow tests when running under emulator (#8189) 26a39bbd6b0bbd66118bb68569f0276d7fe7df6c 38373cfbab5397cc2ab5c3694a3dee12a9e58f45 Xuan Son Nguyen thichthat@gmail.com 2024-06-28T15:11:44+02:00 GitHub noreply@github.com 2024-06-28T15:11:44+02:00 Add MiniCPM, Deepseek V2 chat template + clean up `llama_chat_apply_template_internal` (#8172) 38373cfbab5397cc2ab5c3694a3dee12a9e58f45 b851b3fba0a1b06a1129189bac300e07dd1648c8 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-06-28T12:53:43+02:00 GitHub noreply@github.com 2024-06-28T12:53:43+02:00 Add SPM infill support (#8016) b851b3fba0a1b06a1129189bac300e07dd1648c8 139cc621e90b4f61830515c3c124cf35b3d7a6dc slaren slarengh@gmail.com 2024-06-28T12:37:45+02:00 GitHub noreply@github.com 2024-06-28T12:37:45+02:00 cmake : allow user to override default options (#8178) 139cc621e90b4f61830515c3c124cf35b3d7a6dc e57dc62057d41211ac018056c19c02cd544694df Olivier Chafik ochafik@users.noreply.github.com 2024-06-28T09:26:45+01:00 GitHub noreply@github.com 2024-06-28T09:26:45+01:00 `json`: restore default additionalProperties to false, fix some pattern escapes (#8180) e57dc62057d41211ac018056c19c02cd544694df a27aa50ab7e07fe46aae619076b6e31d5663e914 pculliton phillipculliton@gmail.com 2024-06-28T00:00:43-04:00 GitHub noreply@github.com 2024-06-27T21:00:43-07:00 llama: Add support for Gemma2ForCausalLM (#8156) a27aa50ab7e07fe46aae619076b6e31d5663e914 cb0b06a8a613f7a2ccb7253b2a3c00fdd397ba1c Xuan Son Nguyen thichthat@gmail.com 2024-06-28T02:19:11+02:00 GitHub noreply@github.com 2024-06-28T02:19:11+02:00 Add missing items in makefile (#8177) cb0b06a8a613f7a2ccb7253b2a3c00fdd397ba1c 558f44bf83d78242d4e5c4ab98d0be9125cb9780 Olivier Chafik ochafik@users.noreply.github.com 2024-06-27T22:08:42+01:00 GitHub noreply@github.com 2024-06-27T22:08:42+01:00 `json`: update grammars/README w/ examples & note about additionalProperties (#8132) 558f44bf83d78242d4e5c4ab98d0be9125cb9780 8172ee9da9921ca53d698c7438c2d792b3f3f2c8 loonerin 132926317+loonerin@users.noreply.github.com 2024-06-27T15:01:23-04:00 GitHub noreply@github.com 2024-06-27T21:01:23+02:00 CI: fix release build (Ubuntu+Mac) (#8170) 8172ee9da9921ca53d698c7438c2d792b3f3f2c8 16791b8f0b4526aafbf5d0e5bbbd2e99c2253418 slaren slarengh@gmail.com 2024-06-27T20:04:39+02:00 GitHub noreply@github.com 2024-06-27T20:04:39+02:00 cmake : fix deprecated option names not working (#8171) 16791b8f0b4526aafbf5d0e5bbbd2e99c2253418 ab3679112d4c49a215a3d31550a7720b202e9015 Xuan Son Nguyen thichthat@gmail.com 2024-06-27T18:14:19+02:00 GitHub noreply@github.com 2024-06-27T18:14:19+02:00 Add chatml fallback for cpp `llama_chat_apply_template` (#8160) ab3679112d4c49a215a3d31550a7720b202e9015 97877eb10bd8e7f8023420b5b5300bcbdadd62dc Georgi Gerganov ggerganov@gmail.com 2024-06-27T18:37:29+03:00 GitHub noreply@github.com 2024-06-27T08:37:29-07:00 flake.lock: Update (#8071) 97877eb10bd8e7f8023420b5b5300bcbdadd62dc 387952651a8fc493f8c85ea4c9774bd4a5694f87 jukofyork 69222624+jukofyork@users.noreply.github.com 2024-06-27T15:48:07+01:00 GitHub noreply@github.com 2024-06-27T16:48:07+02:00 Control vector loading fixes (#8137) 387952651a8fc493f8c85ea4c9774bd4a5694f87 6030c61281c8a7eb94eceb7396a608fac8b71555 Raj Hammeer Singh Hada hammeerraj@gmail.com 2024-06-27T20:09:29+05:30 GitHub noreply@github.com 2024-06-27T16:39:29+02:00 Delete examples/llama.android/llama/CMakeLists.txt (#8165) 6030c61281c8a7eb94eceb7396a608fac8b71555 85a267daaa1c6f8fd69160445bcb88717031d10c Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-06-27T16:27:41+02:00 GitHub noreply@github.com 2024-06-27T16:27:41+02:00 Add Qwen2MoE 57B-A14B model identifier (#8158) 85a267daaa1c6f8fd69160445bcb88717031d10c f675b20a3b7f878bf3be766b9a737e2c8321ff0d Johannes Gäßler johannesg@5d6.de 2024-06-27T16:26:05+02:00 GitHub noreply@github.com 2024-06-27T16:26:05+02:00 CUDA: fix MMQ stream-k for --split-mode row (#8167) f675b20a3b7f878bf3be766b9a737e2c8321ff0d 911e35bb8bb2fd1c7d3f40f27e96ff432eae7e14 kustaaya 58045274+kustaaya@users.noreply.github.com 2024-06-27T11:58:54+03:00 GitHub noreply@github.com 2024-06-27T10:58:54+02:00 Added support for Viking pre-tokenizer (#8135) 911e35bb8bb2fd1c7d3f40f27e96ff432eae7e14 ac146628e47451c531a3c7e62e6a973a2bb467a0 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-06-27T09:46:41+02:00 GitHub noreply@github.com 2024-06-27T10:46:41+03:00 llama : fix CodeLlama FIM token checks (#8144) ac146628e47451c531a3c7e62e6a973a2bb467a0 9b31a40c6ddabe552875b811d7127aa039ca9703 Raj Hammeer Singh Hada hammeerraj@gmail.com 2024-06-27T07:27:57+05:30 GitHub noreply@github.com 2024-06-27T03:57:57+02:00 Fix llama-android.cpp for error - "common/common.h not found" (#8145) 9b31a40c6ddabe552875b811d7127aa039ca9703 c70d117c37cc7876e775d1e2722208a50c52edb3 Daniel Bevenius daniel.bevenius@gmail.com 2024-06-27T01:50:09+02:00 GitHub noreply@github.com 2024-06-27T01:50:09+02:00 clip : suppress unused variable warnings (#8105) c70d117c37cc7876e775d1e2722208a50c52edb3 ae5d0f4b899ff2842bfca561370c945ad8d4368b Georgi Gerganov ggerganov@gmail.com 2024-06-26T23:25:22+03:00 Georgi Gerganov ggerganov@gmail.com 2024-06-26T23:25:22+03:00 scripts : fix filename sync ae5d0f4b899ff2842bfca561370c945ad8d4368b 31ec3993f6e050322a249c07af79dbde66ea6ddc slaren slarengh@gmail.com 2024-06-26T21:59:28+02:00 GitHub noreply@github.com 2024-06-26T21:59:28+02:00 ci : publish new docker images only when the files change (#8142) 31ec3993f6e050322a249c07af79dbde66ea6ddc c7ab7b612cbdce04499575e713076a026af4b9c5 slaren slarengh@gmail.com 2024-06-26T21:34:14+02:00 GitHub noreply@github.com 2024-06-26T21:34:14+02:00 ggml : add GGML_CUDA_USE_GRAPHS option, restore GGML_CUDA_FORCE_CUBLAS (cmake) (#8140) c7ab7b612cbdce04499575e713076a026af4b9c5 f2d48fffde76d959fdb0da37316bdc09e5518eb1 slaren slarengh@gmail.com 2024-06-26T20:20:22+02:00 GitHub noreply@github.com 2024-06-26T21:20:22+03:00 make : fix missing -O3 (#8143) f2d48fffde76d959fdb0da37316bdc09e5518eb1 4713bf3093d58a3e12368ab2ab5fc3630f27803e Georgi Gerganov ggerganov@gmail.com 2024-06-26T19:39:19+03:00 Georgi Gerganov ggerganov@gmail.com 2024-06-26T19:39:19+03:00 sync : ggml 4713bf3093d58a3e12368ab2ab5fc3630f27803e 0e814dfc42b4b57ad19598d239557b6a977ca16c Georgi Gerganov ggerganov@gmail.com 2024-06-26T19:36:44+03:00 Georgi Gerganov ggerganov@gmail.com 2024-06-26T19:36:44+03:00 authors : regen 0e814dfc42b4b57ad19598d239557b6a977ca16c a95631ee97bb24861af6bdeec380270459631e8e Georgi Gerganov ggerganov@gmail.com 2024-06-26T19:32:07+03:00 GitHub noreply@github.com 2024-06-26T19:32:07+03:00 devops : remove clblast + LLAMA_CUDA -> GGML_CUDA (#8139) a95631ee97bb24861af6bdeec380270459631e8e f3f65429c44bb195a9195bfdc19a30a79709db7b Georgi Gerganov ggerganov@gmail.com 2024-06-26T19:26:13+03:00 GitHub noreply@github.com 2024-06-26T19:26:13+03:00 readme : update API notes f3f65429c44bb195a9195bfdc19a30a79709db7b 88540445615e77a0177fcca43aaa8e9d8eea6864 Georgi Gerganov ggerganov@gmail.com 2024-06-26T18:33:02+03:00 GitHub noreply@github.com 2024-06-26T18:33:02+03:00 llama : reorganize source code + improve CMake (#8006) 88540445615e77a0177fcca43aaa8e9d8eea6864 c8771ab5f89387cdd7d9a8a69280dac46b45e02f Isaac McFadyen isaac@imcf.me 2024-06-26T02:29:28-04:00 GitHub noreply@github.com 2024-06-26T08:29:28+02:00 Clarify default MMQ for CUDA and LLAMA_CUDA_FORCE_MMQ flag (#8115) c8771ab5f89387cdd7d9a8a69280dac46b45e02f 494165f3b6c4cbcd793123cb57fb3e1f5477f1db Johannes Gäßler johannesg@5d6.de 2024-06-26T08:28:02+02:00 GitHub noreply@github.com 2024-06-26T08:28:02+02:00 CUDA: fix misaligned shared memory read (#8123) 494165f3b6c4cbcd793123cb57fb3e1f5477f1db 9b2f16f8055265c67e074025350736adc1ea0666 Eddie-Wang wangjinheng1120@163.com 2024-06-26T14:27:46+08:00 GitHub noreply@github.com 2024-06-26T09:27:46+03:00 llama : extend llm_build_ffn() to support _scale tensors (#8103) 9b2f16f8055265c67e074025350736adc1ea0666 6777c544bdd8c5d9de3220d6e2557957bbbf2a4f Olivier Chafik ochafik@users.noreply.github.com 2024-06-26T01:46:35+01:00 GitHub noreply@github.com 2024-06-26T01:46:35+01:00 `json`: better support for "type" unions (e.g. nullable arrays w/ typed items) (#7863) 6777c544bdd8c5d9de3220d6e2557957bbbf2a4f 163d50adaf8897d8b734d701ff332de6be63d484 Olivier Chafik ochafik@users.noreply.github.com 2024-06-26T01:45:58+01:00 GitHub noreply@github.com 2024-06-26T01:45:58+01:00 `json`: fix additionalProperties, allow space after enum/const (#7840) 163d50adaf8897d8b734d701ff332de6be63d484 6fcbf6823553efabe52ed83e3c2a3329aa3387d1 jukofyork 69222624+jukofyork@users.noreply.github.com 2024-06-25T21:47:40+01:00 GitHub noreply@github.com 2024-06-25T22:47:40+02:00 fixes #7999 (adds control vectors to all `build_XXX()` functions in `llama.cpp` [needs testing] (#8060) 6fcbf6823553efabe52ed83e3c2a3329aa3387d1 e6bf007744eb06336a231ef39cf08146dd16d2ce fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-06-25T21:14:35+02:00 GitHub noreply@github.com 2024-06-25T21:14:35+02:00 llama : implement Unigram tokenizer needed by T5 and FLAN-T5 model families (#5763) e6bf007744eb06336a231ef39cf08146dd16d2ce 84631fe1504de40427dc4b4cdac92fa7ebf65955 Daniel Bevenius daniel.bevenius@gmail.com 2024-06-25T21:07:28+02:00 GitHub noreply@github.com 2024-06-25T15:07:28-04:00 llama : return nullptr from llama_grammar_init (#8093) 84631fe1504de40427dc4b4cdac92fa7ebf65955 dd047b476c8b904e0c25e5dbc5bee6ffde2f6e17 Olivier Chafik ochafik@users.noreply.github.com 2024-06-25T20:06:20+01:00 GitHub noreply@github.com 2024-06-25T20:06:20+01:00 `json`: support integer minimum, maximum, exclusiveMinimum, exclusiveMaximum (#7797) dd047b476c8b904e0c25e5dbc5bee6ffde2f6e17 925c30956dd17723c3a25297bcd0a609aec60663 slaren slarengh@gmail.com 2024-06-25T19:20:06+02:00 GitHub noreply@github.com 2024-06-25T19:20:06+02:00 disable docker CI on pull requests (#8110) 925c30956dd17723c3a25297bcd0a609aec60663 c8ad35955ad2c68db172dcd0e857423ab128518d joecryptotoo 80373433+joecryptotoo@users.noreply.github.com 2024-06-25T08:13:27-07:00 GitHub noreply@github.com 2024-06-25T17:13:27+02:00 Add healthchecks to llama-server containers (#8081) c8ad35955ad2c68db172dcd0e857423ab128518d 49c03c79cda17913b72260acdc8157b742cee41c Brian mofosyne@gmail.com 2024-06-25T22:03:25+10:00 GitHub noreply@github.com 2024-06-25T22:03:25+10:00 Gguf dump start data offset via --data-offset and some extra refactor (#8054) 49c03c79cda17913b72260acdc8157b742cee41c 48e6b92cc378c937e59719f2c0f482bf76c9ca81 Xuan Son Nguyen thichthat@gmail.com 2024-06-25T13:59:54+02:00 GitHub noreply@github.com 2024-06-25T13:59:54+02:00 cvector: better prompt handling, add "mean vector" method (#8069) 48e6b92cc378c937e59719f2c0f482bf76c9ca81 3791ad219323389106dc3fd80814eb5bbb7b80de Xuan Son Nguyen thichthat@gmail.com 2024-06-25T13:56:49+02:00 GitHub noreply@github.com 2024-06-25T21:56:49+10:00 Add chat template support for llama-cli (#8068) 3791ad219323389106dc3fd80814eb5bbb7b80de f702a90e245499283d6de0b287701c723cda2a87 HanishKVC hanishkvc@gmail.com 2024-06-25T16:57:35+05:30 GitHub noreply@github.com 2024-06-25T21:27:35+10:00 SimpleChat v3.1: Boolean chat request options in Settings UI, cache_prompt (#7950) f702a90e245499283d6de0b287701c723cda2a87 083bacce14c1aaf9976aa40e8266cdc25ac749d3 HatsuneMikuUwU33 173229399+HatsuneMikuUwU33@users.noreply.github.com 2024-06-25T10:44:48+02:00 GitHub noreply@github.com 2024-06-25T10:44:48+02:00 Update control vector help (#8104) 083bacce14c1aaf9976aa40e8266cdc25ac749d3 2df373ac40ea581ccca8a58c713f03ad9d4b658d Meng, Hengyu hengyu.meng@intel.com 2024-06-25T10:19:20+08:00 GitHub noreply@github.com 2024-06-25T10:19:20+08:00 [SYCL] Re-enabled mul_mat_batched_sycl (#8095) 2df373ac40ea581ccca8a58c713f03ad9d4b658d 3b099bcd9cbf2434f90cbe40eba6fa2189ed1d02 Johannes Gäßler johannesg@5d6.de 2024-06-25T01:22:33+02:00 GitHub noreply@github.com 2024-06-25T01:22:33+02:00 CUDA: fix matrix multiplication algorithm choice (#8102) 3b099bcd9cbf2434f90cbe40eba6fa2189ed1d02 a818f3028d1497a51cb2b8eb7d993ad58784940e Johannes Gäßler johannesg@5d6.de 2024-06-24T22:15:33+02:00 GitHub noreply@github.com 2024-06-24T22:15:33+02:00 CUDA: fix MMQ writeback for int8 tensor cores (#8100) a818f3028d1497a51cb2b8eb7d993ad58784940e d62e4aaa02540c89be8b59426340b909d02bbc9e Johannes Gäßler johannesg@5d6.de 2024-06-24T17:43:42+02:00 GitHub noreply@github.com 2024-06-24T17:43:42+02:00 CUDA: use MMQ instead of cuBLAS by default (#8075) d62e4aaa02540c89be8b59426340b909d02bbc9e 9a590c82262dd518137f85406e65e452fdf2aca3 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-06-24T14:13:39+02:00 GitHub noreply@github.com 2024-06-24T14:13:39+02:00 gguf-py : fix tensor groups for encoder-decoder models in gguf-dump.py (#8090) 9a590c82262dd518137f85406e65e452fdf2aca3 52fc8705a0617452df08333e1161838726c322b4 Johannes Gäßler johannesg@5d6.de 2024-06-24T12:41:23+02:00 GitHub noreply@github.com 2024-06-24T12:41:23+02:00 CUDA: optimize MMQ int8 tensor core performance (#8062) 52fc8705a0617452df08333e1161838726c322b4 8cb508d0d5c024e12692370d85237b45469a004b Christian Zhou-Zheng 59622928+christianazinn@users.noreply.github.com 2024-06-24T05:42:03-04:00 GitHub noreply@github.com 2024-06-24T19:42:03+10:00 Option to split during conversion (#6942) 8cb508d0d5c024e12692370d85237b45469a004b 646ef4a9cfb6f40236d3b9ef07ac03700b6efcc7 slaren slarengh@gmail.com 2024-06-24T07:36:11+02:00 GitHub noreply@github.com 2024-06-24T08:36:11+03:00 disable publishing the full-rocm docker image (#8083) 646ef4a9cfb6f40236d3b9ef07ac03700b6efcc7 de0d6a68ac99f307fe889c48e21124bc3b7ca29a Yann Follet 131855179+YannFollet@users.noreply.github.com 2024-06-24T13:30:24+08:00 GitHub noreply@github.com 2024-06-24T08:30:24+03:00 embedding : more cli arguments (#7458) de0d6a68ac99f307fe889c48e21124bc3b7ca29a 95f57bb5d5b18ef0beb2702a0d6c06e46804075c fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-06-24T07:06:05+02:00 GitHub noreply@github.com 2024-06-24T07:06:05+02:00 gguf-py, convert-hf : model conversion support for T5 and FLAN-T5 model variants (#5763) 95f57bb5d5b18ef0beb2702a0d6c06e46804075c e112b610a1a75cb7fa8351e1a933e2e7a755a5ce slaren slarengh@gmail.com 2024-06-24T03:07:59+02:00 GitHub noreply@github.com 2024-06-24T03:07:59+02:00 ggml : remove ggml_task_type and GGML_PERF (#8017) e112b610a1a75cb7fa8351e1a933e2e7a755a5ce 6a2f298bd784403c5c33eebb822217ec5d9b5590 Eddie-Wang wangjinheng1120@163.com 2024-06-24T02:27:57+08:00 GitHub noreply@github.com 2024-06-23T21:27:57+03:00 llama : add support for BitnetForCausalLM (#7931) 6a2f298bd784403c5c33eebb822217ec5d9b5590 11318d9aa1f668aa10407a5cb9614371af32f3ce Aarni Koskela akx@iki.fi 2024-06-23T18:03:08+03:00 GitHub noreply@github.com 2024-06-23T11:03:08-04:00 server : fix JSON-Scheme typo (#7975) 11318d9aa1f668aa10407a5cb9614371af32f3ce b6b9a8e606da7764bd3649c2ea574979c85abd43 Daniel Bevenius daniel.bevenius@gmail.com 2024-06-23T15:39:45+02:00 GitHub noreply@github.com 2024-06-23T15:39:45+02:00 Fix typo in llama_set_embeddings comment (#8077) b6b9a8e606da7764bd3649c2ea574979c85abd43 45c0e2e4c1268c2d7c8c45536f15e3c9a731ecdc slaren slarengh@gmail.com 2024-06-23T13:14:45+02:00 GitHub noreply@github.com 2024-06-23T13:14:45+02:00 fix CI failures (#8066) 45c0e2e4c1268c2d7c8c45536f15e3c9a731ecdc b5a5f34efadec8d8f0057b35cb04742abfeb2ef5 0cc4m picard12@live.de 2024-06-23T10:21:25+02:00 GitHub noreply@github.com 2024-06-23T10:21:25+02:00 Refactor Vulkan backend to allow multiple contexts (#7961) b5a5f34efadec8d8f0057b35cb04742abfeb2ef5 3e58b0ee355f78be41cf5211b68426761339bc3c Clint Herron hanclinto@gmail.com 2024-06-22T14:28:18-04:00 GitHub noreply@github.com 2024-06-22T14:28:18-04:00 Removing extra blank lines that were breaking Lint. (#8067) 3e58b0ee355f78be41cf5211b68426761339bc3c adf480c3ab3abedc964c8ae381476257583ae134 Xuan Son Nguyen thichthat@gmail.com 2024-06-22T18:11:30+02:00 GitHub noreply@github.com 2024-06-22T18:11:30+02:00 cvector: fix CI + correct help message (#8064) adf480c3ab3abedc964c8ae381476257583ae134 3aa184a8c7c553b5dfcc142d919f3db695df297a HatsuneMikuUwU33 173229399+HatsuneMikuUwU33@users.noreply.github.com 2024-06-22T17:19:37+02:00 GitHub noreply@github.com 2024-06-22T17:19:37+02:00 cvector-generator: Moe Moe Fixie-Fixie for Lots of Formats~! ♡(ᐢ ᴥ ᐢ)♡ (#8052) 3aa184a8c7c553b5dfcc142d919f3db695df297a 5b48cd53a87928db0c6447f0c9dac4db5802102d 0xspringtime 110655352+0xspringtime@users.noreply.github.com 2024-06-22T09:37:41-04:00 GitHub noreply@github.com 2024-06-22T15:37:41+02:00 convert-hf : change assert to exception (#8015) 5b48cd53a87928db0c6447f0c9dac4db5802102d c5a8d4b749352645afd4c024f85d6eca2ca72c6d ddh0 dylanhalladay02@icloud.com 2024-06-22T07:16:10-06:00 GitHub noreply@github.com 2024-06-22T15:16:10+02:00 Update llama-quantize ppl/file size output from LLaMA-v1 to Llama-3 values (#8058) c5a8d4b749352645afd4c024f85d6eca2ca72c6d 557b653dc9ed91e8c313e87500e0050c775f81b6 Clint Herron hanclinto@gmail.com 2024-06-21T23:18:36-04:00 GitHub noreply@github.com 2024-06-21T23:18:36-04:00 JSON Schema to GBNF integration tests (#7790) 557b653dc9ed91e8c313e87500e0050c775f81b6 7d5e8777ae1d21af99d4f95be10db4870720da91 k.h.lai adrian.k.h.lai@outlook.com 2024-06-21T16:28:20+08:00 GitHub noreply@github.com 2024-06-21T10:28:20+02:00 vulkan: detect multiple devices by deviceUUID instead of deviceID (#8022) 7d5e8777ae1d21af99d4f95be10db4870720da91 a927b0f3dd9a86ee042cd2bdcc8c9da4a855926b Eve 139727413+netrunnereve@users.noreply.github.com 2024-06-21T05:57:36Z GitHub noreply@github.com 2024-06-21T08:57:36+03:00 ggml : AVX IQ quants (#7845) a927b0f3dd9a86ee042cd2bdcc8c9da4a855926b 80ea089d771f0c2d97afa8bead80ded412f600d7 Georgi Gerganov ggerganov@gmail.com 2024-06-21T08:51:28+03:00 GitHub noreply@github.com 2024-06-21T08:51:28+03:00 llama : optimize long word tokenization with WPM (#8034) 80ea089d771f0c2d97afa8bead80ded412f600d7 0e64591e8290037db6412665a56354b789a0597e Douglas Hanley thesecretaryofwar@gmail.com 2024-06-21T00:38:22-05:00 GitHub noreply@github.com 2024-06-21T08:38:22+03:00 llama : allow pooled embeddings on any model (#7477) 0e64591e8290037db6412665a56354b789a0597e b1ef562bc17fbf8ba436ddf2d89b78efd10d3e03 Shuichi Tsutsumi shuichi0526@gmail.com 2024-06-21T14:30:58+09:00 GitHub noreply@github.com 2024-06-21T08:30:58+03:00 swiftui : enable stream updating (#7754) b1ef562bc17fbf8ba436ddf2d89b78efd10d3e03 17b291a6a581c47f24f99bad926b42617894f99f Hamdoud Hakem 90524568+hamdoudhakem@users.noreply.github.com 2024-06-20T21:01:15+01:00 GitHub noreply@github.com 2024-06-20T22:01:15+02:00 requirements : Bump torch and numpy for python3.12 (#8041) 17b291a6a581c47f24f99bad926b42617894f99f abd894ad96a242043b8e197ec130d8649eead22e Hamdoud Hakem 90524568+hamdoudhakem@users.noreply.github.com 2024-06-20T20:59:59+01:00 GitHub noreply@github.com 2024-06-20T21:59:59+02:00 convert-hf : Fix the encoding in the convert-hf-to-gguf-update.py (#8040) abd894ad96a242043b8e197ec130d8649eead22e de391e4c803383bbea054b6edd016e78c024a74d Johannes Gäßler johannesg@5d6.de 2024-06-20T16:40:13+02:00 GitHub noreply@github.com 2024-06-20T16:40:13+02:00 common: fix warning (#8036) de391e4c803383bbea054b6edd016e78c024a74d d50f8897a797a5a03f31228d1b5a7b8130ee1bc2 luoyu-intel yu.luo@intel.com 2024-06-20T13:19:05Z GitHub noreply@github.com 2024-06-20T21:19:05+08:00 [SYCL] Fix windows build and inference (#8003) d50f8897a797a5a03f31228d1b5a7b8130ee1bc2 2075a66a96cc1b04eabec7cf4b3051193d6f719e Johannes Gäßler johannesg@5d6.de 2024-06-20T14:39:21+02:00 GitHub noreply@github.com 2024-06-20T14:39:21+02:00 CUDA: stream-k decomposition for MMQ (#8018) 2075a66a96cc1b04eabec7cf4b3051193d6f719e ba5899315283eb1df3902363daf79bdc5eefe426 Michael de Gans michael.john.degans@gmail.com 2024-06-19T22:32:01-07:00 GitHub noreply@github.com 2024-06-20T08:32:01+03:00 metal : fix `ggml_metal_supports_op` for BF16 (#8021) ba5899315283eb1df3902363daf79bdc5eefe426 a7854743c5e6216a6178824a603aa9479728ddd5 sasha0552 admin@sasha0552.org 2024-06-19T23:57:10Z GitHub noreply@github.com 2024-06-20T09:57:10+10:00 server : fix smart slot selection (#8020) a7854743c5e6216a6178824a603aa9479728ddd5 9c77ec1d74874ee22bdef8f110e8e8d41389abf2 Michael de Gans michael.john.degans@gmail.com 2024-06-19T13:10:42-07:00 GitHub noreply@github.com 2024-06-19T22:10:42+02:00 un-ignore `build-info.cmake` and `build-info.sh` (#7996) 9c77ec1d74874ee22bdef8f110e8e8d41389abf2 a04a953cab583f0229e7b4b506346e3e9a85c8d0 slaren slarengh@gmail.com 2024-06-19T15:04:15+02:00 GitHub noreply@github.com 2024-06-19T15:04:15+02:00 ggml : synchronize threads using barriers (#7993) a04a953cab583f0229e7b4b506346e3e9a85c8d0 623494a478134432fd2d7ee40135770a3340674f Georgi Gerganov ggerganov@gmail.com 2024-06-19T13:04:36+03:00 GitHub noreply@github.com 2024-06-19T13:04:36+03:00 codecov : remove (#8004) 623494a478134432fd2d7ee40135770a3340674f 37bef8943312d91183ff06d8f1214082a17344a5 Meng, Hengyu hengyu.meng@intel.com 2024-06-19T09:11:51+08:00 GitHub noreply@github.com 2024-06-19T09:11:51+08:00 [SYCL] refactor (#6408) 37bef8943312d91183ff06d8f1214082a17344a5 91c188d6c296bd3384f2a02a83b71187aa3d18b3 jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-06-18T18:40:52+02:00 GitHub noreply@github.com 2024-06-18T18:40:52+02:00 tokenizer : BPE fixes (#7530) 91c188d6c296bd3384f2a02a83b71187aa3d18b3 84f6de17f6a8602e7ff7f7c7bda36a73f510a2dd Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-06-18T14:19:45+02:00 GitHub noreply@github.com 2024-06-18T22:19:45+10:00 Only use FIM middle token if it exists (#7648) 84f6de17f6a8602e7ff7f7c7bda36a73f510a2dd 61665277afde2add00c0d387acb94ed5feb95917 jojorne jojorne@users.noreply.github.com 2024-06-18T09:18:32-03:00 GitHub noreply@github.com 2024-06-18T22:18:32+10:00 Fix no gcc pragma on Windows (#7751) 61665277afde2add00c0d387acb94ed5feb95917 b96f9afb0d58b003ac8d1d0c94cd99393a3bc437 Ulrich Drepper drepper@gmail.com 2024-06-18T14:00:14+02:00 GitHub noreply@github.com 2024-06-18T14:00:14+02:00 Allow compiling with CUDA without CUDA runtime installed (#7989) b96f9afb0d58b003ac8d1d0c94cd99393a3bc437 1193778105c9a81bd38f72c61aaafbaf85dc9c04 Frank Mai thxcode0824@gmail.com 2024-06-18T15:11:40+08:00 GitHub noreply@github.com 2024-06-18T10:11:40+03:00 chore: clean useless beam search param (#7985) 1193778105c9a81bd38f72c61aaafbaf85dc9c04 5326bcceeb7dd34f16d0fe61b134d1e074a8e65d Abheek Gulati abheekg@hotmail.com 2024-06-17T23:57:41-07:00 GitHub noreply@github.com 2024-06-18T09:57:41+03:00 readme : update UI list (#7943) 5326bcceeb7dd34f16d0fe61b134d1e074a8e65d e6ecc2be470e3c5c6c5c9d8b90aa83a1f7725084 Georgi Gerganov ggerganov@gmail.com 2024-06-18T09:50:45+03:00 Georgi Gerganov ggerganov@gmail.com 2024-06-18T09:50:45+03:00 ggml : sync e6ecc2be470e3c5c6c5c9d8b90aa83a1f7725084 a94e6ff8774b7c9f950d9545baf0ce35e8d1ed2f Georgi Gerganov ggerganov@gmail.com 2024-06-18T09:37:20+03:00 Georgi Gerganov ggerganov@gmail.com 2024-06-18T09:50:40+03:00 whisper : use ggml_backend_sched (whisper/2239) a94e6ff8774b7c9f950d9545baf0ce35e8d1ed2f 5b6da187508f49a9fa9d95fa22ae804a0780d256 Ștefan-Gabriel Muscalu legraphista@users.noreply.github.com 2024-06-17T22:08:46+03:00 GitHub noreply@github.com 2024-06-17T21:08:46+02:00 update: support Qwen2-57B-A14B (#7835) 5b6da187508f49a9fa9d95fa22ae804a0780d256 7c26775adb579e92b59c82e8084c07a1d0f75e9c Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-06-17T23:53:17+05:30 GitHub noreply@github.com 2024-06-17T20:23:17+02:00 Make updates to type cast based on compiler instead of OS (#7851) 7c26775adb579e92b59c82e8084c07a1d0f75e9c b473e95084c286780165568cf0f385f21141d68d Georgi Gerganov ggerganov@gmail.com 2024-06-17T19:40:01+03:00 GitHub noreply@github.com 2024-06-17T19:40:01+03:00 llama : disable FA if KV head size do not match (#7982) b473e95084c286780165568cf0f385f21141d68d 99052cd227c7182fcf53343d2e7d33bfa180a9cf Bryan Honof bryanhonof@gmail.com 2024-06-17T17:37:55+02:00 GitHub noreply@github.com 2024-06-17T09:37:55-06:00 Add Nix and Flox install instructions (#7899) 99052cd227c7182fcf53343d2e7d33bfa180a9cf c637fcd34d135a9ff4f97d3a53ad03a910a4a31f slaren slarengh@gmail.com 2024-06-17T16:51:42+02:00 GitHub noreply@github.com 2024-06-17T16:51:42+02:00 sched : offload_op also requires supports_op (#7977) c637fcd34d135a9ff4f97d3a53ad03a910a4a31f 6a2f0b3474d479bda4ac2ee7cfd5dcdcf0be1f79 Frank Mai thxcode0824@gmail.com 2024-06-17T22:11:08+08:00 GitHub noreply@github.com 2024-06-17T16:11:08+02:00 fix: divide 0 exception in mamba (#7932) 6a2f0b3474d479bda4ac2ee7cfd5dcdcf0be1f79 21be9cab94e0b5b53cb6edeeebf8c8c799baad03 Markus Tavenrath mtavenrath@users.noreply.github.com 2024-06-17T16:10:15+02:00 GitHub noreply@github.com 2024-06-17T16:10:15+02:00 Implement non-mapped async IO for CUDA on Windows. (#7896) 21be9cab94e0b5b53cb6edeeebf8c8c799baad03 006167aaf6b6aaa4daa52961035f7460af19f469 Georgi Gerganov ggerganov@gmail.com 2024-06-17T11:09:20+03:00 GitHub noreply@github.com 2024-06-17T11:09:20+03:00 rpc : fix load/store misaligned addresses (#7948) 006167aaf6b6aaa4daa52961035f7460af19f469 df68d4fa5dc929217d3e64d673e099d7a417b206 Brian mofosyne@gmail.com 2024-06-17T15:25:20+10:00 GitHub noreply@github.com 2024-06-17T15:25:20+10:00 gguf-dump.py: add --markdown dump output (#7853) df68d4fa5dc929217d3e64d673e099d7a417b206 43b35e38ba371f9a7faa6dca4c5d1e8f698ffd87 Neo Zhang zhang.jianyu@outlook.com 2024-06-17T11:17:07+08:00 GitHub noreply@github.com 2024-06-17T11:17:07+08:00 [SYCL] Update README-sycl.md for Chapter "Recommended release" and "News" (#7946) 43b35e38ba371f9a7faa6dca4c5d1e8f698ffd87 19b7a836f6658e18e973af532a5cc6ad6b3a27f8 Calvin Laurenson calvin@laurenson.dev 2024-06-16T15:23:04-07:00 GitHub noreply@github.com 2024-06-17T00:23:04+02:00 Add support for sqrt on CUDA (#7953) 19b7a836f6658e18e973af532a5cc6ad6b3a27f8 b5fcf8ef5c29df53cfff60e180b4992a3b2332a6 Georgi Gerganov ggerganov@gmail.com 2024-06-11T17:39:01+03:00 Georgi Gerganov ggerganov@gmail.com 2024-06-16T20:32:49+03:00 cuda : fix bounds check for src0 rows in MMVQ kernel (whisper/2231) b5fcf8ef5c29df53cfff60e180b4992a3b2332a6 398105ff4373eea385ea8e8625cb417b2ae51134 Hong Bo PENG penghb@cn.ibm.com 2024-06-16T16:53:11+08:00 Georgi Gerganov ggerganov@gmail.com 2024-06-16T20:32:49+03:00 ggml : fix and optimize ppc64le (ggml/849) 398105ff4373eea385ea8e8625cb417b2ae51134 bc6c457fa35f6791e9a2bb61108e7d49e8fc98bd Daniel Bevenius daniel.bevenius@gmail.com 2024-06-16T10:51:18+02:00 Georgi Gerganov ggerganov@gmail.com 2024-06-16T20:32:49+03:00 ggml : remove duplicate include of ggml-common.h (ggml/853) bc6c457fa35f6791e9a2bb61108e7d49e8fc98bd 52399254b3bceda279b4ea9111a983e32310166e Georgi Gerganov ggerganov@gmail.com 2024-06-16T19:16:21+03:00 GitHub noreply@github.com 2024-06-16T09:16:21-07:00 flake.lock: Update (#7951) 52399254b3bceda279b4ea9111a983e32310166e 6fe1c627413725ddc1f9e323f6b13fe388c53e0a Georgi Gerganov ggerganov@gmail.com 2024-06-16T14:51:40+03:00 GitHub noreply@github.com 2024-06-16T14:51:40+03:00 unicode : avoid char32_t (#7957) 6fe1c627413725ddc1f9e323f6b13fe388c53e0a cddaf028adc738b5a7ecc60809cb78e0ba0f97c1 hopkins385 98618192+hopkins385@users.noreply.github.com 2024-06-16T13:51:18+02:00 GitHub noreply@github.com 2024-06-16T14:51:18+03:00 readme : update UI list [no ci] (#7958) cddaf028adc738b5a7ecc60809cb78e0ba0f97c1 c8a82194a888f68f259e0d0fa96f3332ac7c5cb6 Georgi Gerganov ggerganov@gmail.com 2024-06-16T14:50:12+03:00 GitHub noreply@github.com 2024-06-16T14:50:12+03:00 ggml : fix handling of zero blocks in IQ quants (#7955) c8a82194a888f68f259e0d0fa96f3332ac7c5cb6 7c7836d9d4062d6858e3fb337b135c417ccee6ce Georgi Gerganov ggerganov@gmail.com 2024-06-16T10:46:51+03:00 Georgi Gerganov ggerganov@gmail.com 2024-06-16T10:46:51+03:00 github : update pr template 7c7836d9d4062d6858e3fb337b135c417ccee6ce 0c7b3595b9e5ad2355818e259f06b0dc3f0065b3 0cc4m picard12@live.de 2024-06-16T07:17:31+02:00 GitHub noreply@github.com 2024-06-16T07:17:31+02:00 Vulkan Shader Refactor, Memory Debugging Option (#7947) 0c7b3595b9e5ad2355818e259f06b0dc3f0065b3 7b2f4a7d193ef2475259bbe7656fcccfab4b1217 Xuan Son Nguyen thichthat@gmail.com 2024-06-15T18:53:40+02:00 GitHub noreply@github.com 2024-06-15T18:53:40+02:00 Add `cvector-generator` example (#7514) 7b2f4a7d193ef2475259bbe7656fcccfab4b1217 f8ec8877b75774fc6c47559d529dac423877bcad Meng, Hengyu hengyu.meng@intel.com 2024-06-15T14:05:10+08:00 GitHub noreply@github.com 2024-06-15T14:05:10+08:00 [SYCL] remove global variables (#7710) f8ec8877b75774fc6c47559d529dac423877bcad 76d66ee0be91e2bec93206e821ee1db8d023cff5 olexiyb olexiyb@gmail.com 2024-06-14T20:28:34+03:00 GitHub noreply@github.com 2024-06-14T20:28:34+03:00 ci : fix macos x86 build (#7940) 76d66ee0be91e2bec93206e821ee1db8d023cff5 66ef1ceedf983773c8ceb4d925285d41d4e50e2a Johannes Gäßler johannesg@5d6.de 2024-06-14T18:41:49+02:00 GitHub noreply@github.com 2024-06-14T18:41:49+02:00 CUDA: faster q2_K, q3_K MMQ + int8 tensor cores (#7921) 66ef1ceedf983773c8ceb4d925285d41d4e50e2a e65bbf606c61f49dc06c7ac060cd5ba7ae446025 Georgi Gerganov ggerganov@gmail.com 2024-06-14T17:14:09+03:00 GitHub noreply@github.com 2024-06-14T17:14:09+03:00 metal : utilize max shared memory for mul_mat_id (#7935) e65bbf606c61f49dc06c7ac060cd5ba7ae446025 6fcd1331efbfbb89c8c96eba2321bb7b4d0c40e4 Radoslav Gerganov rgerganov@gmail.com 2024-06-14T16:47:41+03:00 GitHub noreply@github.com 2024-06-14T16:47:41+03:00 llama-bench : fix RPC indication (#7936) 6fcd1331efbfbb89c8c96eba2321bb7b4d0c40e4 41b9260f18eb7f325c952006ac46afc1d0d8ad2f Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-06-14T12:20:04+02:00 GitHub noreply@github.com 2024-06-14T13:20:04+03:00 llama : more checks before assuming FIM tokens (#7644) 41b9260f18eb7f325c952006ac46afc1d0d8ad2f 172c8256840ffd882ab9992ecedbb587d9b21f15 Elaine elaine.zosa@gmail.com 2024-06-14T13:16:49+03:00 GitHub noreply@github.com 2024-06-14T13:16:49+03:00 convert : add Poro-34B-chat tokenizer support (#7713) 172c8256840ffd882ab9992ecedbb587d9b21f15 a55eb1bf0fa2fd84147bdfd384391e029d988253 Radoslav Gerganov rgerganov@gmail.com 2024-06-13T15:18:44+03:00 GitHub noreply@github.com 2024-06-13T15:18:44+03:00 rpc : fix ggml_backend_rpc_supports_buft() (#7918) a55eb1bf0fa2fd84147bdfd384391e029d988253 f578b86b2123d0f92afbaa98a031df4d4464e582 Galunid karolek1231456@gmail.com 2024-06-13T09:42:41+02:00 GitHub noreply@github.com 2024-06-13T09:42:41+02:00 readme : Remove outdated instructions from README.md (#7914) [no ci] f578b86b2123d0f92afbaa98a031df4d4464e582 1c641e6aac5c18b964e7b32d9dbbb4bf5301d0d7 slaren slarengh@gmail.com 2024-06-13T03:11:35+02:00 GitHub noreply@github.com 2024-06-13T03:11:35+02:00 move BLAS to a separate backend (#6210) 1c641e6aac5c18b964e7b32d9dbbb4bf5301d0d7 963552903f51043ee947a8deeaaa7ec00bc3f1a4 Olivier Chafik ochafik@users.noreply.github.com 2024-06-13T00:41:52+01:00 GitHub noreply@github.com 2024-06-13T00:41:52+01:00 `build`: rename main → llama-cli, server → llama-server, llava-cli → llama-llava-cli, etc... (#7809) 963552903f51043ee947a8deeaaa7ec00bc3f1a4 a9cae48003dfc4fe95b8f5c81682fc6e63425235 Johannes Gäßler johannesg@5d6.de 2024-06-12T17:41:51+02:00 GitHub noreply@github.com 2024-06-12T17:41:51+02:00 CUDA: fix broken oob check for FA vec f32 kernel (#7904) a9cae48003dfc4fe95b8f5c81682fc6e63425235 bfaa676b0841617d4ef3596e63aca6be1a8eb1b5 Georgi Gerganov ggerganov@gmail.com 2024-06-12T16:00:22+03:00 GitHub noreply@github.com 2024-06-12T16:00:22+03:00 tests : add non-cont unary tests (#7857) bfaa676b0841617d4ef3596e63aca6be1a8eb1b5 704a35b183748954013bd875bbbfdd9eaca14e62 Georgi Gerganov ggerganov@gmail.com 2024-06-12T15:24:20+03:00 GitHub noreply@github.com 2024-06-12T15:24:20+03:00 ggml : improve ggml_is_contiguous logic (#7856) 704a35b183748954013bd875bbbfdd9eaca14e62 dcf752707d96eb305f546526c7bc5d01f0831130 Georgi Gerganov ggerganov@gmail.com 2024-06-12T14:42:29+03:00 GitHub noreply@github.com 2024-06-12T14:42:29+03:00 server : restore numeric prompts (#7883) dcf752707d96eb305f546526c7bc5d01f0831130 f2b5764beb35583295e2475479c18f249b139b58 Meng, Hengyu hengyu.meng@intel.com 2024-06-12T17:05:35+08:00 GitHub noreply@github.com 2024-06-12T19:05:35+10:00 update intel docker oneapi-basekit to 2024.1.1-devel-ubuntu22.04 (#7894) f2b5764beb35583295e2475479c18f249b139b58 73bac2b11d7d3e20982fc9ee607625836387db8b Patrice Ferlet metal3d@gmail.com 2024-06-12T03:18:16+02:00 GitHub noreply@github.com 2024-06-12T11:18:16+10:00 Fix a typo and add Fedora 40 pacakge to install for Vulkan (#7794) [no ci] 73bac2b11d7d3e20982fc9ee607625836387db8b ef52d1d16afc695d798396cdd13594ea5e45a9dd k.h.lai adrian.k.h.lai@outlook.com 2024-06-12T03:26:05+08:00 GitHub noreply@github.com 2024-06-11T21:26:05+02:00 vulkan: select only one device for single gpu with multiple drivers (#7582) ef52d1d16afc695d798396cdd13594ea5e45a9dd 14f83526cd27f638c856ea6eff08110b9860eb2a 0cc4m picard12@live.de 2024-06-11T21:20:29+02:00 GitHub noreply@github.com 2024-06-11T21:20:29+02:00 Update Vulkan RoPE implementation (#7818) 14f83526cd27f638c856ea6eff08110b9860eb2a 6fe42d073f0554eada93ac9d40574025aeedb703 Deven Mistry 31466137+deven367@users.noreply.github.com 2024-06-11T12:18:58-04:00 GitHub noreply@github.com 2024-06-12T02:18:58+10:00 fix broken link in pr template (#7880) [no ci] 6fe42d073f0554eada93ac9d40574025aeedb703 148995e5e57b313cce2672f75610db58c6327a51 Brian mofosyne@gmail.com 2024-06-12T00:43:41+10:00 GitHub noreply@github.com 2024-06-11T17:43:41+03:00 github: move PR template to .github/ root (#7868) 148995e5e57b313cce2672f75610db58c6327a51 4bfe50f741479c1df1c377260c3ff5702586719e Johannes Gäßler johannesg@5d6.de 2024-06-11T14:45:40+02:00 GitHub noreply@github.com 2024-06-11T14:45:40+02:00 llama-bench: more compact markdown tables (#7879) 4bfe50f741479c1df1c377260c3ff5702586719e bdcb8f42221bc40c411150a009a3d3a30fa74722 Georgi Gerganov ggerganov@gmail.com 2024-06-11T10:10:20+03:00 GitHub noreply@github.com 2024-06-11T10:10:20+03:00 tests : check the Python version (#7872) bdcb8f42221bc40c411150a009a3d3a30fa74722 c2ce6c47e4f2d891bf29d8810832a3b310a8f205 Johannes Gäßler johannesg@5d6.de 2024-06-11T08:26:07+02:00 GitHub noreply@github.com 2024-06-11T08:26:07+02:00 CUDA: int8 tensor cores for MMQ (q4_K, q5_K, q6_K) (#7860) c2ce6c47e4f2d891bf29d8810832a3b310a8f205 b61eb9644d64e90123ac805436d95b94b3b4cc3f slaren slarengh@gmail.com 2024-06-11T07:59:20+02:00 GitHub noreply@github.com 2024-06-11T08:59:20+03:00 fix CUDA CI by using a windows-2019 image (#7861) b61eb9644d64e90123ac805436d95b94b3b4cc3f 396b18dfec2c56846e80362db70af09b9e1d70ba Olivier Chafik ochafik@users.noreply.github.com 2024-06-11T02:22:57+01:00 GitHub noreply@github.com 2024-06-11T02:22:57+01:00 json: refine constraint for whitespace to avoid runaways yet allow pretty print (#7866) 396b18dfec2c56846e80362db70af09b9e1d70ba 864a99e7a01d9422d2f55618dbe62c8099a2175c Olivier Chafik ochafik@users.noreply.github.com 2024-06-11T01:00:30+01:00 GitHub noreply@github.com 2024-06-11T01:00:30+01:00 `json`: document schema conversion in GBNF readme, align manual grammar examples & converters (#7841) 864a99e7a01d9422d2f55618dbe62c8099a2175c fd5ea0f897ecb3659d6c269ef6f3d833e865ead7 Jared Van Bortel jared@nomic.ai 2024-06-10T18:32:10-04:00 GitHub noreply@github.com 2024-06-10T18:32:10-04:00 cmake : fix CMake requirement for CUDA (#7821) fd5ea0f897ecb3659d6c269ef6f3d833e865ead7 c28a83902cf6ab6a9e085ad6d4cc2e95c4ccfe40 slaren slarengh@gmail.com 2024-06-10T14:18:41+02:00 GitHub noreply@github.com 2024-06-10T15:18:41+03:00 ci : try win-2019 on server windows test (#7854) c28a83902cf6ab6a9e085ad6d4cc2e95c4ccfe40 d9da0e4986f121c727bdd9579a6688097b11602c Georgi Gerganov ggerganov@gmail.com 2024-06-10T15:00:15+03:00 GitHub noreply@github.com 2024-06-10T15:00:15+03:00 examples : remove --instruct remnants (#7846) d9da0e4986f121c727bdd9579a6688097b11602c 1f0dabda8d5c131f9d4632aa41de74317cdd61fb Georgi Gerganov ggerganov@gmail.com 2024-06-10T14:59:55+03:00 GitHub noreply@github.com 2024-06-10T14:59:55+03:00 server : improve "prompt" handling (#7847) 1f0dabda8d5c131f9d4632aa41de74317cdd61fb af4ae502ddaeb03cd5861273ca2e9a5ae4551db7 Johannes Gäßler johannesg@5d6.de 2024-06-10T11:45:13+02:00 GitHub noreply@github.com 2024-06-10T11:45:13+02:00 CUDA: use tensor cores for MMQ (#7676) af4ae502ddaeb03cd5861273ca2e9a5ae4551db7 10ceba354a3b152ff425e9fa97f9caaef99a46b1 Ben Ashbaugh ben.ashbaugh@intel.com 2024-06-10T02:21:31-07:00 GitHub noreply@github.com 2024-06-10T10:21:31+01:00 use the correct SYCL context for host USM allocations (#7777) 10ceba354a3b152ff425e9fa97f9caaef99a46b1 e95beeb1fc4621826ddd616776dbdf717366bf5c Georgi Gerganov ggerganov@gmail.com 2024-06-10T02:04:50+03:00 GitHub noreply@github.com 2024-06-09T16:04:50-07:00 flake.lock: Update (#7838) e95beeb1fc4621826ddd616776dbdf717366bf5c 57bf62ce7cb75cca589943e2050d29bff4026e76 Georgi Gerganov ggerganov@gmail.com 2024-06-09T20:19:35+03:00 GitHub noreply@github.com 2024-06-09T20:19:35+03:00 imatrix : handle partial entries (#7833) 57bf62ce7cb75cca589943e2050d29bff4026e76 3e2ee443159724e2d3a0741f6b167e599ec088aa Nicolás Pérez nicolas_perez@brown.edu 2024-06-09T11:24:29-04:00 GitHub noreply@github.com 2024-06-10T01:24:29+10:00 docs: Added initial PR template with directions for doc only changes and squash merges [no ci] (#7700) 3e2ee443159724e2d3a0741f6b167e599ec088aa 42b53d192f4e3abf1b7c8e424628424504ea5dc5 mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2024-06-09T12:50:35+02:00 GitHub noreply@github.com 2024-06-09T20:50:35+10:00 server: do not remove whitespace at the start of a completion chunk (#7830) 42b53d192f4e3abf1b7c8e424628424504ea5dc5 2decf57bc6e4a6b45176c3727d964a01161beecc Johannes Gäßler johannesg@5d6.de 2024-06-09T09:42:25+02:00 GitHub noreply@github.com 2024-06-09T09:42:25+02:00 CUDA: revise q8_1 data layout for mul_mat_q (#7824) 2decf57bc6e4a6b45176c3727d964a01161beecc 5795b941827fdec6c1662986de962badff456718 sasha0552 admin@sasha0552.org 2024-06-09T06:39:25Z GitHub noreply@github.com 2024-06-09T16:39:25+10:00 convert-hf : set the model name based on cli arg, if present (#7693) 5795b941827fdec6c1662986de962badff456718 ed9f2521185706481501a5e6d5315397b11802ff compilade git@compilade.net 2024-06-08T22:47:25-04:00 GitHub noreply@github.com 2024-06-09T12:47:25+10:00 convert-hf : match model part name prefix and suffix (#7687) ed9f2521185706481501a5e6d5315397b11802ff fe1e3917cfa0f9397a765cfd0aef880674d938d5 compilade git@compilade.net 2024-06-08T22:34:29-04:00 GitHub noreply@github.com 2024-06-09T12:34:29+10:00 gguf-py : decouple adding metadata from writing in GGUFWriter (#7827) fe1e3917cfa0f9397a765cfd0aef880674d938d5 d4d915d351d1f1270d56184bdd46672893e8a5d8 slaren slarengh@gmail.com 2024-06-09T01:43:39+02:00 GitHub noreply@github.com 2024-06-09T01:43:39+02:00 Revert "[SYCL] Update rpc-server.cpp to include SYCL backend (#7682)" (#7808) d4d915d351d1f1270d56184bdd46672893e8a5d8 7a16ce7db2a74a223f0f3b9cee66d4539c5bce8f Olivier Chafik ochafik@users.noreply.github.com 2024-06-08T20:21:08+01:00 GitHub noreply@github.com 2024-06-08T21:21:08+02:00 url: save -mu downloads to new cache location (#7826) 7a16ce7db2a74a223f0f3b9cee66d4539c5bce8f da799b41891e34aac86ce4e173f9c4c0afd4fab3 sasha0552 admin@sasha0552.org 2024-06-08T07:50:31Z GitHub noreply@github.com 2024-06-08T10:50:31+03:00 server : smart slot selection using Longest Common Prefix (#7728) da799b41891e34aac86ce4e173f9c4c0afd4fab3 c00fad71e507ff386d42bd74846fe06d19dd63a4 slaren slarengh@gmail.com 2024-06-07T19:47:49+02:00 GitHub noreply@github.com 2024-06-07T19:47:49+02:00 vulkan : reuse parent extra for views (#7806) c00fad71e507ff386d42bd74846fe06d19dd63a4 27615f5ab21060d96953c9c1e223051ab2188f57 Christian Zhou-Zheng 59622928+christianazinn@users.noreply.github.com 2024-06-07T08:56:01-04:00 GitHub noreply@github.com 2024-06-07T15:56:01+03:00 gguf-split : change binary multi-byte units to decimal (#7803) 27615f5ab21060d96953c9c1e223051ab2188f57 7027b27d765db95d4ac6b569d976e387a8715881 intelmatt 61025942+intelmatt@users.noreply.github.com 2024-06-07T05:15:07-07:00 GitHub noreply@github.com 2024-06-07T15:15:07+03:00 cmake : fix BUILD_SHARED_LIBS=ON build (#7784) 7027b27d765db95d4ac6b569d976e387a8715881 a5cabd76491f07494c5b8267f921c73f5e2bbfb4 Johannes Gäßler johannesg@5d6.de 2024-06-07T11:15:49+02:00 GitHub noreply@github.com 2024-06-07T11:15:49+02:00 server: update cache_prompt documentation [no ci] (#7745) a5cabd76491f07494c5b8267f921c73f5e2bbfb4 d5c938cd7716b9a2ace49a43a469dfbffcff4d28 woodx 124784234+woodx9@users.noreply.github.com 2024-06-07T15:09:45+08:00 GitHub noreply@github.com 2024-06-07T10:09:45+03:00 server : do not get prompt in infill mode (#7286) d5c938cd7716b9a2ace49a43a469dfbffcff4d28 c9ee7118d5644dd3df70ea6878b36a9761616aab pengxin99 pengxin.yuan@intel.com 2024-06-07T14:28:26+08:00 GitHub noreply@github.com 2024-06-07T14:28:26+08:00 [SYCL] fix softmax r2r result wrong issue (#7811) c9ee7118d5644dd3df70ea6878b36a9761616aab ee459f40f65810a810151b24eba5b8bd174ceffe slaren slarengh@gmail.com 2024-06-07T08:01:29+02:00 GitHub noreply@github.com 2024-06-07T09:01:29+03:00 check for nans in imatrix and quantize (#7807) ee459f40f65810a810151b24eba5b8bd174ceffe f83351f9a62a6262f1fc3d08f320033089cddfb5 Georgi Gerganov ggerganov@gmail.com 2024-06-06T19:19:59+03:00 GitHub noreply@github.com 2024-06-06T19:19:59+03:00 server : fix --threads-http arg (#7801) f83351f9a62a6262f1fc3d08f320033089cddfb5 ad675e1c67a05b16e4e12abe30dbecfc808e7b7e Georgi Gerganov ggerganov@gmail.com 2024-06-06T16:30:58+03:00 GitHub noreply@github.com 2024-06-06T16:30:58+03:00 imatrix : migrate to gpt_params (#7771) ad675e1c67a05b16e4e12abe30dbecfc808e7b7e a143c04375828b1f72eb1a326115791b63e79345 Clint Herron hanclinto@gmail.com 2024-06-06T06:08:52-07:00 GitHub noreply@github.com 2024-06-06T06:08:52-07:00 Added support for . (any character) token in grammar engine. (#6467) a143c04375828b1f72eb1a326115791b63e79345 55b2d0849d3ec9e45e4a4d9e480f5fa7977872a6 Mattheus Chediak shammcity00@gmail.com 2024-06-06T09:17:54-03:00 GitHub noreply@github.com 2024-06-06T22:17:54+10:00 README minor fixes (#7798) [no ci] 55b2d0849d3ec9e45e4a4d9e480f5fa7977872a6 f5d7b268ec4bf8628aa6ccc9f6631d0230dde76f Olivier Chafik ochafik@users.noreply.github.com 2024-06-06T10:07:06+01:00 GitHub noreply@github.com 2024-06-06T10:07:06+01:00 grammars: x{min,max} repetition operator (#6640) f5d7b268ec4bf8628aa6ccc9f6631d0230dde76f 2d08b7fbb483c14bd2b173d4cd51ea3a4f862e8f Joan Fontanals joan.fontanals.martinez@jina.ai 2024-06-06T09:22:41+02:00 GitHub noreply@github.com 2024-06-06T10:22:41+03:00 llama : add jina v2 base code (#7596) 2d08b7fbb483c14bd2b173d4cd51ea3a4f862e8f d67caea0d6e6c303d31b01d0a010973e6c908dff slaren slarengh@gmail.com 2024-06-06T07:19:49+02:00 GitHub noreply@github.com 2024-06-06T08:19:49+03:00 docker : build only main and server in their images (#7782) d67caea0d6e6c303d31b01d0a010973e6c908dff 7672adeec7a79ea271058c63106c142ba84f951a slaren slarengh@gmail.com 2024-06-06T07:17:21+02:00 GitHub noreply@github.com 2024-06-06T08:17:21+03:00 docker : add openmp lib (#7780) 7672adeec7a79ea271058c63106c142ba84f951a 7d1a378b8fb266782d9248538a661405aad80768 Galunid karolek1231456@gmail.com 2024-06-05T19:07:24+02:00 GitHub noreply@github.com 2024-06-06T03:07:24+10:00 Fix encoding in python scripts (#7733) 7d1a378b8fb266782d9248538a661405aad80768 2b3389677a833cee0880226533a1768b1a9508d2 Johannes Gäßler johannesg@5d6.de 2024-06-05T16:53:00+02:00 GitHub noreply@github.com 2024-06-05T16:53:00+02:00 CUDA: refactor mmq, dmmv, mmvq (#7716) 2b3389677a833cee0880226533a1768b1a9508d2 9973e81c5ccf4f31b3980f5aa73f5cfea8699860 Georgi Gerganov ggerganov@gmail.com 2024-06-05T11:29:20+03:00 GitHub noreply@github.com 2024-06-05T11:29:20+03:00 ggml : refactor rope norm/neox (#7634) 9973e81c5ccf4f31b3980f5aa73f5cfea8699860 c90dbe026b456a233f8f0fbe752212e6a0503ca2 arch-btw 57669023+arch-btw@users.noreply.github.com 2024-06-04T23:40:49-07:00 GitHub noreply@github.com 2024-06-05T09:40:49+03:00 readme : remove -ins (#7759) c90dbe026b456a233f8f0fbe752212e6a0503ca2 b90dc566c1c615289b05b50d61680f23744a21e7 jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-06-05T01:26:14+02:00 GitHub noreply@github.com 2024-06-05T01:26:14+02:00 Fix per token atrributes bits (#7749) b90dc566c1c615289b05b50d61680f23744a21e7 1442677f92e45a475be7b4d056e3633d1d6f813b agray3 agray3@users.noreply.github.com 2024-06-04T21:06:49+01:00 GitHub noreply@github.com 2024-06-04T22:06:49+02:00 Allow number of nodes in CUDA graph to change (#7738) 1442677f92e45a475be7b4d056e3633d1d6f813b 554c247caffed64465f372661f2826640cb10430 Georgi Gerganov ggerganov@gmail.com 2024-06-04T21:23:39+03:00 GitHub noreply@github.com 2024-06-04T21:23:39+03:00 common : refactor cli arg parsing (#7675) 554c247caffed64465f372661f2826640cb10430 0cd6bd3483fa66124b76a8a8ac794d9ee18c70c1 Georgi Gerganov ggerganov@gmail.com 2024-06-04T21:23:20+03:00 GitHub noreply@github.com 2024-06-04T21:23:20+03:00 ggml : remove OpenCL (#7735) 0cd6bd3483fa66124b76a8a8ac794d9ee18c70c1 5ca0944a153b65724d51b2f484139aa25ccb7a8b Georgi Gerganov ggerganov@gmail.com 2024-06-04T21:23:05+03:00 GitHub noreply@github.com 2024-06-04T21:23:05+03:00 llama : remove beam search (#7736) 5ca0944a153b65724d51b2f484139aa25ccb7a8b adc9ff384121f4d550d28638a646b336d051bf42 Georgi Gerganov ggerganov@gmail.com 2024-06-04T19:43:01+03:00 Georgi Gerganov ggerganov@gmail.com 2024-06-04T19:43:01+03:00 readme : remove obsolete Zig instructions (#7471) adc9ff384121f4d550d28638a646b336d051bf42 987d743d6bc4cee4bde6820733ea33a2abc0afac slaren slarengh@gmail.com 2024-06-04T14:32:42+02:00 GitHub noreply@github.com 2024-06-04T14:32:42+02:00 llama-bench : allow using a different printer for stderr with -oe (#7722) 987d743d6bc4cee4bde6820733ea33a2abc0afac b226c1227bcf6412076ecf787421135fd2c42ef0 Daniele 57776841+daniandtheweb@users.noreply.github.com 2024-06-04T12:09:15Z GitHub noreply@github.com 2024-06-04T14:09:15+02:00 Improve hipBLAS support in CMake (#7696) b226c1227bcf6412076ecf787421135fd2c42ef0 3b38d48609280aa5f8ab7ea135a4351b2a5ee240 zhouwg zhouwg2000@gmail.com 2024-06-04T19:21:26+08:00 GitHub noreply@github.com 2024-06-04T21:21:26+10:00 refine .gitignore (#7688) 3b38d48609280aa5f8ab7ea135a4351b2a5ee240 6d1616944d9efd342ed2a4fd318722adfc9febcd jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-06-04T09:17:17+02:00 GitHub noreply@github.com 2024-06-04T09:17:17+02:00 Per token attributes (#7685) 6d1616944d9efd342ed2a4fd318722adfc9febcd bde7cd3cd949c1a85d3a199498ac98e78039d46f Georgi Gerganov ggerganov@gmail.com 2024-06-04T10:01:09+03:00 GitHub noreply@github.com 2024-06-04T17:01:09+10:00 ggml : prevent builds with -ffinite-math-only (#7726) bde7cd3cd949c1a85d3a199498ac98e78039d46f a5735e4426b19a3ebd0c653ad8ac01420458ee95 Radoslav Gerganov rgerganov@gmail.com 2024-06-03T20:03:26+03:00 GitHub noreply@github.com 2024-06-03T20:03:26+03:00 llama : offload to RPC in addition to other backends (#7640) a5735e4426b19a3ebd0c653ad8ac01420458ee95 0b832d53ba0ffcc759c8d62ede3772dd62321f8e Masaya, Kato 62578291+msy-kato@users.noreply.github.com 2024-06-04T00:14:15+09:00 GitHub noreply@github.com 2024-06-03T17:14:15+02:00 ggml : use OpenMP as a thread pool (#7606) 0b832d53ba0ffcc759c8d62ede3772dd62321f8e 3d7ebf63123b8652fb7bbecef7ba731202309901 Johannes Gäßler johannesg@5d6.de 2024-06-03T16:28:58+02:00 GitHub noreply@github.com 2024-06-03T16:28:58+02:00 make: fix debug options not being applied to NVCC (#7714) 3d7ebf63123b8652fb7bbecef7ba731202309901 a10cda58d3199cd85305e0f03a8c6056714ae2e8 0cc4m picard12@live.de 2024-06-03T10:59:14+02:00 GitHub noreply@github.com 2024-06-03T10:59:14+02:00 Vulkan Mixture of Experts (MoE) support (#7628) a10cda58d3199cd85305e0f03a8c6056714ae2e8 6f28a333c1e3fdfdc7b4f9d0367f2b41a9b7e9d4 Andy Tai andy-tai@users.noreply.github.com 2024-06-03T01:06:24-07:00 GitHub noreply@github.com 2024-06-03T11:06:24+03:00 cmake : add pkg-config spec file for llama.cpp (#7702) 6f28a333c1e3fdfdc7b4f9d0367f2b41a9b7e9d4 549279d8049d78620a2b081e26edb654f83c3bbd zhangkaihuo zhangkaihuo@gmail.com 2024-06-03T15:49:30+08:00 GitHub noreply@github.com 2024-06-03T10:49:30+03:00 llama : MiniCPM support tied embeddings (#7664) 549279d8049d78620a2b081e26edb654f83c3bbd 9e405b6e2ecb888e860f7b92720b4809e21b3915 Georgi Gerganov ggerganov@gmail.com 2024-06-03T08:34:43+03:00 GitHub noreply@github.com 2024-06-03T08:34:43+03:00 llama : avoid double token-to-piece cache (#7654) 9e405b6e2ecb888e860f7b92720b4809e21b3915 3413ae2193d0693f14bead02e5018f442cbf579b woachk 24752637+woachk@users.noreply.github.com 2024-06-03T07:32:16+02:00 GitHub noreply@github.com 2024-06-03T08:32:16+03:00 kompute : implement op_getrows_f32 (#6403) 3413ae2193d0693f14bead02e5018f442cbf579b 1669810d7c2446af8425aa54ff6611bf6f14646c Dave Airlie airlied@redhat.com 2024-06-03T07:59:54+10:00 GitHub noreply@github.com 2024-06-02T17:59:54-04:00 fix bug introduced in using calloc (#7701) 1669810d7c2446af8425aa54ff6611bf6f14646c 7c4e5b7eae26581869e782015d9deca947c34997 Georgi Gerganov ggerganov@gmail.com 2024-06-03T00:13:12+03:00 GitHub noreply@github.com 2024-06-02T14:13:12-07:00 flake.lock: Update (#7686) 7c4e5b7eae26581869e782015d9deca947c34997 9422c5e34bbd302493b77a8f6d546154a1f4fe82 Austin 77757836+teleprint-me@users.noreply.github.com 2024-06-02T13:39:08-04:00 GitHub noreply@github.com 2024-06-02T20:39:08+03:00 chore : add ignore rule for generated server themes (#7689) 9422c5e34bbd302493b77a8f6d546154a1f4fe82 e141ce624af57bdffbaf57014a044eb1d9689230 nickp27 nb.porter@gmail.com 2024-06-02T19:13:54+10:00 GitHub noreply@github.com 2024-06-02T12:13:54+03:00 [SYCL] Update rpc-server.cpp to include SYCL backend (#7682) e141ce624af57bdffbaf57014a044eb1d9689230 2e666832e6ac78194edf030bd1c295e21bdb022c Johannes Gäßler johannesg@5d6.de 2024-06-01T23:26:10+02:00 GitHub noreply@github.com 2024-06-01T23:26:10+02:00 Fix FlashAttention debug test, FP32 assert (#7684) 2e666832e6ac78194edf030bd1c295e21bdb022c 2ac95c9d5678d05e253691fb1f26471675bff5ad Yazan Agha-Schrader mountaiin@icloud.com 2024-06-01T21:31:48+02:00 GitHub noreply@github.com 2024-06-01T22:31:48+03:00 server : new UI (#7633) 2ac95c9d5678d05e253691fb1f26471675bff5ad 750f60c03e4d3f53fa51910551ce87a3d508d2d7 HanishKVC hanishkvc@gmail.com 2024-06-01T21:50:18+05:30 GitHub noreply@github.com 2024-06-02T02:20:18+10:00 SimpleChat: Simple histogram/repeatMatching driven garbageTrimming, Settings UI, Streaming mode, OpenAi Compat (Model, Authorization Bearer), Save/Restore session, Auto Settings UI (#7548) 750f60c03e4d3f53fa51910551ce87a3d508d2d7 9b596417af11c9ac44fcae0fcfbc6f3665089083 Johannes Gäßler johannesg@5d6.de 2024-06-01T15:47:04+02:00 GitHub noreply@github.com 2024-06-01T15:47:04+02:00 CUDA: fix Pascal FA, deq. KV to FP16 for batch > 8 (#7681) 9b596417af11c9ac44fcae0fcfbc6f3665089083 a323ec60af14a33d560df98f2cc41b4112cb4f80 Johannes Gäßler johannesg@5d6.de 2024-06-01T08:44:14+02:00 GitHub noreply@github.com 2024-06-01T08:44:14+02:00 CUDA: quantized KV support for FA vec (#7527) a323ec60af14a33d560df98f2cc41b4112cb4f80 0515ad93f48df63bbff204eddb0cac75e8585c65 Georgi Gerganov ggerganov@gmail.com 2024-05-31T22:23:04+03:00 GitHub noreply@github.com 2024-05-31T22:23:04+03:00 server : update js (#7670) 0515ad93f48df63bbff204eddb0cac75e8585c65 c8047d538f3addab40e3112be60bb92e70ce1a50 Galunid karolek1231456@gmail.com 2024-05-31T17:42:33+02:00 GitHub noreply@github.com 2024-05-31T17:42:33+02:00 convert-hf : Handle NotImplementedError in convert-hf-to-gguf (#7660) c8047d538f3addab40e3112be60bb92e70ce1a50 30e238b246f8002cc6eb7cb79afe242243f1f66d Johannes Gäßler johannesg@5d6.de 2024-05-31T16:26:21+02:00 GitHub noreply@github.com 2024-05-31T16:26:21+02:00 scripts: update compare_llama_bench.py [no ci] (#7673) 30e238b246f8002cc6eb7cb79afe242243f1f66d 16926dff92d6d0efa8cbc0f44d30d63349532b38 Daniele 57776841+daniandtheweb@users.noreply.github.com 2024-05-31T14:00:29Z GitHub noreply@github.com 2024-05-31T16:00:29+02:00 Improve HIP compatibility (#7672) 16926dff92d6d0efa8cbc0f44d30d63349532b38 0c27e6f62eea80140daf152d7b6c154466614e5c Georgi Gerganov ggerganov@gmail.com 2024-05-31T15:04:58+03:00 GitHub noreply@github.com 2024-05-31T15:04:58+03:00 readme : link homebrew discussion 0c27e6f62eea80140daf152d7b6c154466614e5c 2e32f874e675f7bc5307cb7b4470ddbe090bab8f Georgi Gerganov ggerganov@gmail.com 2024-05-31T14:17:10+03:00 GitHub noreply@github.com 2024-05-31T14:17:10+03:00 ggml : fix loongson compile warnings (#7537) 2e32f874e675f7bc5307cb7b4470ddbe090bab8f 1af511fc22cba4959dd8bced5501df9e8af6ddf9 Galunid karolek1231456@gmail.com 2024-05-31T10:24:41+02:00 GitHub noreply@github.com 2024-05-31T18:24:41+10:00 Somehow '**' got lost (#7663) 1af511fc22cba4959dd8bced5501df9e8af6ddf9 0541f06296753dbc59a57379eb54cec865a4c9f9 Galunid karolek1231456@gmail.com 2024-05-31T10:09:20+02:00 GitHub noreply@github.com 2024-05-31T10:09:20+02:00 Add convert.py removal to hot topics (#7662) 0541f06296753dbc59a57379eb54cec865a4c9f9 9022c33646fbf78da35f40c3f98576cc08c40ddf Sertaç Özercan 852750+sozercan@users.noreply.github.com 2024-05-30T16:57:16-07:00 GitHub noreply@github.com 2024-05-31T09:57:16+10:00 [no ci] docs: add aikit to readme (#7650) 9022c33646fbf78da35f40c3f98576cc08c40ddf 5921b8f089d3b7bda86aac5a66825df6a6c10603 JohnnyB jboero@users.noreply.github.com 2024-05-30T21:32:38+01:00 GitHub noreply@github.com 2024-05-30T22:32:38+02:00 Fixed painfully slow single process builds. (#7326) 5921b8f089d3b7bda86aac5a66825df6a6c10603 5dcdf946764fae49a8e2a90bf2f0960bde1c44e8 Georgi Gerganov ggerganov@gmail.com 2024-05-30T19:01:41+03:00 GitHub noreply@github.com 2024-05-31T02:01:41+10:00 llama : cache llama_token_to_piece (#7587) 5dcdf946764fae49a8e2a90bf2f0960bde1c44e8 2e2340de1740b07f2418c04792607387d4dc1442 Martin Delille martin@delille.org 2024-05-30T17:07:39+02:00 GitHub noreply@github.com 2024-05-31T01:07:39+10:00 Fix conan badge display [no ci] (#7645) 2e2340de1740b07f2418c04792607387d4dc1442 7846540bd291e52cd4eee53882315760e05239be Manuel 44313466+makuche@users.noreply.github.com 2024-05-30T16:58:15+02:00 GitHub noreply@github.com 2024-05-31T00:58:15+10:00 Add brew installation instruction to README [no ci] (#7616) 7846540bd291e52cd4eee53882315760e05239be e6157f94c8f835f7f774b98409078867472a34fe Martin Delille martin@delille.org 2024-05-30T14:52:50+02:00 GitHub noreply@github.com 2024-05-30T15:52:50+03:00 readme : add Conan badge (#7638) e6157f94c8f835f7f774b98409078867472a34fe 9c4c9cc83f7297a10bb3b2af54a22ac154fd5b20 Brian mofosyne@gmail.com 2024-05-30T21:55:36+10:00 GitHub noreply@github.com 2024-05-30T21:55:36+10:00 github: add contact links to issues and convert question into research [no ci] (#7612) 9c4c9cc83f7297a10bb3b2af54a22ac154fd5b20 59b0d077662fab430446b3119fa142f3291c45b2 Galunid karolek1231456@gmail.com 2024-05-30T13:40:00+02:00 GitHub noreply@github.com 2024-05-30T21:40:00+10:00 Move convert.py to examples/convert-legacy-llama.py (#7430) 59b0d077662fab430446b3119fa142f3291c45b2 d5c05821f3c3d6cabe8ac45776fe0ecb0da13eca Chris Elrod elrodc@gmail.com 2024-05-30T07:32:55-04:00 GitHub noreply@github.com 2024-05-30T21:32:55+10:00 faster avx512 exp implementation (#7551) d5c05821f3c3d6cabe8ac45776fe0ecb0da13eca 972b555ab935705f3437abd5909a5c46852811f6 junchao-loongson 68935141+junchao-loongson@users.noreply.github.com 2024-05-30T17:30:10+08:00 GitHub noreply@github.com 2024-05-30T12:30:10+03:00 ggml : fix loongarch build (O2 issue) (#7636) 972b555ab935705f3437abd5909a5c46852811f6 3854c9d07f67de7f8cd6d86117bfaef47549b05a Johannes Gäßler johannesg@5d6.de 2024-05-30T09:52:39+02:00 GitHub noreply@github.com 2024-05-30T09:52:39+02:00 README: explain parallel build [no ci] (#7618) 3854c9d07f67de7f8cd6d86117bfaef47549b05a eb57fee51f7b4d78039f003249873c2eb46f12f6 Meng, Hengyu hengyu.meng@intel.com 2024-05-30T14:19:08+08:00 GitHub noreply@github.com 2024-05-30T16:19:08+10:00 [SYCL] fix intel docker (#7630) eb57fee51f7b4d78039f003249873c2eb46f12f6 55d62262a99cd8bc28a1492975791fe433c8cc0f Galunid karolek1231456@gmail.com 2024-05-30T02:10:40+02:00 GitHub noreply@github.com 2024-05-30T02:10:40+02:00 gguf-py : Add tokenizer.ggml.pre to gguf-new-metadata.py (#7627) 55d62262a99cd8bc28a1492975791fe433c8cc0f 975ec63ff26cdf96156d1126d86f75a395fdc43a Georgi Gerganov ggerganov@gmail.com 2024-05-29T22:20:40+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-29T22:21:20+03:00 metal : remove invalid asserts (#7617) 975ec63ff26cdf96156d1126d86f75a395fdc43a fb76ec31a9914b7761c1727303ab30380fd4f05c Georgi Gerganov ggerganov@gmail.com 2024-05-29T20:45:25+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-29T20:45:25+03:00 metal : add missing asserts (#7617) fb76ec31a9914b7761c1727303ab30380fd4f05c cce3dcffc5695bd24835f04e6080070a2a119873 Georgi Gerganov ggerganov@gmail.com 2024-05-29T20:17:31+03:00 GitHub noreply@github.com 2024-05-29T20:17:31+03:00 ggml : fix YARN + add tests + add asserts (#7617) cce3dcffc5695bd24835f04e6080070a2a119873 210d99173dc82aafb48f6e39d787c387951fe3a9 Georgi Gerganov ggerganov@gmail.com 2024-05-29T15:38:26+03:00 GitHub noreply@github.com 2024-05-29T15:38:26+03:00 cuda : non-cont concat support (#7610) 210d99173dc82aafb48f6e39d787c387951fe3a9 87bdf2a199acd62e19814d7a4d0500a04a7f09f3 Radoslav Gerganov rgerganov@gmail.com 2024-05-29T14:45:44+03:00 GitHub noreply@github.com 2024-05-29T14:45:44+03:00 llama-bench : add support for the RPC backend (#7435) 87bdf2a199acd62e19814d7a4d0500a04a7f09f3 00281b7be32462754618c42ed93f95743af46627 slaren slarengh@gmail.com 2024-05-29T13:36:39+02:00 GitHub noreply@github.com 2024-05-29T13:36:39+02:00 ggml : use atomic_flag for critical section (#7598) 00281b7be32462754618c42ed93f95743af46627 2ab977282b02ccd6783fbbaec393c96886cf33b1 Georgi Gerganov ggerganov@gmail.com 2024-05-29T14:31:18+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-29T14:31:18+03:00 scripts : remove mpi remnants 2ab977282b02ccd6783fbbaec393c96886cf33b1 72de268bec49f67e2883880f573c55cea32de736 Georgi Gerganov ggerganov@gmail.com 2024-05-29T14:29:52+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-29T14:29:52+03:00 sync : ggml 72de268bec49f67e2883880f573c55cea32de736 0e8d8bfd6caf1d0a8cbdf9d3d5c06fbbb9dfced8 Georgi Gerganov ggerganov@gmail.com 2024-05-26T18:35:23+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-29T14:29:33+03:00 ggml : restore ggml_rope_xpos_inplace (ggml/0) 0e8d8bfd6caf1d0a8cbdf9d3d5c06fbbb9dfced8 504f0c340f6b5e04de682f6ddefdd3b81208df5d Akarshan Biswas akarshanbiswas@fedoraproject.org 2024-05-29T12:23:47+05:30 GitHub noreply@github.com 2024-05-29T16:53:47+10:00 Add Arc A750 and Arch linux to readme-sycl.md as verified GPU model and Linux distro (#7605) 504f0c340f6b5e04de682f6ddefdd3b81208df5d b864b50ce5e2beefc8c2fd31733e4e1a978b7754 zhouwg zhouwg2000@gmail.com 2024-05-29T10:09:31+08:00 GitHub noreply@github.com 2024-05-29T04:09:31+02:00 ggml : fix typo in ggml.c (#7603) b864b50ce5e2beefc8c2fd31733e4e1a978b7754 02c1ecad07f0e2d2febe8196271bcc64bdc9c006 Meng, Hengyu hengyu.meng@intel.com 2024-05-29T07:00:24+08:00 GitHub noreply@github.com 2024-05-29T07:00:24+08:00 [SYCL] Align GEMM dispatch (#7566) 02c1ecad07f0e2d2febe8196271bcc64bdc9c006 6bd12ce409f949012935b7d1b15a21ffa473a565 jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-05-28T21:46:34+02:00 GitHub noreply@github.com 2024-05-28T21:46:34+02:00 Tokenizer WPM fixes (#7500) 6bd12ce409f949012935b7d1b15a21ffa473a565 5442939fcc5e6ae41abf40612a95fd71377e487e Georgi Gerganov ggerganov@gmail.com 2024-05-28T22:22:50+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-28T22:22:50+03:00 sycl : fix assert (#7563) 5442939fcc5e6ae41abf40612a95fd71377e487e 56411a950f255b523a9edd684fd1632752474399 Giuseppe Scrivano giuseppe@scrivano.org 2024-05-28T20:49:49+02:00 GitHub noreply@github.com 2024-05-28T21:49:49+03:00 llama : support small Granite models (#7481) 56411a950f255b523a9edd684fd1632752474399 2b737caae100cf0ac963206984332e422058f2b9 k.h.lai adrian.k.h.lai@outlook.com 2024-05-29T01:25:08+08:00 GitHub noreply@github.com 2024-05-28T19:25:08+02:00 vulkan: properly initialize vulkan devices for LLAMA_SPLIT_MODE_NONE (#7552) 2b737caae100cf0ac963206984332e422058f2b9 ee3dff6b8e39bb8c1cdea1782a7b95ef0118f970 Radoslav Gerganov rgerganov@gmail.com 2024-05-28T18:13:36+03:00 GitHub noreply@github.com 2024-05-28T18:13:36+03:00 rpc : resource management rework (#7562) ee3dff6b8e39bb8c1cdea1782a7b95ef0118f970 edc29433fa08b4e5aeb67649a29fc7713af13d04 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-05-28T17:07:05+02:00 GitHub noreply@github.com 2024-05-28T17:07:05+02:00 Add support for DeepseekV2ForCausalLM (#7519) edc29433fa08b4e5aeb67649a29fc7713af13d04 8b99e2aa66ba39e4e1114effea6ef7430881eca4 Georgi Gerganov ggerganov@gmail.com 2024-05-28T15:04:09+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-28T15:04:09+03:00 tests : fix test-tokenizer-0.sh 8b99e2aa66ba39e4e1114effea6ef7430881eca4 271ff3fc44a6ecfcea3ebc192e67567d578b7772 Georgi Gerganov ggerganov@gmail.com 2024-05-28T13:55:35+03:00 GitHub noreply@github.com 2024-05-28T13:55:35+03:00 llama : handle unknown utf8 bytes (#7588) 271ff3fc44a6ecfcea3ebc192e67567d578b7772 e2b065071c5fc8ac5697d12ca343551faee465cc Brian mofosyne@gmail.com 2024-05-28T20:27:27+10:00 GitHub noreply@github.com 2024-05-28T20:27:27+10:00 github: add refactor to issue template (#7561) e2b065071c5fc8ac5697d12ca343551faee465cc 0548a4187f2e53b8fc6d9ff0f4c71988f708ff42 Neo Zhang 14088817+arthw@users.noreply.github.com 2024-05-28T17:53:37+08:00 GitHub noreply@github.com 2024-05-28T10:53:37+01:00 [SYCL]fix ggml_sycl_mul_mat_id() to match the change of api (#7436) 0548a4187f2e53b8fc6d9ff0f4c71988f708ff42 9335b969e86a222e247adacedf814d8abfff8847 Georgi Gerganov ggerganov@gmail.com 2024-05-28T11:04:19+03:00 GitHub noreply@github.com 2024-05-28T11:04:19+03:00 ggml : generalize GGML_OP_CONCAT (#7563) 9335b969e86a222e247adacedf814d8abfff8847 c41767154eb82aa3fe7568fc816c3402b78eae94 mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2024-05-28T06:55:51+02:00 GitHub noreply@github.com 2024-05-28T14:55:51+10:00 server: do not remove whitespace at the start of a completion chunk (#7524) c41767154eb82aa3fe7568fc816c3402b78eae94 74b239b3d5f067470d7ef5e26e2e059720572e32 Nathan Epstein nate2@umbc.edu 2024-05-28T00:41:14-04:00 GitHub noreply@github.com 2024-05-28T14:41:14+10:00 Markdownish code block fix (#7571) 74b239b3d5f067470d7ef5e26e2e059720572e32 852aafb163d32d5bad63c10bc323a02c28fec59d Ikko Eltociear Ashimine eltociear@gmail.com 2024-05-28T11:48:16+09:00 GitHub noreply@github.com 2024-05-28T12:48:16+10:00 llava : update clip.h (#7580) 852aafb163d32d5bad63c10bc323a02c28fec59d 0136966dafb452601c23f30395878d5a65ddc559 Djip007 djip.perois@free.fr 2024-05-28T01:40:47+02:00 GitHub noreply@github.com 2024-05-28T01:40:47+02:00 update HIP_UMA #7399 (#7414) 0136966dafb452601c23f30395878d5a65ddc559 10b1e4587670feba2c7730a645accf8234873113 kunnis kunnis@users.noreply.github.com 2024-05-27T18:40:12-05:00 GitHub noreply@github.com 2024-05-28T01:40:12+02:00 adding in x64 targets to cmake presets (#7574) 10b1e4587670feba2c7730a645accf8234873113 197c00681b80f9dea17d11a4436b6b8ef1be0ce8 Johannes Gäßler johannesg@5d6.de 2024-05-27T19:34:40+02:00 GitHub noreply@github.com 2024-05-27T19:34:40+02:00 make: add --device-debug to NVCC debug flags (#7542) 197c00681b80f9dea17d11a4436b6b8ef1be0ce8 95f84d5ce8b449a9b16009434aca800df504a02e agray3 agray3@users.noreply.github.com 2024-05-27T18:33:42+01:00 GitHub noreply@github.com 2024-05-27T19:33:42+02:00 Allow multiple copy function pointers for CUDA graph kernel param updates (#7565) 95f84d5ce8b449a9b16009434aca800df504a02e 5487593bc7ee0b65b9d2e2985b4b61dc77043101 AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-05-27T17:34:51+01:00 GitHub noreply@github.com 2024-05-27T22:04:51+05:30 Fix q_xxs using mul_mat_q (#7459) 5487593bc7ee0b65b9d2e2985b4b61dc77043101 1d8fca72ae9154eec0e1c0a75cfaac3c50f08e4a AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-05-27T13:34:09+01:00 GitHub noreply@github.com 2024-05-27T18:04:09+05:30 Add freq factors (#7495) 1d8fca72ae9154eec0e1c0a75cfaac3c50f08e4a 62bfef5194d5582486d62da3db59bf44981b7912 Georgi Gerganov ggerganov@gmail.com 2024-05-27T12:10:19+03:00 GitHub noreply@github.com 2024-05-27T12:10:19+03:00 metal : add GGML_OP_REPEAT kernels (#7557) 62bfef5194d5582486d62da3db59bf44981b7912 eaf6e031741ca2d3aafeff3e0f4dd7557a974d2b Georgi Gerganov ggerganov@gmail.com 2024-05-27T10:38:39+03:00 GitHub noreply@github.com 2024-05-27T10:38:39+03:00 metal : disable FA kernel for HS=256 (#7556) eaf6e031741ca2d3aafeff3e0f4dd7557a974d2b d6ef0e77dd25f54fb5856af47e3926cf6f36c281 Georgi Gerganov ggerganov@gmail.com 2024-05-27T09:24:13+03:00 GitHub noreply@github.com 2024-05-27T09:24:13+03:00 llama : add comments about experimental flags (#7544) d6ef0e77dd25f54fb5856af47e3926cf6f36c281 dff451cfa1f297348751ce6b538670e1ae9a7d5b Brian mofosyne@gmail.com 2024-05-27T10:54:30+10:00 GitHub noreply@github.com 2024-05-27T10:54:30+10:00 github: add self sorted issue ticket forms (#7543) dff451cfa1f297348751ce6b538670e1ae9a7d5b d298382ad977ec89c8de7b57459b9d7965d2c272 Georgi Gerganov ggerganov@gmail.com 2024-05-26T18:54:56+03:00 GitHub noreply@github.com 2024-05-26T08:54:56-07:00 flake.lock: Update (#7540) d298382ad977ec89c8de7b57459b9d7965d2c272 32a28217f475119926c603341e8273b26932b56a Brian mofosyne@gmail.com 2024-05-27T00:10:17+10:00 GitHub noreply@github.com 2024-05-27T00:10:17+10:00 main: replace --no-special with --special (#7534) 32a28217f475119926c603341e8273b26932b56a c429b33beb35f13934a4dfbe0c138d30b45e5d54 Galunid karolek1231456@gmail.com 2024-05-26T16:02:34+02:00 GitHub noreply@github.com 2024-05-26T16:02:34+02:00 Fix aya-23 conversion scripts (#7539) c429b33beb35f13934a4dfbe0c138d30b45e5d54 9146d36fe7e3e911a07438c07efc1bae082f6390 Bartowski ckealty1182@gmail.com 2024-05-26T08:28:35-04:00 GitHub noreply@github.com 2024-05-26T15:28:35+03:00 llama : add Smaug 70B support (#7402) 9146d36fe7e3e911a07438c07efc1bae082f6390 b9adcbbf92fc7096bee23fe61496d25652ebf765 Aarni Koskela akx@iki.fi 2024-05-26T15:09:42+03:00 GitHub noreply@github.com 2024-05-26T22:09:42+10:00 Readme: add akx/ggify to tools (#1484) b9adcbbf92fc7096bee23fe61496d25652ebf765 9588f196b1d7b21bdff013fcf958c249576b2619 HanishKVC hanishkvc@gmail.com 2024-05-26T06:26:34+05:30 GitHub noreply@github.com 2024-05-26T10:56:34+10:00 SimpleChat Completion Mode flexibility and cleanup, Settings gMe, Optional sliding window (#7480) 9588f196b1d7b21bdff013fcf958c249576b2619 3cbd23ed88c03a27e1eb6090ac4a8186ca9ac29a Georgi Gerganov ggerganov@gmail.com 2024-05-25T15:21:30+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-25T15:22:35+03:00 train : change default FA argument (#7528) 3cbd23ed88c03a27e1eb6090ac4a8186ca9ac29a 00c63907931bb08a0ed2b7e38cf44dd290143cb9 Brian mofosyne@gmail.com 2024-05-25T19:30:42+10:00 GitHub noreply@github.com 2024-05-25T19:30:42+10:00 labeler: added Apple Metal detector (+Kompute) (#7529) 00c63907931bb08a0ed2b7e38cf44dd290143cb9 faa0e6979a11dcb731e9d778ad42ceaa0302015e Justine Tunney jtunney@mozilla.com 2024-05-25T05:04:03-04:00 GitHub noreply@github.com 2024-05-25T19:04:03+10:00 main : don't print special tokens with --grammar (#6923) faa0e6979a11dcb731e9d778ad42ceaa0302015e 9791f402580838d7f8543ae7bc633ef265e436f0 Masaya, Kato 62578291+msy-kato@users.noreply.github.com 2024-05-25T17:42:31+09:00 GitHub noreply@github.com 2024-05-25T11:42:31+03:00 ggml: aarch64: SVE kernels for q8_0_q8_0, q4_0_q8_0 vector dot (#7433) 9791f402580838d7f8543ae7bc633ef265e436f0 902184dd3a9d6685e752b19027a48423742531db Elton Kola eltonkola@gmail.com 2024-05-25T04:11:33-04:00 GitHub noreply@github.com 2024-05-25T11:11:33+03:00 android : module (#7502) 902184dd3a9d6685e752b19027a48423742531db 57684331fc2d685f7d1f5775af0b9e47d1829833 Xuan Son Nguyen thichthat@gmail.com 2024-05-25T05:30:59+02:00 GitHub noreply@github.com 2024-05-25T13:30:59+10:00 fix missing slash in `fs_get_cache_directory()` (#7503) 57684331fc2d685f7d1f5775af0b9e47d1829833 b83bab15a5d2a1e7807d09613a9b34309d86cfaa Mikko Juola mikjuo@gmail.com 2024-05-24T18:14:42-07:00 GitHub noreply@github.com 2024-05-25T11:14:42+10:00 Make tokenize CLI tool have nicer command line arguments. (#6188) b83bab15a5d2a1e7807d09613a9b34309d86cfaa d041d2ceaaf50e058622d92921b3e680ffa4e9e7 compilade git@compilade.net 2024-05-24T21:11:48-04:00 GitHub noreply@github.com 2024-05-25T11:11:48+10:00 gguf-py : fix and simplify quantized shape round-trip (#7483) d041d2ceaaf50e058622d92921b3e680ffa4e9e7 27891f6db03de6e3fd5941983838c29bef253352 Georgi Gerganov ggerganov@gmail.com 2024-05-24T18:59:06+03:00 GitHub noreply@github.com 2024-05-24T08:59:06-07:00 flake.lock: Update (#7232) 27891f6db03de6e3fd5941983838c29bef253352 fbca2f27fc7fa9aa4a8ad0357478fdb908472908 Brian mofosyne@gmail.com 2024-05-24T23:47:56+10:00 GitHub noreply@github.com 2024-05-24T23:47:56+10:00 docker.yml: disable light-intel and server-intel test (#7515) fbca2f27fc7fa9aa4a8ad0357478fdb908472908 0df0aa8e43c3378975269a51f9b876c8692e70da fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-05-24T14:31:13+02:00 GitHub noreply@github.com 2024-05-24T14:31:13+02:00 Add support for ArcticForCausalLM (#7020) 0df0aa8e43c3378975269a51f9b876c8692e70da 74f33adf5f8b20b08fc5a6aa17ce081abe86ef2f Neo Zhang 14088817+arthw@users.noreply.github.com 2024-05-24T10:06:56+08:00 GitHub noreply@github.com 2024-05-24T10:06:56+08:00 add build shared lib in win release package (#7438) 74f33adf5f8b20b08fc5a6aa17ce081abe86ef2f 1debe72737ea131cb52975da3d53ed3a835df3a6 Georgi Gerganov ggerganov@gmail.com 2024-05-23T17:43:18+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-23T17:43:18+03:00 readme : remove trailing space (#7469) 1debe72737ea131cb52975da3d53ed3a835df3a6 007489e895bad02e4e54758bf0bdf2d6a4cdb7c1 Georgi Gerganov ggerganov@gmail.com 2024-05-23T17:17:43+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-23T17:25:38+03:00 ggml : silence UB sanitizer error during iq2_xxs quantization (#0) 007489e895bad02e4e54758bf0bdf2d6a4cdb7c1 8b94e799dfa482adf63419df4905dc79b37e179f Tristan Druyen tristan@vault81.mozmail.com 2024-05-23T16:15:15+02:00 GitHub noreply@github.com 2024-05-23T16:15:15+02:00 Fix phi3 chat template confusion with zephyr (#7449) 8b94e799dfa482adf63419df4905dc79b37e179f 3015851c5ac7334fb544a23a70a284c117b87044 Raj Hammeer Singh Hada hammeerraj@gmail.com 2024-05-23T18:00:13+05:30 GitHub noreply@github.com 2024-05-23T15:30:13+03:00 readme : add Bunny in supported models [no ci] (#7469) 3015851c5ac7334fb544a23a70a284c117b87044 55ac3b7aeaf52f19786ed96e885d89521fc0f6c8 Daniel Bevenius daniel.bevenius@gmail.com 2024-05-23T14:29:26+02:00 GitHub noreply@github.com 2024-05-23T15:29:26+03:00 llama : add getters for n_threads/n_threads_batch (#7464) 55ac3b7aeaf52f19786ed96e885d89521fc0f6c8 dacfcebd6022175848e978f82811a244f1033038 Georgi Gerganov ggerganov@gmail.com 2024-05-23T15:28:14+03:00 GitHub noreply@github.com 2024-05-23T15:28:14+03:00 ci : use Pythia models instead of OpenLlama (#7470) dacfcebd6022175848e978f82811a244f1033038 9b82476ee9e73065a759f8bcc4cf27ec7ab2ed8c Victor Nogueira felladrin@gmail.com 2024-05-23T15:12:43+03:00 GitHub noreply@github.com 2024-05-23T15:12:43+03:00 readme : add GPT-NeoX + Pythia to the list of supported models (#7491) 9b82476ee9e73065a759f8bcc4cf27ec7ab2ed8c a61a94e543e3c6877c087e80fca27a0313ce5fd5 fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-05-23T11:49:53+02:00 GitHub noreply@github.com 2024-05-23T11:49:53+02:00 Add missing inference support for GPTNeoXForCausalLM (Pythia and GPT-NeoX base models) (#7461) a61a94e543e3c6877c087e80fca27a0313ce5fd5 152da28ae54139e3754189b9e6e1c28e11277502 Georgi Gerganov ggerganov@gmail.com 2024-05-23T12:38:18+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-23T12:38:18+03:00 llama : rename n_ctx -> cache.size, less confusing (#0) 152da28ae54139e3754189b9e6e1c28e11277502 d48c88cbd563b6cf0ce972e2f56796896e240736 Brian mofosyne@gmail.com 2024-05-23T17:40:43+10:00 GitHub noreply@github.com 2024-05-23T17:40:43+10:00 labeler.yml: add embedding label detector [no ci] (#7482) d48c88cbd563b6cf0ce972e2f56796896e240736 e84b71c2c6da6e69c8f815168ea836f9716a325e Georgi Gerganov ggerganov@gmail.com 2024-05-23T10:00:44+03:00 GitHub noreply@github.com 2024-05-23T10:00:44+03:00 ggml : remove ggml_flash_attn and ggml_flash_ff (#7463) e84b71c2c6da6e69c8f815168ea836f9716a325e 1b1e27cb49158123ef4902aa41eb368c9e76e6a1 Georgi Gerganov ggerganov@gmail.com 2024-05-23T10:00:21+03:00 GitHub noreply@github.com 2024-05-23T10:00:21+03:00 ggml : drop support for QK_K=64 (#7473) 1b1e27cb49158123ef4902aa41eb368c9e76e6a1 fbf777d2b9c30e7569e3d1c149501c1e31d9b5b9 0cc4m picard12@live.de 2024-05-23T08:59:59+02:00 GitHub noreply@github.com 2024-05-23T08:59:59+02:00 Update vulkan rope implementation to support frequency factors (#7475) fbf777d2b9c30e7569e3d1c149501c1e31d9b5b9 cd93a28cb1446319af5e2f4b416174c3a8e43546 Georgi Gerganov ggerganov@gmail.com 2024-05-23T09:43:24+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-23T09:43:49+03:00 main : minor (#7462) cd93a28cb1446319af5e2f4b416174c3a8e43546 1e374365d170b7f692fd7753c145e21bc14486c8 Johannes Gäßler johannesg@5d6.de 2024-05-23T00:31:20+02:00 GitHub noreply@github.com 2024-05-23T00:31:20+02:00 CUDA: fix FA out-of-bounds reads (#7479) 1e374365d170b7f692fd7753c145e21bc14486c8 197ff91462dd05bb9a3be03578114abf0c355536 HanishKVC hanishkvc@gmail.com 2024-05-22T23:23:21+05:30 GitHub noreply@github.com 2024-05-23T03:53:21+10:00 SimpleChat: a simple and dumb web front end for testing /chat/completions and /completions end points and try chat (#7350) 197ff91462dd05bb9a3be03578114abf0c355536 6ff13987ad1a9519bee13dd98b6a21cd98979aab Georgi Gerganov ggerganov@gmail.com 2024-05-22T20:05:38+03:00 GitHub noreply@github.com 2024-05-22T20:05:38+03:00 build : remove zig (#7471) 6ff13987ad1a9519bee13dd98b6a21cd98979aab 38c03478a37e460ecd3a21155b338a83bfed7f90 Georgi Gerganov ggerganov@gmail.com 2024-05-22T20:04:20+03:00 GitHub noreply@github.com 2024-05-22T20:04:20+03:00 common : normalize naming style (#7462) 38c03478a37e460ecd3a21155b338a83bfed7f90 b18532a4efeca8796fea8e36195c81cbfd596a4a Johannes Gäßler johannesg@5d6.de 2024-05-22T17:58:25+02:00 GitHub noreply@github.com 2024-05-22T17:58:25+02:00 CUDA: fix FA out-of-bounds writes (#7465) b18532a4efeca8796fea8e36195c81cbfd596a4a fcda1128bc5f8eb7e1811708fe9d9867b9aec815 slaren slarengh@gmail.com 2024-05-22T16:10:46+02:00 GitHub noreply@github.com 2024-05-22T16:10:46+02:00 phi3 : duplicate rope factors in each layer (#7447) fcda1128bc5f8eb7e1811708fe9d9867b9aec815 03d8900ebe062355e26a562379daee5f17ea099f k.h.lai adrian.k.h.lai@outlook.com 2024-05-22T20:53:21+08:00 GitHub noreply@github.com 2024-05-22T14:53:21+02:00 vulkan: add workaround for iterator boundary check to fix clang-cl debug build (#7426) 03d8900ebe062355e26a562379daee5f17ea099f 9b3d83318931aa98c487baaa977626931d059e6a Justine Tunney jtunney@mozilla.com 2024-05-22T07:08:18-04:00 GitHub noreply@github.com 2024-05-22T14:08:18+03:00 llama : add missing model type names (#7445) 9b3d83318931aa98c487baaa977626931d059e6a 95fb0aefab568348da159efdd370e064d1b35f97 Georgi Gerganov ggerganov@gmail.com 2024-05-22T12:36:37+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-22T12:36:37+03:00 cuda : fix compile warning (#7454) 95fb0aefab568348da159efdd370e064d1b35f97 3e5faa85032ec3106a2ad831bf412be9ff139f47 Johannes Gäßler johannesg@5d6.de 2024-05-22T10:24:29+02:00 GitHub noreply@github.com 2024-05-22T10:24:29+02:00 CUDA: remove incorrect precision check (#7454) 3e5faa85032ec3106a2ad831bf412be9ff139f47 201cc11afa0a1950e1f632390b2ac6c937a0d8f0 Georgi Gerganov ggerganov@gmail.com 2024-05-22T11:01:35+03:00 GitHub noreply@github.com 2024-05-22T11:01:35+03:00 cuda : fix rope + add tests (#7452) 201cc11afa0a1950e1f632390b2ac6c937a0d8f0 6369bf04336ab60e5c892dd77a3246df91015147 liuwei-git 14815172+liuwei-git@users.noreply.github.com 2024-05-22T04:28:32+08:00 GitHub noreply@github.com 2024-05-21T23:28:32+03:00 llama : add phi3 128K model support (#7225) 6369bf04336ab60e5c892dd77a3246df91015147 e402de364b643cb89ea9f43057733b5d36298670 Georgi Gerganov ggerganov@gmail.com 2024-05-21T23:03:42+03:00 GitHub noreply@github.com 2024-05-21T23:03:42+03:00 metal : handle F16 inf values, fix FA partial offload (#7434) e402de364b643cb89ea9f43057733b5d36298670 fcf6538ba6702c55eaec70da9a75c81d04900a72 Olivier Chafik ochafik@users.noreply.github.com 2024-05-21T20:40:00+01:00 GitHub noreply@github.com 2024-05-21T20:40:00+01:00 `grammars`: fix resampling logic regression (#7424) fcf6538ba6702c55eaec70da9a75c81d04900a72 c3f8d583560b4f261fa21c976793e538c60cd66c Johannes Gäßler johannesg@5d6.de 2024-05-21T19:27:12+02:00 GitHub noreply@github.com 2024-05-21T20:27:12+03:00 CUDA: fix unused warning in mmq.cu (#7442) c3f8d583560b4f261fa21c976793e538c60cd66c 11474e756de3f56b760986e73086d40e787e52f8 Georgi Gerganov ggerganov@gmail.com 2024-05-21T19:53:48+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-21T19:53:48+03:00 tests : test-tokenizer-0.sh print more info (#7402) 11474e756de3f56b760986e73086d40e787e52f8 d8ee90222791afff2ab666ded4cb6195fd94cced Amir amir_zia@outlook.com 2024-05-21T17:13:12+03:00 GitHub noreply@github.com 2024-05-21T17:13:12+03:00 examples: cache hf model when --model not provided (#7353) d8ee90222791afff2ab666ded4cb6195fd94cced d7e852c1bc8e85bf62a6f1aede08cd2de723404a Johannes Gäßler johannesg@5d6.de 2024-05-21T16:02:12+02:00 GitHub noreply@github.com 2024-05-21T16:02:12+02:00 CUDA: deduplicate mmq code (#7397) d7e852c1bc8e85bf62a6f1aede08cd2de723404a 917dc8cfa67a72fb7c8bf7392270da3bf4833af4 jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-05-21T14:39:48+02:00 GitHub noreply@github.com 2024-05-21T14:39:48+02:00 Tokenizer SPM fixes for phi-3 and llama-spm (bugfix) (#7425) 917dc8cfa67a72fb7c8bf7392270da3bf4833af4 fabf30b4c4fca32e116009527180c252919ca922 jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-05-20T20:15:57+02:00 GitHub noreply@github.com 2024-05-20T20:15:57+02:00 Tokenizer SPM fixes for phi-3 and llama-spm (#7375) fabf30b4c4fca32e116009527180c252919ca922 20385cebcc4bb3f6dd10f989573c11864d70d53d Georgi Gerganov ggerganov@gmail.com 2024-05-20T19:35:28+03:00 GitHub noreply@github.com 2024-05-21T02:35:28+10:00 llama : remove Persimmon (#7408) 20385cebcc4bb3f6dd10f989573c11864d70d53d db10f01310beea8a1ef7798651b9d692fd1149d0 Johannes Gäßler johannesg@5d6.de 2024-05-20T18:15:38+02:00 GitHub noreply@github.com 2024-05-20T18:15:38+02:00 perplexity: update README FP16 results [no ci] (#7413) db10f01310beea8a1ef7798651b9d692fd1149d0 3bc10cb485dd7efa4da6c64e73ad0c9e2bfe0821 Radoslav Gerganov rgerganov@gmail.com 2024-05-20T16:36:55+03:00 GitHub noreply@github.com 2024-05-20T16:36:55+03:00 rpc : track allocated buffers (#7411) 3bc10cb485dd7efa4da6c64e73ad0c9e2bfe0821 6bf9b66fa3f263ca2175dcb5f6d0a658581e1dfb Georgi Gerganov ggerganov@gmail.com 2024-05-20T15:10:03+03:00 GitHub noreply@github.com 2024-05-20T22:10:03+10:00 server : fix temperature + disable some tests (#7409) 6bf9b66fa3f263ca2175dcb5f6d0a658581e1dfb 26cd4237bc499f7144d76f440aa775d749b170bb AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-05-20T12:08:23+01:00 GitHub noreply@github.com 2024-05-20T16:38:23+05:30 [SYCL] Update SYCL upscale operation (#7321) 26cd4237bc499f7144d76f440aa775d749b170bb 213e90ed73f8ac3cd3026dc3f086beae0d414f96 Bingan 70050083+binganao@users.noreply.github.com 2024-05-20T17:55:34+08:00 GitHub noreply@github.com 2024-05-20T11:55:34+02:00 Update README.md (#7410) 213e90ed73f8ac3cd3026dc3f086beae0d414f96 65c58207ece92ad213f4bfd0f91dcb2dfb664f5b Herman Semenov GermanAizek@yandex.ru 2024-05-20T07:33:21Z GitHub noreply@github.com 2024-05-20T10:33:21+03:00 ggml-opencl, llama: using reserve() if count already known (#7272) 65c58207ece92ad213f4bfd0f91dcb2dfb664f5b 1cc0155d04918cb3017afa472acea51b77483c4a junchao-loongson 68935141+junchao-loongson@users.noreply.github.com 2024-05-20T15:19:21+08:00 GitHub noreply@github.com 2024-05-20T10:19:21+03:00 ggml : add loongarch lsx and lasx support (#6454) 1cc0155d04918cb3017afa472acea51b77483c4a e932094d58f513d5996c3efc9f6fed8238894c57 Georgi Gerganov ggerganov@gmail.com 2024-05-20T10:16:41+03:00 GitHub noreply@github.com 2024-05-20T10:16:41+03:00 server : tuning tests (#7388) e932094d58f513d5996c3efc9f6fed8238894c57 2789baf480ba7dc9281b65f601f0918e58920f54 Georgi Gerganov ggerganov@gmail.com 2024-05-20T08:56:05+03:00 GitHub noreply@github.com 2024-05-20T08:56:05+03:00 server : return error on too large embedding input (#7389) 2789baf480ba7dc9281b65f601f0918e58920f54 33c8d50accd6dca73c9c4af00a05e24209c160fe Georgi Gerganov ggerganov@gmail.com 2024-05-20T08:55:09+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-20T08:55:09+03:00 tests : fix --keep_split -> --keep-split (#7374) 33c8d50accd6dca73c9c4af00a05e24209c160fe d359f30921a9f62a0fd299c412ff3f270286fea6 Srihari-mcw 96763064+Srihari-mcw@users.noreply.github.com 2024-05-19T19:18:39-07:00 GitHub noreply@github.com 2024-05-20T12:18:39+10:00 Add provisions for windows support for BF16 code including CMake provision for enabling AVX512_BF16 (#7258) d359f30921a9f62a0fd299c412ff3f270286fea6 1ea2a0036e88172d6c8bf7e1a1989a03894dc955 slaren slarengh@gmail.com 2024-05-20T01:17:03+02:00 GitHub noreply@github.com 2024-05-20T01:17:03+02:00 llama : remove MPI backend (#7395) 1ea2a0036e88172d6c8bf7e1a1989a03894dc955 f030ec1f7a72aa825b2104823946551b9ec5dfc1 Fred Douglas 43351173+fredlas@users.noreply.github.com 2024-05-19T11:37:04-05:00 GitHub noreply@github.com 2024-05-19T19:37:04+03:00 quantize : fix --keep-split check (#7374) f030ec1f7a72aa825b2104823946551b9ec5dfc1 e4e6f67be6a8a697f5f89a28c98934e53c99c359 0cc4m picard12@live.de 2024-05-19T17:19:53+02:00 GitHub noreply@github.com 2024-05-19T17:19:53+02:00 Vulkan Embedding Fix (#7360) e4e6f67be6a8a697f5f89a28c98934e53c99c359 5ca49cbecda27ce0a7266658fc3b640bff3ed386 slaren slarengh@gmail.com 2024-05-19T17:08:46+02:00 GitHub noreply@github.com 2024-05-19T17:08:46+02:00 ggml : fix another case of quants nans (#7387) 5ca49cbecda27ce0a7266658fc3b640bff3ed386 1b01f06db0cff5f5f600bb754fc39fde565ed56a Johannes Gäßler johannesg@5d6.de 2024-05-19T16:46:13+02:00 GitHub noreply@github.com 2024-05-19T16:46:13+02:00 ggml: implement quantized KV cache for FA (#7372) 1b01f06db0cff5f5f600bb754fc39fde565ed56a 41858392e17abead21735309bf17cb55183d8c31 Johannes Gäßler johannesg@5d6.de 2024-05-19T16:26:02+02:00 GitHub noreply@github.com 2024-05-19T16:26:02+02:00 server: add test for token probs (#7347) 41858392e17abead21735309bf17cb55183d8c31 6aade19ee74b896c59929676629340b36be3e22c Johannes Gäßler johannesg@5d6.de 2024-05-19T16:06:33+02:00 GitHub noreply@github.com 2024-05-19T17:06:33+03:00 server: fix seed being reported back (#7382) 6aade19ee74b896c59929676629340b36be3e22c ab33f7a338593f6cf1ae98b10b6f8684f63bd72c Anas Ahouzi 112881240+aahouzi@users.noreply.github.com 2024-05-19T14:46:46+02:00 GitHub noreply@github.com 2024-05-19T22:46:46+10:00 Add StableLM2 pre-tokenizer (#7349) ab33f7a338593f6cf1ae98b10b6f8684f63bd72c e23b974f4cf9270d05062d446f406e3ff55d9451 slaren slarengh@gmail.com 2024-05-19T14:19:37+02:00 GitHub noreply@github.com 2024-05-19T14:19:37+02:00 cuda : clear error after buffer allocation failure (#7376) e23b974f4cf9270d05062d446f406e3ff55d9451 854d365abab7194b5013a523f72da19860c3c550 Brian mofosyne@gmail.com 2024-05-19T20:51:03+10:00 GitHub noreply@github.com 2024-05-19T20:51:03+10:00 labeler.yml: Use settings from ggerganov/llama.cpp [no ci] (#7363) 854d365abab7194b5013a523f72da19860c3c550 f5bf761747988ee1832766f7d1433739aff810da Georgi Gerganov ggerganov@gmail.com 2024-05-19T11:01:01+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-19T11:01:01+03:00 cmake : update android comments (#7341) f5bf761747988ee1832766f7d1433739aff810da 059031b8c40e1f4ba60586842c5b1ed3ddf61842 fraxy-v 65565042+fraxy-v@users.noreply.github.com 2024-05-19T01:44:42+03:00 GitHub noreply@github.com 2024-05-19T00:44:42+02:00 Capture CUDA logging output (#7298) 059031b8c40e1f4ba60586842c5b1ed3ddf61842 511182eabb36f6ec9776e2b3c4d7e16d93d0ac0d Georgi Gerganov ggerganov@gmail.com 2024-05-18T18:55:54+03:00 GitHub noreply@github.com 2024-05-18T18:55:54+03:00 ci : re-enable sanitizer runs (#7358) 511182eabb36f6ec9776e2b3c4d7e16d93d0ac0d 133d99c59980139f5bb75922c8b5fca67d7ba9b8 Georgi Gerganov ggerganov@gmail.com 2024-05-18T13:40:39+03:00 GitHub noreply@github.com 2024-05-18T20:40:39+10:00 android : use "ci-android" branch for CI (#7341) 133d99c59980139f5bb75922c8b5fca67d7ba9b8 cb42c294279bc4a0a4e926a7b5a5568049f12fa7 Johannes Gäßler johannesg@5d6.de 2024-05-18T12:36:25+02:00 GitHub noreply@github.com 2024-05-18T12:36:25+02:00 CUDA: deduplicate FlashAttention code (#7352) cb42c294279bc4a0a4e926a7b5a5568049f12fa7 d233b507cd19fcc2d8d8963ecc6a3eb7a33f2ecc Johannes Gäßler johannesg@5d6.de 2024-05-18T11:10:47+02:00 GitHub noreply@github.com 2024-05-18T11:10:47+02:00 server: correct --threads documentation [no ci] (#7362) d233b507cd19fcc2d8d8963ecc6a3eb7a33f2ecc 0f98acfac6cc561dc57586bfff778405e42b576b Engininja2 139037756+Engininja2@users.noreply.github.com 2024-05-18T02:05:17-06:00 GitHub noreply@github.com 2024-05-18T10:05:17+02:00 cuda : add half2 __shfl_xor() for ROCm 5.5 (#7263) 0f98acfac6cc561dc57586bfff778405e42b576b ca57e0f35e33f714b9a6c2c4482b87bfe059c819 Steffen Röcker sroecker@gmail.com 2024-05-18T10:04:55+02:00 GitHub noreply@github.com 2024-05-18T11:04:55+03:00 llama : add support for larger Granite Code Models (20B, 34B) (#7324) ca57e0f35e33f714b9a6c2c4482b87bfe059c819 c1b295eea5c49887a066559527a74e8b94fe9db0 strawberrymelonpanda 152940198+strawberrymelonpanda@users.noreply.github.com 2024-05-18T00:57:08-07:00 GitHub noreply@github.com 2024-05-18T10:57:08+03:00 perplexity : ndot progress and show stats with < 100 tasks (#7348) c1b295eea5c49887a066559527a74e8b94fe9db0 de731963441ff128248259e1b99573d75264d210 0cc4m picard12@live.de 2024-05-18T08:10:58+02:00 GitHub noreply@github.com 2024-05-18T08:10:58+02:00 Update and fix Vulkan soft_max and argsort implementations (#7237) de731963441ff128248259e1b99573d75264d210 b49a13dd2fa9c94c2c19a8c248bb7fa45499f9a8 Brian mofosyne@gmail.com 2024-05-18T16:04:23+10:00 GitHub noreply@github.com 2024-05-18T16:04:23+10:00 github-actions-labeler: initial commit (#7330) b49a13dd2fa9c94c2c19a8c248bb7fa45499f9a8 05834841dcb4f922983ea976539c70472272df9a Georgi Gerganov ggerganov@gmail.com 2024-05-18T08:46:20+03:00 GitHub noreply@github.com 2024-05-18T08:46:20+03:00 convert : fix set_vocab_sentencepiece (#6866) 05834841dcb4f922983ea976539c70472272df9a ef277de2add255a08b2b909ebfbf70364d1f4dc4 slaren slarengh@gmail.com 2024-05-18T02:39:54+02:00 GitHub noreply@github.com 2024-05-18T02:39:54+02:00 ggml : fix quants nans when all the group weights are very close to zero (#7313) ef277de2add255a08b2b909ebfbf70364d1f4dc4 b43272afa29a64dcb8bcf26a96a05bac40792b92 Engininja2 139037756+Engininja2@users.noreply.github.com 2024-05-17T18:39:25-06:00 GitHub noreply@github.com 2024-05-18T02:39:25+02:00 cmake : fix typo in AMDGPU_TARGETS (#7356) b43272afa29a64dcb8bcf26a96a05bac40792b92 0fc1e820a9900a3dd08ddd3c6abe6604c53b689b jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-05-18T01:09:13+02:00 GitHub noreply@github.com 2024-05-18T01:09:13+02:00 Unicode codepoint flags for custom regexs (#7245) 0fc1e820a9900a3dd08ddd3c6abe6604c53b689b 82ca83db3c8d45df559c03a4225b6eb34808a2db Johannes Gäßler johannesg@5d6.de 2024-05-17T18:54:52+02:00 GitHub noreply@github.com 2024-05-17T18:54:52+02:00 CUDA: faster large batch FA without tensor cores (#7314) 82ca83db3c8d45df559c03a4225b6eb34808a2db f4bd8b3d260bb09491ba63c77ab7012b744362ef Gavin Zhao gavinzhaojw@protonmail.com 2024-05-17T11:03:03-04:00 GitHub noreply@github.com 2024-05-17T17:03:03+02:00 ROCm: use native CMake HIP support (#5966) f4bd8b3d260bb09491ba63c77ab7012b744362ef 51e9d02599336e62948d29f1d6c05addeb921ac2 Radoslav Gerganov rgerganov@gmail.com 2024-05-17T17:25:44+03:00 GitHub noreply@github.com 2024-05-17T17:25:44+03:00 rpc : set SO_REUSEADDR for the server socket (#7320) 51e9d02599336e62948d29f1d6c05addeb921ac2 d273c1402b25086fd91aef2467ac13f2e49fa0ea Brian mofosyne@gmail.com 2024-05-17T22:40:14+10:00 GitHub noreply@github.com 2024-05-17T22:40:14+10:00 Added a single test function script and fix debug-test.sh to be more robust (#7279) d273c1402b25086fd91aef2467ac13f2e49fa0ea 27b040691cbe45314147c2745e891a38e9c048d4 Aarni Koskela akx@iki.fi 2024-05-17T15:11:45+03:00 GitHub noreply@github.com 2024-05-17T15:11:45+03:00 py : convert-hf-to-gguf-update improvements (#7340) 27b040691cbe45314147c2745e891a38e9c048d4 29c60d8cddcfd14fa8a6bf023a6c4eb8692c76ba fairydreaming 166155368+fairydreaming@users.noreply.github.com 2024-05-17T13:24:38+02:00 GitHub noreply@github.com 2024-05-17T14:24:38+03:00 llama : use n_embd_head_v when reshaping kqv (#7327) 29c60d8cddcfd14fa8a6bf023a6c4eb8692c76ba 359cbe3f46c90ce6f5151005e411b8fb74f8139e Johannes Gäßler johannesg@5d6.de 2024-05-17T09:59:57+02:00 GitHub noreply@github.com 2024-05-17T09:59:57+02:00 tokenization: add warning for double BOS (#7332) 359cbe3f46c90ce6f5151005e411b8fb74f8139e e18bc6aaf3b547890609ed254ee5248e720e5840 Herman Semenov GermanAizek@yandex.ru 2024-05-17T07:08:49Z GitHub noreply@github.com 2024-05-17T10:08:49+03:00 ggml-quants, llama : removed excess checks (#7274) e18bc6aaf3b547890609ed254ee5248e720e5840 ee94172d33399d2e814ca05c8a3ff8c523ebb093 amd-lalithnc lalithnc@amd.com 2024-05-17T12:31:58+05:30 GitHub noreply@github.com 2024-05-17T10:01:58+03:00 convert : fix Qwen/Qwen-7b conversion (#7308) ee94172d33399d2e814ca05c8a3ff8c523ebb093 934266c0e0b2aa9781fdba2deb112c161ff038a9 Radoslav Gerganov rgerganov@gmail.com 2024-05-17T10:00:17+03:00 GitHub noreply@github.com 2024-05-17T10:00:17+03:00 server : add support for the RPC backend (#7305) 934266c0e0b2aa9781fdba2deb112c161ff038a9 9c4fdcbec8c7fcc428e723b0d8a1cf1f351ba642 Justine Tunney jtunney@mozilla.com 2024-05-17T02:58:52-04:00 GitHub noreply@github.com 2024-05-17T09:58:52+03:00 ggml : rewrite silu and softmax for cpu (#7154) 9c4fdcbec8c7fcc428e723b0d8a1cf1f351ba642 24ecb58168dce81646c2ed425690a106591c8c6d Leon Knauer git@leonknauer.com 2024-05-17T02:11:03+02:00 GitHub noreply@github.com 2024-05-17T10:11:03+10:00 [Server] Added --verbose option to README [no ci] (#7335) 24ecb58168dce81646c2ed425690a106591c8c6d 9afdffe70ebf3166d429b4434783bb0b7f97bdeb Pierrick Hymbert pierrick.hymbert@gmail.com 2024-05-16T20:43:45+02:00 GitHub noreply@github.com 2024-05-16T20:43:45+02:00 Revert "server bench: fix bench not waiting for model load (#7284)" (#7334) 9afdffe70ebf3166d429b4434783bb0b7f97bdeb 3b3963c55c8332e33533c44b2aa882b0e45f8292 Radoslav Gerganov rgerganov@gmail.com 2024-05-15T16:04:40+03:00 Radoslav Gerganov rgerganov@gmail.com 2024-05-16T12:04:08+03:00 rpc : get available mem for the CPU backend 3b3963c55c8332e33533c44b2aa882b0e45f8292 dda64fc17c97820ea9489eb0cc9ae8b8fdce4926 Radoslav Gerganov rgerganov@gmail.com 2024-05-15T15:29:07+03:00 Radoslav Gerganov rgerganov@gmail.com 2024-05-16T09:58:29+03:00 rpc : add command line arg for specifying backend memory dda64fc17c97820ea9489eb0cc9ae8b8fdce4926 0350f5815218c483fb3026a86adc44a115481625 Jared Van Bortel jared@nomic.ai 2024-05-16T02:15:23-04:00 GitHub noreply@github.com 2024-05-16T16:15:23+10:00 convert : get general.name from model dir, not its parent (#5615) 0350f5815218c483fb3026a86adc44a115481625 ad52d5c259344888b06fd5acd3344c663dd0621d Herman Semenov GermanAizek@yandex.ru 2024-05-16T06:14:24Z GitHub noreply@github.com 2024-05-16T16:14:24+10:00 grammar, json, llama: replace push on emplace if it possible (#7273) ad52d5c259344888b06fd5acd3344c663dd0621d 172b78210aae0e54d3668c5de14200efab9fac23 Vaibhav Srivastav vaibhavs10@gmail.com 2024-05-16T07:38:43+02:00 GitHub noreply@github.com 2024-05-16T15:38:43+10:00 doc: add references to hugging face GGUF-my-repo quantisation web tool. (#7288) 172b78210aae0e54d3668c5de14200efab9fac23 13ad16af1231ab2d245d35df3295bcfa23de1305 Max Krasnyansky quic_maxk@quicinc.com 2024-05-15T22:36:43-07:00 GitHub noreply@github.com 2024-05-16T15:36:43+10:00 ci: fix bin/Release path for windows-arm64 builds (#7317) 13ad16af1231ab2d245d35df3295bcfa23de1305 8f7080bf48828b538bc9387c3d150bbd4fb4cf2d Max Krasnyansky max.krasnyansky@gmail.com 2024-05-15T19:47:36-07:00 GitHub noreply@github.com 2024-05-16T12:47:36+10:00 Add support for properly optimized Windows ARM64 builds with LLVM and MSVC (#7191) 8f7080bf48828b538bc9387c3d150bbd4fb4cf2d e1b40ac3b94824d761b5e26ea1bc5692706029d9 Daniel Bevenius daniel.bevenius@gmail.com 2024-05-15T23:41:03+02:00 GitHub noreply@github.com 2024-05-15T23:41:03+02:00 readme : remove stray double quote (#7310) e1b40ac3b94824d761b5e26ea1bc5692706029d9 dc020985b8755dd6aa93a2f002f43c3ede808cce kunnis kunnis@users.noreply.github.com 2024-05-15T12:59:12-05:00 GitHub noreply@github.com 2024-05-15T19:59:12+02:00 ggml : use dynamic thread scheduling for matrix multiplication (#6915) dc020985b8755dd6aa93a2f002f43c3ede808cce 344f9126cc0d15891fde9472fe40b8572628ad7d agray3 agray3@users.noreply.github.com 2024-05-15T14:44:49+01:00 GitHub noreply@github.com 2024-05-15T15:44:49+02:00 Avoid unnecessarily disabling CUDA graphs (#7302) 344f9126cc0d15891fde9472fe40b8572628ad7d 9a17ab914b0aa7353389c656a3f2a0f086726868 slaren slarengh@gmail.com 2024-05-15T15:08:48+02:00 GitHub noreply@github.com 2024-05-15T15:08:48+02:00 ggml : tag ggml_tensor::backend as deprecated (#7290) 9a17ab914b0aa7353389c656a3f2a0f086726868 ea3b0590ee33d3573eb8ef76f88cc60f36d2a38d AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-05-15T13:26:30+01:00 GitHub noreply@github.com 2024-05-15T17:56:30+05:30 Add missing " (#7303) ea3b0590ee33d3573eb8ef76f88cc60f36d2a38d 29499bb59383c2a8c5d557a90abb08b696cef7f6 dm4 sunrisedm4@gmail.com 2024-05-15T20:01:12+08:00 GitHub noreply@github.com 2024-05-15T15:01:12+03:00 embedding : free the batch after execution (#7297) 29499bb59383c2a8c5d557a90abb08b696cef7f6 48aa8fd1f213a69b41569f809cc954f24dbc4366 Georgi Gerganov ggerganov@gmail.com 2024-05-15T13:23:41+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-15T13:23:41+03:00 sync : ggml 48aa8fd1f213a69b41569f809cc954f24dbc4366 583fd6b000ec9ad1b465b5c98524f4a0ae388077 John Balis phobossystems@gmail.com 2024-05-15T03:52:33-05:00 Georgi Gerganov ggerganov@gmail.com 2024-05-15T13:23:33+03:00 ggml : add `ggml_upscale_ext` (ggml/814) 583fd6b000ec9ad1b465b5c98524f4a0ae388077 9f773486ab78d65f5cca3f7e31c862b7043bf721 Johannes Gäßler johannesg@5d6.de 2024-05-15T08:44:16+02:00 GitHub noreply@github.com 2024-05-15T08:44:16+02:00 server bench: fix bench not waiting for model load (#7284) 9f773486ab78d65f5cca3f7e31c862b7043bf721 e8a7fd4fb06d82f663850c21fcf86c0fb98ad9b4 Georgi Gerganov ggerganov@gmail.com 2024-05-14T19:14:38+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-14T19:14:38+03:00 script : sync ggml-rpc e8a7fd4fb06d82f663850c21fcf86c0fb98ad9b4 a5e3fde8578d54b98d941344a4da150669af200d Georgi Gerganov ggerganov@gmail.com 2024-05-14T19:09:30+03:00 GitHub noreply@github.com 2024-05-14T19:09:30+03:00 metal : support FA without mask + add asserts (#7278) a5e3fde8578d54b98d941344a4da150669af200d f308ea705974dff62a1fe5367d776ad9d5109239 Georgi Gerganov ggerganov@gmail.com 2024-05-14T15:33:16+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-14T19:08:09+03:00 sync : ggml f308ea705974dff62a1fe5367d776ad9d5109239 c3c88f296a72432edb697ac8026dbf2ec18f2b21 Georgi Gerganov ggerganov@gmail.com 2024-05-13T11:01:07+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-14T19:08:09+03:00 metal : tune soft_max number of threads (whisper/0) c3c88f296a72432edb697ac8026dbf2ec18f2b21 182adefcf36fc5f4263082ff032c0796fda65578 Georgi Gerganov ggerganov@gmail.com 2024-05-12T20:36:31+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-14T19:08:09+03:00 ggml : try fix ppc64 (whisper/0) 182adefcf36fc5f4263082ff032c0796fda65578 0d26d8ccd8caebab75af697c0275f599075fdacf Przemysław Pawełczyk przemoc@gmail.com 2024-05-08T17:33:43+02:00 Georgi Gerganov ggerganov@gmail.com 2024-05-14T19:08:09+03:00 ggml : expose SSE3 and SSSE3 for MSVC when AVX is available (whisper/2128) 0d26d8ccd8caebab75af697c0275f599075fdacf 4f0263633b40e94e8b69fd6e7e4395cfedfd5c12 Hong Bo PENG penghb@cn.ibm.com 2024-05-12T17:17:18+08:00 Georgi Gerganov ggerganov@gmail.com 2024-05-14T19:08:09+03:00 ggml : optimize for ppc64le using VSX intrinsics (ggml/784) 4f0263633b40e94e8b69fd6e7e4395cfedfd5c12 1265c670fd8e41e1947352c96c5179adda97fb2c Steve Grubb ausearch.1@gmail.com 2024-05-14T10:11:24-04:00 GitHub noreply@github.com 2024-05-14T16:11:24+02:00 server: free sampling contexts on exit (#7264) 1265c670fd8e41e1947352c96c5179adda97fb2c 5e31828d3e35c76ecfee665bc23771a4bec1d130 Brian mofosyne@gmail.com 2024-05-14T23:10:39+10:00 GitHub noreply@github.com 2024-05-14T16:10:39+03:00 Revert "move ndk code to a new library (#6951)" (#7282) 5e31828d3e35c76ecfee665bc23771a4bec1d130 541600201e6480f54ae09e58d16b154d4b4b331d Radoslav Gerganov rgerganov@gmail.com 2024-05-14T14:27:19+03:00 GitHub noreply@github.com 2024-05-14T14:27:19+03:00 ggml : add RPC backend (#6829) 541600201e6480f54ae09e58d16b154d4b4b331d efc8f767c8c8c749a245dd96ad4e2f37c164b54c slaren slarengh@gmail.com 2024-05-14T09:33:42+02:00 GitHub noreply@github.com 2024-05-14T17:33:42+10:00 llama : disable pipeline parallelism with nkvo (#7265) efc8f767c8c8c749a245dd96ad4e2f37c164b54c e0f556186b6e1f2b7032a1479edf5e89e2b1bd86 Elton Kola eltonkola@gmail.com 2024-05-14T03:30:30-04:00 GitHub noreply@github.com 2024-05-14T17:30:30+10:00 move ndk code to a new library (#6951) e0f556186b6e1f2b7032a1479edf5e89e2b1bd86 27f65d6267cf22a44c5ccefa7765d53a05bd1259 Haggai Nuchi h.nuchi@gmail.com 2024-05-13T22:25:56-07:00 GitHub noreply@github.com 2024-05-14T15:25:56+10:00 Add left recursion check: quit early instead of going into an infinite loop (#7083) 27f65d6267cf22a44c5ccefa7765d53a05bd1259 ee52225067622babc277371511b8124884e1c797 Ryuei louixs@users.noreply.github.com 2024-05-14T14:20:47+09:00 GitHub noreply@github.com 2024-05-14T15:20:47+10:00 docs: Fix typo and update description for --embeddings flag (#7026) ee52225067622babc277371511b8124884e1c797 614d3b914e1c3e02596f869649eb4f1d3b68614d compilade git@compilade.net 2024-05-13T14:10:51-04:00 GitHub noreply@github.com 2024-05-13T14:10:51-04:00 convert-hf : support direct Q8_0 conversion (#7234) 614d3b914e1c3e02596f869649eb4f1d3b68614d 30e70334f71b3bd115024affcf98cac3d79aaa95 Georgi Gerganov ggerganov@gmail.com 2024-05-13T17:15:15+03:00 GitHub noreply@github.com 2024-05-13T17:15:15+03:00 llama : less KV padding when FA is off (#7257) 30e70334f71b3bd115024affcf98cac3d79aaa95 1c570d8beeebad95872dc738ea542a4a0022f78a k.h.lai adrian.k.h.lai@outlook.com 2024-05-13T22:02:36+08:00 GitHub noreply@github.com 2024-05-14T00:02:36+10:00 llava-cli: fix base64 prompt (#7248) 1c570d8beeebad95872dc738ea542a4a0022f78a 948f4ec7c5bff92b18e63303f2b2d1645bccd943 Johannes Gäßler johannesg@5d6.de 2024-05-13T13:03:27+02:00 GitHub noreply@github.com 2024-05-13T13:03:27+02:00 perplexity: add BF16 vs. FP16 results (#7150) 948f4ec7c5bff92b18e63303f2b2d1645bccd943 9aa672490c848e45eaa704a554e0f1f6df995fc8 Neo Zhang 14088817+arthw@users.noreply.github.com 2024-05-13T18:11:26+08:00 GitHub noreply@github.com 2024-05-13T18:11:26+08:00 [SYCL] rm wait() (#7233) 9aa672490c848e45eaa704a554e0f1f6df995fc8 b1f8af1886e8187db6bb2a9b87cfc1c0f175f629 Joan Fontanals joan.fontanals.martinez@jina.ai 2024-05-13T10:35:14+02:00 GitHub noreply@github.com 2024-05-13T11:35:14+03:00 llama : rename jina tokenizers to v2 (#7249) b1f8af1886e8187db6bb2a9b87cfc1c0f175f629 e586ee42595500c53938e937b6b6ad5353ad76dc Brian mofosyne@gmail.com 2024-05-13T12:56:47+10:00 GitHub noreply@github.com 2024-05-13T12:56:47+10:00 convert.py: Outfile default name change and additional metadata support (#4858) e586ee42595500c53938e937b6b6ad5353ad76dc cbf75894d256f1861f6409565db599365de3d4b8 Benjamin Findley 39356821+Kartoffelsaft@users.noreply.github.com 2024-05-12T19:40:08-07:00 GitHub noreply@github.com 2024-05-13T12:40:08+10:00 change default temperature of OAI compat API from 0 to 1 (#7226) cbf75894d256f1861f6409565db599365de3d4b8 0d5cef78aeafae4d4e6d56e2d4bcda771af58cc9 Neo Zhang 14088817+arthw@users.noreply.github.com 2024-05-13T08:04:29+08:00 GitHub noreply@github.com 2024-05-13T08:04:29+08:00 [SYCL] Add oneapi runtime dll files to win release package (#7241) 0d5cef78aeafae4d4e6d56e2d4bcda771af58cc9 dc685be46622a8fabfd57cfa804237c8f15679b8 Neo Zhang 14088817+arthw@users.noreply.github.com 2024-05-13T08:02:55+08:00 GitHub noreply@github.com 2024-05-13T08:02:55+08:00 [SYCL] update CI with oneapi 2024.1 (#7235) dc685be46622a8fabfd57cfa804237c8f15679b8 6f1b63606fc68a09d62d1d74dbd156c35219026d Johannes Gäßler johannesg@5d6.de 2024-05-12T19:40:45+02:00 GitHub noreply@github.com 2024-05-12T19:40:45+02:00 CUDA: add FP32 FlashAttention vector kernel (#7188) 6f1b63606fc68a09d62d1d74dbd156c35219026d b228aba91ac2cd9eb90e9d423ba1d0d20e0117e2 Georgi Gerganov ggerganov@gmail.com 2024-05-12T18:30:23+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-12T18:30:23+03:00 cmake : fix version cmp (#7227) b228aba91ac2cd9eb90e9d423ba1d0d20e0117e2 7bd4ffb78062587e4012a1c24186223f09b1bc70 slaren slarengh@gmail.com 2024-05-12T02:29:33+02:00 GitHub noreply@github.com 2024-05-12T02:29:33+02:00 remove convert-lora-to-ggml.py (#7204) 7bd4ffb78062587e4012a1c24186223f09b1bc70 1622ac023f42e5e01c163321cd98c6596aa9402d Georgi Gerganov ggerganov@gmail.com 2024-05-11T21:36:20+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-11T21:38:13+03:00 metal : fix warnings (skipme) (#0) 1622ac023f42e5e01c163321cd98c6596aa9402d 6aeff24f8b91e145e92d17ec7ce3adc4ef60b8e9 Georgi Gerganov ggerganov@gmail.com 2024-05-11T21:35:05+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-11T21:35:05+03:00 sync : ggml 6aeff24f8b91e145e92d17ec7ce3adc4ef60b8e9 325756d28df7d018a7bac424e1b3bc8acb4ecf07 Georgi Gerganov ggerganov@gmail.com 2024-05-11T16:57:53+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-11T21:34:21+03:00 metal : fix indent (ggml/0) 325756d28df7d018a7bac424e1b3bc8acb4ecf07 fed0108491a3a3cbec6c6480dc8667ffff9d7659 Georgi Gerganov ggerganov@gmail.com 2024-05-11T16:25:50+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-11T21:33:08+03:00 ggml : resolve merge (ggml/0) fed0108491a3a3cbec6c6480dc8667ffff9d7659 72c177c1f6c16693eee319d4ebd4eaab5e630dd2 Josh Ramer josh.ramer@icloud.com 2024-05-11T12:26:35-05:00 GitHub noreply@github.com 2024-05-12T03:26:35+10:00 Scripting & documenting debugging one test without anything else in the loop. (#7096) 72c177c1f6c16693eee319d4ebd4eaab5e630dd2 5a419926b0c4efab0531401aea91522aaea9fd07 Xuan Son Nguyen thichthat@gmail.com 2024-05-11T17:28:10+02:00 GitHub noreply@github.com 2024-05-11T17:28:10+02:00 fix system prompt handling (#7153) 5a419926b0c4efab0531401aea91522aaea9fd07 fae9d234b6606693704eca62fe4aefbb6c6abb45 compilade git@compilade.net 2024-05-11T11:06:26-04:00 GitHub noreply@github.com 2024-05-11T11:06:26-04:00 convert-hf : support bfloat16 conversion (#7158) fae9d234b6606693704eca62fe4aefbb6c6abb45 f5ef34e428f3886544590ecb2d532e4d333c114c Georgi Gerganov ggerganov@gmail.com 2024-05-11T12:02:39+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-11T15:38:34+03:00 sync : ggml f5ef34e428f3886544590ecb2d532e4d333c114c ef0d5e3ec9f99003af3ff326384816c02850ea3f Justina Cho justcho5@gmail.com 2024-05-01T14:44:26-07:00 Georgi Gerganov ggerganov@gmail.com 2024-05-11T15:38:34+03:00 feat: implemented sigmoid function (ggml/806) ef0d5e3ec9f99003af3ff326384816c02850ea3f 3292733f95d4632a956890a438af5192e7031c12 Borislav Stanimirov b.stanimirov@abv.bg 2024-04-25T17:24:07+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-11T15:38:34+03:00 build: fix and ignore msvc warnings (ggml/805) 3292733f95d4632a956890a438af5192e7031c12 988631335a20d06497f58be0b8ba13adb4323a22 CrispStrobe 154636388+CrispStrobe@users.noreply.github.com 2024-05-11T10:18:35+02:00 GitHub noreply@github.com 2024-05-11T11:18:35+03:00 convert : skip unaccessible HF repos (#7210) 988631335a20d06497f58be0b8ba13adb4323a22 f99e1e456eaf69cc38c1982a2693ce41c0f897ef Steve Grubb ausearch.1@gmail.com 2024-05-11T04:13:02-04:00 GitHub noreply@github.com 2024-05-11T11:13:02+03:00 server : free llama_batch on exit (#7212) f99e1e456eaf69cc38c1982a2693ce41c0f897ef 5ae3426b0b64672991563d4c28b2018b9f961467 Haoxiang Fei tonyfettes@tonyfettes.com 2024-05-11T16:12:06+08:00 GitHub noreply@github.com 2024-05-11T11:12:06+03:00 llama : lookup word in vocab before doing BPE merges (#7193) 5ae3426b0b64672991563d4c28b2018b9f961467 b83cc3f5b303ff30c52874b2d5864dc6385ebf9f Johannes Gäßler johannesg@5d6.de 2024-05-11T10:11:28+02:00 GitHub noreply@github.com 2024-05-11T10:11:28+02:00 server: fix reported top tokens for temperature 0 (#7203) b83cc3f5b303ff30c52874b2d5864dc6385ebf9f 9cb317f77e53067f7a138cc89ef7657148eae8e6 Joan Fontanals jfontanalsmartinez@gmail.com 2024-05-11T09:46:09+02:00 GitHub noreply@github.com 2024-05-11T10:46:09+03:00 llama : add Jina Embeddings architecture (#6826) 9cb317f77e53067f7a138cc89ef7657148eae8e6 e849648888a11de13aaaa4cb2eda3f5a9c7b444d Georgi Gerganov ggerganov@gmail.com 2024-05-11T10:32:41+03:00 GitHub noreply@github.com 2024-05-11T10:32:41+03:00 ggml : full ALiBi support (#7192) e849648888a11de13aaaa4cb2eda3f5a9c7b444d 18e437665ce626dddbd79119aa7498493e7cb13b slaren slarengh@gmail.com 2024-05-10T18:03:54+02:00 GitHub noreply@github.com 2024-05-10T18:03:54+02:00 llama-bench : add pp+tg test type (#7199) 18e437665ce626dddbd79119aa7498493e7cb13b 8c660242d708d3913a2adc2b6e4a9ee9cf5e4ce7 Georgi Gerganov ggerganov@gmail.com 2024-05-10T18:20:10+03:00 GitHub noreply@github.com 2024-05-10T18:20:10+03:00 metal : fix flash attention kernel requirements (#7169) 8c660242d708d3913a2adc2b6e4a9ee9cf5e4ce7 25c6e82e7a1ad25a42b0894e87d9b5c557409516 Georgi Gerganov ggerganov@gmail.com 2024-05-10T17:53:04+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-10T17:53:04+03:00 convert : print "ignore_merges" field 25c6e82e7a1ad25a42b0894e87d9b5c557409516 4e3880978f8b1bf546dd4e6f3b524d6b8739c49c slaren slarengh@gmail.com 2024-05-10T14:28:01+02:00 GitHub noreply@github.com 2024-05-10T14:28:01+02:00 llama : use n_vocab to differentiate between mistral 7B and llama3 8B (#7200) 4e3880978f8b1bf546dd4e6f3b524d6b8739c49c f89fe2732c5709f6e86d5f4aee2e6d2a561f2eb2 Justine Tunney jtunney@mozilla.com 2024-05-10T07:01:08-04:00 GitHub noreply@github.com 2024-05-10T21:01:08+10:00 Fix memory bug in grammar parser (#7194) f89fe2732c5709f6e86d5f4aee2e6d2a561f2eb2 d11afd665241c1b3910ab5f040d0216403019d87 HanishKVC hanishkvc@gmail.com 2024-05-10T15:51:58+05:30 GitHub noreply@github.com 2024-05-10T20:21:58+10:00 Main+: optionally allow special tokens from user in interactive mode (#7097) d11afd665241c1b3910ab5f040d0216403019d87 8c570c9496212073079476651c7517c02581101f Andrei abetlen@gmail.com 2024-05-10T02:41:10-04:00 GitHub noreply@github.com 2024-05-10T09:41:10+03:00 llava : fix moondream support (#7163) 8c570c9496212073079476651c7517c02581101f eaf4bd8b399a6ed4b834f91d22409d2a05c4d266 Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-05-10T01:32:15+01:00 GitHub noreply@github.com 2024-05-10T08:32:15+08:00 Minor arithmetic improvement to mmvq wrapper kernel (#7172) eaf4bd8b399a6ed4b834f91d22409d2a05c4d266 befddd0f15de6efb15d7e7f5b527dfb671f4196f slaren slarengh@gmail.com 2024-05-10T01:04:12+02:00 GitHub noreply@github.com 2024-05-10T01:04:12+02:00 eval-callback : fix conversion to float (#7184) befddd0f15de6efb15d7e7f5b527dfb671f4196f d46dbc76f8770caec0175f1e57777173c70556a0 0cc4m picard12@live.de 2024-05-09T20:39:54+02:00 GitHub noreply@github.com 2024-05-09T20:39:54+02:00 Vulkan Bugfixes and Improvements (#7084) d46dbc76f8770caec0175f1e57777173c70556a0 0961d866044143eefec5b525e991611f73fd4f6e Georgi Gerganov ggerganov@gmail.com 2024-05-09T16:40:42+03:00 GitHub noreply@github.com 2024-05-09T16:40:42+03:00 readme : add scheduled server workflow status badge 0961d866044143eefec5b525e991611f73fd4f6e 43248e559472556f368988575d9fba906b3eb139 l3utterfly gc.pthzfoldr@gmail.com 2024-05-09T22:32:40+09:00 GitHub noreply@github.com 2024-05-09T16:32:40+03:00 readme : add app (#6371) 43248e559472556f368988575d9fba906b3eb139 a743d76a01f23038b2c85af1e9048ee836767b44 jaime-m-p 167997752+jaime-m-p@users.noreply.github.com 2024-05-09T15:30:44+02:00 GitHub noreply@github.com 2024-05-09T23:30:44+10:00 llama3 custom regex split (#6965) a743d76a01f23038b2c85af1e9048ee836767b44 f31ec120bc36c6270e4948e6a065a7c4cfa0c404 Johannes Gäßler johannesg@5d6.de 2024-05-09T14:32:02+02:00 GitHub noreply@github.com 2024-05-09T14:32:02+02:00 CUDA: generalize FP16 fattn vec kernel (#7061) f31ec120bc36c6270e4948e6a065a7c4cfa0c404 fd9f92b154850014146f61717cd292a59a5cee5a Galunid karolek1231456@gmail.com 2024-05-09T14:13:05+02:00 GitHub noreply@github.com 2024-05-09T14:13:05+02:00 Add warning if token is invalid (#7173) fd9f92b154850014146f61717cd292a59a5cee5a 22842164bcae3251b81ad9e497a16ef66833cb9e Daniel Bevenius daniel.bevenius@gmail.com 2024-05-09T13:03:29+02:00 GitHub noreply@github.com 2024-05-09T14:03:29+03:00 llama : update llama_timings.n_p_eval setting (#7160) 22842164bcae3251b81ad9e497a16ef66833cb9e 47345248827f426038098d016ed11975021b4919 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-05-09T12:56:00+02:00 GitHub noreply@github.com 2024-05-09T13:56:00+03:00 gguf-py : add special token modification capability (#7166) 47345248827f426038098d016ed11975021b4919 07cd41d0965829463eff73eda3348aedbfd3a444 Albert Jin albert.jin@gmail.com 2024-05-09T17:34:37+08:00 GitHub noreply@github.com 2024-05-09T12:34:37+03:00 opencl : alignment size converted from bits to bytes (#7090) 07cd41d0965829463eff73eda3348aedbfd3a444 4426e2987b566f09c7aa96ada9706cc778637620 Ahmet Zeer ahmed.zeer@std.yildiz.edu.tr 2024-05-09T11:16:45+03:00 GitHub noreply@github.com 2024-05-09T10:16:45+02:00 TypoFix (#7162) 4426e2987b566f09c7aa96ada9706cc778637620 f98eb31c517c95960df1d0abc48002787f145f3b Jared Van Bortel jared@nomic.ai 2024-05-08T19:55:32-04:00 GitHub noreply@github.com 2024-05-08T19:55:32-04:00 cmake : fix typo (#7151) f98eb31c517c95960df1d0abc48002787f145f3b bc4bba364fb96d908f2698e908648df5e6f55e02 compilade git@compilade.net 2024-05-08T18:16:38-04:00 GitHub noreply@github.com 2024-05-08T18:16:38-04:00 convert-hf : save memory with lazy evaluation (#7075) bc4bba364fb96d908f2698e908648df5e6f55e02 c12452c7aec8a02264afc00196a13caa591a13ac agray3 agray3@users.noreply.github.com 2024-05-08T21:55:49+01:00 GitHub noreply@github.com 2024-05-08T22:55:49+02:00 Introduction of CUDA Graphs to LLama.cpp (#6766) c12452c7aec8a02264afc00196a13caa591a13ac 9da243b36ac0b9d609adfaaa4c8f1cc8c592f737 Johannes Gäßler johannesg@5d6.de 2024-05-08T21:53:08+02:00 GitHub noreply@github.com 2024-05-08T21:53:08+02:00 JSON: [key] -> .at(key), assert() -> GGML_ASSERT (#7143) 9da243b36ac0b9d609adfaaa4c8f1cc8c592f737 bd1871fa2b1bf8a081b43ba9bc85f8ffd46fac46 Georgi Gerganov ggerganov@gmail.com 2024-05-08T22:14:39+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-08T22:14:39+03:00 Revert "llava : add support for moondream vision language model (#6899)" bd1871fa2b1bf8a081b43ba9bc85f8ffd46fac46 26458af1d63c85195cd96cd1673051e332d06d30 JohnnyB jboero@users.noreply.github.com 2024-05-08T20:12:06+01:00 GitHub noreply@github.com 2024-05-08T22:12:06+03:00 server : add themes + favicon (#6848) 26458af1d63c85195cd96cd1673051e332d06d30 83330d8cd6491e53e1aca4c5dfc47e039b3c04ff Gilad S giladgd@users.noreply.github.com 2024-05-08T22:08:10+03:00 GitHub noreply@github.com 2024-05-08T22:08:10+03:00 metal : use `vm_allocate` instead of `posix_memalign` on macOS (#7078) 83330d8cd6491e53e1aca4c5dfc47e039b3c04ff 465263d0cf1e8f8bc41948332dbd009d27a68590 Dawid Potocki github@dawidpotocki.com 2024-05-09T02:32:32+12:00 GitHub noreply@github.com 2024-05-08T17:32:32+03:00 main : add --conversation / -cnv flag (#7108) 465263d0cf1e8f8bc41948332dbd009d27a68590 911b3900dded9a1cfe0f0e41b82c7a29baf3a217 Eve 139727413+netrunnereve@users.noreply.github.com 2024-05-08T14:29:23Z GitHub noreply@github.com 2024-05-08T17:29:23+03:00 sgemm : AVX Q4_0 and Q8_0 (#6891) 911b3900dded9a1cfe0f0e41b82c7a29baf3a217 ad211edef5db1f1fb955874b7ca6a67bd0c88708 Johan JohanAR@users.noreply.github.com 2024-05-08T14:27:58+02:00 GitHub noreply@github.com 2024-05-08T15:27:58+03:00 server : add_special option for tokenize endpoint (#7059) ad211edef5db1f1fb955874b7ca6a67bd0c88708 229ffff872f8ad0d21c997d18ee7a23692ae60a0 20kdc asdd2808@gmail.com 2024-05-08T13:22:32+01:00 GitHub noreply@github.com 2024-05-08T15:22:32+03:00 convert.py : --vocab-only generates false but valid params (#7027) 229ffff872f8ad0d21c997d18ee7a23692ae60a0 1fd9c1741d864d01cd7ec6d67227b92d7bfabf22 Ren Xuancheng jklj077@users.noreply.github.com 2024-05-08T20:06:43+08:00 GitHub noreply@github.com 2024-05-08T15:06:43+03:00 llama : add BPE pre-tokenization for Qwen2 (#7114) 1fd9c1741d864d01cd7ec6d67227b92d7bfabf22 4cd621c26de2095cd7c4464bdec5fe2e696ef3f3 Xuan Son Nguyen thichthat@gmail.com 2024-05-08T13:24:14+02:00 GitHub noreply@github.com 2024-05-08T13:24:14+02:00 clean up json_value & server_log (#7142) 4cd621c26de2095cd7c4464bdec5fe2e696ef3f3 7e0b6a7b3ba94ff624dc27c1e0e735fded8819b8 DAN™ dranger003@gmail.com 2024-05-08T06:43:23-04:00 GitHub noreply@github.com 2024-05-08T13:43:23+03:00 convert : add BPE pre-tokenization for DBRX (#7132) 7e0b6a7b3ba94ff624dc27c1e0e735fded8819b8 acdce3cdef6fc2f0b7b5623231fd7762c0884d1c Georgi Gerganov ggerganov@gmail.com 2024-05-08T12:47:07+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-08T12:47:07+03:00 py : also print the normalizers acdce3cdef6fc2f0b7b5623231fd7762c0884d1c 3855416027cb25d9a708ffa5581cf503a87856a6 Brian mofosyne@gmail.com 2024-05-08T18:54:39+10:00 GitHub noreply@github.com 2024-05-08T10:54:39+02:00 compare-llama-bench.py: add missing basicConfig (#7138) 3855416027cb25d9a708ffa5581cf503a87856a6 c0e6fbf8c380718102bd25fcb8d2e55f8f9480d1 Justine Tunney jtunney@mozilla.com 2024-05-08T02:30:09-04:00 GitHub noreply@github.com 2024-05-08T09:30:09+03:00 ggml : introduce bfloat16 support (#6412) c0e6fbf8c380718102bd25fcb8d2e55f8f9480d1 c780e75305dba1f67691a8dc0e8bc8425838a452 Georgi Gerganov ggerganov@gmail.com 2024-05-08T09:14:50+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-08T09:14:50+03:00 metal : fix unused warning c780e75305dba1f67691a8dc0e8bc8425838a452 48b2f9c1fc71ab7df5432be2ed9fa7cdf5e8405e Jeximo jeximo@gmail.com 2024-05-07T21:26:43-03:00 GitHub noreply@github.com 2024-05-08T02:26:43+02:00 Further tidy on Android instructions README.md (#7077) 48b2f9c1fc71ab7df5432be2ed9fa7cdf5e8405e af0a5b616359809ce886ea433acedebb39b12969 jukofyork 69222624+jukofyork@users.noreply.github.com 2024-05-08T01:24:16+01:00 GitHub noreply@github.com 2024-05-08T02:24:16+02:00 Fixed save_imatrix to match old behaviour for MoE (#7099) af0a5b616359809ce886ea433acedebb39b12969 b6aa6702030320a3d5fbc2508307af0d7c947e40 Johannes Gäßler johannesg@5d6.de 2024-05-07T23:07:58+02:00 GitHub noreply@github.com 2024-05-07T23:07:58+02:00 server: fix incorrectly reported token probabilities (#7125) b6aa6702030320a3d5fbc2508307af0d7c947e40 260b7c65296fba0568eeb1ff05244ea0be206b54 nopperl 54780682+nopperl@users.noreply.github.com 2024-05-07T19:39:43Z GitHub noreply@github.com 2024-05-07T21:39:43+02:00 Fix OLMo HF to GGUF conversion (#6910) 260b7c65296fba0568eeb1ff05244ea0be206b54 53d6c52e227dedef347b21e28febcfb9caeecdad Kyle Mistele kyle@mistele.com 2024-05-07T13:44:29-05:00 GitHub noreply@github.com 2024-05-07T21:44:29+03:00 server : update readme with undocumented options (#7013) 53d6c52e227dedef347b21e28febcfb9caeecdad 3af34c1d1b0da47f85b95f60922abeded1cb5d33 Georgi Gerganov ggerganov@gmail.com 2024-05-07T21:43:13+03:00 GitHub noreply@github.com 2024-05-07T21:43:13+03:00 readme : update hot topics 3af34c1d1b0da47f85b95f60922abeded1cb5d33 04976db7a819fcf8bfefbfc09a3344210b79dd27 RhinoDevel RhinoDevel@users.noreply.github.com 2024-05-07T19:51:31+02:00 GitHub noreply@github.com 2024-05-07T20:51:31+03:00 main : update log text (EOS to EOG) (#7104) 04976db7a819fcf8bfefbfc09a3344210b79dd27 947d3ad27d94f1addef76b5d64c314618f063933 omahs 73983677+omahs@users.noreply.github.com 2024-05-07T17:20:33+02:00 GitHub noreply@github.com 2024-05-07T18:20:33+03:00 docs: fix typos (#7124) 947d3ad27d94f1addef76b5d64c314618f063933 858f6b73f6e57a62523d16a955d565254be889b4 Georgi Gerganov ggerganov@gmail.com 2024-05-07T11:08:49+03:00 GitHub noreply@github.com 2024-05-07T11:08:49+03:00 ci : add GG_BUILD_EXTRA_TESTS_0 env (#7098) 858f6b73f6e57a62523d16a955d565254be889b4 b3a995b416e13ae3123a117a743e11d0ede0ca4c William Tambellini william.tambellini@gmail.com 2024-05-06T11:12:14-07:00 GitHub noreply@github.com 2024-05-06T20:12:14+02:00 Add an option to build without CUDA VMM (#7067) b3a995b416e13ae3123a117a743e11d0ede0ca4c bcdee0daa7c5e8e086b719e5eb4073b00df70e01 Georgi Gerganov ggerganov@gmail.com 2024-05-06T18:36:06+03:00 GitHub noreply@github.com 2024-05-06T08:36:06-07:00 flake.lock: Update (#7079) bcdee0daa7c5e8e086b719e5eb4073b00df70e01 628b299106d1e9476fdecb3cbe546bf5c60f1b89 Georgi Gerganov ggerganov@gmail.com 2024-05-06T09:31:30+03:00 Georgi Gerganov ggerganov@gmail.com 2024-05-06T09:31:30+03:00 minor : fix trailing whitespace 628b299106d1e9476fdecb3cbe546bf5c60f1b89 8f8acc8683a00ce40fa5a81161a079d2167126e6 kunnis kunnis@users.noreply.github.com 2024-05-05T07:17:47-05:00 GitHub noreply@github.com 2024-05-05T14:17:47+02:00 Adding support for the --numa argument for llama-bench. (#7080) 8f8acc8683a00ce40fa5a81161a079d2167126e6 ca3632602091e959ed2ad4c09c67a7c790b10d31 Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-05-05T13:38:55+02:00 GitHub noreply@github.com 2024-05-05T13:38:55+02:00 Disable benchmark on forked repo (#7034) ca3632602091e959ed2ad4c09c67a7c790b10d31 889bdd76866ea31a7625ec2dcea63ff469f3e981 Lyle Dean dean@lyle.dev 2024-05-05T06:21:46+01:00 GitHub noreply@github.com 2024-05-05T08:21:46+03:00 readme : add note that LLaMA 3 is not supported with convert.py (#7065) 889bdd76866ea31a7625ec2dcea63ff469f3e981 6fbd43221167bf96112f899daf22c127b282cbcf DAN™ dranger003@gmail.com 2024-05-05T01:19:30-04:00 GitHub noreply@github.com 2024-05-05T08:19:30+03:00 command-r : add BPE pre-tokenization (#7063) 6fbd43221167bf96112f899daf22c127b282cbcf 842500144ee02c8b0a46d2cc43880f8d80998fa5 Brian mofosyne@gmail.com 2024-05-05T15:07:48+10:00 GitHub noreply@github.com 2024-05-05T08:07:48+03:00 py : logging and flake8 suppression refactoring (#7081) 842500144ee02c8b0a46d2cc43880f8d80998fa5 cf768b7e71cbcc9886c753ae963c2b68893d02e4 Xuan Son Nguyen thichthat@gmail.com 2024-05-04T18:56:22+02:00 GitHub noreply@github.com 2024-05-04T18:56:22+02:00 gguf-split: add --no-tensor-first-split (#7072) cf768b7e71cbcc9886c753ae963c2b68893d02e4 fcd84a0f5a584ab5271745d7ffef21c8a6bc7b0c Jeximo jeximo@gmail.com 2024-05-04T13:10:15-03:00 GitHub noreply@github.com 2024-05-04T18:10:15+02:00 Tidy Android Instructions README.md (#7016) fcd84a0f5a584ab5271745d7ffef21c8a6bc7b0c 03fb8a002df2e96104f9e06de9c78d2a8ed91e92 viric viric@viric.name 2024-05-04T15:26:53+02:00 GitHub noreply@github.com 2024-05-04T15:26:53+02:00 Fix Linux /sys cpu path to guess number of cores (#7064) 03fb8a002df2e96104f9e06de9c78d2a8ed91e92 92139b90af4841d7fd060b526bdd443b621770ff maor-ps 154728172+maor-ps@users.noreply.github.com 2024-05-04T12:06:40+03:00 GitHub noreply@github.com 2024-05-04T11:06:40+02:00 If first token generated from the server is the stop word the server will crash (#7038) 92139b90af4841d7fd060b526bdd443b621770ff a2ac89d6efb41b535778bfeaecaae8fe295b6ed3 Georgi Gerganov ggerganov@gmail.com 2024-05-04T08:32:32+03:00 GitHub noreply@github.com 2024-05-04T08:32:32+03:00 tests : add test-tokenizer-0.sh + fix some tokenizers (#7036) a2ac89d6efb41b535778bfeaecaae8fe295b6ed3 433def286e98751bf17db75dce53847d075c0be5 Brian mofosyne@gmail.com 2024-05-04T05:36:41+10:00 GitHub noreply@github.com 2024-05-03T22:36:41+03:00 convert.py : add python logging instead of print() (#6511) 433def286e98751bf17db75dce53847d075c0be5 60325fa56f61c228464c9f065db3aa6a61f2156e Daniel Bevenius daniel.bevenius@gmail.com 2024-05-03T15:24:30+02:00 GitHub noreply@github.com 2024-05-03T15:24:30+02:00 llama : rename ctx to user_data in progress_callback (#7045) 60325fa56f61c228464c9f065db3aa6a61f2156e 6ecf3189e00a1e8e737a78b6d10e1d7006e050a2 Bartowski ckealty1182@gmail.com 2024-05-02T19:49:09-04:00 GitHub noreply@github.com 2024-05-03T01:49:09+02:00 Remove .attention from skipped tensors to match more accurately (#7051) 6ecf3189e00a1e8e737a78b6d10e1d7006e050a2 b0d943de179ad5dbd83d51f327fb566066f4ccda alwqx kenan3015@gmail.com 2024-05-02T23:56:41+08:00 GitHub noreply@github.com 2024-05-02T11:56:41-04:00 chore: fix typo in llama.cpp (#7032) b0d943de179ad5dbd83d51f327fb566066f4ccda 8d608a81b7bd170f700648f8214e6f3279d4d715 Andrew Downing andrew2085@gmail.com 2024-05-01T17:31:30-04:00 GitHub noreply@github.com 2024-05-01T23:31:30+02:00 Update LOG_IMPL and LOG_TEE_IMPL (#7029) 8d608a81b7bd170f700648f8214e6f3279d4d715 3ea0d36000e2314baba7e9fc6a97f08670a6f7e4 l3utterfly gc.pthzfoldr@gmail.com 2024-05-02T04:27:41+09:00 GitHub noreply@github.com 2024-05-01T22:27:41+03:00 main : fix off by one error for context shift (#6921) 3ea0d36000e2314baba7e9fc6a97f08670a6f7e4 1613ef8d8eb2479ba55c4d598e08c8f3f18a0fed Johannes Gäßler johannesg@5d6.de 2024-05-01T17:52:55+02:00 GitHub noreply@github.com 2024-05-01T17:52:55+02:00 Server: add tests for batch size, different seeds (#6950) 1613ef8d8eb2479ba55c4d598e08c8f3f18a0fed c4ec9c0d3d67e6b33638e6dad86419e6fd5ffe01 Johannes Gäßler johannesg@5d6.de 2024-05-01T14:46:37+02:00 GitHub noreply@github.com 2024-05-01T14:46:37+02:00 CUDA: CUDART < 11.7 workaround for __hmax, __hmax2 (#7019) c4ec9c0d3d67e6b33638e6dad86419e6fd5ffe01 a8f9b076316e16aadd0791015b3bfd446fe1e904 slaren slarengh@gmail.com 2024-05-01T07:13:59+02:00 GitHub noreply@github.com 2024-05-01T08:13:59+03:00 ci : exempt confirmed bugs from being tagged as stale (#7014) a8f9b076316e16aadd0791015b3bfd446fe1e904 f364eb6fb5d46118a76fa045f487318de4c24961 Johannes Gäßler johannesg@5d6.de 2024-04-30T23:36:27+02:00 GitHub noreply@github.com 2024-04-30T23:36:27+02:00 perplexity: more statistics, added documentation (#6936) f364eb6fb5d46118a76fa045f487318de4c24961 77e15bec6217a39be59b9cc83d6b9afb6b0d8167 Kevin Gibbons bakkot@gmail.com 2024-04-30T08:14:02-07:00 GitHub noreply@github.com 2024-04-30T17:14:02+02:00 switch to using localizedDescription (#7010) 77e15bec6217a39be59b9cc83d6b9afb6b0d8167 a68a1e7ed060ee5af2d638585d19e3510ddbf16c Georgi Gerganov ggerganov@gmail.com 2024-04-30T15:52:21+03:00 GitHub noreply@github.com 2024-04-30T15:52:21+03:00 metal : remove deprecated error code (#7008) a68a1e7ed060ee5af2d638585d19e3510ddbf16c 9c67c2773d4b706cf71d70ecf4aa180b62501960 Kevin Gibbons bakkot@gmail.com 2024-04-30T02:34:50-07:00 GitHub noreply@github.com 2024-04-30T12:34:50+03:00 metal : log more info on error (#6987) 9c67c2773d4b706cf71d70ecf4aa180b62501960 952d03dbead16e4dbdd1d3458486340673cc2465 Georgi Gerganov ggerganov@gmail.com 2024-04-30T12:16:08+03:00 GitHub noreply@github.com 2024-04-30T12:16:08+03:00 ggml : add Flash Attention (#5021) 952d03dbead16e4dbdd1d3458486340673cc2465 8843a98c2ba97a25e93319a104f9ddfaf83ce4c4 Georgi Gerganov ggerganov@gmail.com 2024-04-30T11:05:25+03:00 GitHub noreply@github.com 2024-04-30T11:05:25+03:00 convert : use utf8 encoding (#7000) 8843a98c2ba97a25e93319a104f9ddfaf83ce4c4 b8c1476e44cc1f3a1811613f65251cf779067636 Olivier Chafik ochafik@users.noreply.github.com 2024-04-30T00:52:50+01:00 GitHub noreply@github.com 2024-04-30T00:52:50+01:00 Improve usability of --model-url & related flags (#6930) b8c1476e44cc1f3a1811613f65251cf779067636 5539e6fdd1b97e0c37c54d34df67f334fc344a26 Clint Herron hanclinto@gmail.com 2024-04-29T14:40:14-04:00 GitHub noreply@github.com 2024-04-29T14:40:14-04:00 Extending grammar integration tests (#6644) 5539e6fdd1b97e0c37c54d34df67f334fc344a26 b8a7a5a90fd3187175d84227dad705ade395ba46 Daniel Bevenius daniel.bevenius@gmail.com 2024-04-29T19:56:59+02:00 GitHub noreply@github.com 2024-04-29T13:56:59-04:00 main : fix typo in comment in main.cpp (#6985) b8a7a5a90fd3187175d84227dad705ade395ba46 d2c898f746a527f09effb061829e68b2e1812a28 Olivier Chafik ochafik@users.noreply.github.com 2024-04-29T17:02:45+01:00 GitHub noreply@github.com 2024-04-29T17:02:45+01:00 build(cmake): simplify instructions (`cmake -B build && cmake --build build ...`) (#6964) d2c898f746a527f09effb061829e68b2e1812a28 544f1f10adeec3d5ed91c4d3bd3f903904ecea63 Georgi Gerganov ggerganov@gmail.com 2024-04-29T18:36:39+03:00 GitHub noreply@github.com 2024-04-29T18:36:39+03:00 ci : tmp disable gguf-split (#6983) 544f1f10adeec3d5ed91c4d3bd3f903904ecea63 ffe666572f98a686b17a2cd1dbf4c0a982e5ac0a Georgi Gerganov ggerganov@gmail.com 2024-04-29T17:55:02+03:00 GitHub noreply@github.com 2024-04-29T17:55:02+03:00 ggml : fix __MSC_VER -> _MSC_VER (#6977) ffe666572f98a686b17a2cd1dbf4c0a982e5ac0a 24affa7db3c9db148854b0ab4fd63de8bca7d898 cpumaxx 163466046+cpumaxx@users.noreply.github.com 2024-04-29T07:34:24-07:00 GitHub noreply@github.com 2024-04-29T17:34:24+03:00 llava-cli : multiple images (#6969) 24affa7db3c9db148854b0ab4fd63de8bca7d898 f4ab2a41476600a98067a9474ea8f9e6db41bcfa Georgi Gerganov ggerganov@gmail.com 2024-04-29T17:06:19+03:00 GitHub noreply@github.com 2024-04-29T17:06:19+03:00 readme : update hot topics f4ab2a41476600a98067a9474ea8f9e6db41bcfa 3f167476b11efa7ab08f6cacdeb8cab0935c1249 Georgi Gerganov ggerganov@gmail.com 2024-04-29T16:58:41+03:00 GitHub noreply@github.com 2024-04-29T16:58:41+03:00 llama : fix BPE pre-tokenization (#6920) 3f167476b11efa7ab08f6cacdeb8cab0935c1249 3055a4180557c6cbe29eacc8284c9e070ac10eab David Renshaw dwrenshaw@gmail.com 2024-04-29T09:35:45-04:00 GitHub noreply@github.com 2024-04-29T16:35:45+03:00 sampling : use std::random_device{}() for default random seed (#6962) 3055a4180557c6cbe29eacc8284c9e070ac10eab 577277ffd203b190c3dc2ab3e737946dc432132c Christian Zhou-Zheng 59622928+christianazinn@users.noreply.github.com 2024-04-29T09:34:41-04:00 GitHub noreply@github.com 2024-04-29T16:34:41+03:00 convert : fix conversion of some BERT embedding models (#6937) 577277ffd203b190c3dc2ab3e737946dc432132c ca7f29f568803bee4c92d1b3e41c7d721b0dc570 Przemysław Pawełczyk przemoc@gmail.com 2024-04-29T15:08:20+02:00 GitHub noreply@github.com 2024-04-29T16:08:20+03:00 make : change GNU make default CXX from g++ to c++ (#6966) ca7f29f568803bee4c92d1b3e41c7d721b0dc570 c4f708a93f1df5e35167f9313e000d381298be7f Przemysław Pawełczyk przemoc@gmail.com 2024-04-29T14:59:47+02:00 GitHub noreply@github.com 2024-04-29T15:59:47+03:00 ci : add building in MSYS2 environments (Windows) (#6967) c4f708a93f1df5e35167f9313e000d381298be7f e00b4a8f816ebc45b98a46e5f5231359b9a017e0 Johannes Gäßler johannesg@5d6.de 2024-04-29T14:36:22+02:00 GitHub noreply@github.com 2024-04-29T15:36:22+03:00 llama : fix typo LAMMAFILE -> LLAMAFILE (#6974) e00b4a8f816ebc45b98a46e5f5231359b9a017e0 7bb36ccf91b8a2e92b182dd75624f1fd7cb205ac DAN™ dranger003@gmail.com 2024-04-28T18:38:44-04:00 GitHub noreply@github.com 2024-04-29T00:38:44+02:00 Fix more int overflow during quant (PPL/CUDA). (#6563) 7bb36ccf91b8a2e92b182dd75624f1fd7cb205ac ce023f6f2ff34fbe840e32e65d443d2fed7393de Xuan Son Nguyen thichthat@gmail.com 2024-04-28T17:36:18+02:00 GitHub noreply@github.com 2024-04-28T17:36:18+02:00 gguf : enforce that tensor names are unique (#6905) ce023f6f2ff34fbe840e32e65d443d2fed7393de 6e472f58e40cd4acf6023e15c75a2700535c5f0b Neo Zhang 14088817+arthw@users.noreply.github.com 2024-04-28T22:40:31+08:00 GitHub noreply@github.com 2024-04-28T22:40:31+08:00 add device version in device list (#6959) 6e472f58e40cd4acf6023e15c75a2700535c5f0b 4dba7e8114d84241c842b986e008af8b88d1a019 github-actions[bot] github-actions[bot]@users.noreply.github.com 2024-04-28T00:18:27Z Someone newkozlukov@gmail.com 2024-04-28T11:12:50Z flake.lock: Update 4dba7e8114d84241c842b986e008af8b88d1a019 b7368332e24c5b2c8038bf8267f43632783fcc35 mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2024-04-27T21:02:06+02:00 GitHub noreply@github.com 2024-04-27T21:02:06+02:00 Replace "alternative" boolean operator in conditional compilation directive (#6949) b7368332e24c5b2c8038bf8267f43632783fcc35 928e0b7013c862cf10701957b3d654aa70f11bd8 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-27T17:50:48+02:00 GitHub noreply@github.com 2024-04-27T17:50:48+02:00 ci: server: tests python env on github container ubuntu latest / fix n_predict (#6935) 928e0b7013c862cf10701957b3d654aa70f11bd8 0c4d489e29e53589bf13a801fe7c94b7b546d8f6 agray3 agray3@users.noreply.github.com 2024-04-26T19:08:30+01:00 GitHub noreply@github.com 2024-04-26T20:08:30+02:00 Reset schedule earlier to allow overlap with ggml graph computation on device (#6933) 0c4d489e29e53589bf13a801fe7c94b7b546d8f6 017e6999b5184234370b22a2f868e1be911e8d88 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-26T20:06:33+02:00 GitHub noreply@github.com 2024-04-26T20:06:33+02:00 quantize: add imatrix and dataset metadata in GGUF (#6658) 017e6999b5184234370b22a2f868e1be911e8d88 e2764cd7ca1112d9303eba9e81c9935ee67352ff slaren slarengh@gmail.com 2024-04-26T18:39:58+02:00 GitHub noreply@github.com 2024-04-26T18:39:58+02:00 add basic tensor data validation function (#6884) e2764cd7ca1112d9303eba9e81c9935ee67352ff 4b1c3c98b442a4c84a788fff6323f6fa7e678a5b slaren slarengh@gmail.com 2024-04-26T17:07:42+02:00 GitHub noreply@github.com 2024-04-26T18:07:42+03:00 gguf : fix mismatch between alloc and free functions (#6929) 4b1c3c98b442a4c84a788fff6323f6fa7e678a5b bbe3c6e76157a5d806fdc155451f0ca8936248ee Justine Tunney jtunney@mozilla.com 2024-04-26T10:05:33-04:00 GitHub noreply@github.com 2024-04-26T17:05:33+03:00 llamafile : use 64-bit integers in sgemm (#6928) bbe3c6e76157a5d806fdc155451f0ca8936248ee 7f5ff558eed0f732af8f25c2ab0645610bdec80c Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-26T12:27:25+02:00 GitHub noreply@github.com 2024-04-26T12:27:25+02:00 ci: server: fix python installation (#6925) 7f5ff558eed0f732af8f25c2ab0645610bdec80c 9e4e077ec50fde6049b128662c72d37a3c28e34b Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-26T12:15:30+02:00 GitHub noreply@github.com 2024-04-26T12:15:30+02:00 server: stop generation at `n_ctx_train` if `n_predict` is not set (#6638) 9e4e077ec50fde6049b128662c72d37a3c28e34b 83b72cb086ce46a33dececc86bfe4648b6120aa8 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-26T11:11:51+02:00 GitHub noreply@github.com 2024-04-26T11:11:51+02:00 ci: server: fix python installation (#6922) 83b72cb086ce46a33dececc86bfe4648b6120aa8 d4a9afc1009f0da88d04b2c5f672d81d5ae94675 Georgi Gerganov ggerganov@gmail.com 2024-04-26T10:41:53+03:00 GitHub noreply@github.com 2024-04-26T10:41:53+03:00 Merge pull request from GHSA-p5mv-gjc5-mwqv d4a9afc1009f0da88d04b2c5f672d81d5ae94675 7d641c26ac73956a54b204cabefe85c764823678 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-26T09:27:49+02:00 GitHub noreply@github.com 2024-04-26T09:27:49+02:00 ci: server: fix python installation (#6918) 7d641c26ac73956a54b204cabefe85c764823678 5790c8dac1a4f0aa80b4efee3b962d8c04c829e8 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-26T09:26:59+02:00 GitHub noreply@github.com 2024-04-26T09:26:59+02:00 ci: fix concurrency for pull_request_target (#6917) 5790c8dac1a4f0aa80b4efee3b962d8c04c829e8 46e12c4692a37bdd31a0432fc5153d7d22bc7f72 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-26T09:26:16+02:00 GitHub noreply@github.com 2024-04-26T09:26:16+02:00 bench: server add stop word for PHI-2 (#6916) 46e12c4692a37bdd31a0432fc5153d7d22bc7f72 dba497e0c1eff27cf0e85d1d73c86fa08e1b5557 vik vikhyatk@gmail.com 2024-04-25T12:38:31-07:00 GitHub noreply@github.com 2024-04-25T22:38:31+03:00 llava : add support for moondream vision language model (#6899) dba497e0c1eff27cf0e85d1d73c86fa08e1b5557 fa0b4ad25208d5ec2df6b998ccb29b49b249e82c Georgi Gerganov ggerganov@gmail.com 2024-04-25T21:31:17+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-25T21:37:27+03:00 cmake : restore LLAMA_LLAMAFILE_DEFAULT fa0b4ad25208d5ec2df6b998ccb29b49b249e82c d6e1d44f16e5580cf47f13325ff49960bf13ca37 Georgi Gerganov ggerganov@gmail.com 2024-04-25T18:59:51+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-25T18:59:51+03:00 cmake : remove obsolete ANDROID check d6e1d44f16e5580cf47f13325ff49960bf13ca37 853d06ffe26621176d60909a6e3f7c4cd067b305 slaren slarengh@gmail.com 2024-04-25T17:59:03+02:00 GitHub noreply@github.com 2024-04-25T17:59:03+02:00 llama : synchronize before get/set session data (#6911) 853d06ffe26621176d60909a6e3f7c4cd067b305 3fe0596c1817a6114ffffb6dbfd6c36ca7815dc7 Georgi Gerganov ggerganov@gmail.com 2024-04-25T17:06:27+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-25T17:06:27+03:00 ci : tmp disable slow tests 3fe0596c1817a6114ffffb6dbfd6c36ca7815dc7 0ead1f1072fdc70720f92e008a294dc74a826b1d BarfingLemurs 128182951+BarfingLemurs@users.noreply.github.com 2024-04-25T09:52:28-04:00 GitHub noreply@github.com 2024-04-25T16:52:28+03:00 readme : update model list (#6908) 0ead1f1072fdc70720f92e008a294dc74a826b1d 51543729ff5b1361ebfb164875c93dff0850d5fe slaren slarengh@gmail.com 2024-04-25T15:23:47+02:00 GitHub noreply@github.com 2024-04-25T15:23:47+02:00 llama : check that all the tensor data is in the model file (#6885) 51543729ff5b1361ebfb164875c93dff0850d5fe 4ab99d8d4762869506bb675b36501fd7c2af00b2 Georgi Gerganov ggerganov@gmail.com 2024-04-25T15:48:25+03:00 GitHub noreply@github.com 2024-04-25T15:48:25+03:00 ggml : fix redefinition of vaddvq_f32 for 32-bit ARM (#6906) 4ab99d8d4762869506bb675b36501fd7c2af00b2 54770413c484660d021dd51b5dbacab7880b8827 Daniel Bevenius daniel.bevenius@gmail.com 2024-04-25T14:38:14+02:00 GitHub noreply@github.com 2024-04-25T15:38:14+03:00 clip : rename lerp function to avoid conflict (#6894) 54770413c484660d021dd51b5dbacab7880b8827 aa750c1ede6232c91de890a14a7731d6daa2bc8e Georgi Gerganov ggerganov@gmail.com 2024-04-25T15:12:28+03:00 GitHub noreply@github.com 2024-04-25T15:12:28+03:00 ggml : fix MIN / MAX macros (#6904) aa750c1ede6232c91de890a14a7731d6daa2bc8e 1966eb2615242f224bf9ca939db8905ab6a174a0 Georgi Gerganov ggerganov@gmail.com 2024-04-25T14:27:20+03:00 GitHub noreply@github.com 2024-04-25T14:27:20+03:00 tests : minor bash stuff (#6902) 1966eb2615242f224bf9ca939db8905ab6a174a0 784e11dea1f5ce9638851b2b0dddb107e2a609c8 jiez 373447296@qq.com 2024-04-25T18:29:35+08:00 GitHub noreply@github.com 2024-04-25T13:29:35+03:00 quantize : add '--keep-split' to quantize model into shards (#6688) 784e11dea1f5ce9638851b2b0dddb107e2a609c8 b4e4b8a9351d918a56831c73cf9f25c1837b80d1 Johannes Gäßler johannesg@5d6.de 2024-04-24T21:29:13+02:00 GitHub noreply@github.com 2024-04-24T21:29:13+02:00 README: add graphic for matrix multiplication (#6881) b4e4b8a9351d918a56831c73cf9f25c1837b80d1 3fe847b5747676ee1bf90371c46ed0bc66b57240 Douglas Hanley thesecretaryofwar@gmail.com 2024-04-24T08:10:07-05:00 GitHub noreply@github.com 2024-04-24T16:10:07+03:00 llama : add llama_get_pooling_type function (#6862) 3fe847b5747676ee1bf90371c46ed0bc66b57240 37246b1031b1680c0dcaf20aef736d6b446203fa mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2024-04-24T12:54:24+02:00 GitHub noreply@github.com 2024-04-24T13:54:24+03:00 server : do not apply Markdown formatting in code sections (#6850) 37246b1031b1680c0dcaf20aef736d6b446203fa 28103f4832e301a9c84d44ff0df9d75d46ab6c76 Kyle Mistele kyle@mistele.com 2024-04-24T05:15:29-05:00 GitHub noreply@github.com 2024-04-24T13:15:29+03:00 common : revert showing control tokens by default for server (#6860) 28103f4832e301a9c84d44ff0df9d75d46ab6c76 c0d1b3e03e27634ac2871761f5033cf9324d472d Johannes Gäßler johannesg@5d6.de 2024-04-24T11:08:36+02:00 GitHub noreply@github.com 2024-04-24T11:08:36+02:00 Server: fix seed for multiple slots (#6835) c0d1b3e03e27634ac2871761f5033cf9324d472d abd3314064cd3c513f9eef34c3ba6c23a107442c Georgi Gerganov ggerganov@gmail.com 2024-04-24T12:00:07+03:00 GitHub noreply@github.com 2024-04-24T12:00:07+03:00 ggml : move 32-bit arm compat in ggml-impl.h (#6865) abd3314064cd3c513f9eef34c3ba6c23a107442c 3fec68be4e9577fc53158366d3b3af039c17bb1f Tristan Druyen tristan@vault81.mozmail.com 2024-04-24T10:52:37+02:00 GitHub noreply@github.com 2024-04-24T11:52:37+03:00 llama : add phi 3 chat template (#6857) 3fec68be4e9577fc53158366d3b3af039c17bb1f c8297c6af5693555652c40b95974b95d49d2674d Junyang Lin justinlin930319@hotmail.com 2024-04-24T15:16:21+08:00 GitHub noreply@github.com 2024-04-24T10:16:21+03:00 convert : add support of codeqwen due to tokenizer (#6707) c8297c6af5693555652c40b95974b95d49d2674d 4e96a812b3ce7322a29a3008db2ed73d9087b176 liuwei-git 14815172+liuwei-git@users.noreply.github.com 2024-04-24T15:00:37+08:00 GitHub noreply@github.com 2024-04-24T10:00:37+03:00 llama : add phi3 support (#6852) 4e96a812b3ce7322a29a3008db2ed73d9087b176 192090bae47960f0d38d4967abe398a5d190057e Anas Ahouzi 112881240+aahouzi@users.noreply.github.com 2024-04-23T02:53:18+02:00 GitHub noreply@github.com 2024-04-23T08:53:18+08:00 [SYCL] Windows default build instructions without -DLLAMA_SYCL_F16 flag activated (#6767) 192090bae47960f0d38d4967abe398a5d190057e e931888d5024de814ce7119a18d6a959bfff3821 Justine Tunney jtunney@mozilla.com 2024-04-22T15:00:36-04:00 GitHub noreply@github.com 2024-04-22T22:00:36+03:00 llamafile : improve sgemm.cpp (#6796) e931888d5024de814ce7119a18d6a959bfff3821 8960fe86ae075c846c5df8848230d1904ba8877f Dave Airlie airlied@gmail.com 2024-04-23T00:05:06+10:00 GitHub noreply@github.com 2024-04-22T16:05:06+02:00 ggml : fix calloc argument ordering. (#6820) 8960fe86ae075c846c5df8848230d1904ba8877f c0956b09ba845a7cd787d5580d7c8b96e80f40f5 Georgi Gerganov ggerganov@gmail.com 2024-04-22T15:41:11+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-22T15:41:11+03:00 llama : fix typo in <|im_end|> token text (#6745) c0956b09ba845a7cd787d5580d7c8b96e80f40f5 e9b4a1bf68c18beff4e33f23ea62c1245b296915 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-22T13:22:54+02:00 GitHub noreply@github.com 2024-04-22T13:22:54+02:00 ci: fix job are cancelling each other (#6781) e9b4a1bf68c18beff4e33f23ea62c1245b296915 5cf5e7d490dfdd2e70bface2d35dfd14aa44b4fb github-actions[bot] github-actions[bot]@users.noreply.github.com 2024-04-21T00:17:47Z Someone newkozlukov@gmail.com 2024-04-22T10:42:43Z flake.lock: Update 5cf5e7d490dfdd2e70bface2d35dfd14aa44b4fb 40f74e4d739e9250431cf339ae7588b28d8d0663 Olivier Chafik ochafik@users.noreply.github.com 2024-04-21T18:48:53+01:00 GitHub noreply@github.com 2024-04-21T18:48:53+01:00 `build`: generate hex dump of server assets during build (#6661) 40f74e4d739e9250431cf339ae7588b28d8d0663 b9cc76d87e3d7ae5900f19d4fe8f8976d0a35888 Georgi Gerganov ggerganov@gmail.com 2024-04-21T18:36:45+03:00 GitHub noreply@github.com 2024-04-21T18:36:45+03:00 llama : add option to render special/control tokens (#6807) b9cc76d87e3d7ae5900f19d4fe8f8976d0a35888 7dbdba5690ca61b3ee8c92cfac8e7e251042e787 Georgi Gerganov ggerganov@gmail.com 2024-04-21T16:47:57+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-21T16:48:50+03:00 ggml : fix ggml_backend_cpu_supports_op() for CPY (#0) 7dbdba5690ca61b3ee8c92cfac8e7e251042e787 c1386c936e9fbc38eb2816c711ab28f13355708e Wouter 9594229+DifferentialityDevelopment@users.noreply.github.com 2024-04-21T15:03:39+02:00 GitHub noreply@github.com 2024-04-21T16:03:39+03:00 llama : add llama-3 chat template (#6751) c1386c936e9fbc38eb2816c711ab28f13355708e e8d35f47cb8cb4002fca02e18aaa1cb9fa21d6f1 pmysl piotr.myslinski@outlook.com 2024-04-21T14:49:30+02:00 GitHub noreply@github.com 2024-04-21T15:49:30+03:00 gguf-py : add IQ1_M to GGML_QUANT_SIZES (#6761) e8d35f47cb8cb4002fca02e18aaa1cb9fa21d6f1 2cca09d509c0e114acc16cb347c3cdd86e5f1b40 Jan Boon jan.boon@kaetemi.be 2024-04-21T20:35:40+08:00 GitHub noreply@github.com 2024-04-21T15:35:40+03:00 doc : add link to falcon (#6789) 2cca09d509c0e114acc16cb347c3cdd86e5f1b40 89b0bf0d5dc123cc1053708f5f84b89e52cdc80b Mohammadreza Hendiani mohammad.r.hendiani@gmail.com 2024-04-21T16:02:05+03:30 GitHub noreply@github.com 2024-04-21T15:32:05+03:00 readme : add Fedora instructions (#6783) 89b0bf0d5dc123cc1053708f5f84b89e52cdc80b b97bc3966e852adb626c90be64fd48282800f504 Justine Tunney jtunney@mozilla.com 2024-04-21T08:19:04-04:00 GitHub noreply@github.com 2024-04-21T15:19:04+03:00 llava : use logger in llava-cli (#6797) b97bc3966e852adb626c90be64fd48282800f504 b8109bc0139f15a5b321909f47510b89dca47ffc Pedro Cuenca pedro@huggingface.co 2024-04-21T13:50:41+02:00 GitHub noreply@github.com 2024-04-21T14:50:41+03:00 llama : support Llama 3 HF conversion (#6745) b8109bc0139f15a5b321909f47510b89dca47ffc aed82f6837a3ea515f4d50201cfc77effc7d41b4 Jan Boon jan.boon@kaetemi.be 2024-04-21T00:29:50+08:00 GitHub noreply@github.com 2024-04-20T18:29:50+02:00 doc : server tests require llama to be built with curl enabled (#6788) aed82f6837a3ea515f4d50201cfc77effc7d41b4 0e4802b2ecbaab04b4f829fde4a3096ca19c84b5 Georgi Gerganov ggerganov@gmail.com 2024-04-20T13:27:12+03:00 GitHub noreply@github.com 2024-04-20T13:27:12+03:00 common : try to fix Android CI (#6780) 0e4802b2ecbaab04b4f829fde4a3096ca19c84b5 637e9a86c220718d008b54842dfd294aa96d3b7a loonerin 132926317+loonerin@users.noreply.github.com 2024-04-19T13:03:35-04:00 GitHub noreply@github.com 2024-04-19T19:03:35+02:00 ci: add ubuntu latest release and fix missing build number (mac & ubuntu) (#6748) 637e9a86c220718d008b54842dfd294aa96d3b7a 9958c81b798a5872087b30b360e4674871f2479e Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-19T13:19:01+02:00 GitHub noreply@github.com 2024-04-19T13:19:01+02:00 server: static: upstream upgrade (#6765) 9958c81b798a5872087b30b360e4674871f2479e 8b1b1f4982d3e9b994308d05a1c8b9e45c23edb5 nopperl 54780682+nopperl@users.noreply.github.com 2024-04-19T09:35:54Z GitHub noreply@github.com 2024-04-19T11:35:54+02:00 Implement the OLMo architecture (#6741) 8b1b1f4982d3e9b994308d05a1c8b9e45c23edb5 bca40e98149c7b673558ddd7a3ebeffef789349d Austin 77757836+teleprint-me@users.noreply.github.com 2024-04-19T03:16:45-04:00 GitHub noreply@github.com 2024-04-19T10:16:45+03:00 train : add general name (#6752) bca40e98149c7b673558ddd7a3ebeffef789349d 0d56246f4b9764158525d894b96606f6163c53a8 Neo Zhang 14088817+arthw@users.noreply.github.com 2024-04-19T09:16:31+08:00 GitHub noreply@github.com 2024-04-19T09:16:31+08:00 fix wrong parameter in cmd in readme-sycl.md (#6755) 0d56246f4b9764158525d894b96606f6163c53a8 03c0946d73c63ea73e1d85015b7088298443d438 slaren slarengh@gmail.com 2024-04-18T15:18:48+02:00 GitHub noreply@github.com 2024-04-18T15:18:48+02:00 ggml : group all experts in a single ggml_mul_mat_id (#6505) 03c0946d73c63ea73e1d85015b7088298443d438 e11b2e6e1e18522ca7cf129600875a0f6fb9307d Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-04-18T13:49:01+02:00 GitHub noreply@github.com 2024-04-18T14:49:01+03:00 convert : support models with multiple chat templates (#6588) e11b2e6e1e18522ca7cf129600875a0f6fb9307d c71bfd736ee99a56e697697b39240f2ee06ed26d Ren Xuancheng jklj077@users.noreply.github.com 2024-04-18T19:38:04+08:00 GitHub noreply@github.com 2024-04-18T14:38:04+03:00 Qwen2 : assume tied weights if lm_head/output weights is missing (#6738) c71bfd736ee99a56e697697b39240f2ee06ed26d 3b8f1ec4b18770531d0b1d792f3edf08254e4f0c slaren slarengh@gmail.com 2024-04-18T09:04:47+02:00 GitHub noreply@github.com 2024-04-18T10:04:47+03:00 llama : fix compatibility with old 2 expert models (#6735) 3b8f1ec4b18770531d0b1d792f3edf08254e4f0c 8dd1ec8b3ffbfa2d26e82e672cea89f5eeb2f141 Georgi Gerganov ggerganov@gmail.com 2024-04-17T23:58:26+03:00 GitHub noreply@github.com 2024-04-17T23:58:26+03:00 llamafile : tmp disable + build sgemm.o when needed (#6716) 8dd1ec8b3ffbfa2d26e82e672cea89f5eeb2f141 facb8b56f8fd3bb10a693bf0943ae9d69d0828ef Yaroslav yaroslav.yashin@me.com 2024-04-17T14:47:50+02:00 GitHub noreply@github.com 2024-04-17T15:47:50+03:00 readme : add UI (#6724) facb8b56f8fd3bb10a693bf0943ae9d69d0828ef 532c1737a14bb4b99747e6f460874947df37e450 Zheng.Deng 32841220+dengzheng-cloud@users.noreply.github.com 2024-04-17T04:51:07+08:00 GitHub noreply@github.com 2024-04-16T23:51:07+03:00 convert : fix autoawq gemma (#6704) 532c1737a14bb4b99747e6f460874947df37e450 666867b799ddd9da7dfdc905ece291ecf286effa Georgi Gerganov ggerganov@gmail.com 2024-04-16T23:50:38+03:00 GitHub noreply@github.com 2024-04-16T23:50:38+03:00 llama : make general.name optional (#6709) 666867b799ddd9da7dfdc905ece291ecf286effa 8cc91dc63c0df397d644a581b2cbeea74eb51ae0 Georgi Gerganov ggerganov@gmail.com 2024-04-16T23:50:22+03:00 GitHub noreply@github.com 2024-04-16T23:50:22+03:00 ggml : fix llamafile sgemm wdata offsets (#6710) 8cc91dc63c0df397d644a581b2cbeea74eb51ae0 dbceec87c0221ec952e69448df6a71f1372a7487 Justine Tunney jtunney@mozilla.com 2024-04-16T14:55:30-04:00 GitHub noreply@github.com 2024-04-16T21:55:30+03:00 ggml : add llamafile sgemm (#6414) dbceec87c0221ec952e69448df6a71f1372a7487 f4dea7da1841a92d2788b0535063abf2f0e28461 Ashish 1856117+ashishdatta@users.noreply.github.com 2024-04-16T08:48:35-07:00 GitHub noreply@github.com 2024-04-16T18:48:35+03:00 llama : add StableLM2 12B (#6635) f4dea7da1841a92d2788b0535063abf2f0e28461 8a56075b07a8b571bf95a912ffdce4c928c2b414 Shijie 821898965@qq.com 2024-04-16T23:40:48+08:00 GitHub noreply@github.com 2024-04-16T18:40:48+03:00 llama : add qwen2moe (#6074) 8a56075b07a8b571bf95a912ffdce4c928c2b414 58227ffdeb4fb89cacb0cffaadf76b1914324ad3 Daniel Bevenius daniel.bevenius@gmail.com 2024-04-16T08:34:06+02:00 GitHub noreply@github.com 2024-04-16T09:34:06+03:00 gritlm : add --outdir option to hf.sh script (#6699) 58227ffdeb4fb89cacb0cffaadf76b1914324ad3 4fbd8098e63670c6ae11a8adc350f5ba191cfda3 Georgi Gerganov ggerganov@gmail.com 2024-04-16T09:28:33+03:00 GitHub noreply@github.com 2024-04-16T09:28:33+03:00 perplexity : require positive --ctx-size arg (#6695) 4fbd8098e63670c6ae11a8adc350f5ba191cfda3 7593639ce335e8d7f89aa9a54d616951f273af60 Daniel Bevenius daniel.bevenius@gmail.com 2024-04-16T08:13:13+02:00 GitHub noreply@github.com 2024-04-16T09:13:13+03:00 gguf : add special tokens metadata for FIM/Infill (#6689) 7593639ce335e8d7f89aa9a54d616951f273af60 132f55795e51094954f1b1f647f97648be724a3a Olivier Chafik ochafik@users.noreply.github.com 2024-04-15T18:35:21+01:00 GitHub noreply@github.com 2024-04-15T18:35:21+01:00 `main`: add --json-schema / -j flag (#6659) 132f55795e51094954f1b1f647f97648be724a3a 3272896d79465fd2befef3425dac8e83933b7ea4 compilade git@compilade.net 2024-04-15T08:56:55-04:00 GitHub noreply@github.com 2024-04-15T15:56:55+03:00 llama : fix restoring the number of outputs from state files (#6687) 3272896d79465fd2befef3425dac8e83933b7ea4 7fc16a2c32650d46e79b382ecc6720f58c82f31b Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-15T14:18:47+02:00 GitHub noreply@github.com 2024-04-15T15:18:47+03:00 server : revert "minor layout improvements" (#6684) 7fc16a2c32650d46e79b382ecc6720f58c82f31b 17e98d4c96a583d420f12046bc92102381dbd28e Steven Prichard spprichard20@gmail.com 2024-04-15T05:14:46-05:00 GitHub noreply@github.com 2024-04-15T13:14:46+03:00 swift : linux support (#6590) 17e98d4c96a583d420f12046bc92102381dbd28e 1958f7e06ca2d2e3ab5698cc67513ba359144d8e Neo Zhang Jianyu jianyu.zhang@intel.com 2024-04-15T17:12:26+08:00 GitHub noreply@github.com 2024-04-15T17:12:26+08:00 fix mul_mat_id() for new input, make the ut pass (#6682) 1958f7e06ca2d2e3ab5698cc67513ba359144d8e 04fbc5f23e9708136ff03ebe194c7a7e965a7ca4 David Renshaw dwrenshaw@gmail.com 2024-04-14T15:24:15-04:00 GitHub noreply@github.com 2024-04-14T15:24:15-04:00 llama : add missing kv clear in llama_beam_search (#6664) 04fbc5f23e9708136ff03ebe194c7a7e965a7ca4 f184dd920852d6d372b754f871ee06cfe6f977ad Chao Jiang jc19chaoj@zoho.com 2024-04-15T00:16:34+08:00 GitHub noreply@github.com 2024-04-14T18:16:34+02:00 Add Command R chat template (#6650) f184dd920852d6d372b754f871ee06cfe6f977ad 422c2aff1c9735853c9d8f5162104e41a364adc4 Georgi Gerganov ggerganov@gmail.com 2024-04-14T16:55:30+03:00 GitHub noreply@github.com 2024-04-14T06:55:30-07:00 flake.lock: Update (#6669) 422c2aff1c9735853c9d8f5162104e41a364adc4 8800226d65d5c98cd34eede6a6c05c78405c52da Dave dave-fl@users.noreply.github.com 2024-04-14T07:14:19-04:00 GitHub noreply@github.com 2024-04-14T13:14:19+02:00 Added support for GGML_OP_CLAMP in Metal (#6662) 8800226d65d5c98cd34eede6a6c05c78405c52da e689fc4e912feb19085be6894f475a873759cbfe Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-04-14T13:12:59+02:00 GitHub noreply@github.com 2024-04-14T13:12:59+02:00 Fix --split-max-size (#6655) e689fc4e912feb19085be6894f475a873759cbfe a4ec34e1cd68560e7ff16e21ddc6962b6abd3d1d Jaemin Son woalsdnd@gmail.com 2024-04-14T20:12:36+09:00 GitHub noreply@github.com 2024-04-14T13:12:36+02:00 [bug fix] convert github repository_owner to lowercase (#6673) a4ec34e1cd68560e7ff16e21ddc6962b6abd3d1d de17e3f7455dc7fd298cc61d86798533b9ca7a29 James A Capozzoli 157492257+jac-jim@users.noreply.github.com 2024-04-14T04:40:18-04:00 GitHub noreply@github.com 2024-04-14T11:40:18+03:00 convert : enable the `--use-temp-file` cli flag (#6645) de17e3f7455dc7fd298cc61d86798533b9ca7a29 b5e7285baffb0da8a6619567b52d8e67de41291d Neo Zhang Jianyu jianyu.zhang@intel.com 2024-04-14T10:42:29+08:00 GitHub noreply@github.com 2024-04-14T10:42:29+08:00 fix memcpy() crash, add missed cmd in guide, fix softmax (#6622) b5e7285baffb0da8a6619567b52d8e67de41291d 4bd0f93e4ab4fe6682e7d0241c1bdec1397e954a Johannes Gäßler johannesg@5d6.de 2024-04-14T00:21:55+02:00 GitHub noreply@github.com 2024-04-14T00:21:55+02:00 CUDA: fix matrix multiplication logic for tests (#6667) 4bd0f93e4ab4fe6682e7d0241c1bdec1397e954a ab9a3240a9da941fdef5cd4a25f2b97c2f5a67aa Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-13T11:33:52+02:00 GitHub noreply@github.com 2024-04-13T11:33:52+02:00 model: support arch `DbrxForCausalLM` (#6515) ab9a3240a9da941fdef5cd4a25f2b97c2f5a67aa fbbc030ba93561fac842af994c5c6c4c1147f13b Olivier Chafik ochafik@users.noreply.github.com 2024-04-12T19:43:38+01:00 GitHub noreply@github.com 2024-04-12T19:43:38+01:00 JSON schema conversion: ⚡️ faster repetitions, min/maxLength for strings, cap number length (#6555) fbbc030ba93561fac842af994c5c6c4c1147f13b 4cc120c7443cf9dab898736f3c3b45dc8f14672b slaren slarengh@gmail.com 2024-04-12T18:13:20+02:00 GitHub noreply@github.com 2024-04-12T18:13:20+02:00 metal : unify mul_mv_id kernels (#6556) 4cc120c7443cf9dab898736f3c3b45dc8f14672b 24ee66ed0d908d156bd0d1747b63a636a495cd7a Daniel Bevenius daniel.bevenius@gmail.com 2024-04-12T14:11:46+02:00 GitHub noreply@github.com 2024-04-12T15:11:46+03:00 infill : add download instructions for model (#6626) 24ee66ed0d908d156bd0d1747b63a636a495cd7a 91c736015b66ba1d0b82cbae6313b6d5eaa61b68 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-12T13:49:21+02:00 GitHub noreply@github.com 2024-04-12T14:49:21+03:00 server : coherent log output for KV cache full (#6637) 91c736015b66ba1d0b82cbae6313b6d5eaa61b68 5c4d767ac028c0f9c31cba3fceaf765c6097abfc jiez 373447296@qq.com 2024-04-12T18:45:06+08:00 GitHub noreply@github.com 2024-04-12T13:45:06+03:00 llama : add gguf_remove_key + remove split meta during quantize (#6591) 5c4d767ac028c0f9c31cba3fceaf765c6097abfc ef21ce4ccb41164cb52997bd2210d92bc6a6c5d1 Rene Leonhardt 65483435+reneleonhardt@users.noreply.github.com 2024-04-12T10:52:36+02:00 GitHub noreply@github.com 2024-04-12T10:52:36+02:00 chore: Fix markdown warnings (#6625) ef21ce4ccb41164cb52997bd2210d92bc6a6c5d1 dee7f8d6928cc680cc969f7d93f98c3e24dcad41 Georgi Gerganov ggerganov@gmail.com 2024-04-12T11:49:58+03:00 GitHub noreply@github.com 2024-04-12T11:49:58+03:00 imatrix : remove invalid assert (#6632) dee7f8d6928cc680cc969f7d93f98c3e24dcad41 81da18e71ccfc196d4516fbea5dc3a6a1f92dccb MasterYi1024 39848311+MasterYi1024@users.noreply.github.com 2024-04-12T16:28:12+08:00 GitHub noreply@github.com 2024-04-12T10:28:12+02:00 Correct free memory and total memory. (#6630) 81da18e71ccfc196d4516fbea5dc3a6a1f92dccb 9ed2737acc233716374860e6b2ea7399c4aae29e Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-12T10:26:47+02:00 GitHub noreply@github.com 2024-04-12T10:26:47+02:00 eval-callback: use ggml_op_desc to pretty print unary operator name (#6631) 9ed2737acc233716374860e6b2ea7399c4aae29e 04a5ac211ef40936295980b7cdf0ba6e97093146 Georgi Gerganov ggerganov@gmail.com 2024-04-12T11:15:05+03:00 GitHub noreply@github.com 2024-04-12T11:15:05+03:00 ci : disable Metal for macOS-latest-cmake-x64 (#6628) 04a5ac211ef40936295980b7cdf0ba6e97093146 f7001ccc5aa359fcf41bba19d1c99c3d25c9bcc7 Clint Herron hanclinto@gmail.com 2024-04-11T21:44:50-04:00 GitHub noreply@github.com 2024-04-11T21:44:50-04:00 Optimization: eliminate addition of redundant stacks when advancing grammar. (#6616) f7001ccc5aa359fcf41bba19d1c99c3d25c9bcc7 a474f50ebb3e10be3371562f75f3f573f1a86b5f Clint Herron hanclinto@gmail.com 2024-04-11T17:44:48-04:00 GitHub noreply@github.com 2024-04-11T17:44:48-04:00 As suggested by @slaren, disabling Metal for test to fix CI build on OSX from #6576 (#6619) a474f50ebb3e10be3371562f75f3f573f1a86b5f cbaadc92942c50aab599a9e4c163afc1f44f7c26 Nikolas 127742645+nneubacher@users.noreply.github.com 2024-04-11T21:56:29+02:00 GitHub noreply@github.com 2024-04-11T21:56:29+02:00 Refactor Error Handling for CUDA (#6575) cbaadc92942c50aab599a9e4c163afc1f44f7c26 1bbdaf6ecda6f0a360dfb307b256fcb6838c560b Olivier Chafik ochafik@users.noreply.github.com 2024-04-11T19:47:34+01:00 GitHub noreply@github.com 2024-04-11T19:47:34+01:00 grammars: 1.5x faster inference w/ complex grammars (vector reserves / reuses) (#6609) 1bbdaf6ecda6f0a360dfb307b256fcb6838c560b f4183afe6a22f356ee222a710686ae7f83dbd949 Hugo Roussel hugo.rous@gmail.com 2024-04-11T19:52:21+02:00 GitHub noreply@github.com 2024-04-11T19:52:21+02:00 ci: download artifacts to release directory (#6612) f4183afe6a22f356ee222a710686ae7f83dbd949 b804b1ef77351d2a11be945462c6c251710476cb Daniel Bevenius daniel.bevenius@gmail.com 2024-04-11T15:22:47+02:00 GitHub noreply@github.com 2024-04-11T16:22:47+03:00 scripts : add --outdir option to hf.sh (#6600) b804b1ef77351d2a11be945462c6c251710476cb 8228b66dbc16290c5cbd70e80ab47c068e2569d8 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-11T14:51:07+02:00 GitHub noreply@github.com 2024-04-11T14:51:07+02:00 eval-callback: Example how to use eval callback for debugging (#6576) 8228b66dbc16290c5cbd70e80ab47c068e2569d8 b3a96f27f065a828f08c5d89ff60aab5361188fe Daniel Bevenius daniel.bevenius@gmail.com 2024-04-10T20:16:48+02:00 GitHub noreply@github.com 2024-04-10T21:16:48+03:00 gguf : add option to not check tensor data (#6582) b3a96f27f065a828f08c5d89ff60aab5361188fe 4f407a0a353dae4726c74cc33250b623a4911dd7 Ralph Soika ralph.soika@imixs.com 2024-04-10T19:18:25+02:00 GitHub noreply@github.com 2024-04-10T19:18:25+02:00 minor layout improvements (#6572) 4f407a0a353dae4726c74cc33250b623a4911dd7 65c64dc36f9bca5b3f100614cdd02bf12d6b3e49 slaren slarengh@gmail.com 2024-04-10T17:24:14+02:00 GitHub noreply@github.com 2024-04-10T17:24:14+02:00 llama : add model types for mixtral (#6589) 65c64dc36f9bca5b3f100614cdd02bf12d6b3e49 67fac4b95fcccfda8ab965e9ba4992a9ddf3a25f slaren slarengh@gmail.com 2024-04-10T15:23:12+02:00 GitHub noreply@github.com 2024-04-10T15:23:12+02:00 convert.py : add consolidated.safetensors for mixtral 8x22b (#6587) 67fac4b95fcccfda8ab965e9ba4992a9ddf3a25f 29122d32ac8075ad27b7a8be05dcad2b7e7a5f9e Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-10T08:58:48+02:00 GitHub noreply@github.com 2024-04-10T09:58:48+03:00 docs : how to add a model (#6565) 29122d32ac8075ad27b7a8be05dcad2b7e7a5f9e b231b37b095f172b26b1300ca442a147ea048b64 Artem Zinnatullin ceo@abstractny.gay 2024-04-10T00:49:12-06:00 GitHub noreply@github.com 2024-04-10T09:49:12+03:00 readme : fix ROCm link (#6579) b231b37b095f172b26b1300ca442a147ea048b64 ba5e134e073ec6837078c874aba44a702944a676 sjxx 63994076+ylsdamxssjxxdd@users.noreply.github.com 2024-04-10T14:34:00+08:00 GitHub noreply@github.com 2024-04-10T09:34:00+03:00 readme : update UI list (#6560) ba5e134e073ec6837078c874aba44a702944a676 1b67731e184e27a465b8c5476061294a4af668ea Jiří Sejkora Sejseloid@gmail.com 2024-04-10T00:23:02+02:00 GitHub noreply@github.com 2024-04-10T00:23:02+02:00 readme: fix typo in amdgpu target name (#6573) 1b67731e184e27a465b8c5476061294a4af668ea c4a3a4ff47d62d2503ddf9bd91b58c21f04fe3c3 Jared Van Bortel jared@nomic.ai 2024-04-09T13:44:08-04:00 GitHub noreply@github.com 2024-04-09T13:44:08-04:00 BERT tokenizer fixes (#6498) c4a3a4ff47d62d2503ddf9bd91b58c21f04fe3c3 400d5d722d7edf7de0cf24a18c42b183c65047d2 Georgi Gerganov ggerganov@gmail.com 2024-04-09T20:29:06+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-09T20:29:06+03:00 sync : ggml 400d5d722d7edf7de0cf24a18c42b183c65047d2 5dc9dd7152dedc6046b646855585bd070c91e8c8 Ed Lee edilee@mozilla.com 2024-04-09T01:31:47-07:00 GitHub noreply@github.com 2024-04-09T10:31:47+02:00 server : detect search query to start webchat (#6554) 5dc9dd7152dedc6046b646855585bd070c91e8c8 e11a8999b5690f810c2c99c14347f0834e68c524 Carolinabanana 140120812+Carolinabanana@users.noreply.github.com 2024-04-09T09:16:13+01:00 GitHub noreply@github.com 2024-04-09T11:16:13+03:00 llama : add Command R Plus support (#6491) e11a8999b5690f810c2c99c14347f0834e68c524 cc4a95426d17417d3c83f12bdb514fbe8abe2a88 Georgi Gerganov ggerganov@gmail.com 2024-04-09T09:23:19+03:00 GitHub noreply@github.com 2024-04-09T09:23:19+03:00 license : update copyright notice + add AUTHORS (#6405) cc4a95426d17417d3c83f12bdb514fbe8abe2a88 cecd8d3c98b48f51aaa1d4c729e55bd319f6799c Georgi Gerganov ggerganov@gmail.com 2024-04-08T22:25:49+03:00 GitHub noreply@github.com 2024-04-08T22:25:49+03:00 llama : fix attention layer count sanity check (#6550) cecd8d3c98b48f51aaa1d4c729e55bd319f6799c b73e564b16086845a8b4fffd26e22685d3e0c3db kunnis kunnis@users.noreply.github.com 2024-04-08T10:44:19-05:00 GitHub noreply@github.com 2024-04-08T17:44:19+02:00 Comment explaining a decision (#6531) b73e564b16086845a8b4fffd26e22685d3e0c3db e3c337d87ca650972105a51c6ce302dd236c07ad Georgi Gerganov ggerganov@gmail.com 2024-04-08T16:23:01+03:00 GitHub noreply@github.com 2024-04-08T16:23:01+03:00 quantize : fix precedence of cli args (#6541) e3c337d87ca650972105a51c6ce302dd236c07ad beea6e1b16e783a0886e78dec01002a8c00db24d Rick G 26732651+TheFlipbook@users.noreply.github.com 2024-04-08T06:02:30-07:00 GitHub noreply@github.com 2024-04-08T16:02:30+03:00 llama : support negative ith in llama_get_ API (#6519) beea6e1b16e783a0886e78dec01002a8c00db24d 87fb5b4234d4b9c56ac94cf7aa229c8fd7defdb0 Jan Boon jan.boon@kaetemi.be 2024-04-08T20:43:30+08:00 GitHub noreply@github.com 2024-04-08T15:43:30+03:00 llama : save and restore kv cache for single seq id (#6341) 87fb5b4234d4b9c56ac94cf7aa229c8fd7defdb0 d752327c3338d5b9634121d651c0105f2c933f9b Abhilash Majumder 30946547+abhilash1910@users.noreply.github.com 2024-04-08T13:56:01+05:30 GitHub noreply@github.com 2024-04-08T16:26:01+08:00 remove row=1 cond (#6532) d752327c3338d5b9634121d651c0105f2c933f9b 855f54402e866ed19d8d675b56a81c844c64b325 Firat firatkiral@gmail.com 2024-04-08T00:48:29-07:00 GitHub noreply@github.com 2024-04-08T09:48:29+02:00 Adding KodiBot to UI list (#6535) 855f54402e866ed19d8d675b56a81c844c64b325 b909236c0bf0b6e872af95df9490492ecec310ac Mark Fairbairn thebaron88@gmail.com 2024-04-07T19:52:19+01:00 GitHub noreply@github.com 2024-04-07T20:52:19+02:00 Change Windows AMD example to release build to make inference much faster. (#6525) b909236c0bf0b6e872af95df9490492ecec310ac e0717e751e12af13f4eedaae8bbbd608e40d7e54 Georgi Gerganov ggerganov@gmail.com 2024-04-07T21:25:30+03:00 GitHub noreply@github.com 2024-04-07T11:25:30-07:00 flake.lock: Update (#6517) e0717e751e12af13f4eedaae8bbbd608e40d7e54 c37247796b4d45bdbbc8259afffb80208ad8fe55 DAN™ dranger003@gmail.com 2024-04-07T13:33:59-04:00 GitHub noreply@github.com 2024-04-07T19:33:59+02:00 Add GritLM as supported models. (#6513) c37247796b4d45bdbbc8259afffb80208ad8fe55 f77261a7c525fa1fa47b18a3d78cd308ae41cafc Georgi Gerganov ggerganov@gmail.com 2024-04-07T17:05:51+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-07T17:05:51+03:00 sync : ggml f77261a7c525fa1fa47b18a3d78cd308ae41cafc 43e8995e754b8e04642e92822055d193a3272b37 Slava Primenko primenko.s@gmail.com 2024-04-04T14:49:24+02:00 Georgi Gerganov ggerganov@gmail.com 2024-04-07T17:05:40+03:00 ggml: bypass code incompatible with CUDA < 11.1 (whisper/2020) 43e8995e754b8e04642e92822055d193a3272b37 9472bce30800a581071478a839bf93abf404c893 Georgi Gerganov ggerganov@gmail.com 2024-04-07T16:08:12+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-07T16:08:12+03:00 scripts : sync ggml-cuda folder 9472bce30800a581071478a839bf93abf404c893 d4f220a5ccdc6308173c1a31fad21d7c3fbc96c1 limitedAtonement limitedAtonement@users.noreply.github.com 2024-04-07T07:05:40-04:00 GitHub noreply@github.com 2024-04-07T13:05:40+02:00 Run make to build the project (#6457) d4f220a5ccdc6308173c1a31fad21d7c3fbc96c1 54ea0698fbf87e36a5d68a98c95f6bdd0fb91557 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-04-07T10:55:59+08:00 GitHub noreply@github.com 2024-04-07T10:55:59+08:00 support/fix OPs GGML_TYPE_IQ4_NL, GGML_TYPE_IQ4_XS, GGML_TYPE_IQ3_XXS, GGML_TYPE_IQ3_S, GGML_TYPE_IQ2_XXS, GGML_TYPE_IQ2_XS, GGML_TYPE_IQ2_S, GGML_TYPE_IQ1_S, GGML_TYPE_IQ1_M (#6521) 54ea0698fbf87e36a5d68a98c95f6bdd0fb91557 b66aec675c1571a06b3570b858ae711246f96f84 Georgi Gerganov ggerganov@gmail.com 2024-04-06T17:43:15+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-06T18:27:46+03:00 sync : ggml b66aec675c1571a06b3570b858ae711246f96f84 57dd02c44b2a0eb79e28f6c5eb8242a5d2d3174d Daniel Bevenius daniel.bevenius@gmail.com 2024-04-03T22:57:20+02:00 Georgi Gerganov ggerganov@gmail.com 2024-04-06T17:42:26+03:00 backend : fix typo in scheduler documentation (ggml/781) 57dd02c44b2a0eb79e28f6c5eb8242a5d2d3174d 75cd4c77292034ecec587ecb401366f57338f7c0 Clint Herron hanclinto@gmail.com 2024-04-06T10:31:33-04:00 GitHub noreply@github.com 2024-04-06T10:31:33-04:00 Tests: Added integration tests for GBNF parser (#6472) 75cd4c77292034ecec587ecb401366f57338f7c0 a8bd14d55717754a1f48313a846a2b16fa998ad2 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-06T05:40:47+02:00 GitHub noreply@github.com 2024-04-06T05:40:47+02:00 ci: bench: support sse and fix prompt processing time / server: add tokens usage in stream OAI response (#6495) a8bd14d55717754a1f48313a846a2b16fa998ad2 d0f5deebf898f8186a10148a03a56909ba05fc0b Brian mofosyne@gmail.com 2024-04-06T05:41:38+11:00 GitHub noreply@github.com 2024-04-05T21:41:38+03:00 gguf.py : add licence and version to gguf writer (#6504) d0f5deebf898f8186a10148a03a56909ba05fc0b 87e21bbacd830437ab653cf03b6f26d45c15395d Hoang Nguyen hugo53@users.noreply.github.com 2024-04-05T11:39:43-07:00 GitHub noreply@github.com 2024-04-05T21:39:43+03:00 readme : update UI list (#6503) 87e21bbacd830437ab653cf03b6f26d45c15395d 1b496a745c315022df2d919374052e6004ced8d3 Ting Sun suntcrick@gmail.com 2024-04-06T01:34:53+07:00 GitHub noreply@github.com 2024-04-05T21:34:53+03:00 bench : make n_batch and n_ubatch configurable in Batched bench (#6500) 1b496a745c315022df2d919374052e6004ced8d3 a307375c02cac45cff53cf2520330b43fecc7718 Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-04-05T14:35:06+01:00 GitHub noreply@github.com 2024-04-05T19:05:06+05:30 [SYCL] Fixed minor bug when enabling FP16 for non intel targets (#6464) a307375c02cac45cff53cf2520330b43fecc7718 b660a5729e1e7508671d3d0515fd7efaeaeb85b9 alexpinel 93524949+alexpinel@users.noreply.github.com 2024-04-04T18:22:50+01:00 GitHub noreply@github.com 2024-04-04T13:22:50-04:00 readme : add Dot to UI list (#6487) b660a5729e1e7508671d3d0515fd7efaeaeb85b9 0a1d889e27d6aaa3293dd2c692b849a9bcf4b474 Jun Jie 71215065+junnjiee16@users.noreply.github.com 2024-04-05T01:16:37+08:00 GitHub noreply@github.com 2024-04-04T13:16:37-04:00 readme : fix typo (#6481) 0a1d889e27d6aaa3293dd2c692b849a9bcf4b474 7dda1b727ef1730783a6077136d28b83e70dd397 Ed Lepedus ed.lepedus@googlemail.com 2024-04-04T17:31:22+01:00 GitHub noreply@github.com 2024-04-04T18:31:22+02:00 server: add cURL support to server Dockerfiles (#6474) 7dda1b727ef1730783a6077136d28b83e70dd397 c666ba26c39d5c07e07b4e1e411332f408e309ad Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-04-05T01:30:53+09:00 GitHub noreply@github.com 2024-04-04T18:30:53+02:00 ci: exempt master branch workflows from getting cancelled (#6486) c666ba26c39d5c07e07b4e1e411332f408e309ad 2e66913e5f56209f4c949f98e431925b78e7e84d Ewout ter Hoeven E.M.terHoeven@student.tudelft.nl 2024-04-04T17:08:55+02:00 GitHub noreply@github.com 2024-04-04T17:08:55+02:00 build CI: Name artifacts (#6482) 2e66913e5f56209f4c949f98e431925b78e7e84d 8120efee1d9931b514aeb5a047209d576f23286c Shakhar Dasgupta shakhardasgupta@gmail.com 2024-04-04T11:03:00-04:00 GitHub noreply@github.com 2024-04-04T17:03:00+02:00 server: allow penalizing repetition of newlines on server webpage (#6431) 8120efee1d9931b514aeb5a047209d576f23286c a74401f0e5ebb15fa4d8b6619d1baa6ea9179123 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-04T16:59:04+02:00 GitHub noreply@github.com 2024-04-04T16:59:04+02:00 ci: bench fix concurrency for workflow trigger dispatch with sha1 (#6478) a74401f0e5ebb15fa4d8b6619d1baa6ea9179123 7a2c92637ae265654a68f62e6a7610b358255d3f limitedAtonement limitedAtonement@users.noreply.github.com 2024-04-04T10:30:02-04:00 GitHub noreply@github.com 2024-04-04T16:30:02+02:00 Correct README link (#6458) 7a2c92637ae265654a68f62e6a7610b358255d3f 4bcd6b959ca3991084ad1d8464caf2a734e29b1d Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-04T11:57:58+02:00 GitHub noreply@github.com 2024-04-04T12:57:58+03:00 ci: bench: add more ftype, fix triggers and bot comment (#6466) 4bcd6b959ca3991084ad1d8464caf2a734e29b1d 9b84ae1806cded4d6683c7b810925da5ead40607 Daniel Bevenius daniel.bevenius@gmail.com 2024-04-04T09:49:21+02:00 GitHub noreply@github.com 2024-04-04T09:49:21+02:00 common: remove duplicate check for curl (#6471) 9b84ae1806cded4d6683c7b810925da5ead40607 4399f13fb9462cd06f3f154d0aee738425000fea Clint Herron hanclinto@gmail.com 2024-04-04T03:44:28-04:00 GitHub noreply@github.com 2024-04-04T10:44:28+03:00 examples : add GBNF validator program (#5948) 4399f13fb9462cd06f3f154d0aee738425000fea 1a43c7254ed5de91d09274b853db843c518f1b64 Georgi Gerganov ggerganov@gmail.com 2024-04-04T09:34:58+03:00 Georgi Gerganov ggerganov@gmail.com 2024-04-04T09:34:58+03:00 server : remove obsolete --memory-f32 option 1a43c7254ed5de91d09274b853db843c518f1b64 72d73af65132792a52433952ca4729b01c36cde2 Xiao-Yong Jin jinxiaoyong@gmail.com 2024-04-04T01:33:48-05:00 GitHub noreply@github.com 2024-04-04T09:33:48+03:00 server : add option to disable KV offload (#6468) 72d73af65132792a52433952ca4729b01c36cde2 5fb1574c8112c757fc202fdae279da883f34e610 Clint Herron hanclinto@gmail.com 2024-04-04T02:32:53-04:00 GitHub noreply@github.com 2024-04-04T09:32:53+03:00 convert : fix for lint error complaining of bare except (#6470) 5fb1574c8112c757fc202fdae279da883f34e610 60cdf40cc32f0ad4cb11e0ca8fd38f3b93d8d640 Fattire 528174+fat-tire@users.noreply.github.com 2024-04-03T13:22:57-07:00 GitHub noreply@github.com 2024-04-03T22:22:57+02:00 A few small fixes to server's README docs (#6428) 60cdf40cc32f0ad4cb11e0ca8fd38f3b93d8d640 bb43cf7e9d86d69ffd9c7f008f75db890a35b45a JH23X 165871467+JH23X@users.noreply.github.com 2024-04-03T20:09:52+02:00 GitHub noreply@github.com 2024-04-03T21:09:52+03:00 server : handle exception on wrong type in request (#6452) bb43cf7e9d86d69ffd9c7f008f75db890a35b45a 9f62c0173d964972849251c8ad12fc356f5b7896 bryanSwk 93190252+bryanSwk@users.noreply.github.com 2024-04-04T02:05:10+08:00 GitHub noreply@github.com 2024-04-03T21:05:10+03:00 llama : add SEA-LION support (#6448) 9f62c0173d964972849251c8ad12fc356f5b7896 5d4f12e4624bf4435ba26753814bedd4e9b62803 Ewout ter Hoeven E.M.terHoeven@student.tudelft.nl 2024-04-03T20:01:13+02:00 GitHub noreply@github.com 2024-04-03T21:01:13+03:00 ci : update checkout, setup-python and upload-artifact to latest (#6456) 5d4f12e4624bf4435ba26753814bedd4e9b62803 154d4ee39c38e231fb5c3910df14d2fc920fdf1b Ed Lepedus ed.lepedus@googlemail.com 2024-04-03T18:56:37+01:00 GitHub noreply@github.com 2024-04-03T19:56:37+02:00 server: add cURL support to `server.Dockerfile` (#6461) 154d4ee39c38e231fb5c3910df14d2fc920fdf1b e69945d953b651674d6e6978022edf00c3a34d03 Francisco Melo 43780565+francis2tm@users.noreply.github.com 2024-04-03T18:53:37+01:00 GitHub noreply@github.com 2024-04-03T20:53:37+03:00 readme : add feature-rich rust bindings (#6465) e69945d953b651674d6e6978022edf00c3a34d03 db214fa578e00b01e0884fc2725c9349608bdab5 Joyce joycebrum@google.com 2024-04-03T14:48:07-03:00 GitHub noreply@github.com 2024-04-03T20:48:07+03:00 security : create policy (#6354) db214fa578e00b01e0884fc2725c9349608bdab5 1ff4d9f3d683f02ef8a12e04bfac84300c44bd3a Abhishek Gopinath K 31348521+overtunned@users.noreply.github.com 2024-04-03T21:12:52+05:30 GitHub noreply@github.com 2024-04-03T11:42:52-04:00 Missing tokenizer.model error during gguf conversion (#6443) 1ff4d9f3d683f02ef8a12e04bfac84300c44bd3a 076b08649ecc3b0e1c0709c2a086a63eddd1bf32 kaizau kaizau@users.noreply.github.com 2024-04-03T23:24:31+08:00 GitHub noreply@github.com 2024-04-03T17:24:31+02:00 Add OpenChat, Alpaca, Vicuna chat templates (#6397) 076b08649ecc3b0e1c0709c2a086a63eddd1bf32 08a0c0206075556e82aca0feafad530dcc5f1426 Georgi Gerganov ggerganov@gmail.com 2024-04-03T16:11:15+03:00 GitHub noreply@github.com 2024-04-03T16:11:15+03:00 readme : update hot topics 08a0c0206075556e82aca0feafad530dcc5f1426 52604860f93063ef98863921da697576af1c7665 slaren slarengh@gmail.com 2024-04-03T15:07:05+02:00 GitHub noreply@github.com 2024-04-03T16:07:05+03:00 ggml : mul_mat_id use the same tensor for all the experts (#6387) 52604860f93063ef98863921da697576af1c7665 f87f7b898651339fe173ddf016ca826163e899d8 Meng, Hengyu hengyu.meng@intel.com 2024-04-03T10:34:40+08:00 GitHub noreply@github.com 2024-04-03T10:34:40+08:00 [SYCL] Disable iqx on windows as WA (#6435) f87f7b898651339fe173ddf016ca826163e899d8 33a52448061cfd2ea44da9e6cb30b2ec22e2f6d0 Georgi Gerganov ggerganov@gmail.com 2024-04-01T19:05:57+03:00 GitHub noreply@github.com 2024-04-01T09:05:57-07:00 flake.lock: Update (#6402) 33a52448061cfd2ea44da9e6cb30b2ec22e2f6d0 226e819371eec0f298d9075198394a07b23ecfa9 Johannes Gäßler johannesg@5d6.de 2024-04-01T13:30:43+02:00 GitHub noreply@github.com 2024-04-01T13:30:43+02:00 compare-llama-bench.py: fix long hexsha args (#6424) 226e819371eec0f298d9075198394a07b23ecfa9 c50a82ce0f71558cbb8e555146ba124251504b38 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-04-01T12:36:40+02:00 GitHub noreply@github.com 2024-04-01T12:36:40+02:00 ci: server: verify deps are coherent with the commit (#6409) c50a82ce0f71558cbb8e555146ba124251504b38 37e7854c104301c5b5323ccc40e07699f3a62c3e Georgi Gerganov ggerganov@gmail.com 2024-03-31T11:56:30+03:00 GitHub noreply@github.com 2024-03-31T11:56:30+03:00 readme : update hot topics 37e7854c104301c5b5323ccc40e07699f3a62c3e c342d070c64a1ffe35d22c1b16b672e684a30297 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-30T11:36:07+01:00 GitHub noreply@github.com 2024-03-30T12:36:07+02:00 ci: bench: fix Resource not accessible by integration on PR event (#6393) c342d070c64a1ffe35d22c1b16b672e684a30297 f7fc5f6c6f3700da311de7fe93977c81798f02d3 Mohammadreza Hendiani hendiani.mohammadreza@gmail.com 2024-03-30T01:29:56+03:30 GitHub noreply@github.com 2024-03-29T22:59:56+01:00 Fedora build update (#6388) f7fc5f6c6f3700da311de7fe93977c81798f02d3 ba0c7c70ab5b15f1f2be7fb0dfbe0366dda30d6c Xuan Son Nguyen thichthat@gmail.com 2024-03-29T22:34:44+01:00 GitHub noreply@github.com 2024-03-29T22:34:44+01:00 split: allow --split-max-size option (#6343) ba0c7c70ab5b15f1f2be7fb0dfbe0366dda30d6c d48ccf3ad4fea5b9ede209c7f40be65371987bfe 0cc4m picard12@live.de 2024-03-29T17:29:21+01:00 GitHub noreply@github.com 2024-03-29T17:29:21+01:00 Vulkan k-quant mmq and ggml-backend offload functionality (#6155) d48ccf3ad4fea5b9ede209c7f40be65371987bfe 069574775cea67d8a977904e4d534aff47a671f4 Georgi Gerganov ggerganov@gmail.com 2024-03-29T17:45:46+02:00 GitHub noreply@github.com 2024-03-29T17:45:46+02:00 sync : ggml (#6351) 069574775cea67d8a977904e4d534aff47a671f4 cfde806eb95de06d84162bdee593dad33a1d2693 hxer7963 hxer7963@gmail.com 2024-03-29T21:37:03+08:00 GitHub noreply@github.com 2024-03-29T14:37:03+01:00 [Model] Add support for xverse (#6301) cfde806eb95de06d84162bdee593dad33a1d2693 b910287954d4462fd3d48938336770e258459677 Georgi Gerganov ggerganov@gmail.com 2024-03-29T14:34:28+02:00 GitHub noreply@github.com 2024-03-29T14:34:28+02:00 ci : fix BGE wget (#6383) b910287954d4462fd3d48938336770e258459677 809398709041ee854fbbad9b344bcfdcd3712d59 zhouwg 6889919+zhouwg@users.noreply.github.com 2024-03-29T15:33:46+08:00 GitHub noreply@github.com 2024-03-29T09:33:46+02:00 readme : add project (#6356) 809398709041ee854fbbad9b344bcfdcd3712d59 057400a3fd457f4f214684eeb171444663b47a23 Matt Clayton 156335168+mattjcly@users.noreply.github.com 2024-03-29T03:27:42-04:00 GitHub noreply@github.com 2024-03-29T09:27:42+02:00 cmake : add explicit metal version options (#6370) 057400a3fd457f4f214684eeb171444663b47a23 b75c38166cf0c66793a32d5c3d6cb69929dd083d Daniel Bevenius daniel.bevenius@gmail.com 2024-03-29T08:23:22+01:00 GitHub noreply@github.com 2024-03-29T09:23:22+02:00 llama : remove redundant reshape in build_kv_store (#6369) b75c38166cf0c66793a32d5c3d6cb69929dd083d bfe7dafc9cf96b9a09ead347fed9a547930fc631 Pedro Cuenca pedro@huggingface.co 2024-03-29T08:15:00+01:00 GitHub noreply@github.com 2024-03-29T09:15:00+02:00 convert : allow conversion of Mistral HF models (#6144) bfe7dafc9cf96b9a09ead347fed9a547930fc631 5106ef482c65ac60ac14da9a68c7b37bca4c6993 Georgi Gerganov ggerganov@gmail.com 2024-03-28T22:56:03+02:00 GitHub noreply@github.com 2024-03-28T22:56:03+02:00 readme : add notice for UI list 5106ef482c65ac60ac14da9a68c7b37bca4c6993 be55134a535f7218c53f39211755b1c7550851b2 Ouadie EL FAROUKI ouadie.elfarouki@codeplay.com 2024-03-28T16:01:47Z GitHub noreply@github.com 2024-03-28T16:01:47Z [SYCL] Revisited & updated SYCL build documentation (#6141) be55134a535f7218c53f39211755b1c7550851b2 66ba56025602270152f5ba5234f3a80be3dee1c9 Jared Van Bortel jared@nomic.ai 2024-03-28T11:44:36-04:00 GitHub noreply@github.com 2024-03-28T11:44:36-04:00 convert : refactor vocab selection logic (#6355) 66ba56025602270152f5ba5234f3a80be3dee1c9 0308f5e3d7bf9879f818b1a4ae589ff36b242af5 Ziang Wu 97337387+ZiangWu-77@users.noreply.github.com 2024-03-28T22:33:10+08:00 GitHub noreply@github.com 2024-03-28T16:33:10+02:00 llava : fix MobileVLM (#6364) 0308f5e3d7bf9879f818b1a4ae589ff36b242af5 28cb9a09c4d10a489be1238abe7a858dcd4d65f2 compilade 113953597+compilade@users.noreply.github.com 2024-03-28T08:05:54-04:00 GitHub noreply@github.com 2024-03-28T14:05:54+02:00 llama : fix command-r inference when omitting outputs (#6367) 28cb9a09c4d10a489be1238abe7a858dcd4d65f2 cfc4d75df6399b36153ef739f2c1abee4c114bb8 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-28T11:27:56+01:00 GitHub noreply@github.com 2024-03-28T11:27:56+01:00 ci: bench: fix master not schedule, fix commit status failed on external repo (#6365) cfc4d75df6399b36153ef739f2c1abee4c114bb8 6902cb7f2e3479f364ee177118200fb7e4e9fc92 Ting Sun suntcrick@gmail.com 2024-03-28T16:51:06+08:00 GitHub noreply@github.com 2024-03-28T09:51:06+01:00 doc: fix outdated default value of batch size (#6336) 6902cb7f2e3479f364ee177118200fb7e4e9fc92 d2d8f389960a106b66313ff1621bdb1aaaaaa285 Eric Zhang 34133756+EZForever@users.noreply.github.com 2024-03-28T16:50:48+08:00 GitHub noreply@github.com 2024-03-28T09:50:48+01:00 server : stop gracefully on SIGTERM (#6348) d2d8f389960a106b66313ff1621bdb1aaaaaa285 d39b308eaf0ac91c2e1f432bf66751193a470a56 hutli hutli@hutli.hu 2024-03-27T19:17:30+01:00 Someone newkozlukov@gmail.com 2024-03-28T07:48:27Z nix: removed unnessesary indentation d39b308eaf0ac91c2e1f432bf66751193a470a56 c87397664964e5a2a21de1877d504b23a2a35332 hutli hutli@hutli.hu 2024-03-27T19:14:28+01:00 Someone newkozlukov@gmail.com 2024-03-28T07:48:27Z nix: moved blas availability check to package inputs so it is still overridable c87397664964e5a2a21de1877d504b23a2a35332 dbb03e2b9c4fead73062926b6a134f28d3a3b46d hutli hutli@hutli.hu 2024-03-27T18:10:08+01:00 Someone newkozlukov@gmail.com 2024-03-28T07:48:27Z using blas.meta.available to check host platform dbb03e2b9c4fead73062926b6a134f28d3a3b46d e9f17dc3bf0da76c8b35130f9ca2fda5246c418e hutli jensstaermose@hotmail.com 2024-03-27T17:25:05+01:00 Someone newkozlukov@gmail.com 2024-03-28T07:48:27Z only using explicit blas if hostPlatform is allowed e9f17dc3bf0da76c8b35130f9ca2fda5246c418e 22a462cc1f69873f7d4c6d0201bd93478afa2ecb Someone Serge sergei.kozlukov@aalto.fi 2024-03-26T16:22:42Z Someone newkozlukov@gmail.com 2024-03-28T07:48:27Z nix: .#windows: proper cross-compilation set-up 22a462cc1f69873f7d4c6d0201bd93478afa2ecb f6a0f5c6422200764b7929064c39dcf4bb0e9cd6 Someone Serge sergei.kozlukov@aalto.fi 2024-03-26T16:22:07Z Someone newkozlukov@gmail.com 2024-03-28T07:48:27Z nix: package: don't introduce the dependency on python f6a0f5c6422200764b7929064c39dcf4bb0e9cd6 d0e2f6416bd43eddb70137f6b96c7bc3d0246102 hutli jensstaermose@hotmail.com 2024-02-15T14:25:04+01:00 Someone newkozlukov@gmail.com 2024-03-28T07:48:27Z nix: .#widnows: init d0e2f6416bd43eddb70137f6b96c7bc3d0246102 25f4a613c4ed6451162a87cb90be10d610b49f0f Ziang Wu 97337387+ZiangWu-77@users.noreply.github.com 2024-03-28T12:03:30+08:00 GitHub noreply@github.com 2024-03-28T13:03:30+09:00 doc: fix typo in MobileVLM-README.md (#6181) 25f4a613c4ed6451162a87cb90be10d610b49f0f a016026a3ac16d8c9b993a3573f19b9556d67de4 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-28T08:55:24+08:00 GitHub noreply@github.com 2024-03-28T08:55:24+08:00 [SYCL] fix set main gpu crash (#6339) a016026a3ac16d8c9b993a3573f19b9556d67de4 53c7ec53d5eca26b2c0c648605543a5fa6c12817 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-27T20:26:49+01:00 GitHub noreply@github.com 2024-03-27T20:26:49+01:00 server: continuous performance monitoring and PR comment (#6283) 53c7ec53d5eca26b2c0c648605543a5fa6c12817 e5b89a441af23a74b861b0bf8db3239139041876 Someone Serge sergei.kozlukov@aalto.fi 2024-03-27T16:17:46Z Someone newkozlukov@gmail.com 2024-03-27T19:18:55Z nix: ci: dont test cuda and rocm (for now) e5b89a441af23a74b861b0bf8db3239139041876 3a0345970ed0353fa857df3c8a62a2b3318b1364 slaren slarengh@gmail.com 2024-03-27T15:07:50+01:00 GitHub noreply@github.com 2024-03-27T15:07:50+01:00 ggml : fix bounds checking of zero size views (#6347) 3a0345970ed0353fa857df3c8a62a2b3318b1364 1e13987fba5a536965ef942f2c86549d62cef50b Georgi Gerganov ggerganov@gmail.com 2024-03-27T15:02:49+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-27T15:02:49+02:00 make : whitespace 1e13987fba5a536965ef942f2c86549d62cef50b e82f9e2b833d88cd2b30123ef57346c2cb8abd99 howlger eclipse@voormann.de 2024-03-27T12:15:44+01:00 GitHub noreply@github.com 2024-03-27T13:15:44+02:00 embedding : show full embedding for single prompt (#6342) e82f9e2b833d88cd2b30123ef57346c2cb8abd99 cbc83436197cde617cad696e665879c20df77daa AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-03-27T08:16:40Z GitHub noreply@github.com 2024-03-27T13:46:40+05:30 [SYCL] Fix batched impl for NVidia GPU (#6164) cbc83436197cde617cad696e665879c20df77daa e562b9714b9b3e242361a7f74bbbeb00f6bd99ac Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-27T08:44:27+01:00 GitHub noreply@github.com 2024-03-27T08:44:27+01:00 Make IQ1_M work for QK_K = 64 (#6327) e562b9714b9b3e242361a7f74bbbeb00f6bd99ac 2ab4f00d25c0682a74472412d66454df211e4b0e Sigbjørn Skjæret sigbjorn.skjaeret@scala.com 2024-03-27T08:23:10+01:00 GitHub noreply@github.com 2024-03-27T09:23:10+02:00 common : change --no-penalize-nl to --penalize-nl (#6334) 2ab4f00d25c0682a74472412d66454df211e4b0e 1740d6dd4e00dedda87d6820b0e0d8e70dade340 Georgi Gerganov ggerganov@gmail.com 2024-03-27T09:16:02+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-27T09:16:02+02:00 llama2c : open file as binary (#6332) 1740d6dd4e00dedda87d6820b0e0d8e70dade340 0642b22cd12af278d6e7e459b73411947c169381 Mateusz Charytoniuk mateusz.charytoniuk@protonmail.com 2024-03-27T08:08:59+01:00 GitHub noreply@github.com 2024-03-27T09:08:59+02:00 readme : add php api bindings (#6326) 0642b22cd12af278d6e7e459b73411947c169381 a4f569e8a316cbd33d2b4de94b694d111507475a Eric Zhang 34133756+EZForever@users.noreply.github.com 2024-03-27T13:55:29+08:00 GitHub noreply@github.com 2024-03-27T06:55:29+01:00 server: public: use relative routes for static files (#6325) a4f569e8a316cbd33d2b4de94b694d111507475a 32c8486e1f0297393cb22ac0a0d26a6b17ad4d54 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-27T09:47:06+08:00 GitHub noreply@github.com 2024-03-27T09:47:06+08:00 [SYCL] fix no file in win rel (#6314) 32c8486e1f0297393cb22ac0a0d26a6b17ad4d54 557410b8f06380560155ac7fcb8316d71ddc9837 Jared Van Bortel jared@nomic.ai 2024-03-26T17:46:21-04:00 GitHub noreply@github.com 2024-03-26T17:46:21-04:00 wpm : portable unicode tolower (#6305) 557410b8f06380560155ac7fcb8316d71ddc9837 55c1b2a3bbd470e9e2a3a0618b92cf64a885f806 compilade 113953597+compilade@users.noreply.github.com 2024-03-26T10:46:41-04:00 GitHub noreply@github.com 2024-03-26T16:46:41+02:00 llama : greatly reduce output buffer memory usage (#6122) 55c1b2a3bbd470e9e2a3a0618b92cf64a885f806 e097633f63fdd26d492844f7eff056e4083fd9eb Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-26T15:21:27+01:00 GitHub noreply@github.com 2024-03-26T15:21:27+01:00 IQ1_M: 1.75 bpw quantization (#6302) e097633f63fdd26d492844f7eff056e4083fd9eb d25b1c31b07c3675443a55a828dd58cfef5a241c Pedro Cuenca pedro@huggingface.co 2024-03-26T13:32:19+01:00 GitHub noreply@github.com 2024-03-26T14:32:19+02:00 convert-hf : fix exception in sentencepiece with added tokens (#6320) d25b1c31b07c3675443a55a828dd58cfef5a241c deb7240100da99555b9ab9dc635021e591fceaf5 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-26T13:09:30+01:00 GitHub noreply@github.com 2024-03-26T14:09:30+02:00 quantize : be able to override metadata by key (#6321) deb7240100da99555b9ab9dc635021e591fceaf5 3d032ece8e6973441273601ca2981130608e287d Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-03-26T18:11:46+09:00 GitHub noreply@github.com 2024-03-26T11:11:46+02:00 embedding : adjust `n_ubatch` value (#6296) 3d032ece8e6973441273601ca2981130608e287d e190f1fca6f60d80944f9e8709d343a025c4d245 Jan Boon jan.boon@kaetemi.be 2024-03-26T16:47:43+08:00 GitHub noreply@github.com 2024-03-26T10:47:43+02:00 server : add `n_discard` parameter (#6300) e190f1fca6f60d80944f9e8709d343a025c4d245 280345968dabc00d212d43e31145f5c9961a7604 Joseph Stahl 1269177+josephst@users.noreply.github.com 2024-03-25T20:51:46-04:00 GitHub noreply@github.com 2024-03-25T17:51:46-07:00 nix: make `xcrun` visible in Nix sandbox for precompiling Metal shaders (#6118) 280345968dabc00d212d43e31145f5c9961a7604 b06c16ef9f81d84da520232c125d4d8a1d273736 slaren slarengh@gmail.com 2024-03-26T01:16:01+01:00 GitHub noreply@github.com 2024-03-26T01:16:01+01:00 cuda : rename build flag to LLAMA_CUDA (#6299) b06c16ef9f81d84da520232c125d4d8a1d273736 1f2fd4e727a707f85d58e0b56075c3e6334d18d8 Christian Kögler ck3d@gmx.de 2024-03-25T18:52:45+01:00 GitHub noreply@github.com 2024-03-25T10:52:45-07:00 nix: fix blas support (#6281) 1f2fd4e727a707f85d58e0b56075c3e6334d18d8 43139cc528909c3de8c144ed174e09a1f7912a80 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-25T18:33:15+01:00 GitHub noreply@github.com 2024-03-25T19:33:15+02:00 tests : include IQ2_XXS and IQ2_XS in test-quantize-fns (#6303) 43139cc528909c3de8c144ed174e09a1f7912a80 2f34b865b62b1d2b5eb8a27885e4de220deeacbd Georgi Gerganov ggerganov@gmail.com 2024-03-25T17:22:27+02:00 GitHub noreply@github.com 2024-03-25T08:22:27-07:00 flake.lock: Update (#6266) 2f34b865b62b1d2b5eb8a27885e4de220deeacbd ae1f211ce2138448b47ebb148e25c58406845278 slaren slarengh@gmail.com 2024-03-25T15:43:22+01:00 GitHub noreply@github.com 2024-03-25T16:43:22+02:00 cuda : fix LLAMA_CUDA_F16 build (#6298) ae1f211ce2138448b47ebb148e25c58406845278 ad3a0505e3b6cd777259ee35e61d428357ffc565 slaren slarengh@gmail.com 2024-03-25T13:50:23+01:00 GitHub noreply@github.com 2024-03-25T13:50:23+01:00 cuda : refactor into multiple files (#6269) ad3a0505e3b6cd777259ee35e61d428357ffc565 95ad616cddda50273e955bfe192328acd9aa4896 Xuan Son Nguyen thichthat@gmail.com 2024-03-25T09:42:17+01:00 GitHub noreply@github.com 2024-03-25T09:42:17+01:00 Server: clean up OAI params parsing function (#6284) 95ad616cddda50273e955bfe192328acd9aa4896 64e7b47c6986221f2ff5c57c89dfc018bb0e9e6d Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-25T15:52:41+08:00 GitHub noreply@github.com 2024-03-25T15:52:41+08:00 [SYCL] fix SYCL backend build on windows is break by LOG() error (#6290) 64e7b47c6986221f2ff5c57c89dfc018bb0e9e6d 7733f0c76081b2a69b5f8b192db2db7c43629d58 Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-03-25T16:38:22+09:00 GitHub noreply@github.com 2024-03-25T09:38:22+02:00 examples : add "retrieval" (#6193) 7733f0c76081b2a69b5f8b192db2db7c43629d58 a32b77c4b2c1808654d0b952f26c37d73d2e746b Justine Tunney jtunney@gmail.com 2024-03-25T01:39:56-04:00 GitHub noreply@github.com 2024-03-25T07:39:56+02:00 ggml : support AVX512VNNI (#6280) a32b77c4b2c1808654d0b952f26c37d73d2e746b a0e584defd8c16e7a51ab895f595df0448d710d0 Rick G 26732651+TheFlipbook@users.noreply.github.com 2024-03-24T14:45:56-07:00 GitHub noreply@github.com 2024-03-24T22:45:56+01:00 Fix heap corruption from wmode out-of-bound writes on windows (#6272) a0e584defd8c16e7a51ab895f595df0448d710d0 7aed0ffe6855e9cadcf413f288753af4566bfeb8 Georgi Gerganov ggerganov@gmail.com 2024-03-24T16:18:45+02:00 GitHub noreply@github.com 2024-03-24T16:18:45+02:00 imatrix : fix wname for mul_mat_id ops (#6271) 7aed0ffe6855e9cadcf413f288753af4566bfeb8 ea279d56091b90fbbe063b598f5229d95f58ef68 Johannes Gäßler johannesg@5d6.de 2024-03-24T14:21:17+01:00 GitHub noreply@github.com 2024-03-24T14:21:17+01:00 Fixed lookup compilation issues on Windows (#6273) ea279d56091b90fbbe063b598f5229d95f58ef68 586e7bc561be88e929a9afca7e67d8ead00c53bd Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-24T09:57:06+01:00 GitHub noreply@github.com 2024-03-24T10:57:06+02:00 ci : close inactive issue, increase operations per run (#6270) 586e7bc561be88e929a9afca7e67d8ead00c53bd ddf65685105a39a57b1e7f80c3aa502a6313af24 Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-03-24T17:54:07+09:00 GitHub noreply@github.com 2024-03-24T10:54:07+02:00 sampling : deduplicated code for probability distribution access (#6240) ddf65685105a39a57b1e7f80c3aa502a6313af24 d03224ac9840351023ff8abcf4aa0542258a53df Meng, Hengyu hengyu.meng@intel.com 2024-03-24T12:04:25+08:00 GitHub noreply@github.com 2024-03-24T12:04:25+08:00 [SYCL] offload op (#6217) d03224ac9840351023ff8abcf4aa0542258a53df 94d1b3b4119209efcdd08df0dceaecbd1fe7f85c Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-24T09:44:01+08:00 GitHub noreply@github.com 2024-03-24T09:44:01+08:00 Support build win release for SYCL (#6241) 94d1b3b4119209efcdd08df0dceaecbd1fe7f85c 95562175f83a49755ff6fd3bad09409417c8e6f9 Jared Van Bortel jared@nomic.ai 2024-03-23T18:48:02-04:00 GitHub noreply@github.com 2024-03-23T18:48:02-04:00 use _wfopen instead of fopen on Windows (#6248) 95562175f83a49755ff6fd3bad09409417c8e6f9 f482bb2e4920e544651fb832f2e0bcb4d2ff69ab Georgi Gerganov ggerganov@gmail.com 2024-03-23T21:35:23+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-23T21:35:23+02:00 gitignore : gguf-split f482bb2e4920e544651fb832f2e0bcb4d2ff69ab 1997577d5e121568ae39f538021733ccd4278c23 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-23T18:07:00+01:00 GitHub noreply@github.com 2024-03-23T18:07:00+01:00 common: llama_load_model_from_url split support (#6192) 1997577d5e121568ae39f538021733ccd4278c23 476b0251b27fb64c575507024a671e639d675594 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-23T18:00:38+01:00 GitHub noreply@github.com 2024-03-23T18:00:38+01:00 server: docs: `--threads` and `--threads`, `--ubatch-size`, `--log-disable` (#6254) 476b0251b27fb64c575507024a671e639d675594 21cad01b6e6e1a96f99391f95e8ea8ae25c8288e Julius Arkenberg arki05@users.noreply.github.com 2024-03-23T17:41:53+01:00 GitHub noreply@github.com 2024-03-23T18:41:53+02:00 llama : add grok-1 support (#6204) 21cad01b6e6e1a96f99391f95e8ea8ae25c8288e 1b26aebe4de4f048ac99996efd8a2c9af150904d Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-23T17:18:13+01:00 GitHub noreply@github.com 2024-03-23T17:18:13+01:00 split: add gguf-split in the make build target (#6262) 1b26aebe4de4f048ac99996efd8a2c9af150904d 50ccaf5eacb50a2ca378a4ef0dc7aeb45fead652 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-23T13:18:45+01:00 GitHub noreply@github.com 2024-03-23T13:18:45+01:00 server: flush stdout after logging in both text and json layout (#6253) 50ccaf5eacb50a2ca378a4ef0dc7aeb45fead652 56a00f0a2f48a85376f48b5ce77699df781631ae Johannes Gäßler johannesg@5d6.de 2024-03-23T01:24:36+01:00 GitHub noreply@github.com 2024-03-23T01:24:36+01:00 lookup: complement data from context with general text statistics (#5479) 56a00f0a2f48a85376f48b5ce77699df781631ae 92397d87a45a09b5449d845a64856f177cd7a920 Georgi Gerganov ggerganov@gmail.com 2024-03-22T21:10:39+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-22T21:10:39+02:00 common : default --hf-file to --model (#6234) 92397d87a45a09b5449d845a64856f177cd7a920 1d0331c12a2f2a6296b471232bd4e66fbf06e6a1 fraxy-v 65565042+fraxy-v@users.noreply.github.com 2024-03-22T20:49:06+02:00 GitHub noreply@github.com 2024-03-22T20:49:06+02:00 convert-llama2c-to-ggml : enable conversion of GQA models (#6237) 1d0331c12a2f2a6296b471232bd4e66fbf06e6a1 dba1af612926cbd4ebe2d876277af1e3305177e0 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-22T19:47:14+01:00 GitHub noreply@github.com 2024-03-22T20:47:14+02:00 quantize: options for output and token embedding tensors qtype (#6239) dba1af612926cbd4ebe2d876277af1e3305177e0 ee804f6223777019cf921e0d99cc24669313ab98 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-22T19:00:01+01:00 GitHub noreply@github.com 2024-03-22T19:00:01+01:00 llama_model_loader: support multiple split/shard GGUFs (#6187) ee804f6223777019cf921e0d99cc24669313ab98 80bd33bc2c4be352697dc8473339f25e1085d117 Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-03-23T02:15:06+09:00 GitHub noreply@github.com 2024-03-22T19:15:06+02:00 ci: apply concurrency limit for github workflows (#6243) 80bd33bc2c4be352697dc8473339f25e1085d117 e80f06d2a194be62ab5b1cd7ef7c7a5b241dd4fb Georgi Gerganov ggerganov@gmail.com 2024-03-22T15:33:38+02:00 GitHub noreply@github.com 2024-03-22T15:33:38+02:00 common : add HF arg helpers (#6234) e80f06d2a194be62ab5b1cd7ef7c7a5b241dd4fb f77a8ffd3bbde77b7819823b0c006fd8c2d5cae4 Nexesenex 124105151+Nexesenex@users.noreply.github.com 2024-03-22T14:32:02+01:00 GitHub noreply@github.com 2024-03-22T15:32:02+02:00 llama : correction of the attn.v.weight quantization for IQ3_XS (#6209) f77a8ffd3bbde77b7819823b0c006fd8c2d5cae4 72114edf068a9b2f54d3b09e9a198f611be397e8 Olivier Chafik ochafik@users.noreply.github.com 2024-03-22T13:09:07Z GitHub noreply@github.com 2024-03-22T15:09:07+02:00 tests : conditional python & node json schema tests (#6207) 72114edf068a9b2f54d3b09e9a198f611be397e8 2f0e81e053b41ca28e73a841e7bdbf9820baaa57 Olivier Chafik ochafik@users.noreply.github.com 2024-03-22T13:07:44Z GitHub noreply@github.com 2024-03-22T15:07:44+02:00 json-schema-to-grammar : fix order of props + non-str const/enum (#6232) 2f0e81e053b41ca28e73a841e7bdbf9820baaa57 29ab270e65975785cdca3243a3de71ccebc1252a slaren slarengh@gmail.com 2024-03-22T14:05:31+01:00 GitHub noreply@github.com 2024-03-22T14:05:31+01:00 cuda : add LLAMA_CUDA_NO_PEER_COPY to workaround broken ROCm p2p copy (#6208) 29ab270e65975785cdca3243a3de71ccebc1252a 6b8bb3a31d260a01020497c5f01025c34222fcb9 Xiaoyi Chen cxychina@gmail.com 2024-03-22T04:29:49-07:00 GitHub noreply@github.com 2024-03-22T13:29:49+02:00 readme : add RecurseChat to the list of UIs (#6219) 6b8bb3a31d260a01020497c5f01025c34222fcb9 68e210b3543e0cc71268bee0920441747679ee13 Jan Boon kaetemi@gmail.com 2024-03-22T19:12:05+08:00 GitHub noreply@github.com 2024-03-22T13:12:05+02:00 server : fix n_keep always showing as 0 in response (#6211) 68e210b3543e0cc71268bee0920441747679ee13 b3e94f26bab8e3b5338b5902e5af5bb01894cb4a Georgi Gerganov ggerganov@gmail.com 2024-03-22T13:08:28+02:00 GitHub noreply@github.com 2024-03-22T13:08:28+02:00 server : enable continuous batching by default (#6231) b3e94f26bab8e3b5338b5902e5af5bb01894cb4a b2075fd6a578f5685060df4baa90ae9e48e98c70 Georgi Gerganov ggerganov@gmail.com 2024-03-22T11:35:53+02:00 GitHub noreply@github.com 2024-03-22T11:35:53+02:00 metal : proper assert for mat-mat memory alignment (#6225) b2075fd6a578f5685060df4baa90ae9e48e98c70 95d576b48ebf582b112d1c9cf4eed7142fa4e464 Vaibhav Srivastav vaibhavs10@gmail.com 2024-03-22T08:53:43+01:00 GitHub noreply@github.com 2024-03-22T09:53:43+02:00 ci : add CURL flag for the mac builds (#6214) 95d576b48ebf582b112d1c9cf4eed7142fa4e464 59c17f02de8fdf7b084d6100b875b7e2bc07a83b Georgi Gerganov ggerganov@gmail.com 2024-03-22T09:36:03+02:00 GitHub noreply@github.com 2024-03-22T09:36:03+02:00 metal : pad n_ctx by 32 (#6177) 59c17f02de8fdf7b084d6100b875b7e2bc07a83b fa046eafbc70bf97dcf39843af0323f19a8c9ac3 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-22T15:19:37+08:00 GitHub noreply@github.com 2024-03-22T15:19:37+08:00 add blog link (#6222) fa046eafbc70bf97dcf39843af0323f19a8c9ac3 be07a03217376c53b1d95e5f658adbbbb2831555 DAN™ dranger003@gmail.com 2024-03-21T21:32:42-04:00 GitHub noreply@github.com 2024-03-22T02:32:42+01:00 Fix params underscore convert to dash. (#6203) be07a03217376c53b1d95e5f658adbbbb2831555 d0a71233fbf8ade8ef06ad8e6b81d1d7b254895f Jan Boon kaetemi@gmail.com 2024-03-22T06:41:24+08:00 GitHub noreply@github.com 2024-03-21T23:41:24+01:00 server : update readme doc from `slot_id` to `id_slot` (#6213) d0a71233fbf8ade8ef06ad8e6b81d1d7b254895f f372c49ccdc561ab96fb3c7d2b7cbc0f89a4b359 slaren slarengh@gmail.com 2024-03-21T19:54:28+01:00 GitHub noreply@github.com 2024-03-21T20:54:28+02:00 cuda : disable host register by default (#6206) f372c49ccdc561ab96fb3c7d2b7cbc0f89a4b359 924ce1dce7fdf54894b462d03e7b608a6e574c32 semidark me@semidark.net 2024-03-21T11:52:35-06:00 GitHub noreply@github.com 2024-03-21T18:52:35+01:00 Corrected typo to wrong file (#6199) 924ce1dce7fdf54894b462d03e7b608a6e574c32 03a8f8fafec2e21009be9fe7297d92e5d4538964 Georgi Gerganov ggerganov@gmail.com 2024-03-21T16:20:05+02:00 GitHub noreply@github.com 2024-03-21T16:20:05+02:00 tests : disable system() calls (#6198) 03a8f8fafec2e21009be9fe7297d92e5d4538964 cfd3be76e37dab92c846d75a2421178f20db4a11 slaren slarengh@gmail.com 2024-03-21T13:59:53+01:00 GitHub noreply@github.com 2024-03-21T14:59:53+02:00 cuda : fix LLAMA_CUDA_F16 build (#6197) cfd3be76e37dab92c846d75a2421178f20db4a11 5b7b0ac8dfdd800c0fd0dc69b69991e8cb19fb46 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-21T13:59:38+01:00 GitHub noreply@github.com 2024-03-21T14:59:38+02:00 ggml : same IQ4_NL quantization for CPU/CUDA/Metal (#6196) 5b7b0ac8dfdd800c0fd0dc69b69991e8cb19fb46 1943c0198125a0da1a200390e82cf461f9080d99 Olivier Chafik ochafik@users.noreply.github.com 2024-03-21T11:50:43Z GitHub noreply@github.com 2024-03-21T11:50:43Z json-schema-to-grammar improvements (+ added to server) (#5978) 1943c0198125a0da1a200390e82cf461f9080d99 5e43ba87429d85acbde97d16b2f48d9de992cc80 Vaibhav Srivastav vaibhavs10@gmail.com 2024-03-21T10:30:40+01:00 GitHub noreply@github.com 2024-03-21T11:30:40+02:00 ci : fix indentation error (#6195) 5e43ba87429d85acbde97d16b2f48d9de992cc80 76aa30a26353f597e4fbe3cf776772ae812af89a Vaibhav Srivastav vaibhavs10@gmail.com 2024-03-21T10:13:12+01:00 GitHub noreply@github.com 2024-03-21T11:13:12+02:00 build : add mac pre-build binaries (#6182) 76aa30a26353f597e4fbe3cf776772ae812af89a c5b8595e3f4f4ed319ef71c9c9d868d1b7a27626 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-21T08:27:57+01:00 GitHub noreply@github.com 2024-03-21T08:27:57+01:00 Add ability to use Q5_0, Q5_1, and IQ4_NL for quantized K cache (#6183) c5b8595e3f4f4ed319ef71c9c9d868d1b7a27626 42e21c68826f2e56b9592dccd9f3c43895b6890d AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-03-21T06:10:52Z GitHub noreply@github.com 2024-03-21T11:40:52+05:30 Add nvidia and amd backends (#6157) 42e21c68826f2e56b9592dccd9f3c43895b6890d 1c51f98adcbad40e3c41f0a6ffadeb723190b417 slaren slarengh@gmail.com 2024-03-21T01:47:46+01:00 GitHub noreply@github.com 2024-03-21T01:47:46+01:00 cuda : fix conflict with std::swap (#6186) 1c51f98adcbad40e3c41f0a6ffadeb723190b417 f9c7ba34476ffc4f13ae2cdb1aec493a16eb8d47 slaren slarengh@gmail.com 2024-03-20T21:03:26+01:00 GitHub noreply@github.com 2024-03-20T21:03:26+01:00 cuda : print the returned error when CUDA initialization fails (#6185) f9c7ba34476ffc4f13ae2cdb1aec493a16eb8d47 272935b281fee5c683e3d6d1eb580b84553cf503 Ziang Wu 97337387+ZiangWu-77@users.noreply.github.com 2024-03-20T23:29:51+08:00 GitHub noreply@github.com 2024-03-20T17:29:51+02:00 llava : update MobileVLM-README.md (#6180) 272935b281fee5c683e3d6d1eb580b84553cf503 ccf58aa3ec4d20b10162ba40898dc038ad4c3fad Ziang Wu 97337387+ZiangWu-77@users.noreply.github.com 2024-03-20T23:02:32+08:00 GitHub noreply@github.com 2024-03-20T17:02:32+02:00 llava : add MobileVLM_V2 backup (#6175) ccf58aa3ec4d20b10162ba40898dc038ad4c3fad 91f8ad167dcd24b54615b468d9dd764ebe1d37ad slaren slarengh@gmail.com 2024-03-20T14:42:59+01:00 GitHub noreply@github.com 2024-03-20T14:42:59+01:00 cuda : refactor to remove global resources (#6170) 91f8ad167dcd24b54615b468d9dd764ebe1d37ad 6b7e76d28cdf4361740054140708c71828453522 Xuan Son Nguyen thichthat@gmail.com 2024-03-20T13:30:36+01:00 GitHub noreply@github.com 2024-03-20T13:30:36+01:00 Server: version bump for httplib and json (#6169) 6b7e76d28cdf4361740054140708c71828453522 bc0baab2ea8f806961dd3fb9f534cfeb59a2e1fc Georgi Gerganov ggerganov@gmail.com 2024-03-20T14:17:34+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-20T14:17:34+02:00 gitignore : ignore curl-related files bc0baab2ea8f806961dd3fb9f534cfeb59a2e1fc d795988d9e09f75412efe2134512914c42780ad5 Georgi Gerganov ggerganov@gmail.com 2024-03-20T14:14:32+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-20T14:14:32+02:00 server : allow to override -ngl in tests (#6170) d795988d9e09f75412efe2134512914c42780ad5 f8c4e745e1e728204ab26dbadf52853545e6789c Georgi Gerganov ggerganov@gmail.com 2024-03-20T13:29:49+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-20T13:29:49+02:00 Revert "llava : add a MobileVLM_V2-1.7B backup (#6152)" f8c4e745e1e728204ab26dbadf52853545e6789c 47cc7a7bf9793f81a6dfe7c2096c499403f64dd6 Ziang Wu 97337387+ZiangWu-77@users.noreply.github.com 2024-03-20T19:20:37+08:00 GitHub noreply@github.com 2024-03-20T13:20:37+02:00 llava : add a MobileVLM_V2-1.7B backup (#6152) 47cc7a7bf9793f81a6dfe7c2096c499403f64dd6 bd60d82d0cc8b6852ec535495a5042dbdf05de24 Karthick j.karthic2004@gmail.com 2024-03-20T16:32:34+05:30 GitHub noreply@github.com 2024-03-20T12:02:34+01:00 Server: Handle n_keep parameter in the request (#6174) bd60d82d0cc8b6852ec535495a5042dbdf05de24 6c0b287748327741b113d7d6018b68c63039b1c5 Jared Van Bortel jared@nomic.ai 2024-03-20T01:33:49-04:00 GitHub noreply@github.com 2024-03-20T06:33:49+01:00 server tests : more pythonic process management; fix bare `except:` (#6146) 6c0b287748327741b113d7d6018b68c63039b1c5 d26e8b669dbf1f5f5a0afe4d2d885e86cf566302 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-20T11:21:41+08:00 GitHub noreply@github.com 2024-03-20T11:21:41+08:00 update readme sycl for new update (#6151) d26e8b669dbf1f5f5a0afe4d2d885e86cf566302 d8b009a9456bf5284376149f3deb09300a37701a Abhilash Majumder 30946547+abhilash1910@users.noreply.github.com 2024-03-20T08:28:49+05:30 GitHub noreply@github.com 2024-03-20T08:28:49+05:30 increase igpu cluster limit (#6159) d8b009a9456bf5284376149f3deb09300a37701a d0d5de42e5a65865b5fddb6f5c785083539b74c3 DAN™ dranger003@gmail.com 2024-03-19T12:16:09-04:00 GitHub noreply@github.com 2024-03-19T17:16:09+01:00 Remove undeed header file. (#6158) d0d5de42e5a65865b5fddb6f5c785083539b74c3 b80cf3b2d1dee0ad325f7a794fecc66befce7336 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-19T12:05:44+01:00 GitHub noreply@github.com 2024-03-19T12:05:44+01:00 gguf-split: split and merge gguf per batch of tensors (#6135) b80cf3b2d1dee0ad325f7a794fecc66befce7336 970a48060ab9a6cc67aa063870323781c2a7bd7d Georgi Gerganov ggerganov@gmail.com 2024-03-19T10:21:54+02:00 GitHub noreply@github.com 2024-03-19T10:21:54+02:00 common : disable repeat penalties by default (#6127) 970a48060ab9a6cc67aa063870323781c2a7bd7d 4c28b8252907561165827125d2d1a4bad6926ac6 slaren slarengh@gmail.com 2024-03-19T09:06:54+01:00 GitHub noreply@github.com 2024-03-19T10:06:54+02:00 ci : exempt some labels from being tagged as stale (#6140) 4c28b8252907561165827125d2d1a4bad6926ac6 2d15886bb092c3b780c676b5cc57ff3337af9c83 DAN™ dranger003@gmail.com 2024-03-19T01:59:36-04:00 GitHub noreply@github.com 2024-03-19T07:59:36+02:00 common : print usage on '-h' and '--help' (#6145) 2d15886bb092c3b780c676b5cc57ff3337af9c83 d199ca79f279e84ebe27caafe0aa59c461d88969 github-actions[bot] github-actions[bot]@users.noreply.github.com 2024-03-17T06:37:44Z Someone newkozlukov@gmail.com 2024-03-18T18:51:30Z flake.lock: Update d199ca79f279e84ebe27caafe0aa59c461d88969 104f5e0fc156d48476258295457cafeec2a2af10 Jared Van Bortel jared@nomic.ai 2024-03-18T12:49:02-04:00 GitHub noreply@github.com 2024-03-18T12:49:02-04:00 mpt : implement backwards compatiblity with duped output tensor (#6139) 104f5e0fc156d48476258295457cafeec2a2af10 5e1b7f94a03e0b3b8e4578625bbdadc7bbd2b93c Felix stenbackfelix@gmail.com 2024-03-18T16:40:22+01:00 GitHub noreply@github.com 2024-03-18T17:40:22+02:00 clip : fix memory leak (#6138) 5e1b7f94a03e0b3b8e4578625bbdadc7bbd2b93c ac9ee6a4ad740bc1ee484ede43e9f92b5af244c1 slaren slarengh@gmail.com 2024-03-18T16:33:44+01:00 GitHub noreply@github.com 2024-03-18T16:33:44+01:00 backend : set max split inputs to GGML_MAX_SRC (#6137) ac9ee6a4ad740bc1ee484ede43e9f92b5af244c1 4f6d1337ca5a409dc74aca8c479b7c34408a69c0 Georgi Gerganov ggerganov@gmail.com 2024-03-18T13:45:38+02:00 GitHub noreply@github.com 2024-03-18T13:45:38+02:00 ci : disable stale issue messages (#6126) 4f6d1337ca5a409dc74aca8c479b7c34408a69c0 2bf8d0f7c4cc1235755ad06961ca761e458c5e55 Georgi Gerganov ggerganov@gmail.com 2024-03-18T13:45:27+02:00 GitHub noreply@github.com 2024-03-18T13:45:27+02:00 ci : temporary disable sanitizer builds (#6128) 2bf8d0f7c4cc1235755ad06961ca761e458c5e55 496bc79bc2b79bfd6124b8687a8dbd6a646e9b06 slaren slarengh@gmail.com 2024-03-18T11:03:04+01:00 GitHub noreply@github.com 2024-03-18T11:03:04+01:00 backend : offload large batches to GPU (#6083) 496bc79bc2b79bfd6124b8687a8dbd6a646e9b06 9b03719ad712e2dc36c5c0c20f352bf3e4bda332 DAN™ dranger003@gmail.com 2024-03-18T04:27:44-04:00 GitHub noreply@github.com 2024-03-18T10:27:44+02:00 common : tidy-up argument parsing (#6105) 9b03719ad712e2dc36c5c0c20f352bf3e4bda332 3a6efdd03c46c5ba08e43880d34260c02dd9999b Thérence 13496987+Royalphax@users.noreply.github.com 2024-03-18T09:17:00+01:00 GitHub noreply@github.com 2024-03-18T10:17:00+02:00 convert : add support for CamembertModel architecture (#6119) 3a6efdd03c46c5ba08e43880d34260c02dd9999b d01b3c4c32357567f3531d4e6ceffc5d23e87583 Romain D 90720+Artefact2@users.noreply.github.com 2024-03-18T09:04:41+01:00 GitHub noreply@github.com 2024-03-18T10:04:41+02:00 convert : use f32 outtype for bf16 tensors (#6106) d01b3c4c32357567f3531d4e6ceffc5d23e87583 cd776c37c945bf58efc8fe44b370456680cb1b59 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-17T19:12:37+01:00 GitHub noreply@github.com 2024-03-17T19:12:37+01:00 common: llama_load_model_from_url using --model-url (#6098) cd776c37c945bf58efc8fe44b370456680cb1b59 dc0f6125487dcfbff913360f9d877bc0ccf6aa57 Georgi Gerganov ggerganov@gmail.com 2024-03-17T19:51:57+02:00 GitHub noreply@github.com 2024-03-17T18:51:57+01:00 ci : close all stale issues at once (#6115) dc0f6125487dcfbff913360f9d877bc0ccf6aa57 c47cf414efafb8f60596edc7edb5a2d68065e992 GainLee perfecter.gen@gmail.com 2024-03-18T01:12:22+08:00 GitHub noreply@github.com 2024-03-17T18:12:22+01:00 ggml:fix finding transfer queue family index error (#6094) c47cf414efafb8f60596edc7edb5a2d68065e992 b5f4ae09c3244ae1644b67c03ed9f4227ab25ad2 AmirAli Mirian 37371367+amiralimi@users.noreply.github.com 2024-03-16T11:52:02-04:00 GitHub noreply@github.com 2024-03-16T17:52:02+02:00 ggml : add AVX512F SIMD (#6088) b5f4ae09c3244ae1644b67c03ed9f4227ab25ad2 dfbfdd60f90207404039c6578d709231496831d9 Daniel Bevenius daniel.bevenius@gmail.com 2024-03-16T16:46:29+01:00 GitHub noreply@github.com 2024-03-16T17:46:29+02:00 gritlm : add initial README.md (#6086) dfbfdd60f90207404039c6578d709231496831d9 15961ec04dbd59d21d8984d42e4c0f7e7e7d320a Xuan Son Nguyen thichthat@gmail.com 2024-03-16T16:42:08+01:00 GitHub noreply@github.com 2024-03-16T17:42:08+02:00 readme : add wllama as a wasm binding (#6100) 15961ec04dbd59d21d8984d42e4c0f7e7e7d320a a56d09a4407f29c21e149b44fd5308f83aa1cb09 DAN™ dranger003@gmail.com 2024-03-16T11:39:15-04:00 GitHub noreply@github.com 2024-03-16T17:39:15+02:00 common : refactor nested if causing error C1061 on MSVC (#6101) a56d09a4407f29c21e149b44fd5308f83aa1cb09 d84c48505f60bcd358b82a751d40418c4d235643 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-16T13:20:53+01:00 GitHub noreply@github.com 2024-03-16T14:20:53+02:00 ci : close inactive issue with workflow (#6053) d84c48505f60bcd358b82a751d40418c4d235643 877b4d0c628cc70dddb5df72ed8fc14d126ca7e8 slaren slarengh@gmail.com 2024-03-15T22:14:16+01:00 GitHub noreply@github.com 2024-03-15T23:14:16+02:00 llama : fix Baichuan2 13B (#6092) 877b4d0c628cc70dddb5df72ed8fc14d126ca7e8 12247f4c69a173b9482f68aaa174ec37fc909ccf Theia Vogel theia@vgel.me 2024-03-15T13:43:02-07:00 GitHub noreply@github.com 2024-03-15T22:43:02+02:00 llama : add support for control vectors (#5970) 12247f4c69a173b9482f68aaa174ec37fc909ccf 4e9a7f7f7fb6acbddd1462909c8d696e38edbfcc Andrew Canis andrew.canis@gmail.com 2024-03-15T16:41:22-04:00 GitHub noreply@github.com 2024-03-15T22:41:22+02:00 llama : add Command-R support (#6033) 4e9a7f7f7fb6acbddd1462909c8d696e38edbfcc 3020327f6cd6d2ce50528dd65f4b199d2ea8b1ae Ting Lou ting.lou@gmail.com 2024-03-15T22:31:05+08:00 GitHub noreply@github.com 2024-03-15T16:31:05+02:00 llava : change API to pure C style for Rust FFI bindgen (#6079) 3020327f6cd6d2ce50528dd65f4b199d2ea8b1ae 46acb3676718b983157058aecf729a2064fc7d34 slaren slarengh@gmail.com 2024-03-15T13:24:03+01:00 GitHub noreply@github.com 2024-03-15T14:24:03+02:00 cuda : disable unused cudaLaunchHostFunc code (#6078) 46acb3676718b983157058aecf729a2064fc7d34 131b0584096ee9df4d07cb28759dfea6efe6475f Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-15T18:53:53+08:00 GitHub noreply@github.com 2024-03-15T18:53:53+08:00 fix set main gpu error (#6073) 131b0584096ee9df4d07cb28759dfea6efe6475f 753e36f650fa2a5869f89188d9ee745dc74cf14b Georgi Gerganov ggerganov@gmail.com 2024-03-15T11:36:50+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-15T11:38:40+02:00 make : ggml-metal.o depends on ggml.h 753e36f650fa2a5869f89188d9ee745dc74cf14b 7ce2c77f88e1ca66ec48417e56f91746bac018c2 AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-03-15T09:26:20Z GitHub noreply@github.com 2024-03-15T14:56:20+05:30 [SYCL] Fix non-intel device selection (#6042) 7ce2c77f88e1ca66ec48417e56f91746bac018c2 aab606a11fc0a9740a7f297521c3eef851dfb351 Ondřej Čertík ondrej@certik.us 2024-03-15T02:46:51-06:00 GitHub noreply@github.com 2024-03-15T10:46:51+02:00 gguf : add support for I64 and F64 arrays (#6062) aab606a11fc0a9740a7f297521c3eef851dfb351 b0bc9f4a9da7c19f4779106ea83b23feca747566 Xuan Son Nguyen thichthat@gmail.com 2024-03-15T09:44:57+01:00 GitHub noreply@github.com 2024-03-15T10:44:57+02:00 llama : add Orion chat template (#6066) b0bc9f4a9da7c19f4779106ea83b23feca747566 4755afd1cbd40d93c017e5b98c39796f52345314 slaren slarengh@gmail.com 2024-03-15T09:22:24+01:00 GitHub noreply@github.com 2024-03-15T10:22:24+02:00 llama-bench : use random tokens to improve accuracy with mixtral (#6069) 4755afd1cbd40d93c017e5b98c39796f52345314 6e0438da3cc95b89cdbf55f45fa4e324d9076792 Georgi Gerganov ggerganov@gmail.com 2024-03-14T22:58:41+02:00 GitHub noreply@github.com 2024-03-14T22:58:41+02:00 llama : fix integer overflow during quantization (#6063) 6e0438da3cc95b89cdbf55f45fa4e324d9076792 727107707a73b3dc8a497cf9fc9405722c16dd2b Steve Grubb ausearch.1@gmail.com 2024-03-14T14:29:32-04:00 GitHub noreply@github.com 2024-03-14T20:29:32+02:00 gguf : fix resource leaks (#6061) 727107707a73b3dc8a497cf9fc9405722c16dd2b 69ff61397d2b7b550dcdda4a35b35128892408b0 Ondřej Čertík ondrej@certik.us 2024-03-14T11:57:31-06:00 GitHub noreply@github.com 2024-03-14T19:57:31+02:00 gguf-py : bump version to 0.8.0 (#6060) 69ff61397d2b7b550dcdda4a35b35128892408b0 044ec4b2a567f649459ccd20af2f387c784faa51 Michael Podvitskiy podvitskiymichael@gmail.com 2024-03-14T17:21:56+01:00 GitHub noreply@github.com 2024-03-14T18:21:56+02:00 llama : support models without vocabulary (#5798) 044ec4b2a567f649459ccd20af2f387c784faa51 77178eedc83d49f31bf757d8e12315d76460be78 Georgi Gerganov ggerganov@gmail.com 2024-03-14T15:14:14+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-14T15:14:14+02:00 embedding : add EOS token if not present (#899) 77178eedc83d49f31bf757d8e12315d76460be78 15a333260ab637a040ed0864c206a2ceaf806bb8 Georgi Gerganov ggerganov@gmail.com 2024-03-14T13:32:14+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-14T13:32:14+02:00 gguf-py : fix dtype check (#6045) 15a333260ab637a040ed0864c206a2ceaf806bb8 43241adf22e8231ffaf3827d2c9310cc0ffd5ac5 Jian Liao jianliao@users.noreply.github.com 2024-03-14T04:18:23-07:00 GitHub noreply@github.com 2024-03-14T13:18:23+02:00 readme : improve readme for Llava-1.6 example (#6044) 43241adf22e8231ffaf3827d2c9310cc0ffd5ac5 a44bc969e4cd62ca9f4332e17fe3c51f2093e7c6 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-14T12:15:39+01:00 GitHub noreply@github.com 2024-03-14T13:15:39+02:00 server: disable debug release type sanitizer, simplify trigger (#6047) a44bc969e4cd62ca9f4332e17fe3c51f2093e7c6 2c4fb69246834503db7b78bcbedcef506bbc60c4 Georgi Gerganov ggerganov@gmail.com 2024-03-14T13:13:06+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-14T13:13:06+02:00 llama : fix typo 2c4fb69246834503db7b78bcbedcef506bbc60c4 3ca23481dd309bd51cc31c73a4cc34f922cc372f Michael Podvitskiy podvitskiymichael@gmail.com 2024-03-14T11:56:48+01:00 GitHub noreply@github.com 2024-03-14T12:56:48+02:00 llama : optimize defrag moves + fix fragmentation calculation (#6037) 3ca23481dd309bd51cc31c73a4cc34f922cc372f 3fe8d7a17f84bd721cd4d8db35365da44b69f68b Ondřej Čertík ondrej@certik.us 2024-03-14T04:40:14-06:00 GitHub noreply@github.com 2024-03-14T12:40:14+02:00 gguf-py : add support for I8, I16 and I32 (#6045) 3fe8d7a17f84bd721cd4d8db35365da44b69f68b 68265ebfc6a1bed022973ea0c3145be1450b7e70 Georgi Gerganov ggerganov@gmail.com 2024-03-14T12:38:37+02:00 GitHub noreply@github.com 2024-03-14T12:38:37+02:00 ggml : designate enum vals for integer types (#6050) 68265ebfc6a1bed022973ea0c3145be1450b7e70 381da2d9f0940d7009e3e918bed36338c8ff2fbb Georgi Gerganov ggerganov@gmail.com 2024-03-14T12:37:20+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-14T12:37:20+02:00 embedding : print all resulting embeddings (#899) 381da2d9f0940d7009e3e918bed36338c8ff2fbb 0fd6c1f015f6cccf3b527f7dbd8386a434728126 Georgi Gerganov ggerganov@gmail.com 2024-03-14T11:55:23+02:00 GitHub noreply@github.com 2024-03-14T11:55:23+02:00 metal : build metallib + fix embed path (#6015) 0fd6c1f015f6cccf3b527f7dbd8386a434728126 19885d205e768579ab090d1e99281cae58c21b54 Georgi Gerganov ggerganov@gmail.com 2024-03-14T10:12:29+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-14T10:12:29+02:00 embedding : print cosine similarity (#899) 19885d205e768579ab090d1e99281cae58c21b54 76a936c8939c249a7c3e8e66dfefbab13eae194f Linwei Wang wanix1988@gmail.com 2024-03-14T02:34:40+08:00 GitHub noreply@github.com 2024-03-13T20:34:40+02:00 readme : update details about running llama in Termux on Android (#6039) 76a936c8939c249a7c3e8e66dfefbab13eae194f 463628372d5fe3a0c1e5864aa5fc57deb7387039 Georgi Gerganov ggerganov@gmail.com 2024-03-13T20:33:56+02:00 GitHub noreply@github.com 2024-03-13T20:33:56+02:00 readme : update API changes and hot topics 463628372d5fe3a0c1e5864aa5fc57deb7387039 f30ea47a87ed4446ad55adb265755dc9102956a2 Clint Herron hanclinto@gmail.com 2024-03-13T14:10:40-04:00 GitHub noreply@github.com 2024-03-13T20:10:40+02:00 grammar : handle missing "root" node (#6004) f30ea47a87ed4446ad55adb265755dc9102956a2 d8fd0ccf6ac8b07791ffd1575eed436930854ae3 slaren slarengh@gmail.com 2024-03-13T18:54:21+01:00 GitHub noreply@github.com 2024-03-13T18:54:21+01:00 llama : add pipeline parallelism support (#6017) d8fd0ccf6ac8b07791ffd1575eed436930854ae3 b3d978600f07f22e94f2e797f18a8b5f6df23c89 slaren slarengh@gmail.com 2024-03-13T14:58:30+01:00 GitHub noreply@github.com 2024-03-13T15:58:30+02:00 test-backend-ops : skip CPU backend by default (#6028) b3d978600f07f22e94f2e797f18a8b5f6df23c89 99b71c068f624521ad977e08e41589e2971fa1c7 AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-03-13T13:17:54Z GitHub noreply@github.com 2024-03-13T18:47:54+05:30 Update get version (#6025) 99b71c068f624521ad977e08e41589e2971fa1c7 306d34be7ad19e768975409fc80791a274ea0230 Xuan Son Nguyen thichthat@gmail.com 2024-03-13T11:39:11+01:00 GitHub noreply@github.com 2024-03-13T11:39:11+01:00 Server: Use multi-task for embeddings endpoint (#6001) 306d34be7ad19e768975409fc80791a274ea0230 8030da7afea2d89f997aeadbd14183d399a017b9 slaren slarengh@gmail.com 2024-03-12T16:55:19+01:00 GitHub noreply@github.com 2024-03-12T17:55:19+02:00 ci : remove tidy-review (#6021) 8030da7afea2d89f997aeadbd14183d399a017b9 184215e783dfad76aded2c68244c327a5c507df5 Georgi Gerganov ggerganov@gmail.com 2024-03-12T14:27:20+02:00 GitHub noreply@github.com 2024-03-12T14:27:20+02:00 ggml : reuse quantum structs across backends (#5943) 184215e783dfad76aded2c68244c327a5c507df5 48358b2e5b3983c41ba7e61a493e84d3901dc7b9 Georgi Gerganov ggerganov@gmail.com 2024-03-12T13:49:55+02:00 GitHub noreply@github.com 2024-03-12T13:49:55+02:00 ggml : fix UB in IQ2_S and IQ3_S (#6012) 48358b2e5b3983c41ba7e61a493e84d3901dc7b9 5cdb371731caa2c41fcca42d4d2d43f94f6883b4 Georgi Gerganov ggerganov@gmail.com 2024-03-12T11:15:05+02:00 GitHub noreply@github.com 2024-03-12T11:15:05+02:00 sycl : update IQ1_S kernels (WIP - not working!) (#5995) 5cdb371731caa2c41fcca42d4d2d43f94f6883b4 44ca159faf4fbe1a7ace13a962845ba7cdfd95ec gliptic gliptic@users.noreply.github.com 2024-03-11T20:59:03+01:00 GitHub noreply@github.com 2024-03-11T21:59:03+02:00 grammar : fix unnecessarily retained pointer to rules (#6003) 44ca159faf4fbe1a7ace13a962845ba7cdfd95ec 05b06210c954491cf0f12034b0a62bd4d69ce78b Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-11T16:53:15+01:00 GitHub noreply@github.com 2024-03-11T17:53:15+02:00 1.5 bit: we can do even better (#5999) 05b06210c954491cf0f12034b0a62bd4d69ce78b 83796e62bc9f6caae6228168e359890f51e60fee Georgi Gerganov ggerganov@gmail.com 2024-03-11T17:49:47+02:00 GitHub noreply@github.com 2024-03-11T17:49:47+02:00 llama : more consistent names of count variables (#5994) 83796e62bc9f6caae6228168e359890f51e60fee 828defefb66fc8a25404f5de845897145bf34061 Georgi Gerganov ggerganov@gmail.com 2024-03-11T17:47:47+02:00 GitHub noreply@github.com 2024-03-11T17:47:47+02:00 llama : refactor unicode stuff (#5992) 828defefb66fc8a25404f5de845897145bf34061 caa106d4e05a0ab94225c220b81f9e2cd522339b Jakub N jakubniemczyk97@gmail.com 2024-03-11T14:40:42+01:00 GitHub noreply@github.com 2024-03-11T14:40:42+01:00 Update server docker image URLs (#5997) caa106d4e05a0ab94225c220b81f9e2cd522339b 3202361c5b1ba15e695b31209567ef42c22c5c32 Xuan Son Nguyen thichthat@gmail.com 2024-03-11T10:56:41+01:00 GitHub noreply@github.com 2024-03-11T10:56:41+01:00 Server: format error to json (#5961) 3202361c5b1ba15e695b31209567ef42c22c5c32 332bdfd7980718abf664bfa5460f2288a3314984 Michael Podvitskiy podvitskiymichael@gmail.com 2024-03-11T10:28:51+01:00 GitHub noreply@github.com 2024-03-11T11:28:51+02:00 ggml, ci : Windows ARM runner and build fixes (#5979) 332bdfd7980718abf664bfa5460f2288a3314984 ecab1c75de68de7c41c254e2ae170d3b07bee6d4 Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-03-11T17:09:32+09:00 GitHub noreply@github.com 2024-03-11T10:09:32+02:00 server : maintain chat completion id for streaming responses (#5988) ecab1c75de68de7c41c254e2ae170d3b07bee6d4 ee35600b9061b1ea0c4ea87fce6844297632b2a8 Gilad S giladgd@users.noreply.github.com 2024-03-11T10:00:08+02:00 GitHub noreply@github.com 2024-03-11T10:00:08+02:00 cmake : fix subdir for `LLAMA_METAL_EMBED_LIBRARY` (#5985) ee35600b9061b1ea0c4ea87fce6844297632b2a8 be858f620508385ad12d0e5e862010e666ca729c Georgi Gerganov ggerganov@gmail.com 2024-03-11T09:56:47+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-11T09:56:47+02:00 llama : fix F16/F32 downcast + improve names (#5980) be858f620508385ad12d0e5e862010e666ca729c ef3ced26a3817d92890b97b83acaeb018ade02d0 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-11T07:51:49+01:00 GitHub noreply@github.com 2024-03-11T07:51:49+01:00 Better 1.5 bit quantization (#5971) ef3ced26a3817d92890b97b83acaeb018ade02d0 3814a07392d2bdc22911652bc7c2f9bdb0ce042e Abhilash Majumder 30946547+abhilash1910@users.noreply.github.com 2024-03-11T10:27:56+05:30 GitHub noreply@github.com 2024-03-11T10:27:56+05:30 [SYCL] Add q3_s and q1_s (#5886) 3814a07392d2bdc22911652bc7c2f9bdb0ce042e bb6d00bbf98476215596b9df3870b5504ef5a29a AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-03-11T01:13:57Z GitHub noreply@github.com 2024-03-11T09:13:57+08:00 [SYCL] Add support for SYCL Nvidia target (#5738) bb6d00bbf98476215596b9df3870b5504ef5a29a 7ab7b733bb48250b2df26c12b00256ef42c76932 Georgi Gerganov ggerganov@gmail.com 2024-03-10T23:12:48+02:00 GitHub noreply@github.com 2024-03-10T23:12:48+02:00 metal : move mm_id indices to shared mem (#5982) 7ab7b733bb48250b2df26c12b00256ef42c76932 d9f65c97c3dc3aa6fa27470b8c6e69b437ec1a27 Dean Dean.Sinaean@gmail.com 2024-03-11T04:03:17+08:00 GitHub noreply@github.com 2024-03-10T22:03:17+02:00 android : fix utf8 decoding error (#5935) d9f65c97c3dc3aa6fa27470b8c6e69b437ec1a27 b838b53ad6de2e53f23ddf8f3ad5e6891cc3dd05 Georgi Gerganov ggerganov@gmail.com 2024-03-10T20:58:26+02:00 GitHub noreply@github.com 2024-03-10T20:58:26+02:00 readme : update hot topics b838b53ad6de2e53f23ddf8f3ad5e6891cc3dd05 df4dc3e7cb43162862f339525638ba4febcb6158 Georgi Gerganov ggerganov@gmail.com 2024-03-10T20:10:46+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-10T20:10:46+02:00 sync : ggml df4dc3e7cb43162862f339525638ba4febcb6158 bf47a5eefc669fdba71af096942af999bc1167d4 Georgi Gerganov ggerganov@gmail.com 2024-03-08T23:45:07+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-10T20:10:39+02:00 ggml : try fix 32-bit arm compat (whisper/1938) bf47a5eefc669fdba71af096942af999bc1167d4 fa8a809a9119411bc9c3f00026550d0343f4d9b7 Georgi Gerganov ggerganov@gmail.com 2024-03-10T20:09:24+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-10T20:09:24+02:00 ggml : remove __constant__ specifier for CUDA tables (#5940) fa8a809a9119411bc9c3f00026550d0343f4d9b7 bcebd7dbf62fd7b293d5ed089023e4e733269c71 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-10T18:17:47+01:00 GitHub noreply@github.com 2024-03-10T18:17:47+01:00 server: ci: windows build and tests (#5968) bcebd7dbf62fd7b293d5ed089023e4e733269c71 2960eae847f8dbde23be6d170a61bcf44ebf32de DAN™ dranger003@gmail.com 2024-03-10T11:56:30-04:00 GitHub noreply@github.com 2024-03-10T17:56:30+02:00 llama : add support for GritLM (#5959) 2960eae847f8dbde23be6d170a61bcf44ebf32de c78541479cf835dd9eb568ccd9a2083198a7203d Clint Herron hanclinto@gmail.com 2024-03-10T11:17:43-04:00 GitHub noreply@github.com 2024-03-10T17:17:43+02:00 grammar : verify parsed state (#5950) c78541479cf835dd9eb568ccd9a2083198a7203d 621e86b331f8b0e71f79fd82a4ae1cd54c3e4396 Georgi Gerganov ggerganov@gmail.com 2024-03-10T16:43:08+02:00 GitHub noreply@github.com 2024-03-10T16:43:08+02:00 nix: update flake.lock (#5969) 621e86b331f8b0e71f79fd82a4ae1cd54c3e4396 77d1ac7e00bf049b9f2bba1b5a310a78318c49c4 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-09T23:41:49+01:00 GitHub noreply@github.com 2024-03-09T23:41:49+01:00 server: benchmark: chat/completions scenario and other llm servers comparison (#5941) 77d1ac7e00bf049b9f2bba1b5a310a78318c49c4 d894f352bf433157232dc8dc54eacd50014e898e Georgi Gerganov ggerganov@gmail.com 2024-03-09T22:04:00+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-09T22:04:00+02:00 server : print chat template info d894f352bf433157232dc8dc54eacd50014e898e 098dbaab449f5309a54871ba7e5acef72ae696de slaren slarengh@gmail.com 2024-03-09T19:55:54+01:00 GitHub noreply@github.com 2024-03-09T19:55:54+01:00 perplexity : support using multiple sequences to allow larger batch sizes (#5946) 098dbaab449f5309a54871ba7e5acef72ae696de 8380ecfb219c2e73cc706fcc83935b7c806cc7c3 Georgi Gerganov ggerganov@gmail.com 2024-03-09T18:14:13+02:00 GitHub noreply@github.com 2024-03-09T18:14:13+02:00 readme : update hot topics 8380ecfb219c2e73cc706fcc83935b7c806cc7c3 58308a0ecce7cc261b802f4803c38d420063db21 Georgi Gerganov ggerganov@gmail.com 2024-03-09T17:36:20+02:00 GitHub noreply@github.com 2024-03-09T17:36:20+02:00 ggml : fix unnecessary f32 -> f16 -> f32 casts (mmla) (#5951) 58308a0ecce7cc261b802f4803c38d420063db21 5b09797321430f08caf0473143a962916ab2ea89 Georgi Gerganov ggerganov@gmail.com 2024-03-09T17:34:15+02:00 GitHub noreply@github.com 2024-03-09T17:34:15+02:00 server : fix metrics init (#5964) 5b09797321430f08caf0473143a962916ab2ea89 97c09585d65a95864773b4d25d66d0f708baf38d Georgi Gerganov ggerganov@gmail.com 2024-03-09T15:53:59+02:00 GitHub noreply@github.com 2024-03-09T15:53:59+02:00 ggml : remove old quantization functions (#5942) 97c09585d65a95864773b4d25d66d0f708baf38d fb215c3832236fec7380c4fb618bd7154cb196ef Georgi Gerganov ggerganov@gmail.com 2024-03-09T15:47:47+02:00 GitHub noreply@github.com 2024-03-09T15:47:47+02:00 server : clarify some items in the readme (#5957) fb215c3832236fec7380c4fb618bd7154cb196ef 2c4f566c88322ebf2f9bd11b01b5ebdaa0130b89 SeungWon Jeong 65549245+redlion0929@users.noreply.github.com 2024-03-09T21:27:58+09:00 GitHub noreply@github.com 2024-03-09T14:27:58+02:00 server : normalize embeddings (#5956) 2c4f566c88322ebf2f9bd11b01b5ebdaa0130b89 0db32beaf09d90b8959d3d0cc493ed1e45685353 Georgi Gerganov ggerganov@gmail.com 2024-03-09T14:17:11+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-09T14:17:11+02:00 tests : gitignore ggml-common.h 0db32beaf09d90b8959d3d0cc493ed1e45685353 8a3012a4ad08112bb3dc3f1399afec4e93780c44 Alexey Parfenov zxed@alkatrazstudio.net 2024-03-09T11:16:53Z GitHub noreply@github.com 2024-03-09T13:16:53+02:00 server : fix passing prompt as tokens (#5955) 8a3012a4ad08112bb3dc3f1399afec4e93780c44 9674aaf35cb81478eb38c3f3ebde713ec72fbb79 Georgi Gerganov ggerganov@gmail.com 2024-03-09T12:47:57+02:00 GitHub noreply@github.com 2024-03-09T12:47:57+02:00 ggml : add ggml-common.h to deduplicate shared code (#5940) 9674aaf35cb81478eb38c3f3ebde713ec72fbb79 950ba1ab84db199f0bbdecdb2bb911f35261b321 Georgi Gerganov ggerganov@gmail.com 2024-03-09T12:34:18+02:00 GitHub noreply@github.com 2024-03-09T12:34:18+02:00 server : simplify logic for empty prompts (#5953) 950ba1ab84db199f0bbdecdb2bb911f35261b321 e1fa9569ba8ce276bc7801a3cebdcf8b1aa116ea Xuan Son Nguyen thichthat@gmail.com 2024-03-09T11:27:53+01:00 GitHub noreply@github.com 2024-03-09T11:27:53+01:00 Server: reorganize some http logic (#5939) e1fa9569ba8ce276bc7801a3cebdcf8b1aa116ea fd72d2d2a5e79d61ccef6af3d15f16e5e5cbc352 Gabe Goodhart gabe.l.hart@gmail.com 2024-03-09T02:57:09-07:00 GitHub noreply@github.com 2024-03-09T11:57:09+02:00 server : add SSL support (#5926) fd72d2d2a5e79d61ccef6af3d15f16e5e5cbc352 c2101a2e909ac7c08976d414e64e96c90ee5fa9e Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-09T10:30:04+01:00 GitHub noreply@github.com 2024-03-09T11:30:04+02:00 server: tests: add truncated prompt tests, better kv cache size (#5933) c2101a2e909ac7c08976d414e64e96c90ee5fa9e 515f7d0d4fce41c752fc253acf30707c3be2531e compilade 113953597+compilade@users.noreply.github.com 2024-03-08T17:31:00-05:00 GitHub noreply@github.com 2024-03-08T17:31:00-05:00 llama : support Mamba Selective State Space Models (#5328) 515f7d0d4fce41c752fc253acf30707c3be2531e 76e868821a94072fbc87cb1fcca291694319eae8 compilade 113953597+compilade@users.noreply.github.com 2024-03-08T10:53:37-05:00 GitHub noreply@github.com 2024-03-08T17:53:37+02:00 llama : fix quantization of shared token_embd (#5944) 76e868821a94072fbc87cb1fcca291694319eae8 e457fb3540e0aaec47cfde0abf784c213f9216ee Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-08T12:25:04+01:00 GitHub noreply@github.com 2024-03-08T12:25:04+01:00 server: metrics: add llamacpp:prompt_seconds_total and llamacpp:tokens_predicted_seconds_total, reset bucket only on /metrics. Fix values cast to int. Add Process-Start-Time-Unix header. (#5937) e457fb3540e0aaec47cfde0abf784c213f9216ee af37fd8b30e37ccbffdd82e6f48559e2fb7ce7dd Don Mahurin dmahurin@users.noreply.github.com 2024-03-08T02:41:50-08:00 GitHub noreply@github.com 2024-03-08T12:41:50+02:00 llama : assume tied weights if lm_head/output weights is missing (#5824) af37fd8b30e37ccbffdd82e6f48559e2fb7ce7dd 581ed5c4fe3a8909aaa8313633ac443f471ba755 Georgi Gerganov ggerganov@gmail.com 2024-03-08T12:40:02+02:00 GitHub noreply@github.com 2024-03-08T12:40:02+02:00 server : fix EOS token detection with disabled cache (#5938) 581ed5c4fe3a8909aaa8313633ac443f471ba755 6cdabe652695167263c8b447520987b11856f7ca UEXTM.com 84163508+uextm@users.noreply.github.com 2024-03-08T04:35:04-05:00 GitHub noreply@github.com 2024-03-08T11:35:04+02:00 log : fix MSVC compile errors (#5643) 6cdabe652695167263c8b447520987b11856f7ca 89fb735fcfd21781a8194b211cf32824beb3f71f Georgi Gerganov ggerganov@gmail.com 2024-03-07T16:32:38+02:00 GitHub noreply@github.com 2024-03-07T16:32:38+02:00 llama-bench : add embeddings option (#5924) 89fb735fcfd21781a8194b211cf32824beb3f71f 55a2a900ff4a02fc33708ac7858d595d289a3f2a Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-07T19:14:49+08:00 GitHub noreply@github.com 2024-03-07T12:14:49+01:00 Revert "[SYCL] fix error when set main gpu to non-zero (#5901)" (#5918) 55a2a900ff4a02fc33708ac7858d595d289a3f2a 2002bc96bf2cbf5ab981a17d7e994d817c9801f5 Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-03-07T19:42:39+09:00 GitHub noreply@github.com 2024-03-07T12:42:39+02:00 server : add `/v1/completions` endpoint (#5914) 2002bc96bf2cbf5ab981a17d7e994d817c9801f5 ceca1aef0738b57951cd12c603c3477e75312dec Georgi Gerganov ggerganov@gmail.com 2024-03-07T11:41:53+02:00 GitHub noreply@github.com 2024-03-07T11:41:53+02:00 server : refactor (#5882) ceca1aef0738b57951cd12c603c3477e75312dec e04e04f8fad549bb0b3ec1c91f0413aeb08baf29 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-07T16:34:31+08:00 GitHub noreply@github.com 2024-03-07T16:34:31+08:00 [SYCL] fix error when set main gpu to non-zero (#5901) e04e04f8fad549bb0b3ec1c91f0413aeb08baf29 e25fb4b18fcedb9bed6be4585cf842e9a669b28b Jared Van Bortel jared@nomic.ai 2024-03-06T15:42:23-05:00 GitHub noreply@github.com 2024-03-06T15:42:23-05:00 ggml : use SYS_get_cpu if SYS_getcpu is not defined (#5906) e25fb4b18fcedb9bed6be4585cf842e9a669b28b 1e35d619a6fb0b9c5e3dc955345980ff056ddbaf bobqianic 129547291+bobqianic@users.noreply.github.com 2024-03-06T07:35:07Z GitHub noreply@github.com 2024-03-06T09:35:07+02:00 ggml : use `uint8x16_t` return type for `ggml_vqtbl1q_u8` (#5894) 1e35d619a6fb0b9c5e3dc955345980ff056ddbaf 8ced9f7e3225adb8501e9821ed1bbd92e3a5c7ae Georgi Gerganov ggerganov@gmail.com 2024-03-06T09:12:25+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-06T09:13:42+02:00 convert : remove AWQ remnants (#5768) 8ced9f7e3225adb8501e9821ed1bbd92e3a5c7ae 652ca2bded3c818320d92c70d2b67f64bdbff5e5 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-06T12:08:32+08:00 GitHub noreply@github.com 2024-03-06T12:08:32+08:00 add wait() to make code stable (#5895) 652ca2bded3c818320d92c70d2b67f64bdbff5e5 bd836944f826f07e19b7edcf994a78728da49c1c slaren slarengh@gmail.com 2024-03-05T22:27:29+01:00 GitHub noreply@github.com 2024-03-05T22:27:29+01:00 compare-llama-bench.py : remove mul_mat_q (#5892) bd836944f826f07e19b7edcf994a78728da49c1c 3de31677d36aa4f82d4d99898902d7bcf398e666 Jared Van Bortel jared@nomic.ai 2024-03-05T11:56:37-05:00 GitHub noreply@github.com 2024-03-05T11:56:37-05:00 quants : use MM256_SET_M128I consistently to fix gcc 7 build (#5889) 3de31677d36aa4f82d4d99898902d7bcf398e666 82cb31eb93fd19b74115e0f0133225d1dfdbfdbc ExtReMLapin 3909752+ExtReMLapin@users.noreply.github.com 2024-03-05T17:33:08+01:00 GitHub noreply@github.com 2024-03-05T18:33:08+02:00 grammars : blacklists character control set (#5888) 82cb31eb93fd19b74115e0f0133225d1dfdbfdbc b1a4e994fde929300d4aeb1deb8320c59cb6edec Georgi Gerganov ggerganov@gmail.com 2024-03-05T15:56:24+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-05T15:56:24+02:00 Revert "grammars : don't allow to output unescaped new line in string (#5885)" b1a4e994fde929300d4aeb1deb8320c59cb6edec 61d1c88e155515dd03940913a5707ea84a8b119b ExtReMLapin 3909752+ExtReMLapin@users.noreply.github.com 2024-03-05T14:44:29+01:00 GitHub noreply@github.com 2024-03-05T15:44:29+02:00 grammars : don't allow to output unescaped new line in string (#5885) 61d1c88e155515dd03940913a5707ea84a8b119b 21b08674331e1ea1b599f17c5ca91f0ed173be31 0cc4m picard12@live.de 2024-03-05T13:33:42+01:00 GitHub noreply@github.com 2024-03-05T13:33:42+01:00 Vulkan Improvements (#5835) 21b08674331e1ea1b599f17c5ca91f0ed173be31 6a87ac3a52668e117d97bcea07b529c93188b303 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-05T16:08:35+08:00 GitHub noreply@github.com 2024-03-05T13:38:35+05:30 [SYCL] fix mul_mat fault in CI/unit-test (#5862) 6a87ac3a52668e117d97bcea07b529c93188b303 29eee404746e4696143a4f3a642660a4793a15d8 Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-03-05T15:12:23+09:00 GitHub noreply@github.com 2024-03-05T11:42:23+05:30 fix editorconfig check break (#5879) 29eee404746e4696143a4f3a642660a4793a15d8 1d41d6f7c2a666eb9c18a686a4684c4b03289bf3 Jeffrey Quesnelle emozilla@nousresearch.com 2024-03-04T19:23:06-08:00 GitHub noreply@github.com 2024-03-04T22:23:06-05:00 fix speculative decoding build on windows (#5874) 1d41d6f7c2a666eb9c18a686a4684c4b03289bf3 29ae62d2ae163e2b68aa0ad3bf2ab4636de0c957 hutli 6594598+hutli@users.noreply.github.com 2024-03-05T02:33:08+01:00 GitHub noreply@github.com 2024-03-04T17:33:08-08:00 nix: static build (#5814) 29ae62d2ae163e2b68aa0ad3bf2ab4636de0c957 e0843afe1b37890b631bc7d3d2da2ed36c862b91 Georgi Gerganov ggerganov@gmail.com 2024-03-04T22:31:20+02:00 GitHub noreply@github.com 2024-03-04T22:31:20+02:00 llama : fix embeddings (#5796) e0843afe1b37890b631bc7d3d2da2ed36c862b91 a1c6d96ed8f906aa1cda439f7386b1171a22bf9f Georgi Gerganov ggerganov@gmail.com 2024-03-04T21:50:50+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-04T21:50:50+02:00 flake : fix a1c6d96ed8f906aa1cda439f7386b1171a22bf9f efd8533ef8d0752cef7119eb5dbee412c4dba270 Georgi Gerganov ggerganov@gmail.com 2024-03-04T20:53:27+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-04T20:54:23+02:00 ggml : fix unknown status (#0) efd8533ef8d0752cef7119eb5dbee412c4dba270 9fa262734733573fa629ffc97dfcb971fe3f4832 Georgi Gerganov ggerganov@gmail.com 2024-03-04T11:06:39+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-04T20:54:23+02:00 sync : ggml 9fa262734733573fa629ffc97dfcb971fe3f4832 fe52be11e35358d2fd249f19d7ef5b6f9c08b16b Michael Podvitskiy podvitskiymichael@gmail.com 2024-03-04T10:05:42+01:00 Georgi Gerganov ggerganov@gmail.com 2024-03-04T20:54:23+02:00 ggml : introduce ggml_status (ggml/750) fe52be11e35358d2fd249f19d7ef5b6f9c08b16b 6d341ab6c53cd51f2921d986d0090cc8b049b39a Dane Madsen dane_madsen@hotmail.com 2024-03-05T05:26:55+11:00 GitHub noreply@github.com 2024-03-04T20:26:55+02:00 cmake : handle cases where git index is not found in .git (#5844) 6d341ab6c53cd51f2921d986d0090cc8b049b39a 4ffcdce2ff877ebb683cd217ea38faf20faa5ffe Minsoo Cheong 54794500+mscheong01@users.noreply.github.com 2024-03-05T03:24:00+09:00 GitHub noreply@github.com 2024-03-04T20:24:00+02:00 speculative : implement stochastic speculative sampling (#5625) 4ffcdce2ff877ebb683cd217ea38faf20faa5ffe a0fc62661f0fd2a9edd10ae5617345bbbf972f42 Xuan Son Nguyen thichthat@gmail.com 2024-03-04T12:22:08+01:00 GitHub noreply@github.com 2024-03-04T12:22:08+01:00 add alias for chat template (#5858) a0fc62661f0fd2a9edd10ae5617345bbbf972f42 7d43c585dc174bb586775c22c15e5db9242b5b4b Georgi Gerganov ggerganov@gmail.com 2024-03-04T10:40:04+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-04T10:40:04+02:00 sync : ggml 7d43c585dc174bb586775c22c15e5db9242b5b4b 82f3e668adafba647de703f835991e91a96b5ac4 leejet leejet714@gmail.com 2024-03-03T20:23:52+08:00 Georgi Gerganov ggerganov@gmail.com 2024-03-04T10:39:10+02:00 add some new ops, fix some operators and add batch operations to certain operators. (ggml/747) 82f3e668adafba647de703f835991e91a96b5ac4 5a51cc1bb4592f0d71f9af89cd08b11a066ba447 DAN™ dranger003@gmail.com 2024-03-04T03:08:19-05:00 GitHub noreply@github.com 2024-03-04T10:08:19+02:00 common : use LLAMA_DEFAULT_SEED (#5855) 5a51cc1bb4592f0d71f9af89cd08b11a066ba447 67be2ce1015d070b3b2cd488bcb041eefb61de72 DAN™ dranger003@gmail.com 2024-03-04T02:57:20-05:00 GitHub noreply@github.com 2024-03-04T09:57:20+02:00 main : support special tokens as reverse/anti prompt (#5847) 67be2ce1015d070b3b2cd488bcb041eefb61de72 231ae28f078c3148d097b301f2145f1e3e816cc1 slaren slarengh@gmail.com 2024-03-03T14:26:18+01:00 GitHub noreply@github.com 2024-03-03T14:26:18+01:00 cuda : fix data race in soft max (#5853) 231ae28f078c3148d097b301f2145f1e3e816cc1 475df1d6cf817060028d3ff763cb8097d4ec40d6 Georgi Gerganov ggerganov@gmail.com 2024-03-03T12:44:03+02:00 GitHub noreply@github.com 2024-03-03T12:44:03+02:00 readme : add API changes section 475df1d6cf817060028d3ff763cb8097d4ec40d6 87c2e8b2797860a06af3d6c06b8488a8ff1a09ab Douglas Hanley thesecretaryofwar@gmail.com 2024-03-03T04:40:27-06:00 GitHub noreply@github.com 2024-03-03T12:40:27+02:00 llama : allow for user specified embedding pooling type (#5849) 87c2e8b2797860a06af3d6c06b8488a8ff1a09ab de9692a7d2db66e29e5cb373c6551acc49145ccd Nindaleth Nindaleth@users.noreply.github.com 2024-03-03T09:43:42+01:00 GitHub noreply@github.com 2024-03-03T10:43:42+02:00 gguf-dump : support i-quants (#5841) de9692a7d2db66e29e5cb373c6551acc49145ccd e6029348e86c3810d4435faee54ba822cb43e2ef compilade 113953597+compilade@users.noreply.github.com 2024-03-03T03:41:55-05:00 GitHub noreply@github.com 2024-03-03T10:41:55+02:00 llama : fix llama_copy_state_data with fragmented KV cache (#5840) e6029348e86c3810d4435faee54ba822cb43e2ef 8ef969afcec1645d2d9c3ab1fc82263bba968989 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-03T09:35:23+01:00 GitHub noreply@github.com 2024-03-03T10:35:23+02:00 ci : schedule slow server tests only on Release or on demand (#5839) 8ef969afcec1645d2d9c3ab1fc82263bba968989 fa974646e1a2024fc7dc9e6f27cf1f2f5d4a3763 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-03T08:48:36+01:00 GitHub noreply@github.com 2024-03-03T09:48:36+02:00 server : init http requests thread pool with --parallel if set (#5836) fa974646e1a2024fc7dc9e6f27cf1f2f5d4a3763 9731134296af3a6839cd682e51d9c2109a871de5 Georgi Gerganov ggerganov@gmail.com 2024-03-03T06:11:31+02:00 GitHub noreply@github.com 2024-03-02T20:11:31-08:00 flake.lock: Update (#5842) 9731134296af3a6839cd682e51d9c2109a871de5 4a6e2d6142ab815c964924896891e9ab3e050632 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-02T22:00:14+01:00 GitHub noreply@github.com 2024-03-02T22:00:14+01:00 server: tests: passkey challenge / self-extend with context shift demo (#5832) 4a6e2d6142ab815c964924896891e9ab3e050632 494c87032613e31c0be99b2735e732871f2c4e4d Michael Podvitskiy podvitskiymichael@gmail.com 2024-03-02T20:52:25+01:00 GitHub noreply@github.com 2024-03-02T21:52:25+02:00 llama : add abort_callback to interrupt computation (#5409) 494c87032613e31c0be99b2735e732871f2c4e4d 4d4d2366fc9c54d4a275065cfe9299c6cf7c5b78 Georgi Gerganov ggerganov@gmail.com 2024-03-02T20:00:49+02:00 GitHub noreply@github.com 2024-03-02T20:00:49+02:00 ggml : fix IQ3_S AVX implementation (#5834) 4d4d2366fc9c54d4a275065cfe9299c6cf7c5b78 c7a0ad8ec9ebb5ddb1c1c80c82f2ee041c525d47 Jared Van Bortel jared@nomic.ai 2024-03-02T12:27:26-05:00 GitHub noreply@github.com 2024-03-02T12:27:26-05:00 convert : automatically fall back to HfVocab if tokenizer.model doesn't exist (#5821) c7a0ad8ec9ebb5ddb1c1c80c82f2ee041c525d47 bbde6eb2561153aabbdfac5001c690fe00cad639 Jared Van Bortel jared@nomic.ai 2024-03-02T12:21:47-05:00 GitHub noreply@github.com 2024-03-02T12:21:47-05:00 convert-hf : make model class definitions self-contained (#5825) bbde6eb2561153aabbdfac5001c690fe00cad639 ef2cd694c4155fbf25bae61c5178c47eb3676dba Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-03-02T17:00:51+02:00 GitHub noreply@github.com 2024-03-02T17:00:51+02:00 ggml : IQ3_S improvements (#5829) ef2cd694c4155fbf25bae61c5178c47eb3676dba 6c32d8c7ad8ba7b6ad2a162e929a21dd04fcdca0 Georgi Gerganov ggerganov@gmail.com 2024-03-02T16:54:08+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-02T16:54:20+02:00 scripts : add pod-llama.sh 6c32d8c7ad8ba7b6ad2a162e929a21dd04fcdca0 802da0091ba646ecf02e1a8fae2da0b8e76409bd Xuan Son Nguyen thichthat@gmail.com 2024-03-02T15:19:09+01:00 GitHub noreply@github.com 2024-03-02T16:19:09+02:00 llama : refactor internal quantization functions (#5830) 802da0091ba646ecf02e1a8fae2da0b8e76409bd 715641391dda1ff9762dc5d99d9a30acce99f2c6 compilade 113953597+compilade@users.noreply.github.com 2024-03-02T08:42:56-05:00 GitHub noreply@github.com 2024-03-02T15:42:56+02:00 llama : fix segfault from unknown model arch name (#5820) 715641391dda1ff9762dc5d99d9a30acce99f2c6 9bf297a02bfbd474e51912409a470dd797e2fe13 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-03-02T19:49:30+08:00 GitHub noreply@github.com 2024-03-02T19:49:30+08:00 Support multiple GPUs (split mode) on SYCL backend (#5806) 9bf297a02bfbd474e51912409a470dd797e2fe13 cb5e8f7fc4ee57d4bcccafbe04a82cededd35486 crasm crasm@git.vczf.net 2024-03-02T00:11:06-05:00 GitHub noreply@github.com 2024-03-02T00:11:06-05:00 workflows : remove nocleanup arg for check-requirements.sh (#5826) cb5e8f7fc4ee57d4bcccafbe04a82cededd35486 da3b9ba2b710c0f8b44398a0eb9e5a7ae2ad967a Tushar ditsuke@protonmail.com 2024-03-02T04:48:26+05:30 GitHub noreply@github.com 2024-03-01T15:18:26-08:00 build(nix): Introduce flake.formatter for `nix fmt` (#5687) da3b9ba2b710c0f8b44398a0eb9e5a7ae2ad967a c29af7e2252d288f2ea58a7d437c1cb7c0abf160 nold Nold360@users.noreply.github.com 2024-03-01T22:51:12+01:00 GitHub noreply@github.com 2024-03-01T16:51:12-05:00 convert-hf-to-gguf : require einops for InternLM2ForCausalLM (#5792) c29af7e2252d288f2ea58a7d437c1cb7c0abf160 38d16b142624bdd7c41d9955752b7f7b59c5e048 Sourab Mangrulkar 13534540+pacman100@users.noreply.github.com 2024-03-02T01:00:46+05:30 GitHub noreply@github.com 2024-03-01T21:30:46+02:00 llama : add StarCoder2 support (#5795) 38d16b142624bdd7c41d9955752b7f7b59c5e048 c2224f003bf9cf558b1a3c57033563e11a4de9a5 Georgi Gerganov ggerganov@gmail.com 2024-03-01T20:00:58+02:00 GitHub noreply@github.com 2024-03-01T20:00:58+02:00 server : remove api_like_OAI.py proxy script (#5808) c2224f003bf9cf558b1a3c57033563e11a4de9a5 e7433867288d2f142cffe596f3751bda5d7ee2c7 ddpasa 112642920+ddpasa@users.noreply.github.com 2024-03-01T18:00:00+01:00 GitHub noreply@github.com 2024-03-01T18:00:00+01:00 ggml-vulkan: fix VULKAN_CHECK_RESULTS flag, which was previously broken (#5813) e7433867288d2f142cffe596f3751bda5d7ee2c7 f49a5356865ced0eca1df9f9d84631dfef71b9dc kunal-vaishnavi 115581922+kunal-vaishnavi@users.noreply.github.com 2024-03-01T06:08:08-08:00 GitHub noreply@github.com 2024-03-01T16:08:08+02:00 gemma : fix bfloat16 -> float16 conversion issue (#5810) f49a5356865ced0eca1df9f9d84631dfef71b9dc 3ab8b3a92ede46df88bc5a2dfca3777de4a2b2b6 Miwa / Ensan 63481257+ensan-hcl@users.noreply.github.com 2024-03-01T22:48:56+09:00 GitHub noreply@github.com 2024-03-01T15:48:56+02:00 common : fix flag `--logits-all` to `--all-logits` (#5805) 3ab8b3a92ede46df88bc5a2dfca3777de4a2b2b6 9600d59e010c18f5872580a21734ea1bf1968d04 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-01T12:39:06+01:00 GitHub noreply@github.com 2024-03-01T13:39:06+02:00 llama : cleanup unused mmq flags (#5772) 9600d59e010c18f5872580a21734ea1bf1968d04 5cb02b4a012bb16c6c699c0c62c05ffa653eee0f Douglas Hanley thesecretaryofwar@gmail.com 2024-03-01T03:15:36-06:00 GitHub noreply@github.com 2024-03-01T11:15:36+02:00 unicode : switch to multimap based nfd_map (#5799) 5cb02b4a012bb16c6c699c0c62c05ffa653eee0f 6ea0f010ff6967034528d9e0b8330b9b0f0b7c13 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-03-01T10:08:08+01:00 GitHub noreply@github.com 2024-03-01T10:08:08+01:00 server: allow to override threads server pool with --threads-http (#5794) 6ea0f010ff6967034528d9e0b8330b9b0f0b7c13 f105471ef6aa4727afac8240da398590d7277f45 Eve 139727413+netrunnereve@users.noreply.github.com 2024-03-01T08:54:53Z GitHub noreply@github.com 2024-03-01T10:54:53+02:00 ci : add Ubuntu 22 Vulkan CI run (#5789) f105471ef6aa4727afac8240da398590d7277f45 38d152160898b0173ffe4dc7df5daadcbd2eceb0 Georgi Gerganov ggerganov@gmail.com 2024-03-01T09:59:43+02:00 Georgi Gerganov ggerganov@gmail.com 2024-03-01T09:59:43+02:00 server : fix newlines in help (#5785) 38d152160898b0173ffe4dc7df5daadcbd2eceb0 052051d8ae4639a1c3c61e7da3237bcc572469d4 AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-03-01T07:36:47Z GitHub noreply@github.com 2024-03-01T13:06:47+05:30 [SYCL] Use batched mul_mat pathway (#5591) 052051d8ae4639a1c3c61e7da3237bcc572469d4 d5ab29757ebc59a30f03e408294ec20628a6374e Xuan Son Nguyen thichthat@gmail.com 2024-02-29T21:42:11+01:00 GitHub noreply@github.com 2024-02-29T21:42:11+01:00 Server: normalize naming (#5779) d5ab29757ebc59a30f03e408294ec20628a6374e 87c91c07663b707e831c59ec373b5e665ff9d64a Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2024-02-29T00:17:23-08:00 GitHub noreply@github.com 2024-02-29T10:17:23+02:00 llama : constified `llama_set_state_data`'s `src` (#5774) 87c91c07663b707e831c59ec373b5e665ff9d64a 317709b2a81dbaf87850202686ec5bb2602a504e Georgi Gerganov ggerganov@gmail.com 2024-02-28T21:44:21+02:00 GitHub noreply@github.com 2024-02-28T21:44:21+02:00 ci : reduce 3b ppl chunks to 1 to avoid timeout (#5771) 317709b2a81dbaf87850202686ec5bb2602a504e 08c5ee87e4cceb603ecceac90734fcdade57311b Eve 139727413+netrunnereve@users.noreply.github.com 2024-02-28T19:33:37Z GitHub noreply@github.com 2024-02-28T20:33:37+01:00 make portability_enumeration_ext apple only (#5757) 08c5ee87e4cceb603ecceac90734fcdade57311b 78aacf36344df724cdca9f1e1af849b2d2519cb8 Georgi Gerganov ggerganov@gmail.com 2024-02-28T18:43:38+02:00 GitHub noreply@github.com 2024-02-28T18:43:38+02:00 llama : remove deprecated API (#5770) 78aacf36344df724cdca9f1e1af849b2d2519cb8 8c0e8f4e73e275756ad69f9c99b26ead085ca9f0 Georgi Gerganov ggerganov@gmail.com 2024-02-28T17:36:53+02:00 GitHub noreply@github.com 2024-02-28T17:36:53+02:00 awq-py : remove (#5768) 8c0e8f4e73e275756ad69f9c99b26ead085ca9f0 2774b0c97427ee3ad3e2ee121354d078794e89d9 Georgi Gerganov ggerganov@gmail.com 2024-02-28T11:17:32+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-28T11:17:32+02:00 sync : ggml 2774b0c97427ee3ad3e2ee121354d078794e89d9 5f706718566e3a5147916dc381f3b99de0ffad47 slaren slarengh@gmail.com 2024-02-25T20:41:35+01:00 Georgi Gerganov ggerganov@gmail.com 2024-02-28T11:17:06+02:00 add google magika inference example (ggml/748) 5f706718566e3a5147916dc381f3b99de0ffad47 a693bea1e6762a17b78b6ddf4611e54136941ea2 UEXTM.com 84163508+uextm@users.noreply.github.com 2024-02-24T11:27:36-05:00 Georgi Gerganov ggerganov@gmail.com 2024-02-28T11:17:05+02:00 Introduce backend GUIDs (ggml/743) a693bea1e6762a17b78b6ddf4611e54136941ea2 adcb12a9bad87bc96f2f158c95892b3d04aa7ffb Xuan Son Nguyen thichthat@gmail.com 2024-02-28T09:55:37+01:00 GitHub noreply@github.com 2024-02-28T10:55:37+02:00 server : hit Ctrl+C twice to exit (#5734) adcb12a9bad87bc96f2f158c95892b3d04aa7ffb 177628bfd85565070916ad66a5ac4071ee0527d8 compilade 113953597+compilade@users.noreply.github.com 2024-02-28T03:52:56-05:00 GitHub noreply@github.com 2024-02-28T10:52:56+02:00 llama : fix non-quantization of expert gating tensors (#5754) 177628bfd85565070916ad66a5ac4071ee0527d8 6c4416868df2e5455da7d20547f62bcf9735ba8e Douglas Hanley thesecretaryofwar@gmail.com 2024-02-28T02:51:11-06:00 GitHub noreply@github.com 2024-02-28T10:51:11+02:00 llama : improve BERT tokenization (#5740) 6c4416868df2e5455da7d20547f62bcf9735ba8e efc72253f7987ed7bdc8bde9d9fa5c7cac2f6292 Daniel Bevenius daniel.bevenius@gmail.com 2024-02-28T09:39:39+01:00 GitHub noreply@github.com 2024-02-28T10:39:39+02:00 readme : add link to LLaVA 1.6 models (#5758) efc72253f7987ed7bdc8bde9d9fa5c7cac2f6292 7c4263d4261d6ee6f0539d53eb9e1b4d120ba8af Jorge A 161275481+jorgealias@users.noreply.github.com 2024-02-28T01:39:15-07:00 GitHub noreply@github.com 2024-02-28T10:39:15+02:00 server : add "/chat/completions" alias for "/v1/...` (#5722) 7c4263d4261d6ee6f0539d53eb9e1b4d120ba8af cb49e0f8c906e5da49e9f6d64a57742a9a241c6a Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-28T10:37:02+02:00 GitHub noreply@github.com 2024-02-28T10:37:02+02:00 ggml : make i-quants work with super-blocks of 64 (CPU,Metal) (#5760) cb49e0f8c906e5da49e9f6d64a57742a9a241c6a 0becb22ac05b6542bd9d5f2235691aa1d3d4d307 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-27T19:16:49+02:00 GitHub noreply@github.com 2024-02-27T19:16:49+02:00 Attempt to fix android build (#5752) 0becb22ac05b6542bd9d5f2235691aa1d3d4d307 c24a2a6e6005e5d424301525a42ba45a4a362d30 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-27T16:34:24+02:00 GitHub noreply@github.com 2024-02-27T16:34:24+02:00 IQ4_XS: a 4.25 bpw quantization (#5747) c24a2a6e6005e5d424301525a42ba45a4a362d30 1f30b7a9f1b86baa455072d3182b9ebeee0cd845 Engininja2 139037756+Engininja2@users.noreply.github.com 2024-02-27T07:22:45-06:00 GitHub noreply@github.com 2024-02-27T14:22:45+01:00 cuda : replace remaining shfl_xor with calls to warp_reduce functions (#5744) 1f30b7a9f1b86baa455072d3182b9ebeee0cd845 9d533a77d0c3850ce09d736bc1baa67fd6ad27b3 Engininja2 139037756+Engininja2@users.noreply.github.com 2024-02-27T06:50:18-06:00 GitHub noreply@github.com 2024-02-27T14:50:18+02:00 ggml-quants : fix avx2 iq1_s vec_dot when compiled with gcc (#5742) 9d533a77d0c3850ce09d736bc1baa67fd6ad27b3 cbbd1efa06f8c09f9dff58ff9d9af509cc4c152b Georgi Gerganov ggerganov@gmail.com 2024-02-27T14:35:51+02:00 GitHub noreply@github.com 2024-02-27T14:35:51+02:00 llama : fix defrag bugs + add parameter (#5735) cbbd1efa06f8c09f9dff58ff9d9af509cc4c152b b11a93df41921846a10628a7c306d5c82a549939 le.chang cljs118@126.com 2024-02-27T10:03:06+08:00 GitHub noreply@github.com 2024-02-27T03:03:06+01:00 Makefile: use variables for cublas (#5689) b11a93df41921846a10628a7c306d5c82a549939 a33e6a0d2a66104ea9a906bdbf8a94d050189d91 Xuan Son Nguyen thichthat@gmail.com 2024-02-26T23:15:48+01:00 GitHub noreply@github.com 2024-02-26T23:15:48+01:00 fix server hangs on empty prompt (#5733) a33e6a0d2a66104ea9a906bdbf8a94d050189d91 47bb7b48c7cec9d8f57d56812ce811ec130b89a3 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-26T18:28:38+02:00 GitHub noreply@github.com 2024-02-26T18:28:38+02:00 Adding IQ2_S and IQ2_M to complete coverage of the 2-3 bit quantization range (#5721) 47bb7b48c7cec9d8f57d56812ce811ec130b89a3 c4d7f8178608440506e5489bae0109e4ca12e44a Johannes Gäßler johannesg@5d6.de 2024-02-26T15:36:38+01:00 GitHub noreply@github.com 2024-02-26T15:36:38+01:00 CUDA: fix DEBUG_CUDA_MALLOC (#5729) c4d7f8178608440506e5489bae0109e4ca12e44a e849078c6e09e72fdd2c95ba61f5fba9a7b2d9ef Artem guinmoon@gmail.com 2024-02-26T17:15:28+03:00 GitHub noreply@github.com 2024-02-26T16:15:28+02:00 readme : update ui list (#5731) e849078c6e09e72fdd2c95ba61f5fba9a7b2d9ef 67fd33132fab93e6c2087bd6fa656a8a57419efa AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-02-26T14:02:11Z GitHub noreply@github.com 2024-02-26T19:32:11+05:30 [SYCL] Add support for soft_max ALiBi (#5639) 67fd33132fab93e6c2087bd6fa656a8a57419efa 4804215cb833841ffb15a710a16b77ca0a29eb4b Georgi Gerganov ggerganov@gmail.com 2024-02-26T14:02:12+02:00 GitHub noreply@github.com 2024-02-26T14:02:12+02:00 unicode : reuse iterator (#5726) 4804215cb833841ffb15a710a16b77ca0a29eb4b 8a533f0d9078396ebaee9ba213038a1322976dee Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-26T11:41:34+01:00 GitHub noreply@github.com 2024-02-26T12:41:34+02:00 server: CI fix trailing space (#5728) 8a533f0d9078396ebaee9ba213038a1322976dee 269de86ba073b5dc9ce687c11a3bc4d7d873b962 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-26T09:56:10+01:00 GitHub noreply@github.com 2024-02-26T09:56:10+01:00 server: CI tests reduce build matrix (#5725) 269de86ba073b5dc9ce687c11a3bc4d7d873b962 c39373398803c669056304090050fe3f44b41bf9 Georgi Gerganov ggerganov@gmail.com 2024-02-26T08:30:17+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-26T08:30:17+02:00 llama : fix Gemma rope type (#5691) c39373398803c669056304090050fe3f44b41bf9 e3965cf35aac00d4e24998c8a3d0093ae1d98bd3 github-actions[bot] github-actions[bot]@users.noreply.github.com 2024-02-25T00:17:11Z Someone newkozlukov@gmail.com 2024-02-25T22:24:22Z flake.lock: Update e3965cf35aac00d4e24998c8a3d0093ae1d98bd3 8b350356b28f782deab63d8b0e9ae103ceb25fcd Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-25T22:48:33+01:00 GitHub noreply@github.com 2024-02-25T22:48:33+01:00 server: tests - slow inference causes timeout on the CI (#5715) 8b350356b28f782deab63d8b0e9ae103ceb25fcd bf08e00643fd529f748f0a858fd79f3061e3fa18 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-25T21:46:29+01:00 GitHub noreply@github.com 2024-02-25T21:46:29+01:00 server: docs - refresh and tease a little bit more the http server (#5718) bf08e00643fd529f748f0a858fd79f3061e3fa18 f7625019c51ca437a5840576d92362cfa710e4a2 Georgi Gerganov ggerganov@gmail.com 2024-02-25T22:12:24+02:00 GitHub noreply@github.com 2024-02-25T22:12:24+02:00 llama : refactor k-shift implementation + KV defragmentation (#5691) f7625019c51ca437a5840576d92362cfa710e4a2 abbabc5e51d0d4656b438aec10b7fae9479ef37d compilade 113953597+compilade@users.noreply.github.com 2024-02-25T13:43:50-05:00 GitHub noreply@github.com 2024-02-25T20:43:50+02:00 server : fix crash when system prompt is bigger than batch size (#5714) abbabc5e51d0d4656b438aec10b7fae9479ef37d f1a98c52546d009f742bdec2154c2a314ea950a6 Radosław Gryta radek.gryta@gmail.com 2024-02-25T19:43:00+01:00 GitHub noreply@github.com 2024-02-25T20:43:00+02:00 ggml-quants : provide ggml_vqtbl1q_u8 for 64bit compatibility (#5711) f1a98c52546d009f742bdec2154c2a314ea950a6 7d548a1827f6fc6aece6db74c9d112da42c40d68 kwin1412 42286931+kwin1412@users.noreply.github.com 2024-02-26T00:46:49+08:00 GitHub noreply@github.com 2024-02-25T18:46:49+02:00 make : fix nvcc version is empty (#5713) 7d548a1827f6fc6aece6db74c9d112da42c40d68 930b1780269a69948d106e2d1b838ab7661f679a Ashok Gelal 401055+ashokgelal@users.noreply.github.com 2024-02-25T10:57:34-05:00 GitHub noreply@github.com 2024-02-25T17:57:34+02:00 readme : add Msty to UI list (#5618) 930b1780269a69948d106e2d1b838ab7661f679a d52d7819b8ced70c642a88a59da8c78208dc58ec Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-25T13:50:32+01:00 GitHub noreply@github.com 2024-02-25T13:50:32+01:00 server: logs - unified format and --log-format option (#5700) d52d7819b8ced70c642a88a59da8c78208dc58ec 12894088170f62e4cad4f8d6a3043c185b414bab Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-25T13:49:43+01:00 GitHub noreply@github.com 2024-02-25T13:49:43+01:00 server: concurrency fix + monitoring - add /metrics prometheus compatible endpoint (#5708) 12894088170f62e4cad4f8d6a3043c185b414bab ab336a9d5e5352ecdcdf4c12d2d54cf4ef82ce31 Radosław Gryta radek.gryta@gmail.com 2024-02-25T11:53:11+01:00 GitHub noreply@github.com 2024-02-25T12:53:11+02:00 cmake : fix compilation for Android armeabi-v7a (#5702) ab336a9d5e5352ecdcdf4c12d2d54cf4ef82ce31 69917dfa55674c608360638bb4d6a12a315e2810 Georgi Gerganov ggerganov@gmail.com 2024-02-25T12:09:09+02:00 GitHub noreply@github.com 2024-02-25T12:09:09+02:00 code : normalize enum names (#5697) 69917dfa55674c608360638bb4d6a12a315e2810 9e359a4f47c1b2dceb99e29706c9f7403d32ab5e Anas Ahouzi 112881240+aahouzi@users.noreply.github.com 2024-02-25T10:54:04+01:00 GitHub noreply@github.com 2024-02-25T11:54:04+02:00 py : fix StableLM conversion after config.json changes (#5703) 9e359a4f47c1b2dceb99e29706c9f7403d32ab5e 4c4cb30736582cacb1a164a9d4bc8e17b1014be7 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-24T19:16:04+01:00 GitHub noreply@github.com 2024-02-24T19:16:04+01:00 server: continue to update other slots on embedding concurrent request (#5699) 4c4cb30736582cacb1a164a9d4bc8e17b1014be7 525213d2f5da1eaf4b922b6b792cb52b2c613368 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-24T16:23:52+02:00 GitHub noreply@github.com 2024-02-24T16:23:52+02:00 IQ3_S: a much better alternative to Q3_K (#5676) 525213d2f5da1eaf4b922b6b792cb52b2c613368 fd43d66f46ee3b5345fb8a74a252d86ccd34a409 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-24T12:28:55+01:00 GitHub noreply@github.com 2024-02-24T12:28:55+01:00 server: init functional tests (#5566) fd43d66f46ee3b5345fb8a74a252d86ccd34a409 54fbcd2ce6c48c9e22eca6fbf9e53fb68c3e72ea AlpinDale 52078762+AlpinDale@users.noreply.github.com 2024-02-23T19:31:54Z GitHub noreply@github.com 2024-02-23T21:31:54+02:00 server : add KV cache quantization options (#5684) 54fbcd2ce6c48c9e22eca6fbf9e53fb68c3e72ea 15499eb94227401bdc8875da6eb85c15d37068f7 Jared Van Bortel jared@nomic.ai 2024-02-23T13:39:14-05:00 GitHub noreply@github.com 2024-02-23T20:39:14+02:00 convert : fix missing ftype for gemma (#5690) 15499eb94227401bdc8875da6eb85c15d37068f7 96633eeca1265ed03e57230de54032041c58f9cd Jared Van Bortel jared@nomic.ai 2024-02-22T17:05:23-05:00 GitHub noreply@github.com 2024-02-22T17:05:23-05:00 mpt : do not duplicate token_embd.weight on disk (#5670) 96633eeca1265ed03e57230de54032041c58f9cd 847eedbdb2d1ebf14ef56eb507d4b4b975510908 Georgi Gerganov ggerganov@gmail.com 2024-02-22T23:23:46+02:00 GitHub noreply@github.com 2024-02-22T23:23:46+02:00 gemma : use more bits for the token_embd.weight tensor (#5650) 847eedbdb2d1ebf14ef56eb507d4b4b975510908 7e4f339c404dbe029d4a117c03b37a9bf646cf0e Georgi Gerganov ggerganov@gmail.com 2024-02-22T23:22:48+02:00 GitHub noreply@github.com 2024-02-22T23:22:48+02:00 py : add Gemma conversion from HF models (#5647) 7e4f339c404dbe029d4a117c03b37a9bf646cf0e 334f76fa385ed81095165e5ae068756214893901 Georgi Gerganov ggerganov@gmail.com 2024-02-22T23:21:39+02:00 GitHub noreply@github.com 2024-02-22T23:21:39+02:00 ggml : always define ggml_fp16_t as uint16_t (#5666) 334f76fa385ed81095165e5ae068756214893901 efd56b1c2139d50b9b4381a212feb75d69598fda Georgi Gerganov ggerganov@gmail.com 2024-02-22T23:21:05+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-22T23:21:05+02:00 sync : ggml efd56b1c2139d50b9b4381a212feb75d69598fda 201294ae177b308fb3a99dc504dd6d27e8afa907 Georgi Gerganov ggerganov@gmail.com 2024-02-22T18:31:40+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-22T23:20:50+02:00 ggml : 32-bit arm compat (whisper/1891) 201294ae177b308fb3a99dc504dd6d27e8afa907 5a9e2f60ba3d8362ba17c77ac3092906d49b813f Someone sergei.kozlukov@aalto.fi 2024-02-22T19:44:10Z GitHub noreply@github.com 2024-02-22T11:44:10-08:00 nix: init singularity and docker images (#5056) 5a9e2f60ba3d8362ba17c77ac3092906d49b813f 373ee3fbbabc4c1508eed4f5c3795b23a20939a3 Georgi Gerganov ggerganov@gmail.com 2024-02-22T20:13:25+02:00 GitHub noreply@github.com 2024-02-22T20:13:25+02:00 py : minor fixes (#5668) 373ee3fbbabc4c1508eed4f5c3795b23a20939a3 4cb4d8b22d4fda971621a68c570ce84d66897c37 Xuan Son Nguyen thichthat@gmail.com 2024-02-22T19:10:21+01:00 GitHub noreply@github.com 2024-02-22T19:10:21+01:00 Add Gemma chat template (#5665) 4cb4d8b22d4fda971621a68c570ce84d66897c37 3a03541cedea474fa9d41214484cc3fbcf468a9e Someone sergei.kozlukov@aalto.fi 2024-02-22T16:32:09Z GitHub noreply@github.com 2024-02-22T08:32:09-08:00 workflows: nix: hardcode cachix ids, build unconditionally (#5663) 3a03541cedea474fa9d41214484cc3fbcf468a9e 56d03d92be57f5880b9ed94542d87bb6effae31f Georgi Gerganov ggerganov@gmail.com 2024-02-22T13:54:03+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-22T13:54:03+02:00 minor : fix trailing whitespace (#5638) 56d03d92be57f5880b9ed94542d87bb6effae31f a46f50747b2028f7f9c9883b26bfba12bf92556e Georgi Gerganov ggerganov@gmail.com 2024-02-22T10:35:54+02:00 GitHub noreply@github.com 2024-02-22T10:35:54+02:00 readme : update hot topics a46f50747b2028f7f9c9883b26bfba12bf92556e c5688c6250430d2b8e0259efcf26c16dfa4c1f46 Xuan Son Nguyen thichthat@gmail.com 2024-02-22T09:33:24+01:00 GitHub noreply@github.com 2024-02-22T10:33:24+02:00 server : fallback to chatml, add AlphaMonarch chat template (#5628) c5688c6250430d2b8e0259efcf26c16dfa4c1f46 4ef245a92a968ba0f18a5adfd41e51980ce4fdf5 Alexey Parfenov zxed@alkatrazstudio.net 2024-02-22T08:27:32Z GitHub noreply@github.com 2024-02-22T10:27:32+02:00 server : clarify some params in the docs (#5640) 4ef245a92a968ba0f18a5adfd41e51980ce4fdf5 973053d8b0d04809836b3339a50f68d9c842de90 Dat Quoc Nguyen 2412555+datquocnguyen@users.noreply.github.com 2024-02-22T18:15:13+10:00 GitHub noreply@github.com 2024-02-22T10:15:13+02:00 mpt : add optional bias tensors (#5638) 973053d8b0d04809836b3339a50f68d9c842de90 7c8bcc11dc61cf5930b70cd0168b84afcebe12a9 slaren slarengh@gmail.com 2024-02-22T00:42:09+01:00 GitHub noreply@github.com 2024-02-22T00:42:09+01:00 llama : fix loading models with shared tok_embd and output (#5651) 7c8bcc11dc61cf5930b70cd0168b84afcebe12a9 7fe4678b0244ba7b03eae66ebeaa947e2770bb1a Xuan Son Nguyen thichthat@gmail.com 2024-02-22T00:31:00+01:00 GitHub noreply@github.com 2024-02-22T00:31:00+01:00 Add docs for llama_chat_apply_template (#5645) 7fe4678b0244ba7b03eae66ebeaa947e2770bb1a ba2135ccae7462470b3865c6e41d2e1d734eac05 slaren slarengh@gmail.com 2024-02-21T22:52:39+01:00 GitHub noreply@github.com 2024-02-21T22:52:39+01:00 llama : fix session save/load with quantized KV (#5649) ba2135ccae7462470b3865c6e41d2e1d734eac05 89febfed9322c8849520dc63c93ee4f5fd72556e slaren slarengh@gmail.com 2024-02-21T22:18:23+01:00 GitHub noreply@github.com 2024-02-21T22:18:23+01:00 gemma : allow offloading the output tensor (#5646) 89febfed9322c8849520dc63c93ee4f5fd72556e 5022cf242d689e15defd133f96c4345ad30c5d19 Jared Van Bortel jared@nomic.ai 2024-02-21T10:33:54-05:00 GitHub noreply@github.com 2024-02-21T10:33:54-05:00 examples : do not assume BOS when shifting context (#5622) 5022cf242d689e15defd133f96c4345ad30c5d19 1ecea255ebb70750b52688393f37a63606b90e3f Georgi Gerganov ggerganov@gmail.com 2024-02-21T16:52:39+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-21T16:52:52+02:00 sync : ggml 1ecea255ebb70750b52688393f37a63606b90e3f a00a35cef93e057eace8351a667d14d152a91ebc Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-21T15:47:48+01:00 GitHub noreply@github.com 2024-02-21T15:47:48+01:00 server: health: fix race condition on slots data using tasks queue (#5634) a00a35cef93e057eace8351a667d14d152a91ebc eccd7a26ddbff19e4b8805648f5f14c501957859 Ettore Di Giacinto mudler@users.noreply.github.com 2024-02-21T15:39:10+01:00 GitHub noreply@github.com 2024-02-21T16:39:10+02:00 readme : add LocalAI to the availables UI (#5629) eccd7a26ddbff19e4b8805648f5f14c501957859 c14f72db9c62d71d35eb1c141745c0bd0cb27b49 Georgi Gerganov ggerganov@gmail.com 2024-02-21T16:17:10+02:00 GitHub noreply@github.com 2024-02-21T16:17:10+02:00 sync : ggml (#5633) c14f72db9c62d71d35eb1c141745c0bd0cb27b49 cc6cac08e38e32bf40bbe07e9e8f8f0130b5fd94 Georgi Gerganov ggerganov@gmail.com 2024-02-21T15:39:54+02:00 GitHub noreply@github.com 2024-02-21T15:39:54+02:00 readme : update hot topics cc6cac08e38e32bf40bbe07e9e8f8f0130b5fd94 580111d42b3b6ad0a390bfb267d6e3077506eb31 Daniel Bevenius daniel.bevenius@gmail.com 2024-02-21T14:36:57+01:00 GitHub noreply@github.com 2024-02-21T15:36:57+02:00 llava : add --skip-unknown to 1.6 convert.py (#5632) 580111d42b3b6ad0a390bfb267d6e3077506eb31 88c46cbdac05cebd936511b1d3c74112e721615f postmasters namnguyen@google.com 2024-02-21T05:08:22-08:00 GitHub noreply@github.com 2024-02-21T15:08:22+02:00 llama : add `gemma` model (#5631) 88c46cbdac05cebd936511b1d3c74112e721615f a14679cc30c785e75d38028bae6ec39c6209ddef Meng, Hengyu hengyu.meng@intel.com 2024-02-21T17:52:06+08:00 GitHub noreply@github.com 2024-02-21T17:52:06+08:00 [SYCL] conext add name (#5624) a14679cc30c785e75d38028bae6ec39c6209ddef 6560bed3f066c876682464762cad90f1e28e3f1b Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-21T11:39:52+02:00 GitHub noreply@github.com 2024-02-21T11:39:52+02:00 IQ4_NL: 4-bit non-linear quants with blocks of 32 (#5590) 6560bed3f066c876682464762cad90f1e28e3f1b 06bf2cf8c406e6b70dbf9b431a02fa0ad845b9df CJ Pais cj@cjpais.com 2024-02-20T11:07:22-08:00 GitHub noreply@github.com 2024-02-20T21:07:22+02:00 server : support llava 1.6 (#5553) 06bf2cf8c406e6b70dbf9b431a02fa0ad845b9df 4ed8e4fbef6a15afd993bfcd9ffa279841e18ef1 slaren slarengh@gmail.com 2024-02-20T20:06:17+01:00 GitHub noreply@github.com 2024-02-20T20:06:17+01:00 make : fix debug build with CUDA (#5616) 4ed8e4fbef6a15afd993bfcd9ffa279841e18ef1 9c405c9f9a7cfd23511fd6b2de05dc72481119b4 Daniel Bevenius daniel.bevenius@gmail.com 2024-02-20T18:30:27+01:00 GitHub noreply@github.com 2024-02-20T19:30:27+02:00 llava : add explicit instructions for llava-1.6 (#5611) 9c405c9f9a7cfd23511fd6b2de05dc72481119b4 5207b3fbc500f89dfe528693e96540956dbaed96 Xuan Son Nguyen thichthat@gmail.com 2024-02-20T15:58:27+01:00 GitHub noreply@github.com 2024-02-20T15:58:27+01:00 Server: use llama_chat_apply_template (#5593) 5207b3fbc500f89dfe528693e96540956dbaed96 8dbbd75754d43ec7b4bbe42fb287cc2553fdf0e9 Dane Madsen dane_madsen@hotmail.com 2024-02-20T21:00:23+11:00 GitHub noreply@github.com 2024-02-20T12:00:23+02:00 readme : update UI list (#5605) 8dbbd75754d43ec7b4bbe42fb287cc2553fdf0e9 c0a8c6db371cb3e4379900867b948879f5842201 Haoxiang Fei tonyfettes@tonyfettes.com 2024-02-19T22:58:36-11:00 GitHub noreply@github.com 2024-02-20T11:58:36+02:00 metal : add build system support for embedded metal library (#5604) c0a8c6db371cb3e4379900867b948879f5842201 b9111bd209c7b11b0592450a6ed2e0ca545b2c84 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-20T08:48:19+01:00 GitHub noreply@github.com 2024-02-20T09:48:19+02:00 server : health endpoint configurable failure on no slot (#5594) b9111bd209c7b11b0592450a6ed2e0ca545b2c84 633782b8d949f24b619e6c68ee37b5cc79167173 AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-02-20T07:01:25Z GitHub noreply@github.com 2024-02-20T12:31:25+05:30 Update ggml_sycl_op_mul_mat_vec_q (#5502) 633782b8d949f24b619e6c68ee37b5cc79167173 22f83f0c383e12106692b8afc224d61b8993a52c Mathijs de Bruin mathijs@mathijsfietst.nl 2024-02-13T20:28:02Z Philip Taron philip.taron@gmail.com 2024-02-19T14:49:49-08:00 nix: now that we can do so, allow MacOS to build Vulkan binaries 22f83f0c383e12106692b8afc224d61b8993a52c bb9dcd560a7e81265398b0d463c40f3e467daf19 0cc4m picard12@live.de 2024-02-10T22:18:33+01:00 Philip Taron philip.taron@gmail.com 2024-02-19T14:49:49-08:00 Enable Vulkan MacOS CI bb9dcd560a7e81265398b0d463c40f3e467daf19 f50db6ae0bdcb5f8593ca6ca46dfa03b177faa2f 0cc4m picard12@live.de 2024-02-14T20:57:17+01:00 Philip Taron philip.taron@gmail.com 2024-02-19T14:49:49-08:00 Refactor validation and enumeration platform checks into functions to clean up ggml_vk_instance_init() f50db6ae0bdcb5f8593ca6ca46dfa03b177faa2f d8c054517dc24f1316f3be12a98fff383e1e93e3 0cc4m picard12@live.de 2024-02-10T22:14:52+01:00 Philip Taron philip.taron@gmail.com 2024-02-19T14:49:49-08:00 Add check for VK_KHR_portability_enumeration for MoltenVK support d8c054517dc24f1316f3be12a98fff383e1e93e3 42f664a3825dfde13a32c3577ab66d10c56f3aa6 Mathijs de Bruin mathijs@mathijsfietst.nl 2024-02-06T14:39:22Z Philip Taron philip.taron@gmail.com 2024-02-19T14:49:49-08:00 Add preprocessor checks for Apple devices. 42f664a3825dfde13a32c3577ab66d10c56f3aa6 5dde5408978eda22242b87e22e306d1c2d1a5834 Mathijs de Bruin mathijs@mathijsfietst.nl 2024-02-03T18:00:11Z Philip Taron philip.taron@gmail.com 2024-02-19T14:49:49-08:00 Resolve ErrorIncompatibleDriver with Vulkan on MacOS. 5dde5408978eda22242b87e22e306d1c2d1a5834 40c3a6c1e11040088b4a1ce0abc4651cb3011dd4 Mathijs de Bruin mathijs@mathijsfietst.nl 2024-02-03T17:56:46Z Philip Taron philip.taron@gmail.com 2024-02-19T14:49:49-08:00 Allow for Vulkan build with Accelerate. 40c3a6c1e11040088b4a1ce0abc4651cb3011dd4 f24ed14ee0ce28dfe98115c378b37da144912016 slaren slarengh@gmail.com 2024-02-19T23:40:26+01:00 GitHub noreply@github.com 2024-02-19T23:40:26+01:00 cuda : ignore peer access already enabled errors (#5597) f24ed14ee0ce28dfe98115c378b37da144912016 9d679f0fccd4030779ed3c7684a40122fe41806c Jared Van Bortel jared@nomic.ai 2024-02-19T15:54:12-05:00 GitHub noreply@github.com 2024-02-19T15:54:12-05:00 make : pass CPPFLAGS directly to nvcc, not via -Xcompiler (#5598) 9d679f0fccd4030779ed3c7684a40122fe41806c 1387cf60f758efb218fa06b670182c38ff149b7b nopperl 54780682+nopperl@users.noreply.github.com 2024-02-19T14:14:07Z GitHub noreply@github.com 2024-02-19T16:14:07+02:00 examples : support minItems/maxItems in JSON grammar converter (#5039) 1387cf60f758efb218fa06b670182c38ff149b7b 6fd413791a754598a54a366145960f2e27eec015 Georgi Gerganov ggerganov@gmail.com 2024-02-19T15:23:17+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-19T15:23:17+02:00 llava : remove extra cont (#5587) 6fd413791a754598a54a366145960f2e27eec015 337c9cbd52918ae5fb9a9d9e25d7fae4e238c9f1 slaren slarengh@gmail.com 2024-02-19T14:02:36+01:00 Georgi Gerganov ggerganov@gmail.com 2024-02-19T15:09:43+02:00 llava : replace ggml_cpy with ggml_cont 337c9cbd52918ae5fb9a9d9e25d7fae4e238c9f1 a3145bdc305422973e25f0b066da6f469ed5dc45 Georgi Gerganov ggerganov@gmail.com 2024-02-19T14:54:21+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-19T15:09:43+02:00 sync : ggml a3145bdc305422973e25f0b066da6f469ed5dc45 890559ab28e354052e16e770155ad007fd0856e8 Georgi Gerganov ggerganov@gmail.com 2024-02-19T14:53:48+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-19T15:09:43+02:00 ggml-alloc : apply ggml/731 890559ab28e354052e16e770155ad007fd0856e8 d0e3ce51f45bd6a646da1952d7e5d143a087db3e Didzis Gosko didzis@users.noreply.github.com 2024-02-11T16:41:41+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-19T15:09:43+02:00 metal : option to embed MSL source into compiled binary (whisper/1842) d0e3ce51f45bd6a646da1952d7e5d143a087db3e 68a6b98b3c8af7e5baade3ee45fe1d2c7b9323a9 Georgi Gerganov ggerganov@gmail.com 2024-02-19T14:45:41+02:00 GitHub noreply@github.com 2024-02-19T14:45:41+02:00 ci : enable -Werror for CUDA builds (#5579) 68a6b98b3c8af7e5baade3ee45fe1d2c7b9323a9 70d45af0efce9ed360e1858b827989d971dd9caf Georgi Gerganov ggerganov@gmail.com 2024-02-19T13:41:51+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-19T13:41:51+02:00 make : fix CUDA build (#5580) 70d45af0efce9ed360e1858b827989d971dd9caf 13e2c771aa4212cd5405cf310203848d50f7f859 valiray 133289098+valiray@users.noreply.github.com 2024-02-19T02:37:10-08:00 GitHub noreply@github.com 2024-02-19T12:37:10+02:00 readme : fix typo in README-sycl.md (#5353) 13e2c771aa4212cd5405cf310203848d50f7f859 f53119cec4f073b6d214195ecbe1fad3abdf2b34 Abhilash Majumder 30946547+abhilash1910@users.noreply.github.com 2024-02-19T14:45:18+05:30 GitHub noreply@github.com 2024-02-19T11:15:18+02:00 cmake : remove obsolete sycl compile flags (#5581) f53119cec4f073b6d214195ecbe1fad3abdf2b34 70847553963c85e86051d06df848236829f5f951 Georgi Gerganov ggerganov@gmail.com 2024-02-19T10:34:10+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-19T10:34:10+02:00 minor : fix trailing whitespace (#5538) 70847553963c85e86051d06df848236829f5f951 4480542b2271ba1438f0daff8e5f3a74b1dc8609 Daniel Bevenius daniel.bevenius@gmail.com 2024-02-19T09:31:59+01:00 GitHub noreply@github.com 2024-02-19T10:31:59+02:00 llava : avoid changing the original BakLLaVA model (#5577) 4480542b2271ba1438f0daff8e5f3a74b1dc8609 11b12de39bd787c0494da0cd405958fdfedc29c4 NawafAlansari 72708095+NawafAlansari@users.noreply.github.com 2024-02-19T03:25:38-05:00 GitHub noreply@github.com 2024-02-19T10:25:38+02:00 baby-llama : allocate graphs in ggml_context (#5573) 11b12de39bd787c0494da0cd405958fdfedc29c4 3a9cb4ca6408c29423373dd6cd7aa78a58286c00 Xuan Son Nguyen thichthat@gmail.com 2024-02-19T09:23:37+01:00 GitHub noreply@github.com 2024-02-19T10:23:37+02:00 llama : add llama_chat_apply_template() (#5538) 3a9cb4ca6408c29423373dd6cd7aa78a58286c00 769a716e30ba1da46f709df1c00727d6869d30e7 slaren slarengh@gmail.com 2024-02-19T09:04:45+01:00 GitHub noreply@github.com 2024-02-19T10:04:45+02:00 cuda, metal : fix nans in soft_max (#5574) 769a716e30ba1da46f709df1c00727d6869d30e7 f0d1fafc029a056cd765bdae58dcaa12312e9879 Mirko185 mirkosig@gmail.com 2024-02-19T08:39:31+01:00 GitHub noreply@github.com 2024-02-19T09:39:31+02:00 readme : update (#5572) f0d1fafc029a056cd765bdae58dcaa12312e9879 a0c2dad9d43456c677e205c6240a5f8afb0121ac bmwl brian.marshall@tolko.com 2024-02-18T23:38:32-08:00 GitHub noreply@github.com 2024-02-19T09:38:32+02:00 ggml : android and old glibc NUMA incompatibility bugfixes (#5557) a0c2dad9d43456c677e205c6240a5f8afb0121ac 14278f55d2e2c6a53022075c7f2719b71e1cd61d Jared Van Bortel jared@nomic.ai 2024-02-18T16:21:52-05:00 GitHub noreply@github.com 2024-02-18T16:21:52-05:00 build : pass all warning flags to nvcc via -Xcompiler (#5570) 14278f55d2e2c6a53022075c7f2719b71e1cd61d b1de96824bdbeb91ea458abcb3e5478690ad0727 Georgi Gerganov ggerganov@gmail.com 2024-02-18T22:58:57+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-18T22:58:57+02:00 ggml : restore vec dot stride arg names (#5453) b1de96824bdbeb91ea458abcb3e5478690ad0727 7ad554f90e735cf2a0f612ce44f9aa4fad6ae46a Georgi Gerganov ggerganov@gmail.com 2024-02-18T22:39:30+02:00 GitHub noreply@github.com 2024-02-18T22:39:30+02:00 ci : fix wikitext url + compile warnings (#5569) 7ad554f90e735cf2a0f612ce44f9aa4fad6ae46a 5ee99c32f5e47c8d32634eff9a47fb32a24c276b Georgi Gerganov ggerganov@gmail.com 2024-02-18T21:39:58+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-18T21:39:58+02:00 metal : fix unused warnings (#0) 5ee99c32f5e47c8d32634eff9a47fb32a24c276b c145f8a132b2fe1d1e65987faddbd9a40bef7a12 Robey Holderith robey@flaminglunchbox.net 2024-02-18T11:11:16-08:00 GitHub noreply@github.com 2024-02-18T21:11:16+02:00 common, server : surface min_keep as its own parameter (#5567) c145f8a132b2fe1d1e65987faddbd9a40bef7a12 689a091bbe0537ee9abff3e15a1d74f5f3561165 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-18T18:39:57+01:00 GitHub noreply@github.com 2024-02-18T19:39:57+02:00 server : slots monitoring endpoint (#5550) 689a091bbe0537ee9abff3e15a1d74f5f3561165 f3f28c5395cd25b371617981b341616dbdd31e85 Georgi Gerganov ggerganov@gmail.com 2024-02-18T19:38:06+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-18T19:38:06+02:00 sampling : do not set min_keep to n_probs (#5564) f3f28c5395cd25b371617981b341616dbdd31e85 e75c6279d1c8e7abb82a331f5de7124eed402de2 Georgi Gerganov ggerganov@gmail.com 2024-02-18T19:17:00+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-18T19:17:00+02:00 cmake : fix GGML_USE_SYCL typo (#5555) e75c6279d1c8e7abb82a331f5de7124eed402de2 36376abe05a12a8cb3af548a4af9b8d0e2e69597 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-18T17:31:28+01:00 GitHub noreply@github.com 2024-02-18T18:31:28+02:00 server : enhanced health endpoint (#5548) 36376abe05a12a8cb3af548a4af9b8d0e2e69597 66c1968f7a2e895675425e875b6589f1233a1b52 Pierrick Hymbert pierrick.hymbert@gmail.com 2024-02-18T17:30:09+01:00 GitHub noreply@github.com 2024-02-18T18:30:09+02:00 server : --n-predict option document and cap to max value (#5549) 66c1968f7a2e895675425e875b6589f1233a1b52 1dcc3fde004787e6fc4d84c9de0bb34cd2901a3e Daniel Hiltgen dhiltgen@users.noreply.github.com 2024-02-18T08:23:16-08:00 GitHub noreply@github.com 2024-02-18T18:23:16+02:00 server : graceful server shutdown (#5244) 1dcc3fde004787e6fc4d84c9de0bb34cd2901a3e 5d3de51f972055702a1859186fe7acb8f0b43dc4 Georgi Gerganov ggerganov@gmail.com 2024-02-18T18:21:52+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-18T18:21:52+02:00 common : fix ub (#5530) 5d3de51f972055702a1859186fe7acb8f0b43dc4 fc0c8d286a533363a9a663510b62af85ffad58b3 Herman Semenov GermanAizek@yandex.ru 2024-02-18T16:20:12Z GitHub noreply@github.com 2024-02-18T18:20:12+02:00 ggml, common, examples, tests : fixed type arguments in printf (#5528) fc0c8d286a533363a9a663510b62af85ffad58b3 bd2d4e393b2b7d2a1b2e201058e26017c9728ead Daniel Bevenius daniel.bevenius@gmail.com 2024-02-18T17:19:23+01:00 GitHub noreply@github.com 2024-02-18T18:19:23+02:00 llava : update surgery script to not remove tensors (#5536) bd2d4e393b2b7d2a1b2e201058e26017c9728ead c8e0d7efeb7634ecc2e9832e879ab9fca4510e71 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-18T18:16:55+02:00 GitHub noreply@github.com 2024-02-18T18:16:55+02:00 1.5 bit quantization (#5453) c8e0d7efeb7634ecc2e9832e879ab9fca4510e71 8f1be0d42f23016cb6819dbae01126699c4bd9bc github-actions[bot] github-actions[bot]@users.noreply.github.com 2024-02-18T00:17:07Z Philip Taron philip.taron@gmail.com 2024-02-18T06:39:58-08:00 flake.lock: Update 8f1be0d42f23016cb6819dbae01126699c4bd9bc 6e4e973b2615f8d390b1c4f4a7e05a119078bb0f Georgi Gerganov ggerganov@gmail.com 2024-02-17T23:04:16+02:00 GitHub noreply@github.com 2024-02-17T23:04:16+02:00 ggml : add ALiBi support for ggml_soft_max_ext (#5488) 6e4e973b2615f8d390b1c4f4a7e05a119078bb0f d250c9d61d4d9f7346930814cc4aef3f3673dc3e Ananta Bastola anantarajbastola@gmail.com 2024-02-17T16:03:14-05:00 GitHub noreply@github.com 2024-02-17T23:03:14+02:00 ci : add an option to fail on compile warning (#3952) d250c9d61d4d9f7346930814cc4aef3f3673dc3e 5bf2b94dd4fb74378b78604023b31512fec55f8f clibdev 52199778+clibdev@users.noreply.github.com 2024-02-17T18:28:37+02:00 GitHub noreply@github.com 2024-02-17T18:28:37+02:00 gitignore : update for CLion IDE (#5544) 5bf2b94dd4fb74378b78604023b31512fec55f8f d2819d5577b35507be83d0c3f4d2d3c0ab1488ca Georgi Gerganov ggerganov@gmail.com 2024-02-16T19:05:56+02:00 GitHub noreply@github.com 2024-02-16T19:05:56+02:00 cmake : fix VULKAN and ROCm builds (#5525) d2819d5577b35507be83d0c3f4d2d3c0ab1488ca 4cb072769804c77ab466bc8351c76ede9d5ba49d Georgi Gerganov ggerganov@gmail.com 2024-02-16T15:14:40+02:00 GitHub noreply@github.com 2024-02-16T15:14:40+02:00 scripts : add helpers script for bench comparing commits (#5521) 4cb072769804c77ab466bc8351c76ede9d5ba49d 65085c713e14f78cdda6abc275b1a5d8c2b8ca15 Herman Semenov GermanAizek@yandex.ru 2024-02-16T12:43:23Z GitHub noreply@github.com 2024-02-16T14:43:23+02:00 llava : removed excess free(NULL) operation (#5531) 65085c713e14f78cdda6abc275b1a5d8c2b8ca15 6dcc02d2444c779c18d49c364c5d5c5728b6b484 Herman Semenov GermanAizek@yandex.ru 2024-02-16T11:45:48Z GitHub noreply@github.com 2024-02-16T13:45:48+02:00 llama : minor fixed return int value (#5529) 6dcc02d2444c779c18d49c364c5d5c5728b6b484 5f5808ca7b7f23a1fa7a77241842bb84a0e55108 Alexey Parfenov zxed@alkatrazstudio.net 2024-02-16T11:33:25Z GitHub noreply@github.com 2024-02-16T13:33:25+02:00 server : add "samplers" param to control the samplers order (#5494) 5f5808ca7b7f23a1fa7a77241842bb84a0e55108 f486f6e1e5e9d01603d9325ab3e05f1edb362a95 Rőczey Barnabás 31726601+An0nie@users.noreply.github.com 2024-02-16T11:00:56+01:00 GitHub noreply@github.com 2024-02-16T12:00:56+02:00 server : fix system prompt cli (#5516) f486f6e1e5e9d01603d9325ab3e05f1edb362a95 60ed04cf82dc91ade725dd7ad53f0ee81f76eccf bmwl brian.marshall@tolko.com 2024-02-16T01:31:07-08:00 GitHub noreply@github.com 2024-02-16T11:31:07+02:00 ggml : add numa options (#5377) 60ed04cf82dc91ade725dd7ad53f0ee81f76eccf 594845aab1c6775877f6d9545a51dc0f8d0b3d77 Daniel Bevenius daniel.bevenius@gmail.com 2024-02-16T10:24:39+01:00 GitHub noreply@github.com 2024-02-16T11:24:39+02:00 llava : fix clip-model-is-vision flag in README.md (#5509) 594845aab1c6775877f6d9545a51dc0f8d0b3d77 4524290e87b8e107cc2b56e1251751546f4b9051 Georgi Gerganov ggerganov@gmail.com 2024-02-16T09:57:55+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-16T09:57:55+02:00 ci : fix BERT model download and convert 4524290e87b8e107cc2b56e1251751546f4b9051 c06e45d72983d9ace7b1535f7e7ea258d212169e Douglas Hanley thesecretaryofwar@gmail.com 2024-02-15T11:21:49-06:00 GitHub noreply@github.com 2024-02-15T12:21:49-05:00 Use correct type of pooling for embedding models (#5500) c06e45d72983d9ace7b1535f7e7ea258d212169e 9060a1e9dfca6038906e819be5fa42217f49028c Georgi Gerganov ggerganov@gmail.com 2024-02-15T18:49:08+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-15T18:49:08+02:00 clip : fix wrong loop condition 9060a1e9dfca6038906e819be5fa42217f49028c 9350a1cf21b1492c69b20175b73a419b897d6a3a slaren slarengh@gmail.com 2024-02-15T16:49:01+01:00 GitHub noreply@github.com 2024-02-15T16:49:01+01:00 cuda : print message when initialization fails (#5512) 9350a1cf21b1492c69b20175b73a419b897d6a3a 73122473ffd73030146276dbb85da7c8021a3ee4 Georgi Gerganov ggerganov@gmail.com 2024-02-15T15:41:15+02:00 GitHub noreply@github.com 2024-02-15T15:41:15+02:00 scripts : add hf.sh helper script (#5501) 73122473ffd73030146276dbb85da7c8021a3ee4 0d4177126b0556e202efb85bf3f768be81076400 Michaël de Vries vriesdemichael@gmail.com 2024-02-15T14:14:37+01:00 GitHub noreply@github.com 2024-02-15T14:14:37+01:00 fix(gguf-py): special tokens are no longer skipped when add__token is set to false (#5487) 0d4177126b0556e202efb85bf3f768be81076400 7930a8a6e89a04c77c51e3ae5dc1cd8e845b6b8f Elbios 141279586+Elbios@users.noreply.github.com 2024-02-15T09:01:57+01:00 GitHub noreply@github.com 2024-02-15T10:01:57+02:00 llava : fix memory management bug (#5491) 7930a8a6e89a04c77c51e3ae5dc1cd8e845b6b8f 704359e29985a06a389337a2617b7f3fa8eff908 John 78893154+cmp-nct@users.noreply.github.com 2024-02-15T08:59:18+01:00 GitHub noreply@github.com 2024-02-15T09:59:18+02:00 llaba : hotfix for llava-1.6 image number (#5495) 704359e29985a06a389337a2617b7f3fa8eff908 594fca3fefe27b8e95cfb1656eb0e160ad15a793 Neuman Vong neuman.vong@gmail.com 2024-02-15T17:11:15+11:00 GitHub noreply@github.com 2024-02-15T07:11:15+01:00 vulkan: Find optimal memory type but with fallback (#5381) 594fca3fefe27b8e95cfb1656eb0e160ad15a793 ccbb277f4642fc0d84c72dbc0d51ed2df418d6ce Rune 43761327+Rune-AI@users.noreply.github.com 2024-02-14T16:15:49+01:00 GitHub noreply@github.com 2024-02-14T17:15:49+02:00 readme : fix typo (#5490) ccbb277f4642fc0d84c72dbc0d51ed2df418d6ce 8084d554406b767d36b3250b3b787462d5dd626f John 78893154+cmp-nct@users.noreply.github.com 2024-02-14T15:49:42+01:00 GitHub noreply@github.com 2024-02-14T16:49:42+02:00 llava : update README.md (#5489) 8084d554406b767d36b3250b3b787462d5dd626f aa2341298924ac89778252015efcb792f2df1e20 Michael Podvitskiy podvitskiymichael@gmail.com 2024-02-14T11:49:01+03:00 GitHub noreply@github.com 2024-02-14T10:49:01+02:00 cmake : ARM intrinsics detection for MSVC (#5401) aa2341298924ac89778252015efcb792f2df1e20 f5ca054855dea83f424003162f26de376e5643f6 John 78893154+cmp-nct@users.noreply.github.com 2024-02-14T08:38:35+01:00 GitHub noreply@github.com 2024-02-14T09:38:35+02:00 llava : support v1.6 (#5267) f5ca054855dea83f424003162f26de376e5643f6 6c00a066928b0475b865a2e3e709e2166e02d548 AT manyoso@users.noreply.github.com 2024-02-13T15:44:25-06:00 GitHub noreply@github.com 2024-02-13T22:44:25+01:00 Early return for zero size calls to get_tensor. (#5482) 6c00a066928b0475b865a2e3e709e2166e02d548 ea9c8e11436ad50719987fa23a289c74b7b40d40 John 78893154+cmp-nct@users.noreply.github.com 2024-02-13T18:56:38+01:00 GitHub noreply@github.com 2024-02-13T19:56:38+02:00 gguf : add python reader example (#5216) ea9c8e11436ad50719987fa23a289c74b7b40d40 c4e6dd59e45ef7b14f7763fb073b517395dc176c Jared Van Bortel jared@nomic.ai 2024-02-13T12:03:53-05:00 GitHub noreply@github.com 2024-02-13T12:03:53-05:00 llama : add support for Nomic Embed (#5468) c4e6dd59e45ef7b14f7763fb073b517395dc176c 037259be689353081e7bae3c1ab4ab18e7fbe8c9 Aarni Koskela akx@iki.fi 2024-02-13T18:18:16+02:00 GitHub noreply@github.com 2024-02-13T18:18:16+02:00 llama : allow raw byte in SPM vocabs; don't crash on nl 404 (#5478) 037259be689353081e7bae3c1ab4ab18e7fbe8c9 263978904c7472db1865409a7ff1129599f6a40b Aarni Koskela akx@iki.fi 2024-02-13T15:24:50+02:00 GitHub noreply@github.com 2024-02-13T15:24:50+02:00 llama : make load error reporting more granular (#5477) 263978904c7472db1865409a7ff1129599f6a40b cf45252a7cfcb998bade46a886e20477cecc538a Daniel Bevenius daniel.bevenius@gmail.com 2024-02-13T14:15:42+01:00 GitHub noreply@github.com 2024-02-13T15:15:42+02:00 finetune : rename feed-forward tensors (w1/w2/w3) (#4839) cf45252a7cfcb998bade46a886e20477cecc538a 03bf161eb6dea6400ee49c6dc6b69bdcfa9fd3fc Georgi Gerganov ggerganov@gmail.com 2024-02-13T15:14:22+02:00 GitHub noreply@github.com 2024-02-13T15:14:22+02:00 tests : multi-thread the tokenizer tests (#5474) 03bf161eb6dea6400ee49c6dc6b69bdcfa9fd3fc ad014bba97ef6ef6c3e2f78b2fc463e91ae94579 Douglas Hanley thesecretaryofwar@gmail.com 2024-02-13T06:06:58-06:00 GitHub noreply@github.com 2024-02-13T14:06:58+02:00 llama : support batched embeddings (#5466) ad014bba97ef6ef6c3e2f78b2fc463e91ae94579 49cc1f7d67de2da99f3ac185f9ff1319b7bf35f8 Johannes Gäßler johannesg@5d6.de 2024-02-13T12:38:37+01:00 GitHub noreply@github.com 2024-02-13T12:38:37+01:00 make: add error message for bad CUDA version (#5444) 49cc1f7d67de2da99f3ac185f9ff1319b7bf35f8 99b8b43d7b185a6483f28cf798a2d968b2e16ca7 Georgi Gerganov ggerganov@gmail.com 2024-02-13T13:01:29+02:00 GitHub noreply@github.com 2024-02-13T13:01:29+02:00 bert : add tests + fix quantization (#5475) 99b8b43d7b185a6483f28cf798a2d968b2e16ca7 895407f31b358e3d9335e847d13f033491ec8a5b Georgi Gerganov ggerganov@gmail.com 2024-02-13T11:20:24+02:00 GitHub noreply@github.com 2024-02-13T11:20:24+02:00 tests : disable moe test (#5473) 895407f31b358e3d9335e847d13f033491ec8a5b 099afc6274c859ca67146e725839f2d97a5ef313 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-13T09:07:57+02:00 GitHub noreply@github.com 2024-02-13T09:07:57+02:00 ggml-quants : fix compiler warnings (shadow variable) (#5472) 099afc6274c859ca67146e725839f2d97a5ef313 df334a11251b81fd0b6a0e51e7146e0ba9e973f2 Georgi Gerganov ggerganov@gmail.com 2024-02-12T20:14:39+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-12T20:14:39+02:00 llama : fix quantization when tensors are missing (#5423) df334a11251b81fd0b6a0e51e7146e0ba9e973f2 dbd8828eb03b9aa8d0af7e4c533d3c2f5b38aba6 Georgi Gerganov ggerganov@gmail.com 2024-02-12T19:54:29+02:00 GitHub noreply@github.com 2024-02-12T19:54:29+02:00 swift : package no longer use ggml dependency (#5465) dbd8828eb03b9aa8d0af7e4c533d3c2f5b38aba6 43fe07c1a4f3a58612e1d9543f7c6b556710f5d0 Lee 44310445+lx200916@users.noreply.github.com 2024-02-13T01:29:57+08:00 GitHub noreply@github.com 2024-02-12T19:29:57+02:00 py : fix persimmon `n_rot` conversion (#5460) 43fe07c1a4f3a58612e1d9543f7c6b556710f5d0 4a46d2b7923be83d6019251671ee63aa1fa0d6bc Abhilash Majumder 30946547+abhilash1910@users.noreply.github.com 2024-02-12T20:22:05+05:30 GitHub noreply@github.com 2024-02-12T20:22:05+05:30 ggml-sycl: Replace 3d ops with macro (#5458) 4a46d2b7923be83d6019251671ee63aa1fa0d6bc 3b169441dfe8e420f88d1592708cc2a871daadb9 Daniel Bevenius daniel.bevenius@gmail.com 2024-02-12T09:38:44+01:00 GitHub noreply@github.com 2024-02-12T10:38:44+02:00 llava : remove prog parameter from ArgumentParser (#5457) 3b169441dfe8e420f88d1592708cc2a871daadb9 3bdc4cd0f595a6096cca4a64aa75ffa8a3503465 Georgi Gerganov ggerganov@gmail.com 2024-02-12T09:16:06+02:00 GitHub noreply@github.com 2024-02-12T09:16:06+02:00 sync : ggml (#5452) 3bdc4cd0f595a6096cca4a64aa75ffa8a3503465 2891c8aa9af17f4ff636ff3868bc34ff72b56e25 Johannes Gäßler johannesg@5d6.de 2024-02-11T19:08:39+01:00 GitHub noreply@github.com 2024-02-11T19:08:39+01:00 CUDA: mul_mat_vec_q tiling, refactor mul mat logic (#5434) 2891c8aa9af17f4ff636ff3868bc34ff72b56e25 97a336507ed9b971d72262bec7e2b8b7016a054a Douglas Hanley thesecretaryofwar@gmail.com 2024-02-11T10:21:38-06:00 GitHub noreply@github.com 2024-02-11T11:21:38-05:00 Add support for BERT embedding models (#5423) 97a336507ed9b971d72262bec7e2b8b7016a054a c88c74f967028ae3d5ebade40ae586d20a961abc github-actions[bot] github-actions[bot]@users.noreply.github.com 2024-02-11T00:17:31Z Philip Taron philip.taron@gmail.com 2024-02-11T07:50:41-08:00 flake.lock: Update c88c74f967028ae3d5ebade40ae586d20a961abc a803333a4e6fc534c93afe90d741bc2388bdec87 Sergio López slp@sinrega.org 2024-02-11T15:12:00+01:00 GitHub noreply@github.com 2024-02-11T15:12:00+01:00 vulkan: only use M-sized matmul on Apple GPUs (#5412) a803333a4e6fc534c93afe90d741bc2388bdec87 684780141a08200ec98eba3e982dbafd1d0b5000 Alexey Parfenov zxed@alkatrazstudio.net 2024-02-11T13:43:31Z GitHub noreply@github.com 2024-02-11T15:43:31+02:00 common : use enums for sampler types (#5418) 684780141a08200ec98eba3e982dbafd1d0b5000 85910c5b30f6e268321be8df044f5528a6efac52 Alexey Parfenov zxed@alkatrazstudio.net 2024-02-11T13:38:14Z GitHub noreply@github.com 2024-02-11T15:38:14+02:00 server : allow to specify tokens as strings in logit_bias (#5003) 85910c5b30f6e268321be8df044f5528a6efac52 139b62a839825ef20084ed75ed624db7a5ad554a Georgi Gerganov ggerganov@gmail.com 2024-02-11T15:35:50+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-11T15:35:50+02:00 main : ctrl+C print timing in non-interactive mode (#3873) 139b62a839825ef20084ed75ed624db7a5ad554a 0f2411f154db46780d3aaa3a0664691b2170c83f Georgi Gerganov ggerganov@gmail.com 2024-02-11T15:33:43+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-11T15:33:43+02:00 common : fix compile warning 0f2411f154db46780d3aaa3a0664691b2170c83f a07d0fee1f05c5c1dc49948ae1a3293db017275f Georgi Gerganov ggerganov@gmail.com 2024-02-11T15:33:01+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-11T15:33:01+02:00 ggml : fix compile warnings (unused vars) (#4966) a07d0fee1f05c5c1dc49948ae1a3293db017275f e4640d8fdf56f14a6db3d092bcd3d2d315cb5d04 snadampal 87143774+snadampal@users.noreply.github.com 2024-02-11T07:22:33-06:00 GitHub noreply@github.com 2024-02-11T15:22:33+02:00 ggml : add mmla kernels for quantized GEMM (#4966) e4640d8fdf56f14a6db3d092bcd3d2d315cb5d04 907e08c1109f498b01036367804cff3082c44524 Johannes Gäßler johannesg@5d6.de 2024-02-11T12:44:51+01:00 GitHub noreply@github.com 2024-02-11T12:44:51+01:00 lookup: add print for drafting performance (#5450) 907e08c1109f498b01036367804cff3082c44524 f026f8120f97090d34a52b3dc023c82e0ede3f7d Xuan Son Nguyen thichthat@gmail.com 2024-02-11T11:16:22+01:00 GitHub noreply@github.com 2024-02-11T12:16:22+02:00 server : add llama2 chat template (#5425) f026f8120f97090d34a52b3dc023c82e0ede3f7d cd9aea63b577a83def84dbd6dcd90a6fa02af745 Ian Bull irbull@eclipsesource.com 2024-02-10T02:53:28-08:00 GitHub noreply@github.com 2024-02-10T12:53:28+02:00 metal : use autoreleasepool to avoid memory leaks (#5437) cd9aea63b577a83def84dbd6dcd90a6fa02af745 43b65f5eb85e8741aba573a8f65bb8efad245d31 Georgi Gerganov ggerganov@gmail.com 2024-02-10T09:53:05+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-10T09:53:05+02:00 scripts : update sync scripts with new backends 43b65f5eb85e8741aba573a8f65bb8efad245d31 4633d93af08d890ecd00fa6e4f61d76f21cded4c Georgi Gerganov ggerganov@gmail.com 2024-02-10T09:30:36+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-10T09:30:36+02:00 sync : ggml 4633d93af08d890ecd00fa6e4f61d76f21cded4c 4b7b38bef5addbd31f453871d79647fbae6bec8a Michael Podvitskiy podvitskiymichael@gmail.com 2024-02-09T10:42:27+01:00 Georgi Gerganov ggerganov@gmail.com 2024-02-10T09:29:21+02:00 ggml : add abort_callback for cpu backend (ggml/725) 4b7b38bef5addbd31f453871d79647fbae6bec8a e00d2a62dd1441e3b089570ec06d05c18800d368 Neuman Vong neuman.vong@gmail.com 2024-02-10T05:30:19+11:00 GitHub noreply@github.com 2024-02-09T19:30:19+01:00 vulkan: Set limit for task concurrency (#5427) e00d2a62dd1441e3b089570ec06d05c18800d368 7c777fcd5dd4af7079e33390cf6a19c328a2666f Daniel Bevenius daniel.bevenius@gmail.com 2024-02-09T14:00:59+01:00 GitHub noreply@github.com 2024-02-09T15:00:59+02:00 llava : add requirements.txt and update README.md (#5428) 7c777fcd5dd4af7079e33390cf6a19c328a2666f e5ca3937c685d6e012ac4db40555d6ec100ff03c Riley Stewart ristew@users.noreply.github.com 2024-02-09T02:49:49-08:00 GitHub noreply@github.com 2024-02-09T12:49:49+02:00 server : fix prompt caching for repeated prompts (#5420) e5ca3937c685d6e012ac4db40555d6ec100ff03c e4124c24775f2cb5b3d7acc93bf9dc5471c172ef Paul Tsochantaris ptsochantaris@icloud.com 2024-02-09T10:48:06Z GitHub noreply@github.com 2024-02-09T12:48:06+02:00 llama : do not cap thread count when MoE on CPU (#5419) e4124c24775f2cb5b3d7acc93bf9dc5471c172ef b2f87cb64db47d799b6f3656855c9caf9792ab2a Marko Tasic mtasic85@gmail.com 2024-02-09T11:17:00+01:00 GitHub noreply@github.com 2024-02-09T12:17:00+02:00 readme : add JavaScript/Wasm repo (#5415) b2f87cb64db47d799b6f3656855c9caf9792ab2a 44fbe34360dd760f9e68b4271f21533436397f84 Michael Podvitskiy podvitskiymichael@gmail.com 2024-02-09T10:56:43+01:00 GitHub noreply@github.com 2024-02-09T11:56:43+02:00 ggml : fix `error C2078: too many initializers` for MSVC ARM64 (#5404) 44fbe34360dd760f9e68b4271f21533436397f84 8e6a9d2de0096af7120606c74ee2f26684e87b41 0cc4m picard12@live.de 2024-02-09T06:52:33+01:00 GitHub noreply@github.com 2024-02-09T06:52:33+01:00 Fix Vulkan crash on APUs with very little device memory (#5424) 8e6a9d2de0096af7120606c74ee2f26684e87b41 41f308f58edc2a04bcf9e245100b0a9b10e9a0fb Johannes Gäßler johannesg@5d6.de 2024-02-08T21:56:40+01:00 GitHub noreply@github.com 2024-02-08T21:56:40+01:00 CUDA: more warps for mmvq on NVIDIA (#5394) 41f308f58edc2a04bcf9e245100b0a9b10e9a0fb 6e99f2a04f1871d637dd77eb4d81de31a5510253 slaren slarengh@gmail.com 2024-02-08T21:33:03+01:00 GitHub noreply@github.com 2024-02-08T21:33:03+01:00 llama : do not print "offloading layers" message in CPU-only builds (#5416) 6e99f2a04f1871d637dd77eb4d81de31a5510253 ff4ff05c5ff4311c05a8ce1f984c7d8def4f07a5 Abhilash Majumder 30946547+abhilash1910@users.noreply.github.com 2024-02-08T22:39:10+05:30 GitHub noreply@github.com 2024-02-08T22:39:10+05:30 Fix f16_sycl cpy call from Arc (#5411) ff4ff05c5ff4311c05a8ce1f984c7d8def4f07a5 b7b74cef36a93ae01e0b9af8986d131761742d0e Daniel Bevenius daniel.bevenius@gmail.com 2024-02-08T15:20:03+01:00 GitHub noreply@github.com 2024-02-08T16:20:03+02:00 llava : add missing .py, and fix paths in README.md (#5414) b7b74cef36a93ae01e0b9af8986d131761742d0e 4aa43fab569215a13495a7f1a0f8afc541b16d03 Johannes Gäßler johannesg@5d6.de 2024-02-08T11:36:54+01:00 GitHub noreply@github.com 2024-02-08T11:36:54+01:00 fix trailing whitespace (#5407) 4aa43fab569215a13495a7f1a0f8afc541b16d03 a6e514a85f0fda38ff78ec91782877ea3d19ed98 runfuture runfuture@users.noreply.github.com 2024-02-08T18:36:19+08:00 GitHub noreply@github.com 2024-02-08T12:36:19+02:00 llama : fix MiniCPM (#5392) a6e514a85f0fda38ff78ec91782877ea3d19ed98 26d4efd11e48908e14e2ee9471a7fc4c57079a1d Daniel Bevenius daniel.bevenius@gmail.com 2024-02-08T09:58:19+01:00 GitHub noreply@github.com 2024-02-08T09:58:19+01:00 llava: fix typo/formatting in README.md (#5405) 26d4efd11e48908e14e2ee9471a7fc4c57079a1d 8504d2d0da8cc7a1f2eee0e9e56949f960510b75 Johannes Gäßler johannesg@5d6.de 2024-02-08T09:46:30+01:00 GitHub noreply@github.com 2024-02-08T09:46:30+01:00 sampling: fix top_k <= 0 (#5388) 8504d2d0da8cc7a1f2eee0e9e56949f960510b75 c4fbb6717c684196bd13b72d21747557130914e8 Georgi Gerganov ggerganov@gmail.com 2024-02-08T09:46:47+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-08T09:46:47+02:00 tests : .gitignore obj files c4fbb6717c684196bd13b72d21747557130914e8 8c933b70c21e05b685d476d0a1f36b34cbda7365 Michael Podvitskiy podvitskiymichael@gmail.com 2024-02-07T22:39:23+01:00 GitHub noreply@github.com 2024-02-07T16:39:23-05:00 CMAKE_OSX_ARCHITECTURES for MacOS cross compilation (#5393) 8c933b70c21e05b685d476d0a1f36b34cbda7365 b906596bb775b17656c2e51d5ab1b347faab6860 Ebey Abraham ebey97@gmail.com 2024-02-07T21:11:30Z GitHub noreply@github.com 2024-02-07T22:11:30+01:00 fix typo in readme (#5399) b906596bb775b17656c2e51d5ab1b347faab6860 aa7ab99be29b633263803f2e185265734c2d9427 Kamil Tomšík info@tomsik.cz 2024-02-07T19:44:52+01:00 GitHub noreply@github.com 2024-02-07T13:44:52-05:00 Add Ava in the list of llama.cpp UIs (#4362) aa7ab99be29b633263803f2e185265734c2d9427 10afa6f1d11ebc9fcc1085f468170002cbf6e2b5 Johannes Gäßler johannesg@5d6.de 2024-02-07T12:40:26+01:00 GitHub noreply@github.com 2024-02-07T12:40:26+01:00 CUDA: fixed mmvq kernel for bs 2,3,4 and -sm row (#5386) 10afa6f1d11ebc9fcc1085f468170002cbf6e2b5 0ef46da632c32faa1a538e5dc180994e8bbb46e1 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-02-07T18:16:55+08:00 GitHub noreply@github.com 2024-02-07T18:16:55+08:00 [SYCL] update install make by w64devkit (#5297) 0ef46da632c32faa1a538e5dc180994e8bbb46e1 ee1628bdfea8b0079fed0140ac2f00ef1b465b57 Xiao-Yong Jin jinxiaoyong@gmail.com 2024-02-07T02:17:25-06:00 GitHub noreply@github.com 2024-02-07T10:17:25+02:00 llava-cli : always tokenize special tokens (#5382) ee1628bdfea8b0079fed0140ac2f00ef1b465b57 ed0bf32290ee5b30ffad5becd99cbecef74aedd7 0cc4m picard12@live.de 2024-02-07T07:54:50+01:00 GitHub noreply@github.com 2024-02-07T07:54:50+01:00 Basic Vulkan Multi-GPU implementation (#5321) ed0bf32290ee5b30ffad5becd99cbecef74aedd7 9a697d842bc0cfce8268ebd2ba703ffc1c904f98 Eve 139727413+netrunnereve@users.noreply.github.com 2024-02-07T06:21:30Z GitHub noreply@github.com 2024-02-07T08:21:30+02:00 readme : modernize (#5379) 9a697d842bc0cfce8268ebd2ba703ffc1c904f98 316c7faf7740fa98ea68f1445f4505810f706b9e Ben Williams ben@719ben.com 2024-02-06T22:16:48-08:00 GitHub noreply@github.com 2024-02-07T08:16:48+02:00 readme : update ui list (#5354) 316c7faf7740fa98ea68f1445f4505810f706b9e f3e2b4fa3f81a410ecb7dec929c259ef8d8dbb7d runfuture runfuture@users.noreply.github.com 2024-02-07T14:15:56+08:00 GitHub noreply@github.com 2024-02-07T08:15:56+02:00 llama : add MiniCPM support (#5346) f3e2b4fa3f81a410ecb7dec929c259ef8d8dbb7d f68664ac241a6b5c233d8f1051eef20929b06008 Justin Parker jparkerweb@gmail.com 2024-02-07T01:15:19-05:00 GitHub noreply@github.com 2024-02-07T08:15:19+02:00 server : update `/props` with "total_slots" value (#5373) f68664ac241a6b5c233d8f1051eef20929b06008 213d1439fadefe182f69c5f7e8dd3b4b6572ebcb Sang-Kil Park sang.park@42dot.ai 2024-02-07T13:28:00+09:00 GitHub noreply@github.com 2024-02-06T23:28:00-05:00 convert : fix TypeError on GPT-2 vocab.json (#5288) 213d1439fadefe182f69c5f7e8dd3b4b6572ebcb 17c97fb0620448b37516a3f53fea6c482b0a30a4 Alexey Parfenov zxed@alkatrazstudio.net 2024-02-06T18:08:38Z GitHub noreply@github.com 2024-02-06T20:08:38+02:00 server : remove model.json endpoint (#5371) 17c97fb0620448b37516a3f53fea6c482b0a30a4 b08f22c882a1443e6b97081f3ce718a4d1a741f8 Johannes Gäßler johannesg@5d6.de 2024-02-06T18:43:06+01:00 GitHub noreply@github.com 2024-02-06T19:43:06+02:00 CUDA: mul_mat_vec_q max. batch size 8 -> 4 (#5370) b08f22c882a1443e6b97081f3ce718a4d1a741f8 f57fadc009cbff741a1961cb7896c47d73978d2c Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-06T19:00:16+02:00 GitHub noreply@github.com 2024-02-06T19:00:16+02:00 Update README.md (#5366) f57fadc009cbff741a1961cb7896c47d73978d2c 2e9c0bd6b301155ce749e162527fc55e9fb5b832 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-06T17:28:02+02:00 GitHub noreply@github.com 2024-02-06T17:28:02+02:00 Slight quantization improvement for Q4_K and Q5_K (#5361) 2e9c0bd6b301155ce749e162527fc55e9fb5b832 2c516611f1d0f1e5e9754f8ea1cf97cb1b17bf2c BarfingLemurs 128182951+BarfingLemurs@users.noreply.github.com 2024-02-06T09:06:48-05:00 GitHub noreply@github.com 2024-02-06T16:06:48+02:00 readme : add phi, orion 14b, internlm2, and yi-VL to readme (#5362) 2c516611f1d0f1e5e9754f8ea1cf97cb1b17bf2c 8a79c591de9b7ff3242a94f68b7fb5a17ed8c2be Johannes Gäßler johannesg@5d6.de 2024-02-06T14:44:06+01:00 GitHub noreply@github.com 2024-02-06T14:44:06+01:00 CUDA: mul_mat_vec_q for batch sizes > 1 (#5351) 8a79c591de9b7ff3242a94f68b7fb5a17ed8c2be 31e790322133a4b1d0684527ea446e765e8a96cf Justin Parker jparkerweb@gmail.com 2024-02-06T04:20:59-05:00 GitHub noreply@github.com 2024-02-06T11:20:59+02:00 server : include total "num_slots" in props endpoint (#5349) 31e790322133a4b1d0684527ea446e765e8a96cf 4ffc7a17d4e80c5f3f905139cb570ed9b6934fcb Michael Coppola m18coppola@gmail.com 2024-02-06T04:20:00-05:00 GitHub noreply@github.com 2024-02-06T11:20:00+02:00 server : add `dynatemp_range` and `dynatemp_exponent` (#5352) 4ffc7a17d4e80c5f3f905139cb570ed9b6934fcb 906cff55c2848fda091d888a1585915ec0c9ea9e Niall Coates 1349685+Niall-@users.noreply.github.com 2024-02-06T08:16:23Z GitHub noreply@github.com 2024-02-06T10:16:23+02:00 server : various fixes for the prompt field in /completion (#5300) 906cff55c2848fda091d888a1585915ec0c9ea9e 098f6d737b65134cf220d12b9b706e8cfc5e4610 Georgi Gerganov ggerganov@gmail.com 2024-02-06T07:47:22+02:00 GitHub noreply@github.com 2024-02-06T07:47:22+02:00 py : handle byte tokens in `get_token_type` (#5341) 098f6d737b65134cf220d12b9b706e8cfc5e4610 78b00dda6c0d62c34f5371d47718defff6ed2b22 Johannes Gäßler johannesg@5d6.de 2024-02-05T19:33:00+01:00 GitHub noreply@github.com 2024-02-05T19:33:00+01:00 make: Use ccache for faster compilation (#5318) 78b00dda6c0d62c34f5371d47718defff6ed2b22 c6b395535a6874d749ef47c33eacd466cb252cd5 Johannes Gäßler johannesg@5d6.de 2024-02-05T15:55:10+01:00 GitHub noreply@github.com 2024-02-05T15:55:10+01:00 README: updated introduction (#5343) c6b395535a6874d749ef47c33eacd466cb252cd5 abb61944a5f64dec62c893ed0db10790169b672a Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-05T14:09:47+02:00 GitHub noreply@github.com 2024-02-05T14:09:47+02:00 ggml : make use of ggml-quants.h possible in C++ code (#5338) abb61944a5f64dec62c893ed0db10790169b672a 89503dcb5f764a5cc7093db1f395f5121876a2cc Dr. Tom Murphy VII Ph.D 499244+tom7@users.noreply.github.com 2024-02-05T06:13:57-05:00 GitHub noreply@github.com 2024-02-05T13:13:57+02:00 ggml : avoid duplicating function calls using MIN/MAX macros (#5325) 89503dcb5f764a5cc7093db1f395f5121876a2cc 7e1ae372f36d98fa66b1d778c5862904b4d80c88 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-05T12:32:27+02:00 GitHub noreply@github.com 2024-02-05T12:32:27+02:00 iq3_xxs: quards for the no-imatrix situation (#5334) 7e1ae372f36d98fa66b1d778c5862904b4d80c88 6fdfa2ecc684000a25a4ad91823bc82a6652b645 Guoteng 32697156+SolenoidWGT@users.noreply.github.com 2024-02-05T17:04:06+08:00 GitHub noreply@github.com 2024-02-05T11:04:06+02:00 py : fix internlm2-hf convert to gguf (#5305) 6fdfa2ecc684000a25a4ad91823bc82a6652b645 a2d60c9158435ae9a6f14632f07f1acf7a3becef Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-05T10:46:06+02:00 GitHub noreply@github.com 2024-02-05T10:46:06+02:00 iq2_xxs: tune quantization (#5320) a2d60c9158435ae9a6f14632f07f1acf7a3becef e6f81775323f6f4e4a30abf022a6028fa86b79ac Alexey Parfenov zxed@alkatrazstudio.net 2024-02-05T08:10:22Z GitHub noreply@github.com 2024-02-05T10:10:22+02:00 server : allow to get default generation settings for completion (#5307) e6f81775323f6f4e4a30abf022a6028fa86b79ac 30679d438d5225b3aecf5cec6482cbc9f8f87ba5 l3utterfly gc.pthzfoldr@gmail.com 2024-02-05T17:00:47+09:00 GitHub noreply@github.com 2024-02-05T10:00:47+02:00 common : add dynamic temperature parameters to main example cli (#5295) 30679d438d5225b3aecf5cec6482cbc9f8f87ba5 4be04c8965578edc09194fab769b4b922b8444f5 Georgi Gerganov ggerganov@gmail.com 2024-02-05T09:48:03+02:00 Georgi Gerganov ggerganov@gmail.com 2024-02-05T09:48:03+02:00 scripts : fix typos, cleanup (#5303) 4be04c8965578edc09194fab769b4b922b8444f5 5d55b0cd827bb0fcfedfa329a82bd5d6ef2c93ca Нияз Гарифзянов 112617865+garrnizon@users.noreply.github.com 2024-02-05T10:43:57+03:00 GitHub noreply@github.com 2024-02-05T09:43:57+02:00 scripts : add non-interactive server-llm.sh (#5303) 5d55b0cd827bb0fcfedfa329a82bd5d6ef2c93ca 4833ac209da6a427de64f97e8f403dcdc5de6bc3 chiranko 96988916+chiranko@users.noreply.github.com 2024-02-05T15:41:38+08:00 GitHub noreply@github.com 2024-02-05T09:41:38+02:00 readme : add CodeShell models to the supported models list (#5330) 4833ac209da6a427de64f97e8f403dcdc5de6bc3 9392ebd49ea5ae236a55b47cbf6a13247e8a3b8c AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-02-05T07:08:24Z GitHub noreply@github.com 2024-02-05T12:38:24+05:30 [SYCL] Fix cpy with dims of 3 (#5289) 9392ebd49ea5ae236a55b47cbf6a13247e8a3b8c 5ed26e1fc9fab4ce96ecf2d84183fe45bdcab0d4 github-actions[bot] github-actions[bot]@users.noreply.github.com 2024-02-04T00:17:24Z Philip Taron philip.taron@gmail.com 2024-02-04T08:45:35-08:00 flake.lock: Update 5ed26e1fc9fab4ce96ecf2d84183fe45bdcab0d4 277fad30c60ef3559dc2d01b19d05e659d40a824 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-02-04T10:39:58+02:00 GitHub noreply@github.com 2024-02-04T10:39:58+02:00 Adding some imatrix tools (#5302) 277fad30c60ef3559dc2d01b19d05e659d40a824 3c0d25c4756742ebf15ad44700fabc0700c638bd Welby Seely welbyseely@gmail.com 2024-02-03T23:18:51-05:00 GitHub noreply@github.com 2024-02-03T23:18:51-05:00 cmake : use set() for LLAMA_WIN_VER (#5298) 3c0d25c4756742ebf15ad44700fabc0700c638bd 3cc5ed353c07201d8d5b98b0a4713ab633da6d04 Johannes Gäßler johannesg@5d6.de 2024-02-03T20:15:13+01:00 GitHub noreply@github.com 2024-02-03T20:15:13+01:00 make: add nvcc info print (#5310) 3cc5ed353c07201d8d5b98b0a4713ab633da6d04 60ecf099eddfe70fec797ef6790572e452054add Johannes Gäßler johannesg@5d6.de 2024-02-03T20:14:59+01:00 GitHub noreply@github.com 2024-02-03T20:14:59+01:00 make: fix nvcc optimization flags for host code (#5309) 60ecf099eddfe70fec797ef6790572e452054add e920ed393d989ed35625ddaf182ebb52cda07fcd Martin Schwaighofer mschwaig@users.noreply.github.com 2024-01-28T12:59:43+01:00 Philip Taron philip.taron@gmail.com 2024-02-03T13:13:07-06:00 add Vulkan support to Nix flake e920ed393d989ed35625ddaf182ebb52cda07fcd 52bb63c7082c859c3f1dfc527227e6a95b299c7c 0cc4m picard12@live.de 2024-02-03T18:15:00+01:00 GitHub noreply@github.com 2024-02-03T18:15:00+01:00 Vulkan Intel Fixes, Optimizations and Debugging Flags (#5301) 52bb63c7082c859c3f1dfc527227e6a95b299c7c 1ec3332ade60aeb1494ace2211cf1a966db6d770 Michael Klimenko mklimenko29@gmail.com 2024-02-03T12:23:37+01:00 GitHub noreply@github.com 2024-02-03T13:23:37+02:00 refactor : switch to emplace_back to avoid extra object (#5291) 1ec3332ade60aeb1494ace2211cf1a966db6d770 6a66c5071a74a96c4f52cf1015a092acd18c3714 Jared Van Bortel jared@nomic.ai 2024-02-03T06:22:06-05:00 GitHub noreply@github.com 2024-02-03T13:22:06+02:00 YaRN : store rope scaling type as int32_t in memory (#5285) 6a66c5071a74a96c4f52cf1015a092acd18c3714 a305dba8ff642e57f538f42010868fe0bc5262a1 BADR contact@pythops.com 2024-02-03T12:20:26+01:00 GitHub noreply@github.com 2024-02-03T13:20:26+02:00 readme : add tenere in the ui tools list (#5284) a305dba8ff642e57f538f42010868fe0bc5262a1 191221178f51b6e81122c5bda0fd79620e547d07 AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-02-03T08:11:37Z GitHub noreply@github.com 2024-02-03T16:11:37+08:00 Fix im2col with 32fp (#5286) 191221178f51b6e81122c5bda0fd79620e547d07 e437b37fd0b2b97e6c6ff1045ec7f901faa6498a kalomaze 66376113+kalomaze@users.noreply.github.com 2024-02-02T08:15:30-06:00 GitHub noreply@github.com 2024-02-02T16:15:30+02:00 perplexity : fix KL divergence calculations on Windows (#5273) e437b37fd0b2b97e6c6ff1045ec7f901faa6498a 2d40085c26794e29c434480b9e06738e89e5686f Georgi Gerganov ggerganov@gmail.com 2024-02-02T14:23:40+02:00 GitHub noreply@github.com 2024-02-02T14:23:40+02:00 scripts : parse wtype in server-llm.sh (#5167) 2d40085c26794e29c434480b9e06738e89e5686f b05102fe8cfa9893851c6bf6efd15cdc20b6afa2 Mirror Azure 54669636+MirrorAzure@users.noreply.github.com 2024-02-02T14:39:09+03:00 GitHub noreply@github.com 2024-02-02T13:39:09+02:00 py : add check for '.attn.masked_bias' layers to GPT2model (#5281) b05102fe8cfa9893851c6bf6efd15cdc20b6afa2 6b91b1e0a92ac2e4e269eec6361ca53a61ced6c6 AidanBeltonS 87009434+AidanBeltonS@users.noreply.github.com 2024-02-02T08:39:48Z GitHub noreply@github.com 2024-02-02T16:39:48+08:00 Tidy ggml-sycl (#5261) 6b91b1e0a92ac2e4e269eec6361ca53a61ced6c6 e805f0fa9951081ce0a86378a7aa52b6f636b82d Xuan Son Nguyen thichthat@gmail.com 2024-02-02T08:56:31+01:00 GitHub noreply@github.com 2024-02-02T09:56:31+02:00 docker : add build for SYCL, Vulkan + update readme (#5228) e805f0fa9951081ce0a86378a7aa52b6f636b82d af3ba5d94627d337e32a95129e31a3064c459f6b Meng, Hengyu hengyu.meng@intel.com 2024-02-02T15:54:14+08:00 GitHub noreply@github.com 2024-02-02T15:54:14+08:00 [SYCL] get MAX_MEM_ALLOC from device property (#5270) af3ba5d94627d337e32a95129e31a3064c459f6b e1e721094d8169636d55f68efe37f222cd3f0677 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-02-02T15:53:27+08:00 GitHub noreply@github.com 2024-02-02T15:53:27+08:00 [SYCL] update guide of SYCL backend (#5254) e1e721094d8169636d55f68efe37f222cd3f0677 128dcbd3c9c4b12f42b560a4430427d7b2828628 Ian Bull irbull@gmail.com 2024-02-01T23:20:13-08:00 GitHub noreply@github.com 2024-02-02T09:20:13+02:00 llama : fix memory leak in llama_batch_free (#5252) 128dcbd3c9c4b12f42b560a4430427d7b2828628 4d0924a8902010d31bd737b6f1f594943d120d0f Neo Zhang Jianyu jianyu.zhang@intel.com 2024-02-02T03:48:53+08:00 GitHub noreply@github.com 2024-02-01T20:48:53+01:00 add --no-mmap in llama-bench (#5257) 4d0924a8902010d31bd737b6f1f594943d120d0f 8ca511cadee2c67f0bd8c7034a2513778ee9a1b7 0cc4m picard12@live.de 2024-02-01T19:25:24+01:00 GitHub noreply@github.com 2024-02-01T19:25:24+01:00 Vulkan Phi Fix for AMD Proprietary Drivers (#5260) 8ca511cadee2c67f0bd8c7034a2513778ee9a1b7 d71ac90985854b0905e1abba778e407e17f9f887 slaren slarengh@gmail.com 2024-02-01T18:30:17+01:00 GitHub noreply@github.com 2024-02-01T18:30:17+01:00 cuda : fix LLAMA_CUDA_F16 (#5262) d71ac90985854b0905e1abba778e407e17f9f887 ce32060198b7e2d6a13a9b8e1e1369e3c295ae2a Ali Nehzat ali.nehzat@thanks.dev 2024-02-02T02:18:53+11:00 GitHub noreply@github.com 2024-02-01T17:18:53+02:00 make : generate .a library for static linking (#5205) ce32060198b7e2d6a13a9b8e1e1369e3c295ae2a 1cfb5372cf5707c8ec6dde7c874f4a44a6c4c915 Guoteng 32697156+SolenoidWGT@users.noreply.github.com 2024-02-01T17:19:51+08:00 GitHub noreply@github.com 2024-02-01T11:19:51+02:00 llama : support InternLM2 (#5184) 1cfb5372cf5707c8ec6dde7c874f4a44a6c4c915 d3bac7d58408c602ec1f1e423695f1df8410bb03 Eve 139727413+netrunnereve@users.noreply.github.com 2024-01-31T19:21:55Z GitHub noreply@github.com 2024-01-31T20:21:55+01:00 Fix broken Vulkan Cmake (properly) (#5230) d3bac7d58408c602ec1f1e423695f1df8410bb03 5cb04dbc16d1da38c8fdcc0111b40e67d00dd1c3 Georgi Gerganov ggerganov@gmail.com 2024-01-31T18:47:10+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-31T18:47:10+02:00 llama : reorder build_orion() at correct place (#5118) 5cb04dbc16d1da38c8fdcc0111b40e67d00dd1c3 efb7bdbbd061d087c788598b97992c653f992ddd Georgi Gerganov ggerganov@gmail.com 2024-01-31T17:30:17+02:00 GitHub noreply@github.com 2024-01-31T17:30:17+02:00 llama : remove LLAMA_MAX_DEVICES and LLAMA_SUPPORTS_GPU_OFFLOAD (#5240) efb7bdbbd061d087c788598b97992c653f992ddd 15606309a05ccf7fadbaad5538cb7c32acb1e06b Georgi Gerganov ggerganov@gmail.com 2024-01-31T15:35:41+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-31T15:35:41+02:00 metal : add im2col F32 dst support (#5132) 15606309a05ccf7fadbaad5538cb7c32acb1e06b b2b9f025e7821e78bd501d75d01838c26de07a57 JidongZhang-THU 1119708529@qq.com 2024-01-31T21:10:15+08:00 GitHub noreply@github.com 2024-01-31T15:10:15+02:00 llava : add MobileVLM support (#5132) b2b9f025e7821e78bd501d75d01838c26de07a57 dabcc5b471348e4ae03ddacc41e19ad75fb2f041 Neo Zhang Jianyu jianyu.zhang@intel.com 2024-01-31T21:04:46+08:00 GitHub noreply@github.com 2024-01-31T18:34:46+05:30 format license text, restore apache license by legal suggestion (#5233) dabcc5b471348e4ae03ddacc41e19ad75fb2f041 f8e9140cb46eebaa867e1184a9946e4840eec772 slaren slarengh@gmail.com 2024-01-31T13:43:03+01:00 GitHub noreply@github.com 2024-01-31T13:43:03+01:00 ggml : limit n_threads to the max n_tasks (#5238) f8e9140cb46eebaa867e1184a9946e4840eec772 d62520eb2cc1d7168a30edec6110e1daefbd959f 0cc4m picard12@live.de 2024-01-31T11:44:19+01:00 GitHub noreply@github.com 2024-01-31T11:44:19+01:00 Vulkan Fixes (#5223) d62520eb2cc1d7168a30edec6110e1daefbd959f 01684139c352561840ae55ec627ab58abc3e06ab Yiming Cui conandiy@vip.qq.com 2024-01-31T11:04:21+08:00 GitHub noreply@github.com 2024-01-30T22:04:21-05:00 Fix typos of IQ2_XXS and IQ3_XXS in llama.cpp (#5231) 01684139c352561840ae55ec627ab58abc3e06ab e8dc55d0065d076d4c20f3c4bfca562701b4edfe Neo Zhang Jianyu jianyu.zhang@intel.com 2024-01-31T10:38:07+08:00 GitHub noreply@github.com 2024-01-31T08:08:07+05:30 support SYCL backend windows build (#5208) e8dc55d0065d076d4c20f3c4bfca562701b4edfe e0085fdf7c758f0bc2746fc106fb29dd9df959de Jared Van Bortel jared@nomic.ai 2024-01-30T19:04:37-05:00 GitHub noreply@github.com 2024-01-30T19:04:37-05:00 kompute : llama-bench support and ggml_cpu_has_kompute() (#5226) e0085fdf7c758f0bc2746fc106fb29dd9df959de e6f291d15844398f8326940fe5ad7f2e02b5aa56 Georgi Gerganov ggerganov@gmail.com 2024-01-30T21:19:26+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-30T21:19:26+02:00 Revert "server : change deps.sh xxd files to string literals (#5221)" e6f291d15844398f8326940fe5ad7f2e02b5aa56 4003be0e5feef320f3707786f22722b73cff9356 Georgi Gerganov ggerganov@gmail.com 2024-01-30T20:17:30+02:00 GitHub noreply@github.com 2024-01-30T20:17:30+02:00 server : fix context shift (#5195) 4003be0e5feef320f3707786f22722b73cff9356 fea4fd4ba7f6b754ac795387b275e1a014a77bde JohnnyB jboero@users.noreply.github.com 2024-01-30T12:15:05-06:00 GitHub noreply@github.com 2024-01-30T20:15:05+02:00 server : change deps.sh xxd files to string literals (#5221) fea4fd4ba7f6b754ac795387b275e1a014a77bde 8f8ddfcfadc830b936318c3ea9fe2e8e3365aa85 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-30T19:15:28+02:00 GitHub noreply@github.com 2024-01-30T19:15:28+02:00 ggml : fix IQ3_XXS on Metal (#5219) 8f8ddfcfadc830b936318c3ea9fe2e8e3365aa85 6fb50ebbf036ac57a20fe8d8da31731a543582d5 Georgi Gerganov ggerganov@gmail.com 2024-01-30T16:21:57+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-30T16:21:57+02:00 sync : ggml (#0) 6fb50ebbf036ac57a20fe8d8da31731a543582d5 625a699b5456994bc32a8093d53818f60ceda6d1 Georgi Gerganov ggerganov@gmail.com 2024-01-29T21:08:18+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-30T16:20:25+02:00 gguf : fix comparison (ggml/715) 625a699b5456994bc32a8093d53818f60ceda6d1 a4b07c057a553b1ac253051efc3f040351e2eae1 John Balis phobossystems@gmail.com 2024-01-29T06:37:33-06:00 Georgi Gerganov ggerganov@gmail.com 2024-01-30T16:20:25+02:00 `ggml_cuda_cpy` support for 4d tensors and float16->float32 upcasting (ggml/686) a4b07c057a553b1ac253051efc3f040351e2eae1 549a1e6cd5b39fe0dc3d4ea5515c65f17797a31e Georgi Gerganov ggerganov@gmail.com 2024-01-29T14:00:10+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-30T16:20:25+02:00 gguf : add input validation, prevent integer overflows (ggml/709) 549a1e6cd5b39fe0dc3d4ea5515c65f17797a31e 5f14ee0b0cd06f1c4790e6123df4b38ace637e88 Georgi Gerganov ggerganov@gmail.com 2024-01-29T13:29:46+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-30T16:20:25+02:00 ci : fix yolo URLs + fix metal capture (ggml/712) 5f14ee0b0cd06f1c4790e6123df4b38ace637e88 8e14e3ddb3744566aef7bc0fa734180e47ae6bdf Jack Mousseau jmousseau@users.noreply.github.com 2024-01-29T01:22:23-08:00 Georgi Gerganov ggerganov@gmail.com 2024-01-30T16:20:25+02:00 metal : add debug capture backend function (ggml/694) 8e14e3ddb3744566aef7bc0fa734180e47ae6bdf f4d7e5497485ce6ce0e322533930b7da4657dd2d Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-30T15:15:07+02:00 GitHub noreply@github.com 2024-01-30T15:15:07+02:00 Faster AVX2 dot product for IQ2_XS (#5187) f4d7e5497485ce6ce0e322533930b7da4657dd2d 2256f36b79a932a478d4dcdf02c1e5a60056e5f3 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-30T15:14:12+02:00 GitHub noreply@github.com 2024-01-30T15:14:12+02:00 SOTA 3-bit quants (#5196) 2256f36b79a932a478d4dcdf02c1e5a60056e5f3 7359016c7c0f65dc30cf79791212b06f15866450 0cc4m picard12@live.de 2024-01-30T13:59:30+01:00 GitHub noreply@github.com 2024-01-30T13:59:30+01:00 Vulkan Windows APU Memory Handling (#5199) 7359016c7c0f65dc30cf79791212b06f15866450 813416991ab0d1caa0d12f93ac4e8a24a2add0a3 Vladimir Malyutin first-leon@yandex.ru 2024-01-30T17:57:07+07:00 GitHub noreply@github.com 2024-01-30T12:57:07+02:00 quantize : fix typo (#5211) 813416991ab0d1caa0d12f93ac4e8a24a2add0a3 5589921ef84a4fb1c6d1c9c34d626a5a83033db6 divinity76 divinity76@gmail.com 2024-01-30T10:18:02+01:00 GitHub noreply@github.com 2024-01-30T11:18:02+02:00 main : allow empty --prompt-cache file (#5176) 5589921ef84a4fb1c6d1c9c34d626a5a83033db6 49f44b5c55d801e3d51ddcf409d866047d718905 Romain Neutron romain@neutron.io 2024-01-30T10:16:38+01:00 GitHub noreply@github.com 2024-01-30T11:16:38+02:00 readme : minor (#5204) 49f44b5c55d801e3d51ddcf409d866047d718905 6685cc41c237544623b4b5651eceb9c4280728cc Georgi Gerganov ggerganov@gmail.com 2024-01-30T11:14:44+02:00 GitHub noreply@github.com 2024-01-30T11:14:44+02:00 readme : update hot topics 6685cc41c237544623b4b5651eceb9c4280728cc ceebbb5b21b971941b2533210b74bf359981006c Wu Jian Ping wujp@greatld.com 2024-01-30T17:11:46+08:00 GitHub noreply@github.com 2024-01-30T11:11:46+02:00 server : improve README (#5209) ceebbb5b21b971941b2533210b74bf359981006c 6daa69ee81851ab26ca8aefca1a4202941fc0262 Paul Tsochantaris ptsochantaris@icloud.com 2024-01-29T22:19:29Z GitHub noreply@github.com 2024-01-29T23:19:29+01:00 ggml alloc: Fix for null dereference on alloc failure (#5200) 6daa69ee81851ab26ca8aefca1a4202941fc0262 fbf1ddec69f7001cc707de17fa74d7200813bbac Jared Van Bortel jared@nomic.ai 2024-01-29T17:11:27-05:00 GitHub noreply@github.com 2024-01-29T17:11:27-05:00 kompute : fix fallback to CPU (#5201) fbf1ddec69f7001cc707de17fa74d7200813bbac 2aed77eb06a329f0d82bb1c467f4244904d4073f Jared Van Bortel jared@nomic.ai 2024-01-29T15:50:50-05:00 GitHub noreply@github.com 2024-01-29T15:50:50-05:00 Nomic Vulkan backend (#4456) 2aed77eb06a329f0d82bb1c467f4244904d4073f c82d18e863fcde91b4b1109b1d0c73ea4470c405 divinity76 divinity76@gmail.com 2024-01-29T15:45:41+01:00 GitHub noreply@github.com 2024-01-29T09:45:41-05:00 fix typo "RLIMIT_MLOCK" (#5175) c82d18e863fcde91b4b1109b1d0c73ea4470c405 14fef85e2d5361e6b4dd7a9ecf22bb817362997d Wu Jian Ping wujjpp@hotmail.com 2024-01-29T21:48:10+08:00 GitHub noreply@github.com 2024-01-29T15:48:10+02:00 server : embeddings compatibility for OpenAI (#5190) 14fef85e2d5361e6b4dd7a9ecf22bb817362997d e76627bcce9f77adb6034cb127b7ec93d4287b69 Georgi Gerganov ggerganov@gmail.com 2024-01-29T15:35:54+02:00 GitHub noreply@github.com 2024-01-29T15:35:54+02:00 py : fix except (#5194) e76627bcce9f77adb6034cb127b7ec93d4287b69 fbe7dfa53caff0a7e830b676e6e949917a5c71b4 Sang-Kil Park sang.park@42dot.ai 2024-01-29T18:24:19+09:00 GitHub noreply@github.com 2024-01-29T11:24:19+02:00 py : improve BPE tokenizer support (#5189) fbe7dfa53caff0a7e830b676e6e949917a5c71b4 172ac82629815d038702b049070f4c8c02662da5 slaren slarengh@gmail.com 2024-01-29T09:05:13+01:00 GitHub noreply@github.com 2024-01-29T10:05:13+02:00 ggml : add max buffer sizes to opencl and metal backends (#5181) 172ac82629815d038702b049070f4c8c02662da5 d2f650cb5b04ee2726663e79b47da5efe196ce00 Eve 139727413+netrunnereve@users.noreply.github.com 2024-01-29T08:04:47Z GitHub noreply@github.com 2024-01-29T10:04:47+02:00 cmake : fix Vulkan build (#5182) d2f650cb5b04ee2726663e79b47da5efe196ce00 35dec26cc25a9ff7d8c3ed52326b94f772b911ce Paul Tsochantaris ptsochantaris@icloud.com 2024-01-28T19:50:16Z GitHub noreply@github.com 2024-01-28T21:50:16+02:00 metal : free metal objects (#5161) 35dec26cc25a9ff7d8c3ed52326b94f772b911ce d460510c7222d43a458a17e01d4bbe72437cdd3c Georgi Gerganov ggerganov@gmail.com 2024-01-28T19:48:05+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-28T19:48:05+02:00 sync : ggml d460510c7222d43a458a17e01d4bbe72437cdd3c 2307523d322af762ae06648b29ec5a9eb1c73032 Georgi Gerganov ggerganov@gmail.com 2024-01-28T18:44:58+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-28T19:47:31+02:00 ggml : minor type fix (int64_t -> size_t) 2307523d322af762ae06648b29ec5a9eb1c73032 0f648573dde61c510560f68244f70ece7e60d8c1 0cc4m picard12@live.de 2024-01-28T18:03:59+01:00 GitHub noreply@github.com 2024-01-28T19:03:59+02:00 ggml : add Vulkan backend (#2059) 0f648573dde61c510560f68244f70ece7e60d8c1 b764b8f1d079ba44d912801ce6d29bd0d94d51cf Abhilash Majumder 30946547+abhilash1910@users.noreply.github.com 2024-01-28T21:26:23+05:30 GitHub noreply@github.com 2024-01-28T17:56:23+02:00 ggml : add unified SYCL backend for Intel GPUs (#2690) b764b8f1d079ba44d912801ce6d29bd0d94d51cf 9241c3a2ace544aef708334e54bbdddb90208ee8 Georgi Gerganov ggerganov@gmail.com 2024-01-28T16:54:54+02:00 GitHub noreply@github.com 2024-01-28T14:54:54Z flake.lock: Update (#5162) 9241c3a2ace544aef708334e54bbdddb90208ee8 b2b2bf988c098851b4f3831f0cf38394bff75121 Johannes Gäßler johannesg@5d6.de 2024-01-28T09:59:49+01:00 GitHub noreply@github.com 2024-01-28T09:59:49+01:00 Apply min_p to unsorted tokens (#5115) b2b2bf988c098851b4f3831f0cf38394bff75121 af4980bfedfd8df43b9e4cd1442895e85fee37bc Johannes Gäßler johannesg@5d6.de 2024-01-28T09:35:14+01:00 GitHub noreply@github.com 2024-01-28T09:35:14+01:00 Tests for min_p, sampling queue (#5147) af4980bfedfd8df43b9e4cd1442895e85fee37bc f2e69d28c01303ca9dc79907f89ef120a6ac4a92 Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2024-01-28T00:30:44-08:00 GitHub noreply@github.com 2024-01-28T10:30:44+02:00 readme : add link to rust bindings (#5148) f2e69d28c01303ca9dc79907f89ef120a6ac4a92 39baaf55a160909bb9428bd981014218761a20cb sharpHL 132747147+sharpHL@users.noreply.github.com 2024-01-28T16:00:30+08:00 GitHub noreply@github.com 2024-01-28T10:00:30+02:00 llama : add support for Orion-14B (#5118) 39baaf55a160909bb9428bd981014218761a20cb 6db2b41a76ee78d5efdd5c3cddd5d7ad3f646855 Kyle Mistele kyle@mistele.com 2024-01-28T01:55:31-06:00 GitHub noreply@github.com 2024-01-28T09:55:31+02:00 docker : add server-first container images (#5157) 6db2b41a76ee78d5efdd5c3cddd5d7ad3f646855 753eafed0ebd07af6903771327a1786a7c02cf98 John 78893154+cmp-nct@users.noreply.github.com 2024-01-27T16:09:18+01:00 GitHub noreply@github.com 2024-01-27T17:09:18+02:00 llava : support for Yi-VL and fix for mobileVLM (#5093) 753eafed0ebd07af6903771327a1786a7c02cf98 e9764230054e01553bdead6f2bfd8e001869599d Georgi Gerganov ggerganov@gmail.com 2024-01-27T16:59:20+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-27T17:00:24+02:00 sync : ggml e9764230054e01553bdead6f2bfd8e001869599d 35a2ee914308c85ab5cb576467381443ad23f0ac Judd foldl@users.noreply.github.com 2024-01-26T21:04:01+08:00 Georgi Gerganov ggerganov@gmail.com 2024-01-27T16:59:00+02:00 ggml : check ggml_add src1 type (ggml/708) 35a2ee914308c85ab5cb576467381443ad23f0ac ec903c034131848da9222536ff18da07ec0882a0 Michael Klimenko mklimenko29@gmail.com 2024-01-27T15:25:55+01:00 GitHub noreply@github.com 2024-01-27T15:25:55+01:00 Remove unused data and add fixes (#5154) ec903c034131848da9222536ff18da07ec0882a0 a1d6df129bcd3d42cda38c09217d8d4ec4ea3bdd Maximilian Winter maximilian.winter.91@gmail.com 2024-01-27T14:38:05+01:00 GitHub noreply@github.com 2024-01-27T15:38:05+02:00 server : add self-extend support (#5104) a1d6df129bcd3d42cda38c09217d8d4ec4ea3bdd bbe7c56c9993af86aa2d84cbe1fd69e1b4300cea 0cc4m picard12@live.de 2024-01-26T23:07:32+01:00 GitHub noreply@github.com 2024-01-26T23:07:32+01:00 Add OpenCL add kernel (#5151) bbe7c56c9993af86aa2d84cbe1fd69e1b4300cea 62fead3ea0a30c8d424f4a8373fa14165c7c707f Jared Van Bortel jared@nomic.ai 2024-01-26T15:34:06-05:00 GitHub noreply@github.com 2024-01-26T15:34:06-05:00 cmake : pass CPU architecture flags to nvcc (#5146) 62fead3ea0a30c8d424f4a8373fa14165c7c707f 15b4538ff29b280a395a1406d711497d8eaa2564 slaren slarengh@gmail.com 2024-01-26T18:59:43+01:00 GitHub noreply@github.com 2024-01-26T18:59:43+01:00 cuda : fix tensor size calculation for non-split buffer (#5145) 15b4538ff29b280a395a1406d711497d8eaa2564 7032f4f6349c17a8352f9f93f7d2122f45469e59 slaren slarengh@gmail.com 2024-01-26T18:18:26+01:00 GitHub noreply@github.com 2024-01-26T19:18:26+02:00 ggml-alloc : add 10% margin to the buffer sizes (#5149) 7032f4f6349c17a8352f9f93f7d2122f45469e59 5f1925a8cef81eb9b372faaae34b0dd76d5361d4 snadampal 87143774+snadampal@users.noreply.github.com 2024-01-26T11:17:59-06:00 GitHub noreply@github.com 2024-01-26T19:17:59+02:00 ggml : update softmax n_task calculation (#5126) 5f1925a8cef81eb9b372faaae34b0dd76d5361d4 3b7c914de25c6851396d7f9178249f1ed278120e Georgi Gerganov ggerganov@gmail.com 2024-01-26T17:09:44+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-26T17:09:44+02:00 scripts : move run-with-preset.py from root to scripts folder 3b7c914de25c6851396d7f9178249f1ed278120e 48c857aa10aea73210a4a72da3f1a6f99269e75d Georgi Gerganov ggerganov@gmail.com 2024-01-26T14:48:15+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-26T14:48:15+02:00 tests : gitignore test-c.o 48c857aa10aea73210a4a72da3f1a6f99269e75d 413e7b0559f922bd4de5e9eec548829d111651b1 Xuan Son Nguyen thichthat@gmail.com 2024-01-26T13:42:20+01:00 GitHub noreply@github.com 2024-01-26T14:42:20+02:00 server : refactored the task processing logic (#5065) 413e7b0559f922bd4de5e9eec548829d111651b1 6dd3c28c9cd1ef74b49d79f47d668759346a3c6c crasm crasm@git.vczf.net 2024-01-26T07:18:00-05:00 GitHub noreply@github.com 2024-01-26T14:18:00+02:00 ci : add model tests + script wrapper (#4586) 6dd3c28c9cd1ef74b49d79f47d668759346a3c6c 38b431de232d1b736b5af19b8c7d72f7075a70bc Paul Tsochantaris ptsochantaris@icloud.com 2024-01-26T12:16:07Z GitHub noreply@github.com 2024-01-26T14:16:07+02:00 metal : remove unused `n_buffers` and `buffers` (#5129) 38b431de232d1b736b5af19b8c7d72f7075a70bc aad0b01d7380a7cdfe0dd42307b18c7b6bac9575 Riceball LEE snowyu.lee@gmail.com 2024-01-26T17:10:28+08:00 GitHub noreply@github.com 2024-01-26T11:10:28+02:00 gguf : fix "general.alignment" type in gguf_reader.py (#5136) aad0b01d7380a7cdfe0dd42307b18c7b6bac9575 1182cf4d4f6ee383b92695c2e3fe438086dcdba7 Georgi Gerganov ggerganov@gmail.com 2024-01-26T10:52:33+02:00 GitHub noreply@github.com 2024-01-26T10:52:33+02:00 readme : update hot topics 1182cf4d4f6ee383b92695c2e3fe438086dcdba7 fe54033b69b83164cabb5f3ed92dc0ff7ea47605 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-26T09:14:39+02:00 GitHub noreply@github.com 2024-01-26T09:14:39+02:00 Another bucket sort (#5109) fe54033b69b83164cabb5f3ed92dc0ff7ea47605 5eaf9964fc797d4585c214db32a463d557f3ed33 XiaotaoChen chenxiaotao1234@gmail.com 2024-01-26T04:14:32+08:00 GitHub noreply@github.com 2024-01-25T22:14:32+02:00 readme : add MobileVLM 1.7B/3B to the supported models list (#5107) 5eaf9964fc797d4585c214db32a463d557f3ed33 d292f4f2047963f558dd516f1baaa71793e9acf2 l3utterfly gc.pthzfoldr@gmail.com 2024-01-26T05:06:22+09:00 GitHub noreply@github.com 2024-01-25T22:06:22+02:00 llama : dynamic temperature sampling (#4972) d292f4f2047963f558dd516f1baaa71793e9acf2 256d1bb0ddce6a0a21f5a7503019bdd5c1933cba Jared Van Bortel jared@nomic.ai 2024-01-25T14:51:24-05:00 GitHub noreply@github.com 2024-01-25T14:51:24-05:00 examples : make pydantic scripts pass mypy and support py3.8 (#5099) 256d1bb0ddce6a0a21f5a7503019bdd5c1933cba faa3526a1eba458120987ed8269e5616385a76f4 Valentin Konovalov valle.ketsujin@gmail.com 2024-01-25T12:05:51-05:00 GitHub noreply@github.com 2024-01-25T19:05:51+02:00 android : use release cmake build type by default (#5123) faa3526a1eba458120987ed8269e5616385a76f4 ddc5a5033f948dc7ab0a3a6ec2d914d13c274077 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-25T17:58:53+02:00 GitHub noreply@github.com 2024-01-25T17:58:53+02:00 Fix Q3_K_XS for MoE models (#5113) ddc5a5033f948dc7ab0a3a6ec2d914d13c274077 cd4fddb29f81d6a1f6d51a0c016bc6b486d68def Georgi Gerganov ggerganov@gmail.com 2024-01-25T11:26:17+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-25T11:26:17+02:00 metal : show compile log messages cd4fddb29f81d6a1f6d51a0c016bc6b486d68def c9b316c78fba31e65879a2ec91cbafd341b88cce Engininja2 139037756+Engininja2@users.noreply.github.com 2024-01-24T16:18:15-06:00 GitHub noreply@github.com 2024-01-24T23:18:15+01:00 cuda : fix 2-bit quants on amd hip (#5105) c9b316c78fba31e65879a2ec91cbafd341b88cce bf63d695b804b1c995c7ae4427a8a86936ea6d25 Michael Hueschen m@mhueschen.dev 2024-01-22T16:44:10-07:00 Someone newkozlukov@gmail.com 2024-01-24T12:39:29Z nix-shell: use addToSearchPath bf63d695b804b1c995c7ae4427a8a86936ea6d25 1387ea21178f9f154944013d4dd9764b54c69deb Michael Hueschen m@mhueschen.dev 2024-01-22T03:17:05-07:00 Someone newkozlukov@gmail.com 2024-01-24T12:39:29Z nix: add cc to devShell LD_LIBRARY_PATH 1387ea21178f9f154944013d4dd9764b54c69deb 26d607608d794efa56df3bdb6043a2f94c1d632c slaren slarengh@gmail.com 2024-01-24T12:48:14+01:00 GitHub noreply@github.com 2024-01-24T12:48:14+01:00 llama : pre-allocate input tensors in a separate buffer (#5100) 26d607608d794efa56df3bdb6043a2f94c1d632c 44879ee885f48ecf4675dd216b373dce0a6f3690 Georgi Gerganov ggerganov@gmail.com 2024-01-23T15:50:56+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-23T15:50:56+02:00 metal : disable support for MUL_MAT F32 x F16 44879ee885f48ecf4675dd216b373dce0a6f3690 9ecdd12e95aee20d6dfaf5f5a0f0ce5ac1fb2747 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-23T15:17:20+02:00 GitHub noreply@github.com 2024-01-23T15:17:20+02:00 Additional KL-divergence statistics (#5081) 9ecdd12e95aee20d6dfaf5f5a0f0ce5ac1fb2747 89758723c75ba594e401f6513751beeba7ca1d28 Johannes Gäßler johannesg@5d6.de 2024-01-23T13:31:56+01:00 GitHub noreply@github.com 2024-01-23T13:31:56+01:00 CUDA: more info when no device code (#5088) 89758723c75ba594e401f6513751beeba7ca1d28 2bed4aa3f37cb4e39e16e9ec7b595a7738fd5faf Georgi Gerganov ggerganov@gmail.com 2024-01-23T14:12:57+02:00 GitHub noreply@github.com 2024-01-23T14:12:57+02:00 minor : clean-up some warnings and style (#5094) 2bed4aa3f37cb4e39e16e9ec7b595a7738fd5faf 125d03a5036a02a983c8e98c2cdc126e061afb8e Xuan Son Nguyen thichthat@gmail.com 2024-01-23T08:11:39+01:00 GitHub noreply@github.com 2024-01-23T09:11:39+02:00 devops : add intel oneapi dockerfile (#5068) 125d03a5036a02a983c8e98c2cdc126e061afb8e 011e8ec577fd135cbc02993d3ea9840c516d6a1c Michael Coppola m18coppola@gmail.com 2024-01-23T01:51:27-05:00 GitHub noreply@github.com 2024-01-23T08:51:27+02:00 llama.vim : added api key support (#5090) 011e8ec577fd135cbc02993d3ea9840c516d6a1c 6f9939d119b2d004c264952eb510bd106455531e slaren slarengh@gmail.com 2024-01-22T23:42:41+01:00 GitHub noreply@github.com 2024-01-22T23:42:41+01:00 llama : fix not enough space in buffer with Qwen (#5086) 6f9939d119b2d004c264952eb510bd106455531e 780e24a22eb595b705cbe8284771e9ceff1c4dd2 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-22T16:10:14+02:00 GitHub noreply@github.com 2024-01-22T16:10:14+02:00 KL-divergence (#5076) 780e24a22eb595b705cbe8284771e9ceff1c4dd2 3ce7e8f8e7ccfce07e5947ac5f1f3f4628cf68ea Reinforce-II fate@eastal.com 2024-01-22T21:15:08+08:00 GitHub noreply@github.com 2024-01-22T15:15:08+02:00 ggml : parallelize FP32 conversion when using BLAS (#5045) 3ce7e8f8e7ccfce07e5947ac5f1f3f4628cf68ea b2d80e105a59b54822edf7ce7f3ed5f317e96e21 XiaotaoChen chenxiaotao1234@gmail.com 2024-01-22T21:09:35+08:00 GitHub noreply@github.com 2024-01-22T15:09:35+02:00 llava : MobileVLM support (#4954) b2d80e105a59b54822edf7ce7f3ed5f317e96e21 28603cd2833cedd4434f398d847f87fc83546dbb Someone Serge sergei.kozlukov@aalto.fi 2024-01-21T03:41:37Z Someone newkozlukov@gmail.com 2024-01-22T12:19:30Z flake.nix: add a comment about flakes vs nix 28603cd2833cedd4434f398d847f87fc83546dbb 5e97ec91ae3038720a5b15cde4c52d2a53ec2137 Someone Serge sergei.kozlukov@aalto.fi 2024-01-21T03:29:38Z Someone newkozlukov@gmail.com 2024-01-22T12:19:30Z nix: add a comment on the many nixpkgs-with-cuda instances 5e97ec91ae3038720a5b15cde4c52d2a53ec2137 7251870780e2d572dd6f239d7a0bfe438c82fa74 Someone Serge sergei.kozlukov@aalto.fi 2024-01-21T03:15:13Z Someone newkozlukov@gmail.com 2024-01-22T12:19:30Z nix: add a comment about makeScope 7251870780e2d572dd6f239d7a0bfe438c82fa74 fe8b3c0d4b0d806e8b46660e24eaf4b90b8b385f Someone Serge sergei.kozlukov@aalto.fi 2024-01-13T17:45:01Z Someone newkozlukov@gmail.com 2024-01-22T12:19:30Z nix: refactor the cleanSource rules fe8b3c0d4b0d806e8b46660e24eaf4b90b8b385f f4dd059259d0234913b9e9780e1662811744c09d Someone Serge sergei.kozlukov@aalto.fi 2024-01-13T17:38:32Z Someone newkozlukov@gmail.com 2024-01-22T12:19:30Z workflows: nix-ci: drop the redundant "paths" filter f4dd059259d0234913b9e9780e1662811744c09d f7276f7500f7ea588836dd1fc6f126334c517878 Someone Serge sergei.kozlukov@aalto.fi 2024-01-13T17:16:54Z Someone newkozlukov@gmail.com 2024-01-22T12:19:30Z workflows: nix-build-aarch64: rate limit f7276f7500f7ea588836dd1fc6f126334c517878 15bceec2d73d4166340b46b27677c45ac1b4cdad Someone Serge sergei.kozlukov@aalto.fi 2024-01-13T17:10:19Z Someone newkozlukov@gmail.com 2024-01-22T12:19:30Z workflows: nix-ci: rebuild on flake.lock updates 15bceec2d73d4166340b46b27677c45ac1b4cdad d6bd4d46ddb6926087c11e0f6633ab1c81da58c3 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-22T14:18:43+02:00 GitHub noreply@github.com 2024-01-22T14:18:43+02:00 imatrix : keep intermediate imatrix results (#5077) d6bd4d46ddb6926087c11e0f6633ab1c81da58c3 152d9d05e097e35f1cac21262946d57faec7542a compilade 113953597+compilade@users.noreply.github.com 2024-01-22T06:21:52-05:00 GitHub noreply@github.com 2024-01-22T13:21:52+02:00 llama : support StableLM 2 1.6B (#5052) 152d9d05e097e35f1cac21262946d57faec7542a 66d575c45c5a370d668f9c3283cdf348e2329fa2 Daniel Bevenius daniel.bevenius@gmail.com 2024-01-22T12:11:01+01:00 GitHub noreply@github.com 2024-01-22T13:11:01+02:00 finetune : print sample-start/include-sample-start (#5072) 66d575c45c5a370d668f9c3283cdf348e2329fa2 57744932c64266359ee905518de7e096c0295d8c Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-22T12:43:33+02:00 GitHub noreply@github.com 2024-01-22T12:43:33+02:00 llama : add Q3_K_XS (#5060) 57744932c64266359ee905518de7e096c0295d8c 3466c6ebcf668cac453f891b493ead19283347a8 bobqianic 129547291+bobqianic@users.noreply.github.com 2024-01-22T08:55:05Z GitHub noreply@github.com 2024-01-22T10:55:05+02:00 ci : fix Windows CI by updating Intel SDE version (#5053) 3466c6ebcf668cac453f891b493ead19283347a8 504dc37be8446fb09b1ede70300250ad41be32a2 Shijie 821898965@qq.com 2024-01-22T15:33:19+08:00 GitHub noreply@github.com 2024-01-22T09:33:19+02:00 llama : add more qwen2 models (#5071) 504dc37be8446fb09b1ede70300250ad41be32a2 05490fad7f7f60ff2bed9ad05cd81b44e82ccde3 iSma ismail.senhaji@gmail.com 2024-01-21T22:37:13+01:00 GitHub noreply@github.com 2024-01-21T21:37:13Z Revert LLAMA_NATIVE to OFF in flake.nix (#5066) 05490fad7f7f60ff2bed9ad05cd81b44e82ccde3 6c5629d4d2d15557c38a0e609b30c1a42abad80d kuronekosaiko EvanChanJ@163.com 2024-01-22T00:28:14+08:00 GitHub noreply@github.com 2024-01-21T17:28:14+01:00 add safetensors support to convert-lora-to-ggml.py (#5062) 6c5629d4d2d15557c38a0e609b30c1a42abad80d 7dcbe39d36b76389f6c5cd3b151928472b7e22ff bobqianic 129547291+bobqianic@users.noreply.github.com 2024-01-21T15:17:35Z GitHub noreply@github.com 2024-01-21T10:17:35-05:00 add `#include ` to unicode.h (#5051) 7dcbe39d36b76389f6c5cd3b151928472b7e22ff 726c0fa9a2da976e9c5d5c51e185d9dd453fc9e5 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-21T14:42:44+02:00 GitHub noreply@github.com 2024-01-21T14:42:44+02:00 Add ability to evauate multiple choice tasks (#5047) 726c0fa9a2da976e9c5d5c51e185d9dd453fc9e5 942c0107a7301434c0a5e7da46bc4cf2393aa556 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-21T08:01:20+02:00 GitHub noreply@github.com 2024-01-21T08:01:20+02:00 Slightly faster imatrix (#5050) 942c0107a7301434c0a5e7da46bc4cf2393aa556 b43ebde3b0ccbc42d9dd782b32e2fd8eb35b43b5 Georgi Gerganov ggerganov@gmail.com 2024-01-21T05:17:27+02:00 GitHub noreply@github.com 2024-01-21T03:17:27Z flake.lock: Update (#5054) b43ebde3b0ccbc42d9dd782b32e2fd8eb35b43b5 97c1549808d2742d37584a3c9df28154bdf34417 Jared Van Bortel jared@nomic.ai 2024-01-20T18:14:18-05:00 GitHub noreply@github.com 2024-01-20T18:14:18-05:00 convert : partially revert PR #4818 (#5041) 97c1549808d2742d37584a3c9df28154bdf34417 6df465a91d402370bcba6676b19fad85b06ce7e0 Jared Van Bortel jared@nomic.ai 2024-01-20T10:08:08-05:00 GitHub noreply@github.com 2024-01-20T17:08:08+02:00 perplexity : fix MSVC build after #5020 (#5043) 6df465a91d402370bcba6676b19fad85b06ce7e0 77bc1bbd05f0c31cb45773eb5eb59b9ff2b07e1b slaren slarengh@gmail.com 2024-01-20T16:05:49+01:00 GitHub noreply@github.com 2024-01-20T17:05:49+02:00 llama : run all KQV ops on the CPU with no KV offload (#5049) 77bc1bbd05f0c31cb45773eb5eb59b9ff2b07e1b 48e2b1337257bb77573bf76cfffcff3a5efa8704 Herman Semenov GermanAizek@yandex.ru 2024-01-20T08:11:31Z GitHub noreply@github.com 2024-01-20T10:11:31+02:00 cmake : add support for ccache (#5002) 48e2b1337257bb77573bf76cfffcff3a5efa8704 cca894f16a5eade15afd07b015e4cb3d8658943f adel boussaken netdur@gmail.com 2024-01-20T09:05:43+01:00 GitHub noreply@github.com 2024-01-20T03:05:43-05:00 Add a dart/flutter binding to README.md (#4882) cca894f16a5eade15afd07b015e4cb3d8658943f 381ee195721d8e747ee31a60c0751822b3072f02 Kylin 56434533+KyL0N@users.noreply.github.com 2024-01-20T15:01:46+08:00 GitHub noreply@github.com 2024-01-20T09:01:46+02:00 cuda : fix compile error in jetson platform (#4975) 381ee195721d8e747ee31a60c0751822b3072f02 a5cacb22b2114fd9adf61c00cbb237384d86bced Uzo Nweke uzoechi@gmail.com 2024-01-19T13:20:50-05:00 GitHub noreply@github.com 2024-01-19T20:20:50+02:00 finetune : fix ggml_allocr lifetimes (tmp workaround) (#5033) a5cacb22b2114fd9adf61c00cbb237384d86bced 9b75cb2b3ccbed3df2e14c1202168db3e5145095 Georgi Gerganov ggerganov@gmail.com 2024-01-19T15:24:47+02:00 GitHub noreply@github.com 2024-01-19T15:24:47+02:00 imatrix : add README.md 9b75cb2b3ccbed3df2e14c1202168db3e5145095 de9a147df14e62f54f879d2d15e6c4793107f4fc Shijie 821898965@qq.com 2024-01-19T19:53:13+08:00 GitHub noreply@github.com 2024-01-19T13:53:13+02:00 llama : support upcoming Qwen2 (#5037) de9a147df14e62f54f879d2d15e6c4793107f4fc 7051aacfac0057fa5fac9ea46c55bffc3892d810 Georgi Gerganov ggerganov@gmail.com 2024-01-19T13:52:22+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-19T13:52:22+02:00 py : fix flake8 lint 7051aacfac0057fa5fac9ea46c55bffc3892d810 2b3b999cacc7ad1207c32fbdf3479a19c06e1a34 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-19T11:39:11+02:00 GitHub noreply@github.com 2024-01-19T11:39:11+02:00 winogrande: evaluate log-probs in parallel (#5036) 2b3b999cacc7ad1207c32fbdf3479a19c06e1a34 993fba81807e55d27b570945af8e416d535eced1 chiranko 96988916+chiranko@users.noreply.github.com 2024-01-19T17:07:27+08:00 GitHub noreply@github.com 2024-01-19T11:07:27+02:00 llama : add CodeShell support (#5016) 993fba81807e55d27b570945af8e416d535eced1 8b20858e5e9c44b99b4b31ae9c40b8f20d01d94f Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-19T11:02:39+02:00 GitHub noreply@github.com 2024-01-19T11:02:39+02:00 perplexity: avoid unnecessary alloocations and logit copies (#5035) 8b20858e5e9c44b99b4b31ae9c40b8f20d01d94f 57e2a7a52a819883f40dada8a2edc24ecf48186b Georgi Gerganov ggerganov@gmail.com 2024-01-19T10:45:06+02:00 GitHub noreply@github.com 2024-01-19T10:45:06+02:00 perplexity : faster Winogrande via batching (#5024) 57e2a7a52a819883f40dada8a2edc24ecf48186b 9b6ea4263ab45e02ff905bf7a29dc143ca1facc3 John 78893154+cmp-nct@users.noreply.github.com 2024-01-18T23:12:15+01:00 GitHub noreply@github.com 2024-01-19T00:12:15+02:00 llama : fix falcon arch for tied output embeddings (#4978) 9b6ea4263ab45e02ff905bf7a29dc143ca1facc3 821f0a271e7c9ee737945245dd7abfa22cc9b5b0 Georgi Gerganov ggerganov@gmail.com 2024-01-18T23:36:07+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-18T23:36:07+02:00 cmake : add ggml public headers (#5011) 821f0a271e7c9ee737945245dd7abfa22cc9b5b0 96d7f56d2918ffde1995dbb32392571deb76d7fc Xuan Son Nguyen thichthat@gmail.com 2024-01-18T21:33:05+01:00 GitHub noreply@github.com 2024-01-18T22:33:05+02:00 server : defer tasks when "slot unavailable" (#5018) 96d7f56d2918ffde1995dbb32392571deb76d7fc 2d5419d08ab1131623e6a1d554607b7663435e87 slaren slarengh@gmail.com 2024-01-18T21:12:15+01:00 GitHub noreply@github.com 2024-01-18T21:12:15+01:00 llama : fix mlock with no-mmap with Metal (#5025) 2d5419d08ab1131623e6a1d554607b7663435e87 d391ae9b4919e24624cc963d82162450848beaf4 Georgi Gerganov ggerganov@gmail.com 2024-01-18T21:45:51+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-18T21:45:51+02:00 imatrix : fix assert for src0 non-cont check d391ae9b4919e24624cc963d82162450848beaf4 e9240cdfa06a50c1b5dbafa367cb8cd698e65103 Georgi Gerganov ggerganov@gmail.com 2024-01-18T20:49:00+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-18T20:49:00+02:00 perplexity : fix winogrande N tasks option e9240cdfa06a50c1b5dbafa367cb8cd698e65103 b46757735d30f5c6ed4f20ebeccc684e02d4f3bf Georgi Gerganov ggerganov@gmail.com 2024-01-18T20:45:39+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-18T20:45:39+02:00 scripts : add get-winogrande.sh b46757735d30f5c6ed4f20ebeccc684e02d4f3bf 3e945cc1e9c06d2001031360e4e303e9548fb02c David Sommers 12738+databyte@users.noreply.github.com 2024-01-18T12:20:59-05:00 GitHub noreply@github.com 2024-01-18T19:20:59+02:00 convert.py : fix llama/llama2 conversion due to vocab_size=-1 (#5019) 3e945cc1e9c06d2001031360e4e303e9548fb02c ad19812cda4062c9f154ef16315df41fbe6a770a Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-18T19:18:21+02:00 GitHub noreply@github.com 2024-01-18T19:18:21+02:00 HellaSwag: speed up by parallelizing log-prob evaluation (#5020) ad19812cda4062c9f154ef16315df41fbe6a770a 682986a08eb5cb04865d2e713449f17304d266d8 Georgi Gerganov ggerganov@gmail.com 2024-01-18T15:33:01+02:00 GitHub noreply@github.com 2024-01-18T15:33:01+02:00 perplexity : faster HellaSwag via batching (#5017) 682986a08eb5cb04865d2e713449f17304d266d8 dcad445d0c83ad49bca1b58cf9c139cfcebee5d4 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-18T13:46:27+02:00 GitHub noreply@github.com 2024-01-18T13:46:27+02:00 Add Winogrande evaluation (#5015) dcad445d0c83ad49bca1b58cf9c139cfcebee5d4 1e605f4102c7ea8dc0dff82f5eaa6a71973d549f Georgi Gerganov ggerganov@gmail.com 2024-01-18T11:44:49+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-18T11:44:49+02:00 scritps : add helper script to get hellaswag data in txt format 1e605f4102c7ea8dc0dff82f5eaa6a71973d549f 6b6916b215251e09bd57cdbf870dc8a73345edc2 Paul Tsochantaris ptsochantaris@icloud.com 2024-01-18T08:47:24Z GitHub noreply@github.com 2024-01-18T10:47:24+02:00 metal : fix memory leak, dangling pointer and unused autorel (#5007) 6b6916b215251e09bd57cdbf870dc8a73345edc2 38566680cdfe982a495562332c25b9227de9cf8d Georgi Gerganov ggerganov@gmail.com 2024-01-17T20:54:50+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-17T20:54:50+02:00 sync : ggml 38566680cdfe982a495562332c25b9227de9cf8d ba69bbc84ced580fe4fdb0713ca2d95634325b7a Georgi Gerganov ggerganov@gmail.com 2024-01-17T18:54:56+02:00 GitHub noreply@github.com 2024-01-17T18:54:56+02:00 ggml : add IQ2 to test-backend-ops + refactoring (#4990) ba69bbc84ced580fe4fdb0713ca2d95634325b7a 44a1a4a41a4c0b03afaa7d9e06bcbc7cf95aa1e6 Georgi Gerganov ggerganov@gmail.com 2024-01-17T18:46:30+02:00 GitHub noreply@github.com 2024-01-17T18:46:30+02:00 imatrix : offload to GPU support (#4957) 44a1a4a41a4c0b03afaa7d9e06bcbc7cf95aa1e6 c918fe8dca8fa1c4602427e0a4b88e20046f6c34 Georgi Gerganov ggerganov@gmail.com 2024-01-17T18:39:41+02:00 GitHub noreply@github.com 2024-01-17T18:39:41+02:00 backend : add eval callback (#4935) c918fe8dca8fa1c4602427e0a4b88e20046f6c34 0f83e727af0a7cadf90b7ecc1f8e35de1d0880bc Georgi Gerganov ggerganov@gmail.com 2024-01-17T18:38:39+02:00 GitHub noreply@github.com 2024-01-17T18:38:39+02:00 metal : create autorelease pool during library build (#4970) 0f83e727af0a7cadf90b7ecc1f8e35de1d0880bc 4f4bf35f46600441dec2f941e667291eeb9a18d8 Georgi Gerganov ggerganov@gmail.com 2024-01-17T18:37:36+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-17T18:37:36+02:00 py : fix whitespace 4f4bf35f46600441dec2f941e667291eeb9a18d8 2b3a665d3917edf393761a24c4835447894df74a Georgi Gerganov ggerganov@gmail.com 2024-01-17T15:45:03+02:00 GitHub noreply@github.com 2024-01-17T15:45:03+02:00 py : fix missing added_tokens_dict for SPM and BPE vocabs (#4971) 2b3a665d3917edf393761a24c4835447894df74a 75632936659772d5b2ce54b0b65319fecbaac2e6 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-17T12:36:37+02:00 GitHub noreply@github.com 2024-01-17T12:36:37+02:00 llama : use Q4_K for attn_v for Q2_K_S when n_gqa >= 4 (#4996) 75632936659772d5b2ce54b0b65319fecbaac2e6 f46c0c1b0ea0bc67e24e4bf026a7e898c1af22a9 Paul Tsochantaris ptsochantaris@icloud.com 2024-01-17T08:07:24Z GitHub noreply@github.com 2024-01-17T10:07:24+02:00 metal : remove unnecessary nil check (#4986) f46c0c1b0ea0bc67e24e4bf026a7e898c1af22a9 5c999609013a30c06e6fd28be8db5c2074bcc196 David Renshaw dwrenshaw@gmail.com 2024-01-17T02:17:50-05:00 GitHub noreply@github.com 2024-01-17T09:17:50+02:00 llama : fix copy/paste error in llama_sampling_params comment (#4994) 5c999609013a30c06e6fd28be8db5c2074bcc196 bee938da74c33f42242c3a1058ac0a0a6eeef531 Georgi Gerganov ggerganov@gmail.com 2024-01-16T20:59:31+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-16T20:59:31+02:00 py : remove unnecessary hasattr (#4903) bee938da74c33f42242c3a1058ac0a0a6eeef531 cec8a4847062fbd76253e3b085683f39d91e80d3 Philip Taron philip.taron@gmail.com 2024-01-16T09:56:21-08:00 GitHub noreply@github.com 2024-01-16T09:56:21-08:00 nix: remove nixConfig from flake.nix (#4984) cec8a4847062fbd76253e3b085683f39d91e80d3 334a835a1ccc8106a5fa355683a965efb1bfa24b Daniel Bevenius daniel.bevenius@gmail.com 2024-01-16T18:54:24+01:00 GitHub noreply@github.com 2024-01-16T19:54:24+02:00 finetune : add training data file to log message (#4979) 334a835a1ccc8106a5fa355683a965efb1bfa24b 4feb4b33eeb1756e46084a4db9230b279af1a480 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-16T19:51:26+02:00 GitHub noreply@github.com 2024-01-16T19:51:26+02:00 ggml : importance matrix support for legacy quants (#4969) 4feb4b33eeb1756e46084a4db9230b279af1a480 959ef0c0df725c013c7f712eaa7790b8e38a8e20 Maximilian Winter maximilian.winter.91@gmail.com 2024-01-16T18:41:42+01:00 GitHub noreply@github.com 2024-01-16T19:41:42+02:00 examples : add complete parallel function calling example (#4974) 959ef0c0df725c013c7f712eaa7790b8e38a8e20 c37b3474e61d609d43cccc3bde5d559e80e4f5d1 Georgi Gerganov ggerganov@gmail.com 2024-01-16T19:34:54+02:00 GitHub noreply@github.com 2024-01-16T19:34:54+02:00 perplexity : fix kv cache handling for hellaswag (#4981) c37b3474e61d609d43cccc3bde5d559e80e4f5d1 158f8c9e21302114bac3c646f80ea85b52ffa0bd Georgi Gerganov ggerganov@gmail.com 2024-01-16T19:13:54+02:00 GitHub noreply@github.com 2024-01-16T09:13:54-08:00 flake.lock: update flake-parts, flake-parts/nixpkgs-lib, and nixpkgs (#4920) 158f8c9e21302114bac3c646f80ea85b52ffa0bd 862f5e41ab1fdf12d6f59455aad3f5dd8258f805 Paul Tsochantaris ptsochantaris@icloud.com 2024-01-16T17:05:19Z GitHub noreply@github.com 2024-01-16T19:05:19+02:00 metal : localized logic in `ggml_metal_graph_compute` (#4924) 862f5e41ab1fdf12d6f59455aad3f5dd8258f805 3a48d558a69c88ac17efcaa5900cd9eb19596ac4 Neuman Vong neuman.vong@gmail.com 2024-01-17T00:47:34+11:00 GitHub noreply@github.com 2024-01-16T15:47:34+02:00 android : introduce starter project example (#4926) 3a48d558a69c88ac17efcaa5900cd9eb19596ac4 7c8d3abd1a17c28fc56b1a4814bc4b29f91d7454 Alex Azarov alex@azarov.by 2024-01-16T14:41:27+01:00 GitHub noreply@github.com 2024-01-16T15:41:27+02:00 metal : replace loop of dispatch_async with dispatch_apply (#4934) 7c8d3abd1a17c28fc56b1a4814bc4b29f91d7454 122ed4840cc6d209df6043e027f9f8a03aee01da Alex Azarov alex@azarov.by 2024-01-16T14:33:02+01:00 GitHub noreply@github.com 2024-01-16T15:33:02+02:00 metal : log `recommendedMaxWorkingSetSize` on iOS 16+ (#4936) 122ed4840cc6d209df6043e027f9f8a03aee01da a0b3ac8c48b66206b9c5921ce57bd5c0ea6557c3 Maximilian Winter maximilian.winter.91@gmail.com 2024-01-16T13:10:48+01:00 GitHub noreply@github.com 2024-01-16T14:10:48+02:00 examples : fix and improv docs for the grammar generator (#4909) a0b3ac8c48b66206b9c5921ce57bd5c0ea6557c3 d75c232e1da56f19ac4d2530dadbe0ab3a11fde5 Justine Tunney jtunney@gmail.com 2024-01-16T03:16:33-08:00 GitHub noreply@github.com 2024-01-16T13:16:33+02:00 ggml : introduce GGML_CALL function annotation (#4850) d75c232e1da56f19ac4d2530dadbe0ab3a11fde5 e0324285a569d0583cf2f4a07a2402221ee25f58 Daniel Bevenius daniel.bevenius@gmail.com 2024-01-16T12:14:19+01:00 GitHub noreply@github.com 2024-01-16T13:14:19+02:00 finetune : use LLAMA_FILE_MAGIC_GGLA (#4961) e0324285a569d0583cf2f4a07a2402221ee25f58 3e5ca7931c68152e4ec18d126e9c832dd84914c8 stduhpf stephduh@live.fr 2024-01-16T12:04:32+01:00 GitHub noreply@github.com 2024-01-16T13:04:32+02:00 speculative : threading options (#4959) 3e5ca7931c68152e4ec18d126e9c832dd84914c8 4483396751c79dea540808b9cb9238245d06da2b ngc92 7938269+ngc92@users.noreply.github.com 2024-01-15T20:40:48+02:00 GitHub noreply@github.com 2024-01-15T19:40:48+01:00 pass cpu-architecture arguments only to host code (C;C++) (#4943) 4483396751c79dea540808b9cb9238245d06da2b d9aa4ffa6e0296d42f1f676dd85de97c8491eb73 David Friehs david@friehs.info 2024-01-15T14:06:52+01:00 GitHub noreply@github.com 2024-01-15T15:06:52+02:00 llama : apply classifier-free guidance to logits directly (#4951) d9aa4ffa6e0296d42f1f676dd85de97c8491eb73 ddb008d845cd50bb090bf051f570130524042936 Victor Z. Peng ziliangdotme@gmail.com 2024-01-15T04:41:46-08:00 GitHub noreply@github.com 2024-01-15T14:41:46+02:00 awq-py : fix typo in awq-py/README.md (#4947) ddb008d845cd50bb090bf051f570130524042936 2faaef39799c97a53bec3898141478700da25757 Georgi Gerganov ggerganov@gmail.com 2024-01-15T13:27:00+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-15T13:27:00+02:00 cuda : fix dequantize kernel names (#4938) 2faaef39799c97a53bec3898141478700da25757 4a3156de2fac9a8ee4279de7804d4e352dcfe121 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-15T10:09:38+02:00 GitHub noreply@github.com 2024-01-15T10:09:38+02:00 llama : check for 256 divisibility for IQ2_XS, IQ2_XXS (#4950) 4a3156de2fac9a8ee4279de7804d4e352dcfe121 a836c8f534ab789b02da149fbdaf7735500bff74 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-15T07:48:06+02:00 GitHub noreply@github.com 2024-01-15T07:48:06+02:00 CUDA: faster dequantize kernels for Q4_0 and Q4_1 (#4938) a836c8f534ab789b02da149fbdaf7735500bff74 467a882fd2e5b6172897b49aa45aa29bd3f27685 David Pflug david@pflug.email 2024-01-14T10:46:00-05:00 GitHub noreply@github.com 2024-01-14T17:46:00+02:00 llama : fix missing quotes (#4937) 467a882fd2e5b6172897b49aa45aa29bd3f27685 bb0c1392479398f9aba86d9ec98db0b95ede6e6d Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-14T16:21:12+02:00 GitHub noreply@github.com 2024-01-14T16:21:12+02:00 Add ability to use importance matrix for all k-quants (#4930) bb0c1392479398f9aba86d9ec98db0b95ede6e6d 9408cfdad6b1c090a7e1419d4434edc260b7e47e Georgi Gerganov ggerganov@gmail.com 2024-01-14T13:26:53+02:00 GitHub noreply@github.com 2024-01-14T13:26:53+02:00 llama : check LLAMA_TRACE env for extra logging (#4929) 9408cfdad6b1c090a7e1419d4434edc260b7e47e 03c526749041c863b0cd842b26b8907e1ea0e0b1 Georgi Gerganov ggerganov@gmail.com 2024-01-14T11:08:09+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-14T11:08:41+02:00 scripts : sync-ggml-am.sh option to skip commits 03c526749041c863b0cd842b26b8907e1ea0e0b1 a128c38de862431f1aae9ccc40b792fbc1b8b682 Georgi Gerganov ggerganov@gmail.com 2024-01-14T11:03:19+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-14T11:03:19+02:00 llama : use LLAMA_LOG_ macros for logging a128c38de862431f1aae9ccc40b792fbc1b8b682 5f5fe1bd608fa2ed42af97b5f2ea31be6625fc48 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-14T10:53:39+02:00 GitHub noreply@github.com 2024-01-14T10:53:39+02:00 Fix ffn_down quantization mix for MoE models (#4927) 5f5fe1bd608fa2ed42af97b5f2ea31be6625fc48 ac32902a87147f78d63c931aa8a23dee762660e7 Alex Azarov alex@azarov.by 2024-01-14T09:44:39+01:00 GitHub noreply@github.com 2024-01-14T10:44:39+02:00 metal : correctly set SIMD support flags on iOS (#4923) ac32902a87147f78d63c931aa8a23dee762660e7 147b17ac94a24d524e367cda26a9ff6245689f34 Karthik Kumar Viswanathan 195178+guilt@users.noreply.github.com 2024-01-14T00:41:44-08:00 GitHub noreply@github.com 2024-01-14T10:41:44+02:00 llama : support WinXP build with MinGW 8.1.0 (#3419) 147b17ac94a24d524e367cda26a9ff6245689f34 807179ec583dcb882f97d9704577c06beb2c5ec9 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-14T09:45:56+02:00 GitHub noreply@github.com 2024-01-14T09:45:56+02:00 2-bit quantizations (#4897) 807179ec583dcb882f97d9704577c06beb2c5ec9 76484fbfd355df388f71d6edaa98e1692a74de7e Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-14T09:44:30+02:00 GitHub noreply@github.com 2024-01-14T09:44:30+02:00 Make Q3_K_S be the same as olf Q3_K_L for Mixtral-8x7B (#4906) 76484fbfd355df388f71d6edaa98e1692a74de7e c71d608ce7a1584bf5072f197919dd24f3a6163f Georgi Gerganov ggerganov@gmail.com 2024-01-14T00:14:46+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-14T00:14:46+02:00 sync : ggml c71d608ce7a1584bf5072f197919dd24f3a6163f 4be5ef556de830c5c4f6e45c05ef4427823fe607 Johannes Gäßler johannesg@5d6.de 2024-01-13T21:41:37+01:00 GitHub noreply@github.com 2024-01-13T21:41:37+01:00 ggml: cache sin/cos for RoPE (#4908) 4be5ef556de830c5c4f6e45c05ef4427823fe607 0ea069b87bd296c556824e57455433b6c0357340 Georgi Gerganov ggerganov@gmail.com 2024-01-13T20:45:45+02:00 GitHub noreply@github.com 2024-01-13T20:45:45+02:00 metal : remove old API (#4919) 0ea069b87bd296c556824e57455433b6c0357340 f172de03f11465dc6c5a0fc3a22f8ec254c6832c Georgi Gerganov ggerganov@gmail.com 2024-01-13T19:31:26+02:00 GitHub noreply@github.com 2024-01-13T19:31:26+02:00 server : fix prompt caching with system prompt (#4914) f172de03f11465dc6c5a0fc3a22f8ec254c6832c 2d57de525541247132e354f561ff48775fba5d85 Georgi Gerganov ggerganov@gmail.com 2024-01-13T18:47:38+02:00 GitHub noreply@github.com 2024-01-13T18:47:38+02:00 llama : fix detokenization of non-special added-tokens (#4916) 2d57de525541247132e354f561ff48775fba5d85 df845cc982e7e2ea7b9900e29d55b15338faa78d Georgi Gerganov ggerganov@gmail.com 2024-01-13T18:46:37+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-13T18:46:37+02:00 metal : disable log for loaded kernels (#4794) df845cc982e7e2ea7b9900e29d55b15338faa78d 6b48ed089377330cdb362970a51c1c89b6d857a8 David Friehs david@friehs.info 2024-01-13T17:29:43+01:00 GitHub noreply@github.com 2024-01-13T18:29:43+02:00 llama : minimize size used for state save/load (#4820) 6b48ed089377330cdb362970a51c1c89b6d857a8 722d33f34ec74c6f7046109f936d0928ffe171bc Someone sergei.kozlukov@aalto.fi 2024-01-13T16:29:16Z GitHub noreply@github.com 2024-01-13T16:29:16Z workflows: unbreak nix-build-aarch64, and split it out (#4915) 722d33f34ec74c6f7046109f936d0928ffe171bc c30b1ef39aeba497a943416d2897d69fee055b96 Yann Follet 131855179+YannFollet@users.noreply.github.com 2024-01-14T00:09:08+08:00 GitHub noreply@github.com 2024-01-13T18:09:08+02:00 main : add parameter --no-display-prompt (#4541) c30b1ef39aeba497a943416d2897d69fee055b96 b38b5e93ae31019e87f692b69d27124eae6aac02 texmex76 40733439+texmex76@users.noreply.github.com 2024-01-13T17:06:20+01:00 GitHub noreply@github.com 2024-01-13T18:06:20+02:00 gguf : fix potential infinite for-loop (#4600) b38b5e93ae31019e87f692b69d27124eae6aac02 7dc78764e2ff86512e6e31cb0fcb8087df4b4708 Georgi Gerganov ggerganov@gmail.com 2024-01-13T18:03:45+02:00 GitHub noreply@github.com 2024-01-13T18:03:45+02:00 metal : refactor kernel loading code (#4794) 7dc78764e2ff86512e6e31cb0fcb8087df4b4708 356327feb3f66980ab687040495d722696d98970 Johannes Gäßler johannesg@5d6.de 2024-01-13T15:52:53+01:00 GitHub noreply@github.com 2024-01-13T15:52:53+01:00 compare-llama-bench: tweak output format (#4910) 356327feb3f66980ab687040495d722696d98970 ee8243adaa9a9f51ff449213383874e49efe368f Ziad Ben Hadj-Alouane zied.benhadjalouane@gmail.com 2024-01-13T09:20:46-05:00 GitHub noreply@github.com 2024-01-13T16:20:46+02:00 server : fix deadlock that occurs in multi-prompt scenarios (#4905) ee8243adaa9a9f51ff449213383874e49efe368f 15ebe59210e7fd9817ff67f51fa1a5ee2d004294 makomk makosoft@googlemail.com 2024-01-13T14:16:11Z GitHub noreply@github.com 2024-01-13T16:16:11+02:00 server : fix crash with multimodal models without BOS token (#4904) 15ebe59210e7fd9817ff67f51fa1a5ee2d004294 de473f5f8e19ba5e659cdf5af65fb9251dce16c5 Georgi Gerganov ggerganov@gmail.com 2024-01-13T13:44:37+02:00 GitHub noreply@github.com 2024-01-13T13:44:37+02:00 convert : update phi-2 to latest HF repo (#4903) de473f5f8e19ba5e659cdf5af65fb9251dce16c5 f238461236f4e0e18cac1a554af23c7deadc9b01 Georgi Gerganov ggerganov@gmail.com 2024-01-12T22:02:43+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-12T22:02:43+02:00 sync : ggml f238461236f4e0e18cac1a554af23c7deadc9b01 fa5c1fb44a2724292da545d6b7cf2a1ac0e0b989 Georgi Gerganov ggerganov@gmail.com 2024-01-12T14:02:30+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-12T22:02:11+02:00 ggml : fix 32-bit ARM compat for IQ2_XS (whisper/1758) fa5c1fb44a2724292da545d6b7cf2a1ac0e0b989 52ee4540c0f2e11d52c839db6eb51d014ce060e1 slaren slarengh@gmail.com 2024-01-12T20:38:34+01:00 Georgi Gerganov ggerganov@gmail.com 2024-01-12T22:02:11+02:00 backend_sched : fix assignments 52ee4540c0f2e11d52c839db6eb51d014ce060e1 3fe81781e3bf98b8e44946240a19f3a6ad08a11a Maximilian Winter maximilian.winter.91@gmail.com 2024-01-12T20:46:45+01:00 GitHub noreply@github.com 2024-01-12T21:46:45+02:00 examples : add pydantic models to GBNF grammar generator (#4883) 3fe81781e3bf98b8e44946240a19f3a6ad08a11a e7e4df031b9e29d4b55a4e0b0295187f6b213db1 Johannes Gäßler johannesg@5d6.de 2024-01-12T20:38:54+01:00 GitHub noreply@github.com 2024-01-12T20:38:54+01:00 CUDA: faster q8_0 -> f16 dequantization (#4895) e7e4df031b9e29d4b55a4e0b0295187f6b213db1 584d674be622fbf1578694ada6e62eebedbfd377 slaren slarengh@gmail.com 2024-01-12T20:07:38+01:00 GitHub noreply@github.com 2024-01-12T20:07:38+01:00 llama : ggml-backend integration (#4766) 584d674be622fbf1578694ada6e62eebedbfd377 930f907d3ece1eb5b0a1ec5e209983a66dcbfa68 Georgi Gerganov ggerganov@gmail.com 2024-01-12T20:54:12+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-12T20:54:12+02:00 llama : remove redundant assert for StableLM (#4901) 930f907d3ece1eb5b0a1ec5e209983a66dcbfa68 e790eef21ce659f5c16d59f8a5c8dcf6cde0692a Daniel Bevenius daniel.bevenius@gmail.com 2024-01-12T18:54:53+01:00 GitHub noreply@github.com 2024-01-12T19:54:53+02:00 export-lora : use LLAMA_FILE_MAGIC_GGLA (#4894) e790eef21ce659f5c16d59f8a5c8dcf6cde0692a 5537d9d36bfdb4379555431f574d3d78ce6e7955 Zay 95888118+isaiahbjork@users.noreply.github.com 2024-01-12T05:48:00-07:00 GitHub noreply@github.com 2024-01-12T14:48:00+02:00 llama.swiftui : update models layout (#4826) 5537d9d36bfdb4379555431f574d3d78ce6e7955 1b280c9fffd682b6924010a4437f0275f2921fa9 Georgi Gerganov ggerganov@gmail.com 2024-01-12T14:33:21+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-12T14:33:21+02:00 gitignore : imatrix 1b280c9fffd682b6924010a4437f0275f2921fa9 3cabe80630c7eeb57713cd02249053a8cf6894fa Johannes Gäßler johannesg@5d6.de 2024-01-12T12:30:41+01:00 GitHub noreply@github.com 2024-01-12T12:30:41+01:00 CUDA: fix softmax compile for old CUDA versions (#4862) 3cabe80630c7eeb57713cd02249053a8cf6894fa 4315a94366708828f949f9db89d2a8d99b634459 Georgi Gerganov ggerganov@gmail.com 2024-01-12T13:10:19+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-12T13:11:15+02:00 llama : fix typo "imp_embd" -> "inp_embd" 4315a94366708828f949f9db89d2a8d99b634459 2d00741e12c5db4a33dfccd1125f5de4adec9a5b howlger eclipse@voormann.de 2024-01-12T12:05:32+01:00 GitHub noreply@github.com 2024-01-12T13:05:32+02:00 common : streamline the formatting of help (#4890) 2d00741e12c5db4a33dfccd1125f5de4adec9a5b f445c0e68cf8e1faca0b2aa8dfb9d48231cec301 Georgi Gerganov ggerganov@gmail.com 2024-01-12T13:03:38+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-12T13:03:38+02:00 py : fix lint (#4889) f445c0e68cf8e1faca0b2aa8dfb9d48231cec301 326b418b59b6d48d854c4461a2303e8ac0a311e6 Georgi Gerganov ggerganov@gmail.com 2024-01-12T13:01:56+02:00 GitHub noreply@github.com 2024-01-12T13:01:56+02:00 llama : fix llm_build_k_shift to use correct n_rot (#4889) 326b418b59b6d48d854c4461a2303e8ac0a311e6 1d118386fea031f01550f8cd47a5c86296e5333f Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-12T06:59:57+01:00 GitHub noreply@github.com 2024-01-12T06:59:57+01:00 Importance Matrix calculation (#4861) 1d118386fea031f01550f8cd47a5c86296e5333f 7edefbd79cc6dea96640edc54c6b94b2b2496d8b Georgi Gerganov ggerganov@gmail.com 2024-01-11T23:23:49+02:00 GitHub noreply@github.com 2024-01-11T23:23:49+02:00 server : fix infill when prompt is empty (#4833) 7edefbd79cc6dea96640edc54c6b94b2b2496d8b 3ca63b4538dfc78aaec88cd2c3e3f8417c1924e3 Georgi Gerganov ggerganov@gmail.com 2024-01-11T22:46:26+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T22:46:26+02:00 main : better name for variable n_print (#4874) 3ca63b4538dfc78aaec88cd2c3e3f8417c1924e3 b0377875488b33f7114138687d828da1de61775d Georgi Gerganov ggerganov@gmail.com 2024-01-11T22:43:05+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T22:43:05+02:00 main : disable token count by default (#4874) b0377875488b33f7114138687d828da1de61775d 469e75d0a35b08de549a4fd87f082ca7a8a539ba Georgi Gerganov ggerganov@gmail.com 2024-01-11T21:58:28+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T21:58:28+02:00 swift : track ggml release branch (#4867) 469e75d0a35b08de549a4fd87f082ca7a8a539ba 49662cbed3e95f5976c070b85b9fd53fd577038d Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-11T20:43:15+01:00 GitHub noreply@github.com 2024-01-11T21:43:15+02:00 llama : restore intended k-quants mixes for MoE models (#4872) 49662cbed3e95f5976c070b85b9fd53fd577038d 3ba5b8ca8e6181a5c712c5b77595a29f1d3e2b97 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-11T20:39:39+01:00 GitHub noreply@github.com 2024-01-11T21:39:39+02:00 ggml : SOTA 2-bit quants (add IQ2_XS) (#4856) 3ba5b8ca8e6181a5c712c5b77595a29f1d3e2b97 4330bd83feb39683de4bd7a34cfcf672ff8ac3e4 Georgi Gerganov ggerganov@gmail.com 2024-01-11T21:31:31+02:00 GitHub noreply@github.com 2024-01-11T21:31:31+02:00 swift : pin ggml commit + remove ggml.h from spm-headers (#4878) 4330bd83feb39683de4bd7a34cfcf672ff8ac3e4 27379455c38cb13f24de92dbd6fcdd04eeb1b9d9 Laura Tijntje_7@msn.com 2024-01-11T19:02:48+01:00 GitHub noreply@github.com 2024-01-11T20:02:48+02:00 server : implement credentialed CORS (#4514) 27379455c38cb13f24de92dbd6fcdd04eeb1b9d9 eab67950068e4b125007d027232c47d2a5831cd0 Michael Coppola m18coppola@gmail.com 2024-01-11T12:51:17-05:00 GitHub noreply@github.com 2024-01-11T19:51:17+02:00 server : support for multiple api keys (#4864) eab67950068e4b125007d027232c47d2a5831cd0 d8d90aa343c22fe01429d3540e47ded87e9dcb9d Behnam M 58621210+ibehnam@users.noreply.github.com 2024-01-11T12:41:39-05:00 GitHub noreply@github.com 2024-01-11T19:41:39+02:00 server : add `LOG_INFO` when model is successfully loaded (#4881) d8d90aa343c22fe01429d3540e47ded87e9dcb9d 43f76bf1c362c067fce46bb8dcda0b64af8a9533 Someone sergei.kozlukov@aalto.fi 2024-01-11T17:22:34Z GitHub noreply@github.com 2024-01-11T17:22:34Z ci: nix-flake-update: new token with pr permissions (#4879) 43f76bf1c362c067fce46bb8dcda0b64af8a9533 2f043328e3116724d15b915b5c6078e2df860a69 pudepiedj pudepiedj@gmail.com 2024-01-11T16:14:52Z GitHub noreply@github.com 2024-01-11T18:14:52+02:00 main : print total token count and tokens consumed so far (#4874) 2f043328e3116724d15b915b5c6078e2df860a69 2a7c94db5fb67b2f8882d2d16a11bf5d8d12d397 Isaac McFadyen isaac@imcf.me 2024-01-11T09:33:26-05:00 GitHub noreply@github.com 2024-01-11T16:33:26+02:00 server : fix typo in model name (#4876) 2a7c94db5fb67b2f8882d2d16a11bf5d8d12d397 64802ec00d6383784a9dacf616095eaced16c3c3 Paul Tsochantaris ptsochantaris@icloud.com 2024-01-11T14:31:52Z GitHub noreply@github.com 2024-01-11T16:31:52+02:00 metal : put encoder debug group behind a define (#4873) 64802ec00d6383784a9dacf616095eaced16c3c3 3267c2abc72e34608224408ace3c048831050f97 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:39:08+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:39:08+02:00 sync : ggml 3267c2abc72e34608224408ace3c048831050f97 f85a973aa139ae6f37e8b8e1966f1d278b5e0372 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:34:59+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:39:05+02:00 metal : fix deprecation warning (ggml/690) f85a973aa139ae6f37e8b8e1966f1d278b5e0372 5362e43962e84d61e20b91f34991d7ccaef4a7d5 Timothy Cronin 40186632+4imothy@users.noreply.github.com 2024-01-11T02:27:48-05:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:39:05+02:00 ggml : remove ggml_cpy_inplace and ggml_cont_inplace (ggml/693) 5362e43962e84d61e20b91f34991d7ccaef4a7d5 e739de790921e6abbc8c70398303cacd74913f61 Jack Mousseau jmousseau@users.noreply.github.com 2024-01-10T06:19:19-08:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:39:05+02:00 metal : wrap each operation in debug group (ggml/690) e739de790921e6abbc8c70398303cacd74913f61 c910e3c28a1caee8cb1398143d582dd9ab697e68 leejet leejet714@gmail.com 2024-01-10T21:13:42+08:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:39:05+02:00 ggml : change GGML_MAX_NAME at compile time (ggml/682) c910e3c28a1caee8cb1398143d582dd9ab697e68 f34432ca1e0b288129390c1db8296a82aaf1e632 Halalaluyafail3 55773281+Halalaluyafail3@users.noreply.github.com 2024-01-09T11:16:37-05:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:39:05+02:00 Fix execlp call (ggml/689) f34432ca1e0b288129390c1db8296a82aaf1e632 7a9f75c38b5e62fe27b8a5a3ed823b4a3714024b Erik Scholz Green-Sky@users.noreply.github.com 2024-01-05T16:00:00+01:00 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:39:05+02:00 fix : cuda order of synchronization when setting a buffer (ggml/679) 7a9f75c38b5e62fe27b8a5a3ed823b4a3714024b 5c1980d8d4c4e0c0af77359f81cc44d90b3f250b Behnam M 58621210+ibehnam@users.noreply.github.com 2024-01-11T02:12:05-05:00 GitHub noreply@github.com 2024-01-11T09:12:05+02:00 server : update readme to document the new `/health` endpoint (#4866) 5c1980d8d4c4e0c0af77359f81cc44d90b3f250b cd108e641dbdedd8c5641c4cec1762f751f38136 Georgi Gerganov ggerganov@gmail.com 2024-01-11T09:10:34+02:00 GitHub noreply@github.com 2024-01-11T09:10:34+02:00 server : fix build + rename enums (#4870) cd108e641dbdedd8c5641c4cec1762f751f38136 57d016ba2d46a6e22517a31a75cebb48f9e234b6 Behnam M 58621210+ibehnam@users.noreply.github.com 2024-01-10T14:56:05-05:00 GitHub noreply@github.com 2024-01-10T21:56:05+02:00 server : add a `/health` endpoint (#4860) 57d016ba2d46a6e22517a31a75cebb48f9e234b6 329ff615699d32f596d4ebf8baba654c30064e0d Brian mofosyne@gmail.com 2024-01-11T01:09:53+11:00 GitHub noreply@github.com 2024-01-10T16:09:53+02:00 llama : add additional suffixes for model params (#4834) 329ff615699d32f596d4ebf8baba654c30064e0d d34633d8db6c2e400355de4862cd699154ecc73f Austin 77757836+teleprint-me@users.noreply.github.com 2024-01-10T08:39:09-05:00 GitHub noreply@github.com 2024-01-10T15:39:09+02:00 llama : recognize 1B phi models (#4847) d34633d8db6c2e400355de4862cd699154ecc73f 4f56458d34cb13dcbf69aca650e9bf77d5497e6f John 78893154+cmp-nct@users.noreply.github.com 2024-01-10T14:37:09+01:00 GitHub noreply@github.com 2024-01-10T15:37:09+02:00 clip : support more quantization types (#4846) 4f56458d34cb13dcbf69aca650e9bf77d5497e6f 6efb8eb30e7025b168f3fda3ff83b9b386428ad6 Johannes Gäßler johannesg@5d6.de 2024-01-10T01:04:33+01:00 GitHub noreply@github.com 2024-01-10T01:04:33+01:00 Python script to compare commits with llama-bench (#4844) 6efb8eb30e7025b168f3fda3ff83b9b386428ad6 36e5a08b203542dca53cca4eaf172c5dc4bbc991 Austin 77757836+teleprint-me@users.noreply.github.com 2024-01-09T13:46:46-05:00 GitHub noreply@github.com 2024-01-09T20:46:46+02:00 convert.py : fix vanilla LLaMA model conversion (#4818) 36e5a08b203542dca53cca4eaf172c5dc4bbc991 4dccb38d9abab7f9f2d1f9a6977df4185d490132 Justine Tunney jtunney@gmail.com 2024-01-09T09:59:14-08:00 GitHub noreply@github.com 2024-01-09T19:59:14+02:00 llava-cli : don't crash if --image flag is invalid (#4835) 4dccb38d9abab7f9f2d1f9a6977df4185d490132 9a818f7c42761984ac99e08e613cc20634f8410e Georgi Gerganov ggerganov@gmail.com 2024-01-09T19:37:08+02:00 GitHub noreply@github.com 2024-01-09T19:37:08+02:00 metal : improve dequantize precision to match CPU (#4836) 9a818f7c42761984ac99e08e613cc20634f8410e 18adb4e9bb340b7b4565d8b6715b4449283e7641 Georgi Gerganov ggerganov@gmail.com 2024-01-09T19:20:45+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-09T19:21:13+02:00 scripts : improve get-pg.sh (#4838) 18adb4e9bb340b7b4565d8b6715b4449283e7641 d9653894dffbfd3a58616f31b0967b34faf6f611 iohub rickyang.pro@gmail.com 2024-01-10T00:45:54+08:00 GitHub noreply@github.com 2024-01-09T18:45:54+02:00 readme : add 3rd party collama reference to UI list (#4840) d9653894dffbfd3a58616f31b0967b34faf6f611 128de3585b0f58b1e562733448fc00109f23a95d Georgi Gerganov ggerganov@gmail.com 2024-01-09T16:23:05+02:00 GitHub noreply@github.com 2024-01-09T16:23:05+02:00 scripts : script to get Paul Graham essays in txt format (#4838) 128de3585b0f58b1e562733448fc00109f23a95d 8c5833031857c9e9ada61948bae894ab9c785f86 Behnam M 58621210+ibehnam@users.noreply.github.com 2024-01-09T05:02:05-05:00 GitHub noreply@github.com 2024-01-09T12:02:05+02:00 server : update readme about token probs (#4777) 8c5833031857c9e9ada61948bae894ab9c785f86 18c2e1752c3b387689e9e73d7d8a1a3b1511ce23 Zsapi martin1.zsapka@gmail.com 2024-01-09T10:12:43+01:00 GitHub noreply@github.com 2024-01-09T11:12:43+02:00 server : add api-key flag to documentation (#4832) 18c2e1752c3b387689e9e73d7d8a1a3b1511ce23 8f900abfc09851e281bc9027e0ab2f16bf079b29 Georgi Gerganov ggerganov@gmail.com 2024-01-09T10:42:06+02:00 GitHub noreply@github.com 2024-01-09T10:42:06+02:00 ggml : fix vld1q_s8_x4 32-bit compat (#4828) 8f900abfc09851e281bc9027e0ab2f16bf079b29 1fc2f265ff9377a37fd2c61eae9cd813a3491bea Johannes Gäßler johannesg@5d6.de 2024-01-09T08:58:55+01:00 GitHub noreply@github.com 2024-01-09T08:58:55+01:00 CUDA: faster softmax via shared memory + fp16 math (#4742) 1fc2f265ff9377a37fd2c61eae9cd813a3491bea a9a8c5de3d2028701c239d821b220214fcaefbf1 howlger eclipse@voormann.de 2024-01-08T20:05:53+01:00 GitHub noreply@github.com 2024-01-08T21:05:53+02:00 common : fix the short form of `--grp-attn-w`, not `-gat` (#4825) a9a8c5de3d2028701c239d821b220214fcaefbf1 dd5ae06405c5565b99889bdb3f168f4351252cfb Georgi Gerganov ggerganov@gmail.com 2024-01-08T20:25:17+02:00 GitHub noreply@github.com 2024-01-08T20:25:17+02:00 readme : add link to SOTA models dd5ae06405c5565b99889bdb3f168f4351252cfb 668b31fc7d86245435ad6574e0e1126e734049e2 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2024-01-08T16:02:32+01:00 GitHub noreply@github.com 2024-01-08T16:02:32+01:00 SOTA 2-bit quants (#4773) 668b31fc7d86245435ad6574e0e1126e734049e2 42ea63c5a3da01d4a94e906d8565868012c79f4f Georgi Gerganov ggerganov@gmail.com 2024-01-08T16:40:51+02:00 GitHub noreply@github.com 2024-01-08T16:40:51+02:00 swift : exclude ggml-metal.metal from the package (#4822) 42ea63c5a3da01d4a94e906d8565868012c79f4f 52531fdff88764282c1b233174721aab8347252d Georgi Gerganov ggerganov@gmail.com 2024-01-08T15:57:36+02:00 GitHub noreply@github.com 2024-01-08T15:57:36+02:00 llama.swiftui : update readme 52531fdff88764282c1b233174721aab8347252d b0034d93ce2949ce7d9c098ca02e56f66cd484e2 Georgi Gerganov ggerganov@gmail.com 2024-01-08T11:18:32+02:00 GitHub noreply@github.com 2024-01-08T11:18:32+02:00 main : add self-extend support (#4815) b0034d93ce2949ce7d9c098ca02e56f66cd484e2 b7e7982953f80a656e03feb5cfb17a17a173eb26 Georgi Gerganov ggerganov@gmail.com 2024-01-08T11:14:04+02:00 GitHub noreply@github.com 2024-01-08T11:14:04+02:00 examples : add passkey test (#3856) b7e7982953f80a656e03feb5cfb17a17a173eb26 226460cc0d5b185bc6685fb76f418fd9418d7add Lars Grammel lars.grammel@gmail.com 2024-01-07T21:24:11+01:00 GitHub noreply@github.com 2024-01-07T22:24:11+02:00 readme : add lgrammel/modelfusion JS/TS client for llama.cpp (#4814) 226460cc0d5b185bc6685fb76f418fd9418d7add d5a410e8556191672465f7ff58682ea2474038b0 slaren slarengh@gmail.com 2024-01-07T17:59:01+01:00 GitHub noreply@github.com 2024-01-07T17:59:01+01:00 llama-bench : add no-kv-offload parameter (#4812) d5a410e8556191672465f7ff58682ea2474038b0 9dede37d812604897496dd9d276ae9fbe13d1042 Johannes Gäßler johannesg@5d6.de 2024-01-07T17:24:08+01:00 GitHub noreply@github.com 2024-01-07T17:24:08+01:00 CUDA: fixed redundant value dequantization (#4809) 9dede37d812604897496dd9d276ae9fbe13d1042 3c36213df850a2353e95572b3636797c79b7c815 Georgi Gerganov ggerganov@gmail.com 2024-01-07T14:29:36+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-07T14:29:36+02:00 llama : remove unused vars (#4796) 3c36213df850a2353e95572b3636797c79b7c815 72d8407b3696dd1293bd233b6db392be108bc377 Georgi Gerganov ggerganov@gmail.com 2024-01-07T11:21:53+02:00 GitHub noreply@github.com 2024-01-07T11:21:53+02:00 llama : remove redundant GQA check (#4796) 72d8407b3696dd1293bd233b6db392be108bc377 d117d4dc5dadb46831036bfa4d6e5e8c86babaf1 Alex Azarov alexander.azarov@mapbox.com 2024-01-07T09:20:50+01:00 GitHub noreply@github.com 2024-01-07T10:20:50+02:00 llama.swiftui : use llama.cpp as SPM package (#4804) d117d4dc5dadb46831036bfa4d6e5e8c86babaf1 3418c03ecc149fd657527ebb06776239b60a3f3b Georgi Gerganov ggerganov@gmail.com 2024-01-07T09:50:31+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-07T09:51:12+02:00 llama : print tensor meta for debugging 3418c03ecc149fd657527ebb06776239b60a3f3b 63ee677efd92060b14894b984597c62e3742b8da Alex Azarov alexander.azarov@mapbox.com 2024-01-07T08:46:55+01:00 GitHub noreply@github.com 2024-01-07T09:46:55+02:00 llama.swiftui : add visionOS target (#4805) 63ee677efd92060b14894b984597c62e3742b8da 67984921a70a7e680a24494aeb7575a66e90685d Konstantin Zhuravlyov konstantin.zhuravlyov@amd.com 2024-01-07T01:52:42-05:00 GitHub noreply@github.com 2024-01-07T08:52:42+02:00 ggml : use __builtin_amdgcn_sudot4 in __dp4a for gfx11 (#4787) 67984921a70a7e680a24494aeb7575a66e90685d c75ca5d96f902564cbbbdd7f5cade80d53c288bb Georgi Gerganov ggerganov@gmail.com 2024-01-07T08:45:26+02:00 GitHub noreply@github.com 2024-01-07T08:45:26+02:00 server : fix n_predict check (#4798) c75ca5d96f902564cbbbdd7f5cade80d53c288bb 96e80dabc6e73ff68b09b68947b1fc25883c5094 Daniel Illescas Romero illescas.daniel@protonmail.com 2024-01-06T16:12:59+01:00 GitHub noreply@github.com 2024-01-06T17:12:59+02:00 llama.swiftui : use correct pointer for llama_token_eos (#4797) 96e80dabc6e73ff68b09b68947b1fc25883c5094 eec22a1c6378d9a013943cbddb4330c0da621442 Georgi Gerganov ggerganov@gmail.com 2024-01-06T11:40:24+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-06T11:40:24+02:00 examples : improve base-translate.sh script (#4783) eec22a1c6378d9a013943cbddb4330c0da621442 be36bb946a6336238e92706464de6a30495fe825 a-n-n-a-l-e-e 150648636+a-n-n-a-l-e-e@users.noreply.github.com 2024-01-05T08:04:40-08:00 GitHub noreply@github.com 2024-01-05T18:04:40+02:00 cmake : check for openblas64 (#4134) be36bb946a6336238e92706464de6a30495fe825 91d38876dfa10332359ac671b62353aeceb448d3 Ikko Eltociear Ashimine eltociear@gmail.com 2024-01-06T01:02:44+09:00 GitHub noreply@github.com 2024-01-05T18:02:44+02:00 flake.nix : fix typo (#4700) 91d38876dfa10332359ac671b62353aeceb448d3 d061bf9405cc5fd50792fb2dbdff9c9ea53d6bf9 Georgi Gerganov ggerganov@gmail.com 2024-01-05T16:30:52+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-05T18:02:06+02:00 metal : switch back to default.metallib (ggml/681) d061bf9405cc5fd50792fb2dbdff9c9ea53d6bf9 1bf681f90ef4cf37b36e6d604d3e30fc57eda650 Georgi Gerganov ggerganov@gmail.com 2024-01-05T15:36:04+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-05T18:02:06+02:00 ggml : fix q2_k bpw in comments (ggml/680) 1bf681f90ef4cf37b36e6d604d3e30fc57eda650 c1d7cb28d3fed97fbc95fc3c43f0c5e2113e546c Finn Voorhees finnvoorhees@gmail.com 2024-01-03T08:39:43-05:00 Georgi Gerganov ggerganov@gmail.com 2024-01-05T18:02:06+02:00 ggml : add error handling to graph_compute (whisper/1714) c1d7cb28d3fed97fbc95fc3c43f0c5e2113e546c 3681f22443d917e7328456b69c276d6927dafeec Georgi Gerganov ggerganov@gmail.com 2024-01-05T15:18:21+02:00 GitHub noreply@github.com 2024-01-05T15:18:21+02:00 ggml : do not sched_yield when calling BLAS (#4761) 3681f22443d917e7328456b69c276d6927dafeec b3a7c20b5c035250257d2b62851c379b159c899a Georgi Gerganov ggerganov@gmail.com 2024-01-05T15:11:10+02:00 GitHub noreply@github.com 2024-01-05T15:11:10+02:00 examples : add few-shot translation example (#4783) b3a7c20b5c035250257d2b62851c379b159c899a 012cf349aec8ffb47c9def5dc018240fa3721e8b Daniel Bevenius daniel.bevenius@gmail.com 2024-01-04T20:45:37+01:00 GitHub noreply@github.com 2024-01-04T21:45:37+02:00 finetune : remove unused includes (#4756) 012cf349aec8ffb47c9def5dc018240fa3721e8b a91928014fcf51fe297823fcff0788de4f14206e Georgi Gerganov ggerganov@gmail.com 2024-01-04T19:56:33+02:00 GitHub noreply@github.com 2024-01-04T19:56:33+02:00 server : send token probs for "stream == false" (#4714) a91928014fcf51fe297823fcff0788de4f14206e 3c0b585561d74a56977cf3a3844535ecc9e37972 Johannes Gäßler johannesg@5d6.de 2024-01-04T09:43:23+01:00 GitHub noreply@github.com 2024-01-04T09:43:23+01:00 Print backend name on test-backend-ops failure (#4751) 3c0b585561d74a56977cf3a3844535ecc9e37972 e5804313a1edaf00726ed0b96ecced07accbf50c singularity 12184989+singularity-s0@users.noreply.github.com 2024-01-04T16:22:38+08:00 GitHub noreply@github.com 2024-01-04T10:22:38+02:00 llama.swiftui : support loading custom model from file picker (#4767) e5804313a1edaf00726ed0b96ecced07accbf50c dc891b7f7a23158d54f9383790b92c79cc5906c1 Michael Coppola m18coppola@gmail.com 2024-01-04T03:17:09-05:00 GitHub noreply@github.com 2024-01-04T10:17:09+02:00 server : fix options in README.md (#4765) dc891b7f7a23158d54f9383790b92c79cc5906c1 46cea79e1f32499bb24b9fab12123cd386e96728 Georgi Gerganov ggerganov@gmail.com 2024-01-04T10:12:26+02:00 GitHub noreply@github.com 2024-01-04T10:12:26+02:00 ggml : include stdlib.h before intrin.h (#4736) 46cea79e1f32499bb24b9fab12123cd386e96728 cb1e2818e0e12ec99f7236ec5d4f3ffd8bcc2f4a singularity 12184989+singularity-s0@users.noreply.github.com 2024-01-04T15:58:16+08:00 GitHub noreply@github.com 2024-01-04T09:58:16+02:00 llama.swiftui : fix build of ggml.metallib (#4754) cb1e2818e0e12ec99f7236ec5d4f3ffd8bcc2f4a ece9a45e8ffb73ad461c792720c2fec28b0137bc Daniel Bevenius daniel.bevenius@gmail.com 2024-01-03T18:53:40+01:00 GitHub noreply@github.com 2024-01-03T19:53:40+02:00 train : fix typo in overlapping-samples help msg (#4758) ece9a45e8ffb73ad461c792720c2fec28b0137bc 7bed7eba359b0fa8e575345dc5467a46b4ba509f Ashraful Islam ashraful.meche@gmail.com 2024-01-03T11:30:02-06:00 GitHub noreply@github.com 2024-01-03T19:30:02+02:00 swift : update Package.swift to use ggml as dependency (#4691) 7bed7eba359b0fa8e575345dc5467a46b4ba509f d55356d3baa58a6c3a9171cb67a67094b9aa9dff Georgi Gerganov ggerganov@gmail.com 2024-01-03T14:18:46+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-03T14:38:38+02:00 cuda : simplify expression d55356d3baa58a6c3a9171cb67a67094b9aa9dff 75e3fd85814c367b55aea11e7bb38cb7b82c6aa0 Georgi Gerganov ggerganov@gmail.com 2024-01-03T13:01:44+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-03T14:38:38+02:00 cuda : mark I16 and I32 ops as unsupported 75e3fd85814c367b55aea11e7bb38cb7b82c6aa0 289313716ff7ccf6aee284f686a0fe8cbc7714af Georgi Gerganov ggerganov@gmail.com 2024-01-03T11:37:44+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-03T14:38:38+02:00 sync : ggml 289313716ff7ccf6aee284f686a0fe8cbc7714af ab62fc3e5520f5a143c36cb23c269f11aa4dafd6 Georgi Gerganov ggerganov@gmail.com 2024-01-03T11:35:46+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-03T14:38:38+02:00 metal : add kernel_get_rows_i32 ab62fc3e5520f5a143c36cb23c269f11aa4dafd6 5f66ebca9c41a17385341da4b553a8eb5f07edee Georgi Gerganov ggerganov@gmail.com 2024-01-03T11:25:54+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-03T14:38:38+02:00 scripts : fix sync order + metal sed 5f66ebca9c41a17385341da4b553a8eb5f07edee f2eb19bd8bc9f5730d6e05d7a52a9e19bf5ac099 Guillaume Wenzek gwenzek@users.noreply.github.com 2023-12-29T18:07:03+01:00 Georgi Gerganov ggerganov@gmail.com 2024-01-03T14:38:38+02:00 ggml : extend ggml_get_rows, ggml_repeat, ggml_concat (ggml/639) f2eb19bd8bc9f5730d6e05d7a52a9e19bf5ac099 f3f62f0d835d559e80714bbeb05d03125574e3dd Justin Parker jparkerweb@gmail.com 2024-01-03T03:43:19-05:00 GitHub noreply@github.com 2024-01-03T10:43:19+02:00 server : throw an error when `slot unavailable` (#4741) f3f62f0d835d559e80714bbeb05d03125574e3dd 0ef3ca2ac62016c0c545de1c89dc2e3e130f4a99 Georgi Gerganov ggerganov@gmail.com 2024-01-02T21:07:47+02:00 GitHub noreply@github.com 2024-01-02T21:07:47+02:00 metal : optimize ggml_mul_mat_id (faster Mixtral PP) (#4725) 0ef3ca2ac62016c0c545de1c89dc2e3e130f4a99 540938f8904707dd74cb3be18495f853b312e72f Phil H 5756783+phiharri@users.noreply.github.com 2024-01-02T15:48:49Z GitHub noreply@github.com 2024-01-02T17:48:49+02:00 server : add token counts to html footer (#4738) 540938f8904707dd74cb3be18495f853b312e72f 0040d42eeb237197054cc7790df5776eacfa608e Georgi Gerganov ggerganov@gmail.com 2024-01-02T16:26:45+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-02T16:26:45+02:00 llama : llama_model_desc print number of experts 0040d42eeb237197054cc7790df5776eacfa608e 83e633c27efdf0eb0ba54249e784b0ea760b1007 Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2024-01-02T06:15:16-08:00 GitHub noreply@github.com 2024-01-02T16:15:16+02:00 llama : replace all API facing `int`'s with `int32_t` (#4577) 83e633c27efdf0eb0ba54249e784b0ea760b1007 32866c5edde402f42ff4233bb89dcfcede34fd22 postmasters namnguyen@google.com 2024-01-02T03:51:28-08:00 GitHub noreply@github.com 2024-01-02T13:51:28+02:00 llama : differentiate the KV dims in the attention (#4657) 32866c5edde402f42ff4233bb89dcfcede34fd22 5d7002d4372ebf107cfaf46fcd90df27b204f330 Georgi Gerganov ggerganov@gmail.com 2024-01-02T13:28:15+02:00 Georgi Gerganov ggerganov@gmail.com 2024-01-02T13:28:15+02:00 editorconfig : fix whitespace and indentation #4710 5d7002d4372ebf107cfaf46fcd90df27b204f330 26f3071d714f0b27ad7f021a46a66a1085480258 minarchist minarchist@users.noreply.github.com 2024-01-02T04:38:15-06:00 GitHub noreply@github.com 2024-01-02T12:38:15+02:00 server : add --override-kv parameter (#4710) 26f3071d714f0b27ad7f021a46a66a1085480258 775ac8712a7b42cfead2585f42cec0dfd56644ab Nam D. Tran 42194884+namtranase@users.noreply.github.com 2024-01-02T16:23:38+07:00 GitHub noreply@github.com 2024-01-02T11:23:38+02:00 py : re-enable mmap in convert hf (#4732) 775ac8712a7b42cfead2585f42cec0dfd56644ab 58ba655af054715c0516ee270ad028ad9e74f357 Daniel Bevenius daniel.bevenius@gmail.com 2024-01-02T10:16:55+01:00 GitHub noreply@github.com 2024-01-02T10:16:55+01:00 finetune: fix typo in README.md (#4733) 58ba655af054715c0516ee270ad028ad9e74f357 edd1ab7bc34c10a780ee7f9a4499f7689cdad36d Georgi Gerganov ggerganov@gmail.com 2024-01-02T10:57:44+02:00 GitHub noreply@github.com 2024-01-02T10:57:44+02:00 metal : enable shader debugging (cmake option) (#4705) edd1ab7bc34c10a780ee7f9a4499f7689cdad36d 198ed7ebfc89b8f2b35a8b1655d57bfb57530c1a Someone Serge sergei.kozlukov@aalto.fi 2023-12-31T17:42:22Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 flake.lock: update 198ed7ebfc89b8f2b35a8b1655d57bfb57530c1a d8361747317c5cb2e00e7fb3b59ff4dce5a176a5 Someone Serge sergei.kozlukov@aalto.fi 2023-12-30T18:25:25Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 flake.nix: suggest the binary caches d8361747317c5cb2e00e7fb3b59ff4dce5a176a5 06f2a5d1909a1385b1a16dab4ade68377e121bdd Someone Serge sergei.kozlukov@aalto.fi 2023-12-30T18:01:07Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 workflows: nix-ci: add a qemu job for jetsons 06f2a5d1909a1385b1a16dab4ade68377e121bdd c5239944bab0ff71915df8f2dc7e42fc2c138ff6 Someone Serge sergei.kozlukov@aalto.fi 2023-12-30T17:36:08Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 workflows: nix-flakestry: drop tag filters c5239944bab0ff71915df8f2dc7e42fc2c138ff6 1e9ae54cf24d27afe3900d1250634a2a33423db1 Someone Serge sergei.kozlukov@aalto.fi 2023-12-30T16:38:36Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 workflows: weekly `nix flake update` 1e9ae54cf24d27afe3900d1250634a2a33423db1 7adedecbe39bd552bc14142f496246d55a43ac4e Someone Serge sergei.kozlukov@aalto.fi 2023-12-30T17:19:11Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 workflows: nix-ci: add a job for eval 7adedecbe39bd552bc14142f496246d55a43ac4e 356ea17e0f92bfbbf28a4f69261bed48eff68d9c Someone Serge sergei.kozlukov@aalto.fi 2023-12-26T19:17:26Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 workflows: nix-ci: init; build flake outputs 356ea17e0f92bfbbf28a4f69261bed48eff68d9c a5c088d8c698299b973d2709153e5d95295606d9 Someone Serge sergei.kozlukov@aalto.fi 2023-12-29T16:21:50Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 flake.nix: expose checks a5c088d8c698299b973d2709153e5d95295606d9 1e3900ebacb3a0b385271389686403c97ad76d88 Someone Serge sergei.kozlukov@aalto.fi 2023-12-26T23:34:40Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 flake.nix: rocm not yet supported on aarch64, so hide the output 1e3900ebacb3a0b385271389686403c97ad76d88 e39106c0554cbd0e9310e08fb3b2a577ea4b6273 Someone Serge sergei.kozlukov@aalto.fi 2023-12-29T16:15:37Z Philip Taron philip.taron@gmail.com 2023-12-31T13:14:58-08:00 flake.nix: expose full scope in legacyPackages e39106c0554cbd0e9310e08fb3b2a577ea4b6273 9fbda719de18a9400a064c28759c39d55d687d3e Georgi Gerganov ggerganov@gmail.com 2023-12-31T11:43:31+02:00 GitHub noreply@github.com 2023-12-31T11:43:31+02:00 ggml : add ggml_vdotq_s32 alias (#4715) 9fbda719de18a9400a064c28759c39d55d687d3e 39d8bc71edcb8b6f99d46fa4216af7a15232e218 Georgi Gerganov ggerganov@gmail.com 2023-12-30T23:24:42+02:00 GitHub noreply@github.com 2023-12-30T23:24:42+02:00 clip : refactor + bug fixes (#4696) 39d8bc71edcb8b6f99d46fa4216af7a15232e218 24a447e20af425fa44cf10feaa632b6bb596c80f Johannes Gäßler johannesg@5d6.de 2023-12-30T13:52:01+01:00 GitHub noreply@github.com 2023-12-30T13:52:01+01:00 CUDA: fixed tensor cores not being used on RDNA3 (#4697) 24a447e20af425fa44cf10feaa632b6bb596c80f a20f3c7465d6d1b33767757c2760643b799a81bf automaticcat daogiatuank54@gmail.com 2023-12-30T15:07:48+07:00 GitHub noreply@github.com 2023-12-30T10:07:48+02:00 ggml : add ggml_cpu_has_avx_vnni() (#4589) a20f3c7465d6d1b33767757c2760643b799a81bf 0235b9b571f3cc7d2b8836409a5404b41ce1379c Johannes Gäßler johannesg@5d6.de 2023-12-29T23:12:53+01:00 GitHub noreply@github.com 2023-12-29T23:12:53+01:00 CUDA: fix tensor core logic for Pascal and HIP (#4682) 0235b9b571f3cc7d2b8836409a5404b41ce1379c ce18d727a47f2473ca863a6f78bf3ad480008f72 Georgi Gerganov ggerganov@gmail.com 2023-12-29T18:53:34+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-29T18:53:34+02:00 clip : use ggml_backend_buffer_is_host (#4205) ce18d727a47f2473ca863a6f78bf3ad480008f72 91bb39cec7e4dfb9e2293509ef60298a67f0b1b7 Steward Garcia 57494570+FSSRepo@users.noreply.github.com 2023-12-29T11:52:15-05:00 GitHub noreply@github.com 2023-12-29T18:52:15+02:00 clip : enable gpu backend (#4205) 91bb39cec7e4dfb9e2293509ef60298a67f0b1b7 04ac0607e913ab91234dfb240e12a76509e30982 hydai z54981220@gmail.com 2023-12-30T00:31:19+08:00 GitHub noreply@github.com 2023-12-29T17:31:19+01:00 cuda: fix vmm oom issue on NVIDIA AGX Orin (#4687) 04ac0607e913ab91234dfb240e12a76509e30982 68eccbdc5b56f2a2450f9a8463f9934388cafabf crasm crasm@git.vczf.us 2023-12-29T09:50:29-05:00 GitHub noreply@github.com 2023-12-29T16:50:29+02:00 python : add check-requirements.sh and GitHub workflow (#4585) 68eccbdc5b56f2a2450f9a8463f9934388cafabf 97bbca6e8522d18041fcde6c3d0907a52ce36446 Philip Taron philip.taron@gmail.com 2023-12-29T06:42:26-08:00 GitHub noreply@github.com 2023-12-29T16:42:26+02:00 flake.nix : rewrite (#4605) 97bbca6e8522d18041fcde6c3d0907a52ce36446 4af4801566bc262a38fb77f51edf278ac323c2bd Cuong Trinh Manh nguoithichkhampha@gmail.com 2023-12-29T21:39:15+07:00 GitHub noreply@github.com 2023-12-29T16:39:15+02:00 cmake : fix ld warning duplicate libraries libllama.a (#4671) 4af4801566bc262a38fb77f51edf278ac323c2bd db49ff8ed7f0bb201176703441cc02911b08ef2a Justine Tunney jtunney@gmail.com 2023-12-29T06:38:38-08:00 GitHub noreply@github.com 2023-12-29T16:38:38+02:00 llava-cli : refactor to use sampling library (#4669) db49ff8ed7f0bb201176703441cc02911b08ef2a 60f55e888c29cbd87c4238dd19e85d0eef87245d Justine Tunney jtunney@gmail.com 2023-12-29T06:24:12-08:00 GitHub noreply@github.com 2023-12-29T16:24:12+02:00 server : replace sleep with condition variables (#4673) 60f55e888c29cbd87c4238dd19e85d0eef87245d b93edd22f55d3e5268263c3edcdae1818505c078 SakuraUmi yukinon244@gmail.com 2023-12-29T22:22:44+08:00 GitHub noreply@github.com 2023-12-29T16:22:44+02:00 server : fix OpenAI server sampling w.r.t. penalty. (#4675) b93edd22f55d3e5268263c3edcdae1818505c078 82d6eab224862a7044069fb9211dc4b29124264b Karthik Sethuraman k.seth1993@gmail.com 2023-12-29T06:22:10-08:00 GitHub noreply@github.com 2023-12-29T16:22:10+02:00 server : allow to generate multimodal embeddings (#4681) 82d6eab224862a7044069fb9211dc4b29124264b afd997ab6011dfefe9e917425b04ef4d83614841 andrijdavid david@geek.mg 2023-12-29T15:18:20+01:00 GitHub noreply@github.com 2023-12-29T16:18:20+02:00 main-cmake-pkg : fix build issue (#4665) afd997ab6011dfefe9e917425b04ef4d83614841 c8255f8a6b2a3b3ebc6cb340cc2487f39fc95ffc Peter Sugihara peter@campsh.com 2023-12-29T05:58:56-08:00 GitHub noreply@github.com 2023-12-29T15:58:56+02:00 llama.swiftui : fix infinite loop, ouput timings, buff UI (#4674) c8255f8a6b2a3b3ebc6cb340cc2487f39fc95ffc 441f51dca004debf8b275f1bdc08e0f1af7fd8f8 Georgi Gerganov ggerganov@gmail.com 2023-12-29T15:12:35+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-29T15:12:35+02:00 scripts : print list of sync commits 441f51dca004debf8b275f1bdc08e0f1af7fd8f8 38b3de4658292582a8941a2be5c77b40ce6ac0f2 Tamotsu Takahashi ttakah+github@gmail.com 2023-12-29T19:23:27+09:00 Georgi Gerganov ggerganov@gmail.com 2023-12-29T15:11:53+02:00 ci : build with CLBlast + ggml-opencl use GGML_API (whisper/1576) 38b3de4658292582a8941a2be5c77b40ce6ac0f2 afc8c192919f04613a92d40391bff4c8cd99856b Georgi Gerganov ggerganov@gmail.com 2023-12-29T14:56:41+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-29T14:56:41+02:00 sync : ggml afc8c192919f04613a92d40391bff4c8cd99856b ca38b8d334baa724bd6c9402470931d26427466f bssrdf merlintiger@hotmail.com 2023-12-29T03:32:31-05:00 Georgi Gerganov ggerganov@gmail.com 2023-12-29T14:54:19+02:00 ggml : fix some mul mat cases + add tests for src1 F16 (ggml/669) ca38b8d334baa724bd6c9402470931d26427466f 65e5f6dadbba4b496bba27f573e473c66b446496 Georgi Gerganov ggerganov@gmail.com 2023-12-29T14:41:36+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-29T14:54:05+02:00 scripts : do not sync commits from this repo 65e5f6dadbba4b496bba27f573e473c66b446496 ea5497df5d138c83b2b0ca70aefdc4b1175c1001 Justine Tunney jtunney@gmail.com 2023-12-28T11:20:00-08:00 GitHub noreply@github.com 2023-12-28T15:20:00-04:00 Fix OpenAI server sampling w.r.t. temp and seed (#4668) ea5497df5d138c83b2b0ca70aefdc4b1175c1001 f6793491b5af6da75edad34d6f503ef86d31b09f manikbhandari mbbhandarimanik2@gmail.com 2023-12-28T09:03:57-05:00 GitHub noreply@github.com 2023-12-28T15:03:57+01:00 gpt2 : Add gpt2 architecture integration (#4555) f6793491b5af6da75edad34d6f503ef86d31b09f 879b690a9e1eb1ab0a29b58236fc76978fb4d902 Nam D. Tran 42194884+namtranase@users.noreply.github.com 2023-12-27T22:39:45+07:00 GitHub noreply@github.com 2023-12-27T17:39:45+02:00 llama : add AWQ for llama, llama2, mpt, and mistral models (#4593) 879b690a9e1eb1ab0a29b58236fc76978fb4d902 b47879b0dda43f2d26415e88b6840295817e552a Daniel Bevenius daniel.bevenius@gmail.com 2023-12-27T15:16:55+01:00 GitHub noreply@github.com 2023-12-27T16:16:55+02:00 finetune : fix output formatting in print_params (#4653) b47879b0dda43f2d26415e88b6840295817e552a 951010fa53a0ffe81b7d2e87c4349e0d3cb3d19d Georgi Gerganov ggerganov@gmail.com 2023-12-27T11:15:31+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-27T11:44:22+02:00 scripts : add sync-ggml-am.sh 951010fa53a0ffe81b7d2e87c4349e0d3cb3d19d f56d6077d0c37e6606ac0a4fa3169de70593acfe Georgi Gerganov ggerganov@gmail.com 2023-12-27T11:02:13+02:00 GitHub noreply@github.com 2023-12-27T11:02:13+02:00 ggml : fix dot product for ARM (#4630) f56d6077d0c37e6606ac0a4fa3169de70593acfe dc68f0054cd279cddddb0cae0c9ef4f9cbaa512a wonjun Jang strutive07@gmail.com 2023-12-27T17:37:25+09:00 GitHub noreply@github.com 2023-12-27T17:37:25+09:00 Add byte token type when tokenizer.model is not exists (#4641) dc68f0054cd279cddddb0cae0c9ef4f9cbaa512a de8e496437c59e7d1cc84109e3e49a3478aee25a slaren slarengh@gmail.com 2023-12-26T21:23:59+01:00 GitHub noreply@github.com 2023-12-26T21:23:59+01:00 cuda : fix vmm pool with multi GPU (#4620) de8e496437c59e7d1cc84109e3e49a3478aee25a 77465dad48d7c945c367ab46b6f2ea98ae9b7b15 WillCorticesAI 150854901+WillCorticesAI@users.noreply.github.com 2023-12-26T05:42:08-05:00 GitHub noreply@github.com 2023-12-26T11:42:08+01:00 Update comment for AdamW implementation reference. (#4604) 77465dad48d7c945c367ab46b6f2ea98ae9b7b15 a206137f927daef1752753cf5e281220b449a468 FantasyGmm 16450052+FantasyGmm@users.noreply.github.com 2023-12-26T18:38:36+08:00 GitHub noreply@github.com 2023-12-26T11:38:36+01:00 Fix new CUDA10 compilation errors (#4635) a206137f927daef1752753cf5e281220b449a468 b9f47952ffae4e0d3420905526003c23333f6c98 Paul Tsochantaris ptsochantaris@icloud.com 2023-12-25T16:09:53Z GitHub noreply@github.com 2023-12-25T18:09:53+02:00 Adding Emeltal reference to UI list (#4629) b9f47952ffae4e0d3420905526003c23333f6c98 753be377b69bda2d65a7e089f2b7f0c53ef3495e slaren slarengh@gmail.com 2023-12-24T21:01:12+01:00 GitHub noreply@github.com 2023-12-24T22:01:12+02:00 simplify bug issue template (#4623) 753be377b69bda2d65a7e089f2b7f0c53ef3495e 5bf3953d7e9831ea22b0bc017ce97409b801ccf1 Shintarou Okada kokuzen@gmail.com 2023-12-24T22:35:49+09:00 GitHub noreply@github.com 2023-12-24T15:35:49+02:00 llama : add PLaMo model (#3557) 5bf3953d7e9831ea22b0bc017ce97409b801ccf1 708e179e8562c2604240df95a2241dea17fd808b slaren slarengh@gmail.com 2023-12-24T14:34:22+01:00 GitHub noreply@github.com 2023-12-24T14:34:22+01:00 cuda : improve cuda pool efficiency using virtual memory (#4606) 708e179e8562c2604240df95a2241dea17fd808b 925e5584a058afb612f9c20bc472c130f5d0f891 slaren slarengh@gmail.com 2023-12-23T16:10:51+01:00 GitHub noreply@github.com 2023-12-23T16:10:51+01:00 fallback to CPU buffer if host buffer alloc fails (#4610) 925e5584a058afb612f9c20bc472c130f5d0f891 6123979952385847d8348e295d77d6e01da8aa84 Samuel Maynard samwmaynard@gmail.com 2023-12-23T11:35:55+02:00 GitHub noreply@github.com 2023-12-23T11:35:55+02:00 ci(docker): fix tags in "Build and push docker image (tagged)" (#4603) 6123979952385847d8348e295d77d6e01da8aa84 b9ec82d262cb20d7f0a8a1157bfa9aace40e2625 Alexey Parfenov zxed@alkatrazstudio.net 2023-12-23T09:31:49Z GitHub noreply@github.com 2023-12-23T11:31:49+02:00 server : allow to specify custom prompt for penalty calculation (#3727) b9ec82d262cb20d7f0a8a1157bfa9aace40e2625 e0a4002273907b2c414b6b5442d99e08bfe2df35 kalomaze 66376113+kalomaze@users.noreply.github.com 2023-12-23T03:27:07-06:00 GitHub noreply@github.com 2023-12-23T11:27:07+02:00 grammar : check the full vocab only if necessary (opt) (#4306) e0a4002273907b2c414b6b5442d99e08bfe2df35 7082d24cec35e9ce9147535a2224dfc67ee0a78c Johannes Gäßler johannesg@5d6.de 2023-12-23T09:16:33+01:00 GitHub noreply@github.com 2023-12-23T09:16:33+01:00 CUDA: fixed row rounding for 0 tensor splits (#4594) 7082d24cec35e9ce9147535a2224dfc67ee0a78c ba661751322a7c201fd3bef71af077c5aebfaa2a LeonEricsson 70749762+LeonEricsson@users.noreply.github.com 2023-12-22T17:05:56+01:00 GitHub noreply@github.com 2023-12-22T18:05:56+02:00 lookup : add prompt lookup decoding example (#4484) ba661751322a7c201fd3bef71af077c5aebfaa2a a55876955b1a83464171de8d578d3ab062a7b62d Georgi Gerganov ggerganov@gmail.com 2023-12-22T17:53:43+02:00 GitHub noreply@github.com 2023-12-22T17:53:43+02:00 sync : ggml (fix im2col) (#4591) a55876955b1a83464171de8d578d3ab062a7b62d 6724ef16573ec7ecce620be56cbbff145856b2fb FantasyGmm 16450052+FantasyGmm@users.noreply.github.com 2023-12-22T23:11:12+08:00 GitHub noreply@github.com 2023-12-22T17:11:12+02:00 cuda : fix jetson compile error (#4560) 6724ef16573ec7ecce620be56cbbff145856b2fb 48b7ff193e64c97ab174280ba0eb8d14b47c49ba Henrik Forstén henrik.forsten@gmail.com 2023-12-22T15:34:05+02:00 GitHub noreply@github.com 2023-12-22T14:34:05+01:00 Fix CudaMemcpy direction (#4599) 48b7ff193e64c97ab174280ba0eb8d14b47c49ba 48b24b170e3b4f9dc28200306840cb07d1c123df slaren slarengh@gmail.com 2023-12-22T12:12:53+01:00 GitHub noreply@github.com 2023-12-22T13:12:53+02:00 llama : fix platforms without mmap (#4578) 48b24b170e3b4f9dc28200306840cb07d1c123df 28cb35a0ecb9852adc3494aa51dde60141939d64 Herman Semenov GermanAizek@yandex.ru 2023-12-22T09:26:49Z GitHub noreply@github.com 2023-12-22T11:26:49+02:00 ggml : add comment about backward GGML_OP_DIAG_MASK_INF (#4203) 28cb35a0ecb9852adc3494aa51dde60141939d64 f31b98489824a86c937fa62ccf5dfd4bb0327b86 Michael Kesper mkesper@schokokeks.org 2023-12-22T09:03:25+01:00 GitHub noreply@github.com 2023-12-22T10:03:25+02:00 make : add LLAMA_HIP_UMA option (#4587) f31b98489824a86c937fa62ccf5dfd4bb0327b86 2bb98279c5a087d62949972b35cf63ff974ffe6a rhuddleston ryan.huddleston@percona.com 2023-12-21T23:56:34-07:00 GitHub noreply@github.com 2023-12-22T08:56:34+02:00 ci : tag docker image with build number (#4584) 2bb98279c5a087d62949972b35cf63ff974ffe6a 0137ef88ea9f8fd837a065700814329d24adeec3 Deins deinsegle@gmail.com 2023-12-22T08:49:54+02:00 GitHub noreply@github.com 2023-12-22T08:49:54+02:00 readme : add zig bindings (#4581) 0137ef88ea9f8fd837a065700814329d24adeec3 c7e9701f86564088350209d2f9d71c96ea00527f bobqianic 129547291+bobqianic@users.noreply.github.com 2023-12-22T06:47:01Z GitHub noreply@github.com 2023-12-22T08:47:01+02:00 ggml : extend `enum ggml_log_level` with `GGML_LOG_LEVEL_DEBUG` (#4579) c7e9701f86564088350209d2f9d71c96ea00527f afefa319f1f59b002dfa0d1ef407a2c74bd9770b crasm crasm@git.vczf.us 2023-12-22T01:19:36-05:00 GitHub noreply@github.com 2023-12-22T08:19:36+02:00 llama : add ability to cancel model loading (#4462) afefa319f1f59b002dfa0d1ef407a2c74bd9770b 769a7bc85eaa44e3d7eadf39abfeff7bb0b9cc2f Georgi Gerganov ggerganov@gmail.com 2023-12-21T23:20:49+02:00 GitHub noreply@github.com 2023-12-21T23:20:49+02:00 ggml : change ggml_scale to take a float instead of tensor (#4573) 769a7bc85eaa44e3d7eadf39abfeff7bb0b9cc2f 32259b2dade6f6856739bf7ba0a4ff7b474dc760 Georgi Gerganov ggerganov@gmail.com 2023-12-21T23:20:36+02:00 GitHub noreply@github.com 2023-12-21T23:20:36+02:00 gguf-py : fix broken link 32259b2dade6f6856739bf7ba0a4ff7b474dc760 4a5f9d629ecfd0a53afdddbaf54a4fa02d9a9ce9 Georgi Gerganov ggerganov@gmail.com 2023-12-21T23:07:58+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-21T23:08:14+02:00 gguf : simplify example dependencies 4a5f9d629ecfd0a53afdddbaf54a4fa02d9a9ce9 d232aca5a73b290e218a2e48b91023d5e994203f Samuel Maynard samwmaynard@gmail.com 2023-12-21T22:36:26+02:00 GitHub noreply@github.com 2023-12-21T22:36:26+02:00 ci : add `jlumbroso/free-disk-space` to docker workflow (#4150) d232aca5a73b290e218a2e48b91023d5e994203f 31f27758faf4a4bd08101a57c7ec3a473f771f86 slaren slarengh@gmail.com 2023-12-21T21:07:46+01:00 GitHub noreply@github.com 2023-12-21T21:07:46+01:00 llama : initial ggml-backend integration (#4520) 31f27758faf4a4bd08101a57c7ec3a473f771f86 56fa50819f7a3ca2128f63b81c17c08a4454479e Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2023-12-21T11:57:48-08:00 GitHub noreply@github.com 2023-12-21T21:57:48+02:00 llama : allow getting n_batch from llama_context in c api (#4540) 56fa50819f7a3ca2128f63b81c17c08a4454479e 0f630fbc924aaabeea6eaf466bb4b47d13015c3e Finn Voorhees finnvoorhees@gmail.com 2023-12-21T14:55:02-05:00 GitHub noreply@github.com 2023-12-21T21:55:02+02:00 metal : fix `ggml_metal_log` vargs (#4373) 0f630fbc924aaabeea6eaf466bb4b47d13015c3e 562cf222b5129e40b312877e928eac3a02e4ec33 Erik Garrison erik.garrison@gmail.com 2023-12-21T13:45:32-06:00 GitHub noreply@github.com 2023-12-21T21:45:32+02:00 cuda : ROCm AMD Unified Memory Architecture (UMA) handling (#4449) 562cf222b5129e40b312877e928eac3a02e4ec33 8fe03ffddaaa0ab5d48feaafe398151c9f22d4f6 arlo-phoenix 140345165+arlo-phoenix@users.noreply.github.com 2023-12-21T20:13:25+01:00 GitHub noreply@github.com 2023-12-21T20:13:25+01:00 ggml-cuda: Fix HIP build by adding define for __trap (#4569) 8fe03ffddaaa0ab5d48feaafe398151c9f22d4f6 9154494808dc865475c59022c29060b4947a803b Jared Van Bortel jared@nomic.ai 2023-12-21T12:55:34-05:00 GitHub noreply@github.com 2023-12-21T19:55:34+02:00 common : remove incorrect --model-draft default (#4568) 9154494808dc865475c59022c29060b4947a803b c083718c895b7c8c7fb2a4660643fb78d0c64dfd Johannes Gäßler johannesg@5d6.de 2023-12-21T18:42:59+01:00 GitHub noreply@github.com 2023-12-21T18:42:59+01:00 CUDA: mul_mat_id always on GPU for batches >= 32 (#4553) c083718c895b7c8c7fb2a4660643fb78d0c64dfd 880e352277fc017df4d5794f0c21c44e1eae2b84 Georgi Gerganov ggerganov@gmail.com 2023-12-21T19:27:14+02:00 GitHub noreply@github.com 2023-12-21T19:27:14+02:00 readme : update coding guidelines 880e352277fc017df4d5794f0c21c44e1eae2b84 66f35a2f48e1965a13835a523e677223dbf148be howlger github@voormann.de 2023-12-21T18:07:34+01:00 GitHub noreply@github.com 2023-12-21T19:07:34+02:00 py : open merges file as 'utf-8' (#4566) 66f35a2f48e1965a13835a523e677223dbf148be 139882392258671ffe5acdfcadc0bc08572d6eef bobqianic 129547291+bobqianic@users.noreply.github.com 2023-12-21T17:06:44Z GitHub noreply@github.com 2023-12-21T19:06:44+02:00 cuda : better error message for ggml_get_rows (#4561) 139882392258671ffe5acdfcadc0bc08572d6eef d3223afdad0ed2821a8ddf739c291cd410c92a11 slaren slarengh@gmail.com 2023-12-21T18:02:30+01:00 GitHub noreply@github.com 2023-12-21T18:02:30+01:00 cuda : replace asserts in wrong architecture checks with __trap (#4556) d3223afdad0ed2821a8ddf739c291cd410c92a11 1d7a1912cea2227f9a1a449758ed622c560542f9 Johannes Gäßler johannesg@5d6.de 2023-12-21T17:34:17+01:00 GitHub noreply@github.com 2023-12-21T18:34:17+02:00 llama : disable per-tensor info prints on model load (#4562) 1d7a1912cea2227f9a1a449758ed622c560542f9 799fc2268989482054944c902874cca76337580f LoganDark github@logandark.mozmail.com 2023-12-21T01:59:27-08:00 GitHub noreply@github.com 2023-12-21T10:59:27+01:00 Fix access violation in ggml_cuda_free_data if tensor->extra is NULL (#4554) 799fc2268989482054944c902874cca76337580f 328b83de23b33240e28f4e74900d1d06726f5eb1 Johannes Gäßler johannesg@5d6.de 2023-12-20T15:41:22+01:00 GitHub noreply@github.com 2023-12-20T15:41:22+01:00 CUDA: Faster Mixtral prompt processing (#4538) 328b83de23b33240e28f4e74900d1d06726f5eb1 a7aee47b98e45539d491071b25778b833b77e387 Eric Sommerlade es0m@users.noreply.github.com 2023-12-19T16:17:01Z GitHub noreply@github.com 2023-12-19T18:17:01+02:00 ggml : fixed check for _MSC_VER (#4535) a7aee47b98e45539d491071b25778b833b77e387 0e18b2e7d0b5c0a509ea40098def234b8d4a938a arlo-phoenix 140345165+arlo-phoenix@users.noreply.github.com 2023-12-18T22:33:45+01:00 GitHub noreply@github.com 2023-12-18T22:33:45+01:00 ggml-cuda: Fix HIP build (#4528) 0e18b2e7d0b5c0a509ea40098def234b8d4a938a 6ff39b129d0281d045f83d515e51b7197b44b253 Georgi Gerganov ggerganov@gmail.com 2023-12-18T20:17:43+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-18T20:17:43+02:00 llama.swiftui : add tinyllama 1.1B F16 6ff39b129d0281d045f83d515e51b7197b44b253 b9e74f9bca5fdf7d0a22ed25e7a9626335fdfa48 Georgi Gerganov ggerganov@gmail.com 2023-12-18T20:05:12+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-18T20:05:12+02:00 llama.swiftui : add more models b9e74f9bca5fdf7d0a22ed25e7a9626335fdfa48 3c04bf6da89eaf4c7d317e0518f0687dfcbf2de7 Ebey Abraham ebey97@gmail.com 2023-12-18T17:27:47Z GitHub noreply@github.com 2023-12-18T19:27:47+02:00 llama : add phi-2 + fix NeoX rope + ggml_mul_mat_set_prec (#4490) 3c04bf6da89eaf4c7d317e0518f0687dfcbf2de7 2994f0c5a2e8c96955b422dedc93ec2595d16b82 hankcs cnhankmc@gmail.com 2023-12-18T05:14:58-08:00 GitHub noreply@github.com 2023-12-18T15:14:58+02:00 llama : fix try_override for bool_value which always return true (#4519) 2994f0c5a2e8c96955b422dedc93ec2595d16b82 b1306c439490c7fa4ec33594500d980d1e9e15e6 Jared Van Bortel jared@nomic.ai 2023-12-17T19:39:02-05:00 GitHub noreply@github.com 2023-12-17T19:39:02-05:00 decode : fix logits_valid for legacy API (#4516) b1306c439490c7fa4ec33594500d980d1e9e15e6 800a489e4a8be199122259a995b1ee9dd7fae320 Georgi Gerganov ggerganov@gmail.com 2023-12-17T20:16:23+02:00 GitHub noreply@github.com 2023-12-17T20:16:23+02:00 readme : update hot topics 800a489e4a8be199122259a995b1ee9dd7fae320 f7f468a97dceec2f8fe8b1ed7a2091083446ebc7 Georgi Gerganov ggerganov@gmail.com 2023-12-17T19:38:41+02:00 GitHub noreply@github.com 2023-12-17T19:38:41+02:00 llama.swiftui : add bench functionality (#4483) f7f468a97dceec2f8fe8b1ed7a2091083446ebc7 919c40660fd27157b391b5832d2a577d5afef4cb Jared Van Bortel jared@nomic.ai 2023-12-17T10:45:46-05:00 GitHub noreply@github.com 2023-12-17T10:45:46-05:00 gguf-py : fail fast on nonsensical special token IDs (#4489) 919c40660fd27157b391b5832d2a577d5afef4cb 45668633fdb522a925c3dafc1ecf426f539efb27 Matheus Gabriel Alves Silva matheusgasource@gmail.com 2023-12-17T12:23:33-03:00 GitHub noreply@github.com 2023-12-17T17:23:33+02:00 build : Check the ROCm installation location (#4485) 45668633fdb522a925c3dafc1ecf426f539efb27 0ffc92d2d23a789625f018840469af045be1e3c0 slaren slarengh@gmail.com 2023-12-17T16:05:56+01:00 GitHub noreply@github.com 2023-12-17T16:05:56+01:00 finetune : keep allocs alive until all allocations are done (#4486) 0ffc92d2d23a789625f018840469af045be1e3c0 8edd2b40fdbcafbf630f2cf29306b29d5cb48c42 olexiyb olexiyb@gmail.com 2023-12-17T17:02:16+02:00 GitHub noreply@github.com 2023-12-17T17:02:16+02:00 server : disable llm logs if SERVER_VERBOSE is off (#3792) 8edd2b40fdbcafbf630f2cf29306b29d5cb48c42 eb16dae7e70ca97396190698b29c0f9ee3388e88 AdithyanI adithyan.i4internet@gmail.com 2023-12-17T15:57:56+01:00 GitHub noreply@github.com 2023-12-17T16:57:56+02:00 server : fix grammar being ignored (#4494) eb16dae7e70ca97396190698b29c0f9ee3388e88 62bd52b7bf90819e75f427a95a484cd5eee0b3c7 Alexey Parfenov zxed@alkatrazstudio.net 2023-12-17T14:56:09Z GitHub noreply@github.com 2023-12-17T16:56:09+02:00 server : fix possible ambiguity in content type charset (#4501) 62bd52b7bf90819e75f427a95a484cd5eee0b3c7 5daa5f54fdcd2b5228add1a4c43a1897b2168f35 mzcu milos.cubrilo@gmail.com 2023-12-17T15:54:37+01:00 GitHub noreply@github.com 2023-12-17T16:54:37+02:00 server : allow requests larger than 8K (#4500) 5daa5f54fdcd2b5228add1a4c43a1897b2168f35 c6c4fc081c1df1c60a9bfe3e6a3fd086f1a29ec7 Bach Le bach@bullno1.com 2023-12-17T18:57:33+08:00 GitHub noreply@github.com 2023-12-17T11:57:33+01:00 Link to cublas dynamically on Windows even with LLAMA_STATIC (#4506) c6c4fc081c1df1c60a9bfe3e6a3fd086f1a29ec7 8a5be3bd5885d79ad84aadf32bb8c1a67bd43c19 slaren slarengh@gmail.com 2023-12-16T18:58:46+01:00 GitHub noreply@github.com 2023-12-16T18:58:46+01:00 lora : add support for non-llama models (#3333) 8a5be3bd5885d79ad84aadf32bb8c1a67bd43c19 88ae8952b65cbf32eb1f5703681ea592e510e570 Jared Van Bortel jared@nomic.ai 2023-12-15T22:16:15-05:00 GitHub noreply@github.com 2023-12-15T22:16:15-05:00 llama : sanity checks for access to logits (#4274) 88ae8952b65cbf32eb1f5703681ea592e510e570 ee4725a686643669a8587142fa068cbf29de3ce2 ShadovvBeast ShadovvBeast@gmail.com 2023-12-15T13:49:01+02:00 GitHub noreply@github.com 2023-12-15T13:49:01+02:00 server : add optional API Key Authentication example (#4441) ee4725a686643669a8587142fa068cbf29de3ce2 6744dbe924a317e3e2a5a2a4a2037061b2223449 slaren slarengh@gmail.com 2023-12-15T12:45:50+01:00 GitHub noreply@github.com 2023-12-15T12:45:50+01:00 ggml : group mul_mat_id rows by matrix (cpu only) (#4480) 6744dbe924a317e3e2a5a2a4a2037061b2223449 cafcd4f89500b8afef722cdb08088eceb8a22572 slaren slarengh@gmail.com 2023-12-14T20:05:21+01:00 GitHub noreply@github.com 2023-12-14T20:05:21+01:00 ggml : use ggml_row_size where possible (#4472) cafcd4f89500b8afef722cdb08088eceb8a22572 c50e40016394f124b97ce39da48148b1f6c01833 slaren slarengh@gmail.com 2023-12-14T16:52:08+01:00 GitHub noreply@github.com 2023-12-14T16:52:08+01:00 ggml : remove n_dims from ggml_tensor (#4469) c50e40016394f124b97ce39da48148b1f6c01833 20a68a7030ee06e8eb7eb8e24ae4ac52dc17803f wonjun Jang strutive07@gmail.com 2023-12-14T21:44:49+09:00 GitHub noreply@github.com 2023-12-14T14:44:49+02:00 py : add protobuf dependency (#4466) 20a68a7030ee06e8eb7eb8e24ae4ac52dc17803f 55e87c3749cb4985c3b316984d40e00e4df4a5d0 LostRuins 39025047+LostRuins@users.noreply.github.com 2023-12-14T20:13:33+08:00 GitHub noreply@github.com 2023-12-14T14:13:33+02:00 ggml : add ggml_row_size() (fixes llama out of space) (#4461) 55e87c3749cb4985c3b316984d40e00e4df4a5d0 873637afc7924f435ac44c067630a28e82eefa7b Georgi Gerganov ggerganov@gmail.com 2023-12-14T10:35:29+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-14T10:35:29+02:00 ggml : fix OpenCL broadcast requirement for ggml_mul (close #4453) 873637afc7924f435ac44c067630a28e82eefa7b 0353a1840134b24b07ab61fd4490192f28c4db6b wonjun Jang strutive07@gmail.com 2023-12-14T17:09:34+09:00 GitHub noreply@github.com 2023-12-14T10:09:34+02:00 convert : support loading vocab from fast tokenizer config (#3633) 0353a1840134b24b07ab61fd4490192f28c4db6b 948ff137ec37f1ec74c02905917fa0afc9b97514 BarfingLemurs 128182951+BarfingLemurs@users.noreply.github.com 2023-12-14T02:38:49-05:00 GitHub noreply@github.com 2023-12-14T09:38:49+02:00 readme : update supported model list (#4457) 948ff137ec37f1ec74c02905917fa0afc9b97514 4d98d9a65665eee3838cef936641f640e3f5b649 shibe2 shibe@tuta.io 2023-12-13T23:57:15+04:00 GitHub noreply@github.com 2023-12-13T21:57:15+02:00 server : fix handling of characters that span multiple tokens when streaming (#4446) 4d98d9a65665eee3838cef936641f640e3f5b649 70f806b821f603cafb6f634c93a6729dc21bb354 Georgi Gerganov ggerganov@gmail.com 2023-12-13T21:54:54+02:00 GitHub noreply@github.com 2023-12-13T21:54:54+02:00 sync : ggml (SD ops, tests, kernels) (#4444) 70f806b821f603cafb6f634c93a6729dc21bb354 9fb13f95840c722ad419f390dc8a9c86080a3700 Jared Van Bortel jared@nomic.ai 2023-12-13T12:10:10-05:00 GitHub noreply@github.com 2023-12-13T12:10:10-05:00 build : detect host compiler and cuda compiler separately (#4414) 9fb13f95840c722ad419f390dc8a9c86080a3700 113f9942fc73a262c85e9dcf7c2ea7336250bba0 Siwen Yu yusiwen@gmail.com 2023-12-13T20:50:14+08:00 GitHub noreply@github.com 2023-12-13T14:50:14+02:00 common : add `--version` option to show build info in CLI (#4433) 113f9942fc73a262c85e9dcf7c2ea7336250bba0 799a1cb13b0b1b560ab0ceff485caed68faa8f1f Georgi Gerganov ggerganov@gmail.com 2023-12-13T14:05:38+02:00 GitHub noreply@github.com 2023-12-13T14:05:38+02:00 readme : update hot topics 799a1cb13b0b1b560ab0ceff485caed68faa8f1f fecac45658a99eddc4d6e36ba0310ca8f87a77f0 slaren slarengh@gmail.com 2023-12-13T13:04:25+01:00 GitHub noreply@github.com 2023-12-13T14:04:25+02:00 llama : add Mixtral support (#4406) fecac45658a99eddc4d6e36ba0310ca8f87a77f0 9494d7c4774ab745490b5a19570ff7747a194143 kalomaze 66376113+kalomaze@users.noreply.github.com 2023-12-12T04:12:35-06:00 GitHub noreply@github.com 2023-12-12T12:12:35+02:00 server : tweak default sampling parameters (#4367) 9494d7c4774ab745490b5a19570ff7747a194143 6138963fb232cbae70c9d181db0ba125708f473d Richard Kiss him@richardkiss.com 2023-12-12T01:53:36-08:00 GitHub noreply@github.com 2023-12-12T11:53:36+02:00 english : use `typos` to fix comments and logs (#4354) 6138963fb232cbae70c9d181db0ba125708f473d 6391817cd19a4507c6c941a1fd08756268662b2d Jared Van Bortel jared@nomic.ai 2023-12-12T04:27:26-05:00 GitHub noreply@github.com 2023-12-12T11:27:26+02:00 build : target Windows 8 for standard mingw-w64 (#4405) 6391817cd19a4507c6c941a1fd08756268662b2d d9d4cfef64ea416dd66632173787d03ffb180cc7 crasm crasm@git.vczf.us 2023-12-12T04:25:57-05:00 GitHub noreply@github.com 2023-12-12T11:25:57+02:00 llama : document logits_all deprecation (#4418) d9d4cfef64ea416dd66632173787d03ffb180cc7 41a11aaf99feff4901e4c8dc48ad00766c5da4e9 Vladimir Zorin vladimir@deviant.guru 2023-12-12T11:25:29+02:00 GitHub noreply@github.com 2023-12-12T11:25:29+02:00 server : fix local model name in server (#4420) 41a11aaf99feff4901e4c8dc48ad00766c5da4e9 8a7b2fa528f130631a5f43648481596ab320ed5a Taikono-Himazin kazu@po.harenet.ne.jp 2023-12-12T18:24:32+09:00 GitHub noreply@github.com 2023-12-12T11:24:32+02:00 ggml : increased GGML_MAX_PARAMS to allow finetuning of 70b models (#4424) 8a7b2fa528f130631a5f43648481596ab320ed5a e18f7345a300920e234f732077bda660cc6cda9c Yueh-Po Peng 94939112+y10ab1@users.noreply.github.com 2023-12-11T06:27:38+08:00 GitHub noreply@github.com 2023-12-10T23:27:38+01:00 Update README.md (#4388) e18f7345a300920e234f732077bda660cc6cda9c fe680e3d1080a765e5d3150ffd7bab189742898d Xiang (Kevin) Li kevinli020508@gmail.com 2023-12-09T16:29:27-05:00 GitHub noreply@github.com 2023-12-09T23:29:27+02:00 grammar : revert the replacement of llama_token_to_piece with id_to_token (#4396) fe680e3d1080a765e5d3150ffd7bab189742898d bcc0eb4591bec5ec02fad3f2bdcb1b265052ea56 Georgi Gerganov ggerganov@gmail.com 2023-12-07T22:26:54+02:00 GitHub noreply@github.com 2023-12-07T22:26:54+02:00 sync : ggml (new ops, tests, backend, etc.) (#4359) bcc0eb4591bec5ec02fad3f2bdcb1b265052ea56 81bc9214a389362010f7a57f4cbc30e5f83a2d28 Georgi Gerganov ggerganov@gmail.com 2023-12-07T13:03:17+02:00 GitHub noreply@github.com 2023-12-07T13:03:17+02:00 llama : per-layer KV cache + quantum K cache (#4309) 81bc9214a389362010f7a57f4cbc30e5f83a2d28 05cd6e5036d72d0930de4d8f6be7bce09e8dda24 Hongyu Ouyang 96765450+casavaca@users.noreply.github.com 2023-12-07T02:25:22-08:00 GitHub noreply@github.com 2023-12-07T12:25:22+02:00 train : fix #4227 (double free in examples/train-text-from-scratch/train-text-from-scratch.cpp) (#4351) 05cd6e5036d72d0930de4d8f6be7bce09e8dda24 caa9249217c5fd524b900add5ddcbeaa20cbcb12 Georgi Gerganov ggerganov@gmail.com 2023-12-06T20:21:59+02:00 GitHub noreply@github.com 2023-12-06T20:21:59+02:00 server : recognize cache_prompt parameter in OAI API (#4347) caa9249217c5fd524b900add5ddcbeaa20cbcb12 da5eaef1f34d0a1f584cd4a092e7691ea46a9d91 Georgi Gerganov ggerganov@gmail.com 2023-12-06T10:41:03+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-06T10:41:03+02:00 common : fix compile warning da5eaef1f34d0a1f584cd4a092e7691ea46a9d91 5f6e0c0dff1e7a89331e6b25eca9a9fd71324069 stduhpf stephduh@live.fr 2023-12-06T09:08:17+01:00 GitHub noreply@github.com 2023-12-06T10:08:17+02:00 speculative : support `--color` (#4343) 5f6e0c0dff1e7a89331e6b25eca9a9fd71324069 5aa365d88fdb8fdd430ef3fc141c7a5fd37c3502 Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2023-12-05T10:55:12-10:00 GitHub noreply@github.com 2023-12-05T22:55:12+02:00 grammar : pre-computed pieces + reserve mem + less string copies (#4330) 5aa365d88fdb8fdd430ef3fc141c7a5fd37c3502 52c8bc3cf312e1caf02d37bfb9d9d865cbe33594 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-12-05T10:19:18-07:00 GitHub noreply@github.com 2023-12-05T19:19:18+02:00 llama : allow overriding GGUF metadata when loading model (#4092) 52c8bc3cf312e1caf02d37bfb9d9d865cbe33594 e4b76bbe316ee50fb17d9ac29e654c0edf830eba MaggotHATE clay1326@gmail.com 2023-12-05T15:05:51+05:00 GitHub noreply@github.com 2023-12-05T12:05:51+02:00 sampling : custom samplers order (#4285) e4b76bbe316ee50fb17d9ac29e654c0edf830eba 23b5e12eb5a76489b4c3ee22213a081da68b1809 kchro3 62481661+kchro3@users.noreply.github.com 2023-12-04T23:29:46-08:00 GitHub noreply@github.com 2023-12-05T09:29:46+02:00 swift : revert compiler checks for swift package (#4332) 23b5e12eb5a76489b4c3ee22213a081da68b1809 d208995c6da66f252d4054c1c5a90eb8ccb7a2f7 Daniel Bevenius daniel.bevenius@gmail.com 2023-12-04T17:04:21+01:00 GitHub noreply@github.com 2023-12-04T18:04:21+02:00 simple : update error message for KV cache check (#4324) d208995c6da66f252d4054c1c5a90eb8ccb7a2f7 5c9f90cba1cc6b0a2a7d19ee5dcb73cad6331d30 Miwa / Ensan 63481257+ensan-hcl@users.noreply.github.com 2023-12-05T01:03:49+09:00 GitHub noreply@github.com 2023-12-04T18:03:49+02:00 swift : fix concatenation method to avoid invalid UTF8 stringfication (#4325) 5c9f90cba1cc6b0a2a7d19ee5dcb73cad6331d30 4fa44e84adb4c78e1885694cc3513982d4af2b08 Miwa / Ensan 63481257+ensan-hcl@users.noreply.github.com 2023-12-04T22:43:45+09:00 GitHub noreply@github.com 2023-12-04T15:43:45+02:00 swift : fix prompt tokenization logic (#4321) 4fa44e84adb4c78e1885694cc3513982d4af2b08 fbbc42827b2949b95bcde23ce47bb47d006c895d Ikko Eltociear Ashimine eltociear@gmail.com 2023-12-04T16:57:35+09:00 GitHub noreply@github.com 2023-12-04T09:57:35+02:00 grammar-parser : fix typo (#4318) fbbc42827b2949b95bcde23ce47bb47d006c895d adf3de4f69ff7e44131222f05f9c7447ac0be3cb Georgi Gerganov ggerganov@gmail.com 2023-12-03T15:56:35+02:00 GitHub noreply@github.com 2023-12-03T15:56:35+02:00 ggml : reuse ggml_get_n_tasks() in ggml_graph_plan() (#4308) adf3de4f69ff7e44131222f05f9c7447ac0be3cb 33e171d1e9fc4903f9314b490d77fb8d58331b63 Georgi Gerganov ggerganov@gmail.com 2023-12-03T15:56:22+02:00 GitHub noreply@github.com 2023-12-03T15:56:22+02:00 ggml : fix soft max out-of-bounds access (#4307) 33e171d1e9fc4903f9314b490d77fb8d58331b63 6949b50df56ee58a2d76d45487942cb211c08629 Ed Lee edilee@mozilla.com 2023-12-03T01:10:43-08:00 GitHub noreply@github.com 2023-12-03T11:10:43+02:00 server : fix OpenAI API `stop` field to be optional (#4299) 6949b50df56ee58a2d76d45487942cb211c08629 d7b800b8bc490a221acbd83c575206a907f2f6e2 Rickard Edén rickardeden@gmail.com 2023-12-03T10:03:25+01:00 GitHub noreply@github.com 2023-12-03T11:03:25+02:00 py : add grammar to oai like api (#4294) d7b800b8bc490a221acbd83c575206a907f2f6e2 5a7d3125e7c24f223659b7f0b7aa7736986e92c0 Georgi Gerganov ggerganov@gmail.com 2023-12-03T10:58:16+02:00 GitHub noreply@github.com 2023-12-03T10:58:16+02:00 llama : pad KV cache size (#4280) 5a7d3125e7c24f223659b7f0b7aa7736986e92c0 d5a1cbde60531d02ac74da27ea355182e3a4d516 Georgi Gerganov ggerganov@gmail.com 2023-12-01T20:39:12+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-01T20:39:12+02:00 llama : avoid using "optional" keyword (#4283) d5a1cbde60531d02ac74da27ea355182e3a4d516 b220222a64ce760bfbec9c770f11db3ec6a6abb6 Georgi Gerganov ggerganov@gmail.com 2023-12-01T20:35:03+02:00 Georgi Gerganov ggerganov@gmail.com 2023-12-01T20:35:47+02:00 llama : support optional tensors (#4283) b220222a64ce760bfbec9c770f11db3ec6a6abb6 511f52c334e37033f9c9de07b98fca4abc9470bd Miwa / Ensan 63481257+ensan-hcl@users.noreply.github.com 2023-12-02T03:19:45+09:00 GitHub noreply@github.com 2023-12-01T20:19:45+02:00 swift : fix token_to_piece implementation (#4278) 511f52c334e37033f9c9de07b98fca4abc9470bd 03562f3a86d6706eea9f4fc09b532946c191b34e Jared Van Bortel jared@nomic.ai 2023-12-01T13:18:35-05:00 GitHub noreply@github.com 2023-12-01T20:18:35+02:00 build : enable libstdc++ assertions for debug builds (#4275) 03562f3a86d6706eea9f4fc09b532946c191b34e 37c746d687d877bc11803e96b4dc5f378b83c0a0 CausalLM 148736309+CausalLM@users.noreply.github.com 2023-12-02T02:17:06+08:00 GitHub noreply@github.com 2023-12-01T20:17:06+02:00 llama : support attention bias on LLaMA architecture (#4283) 37c746d687d877bc11803e96b4dc5f378b83c0a0 880f57973b8e0091d0f9f50eb5ab4cd4e31582ca Shijie 821898965@qq.com 2023-12-02T02:16:31+08:00 GitHub noreply@github.com 2023-12-01T20:16:31+02:00 llama : add Qwen support (#4281) 880f57973b8e0091d0f9f50eb5ab4cd4e31582ca 8d6d9f033b8101f929e445cf45b39e1557ca7934 Georgi Gerganov ggerganov@gmail.com 2023-12-01T18:42:11+02:00 GitHub noreply@github.com 2023-12-01T18:42:11+02:00 llama : fix integer overflow during quantization (#4284) 8d6d9f033b8101f929e445cf45b39e1557ca7934 ef47ec18da469423c276b683dd9b5741cee7023e Daniel Bevenius daniel.bevenius@gmail.com 2023-12-01T10:41:56+01:00 GitHub noreply@github.com 2023-12-01T11:41:56+02:00 py : add requirements file for convert-hf-to-gguf.py (#4277) ef47ec18da469423c276b683dd9b5741cee7023e 1d144112c0fbbb4ecc07dbcf4f05a380148bd6de Georgi Gerganov ggerganov@gmail.com 2023-12-01T10:51:24+02:00 GitHub noreply@github.com 2023-12-01T10:51:24+02:00 ggml : add ggml_soft_max_ext (#4256) 1d144112c0fbbb4ecc07dbcf4f05a380148bd6de f43f09366dfd018e4568e23a232aaa8c4f7cfc78 Ziad Ben Hadj-Alouane zied.benhadjalouane@gmail.com 2023-11-30T17:25:49-05:00 GitHub noreply@github.com 2023-12-01T00:25:49+02:00 server : add --log-disable to disable logging to file (#4260) f43f09366dfd018e4568e23a232aaa8c4f7cfc78 d2809a3ba2780e00fce5a6149a7eda09f1c0e906 Ziad Ben Hadj-Alouane zied.benhadjalouane@gmail.com 2023-11-30T17:25:04-05:00 GitHub noreply@github.com 2023-12-01T00:25:04+02:00 server : add single-client multi-prompt support (#4232) d2809a3ba2780e00fce5a6149a7eda09f1c0e906 15f5d96037e597523b721aa39c874d69de2acf85 WillCorticesAI 150854901+WillCorticesAI@users.noreply.github.com 2023-11-30T17:23:44-05:00 GitHub noreply@github.com 2023-12-01T00:23:44+02:00 make : fix Apple clang determination bug (#4272) 15f5d96037e597523b721aa39c874d69de2acf85 33c9892af58b7b161f2a532935dcccff8c8048c6 Jared Van Bortel jared@nomic.ai 2023-11-30T17:23:08-05:00 GitHub noreply@github.com 2023-12-01T00:23:08+02:00 build : fix build info generation and cleanup Makefile (#3920) 33c9892af58b7b161f2a532935dcccff8c8048c6 8efa0f6ebed53c9453e6721da86fb294e5015909 John 78893154+cmp-nct@users.noreply.github.com 2023-11-30T23:11:14+01:00 GitHub noreply@github.com 2023-11-30T23:11:14+01:00 llava : ShareGPT4V compatibility (vision encoder only loading) (#4172) 8efa0f6ebed53c9453e6721da86fb294e5015909 524907aa768a26cbf83d8e2eb30547e2ee1d1b1a Andrew Godfrey AndrewGodfrey@users.noreply.github.com 2023-11-30T13:56:19-08:00 GitHub noreply@github.com 2023-11-30T23:56:19+02:00 main : pass LOG_TEE callback to llama.cpp log (#4033) 524907aa768a26cbf83d8e2eb30547e2ee1d1b1a 3bd2c7ce1b752973cf937482a0333e85d1681e2b vodkaslime 646329483@qq.com 2023-12-01T05:49:21+08:00 GitHub noreply@github.com 2023-11-30T23:49:21+02:00 readme : fix (#4135) 3bd2c7ce1b752973cf937482a0333e85d1681e2b bde629bb53b85886ee0fe83524c1efe2689bc618 Juraj Bednar juraj@bednar.io 2023-11-30T22:46:01+01:00 GitHub noreply@github.com 2023-11-30T23:46:01+02:00 docker : add finetune option (#4211) bde629bb53b85886ee0fe83524c1efe2689bc618 f7f9e06212d44530b3200033286049dbdf84b3d3 Miwa / Ensan 63481257+ensan-hcl@users.noreply.github.com 2023-12-01T06:45:17+09:00 GitHub noreply@github.com 2023-11-30T23:45:17+02:00 batched.swift : update README.md (#4214) f7f9e06212d44530b3200033286049dbdf84b3d3 74daabae6927b99e7333d6126dee35193c418457 Li Tan tanliboy@gmail.com 2023-11-30T13:44:11-08:00 GitHub noreply@github.com 2023-11-30T23:44:11+02:00 cmake : fix the metal file foder path (#4217) 74daabae6927b99e7333d6126dee35193c418457 b18c66ca6eee4fe0465cff5042daf05005dc9ab2 Dawid Wysocki 62249621+TortillaZHawaii@users.noreply.github.com 2023-11-30T22:43:32+01:00 GitHub noreply@github.com 2023-11-30T23:43:32+02:00 readme : fix typo (#4253) b18c66ca6eee4fe0465cff5042daf05005dc9ab2 f4d973cecb7368c985720ba9100ae6abba14806d Daniel Bevenius daniel.bevenius@gmail.com 2023-11-30T22:43:08+01:00 GitHub noreply@github.com 2023-11-30T23:43:08+02:00 llama : fix alignment of general.name in print meta (#4254) f4d973cecb7368c985720ba9100ae6abba14806d 954e22858c5cea1dc03e9172d3879402af2b5990 slaren slarengh@gmail.com 2023-11-30T22:42:23+01:00 GitHub noreply@github.com 2023-11-30T23:42:23+02:00 convert.py : fix llama/llama2 conversion due to vocab_size=-1 (#4258) 954e22858c5cea1dc03e9172d3879402af2b5990 e2bd725f4b39bc5c6234858d158e01248f5ab5bd tarcey cey.tarik@gmail.com 2023-11-30T22:40:23+01:00 GitHub noreply@github.com 2023-11-30T23:40:23+02:00 llama : fix typical sampling (#4261) e2bd725f4b39bc5c6234858d158e01248f5ab5bd 1f5cd83275fabb43f2ae92c30033b384a3eb37b4 rhjdvsgsgks 26178113+rhjdvsgsgks@users.noreply.github.com 2023-11-30T20:50:40Z GitHub noreply@github.com 2023-11-30T22:50:40+02:00 py : fix oai proxy (#3972) 1f5cd83275fabb43f2ae92c30033b384a3eb37b4 4fea3420ee3918d125d74c94d962a6ea82875351 Georgi Gerganov ggerganov@gmail.com 2023-11-29T11:00:17+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-29T11:00:17+02:00 examples : add readme files 4fea3420ee3918d125d74c94d962a6ea82875351 64e64aa2557d97490b2fe1262b313e2f4a1607e3 Peter Sugihara peter@campsh.com 2023-11-28T23:16:34-08:00 GitHub noreply@github.com 2023-11-29T09:16:34+02:00 readme : add FreeChat (#4248) 64e64aa2557d97490b2fe1262b313e2f4a1607e3 8406b0924bf323f37d536dee8b8165c1f3d9d11d Jared Van Bortel jared@nomic.ai 2023-11-28T04:51:11-05:00 GitHub noreply@github.com 2023-11-28T11:51:11+02:00 ggml : restore abort() in GGML_ASSERT (#4242) 8406b0924bf323f37d536dee8b8165c1f3d9d11d b38a16dfcff88d547f78f52d1bea31b84a05aff7 Georgi Gerganov ggerganov@gmail.com 2023-11-28T10:32:03+02:00 GitHub noreply@github.com 2023-11-28T10:32:03+02:00 ggml : re-enable BLAS for CPU when src0 != F32 + remove redundant full offload checks in llama.cpp (#4240) b38a16dfcff88d547f78f52d1bea31b84a05aff7 0dab8cd7cca7e1bc3550dcb4797b9062cdbb1ebd bandoti 141645996+bandoti@users.noreply.github.com 2023-11-27T15:25:42-04:00 GitHub noreply@github.com 2023-11-27T21:25:42+02:00 cmake : fix issue with version info not getting baked into LlamaConfig.cmake (#3970) 0dab8cd7cca7e1bc3550dcb4797b9062cdbb1ebd bb03290c17540768a16000a2b01ee4f22440aba1 Kasumi 90275229+kasumi-1@users.noreply.github.com 2023-11-28T01:39:42+08:00 GitHub noreply@github.com 2023-11-27T19:39:42+02:00 readme : add Amica to UI list (#4230) bb03290c17540768a16000a2b01ee4f22440aba1 f3b269813f6147c5b5cda082e6b45cf04a932e0d Bailey Chittle 39804642+bachittle@users.noreply.github.com 2023-11-27T09:56:52-05:00 GitHub noreply@github.com 2023-11-27T16:56:52+02:00 examples : iOS example with swift ui (#4159) f3b269813f6147c5b5cda082e6b45cf04a932e0d 3e73d31d9cc0232882ce61c64742aff3ecfec416 Jared Van Bortel jared@nomic.ai 2023-11-26T22:58:43-05:00 GitHub noreply@github.com 2023-11-26T22:58:43-05:00 ggml : fix -Warray-bounds warning with gcc (#4231) 3e73d31d9cc0232882ce61c64742aff3ecfec416 9656026b53236ed7328458269c4c798dd50ac8d1 Georgi Gerganov ggerganov@gmail.com 2023-11-26T21:51:46+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-26T21:52:23+02:00 lookahead : support `-n -1` infinite generation 9656026b53236ed7328458269c4c798dd50ac8d1 922754a8d60080e956891f6cee1fb03aa48d57c6 Georgi Gerganov ggerganov@gmail.com 2023-11-26T20:42:51+02:00 GitHub noreply@github.com 2023-11-26T20:42:51+02:00 readme : update hot topics 922754a8d60080e956891f6cee1fb03aa48d57c6 22da05536ff4ad963080773bef1fb839fdab95d3 Georgi Gerganov ggerganov@gmail.com 2023-11-26T20:33:07+02:00 GitHub noreply@github.com 2023-11-26T20:33:07+02:00 lookahead : add example for lookahead decoding (#4207) 22da05536ff4ad963080773bef1fb839fdab95d3 1ddb52ec38f9931925a587f45a23b1c37152c028 Xiao-Yong Jin jinxiaoyong@gmail.com 2023-11-26T02:30:02-06:00 GitHub noreply@github.com 2023-11-26T10:30:02+02:00 metal : fix yarn (#4220) 1ddb52ec38f9931925a587f45a23b1c37152c028 f837c3a992b2b6146936cb120871a8cf9d0e3857 Galunid karolek1231456@gmail.com 2023-11-25T22:45:02+01:00 GitHub noreply@github.com 2023-11-25T22:45:02+01:00 scripts : Use mmap in torch load (#4202) f837c3a992b2b6146936cb120871a8cf9d0e3857 3014b5415d08e3dff961da6eea835b9760a701b8 Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2023-11-25T08:58:23-08:00 GitHub noreply@github.com 2023-11-25T18:58:23+02:00 llama : grammar `reserve` space in `decode_utf8` (#4210) 3014b5415d08e3dff961da6eea835b9760a701b8 04814e718edb13bdf8cca861dc2e5ab4e1995c30 crasm crasm@git.vczf.us 2023-11-25T10:47:07-05:00 GitHub noreply@github.com 2023-11-25T10:47:07-05:00 Update docs for yarn_ext_factor <0.0 as unspecified instead of NaN (#4189) 04814e718edb13bdf8cca861dc2e5ab4e1995c30 af19d3573481d409b3c4e55494810eb1f65a9aae Georgi Gerganov ggerganov@gmail.com 2023-11-25T12:02:13+02:00 GitHub noreply@github.com 2023-11-25T12:02:13+02:00 readme : update hot topics af19d3573481d409b3c4e55494810eb1f65a9aae e9c13ff78114af6fc6a4f27cc8dcdda0f3d389fb Georgi Gerganov ggerganov@gmail.com 2023-11-25T11:29:06+02:00 GitHub noreply@github.com 2023-11-25T11:29:06+02:00 server : OAI API compatibility (#4198) e9c13ff78114af6fc6a4f27cc8dcdda0f3d389fb 8a052c131ed3525313cdb84e5ae4e2b6cf8d2e24 slaren slarengh@gmail.com 2023-11-24T18:10:01+01:00 GitHub noreply@github.com 2023-11-24T18:10:01+01:00 llama : set metal log callback correctly (#4204) 8a052c131ed3525313cdb84e5ae4e2b6cf8d2e24 189d68446e7ef21e8f3af3c0a3d91c35a39aec89 slaren slarengh@gmail.com 2023-11-24T18:04:31+01:00 GitHub noreply@github.com 2023-11-24T18:04:31+01:00 ggml-cuda : support stablelm rope (#4156) 189d68446e7ef21e8f3af3c0a3d91c35a39aec89 2568a4bf548d7392e9c78c008b33b4c11d53fe95 Galunid karolek1231456@gmail.com 2023-11-24T15:02:49+01:00 GitHub noreply@github.com 2023-11-24T15:02:49+01:00 convert : fix tensors using grad in some models (#4173) 2568a4bf548d7392e9c78c008b33b4c11d53fe95 b35f3d0def3efde92ed465d92a267430d957e87d eastriver lee@eastriver.dev 2023-11-24T18:25:10+09:00 GitHub noreply@github.com 2023-11-24T11:25:10+02:00 main.swift : fix eos checking (#4197) b35f3d0def3efde92ed465d92a267430d957e87d 55978ce09b69d3987d17d08d92d8cc27193e0773 Aaryaman Vasishta aaryaman.vasishta@amd.com 2023-11-24T16:52:39+09:00 GitHub noreply@github.com 2023-11-24T09:52:39+02:00 readme : use PATH for Windows ROCm (#4195) 55978ce09b69d3987d17d08d92d8cc27193e0773 6b0a7420d03b9d13cb0e9439a01ce8476d8bf093 Haohui Mai ricetons@gmail.com 2023-11-23T13:56:53-08:00 GitHub noreply@github.com 2023-11-23T22:56:53+01:00 Fix incorrect format strings and uninitialized variables. (#4133) 6b0a7420d03b9d13cb0e9439a01ce8476d8bf093 d103d935c0e75769a6a597f7a64cab72c6cc3e79 Georgi Gerganov ggerganov@gmail.com 2023-11-23T19:07:56+02:00 GitHub noreply@github.com 2023-11-23T19:07:56+02:00 llama : KV cache view API + better KV cache management (#4170) d103d935c0e75769a6a597f7a64cab72c6cc3e79 9d5949f04b1f8b76184818abbd99938c2020803f Georgi Gerganov ggerganov@gmail.com 2023-11-23T13:51:22+02:00 GitHub noreply@github.com 2023-11-23T13:51:22+02:00 readme : update hot topics 9d5949f04b1f8b76184818abbd99938c2020803f ff8238f71d56245f4a6dbea693f4ebd81263464d Daniel Bevenius daniel.bevenius@gmail.com 2023-11-23T12:34:20+01:00 GitHub noreply@github.com 2023-11-23T13:34:20+02:00 examples : fix typo in parallel example doc comment (#4181) ff8238f71d56245f4a6dbea693f4ebd81263464d 8e672efe632bb6a7333964a255c4b96f018b9a65 Georgi Gerganov ggerganov@gmail.com 2023-11-23T11:35:04+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-23T11:35:04+02:00 docs : add llama-star arch idea 8e672efe632bb6a7333964a255c4b96f018b9a65 0b871f1a04ef60e114bbe43004fd9c21114e802d Galunid karolek1231456@gmail.com 2023-11-21T16:22:30+01:00 GitHub noreply@github.com 2023-11-21T16:22:30+01:00 stablelm : simplify + speedup generation (#4153) 0b871f1a04ef60e114bbe43004fd9c21114e802d dfc7cd48b1cc31d759c093e917a18c0efe03d0e8 Galunid karolek1231456@gmail.com 2023-11-20T19:30:00+01:00 GitHub noreply@github.com 2023-11-20T19:30:00+01:00 finetune - update readme to mention llama support only (#4148) dfc7cd48b1cc31d759c093e917a18c0efe03d0e8 881800d1f083c39431cef288347082be516d1c80 Aaryaman Vasishta aaryaman.vasishta@amd.com 2023-11-21T00:02:46+09:00 GitHub noreply@github.com 2023-11-20T17:02:46+02:00 readme : update ROCm Windows instructions (#4122) 881800d1f083c39431cef288347082be516d1c80 f23c0359a32871947169a044eb1dc4dbffd0f405 Seb C 47074056+Sebby37@users.noreply.github.com 2023-11-21T00:26:59+10:30 GitHub noreply@github.com 2023-11-20T14:56:59+01:00 main : Add ChatML functionality to main example (#4046) f23c0359a32871947169a044eb1dc4dbffd0f405 40a34fe8d034bd484efd79ccbb95059ca6308dcb Galunid karolek1231456@gmail.com 2023-11-20T11:35:47+01:00 GitHub noreply@github.com 2023-11-20T11:35:47+01:00 ci : add flake8 to github actions (python linting) (#4129) 40a34fe8d034bd484efd79ccbb95059ca6308dcb dae06c06e5c6232ae2be4d567dd5101e1e96c814 Branden Butler bwtbutler@hotmail.com 2023-11-20T03:50:04-06:00 GitHub noreply@github.com 2023-11-20T11:50:04+02:00 speculative : fix prompt tokenization in speculative example (#4025) dae06c06e5c6232ae2be4d567dd5101e1e96c814 05e8301e4593e2a67b4bae24f093dd12ce5cc7c2 Georgi Gerganov ggerganov@gmail.com 2023-11-19T19:16:07+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-19T19:16:07+02:00 Revert "finetune : add --n-gpu-layers flag info to --help (#4128)" 05e8301e4593e2a67b4bae24f093dd12ce5cc7c2 936c79b2275a8f15f3512e63de615c676904d650 Clark Saben 76020733+csaben@users.noreply.github.com 2023-11-19T11:56:38-05:00 GitHub noreply@github.com 2023-11-19T18:56:38+02:00 finetune : add --n-gpu-layers flag info to --help (#4128) 936c79b2275a8f15f3512e63de615c676904d650 262005ad9ded375e9c544a02c18ef6254fe185a2 SoftwareRenderer 138734813+SoftwareRenderer@users.noreply.github.com 2023-11-19T11:54:10-05:00 GitHub noreply@github.com 2023-11-19T18:54:10+02:00 server : relay error messages (#4131) 262005ad9ded375e9c544a02c18ef6254fe185a2 35985acffaab1eb4ffcbc22c86063bc630f24a89 kchro3 62481661+kchro3@users.noreply.github.com 2023-11-19T08:52:57-08:00 GitHub noreply@github.com 2023-11-19T18:52:57+02:00 common : comma should be semicolon (#4137) 35985acffaab1eb4ffcbc22c86063bc630f24a89 e937066420b79a757bf80e9836eb12b88420a218 Georgi Gerganov ggerganov@gmail.com 2023-11-19T18:50:49+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-19T18:50:49+02:00 gitignore : tokenize e937066420b79a757bf80e9836eb12b88420a218 28a2e6e7d476717881be6eb9e2d3331342cec57b slaren slarengh@gmail.com 2023-11-19T11:10:52+01:00 GitHub noreply@github.com 2023-11-19T11:10:52+01:00 gguf-py : export chat templates (#4125) 28a2e6e7d476717881be6eb9e2d3331342cec57b 0b5c3b04572a05f80163a365070fb377a837ac27 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-18T14:48:17-07:00 GitHub noreply@github.com 2023-11-18T14:48:17-07:00 tokenize example: Respect normal add BOS token behavior (#4126) 0b5c3b04572a05f80163a365070fb377a837ac27 2923f17f6fec049a71186636c3c4d96408856194 Galunid karolek1231456@gmail.com 2023-11-18T21:08:33+01:00 GitHub noreply@github.com 2023-11-18T21:08:33+01:00 scripts : Remove missed baichuan convert script (#4127) 2923f17f6fec049a71186636c3c4d96408856194 bbecf3f415797f812893947998bda4f866fa900e Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-18T08:11:18-07:00 GitHub noreply@github.com 2023-11-18T08:11:18-07:00 Clean up ggml-cuda.cu warnings when compiling with clang (for ROCM) (#4124) bbecf3f415797f812893947998bda4f866fa900e 8e9361089dd31ae9ae59452a8ee409fd51a16371 slaren slarengh@gmail.com 2023-11-17T20:39:11+01:00 GitHub noreply@github.com 2023-11-17T21:39:11+02:00 llama : increase max nodes (#4115) 8e9361089dd31ae9ae59452a8ee409fd51a16371 5ad387e994dde77a47ec547a4a65f7611dc325f4 Roger Meier r.meier@siemens.com 2023-11-17T17:11:23+01:00 GitHub noreply@github.com 2023-11-17T18:11:23+02:00 build : support ppc64le build for make and CMake (#3963) 5ad387e994dde77a47ec547a4a65f7611dc325f4 2fa02b4b3d86182381311c98b75065ee1b7c2930 Georgi Gerganov ggerganov@gmail.com 2023-11-17T18:01:38+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-17T18:01:38+02:00 tokenize : fix trailing whitespace 2fa02b4b3d86182381311c98b75065ee1b7c2930 2ab0707acbf3a3ca9e5bc5959c7920c22eba2257 zakkor edward.partenie@gmail.com 2023-11-17T17:36:44+02:00 GitHub noreply@github.com 2023-11-17T17:36:44+02:00 examples : add tokenize (#4039) 2ab0707acbf3a3ca9e5bc5959c7920c22eba2257 11173c92d6eaa2bd1308c2389f44f838480836ac Don Mahurin dmahurin@users.noreply.github.com 2023-11-17T07:32:34-08:00 GitHub noreply@github.com 2023-11-17T17:32:34+02:00 convert : use 'model' value if it exists. This allows karpathy/tinyllamas to load (#4089) 11173c92d6eaa2bd1308c2389f44f838480836ac 9e87ef60e18d69338c5efea314aa7e718bf2040a John 78893154+cmp-nct@users.noreply.github.com 2023-11-17T16:24:30+01:00 GitHub noreply@github.com 2023-11-17T17:24:30+02:00 py : Falcon HF compatibility (#4104) 9e87ef60e18d69338c5efea314aa7e718bf2040a c7cce1246e248124117ae5bc058923e3ade95f11 Jannis Schönleber joennlae@gmail.com 2023-11-17T16:24:07+01:00 GitHub noreply@github.com 2023-11-17T17:24:07+02:00 common : improve yaml log escaping (#4080) c7cce1246e248124117ae5bc058923e3ade95f11 f7d5e975424ff0eea55ca5a9181ac8e15553c1fc Huawei Lin huaweilin.cs@gmail.com 2023-11-17T10:22:56-05:00 GitHub noreply@github.com 2023-11-17T17:22:56+02:00 llava : fix compilation warning that fread return value is not used (#4069) f7d5e975424ff0eea55ca5a9181ac8e15553c1fc ba4cf5c0bf37a729d29e899dadf14541cddd23d4 Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2023-11-17T16:20:53+01:00 GitHub noreply@github.com 2023-11-17T17:20:53+02:00 py : remove superfluous import statements (#4076) ba4cf5c0bf37a729d29e899dadf14541cddd23d4 e85bb1a8e736228a1f0d965777de5f77f22834b8 Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2023-11-17T16:19:16+01:00 GitHub noreply@github.com 2023-11-17T17:19:16+02:00 train : move number of gpu layers argument parsing to common/train.cpp (#4074) e85bb1a8e736228a1f0d965777de5f77f22834b8 3e916a07ac093045d88ef0c4fa78647ae0efc010 slaren slarengh@gmail.com 2023-11-17T16:17:37+01:00 GitHub noreply@github.com 2023-11-17T17:17:37+02:00 llama : add functions to get the model's metadata (#4013) 3e916a07ac093045d88ef0c4fa78647ae0efc010 947f64f1630bb8b0b363a3bb5e29e11425312d57 gwjr 502526+gwjr@users.noreply.github.com 2023-11-17T14:48:19Z GitHub noreply@github.com 2023-11-17T16:48:19+02:00 finetune : speed-up ggml_compute_forward_out_prod_f32 via BLAS (#4079) 947f64f1630bb8b0b363a3bb5e29e11425312d57 b83e149ec6264d078e6a47412e7347bf5c2bfcc9 Andrew Godfrey AndrewGodfrey@users.noreply.github.com 2023-11-17T02:23:11-08:00 GitHub noreply@github.com 2023-11-17T11:23:11+01:00 finetune : zero the loraB initial vectors (#4082) b83e149ec6264d078e6a47412e7347bf5c2bfcc9 4f447a48339977073a1af4f33ae873465ff64994 Andrew Godfrey AndrewGodfrey@users.noreply.github.com 2023-11-17T00:01:15-08:00 GitHub noreply@github.com 2023-11-17T10:01:15+02:00 cuda : get_row_rounding F32 (#4095) 4f447a48339977073a1af4f33ae873465ff64994 91f6499393d2d999331fbfdba47a7f8b9f913f0d Georgi Gerganov ggerganov@gmail.com 2023-11-17T10:00:15+02:00 GitHub noreply@github.com 2023-11-17T10:00:15+02:00 llama : fix data units (#4101) 91f6499393d2d999331fbfdba47a7f8b9f913f0d 8da46278e1a57107591653275f8e03a281de94f0 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-16T19:14:37-07:00 GitHub noreply@github.com 2023-11-16T19:14:37-07:00 Respect tokenizer.ggml.add_bos_token value when tokenizing (#4040) 8da46278e1a57107591653275f8e03a281de94f0 a6fc554e268634494f33b0de76f9dde650dd292f texmex76 40733439+texmex76@users.noreply.github.com 2023-11-16T16:01:48+01:00 GitHub noreply@github.com 2023-11-16T17:01:48+02:00 gguf : fix potential infinite loops while parsing (#4100) a6fc554e268634494f33b0de76f9dde650dd292f 1cf2850d52bb027aa215e039ed9a0c61beeef8d3 Jared Van Bortel jared@nomic.ai 2023-11-15T11:34:47-05:00 GitHub noreply@github.com 2023-11-15T11:34:47-05:00 llama : restore prefix space in llama tokenizer (#4081) 1cf2850d52bb027aa215e039ed9a0c61beeef8d3 6bb4908a17150b49373b5f977685b2e180a04f6f slaren slarengh@gmail.com 2023-11-15T13:58:13+01:00 GitHub noreply@github.com 2023-11-15T14:58:13+02:00 ggml-cuda : increase max graph size (#4084) 6bb4908a17150b49373b5f977685b2e180a04f6f 36eed0c42c5b0bf74af81fb9243d262014f9382f Michael Potter NanoTekGuy@Gmail.com 2023-11-14T09:34:41-08:00 GitHub noreply@github.com 2023-11-14T12:34:41-05:00 Fix MacOS Sonoma model quantization (#4052) 36eed0c42c5b0bf74af81fb9243d262014f9382f b46d12f86d56bef3dc8b596dfb3d22f3b08102be Galunid karolek1231456@gmail.com 2023-11-14T11:17:12+01:00 GitHub noreply@github.com 2023-11-14T11:17:12+01:00 stablelm : StableLM support (#3586) b46d12f86d56bef3dc8b596dfb3d22f3b08102be bd90eca237b498dd106d315dcb9ad3e6fae3906f afrideva 95653597+afrideva@users.noreply.github.com 2023-11-13T17:03:40-08:00 GitHub noreply@github.com 2023-11-13T18:03:40-07:00 convert.py: also look for plain model.safetensors (#4043) bd90eca237b498dd106d315dcb9ad3e6fae3906f 3d68f364f15778dc326f5024f2e5af1ad6dfddef M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-11-13T18:20:52+03:00 GitHub noreply@github.com 2023-11-13T18:20:52+03:00 llava : fix regression for square images in #3613 (#4056) 3d68f364f15778dc326f5024f2e5af1ad6dfddef c049b37d7baf558944501705b91ac89b26ee3e41 Georgi Gerganov ggerganov@gmail.com 2023-11-13T16:55:52+02:00 GitHub noreply@github.com 2023-11-13T16:55:52+02:00 ggml : sync (im2col, GPU conv, 32-bit arm compat) (#4060) c049b37d7baf558944501705b91ac89b26ee3e41 4760e7cc0b68570d58f55e8dda469805d1759d0d Georgi Gerganov ggerganov@gmail.com 2023-11-13T14:18:08+02:00 GitHub noreply@github.com 2023-11-13T14:18:08+02:00 readme : update hot topics 4760e7cc0b68570d58f55e8dda469805d1759d0d bb50a792ec2a49944470c82694fa364345e95170 Georgi Gerganov ggerganov@gmail.com 2023-11-13T14:16:23+02:00 GitHub noreply@github.com 2023-11-13T14:16:23+02:00 sync : ggml (backend v2) (#3912) bb50a792ec2a49944470c82694fa364345e95170 21fd874c8d2a14dea2d56724e4357c0824aee6a8 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-13T01:58:15-07:00 GitHub noreply@github.com 2023-11-13T01:58:15-07:00 Add ReLU and SQR CUDA ops to (partially) fix Persimmon offloading (#4041) 21fd874c8d2a14dea2d56724e4357c0824aee6a8 532dd74e38c29e16ea1cfc4e7eedb4f2fab3f3cd Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-12T16:39:37-07:00 GitHub noreply@github.com 2023-11-12T16:39:37-07:00 gguf-py: gguf_writer: Use bytearray to build metadata (#4051) 532dd74e38c29e16ea1cfc4e7eedb4f2fab3f3cd e86fc56f7521ca4b18d1d9939e82abd40c2f1c01 Richard Kiss him@richardkiss.com 2023-11-11T22:04:58-08:00 GitHub noreply@github.com 2023-11-11T23:04:58-07:00 Fix some documentation typos/grammar mistakes (#4032) e86fc56f7521ca4b18d1d9939e82abd40c2f1c01 d96ca7ded77df764db797b68b4a29e34c5b56285 M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-11-11T18:35:31+03:00 GitHub noreply@github.com 2023-11-11T08:35:31-07:00 Fix gguf-convert-endian script (#4037) d96ca7ded77df764db797b68b4a29e34c5b56285 34b0a082074b073eb14c2bd93c0c070e20ddcd16 Alexey Parfenov zxed@alkatrazstudio.net 2023-11-11T05:48:21Z GitHub noreply@github.com 2023-11-10T23:48:21-06:00 server : fix crash when prompt exceeds context size (#3996) 34b0a082074b073eb14c2bd93c0c070e20ddcd16 4a4fd3eefad5bd17ab6bcd8e2181b4f62eae76cf Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-10T22:04:50-07:00 GitHub noreply@github.com 2023-11-11T08:04:50+03:00 gguf-py: Refactor and allow reading/modifying existing GGUF files (#3981) 4a4fd3eefad5bd17ab6bcd8e2181b4f62eae76cf df9d1293defe783f42bc83af732d3c670552c541 Jhen-Jie Hong iainst0409@gmail.com 2023-11-11T06:49:33+08:00 GitHub noreply@github.com 2023-11-10T16:49:33-06:00 server : allow continue edit on completion mode (#3950) df9d1293defe783f42bc83af732d3c670552c541 a75fa576abba9d37f463580c379e4bbf1e1ad03c Galunid karolek1231456@gmail.com 2023-11-10T14:24:54+01:00 GitHub noreply@github.com 2023-11-10T14:24:54+01:00 Unbreak persimmon after #3837 (#4010) a75fa576abba9d37f463580c379e4bbf1e1ad03c 57ad015dc3011b046ed5a23186c86ea55f987c54 Galunid karolek1231456@gmail.com 2023-11-09T11:09:29+01:00 GitHub noreply@github.com 2023-11-09T11:09:29+01:00 scripts: Generalize convert scripts (#3838) 57ad015dc3011b046ed5a23186c86ea55f987c54 875fb42871a0f5a88fbe31a0b5edd697b84038e4 Mihai mihai.chirculescu@yahoo.com 2023-11-09T04:00:34+02:00 GitHub noreply@github.com 2023-11-08T20:00:34-06:00 server : add min_p param (#3877) 875fb42871a0f5a88fbe31a0b5edd697b84038e4 0a7c980b6f94a049cb804573df2d8092a34df8e4 slaren slarengh@gmail.com 2023-11-08T13:15:14+01:00 GitHub noreply@github.com 2023-11-08T13:15:14+01:00 ggml-alloc : fix backend assignments of views (#3982) 0a7c980b6f94a049cb804573df2d8092a34df8e4 413503d4b92500d82b002d03c580a71a54747138 Jared Van Bortel cebtenzzre@gmail.com 2023-11-07T12:43:04-05:00 GitHub noreply@github.com 2023-11-07T12:43:04-05:00 gguf : track writer state, free unneeded tensors, cleanup (#3871) 413503d4b92500d82b002d03c580a71a54747138 e9c1cecb9d7d743d30b4a29ecd56a411437def0a Georgi Gerganov ggerganov@gmail.com 2023-11-07T19:25:32+02:00 GitHub noreply@github.com 2023-11-07T20:25:32+03:00 make : do not add linker flags when compiling static llava lib (#3977) e9c1cecb9d7d743d30b4a29ecd56a411437def0a 54b4df8886103b436a4bb3b60f4d84824f9e8868 xaedes xaedes@gmail.com 2023-11-07T09:04:51+01:00 GitHub noreply@github.com 2023-11-07T10:04:51+02:00 ggml : fix backward rope after YaRN (#3974) 54b4df8886103b436a4bb3b60f4d84824f9e8868 46876d2a2c92e60579dc732cdb8cbd243b06f317 Matthew Tejo matthew.tejo@gmail.com 2023-11-06T23:43:59-08:00 GitHub noreply@github.com 2023-11-07T10:43:59+03:00 Use params when loading models in llava-cli (#3976) 46876d2a2c92e60579dc732cdb8cbd243b06f317 381efbf480959bb6d1e247a8b0c2328f22e350f8 Meng Zhang meng@tabbyml.com 2023-11-06T22:49:08-08:00 GitHub noreply@github.com 2023-11-07T08:49:08+02:00 cuda : supports running on CPU for GGML_USE_CUBLAS=ON build (#3946) 381efbf480959bb6d1e247a8b0c2328f22e350f8 2833a6f63c1b87c7f4ac574bcf7a15a2f3bf3ede Damian Stewart d@damianstewart.com 2023-11-06T22:36:23+01:00 GitHub noreply@github.com 2023-11-07T00:36:23+03:00 llava : expose as a shared library for downstream projects (#3613) 2833a6f63c1b87c7f4ac574bcf7a15a2f3bf3ede d9ccce2e339ca0396560d18b8637f2c848d72a08 slaren slarengh@gmail.com 2023-11-05T18:45:16+01:00 GitHub noreply@github.com 2023-11-05T18:45:16+01:00 ggml-cuda : fix f16 mul mat (#3961) d9ccce2e339ca0396560d18b8637f2c848d72a08 bb60fd0bf6bb270744d86dd45b3a95af01b7de45 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-05T10:06:06-07:00 GitHub noreply@github.com 2023-11-05T10:06:06-07:00 Allow common process_escapes to handle \x sequences (#3928) bb60fd0bf6bb270744d86dd45b3a95af01b7de45 132d25b8a62ea084447e0014a0112c1b371fb3f8 Thái Hoàng Tâm 75922889+RoyalHeart@users.noreply.github.com 2023-11-05T23:15:27+07:00 GitHub noreply@github.com 2023-11-05T18:15:27+02:00 server : fix typo for --alias shortcut from -m to -a (#3958) 132d25b8a62ea084447e0014a0112c1b371fb3f8 3d48f42efcd05381221654376e9f6f69d76af739 Jared Van Bortel cebtenzzre@gmail.com 2023-11-05T10:08:57-05:00 GitHub noreply@github.com 2023-11-05T10:08:57-05:00 cuda : fix disabling device with --tensor-split 1,0 (#3951) 3d48f42efcd05381221654376e9f6f69d76af739 c41ea36eaa3548776de4cb3d5d49b925cd3fc0f2 Meng Zhang meng@tabbyml.com 2023-11-05T04:40:08-08:00 GitHub noreply@github.com 2023-11-05T14:40:08+02:00 llama : mark LLM_ARCH_STARCODER as full offload supported (#3945) c41ea36eaa3548776de4cb3d5d49b925cd3fc0f2 a7fac013cf1cc7bbc0160a226aa2412e9f22e78a Eve 139727413+netrunnereve@users.noreply.github.com 2023-11-05T08:03:09Z GitHub noreply@github.com 2023-11-05T10:03:09+02:00 cmake : MSVC instruction detection (fixed up #809) (#3923) a7fac013cf1cc7bbc0160a226aa2412e9f22e78a 48ade94538fa509465d71023e49d07aab0ec8cd5 Eve 139727413+netrunnereve@users.noreply.github.com 2023-11-05T07:46:44Z GitHub noreply@github.com 2023-11-05T09:46:44+02:00 ci : use intel sde when ci cpu doesn't support avx512 (#3949) 48ade94538fa509465d71023e49d07aab0ec8cd5 f28af0d81aa1010afa5de74cf627dcb04bea3157 slaren slarengh@gmail.com 2023-11-05T08:12:13+01:00 GitHub noreply@github.com 2023-11-05T09:12:13+02:00 cuda : revert CUDA pool stuff (#3944) f28af0d81aa1010afa5de74cf627dcb04bea3157 d9b33fe95bd257b36c84ee5769cc048230067d6f Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-04T16:20:34-06:00 GitHub noreply@github.com 2023-11-04T16:20:34-06:00 gguf-py: Support 01.AI Yi models (#3943) d9b33fe95bd257b36c84ee5769cc048230067d6f 5ba37461711095c0284233dbd14f0d9010cdbf56 Peter Sugihara peter@campsh.com 2023-11-03T12:18:18-07:00 GitHub noreply@github.com 2023-11-03T21:18:18+02:00 metal : round up to 16 to fix MTLDebugComputeCommandEncoder assertion (#3938) 5ba37461711095c0284233dbd14f0d9010cdbf56 abb77e7319aabc0b5cfb7c22da690a692489b6b7 Xiao-Yong Jin jinxiaoyong@gmail.com 2023-11-03T13:00:31-05:00 GitHub noreply@github.com 2023-11-03T14:00:31-04:00 ggml-metal: fix yarn rope (#3937) abb77e7319aabc0b5cfb7c22da690a692489b6b7 8f961abdc4e134c83bf8c2ad618ab256b4cae0f9 slaren slarengh@gmail.com 2023-11-03T12:13:09+01:00 GitHub noreply@github.com 2023-11-03T12:13:09+01:00 ggml-cuda : move row numbers to x grid dim in mmv kernels (#3921) 8f961abdc4e134c83bf8c2ad618ab256b4cae0f9 05816027d649f977468fc804cdb54e99eac246d1 Georgi Gerganov ggerganov@gmail.com 2023-11-03T09:41:17+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-03T09:41:56+02:00 speculative : change default p_accept to 0.5 + CLI args (#3919) 05816027d649f977468fc804cdb54e99eac246d1 3fdbe6b66b7b5c6ad3b2f245cbad1517c27ff776 Georgi Gerganov ggerganov@gmail.com 2023-11-03T09:24:00+02:00 GitHub noreply@github.com 2023-11-03T09:24:00+02:00 common : YAYF (yet another YARN fix) (#3925) 3fdbe6b66b7b5c6ad3b2f245cbad1517c27ff776 629f917cd6b96ba1274c49a8aab163b1b189229d cebtenzzre cebtenzzre@gmail.com 2023-11-03T02:31:58-04:00 GitHub noreply@github.com 2023-11-03T08:31:58+02:00 llama : change yarn_ext_factor placeholder to -1 (#3922) 629f917cd6b96ba1274c49a8aab163b1b189229d 51b2fc11f7f605fff49725a4540e9a6ef7b51b70 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-11-02T13:58:22-06:00 GitHub noreply@github.com 2023-11-02T21:58:22+02:00 cuda : add ROCM aliases for CUDA pool stuff (#3918) 51b2fc11f7f605fff49725a4540e9a6ef7b51b70 224e7d5b14cbabab7ae45c64db2cfde979c8455d Andrei abetlen@gmail.com 2023-11-02T15:40:31-04:00 GitHub noreply@github.com 2023-11-02T21:40:31+02:00 cmake : fix relative path to git submodule index (#3915) 224e7d5b14cbabab7ae45c64db2cfde979c8455d c7743fe1c1cbda5a886362aa371480360580fdf0 Georgi Gerganov ggerganov@gmail.com 2023-11-02T20:44:12+02:00 GitHub noreply@github.com 2023-11-02T20:44:12+02:00 readme : add notice about #3912 c7743fe1c1cbda5a886362aa371480360580fdf0 d6069051de7165a4e06662c89257f5d2905bb156 Georgi Gerganov ggerganov@gmail.com 2023-11-02T20:32:11+02:00 GitHub noreply@github.com 2023-11-02T20:32:11+02:00 cuda : fix const ptrs warning causing ROCm build issues (#3913) d6069051de7165a4e06662c89257f5d2905bb156 4ff1046d75e64f0e556d8dcd930ea25c23eb8b18 Oleksii Maryshchenko oleksii.maryshchenko@gmail.com 2023-11-02T18:10:39+01:00 GitHub noreply@github.com 2023-11-02T19:10:39+02:00 cuda : use CUDA memory pool with async memory allocation/deallocation when available (#3903) 4ff1046d75e64f0e556d8dcd930ea25c23eb8b18 21958bb393a654591ed26f339791b752d58f5c8b Georgi Gerganov ggerganov@gmail.com 2023-11-02T16:22:30+02:00 GitHub noreply@github.com 2023-11-02T16:22:30+02:00 gguf : print error for GGUFv1 files (#3908) 21958bb393a654591ed26f339791b752d58f5c8b 2756c4fbffab097736d5116007872d86456a544a slaren slarengh@gmail.com 2023-11-02T13:10:33+01:00 GitHub noreply@github.com 2023-11-02T14:10:33+02:00 cmake : disable LLAMA_NATIVE by default (#3906) 2756c4fbffab097736d5116007872d86456a544a 1efae9b7dca2a5cc5aa21c1997b538022964ea19 Georgi Gerganov ggerganov@gmail.com 2023-11-02T11:20:21+02:00 GitHub noreply@github.com 2023-11-02T11:20:21+02:00 gguf : remove special-case code for GGUFv1 (#3901) 1efae9b7dca2a5cc5aa21c1997b538022964ea19 b12fa0d1c13596869c512f49a526b979c94787cc Georgi Gerganov ggerganov@gmail.com 2023-11-02T09:54:18+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-02T09:54:44+02:00 llm : prevent from 1-D tensors being GPU split (#3697) b12fa0d1c13596869c512f49a526b979c94787cc 4d719a6d4e74b9a98e75f826f865f3153717d54b cebtenzzre cebtenzzre@gmail.com 2023-11-02T02:50:16-04:00 GitHub noreply@github.com 2023-11-02T08:50:16+02:00 build : link against build info instead of compiling against it (#3879) 4d719a6d4e74b9a98e75f826f865f3153717d54b 183b3fac6c28e65d23ac0230c1dd6fb84bf0154d Georgi Gerganov ggerganov@gmail.com 2023-11-02T08:35:10+02:00 GitHub noreply@github.com 2023-11-02T08:35:10+02:00 cuda : check if this fixes Pascal card regression (#3882) 183b3fac6c28e65d23ac0230c1dd6fb84bf0154d 2fffa0d61fa10e4b466e78cabcc6a4e16717b580 Georgi Gerganov ggerganov@gmail.com 2023-11-02T08:33:37+02:00 GitHub noreply@github.com 2023-11-02T08:33:37+02:00 metal : fix build errors and kernel sig after #2268 (#3898) 2fffa0d61fa10e4b466e78cabcc6a4e16717b580 0eb332a10f3f14a3746c391bf80ff5e7bdf29d5d cebtenzzre cebtenzzre@gmail.com 2023-11-02T01:49:44-04:00 GitHub noreply@github.com 2023-11-02T07:49:44+02:00 cuda : fix RoPE after #2268 (#3897) 0eb332a10f3f14a3746c391bf80ff5e7bdf29d5d d02e98cde035d91ed8032ab943d1d504fe9da394 cebtenzzre cebtenzzre@gmail.com 2023-11-01T19:29:14-04:00 GitHub noreply@github.com 2023-11-01T19:29:14-04:00 llama : fix llama_context_default_params after #2268 (#3893) d02e98cde035d91ed8032ab943d1d504fe9da394 898aeca90a9bb992f506234cf3b8b7f7fa28a1df slaren slarengh@gmail.com 2023-11-01T23:10:09+01:00 GitHub noreply@github.com 2023-11-01T23:10:09+01:00 ggml-cuda : compute ptrs for cublasGemmBatchedEx in a kernel (#3891) 898aeca90a9bb992f506234cf3b8b7f7fa28a1df c43c2da8afacaddfe51c09b21dbd9922cd0ea46b cebtenzzre cebtenzzre@gmail.com 2023-11-01T18:04:33-04:00 GitHub noreply@github.com 2023-11-01T18:04:33-04:00 llama : implement YaRN RoPE scaling (#2268) c43c2da8afacaddfe51c09b21dbd9922cd0ea46b 523e49b11174368cd73460fa5eae7b39d856f300 Georgi Gerganov ggerganov@gmail.com 2023-11-01T23:08:30+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-01T23:08:30+02:00 llm : fix llm_build_kqv taking unused tensor (benign, #3837) 523e49b11174368cd73460fa5eae7b39d856f300 e16b9fa4baa8a09c6619b116159830e898050942 Georgi Gerganov ggerganov@gmail.com 2023-11-01T23:00:50+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-01T23:00:50+02:00 llm : fix falcon norm after refactoring (#3837) e16b9fa4baa8a09c6619b116159830e898050942 ff8f9a88da0018972dfdf6fe64b5c8992caabd9c Georgi Gerganov ggerganov@gmail.com 2023-11-01T21:25:00+02:00 GitHub noreply@github.com 2023-11-01T21:25:00+02:00 metal : multi-simd softmax (#3710) ff8f9a88da0018972dfdf6fe64b5c8992caabd9c 50337961a678fce4081554b24e56e86b67660163 Georgi Gerganov ggerganov@gmail.com 2023-11-01T21:15:55+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-01T21:15:55+02:00 common : minor (#3715) 50337961a678fce4081554b24e56e86b67660163 0e40806c1cb3bdf9955ed807ffbe212be85b4c67 Georgi Gerganov ggerganov@gmail.com 2023-11-01T20:11:02+02:00 GitHub noreply@github.com 2023-11-01T20:11:02+02:00 llm : add llm_build_context (#3881) 0e40806c1cb3bdf9955ed807ffbe212be85b4c67 a2758d08e44ce3624d233af4d23c6843e2e735b5 bandoti 141645996+bandoti@users.noreply.github.com 2023-11-01T14:42:01-03:00 GitHub noreply@github.com 2023-11-01T19:42:01+02:00 common : allow caller to handle help/argument exceptions (#3715) a2758d08e44ce3624d233af4d23c6843e2e735b5 e75dfdd31b6a3dfa0627ba4ac3bb4b36e9db588e staviq staviq@gmail.com 2023-11-01T15:18:27+01:00 GitHub noreply@github.com 2023-11-01T16:18:27+02:00 log : make generating separate log files optional (#3787) e75dfdd31b6a3dfa0627ba4ac3bb4b36e9db588e 9a3b4f6c86503c9cfc049d4d0fdeafef12806f5e l3utterfly gc.pthzfoldr@gmail.com 2023-11-01T21:40:43+08:00 GitHub noreply@github.com 2023-11-01T15:40:43+02:00 sampling : null grammar field after reset (#3885) 9a3b4f6c86503c9cfc049d4d0fdeafef12806f5e 73bdcb395ef9a997d9c02950c7cd4249546162cd Georgi Gerganov ggerganov@gmail.com 2023-11-01T13:50:45+02:00 Georgi Gerganov ggerganov@gmail.com 2023-11-01T13:50:45+02:00 ggml : fix UNUSED macro (#3762) 73bdcb395ef9a997d9c02950c7cd4249546162cd f0e209324a7f663225791897877bf610f1af152d Andrew Godfrey AndrewGodfrey@users.noreply.github.com 2023-11-01T04:49:04-07:00 GitHub noreply@github.com 2023-11-01T13:49:04+02:00 finetune : add -ngl parameter (#3762) f0e209324a7f663225791897877bf610f1af152d ca190bca8e844d171020d6147687e71472d71734 Georgi Gerganov ggerganov@gmail.com 2023-11-01T11:29:07+02:00 GitHub noreply@github.com 2023-11-01T11:29:07+02:00 scripts : add server-llm.sh (#3868) ca190bca8e844d171020d6147687e71472d71734 71e3718abdb2771b50c9606d3a7569623a0b0afe Adrian Hesketh a-h@users.noreply.github.com 2023-11-01T09:28:28Z GitHub noreply@github.com 2023-11-01T11:28:28+02:00 server : re-enable completion and embedded at the same time (#3876) 71e3718abdb2771b50c9606d3a7569623a0b0afe 238657db2364cfb728c694470a4a81702afea760 Georgi Gerganov ggerganov@gmail.com 2023-11-01T08:04:02+02:00 GitHub noreply@github.com 2023-11-01T08:04:02+02:00 llama : refactor graph build code (#3837) 238657db2364cfb728c694470a4a81702afea760 07178c98e1b61a5e2af39d347add12e7eb9e08e1 kalomaze 66376113+kalomaze@users.noreply.github.com 2023-10-31T14:44:49-05:00 GitHub noreply@github.com 2023-10-31T20:44:49+01:00 samplers : Min-P sampler implementation [alternative to Top P/Top K] (#3841) 07178c98e1b61a5e2af39d347add12e7eb9e08e1 207b51900e15cc7f89763a3bb1c565fe11cbb45d Tungsten842 886724vf@anonaddy.me 2023-10-31T18:24:03+01:00 GitHub noreply@github.com 2023-10-31T19:24:03+02:00 flake.nix: fix for rocm 5.7 (#3853) 207b51900e15cc7f89763a3bb1c565fe11cbb45d 6e08281e588bbba1a5d180290a94a43f167f3a1a Georgi Gerganov ggerganov@gmail.com 2023-10-30T19:19:15+02:00 GitHub noreply@github.com 2023-10-30T19:19:15+02:00 ggml : move FP16 <-> FP32 code to ggml-impl.h (#3861) 6e08281e588bbba1a5d180290a94a43f167f3a1a 2046eb4345e62c4575b3cdc0115a51db89f3fb70 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-29T11:31:40-06:00 GitHub noreply@github.com 2023-10-29T11:31:40-06:00 Extend llama_kv_cache_seq_rm to allow matching any sequence (#3843) 2046eb4345e62c4575b3cdc0115a51db89f3fb70 71a09da301705b9c5ad4ca3cf3fbd966dd3f1ec5 cebtenzzre cebtenzzre@gmail.com 2023-10-29T12:33:47-04:00 GitHub noreply@github.com 2023-10-29T18:33:47+02:00 make : remove unnecessary dependency on build-info.h (#3842) 71a09da301705b9c5ad4ca3cf3fbd966dd3f1ec5 d69d777c02b9ac405a95f3cbfba219a990caefff Georgi Gerganov ggerganov@gmail.com 2023-10-29T18:32:51+02:00 GitHub noreply@github.com 2023-10-29T18:32:51+02:00 llama : fix kv shift bug (#3835) d69d777c02b9ac405a95f3cbfba219a990caefff ff3bad83e29e3009010cbc923bebd769055eaa7f Georgi Gerganov ggerganov@gmail.com 2023-10-29T18:32:28+02:00 GitHub noreply@github.com 2023-10-29T18:32:28+02:00 ggml : quantization refactoring (#3833) ff3bad83e29e3009010cbc923bebd769055eaa7f 82a6646e0221216c41edcdf99f5a44bb051391f5 Erik Scholz Green-Sky@users.noreply.github.com 2023-10-28T16:41:07+02:00 GitHub noreply@github.com 2023-10-28T16:41:07+02:00 flake : update flake.lock for newer transformers version + provide extra dev shell (#3797) 82a6646e0221216c41edcdf99f5a44bb051391f5 ba231e8a6dd8ad82acfe0e4d492ff7cef6b3f0a1 Aarni Koskela akx@iki.fi 2023-10-28T15:43:01+03:00 GitHub noreply@github.com 2023-10-28T15:43:01+03:00 metal : try cwd for ggml-metal.metal if bundle lookup fails (#3793) ba231e8a6dd8ad82acfe0e4d492ff7cef6b3f0a1 8a2f2fea2914aaa3f4b2f82800c7de15f15bdb09 Georgi Gerganov ggerganov@gmail.com 2023-10-28T15:25:33+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-28T15:35:26+03:00 issues : change label from bug to bug-unconfirmed (#3748) 8a2f2fea2914aaa3f4b2f82800c7de15f15bdb09 bd6d9e205982b34e0ba2c3b22bbf31a1ef1a1bb5 Georgi Gerganov ggerganov@gmail.com 2023-10-28T15:25:15+03:00 GitHub noreply@github.com 2023-10-28T06:25:15-06:00 convert : ignore tokens if their IDs are within [0, vocab_size) (#3831) bd6d9e205982b34e0ba2c3b22bbf31a1ef1a1bb5 ee1a0ec9cb367ba41d138134795cbbbe93d2bf1c Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-28T05:54:24-06:00 GitHub noreply@github.com 2023-10-28T14:54:24+03:00 llama : allow quantizing k-quants to fall back when tensor size incompatible (#3747) ee1a0ec9cb367ba41d138134795cbbbe93d2bf1c 177461104b454163473dced2a5038f4e016cdb7e Georgi Gerganov ggerganov@gmail.com 2023-10-28T14:23:11+03:00 GitHub noreply@github.com 2023-10-28T14:23:11+03:00 llama : add option for greedy sampling with probs (#3813) 177461104b454163473dced2a5038f4e016cdb7e fdee152e4eebb78c191df0b074857111d7f2aba7 Henk Poley HenkPoley@gmail.com 2023-10-28T12:16:33+02:00 GitHub noreply@github.com 2023-10-28T13:16:33+03:00 common : print that one line of the syntax help *also* to standard output (#3823) fdee152e4eebb78c191df0b074857111d7f2aba7 41aee4df821854f37d90a45281f03b6db8d27de2 Georgi Gerganov ggerganov@gmail.com 2023-10-28T12:06:08+03:00 GitHub noreply@github.com 2023-10-28T12:06:08+03:00 starcoder : add GPU offloading (#3827) 41aee4df821854f37d90a45281f03b6db8d27de2 6d459cbfbe5a011dfca94f9550527a504b6f9aa1 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-27T15:40:07-06:00 GitHub noreply@github.com 2023-10-28T00:40:07+03:00 speculative : ensure draft and target model vocab matches (#3812) 6d459cbfbe5a011dfca94f9550527a504b6f9aa1 c8d6a1f34ab6f1b6bd468d256e535a61f98f114c cebtenzzre cebtenzzre@gmail.com 2023-10-27T17:33:53-04:00 GitHub noreply@github.com 2023-10-27T17:33:53-04:00 llama : correctly report GGUFv3 format (#3818) c8d6a1f34ab6f1b6bd468d256e535a61f98f114c 2f9ec7e271220a78fe27c9e6ccbcc0dda31cda0f Thibault Terrasson thibault.terrasson@gmail.com 2023-10-27T16:37:41+02:00 GitHub noreply@github.com 2023-10-27T08:37:41-06:00 simple : fix batch handling (#3803) 2f9ec7e271220a78fe27c9e6ccbcc0dda31cda0f 34b2a5e1ee4fe6295fb4420eb91131d743694c65 Georgi Gerganov ggerganov@gmail.com 2023-10-27T17:01:23+03:00 GitHub noreply@github.com 2023-10-27T17:01:23+03:00 cuda : improve text-generation and batched decoding performance (#3776) 34b2a5e1ee4fe6295fb4420eb91131d743694c65 6961c4bd0b5176e10ab03b35394f1e9eab761792 Georgi Gerganov ggerganov@gmail.com 2023-10-26T22:53:37+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-26T22:54:17+03:00 server : do not release slot on image input (#3798) 6961c4bd0b5176e10ab03b35394f1e9eab761792 cc448774866e6479c750bd7c135cd8f92cedee67 Georgi Gerganov ggerganov@gmail.com 2023-10-25T10:26:27+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-25T10:26:27+03:00 batched-bench : print params at start cc448774866e6479c750bd7c135cd8f92cedee67 ad939626577cd25b462e8026cc543efb71528472 Georgi Gerganov ggerganov@gmail.com 2023-10-25T10:09:16+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-25T10:09:16+03:00 log : disable pid in log filenames ad939626577cd25b462e8026cc543efb71528472 1717521cdb976a2219888b0e5cba36e210eee9df cebtenzzre cebtenzzre@gmail.com 2023-10-24T16:10:43-04:00 GitHub noreply@github.com 2023-10-24T23:10:43+03:00 server : add parameter -tb N, --threads-batch N (#3584) (#3768) 1717521cdb976a2219888b0e5cba36e210eee9df b2f7e04bd312eaf97eee0523aa09d950d585626b Georgi Gerganov ggerganov@gmail.com 2023-10-24T23:08:20+03:00 GitHub noreply@github.com 2023-10-24T23:08:20+03:00 server : do not block system prompt update (#3767) b2f7e04bd312eaf97eee0523aa09d950d585626b abd21fc99f1d35e2081e4c01dc09c71a86bf3c5a Georgi Gerganov ggerganov@gmail.com 2023-10-24T21:51:20+03:00 GitHub noreply@github.com 2023-10-24T21:51:20+03:00 sync : ggml (conv ops + cuda MSVC fixes) (#3765) abd21fc99f1d35e2081e4c01dc09c71a86bf3c5a 2b4ea35e56792064598e922e46d081e02bc96b94 John Smith 67539080+kingsidelee@users.noreply.github.com 2023-10-25T01:48:45+08:00 GitHub noreply@github.com 2023-10-24T20:48:45+03:00 cmake : add missed dependencies (#3763) 2b4ea35e56792064598e922e46d081e02bc96b94 daab3d7f45832e10773c99f3484b0d5b14d86c0c Georgi Gerganov ggerganov@gmail.com 2023-10-24T16:48:37+03:00 GitHub noreply@github.com 2023-10-24T16:48:37+03:00 cuda : add batched cuBLAS GEMM for faster attention (#3749) daab3d7f45832e10773c99f3484b0d5b14d86c0c 469c9addef75893e6be12edda852d12e840bf064 Galunid karolek1231456@gmail.com 2023-10-24T09:17:17+02:00 GitHub noreply@github.com 2023-10-24T09:17:17+02:00 Add more tokenizer tests (#3742) 469c9addef75893e6be12edda852d12e840bf064 e3932593d46c30145301a13097895f9376cba509 Georgi Gerganov ggerganov@gmail.com 2023-10-24T09:46:50+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-24T09:47:22+03:00 metal : handle ggml_scale for n%4 != 0 (close #3754) e3932593d46c30145301a13097895f9376cba509 9d02956443e5c1ded29b7b5ed8a21bc01ba6f563 Georgi Gerganov ggerganov@gmail.com 2023-10-23T23:46:05+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-23T23:46:05+03:00 Revert "make : add optional CUDA_NATIVE_ARCH (#2482)" 9d02956443e5c1ded29b7b5ed8a21bc01ba6f563 69a6735087c3634963c642fd69f0851ac479cd78 M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-10-23T22:57:16+03:00 GitHub noreply@github.com 2023-10-23T22:57:16+03:00 issues : separate bug and enhancement template + no default title (#3748) 69a6735087c3634963c642fd69f0851ac479cd78 5be6c803fa5378f62a1590f3ad8c6b64c7c0c2ce Galunid karolek1231456@gmail.com 2023-10-23T21:46:00+02:00 GitHub noreply@github.com 2023-10-23T21:46:00+02:00 Update special token handling in conversion scripts for gpt2 derived tokenizers (#3746) 5be6c803fa5378f62a1590f3ad8c6b64c7c0c2ce 6336701c9378c23c85d1c0e464b663ca2bbb8e60 Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2023-10-23T12:40:03-07:00 GitHub noreply@github.com 2023-10-23T22:40:03+03:00 llama : remove token functions with `context` args in favor of `model` (#3720) 6336701c9378c23c85d1c0e464b663ca2bbb8e60 96981f37b1e3f450d9e63e571514217bf60f0a7f Galunid karolek1231456@gmail.com 2023-10-23T17:47:03+02:00 GitHub noreply@github.com 2023-10-23T17:47:03+02:00 Fix baichuan convert script not detecing model (#3739) 96981f37b1e3f450d9e63e571514217bf60f0a7f 438c2ca83045a00ef244093d27e9ed41a8cb4ea9 Alex awhill19@icloud.com 2023-10-22T15:56:53-04:00 GitHub noreply@github.com 2023-10-22T22:56:53+03:00 make : add optional CUDA_NATIVE_ARCH (#2482) 438c2ca83045a00ef244093d27e9ed41a8cb4ea9 9e70cc03229df19ca2d28ce23cc817198f897278 Georgi Gerganov ggerganov@gmail.com 2023-10-22T22:53:08+03:00 GitHub noreply@github.com 2023-10-22T22:53:08+03:00 server : parallel decoding and multimodal (#3677) 9e70cc03229df19ca2d28ce23cc817198f897278 5a42a5f8e8a86da9ac88008d748cf232a83aa0e1 goerch jhr.walter@t-online.de 2023-10-22T21:21:42+02:00 GitHub noreply@github.com 2023-10-22T21:21:42+02:00 Add test for MPT tokenization (#3728) 5a42a5f8e8a86da9ac88008d748cf232a83aa0e1 a5e7dbd6141128bfa3c40a19c2945a181df625d3 Ian Scrivener github@zilogy.asia 2023-10-23T05:16:43+11:00 GitHub noreply@github.com 2023-10-22T21:16:43+03:00 readme : remove unsupported node.js library (#3703) a5e7dbd6141128bfa3c40a19c2945a181df625d3 d3956aea53369455008159cc405ed4c496976692 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-22T12:14:56-06:00 GitHub noreply@github.com 2023-10-22T21:14:56+03:00 llama : validate special token ids are in range when loading GGUF model (#3635) d3956aea53369455008159cc405ed4c496976692 22c69a27945e7acf9690dd3210d316f22182751c vvhg1 94630311+vvhg1@users.noreply.github.com 2023-10-22T20:09:51+02:00 GitHub noreply@github.com 2023-10-22T21:09:51+03:00 main : escape prompt for cfg_negative_prompt and consecutive inputs in main with interactive (#3623) 22c69a27945e7acf9690dd3210d316f22182751c 465219b9143ac01db0990bbcb0a081ef72ec2008 Georgi Gerganov ggerganov@gmail.com 2023-10-22T08:37:20+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-22T08:37:20+03:00 batched : add len CLI argument 465219b9143ac01db0990bbcb0a081ef72ec2008 d1031cf49c3b958b915fd558e23453471c29ac33 shibe2 shibe@tuta.io 2023-10-12T16:01:23+04:00 shibe2 shibe@tuta.io 2023-10-20T22:30:52+04:00 CLBlast: Add outer loops over src0 for broadcasting in mulmat d1031cf49c3b958b915fd558e23453471c29ac33 8cf19d60dc93809db8e51fedc811595eed9134c5 Georgi Gerganov ggerganov@gmail.com 2023-10-20T21:07:23+03:00 GitHub noreply@github.com 2023-10-20T21:07:23+03:00 sampling : refactor init to use llama_sampling_params (#3696) 8cf19d60dc93809db8e51fedc811595eed9134c5 a0edf73bda31c7c4e649e6f07c6fd30a729929cd Qin Yue Chen 71813199+chenqiny@users.noreply.github.com 2023-10-20T06:19:40-05:00 GitHub noreply@github.com 2023-10-20T14:19:40+03:00 gguf : support big endian platform (#3552) a0edf73bda31c7c4e649e6f07c6fd30a729929cd f439e506e8ae8b01df2ae2156380f8156d7553e3 Georgi Gerganov ggerganov@gmail.com 2023-10-20T13:06:10+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-20T13:06:10+03:00 server : fix uninitialized sampling context (close #3685) f439e506e8ae8b01df2ae2156380f8156d7553e3 e78f3ef24af4ca74e77e725644b41ae8ca3b10a5 Herman Semenov GermanAizek@yandex.ru 2023-10-20T10:02:12Z GitHub noreply@github.com 2023-10-20T13:02:12+03:00 ggml : fix rope + llama minor optimizations (#3560) e78f3ef24af4ca74e77e725644b41ae8ca3b10a5 f3b25e40438b3c8383caabf4e7b89863145a9f0e cebtenzzre cebtenzzre@gmail.com 2023-10-20T01:32:08-04:00 GitHub noreply@github.com 2023-10-20T08:32:08+03:00 convert : restore compat with old Falcon models (#3680) f3b25e40438b3c8383caabf4e7b89863145a9f0e 60abea9798f47b918a9f38c66edfd88c526d20f6 M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-10-19T19:40:41+03:00 GitHub noreply@github.com 2023-10-19T19:40:41+03:00 multimodal : add BakLLaVA conversion support (#3682) 60abea9798f47b918a9f38c66edfd88c526d20f6 004797f6ac135383f8c1d1f5bd415ddee2f79318 M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-10-19T16:59:11+03:00 GitHub noreply@github.com 2023-10-19T16:59:11+03:00 llava : avoid segfault in case of non-existent mmproj file (#3674) 004797f6ac135383f8c1d1f5bd415ddee2f79318 4e82b2ea3fa6482915d147bc9f46e70b9ada7700 Georgi Gerganov ggerganov@gmail.com 2023-10-18T21:44:43+03:00 GitHub noreply@github.com 2023-10-18T21:44:43+03:00 readme : update hot topics 4e82b2ea3fa6482915d147bc9f46e70b9ada7700 0e89203b517c95ec6675eda75d200a60d1e8921d Georgi Gerganov ggerganov@gmail.com 2023-10-18T18:49:40+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-18T18:49:40+03:00 speculative : bug fixes 0e89203b517c95ec6675eda75d200a60d1e8921d c67fe68e417f766970fb1feaf2e66458aa24116a Georgi Gerganov ggerganov@gmail.com 2023-10-18T16:21:57+03:00 GitHub noreply@github.com 2023-10-18T16:21:57+03:00 speculative : add tree-based sampling example (#3624) c67fe68e417f766970fb1feaf2e66458aa24116a 1117d06607d2d885640ac501f05f0aae5494e2c5 Jhen-Jie Hong iainst0409@gmail.com 2023-10-18T07:21:48-05:00 GitHub noreply@github.com 2023-10-18T15:21:48+03:00 metal : implement q5_0 and q5_1 kernels (#3648) 1117d06607d2d885640ac501f05f0aae5494e2c5 cb33f43a2a9f5a5a5f8d290dd97c625d9ba97a2f shibe2 shibe@tuta.io 2023-10-18T16:09:22+04:00 GitHub noreply@github.com 2023-10-18T15:09:22+03:00 opencl : fix element-wise multiplication (#3656) cb33f43a2a9f5a5a5f8d290dd97c625d9ba97a2f e1675d133c31e1c8de2f06be7164e12c0ba6cf2c slaren slarengh@gmail.com 2023-10-17T22:24:50+02:00 GitHub noreply@github.com 2023-10-17T22:24:50+02:00 fix embeddings when using CUDA (#3657) e1675d133c31e1c8de2f06be7164e12c0ba6cf2c 8402566a7c436bfbde8e7b0461faee50298106a0 Georgi Gerganov ggerganov@gmail.com 2023-10-17T22:34:26+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-17T22:34:26+03:00 llama : avoid fprintf in favor of LLAMA_LOG (#3538) 8402566a7c436bfbde8e7b0461faee50298106a0 40e5ce054f4c4fa555e4510ea5f760bb29185332 BarfingLemurs 128182951+BarfingLemurs@users.noreply.github.com 2023-10-17T14:13:21-04:00 GitHub noreply@github.com 2023-10-17T21:13:21+03:00 readme : update hot-topics & models, detail windows release in usage (#3615) 40e5ce054f4c4fa555e4510ea5f760bb29185332 a5e8c1d8c71f01d98ae2ec63a57c118664f9764d shibe2 shibe@tuta.io 2023-10-11T21:30:06+04:00 shibe2 shibe@tuta.io 2023-10-17T21:02:30+04:00 CLBlast: Fix temporary buffer size for f16 conversion (wsize) a5e8c1d8c71f01d98ae2ec63a57c118664f9764d e74c705e15cd228ad696c4a3cdea6d6fb4ff434c slaren slarengh@gmail.com 2023-10-17T19:00:58+02:00 GitHub noreply@github.com 2023-10-17T20:00:58+03:00 train-text-from-scratch : fix assert failure in ggml-alloc (#3618) e74c705e15cd228ad696c4a3cdea6d6fb4ff434c 3ad1e3f1a10c1f66b4f1cd7510e0977fadbc0dfd Georgi Gerganov ggerganov@gmail.com 2023-10-17T19:52:53+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-17T19:52:53+03:00 editorconfig : remove trailing spaces 3ad1e3f1a10c1f66b4f1cd7510e0977fadbc0dfd 1142013da40e98946a109b141dd858f0ed996051 coezbek c.oezbek@gmail.com 2023-10-17T18:51:02+02:00 GitHub noreply@github.com 2023-10-17T19:51:02+03:00 server : documentation of JSON return value of /completion endpoint (#3632) 1142013da40e98946a109b141dd858f0ed996051 5fe268a4d9ce09f3a6c77239af583d3a8e49d54c Georgi Gerganov ggerganov@gmail.com 2023-10-17T19:12:46+03:00 GitHub noreply@github.com 2023-10-17T19:12:46+03:00 save-load-state : fix example + add ci test (#3655) 5fe268a4d9ce09f3a6c77239af583d3a8e49d54c 1a159553f921a9209fed8c714494e57b3649f232 ldwang ftgreat@163.com 2023-10-17T23:52:33+08:00 GitHub noreply@github.com 2023-10-17T18:52:33+03:00 readme : add Aquila2 links (#3610) 1a159553f921a9209fed8c714494e57b3649f232 281ef73c258cc1eebec8a64264240432d5878c4b staviq staviq@gmail.com 2023-10-17T17:11:01+02:00 GitHub noreply@github.com 2023-10-17T18:11:01+03:00 tokenizer : special token handling (#3538) 281ef73c258cc1eebec8a64264240432d5878c4b 940efa95fec0b8a98c226a889d2ad839dfeeae0d Georgi Gerganov ggerganov@gmail.com 2023-10-17T09:19:28+03:00 GitHub noreply@github.com 2023-10-17T09:19:28+03:00 k-quants : fix quantization ranges (#3646) 940efa95fec0b8a98c226a889d2ad839dfeeae0d 11bff290458f12f020b588792707f76ec658a27a Georgi Gerganov ggerganov@gmail.com 2023-10-16T23:58:00+03:00 GitHub noreply@github.com 2023-10-16T23:58:00+03:00 llava : fix tokenization to not add bos between image embeddings and user prompt (#3645) 11bff290458f12f020b588792707f76ec658a27a 11dc1091f64b24ca6d643acc6d0051117ba60161 cebtenzzre cebtenzzre@gmail.com 2023-10-15T02:32:06-04:00 GitHub noreply@github.com 2023-10-15T09:32:06+03:00 MPT : support GQA for replit-code-v1.5 (#3627) 11dc1091f64b24ca6d643acc6d0051117ba60161 2a4bcbacead886996f175f33479d1d874a3e577f M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-10-14T13:52:44+03:00 GitHub noreply@github.com 2023-10-14T04:52:44-06:00 Honor -ngl option for Cuda offloading in llava (#3621) 2a4bcbacead886996f175f33479d1d874a3e577f 424b6381c4daeed62e6600e0402e72f39845b58d Daniel Bevenius daniel.bevenius@gmail.com 2023-10-13T12:33:16+02:00 GitHub noreply@github.com 2023-10-13T13:33:16+03:00 llama : remove n_threads from llama_decode_internal (#3614) 424b6381c4daeed62e6600e0402e72f39845b58d 1e0e873c373c33989beb6bc64d83cd572ab7fe2b slaren slarengh@gmail.com 2023-10-13T12:23:10+02:00 GitHub noreply@github.com 2023-10-13T12:23:10+02:00 ggml : add context enumeration functions (#3605) 1e0e873c373c33989beb6bc64d83cd572ab7fe2b 370359e5baf619f3a8d461023143d1494b1e8fde shibe2 shibe@tuta.io 2023-10-12T23:59:47+04:00 GitHub noreply@github.com 2023-10-12T21:59:47+02:00 CLBlast: Fix matrix-vector multiplication (#3544) 370359e5baf619f3a8d461023143d1494b1e8fde 9e24cc6e2e589d405bd1720c400f5b0b9d0ca3ee M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-10-12T18:23:18+03:00 GitHub noreply@github.com 2023-10-12T18:23:18+03:00 examples: support LLaVA v1.5 (multimodal model) (#3436) 9e24cc6e2e589d405bd1720c400f5b0b9d0ca3ee d28e572c0270eb72649dbcc3a347e36c05c2934a uint256_t maekawatoshiki1017@gmail.com 2023-10-12T22:36:16+09:00 GitHub noreply@github.com 2023-10-12T16:36:16+03:00 docs : fix typo GOMP_CPU_AFFINITY (#3597) d28e572c0270eb72649dbcc3a347e36c05c2934a f3040beaab5228b1a9dfe5675a200379478f7204 Georgi Gerganov ggerganov@gmail.com 2023-10-12T14:31:05+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-12T14:31:05+03:00 cmake : fix add_compile_options on macOS f3040beaab5228b1a9dfe5675a200379478f7204 1a8c8795d64b04df96c28f29faac2d6e256f53bc Ian Scrivener github@zilogy.asia 2023-10-12T22:10:50+11:00 GitHub noreply@github.com 2023-10-12T14:10:50+03:00 typo : it is `--n-gpu-layers` not `--gpu-layers` (#3592) 1a8c8795d64b04df96c28f29faac2d6e256f53bc b016596d903641f8825cd94bb6742e1de0c21017 Georgi Gerganov ggerganov@gmail.com 2023-10-12T13:44:56+03:00 GitHub noreply@github.com 2023-10-12T13:44:56+03:00 ci : check if there is enough VRAM (#3596) b016596d903641f8825cd94bb6742e1de0c21017 6b3ae4da92485f979a0f45774fcf68597634db0b Aarni Koskela akx@iki.fi 2023-10-12T15:51:53+09:00 GitHub noreply@github.com 2023-10-12T09:51:53+03:00 server : add completion mode (no chat) (#3582) 6b3ae4da92485f979a0f45774fcf68597634db0b 57dd55e2c742bfc50e0f5c6fb95c14118cff44f6 Georgi Gerganov ggerganov@gmail.com 2023-10-12T09:35:19+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-12T09:35:30+03:00 prompts : add mnemonics.txt 57dd55e2c742bfc50e0f5c6fb95c14118cff44f6 b8fe4b5cc9cb237ca98e5bc51b5d189e3c446d13 Georgi Gerganov ggerganov@gmail.com 2023-10-12T09:29:04+03:00 GitHub noreply@github.com 2023-10-12T09:29:04+03:00 server : fix kv cache management (#3588) b8fe4b5cc9cb237ca98e5bc51b5d189e3c446d13 a8bdd65525ae86dea905e9866ad369b53e30ac14 Georgi Gerganov ggerganov@gmail.com 2023-10-11T23:55:08+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-11T23:55:41+03:00 main : fix session loading bug (#3400) a8bdd65525ae86dea905e9866ad369b53e30ac14 70c29da118cdb02bfcbd0376c32b5b2236e48e48 Michael Coppola m18coppola@gmail.com 2023-10-11T15:42:22-04:00 GitHub noreply@github.com 2023-10-11T22:42:22+03:00 server : add parameter -tb N, --threads-batch N (#3584) 70c29da118cdb02bfcbd0376c32b5b2236e48e48 8c70a5ff25964f0a81e20d142a2f5ac5baff22fc Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-11T13:35:46-06:00 GitHub noreply@github.com 2023-10-11T22:35:46+03:00 common : fix mirostat state when using multiple sequences (#3543) 8c70a5ff25964f0a81e20d142a2f5ac5baff22fc 24ba3d829e31a6eda3fa1723f692608c2fa3adda Georgi Gerganov ggerganov@gmail.com 2023-10-11T21:25:33+03:00 GitHub noreply@github.com 2023-10-11T21:25:33+03:00 batched : add bench tool (#3545) 24ba3d829e31a6eda3fa1723f692608c2fa3adda 9f6ede19f3cfa50d4a51a5babb056c3f8a450b80 Zane Shannon z@zcs.me 2023-10-11T04:14:05-07:00 GitHub noreply@github.com 2023-10-11T06:14:05-05:00 examples : add batched.swift + improve CI for swift (#3562) 9f6ede19f3cfa50d4a51a5babb056c3f8a450b80 233fc1c69f6f415f35363e18a755f9610e89161b Galunid karolek1231456@gmail.com 2023-10-11T01:02:49+02:00 GitHub noreply@github.com 2023-10-10T19:02:49-04:00 Add MPT model to supported models in README.md (#3574) 233fc1c69f6f415f35363e18a755f9610e89161b c5b49360d0d9e49f32e05a9116e90bd0b39a282d goerch jhr.walter@t-online.de 2023-10-10T18:59:52+02:00 GitHub noreply@github.com 2023-10-10T18:59:52+02:00 Minor improvements in GPT2 tokenizer (#3567) c5b49360d0d9e49f32e05a9116e90bd0b39a282d 02d2875deff28599c6c2c6e1886fab002ffe43b1 Xingchen Song(宋星辰) xingchensong1996@163.com 2023-10-11T00:28:50+08:00 GitHub noreply@github.com 2023-10-10T19:28:50+03:00 readme : add bloom (#3570) 02d2875deff28599c6c2c6e1886fab002ffe43b1 0aa6595ae02f97f2e5ffd74bf57a8b21ac83b272 Xingchen Song(宋星辰) xingchensong1996@163.com 2023-10-10T22:48:21+08:00 GitHub noreply@github.com 2023-10-10T17:48:21+03:00 llm : add bloom models (#3553) 0aa6595ae02f97f2e5ffd74bf57a8b21ac83b272 f5f9121de140eff558f13b5c5e78a3a3b6b94377 Jhen-Jie Hong iainst0409@gmail.com 2023-10-10T06:31:13-05:00 GitHub noreply@github.com 2023-10-10T14:31:13+03:00 swift : improvements and fixes (#3564) f5f9121de140eff558f13b5c5e78a3a3b6b94377 11ea5c7d96f2c28e1c99659e08ec0a44574056e2 Jan Ploski jpl@plosquare.com 2023-10-10T09:50:23+02:00 GitHub noreply@github.com 2023-10-10T10:50:23+03:00 llm : add MPT support (#3417) 11ea5c7d96f2c28e1c99659e08ec0a44574056e2 95bd60a0a69f57e9a2ff1269667ea484a1a9bb40 vvhg1 94630311+vvhg1@users.noreply.github.com 2023-10-10T09:31:21+02:00 GitHub noreply@github.com 2023-10-10T10:31:21+03:00 infill. : fix tokenization (#3508) 95bd60a0a69f57e9a2ff1269667ea484a1a9bb40 fcca0a700487999d52a525c96d6661e9f6a8703a slaren slarengh@gmail.com 2023-10-09T14:44:58+02:00 GitHub noreply@github.com 2023-10-09T15:44:58+03:00 ggml-alloc : fix assert in debug builds (#3555) fcca0a700487999d52a525c96d6661e9f6a8703a dcc09d25961c5d0626bc148e558ee841141748f7 Georgi Gerganov ggerganov@gmail.com 2023-10-09T14:32:17+03:00 GitHub noreply@github.com 2023-10-09T14:32:17+03:00 refact : fix convert script + zero out KV cache to avoid nans (#3523) dcc09d25961c5d0626bc148e558ee841141748f7 db3abcc114d5d1790ba814aa1a80ac673d4ccc3e Georgi Gerganov ggerganov@gmail.com 2023-10-09T14:28:27+03:00 GitHub noreply@github.com 2023-10-09T14:28:27+03:00 metal : do not use mul_mm kernels when ne00 < 64 (#3542) db3abcc114d5d1790ba814aa1a80ac673d4ccc3e eee42c670e6fa6df9cf17e7ffc319f74cbd81354 Georgi Gerganov ggerganov@gmail.com 2023-10-08T20:19:14+03:00 GitHub noreply@github.com 2023-10-08T20:19:14+03:00 sync : ggml (ggml-backend) (#3548) eee42c670e6fa6df9cf17e7ffc319f74cbd81354 8e6716a102e390e930594d51302730184dac83cc Matheus C. França matheus-catarino@hotmail.com 2023-10-08T10:59:20-03:00 GitHub noreply@github.com 2023-10-08T16:59:20+03:00 ci : add Zig CI/CD and fix build (#2996) 8e6716a102e390e930594d51302730184dac83cc 9c38d181d40b9d27f8f42152c18e7c70bfffcf37 Ryder Wishart ryderwishart@gmail.com 2023-10-08T03:55:58-07:00 GitHub noreply@github.com 2023-10-08T13:55:58+03:00 api_like_OAI.py : compat with Microsoft Guidance (#2746) 9c38d181d40b9d27f8f42152c18e7c70bfffcf37 a1202a31ed8c35705bd09fe91c3e7410c619bd70 arcrank arcrank@gmail.com 2023-10-08T06:52:57-04:00 GitHub noreply@github.com 2023-10-08T13:52:57+03:00 api_like_OAI.py : simplify function (#2796) a1202a31ed8c35705bd09fe91c3e7410c619bd70 94e502dfb79430870b42b8e8ee132b4aaa93e4a8 Johannes Rudolph johannes.rudolph@gmail.com 2023-10-08T12:21:19+02:00 GitHub noreply@github.com 2023-10-08T13:21:19+03:00 k-quants : fix comments about block sizing (#3499) 94e502dfb79430870b42b8e8ee132b4aaa93e4a8 7d8b24932fe788a4cda76459a0c5df3e0073cb98 Georgi Gerganov ggerganov@gmail.com 2023-10-08T11:24:50+03:00 GitHub noreply@github.com 2023-10-08T11:24:50+03:00 ci : enable on obj-c changes + fix metal build (#3540) 7d8b24932fe788a4cda76459a0c5df3e0073cb98 b0ec5218c3d24755786b80ecce9cf4ffc07583f8 Luo Tian lt@basecity.com 2023-10-08T16:24:01+08:00 GitHub noreply@github.com 2023-10-08T11:24:01+03:00 zig : fix build by introducing train.cpp (#3539) b0ec5218c3d24755786b80ecce9cf4ffc07583f8 63d3b06a4318329f92b078e8aa0be7ab6e9f871f Georgi Gerganov ggerganov@gmail.com 2023-10-08T10:01:53+03:00 GitHub noreply@github.com 2023-10-08T10:01:53+03:00 metal : support MTLGPUFamily < Apple7, formatting, style (#3524) 63d3b06a4318329f92b078e8aa0be7ab6e9f871f a16e89cec83b4bd5f6af8f1ce1400f94c12356f9 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-07T23:22:17-06:00 GitHub noreply@github.com 2023-10-08T08:22:17+03:00 llama : fix missing break in Persimmon arch case statements (#3535) a16e89cec83b4bd5f6af8f1ce1400f94c12356f9 4d0383321184aadf91968d9e3c6a45286ed2473b Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-07T15:31:41-06:00 GitHub noreply@github.com 2023-10-07T15:31:41-06:00 Fix trying to strip newline from empty prompt and cfg prompt file content (#3534) 4d0383321184aadf91968d9e3c6a45286ed2473b c47066d833c6c112e0d23342aa62c3250dd33c81 M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-10-07T22:14:10+03:00 GitHub noreply@github.com 2023-10-07T22:14:10+03:00 gguf.py : fix CI for publishing GGUF package (#3532) c47066d833c6c112e0d23342aa62c3250dd33c81 f1782c68de13b64bb5283fc2038f584e47be9fd2 Tom C tom.corelis@gmail.com 2023-10-07T02:56:15-07:00 GitHub noreply@github.com 2023-10-07T12:56:15+03:00 py : change version of numpy requirement to 1.24.4 (#3515) f1782c68de13b64bb5283fc2038f584e47be9fd2 c26765a0a148b47e5b541df32438c3ad2a0a8314 cebtenzzre cebtenzzre@gmail.com 2023-10-07T04:41:52-04:00 GitHub noreply@github.com 2023-10-07T11:41:52+03:00 quantize : fail fast on write errors (#3521) c26765a0a148b47e5b541df32438c3ad2a0a8314 0e797c2fc571b866090f7d60ac7d39d8533593f2 Jhen-Jie Hong iainst0409@gmail.com 2023-10-07T03:40:27-05:00 GitHub noreply@github.com 2023-10-07T11:40:27+03:00 metal : support default.metallib load & reuse code for swift package (#3522) 0e797c2fc571b866090f7d60ac7d39d8533593f2 3a716b4dae545c3db307594fbc509a95d3e21b6e Phillip Kravtsov phillip@kravtsov.net 2023-10-07T00:12:43-07:00 GitHub noreply@github.com 2023-10-07T10:12:43+03:00 llm : support Adept Persimmon 8B (#3410) 3a716b4dae545c3db307594fbc509a95d3e21b6e 1faaae8c2bdc4a21302e367e0754c3fe74a8113e goerch jhr.walter@t-online.de 2023-10-07T06:57:01+02:00 GitHub noreply@github.com 2023-10-07T06:57:01+02:00 Fix for #3454 (#3455) 1faaae8c2bdc4a21302e367e0754c3fe74a8113e cb13d73a720c42d1958bff79b6869d77b26b8cea BarfingLemurs 128182951+BarfingLemurs@users.noreply.github.com 2023-10-06T15:13:36-04:00 GitHub noreply@github.com 2023-10-06T22:13:36+03:00 readme : update models, cuda + ppl instructions (#3510) cb13d73a720c42d1958bff79b6869d77b26b8cea 9ca79d5cbbc8d43f2bff951404b6a40ff1ee3788 Mihai mihai.chirculescu@yahoo.com 2023-10-06T21:39:33+03:00 GitHub noreply@github.com 2023-10-06T21:39:33+03:00 server : docs fix default values and add n_probs (#3506) 9ca79d5cbbc8d43f2bff951404b6a40ff1ee3788 0c731ca4039ccff86ffab90eaae4ca98037c4496 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-06T10:10:13-06:00 GitHub noreply@github.com 2023-10-06T10:10:13-06:00 kv cache slot search improvements (#3493) 0c731ca4039ccff86ffab90eaae4ca98037c4496 a8777ad84e00cda0399e827cdf971e2c3fab1da2 Georgi Gerganov ggerganov@gmail.com 2023-10-06T16:35:55+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-06T16:36:32+03:00 prompts : fix editorconfig checks after #3416 a8777ad84e00cda0399e827cdf971e2c3fab1da2 97af49fa395df77e4c18af0e1655b2fee67c9686 pudepiedj pudepiedj@gmail.com 2023-10-06T14:16:38+01:00 GitHub noreply@github.com 2023-10-06T16:16:38+03:00 parallel : add option to load external prompt file (#3416) 97af49fa395df77e4c18af0e1655b2fee67c9686 16820a5a0d885113f21021ce934f0b0027b9d69a Jhen-Jie Hong iainst0409@gmail.com 2023-10-06T07:44:24-05:00 GitHub noreply@github.com 2023-10-06T15:44:24+03:00 server : reuse llama_sample_token common util (#3494) 16820a5a0d885113f21021ce934f0b0027b9d69a 04b2f4386eda0264287156104cbf9d1b87895422 l3utterfly gc.pthzfoldr@gmail.com 2023-10-06T18:47:59+08:00 GitHub noreply@github.com 2023-10-06T13:47:59+03:00 llama : correct hparams comparison (#3446) 04b2f4386eda0264287156104cbf9d1b87895422 48edda30ee545fdac2e7a33d505382888f748bbf Jhen-Jie Hong iainst0409@gmail.com 2023-10-06T05:36:43-05:00 GitHub noreply@github.com 2023-10-06T13:36:43+03:00 ci : fix xcodebuild destinations (#3491) 48edda30ee545fdac2e7a33d505382888f748bbf 45eba9369fbcbd7f677eba9a2d3e4ffcfdc81824 cebtenzzre cebtenzzre@gmail.com 2023-10-05T15:00:34-04:00 GitHub noreply@github.com 2023-10-05T15:00:34-04:00 convert : update Falcon script for new HF config (#3448) 45eba9369fbcbd7f677eba9a2d3e4ffcfdc81824 acec9eaaa93315711c11d15afa8d245d164b7cff Kenvix ⭐ kenvixzure@live.com 2023-10-06T01:16:39+08:00 GitHub noreply@github.com 2023-10-05T20:16:39+03:00 build : use std::make_tuple() for compatibility with older GCC versions (#3488) acec9eaaa93315711c11d15afa8d245d164b7cff e2583cbc29cd7d6d1403f338842c07dfc0467e6c staviq staviq@gmail.com 2023-10-05T18:17:29+02:00 GitHub noreply@github.com 2023-10-05T19:17:29+03:00 common : process escape sequences in reverse prompts (#3461) e2583cbc29cd7d6d1403f338842c07dfc0467e6c e8b8d32e8663ffc55a02c9721af3a5190382cbb0 shibe2 shibe@tuta.io 2023-10-05T15:57:03+04:00 shibe2 shibe@tuta.io 2023-10-05T18:25:23+04:00 CLBlast: Fix handling of on-device tensor data e8b8d32e8663ffc55a02c9721af3a5190382cbb0 8f3a642ec1d878b2d0a0d15e3a4277f522790d4c Jhen-Jie Hong iainst0409@gmail.com 2023-10-05T09:02:55-05:00 GitHub noreply@github.com 2023-10-05T17:02:55+03:00 server : fix incorrect num_tokens_predicted (#3480) 8f3a642ec1d878b2d0a0d15e3a4277f522790d4c 0745384449fe8d89d6d99c93153569079e853247 Jhen-Jie Hong iainst0409@gmail.com 2023-10-05T09:00:07-05:00 GitHub noreply@github.com 2023-10-05T17:00:07+03:00 swift : disable ACCELERATE_NEW_LAPACK (#3481) 0745384449fe8d89d6d99c93153569079e853247 019ba1dcd0c7775a5ac0f7442634a330eb0173cc Jhen-Jie Hong iainst0409@gmail.com 2023-10-05T08:56:21-05:00 GitHub noreply@github.com 2023-10-05T16:56:21+03:00 ci : add swift build via xcodebuild (#3482) 019ba1dcd0c7775a5ac0f7442634a330eb0173cc beabc8cfb0145b48aad68fefc573d316fe9c3a8a Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-10-04T08:20:28-06:00 GitHub noreply@github.com 2023-10-04T17:20:28+03:00 convert : fix Baichuan2 models by using vocab size in config.json (#3299) beabc8cfb0145b48aad68fefc573d316fe9c3a8a 0d152b37fecd5a4838330d47bb034cebf1681779 Georgi Gerganov ggerganov@gmail.com 2023-10-04T16:50:44+03:00 GitHub noreply@github.com 2023-10-04T16:50:44+03:00 readme : add project status link 0d152b37fecd5a4838330d47bb034cebf1681779 f8c90cdbaa729e64493164c1aba7ea80da7b716f Georgi Gerganov ggerganov@gmail.com 2023-10-04T16:25:41+03:00 Georgi Gerganov ggerganov@gmail.com 2023-10-04T16:25:41+03:00 ggml : fix build after #3329 f8c90cdbaa729e64493164c1aba7ea80da7b716f f93af02488179b9c52d0d391b08ae4c4d891b8d3 ds5t5 145942675+ds5t5@users.noreply.github.com 2023-10-04T06:23:39-07:00 GitHub noreply@github.com 2023-10-04T16:23:39+03:00 llm : add Refact model (#3329) f93af02488179b9c52d0d391b08ae4c4d891b8d3 f72f8f22c9cb60465b2e79df2767e4ba9604e576 Georgi Gerganov ggerganov@gmail.com 2023-10-04T15:29:58+03:00 GitHub noreply@github.com 2023-10-04T15:29:58+03:00 sync : ggml (conv 1d + 2d updates, UB fixes) (#3468) f72f8f22c9cb60465b2e79df2767e4ba9604e576 79f34abddb72ac5ddbf118f3d87520b611a10a7d Merrick Christensen merrick.christensen@gmail.com 2023-10-04T00:33:13-06:00 GitHub noreply@github.com 2023-10-04T09:33:13+03:00 finetune : readme fix typo (#3465) 79f34abddb72ac5ddbf118f3d87520b611a10a7d 8186242b6d67cf87ae179fb1a62f52fdf0e5c5eb Tameem 113388789+AhmadTameem@users.noreply.github.com 2023-10-03T23:38:19+05:00 GitHub noreply@github.com 2023-10-03T21:38:19+03:00 ggml : add RISC-V Vector Support for K-Quants and improved the existing intrinsics (#3453) 8186242b6d67cf87ae179fb1a62f52fdf0e5c5eb ac2219fef34eb5b713c286c34c6e4162c39c8f3b h-h-h-h 13482553+h-h-h-h@users.noreply.github.com 2023-10-03T20:16:15+02:00 GitHub noreply@github.com 2023-10-03T21:16:15+03:00 main : consistent prefix/suffix coloring (#3425) ac2219fef34eb5b713c286c34c6e4162c39c8f3b 48be797ffbd80b062f55778e09e97180eb25d2ab Georgi Gerganov ggerganov@gmail.com 2023-10-03T21:04:01+03:00 GitHub noreply@github.com 2023-10-03T21:04:01+03:00 llama : fix session saving/loading (#3400) 48be797ffbd80b062f55778e09e97180eb25d2ab f56e1baec361b5381e32ee6b6e56e4f00e002dfe Alex Klinkhamer git@grencez.dev 2023-10-03T10:09:28-07:00 GitHub noreply@github.com 2023-10-03T20:09:28+03:00 llama : expose model's rope_freq_scale in the API (#3418) f56e1baec361b5381e32ee6b6e56e4f00e002dfe 017efe899d8fa76118aef88e963210d48da01172 Jiahao Li liplus17@163.com 2023-10-04T00:55:21+08:00 GitHub noreply@github.com 2023-10-03T19:55:21+03:00 metal : alibi for arbitrary number of heads (#3426) 017efe899d8fa76118aef88e963210d48da01172 ff5a3f0c09dfa0a8e0bf76d1748df5c6dee0e8ff Eve 139727413+netrunnereve@users.noreply.github.com 2023-10-03T16:53:15Z GitHub noreply@github.com 2023-10-03T19:53:15+03:00 cmake : make LLAMA_NATIVE flag actually use the instructions supported by the processor (#3273) ff5a3f0c09dfa0a8e0bf76d1748df5c6dee0e8ff 1c84003c08027f5d3a4cb876f51d6b6224a34d0e goerch jhr.walter@t-online.de 2023-10-03T09:16:26+02:00 GitHub noreply@github.com 2023-10-03T09:16:26+02:00 Work on the BPE tokenizer (#3252) 1c84003c08027f5d3a4cb876f51d6b6224a34d0e e78f0b0d0572168f328dd0e2ed3175a53fe52acc cebtenzzre cebtenzzre@gmail.com 2023-10-02T18:07:24-04:00 GitHub noreply@github.com 2023-10-02T18:07:24-04:00 convert : fix vocab size when not defined in hparams (#3421) e78f0b0d0572168f328dd0e2ed3175a53fe52acc 665018c749101e81c816675198e731e47d6b1dbe cebtenzzre cebtenzzre@gmail.com 2023-10-02T15:38:43-04:00 GitHub noreply@github.com 2023-10-02T22:38:43+03:00 cmake : increase minimum version for add_link_options (#3444) 665018c749101e81c816675198e731e47d6b1dbe 29a404a951fb0b3f9c3b6ab8c4c9c76ac50d2bb3 shibe2 shibe@tuta.io 2023-10-02T23:26:15+04:00 GitHub noreply@github.com 2023-10-02T21:26:15+02:00 CLBlast: Add broadcast support for matrix multiplication (#3402) 29a404a951fb0b3f9c3b6ab8c4c9c76ac50d2bb3 0fe321031a5c670ab5fb5f49d69c4c91d783c93f cebtenzzre cebtenzzre@gmail.com 2023-10-02T15:20:28-04:00 GitHub noreply@github.com 2023-10-02T15:20:28-04:00 gguf : add BERT, MPT, and GPT-J arch info (#3408) 0fe321031a5c670ab5fb5f49d69c4c91d783c93f 9476b012260a2fb6c67976582d64484ce7406ed9 cebtenzzre cebtenzzre@gmail.com 2023-10-02T14:58:46-04:00 GitHub noreply@github.com 2023-10-02T14:58:46-04:00 gguf : general usability improvements (#3409) 9476b012260a2fb6c67976582d64484ce7406ed9 a03ce38455544121c5c00cf845def1443acd6ac8 cebtenzzre cebtenzzre@gmail.com 2023-10-02T09:16:50-04:00 GitHub noreply@github.com 2023-10-02T16:16:50+03:00 cmake : make CUDA flags more similar to the Makefile (#3420) a03ce38455544121c5c00cf845def1443acd6ac8 a84767698495d72e44044f1f6db1c1cc721bfd15 xaedes xaedes@gmail.com 2023-10-02T15:15:45+02:00 GitHub noreply@github.com 2023-10-02T16:15:45+03:00 finetune : fix #3404 (#3437) a84767698495d72e44044f1f6db1c1cc721bfd15 095231dfd32679e32300f8ffaf1770b693ea64b0 Adrian smith.adriane@gmail.com 2023-10-02T03:49:59-07:00 GitHub noreply@github.com 2023-10-02T13:49:59+03:00 metal : set log callback before initializing (#3427) 095231dfd32679e32300f8ffaf1770b693ea64b0 ea55295a745c084f588be20710f5a1a12abb1109 bandoti 141645996+bandoti@users.noreply.github.com 2023-10-02T06:51:49-03:00 GitHub noreply@github.com 2023-10-02T12:51:49+03:00 cmake : fix transient definitions in find pkg (#3411) ea55295a745c084f588be20710f5a1a12abb1109 c97f01c362ac102c6994edb80008f8608539553a Kevin Ji 1146876+kevinji@users.noreply.github.com 2023-10-02T04:53:53-04:00 GitHub noreply@github.com 2023-10-02T11:53:53+03:00 docker : ignore Git files (#3314) c97f01c362ac102c6994edb80008f8608539553a f5ef5cfb18148131fcf45bdd2331f0db5ab7c3d0 vvhg1 94630311+vvhg1@users.noreply.github.com 2023-10-02T09:42:02+02:00 GitHub noreply@github.com 2023-10-02T10:42:02+03:00 infill : add new example + extend server API (#3296) f5ef5cfb18148131fcf45bdd2331f0db5ab7c3d0 40e07a60f9ce06e79f3ccd4c903eba300fb31b5e slaren slarengh@gmail.com 2023-09-30T18:12:57+02:00 GitHub noreply@github.com 2023-09-30T18:12:57+02:00 ggml-cuda : perform cublas mat mul of quantized types as f16 (#3412) 40e07a60f9ce06e79f3ccd4c903eba300fb31b5e bc34dd4f5b5a7c10ae3ed85a265ce6f2ed2fab79 slaren slarengh@gmail.com 2023-09-29T18:42:32+02:00 GitHub noreply@github.com 2023-09-29T18:42:32+02:00 llama.cpp : add documentation about rope_freq_base and scale values (#3401) bc34dd4f5b5a7c10ae3ed85a265ce6f2ed2fab79 2777a84be429401a2b7d33c2b6a4ada1f0776f1b Georgi Gerganov ggerganov@gmail.com 2023-09-29T19:05:18+03:00 GitHub noreply@github.com 2023-09-29T19:05:18+03:00 train : fix KQ_pos allocation (#3392) 2777a84be429401a2b7d33c2b6a4ada1f0776f1b 0a4a4a098261ddd26480371eaccfe90d1bf6488a Cebtenzzre cebtenzzre@gmail.com 2023-09-29T09:48:45-04:00 GitHub noreply@github.com 2023-09-29T16:48:45+03:00 llama : quantize up to 31% faster on Linux and Windows with mmap (#3206) 0a4a4a098261ddd26480371eaccfe90d1bf6488a 569550df20c1ede59ff195a6b6e900957ad84d16 BarfingLemurs 128182951+BarfingLemurs@users.noreply.github.com 2023-09-29T08:50:35-04:00 GitHub noreply@github.com 2023-09-29T15:50:35+03:00 readme : update hot topics + model links (#3399) 569550df20c1ede59ff195a6b6e900957ad84d16 c71bf2c45c5140203184f50b259828107658e900 Andrew Duffy a10y@users.noreply.github.com 2023-09-29T07:15:57-04:00 GitHub noreply@github.com 2023-09-29T14:15:57+03:00 readme : add link to grammars app (#3388) c71bf2c45c5140203184f50b259828107658e900 bc39553c901a91cfcb757863586250838c83eeab Jhen-Jie Hong iainst0409@gmail.com 2023-09-29T13:25:13+08:00 GitHub noreply@github.com 2023-09-29T08:25:13+03:00 swift : fix build on xcode 15 (#3387) bc39553c901a91cfcb757863586250838c83eeab 0ccfc62a96a6b59a8faa14d1b350493f4cd51ae2 Cebtenzzre cebtenzzre@gmail.com 2023-09-28T17:41:44-04:00 GitHub noreply@github.com 2023-09-28T17:41:44-04:00 build : enable more non-default compiler warnings (#3200) 0ccfc62a96a6b59a8faa14d1b350493f4cd51ae2 7f1a0fe709ea1a861da2f3759f58a28bf8953c12 Hua Jiang allenhjiang@outlook.com 2023-09-28T13:06:18-07:00 GitHub noreply@github.com 2023-09-28T23:06:18+03:00 ggml_tensor: update the structure comments. (#3283) 7f1a0fe709ea1a861da2f3759f58a28bf8953c12 16bc66d9479edd5ee12ec734973554d4493c5dfa Qu Zongfu 43257352+yancaoweidaode@users.noreply.github.com 2023-09-29T03:51:52+08:00 GitHub noreply@github.com 2023-09-28T22:51:52+03:00 ggml : release the requested thread pool resource (#3292) 16bc66d9479edd5ee12ec734973554d4493c5dfa 0512d66670de3f650c579519833c085014b0f200 slaren slarengh@gmail.com 2023-09-28T21:42:38+02:00 GitHub noreply@github.com 2023-09-28T22:42:38+03:00 llama.cpp : split llama_context_params into model and context params (#3301) 0512d66670de3f650c579519833c085014b0f200 0e76a8992c8200237bbc6471a53fb8796b3872f7 Eve 139727413+netrunnereve@users.noreply.github.com 2023-09-28T19:31:04Z GitHub noreply@github.com 2023-09-28T22:31:04+03:00 ci : multithreaded builds (#3311) 0e76a8992c8200237bbc6471a53fb8796b3872f7 2db94d98eda56982d80238840b0652b4137a2a84 xaedes xaedes@gmail.com 2023-09-28T20:40:11+02:00 GitHub noreply@github.com 2023-09-28T21:40:11+03:00 train : finetune LORA (#2632) 2db94d98eda56982d80238840b0652b4137a2a84 ecf90b1a5114034bc0939b3968f549fe4d63cf6d Cebtenzzre cebtenzzre@gmail.com 2023-09-28T14:30:31-04:00 GitHub noreply@github.com 2023-09-28T14:30:31-04:00 gguf : basic type checking in gguf_get_* (#3346) ecf90b1a5114034bc0939b3968f549fe4d63cf6d 2619109ad57d7a75388a9cce51e5da645410d92e Cebtenzzre cebtenzzre@gmail.com 2023-09-28T14:30:15-04:00 GitHub noreply@github.com 2023-09-28T14:30:15-04:00 gguf : make token scores and types optional (#3347) 2619109ad57d7a75388a9cce51e5da645410d92e ec893798b7a2a803466cc8f063051499ec3d96f7 Georgi Gerganov ggerganov@gmail.com 2023-09-28T19:36:36+03:00 GitHub noreply@github.com 2023-09-28T19:36:36+03:00 ci : disable freeBSD builds due to lack of VMs (#3381) ec893798b7a2a803466cc8f063051499ec3d96f7 45855b3f1c7bdd0320aa632334d0b3e8965c26c4 Georgi Gerganov ggerganov@gmail.com 2023-09-28T19:04:36+03:00 GitHub noreply@github.com 2023-09-28T19:04:36+03:00 llama : custom attention mask + parallel decoding + no context swaps (#3228) 45855b3f1c7bdd0320aa632334d0b3e8965c26c4 4aea3b846ec151cc6d08f93a8889eae13b286b06 Kevin Ji 1146876+kevinji@users.noreply.github.com 2023-09-28T09:11:32-04:00 GitHub noreply@github.com 2023-09-28T09:11:32-04:00 docs : mark code as Bash (#3375) 4aea3b846ec151cc6d08f93a8889eae13b286b06 da0400344be12074e67dcabc565140289cf7efaa Pierre Alexandre SCHEMBRI pa.schembri@gmail.com 2023-09-28T14:13:37+02:00 GitHub noreply@github.com 2023-09-28T15:13:37+03:00 readme : add Mistral AI release 0.1 (#3362) da0400344be12074e67dcabc565140289cf7efaa e519621010cac02c6fec0f8f3b16cda0591042c0 slaren slarengh@gmail.com 2023-09-28T12:08:28+02:00 GitHub noreply@github.com 2023-09-28T13:08:28+03:00 ggml-cuda : perform cublas fp16 matrix multiplication as fp16 (#3370) e519621010cac02c6fec0f8f3b16cda0591042c0 ac43576124a75c2de6e333ac31a3444ff9eb9458 Zhang Peiyuan a1286225768@gmail.com 2023-09-28T02:45:20+08:00 GitHub noreply@github.com 2023-09-27T20:45:20+02:00 convert : remove bug in convert.py permute function (#3364) ac43576124a75c2de6e333ac31a3444ff9eb9458 20c7e1e804690f3db58bd33eb56f8c6aa4735c63 Richard Roberson richardr1126@gmail.com 2023-09-27T10:25:12-06:00 GitHub noreply@github.com 2023-09-27T19:25:12+03:00 make-ggml.py : compatibility with more models and GGUF (#3290) 20c7e1e804690f3db58bd33eb56f8c6aa4735c63 dc6897404e141c74cbbf8030ecfebd74e1815411 Cebtenzzre cebtenzzre@gmail.com 2023-09-27T12:18:07-04:00 GitHub noreply@github.com 2023-09-27T12:18:07-04:00 gguf : fix a few general keys (#3341) dc6897404e141c74cbbf8030ecfebd74e1815411 527e57cfd8a9a26bf622c0510c21c2508a24be26 Rickard Hallerbäck rickard.hallerback@gmail.com 2023-09-27T17:48:33+02:00 GitHub noreply@github.com 2023-09-27T18:48:33+03:00 metal : reusing llama.cpp logging (#3152) 527e57cfd8a9a26bf622c0510c21c2508a24be26 ffe88a36a913e5792aa383f0726bdbcf632e7191 Jag Chadha jagtesh@gmail.com 2023-09-27T11:34:32-04:00 GitHub noreply@github.com 2023-09-27T18:34:32+03:00 build : add ACCELERATE_NEW_LAPACK to fix warning on macOS Sonoma (#3342) ffe88a36a913e5792aa383f0726bdbcf632e7191 99115f3fa654b593099c6719ad30e3f54ce231e1 BarfingLemurs 128182951+BarfingLemurs@users.noreply.github.com 2023-09-27T11:30:36-04:00 GitHub noreply@github.com 2023-09-27T18:30:36+03:00 readme : add some recent perplexity and bpw measurements to READMES, link for k-quants (#3340) 99115f3fa654b593099c6719ad30e3f54ce231e1 1726f9626f21f102d8e01df06c23a7f94add7990 DAN™ dranger003@gmail.com 2023-09-25T18:45:33-04:00 GitHub noreply@github.com 2023-09-25T18:45:33-04:00 cmake : fix build-info.h on MSVC (#3309) 1726f9626f21f102d8e01df06c23a7f94add7990 a98b1633d5a94d0aa84c7c16e1f8df5ac21fc850 2f38b454 dxf@protonmail.com 2023-09-26T02:24:52+08:00 GitHub noreply@github.com 2023-09-25T20:24:52+02:00 docs: Fix typo CLBlast_DIR var. (#3330) a98b1633d5a94d0aa84c7c16e1f8df5ac21fc850 c091cdfb24621710c617ea85c92fcd347d0bf340 Erik Scholz Green-Sky@users.noreply.github.com 2023-09-25T13:48:30+02:00 GitHub noreply@github.com 2023-09-25T13:48:30+02:00 nix : add cuda, use a symlinked toolkit for cmake (#3202) c091cdfb24621710c617ea85c92fcd347d0bf340 51a7cf5c6e490b2f51c82daa76c4ca4f8d845826 slaren slarengh@gmail.com 2023-09-23T21:48:24+02:00 GitHub noreply@github.com 2023-09-23T21:48:24+02:00 llama-bench : add README (#3317) 51a7cf5c6e490b2f51c82daa76c4ca4f8d845826 bedb92b603886768ad51e629f81eda15ff6b86f5 Cebtenzzre cebtenzzre@gmail.com 2023-09-23T05:28:50-04:00 GitHub noreply@github.com 2023-09-23T12:28:50+03:00 examples : fix RoPE defaults to match PR #3240 (#3315) bedb92b603886768ad51e629f81eda15ff6b86f5 bc9d3e3971e5607a10ff4c24e39568ce1ac87271 Kevin Ji 1146876+kevinji@users.noreply.github.com 2023-09-22T23:52:23-04:00 GitHub noreply@github.com 2023-09-22T23:52:23-04:00 scripts : use `/usr/bin/env` in shebang (#3313) bc9d3e3971e5607a10ff4c24e39568ce1ac87271 36b904e20003017f50108ae68359ef87a192dae2 Lee Drake b.lee.drake@gmail.com 2023-09-21T13:00:24-06:00 GitHub noreply@github.com 2023-09-21T21:00:24+02:00 Update README.md (#3289) 36b904e20003017f50108ae68359ef87a192dae2 324f3403d54ae4499a1d68623161015f7419fb76 shibe2 shibe@tuta.io 2023-09-21T22:10:26+04:00 GitHub noreply@github.com 2023-09-21T14:10:26-04:00 ggml-opencl.cpp: Make private functions static (#3300) 324f3403d54ae4499a1d68623161015f7419fb76 f56c418ab0a635c020bcb5bf44b8f00cb3c9e514 Edward Taylor edeetee@gmail.com 2023-09-21T21:08:20+12:00 GitHub noreply@github.com 2023-09-21T12:08:20+03:00 zig : fix for updated c lib (#3259) f56c418ab0a635c020bcb5bf44b8f00cb3c9e514 8185710a80531e9ee0c0cb99d3a9c9af1019ab67 yuiseki yuiseki@gmail.com 2023-09-21T17:57:40+09:00 GitHub noreply@github.com 2023-09-21T11:57:40+03:00 embedding : update README.md (#3224) 8185710a80531e9ee0c0cb99d3a9c9af1019ab67 7eb41179edc56083ef4eb2df7967ac9ff38b34fb Johannes Gäßler johannesg@5d6.de 2023-09-21T10:43:53+02:00 GitHub noreply@github.com 2023-09-21T11:43:53+03:00 CUDA: use only 1 thread if fully offloaded (#2915) 7eb41179edc56083ef4eb2df7967ac9ff38b34fb a5661d7e71d15b8dfc81bc0510ba912ebe85dfa3 Georgi Gerganov ggerganov@gmail.com 2023-09-20T20:48:22+03:00 GitHub noreply@github.com 2023-09-20T20:48:22+03:00 readme : update hot topics a5661d7e71d15b8dfc81bc0510ba912ebe85dfa3 65c2c1c5ab7c5089dbc6d10bc49b9c58f0164317 Cebtenzzre cebtenzzre@gmail.com 2023-09-20T12:12:47-04:00 GitHub noreply@github.com 2023-09-20T12:12:47-04:00 llama : allow gguf RoPE keys to be overridden with defaults (#3240) 65c2c1c5ab7c5089dbc6d10bc49b9c58f0164317 80834daecf4b9021770361a6d5e1b9c7a60e6854 Cebtenzzre cebtenzzre@gmail.com 2023-09-20T12:06:08-04:00 GitHub noreply@github.com 2023-09-20T12:06:08-04:00 benchmark-matmult : do not use integer abs() on a float (#3277) 80834daecf4b9021770361a6d5e1b9c7a60e6854 a40f2b656fab364ce0aff98dbefe9bd9c3721cc9 kang tpdns9032100@gmail.com 2023-09-20T22:48:22+09:00 GitHub noreply@github.com 2023-09-20T15:48:22+02:00 flake : Restore default package's buildInputs (#3262) a40f2b656fab364ce0aff98dbefe9bd9c3721cc9 d119c04c159d015a93567df7e73e0e45a22d0f1d Alon alonfaraj@gmail.com 2023-09-20T15:06:36+03:00 GitHub noreply@github.com 2023-09-20T14:06:36+02:00 CI: FreeBSD fix (#3258) d119c04c159d015a93567df7e73e0e45a22d0f1d 8781013ef654270cbead3e0011e33a6d690fb168 Georgi Gerganov ggerganov@gmail.com 2023-09-20T10:02:39+03:00 GitHub noreply@github.com 2023-09-20T10:02:39+03:00 examples : fix benchmark-matmult (#1554) 8781013ef654270cbead3e0011e33a6d690fb168 7ddf185537b712ea0ccbc5f222ee92bed654914e Cebtenzzre cebtenzzre@gmail.com 2023-09-18T10:03:53-04:00 GitHub noreply@github.com 2023-09-18T10:03:53-04:00 make : restore build-info.h dependency for several targets (#3205) 7ddf185537b712ea0ccbc5f222ee92bed654914e ee66942d7ef7c259528158f9a3bd1c314984d32f Erik Scholz Green-Sky@users.noreply.github.com 2023-09-18T02:21:47+02:00 GitHub noreply@github.com 2023-09-18T02:21:47+02:00 ci : switch cudatoolkit install on windows to networked (#3236) ee66942d7ef7c259528158f9a3bd1c314984d32f 111163e2463171891680feed94371eb9becd9817 Johannes Gäßler johannesg@5d6.de 2023-09-17T23:35:20+02:00 GitHub noreply@github.com 2023-09-17T23:35:20+02:00 CUDA: fix peer access logic (#3231) 111163e2463171891680feed94371eb9becd9817 8b428c9bc84be6887d904600d1298b28baffd552 Johannes Gäßler johannesg@5d6.de 2023-09-17T16:37:53+02:00 GitHub noreply@github.com 2023-09-17T16:37:53+02:00 CUDA: enable peer access between devices (#2470) 8b428c9bc84be6887d904600d1298b28baffd552 578d8c8f5cb72f354bc115ba230ee5b2d803eee7 slaren slarengh@gmail.com 2023-09-17T14:33:28+02:00 GitHub noreply@github.com 2023-09-17T14:33:28+02:00 llama.cpp : show model size and BPW on load (#3223) 578d8c8f5cb72f354bc115ba230ee5b2d803eee7 b541b4f0b1d4d9871c831e47cd5ff661039d6101 Johannes Gäßler johannesg@5d6.de 2023-09-17T14:16:22+02:00 GitHub noreply@github.com 2023-09-17T14:16:22+02:00 CUDA: fix scratch malloced on non-main device (#3220) b541b4f0b1d4d9871c831e47cd5ff661039d6101 5dbc2b3213126a31d3be4ade8ca042cb93019682 IsaacDynamo 61521674+IsaacDynamo@users.noreply.github.com 2023-09-16T19:35:25+02:00 GitHub noreply@github.com 2023-09-16T19:35:25+02:00 Enable BUILD_SHARED_LIBS=ON on all Windows builds (#3215) 5dbc2b3213126a31d3be4ade8ca042cb93019682 b08e75baea294e366628b898e85c0bd359b58115 Vlad spitfireage@gmail.com 2023-09-16T17:55:43+03:00 GitHub noreply@github.com 2023-09-16T16:55:43+02:00 Enable build with CUDA 11.0 (make) (#3132) b08e75baea294e366628b898e85c0bd359b58115 e6616cf0db2b63189fc34d0076f654af9adecdf8 goerch jhr.walter@t-online.de 2023-09-16T13:41:33+02:00 GitHub noreply@github.com 2023-09-16T13:41:33+02:00 Fixing the last deviations from sentencepiece indicated by test-tokenizer-1 (#3170) e6616cf0db2b63189fc34d0076f654af9adecdf8 3aefaab9e59335ebb07d5205dbc8633efd680e58 Cebtenzzre cebtenzzre@gmail.com 2023-09-15T16:59:49-04:00 GitHub noreply@github.com 2023-09-15T16:59:49-04:00 examples : add compiler version and target to build info (#2998) 3aefaab9e59335ebb07d5205dbc8633efd680e58 69eb67e28275cd2d57693405f768754a7b2245ad Cebtenzzre cebtenzzre@gmail.com 2023-09-15T15:38:27-04:00 GitHub noreply@github.com 2023-09-15T15:38:27-04:00 check C++ code with -Wmissing-declarations (#3184) 69eb67e28275cd2d57693405f768754a7b2245ad 4fe09dfe665c58a753dc9eb638dd4dca1cd35488 Cebtenzzre cebtenzzre@gmail.com 2023-09-15T15:18:15-04:00 GitHub noreply@github.com 2023-09-15T15:18:15-04:00 fix build numbers by setting fetch-depth=0 (#3197) 4fe09dfe665c58a753dc9eb638dd4dca1cd35488 80291a1d02a07f7f66666fb576c5b1e75aa48b46 Meng Zhang meng@tabbyml.com 2023-09-16T03:02:13+08:00 GitHub noreply@github.com 2023-09-15T22:02:13+03:00 llama : add support for StarCoder model architectures (#3187) 80291a1d02a07f7f66666fb576c5b1e75aa48b46 c6f1491da032238241e01021c8c58d7b540a043f Cebtenzzre cebtenzzre@gmail.com 2023-09-15T14:02:01-04:00 GitHub noreply@github.com 2023-09-15T21:02:01+03:00 common : do not use GNU zero-length __VA_ARGS__ extension (#3195) c6f1491da032238241e01021c8c58d7b540a043f e3d87a6c36eadd84d58763143c6d56a0c771ca40 Georgi Gerganov ggerganov@gmail.com 2023-09-15T20:17:24+03:00 GitHub noreply@github.com 2023-09-15T20:17:24+03:00 metal : fix bug in soft_max kernels (out-of-bounds access) (#3194) e3d87a6c36eadd84d58763143c6d56a0c771ca40 8c00b7a6ff38e27fa1e471452b8a480913772c2a Cebtenzzre cebtenzzre@gmail.com 2023-09-15T12:29:02-04:00 GitHub noreply@github.com 2023-09-15T12:29:02-04:00 convert : make ftype optional in simple scripts (#3185) 8c00b7a6ff38e27fa1e471452b8a480913772c2a 7e50d34be68aae2cc766203703dd188e910e033a Georgi Gerganov ggerganov@gmail.com 2023-09-15T19:06:03+03:00 GitHub noreply@github.com 2023-09-15T19:06:03+03:00 sync : ggml (Metal F32 support + reduce ggml-alloc size) (#3192) 7e50d34be68aae2cc766203703dd188e910e033a 235f7c193b02dacfb56319e41a28684b3a2c6db0 Engininja2 139037756+Engininja2@users.noreply.github.com 2023-09-15T06:24:30-06:00 GitHub noreply@github.com 2023-09-15T15:24:30+03:00 cmake : fix building shared libs for clang (rocm) on windows (#3176) 235f7c193b02dacfb56319e41a28684b3a2c6db0 a51b68765799e75e17c7622f376bfbeb66f1bd70 Evgeny Kurnevsky kurnevsky@gmail.com 2023-09-15T10:10:22+02:00 GitHub noreply@github.com 2023-09-15T11:10:22+03:00 flake : use pkg-config instead of pkgconfig (#3188) a51b68765799e75e17c7622f376bfbeb66f1bd70 76164fe2e65c058e9ee2c3afd0ad6b182ca57e25 Georgi Gerganov ggerganov@gmail.com 2023-09-15T11:09:24+03:00 GitHub noreply@github.com 2023-09-15T11:09:24+03:00 metal : relax conditions on fast matrix multiplication kernel (#3168) 76164fe2e65c058e9ee2c3afd0ad6b182ca57e25 c2ab6fe661af9834ca6dd75f14e2439938cc22ac Andrei abetlen@gmail.com 2023-09-15T04:07:40-04:00 GitHub noreply@github.com 2023-09-15T11:07:40+03:00 cmake : fix llama.h location when built outside of root directory (#3179) c2ab6fe661af9834ca6dd75f14e2439938cc22ac 2d770505a89a99ce78a5950cf14fc06d3176ffa4 Ali Tariq ali.tariq@10xengineers.ai 2023-09-15T13:06:56+05:00 GitHub noreply@github.com 2023-09-15T11:06:56+03:00 ci : Cloud-V for RISC-V builds (#3160) 2d770505a89a99ce78a5950cf14fc06d3176ffa4 98311c427739e3b06527c3ce6b5c021ab6692740 Roland 14355895+rbur0425@users.noreply.github.com 2023-09-15T03:28:45-04:00 GitHub noreply@github.com 2023-09-15T10:28:45+03:00 llama : remove mtest (#3177) 98311c427739e3b06527c3ce6b5c021ab6692740 feea179e9f9921e96e8fb1b8855d4a8f83682455 Cebtenzzre cebtenzzre@gmail.com 2023-09-14T21:09:53-04:00 GitHub noreply@github.com 2023-09-14T21:09:53-04:00 llama : make quantize example up to 2.7x faster (#3115) feea179e9f9921e96e8fb1b8855d4a8f83682455 769266a543f68377a1d904ec2a8c27b38a4025ab jneem joeneeman@gmail.com 2023-09-14T13:54:47-05:00 GitHub noreply@github.com 2023-09-14T21:54:47+03:00 flake : allow $out/include to already exist (#3175) 769266a543f68377a1d904ec2a8c27b38a4025ab cf8238e7f43cb82a36426af392037e85cd2a3df6 Andrei abetlen@gmail.com 2023-09-14T13:38:16-04:00 GitHub noreply@github.com 2023-09-14T20:38:16+03:00 cmake : compile ggml-rocm with -fpic when building shared library (#3158) cf8238e7f43cb82a36426af392037e85cd2a3df6 4b8560e72a936b5d536ebd1e7a5dd579984769f3 Asbjørn Olling asbjornolling@gmail.com 2023-09-14T19:25:00+02:00 GitHub noreply@github.com 2023-09-14T20:25:00+03:00 flake : include llama.h in nix output (#3159) 4b8560e72a936b5d536ebd1e7a5dd579984769f3 83a53b753a9499a2a3535c93975b430cb2c828a9 Cebtenzzre cebtenzzre@gmail.com 2023-09-14T13:22:47-04:00 GitHub noreply@github.com 2023-09-14T20:22:47+03:00 make : fix clang++ detection, move some definitions to CPPFLAGS (#3155) 83a53b753a9499a2a3535c93975b430cb2c828a9 5c872dbca2c7979b1f6dafc97db0774b8bbf9372 Alon alonfaraj@gmail.com 2023-09-14T20:21:25+03:00 GitHub noreply@github.com 2023-09-14T19:21:25+02:00 CI: add FreeBSD & simplify CUDA windows (#3053) 5c872dbca2c7979b1f6dafc97db0774b8bbf9372 990a5e226a1a0ac858abe3aa7e5f3b000d4fa665 akawrykow 142945436+akawrykow@users.noreply.github.com 2023-09-14T10:19:42-07:00 GitHub noreply@github.com 2023-09-14T20:19:42+03:00 falcon : use stated vocab size (#2914) 990a5e226a1a0ac858abe3aa7e5f3b000d4fa665 980ab41afba96106cd29cdf3aa6f948c251cb71f bandoti 141645996+bandoti@users.noreply.github.com 2023-09-14T14:04:40-03:00 GitHub noreply@github.com 2023-09-14T20:04:40+03:00 cmake : add relocatable Llama package (#2960) 980ab41afba96106cd29cdf3aa6f948c251cb71f e394084166baac09e8ee9a08a4686f907f7e5291 dylan canardleteer@users.noreply.github.com 2023-09-14T09:47:00-07:00 GitHub noreply@github.com 2023-09-14T19:47:00+03:00 docker : add gpu image CI builds (#3103) e394084166baac09e8ee9a08a4686f907f7e5291 4c8643dd6ea1a163bc5979cb69c1e7ab0975bc93 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-09-14T10:32:26-06:00 GitHub noreply@github.com 2023-09-14T19:32:26+03:00 gguf-py : support identity operation in TensorNameMap (#3095) 4c8643dd6ea1a163bc5979cb69c1e7ab0975bc93 35f73049af6c676a106a5a990a819ae0bc3fcd7d jameswu2014 545426914@qq.com 2023-09-15T00:32:10+08:00 GitHub noreply@github.com 2023-09-14T12:32:10-04:00 feature : support Baichuan serial models (#3009) 35f73049af6c676a106a5a990a819ae0bc3fcd7d 71ca2fad7d6c0ef95ef9944fb3a1a843e481f314 Leng Yue lengyue@lengyue.me 2023-09-14T09:14:44-07:00 GitHub noreply@github.com 2023-09-14T19:14:44+03:00 speculative : add heuristic algorithm (#3006) 71ca2fad7d6c0ef95ef9944fb3a1a843e481f314 1b6c650d16048d6427dd502a9627e72837265844 goerch jhr.walter@t-online.de 2023-09-13T15:19:44+02:00 GitHub noreply@github.com 2023-09-13T16:19:44+03:00 whisper : tokenizer fix + re-enable tokenizer test for LLaMa (#3096) 1b6c650d16048d6427dd502a9627e72837265844 0a5eebb45d5697127b84418576dc479c400c4b3d Tristan Ross rosscomputerguy@protonmail.com 2023-09-13T06:08:52-07:00 GitHub noreply@github.com 2023-09-13T16:08:52+03:00 cmake : add a compiler flag check for FP16 format (#3086) 0a5eebb45d5697127b84418576dc479c400c4b3d 84e723653ca99d51a74b454984acf2c077468561 Johannes Gäßler johannesg@5d6.de 2023-09-13T11:20:24+02:00 GitHub noreply@github.com 2023-09-13T11:20:24+02:00 CUDA: mul_mat_q RDNA2 tunings (#2910) 84e723653ca99d51a74b454984acf2c077468561 b52b29ab9d601bb298050bcd2261169bc917ba2c FK sozforex@gmail.com 2023-09-13T08:50:46+02:00 GitHub noreply@github.com 2023-09-13T08:50:46+02:00 speculative: add --n-gpu-layers-draft option (#3063) b52b29ab9d601bb298050bcd2261169bc917ba2c 4f7cd6ba9c88d3ca9a207b6e04f8b2b1efd707b8 Eric Sommerlade es0m@users.noreply.github.com 2023-09-13T02:54:20+01:00 GitHub noreply@github.com 2023-09-12T21:54:20-04:00 arm64 support for windows (#3007) 4f7cd6ba9c88d3ca9a207b6e04f8b2b1efd707b8 89e89599fd095172f8d67903b5e227467420f036 Johannes Gäßler johannesg@5d6.de 2023-09-13T00:15:33+02:00 GitHub noreply@github.com 2023-09-13T00:15:33+02:00 CUDA: fix LoRAs (#3130) 89e89599fd095172f8d67903b5e227467420f036 d54a4027a6ebda98ab0fef7fa0c2247d0bef132a Johannes Gäßler johannesg@5d6.de 2023-09-11T22:58:41+02:00 GitHub noreply@github.com 2023-09-11T22:58:41+02:00 CUDA: fix mul_mat_q not used for output tensor (#3127) d54a4027a6ebda98ab0fef7fa0c2247d0bef132a 1b0d09259e37898c519edb6c52d58f4d096f10bd Johannes Gäßler johannesg@5d6.de 2023-09-11T19:55:51+02:00 GitHub noreply@github.com 2023-09-11T19:55:51+02:00 CUDA: lower GPU latency + fix Windows performance (#3110) 1b0d09259e37898c519edb6c52d58f4d096f10bd 8a4ca9af569853023ce87f047eb5165df13f2ff1 Jhen-Jie Hong iainst0409@gmail.com 2023-09-11T19:49:06+08:00 GitHub noreply@github.com 2023-09-11T19:49:06+08:00 cmake : support build for iOS/tvOS (#3116) 8a4ca9af569853023ce87f047eb5165df13f2ff1 f31b6f4e2d6def3c0bd7c75f75c0c1e8698e0589 Johannes Gäßler johannesg@5d6.de 2023-09-11T13:00:24+02:00 GitHub noreply@github.com 2023-09-11T13:00:24+02:00 CUDA: add device number to error messages (#3112) f31b6f4e2d6def3c0bd7c75f75c0c1e8698e0589 6eeb4d90839bac1e6085e5544654ab5c319ad09a Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-09-11T09:30:11+02:00 GitHub noreply@github.com 2023-09-11T10:30:11+03:00 metal : PP speedup (#3084) 6eeb4d90839bac1e6085e5544654ab5c319ad09a 21ac3a1503001020122db5dce6adf34b761675f5 Erik Scholz Green-Sky@users.noreply.github.com 2023-09-10T17:06:53+02:00 GitHub noreply@github.com 2023-09-10T11:06:53-04:00 convert: remove most of the n_mult usage in convert.py (#3098) 21ac3a1503001020122db5dce6adf34b761675f5 4fd54779550e43e2a29f6840ebcf8f395a2f879e kchro3 62481661+kchro3@users.noreply.github.com 2023-09-09T02:12:10-07:00 GitHub noreply@github.com 2023-09-09T17:12:10+08:00 metal : support for Swift (#3078) 4fd54779550e43e2a29f6840ebcf8f395a2f879e ec2a24fedf1de8ebd5f170016953b09ff2806924 Jhen-Jie Hong iainst0409@gmail.com 2023-09-09T16:46:04+08:00 GitHub noreply@github.com 2023-09-09T11:46:04+03:00 metal : support build for iOS/tvOS (#3089) ec2a24fedf1de8ebd5f170016953b09ff2806924 7d99aca759f2f8a1ff39f3bb02a840f69863428b takov751 40316768+takov751@users.noreply.github.com 2023-09-08T17:06:26+01:00 GitHub noreply@github.com 2023-09-08T19:06:26+03:00 flake : add train-text-from-scratch to flake.nix (#3042) 7d99aca759f2f8a1ff39f3bb02a840f69863428b ba7ffbb2517ff8cf4c689f94a9ad866f3ee71225 Ikko Eltociear Ashimine eltociear@gmail.com 2023-09-09T01:04:32+09:00 GitHub noreply@github.com 2023-09-08T19:04:32+03:00 readme : fix typo (#3043) ba7ffbb2517ff8cf4c689f94a9ad866f3ee71225 e64f5b55783e910d8287363895d652b4bea6527a Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-09-08T18:01:04+02:00 GitHub noreply@github.com 2023-09-08T19:01:04+03:00 metal : Q3_K speedup (#2995) e64f5b55783e910d8287363895d652b4bea6527a 94f10b91ed69980f299441e49c8dbdb448f0ccc6 Cebtenzzre cebtenzzre@gmail.com 2023-09-08T11:43:35-04:00 GitHub noreply@github.com 2023-09-08T11:43:35-04:00 examples : make n_ctx warning work again (#3066) 94f10b91ed69980f299441e49c8dbdb448f0ccc6 b3e9852e471d12cbbe5dad20c81c4766d969739a Georgi Gerganov ggerganov@gmail.com 2023-09-08T18:18:04+03:00 GitHub noreply@github.com 2023-09-08T18:18:04+03:00 readme : update hot tpoics b3e9852e471d12cbbe5dad20c81c4766d969739a cb6c44c5e045709b6bb5cc9bb8c9be107c771a78 Georgi Gerganov ggerganov@gmail.com 2023-09-08T17:58:07+03:00 GitHub noreply@github.com 2023-09-08T17:58:07+03:00 sync : ggml (CUDA GLM RoPE + POSIX) (#3082) cb6c44c5e045709b6bb5cc9bb8c9be107c771a78 a21baeb12202a9020b48c53beaaf4b355228e8ba Przemysław Pawełczyk przemoc@gmail.com 2023-09-08T14:09:21+02:00 GitHub noreply@github.com 2023-09-08T15:09:21+03:00 build : do not use _GNU_SOURCE gratuitously (#2035) a21baeb12202a9020b48c53beaaf4b355228e8ba 6ff712a6d1a0c85d996e2f681df57a2554cfe5c1 hongbo.mo 352280764@qq.com 2023-09-08T18:57:55+08:00 GitHub noreply@github.com 2023-09-08T13:57:55+03:00 docker : add git to full-cuda.Dockerfile main-cuda.Dockerfile (#3044) 6ff712a6d1a0c85d996e2f681df57a2554cfe5c1 ebc96086af49fe70108cafcea6ab4bebd658a41a Yui dev@sleepyyui.com 2023-09-08T12:32:55+02:00 GitHub noreply@github.com 2023-09-08T12:32:55+02:00 Update deprecated GGML TheBloke links to GGUF (#3079) ebc96086af49fe70108cafcea6ab4bebd658a41a 7f412dab9c8801f5d37904f7dce1faf4c2b43b42 slaren slarengh@gmail.com 2023-09-08T04:04:56+02:00 GitHub noreply@github.com 2023-09-08T04:04:56+02:00 ggml-alloc : correctly check mmap return value for errors (#3075) 7f412dab9c8801f5d37904f7dce1faf4c2b43b42 6336d834ec7bff3e93e24182c0f609d2f2bdce26 Kunshang Ji kunshang.ji@intel.com 2023-09-08T09:46:56+08:00 GitHub noreply@github.com 2023-09-08T03:46:56+02:00 enable CPU HBM (#2603) 6336d834ec7bff3e93e24182c0f609d2f2bdce26 00d62adb79bf914a95fb9a2e8f42f3029e76d62c Cebtenzzre cebtenzzre@gmail.com 2023-09-07T14:27:42-04:00 GitHub noreply@github.com 2023-09-07T14:27:42-04:00 convert : fix F32 ftype not being saved (#3048) 00d62adb79bf914a95fb9a2e8f42f3029e76d62c 4fa2cc1750b861880de42515cb19c13b2d776ee2 Cebtenzzre cebtenzzre@gmail.com 2023-09-07T13:22:29-04:00 GitHub noreply@github.com 2023-09-07T13:22:29-04:00 fix some warnings from gcc and clang-tidy (#3038) 4fa2cc1750b861880de42515cb19c13b2d776ee2 5ffab089a54bc06ae4a9ab533893b558756a1e80 Cebtenzzre cebtenzzre@gmail.com 2023-09-07T10:15:01-04:00 GitHub noreply@github.com 2023-09-07T10:15:01-04:00 make : improve test target (#3031) 5ffab089a54bc06ae4a9ab533893b558756a1e80 15b67a66c2f2d6032415b28a699b5131962318f1 Cebtenzzre cebtenzzre@gmail.com 2023-09-07T10:13:50-04:00 GitHub noreply@github.com 2023-09-07T10:13:50-04:00 make : fix CPPFLAGS (#3035) 15b67a66c2f2d6032415b28a699b5131962318f1 be8c9c245bd129ebabb80e0a7a8dd7daeb4d30af slaren slarengh@gmail.com 2023-09-07T15:52:34+02:00 GitHub noreply@github.com 2023-09-07T15:52:34+02:00 llama-bench : use two tokens in the warmup run for prompt evals (#3059) be8c9c245bd129ebabb80e0a7a8dd7daeb4d30af be6beeb8d75294552c4918fce06d7b84eebf3d79 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-09-07T15:45:01+02:00 GitHub noreply@github.com 2023-09-07T16:45:01+03:00 metal : parallel RoPE on Metal (#3024) be6beeb8d75294552c4918fce06d7b84eebf3d79 c4f496648c1e32efeb714200e7eae7fc7cfbb223 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-09-07T15:42:42+02:00 GitHub noreply@github.com 2023-09-07T16:42:42+03:00 metal : correct fix of kernel_norm (#3060) c4f496648c1e32efeb714200e7eae7fc7cfbb223 fec2fb19e4229aac58c98171c46e77144b99f8a3 Georgi Gerganov ggerganov@gmail.com 2023-09-07T15:49:09+03:00 GitHub noreply@github.com 2023-09-07T15:49:09+03:00 metal : fix kernel_norm (fixes Falcon on Metal) (#3057) fec2fb19e4229aac58c98171c46e77144b99f8a3 178b1850ebd21b349cebbee887950e435c5aa2d3 Przemysław Pawełczyk przemoc@gmail.com 2023-09-07T10:15:06+02:00 GitHub noreply@github.com 2023-09-07T11:15:06+03:00 ggml : posixify madvise and pagesize (#3037) 178b1850ebd21b349cebbee887950e435c5aa2d3 ea2c85d5d2a93d39d0172222917f3195f0e456ff Georgi Gerganov ggerganov@gmail.com 2023-09-06T12:40:57+03:00 Georgi Gerganov ggerganov@gmail.com 2023-09-06T12:40:57+03:00 k-quants : fix zero-weight guard in Q6_K (ref #3040) ea2c85d5d2a93d39d0172222917f3195f0e456ff 9912b9efc8922321fe7202ab42ba913833cbe9cd Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-09-06T02:49:11-06:00 GitHub noreply@github.com 2023-09-06T02:49:11-06:00 convert-llama-ggml-to-gguf: Try to handle files older than GGJTv3 (#3023) 9912b9efc8922321fe7202ab42ba913833cbe9cd 9e2023156e5b5acabaf8632e66c6ae68d3703c31 Cebtenzzre cebtenzzre@gmail.com 2023-09-05T18:21:10-04:00 GitHub noreply@github.com 2023-09-05T18:21:10-04:00 build : add LLAMA_METAL_NDEBUG flag (#3033) 9e2023156e5b5acabaf8632e66c6ae68d3703c31 de2fe892af92a5c7b5ef1beb7efbc0524343fbab Cebtenzzre cebtenzzre@gmail.com 2023-09-05T15:12:00-04:00 GitHub noreply@github.com 2023-09-05T15:12:00-04:00 make : use new flag variables for recent changes (#3019) de2fe892af92a5c7b5ef1beb7efbc0524343fbab c9c3220c485c7bea740a07cda7343677fb3beaae Cebtenzzre cebtenzzre@gmail.com 2023-09-05T15:10:27-04:00 GitHub noreply@github.com 2023-09-05T15:10:27-04:00 examples : replace fprintf to stdout with printf (#3017) c9c3220c485c7bea740a07cda7343677fb3beaae d59bd97065cd7ded6c4ecab54b1d5e0b1b11e318 Erik Scholz Green-Sky@users.noreply.github.com 2023-09-05T19:41:00+02:00 GitHub noreply@github.com 2023-09-05T19:41:00+02:00 convert: fix convert.py not working with int filename_stem (#3028) d59bd97065cd7ded6c4ecab54b1d5e0b1b11e318 35938ee3b0c16f1fbbf240dae21e0228864b938c Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-09-05T09:55:33+02:00 GitHub noreply@github.com 2023-09-05T09:55:33+02:00 Guard against all weights in a super-block being zero (#3010) 35938ee3b0c16f1fbbf240dae21e0228864b938c 921772104ba2219bfdc2b2980d05ebc0aa0c92a4 Georgi Gerganov ggerganov@gmail.com 2023-09-05T10:46:39+03:00 Georgi Gerganov ggerganov@gmail.com 2023-09-05T10:46:39+03:00 llama : update logic for number of threads when using BLAS 921772104ba2219bfdc2b2980d05ebc0aa0c92a4 2ba85c8609309a59d49c45ab43c31800b7ba141c Georgi Gerganov ggerganov@gmail.com 2023-09-05T08:46:17+03:00 GitHub noreply@github.com 2023-09-05T08:46:17+03:00 speculative : add grammar support (#2991) 2ba85c8609309a59d49c45ab43c31800b7ba141c e36ecdccc8754783f93ad3ac8a09e540101f2ca0 Georgi Gerganov ggerganov@gmail.com 2023-09-04T22:50:50+03:00 Georgi Gerganov ggerganov@gmail.com 2023-09-04T22:50:50+03:00 py : minor e36ecdccc8754783f93ad3ac8a09e540101f2ca0 bd33e5ab92e7f214205792fc1cd9ca28e810f897 Georgi Gerganov ggerganov@gmail.com 2023-09-04T22:26:24+03:00 GitHub noreply@github.com 2023-09-04T22:26:24+03:00 build : on Mac OS enable Metal by default (#2901) bd33e5ab92e7f214205792fc1cd9ca28e810f897 31035681445181fb414e0def7ec3f84462b3bd97 slaren slarengh@gmail.com 2023-09-04T14:59:52+02:00 GitHub noreply@github.com 2023-09-04T14:59:52+02:00 ggml-opencl : store GPU buffer in ggml_tensor::extra (#2994) 31035681445181fb414e0def7ec3f84462b3bd97 5b8530d88c489f9d0c0ef3d0886b369f655b792e Cebtenzzre cebtenzzre@gmail.com 2023-09-04T06:40:18-04:00 GitHub noreply@github.com 2023-09-04T13:40:18+03:00 llama-bench : make cpp file non-executable (#2999) 5b8530d88c489f9d0c0ef3d0886b369f655b792e e4386f417faf894f6706eec005e24d142b577fcb Leng Yue lengyue@lengyue.me 2023-09-04T03:39:57-07:00 GitHub noreply@github.com 2023-09-04T13:39:57+03:00 make : add speculative example (#3003) e4386f417faf894f6706eec005e24d142b577fcb 35195689cd835464779c247b1c22ab9247418fd1 Aarni Koskela akx@iki.fi 2023-09-04T10:28:55+02:00 GitHub noreply@github.com 2023-09-04T16:28:55+08:00 server : add a subtle loading animation to the edit box (#2466) 35195689cd835464779c247b1c22ab9247418fd1 cf9b08485c4c2d4d945c6e74fe20f273a38b6104 Jiahao Li liplus17@163.com 2023-09-04T14:53:30+08:00 GitHub noreply@github.com 2023-09-04T08:53:30+02:00 2x faster (rms) norm cuda kernels (3.7% e2e improvement) (#2985) cf9b08485c4c2d4d945c6e74fe20f273a38b6104 47068e517004d90f13c16352bb3b4cafd53a00cd slaren slarengh@gmail.com 2023-09-03T20:34:09+02:00 GitHub noreply@github.com 2023-09-03T20:34:09+02:00 ggml-alloc : use virtual memory for measurement (#2973) 47068e517004d90f13c16352bb3b4cafd53a00cd 8f429fa5111901f9646cf998643ac5310846d487 Georgi Gerganov ggerganov@gmail.com 2023-09-03T15:12:08+03:00 GitHub noreply@github.com 2023-09-03T15:12:08+03:00 speculative : PoC for speeding-up inference via speculative sampling (#2926) 8f429fa5111901f9646cf998643ac5310846d487 6519e9c99cffbad19b31bcba86df48c500628c09 Georgi Gerganov ggerganov@gmail.com 2023-09-03T13:42:56+03:00 Georgi Gerganov ggerganov@gmail.com 2023-09-03T13:43:17+03:00 perplexity : fix ETA by warming up the model with an empty run 6519e9c99cffbad19b31bcba86df48c500628c09 b7f2aa9e512c3be2e863d877cbb1056d7c4a03f8 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-09-03T04:38:43-06:00 GitHub noreply@github.com 2023-09-03T04:38:43-06:00 gguf(python): Fix special vocab handling when id < 0 (#2984) b7f2aa9e512c3be2e863d877cbb1056d7c4a03f8 73a12a6344d5da4d8e2eba5d12221b8bc6895931 Georgi Gerganov ggerganov@gmail.com 2023-09-03T13:23:33+03:00 GitHub noreply@github.com 2023-09-03T13:23:33+03:00 metal : restore 363f0bf and fix reduce in F16_F32 kernels (#2986) 73a12a6344d5da4d8e2eba5d12221b8bc6895931 37301347767d555d0a66c043ce4ef6ead8e61c55 Alon alonfaraj@gmail.com 2023-09-03T13:19:01+03:00 GitHub noreply@github.com 2023-09-03T13:19:01+03:00 cov : disable comment in PRs (#2989) 37301347767d555d0a66c043ce4ef6ead8e61c55 d9151e6f570eb20bfd54427bd8a337d9b1a08018 opparco parco.opaai@gmail.com 2023-09-03T19:18:09+09:00 GitHub noreply@github.com 2023-09-03T13:18:09+03:00 llama : fix bpe tokenize from byte (#2889) d9151e6f570eb20bfd54427bd8a337d9b1a08018 afc43d5f82588d2ed71ea104e8262f5e5da13980 Georgi Gerganov ggerganov@gmail.com 2023-09-03T12:40:56+03:00 Georgi Gerganov ggerganov@gmail.com 2023-09-03T12:40:56+03:00 metal : revert 6af0bab until we fix it afc43d5f82588d2ed71ea104e8262f5e5da13980 6460f758dbd472653296044d36bed8c4554988f5 Alon alonfaraj@gmail.com 2023-09-03T11:48:49+03:00 GitHub noreply@github.com 2023-09-03T11:48:49+03:00 cov : add Code Coverage and codecov.io integration (#2928) 6460f758dbd472653296044d36bed8c4554988f5 ca82cf7bac0c91d03e3d320b3a865dd006f854ac Wentai Zhang rchardx@gmail.com 2023-09-03T16:46:44+08:00 GitHub noreply@github.com 2023-09-03T11:46:44+03:00 opencl : fix a bug in ggml_cl_pool_malloc() for ggml_cl_mul_mat_f32() (#2955) ca82cf7bac0c91d03e3d320b3a865dd006f854ac 6a31a3bd9806c85ed08266f6ab65181da0f30d03 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-09-03T11:06:22+03:00 GitHub noreply@github.com 2023-09-03T11:06:22+03:00 metal : more optimizations (#2959) 6a31a3bd9806c85ed08266f6ab65181da0f30d03 cff7b0bf07cb46e1ad4fd199f6bdeb538925c8c4 kchro3 62481661+kchro3@users.noreply.github.com 2023-09-02T23:21:05-07:00 GitHub noreply@github.com 2023-09-03T09:21:05+03:00 swift : add support for k-quants (#2983) cff7b0bf07cb46e1ad4fd199f6bdeb538925c8c4 340af42f09a80e32f4998857b4f0543e41124525 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-09-02T23:52:13-06:00 GitHub noreply@github.com 2023-09-03T08:52:13+03:00 convert.py : BPE fixes (#2938) 340af42f09a80e32f4998857b4f0543e41124525 c42f0ec6b344e14bd81c8612ab1445b3ff77358b Ido S ido.pluto@gmail.com 2023-09-03T08:50:51+03:00 GitHub noreply@github.com 2023-09-03T08:50:51+03:00 docs : add `catai` to `README.md` (#2967) c42f0ec6b344e14bd81c8612ab1445b3ff77358b 2753415afdaf22a18c49608bd9d93cfffc05d435 momonga 115213907+mmnga@users.noreply.github.com 2023-09-03T14:36:28+09:00 GitHub noreply@github.com 2023-09-03T08:36:28+03:00 examples : fix gpt-neox (#2943) 2753415afdaf22a18c49608bd9d93cfffc05d435 bc054af97ac68a4b726e972cb283eb9565253ed5 kchro3 62481661+kchro3@users.noreply.github.com 2023-09-02T22:27:25-07:00 GitHub noreply@github.com 2023-09-03T08:27:25+03:00 swift : add missing c file to Package.swift (#2978) bc054af97ac68a4b726e972cb283eb9565253ed5 3358c381f6251bf6e65855e1c93bfaa9ec82ddb3 Cebtenzzre cebtenzzre@gmail.com 2023-09-03T01:26:59-04:00 GitHub noreply@github.com 2023-09-03T08:26:59+03:00 make : support overriding CFLAGS/CXXFLAGS/CPPFLAGS/LDFLAGS (#2886) 3358c381f6251bf6e65855e1c93bfaa9ec82ddb3 52315a421674ff64305dbf082f69e4ec77f0a3f3 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-09-02T11:53:55-06:00 GitHub noreply@github.com 2023-09-02T11:53:55-06:00 logging: Fix creating empty file even when disabled (#2966) 52315a421674ff64305dbf082f69e4ec77f0a3f3 8b56b4f2c396eae1f4417e5a859557fed989e0ee bandoti 141645996+bandoti@users.noreply.github.com 2023-09-02T09:53:18-03:00 GitHub noreply@github.com 2023-09-02T15:53:18+03:00 readme : update clblast instructions (#2903) 8b56b4f2c396eae1f4417e5a859557fed989e0ee 21f3d1be867b4d7be07c26f5da6e4bc69bcf4d27 Karsten Weiss knweiss@gmail.com 2023-09-02T14:29:09+02:00 GitHub noreply@github.com 2023-09-02T15:29:09+03:00 metal : show all Metal device instances in the system (#2952) 21f3d1be867b4d7be07c26f5da6e4bc69bcf4d27 571083f508266c4eb5cb5457d836df5dd3c173ce Jhen-Jie Hong iainst0409@gmail.com 2023-09-02T20:23:45+08:00 GitHub noreply@github.com 2023-09-02T15:23:45+03:00 k-quants : fix build on armv7 (android only) (#2920) 571083f508266c4eb5cb5457d836df5dd3c173ce f04d0028444bc9b3d4225fba47e19d4c3aeb3741 Jhen-Jie Hong iainst0409@gmail.com 2023-09-02T08:31:46+08:00 GitHub noreply@github.com 2023-09-02T08:31:46+08:00 server : avoid aniprompt in probabilities of final response (#2849) f04d0028444bc9b3d4225fba47e19d4c3aeb3741 69fdbb9abc8907dd2a9ffdd840cba92d678a660a Engininja2 139037756+Engininja2@users.noreply.github.com 2023-09-01T15:33:19-06:00 GitHub noreply@github.com 2023-09-01T23:33:19+02:00 cuda : vsubss4 for older versions of ROCm/clang (#2942) 69fdbb9abc8907dd2a9ffdd840cba92d678a660a 5d6f19f16b2173afe2d5c6aee2f5c9fc31038eba ZHAOKAI WANG sanxianwei@163.com 2023-09-01T22:06:44+08:00 GitHub noreply@github.com 2023-09-01T17:06:44+03:00 readme : quick start command fix (#2908) 5d6f19f16b2173afe2d5c6aee2f5c9fc31038eba 0d5893668625456c94bbadfddc53fc69cd51c223 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-09-01T08:02:48-06:00 GitHub noreply@github.com 2023-09-01T08:02:48-06:00 Allow quantize to only copy tensors, some other improvements (#2931) 0d5893668625456c94bbadfddc53fc69cd51c223 6c9c23429bf4e4fcaaddbebadc4638558430a7f2 Georgi Gerganov ggerganov@gmail.com 2023-09-01T17:00:40+03:00 Georgi Gerganov ggerganov@gmail.com 2023-09-01T17:01:11+03:00 llama2c : rename function 6c9c23429bf4e4fcaaddbebadc4638558430a7f2 ee8654bcd0146708988a703e54406d5b553712ea Cebtenzzre cebtenzzre@gmail.com 2023-09-01T09:53:14-04:00 GitHub noreply@github.com 2023-09-01T16:53:14+03:00 make : use unaligned vector moves on MinGW (#2945) ee8654bcd0146708988a703e54406d5b553712ea 49bb9cbe0f598bc43be539b0df8eafb2130cfad3 m3ndax adrian.goessl@outlook.com 2023-09-01T15:47:27+02:00 GitHub noreply@github.com 2023-09-01T16:47:27+03:00 minor : add const qualifiers (#2853) 49bb9cbe0f598bc43be539b0df8eafb2130cfad3 ef156499721c67748cde01a5436cb6f0648bb4b4 Konstantin Herud konstantin.herud@denkbares.com 2023-09-01T15:36:14+02:00 GitHub noreply@github.com 2023-09-01T16:36:14+03:00 docs : add java-llama.cpp to README.md (#2935) ef156499721c67748cde01a5436cb6f0648bb4b4 d8d6977f48f1fa402ade38ad32c5b5fb1358d059 Cebtenzzre cebtenzzre@gmail.com 2023-09-01T09:34:50-04:00 GitHub noreply@github.com 2023-09-01T16:34:50+03:00 build : fix most gcc and clang warnings (#2861) d8d6977f48f1fa402ade38ad32c5b5fb1358d059 5aec2cfaac386eb09aebb75b805860828f00de91 Ben Siraphob bensiraphob@gmail.com 2023-09-01T09:32:14-04:00 GitHub noreply@github.com 2023-09-01T16:32:14+03:00 examples : add C grammar (#2357) 5aec2cfaac386eb09aebb75b805860828f00de91 13268c533177a4dc76bce0b465645d74f0d51d55 Tameem 113388789+AhmadTameem@users.noreply.github.com 2023-09-01T18:27:40+05:00 GitHub noreply@github.com 2023-09-01T16:27:40+03:00 ggml : add RISC-V vector intrinsics support (#2929) 13268c533177a4dc76bce0b465645d74f0d51d55 4dcd47d71df8ca4edcc31302744bd93f0c31298e Georgi Gerganov ggerganov@gmail.com 2023-09-01T13:42:41+03:00 GitHub noreply@github.com 2023-09-01T13:42:41+03:00 metal : slight speed-up for add and mul kernels (#2917) 4dcd47d71df8ca4edcc31302744bd93f0c31298e 18705a30ef3d6a89e1d7c6cb8cfe8633f760cb53 staviq staviq@gmail.com 2023-09-01T11:07:06+02:00 GitHub noreply@github.com 2023-09-01T12:07:06+03:00 logs : fix mingw-like builds (fixes #2898) (#2911) 18705a30ef3d6a89e1d7c6cb8cfe8633f760cb53 e8d91589258f9204397a7ac5f9b3c857835c98f8 Cebtenzzre cebtenzzre@gmail.com 2023-09-01T05:03:49-04:00 GitHub noreply@github.com 2023-09-01T12:03:49+03:00 llama2c : fix segfault and alloc-dealloc-mismatch (#2913) e8d91589258f9204397a7ac5f9b3c857835c98f8 bce1fef328941499dc0acb76cc7fd7ac90449c2f Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-09-01T11:15:57+03:00 GitHub noreply@github.com 2023-09-01T11:15:57+03:00 metal: somewhat faster f16 x f32 matrix multiply kernel (#2951) bce1fef328941499dc0acb76cc7fd7ac90449c2f 528134dd0267838d9c0250cf1d9621631dff09b2 Cebtenzzre cebtenzzre@gmail.com 2023-08-31T22:13:51-04:00 GitHub noreply@github.com 2023-08-31T22:13:51-04:00 convert : fix another python 3.8 issue (#2949) 528134dd0267838d9c0250cf1d9621631dff09b2 aeefac4ff760acea5afe66fbfe8d7eca1937b79c slaren slarengh@gmail.com 2023-09-01T01:32:09+02:00 GitHub noreply@github.com 2023-09-01T01:32:09+02:00 remove convert-llama-7b-pth-to-gguf.py and convert-llama-hf-to-gguf.py (#2906) aeefac4ff760acea5afe66fbfe8d7eca1937b79c e8422de39e4aa2f7e50574124b060a80607e654a Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-08-31T16:49:24-06:00 GitHub noreply@github.com 2023-08-31T16:49:24-06:00 scripts: Use local gguf package when running from repo (#2927) e8422de39e4aa2f7e50574124b060a80607e654a 92d0b751a77a089e650983e9f1564ef4d31b32b9 DannyDaemonic DannyDaemonic@gmail.com 2023-08-31T04:21:45-07:00 GitHub noreply@github.com 2023-08-31T04:21:45-07:00 @vxiiduu's fix for PrefetchVirtualMemory (#2930) 92d0b751a77a089e650983e9f1564ef4d31b32b9 8afe2280009ecbfc9de2c93b8f41283dc810609a Cebtenzzre cebtenzzre@gmail.com 2023-08-31T01:02:23-04:00 GitHub noreply@github.com 2023-08-31T08:02:23+03:00 convert : fix python 3.8 support, modernize type annotations (#2916) 8afe2280009ecbfc9de2c93b8f41283dc810609a 71d6975559acfd6c8407a4ef8275a9979c737765 Johannes Gäßler johannesg@5d6.de 2023-08-30T21:46:19+02:00 GitHub noreply@github.com 2023-08-30T21:46:19+02:00 CUDA: mul_mat_q=true llama_context_params default (#2912) 71d6975559acfd6c8407a4ef8275a9979c737765 b532a69b2fd08067f34f32f37a2fd9b37678a34a Henri Vasserman henv@hot.ee 2023-08-30T19:14:53+03:00 GitHub noreply@github.com 2023-08-30T19:14:53+03:00 [Docker] fix tools.sh argument passing. (#2884) b532a69b2fd08067f34f32f37a2fd9b37678a34a c90d135eb433cf0d40fb95e46a48d1391d2352b5 Georgi Gerganov ggerganov@gmail.com 2023-08-30T13:29:40+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-30T13:29:40+03:00 convert.py : use dir name to name the llama c90d135eb433cf0d40fb95e46a48d1391d2352b5 0d1c706181cd31e7f368dd14eeb16c1a2569e4df Georgi Gerganov ggerganov@gmail.com 2023-08-30T12:52:46+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-30T12:53:24+03:00 examples : fix underscore in beam-search + .gitignore (close #2900) 0d1c706181cd31e7f368dd14eeb16c1a2569e4df 950929442070874d45561d2a4b68b010457767de M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-08-30T12:47:40+03:00 GitHub noreply@github.com 2023-08-30T12:47:40+03:00 gguf : add workflow for Pypi publishing (#2896) 950929442070874d45561d2a4b68b010457767de 35092fb54712d032860f3976a6fc1ae1f84a4a28 alonfaraj alonfaraj@gmail.com 2023-08-30T12:42:51+03:00 GitHub noreply@github.com 2023-08-30T12:42:51+03:00 make : add test and update CI (#2897) 35092fb54712d032860f3976a6fc1ae1f84a4a28 dc07dc492ef9640bbb82904d7c7679f7bdcf6d76 Gilad S giladgd@users.noreply.github.com 2023-08-30T11:40:12+03:00 GitHub noreply@github.com 2023-08-30T11:40:12+03:00 docs : add `node-llama-cpp` to `README.md` (#2885) dc07dc492ef9640bbb82904d7c7679f7bdcf6d76 ad9ddcff6ef322db5cf13785bd7c856b610d242e Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-08-30T02:25:50-06:00 GitHub noreply@github.com 2023-08-30T11:25:50+03:00 convert : various script cleanups/fixes + merges and special token handling (#2842) ad9ddcff6ef322db5cf13785bd7c856b610d242e 8341a25957b319a03d4a811176cd5ad7f2b0fbd4 chaihahaha chai836275709@gmail.com 2023-08-30T14:50:55+08:00 GitHub noreply@github.com 2023-08-30T09:50:55+03:00 llm.vim : stop generation at multiple linebreaks, bind to (#2879) 8341a25957b319a03d4a811176cd5ad7f2b0fbd4 849408957c687cde4ab32c147107f643fc55130b staviq staviq@gmail.com 2023-08-30T08:29:32+02:00 GitHub noreply@github.com 2023-08-30T09:29:32+03:00 main : log file (#2748) 849408957c687cde4ab32c147107f643fc55130b 06abf8eebabe086ca4003dee2754ab45032cd3fd Cebtenzzre cebtenzzre@gmail.com 2023-08-30T02:20:26-04:00 GitHub noreply@github.com 2023-08-30T09:20:26+03:00 tests : add a C compliance test (#2848) 06abf8eebabe086ca4003dee2754ab45032cd3fd c03a243abf9f30889f31fefdfa94fe9d7034820c slaren slarengh@gmail.com 2023-08-29T23:24:42+02:00 GitHub noreply@github.com 2023-08-29T23:24:42+02:00 ggml : add view_src and view_offs to ggml_tensor for views (#2874) c03a243abf9f30889f31fefdfa94fe9d7034820c fa3582f509a2715e80a473e79f88dcd1ebff44c2 slaren slarengh@gmail.com 2023-08-29T23:17:34+02:00 GitHub noreply@github.com 2023-08-29T23:17:34+02:00 remove outdated references to -eps and -gqa from README (#2881) fa3582f509a2715e80a473e79f88dcd1ebff44c2 e37e69dcc3d52f21222a63cafed2a71b3f6b53c6 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-29T23:55:45+03:00 GitHub noreply@github.com 2023-08-29T23:55:45+03:00 Tell users attmepting to run perplexity with too few tokens to use more (#2882) e37e69dcc3d52f21222a63cafed2a71b3f6b53c6 53885d7256909ec3e2176cdc2477f3986c15ec69 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-29T23:55:03+03:00 GitHub noreply@github.com 2023-08-29T23:55:03+03:00 10X faster BPE tokenizer (#2876) 53885d7256909ec3e2176cdc2477f3986c15ec69 bcce96ba4dd95482824700c4ce2455fe8c49055a maddes8cht 55592906+maddes8cht@users.noreply.github.com 2023-08-29T15:51:02+02:00 GitHub noreply@github.com 2023-08-29T16:51:02+03:00 py : fix "usage" messages (#2873) bcce96ba4dd95482824700c4ce2455fe8c49055a 74e0caeb82fc9db77fa2cc93070bb919a9a935dd jameswu2014 545426914@qq.com 2023-08-29T17:48:41+08:00 GitHub noreply@github.com 2023-08-29T12:48:41+03:00 convert.py : fix baichuan7B support (#2870) 74e0caeb82fc9db77fa2cc93070bb919a9a935dd d4b5e16c32ba9c5fa6bbd035e80a99c113050cde Jhen-Jie Hong iainst0409@gmail.com 2023-08-29T17:30:10+08:00 GitHub noreply@github.com 2023-08-29T12:30:10+03:00 readme : add react-native binding (#2869) d4b5e16c32ba9c5fa6bbd035e80a99c113050cde 3a007648f230ea37d6cca5e63850f04ebb12d2cf Cebtenzzre cebtenzzre@gmail.com 2023-08-29T04:42:41-04:00 GitHub noreply@github.com 2023-08-29T11:42:41+03:00 make : fix clang tests build, add missing examples (#2859) 3a007648f230ea37d6cca5e63850f04ebb12d2cf 611363ac791435497e66278dfe31ac8a4e11fa4f Georgi Gerganov ggerganov@gmail.com 2023-08-29T11:33:46+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-29T11:33:46+03:00 metal : add option to disable debug logs (close #2764) 611363ac791435497e66278dfe31ac8a4e11fa4f 95b6e5212f5e4e1419de1d833d7f8d788f9f2227 Georgi Gerganov ggerganov@gmail.com 2023-08-29T10:50:30+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-29T10:50:30+03:00 scripts : add pipefail 95b6e5212f5e4e1419de1d833d7f8d788f9f2227 44c117f41ee01c5ac8fb86bba041f08d8b87b46d Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2023-08-28T23:33:27-07:00 GitHub noreply@github.com 2023-08-29T09:33:27+03:00 added `struct` to llama_dump_timing_info_yaml's `llama_context` (#2857) 44c117f41ee01c5ac8fb86bba041f08d8b87b46d 43033b7bb4858da4f591715b3babdf906c9b7cbc xaedes xaedes@gmail.com 2023-08-28T21:51:47+02:00 GitHub noreply@github.com 2023-08-28T22:51:47+03:00 train : mem usage and other improvements (#2439) 43033b7bb4858da4f591715b3babdf906c9b7cbc 6b73ef120114beb5664ea94aab48d07ed248ee52 slaren slarengh@gmail.com 2023-08-28T19:19:18+02:00 GitHub noreply@github.com 2023-08-28T19:19:18+02:00 llama-bench : set locale to utf8 (#2832) 6b73ef120114beb5664ea94aab48d07ed248ee52 75fafcbcccc280a5b3883bc76d0a2dabf474d094 Johannes Gäßler johannesg@5d6.de 2023-08-28T17:59:39+02:00 GitHub noreply@github.com 2023-08-28T17:59:39+02:00 YAML result logging + preset script (#2657) 75fafcbcccc280a5b3883bc76d0a2dabf474d094 be475f60af1a54e8de81466ccc907d080cf6df1a alonfaraj alonfaraj@gmail.com 2023-08-28T18:38:35+03:00 GitHub noreply@github.com 2023-08-28T18:38:35+03:00 make : fix tests build (#2855) be475f60af1a54e8de81466ccc907d080cf6df1a 3af6b86301ddfb11bb68e91dfc030b611b0d8426 grahameth 96447521+grahameth@users.noreply.github.com 2023-08-28T17:38:12+02:00 GitHub noreply@github.com 2023-08-28T18:38:12+03:00 llama.cpp : fix wrong vsnprintf call in MS compiler (#2856) 3af6b86301ddfb11bb68e91dfc030b611b0d8426 35feac6560387cf0484371af3d9b12bff678e0b9 Ronny Brendel ronnybrendel@gmail.com 2023-08-28T14:51:08+02:00 GitHub noreply@github.com 2023-08-28T15:51:08+03:00 ggml : tiny ggml_vec_dot_q4_K_q8_K AVX2 improvement (#2819) 35feac6560387cf0484371af3d9b12bff678e0b9 92b1bbd2ec43c82ec0530ba3c8758846c5790c75 Georgi Gerganov ggerganov@gmail.com 2023-08-28T14:24:53+03:00 GitHub noreply@github.com 2023-08-28T14:24:53+03:00 ggml : sync (mem align to header + conv_transpose_2d fixes + ggml_alloc) (#2852) 92b1bbd2ec43c82ec0530ba3c8758846c5790c75 dd0dc366dab10e8df28d3924e7f313b5c695e908 Johannes Gäßler johannesg@5d6.de 2023-08-28T13:23:55+02:00 GitHub noreply@github.com 2023-08-28T14:23:55+03:00 CUDA: fix RoPE asserts, block sizes (#2833) dd0dc366dab10e8df28d3924e7f313b5c695e908 f55538c3ccba9b926846ef862fa830cea08c433e igarnier igarnier@protonmail.com 2023-08-28T10:19:59+02:00 GitHub noreply@github.com 2023-08-28T11:19:59+03:00 llama.h : add missing struct keyword for C compat in callback type (#2847) f55538c3ccba9b926846ef862fa830cea08c433e ebcee207b6058b7f695bb5c203ad87b1066a9790 Georgi Gerganov ggerganov@gmail.com 2023-08-28T10:59:08+03:00 GitHub noreply@github.com 2023-08-28T10:59:08+03:00 metal : fix memory leak (#2762) ebcee207b6058b7f695bb5c203ad87b1066a9790 3e8ff47af620a31e0810c58a41e4b089145982ef Cebtenzzre cebtenzzre@gmail.com 2023-08-28T02:32:25-04:00 GitHub noreply@github.com 2023-08-28T09:32:25+03:00 quantize : make output filename optional again (#2823) 3e8ff47af620a31e0810c58a41e4b089145982ef 103cfafc774f6feb3172b5d4d39681c965b17eba JohnnyB jboero@users.noreply.github.com 2023-08-28T07:31:24+01:00 GitHub noreply@github.com 2023-08-28T09:31:24+03:00 devops : added systemd units and set versioning to use date. (#2835) 103cfafc774f6feb3172b5d4d39681c965b17eba c10704d01e21e3dbe4d6ca1026ebff85349dd239 Georgi Gerganov ggerganov@gmail.com 2023-08-27T21:50:22+03:00 GitHub noreply@github.com 2023-08-27T21:50:22+03:00 gguf : fix strings to not be null-terminated (#2839) c10704d01e21e3dbe4d6ca1026ebff85349dd239 230d46c723edf5999752e4cb67fd94edb19ef9c7 Georgi Gerganov ggerganov@gmail.com 2023-08-27T18:55:41+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-27T18:55:41+03:00 llama : fix MPI threads (close #2827) 230d46c723edf5999752e4cb67fd94edb19ef9c7 463173a6c0ff353055eb90665794884c888c790f Olivier Chafik ochafik@users.noreply.github.com 2023-08-27T15:13:31+01:00 GitHub noreply@github.com 2023-08-27T17:13:31+03:00 examples : update llama2.c converter to read vocab and write models in GGUF format (#2751) 463173a6c0ff353055eb90665794884c888c790f eaa13a48ff4136f01c1cdb79cacd61b67ec53095 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-27T16:50:33+03:00 GitHub noreply@github.com 2023-08-27T16:50:33+03:00 llama : speedup tokenization (#2831) eaa13a48ff4136f01c1cdb79cacd61b67ec53095 da7455d0467b5f5cc2e45d0dcffaf098df13db63 Georgi Gerganov ggerganov@gmail.com 2023-08-27T16:40:48+03:00 GitHub noreply@github.com 2023-08-27T16:40:48+03:00 falcon : fix CUDA inference by making K and Q contiguous (#2830) da7455d0467b5f5cc2e45d0dcffaf098df13db63 25423e9185b7c2a1881ed8f85cc752a12370be9d Georgi Gerganov ggerganov@gmail.com 2023-08-27T15:52:34+03:00 GitHub noreply@github.com 2023-08-27T15:52:34+03:00 readme : fix headings 25423e9185b7c2a1881ed8f85cc752a12370be9d a6d1189fdd4c1ab4ba23f9d777f8950901dcffb2 Georgi Gerganov ggerganov@gmail.com 2023-08-27T15:24:40+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-27T15:24:58+03:00 scripts : helper convert script a6d1189fdd4c1ab4ba23f9d777f8950901dcffb2 c48c5bb0b06385f6c708339188d2aaf2bc278477 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-27T15:19:59+03:00 GitHub noreply@github.com 2023-08-27T15:19:59+03:00 k_quants tuning for Falcon-7b (#2816) c48c5bb0b06385f6c708339188d2aaf2bc278477 d0cee0d36d5be95a0d9088b674dbb27354107221 Georgi Gerganov ggerganov@gmail.com 2023-08-27T14:44:35+03:00 GitHub noreply@github.com 2023-08-27T14:44:35+03:00 readme : update hot topics d0cee0d36d5be95a0d9088b674dbb27354107221 edd4c1481708fcd788b0e423268304fd26e2b125 Georgi Gerganov ggerganov@gmail.com 2023-08-27T14:19:54+03:00 GitHub noreply@github.com 2023-08-27T14:19:54+03:00 gguf : add 64-bit support (GGUF v2) (#2821) edd4c1481708fcd788b0e423268304fd26e2b125 1591e2e590762011b43b10a9b6e04f13f98f2aa5 Georgi Gerganov ggerganov@gmail.com 2023-08-27T14:19:19+03:00 GitHub noreply@github.com 2023-08-27T14:19:19+03:00 llama : more tokenizer fixes (#2810) 1591e2e590762011b43b10a9b6e04f13f98f2aa5 789c8c945a2814e1487e18e68823d9926e3b1454 Przemysław Pawełczyk przemoc@gmail.com 2023-08-27T10:10:25+02:00 GitHub noreply@github.com 2023-08-27T11:10:25+03:00 ggml : detect SSSE3 (#2825) 789c8c945a2814e1487e18e68823d9926e3b1454 c1ac54b77aaba10d029084d152be786102010eb2 slaren slarengh@gmail.com 2023-08-27T09:03:27+02:00 GitHub noreply@github.com 2023-08-27T10:03:27+03:00 ci : add LoRA test to CI (#2650) c1ac54b77aaba10d029084d152be786102010eb2 730d9c681e339b76407659344e5a2cd50af7d7d5 Bruce MacDonald brucewmacdonald@gmail.com 2023-08-26T16:11:45-07:00 GitHub noreply@github.com 2023-08-27T07:11:45+08:00 server : add `/detokenize` endpoint (#2802) 730d9c681e339b76407659344e5a2cd50af7d7d5 c7d92e6dfec3f54849f3a0ba373054d29f321ea2 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-08-26T14:13:36-06:00 GitHub noreply@github.com 2023-08-26T23:13:36+03:00 convert.py : advanced option (#2753) c7d92e6dfec3f54849f3a0ba373054d29f321ea2 61d1a2895eeca55e0c8b7018492f6ab9c90cff78 Tim Miller drasticactions@users.noreply.github.com 2023-08-27T03:27:07+09:00 GitHub noreply@github.com 2023-08-26T21:27:07+03:00 llama : use Unicode Escape Sequence to replace encoded characters (#2814) 61d1a2895eeca55e0c8b7018492f6ab9c90cff78 741ca7dd1cec0a0349494742b9083d6ef4cd73c5 Tungsten842 quantmint@protonmail.com 2023-08-26T20:19:44+02:00 GitHub noreply@github.com 2023-08-26T21:19:44+03:00 flake.nix : add rocm support and cleanup (#2808) 741ca7dd1cec0a0349494742b9083d6ef4cd73c5 72f895c923ba98b8f2af294440206f35915c0501 Cebtenzzre cebtenzzre@gmail.com 2023-08-26T14:17:51-04:00 GitHub noreply@github.com 2023-08-26T21:17:51+03:00 llama : move #includes out of _GNU_SOURCE conditional (#2817) 72f895c923ba98b8f2af294440206f35915c0501 50526f37eba0b28336700890242ff282b949cd83 Dr. Tom Murphy VII Ph.D 499244+tom7@users.noreply.github.com 2023-08-26T14:12:56-04:00 GitHub noreply@github.com 2023-08-26T21:12:56+03:00 main : fix bug (penalize_nl=false doesn't work) + suppress warning on mingw (#1528) 50526f37eba0b28336700890242ff282b949cd83 04f4b1eb10f3e25750ca3e530265ce2841730e6b Cebtenzzre cebtenzzre@gmail.com 2023-08-26T12:53:52-04:00 GitHub noreply@github.com 2023-08-26T19:53:52+03:00 llama : use std::abs in llama_sample_tail_free (#2800) 04f4b1eb10f3e25750ca3e530265ce2841730e6b 7592375403a0bd0456d5ec2cdf8350e591f04fb0 Georgi Gerganov ggerganov@gmail.com 2023-08-26T17:37:35+03:00 GitHub noreply@github.com 2023-08-26T17:37:35+03:00 k-quants : remove unnecessary tensor shape restrictions (#2811) 7592375403a0bd0456d5ec2cdf8350e591f04fb0 771551a793c9976ed9cdfe7b8c69536af32af9f9 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-26T17:27:49+03:00 GitHub noreply@github.com 2023-08-26T17:27:49+03:00 Better perplexity for 2- and 3-bit quantization for LLaMA-v2-70B (#2807) 771551a793c9976ed9cdfe7b8c69536af32af9f9 f305bad11e10ad09e396faed2e37f4f845f5d566 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-26T16:48:53+03:00 GitHub noreply@github.com 2023-08-26T16:48:53+03:00 Fix HellaSwag (#2805) f305bad11e10ad09e396faed2e37f4f845f5d566 a2ca4e9de9da45ed0bb1c34935d5ec80cebc22d5 Volodymyr Vitvitskyi 72226+signalpillar@users.noreply.github.com 2023-08-26T14:25:39+01:00 GitHub noreply@github.com 2023-08-26T16:25:39+03:00 flake : build llama.cpp on Intel with nix (#2795) a2ca4e9de9da45ed0bb1c34935d5ec80cebc22d5 2ba83c8685177faea3399db9564f9c52df75c366 Nigel Bosch pnigelb@gmail.com 2023-08-26T07:11:17-05:00 GitHub noreply@github.com 2023-08-26T14:11:17+02:00 Handle null rope scaling value (#2793) 2ba83c8685177faea3399db9564f9c52df75c366 bae5c5f679e043371bc2b4dffff8d4964d6cb953 klosax 131523366+klosax@users.noreply.github.com 2023-08-26T13:45:53+02:00 GitHub noreply@github.com 2023-08-26T13:45:53+02:00 Fix spm whitespaces (#2806) bae5c5f679e043371bc2b4dffff8d4964d6cb953 232caf3c1581a6cb023571780ff41dc2d66d1ca0 lon 114724657+longregen@users.noreply.github.com 2023-08-26T10:07:43+02:00 GitHub noreply@github.com 2023-08-26T16:07:43+08:00 examples : skip unnecessary external lib in server README.md how-to (#2804) 232caf3c1581a6cb023571780ff41dc2d66d1ca0 d046dcee081118c9071bbc63dacdb359a58c467a Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2023-08-25T09:17:15-07:00 GitHub noreply@github.com 2023-08-25T19:17:15+03:00 llama : fix struct decl (#2790) d046dcee081118c9071bbc63dacdb359a58c467a c82742ac9cd96fd34aa961978805c1d8a361d589 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-25T19:05:02+03:00 GitHub noreply@github.com 2023-08-25T19:05:02+03:00 Faster perplexity computation (#2786) c82742ac9cd96fd34aa961978805c1d8a361d589 28b2c996ca0ab90a5669946084f13443ec98e241 Matt Pulver matt.pulver@heavy.ai 2023-08-25T11:18:48-04:00 GitHub noreply@github.com 2023-08-25T18:18:48+03:00 llama : add llama_beam_search() (#2267) 28b2c996ca0ab90a5669946084f13443ec98e241 154725c5436808e5c519685d0279e850596dbe62 Nigel Bosch pnigelb@gmail.com 2023-08-25T09:41:52-05:00 GitHub noreply@github.com 2023-08-25T16:41:52+02:00 convert.py : Get rope scale from HuggingFace models (#2772) 154725c5436808e5c519685d0279e850596dbe62 12e2e33a977af73e75885eeee91c5575a77f4e5f slaren slarengh@gmail.com 2023-08-25T15:16:19+02:00 GitHub noreply@github.com 2023-08-25T15:16:19+02:00 llama-bench : add model sizes (#2771) 12e2e33a977af73e75885eeee91c5575a77f4e5f 29674ab4e847fcaba60cc6558f0d46d5f74ae279 slaren slarengh@gmail.com 2023-08-25T14:08:53+02:00 GitHub noreply@github.com 2023-08-25T14:08:53+02:00 convert.py : export rope freq_base when converting CodeLlama from an HF model (#2773) 29674ab4e847fcaba60cc6558f0d46d5f74ae279 5439a0ab57c16b556ffa91a0953df5e46b1e7fb4 Jhen-Jie Hong iainst0409@gmail.com 2023-08-25T18:32:45+08:00 GitHub noreply@github.com 2023-08-25T18:32:45+08:00 server : display token probabilities in the UI (#2489) 5439a0ab57c16b556ffa91a0953df5e46b1e7fb4 8194cd8772c58b8a43aa07a2fc468f5366d7e320 Georgi Gerganov ggerganov@gmail.com 2023-08-25T13:03:25+03:00 GitHub noreply@github.com 2023-08-25T13:03:25+03:00 ci : pip install gguf in editable mode (#2782) 8194cd8772c58b8a43aa07a2fc468f5366d7e320 6bbc598a632560cb45dd2c51ad403bda8723b629 M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-08-25T12:43:41+03:00 GitHub noreply@github.com 2023-08-25T12:43:41+03:00 gguf : export objects to user code (#2780) 6bbc598a632560cb45dd2c51ad403bda8723b629 3f460a2b723c8b936ac29ecfd02f244b3adeba55 Henri Vasserman henv@hot.ee 2023-08-25T12:09:42+03:00 GitHub noreply@github.com 2023-08-25T12:09:42+03:00 ROCm Port (#1087) 3f460a2b723c8b936ac29ecfd02f244b3adeba55 87e3733f24a85d894cc16e1cbdfa1ea1e81a76f3 Georgi Gerganov ggerganov@gmail.com 2023-08-25T11:55:59+03:00 GitHub noreply@github.com 2023-08-25T11:55:59+03:00 cuda : add RoPE kernel for mode == 2 (NeoX) (#2760) 87e3733f24a85d894cc16e1cbdfa1ea1e81a76f3 b91ad7f46134d0d051dc516eb59a76f402de55c2 M. Yusuf Sarıgöz yusufsarigoz@gmail.com 2023-08-25T09:26:05+03:00 GitHub noreply@github.com 2023-08-25T09:26:05+03:00 gguf : make gguf pip-installable b91ad7f46134d0d051dc516eb59a76f402de55c2 2e5f70a25fc4576e9ed78603fe493eb7702c37a3 Shouzheng Liu lshzh.hi@gmail.com 2023-08-25T01:58:00-04:00 GitHub noreply@github.com 2023-08-25T08:58:00+03:00 ggml-alloc : enlarge size of parse_seq (#2776) 2e5f70a25fc4576e9ed78603fe493eb7702c37a3 d0f77b1353fc820d1ff1e6b87bc6bedde315938d Marcus Dunn 51931484+MarcusDunn@users.noreply.github.com 2023-08-24T14:49:30-07:00 GitHub noreply@github.com 2023-08-24T23:49:30+02:00 Added `enum` to `llama_token_get_type` return type (#2774) d0f77b1353fc820d1ff1e6b87bc6bedde315938d 0d3094f0c742ce61f84feb6e4f0b59beee6194d7 slaren slarengh@gmail.com 2023-08-24T21:10:39+02:00 GitHub noreply@github.com 2023-08-24T21:10:39+02:00 convert.py : try to determine n_ctx automatically for CodeLlama (#2770) 0d3094f0c742ce61f84feb6e4f0b59beee6194d7 01f2224682b08185af609b28b1268b95c8b4cfa2 slaren slarengh@gmail.com 2023-08-24T20:04:05+02:00 GitHub noreply@github.com 2023-08-24T21:04:05+03:00 gguf : add rope_freq_base parameter for CodeLlama (#2769) 01f2224682b08185af609b28b1268b95c8b4cfa2 38b16dfca6e5032e6cfb90c1653bf1ba4cf647b4 Georgi Gerganov ggerganov@gmail.com 2023-08-24T19:58:30+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-24T19:58:30+03:00 falcon : write file type 38b16dfca6e5032e6cfb90c1653bf1ba4cf647b4 8f8c28e89cb9531211783da697d6e7c445e2af1d Shouzheng Liu lshzh.hi@gmail.com 2023-08-24T12:27:25-04:00 GitHub noreply@github.com 2023-08-24T19:27:25+03:00 metal : bug-fix when enable ggml-alloc (#2757) 8f8c28e89cb9531211783da697d6e7c445e2af1d 7694adda8d1111b3cf758ad6c91d754a0a4cacff Georgi Gerganov ggerganov@gmail.com 2023-08-24T19:26:19+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-24T19:26:47+03:00 convert : auto-determine model name based on dir + scripts update 7694adda8d1111b3cf758ad6c91d754a0a4cacff fea95c682d0028fdd25853bea58035794a0c964d Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-08-24T10:11:13-06:00 GitHub noreply@github.com 2023-08-24T10:11:13-06:00 Fix for main example getting stuck when -n -2 and --interactive (#2767) fea95c682d0028fdd25853bea58035794a0c964d ef955fbd230c571cc1cda0d19baaeec347523175 slaren slarengh@gmail.com 2023-08-24T17:44:11+02:00 GitHub noreply@github.com 2023-08-24T17:44:11+02:00 fix convert.py for codellama, add llama 34B to the list of recognized models (#2768) ef955fbd230c571cc1cda0d19baaeec347523175 d67777c202c03bcb74372690599ef3c03affb3ba DannyDaemonic DannyDaemonic@gmail.com 2023-08-24T06:58:02-07:00 GitHub noreply@github.com 2023-08-24T15:58:02+02:00 Tag release with build number (#2732) d67777c202c03bcb74372690599ef3c03affb3ba c3e53b421a9910548be0345f85712c535f467a98 Georgi Gerganov ggerganov@gmail.com 2023-08-24T16:19:57+03:00 GitHub noreply@github.com 2023-08-24T16:19:57+03:00 metal : add Q8_0 support (#2763) c3e53b421a9910548be0345f85712c535f467a98 6e91a1b0706c2e0e52b9d9be7ee82d3c1e7a33c1 Georgi Gerganov ggerganov@gmail.com 2023-08-24T12:26:01+03:00 GitHub noreply@github.com 2023-08-24T12:26:01+03:00 llama : escape all U+2581 in a string (#2750) 6e91a1b0706c2e0e52b9d9be7ee82d3c1e7a33c1 44d5462b5cddc1c5cbcd7647646f7b55b175b01f Evan Jones evan.q.jones@gmail.com 2023-08-24T00:07:13-04:00 GitHub noreply@github.com 2023-08-24T07:07:13+03:00 llama : fix grammar sometimes generating null char (#2756) 44d5462b5cddc1c5cbcd7647646f7b55b175b01f c7868b075377c8c3fa916ea7c1aca600f44bed55 Georgi Gerganov ggerganov@gmail.com 2023-08-23T23:44:19+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-23T23:44:19+03:00 readme : fix link c7868b075377c8c3fa916ea7c1aca600f44bed55 79da24b58c1ea72340e64f799a4717d372207676 Georgi Gerganov ggerganov@gmail.com 2023-08-23T23:43:00+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-23T23:43:00+03:00 minor : fix trailing whitespace 79da24b58c1ea72340e64f799a4717d372207676 cf658adc832badaaa2ca119fe86070e5a830f8f6 Georgi Gerganov ggerganov@gmail.com 2023-08-23T23:41:16+03:00 GitHub noreply@github.com 2023-08-23T23:41:16+03:00 readme : update hot topics cf658adc832badaaa2ca119fe86070e5a830f8f6 a192860cfec89a38d59a943623bf595b1fe4495b Georgi Gerganov ggerganov@gmail.com 2023-08-23T23:08:04+03:00 GitHub noreply@github.com 2023-08-23T23:08:04+03:00 llm : add Falcon support (#2717) a192860cfec89a38d59a943623bf595b1fe4495b 95385241a91a616788a3bb76d12c9b7b2379ca2d Georgi Gerganov ggerganov@gmail.com 2023-08-23T22:37:39+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-23T22:37:39+03:00 minor : fix trailing whitespace 95385241a91a616788a3bb76d12c9b7b2379ca2d 335acd2ffd7b04501c6d8773ab9fcee6e7bf8639 Olivier Chafik ochafik@users.noreply.github.com 2023-08-23T20:33:05+01:00 GitHub noreply@github.com 2023-08-23T22:33:05+03:00 examples : restore the functionality to import llama2.c models (#2685) 335acd2ffd7b04501c6d8773ab9fcee6e7bf8639 5290c38e6e9b66ee2b543e560e301c1a1a90929c slaren slarengh@gmail.com 2023-08-23T16:46:54+02:00 GitHub noreply@github.com 2023-08-23T16:46:54+02:00 fix convert-lora-to-ggml.py (#2738) 5290c38e6e9b66ee2b543e560e301c1a1a90929c cc34dbda9681418a2b18382446b90cdcec398d82 klosax 131523366+klosax@users.noreply.github.com 2023-08-23T16:46:03+02:00 GitHub noreply@github.com 2023-08-23T16:46:03+02:00 main : insert bos if no tokens (#2727) cc34dbda9681418a2b18382446b90cdcec398d82 7c2227a1972a4add4b5c118e4914c086513d0382 akawrykow 142945436+akawrykow@users.noreply.github.com 2023-08-23T07:31:34-07:00 GitHub noreply@github.com 2023-08-23T17:31:34+03:00 gitignore : fix for windows (#2729) 7c2227a1972a4add4b5c118e4914c086513d0382 f19dca04ea5fbf9a0b2753091d93464585d5c73b Cebtenzzre cebtenzzre@gmail.com 2023-08-23T10:29:09-04:00 GitHub noreply@github.com 2023-08-23T17:29:09+03:00 chmod : make scripts executable (#2675) f19dca04ea5fbf9a0b2753091d93464585d5c73b 8207214b6a37a46526cee9e72d4c9092b9d1872f JohnnyB jboero@users.noreply.github.com 2023-08-23T15:28:22+01:00 GitHub noreply@github.com 2023-08-23T17:28:22+03:00 devops : RPM Specs (#2723) 8207214b6a37a46526cee9e72d4c9092b9d1872f 62959e740e8759d246ac8d09036950efde09981c Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-23T12:57:12+03:00 GitHub noreply@github.com 2023-08-23T12:57:12+03:00 Fix values shown in the quantize tool help (#2735) 62959e740e8759d246ac8d09036950efde09981c 7f7ddd5002040804e33fcdbde44aa22f8635f57d Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-23T12:56:42+03:00 GitHub noreply@github.com 2023-08-23T12:56:42+03:00 Strided perplexity (#2714) 7f7ddd5002040804e33fcdbde44aa22f8635f57d b8ad1b66b23f9b2e6e4531e9a62753323036a556 IgnacioFDM ignaciofdm@gmail.com 2023-08-23T06:31:09-03:00 GitHub noreply@github.com 2023-08-23T03:31:09-06:00 Fix ggml to gguf conversion on Windows (#2733) b8ad1b66b23f9b2e6e4531e9a62753323036a556 f5fe98d11bdf9e7797bcfb05c0c3601ffc4b9d26 Xiao-Yong Jin jinxiaoyong@gmail.com 2023-08-23T02:12:12-05:00 GitHub noreply@github.com 2023-08-23T15:12:12+08:00 server : allow json array in prompt or content for direct token input (#2306) f5fe98d11bdf9e7797bcfb05c0c3601ffc4b9d26 777f42ba18b29f25c71ff8de3ecf97b8017304c0 Evan Jones evan.q.jones@gmail.com 2023-08-22T21:01:57-04:00 GitHub noreply@github.com 2023-08-22T21:01:57-04:00 docs : add grammar docs (#2701) 777f42ba18b29f25c71ff8de3ecf97b8017304c0 46ef5b5fcf4c366e1fb27726b6394adbbf8fd0ea Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-08-22T17:39:39-06:00 GitHub noreply@github.com 2023-08-22T17:39:39-06:00 Improve handling of special tokens in GGML to GGUF converter (#2725) 46ef5b5fcf4c366e1fb27726b6394adbbf8fd0ea c63bb1d16a70c03440671b76954bb767513cead8 goerch jhr.walter@t-online.de 2023-08-22T23:10:42+02:00 GitHub noreply@github.com 2023-08-23T00:10:42+03:00 llama : fix whitespace escaping in tokenizer (#2724) c63bb1d16a70c03440671b76954bb767513cead8 3b6cfe7c927df178ca3c11643c3ec93e143471c9 Johannes Gäßler johannesg@5d6.de 2023-08-22T22:47:05+02:00 GitHub noreply@github.com 2023-08-22T22:47:05+02:00 CUDA: use mul_mat_q kernels by default (#2683) 3b6cfe7c927df178ca3c11643c3ec93e143471c9 800c9635b4a9390126f397870f3a825fc7455bd1 Alex Petenchea alex.petenchea@gmail.com 2023-08-22T21:58:16+03:00 GitHub noreply@github.com 2023-08-22T21:58:16+03:00 convert.py : clarifying error message (#2718) 800c9635b4a9390126f397870f3a825fc7455bd1 deb7dfca4b9725cd295d1426db75fe8e0a6d5312 Jiahao Li liplus17@163.com 2023-08-23T02:27:06+08:00 GitHub noreply@github.com 2023-08-22T20:27:06+02:00 Fix CUDA softmax by subtracting max value before exp (#2665) deb7dfca4b9725cd295d1426db75fe8e0a6d5312 bac66994cf356cf488078c056831396eb4ce31d5 Georgi Gerganov ggerganov@gmail.com 2023-08-22T20:05:59+03:00 GitHub noreply@github.com 2023-08-22T20:05:59+03:00 gguf : add ftype meta info to the model (#2710) bac66994cf356cf488078c056831396eb4ce31d5 519c981f8b65ee6c87c2965539685ced0a17223b Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-22T19:14:09+03:00 GitHub noreply@github.com 2023-08-22T19:14:09+03:00 Quantization imrovements for k_quants (#2707) 519c981f8b65ee6c87c2965539685ced0a17223b 1123f7fbdfb8012e46f05e903e6f675922916378 slaren slarengh@gmail.com 2023-08-22T16:03:12+02:00 GitHub noreply@github.com 2023-08-22T16:03:12+02:00 embedding : evaluate prompt in batches (#2713) 1123f7fbdfb8012e46f05e903e6f675922916378 ef3f333d3775600d1646a9fa249aca532d15fb89 slaren slarengh@gmail.com 2023-08-22T15:25:19+02:00 GitHub noreply@github.com 2023-08-22T15:25:19+02:00 ggml-cuda : use graph allocator (#2684) ef3f333d3775600d1646a9fa249aca532d15fb89 8e4364f2af9cd5d57240f23e83c0e29bc068bc02 Georgi Gerganov ggerganov@gmail.com 2023-08-22T14:22:08+03:00 GitHub noreply@github.com 2023-08-22T14:22:08+03:00 ggml : sync latest (SAM + SD operators, CUDA alibi) (#2709) 8e4364f2af9cd5d57240f23e83c0e29bc068bc02 1e3bc523d8053a77df3ac7126a84d0297ee97ef6 slaren slarengh@gmail.com 2023-08-22T09:56:03+02:00 GitHub noreply@github.com 2023-08-22T10:56:03+03:00 llama-bench : minor fixes (#2695) 1e3bc523d8053a77df3ac7126a84d0297ee97ef6 14b1d7e6f720dee41ce5a826376df738096d9033 Kylin 56434533+KyL0N@users.noreply.github.com 2023-08-22T15:14:23+08:00 GitHub noreply@github.com 2023-08-22T10:14:23+03:00 ggml : support CUDA's half type for aarch64(#1455) (#2670) 14b1d7e6f720dee41ce5a826376df738096d9033 226255b44ef2c2794bfac48d101d35a9c2dbb965 Shouzheng Liu lshzh.hi@gmail.com 2023-08-22T02:18:40-04:00 GitHub noreply@github.com 2023-08-22T09:18:40+03:00 metal : add missing barriers for mul-mat (#2699) 226255b44ef2c2794bfac48d101d35a9c2dbb965 930523c8e1cbbee5449c055daa894917fac6805e Jhen-Jie Hong iainst0409@gmail.com 2023-08-22T08:32:00+08:00 GitHub noreply@github.com 2023-08-22T08:32:00+08:00 server : fallback to default if client param is null (#2688) 930523c8e1cbbee5449c055daa894917fac6805e c8dba409e6d6a754090f08e6a862c5ffdd52e421 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-08-21T18:01:34-06:00 GitHub noreply@github.com 2023-08-21T18:01:34-06:00 Fix convert-llama-ggmlv3-to-gguf.py vocab conversion (#2698) c8dba409e6d6a754090f08e6a862c5ffdd52e421 6381d4e110bd0ec02843a60bbeb8b6fc37a9ace9 Georgi Gerganov ggerganov@gmail.com 2023-08-21T23:40:22+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-21T23:40:22+03:00 py : remove obsolete script 6381d4e110bd0ec02843a60bbeb8b6fc37a9ace9 dadbed99e65252d79f81101a392d0d6497b86caa Georgi Gerganov ggerganov@gmail.com 2023-08-21T23:07:43+03:00 GitHub noreply@github.com 2023-08-21T23:07:43+03:00 gguf : new file format with flexible meta data (beta) (#2398) dadbed99e65252d79f81101a392d0d6497b86caa cb1c0727bd59803b439b6a3af121c99e6393ff3d Shouzheng Liu lshzh.hi@gmail.com 2023-08-21T06:59:29-04:00 GitHub noreply@github.com 2023-08-21T13:59:29+03:00 metal : fix synchronization in new matrix multiplication kernel (#2686) cb1c0727bd59803b439b6a3af121c99e6393ff3d 9e232f0234073358e7031c1b8d7aa45020469a3b Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-21T11:11:31+03:00 GitHub noreply@github.com 2023-08-21T11:11:31+03:00 HellaSwag: split token evaluation into batches if needed (#2681) 9e232f0234073358e7031c1b8d7aa45020469a3b 5e9ff54a675d163d9f42aad1b5b3e734f17b2701 slaren slarengh@gmail.com 2023-08-20T22:17:53+02:00 GitHub noreply@github.com 2023-08-20T22:17:53+02:00 ggml : move all type info to ggml_type_traits (#2663) 5e9ff54a675d163d9f42aad1b5b3e734f17b2701 1f0bccb27929e261744c979bc75114955da49e98 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-08-20T16:44:46+03:00 GitHub noreply@github.com 2023-08-20T16:44:46+03:00 More efficient Hellaswag implementation (#2677) 1f0bccb27929e261744c979bc75114955da49e98 f63564adfaa157ca387071d6b9a06cfaef0ef576 Georgi Gerganov ggerganov@gmail.com 2023-08-19T00:45:36+03:00 GitHub noreply@github.com 2023-08-19T05:45:36+08:00 server : better default prompt (#2646) f63564adfaa157ca387071d6b9a06cfaef0ef576 2d8b76a110d76ff6b5728ff0af8477531e4db60e Jhen-Jie Hong iainst0409@gmail.com 2023-08-19T05:41:32+08:00 GitHub noreply@github.com 2023-08-19T05:41:32+08:00 server : update xxd usage for older versions compatibility (#2649) 2d8b76a110d76ff6b5728ff0af8477531e4db60e 7af633aec339367e36c867ae709088d6a801aa75 Adrian smith.adriane@gmail.com 2023-08-18T12:39:22-07:00 GitHub noreply@github.com 2023-08-18T21:39:22+02:00 Add link to clojure bindings to Readme. (#2659) 7af633aec339367e36c867ae709088d6a801aa75 097e121e2f17ed3541cf02c55ff7e9febc091b19 Georgi Gerganov ggerganov@gmail.com 2023-08-18T17:48:31+03:00 GitHub noreply@github.com 2023-08-18T17:48:31+03:00 readme : incoming BREAKING CHANGE 097e121e2f17ed3541cf02c55ff7e9febc091b19 eaf98c2649d7da705de255712f0038ac7e47c610 slaren slarengh@gmail.com 2023-08-18T12:44:58+02:00 GitHub noreply@github.com 2023-08-18T12:44:58+02:00 llama : add benchmark example (#2626) eaf98c2649d7da705de255712f0038ac7e47c610 e9b12c332ec6e215fbac4b2ef165353acbcd8319 mdrokz mohammadmunshi@gmail.com 2023-08-18T15:47:58+05:30 GitHub noreply@github.com 2023-08-18T13:17:58+03:00 readme : add link to Rust bindings (#2656) e9b12c332ec6e215fbac4b2ef165353acbcd8319 604b8bdfa6320bbcb018eebcc1252dfede603c6b Georgi Gerganov ggerganov@gmail.com 2023-08-18T12:48:55+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-18T12:48:55+03:00 perplexity : more meaningful ETA number - 2 decimal points 604b8bdfa6320bbcb018eebcc1252dfede603c6b 10151bee2e38b5711335c4a38f6ca93b50223acf Evan Jones evan.q.jones@gmail.com 2023-08-17T19:54:44-04:00 GitHub noreply@github.com 2023-08-17T19:54:44-04:00 Fix unicode in grammars (fixes #2501) (#2553) 10151bee2e38b5711335c4a38f6ca93b50223acf 0992a7b8b18a89e29a205efb48ceb559c9a08203 staviq staviq@gmail.com 2023-08-17T23:34:01Z GitHub noreply@github.com 2023-08-18T07:34:01+08:00 server : support for saving templates in browser LocalStorage (#2486) 0992a7b8b18a89e29a205efb48ceb559c9a08203 6ddeefad9b634c5c79e6bcf046523493ff1fdf7d Johannes Gäßler johannesg@5d6.de 2023-08-17T23:57:59+02:00 GitHub noreply@github.com 2023-08-17T23:57:59+02:00 README: fix LLAMA_CUDA_MMV_Y documentation (#2647) 6ddeefad9b634c5c79e6bcf046523493ff1fdf7d 8dae7ce68437faf1fa96ec0e7687b8700956ef20 Henri Vasserman henv@hot.ee 2023-08-17T23:11:18+03:00 GitHub noreply@github.com 2023-08-17T23:11:18+03:00 [Zig] Fixing Zig build and improvements (#2554) 8dae7ce68437faf1fa96ec0e7687b8700956ef20 a73ccf1aa34de49f61bfeb7f8a679c3bfdb3abe3 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-08-17T07:29:44-06:00 GitHub noreply@github.com 2023-08-17T07:29:44-06:00 Add --cfg-negative-prompt-file option for examples (#2591) a73ccf1aa34de49f61bfeb7f8a679c3bfdb3abe3 7cf54e1f746941279d81d485796777c01f88049c Georgi Gerganov ggerganov@gmail.com 2023-08-17T10:47:09+03:00 GitHub noreply@github.com 2023-08-17T10:47:09+03:00 llama : replace (permute + reshape + view_1d) with (view_3d) (#2538) 7cf54e1f746941279d81d485796777c01f88049c a872a2b28eaefc8d464eaa535c94deeb501666f9 drbh david.richard.holtz@gmail.com 2023-08-17T03:41:01-04:00 GitHub noreply@github.com 2023-08-17T10:41:01+03:00 tests : adds simple llama grammar tests (#2618) a872a2b28eaefc8d464eaa535c94deeb501666f9 0919a0f73d95cfb93a1646a1d1741a0615fe2c5e Shouzheng Liu lshzh.hi@gmail.com 2023-08-17T03:35:53-04:00 GitHub noreply@github.com 2023-08-17T10:35:53+03:00 ggml-alloc : fix discrepency between measure&eval (#2639) 0919a0f73d95cfb93a1646a1d1741a0615fe2c5e ed53db86c3b0e0815331a96d7a379edb5e62472c Kolen Cheung ickc@users.noreply.github.com 2023-08-16T21:09:49+01:00 GitHub noreply@github.com 2023-08-16T23:09:49+03:00 cmake : install ggml-meta.metal if LLAMA_METAL (#2449) ed53db86c3b0e0815331a96d7a379edb5e62472c fc8ef549e50087762a0b4f901cd74b2defcc6ae3 Jhen-Jie Hong iainst0409@gmail.com 2023-08-17T04:09:03+08:00 GitHub noreply@github.com 2023-08-16T23:09:03+03:00 metal : print error of load pipeline state (#2564) fc8ef549e50087762a0b4f901cd74b2defcc6ae3 bf83bff6742c0f1795b4c18695a13a34ac7adf62 Shouzheng Liu lshzh.hi@gmail.com 2023-08-16T16:08:28-04:00 GitHub noreply@github.com 2023-08-16T23:08:28+03:00 metal : enable ggml-alloc (#2627) bf83bff6742c0f1795b4c18695a13a34ac7adf62 b5ffb2849d23afe73647f68eec7b68187af09be6 Shouzheng Liu lshzh.hi@gmail.com 2023-08-16T16:07:04-04:00 GitHub noreply@github.com 2023-08-16T23:07:04+03:00 metal : matrix-matrix multiplication kernel (#2615) b5ffb2849d23afe73647f68eec7b68187af09be6 3ebb00935f3f0522b75df49c2769ab1774b91380 Georgi Gerganov ggerganov@gmail.com 2023-08-15T10:04:58+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-15T10:05:25+03:00 scripts : add helper script to get wikitext 3ebb00935f3f0522b75df49c2769ab1774b91380 d783f7982e0e823a2626a9956359c0d36c1a7e21 Jhen-Jie Hong iainst0409@gmail.com 2023-08-15T06:14:14+08:00 GitHub noreply@github.com 2023-08-15T06:14:14+08:00 server : add missing /json-schema-to-grammar.mjs (#2616) d783f7982e0e823a2626a9956359c0d36c1a7e21 d75561df207d22790609ee0ad924302f66ac2599 Jhen-Jie Hong iainst0409@gmail.com 2023-08-14T21:37:39+08:00 GitHub noreply@github.com 2023-08-14T16:37:39+03:00 metal : return null instead of exit(1) (#2573) d75561df207d22790609ee0ad924302f66ac2599 348acf188c9fbe66396990f2dc83229df367969b Cheng Shao terrorjack@type.dance 2023-08-14T15:36:42+02:00 GitHub noreply@github.com 2023-08-14T16:36:42+03:00 server : add --numa support (#2524) 348acf188c9fbe66396990f2dc83229df367969b 1cd06fa25eb859b14b3427a1d815a48f25fc3c34 Kamil Tomšík info@tomsik.cz 2023-08-14T15:35:16+02:00 GitHub noreply@github.com 2023-08-14T16:35:16+03:00 llama : add missing enum keyword in function signatures (#2610) 1cd06fa25eb859b14b3427a1d815a48f25fc3c34 2feb8934eb75ca63f3c42724229cce1df9579c8e Johannes Gäßler johannesg@5d6.de 2023-08-14T10:41:22+02:00 GitHub noreply@github.com 2023-08-14T10:41:22+02:00 CUDA: launch_bounds, small q4_K, q5_K mmq refactor (#2596) 2feb8934eb75ca63f3c42724229cce1df9579c8e 5517d6e69214cdead000a76983b9fe175c3f8329 Jhen-Jie Hong iainst0409@gmail.com 2023-08-14T16:20:17+08:00 GitHub noreply@github.com 2023-08-14T16:20:17+08:00 server : fix default grammar by use empty string in the UI (#2604) 5517d6e69214cdead000a76983b9fe175c3f8329 f31b5397143009d682db90fd2a6cde83f1ef00eb Jhen-Jie Hong iainst0409@gmail.com 2023-08-14T15:16:54+08:00 GitHub noreply@github.com 2023-08-14T15:16:54+08:00 server : implement json-schema-to-grammar.mjs & add grammar param in the UI (#2588) f31b5397143009d682db90fd2a6cde83f1ef00eb ee77efea2a1e3f7d153976b0934522b6bbaa62e6 vxiiduu 73044267+vxiiduu@users.noreply.github.com 2023-08-14T13:59:16+10:00 GitHub noreply@github.com 2023-08-13T20:59:16-07:00 Enhance Windows 7 and below compatibility. (#2592) ee77efea2a1e3f7d153976b0934522b6bbaa62e6 f64d44a9b9581cd58f7ec40f4fa1c3ca5ca18e1e drbh david.richard.holtz@gmail.com 2023-08-13T10:00:48-04:00 GitHub noreply@github.com 2023-08-13T17:00:48+03:00 test : add simple grammar parsing tests (#2594) f64d44a9b9581cd58f7ec40f4fa1c3ca5ca18e1e b19edd54d51cef5e3616c18b1d0d8626895b2cba Johannes Gäßler johannesg@5d6.de 2023-08-13T00:24:45+02:00 GitHub noreply@github.com 2023-08-13T00:24:45+02:00 CUDA: Fixed OpenLLaMA 3b mmq, reduced compile time (#2590) b19edd54d51cef5e3616c18b1d0d8626895b2cba 53dc399472d5bd35ee739b865e843b1996bd3814 byte-6174 88070277+byte-6174@users.noreply.github.com 2023-08-11T19:17:25-04:00 GitHub noreply@github.com 2023-08-12T01:17:25+02:00 Adding support for llama2.c models (#2559) 53dc399472d5bd35ee739b865e843b1996bd3814 9ca4abed893685692f90413e4d43153af12342d9 Equim sayaka@ekyu.moe 2023-08-12T06:35:14+08:00 GitHub noreply@github.com 2023-08-12T00:35:14+02:00 server: fixed wrong variable name in timing json (#2579) 9ca4abed893685692f90413e4d43153af12342d9 e59fcb2bc129881f4a269fee748fb38bce0a64de DannyDaemonic DannyDaemonic@gmail.com 2023-08-10T13:11:36-07:00 GitHub noreply@github.com 2023-08-10T13:11:36-07:00 Handle `ENABLE_VIRTUAL_TERMINAL_PROCESSING` more gracefully on earlier versions of Windows. e59fcb2bc129881f4a269fee748fb38bce0a64de 1638757767072a4957f52b9e3594f0b67610631b Christian Demsar crasm@git.vczf.us 2023-08-10T10:28:27-04:00 GitHub noreply@github.com 2023-08-10T16:28:27+02:00 Add --n-predict -2 for stopping generation on full context (#2565) 1638757767072a4957f52b9e3594f0b67610631b 916a9acdd0a411426690400ebe2bb7ce840a6bba Martin Krasser krasserm@googlemail.com 2023-08-10T12:16:38+02:00 GitHub noreply@github.com 2023-08-10T13:16:38+03:00 Fix grammar-based sampling issue in server (#2566) 916a9acdd0a411426690400ebe2bb7ce840a6bba ea04a4ca1940d92becc0ee26523aa2c4a18cf938 Sam Spilsbury smspillaz@gmail.com 2023-08-09T23:47:42+03:00 GitHub noreply@github.com 2023-08-09T22:47:42+02:00 ggml-alloc: Don't try to re-use buffers of external tensors (#2562) ea04a4ca1940d92becc0ee26523aa2c4a18cf938 25d43e0eb578b6e73046d9d6644a3a14d460600d grahameth 96447521+grahameth@users.noreply.github.com 2023-08-09T22:46:40+02:00 GitHub noreply@github.com 2023-08-09T22:46:40+02:00 add log_callback to llama_context_params for custom logging. (#2234) 25d43e0eb578b6e73046d9d6644a3a14d460600d f5bfea0580e417f99850d5456ca541d871a3e48c Johannes Gäßler johannesg@5d6.de 2023-08-09T09:42:34+02:00 GitHub noreply@github.com 2023-08-09T09:42:34+02:00 CUDA: tuned mul_mat_q kernels (#2546) f5bfea0580e417f99850d5456ca541d871a3e48c acfc5478ff3446ca3b54553967a3dea09b7c771a Martin Krasser krasserm@googlemail.com 2023-08-08T15:29:19+02:00 GitHub noreply@github.com 2023-08-08T16:29:19+03:00 Allow passing grammar to completion endpoint (#2532) acfc5478ff3446ca3b54553967a3dea09b7c771a 7ed8d1fe7f8cbe6a6763e6b46759795ac8d21e12 Johannes Gäßler johannesg@5d6.de 2023-08-08T14:38:16+02:00 GitHub noreply@github.com 2023-08-08T14:38:16+02:00 CUDA: tighter VRAM scratch size for 65b/70b (#2551) 7ed8d1fe7f8cbe6a6763e6b46759795ac8d21e12 e7f94d6fdc83b41ba449b4b8c80821673dd12ffc chaihahaha chai836275709@gmail.com 2023-08-08T20:07:02+08:00 GitHub noreply@github.com 2023-08-08T15:07:02+03:00 llm.vim : multiline autocompletion, get rid of "^@" (#2543) e7f94d6fdc83b41ba449b4b8c80821673dd12ffc 2d7baaf50f3277e65cf71071f61ea34823d14c30 Georgi Gerganov ggerganov@gmail.com 2023-08-08T15:05:30+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-08T15:06:18+03:00 vim : bring back simple llm.vim example 2d7baaf50f3277e65cf71071f61ea34823d14c30 f3c3b4b1672d860800639c87d3b5d17564692469 AustinMroz austinmroz@utexas.edu 2023-08-08T06:44:48-05:00 GitHub noreply@github.com 2023-08-08T14:44:48+03:00 vim : streaming and more (#2495) f3c3b4b1672d860800639c87d3b5d17564692469 93356bdb7a324a8f6570f99d02af392cd4c45796 klosax 131523366+klosax@users.noreply.github.com 2023-08-07T19:07:19+02:00 GitHub noreply@github.com 2023-08-07T19:07:19+02:00 Add --rope-scale parameter (#2544) 93356bdb7a324a8f6570f99d02af392cd4c45796 60baff7c8584ec369e53469cad5f92e102b1efe4 Georgi Gerganov ggerganov@gmail.com 2023-08-07T14:25:58+03:00 GitHub noreply@github.com 2023-08-07T14:25:58+03:00 ggml : mul mat tweaks (#2372) 60baff7c8584ec369e53469cad5f92e102b1efe4 9082b5dfbfae01243a0b822dcd2812877e63bf1b Georgi Gerganov ggerganov@gmail.com 2023-08-07T14:24:42+03:00 Georgi Gerganov ggerganov@gmail.com 2023-08-07T14:24:42+03:00 ggml : pad result of ggml_nbytes() 9082b5dfbfae01243a0b822dcd2812877e63bf1b 99d29c0094476c4962023036ecd61a3309d0e16b Georgi Gerganov ggerganov@gmail.com 2023-08-07T13:55:18+03:00 GitHub noreply@github.com 2023-08-07T13:55:18+03:00 ggml : change params pointer (style change) (#2539) 99d29c0094476c4962023036ecd61a3309d0e16b 3d9a55181603e85a26378a850a14068034e5002d Georgi Gerganov ggerganov@gmail.com 2023-08-07T13:20:09+03:00 GitHub noreply@github.com 2023-08-07T13:20:09+03:00 ggml : sync (custom ops) (#2537) 3d9a55181603e85a26378a850a14068034e5002d f6f9896ac3d2ff207e18f87dab85d126ceef5236 Johannes Gäßler johannesg@5d6.de 2023-08-07T10:09:40+02:00 GitHub noreply@github.com 2023-08-07T10:09:40+02:00 Fixed mmap prefetch for GPU offloading (#2529) f6f9896ac3d2ff207e18f87dab85d126ceef5236 34a14b28ff7f3c98730339bacee035091b2a812a Georgi Gerganov ggerganov@gmail.com 2023-08-07T10:52:57+03:00 GitHub noreply@github.com 2023-08-07T10:52:57+03:00 metal : fix out-of-bounds access + inc concurrency nodes (#2416) 34a14b28ff7f3c98730339bacee035091b2a812a 7297128db8159c7b12db4c28a4532b993025c2e5 GiviMAD GiviMAD@users.noreply.github.com 2023-08-06T23:21:46-07:00 GitHub noreply@github.com 2023-08-07T09:21:46+03:00 [Makefile] Move ARM CFLAGS before compilation (#2536) 7297128db8159c7b12db4c28a4532b993025c2e5 86c32198954a2bc482025703d6875e11f1c2a574 Henri Vasserman henv@hot.ee 2023-08-07T08:35:53+03:00 GitHub noreply@github.com 2023-08-07T08:35:53+03:00 [Zig] Rewrite build for Zig 0.11 (#2514) 86c32198954a2bc482025703d6875e11f1c2a574 2e8265ae1764d6288aab0e2df641909072e2d58e DannyDaemonic DannyDaemonic@gmail.com 2023-08-05T23:49:34-07:00 GitHub noreply@github.com 2023-08-06T09:49:34+03:00 console : fix issue related to Windows 11 PowerShell console mode persistence (#2521) 2e8265ae1764d6288aab0e2df641909072e2d58e f514d1b306e1114c2884fcb25dd9bd48ae64ba32 Keiichi Tabata keiichi.tabata@outlook.com 2023-08-06T15:34:05+09:00 GitHub noreply@github.com 2023-08-06T09:34:05+03:00 convert.py : add missing abstract methods for quantized data (#2491) f514d1b306e1114c2884fcb25dd9bd48ae64ba32 332311234a0aa2974b2450710e22e09d90dd6b0b Johannes Gäßler johannesg@5d6.de 2023-08-05T18:20:44+02:00 GitHub noreply@github.com 2023-08-05T18:20:44+02:00 CUDA: faster k-quant mul_mat_q kernels (#2525) 332311234a0aa2974b2450710e22e09d90dd6b0b 182af739c4ce237f7579facfe8f94dc53a1f573f Jonas Wunderlich 32615971+jonas-w@users.noreply.github.com 2023-08-04T20:16:11Z GitHub noreply@github.com 2023-08-04T22:16:11+02:00 fix firefox autoscroll (#2519) 182af739c4ce237f7579facfe8f94dc53a1f573f 4329d1acb01c353803a54733b8eef9d93d0b84b2 Cebtenzzre cebtenzzre@gmail.com 2023-08-04T15:00:57-04:00 GitHub noreply@github.com 2023-08-04T21:00:57+02:00 server: regenerate completion.js.hpp (#2515) 4329d1acb01c353803a54733b8eef9d93d0b84b2 02f9d96a866268700b8d8e7acbbcb4392c5ff345 Cebtenzzre cebtenzzre@gmail.com 2023-08-04T11:35:22-04:00 GitHub noreply@github.com 2023-08-04T17:35:22+02:00 CUDA: use min compute capability of GPUs actually used (#2506) 02f9d96a866268700b8d8e7acbbcb4392c5ff345 3498588e0fb4daf040c4e3c698595cb0bfd345c0 Cebtenzzre cebtenzzre@gmail.com 2023-08-04T11:34:32-04:00 GitHub noreply@github.com 2023-08-04T17:34:32+02:00 CUDA: check if event is NULL before cudaStreamWaitEvent (#2505) 3498588e0fb4daf040c4e3c698595cb0bfd345c0 5f631c26794b6371fcf2660e8d0c53494a5575f7 DannyDaemonic DannyDaemonic@gmail.com 2023-08-04T08:20:12-07:00 GitHub noreply@github.com 2023-08-04T08:20:12-07:00 Add --simple-io option for subprocesses and break out console.h and cpp (#1558) 5f631c26794b6371fcf2660e8d0c53494a5575f7 415e99fec27be5a2e4283f1937afd17eb33fbd66 Stephen Nichols snichols@users.noreply.github.com 2023-08-04T06:37:24-05:00 GitHub noreply@github.com 2023-08-04T13:37:24+02:00 Fixing race condition in server and partial stream handling in frontend. (#2391) 415e99fec27be5a2e4283f1937afd17eb33fbd66 ff966e7ca6af127c9405523cdb07ef8fa01bf6d6 l3utterfly gc.pthzfoldr@gmail.com 2023-08-04T19:29:52+08:00 GitHub noreply@github.com 2023-08-04T13:29:52+02:00 Stream save llama context data to file instead of allocating entire buffer upfront (#2488) ff966e7ca6af127c9405523cdb07ef8fa01bf6d6 8183159cf3def112f6d1fe94815fce70e1bffa12 Borislav Stanimirov b.stanimirov@abv.bg 2023-08-04T13:07:21+03:00 GitHub noreply@github.com 2023-08-04T13:07:21+03:00 build : fix several cast and printf warnings (#2499) 8183159cf3def112f6d1fe94815fce70e1bffa12 468ea24fb4633a0d681f7ac84089566c1c6190cb Evan Jones evan.q.jones@gmail.com 2023-08-02T22:05:44-04:00 GitHub noreply@github.com 2023-08-02T22:05:44-04:00 examples : generate JSON according to schema (#1887) 468ea24fb4633a0d681f7ac84089566c1c6190cb 4f6b60c77652cdfc9d5545fe247ae5d764815598 Johannes Gäßler johannesg@5d6.de 2023-08-02T18:04:04+02:00 GitHub noreply@github.com 2023-08-02T18:04:04+02:00 CUDA: faster non k-quant mul_mat_q kernels (#2483) 4f6b60c77652cdfc9d5545fe247ae5d764815598 220d9318647a8ce127dbf7c9de5400455f41e7d8 Johannes Gäßler johannesg@5d6.de 2023-08-02T16:48:10+02:00 GitHub noreply@github.com 2023-08-02T16:48:10+02:00 CUDA: Fix models with output size != 32000 (#2480) 220d9318647a8ce127dbf7c9de5400455f41e7d8 81844fbcfd93a162b7aeaea9e4f2ab1358f7f97e ldwang ftgreat@163.com 2023-08-02T16:21:11+08:00 GitHub noreply@github.com 2023-08-02T11:21:11+03:00 readme : add Aquila-7B model series to supported models (#2487) 81844fbcfd93a162b7aeaea9e4f2ab1358f7f97e a312193e184b919047f33a5e844d846c5538dbe6 Eve 139727413+netrunnereve@users.noreply.github.com 2023-08-02T04:06:19-04:00 GitHub noreply@github.com 2023-08-02T11:06:19+03:00 tests : Fix compilation warnings (Linux/GCC) (#2451) a312193e184b919047f33a5e844d846c5538dbe6 c574bddb368424b5996cbee2ec45ec050967d404 Yiming Cui conandiy@vip.qq.com 2023-08-02T14:18:31+08:00 GitHub noreply@github.com 2023-08-02T09:18:31+03:00 readme : Add Chinese LLaMA-2 / Alpaca-2 to supported models (#2475) c574bddb368424b5996cbee2ec45ec050967d404 86aeb27734481751592abd85590020b40d9224c8 Bono Lv lvscar@users.noreply.github.com 2023-08-01T20:54:28+08:00 GitHub noreply@github.com 2023-08-01T14:54:28+02:00 fix a typo in examples/server/README.md (#2478) 86aeb27734481751592abd85590020b40d9224c8 1873ff586bd8499a18f763632711bf15d253585e ebraminio ebraminio@gmail.com 2023-08-01T01:56:23-07:00 GitHub noreply@github.com 2023-08-01T10:56:23+02:00 server : Support dark mode (#2414) 1873ff586bd8499a18f763632711bf15d253585e 49e7cb5bb1f75c91dd5db7d2d88cbc11bd9ee0c5 Matteo Boschini 12133566+mbosc@users.noreply.github.com 2023-08-01T09:43:12+02:00 GitHub noreply@github.com 2023-08-01T10:43:12+03:00 metal : add gqa8 kernel to allow llama-2-70B on metal (#2459) 49e7cb5bb1f75c91dd5db7d2d88cbc11bd9ee0c5 b772bba42e3bbca3cdab224456f8ff2ce427fd0b Johannes Gäßler johannesg@5d6.de 2023-07-31T21:02:19+02:00 GitHub noreply@github.com 2023-07-31T21:02:19+02:00 CUDA: fixed LLAMA_FAST compilation option (#2473) b772bba42e3bbca3cdab224456f8ff2ce427fd0b 0728c5a8b9569183ffca0399caac099afef87595 Johannes Gäßler johannesg@5d6.de 2023-07-31T19:52:22+02:00 GitHub noreply@github.com 2023-07-31T19:52:22+02:00 CUDA: fixed cmake F16 option (#2471) 0728c5a8b9569183ffca0399caac099afef87595 1215ed7d5ccf854a55eccb52661427bb985e7f2c Johannes Gäßler johannesg@5d6.de 2023-07-31T15:44:35+02:00 GitHub noreply@github.com 2023-07-31T15:44:35+02:00 CUDA: mmq CLI option, fixed mmq build issues (#2453) 1215ed7d5ccf854a55eccb52661427bb985e7f2c 2dbf518911926ef5a30f43aa83a0b1b1cdeaaa11 Johannes Gäßler johannesg@5d6.de 2023-07-31T14:32:30+02:00 GitHub noreply@github.com 2023-07-31T14:32:30+02:00 CUDA: Implemented row flattening for non-glm RoPE (#2468) 2dbf518911926ef5a30f43aa83a0b1b1cdeaaa11 9d2382b3e45b5815fc6a054045a2f2c2b18c22a2 Johannes Gäßler johannesg@5d6.de 2023-07-31T13:18:51+02:00 GitHub noreply@github.com 2023-07-31T13:18:51+02:00 CUDA: fewer memory bank conflicts for mul_mat_q (#2458) 9d2382b3e45b5815fc6a054045a2f2c2b18c22a2 a113689571420fb4d6540f1a324d12965781356a slaren slarengh@gmail.com 2023-07-31T11:02:53+02:00 GitHub noreply@github.com 2023-07-31T11:02:53+02:00 Fix Metal backend broken from the allocator changes (#2455) a113689571420fb4d6540f1a324d12965781356a 11f3ca06b8c66b0427aab0a472479da22553b472 slaren slarengh@gmail.com 2023-07-30T15:58:01+02:00 GitHub noreply@github.com 2023-07-30T15:58:01+02:00 ggml : add graph tensor allocator (#2411) 11f3ca06b8c66b0427aab0a472479da22553b472 9baf9ef304f330009d5a93b7390280a0fd27c9a1 Johannes Gäßler johannesg@5d6.de 2023-07-29T23:04:44+02:00 GitHub noreply@github.com 2023-07-29T23:04:44+02:00 CUDA: Quantized matrix matrix multiplication (#2160) 9baf9ef304f330009d5a93b7390280a0fd27c9a1 8a88e5855c3b93024be0f93290b01a4206b65b38 Johannes Gäßler johannesg@5d6.de 2023-07-29T23:04:10+02:00 GitHub noreply@github.com 2023-07-29T23:04:10+02:00 CUDA: faster multi GPU synchronization (#2448) 8a88e5855c3b93024be0f93290b01a4206b65b38 a9559bf77b903d94eb21614ceae5ae8950f0f1fc klosax 131523366+klosax@users.noreply.github.com 2023-07-28T20:25:36+02:00 GitHub noreply@github.com 2023-07-28T21:25:36+03:00 perplexity : add Hellaswag calculation (#2389) a9559bf77b903d94eb21614ceae5ae8950f0f1fc ee1b497c985f61d6ec519c39fcfed78a3c6f1d06 Lee 44310445+lx200916@users.noreply.github.com 2023-07-29T02:17:45+08:00 GitHub noreply@github.com 2023-07-28T21:17:45+03:00 ggml : workaround for missing _mm256_setr_m128i in GCC < 8 in k_quants.c (#2405) ee1b497c985f61d6ec519c39fcfed78a3c6f1d06 d73b8d48b45d6e2c0ae9bb8c39623c4024adc275 eric8607242 e0928021388@gmail.com 2023-07-29T02:10:05+08:00 GitHub noreply@github.com 2023-07-28T21:10:05+03:00 llama : support more diverse tokenizers? (#2420) d73b8d48b45d6e2c0ae9bb8c39623c4024adc275 34ae1caf7fdea4c4c09087002e15e6230102e6a9 Georgi Gerganov ggerganov@gmail.com 2023-07-28T21:05:08+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-28T21:05:08+03:00 examples : fix whitespace 34ae1caf7fdea4c4c09087002e15e6230102e6a9 d91f3f0c55663719ea03b76311e8c36ed55eb0e2 nhamanasu 45545786+nhamanasu@users.noreply.github.com 2023-07-29T03:02:10+09:00 GitHub noreply@github.com 2023-07-28T21:02:10+03:00 examples : server chat mode with llama2 (#2400) d91f3f0c55663719ea03b76311e8c36ed55eb0e2 65cdf34bdc469fa86248e667a5880992684ef114 Weird Constructor weirdconstructor@gmail.com 2023-07-28T10:44:43+02:00 GitHub noreply@github.com 2023-07-28T11:44:43+03:00 readme : fix the description of the Tail free sampling (TFS) method (#2431) 65cdf34bdc469fa86248e667a5880992684ef114 edcc7ae7d26007bbf83136e9d33f863fcad9b871 Rand Xie randxiexyy29@gmail.com 2023-07-28T01:42:53-07:00 GitHub noreply@github.com 2023-07-28T11:42:53+03:00 llama : use n_embd_gqa instead of n_embd to handle llama-2 70B (#2433) edcc7ae7d26007bbf83136e9d33f863fcad9b871 7c529cede6e84054e77a3eceab31c53de7b2f55b niansa/tuxifan tuxifan@posteo.de 2023-07-28T03:14:11+02:00 GitHub noreply@github.com 2023-07-28T03:14:11+02:00 Obtaining LLaMA 2 instructions (#2308) 7c529cede6e84054e77a3eceab31c53de7b2f55b 1a941869cbef8e9cc351a6c6987e4ae3b0f021f7 mj-shifu 77107165+mj-shifu@users.noreply.github.com 2023-07-27T22:39:17+02:00 GitHub noreply@github.com 2023-07-27T14:39:17-06:00 convert.py : Update to support 70B HF format model files (#2427) 1a941869cbef8e9cc351a6c6987e4ae3b0f021f7 b5472ea0ada081a6e1c06998ebbc9a24aa2cd4a4 Georgi Gerganov ggerganov@gmail.com 2023-07-27T11:00:54+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-27T11:00:54+03:00 metal : disable graph concurrency optimization due to bug (#2413) b5472ea0ada081a6e1c06998ebbc9a24aa2cd4a4 6df1f5940f889adde32fe47dc8881f010dcf9aba slaren slarengh@gmail.com 2023-07-26T23:57:23+02:00 GitHub noreply@github.com 2023-07-26T23:57:23+02:00 ggml : fix assert in ggml_set_unary_op (#2410) 6df1f5940f889adde32fe47dc8881f010dcf9aba 5488fb789ea5692268309baa76f67598155060be Cebtenzzre cebtenzzre@gmail.com 2023-07-26T14:00:04-04:00 GitHub noreply@github.com 2023-07-26T21:00:04+03:00 make : build with -Wmissing-prototypes (#2394) 5488fb789ea5692268309baa76f67598155060be eb542d39324574a6778fad9ba9e34ba7a14a82a3 slaren slarengh@gmail.com 2023-07-26T15:56:53+02:00 GitHub noreply@github.com 2023-07-26T15:56:53+02:00 ggml : allocate graphs in a context (#2392) eb542d39324574a6778fad9ba9e34ba7a14a82a3 07aaa0f63fccaeab099b3a732abda20b921bc5a5 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-25T18:35:53+03:00 GitHub noreply@github.com 2023-07-25T18:35:53+03:00 Add LLAMA_DEFAULT_RMS_EPS so we can change the default (#2384) 07aaa0f63fccaeab099b3a732abda20b921bc5a5 fce48caf9a6b9930eee9e2a5971428cdff403ba8 slaren slarengh@gmail.com 2023-07-25T16:20:12+02:00 GitHub noreply@github.com 2023-07-25T16:20:12+02:00 ggml : fix ggml_flash_attn to use op_params (#2387) fce48caf9a6b9930eee9e2a5971428cdff403ba8 875086bdb95ce1f3294439811536533199e3b579 ldwang ftgreat@163.com 2023-07-25T21:22:09+08:00 GitHub noreply@github.com 2023-07-25T16:22:09+03:00 convert.py : support bpe tokenizer (#2228) 875086bdb95ce1f3294439811536533199e3b579 da1889834a036a63ead2b0ca5c9ed8967712568c Jiahao Li liplus17@163.com 2023-07-25T20:58:32+08:00 GitHub noreply@github.com 2023-07-25T15:58:32+03:00 ggml : relax contiguous constraints in activation function (#2371) da1889834a036a63ead2b0ca5c9ed8967712568c 82552b7f5403ca13957ac9a2cdc1732470057b62 slaren slarengh@gmail.com 2023-07-25T14:32:20+02:00 GitHub noreply@github.com 2023-07-25T15:32:20+03:00 ggml : improve graph build time via hash table lookup (#2329) 82552b7f5403ca13957ac9a2cdc1732470057b62 0c06204fb39aa5560e883e0ae74be9518c57d88e Hesen Peng hesen.peng@gmail.com 2023-07-25T05:24:09-07:00 GitHub noreply@github.com 2023-07-25T15:24:09+03:00 build : fix line breaking error in build-info.sh (#2349) 0c06204fb39aa5560e883e0ae74be9518c57d88e 1fed755b1fb9babb6dbc1b4023e492950cd5a5be Xiao-Yong Jin jinxiaoyong@gmail.com 2023-07-25T07:19:11-05:00 GitHub noreply@github.com 2023-07-25T15:19:11+03:00 main : add `--in-prefix-bos` to prefix BOS to user inputs; keep EOS (#2304) 1fed755b1fb9babb6dbc1b4023e492950cd5a5be be2301bcdaf6c9f0921141bd071de7788e2a351a Eve 139727413+netrunnereve@users.noreply.github.com 2023-07-25T08:16:13-04:00 GitHub noreply@github.com 2023-07-25T15:16:13+03:00 ci : add non-AVX scalar build/test (#2356) be2301bcdaf6c9f0921141bd071de7788e2a351a 1aa18ef994a6a2b531434eb13251ef48e56d345b katsu560 118887472+katsu560@users.noreply.github.com 2023-07-25T21:13:41+09:00 GitHub noreply@github.com 2023-07-25T15:13:41+03:00 k_quants : add AVX support to dot functions with QK_K as 64 (#2339) 1aa18ef994a6a2b531434eb13251ef48e56d345b 9a08eaf3c4010962d0126e9e5bfbe9af64b2ac90 Shouzheng Liu lshzh.hi@gmail.com 2023-07-25T08:00:19-04:00 GitHub noreply@github.com 2023-07-25T15:00:19+03:00 metal : concurrently dispatch commands (#2358) 9a08eaf3c4010962d0126e9e5bfbe9af64b2ac90 129d844c87d90e74aafc23dcc84c980fd408def4 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-25T13:48:29+03:00 GitHub noreply@github.com 2023-07-25T13:48:29+03:00 Another speed gain for Q4_0 and Q4_1 on Metal (#2375) 129d844c87d90e74aafc23dcc84c980fd408def4 d5512b782b27ff698007dcd175da18959d5f163f Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-25T13:48:04+03:00 GitHub noreply@github.com 2023-07-25T13:48:04+03:00 Fix Q4_K and Q5_K for QK_K = 64 on CUDA (#2359) d5512b782b27ff698007dcd175da18959d5f163f c798308e3a425eae050a1f249a576fa8c6433327 slaren slarengh@gmail.com 2023-07-25T11:36:17+02:00 GitHub noreply@github.com 2023-07-25T12:36:17+03:00 server: add rms_norm_eps parameter (#2380) c798308e3a425eae050a1f249a576fa8c6433327 41c674161fb2459bdf7806d1eebead15bc5d046e Henri Vasserman henv@hot.ee 2023-07-25T10:27:34+03:00 GitHub noreply@github.com 2023-07-25T10:27:34+03:00 [Server] Escape HTML in webchat (#2368) 41c674161fb2459bdf7806d1eebead15bc5d046e b3f138d05849ccbce67303ac17b50ebbc268128a slaren slarengh@gmail.com 2023-07-24T17:57:12+02:00 GitHub noreply@github.com 2023-07-24T17:57:12+02:00 make rms_norm_eps a parameter (#2374) b3f138d05849ccbce67303ac17b50ebbc268128a 5b2b2dc6ae8086bff7c9b3c17fb435cf319b7185 Aarni Koskela akx@iki.fi 2023-07-24T17:54:22+03:00 GitHub noreply@github.com 2023-07-24T17:54:22+03:00 Chat UI extras (#2366) 5b2b2dc6ae8086bff7c9b3c17fb435cf319b7185 42f70cb2f6a8089e0a0560a459e4ba317bac4d49 Georgi Gerganov ggerganov@gmail.com 2023-07-24T14:46:21+03:00 GitHub noreply@github.com 2023-07-24T14:46:21+03:00 ggml : sync (unary ops refactor, static-correctness) (#2370) 42f70cb2f6a8089e0a0560a459e4ba317bac4d49 84e09a7d8bc4ab6d658b5cd81295ac0add60be78 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-24T12:55:02+03:00 GitHub noreply@github.com 2023-07-24T12:55:02+03:00 Fix scalar version of Q5_K when QK_K = 64 (#2362) 84e09a7d8bc4ab6d658b5cd81295ac0add60be78 2f9cf974a066ac0e03fbb235d834b01b0164d743 Evan Jones evan.q.jones@gmail.com 2023-07-23T23:58:10-04:00 GitHub noreply@github.com 2023-07-23T23:58:10-04:00 llama : add grammar-based sampling (#1773) 2f9cf974a066ac0e03fbb235d834b01b0164d743 4f06592cc6b83979e4b442e8cb97b3948c857188 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-24T00:19:47+03:00 GitHub noreply@github.com 2023-07-24T00:19:47+03:00 Some more Q4_K and Q5_K speedup on CUDA (#2346) 4f06592cc6b83979e4b442e8cb97b3948c857188 70d26ac3883009946e737525506fa88f52727132 IgnacioFDM ignaciofdm@gmail.com 2023-07-23T17:31:17-03:00 GitHub noreply@github.com 2023-07-23T23:31:17+03:00 Add gqa parameter support to the server (#2351) 70d26ac3883009946e737525506fa88f52727132 57921ca6db53e08eb90010fba99add85be28b5a1 Johannes Gäßler johannesg@5d6.de 2023-07-23T17:49:06+02:00 GitHub noreply@github.com 2023-07-23T17:49:06+02:00 Fix __dp4a documentation (#2348) 57921ca6db53e08eb90010fba99add85be28b5a1 3602ac4255fd4cd589821346290b464a64b955d1 wzy 32936898+Freed-Wu@users.noreply.github.com 2023-07-23T21:33:02+08:00 GitHub noreply@github.com 2023-07-23T16:33:02+03:00 common : n_threads == -1 uses std::thread::hardware_concurrency() (#2347) 3602ac4255fd4cd589821346290b464a64b955d1 95a6c595e7ca8dbe47ccf8824e04213e10357f9a slaren slarengh@gmail.com 2023-07-23T15:19:39+02:00 GitHub noreply@github.com 2023-07-23T15:19:39+02:00 fix n_tasks (#2342) 95a6c595e7ca8dbe47ccf8824e04213e10357f9a e76d630df17e235e6b9ef416c45996765d2e36fb slaren slarengh@gmail.com 2023-07-23T14:36:02+02:00 GitHub noreply@github.com 2023-07-23T14:36:02+02:00 ggml: move op parameters from tensors to ggml_tensor::op_params (#2333) e76d630df17e235e6b9ef416c45996765d2e36fb 1d0824b2476e7fda09751a0235c9e571b76d6f2c Georgi Gerganov ggerganov@gmail.com 2023-07-23T15:09:47+03:00 GitHub noreply@github.com 2023-07-23T15:09:47+03:00 llama : grouped-query attention + LLaMAv2 70B support (#2276) 1d0824b2476e7fda09751a0235c9e571b76d6f2c bc3ec2cdc9ea20b0faba2e1b4576fab3a911e4d1 maddes8cht 55592906+maddes8cht@users.noreply.github.com 2023-07-23T13:59:48+02:00 GitHub noreply@github.com 2023-07-23T14:59:48+03:00 llama : print help to stdout (#2338) bc3ec2cdc9ea20b0faba2e1b4576fab3a911e4d1 a940458e4814e87bd0d3fbdb3f3d2733b4a3ccb1 wzy 32936898+Freed-Wu@users.noreply.github.com 2023-07-23T19:57:02+08:00 GitHub noreply@github.com 2023-07-23T14:57:02+03:00 flake : support `nix build '.#opencl'` (#2337) a940458e4814e87bd0d3fbdb3f3d2733b4a3ccb1 91171b8072f6f0c8ae3a61e23451acb538bb9ece Christian Demsar christian@github.email.demsar.us 2023-07-23T07:56:34-04:00 GitHub noreply@github.com 2023-07-23T14:56:34+03:00 llama : print max tensor size to stderr (#2336) 91171b8072f6f0c8ae3a61e23451acb538bb9ece 355c80f49e32b0b15c0a457f3bad380e57f5b9ac Jose Maldonado 63384398+yukiteruamano@users.noreply.github.com 2023-07-23T07:52:08-04:00 GitHub noreply@github.com 2023-07-23T14:52:08+03:00 make : fix CLBLAST compile support in FreeBSD (#2331) 355c80f49e32b0b15c0a457f3bad380e57f5b9ac 83a00ce69bef9124c0702424a012ea799128b77d AustinMroz austinmroz@utexas.edu 2023-07-23T06:16:48-05:00 GitHub noreply@github.com 2023-07-23T14:16:48+03:00 examples : simplify vim plugin (#2327) 83a00ce69bef9124c0702424a012ea799128b77d d2a43664f93ba30a84e42713bb69f936cbdacf2a Jiahao Li liplus17@163.com 2023-07-23T19:00:37+08:00 GitHub noreply@github.com 2023-07-23T14:00:37+03:00 metal : support bcast add & dup & cont op (#2323) d2a43664f93ba30a84e42713bb69f936cbdacf2a b9b7d94fc10a8039befd1bc3af4f4b09c620c351 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-23T08:49:20+03:00 GitHub noreply@github.com 2023-07-23T08:49:20+03:00 Speed up Q4_K (#2322) b9b7d94fc10a8039befd1bc3af4f4b09c620c351 b47b8a9cfeb439d271bf997fb985fd6d82b3af5e Johannes Gäßler johannesg@5d6.de 2023-07-22T21:27:34+02:00 GitHub noreply@github.com 2023-07-22T21:27:34+02:00 CUDA: Fixed 7b q3_K_S with mul_mat_vec_q (#2313) b47b8a9cfeb439d271bf997fb985fd6d82b3af5e b5fe67f8c69113bd9354bc1adcfe2df6be323740 Georgi Gerganov ggerganov@gmail.com 2023-07-22T21:17:57+03:00 GitHub noreply@github.com 2023-07-22T21:17:57+03:00 llama : optimize memory buffers (#2325) b5fe67f8c69113bd9354bc1adcfe2df6be323740 24baa54ac1ff3d4156a2360deb1473af04a9b1a2 klosax 131523366+klosax@users.noreply.github.com 2023-07-22T14:21:24+02:00 GitHub noreply@github.com 2023-07-22T14:21:24+02:00 Perplexity: Compute scores correlated to HellaSwag (#2312) 24baa54ac1ff3d4156a2360deb1473af04a9b1a2 dd6c67d3cbb7b360747f776412bf01976aa32f4b whoreson 139810751+whoreson@users.noreply.github.com 2023-07-22T12:34:51+02:00 GitHub noreply@github.com 2023-07-22T13:34:51+03:00 examples : basic VIM plugin dd6c67d3cbb7b360747f776412bf01976aa32f4b 5d500e8ccf5eee3de3ae66685cc3be75e43e08b9 Georgi Gerganov ggerganov@gmail.com 2023-07-22T12:00:56+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-22T12:00:56+03:00 ci : fix args 5d500e8ccf5eee3de3ae66685cc3be75e43e08b9 7d5f18468ceabd7a38f414f9f21b26b0c137f994 Georgi Gerganov ggerganov@gmail.com 2023-07-22T11:48:22+03:00 GitHub noreply@github.com 2023-07-22T11:48:22+03:00 ci : add 7B CUDA tests (#2319) 7d5f18468ceabd7a38f414f9f21b26b0c137f994 d924522a46c5ef097af4a88087d91673e8e87e4d Richard Roberson richardr1126@gmail.com 2023-07-21T13:01:10-06:00 GitHub noreply@github.com 2023-07-21T22:01:10+03:00 examples : add easy python script to create quantized (k-bit support) GGML models from local HF Transformer models (#2311) d924522a46c5ef097af4a88087d91673e8e87e4d 4d76a5f49b9b5382dba5d13d92edb9159536c225 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-21T17:27:51+03:00 GitHub noreply@github.com 2023-07-21T17:27:51+03:00 Custom RoPE + bettter memory management for CUDA (#2295) 4d76a5f49b9b5382dba5d13d92edb9159536c225 0db14fef06836caaa13cc123c0a24dc598bdb9f0 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-21T17:05:30+03:00 GitHub noreply@github.com 2023-07-21T17:05:30+03:00 Faster Q3_K implementation on Metal (#2307) 0db14fef06836caaa13cc123c0a24dc598bdb9f0 03e566977b277937c5f706180171c5d12b597b0b Georgi Gerganov ggerganov@gmail.com 2023-07-21T15:16:55+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-21T15:16:55+03:00 ggml : fix the rope fix (513f8619535a64fa9ace808cdcbcf66211535f5c) 03e566977b277937c5f706180171c5d12b597b0b 513f8619535a64fa9ace808cdcbcf66211535f5c Ikko Eltociear Ashimine eltociear@gmail.com 2023-07-21T20:53:07+09:00 GitHub noreply@github.com 2023-07-21T14:53:07+03:00 examples : fix typo in minigpt4.py (#2298) 513f8619535a64fa9ace808cdcbcf66211535f5c 3973b25a64a37a47eac156a3fd28f83c16f14bf2 Georgi Gerganov ggerganov@gmail.com 2023-07-21T14:51:34+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-21T14:51:34+03:00 ggml : fix rope args order + assert (#2054) 3973b25a64a37a47eac156a3fd28f83c16f14bf2 ab0e26bdfb7b3adb1e3145c61a0fa92d1abd21d0 Georgi Gerganov ggerganov@gmail.com 2023-07-21T14:42:41+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-21T14:42:41+03:00 gitignore : fix final newline ab0e26bdfb7b3adb1e3145c61a0fa92d1abd21d0 73643f5fb1136dc2b65ae910bdc5a431520d70a2 Guillaume "Vermeille" Sanchez Guillaume.V.Sanchez@gmail.com 2023-07-21T12:58:36+02:00 GitHub noreply@github.com 2023-07-21T13:58:36+03:00 llama : remove cfg smooth factor as it is only a reparameterization of the guidance scale (#2280) 73643f5fb1136dc2b65ae910bdc5a431520d70a2 a814d04f81121e0429b39a61fe4afd946cd42046 Jose Maldonado 63384398+yukiteruamano@users.noreply.github.com 2023-07-21T06:53:27-04:00 GitHub noreply@github.com 2023-07-21T13:53:27+03:00 gitignore : changes for Poetry users + chat examples (#2284) a814d04f81121e0429b39a61fe4afd946cd42046 4c013bb7385a0e52ce721480c40c45bec5ef103f Georgi Gerganov ggerganov@gmail.com 2023-07-21T13:50:55+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-21T13:50:55+03:00 make : fix indentation 4c013bb7385a0e52ce721480c40c45bec5ef103f 42c7c2e2e9cae79330f57456fbc0eae1eaff17fa Georgi Gerganov ggerganov@gmail.com 2023-07-21T13:48:18+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-21T13:49:18+03:00 ci : fix MNT realpath usage (#2250) 42c7c2e2e9cae79330f57456fbc0eae1eaff17fa 78a3d13424b01c3f8ea94ea7e59650ab0501e902 Sky Yan skyan83@gmail.com 2023-07-21T18:38:57+08:00 GitHub noreply@github.com 2023-07-21T13:38:57+03:00 make : support customized LLAMA_CUDA_NVCC and LLAMA_CUDA_CCBIN (#2275) 78a3d13424b01c3f8ea94ea7e59650ab0501e902 ae178ab46bfd6ecb2422da5dad441a4e2fef8b7e wzy 32936898+Freed-Wu@users.noreply.github.com 2023-07-21T18:26:34+08:00 GitHub noreply@github.com 2023-07-21T13:26:34+03:00 flake : remove intel mkl from flake.nix due to missing files (#2277) ae178ab46bfd6ecb2422da5dad441a4e2fef8b7e 54e3bc76fed914f8d4a30a7a50c19867cccb1338 Georgi Gerganov ggerganov@gmail.com 2023-07-21T13:10:51+03:00 GitHub noreply@github.com 2023-07-21T13:10:51+03:00 llama : make tensor_split ptr instead of array (#2272) 54e3bc76fed914f8d4a30a7a50c19867cccb1338 019fe257bbf699f400231683a8b816ad90281275 Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2023-07-21T12:09:16+02:00 GitHub noreply@github.com 2023-07-21T13:09:16+03:00 make : add new target for test binaries (#2244) 019fe257bbf699f400231683a8b816ad90281275 e68c96f7fee8fc22814a4a1209ffc97bbf35f7bd Hatsune Miku 129688334+at8u@users.noreply.github.com 2023-07-21T08:13:18Z GitHub noreply@github.com 2023-07-21T11:13:18+03:00 MIKU MAYHEM: Upgrading the Default Model for Maximum Fun 🎉 (#2287) e68c96f7fee8fc22814a4a1209ffc97bbf35f7bd 9cf022a1889e50113fd348dc96b4557fc75a6296 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-21T10:44:40+03:00 GitHub noreply@github.com 2023-07-21T10:44:40+03:00 Faster Q2_K on Metal (#2297) 9cf022a1889e50113fd348dc96b4557fc75a6296 e782c9e735f93ab4767ffc37462c523b73a17ddc Przemysław Pawełczyk przemoc@gmail.com 2023-07-21T09:42:21+02:00 GitHub noreply@github.com 2023-07-21T10:42:21+03:00 make : fix embdinput library and server examples building on MSYS2 (#2235) e782c9e735f93ab4767ffc37462c523b73a17ddc 785829dfe8baf0213f2ff66963d28c62f92d7930 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-20T18:19:45+03:00 GitHub noreply@github.com 2023-07-20T18:19:45+03:00 Faster Q5_K and Q6_K on Metal (#2294) 785829dfe8baf0213f2ff66963d28c62f92d7930 fff0e0eafe817eef429ecb64f892ab7bdae31846 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-20T15:18:43+03:00 GitHub noreply@github.com 2023-07-20T15:18:43+03:00 Faster Q4_K on Metal (#2290) fff0e0eafe817eef429ecb64f892ab7bdae31846 417a85a0010519224cf154eb85d383ffeafeeead Georgi Gerganov ggerganov@gmail.com 2023-07-20T13:47:26+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-20T13:47:26+03:00 llama : fix regression from #2000 - could not load no-mmap models 417a85a0010519224cf154eb85d383ffeafeeead 294f424554c1599784ac9962462fc39ace92d8a5 Shouzheng Liu lshzh.hi@gmail.com 2023-07-20T06:32:22-04:00 GitHub noreply@github.com 2023-07-20T13:32:22+03:00 metal: minor q4 optimization and reduce code size (#2248) 294f424554c1599784ac9962462fc39ace92d8a5 45a1b07e9b20c33d71d8c849ff27d693a75a0269 Rinne AsakusaRinne@gmail.com 2023-07-19T15:06:40+08:00 GitHub noreply@github.com 2023-07-19T10:06:40+03:00 llama : extend API to get max devices at runtime (#2253) 45a1b07e9b20c33d71d8c849ff27d693a75a0269 b1f429095328a34556c0e9a7a2fefced3db3368c wzy 32936898+Freed-Wu@users.noreply.github.com 2023-07-19T15:01:55+08:00 GitHub noreply@github.com 2023-07-19T10:01:55+03:00 flake : update flake.nix (#2270) b1f429095328a34556c0e9a7a2fefced3db3368c d01bccde9f759b24449fdaa16306b406a50eb367 wzy 32936898+Freed-Wu@users.noreply.github.com 2023-07-19T15:01:11+08:00 GitHub noreply@github.com 2023-07-19T10:01:11+03:00 cmake : install targets (#2256) d01bccde9f759b24449fdaa16306b406a50eb367 6cbf9dfb32f0e23ed3afd02d30ab066ed53e2c4d Georgi Gerganov ggerganov@gmail.com 2023-07-18T14:24:43+03:00 GitHub noreply@github.com 2023-07-18T14:24:43+03:00 ci : integrate with ggml-org/ci (#2250) 6cbf9dfb32f0e23ed3afd02d30ab066ed53e2c4d 7568d1a2b206331412106ea66da3f871025e0c3c Georgi Gerganov ggerganov@gmail.com 2023-07-18T11:50:49+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-18T11:50:49+03:00 llama : shorten quantization descriptions 7568d1a2b206331412106ea66da3f871025e0c3c b7647436ccc80970b44a270f70f4f2ea139054d1 Jiahao Li liplus17@163.com 2023-07-18T01:39:29+08:00 GitHub noreply@github.com 2023-07-17T20:39:29+03:00 Support dup & cont ops on CUDA (#2242) b7647436ccc80970b44a270f70f4f2ea139054d1 672dda10e4d8ac79df5d5970da7fb69d242ca9a7 Alex Klinkhamer git@grencez.dev 2023-07-16T14:01:45-07:00 GitHub noreply@github.com 2023-07-17T00:01:45+03:00 llama : fix t_start_sample_us initialization warning (#2238) 672dda10e4d8ac79df5d5970da7fb69d242ca9a7 27ab66e437797aedbb23b3599385756b6c26ac39 Qingyou Meng meng.qingyou@gmail.com 2023-07-17T03:57:28+08:00 GitHub noreply@github.com 2023-07-16T22:57:28+03:00 ggml : fixed runtime bugs and compile errors related to GGML_PERF and GGML_DEBUG (#2219) 27ab66e437797aedbb23b3599385756b6c26ac39 6e7cca404748dd4b1a3affd0d1296e37f4ac0a6f Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2023-07-16T21:54:47+02:00 GitHub noreply@github.com 2023-07-16T22:54:47+03:00 py : turn verify-checksum-models.py into executable (#2245) 6e7cca404748dd4b1a3affd0d1296e37f4ac0a6f a6803cab946c817fb7aaf2a40b317f5d3e373bd1 Xiao-Yong Jin jinxiaoyong@gmail.com 2023-07-15T06:34:16-04:00 GitHub noreply@github.com 2023-07-15T13:34:16+03:00 llama : add custom RoPE (#2054) a6803cab946c817fb7aaf2a40b317f5d3e373bd1 7dabc66f3c63f8ea0f61bac346fa138e01df675f Dave Della Costa ddellacosta+github@gmail.com 2023-07-14T15:13:38-04:00 GitHub noreply@github.com 2023-07-14T22:13:38+03:00 flake : add runHook preInstall/postInstall to installPhase so hooks function (#2224) 7dabc66f3c63f8ea0f61bac346fa138e01df675f 7cdd30bf1f84339c55a5e3de29384f6bbdebb61c wzy 32936898+Freed-Wu@users.noreply.github.com 2023-07-15T03:05:08+08:00 GitHub noreply@github.com 2023-07-14T22:05:08+03:00 make : use pkg-config for OpenBLAS (#2222) 7cdd30bf1f84339c55a5e3de29384f6bbdebb61c e8035f141e1f71d739fa5cfc9c01531cdee6fc16 Bach Le bach@bullno1.com 2023-07-15T03:00:58+08:00 GitHub noreply@github.com 2023-07-14T22:00:58+03:00 cuda : allocate all temporary ggml_tensor_extra_gpu from a fixed-size buffer (#2220) e8035f141e1f71d739fa5cfc9c01531cdee6fc16 7513b7b0a1c11faa00ad5a34d22681e5f07d32e4 Evan Miller emmiller@gmail.com 2023-07-14T14:55:56-04:00 GitHub noreply@github.com 2023-07-14T21:55:56+03:00 ggml : fix static_assert with older compilers #2024 (#2218) 7513b7b0a1c11faa00ad5a34d22681e5f07d32e4 de8342423d9600cf6e15455c1a27bae441262b45 Bach Le bach@bullno1.com 2023-07-15T02:55:24+08:00 GitHub noreply@github.com 2023-07-14T21:55:24+03:00 llama : add functions that work directly on model (#2197) de8342423d9600cf6e15455c1a27bae441262b45 c48c525f8711780f3f7c59bf92f1760f38317218 Ali Chraghi 63465728+alichraghi@users.noreply.github.com 2023-07-14T11:50:58-07:00 GitHub noreply@github.com 2023-07-14T21:50:58+03:00 build.zig : install config header (#2216) c48c525f8711780f3f7c59bf92f1760f38317218 206e01de117cc65ed8713ac9fdebc57ba4532ec3 Shangning Xu 32517059+xushangning@users.noreply.github.com 2023-07-15T02:40:05+08:00 GitHub noreply@github.com 2023-07-14T21:40:05+03:00 examples : fixed path typos in embd-input (#2214) 206e01de117cc65ed8713ac9fdebc57ba4532ec3 4304bd3cded73c867a882ea5ca4517e3995cc996 Jiahao Li liplus17@163.com 2023-07-15T02:38:24+08:00 GitHub noreply@github.com 2023-07-14T21:38:24+03:00 cuda : support broadcast add & mul (#2192) 4304bd3cded73c867a882ea5ca4517e3995cc996 229aab351c375899debad45fcb213bf0565bba4e Johannes Gäßler johannesg@5d6.de 2023-07-14T19:44:08+02:00 GitHub noreply@github.com 2023-07-14T19:44:08+02:00 CUDA: mul_mat_vec_q kernels for k-quants (#2203) 229aab351c375899debad45fcb213bf0565bba4e 697966680b27d9b4f05668605b863cb9aea3e15f James Reynolds magnusviri@users.noreply.github.com 2023-07-14T11:34:40-06:00 GitHub noreply@github.com 2023-07-14T20:34:40+03:00 make : fix combination of LLAMA_METAL and LLAMA_MPI (#2208) 697966680b27d9b4f05668605b863cb9aea3e15f 27ad57a69b85bf12420a27e9945e580cc280be57 Georgi Gerganov ggerganov@gmail.com 2023-07-14T16:36:41+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-14T16:36:41+03:00 ggml : sync (ggml_conv_2d, fix mul_mat bug, CUDA GLM rope) 27ad57a69b85bf12420a27e9945e580cc280be57 32c54116318929c90fd7ae814cf9b5232cd44c36 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-07-14T12:46:21+03:00 GitHub noreply@github.com 2023-07-14T11:46:21+02:00 Metal: faster Q4_0 and Q4_1 matrix x vector kernels (#2212) 32c54116318929c90fd7ae814cf9b5232cd44c36 ff5d58faecf1f02b05bd015bdfc6a394cf2bc9ba Howard Su howard0su@gmail.com 2023-07-13T21:58:25+08:00 GitHub noreply@github.com 2023-07-13T21:58:25+08:00 Revert "Support using mmap when applying LoRA (#2095)" (#2206) ff5d58faecf1f02b05bd015bdfc6a394cf2bc9ba b782422a3e090d0aeab84bfa03ba008dcd1c2a3d Howard Su howard0su@gmail.com 2023-07-13T21:58:09+08:00 GitHub noreply@github.com 2023-07-13T21:58:09+08:00 Fix compile error on Windows CUDA (#2207) b782422a3e090d0aeab84bfa03ba008dcd1c2a3d 1cbf561466e957b25f0e8163c2386683f8674369 Bodo Graumann mail@bodograumann.de 2023-07-13T15:49:14+02:00 GitHub noreply@github.com 2023-07-13T16:49:14+03:00 devops : add missing quotes to bash script (#2193) 1cbf561466e957b25f0e8163c2386683f8674369 975221e9548ef6d9f4af8d39cdffc4811c050beb Shouzheng Liu 61452103+lshzh-ww@users.noreply.github.com 2023-07-12T16:10:55-04:00 GitHub noreply@github.com 2023-07-12T23:10:55+03:00 metal : new q4_0 matrix-vector kernel (#2188) 975221e9548ef6d9f4af8d39cdffc4811c050beb 4523d10d0cf8c088f1b26c76d38d73290eb3b444 Georgi Gerganov ggerganov@gmail.com 2023-07-12T20:51:29+03:00 GitHub noreply@github.com 2023-07-12T20:51:29+03:00 ggml : broadcast mul_mat + conv batch support (#2199) 4523d10d0cf8c088f1b26c76d38d73290eb3b444 680e6f91775f972f0df34f56807f30826370db59 Georgi Gerganov ggerganov@gmail.com 2023-07-12T20:27:03+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-12T20:32:15+03:00 ggml : add ggml_pool_1d and ggml_pool_2d 680e6f91775f972f0df34f56807f30826370db59 4e7464ef88885cb3532738b03cac890f4077fa20 Georgi Gerganov ggerganov@gmail.com 2023-07-12T20:26:18+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-12T20:32:15+03:00 cuda : add gelu support 4e7464ef88885cb3532738b03cac890f4077fa20 2b5eb72e109577ed84e44bb8fa47e4956f337300 Howard Su howard0su@gmail.com 2023-07-12T20:18:40+08:00 GitHub noreply@github.com 2023-07-12T20:18:40+08:00 FP16 is supported in CM=6.0 (#2177) 2b5eb72e109577ed84e44bb8fa47e4956f337300 f7d278faf308cb989c221895968f2a26f14b2155 Johannes Gäßler johannesg@5d6.de 2023-07-12T10:38:52+02:00 GitHub noreply@github.com 2023-07-12T10:38:52+02:00 Fixed __dp4a compute capability: 6.0 -> 6.1 (#2189) f7d278faf308cb989c221895968f2a26f14b2155 20d7740a9b45f6e5b247fa3738fdda35e18c2e8a Georgi Gerganov ggerganov@gmail.com 2023-07-12T10:54:19+03:00 GitHub noreply@github.com 2023-07-12T10:54:19+03:00 ggml : revert CUDA broadcast changes from #2183 (#2191) 20d7740a9b45f6e5b247fa3738fdda35e18c2e8a 5bf2a2771886ee86137e01dbc7492f78fb392066 Georgi Gerganov ggerganov@gmail.com 2023-07-11T22:53:34+03:00 GitHub noreply@github.com 2023-07-11T22:53:34+03:00 ggml : sync (abort callback, mul / add broadcast, fix alibi) (#2183) 5bf2a2771886ee86137e01dbc7492f78fb392066 c9c74b4e3f9dcfab8b0032749ff8a579ab4e4d8d Spencer Sutton spencersutton@users.noreply.github.com 2023-07-11T12:31:10-04:00 GitHub noreply@github.com 2023-07-11T19:31:10+03:00 ggml : remove src0 and src1 from ggml_tensor and rename opt to src (#2178) c9c74b4e3f9dcfab8b0032749ff8a579ab4e4d8d 3ec7e596b2ba3f43c22f441254ca2bcfa91102ba Bach Le bach@bullno1.com 2023-07-12T00:18:43+08:00 GitHub noreply@github.com 2023-07-11T19:18:43+03:00 llama : add classifier-free guidance (#2135) 3ec7e596b2ba3f43c22f441254ca2bcfa91102ba 917831c63a4138814d23da1917bf2b5d5b9faa6c Jinwoo Jeong 33892306+williamjeong2@users.noreply.github.com 2023-07-12T01:12:35+09:00 GitHub noreply@github.com 2023-07-11T19:12:35+03:00 docker : add '--server' option (#2174) 917831c63a4138814d23da1917bf2b5d5b9faa6c 2347463201a9f4159ae95b737e1544dd300569c8 Chad Brewbaker crb002@gmail.com 2023-07-11T11:03:06-05:00 GitHub noreply@github.com 2023-07-11T19:03:06+03:00 readme : fix zig build instructions (#2171) 2347463201a9f4159ae95b737e1544dd300569c8 bbef28218fe827265716b66977719b9ee2b21165 Howard Su howard0su@gmail.com 2023-07-11T22:37:01+08:00 GitHub noreply@github.com 2023-07-11T22:37:01+08:00 Support using mmap when applying LoRA (#2095) bbef28218fe827265716b66977719b9ee2b21165 5656d10599bd756dc0f17284e418e704200b43f3 LostRuins 39025047+LostRuins@users.noreply.github.com 2023-07-11T22:01:08+08:00 GitHub noreply@github.com 2023-07-11T22:01:08+08:00 Possible solution to allow K-quants on models with n_vocab!=32000 (#2148) 5656d10599bd756dc0f17284e418e704200b43f3 1d1630996920f889cdc08de26cebf2415958540e Evan Miller emmiller@gmail.com 2023-07-10T11:49:56-04:00 GitHub noreply@github.com 2023-07-10T18:49:56+03:00 mpi : add support for distributed inference via MPI (#2099) 1d1630996920f889cdc08de26cebf2415958540e db4047ad5cd8eae04db3b2efe0245e69a376601a oobabooga 112222186+oobabooga@users.noreply.github.com 2023-07-09T05:59:53-03:00 GitHub noreply@github.com 2023-07-09T11:59:53+03:00 llama : remove "first token must be BOS" restriction (#2153) db4047ad5cd8eae04db3b2efe0245e69a376601a 18780e0a5e17348236230bbe891901b9b5718709 Nigel Bosch pnigelb@gmail.com 2023-07-09T03:56:18-05:00 GitHub noreply@github.com 2023-07-09T11:56:18+03:00 main : escape prompt prefix/suffix (#2151) 18780e0a5e17348236230bbe891901b9b5718709 3bbc1a11f04a9adc0d0e08c2940ba4d2978755ab JackJollimore 130917767+JackJollimore@users.noreply.github.com 2023-07-09T05:20:43-03:00 GitHub noreply@github.com 2023-07-09T11:20:43+03:00 readme : update Termux instructions (#2147) 3bbc1a11f04a9adc0d0e08c2940ba4d2978755ab 2492a53fd0d8372ecc67f49f07b581905175eea8 clyang clyang@clyang.net 2023-07-09T16:12:20+08:00 GitHub noreply@github.com 2023-07-09T11:12:20+03:00 ggml : fix buidling with Intel MKL but ask for "cblas.h" issue (#2104) (#2115) 2492a53fd0d8372ecc67f49f07b581905175eea8 64639555ff93c8ead2b80becb49cc6b60aeac240 rankaiyx rankaiyx@rankaiyx.com 2023-07-09T15:38:42+08:00 GitHub noreply@github.com 2023-07-09T10:38:42+03:00 readme : add more docs indexes (#2127) 64639555ff93c8ead2b80becb49cc6b60aeac240 061f5f8d2109bb7adcbd40f1b456d887c5a1df25 Johannes Gäßler johannesg@5d6.de 2023-07-08T20:01:44+02:00 GitHub noreply@github.com 2023-07-08T20:01:44+02:00 Fixed OpenLLaMA 3b CUDA mul_mat_vec_q (#2144) 061f5f8d2109bb7adcbd40f1b456d887c5a1df25 84525e7962bee0abef91108948bbf7f7bfdcf421 Johannes Gäßler johannesg@5d6.de 2023-07-08T00:25:15+02:00 GitHub noreply@github.com 2023-07-08T00:25:15+02:00 CUDA: add __restrict__ to mul mat vec kernels (#2140) 84525e7962bee0abef91108948bbf7f7bfdcf421 a7e20edf2266169ccd97a4eb949a593d628fbd64 dylan canardleteer@users.noreply.github.com 2023-07-07T11:25:25-07:00 GitHub noreply@github.com 2023-07-07T21:25:25+03:00 docker : add support for CUDA in docker (#1461) a7e20edf2266169ccd97a4eb949a593d628fbd64 1d656d6360359cfdaaf5d64ed9690047b600dbcb Georgi Gerganov ggerganov@gmail.com 2023-07-07T21:23:57+03:00 GitHub noreply@github.com 2023-07-07T21:23:57+03:00 ci : switch threads to 1 (#2138) 1d656d6360359cfdaaf5d64ed9690047b600dbcb 72421402834141df6cbdcf595fe46dbd11874dce Qingyou Meng meng.qingyou@gmail.com 2023-07-08T00:24:01+08:00 GitHub noreply@github.com 2023-07-07T19:24:01+03:00 ggml : change ggml_graph_compute() API to not require context (#1999) 72421402834141df6cbdcf595fe46dbd11874dce 3e08ae99ceb143d67f9273fda47541e9d98ff23f Georgi Gerganov ggerganov@gmail.com 2023-07-07T18:36:37+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-07T18:37:10+03:00 ggml : remove sched_yield() call in ggml_graph_compute_thread() (#2134) 3e08ae99ceb143d67f9273fda47541e9d98ff23f 481f793acc3882a09d45d8d2c3076ad3d1c60cfc Aarni Koskela akx@iki.fi 2023-07-07T16:12:49+03:00 GitHub noreply@github.com 2023-07-07T09:12:49-04:00 convert.py: add mapping for safetensors bf16 (#1598) 481f793acc3882a09d45d8d2c3076ad3d1c60cfc dfd9fce6d65599bf33df43e616e85aa639bdae4c Howard Su howard0su@gmail.com 2023-07-07T11:34:18+08:00 GitHub noreply@github.com 2023-07-07T05:34:18+02:00 Fix opencl by wrap #if-else-endif with \n (#2086) dfd9fce6d65599bf33df43e616e85aa639bdae4c 36680f6e40e4440c3ec3385d0b7e5ca8bb6c37f7 Georgi Gerganov ggerganov@gmail.com 2023-07-06T19:41:31+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-06T19:41:31+03:00 ggml : fix restrict usage 36680f6e40e4440c3ec3385d0b7e5ca8bb6c37f7 a17a2683d8fdb899ba497d0c28ccafb28c62efb6 Judd foldl@users.noreply.github.com 2023-07-07T00:23:49+08:00 GitHub noreply@github.com 2023-07-06T19:23:49+03:00 convert : update for baichuan (#2081) a17a2683d8fdb899ba497d0c28ccafb28c62efb6 31cfbb1013a482e89c72146e2063ac4362becae7 tslmy tslmy@users.noreply.github.com 2023-07-06T09:17:50-07:00 GitHub noreply@github.com 2023-07-06T19:17:50+03:00 alpaca.sh : update model file name (#2074) 31cfbb1013a482e89c72146e2063ac4362becae7 983b555e9ddb36703cee4d22642afe958de093b7 Tobias Lütke tobi@shopify.com 2023-07-05T16:51:13-04:00 GitHub noreply@github.com 2023-07-05T16:51:13-04:00 Expose generation timings from server & update completions.js (#2116) 983b555e9ddb36703cee4d22642afe958de093b7 ec326d350c72afd23709a409944728a607188cc0 Jesse Jojo Johnson williamsaintgeorge@gmail.com 2023-07-05T18:03:19Z GitHub noreply@github.com 2023-07-05T21:03:19+03:00 Update Server Instructions (#2113) ec326d350c72afd23709a409944728a607188cc0 1b6efeab829f3eeda5b39bd47624bb60b3531b88 Georgi Gerganov ggerganov@gmail.com 2023-07-05T20:44:11+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-05T20:44:11+03:00 ggml : fix bug introduced in #1237 1b6efeab829f3eeda5b39bd47624bb60b3531b88 1b107b8550dced48dc5f41184640061354226b96 Georgi Gerganov ggerganov@gmail.com 2023-07-05T20:20:05+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-05T20:20:25+03:00 tests : fix test-grad0 1b107b8550dced48dc5f41184640061354226b96 8567c76b5326e862be0755a8dc1dd988223fcae3 Stephan Walter stephan@walter.name 2023-07-05T16:13:06Z GitHub noreply@github.com 2023-07-05T19:13:06+03:00 ggml : generalize `quantize_fns` for simpler FP16 handling (#1237) 8567c76b5326e862be0755a8dc1dd988223fcae3 924dd22fd3ba93e097f8d19ba5cda919ca2fe2fb Jesse Jojo Johnson williamsaintgeorge@gmail.com 2023-07-05T15:13:35Z GitHub noreply@github.com 2023-07-05T18:13:35+03:00 Update server instructions for web front end (#2103) 924dd22fd3ba93e097f8d19ba5cda919ca2fe2fb 051c70dcd55709c9cbbfa849af035951fe720433 Johannes Gäßler johannesg@5d6.de 2023-07-05T14:19:42+02:00 GitHub noreply@github.com 2023-07-05T14:19:42+02:00 Quantized dot products for CUDA mul mat vec (#2067) 051c70dcd55709c9cbbfa849af035951fe720433 9e4475f5cf639315f61ed7b8da6258bb0c7c5ca9 Howard Su howard0su@gmail.com 2023-07-05T18:31:23+08:00 GitHub noreply@github.com 2023-07-05T18:31:23+08:00 llama: Don't double count the sampling time (#2107) 9e4475f5cf639315f61ed7b8da6258bb0c7c5ca9 7f0e9a775ecc4c6ade271c217f63d6dc93e79eaa Johannes Gäßler johannesg@5d6.de 2023-07-05T08:58:05+02:00 GitHub noreply@github.com 2023-07-05T08:58:05+02:00 Fixed OpenCL offloading prints (#2082) 7f0e9a775ecc4c6ade271c217f63d6dc93e79eaa b472f3fca558b6335adbd87210ed58cfb5da37cb Nigel Bosch pnigelb@gmail.com 2023-07-04T18:33:33-05:00 GitHub noreply@github.com 2023-07-05T07:33:33+08:00 embd-input: Fix input embedding example unsigned int seed (#2105) b472f3fca558b6335adbd87210ed58cfb5da37cb ed9a54e5129a11c2a5b555e1dc65e875e3c37b4f Georgi Gerganov ggerganov@gmail.com 2023-07-04T22:25:22+03:00 GitHub noreply@github.com 2023-07-04T22:25:22+03:00 readme : add link web chat PR ed9a54e5129a11c2a5b555e1dc65e875e3c37b4f f257fd255044decffad93dee2502875ce66ad80c Georgi Gerganov ggerganov@gmail.com 2023-07-04T21:54:11+03:00 GitHub noreply@github.com 2023-07-04T21:54:11+03:00 ggml : sync latest (new ops, macros, refactoring) (#2106) f257fd255044decffad93dee2502875ce66ad80c 7ee76e45afae7f9a7a53e93393accfb5b36684e1 jwj7140 32943891+jwj7140@users.noreply.github.com 2023-07-05T03:06:12+09:00 GitHub noreply@github.com 2023-07-04T21:06:12+03:00 Add an API example using server.cpp similar to OAI. (#2009) 7ee76e45afae7f9a7a53e93393accfb5b36684e1 acc111caf93fc6681450924df9f99679c384c59e Tobias Lütke tobi@shopify.com 2023-07-04T10:05:27-04:00 GitHub noreply@github.com 2023-07-04T16:05:27+02:00 Simple webchat for server (#1998) acc111caf93fc6681450924df9f99679c384c59e 23c7c6fc9182b041f006b86ea1e7f99911ecf344 Henri Vasserman henv@hot.ee 2023-07-04T15:38:04+03:00 GitHub noreply@github.com 2023-07-04T15:38:04+03:00 Allow old Make to build server. (#2098) 23c7c6fc9182b041f006b86ea1e7f99911ecf344 698efad5fbbf326f01288649b123eff5f79b417e ZhouYuChen zhouyuchen@naver.com 2023-07-04T20:15:16+08:00 GitHub noreply@github.com 2023-07-04T14:15:16+02:00 Update Makefile: clean simple (#2097) 698efad5fbbf326f01288649b123eff5f79b417e 14a2cc71f62e45803ae70890ffbdeb0a172e6210 Erik Scholz Green-Sky@users.noreply.github.com 2023-07-04T01:50:12+02:00 GitHub noreply@github.com 2023-07-04T01:50:12+02:00 CI: make the brew update temporarily optional. (#2092) 14a2cc71f62e45803ae70890ffbdeb0a172e6210 1cf14ccef12e19c5a5b0b17ab456242d1f8c7fdd Govlzkoy gotope@users.noreply.github.com 2023-07-04T07:50:00+08:00 GitHub noreply@github.com 2023-07-04T07:50:00+08:00 [ggml] fix index for ne03 value in ggml_cl_mul_f32 (#2088) 1cf14ccef12e19c5a5b0b17ab456242d1f8c7fdd cc45a7feb8412e84ff292207621412fffc0d3d51 Henri Vasserman henv@hot.ee 2023-07-04T00:05:23+03:00 GitHub noreply@github.com 2023-07-04T00:05:23+03:00 fix server crashes (#2076) cc45a7feb8412e84ff292207621412fffc0d3d51 55dbb915cc2a95048f56e667b09dfad38d840421 Howard Su howard0su@gmail.com 2023-07-04T02:43:55+08:00 GitHub noreply@github.com 2023-07-03T20:43:55+02:00 Fix crash of test-tokenizer-0 under Debug build (#2064) 55dbb915cc2a95048f56e667b09dfad38d840421 d7d2e6a0f0c74f7a570dae384dfff371ac744d2a Howard Su howard0su@gmail.com 2023-07-03T19:58:58+08:00 GitHub noreply@github.com 2023-07-03T19:58:58+08:00 [llama] No need to check file version when loading vocab score (#2079) d7d2e6a0f0c74f7a570dae384dfff371ac744d2a 46088f72318981341a2d646f12f6eee6aec06d65 WangHaoranRobin 56047610+WangHaoranRobin@users.noreply.github.com 2023-07-03T05:38:44+08:00 GitHub noreply@github.com 2023-07-03T00:38:44+03:00 server: add option to output probabilities for completion (#1962) 46088f72318981341a2d646f12f6eee6aec06d65 0bc2cdfc875fa7877d8e01c8bb17066f99c08f21 Georgi Gerganov ggerganov@gmail.com 2023-07-02T09:46:46+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-02T09:46:46+03:00 ggml : fix build with OpenBLAS (close #2066) 0bc2cdfc875fa7877d8e01c8bb17066f99c08f21 befb3a35627432473f143c90994557d78ff5bc67 Johannes Gäßler johannesg@5d6.de 2023-07-01T21:49:44+02:00 GitHub noreply@github.com 2023-07-01T21:49:44+02:00 Better CUDA synchronization logic (#2057) befb3a35627432473f143c90994557d78ff5bc67 b2132270678c473f7cd9ba871b03d694126bc33a Johannes Gäßler johannesg@5d6.de 2023-07-01T21:47:26+02:00 GitHub noreply@github.com 2023-07-01T21:47:26+02:00 Test-based VRAM scratch size + context adjustment (#2056) b2132270678c473f7cd9ba871b03d694126bc33a 2f8cd979ecd1fa582852e7136e92ff8990b98fd8 Daniel Drake drake@endlessos.org 2023-07-01T20:31:44+02:00 GitHub noreply@github.com 2023-07-01T21:31:44+03:00 cmake : don't force -mcpu=native on aarch64 (#2063) 2f8cd979ecd1fa582852e7136e92ff8990b98fd8 471aab6e4cb89d8ef6d043f1bc93acb6eb78ab67 Aaron Miller apage43@ninjawhale.com 2023-07-01T11:14:59-07:00 GitHub noreply@github.com 2023-07-01T21:14:59+03:00 metal : release buffers when freeing metal context (#2062) 471aab6e4cb89d8ef6d043f1bc93acb6eb78ab67 463f2f4c4f8dd5ca9594b7d65849f346f0effe05 Judd foldl@users.noreply.github.com 2023-07-02T01:00:25+08:00 GitHub noreply@github.com 2023-07-01T20:00:25+03:00 convert : add support of baichuan-7b (#2055) 463f2f4c4f8dd5ca9594b7d65849f346f0effe05 cb44dbc7de287b3d17772cfb1aa49d55e082ce5b Georgi Gerganov ggerganov@gmail.com 2023-07-01T19:05:09+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-01T19:05:09+03:00 llama : fix return value of llama_load_session_file_internal (#2022) cb44dbc7de287b3d17772cfb1aa49d55e082ce5b 79f634a19d1c32a6cfb1befc21551ee684fced6b Rand Xie randxiexyy29@gmail.com 2023-07-02T00:02:58+08:00 GitHub noreply@github.com 2023-07-01T19:02:58+03:00 llama : catch llama_load_session_file_internal exceptions (#2022) 79f634a19d1c32a6cfb1befc21551ee684fced6b 04606a159947566b27810508433e6ca5dbc684ba Georgi Gerganov ggerganov@gmail.com 2023-07-01T18:46:00+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-01T18:46:00+03:00 embd-input : fix returning ptr to temporary 04606a159947566b27810508433e6ca5dbc684ba b1ca8f36a9cdbcee5f5c425df717611a1040a897 Georgi Gerganov ggerganov@gmail.com 2023-07-01T18:45:44+03:00 Georgi Gerganov ggerganov@gmail.com 2023-07-01T18:45:44+03:00 train : fix compile warning b1ca8f36a9cdbcee5f5c425df717611a1040a897 b8c8dda75fdf5fdea49c80af36818e7c30fe0ddf Qingyou Meng meng.qingyou@gmail.com 2023-07-01T23:42:43+08:00 GitHub noreply@github.com 2023-07-01T18:42:43+03:00 ggml : disable GGML_TASK_INIT and GGML_TASK_FINALIZE by default (#1995) b8c8dda75fdf5fdea49c80af36818e7c30fe0ddf 96a712ca1b7f427e3bd7ffc0c70b2105cfc7fbf1 Howard Su howard0su@gmail.com 2023-06-29T21:15:15+08:00 GitHub noreply@github.com 2023-06-29T06:15:15-07:00 Use unsigned for random seed (#2006) 96a712ca1b7f427e3bd7ffc0c70b2105cfc7fbf1 d3494bb86bf7ad5b0b60aae0220ea576f273b5c0 LostRuins 39025047+LostRuins@users.noreply.github.com 2023-06-29T11:56:43+08:00 GitHub noreply@github.com 2023-06-29T05:56:43+02:00 Porting the improved K-Quant CUDA kernels to OpenCL (#1966) d3494bb86bf7ad5b0b60aae0220ea576f273b5c0 5b351e94d041742cd50ffcf2d44718d63bab398a m3ndax adrian.goessl@outlook.com 2023-06-28T20:39:08+02:00 GitHub noreply@github.com 2023-06-28T21:39:08+03:00 llama : replacing auto &kv with const auto &kv (#2041) 5b351e94d041742cd50ffcf2d44718d63bab398a 6432aabb6dc887436e4d57414b63116189c3b13b Salvador E. Tropea stropea@inti.gob.ar 2023-06-28T14:27:31-03:00 GitHub noreply@github.com 2023-06-28T20:27:31+03:00 cuda : remove nchannels_x argument from mul_mat_vec_nc_f16_f32 (#2028) 6432aabb6dc887436e4d57414b63116189c3b13b b922bc351b69770cec2d35d2aa50fa052b95ca93 Salvador E. Tropea stropea@inti.gob.ar 2023-06-28T14:26:26-03:00 GitHub noreply@github.com 2023-06-28T20:26:26+03:00 cuda : fix missing const qualifier in casts (#2027) b922bc351b69770cec2d35d2aa50fa052b95ca93 7f9753fa1263c4eded9a3de19778562f0e1093d7 Howard Su howard0su@gmail.com 2023-06-28T10:13:02-07:00 GitHub noreply@github.com 2023-06-28T20:13:02+03:00 llama : remove shards weight file support (#2000) 7f9753fa1263c4eded9a3de19778562f0e1093d7 cfa0750bc9dbc2d957a91b8ed09ab0035d8f3d4e Johannes Gäßler johannesg@5d6.de 2023-06-28T18:35:54+02:00 GitHub noreply@github.com 2023-06-28T18:35:54+02:00 CUDA GPU acceleration for LoRAs + f16 models (#1970) cfa0750bc9dbc2d957a91b8ed09ab0035d8f3d4e 9d23589d638dc74577d5ff880e6d4248b795f12e ningshanwutuobang ningshanwutuobang@gmail.com 2023-06-28T23:53:37+08:00 GitHub noreply@github.com 2023-06-28T18:53:37+03:00 llama : support input embeddings directly (#1910) 9d23589d638dc74577d5ff880e6d4248b795f12e 0be54f75a6c3e9a09ea71bdfcdabf9a996a0549b Erik Scholz Green-Sky@users.noreply.github.com 2023-06-27T19:06:33+02:00 GitHub noreply@github.com 2023-06-27T19:06:33+02:00 fix pthreads setaffinity usage on android (#2020) 0be54f75a6c3e9a09ea71bdfcdabf9a996a0549b 181e8d975528a4e27eabb8ae6e9865f9ceae4b37 Howard Su howard0su@gmail.com 2023-06-27T13:07:13+08:00 GitHub noreply@github.com 2023-06-27T08:07:13+03:00 baby-llama : fix build after ggml_rope change (#2016) 181e8d975528a4e27eabb8ae6e9865f9ceae4b37 d9779021bd59ed96daae75e820a5ac5da47ca8ff Georgi Gerganov ggerganov@gmail.com 2023-06-27T00:37:13+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-27T00:37:33+03:00 llama : fix rope usage after ChatGLM change d9779021bd59ed96daae75e820a5ac5da47ca8ff d38e45157862b58a1824387e64860d68ca3533a7 Georgi Gerganov ggerganov@gmail.com 2023-06-27T00:06:51+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-27T00:06:51+03:00 ggml : add support for ChatGLM RoPE d38e45157862b58a1824387e64860d68ca3533a7 eaa6ca5a61b8c9501df9ebe3d264f45b75a5f8aa Roman Parykin donderom@gmail.com 2023-06-26T22:47:59+03:00 GitHub noreply@github.com 2023-06-26T22:47:59+03:00 readme : add Scala 3 bindings repo (#2010) eaa6ca5a61b8c9501df9ebe3d264f45b75a5f8aa aa777abbb73655c4e1e9237b7c0ad66745e8e48c David Yang davidyang6us@gmail.com 2023-06-27T03:45:32+08:00 GitHub noreply@github.com 2023-06-26T22:45:32+03:00 ggml : increase max tensor name + clean up compiler warnings in train-text (#1988) aa777abbb73655c4e1e9237b7c0ad66745e8e48c c824d2e368d193d9f564ff29880a51cda9f90527 Gustavo Rocha Dias 91472747+gustrd@users.noreply.github.com 2023-06-26T16:34:45-03:00 GitHub noreply@github.com 2023-06-26T22:34:45+03:00 readme : LD_LIBRARY_PATH complement for some Android devices when building with CLBlast inside Termux (#2007) c824d2e368d193d9f564ff29880a51cda9f90527 b853d456018b10820686362af41b2f2f75f1eec6 Georgi Gerganov ggerganov@gmail.com 2023-06-26T21:03:59+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-26T21:03:59+03:00 ggml : avoid conv 2d kernel round up b853d456018b10820686362af41b2f2f75f1eec6 9225baef71407d799a6f7f563b77fd7f82791416 zrm trustiosity.zrm@gmail.com 2023-06-26T13:57:59-04:00 GitHub noreply@github.com 2023-06-26T20:57:59+03:00 ggml : add NUMA support (#1556) 9225baef71407d799a6f7f563b77fd7f82791416 a84ab1da8dc6a59a5b67420ae1322f09503ffc72 Georgi Gerganov ggerganov@gmail.com 2023-06-26T20:10:52+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-26T20:10:52+03:00 k-quants : fix indentation a84ab1da8dc6a59a5b67420ae1322f09503ffc72 5743ca80928d8410754ec64a5673d5c2dd6cfbb7 katsu560 118887472+katsu560@users.noreply.github.com 2023-06-27T01:47:02+09:00 GitHub noreply@github.com 2023-06-26T19:47:02+03:00 tests : fix quantize perf (#1990) 5743ca80928d8410754ec64a5673d5c2dd6cfbb7 412c60e4739367144e51e59add5dc7749d084115 katsu560 118887472+katsu560@users.noreply.github.com 2023-06-27T01:46:07+09:00 GitHub noreply@github.com 2023-06-26T19:46:07+03:00 k-quants : add AVX support to dot functions (#1916) 412c60e4739367144e51e59add5dc7749d084115 6769e944c727c63612dcafbef52009d21ae00fff Georgi Gerganov ggerganov@gmail.com 2023-06-26T19:45:09+03:00 GitHub noreply@github.com 2023-06-26T19:45:09+03:00 readme : add link to new k-quants for visibility 6769e944c727c63612dcafbef52009d21ae00fff cbebf61ca7584e9709265395f0127ae7fc0f1882 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-26T19:43:07+03:00 GitHub noreply@github.com 2023-06-26T19:43:07+03:00 k-quants : support for super-block size of 64 (#2001) cbebf61ca7584e9709265395f0127ae7fc0f1882 447ccbe8c39332fcdd0d98a041b6e2ff6f06219d Howard Su howard0su@gmail.com 2023-06-26T23:15:47+08:00 GitHub noreply@github.com 2023-06-26T23:15:47+08:00 Fix assert when free invalid cuda pointer (#2005) 447ccbe8c39332fcdd0d98a041b6e2ff6f06219d bd34cdde38f8fd661890ddd5f57ca30bf279877b Georgi Gerganov ggerganov@gmail.com 2023-06-25T16:08:12+03:00 GitHub noreply@github.com 2023-06-25T16:08:12+03:00 readme : add new roadmap + manifesto bd34cdde38f8fd661890ddd5f57ca30bf279877b c2a08f87b8d180115d04b8688f383d1b2761b16d Georgi Gerganov ggerganov@gmail.com 2023-06-25T14:25:08+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-25T14:25:08+03:00 ggml : sync latest ggml (custom operators) c2a08f87b8d180115d04b8688f383d1b2761b16d 66a2555ba6cab954c56d653b29c27bfbbacfbfb1 anon998 131767832+anon998@users.noreply.github.com 2023-06-25T08:48:36Z GitHub noreply@github.com 2023-06-25T10:48:36+02:00 fix server sampling: top k sampler first (#1977) 66a2555ba6cab954c56d653b29c27bfbbacfbfb1 e65ca7e14ac76c4046091da39d41a9017abaa9b3 Georgi Gerganov ggerganov@gmail.com 2023-06-25T09:07:03+03:00 GitHub noreply@github.com 2023-06-25T09:07:03+03:00 readme : add Azure CI discussion link e65ca7e14ac76c4046091da39d41a9017abaa9b3 5ec8dd5a3c6a9a109351d2257bb9d53869bd0a94 sjinzh sjinzh@gmail.com 2023-06-25T13:45:44+08:00 GitHub noreply@github.com 2023-06-25T08:45:44+03:00 zig : upgrade build system support (#1981) 5ec8dd5a3c6a9a109351d2257bb9d53869bd0a94 65bdd52a867539691007f85c5508146d507f72c1 Robyn robyngraf@users.noreply.github.com 2023-06-25T04:10:29+10:00 GitHub noreply@github.com 2023-06-24T20:10:29+02:00 #1869 Fix null reference errors when training from scratch with CUDA (#1907) 65bdd52a867539691007f85c5508146d507f72c1 fdd18609113862dc6eb34dfc44a093d54c59ff1f Georgi Gerganov ggerganov@gmail.com 2023-06-24T19:40:18+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-24T19:40:18+03:00 tests : sync test-grad0 from ggml fdd18609113862dc6eb34dfc44a093d54c59ff1f c943d823c14cef33092205ca3944de6fdf7abf99 Rowan Hart rowanbhart@gmail.com 2023-06-24T04:07:08-07:00 GitHub noreply@github.com 2023-06-24T14:07:08+03:00 flake : fix ggml-metal.metal path and run nixfmt (#1974) c943d823c14cef33092205ca3944de6fdf7abf99 f2c754e1c38936fdde74e4848ac468a696eb73c6 AN Long aisk@users.noreply.github.com 2023-06-24T19:02:06+08:00 GitHub noreply@github.com 2023-06-24T14:02:06+03:00 convert : fix invalid params in write_vocab_only (#1975) f2c754e1c38936fdde74e4848ac468a696eb73c6 11da1a85cd69af84b5861134738c7e9e20907470 slaren slarengh@gmail.com 2023-06-24T12:57:18+02:00 GitHub noreply@github.com 2023-06-24T13:57:18+03:00 ggml : improve ggml_graph_dump_dot, add ggml_format_name (#1978) 11da1a85cd69af84b5861134738c7e9e20907470 235b610d650cbfed6dbd5d671f750d35fc18cd7d Georgi Gerganov ggerganov@gmail.com 2023-06-24T13:38:18+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-24T13:38:18+03:00 readme : fix whitespaces 235b610d650cbfed6dbd5d671f750d35fc18cd7d b061ba9e2a7a2c335a200df8c11aed5e31e4ccbb Alberto 57916483+albbus-stack@users.noreply.github.com 2023-06-24T12:32:13+02:00 GitHub noreply@github.com 2023-06-24T13:32:13+03:00 readme : fixed termux instructions (#1973) b061ba9e2a7a2c335a200df8c11aed5e31e4ccbb 527b6fba1d237befb324fd846bda7418c0fa394d Alex Renda alexrenda@users.noreply.github.com 2023-06-24T03:15:01-07:00 GitHub noreply@github.com 2023-06-24T13:15:01+03:00 llama : fix top-p sampling to match the canonical definition (#1953) 527b6fba1d237befb324fd846bda7418c0fa394d d7b7484f74d486f77feb4c0b7af7e1718ed91651 Didzis Gosko didzis@users.noreply.github.com 2023-06-24T11:47:58+03:00 GitHub noreply@github.com 2023-06-24T11:47:58+03:00 llama : make model stateless and context stateful (llama_state) (#1797) d7b7484f74d486f77feb4c0b7af7e1718ed91651 7487137227eb32ed9b12156338b865cb29b2dfd1 eiery 19350831+eiery@users.noreply.github.com 2023-06-23T04:38:01-04:00 GitHub noreply@github.com 2023-06-23T10:38:01+02:00 Add OpenLLaMA instructions to the README (#1954) 7487137227eb32ed9b12156338b865cb29b2dfd1 bbca06e26949686d61a5126332680ba3cccf235c Erik Scholz Green-Sky@users.noreply.github.com 2023-06-22T14:20:47+02:00 GitHub noreply@github.com 2023-06-22T14:20:47+02:00 rework convert.py to read hyper-parameters from config.json (#1958) bbca06e26949686d61a5126332680ba3cccf235c fb98254f99d769fcbbf20966ef386abdb48ef601 Johannes Gäßler johannesg@5d6.de 2023-06-21T23:49:25+02:00 GitHub noreply@github.com 2023-06-21T23:49:25+02:00 cmake: revert CUDA arch default to 52, 61 if f16 (#1959) fb98254f99d769fcbbf20966ef386abdb48ef601 049aa16b8c5c6d086246e4e6b9feb18de4fbd663 Rahul Vivek Nair 68507071+RahulVivekNair@users.noreply.github.com 2023-06-22T03:18:43+05:30 GitHub noreply@github.com 2023-06-21T23:48:43+02:00 Fix typo in README.md (#1961) 049aa16b8c5c6d086246e4e6b9feb18de4fbd663 2322ec223a21625dfe9bd73ee677444a98a24ac9 Georgi Gerganov ggerganov@gmail.com 2023-06-20T19:05:54+03:00 GitHub noreply@github.com 2023-06-20T19:05:54+03:00 readme : add link to p1 2322ec223a21625dfe9bd73ee677444a98a24ac9 aacdbd40562684665b6f7b8ba6695b7a2088bbb0 Xiake Sun xiake.sun@intel.com 2023-06-20T05:42:40-07:00 GitHub noreply@github.com 2023-06-20T15:42:40+03:00 Fix typo (#1949) aacdbd40562684665b6f7b8ba6695b7a2088bbb0 20568fe60f00155fa25e92eb3a7f6b911d557967 Ettore Di Giacinto mudler@users.noreply.github.com 2023-06-20T03:24:39+02:00 GitHub noreply@github.com 2023-06-20T04:24:39+03:00 llama : fix params struct slignment (#1936) 20568fe60f00155fa25e92eb3a7f6b911d557967 18b35625c3c19c64b7818a12460ba5ddb006dfdc Henri Vasserman henv@hot.ee 2023-06-20T01:12:39+03:00 GitHub noreply@github.com 2023-06-20T01:12:39+03:00 [Fix] Reenable server embedding endpoint (#1937) 18b35625c3c19c64b7818a12460ba5ddb006dfdc ba4e85a8339b9dd7cdffad31838235f2fe45a8ea Georgi Gerganov ggerganov@gmail.com 2023-06-19T20:43:30+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-19T20:43:30+03:00 ggml : fix bug in LBFGS optimizer (found by ggml tests) ba4e85a8339b9dd7cdffad31838235f2fe45a8ea 23fc5c219a9aebd57c8af3fac454062cc4622980 l3utterfly gc.pthzfoldr@gmail.com 2023-06-19T23:20:06+08:00 GitHub noreply@github.com 2023-06-19T18:20:06+03:00 llama : use aligned memory during ggml_init call from loading saved sessions (#1934) 23fc5c219a9aebd57c8af3fac454062cc4622980 cb40dfca694b5cb849837548fd69932117c78362 Georgi Gerganov ggerganov@gmail.com 2023-06-19T18:18:34+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-19T18:18:34+03:00 cmake : fix trailing whitespaces cb40dfca694b5cb849837548fd69932117c78362 ca7c3f4da5d144d4cd1dd44903552e6ba49b8ec8 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-19T18:17:03+03:00 GitHub noreply@github.com 2023-06-19T18:17:03+03:00 llama : only use Q6_K for output weights if tensor size is multiple of 256 (#1932) ca7c3f4da5d144d4cd1dd44903552e6ba49b8ec8 b97ca431db35ec96a339a721acb1219c1dd78bed Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-19T18:14:09+03:00 GitHub noreply@github.com 2023-06-19T18:14:09+03:00 cuda : faster k-quants on older GPUs (#1930) b97ca431db35ec96a339a721acb1219c1dd78bed 1e3abfcef073e73c2b31e8570cb06c5cb2fd1f55 Georgi Gerganov ggerganov@gmail.com 2023-06-19T18:12:33+03:00 GitHub noreply@github.com 2023-06-19T18:12:33+03:00 ggml : sync latest ggml repo (#1924) 1e3abfcef073e73c2b31e8570cb06c5cb2fd1f55 16b9cd193965769089881bb8ec012fccca7b37b6 Howard Su howard0su@gmail.com 2023-06-19T23:10:37+08:00 GitHub noreply@github.com 2023-06-19T18:10:37+03:00 cmake : fix build shared ggml when CUDA is enabled (#1929) 16b9cd193965769089881bb8ec012fccca7b37b6 b24c3049d96557c24782e4d32feaae65f47277af Johannes Gäßler johannesg@5d6.de 2023-06-19T10:23:56+02:00 GitHub noreply@github.com 2023-06-19T10:23:56+02:00 Convert vector to f16 for dequantize mul mat vec (#1913) b24c3049d96557c24782e4d32feaae65f47277af 0ede372a51fd8160688e01b587582666c14e94e5 Johannes Gäßler johannesg@5d6.de 2023-06-18T17:41:26+02:00 GitHub noreply@github.com 2023-06-18T17:41:26+02:00 Added tokens per second to info prints (#1928) 0ede372a51fd8160688e01b587582666c14e94e5 8596af427722775f0df4a7c90b9af067ba90d4ef Johannes Gäßler johannesg@5d6.de 2023-06-18T16:07:09+02:00 GitHub noreply@github.com 2023-06-18T16:07:09+02:00 Fixed incorrectly applying RMS norm twice (#1925) 8596af427722775f0df4a7c90b9af067ba90d4ef e1886cf4fe0d0f31661dda52a4a9f34bd9b9009a l3utterfly gc.pthzfoldr@gmail.com 2023-06-18T19:19:16+08:00 GitHub noreply@github.com 2023-06-18T14:19:16+03:00 ggml : fix bug in ggml_compute_forward_add_q_f32 (#1918) e1886cf4fe0d0f31661dda52a4a9f34bd9b9009a 8ab8ba62eb27cc340be2edf3418e051b1d967416 Mike ytianhui2004@gmail.com 2023-06-18T16:28:26+08:00 GitHub noreply@github.com 2023-06-18T11:28:26+03:00 readme : update Android build instructions (#1922) 8ab8ba62eb27cc340be2edf3418e051b1d967416 90cc59d6ab1363a5c69c60c4b94db647d3a54a18 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-18T11:13:43+03:00 GitHub noreply@github.com 2023-06-18T11:13:43+03:00 llama : prevent usage of k-quants when tensor size is not a multiple of 256 (#1921) 90cc59d6ab1363a5c69c60c4b94db647d3a54a18 ce2c7d72e2d06988b5ddec6811ab923254542077 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-18T10:52:10+03:00 GitHub noreply@github.com 2023-06-18T10:52:10+03:00 examples : fix examples/metal (#1920) ce2c7d72e2d06988b5ddec6811ab923254542077 57cd69460f736031a3fc54af1e97c03f80128478 Georgi Gerganov ggerganov@gmail.com 2023-06-18T09:09:47+03:00 GitHub noreply@github.com 2023-06-18T09:09:47+03:00 metal : handle buffers larger than device's maxBufferLength (#1826) 57cd69460f736031a3fc54af1e97c03f80128478 b2416493ab3ab21686d47c96669da6d6c6af08a4 Howard Su howard0su@gmail.com 2023-06-18T12:29:47+08:00 GitHub noreply@github.com 2023-06-18T07:29:47+03:00 cmake : add CUDA_ARCHITECTURES to new target ggml_static (#1917) b2416493ab3ab21686d47c96669da6d6c6af08a4 4f9c43e3bd488b7561119785485e1155dba338d7 Georgi Gerganov ggerganov@gmail.com 2023-06-17T20:55:03+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-17T20:55:03+03:00 make : do not print help for simple example 4f9c43e3bd488b7561119785485e1155dba338d7 2c9380dd2f77e41149340f3ecb09764d793b16db Georgi Gerganov ggerganov@gmail.com 2023-06-17T20:24:11+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-17T20:24:11+03:00 minor : warning fixes 2c9380dd2f77e41149340f3ecb09764d793b16db 051e1b0e6a6e3aee7d989b47760980e6fda5861c Johannes Gäßler johannesg@5d6.de 2023-06-17T19:15:02+02:00 GitHub noreply@github.com 2023-06-17T19:15:02+02:00 Only one CUDA stream per device for async compute (#1898) 051e1b0e6a6e3aee7d989b47760980e6fda5861c 86c7571864ff331f8cdb9e092f3abeb123729a56 Georgi Gerganov ggerganov@gmail.com 2023-06-17T19:30:22+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-17T19:31:20+03:00 llama : fix kv_cache `n` init (close #1903) 86c7571864ff331f8cdb9e092f3abeb123729a56 3d59ec5935ea1d33e9d51060a8dd737169b9b89b DaniAndTheWeb 57776841+DaniAndTheWeb@users.noreply.github.com 2023-06-17T18:17:22+02:00 GitHub noreply@github.com 2023-06-17T19:17:22+03:00 make : update for latest Arch (#1701) 3d59ec5935ea1d33e9d51060a8dd737169b9b89b 0711a5f6dce7f04c2a791b14bc47f7d4cb545408 Howard Su howard0su@gmail.com 2023-06-17T23:46:15+08:00 GitHub noreply@github.com 2023-06-17T18:46:15+03:00 ggml : fix warnings under MSVC (#1908) 0711a5f6dce7f04c2a791b14bc47f7d4cb545408 fc45a81bc642b9ef33d9004f2b363d558438a6c9 Aaron Miller apage43@ninjawhale.com 2023-06-17T07:37:49-07:00 GitHub noreply@github.com 2023-06-17T17:37:49+03:00 metal : add norm, cpy f16->f16, alibi kernels (#1823) fc45a81bc642b9ef33d9004f2b363d558438a6c9 794db3e7b982fee37e3995db9c3a216a57ff65e3 Faez Shakil faez.shakil@gmail.com 2023-06-17T17:13:05+05:00 GitHub noreply@github.com 2023-06-17T14:13:05+02:00 exposed modules so that they can be invoked by nix run github:ggerganov/llama.cpp#server etc (#1863) 794db3e7b982fee37e3995db9c3a216a57ff65e3 5ddf7ea1fb42bac21026de2f77e0f9c069b92234 Randall Fitzgerald randall@dasaku.net 2023-06-17T07:53:04-04:00 GitHub noreply@github.com 2023-06-17T14:53:04+03:00 Server Example Refactor and Improvements (#1570) 5ddf7ea1fb42bac21026de2f77e0f9c069b92234 bac19927c302737465a1deb14ac0943a221863e8 Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2023-06-17T12:32:48+02:00 GitHub noreply@github.com 2023-06-17T13:32:48+03:00 hooks : setting up flake8 and pre-commit hooks (#1681) bac19927c302737465a1deb14ac0943a221863e8 b4c6f46f17b6e02f1cd55a81339e7e64f3aaa688 Gustavo Rocha Dias 91472747+gustrd@users.noreply.github.com 2023-06-17T06:01:06-03:00 GitHub noreply@github.com 2023-06-17T12:01:06+03:00 readme : alternative way to build for Android with CLBlast. (#1828) b4c6f46f17b6e02f1cd55a81339e7e64f3aaa688 92f20d9942c86daeb78637bdad7296a572f4da28 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-06-17T01:49:42-06:00 GitHub noreply@github.com 2023-06-17T01:49:42-06:00 Allow cmake to build ggml as a library (#1896) 92f20d9942c86daeb78637bdad7296a572f4da28 d411968e990c37f51328849c96a743dd78f3c3dd David Yang davidyang6us@gmail.com 2023-06-17T14:51:54+08:00 GitHub noreply@github.com 2023-06-17T09:51:54+03:00 train : get raw text instead of page with html (#1905) d411968e990c37f51328849c96a743dd78f3c3dd b41b4cad6f956b5f501db0711dd7007c32b5eee5 0cc4m picard12@live.de 2023-06-16T20:59:49+02:00 GitHub noreply@github.com 2023-06-16T21:59:49+03:00 opencl : support k-quants (#1836) b41b4cad6f956b5f501db0711dd7007c32b5eee5 13fe9d2d84f30cab613c960bf66ac83916006694 SuperUserNameMan yoann@terminajones.com 2023-06-16T20:58:09+02:00 GitHub noreply@github.com 2023-06-16T21:58:09+03:00 examples : add "simple" (#1840) 13fe9d2d84f30cab613c960bf66ac83916006694 ac3b8869538c7fbdb48ff141d78c4dea091789f0 Zenix zenixls2@gmail.com 2023-06-17T03:53:04+09:00 GitHub noreply@github.com 2023-06-16T21:53:04+03:00 cmake : add auto detection of BLAS_INCLUDE_DIRS (#1886) ac3b8869538c7fbdb48ff141d78c4dea091789f0 5b9ccaf104cc1054d4f8f17bc8a4b8dc949e5527 Johannes Gäßler johannesg@5d6.de 2023-06-16T20:25:51+02:00 GitHub noreply@github.com 2023-06-16T21:25:51+03:00 llama : fix embd when offloading non-repeating layers (#1891) 5b9ccaf104cc1054d4f8f17bc8a4b8dc949e5527 9cbf50c041a525d781c7764f493a5443924e4e38 FrankHB frankhb1989@gmail.com 2023-06-17T02:25:01+08:00 GitHub noreply@github.com 2023-06-16T21:25:01+03:00 Fixed possible macro redefinition (#1892) 9cbf50c041a525d781c7764f493a5443924e4e38 3d0112261042b356621e93db3fa4c6798a5d098f Borislav Stanimirov b.stanimirov@abv.bg 2023-06-16T21:23:53+03:00 GitHub noreply@github.com 2023-06-16T21:23:53+03:00 build : fix and ignore MSVC warnings (#1889) 3d0112261042b356621e93db3fa4c6798a5d098f 602c748863e15270d80d74aa2c3bf86ab8139e07 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-16T20:08:44+03:00 GitHub noreply@github.com 2023-06-16T20:08:44+03:00 CUDA : faster k-quant dot kernels (#1862) 602c748863e15270d80d74aa2c3bf86ab8139e07 a09f9195be39afb4b023b646c0a6ec8a86915174 Borislav Stanimirov b.stanimirov@abv.bg 2023-06-16T09:58:11+03:00 GitHub noreply@github.com 2023-06-16T09:58:11+03:00 gitignore : add several entries specific to Visual Studio (#1888) a09f9195be39afb4b023b646c0a6ec8a86915174 bed92756172d4514b23aaf9744cf8e2dc892fc7b Johannes Gäßler johannesg@5d6.de 2023-06-15T21:49:08+02:00 GitHub noreply@github.com 2023-06-15T21:49:08+02:00 Fixed CUDA runtime version check (#1879) bed92756172d4514b23aaf9744cf8e2dc892fc7b c36e81da62ebfe09a768201cc44fa8d712dd00ed Georgi Gerganov ggerganov@gmail.com 2023-06-15T21:56:50+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-15T21:56:50+03:00 cmake : remove whitespaces c36e81da62ebfe09a768201cc44fa8d712dd00ed 3559433fecedf365e7aba2fe3d5f89d9abb817c1 yangli2 yangli2@gmail.com 2023-06-15T11:05:53-07:00 GitHub noreply@github.com 2023-06-15T21:05:53+03:00 examples : add chat-vicuna.sh (#1854) 3559433fecedf365e7aba2fe3d5f89d9abb817c1 69b34a0e80300bfb3e996983ac3ea075f5526675 Igor Okulist okigan@gmail.com 2023-06-15T12:51:26-05:00 GitHub noreply@github.com 2023-06-15T20:51:26+03:00 cmake : set include path for OpenBlas (#1830) 69b34a0e80300bfb3e996983ac3ea075f5526675 cf267d1c71a781700698f8518e903239c3bcc929 Frederik Vogel Schaltfehler@users.noreply.github.com 2023-06-16T02:47:04+09:00 GitHub noreply@github.com 2023-06-15T20:47:04+03:00 swift : Package compile breaks due to ggml-metal.metal (#1831) cf267d1c71a781700698f8518e903239c3bcc929 9dda13e5e1f70bdfc25fbc0f0378f27c8b67e983 daboe01 daboe01@googlemail.com 2023-06-15T19:42:48+02:00 GitHub noreply@github.com 2023-06-15T20:42:48+03:00 make : add train-text-from-scratch (#1850) 9dda13e5e1f70bdfc25fbc0f0378f27c8b67e983 37e257c48e350cf03c353c10d31e777f8d00123d Srinivas Billa nivibilla@gmail.com 2023-06-15T18:36:38+01:00 GitHub noreply@github.com 2023-06-15T20:36:38+03:00 readme : server compile flag (#1874) 37e257c48e350cf03c353c10d31e777f8d00123d 64cc19b4fe3df03bc20e520aa111c30cff3a655e sandyiscool sandyiscool@gmail.com 2023-06-15T23:06:06+05:30 GitHub noreply@github.com 2023-06-15T20:36:06+03:00 make : clean *.so files (#1857) 64cc19b4fe3df03bc20e520aa111c30cff3a655e 4bfcc855abdb2c9fcc3c5a84747974521909fa41 Howard Su howard0su@gmail.com 2023-06-16T01:29:59+08:00 GitHub noreply@github.com 2023-06-15T19:29:59+02:00 Fix the validation of main device (#1872) 4bfcc855abdb2c9fcc3c5a84747974521909fa41 6b8312e7979b852f6b6ac9d29cd51fda16c17948 Georgi Gerganov ggerganov@gmail.com 2023-06-15T20:29:48+03:00 GitHub noreply@github.com 2023-06-15T20:29:48+03:00 metal : parallel command buffer encoding (#1860) 6b8312e7979b852f6b6ac9d29cd51fda16c17948 254a7a7a5ff4c874ff8488f1f5cbdd7e9c89d682 Johannes Gäßler johannesg@5d6.de 2023-06-15T19:06:46+02:00 GitHub noreply@github.com 2023-06-15T19:06:46+02:00 Better error when using both LoRA + GPU layers (#1861) 254a7a7a5ff4c874ff8488f1f5cbdd7e9c89d682 92549202659fc23ba9fec5e688227d0da9b06b40 Johannes Gäßler johannesg@5d6.de 2023-06-14T19:47:19+02:00 GitHub noreply@github.com 2023-06-14T19:47:19+02:00 CUDA full GPU acceleration, KV cache in VRAM (#1827) 92549202659fc23ba9fec5e688227d0da9b06b40 e32089b2c20b1b87b22912f4a8b93fe01647d5b9 0xspringtime 110655352+0xspringtime@users.noreply.github.com 2023-06-13T15:37:54-04:00 GitHub noreply@github.com 2023-06-13T22:37:54+03:00 baby-llama : fix operator!= (#1821) e32089b2c20b1b87b22912f4a8b93fe01647d5b9 2347e45e7bdb09c9a7d74b2c0bc86c2b65f0c343 xaedes xaedes@gmail.com 2023-06-13T21:04:40+02:00 GitHub noreply@github.com 2023-06-13T22:04:40+03:00 train : improved training-from-scratch example (#1652) 2347e45e7bdb09c9a7d74b2c0bc86c2b65f0c343 74d4cfa3438cb58bd177eed30014e6588694aaa8 Georgi Gerganov ggerganov@gmail.com 2023-06-13T20:20:07+03:00 GitHub noreply@github.com 2023-06-13T20:20:07+03:00 llama : do a warm-up eval at start for better timings (#1824) 74d4cfa3438cb58bd177eed30014e6588694aaa8 74a6d922f12ccfe16b0c265f43be8978c6f25e98 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-06-13T04:23:23-06:00 GitHub noreply@github.com 2023-06-13T04:23:23-06:00 Allow "quantizing" to f16 and f32 (#1787) 74a6d922f12ccfe16b0c265f43be8978c6f25e98 e4caa8da59c1c97dc23fa336f4d726984a20560f Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-12T22:39:21+03:00 GitHub noreply@github.com 2023-06-12T22:39:21+03:00 Metal implementation for all k_quants (#1807) e4caa8da59c1c97dc23fa336f4d726984a20560f 58970a4c39124a647ac2a640d9e178ea6c961e65 slaren slarengh@gmail.com 2023-06-12T19:12:47+02:00 GitHub noreply@github.com 2023-06-12T20:12:47+03:00 ci : run when changing only the CUDA sources (#1800) 58970a4c39124a647ac2a640d9e178ea6c961e65 8c0a10e64dbf60fd9946c0cd5e6f59690800b123 Howard Su howard0su@gmail.com 2023-06-12T20:44:16+08:00 GitHub noreply@github.com 2023-06-12T14:44:16+02:00 Leverage mmap for offloading tensors to GPU (#1597) 8c0a10e64dbf60fd9946c0cd5e6f59690800b123 fa84c4b3e80199a5683438f062009c031a06c4fa Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-12T14:31:36+03:00 GitHub noreply@github.com 2023-06-12T14:31:36+03:00 metal : fix failure to load model (#1817) fa84c4b3e80199a5683438f062009c031a06c4fa 12b063f0ecf280e98028e444fc492ee6222cdcdc Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-06-11T08:19:17-06:00 GitHub noreply@github.com 2023-06-11T08:19:17-06:00 Fix issue where interactive mode crashes when input exceeds ctx size (#1789) 12b063f0ecf280e98028e444fc492ee6222cdcdc 31d2b5f4a4bae081e59b36ab37c6ff6f5b5940ad Kyle Liang liangmanlai@gmail.com 2023-06-11T21:20:52+08:00 GitHub noreply@github.com 2023-06-11T15:20:52+02:00 Fixed WSL cuda's OOM error (#1594) 31d2b5f4a4bae081e59b36ab37c6ff6f5b5940ad 4de0334f5cabf4696eced2e5d6e279fdfaa6c0f2 Ryan Landay rlanday@gmail.com 2023-06-11T17:38:53+08:00 GitHub noreply@github.com 2023-06-11T12:38:53+03:00 Update SHA256SUMS with current hashes for models quantized using q4_0 (#1798) 4de0334f5cabf4696eced2e5d6e279fdfaa6c0f2 3f1223155a462477ac933474ebc4eab0ce3ca264 Georgi Gerganov ggerganov@gmail.com 2023-06-10T22:56:53+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-10T22:56:53+03:00 cmake : fix Metal build (close #1791) 3f1223155a462477ac933474ebc4eab0ce3ca264 303f5809f1b4ec49823dbe70cacd2124ec1d0df0 Artyom Lebedev vagran.ast@gmail.com 2023-06-10T22:51:36+03:00 GitHub noreply@github.com 2023-06-10T22:51:36+03:00 k-quants : GCC12 compilation fix (#1792) 303f5809f1b4ec49823dbe70cacd2124ec1d0df0 059e99066d95d73d1ca26c3375d47c0e35596229 Andrei abetlen@gmail.com 2023-06-10T10:47:34-04:00 GitHub noreply@github.com 2023-06-10T17:47:34+03:00 metal : fix issue with ggml-metal.metal path. Closes #1769 (#1782) 059e99066d95d73d1ca26c3375d47c0e35596229 17c10acfb44ecb7af25e37fb67b9501cbc0034d2 Aisuko urakiny@gmail.com 2023-06-11T00:08:11+10:00 GitHub noreply@github.com 2023-06-10T17:08:11+03:00 doc : fix wrong address of BLIS.md (#1772) 17c10acfb44ecb7af25e37fb67b9501cbc0034d2 e9b66ee9829039d4ab54550d6222e42a0b31e52a Georgi Gerganov ggerganov@gmail.com 2023-06-10T12:06:45+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-10T12:08:15+03:00 ggml : force no_alloc == false when creating opt tensors (close #1699) e9b66ee9829039d4ab54550d6222e42a0b31e52a 4f0154b0bad775ac4651bf73b5c216eb43c45cdc Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-10T11:28:11+03:00 GitHub noreply@github.com 2023-06-10T11:28:11+03:00 metal : add Q4_1 implementation (#1785) 4f0154b0bad775ac4651bf73b5c216eb43c45cdc ef3171d16241c18581d4d08374f0b9e396ade6b7 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-06-10T01:59:17-06:00 GitHub noreply@github.com 2023-06-10T10:59:17+03:00 llama : support requantizing models instead of only allowing quantization from 16/32bit (#1691) ef3171d16241c18581d4d08374f0b9e396ade6b7 555275a693843273759230547001f9ae07fb537e Xingchen Song(宋星辰) xingchensong1996@163.com 2023-06-10T15:49:40+08:00 GitHub noreply@github.com 2023-06-10T10:49:40+03:00 ggml : workaround for missing _mm256_setr_m128i in GCC < 8 (#1638) 555275a693843273759230547001f9ae07fb537e 98ed16557432d7a5179c57eddcc3a08a7ae6d54d rankaiyx rankaiyx@rankaiyx.com 2023-06-10T14:41:59+08:00 GitHub noreply@github.com 2023-06-10T09:41:59+03:00 make : add SSSE3 compilation use case (#1659) 98ed16557432d7a5179c57eddcc3a08a7ae6d54d ae9663f1887513e152839e91f61c513075a19422 Robert Sung-wook Shin edp1096@users.noreply.github.com 2023-06-10T01:24:40+09:00 GitHub noreply@github.com 2023-06-09T18:24:40+02:00 OpenCL: Add release memory (#1741) ae9663f1887513e152839e91f61c513075a19422 b33dee282f5d8032b5f780152732dc45cbf2d349 Johannes Gäßler johannesg@5d6.de 2023-06-09T13:58:15+02:00 GitHub noreply@github.com 2023-06-09T13:58:15+02:00 Windows nvcc workaround (#1753) b33dee282f5d8032b5f780152732dc45cbf2d349 92f44ff7f778ef1b94028b2ba6d39943b5ca0ada Georgi Gerganov ggerganov@gmail.com 2023-06-09T11:11:04+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-09T11:11:04+03:00 metal : fix build "tanhf" -> "tanh" 92f44ff7f778ef1b94028b2ba6d39943b5ca0ada 245fc3c37da5ac5963f9f11a9f4f2ac08d96afc6 AT manyoso@users.noreply.github.com 2023-06-09T04:00:51-04:00 GitHub noreply@github.com 2023-06-09T11:00:51+03:00 metal : add GELU implementation (#1770) 245fc3c37da5ac5963f9f11a9f4f2ac08d96afc6 72ff5282bf0388c60821f504c4c8cc2b1f491aa6 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-09T10:39:59+03:00 GitHub noreply@github.com 2023-06-09T10:39:59+03:00 metal : faster q4_0 (#1775) 72ff5282bf0388c60821f504c4c8cc2b1f491aa6 0bf7cf1b296fc9fca05411b37afdf08a531487d2 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-08T22:28:21+03:00 GitHub noreply@github.com 2023-06-08T22:28:21+03:00 metal : add Q2_K implementation (#1762) 0bf7cf1b296fc9fca05411b37afdf08a531487d2 8432d4d9f716b25133e3ed671d91e21f6f3be867 Georgi Gerganov ggerganov@gmail.com 2023-06-08T20:48:14+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-08T20:48:14+03:00 Revert "ggml : load data into int8x16x4_t using vld4q_s8 on arm64 (#1738)" 8432d4d9f716b25133e3ed671d91e21f6f3be867 0f291e1f65c1d68201e71ce99c89562a36686b6d le.chang cljs118@126.com 2023-06-09T00:47:56+08:00 GitHub noreply@github.com 2023-06-08T19:47:56+03:00 ggml : load data into int8x16x4_t using vld4q_s8 on arm64 (#1738) 0f291e1f65c1d68201e71ce99c89562a36686b6d 8fc8179919a11738910db07a800f2b176f8adf09 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-08T19:46:22+03:00 GitHub noreply@github.com 2023-06-08T19:46:22+03:00 metal : Q6_K implementation (#1752) 8fc8179919a11738910db07a800f2b176f8adf09 b50b570ed9d699d3d126d72fc02de92926bcd937 qingfengfenga 41416092+qingfengfenga@users.noreply.github.com 2023-06-08T15:58:53+08:00 GitHub noreply@github.com 2023-06-08T00:58:53-07:00 Add llama.cpp docker support for non-latin languages (#1673) b50b570ed9d699d3d126d72fc02de92926bcd937 53aba3f393f2e02a78ddaba2e934893a8bbf3246 Steven Roussey sroussey@gmail.com 2023-06-08T00:12:28-07:00 GitHub noreply@github.com 2023-06-08T10:12:28+03:00 ggml : fix fprintf warnings (#1720) 53aba3f393f2e02a78ddaba2e934893a8bbf3246 4161bdc04debb70bf5f275492b4d89fd9330087c Georgi Gerganov ggerganov@gmail.com 2023-06-08T10:09:08+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-08T10:09:08+03:00 clang-tidy : restore dot file from accidental deletion 4161bdc04debb70bf5f275492b4d89fd9330087c 0035858273ebe0694926bf4414d279f3e1cd109d Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-08T10:08:23+03:00 GitHub noreply@github.com 2023-06-08T10:08:23+03:00 metal : add Q4_K implementation (#1733) 0035858273ebe0694926bf4414d279f3e1cd109d 5c64a0952ee58b2d742ee84e8e3d43cce5d366db johnson442 56517414+johnson442@users.noreply.github.com 2023-06-08T08:02:48+01:00 GitHub noreply@github.com 2023-06-08T10:02:48+03:00 k-quants : add missing compile definition to CMakeLists (#1748) 5c64a0952ee58b2d742ee84e8e3d43cce5d366db 5b57a5b72676540b6a45a3f527126299969ad241 Georgi Gerganov ggerganov@gmail.com 2023-06-07T10:59:52+03:00 GitHub noreply@github.com 2023-06-07T10:59:52+03:00 k-quants : allow to optionally disable at compile time (#1734) 5b57a5b72676540b6a45a3f527126299969ad241 4dc62c545df0af60635d579e9e4dd91bc5afff51 jacobi petrucciani 8117202+jpetrucciani@users.noreply.github.com 2023-06-07T00:15:31-04:00 GitHub noreply@github.com 2023-06-07T07:15:31+03:00 flake : update to support metal on m1/m2 (#1724) 4dc62c545df0af60635d579e9e4dd91bc5afff51 35a84916fb029905c44746127026079268216e7a Georgi Gerganov ggerganov@gmail.com 2023-06-07T07:15:08+03:00 GitHub noreply@github.com 2023-06-07T07:15:08+03:00 readme : add June roadmap 35a84916fb029905c44746127026079268216e7a 2d7bf110edd8c49209401a16132052cba706ffd0 Willy Tarreau w@1wt.eu 2023-06-07T04:10:17+02:00 GitHub noreply@github.com 2023-06-06T22:10:17-04:00 main: add the possibility to open the prompt cache read-only (#1640) 2d7bf110edd8c49209401a16132052cba706ffd0 2a4e41a086ce80da68c402457c75c77e52dcc698 Georgi Gerganov ggerganov@gmail.com 2023-06-06T22:54:39+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-06T22:54:39+03:00 llama : fix vram_scratch var 2a4e41a086ce80da68c402457c75c77e52dcc698 17366df842e358768c0df7024484fffecfc7865b Georgi Gerganov ggerganov@gmail.com 2023-06-06T22:41:53+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-06T22:41:53+03:00 llama : fix compile warnings 17366df842e358768c0df7024484fffecfc7865b 44f906e8537fcec965e312d621c80556d6aa9bec Johannes Gäßler johannesg@5d6.de 2023-06-06T21:33:23+02:00 GitHub noreply@github.com 2023-06-06T21:33:23+02:00 Multi GPU support, CUDA refactor, CUDA scratch buffer (#1703) 44f906e8537fcec965e312d621c80556d6aa9bec d5b111f53d14972669eb52055f9df2567663ad8b Georgi Gerganov ggerganov@gmail.com 2023-06-06T20:16:57+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-06T20:21:56+03:00 metal : add f16 support d5b111f53d14972669eb52055f9df2567663ad8b 2d43387dafe9c60f15f57aa23ee0b37864b98b32 LostRuins 39025047+LostRuins@users.noreply.github.com 2023-06-07T01:00:01+08:00 GitHub noreply@github.com 2023-06-06T19:00:01+02:00 Clblast fixes + enhancements to save VRAM and offload more layers (#1675) 2d43387dafe9c60f15f57aa23ee0b37864b98b32 7ad7750c5c9f6bcea73a1895ffc57e7d21f2ab95 Georgi Gerganov ggerganov@gmail.com 2023-06-06T10:18:03+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-06T10:18:03+03:00 ggml : fix builds, add ggml-quants-k.o (close #1712, close #1710) 7ad7750c5c9f6bcea73a1895ffc57e7d21f2ab95 7a74dee6b4e0e80862191141c0037abe28967d5c Georgi Gerganov ggerganov@gmail.com 2023-06-06T09:55:10+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-06T09:55:25+03:00 gitignore : add .clang-tidy 7a74dee6b4e0e80862191141c0037abe28967d5c 590250f7a9847bc9c83aa063dbaac8fa0fea27c8 Georgi Gerganov ggerganov@gmail.com 2023-06-06T09:39:38+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-06T09:39:38+03:00 llama : temporary disable Q6_K output quantization (#1711) 590250f7a9847bc9c83aa063dbaac8fa0fea27c8 f4c55d3bd7e124b101bc974cbbf0e0dbbc32d5a3 Spencer Sutton spencersutton@users.noreply.github.com 2023-06-05T23:28:17-04:00 GitHub noreply@github.com 2023-06-06T06:28:17+03:00 metal : add checks for buffer size (#1706) f4c55d3bd7e124b101bc974cbbf0e0dbbc32d5a3 f1465624c2cbc8ee65b566e3d87f2af27796d4c4 Yuval Peled 31162840+Yuval-Peled@users.noreply.github.com 2023-06-05T23:32:36+03:00 GitHub noreply@github.com 2023-06-05T23:32:36+03:00 docs : add performance troubleshoot + example benchmark documentation (#1674) f1465624c2cbc8ee65b566e3d87f2af27796d4c4 c2df36d60dc0ff1576541b965d751eadbacbeada Foul-Tarnished 107711110+Foul-Tarnished@users.noreply.github.com 2023-06-05T22:28:37+02:00 GitHub noreply@github.com 2023-06-05T23:28:37+03:00 readme : fix typo (#1700) c2df36d60dc0ff1576541b965d751eadbacbeada 9d0693bce38013364b1042568d9083353bfff48f mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2023-06-05T22:24:29+02:00 GitHub noreply@github.com 2023-06-05T23:24:29+03:00 llama : consistently catch and throw only exceptions deriving from std::exception (#1599) 9d0693bce38013364b1042568d9083353bfff48f efe05076323f5c6bafece109e21cce046f5e4b07 kiltyj kiltyj@gmail.com 2023-06-05T13:24:04-07:00 GitHub noreply@github.com 2023-06-05T23:24:04+03:00 metal : use shared buffers between CPU and GPU (#1696) efe05076323f5c6bafece109e21cce046f5e4b07 e7fe66e670537990ccc075cce9286df88bba052a grahameth 96447521+grahameth@users.noreply.github.com 2023-06-05T22:11:49+02:00 GitHub noreply@github.com 2023-06-05T23:11:49+03:00 ggml : fix internal overflow in ggml_time_us on Windows (#1702) e7fe66e670537990ccc075cce9286df88bba052a 99009e72f8072fa552eb02efee436be596c71cdd Georgi Gerganov ggerganov@gmail.com 2023-06-05T23:05:05+03:00 GitHub noreply@github.com 2023-06-05T23:05:05+03:00 ci : disable auto tidy (#1705) 99009e72f8072fa552eb02efee436be596c71cdd 5220a991a5e92bddad9542267ab445a2c033681c Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-06-05T22:56:18+03:00 GitHub noreply@github.com 2023-06-05T22:56:18+03:00 ggml : add SOTA 2,3,4,5,6 bit k-quantizations (#1684) 5220a991a5e92bddad9542267ab445a2c033681c d1f563a743a83dabc11e125d4a7d64189c16498c Henri Vasserman henv@hot.ee 2023-06-05T13:43:08+03:00 GitHub noreply@github.com 2023-06-05T13:43:08+03:00 Increase 3B scratch buffers. (#1698) d1f563a743a83dabc11e125d4a7d64189c16498c 827f5eda91e5b7299848ee2c7179d873bdee0f7b Georgi Gerganov ggerganov@gmail.com 2023-06-05T10:19:03+03:00 Georgi Gerganov ggerganov@gmail.com 2023-06-05T10:19:03+03:00 llama : fix Metal KV cache sync (close #1695) 827f5eda91e5b7299848ee2c7179d873bdee0f7b ecb217db4fcfa3880300ad08531a5fb6bb142d45 Georgi Gerganov ggerganov@gmail.com 2023-06-04T23:38:19+03:00 GitHub noreply@github.com 2023-06-04T23:38:19+03:00 readme : update hot topics ecb217db4fcfa3880300ad08531a5fb6bb142d45 dcb2ed48268e421baf25adc00d602dad0f415564 Georgi Gerganov ggerganov@gmail.com 2023-06-04T23:34:30+03:00 GitHub noreply@github.com 2023-06-04T23:34:30+03:00 llama : Metal inference (#1642) dcb2ed48268e421baf25adc00d602dad0f415564 d8bd0013e8768aaa3dc9cfc1ff01499419d5348e 0cc4m picard12@live.de 2023-06-04T08:12:05+02:00 GitHub noreply@github.com 2023-06-04T08:12:05+02:00 OpenCL: Fix duplication of layers in VRAM and RAM, add GPU mul kernel (#1653) d8bd0013e8768aaa3dc9cfc1ff01499419d5348e b5c85468a3eadf424420af5bf11c2353ff828cda Henri Vasserman henv@hot.ee 2023-06-03T16:35:20+03:00 GitHub noreply@github.com 2023-06-03T16:35:20+03:00 Add info about CUDA_VISIBLE_DEVICES (#1682) b5c85468a3eadf424420af5bf11c2353ff828cda 136476e898fb96c302b0829ee3e79267ae12660f Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2023-06-03T14:11:53+02:00 GitHub noreply@github.com 2023-06-03T15:11:53+03:00 Docker: change to calling convert.py (#1641) 136476e898fb96c302b0829ee3e79267ae12660f ffb06a345e3a9e30d39aaa5b46a23201a74be6de Evan Jones evan.q.jones@gmail.com 2023-06-03T07:28:45-04:00 GitHub noreply@github.com 2023-06-03T07:28:45-04:00 Fix prompt cache saving and chat-persistent rollover (#1678) ffb06a345e3a9e30d39aaa5b46a23201a74be6de 7552ac586380f202b75b18aa216ecfefbd438d94 Henri Vasserman henv@hot.ee 2023-05-30T21:24:22+03:00 GitHub noreply@github.com 2023-05-30T21:24:22+03:00 OpenLLaMA 3B support (#1588) 7552ac586380f202b75b18aa216ecfefbd438d94 5d1830b99dfd85bb6279adb4dd94aa444afd5b5e Georgi Gerganov ggerganov@gmail.com 2023-05-29T19:31:44+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-29T19:31:44+03:00 ggml : sync cgraph import / export API 5d1830b99dfd85bb6279adb4dd94aa444afd5b5e 248367605ead6fb7c36d2bfb1ebd8f00a23f7c71 Georgi Gerganov ggerganov@gmail.com 2023-05-29T19:30:49+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-29T19:30:49+03:00 ggml : fix bug in ggml_alibi 248367605ead6fb7c36d2bfb1ebd8f00a23f7c71 0e730dd23b0fb5f93dba574e0a48d9a69dc5dbae DannyDaemonic DannyDaemonic@gmail.com 2023-05-29T05:13:40-07:00 GitHub noreply@github.com 2023-05-29T05:13:40-07:00 Work around for recalculating logits in cached prompts (Fixes #1585) (#1609) 0e730dd23b0fb5f93dba574e0a48d9a69dc5dbae 3b126f654fceb4f5f195a1c2e825bb18e101188c Jiří Podivín 66251151+jpodivin@users.noreply.github.com 2023-05-29T06:45:50+02:00 GitHub noreply@github.com 2023-05-28T21:45:50-07:00 Adding git in container package dependencies (#1621) 3b126f654fceb4f5f195a1c2e825bb18e101188c 1b78ed20818b72306edc7208b9bfb69a1a0d3297 Johannes Gäßler johannesg@5d6.de 2023-05-28T21:01:02+02:00 GitHub noreply@github.com 2023-05-28T21:01:02+02:00 LLAMA_DEBUG adds debug symbols (#1617) 1b78ed20818b72306edc7208b9bfb69a1a0d3297 337aea11390221bc925e4acb1f603f1649af2735 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-05-28T11:48:57-06:00 GitHub noreply@github.com 2023-05-28T11:48:57-06:00 Only show -ngl option when relevant + other doc/arg handling updates (#1625) 337aea11390221bc925e4acb1f603f1649af2735 bb051d9723d628414b9e929e5264e23262a2f1b2 Vladimir Zorin vladimir@deviant.guru 2023-05-28T20:14:24+03:00 GitHub noreply@github.com 2023-05-28T20:14:24+03:00 examples : add --alias option to gpt_params to set use friendly model name (#1614) bb051d9723d628414b9e929e5264e23262a2f1b2 ca74884f6625b15ef69832f07fc60fe00db5f90c Howard Su howard0su@gmail.com 2023-05-29T01:13:36+08:00 GitHub noreply@github.com 2023-05-28T20:13:36+03:00 opencl : no need to allocate cl_mem on heap (#1612) ca74884f6625b15ef69832f07fc60fe00db5f90c a6704643b62243bc4b6bbcd727d63d44e01a1002 Howard Su howard0su@gmail.com 2023-05-29T01:09:56+08:00 GitHub noreply@github.com 2023-05-28T20:09:56+03:00 opencl : use strstr to check if fp16 supported (#1611) a6704643b62243bc4b6bbcd727d63d44e01a1002 0df7d63e5ba0ab8856476e121a03b985d6f15c9d apcameron 37645737+apcameron@users.noreply.github.com 2023-05-27T21:03:25+01:00 GitHub noreply@github.com 2023-05-27T23:03:25+03:00 ggml : add support for the RISCV architecture (#1616) 0df7d63e5ba0ab8856476e121a03b985d6f15c9d 97c9b77c4fc5e2283755c4418759cfc5fc73ad05 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-05-27T11:04:14-06:00 GitHub noreply@github.com 2023-05-27T11:04:14-06:00 Include server in releases + other build system cleanups (#1610) 97c9b77c4fc5e2283755c4418759cfc5fc73ad05 0ecb1bbbeb16e36a2ea7a5ce525c6c59ef74312b Henri Vasserman henv@hot.ee 2023-05-27T18:47:55+03:00 GitHub noreply@github.com 2023-05-27T18:47:55+03:00 Add documentation about CLBlast (#1604) 0ecb1bbbeb16e36a2ea7a5ce525c6c59ef74312b 93618031c7ccdd949d976370f24953d261048575 Henri Vasserman henv@hot.ee 2023-05-27T17:24:06+03:00 GitHub noreply@github.com 2023-05-27T17:24:06+03:00 [CI] Fix openblas (#1613) 93618031c7ccdd949d976370f24953d261048575 83c54e6da58f1970556741b143bd26e30b1f46af Georgi Gerganov ggerganov@gmail.com 2023-05-27T16:19:56+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-27T16:19:56+03:00 ggml : add ggml_tensor_overhead() 83c54e6da58f1970556741b143bd26e30b1f46af bdbda1b17afb78e8613d03c8210a57fac632397b Henri Vasserman henv@hot.ee 2023-05-27T15:18:25+03:00 GitHub noreply@github.com 2023-05-27T14:18:25+02:00 [CI] CLBlast: Fix directory name (#1606) bdbda1b17afb78e8613d03c8210a57fac632397b 66874d4fbcc7866377246efbcee938e8cc9c7d76 Georgi Gerganov ggerganov@gmail.com 2023-05-27T12:22:05+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-27T12:23:16+03:00 ggml : sync ggml core (minor additions, e.g. ggml_get_tensor_by_name()) 66874d4fbcc7866377246efbcee938e8cc9c7d76 1fcdcc28b119a6608774d52de905931bd5f8a43d Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-05-25T20:18:01-06:00 GitHub noreply@github.com 2023-05-25T20:18:01-06:00 Some improvements to loading the session with --prompt-cache (#1550) 1fcdcc28b119a6608774d52de905931bd5f8a43d ac7876ac20124a15a44fd6317721ff1aa2538806 Johannes Gäßler johannesg@5d6.de 2023-05-25T23:07:29+02:00 GitHub noreply@github.com 2023-05-26T00:07:29+03:00 cuda : performance optimizations (#1530) ac7876ac20124a15a44fd6317721ff1aa2538806 c31bbe934b9666af42f32ce12d32cae9160e5dc4 Henri Vasserman henv@hot.ee 2023-05-24T10:30:09+03:00 GitHub noreply@github.com 2023-05-24T10:30:09+03:00 Update CLBlast to 1.6.0 (#1580) c31bbe934b9666af42f32ce12d32cae9160e5dc4 1359b6aba55d5b0410f6adaa0aa2e49bbfd01d84 Evan Jones evan.q.jones@gmail.com 2023-05-24T02:24:01-04:00 GitHub noreply@github.com 2023-05-24T09:24:01+03:00 readme : add docs for chat-persistent.sh (#1568) 1359b6aba55d5b0410f6adaa0aa2e49bbfd01d84 7d873811f31d4d8c909015c946a862c0089cda7d Senemu 10880819+Senemu@users.noreply.github.com 2023-05-24T06:16:22Z GitHub noreply@github.com 2023-05-24T09:16:22+03:00 chat-persistent.sh : use bracket expressions in grep (#1564) 7d873811f31d4d8c909015c946a862c0089cda7d 2e6cd4b02549e343bef3768e6b946f999c82e823 Maarten ter Huurne maarten@treewalker.org 2023-05-23T18:01:15+02:00 GitHub noreply@github.com 2023-05-23T19:01:15+03:00 Fix handling of "invalid property" when creating OpenCL command queue (#1565) 2e6cd4b02549e343bef3768e6b946f999c82e823 7e4ea5beff567f53be92f75f9089e6f11fa5dabd 0cc4m picard12@live.de 2023-05-22T23:33:24+02:00 GitHub noreply@github.com 2023-05-23T00:33:24+03:00 OpenCL Token Generation Acceleration (#1459) 7e4ea5beff567f53be92f75f9089e6f11fa5dabd 7780e4f479dc5af106287c164b8e186cd9b6215c Steward Garcia 57494570+FSSRepo@users.noreply.github.com 2023-05-21T11:51:18-06:00 GitHub noreply@github.com 2023-05-21T20:51:18+03:00 examples : add server example with REST API (#1443) 7780e4f479dc5af106287c164b8e186cd9b6215c 265db9834e761b7c8210ea1888117efcd3262f52 Stefan Sydow stefan@sydow.email 2023-05-21T16:03:44+02:00 GitHub noreply@github.com 2023-05-21T17:03:44+03:00 make : .PHONY clean (#1553) 265db9834e761b7c8210ea1888117efcd3262f52 fab49c685e09d95942de34e3eadd72f880de21d5 Georgi Gerganov ggerganov@gmail.com 2023-05-21T11:56:23+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-21T11:56:23+03:00 ggml : output 3d sizes in ggml_graph_dump_dot() fab49c685e09d95942de34e3eadd72f880de21d5 b8ee340abe4a46143eb8cc4a135b976856c9136c Georgi Gerganov ggerganov@gmail.com 2023-05-20T20:00:41+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-20T20:00:41+03:00 ggml : update WASM SIMD b8ee340abe4a46143eb8cc4a135b976856c9136c 9ecb30f9594f222d8318fb1e803a4c363b0c39e5 Zenix zenixls2@gmail.com 2023-05-20T23:58:31+09:00 GitHub noreply@github.com 2023-05-20T17:58:31+03:00 feature : support blis and other blas implementation (#1536) 9ecb30f9594f222d8318fb1e803a4c363b0c39e5 29cf5596fe0c37213f9b74e80d8f631193a93f0f Henri Vasserman henv@hot.ee 2023-05-20T17:57:39+03:00 GitHub noreply@github.com 2023-05-20T17:57:39+03:00 OpenCL: Fixes for older devices. (#1435) 29cf5596fe0c37213f9b74e80d8f631193a93f0f 3de84b26066d95068409c1dc79bcc41c1eea2a03 Juuso Alasuutari juuso.alasuutari@gmail.com 2023-05-20T15:58:15+03:00 GitHub noreply@github.com 2023-05-20T15:58:15+03:00 llama : define magic numbers as integer constants (#1518) (#1520) 3de84b26066d95068409c1dc79bcc41c1eea2a03 affc76edfdefa7b326f526e463cc65ff13fcfb92 Georgi Gerganov ggerganov@gmail.com 2023-05-20T15:34:45+03:00 GitHub noreply@github.com 2023-05-20T15:34:45+03:00 ggml : add ggml_clamp() (#1539) affc76edfdefa7b326f526e463cc65ff13fcfb92 ea600071cb005267e9e8f2629c1e406dd5fde083 Johannes Gäßler johannesg@5d6.de 2023-05-20T14:19:28+02:00 GitHub noreply@github.com 2023-05-20T15:19:28+03:00 cuda : loading models directly into VRAM, norm calculation on GPU, broadcasting for ggml_mul (#1483) ea600071cb005267e9e8f2629c1e406dd5fde083 07e9ace0f9da424d82e75df969642522880feb92 Georgi Gerganov ggerganov@gmail.com 2023-05-20T12:03:48+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-20T12:03:48+03:00 Revert "feature : add blis and other BLAS implementation support (#1502)" 07e9ace0f9da424d82e75df969642522880feb92 ec2e10c4443209da56b431b24dd0845b60e757fb Zenix zenixls2@gmail.com 2023-05-20T18:02:48+09:00 GitHub noreply@github.com 2023-05-20T12:02:48+03:00 feature : add blis and other BLAS implementation support (#1502) ec2e10c4443209da56b431b24dd0845b60e757fb d2c59b8ba498ab01e65203dde6fe95236d20f6e7 Georgi Gerganov ggerganov@gmail.com 2023-05-20T11:06:11+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-20T11:06:37+03:00 llama : add llama_init_backend() API (close #1527) d2c59b8ba498ab01e65203dde6fe95236d20f6e7 503db28849d8641d66244385e7e9649608a2e4d0 DannyDaemonic DannyDaemonic@gmail.com 2023-05-20T00:40:02-07:00 GitHub noreply@github.com 2023-05-20T00:40:02-07:00 Fix for mingw (#1462) 503db28849d8641d66244385e7e9649608a2e4d0 8a203f9fa1b24e010be8f35ebbbd6786293684cb Maxime 672982+maximegmd@users.noreply.github.com 2023-05-20T09:22:37+02:00 GitHub noreply@github.com 2023-05-20T10:22:37+03:00 llama : fix name shadowing and C4146 (#1526) 8a203f9fa1b24e010be8f35ebbbd6786293684cb 4fd3e29297e3246a7be291932c115636fadb0f52 Georgi Gerganov ggerganov@gmail.com 2023-05-20T10:14:31+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-20T10:14:43+03:00 llama : fix compile warnings in llama_set_state_data() 4fd3e29297e3246a7be291932c115636fadb0f52 2d5db48371052087a83974abda3767d1aedec598 Georgi Gerganov ggerganov@gmail.com 2023-05-20T10:13:19+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-20T10:13:19+03:00 ggml : fix scalar implementation of Q4_1 dot 2d5db48371052087a83974abda3767d1aedec598 6986c7835adc13ba3f9d933b95671bb1f3984dc6 Georgi Gerganov ggerganov@gmail.com 2023-05-19T22:17:18+03:00 GitHub noreply@github.com 2023-05-19T22:17:18+03:00 ggml : use F16 instead of F32 in Q4_0, Q4_1, Q8_0 (#1508) 6986c7835adc13ba3f9d933b95671bb1f3984dc6 943e6081cc939df7584f8f0ab7057a39c2ef3271 Georgi Gerganov ggerganov@gmail.com 2023-05-19T21:17:28+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-19T21:17:28+03:00 tests : add missing header 943e6081cc939df7584f8f0ab7057a39c2ef3271 7694b52b9a206b93d59139c3c7c9b55da0f5aa59 Evan Jones evan.q.jones@gmail.com 2023-05-19T13:39:51-04:00 GitHub noreply@github.com 2023-05-19T20:39:51+03:00 examples : add persistent chat (#1495) 7694b52b9a206b93d59139c3c7c9b55da0f5aa59 79e3efb0e97b65b6cc72cd9ee970fa8189ad79a4 Jason McCartney jmac@theroot.org 2023-05-19T10:24:59-07:00 GitHub noreply@github.com 2023-05-19T20:24:59+03:00 main : make reverse prompt option act as a stop token in non-interactive mode (#1032) 79e3efb0e97b65b6cc72cd9ee970fa8189ad79a4 4b7e245adf63db675c3daab4a9bfddd451ef4097 David Kennedy dakennedyd@gmail.com 2023-05-19T13:16:30-04:00 GitHub noreply@github.com 2023-05-19T20:16:30+03:00 readme : adds WizardLM to the list of supported models (#1485) 4b7e245adf63db675c3daab4a9bfddd451ef4097 5ea43392731040b454c293123839b90e159cbb99 Georgi Gerganov ggerganov@gmail.com 2023-05-19T20:14:51+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-19T20:14:51+03:00 minor : fix compile warnings 5ea43392731040b454c293123839b90e159cbb99 ee9654138ab0ae5f138f4abddf56ca234ea3c352 Erik Scholz Green-Sky@users.noreply.github.com 2023-05-18T19:31:01+02:00 GitHub noreply@github.com 2023-05-18T19:31:01+02:00 make kv_f16 the default for api users (#1517) ee9654138ab0ae5f138f4abddf56ca234ea3c352 dc271c52ed65e7c8dfcbaaf84dabb1f788e4f3d0 DannyDaemonic DannyDaemonic@gmail.com 2023-05-18T10:30:40-07:00 GitHub noreply@github.com 2023-05-18T19:30:40+02:00 Fixes #1511 lambda issue for w64devkit (mingw) (#1513) dc271c52ed65e7c8dfcbaaf84dabb1f788e4f3d0 c238b5873a1ea496db03ffcfe124c9d0d83afbc6 Stephan Walter stephan@walter.name 2023-05-17T22:12:01Z GitHub noreply@github.com 2023-05-17T22:12:01Z Remove unused n_parts parameter (#1509) c238b5873a1ea496db03ffcfe124c9d0d83afbc6 2b2646931bd2a2eb3e21c6f3733cc0e090b2e24b rankaiyx rankaiyx@rankaiyx.com 2023-05-17T22:47:58+08:00 GitHub noreply@github.com 2023-05-17T16:47:58+02:00 benchmark-matmul: Print the average of the test results (#1490) 2b2646931bd2a2eb3e21c6f3733cc0e090b2e24b 42627421ece816e632e6a0d757fa75150c687f87 Tom Jobbins 784313+TheBloke@users.noreply.github.com 2023-05-16T23:04:35+01:00 GitHub noreply@github.com 2023-05-17T00:04:35+02:00 convert.py: Support models which are stored in a single pytorch_model.bin (#1469) 42627421ece816e632e6a0d757fa75150c687f87 9560655409dc80771a9b19e838ff47c5c1df6483 Ilya Kurdyukov 59548320+ilyakurdyukov@users.noreply.github.com 2023-05-17T01:36:47+07:00 GitHub noreply@github.com 2023-05-16T18:36:47Z ~7% faster Q5_1 AVX2 code (#1477) 9560655409dc80771a9b19e838ff47c5c1df6483 2a5ee023ad3022bc0b505343394b9754587fb731 András Salamon ott2@users.noreply.github.com 2023-05-16T16:46:34+01:00 GitHub noreply@github.com 2023-05-16T17:46:34+02:00 define default model path once, sync path with readme (#1366) 2a5ee023ad3022bc0b505343394b9754587fb731 63d20469b85467c5729cc9a97bd44cc3da63423f sandyiscool sandyiscool@gmail.com 2023-05-16T14:00:15+05:30 GitHub noreply@github.com 2023-05-16T10:30:15+02:00 Add alternate include path for openblas (#1476) 63d20469b85467c5729cc9a97bd44cc3da63423f b5c9295eef2b56e307393b35b3a923e3518d226e zrm trustiosity.zrm@gmail.com 2023-05-14T22:25:42-04:00 GitHub noreply@github.com 2023-05-15T04:25:42+02:00 fix get_num_physical_cores() (#1436) b5c9295eef2b56e307393b35b3a923e3518d226e eb363627fda5f47de8ab5e9be8abd426049d00df slaren slarengh@gmail.com 2023-05-14T22:46:00+02:00 GitHub noreply@github.com 2023-05-14T22:46:00+02:00 benchmark-matmul: fix clang-tidy issues, report results in GFLOPS (#1458) eb363627fda5f47de8ab5e9be8abd426049d00df 79b2d5b69d80be0bf29312fb9a95854876b0a8a5 Johannes Gäßler johannesg@5d6.de 2023-05-14T20:53:23+02:00 GitHub noreply@github.com 2023-05-14T21:53:23+03:00 cuda : deduplicated dequantization code (#1453) 79b2d5b69d80be0bf29312fb9a95854876b0a8a5 13c351ad7292c5b5ab35db25c7a4f993e75d9cfd xaedes xaedes@gmail.com 2023-05-14T17:55:02+02:00 GitHub noreply@github.com 2023-05-14T18:55:02+03:00 ggml : alternative fix for race condition bug in non-inplace ggml_compute_forward_diag_mask_f32 (#1454) 13c351ad7292c5b5ab35db25c7a4f993e75d9cfd 60f8c361ca26328ef8523dfb08077fe2f1034490 Georgi Gerganov ggerganov@gmail.com 2023-05-14T18:22:50+03:00 GitHub noreply@github.com 2023-05-14T18:22:50+03:00 ggml : various fixes (#1450) 60f8c361ca26328ef8523dfb08077fe2f1034490 601a033475645370483973817d987928ea95f36c katsu560 118887472+katsu560@users.noreply.github.com 2023-05-14T19:03:51+09:00 GitHub noreply@github.com 2023-05-14T10:03:51Z ggml : add AVX support based on AVX2 code (#1430) 601a033475645370483973817d987928ea95f36c 08737ef720f0510c7ec2aa84d7f70c691073c35d Georgi Gerganov ggerganov@gmail.com 2023-05-14T10:20:19+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-14T10:20:19+03:00 ggml : add GGML_QNT_VERSION to track quantization format changes 08737ef720f0510c7ec2aa84d7f70c691073c35d bda4d7c215aa16b2a78e522521dfc0e1c2e8b194 Georgi Gerganov ggerganov@gmail.com 2023-05-13T17:40:58+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-13T17:40:58+03:00 cuda : fix convert function (#1412) bda4d7c215aa16b2a78e522521dfc0e1c2e8b194 5a5aeb1e91009c72bf816400b758bb8a305616d7 Georgi Gerganov ggerganov@gmail.com 2023-05-13T17:25:09+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-13T17:25:09+03:00 make : fix PERF build with cuBLAS 5a5aeb1e91009c72bf816400b758bb8a305616d7 66841fdb0eaf0cc210757cc7f683d0f4eebadc21 Georgi Gerganov ggerganov@gmail.com 2023-05-13T16:55:14+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-13T16:55:14+03:00 llama : fix unused warning 66841fdb0eaf0cc210757cc7f683d0f4eebadc21 905d87b70aa189623d500a28602d7a3a755a4769 Georgi Gerganov ggerganov@gmail.com 2023-05-13T16:48:03+03:00 GitHub noreply@github.com 2023-05-13T16:48:03+03:00 ggml : multi-thread mul and diag_mask ops (#1428) 905d87b70aa189623d500a28602d7a3a755a4769 f954edda935a70a14cf0cc45ecc7fe7d60cf3e4b Johannes Gäßler johannesg@5d6.de 2023-05-13T15:38:36+02:00 GitHub noreply@github.com 2023-05-13T16:38:36+03:00 ggml : GPU-accelerated token generation (#1412) f954edda935a70a14cf0cc45ecc7fe7d60cf3e4b f048af0230ad5381bb20b9e3c1467ec6fc7debdf xaedes xaedes@gmail.com 2023-05-13T14:56:40+02:00 GitHub noreply@github.com 2023-05-13T15:56:40+03:00 ggml : implement backward pass for llama + small training-llama-from-scratch example (#1360) f048af0230ad5381bb20b9e3c1467ec6fc7debdf ac0cd259d54be7e45ffa2c7b2508812cf4f83ccf Georgi Gerganov ggerganov@gmail.com 2023-05-13T11:54:33+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-13T11:54:33+03:00 ggml : sync alibi fix from ggml repo ac0cd259d54be7e45ffa2c7b2508812cf4f83ccf 0cd22e190aeaef867fa5db025b4d274f2fcfdcf6 3ooabkhxtn 31479382+3ooabkhxtn@users.noreply.github.com 2023-05-13T10:43:33+02:00 GitHub noreply@github.com 2023-05-13T08:43:33Z Adding SSE instructions to ggml_vec_dot_q4_0_q8_0 (#1413) 0cd22e190aeaef867fa5db025b4d274f2fcfdcf6 6456a4eb9f9c1f4de8f6908ef100daa78802ad00 Georgi Gerganov ggerganov@gmail.com 2023-05-13T11:23:15+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-13T11:23:15+03:00 llama : fix various warnings 6456a4eb9f9c1f4de8f6908ef100daa78802ad00 cdd5350892b1d4e521e930c77341f858fcfcd433 Rinne AsakusaRinne@gmail.com 2023-05-13T15:24:20+08:00 GitHub noreply@github.com 2023-05-13T10:24:20+03:00 embedding : remove unused code (#1426) cdd5350892b1d4e521e930c77341f858fcfcd433 738ace394a6f8cf0174e90a97185d9e512c0e200 Georgi Gerganov ggerganov@gmail.com 2023-05-13T09:12:44+03:00 GitHub noreply@github.com 2023-05-13T09:12:44+03:00 readme : update Q4_0 perplexities 738ace394a6f8cf0174e90a97185d9e512c0e200 699b1ad7fe6f7b9e41d3cb41e61a8cc3ea5fc6b5 Georgi Gerganov ggerganov@gmail.com 2023-05-13T09:08:52+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-13T09:08:52+03:00 llama : free ggml context in set / copy state data (close #1425) 699b1ad7fe6f7b9e41d3cb41e61a8cc3ea5fc6b5 fb62f924336c9746da9976c6ab3c2e6460258d54 Henri Vasserman henv@hot.ee 2023-05-13T09:01:15+03:00 GitHub noreply@github.com 2023-05-13T09:01:15+03:00 opencl : fix kernels for the new formats (#1422) fb62f924336c9746da9976c6ab3c2e6460258d54 773ee249fb6c14f791ac39f6ec05536f40dedc54 Georgi Gerganov ggerganov@gmail.com 2023-05-12T21:44:20+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-12T21:44:20+03:00 llama : fix --mtest option (close #1414) 773ee249fb6c14f791ac39f6ec05536f40dedc54 553fd4d4b5f3314f338be8006f62a4fdf7670186 Johannes Gäßler johannesg@5d6.de 2023-05-12T16:34:55+02:00 GitHub noreply@github.com 2023-05-12T14:34:55Z CLI args use - instead of _, backwards compatible (#1416) 553fd4d4b5f3314f338be8006f62a4fdf7670186 089b1c93ba2b93bc9a605af293730a028fad2c4e slaren slarengh@gmail.com 2023-05-12T15:40:53+02:00 GitHub noreply@github.com 2023-05-12T15:40:53+02:00 Add clang-tidy reviews to CI (#1407) 089b1c93ba2b93bc9a605af293730a028fad2c4e b9fd7eee57df101d4a3e3eabc9fd6c2cb13c9ca1 Rinne liu_yaohui1998@126.com 2023-05-12T13:39:40+08:00 GitHub noreply@github.com 2023-05-12T08:39:40+03:00 readme : add C#/.NET bindings repo (#1409) b9fd7eee57df101d4a3e3eabc9fd6c2cb13c9ca1 b608b55a3ea8e4760c617418538465449175bdb8 Georgi Gerganov ggerganov@gmail.com 2023-05-12T00:23:08+03:00 GitHub noreply@github.com 2023-05-12T00:23:08+03:00 ggml : remove bit shuffling (#1405) b608b55a3ea8e4760c617418538465449175bdb8 cf348a60e0af3905acd1d297cb064b918265b7ac CRD716 crd716@gmail.com 2023-05-11T10:10:19-05:00 GitHub noreply@github.com 2023-05-11T18:10:19+03:00 prompts : model agnostic DAN (#1304) cf348a60e0af3905acd1d297cb064b918265b7ac e6a46b0ed1884c77267dc70693183e3b7164e0e0 Evan Jones evan.q.jones@gmail.com 2023-05-10T11:37:14-04:00 GitHub noreply@github.com 2023-05-10T11:37:14-04:00 main : add option to save full output to session (#1338) e6a46b0ed1884c77267dc70693183e3b7164e0e0 9f8dbc4787f32cd9c13b5c647d497d13c1a06db2 DannyDaemonic DannyDaemonic@gmail.com 2023-05-09T10:53:28-07:00 GitHub noreply@github.com 2023-05-09T19:53:28+02:00 Locale fix for Windows (#1379) 9f8dbc4787f32cd9c13b5c647d497d13c1a06db2 41654efea879bbdf4fd794e13335929d4cf0eb90 Sami Farin 3876865+Safari77@users.noreply.github.com 2023-05-09T15:29:20+03:00 GitHub noreply@github.com 2023-05-09T14:29:20+02:00 use pause asm insn in busyloop to run the CPU (13600K) 10 °C cooler (#1314) 41654efea879bbdf4fd794e13335929d4cf0eb90 56551bc11f46b2716fdf61bb48ac28414889dc0a DannyDaemonic DannyDaemonic@gmail.com 2023-05-08T19:45:48-07:00 GitHub noreply@github.com 2023-05-08T19:45:48-07:00 Interface improvements and `--multiline-input` (previously `--author-mode`) (#1040) 56551bc11f46b2716fdf61bb48ac28414889dc0a fe60904eef4b504685fa0406cb19864ae619fb4f Georgi Gerganov ggerganov@gmail.com 2023-05-08T22:52:18+03:00 GitHub noreply@github.com 2023-05-08T22:52:18+03:00 readme : add notice about upcoming breaking change fe60904eef4b504685fa0406cb19864ae619fb4f 003ba2fb4309e2339487564bd249e4fcc8d7ea01 AlpinDale 52078762+AlpinDale@users.noreply.github.com 2023-05-08T21:03:30+04:30 GitHub noreply@github.com 2023-05-08T19:33:30+03:00 readme : add TOC and Pygmalion instructions (#1359) 003ba2fb4309e2339487564bd249e4fcc8d7ea01 f9a6364912fd0463fddfdbc9ef9f79fdc281570d Pavol Rusnak pavol@rusnak.io 2023-05-08T16:48:21+02:00 GitHub noreply@github.com 2023-05-08T17:48:21+03:00 llama : fix hparams shadow (#1367) f9a6364912fd0463fddfdbc9ef9f79fdc281570d 95078cc554fe03d4512363c7e4dec963f0047c72 Georgi Gerganov ggerganov@gmail.com 2023-05-08T17:41:54+03:00 GitHub noreply@github.com 2023-05-08T17:41:54+03:00 llama : require first token to be BOS (#1303) 95078cc554fe03d4512363c7e4dec963f0047c72 1f48b0abcfbd6cc99571e42348e0ec97e4be8b93 ubik2 ubik2@users.noreply.github.com 2023-05-08T04:54:26-07:00 GitHub noreply@github.com 2023-05-08T13:54:26+02:00 convert: add ability to convert safetensors files (#1276) 1f48b0abcfbd6cc99571e42348e0ec97e4be8b93 e1295513a48ae8254d8af5ec0250b56d6eaffefd Johannes Gäßler johannesg@5d6.de 2023-05-08T02:42:01+02:00 GitHub noreply@github.com 2023-05-08T02:42:01+02:00 Documented CUDA reproducibility, added warning (#1346) e1295513a48ae8254d8af5ec0250b56d6eaffefd 1b0fd454650ef4d68a980e3225488b79e6e9af25 Henri Vasserman henv@hot.ee 2023-05-07T14:20:09+03:00 GitHub noreply@github.com 2023-05-07T13:20:09+02:00 CI: add Windows CLBlast and OpenBLAS builds (#1277) 1b0fd454650ef4d68a980e3225488b79e6e9af25 3924088512d9e12e90ed6dbf28a6c5712481d33e swittk switt1995@gmail.com 2023-05-07T10:03:23+07:00 GitHub noreply@github.com 2023-05-06T23:03:23-04:00 ggml : Allow usage of CLBlast alongside Accelerate.framework (#1336) 3924088512d9e12e90ed6dbf28a6c5712481d33e 173d0e6419e8f8f3c1f4f13201b777f4c60629f3 Jed Fox git@jedfox.com 2023-05-06T17:01:47-04:00 GitHub noreply@github.com 2023-05-06T17:01:47-04:00 Remove default arguments from sampling functions (#1343) 173d0e6419e8f8f3c1f4f13201b777f4c60629f3 a3b85b28da84c67c3406807aef5e0457bcc4b00f DaniAndTheWeb 57776841+DaniAndTheWeb@users.noreply.github.com 2023-05-05T23:57:14+02:00 GitHub noreply@github.com 2023-05-05T23:57:14+02:00 makefile: automatic Arch Linux detection (#1332) a3b85b28da84c67c3406807aef5e0457bcc4b00f 921dcee00a55d9aba3b3026d0509d31ac8386e2a Erik Scholz Green-Sky@users.noreply.github.com 2023-05-05T22:56:09+02:00 GitHub noreply@github.com 2023-05-05T22:56:09+02:00 ci : add cublas to windows release (#1271) 921dcee00a55d9aba3b3026d0509d31ac8386e2a 2d13786e91ec9fd28ddf737053822042a824da78 Pavol Rusnak pavol@rusnak.io 2023-05-05T16:43:36+02:00 GitHub noreply@github.com 2023-05-05T16:43:36+02:00 readme: add missing info (#1324) 2d13786e91ec9fd28ddf737053822042a824da78 a90e96b266873ebb5e947c9864b12193bdada0fb Ionoclast Laboratories brigham@ionoclast.com 2023-05-05T08:18:21-04:00 GitHub noreply@github.com 2023-05-05T14:18:21+02:00 Fix for OpenCL / clbast builds on macOS. (#1329) a90e96b266873ebb5e947c9864b12193bdada0fb 94c5652fc0f4d04ac54412c4d81e2ebcdafb6ede Benjamin Lecaillon 84293038+blecaillon@users.noreply.github.com 2023-05-05T02:17:07+02:00 GitHub noreply@github.com 2023-05-05T03:17:07+03:00 Convert.py @staticmethod (#1327) 94c5652fc0f4d04ac54412c4d81e2ebcdafb6ede 34d9f22f44c42d345cc72c8f3aa4cb71c5df0acb slaren slarengh@gmail.com 2023-05-05T00:58:56+02:00 GitHub noreply@github.com 2023-05-05T00:58:56+02:00 quantize: make output filename optional, default to ggml-model-.bin (#1301) 34d9f22f44c42d345cc72c8f3aa4cb71c5df0acb d3e8093e9b5845514b049ede3b12728c8f013eba Ivan Stepanov ivanstepanovftw@gmail.com 2023-05-04T19:56:27+03:00 GitHub noreply@github.com 2023-05-04T18:56:27+02:00 Wrap exceptions in std::exception to verbose output on exception. (#1316) d3e8093e9b5845514b049ede3b12728c8f013eba 360cfe5bec852805b84eec799102fc6f45df9fef Ivan Stepanov ivanstepanovftw@gmail.com 2023-05-04T19:54:37+03:00 GitHub noreply@github.com 2023-05-04T18:54:37+02:00 convert: support DT_BF16 tensors (#1309) 360cfe5bec852805b84eec799102fc6f45df9fef 2edbdb0f99336cb41f0995061c7602ed54beb863 44670 44670@users.noreply.github.com 2023-05-05T00:33:31+08:00 GitHub noreply@github.com 2023-05-04T19:33:31+03:00 readme : add OpenBuddy link (#1321) 2edbdb0f99336cb41f0995061c7602ed54beb863 20fbf2a2a08d8edefe9b3435fa86f8b2f63f8588 44670 44670@users.noreply.github.com 2023-05-04T23:41:12+08:00 GitHub noreply@github.com 2023-05-04T18:41:12+03:00 main : add --in-suffix option (#1318) 20fbf2a2a08d8edefe9b3435fa86f8b2f63f8588 db1080876a62ec3bb4119d90b16e7dce7594b733 Ron Jailall rojailal@gmail.com 2023-05-04T11:05:59-04:00 GitHub noreply@github.com 2023-05-04T18:05:59+03:00 ggml : change immintrin.h to intrin.h for compatibility (#1307) db1080876a62ec3bb4119d90b16e7dce7594b733 c65a7fbfa9c736416a25369cc05d356789df4c15 DannyDaemonic DannyDaemonic@gmail.com 2023-05-04T05:08:25-07:00 GitHub noreply@github.com 2023-05-04T05:08:25-07:00 Only escape prompts when used with `-e` (#1311) c65a7fbfa9c736416a25369cc05d356789df4c15 f647ce040ff06348d2ceaa5443a6a7a8b80c70c9 DannyDaemonic DannyDaemonic@gmail.com 2023-05-04T03:02:59-07:00 GitHub noreply@github.com 2023-05-04T03:02:59-07:00 Update main's README.md with new features (#1296) f647ce040ff06348d2ceaa5443a6a7a8b80c70c9 799fdc1b5d888b8a8682baf112e1c2a2df0df1c4 Tomas tom.tomas.36478119@gmail.com 2023-05-04T17:02:30+07:00 GitHub noreply@github.com 2023-05-04T03:02:30-07:00 fix #1224 reverse prompt and multi line (#1297) 799fdc1b5d888b8a8682baf112e1c2a2df0df1c4 6daa09d87926fe654385c2887e39ec3eeaa58120 Georgi Gerganov ggerganov@gmail.com 2023-05-03T23:24:20+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-03T23:24:20+03:00 ggml : vectorize Q8_0 quantization 6daa09d87926fe654385c2887e39ec3eeaa58120 bca9ad938a2a43621cf406d993b755cc91728dd5 khimaros me@khimaros.com 2023-05-03T10:58:11-07:00 GitHub noreply@github.com 2023-05-03T20:58:11+03:00 examples : read chat prompts from a template file (#1196) bca9ad938a2a43621cf406d993b755cc91728dd5 e2a937ca6abadc7e01e139db31e6db9dce16e3e9 Georgi Gerganov ggerganov@gmail.com 2023-05-03T20:09:42+03:00 GitHub noreply@github.com 2023-05-03T20:09:42+03:00 minor : fix whitespaces (#1302) e2a937ca6abadc7e01e139db31e6db9dce16e3e9 b0c71c7b6dc0c0adb507d78f401e95e7ab0f5a38 Georgi Gerganov ggerganov@gmail.com 2023-05-03T18:43:23+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-03T18:43:23+03:00 minor : fix trailing whitespaces b0c71c7b6dc0c0adb507d78f401e95e7ab0f5a38 a8a2efdc8161d4f69a0dd863e741c11fbd5df85c KASR karim.asrih@gmail.com 2023-05-03T17:31:28+02:00 GitHub noreply@github.com 2023-05-03T18:31:28+03:00 scripts : platform independent script to verify sha256 checksums (#1203) a8a2efdc8161d4f69a0dd863e741c11fbd5df85c e216aa04633892b972d013719e38b59fd4917341 CRD716 crd716@gmail.com 2023-05-03T10:26:47-05:00 GitHub noreply@github.com 2023-05-03T18:26:47+03:00 examples : various prompt and example fixes (#1298) e216aa04633892b972d013719e38b59fd4917341 2485d7a4d39406cd0f468e35551b472cceb5bd61 Evan Jones evan.q.jones@gmail.com 2023-05-02T22:26:13-04:00 GitHub noreply@github.com 2023-05-02T22:26:13-04:00 llama : only copy used KV cache in get / set state (#1272) 2485d7a4d39406cd0f468e35551b472cceb5bd61 13b0c68ed7a9948db0720f7393df094ab1005b14 DannyDaemonic DannyDaemonic@gmail.com 2023-05-02T18:46:20-07:00 GitHub noreply@github.com 2023-05-02T18:46:20-07:00 Process escape sequences given in prompts (#1173) 13b0c68ed7a9948db0720f7393df094ab1005b14 55bc5f0900d925c539488901c5538b637d68665c DannyDaemonic DannyDaemonic@gmail.com 2023-05-02T18:01:57-07:00 GitHub noreply@github.com 2023-05-02T18:01:57-07:00 Handle signals properly on Windows (#1123) 55bc5f0900d925c539488901c5538b637d68665c 9daff419f6be818595ddbea293a0ea7283af726e DannyDaemonic DannyDaemonic@gmail.com 2023-05-02T17:52:35-07:00 GitHub noreply@github.com 2023-05-02T17:52:35-07:00 Call sh on build-info.sh (#1294) 9daff419f6be818595ddbea293a0ea7283af726e bf4b22ffe433dc5e2fba7588c4485a7e51b1a30d kuvaus 22169537+kuvaus@users.noreply.github.com 2023-05-03T03:43:43+03:00 GitHub noreply@github.com 2023-05-03T02:43:43+02:00 fix build-info.h for git submodules (#1289) bf4b22ffe433dc5e2fba7588c4485a7e51b1a30d 67c77799e025a8425c23a6a0599c007f46ded590 slaren slarengh@gmail.com 2023-05-03T01:36:45+02:00 GitHub noreply@github.com 2023-05-03T01:36:45+02:00 fix missing parameters in `llama_init_from_gpt_params` (#1293) 67c77799e025a8425c23a6a0599c007f46ded590 0e6cbff1b7509628c588e661166f6e187137734d Ron Evans ron@hybridgroup.com 2023-05-02T22:39:51+02:00 GitHub noreply@github.com 2023-05-02T23:39:51+03:00 examples : add llama_init_from_gpt_params() common function (#1290) 0e6cbff1b7509628c588e661166f6e187137734d 5d5817ca603d4cb451bed26594aa3dcd93f4ec56 Georgi Gerganov ggerganov@gmail.com 2023-05-02T23:09:08+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-02T23:09:08+03:00 llama : fix compile warnings 5d5817ca603d4cb451bed26594aa3dcd93f4ec56 8c9be35ff998cbb4178b0fedcb9afd85cb6852e2 Georgi Gerganov ggerganov@gmail.com 2023-05-02T22:14:50+03:00 Georgi Gerganov ggerganov@gmail.com 2023-05-02T22:14:50+03:00 ggml : fix 32-bit ARM 8c9be35ff998cbb4178b0fedcb9afd85cb6852e2 cc0bb7235c72e50a621800e366d0e4fe315f0e11 Ron Evans ron@hybridgroup.com 2023-05-02T19:53:52+02:00 GitHub noreply@github.com 2023-05-02T20:53:52+03:00 examples : improve vertical alignment of a few variables (#1286) cc0bb7235c72e50a621800e366d0e4fe315f0e11 2bb992f034689e2ddd7b9ac05163b0359a5957b3 Marvin Gießing marvin.giessing@gmail.com 2023-05-02T18:42:16+02:00 GitHub noreply@github.com 2023-05-02T19:42:16+03:00 ggml : fix ppc64le build error and make cmake detect Power processors (#1284) 2bb992f034689e2ddd7b9ac05163b0359a5957b3 e2cd5069999181a9e4a22cf420e0491878b3062f Robert Brisita 986796+rbrisita@users.noreply.github.com 2023-05-02T12:23:44-04:00 GitHub noreply@github.com 2023-05-02T19:23:44+03:00 llama : allow 0 as a seed number. (#1275) e2cd5069999181a9e4a22cf420e0491878b3062f 2d099e5193d73f800b646c39e2fad08c1c1f1096 Ron Evans ron@hybridgroup.com 2023-05-02T18:13:26+02:00 GitHub noreply@github.com 2023-05-02T19:13:26+03:00 main : switch input_noecho to input_echo to remove negation (#979) 2d099e5193d73f800b646c39e2fad08c1c1f1096 f4cef87edfd1b2f8d5befd4fde54ca2e03987bea slaren slarengh@gmail.com 2023-05-02T16:03:00+02:00 GitHub noreply@github.com 2023-05-02T16:03:00+02:00 ggml: add names to tensors (#1268) f4cef87edfd1b2f8d5befd4fde54ca2e03987bea 58b367c2d757c0ea12aec672382462b42204c724 DannyDaemonic DannyDaemonic@gmail.com 2023-05-01T09:23:47-07:00 GitHub noreply@github.com 2023-05-01T18:23:47+02:00 Add git-based build information for better issue tracking (#1232) 58b367c2d757c0ea12aec672382462b42204c724 ea3a0ad6b6b5ca4693b94acd4cb32e2803f66fae slaren 2141330+slaren@users.noreply.github.com 2023-05-01T18:11:07+02:00 GitHub noreply@github.com 2023-05-01T18:11:07+02:00 cuBLAS: refactor and optimize f16 mat mul performance (#1259) ea3a0ad6b6b5ca4693b94acd4cb32e2803f66fae 2bdc09646d8c6cb74a6f573e9081586b4b83b9d1 xloem 0xloem@gmail.com 2023-05-01T08:58:51-04:00 GitHub noreply@github.com 2023-05-01T15:58:51+03:00 llama : update stubs for systems without mmap and mlock (#1266) 2bdc09646d8c6cb74a6f573e9081586b4b83b9d1 70269cae37538461ff816e714afbb3ebcdcdc26b Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-05-01T05:56:07-06:00 GitHub noreply@github.com 2023-05-01T14:56:07+03:00 ggml : fix ggml_used_mem() (#1264) 70269cae37538461ff816e714afbb3ebcdcdc26b b925f1f1b082319ee69943f8d1a83ac9b6ff09ca Georgi Gerganov ggerganov@gmail.com 2023-05-01T14:54:59+03:00 GitHub noreply@github.com 2023-05-01T14:54:59+03:00 llama : fix session load / save (#1263) b925f1f1b082319ee69943f8d1a83ac9b6ff09ca 90b19bd6eee943832584f9cac0b6f9ea29cc42a4 slaren 2141330+slaren@users.noreply.github.com 2023-05-01T13:32:22+02:00 GitHub noreply@github.com 2023-05-01T13:32:22+02:00 cuBLAS: fall back to pageable memory if pinned alloc fails (#1233) 90b19bd6eee943832584f9cac0b6f9ea29cc42a4 7ff0dcd32091c703a12adb0c57c32c565ce17664 Alex Klinkhamer git@grencez.dev 2023-05-01T00:24:20-07:00 GitHub noreply@github.com 2023-05-01T10:24:20+03:00 llama : let context be const when accessing const data (#1261) 7ff0dcd32091c703a12adb0c57c32c565ce17664 6f796992869f306c48484d62a39f2a181ae2fd6f Georgi Gerganov ggerganov@gmail.com 2023-04-30T22:28:51+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-30T22:28:51+03:00 ggml : fix UB (int << 31) 6f796992869f306c48484d62a39f2a181ae2fd6f a5d30b1f53677cb50791fec41c43e93274347303 Pavol Rusnak pavol@rusnak.io 2023-04-30T20:48:38+02:00 GitHub noreply@github.com 2023-04-30T20:48:38+02:00 build: add armv{6,7,8} support to cmake (#1251) a5d30b1f53677cb50791fec41c43e93274347303 76a884920aa1d2fc0dc7a7ac12dfc5ec5816377c jon-chuang 9093549+jon-chuang@users.noreply.github.com 2023-04-30T14:41:35-04:00 GitHub noreply@github.com 2023-04-30T21:41:35+03:00 common : better default number of threads (#934) 76a884920aa1d2fc0dc7a7ac12dfc5ec5816377c 6bc4400e67e6bc4faad3ad3d5e9d8a6576a9752d 0cc4m picard12@live.de 2023-04-30T20:34:52+02:00 GitHub noreply@github.com 2023-04-30T21:34:52+03:00 ggml : add CLBlast q5_0, q5_1, q8_0 dequant kernels (#1225) 6bc4400e67e6bc4faad3ad3d5e9d8a6576a9752d f0d70f147d969e41fa410b8af2965a27aa901eb9 Georgi Gerganov ggerganov@gmail.com 2023-04-30T19:07:00+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-30T19:07:43+03:00 ggml : add Q5 WASM SIMD + GGML_FTYPE f0d70f147d969e41fa410b8af2965a27aa901eb9 3e5aa8a1c44051153d6d7b3eeca2f4b4e5fb310c Stephan Walter stephan@walter.name 2023-04-30T12:32:37Z GitHub noreply@github.com 2023-04-30T12:32:37Z Various fixes to mat_mul benchmark (#1253) 3e5aa8a1c44051153d6d7b3eeca2f4b4e5fb310c c3ca7a5f0546c561eb278be3f2fe335795679e01 Georgi Gerganov ggerganov@gmail.com 2023-04-30T10:25:46+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-30T10:25:46+03:00 ggml : fix labels for GGML_OP_ALIBI c3ca7a5f0546c561eb278be3f2fe335795679e01 e8c051611abfc9a7f37fd4bba48217180893bd68 Georgi Gerganov ggerganov@gmail.com 2023-04-29T21:34:23+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-29T21:34:23+03:00 ggml : fix 32-bit ARM NEON e8c051611abfc9a7f37fd4bba48217180893bd68 0b5a9350993e6fc8be45dc2a3eafc1fd0812d392 Georgi Gerganov ggerganov@gmail.com 2023-04-29T21:12:56+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-29T21:12:56+03:00 ggml : use vzip instead of vuzp for consistency 0b5a9350993e6fc8be45dc2a3eafc1fd0812d392 ec728e44d7488c2da3560970317708b2b12b9c04 Georgi Gerganov ggerganov@gmail.com 2023-04-29T19:28:36+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-29T19:28:36+03:00 ggml : fix visibility and unused warnings ec728e44d7488c2da3560970317708b2b12b9c04 214b6a35702a489e3738acd81fad6d46182d3036 Georgi Gerganov ggerganov@gmail.com 2023-04-29T18:43:42+03:00 GitHub noreply@github.com 2023-04-29T18:43:42+03:00 ggml : fix #if for f32_f32 mul_mat (CLBlast) (#1229) 214b6a35702a489e3738acd81fad6d46182d3036 305eb5afd51325e3142c01c17431febb7c67de87 Georgi Gerganov ggerganov@gmail.com 2023-04-29T18:43:28+03:00 GitHub noreply@github.com 2023-04-29T18:43:28+03:00 ggml : adjust mul_mat_f16 work memory (#1226) 305eb5afd51325e3142c01c17431febb7c67de87 84ca9c2ecf3391d911589d0fe2b483cbfb4b82a6 Georgi Gerganov ggerganov@gmail.com 2023-04-29T13:53:12+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-29T13:53:12+03:00 build : fix reference to old llama_util.h 84ca9c2ecf3391d911589d0fe2b483cbfb4b82a6 334637e43e3a0529b4b50e2c22968b1ed1633353 Georgi Gerganov ggerganov@gmail.com 2023-04-29T13:48:11+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-29T13:48:11+03:00 examples : fix save-load-state + rename llama-util.h 334637e43e3a0529b4b50e2c22968b1ed1633353 dd7eff57d8491792010b1002b8de6a4b54912e5c Georgi Gerganov ggerganov@gmail.com 2023-04-29T09:51:06+03:00 GitHub noreply@github.com 2023-04-29T09:51:06+03:00 common : change default parameters to pre-#1126 (#1223) dd7eff57d8491792010b1002b8de6a4b54912e5c 7fc50c051ae8a78e9643fdf172d12e20f2dd9b6c Ivan Stepanov ivanstepanovftw@gmail.com 2023-04-29T08:34:41+03:00 GitHub noreply@github.com 2023-04-29T08:34:41+03:00 llama : new sampling algorithms (#1126) 7fc50c051ae8a78e9643fdf172d12e20f2dd9b6c b1ee8f59b4101b46999a0995d9a34506f7285466 slaren 2141330+slaren@users.noreply.github.com 2023-04-29T02:04:18+02:00 GitHub noreply@github.com 2023-04-29T02:04:18+02:00 cuBLAS: use host pinned memory and dequantize while copying (#1207) b1ee8f59b4101b46999a0995d9a34506f7285466 36d19a603b221d1bd7897fcb10e823e2103b052d Henri Vasserman henv@hot.ee 2023-04-29T02:31:56+03:00 GitHub noreply@github.com 2023-04-29T01:31:56+02:00 cuBLAS: non-contiguous tensor support (#1215) 36d19a603b221d1bd7897fcb10e823e2103b052d 7f15c5c477d9933689a9d1c40794483e350c2f19 Stephan Walter stephan@walter.name 2023-04-28T23:10:43Z GitHub noreply@github.com 2023-04-28T23:10:43Z Remove Q4_3 which is no better than Q5 (#1218) 7f15c5c477d9933689a9d1c40794483e350c2f19 55390bcaf2579a5435564d7267ae3ed367837fd6 Georgi Gerganov ggerganov@gmail.com 2023-04-28T21:32:52+03:00 GitHub noreply@github.com 2023-04-28T21:32:52+03:00 readme : update hot topics 55390bcaf2579a5435564d7267ae3ed367837fd6 5fba3c016bfd1d73a070e7c93dac14162ce118d0 Georgi Gerganov ggerganov@gmail.com 2023-04-28T20:37:43+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-28T20:51:05+03:00 ggml : sync ggml (ggml_alibi) 5fba3c016bfd1d73a070e7c93dac14162ce118d0 1481a9cf25ea2e4abef6b13a57660a35f3e66af1 CRD716 crd716@gmail.com 2023-04-28T11:13:33-05:00 GitHub noreply@github.com 2023-04-28T19:13:33+03:00 examples : add Jeopardy example (#1168) 1481a9cf25ea2e4abef6b13a57660a35f3e66af1 11d902364b0e3b503a02a4e757ee2dc38aacb68f Evan Jones evan.q.jones@gmail.com 2023-04-28T11:59:37-04:00 GitHub noreply@github.com 2023-04-28T18:59:37+03:00 llama : add session file format and saved sessions in main (#1169) 11d902364b0e3b503a02a4e757ee2dc38aacb68f 7296c961d9303010a2b98379f738da2a8a55aa1b Georgi Gerganov ggerganov@gmail.com 2023-04-28T17:58:44+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-28T17:59:08+03:00 ggml : add helper debug printf in soft_max 7296c961d9303010a2b98379f738da2a8a55aa1b 78ec543733d10a1629f984fd0302fdaa4e87fe66 0cc4m picard12@live.de 2023-04-28T16:57:16+02:00 GitHub noreply@github.com 2023-04-28T17:57:16+03:00 ggml : add CLBlast support (#1164) 78ec543733d10a1629f984fd0302fdaa4e87fe66 92a6e13a31ba052abd9062af6cb8df2a293ce661 Folko-Ven 71110216+Folko-Ven@users.noreply.github.com 2023-04-28T19:22:48+05:00 GitHub noreply@github.com 2023-04-28T16:22:48+02:00 Correcting link to w64devkit (#1214) 92a6e13a31ba052abd9062af6cb8df2a293ce661 04aaae1d79482cad2564412f3b32e70298ac7789 Johannes Gäßler johannesg@5d6.de 2023-04-28T15:40:32+02:00 GitHub noreply@github.com 2023-04-28T15:40:32+02:00 Add Manjaro CUDA include and lib dirs to Makefile (#1212) 04aaae1d79482cad2564412f3b32e70298ac7789 0b2da20538d01926b77ea237dd1c930c4d20b686 Yann Follet 131855179+YannFollet@users.noreply.github.com 2023-04-28T19:59:48+08:00 GitHub noreply@github.com 2023-04-28T11:59:48Z add avx2 for dot_q8_0_q8_0, 2x faster than scalar (#1211) 0b2da20538d01926b77ea237dd1c930c4d20b686 f9be42add0bf3ce61814b7ede0e6d0dda9ff22c6 Stephan Walter stephan@walter.name 2023-04-26T20:26:42Z GitHub noreply@github.com 2023-04-26T23:26:42+03:00 ggml : slightly faster AVX2 implementation for Q5 (#1197) f9be42add0bf3ce61814b7ede0e6d0dda9ff22c6 574406dc7e350ddbffaeca33bf0392b7bfeb1436 Georgi Gerganov ggerganov@gmail.com 2023-04-26T23:24:42+03:00 GitHub noreply@github.com 2023-04-26T23:24:42+03:00 readme : add quantization info 574406dc7e350ddbffaeca33bf0392b7bfeb1436 87a6f846d3e929632c45916dd08f1e2a9c72d2a3 Georgi Gerganov ggerganov@gmail.com 2023-04-26T23:14:13+03:00 GitHub noreply@github.com 2023-04-26T23:14:13+03:00 ggml : add Q5_0 and Q5_1 quantization (#1187) 87a6f846d3e929632c45916dd08f1e2a9c72d2a3 ea3ad7eb60cfb44526a58122e8019850f437cd1b Ásgeir Bjarni Ingvarsson asgeir@fundinn.org 2023-04-26T20:08:43Z GitHub noreply@github.com 2023-04-26T22:08:43+02:00 Allow setting the rng seed after initialization. (#1184) ea3ad7eb60cfb44526a58122e8019850f437cd1b 859fee6dfb00fab7ce6bc215b4adae78d82f4759 DaniAndTheWeb 57776841+DaniAndTheWeb@users.noreply.github.com 2023-04-26T22:03:03+02:00 GitHub noreply@github.com 2023-04-26T22:03:03+02:00 Updating build instructions to include BLAS support (#1183) 859fee6dfb00fab7ce6bc215b4adae78d82f4759 4afcc378698e057fcde64e23eb664e5af8dd6956 Pavol Rusnak pavol@rusnak.io 2023-04-26T18:43:27+02:00 GitHub noreply@github.com 2023-04-26T18:43:27+02:00 quantize : use `map` to assign quantization type from `string` (#1191) 4afcc378698e057fcde64e23eb664e5af8dd6956 667c501334ace706e3abc3f7a37cf1d6b4228745 Stephan Walter stephan@walter.name 2023-04-25T21:41:56Z GitHub noreply@github.com 2023-04-25T23:41:56+02:00 Update SHA256SUMS after quantization change (#1181) 667c501334ace706e3abc3f7a37cf1d6b4228745 bb98e77be704584fb40b0400394b4c16ae75f8e2 ostix360 55257054+ostix360@users.noreply.github.com 2023-04-25T23:33:08+02:00 GitHub noreply@github.com 2023-04-25T23:33:08+02:00 py : cast lora_alpha to int in convert-lora-to-ggml (#1170) bb98e77be704584fb40b0400394b4c16ae75f8e2 7a32fcb3b29f4db8aed8a85dc58eb958fb118153 Pavol Rusnak pavol@rusnak.io 2023-04-25T23:19:57+02:00 GitHub noreply@github.com 2023-04-25T23:19:57+02:00 nix: use convert.py instead of legacy wrapper convert-pth-to-ggml.py (#981) 7a32fcb3b29f4db8aed8a85dc58eb958fb118153 dd0eabc049fb1efc631cab8eb0a646808d704e18 Georgi Gerganov ggerganov@gmail.com 2023-04-25T23:40:51+03:00 GitHub noreply@github.com 2023-04-25T23:40:51+03:00 ggml : add Q8_0 quantization format (rename the old one to Q8_1) (ARM NEON) (#1179) dd0eabc049fb1efc631cab8eb0a646808d704e18 54bb60e26858be251a0eb3cb70f80322aff804a0 unbounded haakon@likedan.net 2023-04-25T19:20:46+02:00 GitHub noreply@github.com 2023-04-25T20:20:46+03:00 ggml : use full range for Q4_0 and Q4_2 quantization (#729) 54bb60e26858be251a0eb3cb70f80322aff804a0 8a0f8673ba1cdc6aa6df27a9fbc698431ca70e8d xaedes xaedes@gmail.com 2023-04-24T23:02:02+02:00 GitHub noreply@github.com 2023-04-24T23:02:02+02:00 ggml : fix bug in ggml_compute_forward_sum_f32 (#1162) 8a0f8673ba1cdc6aa6df27a9fbc698431ca70e8d 0c5692345d5c046dbc6a7d311a00ae5842ac39c3 Georgi Gerganov ggerganov@gmail.com 2023-04-24T22:18:25+03:00 GitHub noreply@github.com 2023-04-24T22:18:25+03:00 ggml : export symbols (#1155) 0c5692345d5c046dbc6a7d311a00ae5842ac39c3 957c8ae21d1e7052ea45a40ee8c0407b909e90cc xaedes xaedes@gmail.com 2023-04-24T18:23:31+02:00 GitHub noreply@github.com 2023-04-24T19:23:31+03:00 examples : add save_load_state example (#1150) 957c8ae21d1e7052ea45a40ee8c0407b909e90cc 9b0a4d421459f4e5e1af735c9784c3247b379025 Georgi Gerganov ggerganov@gmail.com 2023-04-24T18:47:03+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-24T18:47:30+03:00 llama : increase scratch buffer size for 65B (ref #1152) 9b0a4d421459f4e5e1af735c9784c3247b379025 2ec83428de7a876ecbbe484e1de42b73b5a40e25 mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2023-04-24T17:45:32+02:00 GitHub noreply@github.com 2023-04-24T15:45:32Z examples/main README improvements and some light refactoring (#1131) 2ec83428de7a876ecbbe484e1de42b73b5a40e25 e4cf982e0d4fcfbb4b977a52dbeacd115da10c3b Stephan Walter stephan@walter.name 2023-04-24T15:38:26Z GitHub noreply@github.com 2023-04-24T15:38:26Z Fix build for gcc 8 and test in CI (#1154) e4cf982e0d4fcfbb4b977a52dbeacd115da10c3b c4fe84fb0d28851a5c10e5a633f82ae2ba3b7fae slaren 2141330+slaren@users.noreply.github.com 2023-04-24T17:29:58+02:00 GitHub noreply@github.com 2023-04-24T17:29:58+02:00 Fix cuda compilation (#1128) c4fe84fb0d28851a5c10e5a633f82ae2ba3b7fae 1d78fecdab4087028a38517e86ed129f077174d8 Georgi Gerganov ggerganov@gmail.com 2023-04-24T07:40:02+03:00 GitHub noreply@github.com 2023-04-24T07:40:02+03:00 llama : refactor get / set state + remove redundant kv cache API (#1143) 1d78fecdab4087028a38517e86ed129f077174d8 284685f1692258c2bcf08b86b723b80ba2e66c7a slaren 2141330+slaren@users.noreply.github.com 2023-04-23T23:03:44+02:00 GitHub noreply@github.com 2023-04-23T23:03:44+02:00 Fix LoRA acronym (#1145) 284685f1692258c2bcf08b86b723b80ba2e66c7a edce63baa9dbd3963c3441bce07ee0acbb635697 Georgi Gerganov ggerganov@gmail.com 2023-04-23T19:57:09+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-23T19:57:09+03:00 scripts : add helper scripts to synch ggml repo edce63baa9dbd3963c3441bce07ee0acbb635697 ec9cdb6752dd96b3cc74d90ad1adeba5b4fa2b0e DannyDaemonic DannyDaemonic@gmail.com 2023-04-23T08:37:02-07:00 GitHub noreply@github.com 2023-04-23T15:37:02Z Added README.md for main with examples and explanations (#1139) ec9cdb6752dd96b3cc74d90ad1adeba5b4fa2b0e e4422e299c10c7e84c8e987770ef40d31905a76b Georgi Gerganov ggerganov@gmail.com 2023-04-23T18:32:52+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-23T18:32:52+03:00 ggml : do not print perf ops that have not been used at all e4422e299c10c7e84c8e987770ef40d31905a76b 53c8434398b3cba7ac6298cdd44abd40f0e640b1 Georgi Gerganov ggerganov@gmail.com 2023-04-23T18:15:39+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-23T18:15:39+03:00 ggml : better PERF prints + support "LLAMA_PERF=1 make" 53c8434398b3cba7ac6298cdd44abd40f0e640b1 c6524f46eb93fdb949330293a8469fd70080bd5a Stephan Walter stephan@walter.name 2023-04-23T11:01:03Z GitHub noreply@github.com 2023-04-23T11:01:03Z Improve AVX2 for vec_dot_q4_3_q8_0 (#1138) c6524f46eb93fdb949330293a8469fd70080bd5a c9e2c26f413377b352845f442cdab976ce85a05d Pavol Rusnak pavol@rusnak.io 2023-04-23T10:21:26+02:00 GitHub noreply@github.com 2023-04-23T10:21:26+02:00 readme : update gpt4all instructions (#980) c9e2c26f413377b352845f442cdab976ce85a05d 0e018fe008eacebdbcfa2d61b6c988c245c961cd Yishuo Wang MeouSker77@outlook.com 2023-04-23T15:57:05+08:00 GitHub noreply@github.com 2023-04-23T07:57:05Z A better `packNibbles` and `mul_sum_i8_pairs_float` implementation using AVX512 (#1119) 0e018fe008eacebdbcfa2d61b6c988c245c961cd 857308d1e8fb6afe33edb481d48560eee8fe7d7c Georgi Gerganov ggerganov@gmail.com 2023-04-22T16:31:56+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-22T16:32:07+03:00 ggml : fix Q4_3 cuBLAS 857308d1e8fb6afe33edb481d48560eee8fe7d7c c50b628810f36a3e6e0324371f6db579eacefa0e Stephan Walter stephan@walter.name 2023-04-22T13:12:29Z GitHub noreply@github.com 2023-04-22T16:12:29+03:00 ci : trigger CI for drafts, but not most PR actions (#1125) c50b628810f36a3e6e0324371f6db579eacefa0e 5f939498d517b4dddbe904f202e895a3ecfb9dc4 Stephan Walter stephan@walter.name 2023-04-22T10:54:13Z GitHub noreply@github.com 2023-04-22T10:54:13Z Fix CI: ARM NEON, quantization unit tests, editorconfig (#1122) 5f939498d517b4dddbe904f202e895a3ecfb9dc4 36b4f7e06406eed8a605cc9f2921d9244ef6a8e5 unbounded haakon@likedan.net 2023-04-22T11:10:39+02:00 GitHub noreply@github.com 2023-04-22T12:10:39+03:00 ggml : unit test for quantization functions (#953) 36b4f7e06406eed8a605cc9f2921d9244ef6a8e5 10f19c1121068ce3dab9bece03a8b9caaea2db36 wbpxre150 100937007+wbpxre150@users.noreply.github.com 2023-04-22T16:56:35+08:00 GitHub noreply@github.com 2023-04-22T11:56:35+03:00 llama : print timings on ctrl+c exit (#1021) 10f19c1121068ce3dab9bece03a8b9caaea2db36 7e312f165c5047d6e16680d1eebc83055e95c313 eiery 19350831+eiery@users.noreply.github.com 2023-04-22T04:27:05-04:00 GitHub noreply@github.com 2023-04-22T11:27:05+03:00 llama : have n_batch default to 512 (#1091) 7e312f165c5047d6e16680d1eebc83055e95c313 872c365a9176a011b13d31269bb3121fa89c37e1 Howard Su howard0su@gmail.com 2023-04-22T16:18:20+08:00 GitHub noreply@github.com 2023-04-22T11:18:20+03:00 cmake : fix build under Windows when enable BUILD_SHARED_LIBS (#1100) 872c365a9176a011b13d31269bb3121fa89c37e1 955ef9a5d53d8f911fe00580ac9bd0caa56430af Georgi Gerganov ggerganov@gmail.com 2023-04-22T11:08:12+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-22T11:08:12+03:00 ggml : fix AVX build + update to new Q8_0 format 955ef9a5d53d8f911fe00580ac9bd0caa56430af c5aa5e577741d0359ad26ec50b9e21a74c65d911 Georgi Gerganov ggerganov@gmail.com 2023-04-22T10:55:35+03:00 GitHub noreply@github.com 2023-04-22T10:55:35+03:00 ggml : alternative Q4_3 implementation using modified Q8_0 (#1109) c5aa5e577741d0359ad26ec50b9e21a74c65d911 e9a9cb0c54461ffbda75b7b2f99f3ea5562291c2 Stephan Walter stephan@walter.name 2023-04-22T07:37:05Z GitHub noreply@github.com 2023-04-22T10:37:05+03:00 ggml : AVX2 optimization for vec_dot_q4_3_q8_0 and refactoring (#1099) e9a9cb0c54461ffbda75b7b2f99f3ea5562291c2 b6e7f9b09e9c340ec97a2fae61c1eb8db861f2f9 Clint Herron hanclinto@gmail.com 2023-04-22T02:54:33-04:00 GitHub noreply@github.com 2023-04-22T09:54:33+03:00 examples : Improve Alpaca Default Repeat Penalty: Better Match Alpaca.cpp Experience (#1107) b6e7f9b09e9c340ec97a2fae61c1eb8db861f2f9 50cb666b8a2e35a49b08c0f6bc81138c8f6f2ac1 xaedes xaedes@gmail.com 2023-04-22T08:21:32+02:00 GitHub noreply@github.com 2023-04-22T09:21:32+03:00 llama : add api for getting/setting the complete state: rng, logits, embedding and kv_cache (#1105) 50cb666b8a2e35a49b08c0f6bc81138c8f6f2ac1 25d7abbd1f73582b7e0fdc422a936e8541c0780b slaren 2141330+slaren@users.noreply.github.com 2023-04-21T21:59:17+02:00 GitHub noreply@github.com 2023-04-21T21:59:17+02:00 Improve cuBLAS performance by using a memory pool (#1094) 25d7abbd1f73582b7e0fdc422a936e8541c0780b 018f2279f5fe3ef743bd8254b23ea8f0efae7e73 apaz aarpazdera@gmail.com 2023-04-21T13:48:06-05:00 GitHub noreply@github.com 2023-04-21T21:48:06+03:00 llama : fixed rlimit error message (#888) 018f2279f5fe3ef743bd8254b23ea8f0efae7e73 9411288271ab548216902a029f42a0a38ebcedb7 源文雨 41315874+fumiama@users.noreply.github.com 2023-04-22T02:27:06+08:00 GitHub noreply@github.com 2023-04-21T21:27:06+03:00 cmake : link threads publicly to ggml (#1042) 9411288271ab548216902a029f42a0a38ebcedb7 8687c1f2581d059cd5b6a9502f89bd343566062a Alex Klinkhamer from.github.com.917@grencez.dev 2023-04-21T11:18:09-07:00 GitHub noreply@github.com 2023-04-21T21:18:09+03:00 main : evaluate tokens in batches after swapping context (#1014) 8687c1f2581d059cd5b6a9502f89bd343566062a 1bfc153e2f35ddd9d64b084e8d1a5e6fa57ad1c9 xaedes xaedes@googlemail.com 2023-04-21T17:25:21+02:00 GitHub noreply@github.com 2023-04-21T18:25:21+03:00 llama : remember and restore kv cache data pointers (#1104) 1bfc153e2f35ddd9d64b084e8d1a5e6fa57ad1c9 3d59769c3bb7e72c915646ddb1e239b1face19f5 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-04-21T17:18:26+02:00 GitHub noreply@github.com 2023-04-21T18:18:26+03:00 ggml : a faster version for Q4_1 x Q8_0 dot products (#1083) 3d59769c3bb7e72c915646ddb1e239b1face19f5 d40fded93e1a533e969768e1e335c15c61c296ce slaren 2141330+slaren@users.noreply.github.com 2023-04-21T14:57:57+02:00 GitHub noreply@github.com 2023-04-21T14:57:57+02:00 Show perplexity ETA in hours and minutes (#1096) d40fded93e1a533e969768e1e335c15c61c296ce 2510c1831fac874f32e272f6079f01b5461f3986 Georgi Gerganov ggerganov@gmail.com 2023-04-21T10:23:36+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-21T10:24:02+03:00 llama : fix comment for "output.weight" tensor 2510c1831fac874f32e272f6079f01b5461f3986 12b5900dbc9743dee3ce83513cf5c3a44523a1b6 Stephan Walter stephan@walter.name 2023-04-20T21:56:44Z GitHub noreply@github.com 2023-04-20T23:56:44+02:00 Add ggml-model-*.bin checksums for 7B, 13B, 30B, 65B (#1088) 12b5900dbc9743dee3ce83513cf5c3a44523a1b6 9ff334f3c9b960a44c5e149b08c748a2914fb882 Georgi Gerganov ggerganov@gmail.com 2023-04-20T23:32:59+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-20T23:32:59+03:00 ggml : sync ggml (add GPT-NeoX RoPE implementation) 9ff334f3c9b960a44c5e149b08c748a2914fb882 2005469ea130cf920c50175d4f47a87bfd8aaf4d Georgi Gerganov ggerganov@gmail.com 2023-04-20T21:58:05+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-20T21:58:38+03:00 ggml : fix bug in ggml_compute_forward_dup_f32() 2005469ea130cf920c50175d4f47a87bfd8aaf4d 8a1756abdf1f48cb4dcb898bc8fbe9102ef49dc6 slaren 2141330+slaren@users.noreply.github.com 2023-04-20T20:49:53+02:00 GitHub noreply@github.com 2023-04-20T20:49:53+02:00 Add Q4_3 support to cuBLAS (#1086) 8a1756abdf1f48cb4dcb898bc8fbe9102ef49dc6 66aab46079609972ee1f7bd6f319d826205a2fbd Georgi Gerganov ggerganov@gmail.com 2023-04-20T21:43:50+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-20T21:43:50+03:00 ggml : do not break cuBLAS build (Q4_3 is not yet implemented) 66aab46079609972ee1f7bd6f319d826205a2fbd 38de86a7114c97ecf3644e3a60159f1ed893e1b0 Georgi Gerganov ggerganov@gmail.com 2023-04-20T20:44:05+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-20T20:44:05+03:00 ggml : fix Q4_3 quantization 38de86a7114c97ecf3644e3a60159f1ed893e1b0 e0305ead3a072db9c08b35c9600c49273b38a4b5 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-04-20T19:42:27+02:00 GitHub noreply@github.com 2023-04-20T20:42:27+03:00 llama : multi-threaded quantization (#1075) e0305ead3a072db9c08b35c9600c49273b38a4b5 6a9661ea5ad72166b700ae5e87976e4452499dda Georgi Gerganov ggerganov@gmail.com 2023-04-20T20:35:53+03:00 GitHub noreply@github.com 2023-04-20T20:35:53+03:00 ggml : add Q4_3 quantization (#1082) 6a9661ea5ad72166b700ae5e87976e4452499dda 5addcb120cf2682c7ede0b1c520592700d74c87c Ivan Komarov Ivan.Komarov@dfyz.info 2023-04-20T17:15:18+02:00 GitHub noreply@github.com 2023-04-20T18:15:18+03:00 ci : remove the LLAMA_ACCELERATE matrix dimension from Ubuntu builds in the CI (#1074) 5addcb120cf2682c7ede0b1c520592700d74c87c c8c2c524827be8fd681a63f0e5a697b0bf4c587b 源文雨 41315874+fumiama@users.noreply.github.com 2023-04-20T21:28:43+08:00 GitHub noreply@github.com 2023-04-20T15:28:43+02:00 fix: LLAMA_CUBLAS=1 undefined reference 'shm_open' (#1080) c8c2c524827be8fd681a63f0e5a697b0bf4c587b 02d6988121510c067e06d498a273a351a888f5b9 Stephan Walter stephan@walter.name 2023-04-20T06:45:41Z GitHub noreply@github.com 2023-04-20T08:45:41+02:00 AVX2 optimization for vec_dot_q4_2_q8_0 (#1068) 02d6988121510c067e06d498a273a351a888f5b9 834695fe3a3ed2a962e774c9615e3f7b41d360a8 slaren 2141330+slaren@users.noreply.github.com 2023-04-20T03:14:14+02:00 GitHub noreply@github.com 2023-04-20T03:14:14+02:00 Improve cuBLAS performance by dequantizing on the GPU (#1065) 834695fe3a3ed2a962e774c9615e3f7b41d360a8 f7d05095b404b5500b4a702ea16f67fc22446e49 CRD716 crd716@gmail.com 2023-04-19T14:52:14-05:00 GitHub noreply@github.com 2023-04-19T19:52:14Z Minor: Readme fixed grammar, spelling, and misc updates (#1071) f7d05095b404b5500b4a702ea16f67fc22446e49 884e7d7a2bfd7325b107442d6758983f5886ed3d Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-04-19T20:20:14+02:00 GitHub noreply@github.com 2023-04-19T20:20:14+02:00 Q4_2 quantization with rmse-optimized scale and quants (#1062) 884e7d7a2bfd7325b107442d6758983f5886ed3d 7cd5c4a3e9106151d48f328bb3c94c298a211f18 Georgi Gerganov ggerganov@gmail.com 2023-04-19T20:10:08+03:00 GitHub noreply@github.com 2023-04-19T20:10:08+03:00 ggml : use 8-bit precision for Q4_1 intermediate results (#1047) 7cd5c4a3e9106151d48f328bb3c94c298a211f18 f3d4edf504c19ee9d1381c5727fe38667205d979 Georgi Gerganov ggerganov@gmail.com 2023-04-19T19:07:54+03:00 GitHub noreply@github.com 2023-04-19T19:07:54+03:00 readme : add warning about Q4_2 and Q4_3 f3d4edf504c19ee9d1381c5727fe38667205d979 8944a1329648c57bb7d66851170938230587a52c Stephan Walter stephan@walter.name 2023-04-19T16:06:37Z GitHub noreply@github.com 2023-04-19T19:06:37+03:00 ggml : Q4 cleanup - remove 4-bit dot product code (#1061) 8944a1329648c57bb7d66851170938230587a52c 66674012389f9537140044290f8517bc4a5e0b74 slaren 2141330+slaren@users.noreply.github.com 2023-04-19T11:22:45+02:00 GitHub noreply@github.com 2023-04-19T11:22:45+02:00 Add NVIDIA cuBLAS support (#1044) 66674012389f9537140044290f8517bc4a5e0b74 77a73403ca8eaced2590559d0f9cebd2b3649d32 slaren 2141330+slaren@users.noreply.github.com 2023-04-19T00:53:24+02:00 GitHub noreply@github.com 2023-04-19T00:53:24+02:00 Multi-threaded ggml_cpy (#1035) 77a73403ca8eaced2590559d0f9cebd2b3649d32 50a8a2af97cb92e53e7a3195aa201c3d87da5415 Georgi Gerganov ggerganov@gmail.com 2023-04-18T23:54:57+03:00 GitHub noreply@github.com 2023-04-18T23:54:57+03:00 ggml : add new Q4_2 quantization (ARM only) (#1046) 50a8a2af97cb92e53e7a3195aa201c3d87da5415 4caebf6d408b91c2d29d0abc7b1e867b5de64db5 Georgi Gerganov ggerganov@gmail.com 2023-04-18T23:11:23+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-18T23:11:23+03:00 ggml : scratch that - vmlaq_n_f32 is always better 4caebf6d408b91c2d29d0abc7b1e867b5de64db5 dcdd65e2969bc03c91a1ebd1160162d5054e6923 Georgi Gerganov ggerganov@gmail.com 2023-04-18T23:00:08+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-18T23:00:08+03:00 gitignore : vdot dcdd65e2969bc03c91a1ebd1160162d5054e6923 5ecff35151156118c2df74899637ad34ee384b9b Georgi Gerganov ggerganov@gmail.com 2023-04-18T22:59:17+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-18T22:59:17+03:00 ggml : optimize ggml_vec_dot_q4_0_q8_0() using vectorized accumulators 5ecff35151156118c2df74899637ad34ee384b9b 7faa7460f03bdd88becf1e659cf359f274055404 Kawrakow 48489457+ikawrakow@users.noreply.github.com 2023-04-18T21:00:14+02:00 GitHub noreply@github.com 2023-04-18T19:00:14Z Adding a simple program to measure speed of dot products (#1041) 7faa7460f03bdd88becf1e659cf359f274055404 5af8e32238c7d9c4cdb7fc640472c9a26538b9da Georgi Gerganov ggerganov@gmail.com 2023-04-18T20:10:26+03:00 GitHub noreply@github.com 2023-04-18T20:10:26+03:00 readme : update hot topics about new LoRA functionality 5af8e32238c7d9c4cdb7fc640472c9a26538b9da 42747220b4cac548b6e3059b66b3e960b517cfa4 Georgi Gerganov ggerganov@gmail.com 2023-04-17T18:00:10+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-18T19:57:06+03:00 ci : do not run on drafts 42747220b4cac548b6e3059b66b3e960b517cfa4 e9298af3896b536d0c6d740447dc764e56b22102 Ivan Komarov Ivan.Komarov@dfyz.info 2023-04-18T03:15:50+02:00 GitHub noreply@github.com 2023-04-18T03:15:50+02:00 Do not close file after mmap (Windows version) (#1034) e9298af3896b536d0c6d740447dc764e56b22102 4ad73137a1aadc40a62f7101aab883f69e7172c6 Atsushi Tatsuma yoshoku@outlook.com 2023-04-18T04:34:35+09:00 GitHub noreply@github.com 2023-04-17T22:34:35+03:00 readme : add Ruby bindings (#1029) 4ad73137a1aadc40a62f7101aab883f69e7172c6 315a95a4d30db726fb7d244dd3b9e90a83fb1616 Cameron csteele@steelecameron.com 2023-04-17T11:26:23-07:00 GitHub noreply@github.com 2023-04-17T20:26:23+02:00 add 4_0 to default outfile namestr dict (#1031) 315a95a4d30db726fb7d244dd3b9e90a83fb1616 efd05648c88a0923a55f56e7ce1b0f9c33410afb slaren 2141330+slaren@users.noreply.github.com 2023-04-17T17:28:55+02:00 GitHub noreply@github.com 2023-04-17T17:28:55+02:00 Add LoRA support (#820) efd05648c88a0923a55f56e7ce1b0f9c33410afb eb17a026fd23d1c1b612fa4600f7f5c58e501a28 Arik Poznanski arikpoz@users.noreply.github.com 2023-04-17T17:41:53+03:00 GitHub noreply@github.com 2023-04-17T17:41:53+03:00 llama : well-defined static initialization of complex objects (#927) eb17a026fd23d1c1b612fa4600f7f5c58e501a28 69b740289f9b3756ea9dd2a23f241c6f688d88b9 Georgi Gerganov ggerganov@gmail.com 2023-04-17T17:31:06+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-17T17:31:06+03:00 quantize-stats : fix bug in --type argument 69b740289f9b3756ea9dd2a23f241c6f688d88b9 f266259ad9a2bce5a34d919592310147af23f3dc Georgi Gerganov ggerganov@gmail.com 2023-04-17T16:16:23+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-17T16:16:23+03:00 ggml : avoid using ggml_fp16_to_fp32() and ggml_fp32_to_fp16() in ggml.c f266259ad9a2bce5a34d919592310147af23f3dc 47f61aaa5f76d04286792e2fbd0c95b659ab2af0 Ivan Komarov Ivan.Komarov@dfyz.info 2023-04-17T15:10:57+02:00 GitHub noreply@github.com 2023-04-17T15:10:57+02:00 Speedup the AVX-512 implementation of ggml_vec_dot_q4_0() (#933) 47f61aaa5f76d04286792e2fbd0c95b659ab2af0 3173a62eb9f90b94fb3184131032c1c8b7aa8d86 slaren 2141330+slaren@users.noreply.github.com 2023-04-16T21:27:38+02:00 GitHub noreply@github.com 2023-04-16T21:27:38+02:00 Fix: do not close file on mmap (#1017) 3173a62eb9f90b94fb3184131032c1c8b7aa8d86 489537e6cf6c93b74a029a11533dbcaa89791dcc Georgi Gerganov ggerganov@gmail.com 2023-04-16T13:58:48+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-16T13:59:27+03:00 stdout : vertical align outputs for better readibility 489537e6cf6c93b74a029a11533dbcaa89791dcc 2d3481c72125cd388258864c7ad8d7d36777bad7 Pavol Rusnak pavol@rusnak.io 2023-04-16T12:13:00+02:00 GitHub noreply@github.com 2023-04-16T10:13:00Z examples: add missing include for time() (#1011) 2d3481c72125cd388258864c7ad8d7d36777bad7 74f5899df4a6083fc467b620baa1cf821e37799d nanahi 130121847+na-na-hi@users.noreply.github.com 2023-04-16T17:13:42+08:00 GitHub noreply@github.com 2023-04-16T11:13:42+02:00 Fix msys2 build error and warnings (#1009) 74f5899df4a6083fc467b620baa1cf821e37799d 2f7c8e014e3c0ceaf39688845c2ff6f919fb03b7 comex comexk@gmail.com 2023-04-15T14:53:21-07:00 GitHub noreply@github.com 2023-04-15T23:53:21+02:00 convert.py: Fix loading safetensors and ggml format on Windows (#991) 2f7c8e014e3c0ceaf39688845c2ff6f919fb03b7 0ad964631f9b3970f1936008fcfb1eadef59c7ed Stephan Walter stephan@walter.name 2023-04-15T18:28:56Z GitHub noreply@github.com 2023-04-15T18:28:56Z Fix potential int8 overflow in non-SIMD vec_dot (#986) 0ad964631f9b3970f1936008fcfb1eadef59c7ed e95b6554b493e71a0275764342e09bd5784a7026 Stephan Walter stephan@walter.name 2023-04-15T16:25:38Z GitHub noreply@github.com 2023-04-15T16:25:38Z Refactor ggml.c for future tensor types (#1001) e95b6554b493e71a0275764342e09bd5784a7026 aa485cee334e84437e21681c14b6f80b65876d8b Georgi Gerganov ggerganov@gmail.com 2023-04-15T17:53:22+03:00 GitHub noreply@github.com 2023-04-15T17:53:22+03:00 ggml : add Q8_0 quantization for intermediate results (#951) aa485cee334e84437e21681c14b6f80b65876d8b c12b14b77fced0ce9a0e2d81f670c3a746dec251 Georgi Gerganov ggerganov@gmail.com 2023-04-15T14:25:45+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-15T14:25:45+03:00 ggml : use posix_memalign on non-Windows env c12b14b77fced0ce9a0e2d81f670c3a746dec251 106faaf2971d6c89d6010279a9a95737772470ef Ivan Komarov Ivan.Komarov@dfyz.info 2023-04-15T07:51:54+02:00 GitHub noreply@github.com 2023-04-15T08:51:54+03:00 benchmark : fix result validation in benchmark-q4_0-matmult (#987) 106faaf2971d6c89d6010279a9a95737772470ef c85e03d12e4b8af22cb13aa9c618dcd5935862fd katsu560 118887472+katsu560@users.noreply.github.com 2023-04-15T14:51:11+09:00 GitHub noreply@github.com 2023-04-15T08:51:11+03:00 cmake : add finding the OpenBLAS header file (#992) c85e03d12e4b8af22cb13aa9c618dcd5935862fd 489093548c89c67520109ab25c4df4a4614a32a0 Pavol Rusnak pavol@rusnak.io 2023-04-14T21:58:43+02:00 GitHub noreply@github.com 2023-04-14T22:58:43+03:00 Revert "main : alternative instruct mode (Vicuna support, etc.) (#863)" (#982) 489093548c89c67520109ab25c4df4a4614a32a0 93265e988af32b8be314bfed334f795a3037555d Pavol Rusnak pavol@rusnak.io 2023-04-14T21:46:49+02:00 GitHub noreply@github.com 2023-04-14T19:46:49Z py : bump sentencepiece to 0.1.98 to support Python 3.11 (#976) 93265e988af32b8be314bfed334f795a3037555d c56b7152690ca25cfd66b20210b3629e6c1e739b Stephan Walter stephan@walter.name 2023-04-14T19:39:48Z GitHub noreply@github.com 2023-04-14T22:39:48+03:00 make : fix dependencies, use auto variables (#983) c56b7152690ca25cfd66b20210b3629e6c1e739b f4d277ae17247ee51129ef1a9ff74d377cc90b1b Pavol Rusnak pavol@rusnak.io 2023-04-14T20:05:37+02:00 GitHub noreply@github.com 2023-04-14T20:05:37+02:00 Expose type name from ggml (#970) f4d277ae17247ee51129ef1a9ff74d377cc90b1b c9a59b70a54e0bc05777df287feaea3dbe0310c4 Tomáš Pazdiora tomas.pazdiora@gmail.com 2023-04-14T17:19:17+02:00 GitHub noreply@github.com 2023-04-14T18:19:17+03:00 main : alternative instruct mode (Vicuna support, etc.) (#863) c9a59b70a54e0bc05777df287feaea3dbe0310c4 a32f7acc9f54dba1c728cb1e596bd00bf3b4eb5f Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-04-14T08:43:55-06:00 GitHub noreply@github.com 2023-04-14T17:43:55+03:00 ggml : add unary and binary map operations (#874) a32f7acc9f54dba1c728cb1e596bd00bf3b4eb5f 43ffdefb7424f79a3d510c199e2ea86684b4f824 Pavol Rusnak pavol@rusnak.io 2023-04-14T15:37:11+02:00 GitHub noreply@github.com 2023-04-14T15:37:11+02:00 py : cleanup dependencies (#962) 43ffdefb7424f79a3d510c199e2ea86684b4f824 1623a6e9b46453bff30afd7d0f6c3fd188499c2f Pavol Rusnak pavol@rusnak.io 2023-04-14T14:23:21+02:00 GitHub noreply@github.com 2023-04-14T14:23:21+02:00 py : fix flake8 and isort nitpicks (#960) 1623a6e9b46453bff30afd7d0f6c3fd188499c2f c14e0d2f23e6d1e785255f4da8c253c1b4723659 Georgi Gerganov ggerganov@gmail.com 2023-04-14T13:31:29+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-14T13:31:29+03:00 ggml : minor c14e0d2f23e6d1e785255f4da8c253c1b4723659 723dac55fa2ba7adc6e3fc8609781d1ad0378906 Georgi Gerganov ggerganov@gmail.com 2023-04-14T13:31:15+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-14T13:31:15+03:00 ggml : always allocate buffers with size multiple of GGML_MEM_ALIGN 723dac55fa2ba7adc6e3fc8609781d1ad0378906 0f07cacb05f49704d35a39aa27cfd4b419eb6f8d comex comexk@gmail.com 2023-04-14T00:03:03-07:00 GitHub noreply@github.com 2023-04-14T10:03:03+03:00 py : new conversion script (#545) 0f07cacb05f49704d35a39aa27cfd4b419eb6f8d c5d70f5c9ea5a8f0f6b0d6aa741455978a1dabfd Georgi Gerganov ggerganov@gmail.com 2023-04-14T09:45:42+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-14T09:45:42+03:00 ggml : fix q4_1 dot product types c5d70f5c9ea5a8f0f6b0d6aa741455978a1dabfd be87b6ed20a5f7528bf491a83e759a9fc6a24fea Howard Su howard0su@gmail.com 2023-04-14T14:24:52+08:00 GitHub noreply@github.com 2023-04-14T09:24:52+03:00 ggml : optimize rope function to avoid call powf in the tight loop (#807) be87b6ed20a5f7528bf491a83e759a9fc6a24fea 0e07e6a8399fd993739a3ba3c6f95f92bfab6f58 Gary Linscott glinscott@gmail.com 2023-04-13T14:50:42-07:00 GitHub noreply@github.com 2023-04-14T00:50:42+03:00 perplexity : add support for batch size to `--perplexity` (#407) 0e07e6a8399fd993739a3ba3c6f95f92bfab6f58 a3a2a0eda8828b60436e9f69d9ac2c1060d03e7a CRD716 crd716@gmail.com 2023-04-13T10:39:25-05:00 GitHub noreply@github.com 2023-04-13T18:39:25+03:00 common : remove unnecessary includes (#947) a3a2a0eda8828b60436e9f69d9ac2c1060d03e7a d990e3fffc5b0f5448e90a16c79a4f2675100af0 Georgi Gerganov ggerganov@gmail.com 2023-04-13T18:36:40+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-13T18:36:48+03:00 ggml : add GGML_DEFAULT_N_THREADS d990e3fffc5b0f5448e90a16c79a4f2675100af0 9190e8eac8bdc108c40d2d7505e9b45fa773251f Georgi Gerganov ggerganov@gmail.com 2023-04-13T18:32:36+03:00 GitHub noreply@github.com 2023-04-13T18:32:36+03:00 ggml : speed-up ggml_vec_dot_q4_1() ARM_NEON + 32-bit ARM support (#900) 9190e8eac8bdc108c40d2d7505e9b45fa773251f c85980acd04631a7c43d13676276f76ec72f5dfe Georgi Gerganov ggerganov@gmail.com 2023-04-13T18:04:45+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-13T18:04:45+03:00 llama : merge llama_internal.h into llama.h c85980acd04631a7c43d13676276f76ec72f5dfe 6232f2d7fd7a22d5eeb62182b2f21fcf01359754 Georgi Gerganov ggerganov@gmail.com 2023-04-13T18:01:22+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-13T18:01:33+03:00 gitignore : benchmark 6232f2d7fd7a22d5eeb62182b2f21fcf01359754 6c248707f51c8a50f7792e7f7787ec481881db88 Stephan Walter stephan@walter.name 2023-04-13T14:59:50Z GitHub noreply@github.com 2023-04-13T17:59:50+03:00 ggml : optimize non-SIMD Q4_0 vector dot product (#703) 6c248707f51c8a50f7792e7f7787ec481881db88 8cda5c981d0bf4dcb7664194b2cb9a06e2dbdd54 Pavol Rusnak pavol@rusnak.io 2023-04-13T16:08:32+02:00 GitHub noreply@github.com 2023-04-13T17:08:32+03:00 ggml : introduce GGML_ALIGNED_MALLOC/GGML_ALIGNED_FREE macros (#884) 8cda5c981d0bf4dcb7664194b2cb9a06e2dbdd54 ec29272175d7a79681d9919f3e755b1bcefa0478 CRD716 crd716@gmail.com 2023-04-13T09:03:57-05:00 GitHub noreply@github.com 2023-04-13T16:03:57+02:00 fix whitespace (#944) ec29272175d7a79681d9919f3e755b1bcefa0478 7e941b95eba067cb5b92785e642fd803657376ee CRD716 crd716@gmail.com 2023-04-13T08:59:53-05:00 GitHub noreply@github.com 2023-04-13T16:59:53+03:00 readme : remove python 3.10 warning (#929) 7e941b95eba067cb5b92785e642fd803657376ee c729ff730a46a135817a3d9988a097e3678a9722 Genkagaku.GPT hlhr202@163.com 2023-04-13T21:54:27+08:00 GitHub noreply@github.com 2023-04-13T16:54:27+03:00 readme : llama node binding (#911) c729ff730a46a135817a3d9988a097e3678a9722 4579af95e8e16910f6dbab0994917a5b3901f0cf Pavol Rusnak pavol@rusnak.io 2023-04-13T15:49:05+02:00 GitHub noreply@github.com 2023-04-13T15:49:05+02:00 flake.nix: add all binaries from bin (#848) 4579af95e8e16910f6dbab0994917a5b3901f0cf 8c3ffc2f048a372639906fb30ec3c2070288d3be Judd foldl@users.noreply.github.com 2023-04-13T21:43:22+08:00 GitHub noreply@github.com 2023-04-13T16:43:22+03:00 zig : update build.zig (#872) 8c3ffc2f048a372639906fb30ec3c2070288d3be 107980d970808c2ccf9334ad033e2782a560b911 Vladimir bogdad@gmail.com 2023-04-13T15:24:30+02:00 GitHub noreply@github.com 2023-04-13T16:24:30+03:00 ggml : update cblas_sgemm columns var to be more reasonable (#838) 107980d970808c2ccf9334ad033e2782a560b911 585d91a156794d30eec16ebe67c8d7a1d41406c1 niansa/tuxifan anton-sa@web.de 2023-04-13T15:03:39+02:00 GitHub noreply@github.com 2023-04-13T16:03:39+03:00 examples : add -n to alpaca and gpt4all scripts (#706) 585d91a156794d30eec16ebe67c8d7a1d41406c1 95ea26f6e92d620a5437f576b80868aee7f808d6 anzz1 anzz1@live.com 2023-04-13T15:48:21+03:00 GitHub noreply@github.com 2023-04-13T15:48:21+03:00 cmake : add explicit F16C option (x86) (#576) 95ea26f6e92d620a5437f576b80868aee7f808d6 82d146df9b43cf677e0dbce20b03cf864958a0cc SebastianApel 13675545+SebastianApel@users.noreply.github.com 2023-04-13T14:46:23+02:00 GitHub noreply@github.com 2023-04-13T15:46:23+03:00 benchmark : add tool for timing q4_0 matrix multiplication (#653) 82d146df9b43cf677e0dbce20b03cf864958a0cc e7f6997f897a18b6372a6460e25c5f89e1469f1d Pavol Rusnak pavol@rusnak.io 2023-04-13T11:33:16+02:00 GitHub noreply@github.com 2023-04-13T11:33:16+02:00 do not force the prompt file to end with a new line (#908) e7f6997f897a18b6372a6460e25c5f89e1469f1d f76cb3a34d6a6b03afb96650e39495f201eac042 Stephan Walter stephan@walter.name 2023-04-12T15:06:16Z GitHub noreply@github.com 2023-04-12T15:06:16Z Don't crash on ftype (formerly f16) == 4 (#917) f76cb3a34d6a6b03afb96650e39495f201eac042 782438070f7568380755ffc7bf5e09b20c1e8272 Georgi Gerganov ggerganov@gmail.com 2023-04-12T14:48:57+03:00 GitHub noreply@github.com 2023-04-12T14:48:57+03:00 readme : change "GPU support" link to discussion 782438070f7568380755ffc7bf5e09b20c1e8272 4dbbd407500cf500ca5f47e4e947635797997c05 Georgi Gerganov ggerganov@gmail.com 2023-04-12T14:31:12+03:00 GitHub noreply@github.com 2023-04-12T14:31:12+03:00 readme : update hot topics with link to "GPU support" issue 4dbbd407500cf500ca5f47e4e947635797997c05 8b679987cdce292ff36bd741f6715e4927e26f9b Nicolai Weitkemper kontakt@nicolaiweitkemper.de 2023-04-12T08:46:20+02:00 GitHub noreply@github.com 2023-04-12T08:46:20+02:00 readme: link to sha256sums file (#902) 8b679987cdce292ff36bd741f6715e4927e26f9b 3e6e70d8e8917b5bd14c7c9f9b89a585f1ff0b31 Pavol Rusnak pavol@rusnak.io 2023-04-11T21:45:44+02:00 GitHub noreply@github.com 2023-04-11T19:45:44Z Fix whitespace, add .editorconfig, add GitHub workflow (#883) 3e6e70d8e8917b5bd14c7c9f9b89a585f1ff0b31 2663d2c6784ad7b77998c6874df25648d597f74b Stephan Walter stephan@walter.name 2023-04-11T15:03:51Z GitHub noreply@github.com 2023-04-11T15:03:51Z Add enum llama_ftype, sync ggml_type to model files (#709) 2663d2c6784ad7b77998c6874df25648d597f74b a0caa34b162449b5c13b8d604573053300ff54a1 comex comexk@gmail.com 2023-04-11T06:19:54-07:00 GitHub noreply@github.com 2023-04-11T15:19:54+02:00 Windows fixes (#890) a0caa34b162449b5c13b8d604573053300ff54a1 461ba9e66ed3885f80680d71495e055580573c74 qouoq qouoq@fastmail.com 2023-04-11T04:41:53+08:00 GitHub noreply@github.com 2023-04-10T22:41:53+02:00 Add BAIR's Koala to supported models (#877) 461ba9e66ed3885f80680d71495e055580573c74 c3ac702e5ee3533457e0489df4906ee112fe88e7 Georgi Gerganov ggerganov@gmail.com 2023-04-10T23:20:01+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-10T23:20:01+03:00 ggml : fix WASM build c3ac702e5ee3533457e0489df4906ee112fe88e7 9d634ef452d0fc24fcd49592952d13d0ab0f41b7 Georgi Gerganov ggerganov@gmail.com 2023-04-10T22:40:28+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-10T22:42:28+03:00 ggml : add ggml_cont() + optimize ggml_cpy() for contiguous dst 9d634ef452d0fc24fcd49592952d13d0ab0f41b7 d9a239c4104c888eafda672c1e42c9bbc5084cb8 Georgi Gerganov ggerganov@gmail.com 2023-04-10T19:32:45+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-10T22:42:28+03:00 ggml : remove trailing whitespaces d9a239c4104c888eafda672c1e42c9bbc5084cb8 684da25926e5c505f725b4f10b5485b218fa1fc7 Marco Matthies 71844+marcom@users.noreply.github.com 2023-04-10T19:57:59+02:00 GitHub noreply@github.com 2023-04-10T19:57:59+02:00 Simplify to include lower-case windows.h always, fix compile on mingw32 (#747) 684da25926e5c505f725b4f10b5485b218fa1fc7 180b693a47b6b825288ef9f2c39d24b6eea4eea6 Georgi Gerganov ggerganov@gmail.com 2023-04-10T19:29:48+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-10T19:29:48+03:00 ggml : fix quantize_row_q4_1() ARM_NEON (close #876) 180b693a47b6b825288ef9f2c39d24b6eea4eea6 f963b63afa0e057cfb9eba4d88407c6a0850a0d8 comex comexk@gmail.com 2023-04-08T13:08:21-07:00 Matvey Soloviev blackhole89@gmail.com 2023-04-10T01:10:46+02:00 Print model version. f963b63afa0e057cfb9eba4d88407c6a0850a0d8 aaf3b23debc1fe1a06733c8c6468fb84233cc44f comex comexk@gmail.com 2023-04-08T12:24:37-07:00 Matvey Soloviev blackhole89@gmail.com 2023-04-10T01:10:46+02:00 Rewrite loading code to try to satisfy everyone: aaf3b23debc1fe1a06733c8c6468fb84233cc44f f2d1c472946dee2aba9077e8df73346796752b10 Tomáš Pazdiora tomas.pazdiora@gmail.com 2023-04-08T17:49:39+02:00 GitHub noreply@github.com 2023-04-08T17:49:39+02:00 fix for windows utf-8 input (#840) f2d1c472946dee2aba9077e8df73346796752b10 317fb12fbd7cef5d86476574bffe0e904af884ca eiery 19350831+eiery@users.noreply.github.com 2023-04-08T07:15:17-04:00 GitHub noreply@github.com 2023-04-08T11:15:17Z cmake should link openblas properly with -lopenblas like how it's done in the makefile (#839) 317fb12fbd7cef5d86476574bffe0e904af884ca 62cfc54f77e519057110265b52b0d614fa363e2a lon 114724657+longregen@users.noreply.github.com 2023-04-08T07:04:23-03:00 GitHub noreply@github.com 2023-04-08T12:04:23+02:00 Add new binaries to flake.nix (#847) 62cfc54f77e519057110265b52b0d614fa363e2a 698f7b5d6316a1f8453b3b32fd0d637d24952ffd unbounded haakon@likedan.net 2023-04-08T00:09:18+02:00 GitHub noreply@github.com 2023-04-08T00:09:18+02:00 Add quantize-stats command for testing quantization (#728) 698f7b5d6316a1f8453b3b32fd0d637d24952ffd c1950c343109ab1fd15fc2ae1c83650c85d4eeef bhubbb 79117352+bhubbb@users.noreply.github.com 2023-04-08T02:11:58+10:00 GitHub noreply@github.com 2023-04-07T19:11:58+03:00 make : add libllama.so target for llama-cpp-python (#797) c1950c343109ab1fd15fc2ae1c83650c85d4eeef 4953e9007f86327aabc8312a7211c18019a3a40e iacore 74560659+iacore@users.noreply.github.com 2023-04-07T16:05:29Z GitHub noreply@github.com 2023-04-07T19:05:29+03:00 zig : don't link examples/common.cpp for non-example (#814) 4953e9007f86327aabc8312a7211c18019a3a40e cc9cee8e9e7598bd280295f6264f36d3a9224006 Ivan Stepanov ivanstepanovftw@gmail.com 2023-04-07T19:02:12+03:00 GitHub noreply@github.com 2023-04-07T19:02:12+03:00 llama : always sort logits before nucleus sampling (#812) cc9cee8e9e7598bd280295f6264f36d3a9224006 d2beca95dcfcd6f1145886e914b879ffc3604b7a Sergey Alirzaev zl29ah@gmail.com 2023-04-06T17:59:11+02:00 GitHub noreply@github.com 2023-04-06T17:59:11+02:00 Do not crash when it has nothing to say. (#796) d2beca95dcfcd6f1145886e914b879ffc3604b7a eeaa7b0492fc79baab8bb1fe195d6c87159f2bd3 Pavol Rusnak pavol@rusnak.io 2023-04-06T08:56:58+02:00 GitHub noreply@github.com 2023-04-06T08:56:58+02:00 Make docker instructions more explicit (#785) eeaa7b0492fc79baab8bb1fe195d6c87159f2bd3 986b6ce9f99503c51ec5afd8a10baa32359434c6 Georgi Gerganov ggerganov@gmail.com 2023-04-05T22:11:03+03:00 GitHub noreply@github.com 2023-04-05T22:11:03+03:00 ggml : multi-thread ggml_rope() (~3-4 times faster on M1) (#781) 986b6ce9f99503c51ec5afd8a10baa32359434c6 34162989297fdfe3ab7305451ce55bc87e3f4c9c Georgi Gerganov ggerganov@gmail.com 2023-04-05T22:07:33+03:00 GitHub noreply@github.com 2023-04-05T22:07:33+03:00 ggml, llama : avoid heavy V transpose + improvements (#775) 34162989297fdfe3ab7305451ce55bc87e3f4c9c 5a8c4f624077373a198cd562146ffa67b02ebc75 Georgi Gerganov ggerganov@gmail.com 2023-04-05T19:54:30+03:00 GitHub noreply@github.com 2023-04-05T19:54:30+03:00 Update README.md 5a8c4f624077373a198cd562146ffa67b02ebc75 ff05d05c960076b42f027e4d318cbd6ea59b3030 Ivan Stepanov ivanstepanovftw@gmail.com 2023-04-05T19:20:05+03:00 GitHub noreply@github.com 2023-04-05T19:20:05+03:00 llama : define non-positive top_k; top_k range check (#779) ff05d05c960076b42f027e4d318cbd6ea59b3030 62b3e81aaeafb282934de8b21de13b0104f12f8c at8u 129688334+at8u@users.noreply.github.com 2023-04-05T15:59:13Z GitHub noreply@github.com 2023-04-05T18:59:13+03:00 miku.sh : add executable bit (#780) 62b3e81aaeafb282934de8b21de13b0104f12f8c 8d10406d6ee230e6c1a96590cc8273f86ae606f8 Georgi Gerganov ggerganov@gmail.com 2023-04-05T18:58:06+03:00 Georgi Gerganov ggerganov@gmail.com 2023-04-05T18:58:31+03:00 media : add logos and banners 8d10406d6ee230e6c1a96590cc8273f86ae606f8 ed1c214e667d58ac442b3c6664831fe0eed2941d Georgi Gerganov ggerganov@gmail.com 2023-04-05T18:56:20+03:00 GitHub noreply@github.com 2023-04-05T18:56:20+03:00 readme : change logo + add bindings + add uis + add wiki ed1c214e667d58ac442b3c6664831fe0eed2941d 0c44427df10ee024b4e7ef7bfec56e993daff1db iacore 74560659+iacore@users.noreply.github.com 2023-04-05T15:06:02Z GitHub noreply@github.com 2023-04-05T18:06:02+03:00 zig : add build.zig (#773) 0c44427df10ee024b4e7ef7bfec56e993daff1db 594cc95fabab0b662dabba3ea619ca5dca18bf6b Ivan Stepanov ivanstepanovftw@gmail.com 2023-04-05T17:38:37+03:00 GitHub noreply@github.com 2023-04-05T17:38:37+03:00 make : missing host optimizations in CXXFLAGS (#763) 594cc95fabab0b662dabba3ea619ca5dca18bf6b 88ed5761b869a221bc26847ff3f6977e7ee6425e Adithya Balaji adithya.b94@gmail.com 2023-04-05T16:36:12+02:00 GitHub noreply@github.com 2023-04-05T17:36:12+03:00 readme : update with CMake and windows example (#748) 88ed5761b869a221bc26847ff3f6977e7ee6425e 58c438cf7dfbbef710b1905a453a38a8a9ced07d at8u 129688334+at8u@users.noreply.github.com 2023-04-05T14:32:42Z GitHub noreply@github.com 2023-04-05T17:32:42+03:00 examples : add Miku.sh (#724) 58c438cf7dfbbef710b1905a453a38a8a9ced07d 53dbba769537e894ead5c6913ab2fd3a4658b738 Andrew Duffy a10y@users.noreply.github.com 2023-04-05T11:44:24+01:00 GitHub noreply@github.com 2023-04-05T06:44:24-04:00 Add Accelerate/BLAS when using Swift (#765) 53dbba769537e894ead5c6913ab2fd3a4658b738 437e77855a54e69c86fe03bc501f63d9a3fddb0e mgroeber9110 45620825+mgroeber9110@users.noreply.github.com 2023-04-03T18:00:55+02:00 GitHub noreply@github.com 2023-04-03T18:00:55+02:00 Windows: reactive sigint handler after each Ctrl-C (#736) 437e77855a54e69c86fe03bc501f63d9a3fddb0e cd7fa956904cb8e321b72b3499f4a3a82e43c266 SebastianApel 13675545+SebastianApel@users.noreply.github.com 2023-04-03T09:52:28+02:00 GitHub noreply@github.com 2023-04-03T09:52:28+02:00 10+% performance improvement of ggml_vec_dot_q4_0 on AVX2 (#654) cd7fa956904cb8e321b72b3499f4a3a82e43c266 a0c05164168297c04737936ad0cad849a512547a Ivan Stepanov ivanstepanovftw@gmail.com 2023-04-03T03:19:04+03:00 GitHub noreply@github.com 2023-04-03T02:19:04+02:00 Define non-positive temperature behavior (#720) a0c05164168297c04737936ad0cad849a512547a d8d4e865cd481b18f10508ffee35db903767ef5c bsilvereagle bsilvereagle@users.noreply.github.com 2023-04-02T15:13:03-07:00 GitHub noreply@github.com 2023-04-03T00:13:03+02:00 Remove torch GPU dependencies from the Docker.full image (#665) d8d4e865cd481b18f10508ffee35db903767ef5c e986f94829bae0b9e66b326acbbba179931c84f1 Thatcher Chamberlin j.thatcher.c@gmail.com 2023-04-02T06:48:57-04:00 GitHub noreply@github.com 2023-04-02T12:48:57+02:00 Add a missing step to the gpt4all instructions (#690) e986f94829bae0b9e66b326acbbba179931c84f1 c0bb1d3ce21005ab21d686626ba87261a6e3a660 Christian Falch 875252+chrfalch@users.noreply.github.com 2023-04-02T12:23:04+02:00 GitHub noreply@github.com 2023-04-02T12:23:04+02:00 Added api for getting/setting the kv_cache (#685) c0bb1d3ce21005ab21d686626ba87261a6e3a660 6e7801d08d81c931a5427bae46f00763e993f54a Marian Cepok marian.cepok@gmail.com 2023-04-02T12:21:31+02:00 GitHub noreply@github.com 2023-04-02T13:21:31+03:00 ggml : change ne to int64_t (#626) 6e7801d08d81c931a5427bae46f00763e993f54a 81040f10aae3160317c5787c9c59acb219927826 Leonardo Neumann leonardo@neumann.dev.br 2023-04-02T04:56:20-03:00 GitHub noreply@github.com 2023-04-02T10:56:20+03:00 examples : add gpt4all script (#658) 81040f10aae3160317c5787c9c59acb219927826 c4f89d8d73aab4318a6c61e3835135adfcf55407 Stephan Walter stephan@walter.name 2023-04-02T07:18:53Z GitHub noreply@github.com 2023-04-02T10:18:53+03:00 llama : do not allocate KV cache for "vocab_only == true" (#682) c4f89d8d73aab4318a6c61e3835135adfcf55407 5b70e7de4c0b8186669d0c5609ba61a2d46de562 Fabian cmdrf@users.noreply.github.com 2023-04-02T09:17:05+02:00 GitHub noreply@github.com 2023-04-02T10:17:05+03:00 make : use -march=native -mtune=native on x86 (#609) 5b70e7de4c0b8186669d0c5609ba61a2d46de562 a717cba8440b380f43cd3e2510862fc1ea3de9a2 Murilo Santana mvrilo@gmail.com 2023-04-01T23:41:12-03:00 GitHub noreply@github.com 2023-04-02T04:41:12+02:00 fix default params for examples/main (#697) a717cba8440b380f43cd3e2510862fc1ea3de9a2 d0a7f742e76bb48c0bd852f0b3bf09ec0b75b200 Ikko Eltociear Ashimine eltociear@gmail.com 2023-04-02T01:38:18+09:00 GitHub noreply@github.com 2023-04-01T18:38:18+02:00 py: huggingface -> Hugging Face (#686) d0a7f742e76bb48c0bd852f0b3bf09ec0b75b200 0d054e292e5492981867be69c788edd04dc8adeb rimoliga 53384203+rimoliga@users.noreply.github.com 2023-04-01T11:57:30-03:00 GitHub noreply@github.com 2023-04-01T16:57:30+02:00 readme: replace termux links with homepage, play store is deprecated (#680) 0d054e292e5492981867be69c788edd04dc8adeb 3525899277d2e2bdc8ec3f0e6e40c47251608700 Slaren 2141330+slaren@users.noreply.github.com 2023-03-31T20:03:48+02:00 Pavol Rusnak pavol@rusnak.io 2023-04-01T16:08:40+02:00 Show error message when -f fails 3525899277d2e2bdc8ec3f0e6e40c47251608700 1d08882afa647c44195f4f6495a68ea455650cae Stephan Walter stephan@walter.name 2023-03-31T19:19:16Z GitHub noreply@github.com 2023-03-31T19:19:16Z Enable -std= for cmake builds, fix warnings (#598) 1d08882afa647c44195f4f6495a68ea455650cae 02c5b27e91a6d18cf1043d3a2d8dbc59610ac257 slaren 2141330+slaren@users.noreply.github.com 2023-03-31T17:55:52+02:00 GitHub noreply@github.com 2023-03-31T15:55:52Z Optimize AVX2 ggml_vec_dot_q4_0 (#642) 02c5b27e91a6d18cf1043d3a2d8dbc59610ac257 cbef542879962fdc491656cd0c8cadd65a5f1356 perserk perserk@gmail.com 2023-03-31T16:55:44+05:00 GitHub noreply@github.com 2023-03-31T13:55:44+02:00 Add AVX acceleration (#617) cbef542879962fdc491656cd0c8cadd65a5f1356 9733104be5389ebb1ff05095eca2a70280cd875a Pavol Rusnak pavol@rusnak.io 2023-03-29T21:31:24+02:00 Pavol Rusnak pavol@rusnak.io 2023-03-31T10:32:01+02:00 py : cleanup the code 9733104be5389ebb1ff05095eca2a70280cd875a 3df890aef432ce68143cfafcd7caf828bc4c3e55 Pavol Rusnak pavol@rusnak.io 2023-03-31T00:52:06+02:00 Pavol Rusnak pavol@rusnak.io 2023-03-31T01:07:32+02:00 drop quantize.py (now that models are using a single file) 3df890aef432ce68143cfafcd7caf828bc4c3e55 ee0c40dd6de8c3c658ae43199939ef40bb1cf408 Georgi Gerganov ggerganov@gmail.com 2023-03-30T22:31:54+03:00 GitHub noreply@github.com 2023-03-30T22:31:54+03:00 readme : update supported models ee0c40dd6de8c3c658ae43199939ef40bb1cf408 6f23ba5ee235cbcb1eedd63b98422dd8d4392a78 Justine Tunney jtunney@gmail.com 2023-03-30T05:42:56-07:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Introduce GGML migration tool for new file format 6f23ba5ee235cbcb1eedd63b98422dd8d4392a78 78ca9838ee36660a776e97e3391b6fb5dcaacf7f Justine Tunney jtunney@gmail.com 2023-03-30T01:53:36-07:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Ensure --mlock works properly with mmap() support 78ca9838ee36660a776e97e3391b6fb5dcaacf7f a017390358cdb23fffb30988dc84bb190d0403ca Justine Tunney jtunney@gmail.com 2023-03-29T13:51:37-07:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Make loading weights 10-100x faster a017390358cdb23fffb30988dc84bb190d0403ca ac184d514723902f9b05b688703b1be6e8dc65de Slaren 2141330+slaren@users.noreply.github.com 2023-03-29T22:22:36+02:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Initial windows support (untested) ac184d514723902f9b05b688703b1be6e8dc65de 276e5b781155e3bbe6834472c58f03dfe62efabe Slaren 2141330+slaren@users.noreply.github.com 2023-03-29T08:53:14+02:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Always initialize mm_addr and mm_length in llama_model 276e5b781155e3bbe6834472c58f03dfe62efabe d68c5dc4356c8f49e933df210f2ceca5002a8118 Slaren 2141330+slaren@users.noreply.github.com 2023-03-29T08:31:26+02:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Unmap the file in llama_free d68c5dc4356c8f49e933df210f2ceca5002a8118 64bde3ffd4aef799acb790a3eedddbd0a0612108 Slaren 2141330+slaren@users.noreply.github.com 2023-03-29T06:18:18+02:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Make mmap_file static 64bde3ffd4aef799acb790a3eedddbd0a0612108 c03ae8dca1d7c451054754979e60a6de1f64c3cd Slaren 2141330+slaren@users.noreply.github.com 2023-03-29T05:38:57+02:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Fix ggml_init_params in quantize c03ae8dca1d7c451054754979e60a6de1f64c3cd 3bcc129ba881c99795e850b0a23707a4dfdabe9d Slaren 2141330+slaren@users.noreply.github.com 2023-03-29T02:03:43+02:00 Justine Tunney jtunney@gmail.com 2023-03-30T12:28:25-07:00 Add mmap support for model files 3bcc129ba881c99795e850b0a23707a4dfdabe9d a4755cf288deb83df646f91f8fc98613271322db Stephan Walter stephan@walter.name 2023-03-30T17:56:59Z GitHub noreply@github.com 2023-03-30T20:56:59+03:00 cmake : properly invoke CTest (#629) a4755cf288deb83df646f91f8fc98613271322db 1f0414feecc336482163af6c1e5650f9373ed8c9 Casey Primozic me@ameo.link 2023-03-30T10:53:35-07:00 GitHub noreply@github.com 2023-03-30T17:53:35Z Remove unused variable (#607) 1f0414feecc336482163af6c1e5650f9373ed8c9 77efdf5a501b1140801da5cd8751e9f9b259ec32 david raistrick keen99@users.noreply.github.com 2023-03-30T13:34:45-04:00 GitHub noreply@github.com 2023-03-30T20:34:45+03:00 make : fix darwin f16c flags check (#615) 77efdf5a501b1140801da5cd8751e9f9b259ec32 ed3c680bcd0e8ce6e574573ba95880b694449878 Georgi Gerganov ggerganov@gmail.com 2023-03-30T20:27:32+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-30T20:27:32+03:00 ggml : fix NEON signs (close #620, #622) ed3c680bcd0e8ce6e574573ba95880b694449878 9cbc404ba6699a9ba4925ea25a60552b13491c7a slaren 2141330+slaren@users.noreply.github.com 2023-03-30T11:16:30+02:00 GitHub noreply@github.com 2023-03-30T11:16:30+02:00 Fix GGML_F32Cx8_STORE in AVX without F16C path (#619) 9cbc404ba6699a9ba4925ea25a60552b13491c7a b51c717d5cf9181c33afcb84554e47f6d539c891 anzz1 anzz1@live.com 2023-03-29T23:44:39+03:00 GitHub noreply@github.com 2023-03-29T23:44:39+03:00 ci : re-enable AVX512 testing (Windows-MSVC) (#584) b51c717d5cf9181c33afcb84554e47f6d539c891 0ba76c1e73ae21038b80bfb5a746157376c88173 Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:15:34+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:15:34+03:00 ggml : init time on first ggml_init() call 0ba76c1e73ae21038b80bfb5a746157376c88173 cea1c859483a5cfc7e2b31a06f8561d7a7604870 Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:13:12+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:13:12+03:00 llama : fix compile warnings when reading the vocab cea1c859483a5cfc7e2b31a06f8561d7a7604870 f202ada131f60059112a948f660b2e0ac93d049a Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:10:01+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:10:01+03:00 ggml : add ARM_NEON dequantize_row_q4_1() f202ada131f60059112a948f660b2e0ac93d049a 3b44d30d9b618f0f2eb9abcfe912770a4e7d85d4 Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:03:02+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:03:07+03:00 ggml : add ARM_NEON quantize_row_q4_1() 3b44d30d9b618f0f2eb9abcfe912770a4e7d85d4 61cbfff5c95e45236883b1b60e025f8f6fa8c8a3 Georgi Gerganov ggerganov@gmail.com 2023-03-29T21:47:33+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-29T22:03:07+03:00 ggml : add ARM_NEON ggml_vec_dot_q4_1() 61cbfff5c95e45236883b1b60e025f8f6fa8c8a3 d9ad104440d84a0cc0734bff47ef0ba41ba740c4 Pavol Rusnak pavol@rusnak.io 2023-03-29T20:09:25+02:00 GitHub noreply@github.com 2023-03-29T20:09:25+02:00 rename convert_ggml_to_pth.py -> convert-ggml-to-pth.py (#600) d9ad104440d84a0cc0734bff47ef0ba41ba740c4 b467702b87461543c75013207e9adc6d20dcc01d Thérence 13496987+Royalphax@users.noreply.github.com 2023-03-29T19:21:09+02:00 GitHub noreply@github.com 2023-03-29T20:21:09+03:00 Create chat-13B.bat (#592) b467702b87461543c75013207e9adc6d20dcc01d 516d88e75c9e768c0001a452dbad212494c586b3 Georgi Gerganov ggerganov@gmail.com 2023-03-29T19:38:31+03:00 GitHub noreply@github.com 2023-03-29T19:38:31+03:00 readme : fix typos 516d88e75c9e768c0001a452dbad212494c586b3 53635c081c49321d523567112f9fddfbba6b787b Georgi Gerganov ggerganov@gmail.com 2023-03-29T19:37:20+03:00 GitHub noreply@github.com 2023-03-29T19:37:20+03:00 readme : add GPT4All instructions (close #588) 53635c081c49321d523567112f9fddfbba6b787b 41318d708ed196ff727dce14d263a64b23c7333d Georgi Gerganov ggerganov@gmail.com 2023-03-29T19:29:26+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-29T19:29:52+03:00 py : add GPT4All conversion script 41318d708ed196ff727dce14d263a64b23c7333d a6956b25a1c783e5e96fe06c9c00438f846ef047 Maël Kerbiriou m431.kerbiriou@gmail.com 2023-03-29T18:10:07+02:00 GitHub noreply@github.com 2023-03-29T19:10:07+03:00 llama : use the same threshold for OpenBLAS and ggml thread limiting (#577) a6956b25a1c783e5e96fe06c9c00438f846ef047 83df5639eb182ed7c122382907691d8baa3c32df Tobias Lütke tobi@shopify.com 2023-03-29T17:10:24+02:00 GitHub noreply@github.com 2023-03-29T10:10:24-05:00 add example of re-act pattern (#583) 83df5639eb182ed7c122382907691d8baa3c32df a5c42c4b13b3be9e58fe8f9adbb6ee60417674a6 anzz1 anzz1@live.com 2023-03-29T16:20:07+03:00 GitHub noreply@github.com 2023-03-29T13:20:07Z Fix GCC warning about binary literal (#595) a5c42c4b13b3be9e58fe8f9adbb6ee60417674a6 5a5f8b1501fbb34367225544010ddfc306d6d2fe anzz1 anzz1@live.com 2023-03-29T16:19:29+03:00 GitHub noreply@github.com 2023-03-29T13:19:29Z Fix typo in llama.h (#593) 5a5f8b1501fbb34367225544010ddfc306d6d2fe f1217055eaedfc7214be93d98e529cae89830430 anzz1 anzz1@live.com 2023-03-28T22:44:29+03:00 GitHub noreply@github.com 2023-03-28T22:44:29+03:00 Enable Fused-Multiply-Add (FMA) and F16C/CVT16 vector extensions on MSVC (#375) f1217055eaedfc7214be93d98e529cae89830430 7f4c5c66514227c3870c2bd189fb0609fdd0de10 anzz1 anzz1@live.com 2023-03-28T22:43:25+03:00 GitHub noreply@github.com 2023-03-28T22:43:25+03:00 CI: fix subdirectory path globbing (#546) 7f4c5c66514227c3870c2bd189fb0609fdd0de10 2a98bc18ea34dbf15f261a0df37080e588a189d1 anzz1 anzz1@live.com 2023-03-28T21:23:09+03:00 GitHub noreply@github.com 2023-03-28T21:23:09+03:00 llama : fix linkage with mingw (#551) 2a98bc18ea34dbf15f261a0df37080e588a189d1 d0aaff571cd5c316b68e3e11d57e274bfd2bd457 slaren 2141330+slaren@users.noreply.github.com 2023-03-28T20:06:03+02:00 GitHub noreply@github.com 2023-03-28T21:06:03+03:00 ggml : add AVX2 implementation of quantize_row_q4_1 (#515) d0aaff571cd5c316b68e3e11d57e274bfd2bd457 d0330fd783d7c67349cdcce4a56604ef0aeccdb5 thement 40525767+thement@users.noreply.github.com 2023-03-28T19:55:42+02:00 GitHub noreply@github.com 2023-03-28T20:55:42+03:00 py : add temporary script to convert old ggml files to newer version (#539) d0330fd783d7c67349cdcce4a56604ef0aeccdb5 99c5b2765422232ebb4414f5a63693d734406a7f Tai Duc Nguyen taiducnguyen.drexel@gmail.com 2023-03-28T13:51:29-04:00 GitHub noreply@github.com 2023-03-28T20:51:29+03:00 py : add capabiliy to convert from ggml back to torch or hf format for further consumption/training/finetuning (#403) 99c5b2765422232ebb4414f5a63693d734406a7f 692ce3164ef1201ecb9cfad315cc0a08b965adb8 Stephan Walter stephan@walter.name 2023-03-28T17:13:01Z GitHub noreply@github.com 2023-03-28T20:13:01+03:00 ggml : refactor quantized processing functions (#509) 692ce3164ef1201ecb9cfad315cc0a08b965adb8 96f9c0506fa81cada6f96f45768c34f45406c4bb DooWoong Lee (David) manics99@naver.com 2023-03-29T02:02:34+09:00 GitHub noreply@github.com 2023-03-28T20:02:34+03:00 py : removed unused `model` variable and verified that the code functions correctly with `vocab_only` setting. Also confirmed that the code works as expected after running with reduced memory usage due to deletion of no-longer-needed variable. (#547) 96f9c0506fa81cada6f96f45768c34f45406c4bb d502bc7c9d9d6dfb3a09aea404395b666d7b374d Georgi Gerganov ggerganov@gmail.com 2023-03-28T20:01:09+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-28T20:01:09+03:00 ci : make ctest verbose, hopefully we see what is wrong with the sanitizer d502bc7c9d9d6dfb3a09aea404395b666d7b374d 436e56193199a1625f8c561069f702e8840a9e08 Georgi Gerganov ggerganov@gmail.com 2023-03-28T19:51:55+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-28T19:51:55+03:00 tests : free llama context at the end of the test 436e56193199a1625f8c561069f702e8840a9e08 20e1e84884376b3fb44ffbfd48d478b2934b0b5e Stephan Walter stephan@walter.name 2023-03-28T16:48:20Z GitHub noreply@github.com 2023-03-28T19:48:20+03:00 all : be more strict about converting float to double (#458) 20e1e84884376b3fb44ffbfd48d478b2934b0b5e c1f885067c61191a07a1aedf684168dda62f3f71 Jed Fox git@jedfox.com 2023-03-28T11:39:01-05:00 GitHub noreply@github.com 2023-03-28T19:39:01+03:00 deploy : add a Package.swift for SwiftPM support (#393) c1f885067c61191a07a1aedf684168dda62f3f71 e0670260fb50a882b37074112b1881fb0820cf77 Stephan Walter stephan@walter.name 2023-03-28T15:56:03Z GitHub noreply@github.com 2023-03-28T18:56:03+03:00 ggml : introduce structs for the q4 data blocks (#356) e0670260fb50a882b37074112b1881fb0820cf77 28ba975aea1dcae2f31770516f5d542ff177771e Georgi Gerganov ggerganov@gmail.com 2023-03-28T18:34:35+03:00 Georgi Gerganov ggerganov@gmail.com 2023-03-28T18:34:35+03:00 gitignore : add "embedding" 28ba975aea1dcae2f31770516f5d542ff177771e a6bdc47cba23713a22ade47dd65b6afeb8009ff4 dotpy314 33351922+dotpy314@users.noreply.github.com 2023-03-28T23:06:28+08:00 GitHub noreply@github.com 2023-03-28T18:06:28+03:00 Check the existence of f16_model_path_base in quantize.py (#574) a6bdc47cba23713a22ade47dd65b6afeb8009ff4 7b8dbcb78b2f65c4676e41da215800d65846edd0 slaren 2141330+slaren@users.noreply.github.com 2023-03-28T16:26:55+02:00 GitHub noreply@github.com 2023-03-28T17:26:55+03:00 Fix usage of F16C intrinsics in AVX code (#563) 7b8dbcb78b2f65c4676e41da215800d65846edd0 4b8efff0e3945090379aa2f897ff125c8f9cdbae anzz1 anzz1@live.com 2023-03-28T17:09:55+03:00 GitHub noreply@github.com 2023-03-28T17:09:55+03:00 main.cpp fixes, refactoring (#571) 4b8efff0e3945090379aa2f897ff125c8f9cdbae 7e5395575a3360598f2565c73c8a2ec0c0abbdb8 RJ Adriaansen adriaansen@eshcc.eur.nl 2023-03-28T08:11:09+02:00 GitHub noreply@github.com 2023-03-28T09:11:09+03:00 Add embedding example to Makefile (#540) 7e5395575a3360598f2565c73c8a2ec0c0abbdb8 34c1072e497eb92d81ee7c0e12aa6741496a41c6 Marco Matthies 71844+marcom@users.noreply.github.com 2023-03-27T06:55:26+02:00 GitHub noreply@github.com 2023-03-27T07:55:26+03:00 Fix missing ggml link in cmake for examples/* on w64-mingw32 (#542) 34c1072e497eb92d81ee7c0e12aa6741496a41c6 939ad2d3a56815f480b6fd5ea432a7ee576a7e6b Erik Scholz Green-Sky@users.noreply.github.com 2023-03-26T17:48:40+02:00 GitHub noreply@github.com 2023-03-26T15:48:40Z ci: add debug build to sanitizer build matrix (#527) 939ad2d3a56815f480b6fd5ea432a7ee576a7e6b 8c2ec5e21d580c99e257c3cfddcf21fa53229aa4 Stephan Walter stephan@walter.name 2023-03-26T15:34:02Z GitHub noreply@github.com 2023-03-26T15:34:02Z Fix undefined variables in debug build, remove unused variables (#531) 8c2ec5e21d580c99e257c3cfddcf21fa53229aa4 b391579db92f095666be1d979899b54ae0981573 Juan Calderon-Perez 835733+gaby@users.noreply.github.com 2023-03-26T10:48:42-04:00 GitHub noreply@github.com 2023-03-26T14:48:42Z Add support for linux/arm64 platform during Docker Builds (#514) b391579db92f095666be1d979899b54ae0981573 7a87d31f4f0c37bbb2ea695929fa4fe3ad579cda Stephan Walter stephan@walter.name 2023-03-26T13:14:01Z GitHub noreply@github.com 2023-03-26T16:14:01+03:00 Update README and comments for standalone perplexity tool (#525) 7a87d31f4f0c37bbb2ea695929fa4fe3ad579cda 348d6926ee31d4476f9b90e1a627b0925a70f847 anzz1 anzz1@live.com 2023-03-26T16:06:10+03:00 GitHub noreply@github.com 2023-03-26T16:06:10+03:00 [main] fix infinite generation (-n == -1) (#523) 348d6926ee31d4476f9b90e1a627b0925a70f847 33e35b8fe8f09adcac0632e9cece62e1dd629f7d Georgi Gerganov ggerganov@gmail.com 2023-03-26T10:20:49+03:00 GitHub noreply@github.com 2023-03-26T10:20:49+03:00 Add logo to README.md 33e35b8fe8f09adcac0632e9cece62e1dd629f7d 19726169b379bebc96189673a19b89ab1d307659 Harald Fernengel harald.fernengel@here.com 2023-03-26T07:25:46+02:00 GitHub noreply@github.com 2023-03-26T08:25:46+03:00 Exit from interactive mode if input stream is bad (#491) 19726169b379bebc96189673a19b89ab1d307659 f732695cd57fb41e3a1be625cec4edf5be45b40a anzz1 anzz1@live.com 2023-03-26T00:13:28+02:00 GitHub noreply@github.com 2023-03-26T00:13:28+02:00 CI: Run other sanitizer builds even if one fails (#511) f732695cd57fb41e3a1be625cec4edf5be45b40a 2f7bf7dd7cd7299874d582f7f34834418abf4057 jp-x-g jpxg-dev@protonmail.com 2023-03-25T14:53:55-07:00 GitHub noreply@github.com 2023-03-25T23:53:55+02:00 Clarify console output in convert-pth-to-ggml.py (#512) 2f7bf7dd7cd7299874d582f7f34834418abf4057 34ab5268432fd287caa68d60bdd8aef411def3fa anzz1 anzz1@live.com 2023-03-25T23:38:11+02:00 GitHub noreply@github.com 2023-03-25T23:38:11+02:00 CMake / CI additions (#497) 34ab5268432fd287caa68d60bdd8aef411def3fa c2b25b6912662d2637d9c6e6df3a5de931e0d7ce anzz1 anzz1@live.com 2023-03-25T22:29:22+02:00 GitHub noreply@github.com 2023-03-25T22:29:22+02:00 (Windows) Set console to UTF-8 on init (#420) c2b25b6912662d2637d9c6e6df3a5de931e0d7ce 79b2b266db6b198b5af450982c3cd61120fac951 Georgi Gerganov ggerganov@gmail.com 2023-03-25T21:53:39+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T21:53:39+02:00 Fix colors enabling on WIN32 79b2b266db6b198b5af450982c3cd61120fac951 e2d490dafd860eaaaf9aa8008ab790527d556daf Georgi Gerganov ggerganov@gmail.com 2023-03-25T21:51:41+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T21:51:41+02:00 If n_predict == -1, generate forever e2d490dafd860eaaaf9aa8008ab790527d556daf 03f7e335604b3d68f74995aa2ccb4955833ee423 Georgi Gerganov ggerganov@gmail.com 2023-03-25T21:36:22+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T21:36:22+02:00 Inifinite generation via context swapping (#71) 03f7e335604b3d68f74995aa2ccb4955833ee423 55ad42af845127bd0eb0c1f36f327ecec83f4bca Georgi Gerganov ggerganov@gmail.com 2023-03-25T20:51:14+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T20:51:14+02:00 Cleanup STL headers + fix embedding examples + minor stuff 55ad42af845127bd0eb0c1f36f327ecec83f4bca 459e93cce07cab9052c06b5bf360819893442e1e Georgi Gerganov ggerganov@gmail.com 2023-03-25T20:36:52+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T20:37:09+02:00 Move chat scripts into "./examples" 459e93cce07cab9052c06b5bf360819893442e1e a316a425d04027453dc0fd45f003b647c12f66f9 slaren 2141330+slaren@users.noreply.github.com 2023-03-25T19:31:48+01:00 GitHub noreply@github.com 2023-03-25T20:31:48+02:00 Add AVX2 implementation of dequantize_row_q4_1 (#505) a316a425d04027453dc0fd45f003b647c12f66f9 ecbe466a364876927994e2f1ec14f4d82301d201 Georgi Gerganov ggerganov@gmail.com 2023-03-25T20:26:40+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T20:26:40+02:00 Overhaul the examples structure ecbe466a364876927994e2f1ec14f4d82301d201 502a400192013d3e95ed87b777e8fa3bec45713c Georgi Gerganov ggerganov@gmail.com 2023-03-25T19:47:21+02:00 GitHub noreply@github.com 2023-03-25T19:47:21+02:00 Retire the ggml_mul_mat() branch for transposed src0 (#500) 502a400192013d3e95ed87b777e8fa3bec45713c 09aecbf6283bbce9449e2d96000073145aaaf5fc Georgi Gerganov ggerganov@gmail.com 2023-03-25T17:16:50+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T17:17:16+02:00 Disable prompt verbosity by default and add option to enable (#480) 09aecbf6283bbce9449e2d96000073145aaaf5fc 4640eff23d341a0273587800e17ff4a378132d60 slaren 2141330+slaren@users.noreply.github.com 2023-03-25T16:06:49+01:00 GitHub noreply@github.com 2023-03-25T17:06:49+02:00 Add AVX2 implementation of dequantize_row_q4_0 (#467) 4640eff23d341a0273587800e17ff4a378132d60 ab77d7631211b299cb734bea6ad1f74324154150 Georgi Gerganov ggerganov@gmail.com 2023-03-25T17:03:10+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T17:03:10+02:00 Don't interefe with BLAS for large prompts by running only 1 thread ab77d7631211b299cb734bea6ad1f74324154150 29b7baab670ae8b76ac0da21c2ded69ff18971ee Georgi Gerganov ggerganov@gmail.com 2023-03-25T16:47:59+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T16:49:09+02:00 Add longer DAN prompt for testing big batch numbers 29b7baab670ae8b76ac0da21c2ded69ff18971ee 4a7129acd2e939b92d70dd568c746f2fa078232c slaren 2141330+slaren@users.noreply.github.com 2023-03-25T15:34:23+01:00 GitHub noreply@github.com 2023-03-25T16:34:23+02:00 Add timings for the prompt evaluation (#478) 4a7129acd2e939b92d70dd568c746f2fa078232c 6b6dbc8910c6d53f4d96c46c8fcec70e2cd435d8 Georgi Gerganov ggerganov@gmail.com 2023-03-25T16:30:32+02:00 GitHub noreply@github.com 2023-03-25T16:30:32+02:00 Remove obsolete information from README 6b6dbc8910c6d53f4d96c46c8fcec70e2cd435d8 2a2e63ce0503d9bf3e55283e40a052c78c1cc3a8 Georgi Gerganov ggerganov@gmail.com 2023-03-25T16:22:05+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T16:22:05+02:00 Remove obsolete assert and fix compiler warning 2a2e63ce0503d9bf3e55283e40a052c78c1cc3a8 e899bf54b291e8c84173a0e534a2c262f3f63229 Georgi Gerganov ggerganov@gmail.com 2023-03-25T16:09:54+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-25T16:10:14+02:00 Fix nasty bug in ggml_compute_forward_mul_mat_f32() and reenable BLAS e899bf54b291e8c84173a0e534a2c262f3f63229 fbd4d38c647f82b2598291ea9b8d0c09ac1ffb8c anzz1 anzz1@live.com 2023-03-25T14:42:09+02:00 GitHub noreply@github.com 2023-03-25T14:42:09+02:00 bounds checking for input prefix (#492) fbd4d38c647f82b2598291ea9b8d0c09ac1ffb8c 58e6c9f36f97d0a3e287b97256dc5f6b0e9fb5ae anzz1 anzz1@live.com 2023-03-25T14:03:19+02:00 GitHub noreply@github.com 2023-03-25T14:03:19+02:00 feat: '--in-prefix STRING' option (#426) 58e6c9f36f97d0a3e287b97256dc5f6b0e9fb5ae 36d07532ef7ccf0bdc12e050472f359a6794957f Jed Fox git@jedfox.com 2023-03-25T01:26:28-04:00 GitHub noreply@github.com 2023-03-25T07:26:28+02:00 Add support for file load progress reporting callbacks (#434) 36d07532ef7ccf0bdc12e050472f359a6794957f 6f1ee4b640912211a4b07965c585d327e32e734d Doomsdayrs 38189170+Doomsdayrs@users.noreply.github.com 2023-03-25T01:21:24-04:00 GitHub noreply@github.com 2023-03-25T07:21:24+02:00 Add missing struct annotation (#483) 6f1ee4b640912211a4b07965c585d327e32e734d 8520fc310eab87f2c4612f2a00d4adbd44a20d0d Chris Kuehl ckuehl@ckuehl.me 2023-03-24T23:38:14-05:00 GitHub noreply@github.com 2023-03-25T06:38:14+02:00 Fix crash for 65B model with pre-allocated memory (#485) 8520fc310eab87f2c4612f2a00d4adbd44a20d0d b3f460e94139cb24b0af81cc8bc10eb86269d704 Georgi Gerganov ggerganov@gmail.com 2023-03-24T23:47:06+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-24T23:47:06+02:00 Disable BLAS altogether - the bug is not just for qunatized mat mul b3f460e94139cb24b0af81cc8bc10eb86269d704 04c6f5ed6fafd63601fa06757877ed5ccf9d5991 Georgi Gerganov ggerganov@gmail.com 2023-03-24T23:39:17+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-24T23:39:17+02:00 Disable BLAS branch in mul_mat - seems there is a bug 04c6f5ed6fafd63601fa06757877ed5ccf9d5991 7a9b6c3a8bdc1cb75fefc826dfaa7331eb63695d Georgi Gerganov ggerganov@gmail.com 2023-03-24T23:17:58+02:00 GitHub noreply@github.com 2023-03-24T23:17:58+02:00 Immediately start processing the prompt before user input has been provided (#476) 7a9b6c3a8bdc1cb75fefc826dfaa7331eb63695d 31572d966531f7d768eb773322016ab78eb6e835 Georgi Gerganov ggerganov@gmail.com 2023-03-24T23:17:37+02:00 GitHub noreply@github.com 2023-03-24T23:17:37+02:00 Reduce memory usage and allocate enough memory for largest context (#473) 31572d966531f7d768eb773322016ab78eb6e835 f4f5362edb01b05c383b23f36d7b3489c77061b5 Georgi Gerganov ggerganov@gmail.com 2023-03-24T18:23:56+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-24T18:23:56+02:00 Temporary bump the memory buffer size - hopefully fix issues from 483bab2e f4f5362edb01b05c383b23f36d7b3489c77061b5 863f65e2e32dc1e6d23c96a4811bf382d6b2a548 Gary Mulder gjmulder@gmail.com 2023-03-24T15:23:09Z GitHub noreply@github.com 2023-03-24T15:23:09Z Update README.md (#444) 863f65e2e32dc1e6d23c96a4811bf382d6b2a548 afd220d9c665e4c19107120ace2f0cb742e28aa1 rabidcopy rabidcopy@yahoo.com 2023-03-24T10:22:39-05:00 GitHub noreply@github.com 2023-03-24T17:22:39+02:00 fix instruct mode (#445) afd220d9c665e4c19107120ace2f0cb742e28aa1 481044d50cfe8eaa6cd0c1a1b445680e4b0b3ebc Georgi Gerganov ggerganov@gmail.com 2023-03-24T17:21:01+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-24T17:21:01+02:00 Properly free llama_context on failure 481044d50cfe8eaa6cd0c1a1b445680e4b0b3ebc 563cdc391dde140f1084d1012234e8e6f57f881f Cameron Kaiser classilla@users.noreply.github.com 2023-03-24T08:19:26-07:00 GitHub noreply@github.com 2023-03-24T17:19:26+02:00 additional optimizations for POWER9 (#454) 563cdc391dde140f1084d1012234e8e6f57f881f 8d4a855c241ecb0f3ddc03447fe56002ebf27a37 comex comexk@gmail.com 2023-03-24T08:19:05-07:00 GitHub noreply@github.com 2023-03-24T17:19:05+02:00 Support calling mlock() on loaded model data on Linux and macOS (#453) 8d4a855c241ecb0f3ddc03447fe56002ebf27a37 b6b268d4415fd3b3e53f22b6619b724d4928f713 Luciano lucianostrika44@gmail.com 2023-03-24T08:05:13-07:00 GitHub noreply@github.com 2023-03-24T17:05:13+02:00 Add embedding mode with arg flag. Currently working (#282) b6b268d4415fd3b3e53f22b6619b724d4928f713 3cd8dde0d1357b7f11bdd25c45d5bf5e97e284a0 Georgi Gerganov ggerganov@gmail.com 2023-03-24T09:13:35+02:00 GitHub noreply@github.com 2023-03-24T09:13:35+02:00 Add link to Roadmap discussion 3cd8dde0d1357b7f11bdd25c45d5bf5e97e284a0 4870e455b3653f7d7769fa5772b2c90ffad088df Georgi Gerganov ggerganov@gmail.com 2023-03-24T06:22:28+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-24T06:22:28+02:00 Revert "Fix memory allocation issues and seg faults" 4870e455b3653f7d7769fa5772b2c90ffad088df 483bab2e3d4a868fe679d8bb32827d2a4df214dc Georgi Gerganov ggerganov@gmail.com 2023-03-24T00:11:53+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-24T00:11:53+02:00 Fix memory allocation issues and seg faults 483bab2e3d4a868fe679d8bb32827d2a4df214dc 404e1da38ec8025707031a8027da14dc1590f952 Georgi Gerganov ggerganov@gmail.com 2023-03-23T23:22:01+02:00 GitHub noreply@github.com 2023-03-23T23:22:01+02:00 Avoid the transposed X branch in the Z = X * Y matrix multiplication (#439) 404e1da38ec8025707031a8027da14dc1590f952 4cc053b6d5e9df7ac21fa06b7208a70c156d4d7a Jed Fox git@jedfox.com 2023-03-23T16:42:52-04:00 GitHub noreply@github.com 2023-03-23T22:42:52+02:00 Fix quantize script not finding models in parent directory (#428) 4cc053b6d5e9df7ac21fa06b7208a70c156d4d7a 0ba5a3a9a5efedb1aeecbbc70a4e9825542472d5 Georgi Gerganov ggerganov@gmail.com 2023-03-23T22:39:44+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-23T22:39:44+02:00 Remove oboslete command from Docker script 0ba5a3a9a5efedb1aeecbbc70a4e9825542472d5 2e17dfd80a473099dacc0f41c9146d233c6a5972 Georgi Gerganov ggerganov@gmail.com 2023-03-23T22:32:02+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-23T22:32:21+02:00 Obsolete 2e17dfd80a473099dacc0f41c9146d233c6a5972 20a1a4e09c522a80e2a0db51643d25fa38326065 rabidcopy rabidcopy@yahoo.com 2023-03-23T15:22:47-05:00 GitHub noreply@github.com 2023-03-23T22:22:47+02:00 Replace EOS with newline to prevent context/memory being flushed by EOS in interactive mode (#333) 20a1a4e09c522a80e2a0db51643d25fa38326065 ad072fc5ad6f6905a7224ff6ea07c0644aa075b1 Timmy Knight r2d2fish@gmail.com 2023-03-23T10:18:13-10:00 GitHub noreply@github.com 2023-03-23T22:18:13+02:00 Fix GPTQ converter (#423) ad072fc5ad6f6905a7224ff6ea07c0644aa075b1 ea10d3ded2994106596ddf8e4ed02741b3e053e6 nusu-github 29514220+nusu-github@users.noreply.github.com 2023-03-24T05:16:48+09:00 GitHub noreply@github.com 2023-03-23T21:16:48+01:00 Generate library with CMake (#430) ea10d3ded2994106596ddf8e4ed02741b3e053e6 a18c19259a3cb9dec332d613e8f15704f678a468 anzz1 anzz1@live.com 2023-03-23T19:54:28+02:00 GitHub noreply@github.com 2023-03-23T19:54:28+02:00 Command line args bounds checking (#424) a18c19259a3cb9dec332d613e8f15704f678a468 a50e39c6fe36be3de0941b3c05aaf9c37912fd47 Ben Siraphob bensiraphob@gmail.com 2023-03-22T00:37:02-05:00 Pavol Rusnak pavol@rusnak.io 2023-03-23T17:51:26+01:00 Fix Nix build a50e39c6fe36be3de0941b3c05aaf9c37912fd47 a140219e81cfb80356438112cd2290d701b282bb Stephan Walter stephan@walter.name 2023-03-23T14:15:48Z GitHub noreply@github.com 2023-03-23T15:15:48+01:00 Revert "Delete SHA256SUMS for now" (#429) a140219e81cfb80356438112cd2290d701b282bb 8a3e5ef801339e57b9b0449220e9ffb11a6648e2 Kerfuffle 44031344+KerfuffleV2@users.noreply.github.com 2023-03-23T05:41:32-06:00 GitHub noreply@github.com 2023-03-23T12:41:32+01:00 Fix Makefile echo escape codes (by removing them). (#418) 8a3e5ef801339e57b9b0449220e9ffb11a6648e2 8eea5ae0e5f31238a97c79ea9103c27647380e37 Gary Mulder gjmulder@gmail.com 2023-03-23T11:30:40Z GitHub noreply@github.com 2023-03-23T11:30:40Z Move model section from issue template to README.md (#421) 8eea5ae0e5f31238a97c79ea9103c27647380e37 93208cfb929c2323e5d2ac6bf354e278040e70ed anzz1 anzz1@live.com 2023-03-23T12:26:19+02:00 GitHub noreply@github.com 2023-03-23T11:26:19+01:00 Delete SHA256SUMS for now (#416) 93208cfb929c2323e5d2ac6bf354e278040e70ed 03ace14cfd68a1289ac3b76563534c8ee72a2e53 Georgi Gerganov ggerganov@gmail.com 2023-03-23T10:46:58+02:00 GitHub noreply@github.com 2023-03-23T10:46:58+02:00 Adjust repetition penalty .. 03ace14cfd68a1289ac3b76563534c8ee72a2e53 e4412b45e395981068d2850d3fa04cc16c77d70d Georgi Gerganov ggerganov@gmail.com 2023-03-23T09:48:51+02:00 GitHub noreply@github.com 2023-03-23T09:48:51+02:00 Add link to recent podcast about whisper.cpp and llama.cpp e4412b45e395981068d2850d3fa04cc16c77d70d f7dc43bc0d759732815856183246f167111587ad anzz1 anzz1@live.com 2023-03-23T04:20:34+02:00 GitHub noreply@github.com 2023-03-23T04:20:34+02:00 CI: CMake: Separate build and test steps (#376) f7dc43bc0d759732815856183246f167111587ad ee8a7887865a893be208e0a92d6d94d2cb66a789 tjohnman tjohnman@users.noreply.github.com 2023-03-23T01:30:23+01:00 GitHub noreply@github.com 2023-03-23T01:30:23+01:00 Fix instruct mode broken by PR #354 (#409) ee8a7887865a893be208e0a92d6d94d2cb66a789 69c92298a9e36dc2363b3bf50452976ce49487b3 Gary Mulder gjmulder@gmail.com 2023-03-22T19:06:18Z GitHub noreply@github.com 2023-03-22T19:06:18Z Update issue template so people will use it (#404) 69c92298a9e36dc2363b3bf50452976ce49487b3 97940520e8fd49c56bb29b71cc350190b723513f Stephan Walter stephan@walter.name 2023-03-22T17:29:06Z GitHub noreply@github.com 2023-03-22T19:29:06+02:00 Deduplicate q4 quantization functions (#383) 97940520e8fd49c56bb29b71cc350190b723513f 305ba6f0e6daa3796aad9dd18053a1945dd4cc58 Valentyn Bezshapkin 61702053+valentynbez@users.noreply.github.com 2023-03-22T18:20:25+01:00 GitHub noreply@github.com 2023-03-22T19:20:25+02:00 fix: add POSIX functionality for Linux compilation (#51) 305ba6f0e6daa3796aad9dd18053a1945dd4cc58 4122dffff958cd137175b58f1f27c0913528d7ba tjohnman tjohnman@users.noreply.github.com 2023-03-22T18:16:35+01:00 GitHub noreply@github.com 2023-03-22T19:16:35+02:00 Don't force immediate interactive without `-i` (#354) 4122dffff958cd137175b58f1f27c0913528d7ba 56e659a0b271436e24813a801640d015e7b05328 Erik Scholz Green-Sky@users.noreply.github.com 2023-03-22T17:37:10+01:00 GitHub noreply@github.com 2023-03-22T18:37:10+02:00 cmake: make llama an actual library (#392) 56e659a0b271436e24813a801640d015e7b05328 40ea807a972ec7b5a426f034ebfa593b5e7a06ed Erik Scholz Green-Sky@users.noreply.github.com 2023-03-22T17:09:38+01:00 GitHub noreply@github.com 2023-03-22T18:09:38+02:00 fix perplexity after c-api refactor (#390) 40ea807a972ec7b5a426f034ebfa593b5e7a06ed d5850c53ca179b9674b98f35d359763416a3cc11 Gary Linscott glinscott@gmail.com 2023-03-22T08:53:54-07:00 GitHub noreply@github.com 2023-03-22T08:53:54-07:00 Add details on perplexity to README.md (#395) d5850c53ca179b9674b98f35d359763416a3cc11 ae44e23ee36c02da0e37ab508a4b473ace724f8e Yusuf Kağan Hanoğlu hanoglu@yahoo.com 2023-03-22T11:55:45+03:00 GitHub noreply@github.com 2023-03-22T10:55:45+02:00 Add missing header for memcpy (#386) ae44e23ee36c02da0e37ab508a4b473ace724f8e 928480ef5b7b03d7a07e98286aebe3d8b24457d9 Georgi Gerganov ggerganov@gmail.com 2023-03-22T07:47:15+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-22T07:47:15+02:00 When seed <= 0 - use the clock to generate one 928480ef5b7b03d7a07e98286aebe3d8b24457d9 56817b1f882b1894daa4051d0de0bf9a0926d315 Georgi Gerganov ggerganov@gmail.com 2023-03-22T07:45:00+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-22T07:45:14+02:00 Init llama_context_params properly from CLI (#370) 56817b1f882b1894daa4051d0de0bf9a0926d315 f5a77a629bd0f37ae1696747633ab42a5530ec15 Georgi Gerganov ggerganov@gmail.com 2023-03-22T07:34:02+02:00 GitHub noreply@github.com 2023-03-22T07:34:02+02:00 Remove temporary notice and update hot topics f5a77a629bd0f37ae1696747633ab42a5530ec15 da0e9fe90ccf6e73597eb19dd0cfc0a28363fb3b Georgi Gerganov ggerganov@gmail.com 2023-03-22T07:32:36+02:00 GitHub noreply@github.com 2023-03-22T07:32:36+02:00 Introduce C-style API (#370) da0e9fe90ccf6e73597eb19dd0cfc0a28363fb3b e6c9e0986c79ba1cc8848879b2fcce979f9b4672 Gary Mulder gjmulder@gmail.com 2023-03-20T20:14:06Z Pavol Rusnak pavol@rusnak.io 2023-03-21T23:19:11+01:00 Add SHA256SUMS file and instructions to README how to obtain and verify the downloads e6c9e0986c79ba1cc8848879b2fcce979f9b4672 01a297b09932e29f3319d6588977c32a926c7907 anzz1 anzz1@live.com 2023-03-21T23:49:24+02:00 anzz1 anzz1@live.com 2023-03-22T00:01:08+02:00 Fix bin dir for win ci 01a297b09932e29f3319d6588977c32a926c7907 3366853e41fcc818222a0271c76b6106179106fb Erik Scholz Green-Sky@users.noreply.github.com 2023-03-21T22:34:25+01:00 GitHub noreply@github.com 2023-03-21T23:34:25+02:00 specify build type for ctest on windows (#371) 3366853e41fcc818222a0271c76b6106179106fb 3f9c6135e45ae3f520b1e17197004cc60c9ca45b Georgi Gerganov ggerganov@gmail.com 2023-03-21T22:57:35+02:00 GitHub noreply@github.com 2023-03-21T22:57:35+02:00 Add notice about pending change 3f9c6135e45ae3f520b1e17197004cc60c9ca45b 0f6135270839f0715843c4d480c63ae150def419 Mathieu Nayrolles MathieuNls@users.noreply.github.com 2023-03-21T16:52:27-04:00 GitHub noreply@github.com 2023-03-21T22:52:27+02:00 fix typo in chatLLaMa (#368) 0f6135270839f0715843c4d480c63ae150def419 353ec251a42491f5192c48561da4b444ef67f23c Georgi Gerganov ggerganov@gmail.com 2023-03-21T19:47:27+02:00 GitHub noreply@github.com 2023-03-21T19:47:27+02:00 Update issue templates 353ec251a42491f5192c48561da4b444ef67f23c 89d5d90f3b6d25f134da7a8e252c3432bffcf674 Fabio R. Sluzala Fabio3rs@users.noreply.github.com 2023-03-21T14:21:50-03:00 GitHub noreply@github.com 2023-03-21T19:21:50+02:00 We could use std::unordered_map over std::map (#305) 89d5d90f3b6d25f134da7a8e252c3432bffcf674 16ffc013c62f22bdaa3cdc022d7a13fd952d73fc Matvey Soloviev blackhole89@gmail.com 2023-03-21T18:11:01+01:00 GitHub noreply@github.com 2023-03-21T19:11:01+02:00 Fix color codes emitting mid-UTF8 code. (#312) 16ffc013c62f22bdaa3cdc022d7a13fd952d73fc 486ae645fd3eda8b9d7413d5ff34fb65a3e337fb comex comexk@gmail.com 2023-03-21T09:42:25-07:00 GitHub noreply@github.com 2023-03-21T18:42:25+02:00 Importer for GPTQ quantized LLaMA models (#301) 486ae645fd3eda8b9d7413d5ff34fb65a3e337fb 3ab3e6582f7320c2b6568c892fdfc8215caf7e6c Gary Linscott glinscott@gmail.com 2023-03-21T09:27:42-07:00 GitHub noreply@github.com 2023-03-21T18:27:42+02:00 Compute perplexity over prompt (#270) 3ab3e6582f7320c2b6568c892fdfc8215caf7e6c f157088cb75f23208abc92b473a132ef3f7a7f15 Jean-Christophe Hoelt hoelt@fovea.cc 2023-03-21T18:23:15+02:00 GitHub noreply@github.com 2023-03-21T18:23:15+02:00 Add chatLLaMa script (#198) f157088cb75f23208abc92b473a132ef3f7a7f15 c86ba036e613d46815501a4c6775117c9fc7afce Alex von Gluck IV kallisti5@unixzen.com 2023-03-21T11:21:06-05:00 GitHub noreply@github.com 2023-03-21T18:21:06+02:00 makefile: Fix CPU feature detection on Haiku (#218) c86ba036e613d46815501a4c6775117c9fc7afce 1daf4dd71235dbbf537738e7ad53daad8d97586f anzz1 anzz1@live.com 2023-03-21T18:14:46+02:00 GitHub noreply@github.com 2023-03-21T18:14:46+02:00 Enable ANSI colors on Windows 10+ (#311) 1daf4dd71235dbbf537738e7ad53daad8d97586f dc6a845b8573cd7d06c6b295241d26f311602a1f Georgi Gerganov ggerganov@gmail.com 2023-03-21T18:10:32+02:00 GitHub noreply@github.com 2023-03-21T18:10:32+02:00 Minor style changes dc6a845b8573cd7d06c6b295241d26f311602a1f 6a612959e1b6c37b68b6b141329751a2902b1030 Georgi Gerganov ggerganov@gmail.com 2023-03-21T18:09:37+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-21T18:09:46+02:00 Add chat.sh script 6a612959e1b6c37b68b6b141329751a2902b1030 d5f56a5e5a0069329a81f96460221e7afb1daddc tjohnman tjohnman@users.noreply.github.com 2023-03-21T17:05:06+01:00 GitHub noreply@github.com 2023-03-21T18:05:06+02:00 Check for reverse prompt by characters instead of tokens (#292) (#330) d5f56a5e5a0069329a81f96460221e7afb1daddc 3bfa3b43b7319b71853bfc7d3cf4e9767c24bbc8 tjohnman tjohnman@users.noreply.github.com 2023-03-21T17:04:43+01:00 GitHub noreply@github.com 2023-03-21T18:04:43+02:00 Check for reverse prompt by characters instead of tokens (#292) (#330) 3bfa3b43b7319b71853bfc7d3cf4e9767c24bbc8 715d292ee0e34d27f27af43d7feaad1f1344981d Georgi Gerganov ggerganov@gmail.com 2023-03-21T17:59:16+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-21T17:59:16+02:00 Fix convert script, warnings alpaca instructions, default params 715d292ee0e34d27f27af43d7feaad1f1344981d c98ae02668a25916954b1653e25a5a35ca048d63 Kevin Lo kevlo@kevlo.org 2023-03-21T09:50:09-06:00 GitHub noreply@github.com 2023-03-21T17:50:09+02:00 Add OpenBSD support (#314) c98ae02668a25916954b1653e25a5a35ca048d63 c3b2306b18a087799acc431e485b8a2e3162cd52 Mack Straight eiz@users.noreply.github.com 2023-03-21T08:49:43-07:00 GitHub noreply@github.com 2023-03-21T17:49:43+02:00 fix typo in comment (#318) c3b2306b18a087799acc431e485b8a2e3162cd52 975d2cebf97ce888fa0aeee6f5ac774d7135891f Qingyou Meng meng.qingyou@gmail.com 2023-03-21T23:44:11+08:00 GitHub noreply@github.com 2023-03-21T17:44:11+02:00 Makefile: slightly cleanup for Mac Intel; echo instead of run ./main -h (#335) 975d2cebf97ce888fa0aeee6f5ac774d7135891f e0ffc861fae5ac8b40ce973f822d03db02929d36 anzz1 anzz1@live.com 2023-03-21T17:42:43+02:00 GitHub noreply@github.com 2023-03-21T17:42:43+02:00 cmdline option for custom amount of model parts (--n_parts N) (#348) e0ffc861fae5ac8b40ce973f822d03db02929d36 8f644a0a859938c787d329d27f98e03c58d7df27 Kevin Kwok antimatter15@gmail.com 2023-03-21T08:34:49-07:00 GitHub noreply@github.com 2023-03-21T17:34:49+02:00 Update IPFS links to quantized alpaca with new tokenizer format (#352) 8f644a0a859938c787d329d27f98e03c58d7df27 eb34620aeceaf9d9df7fcb19acc17ad41b9f60f8 Georgi Gerganov ggerganov@gmail.com 2023-03-21T17:32:14+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-21T17:32:14+02:00 Change default repeat_penalty to 1.0 eb34620aeceaf9d9df7fcb19acc17ad41b9f60f8 2e664f1ff413995506c9a54f3a8d5b8c64e37a91 Georgi Gerganov ggerganov@gmail.com 2023-03-21T17:29:41+02:00 GitHub noreply@github.com 2023-03-21T17:29:41+02:00 Add tokenizer test + revert to C++11 (#355) 2e664f1ff413995506c9a54f3a8d5b8c64e37a91 8cf9f34eddc124d4ab28f4d2fe8e99d574510bde Casey Primozic casey@cprimozic.net 2023-03-21T07:35:42-07:00 GitHub noreply@github.com 2023-03-21T15:35:42+01:00 Add initial AVX512 support for dot product on Linux (#320) 8cf9f34eddc124d4ab28f4d2fe8e99d574510bde bd4b46d6ba504b99c936f43fc014529adffb6048 nusu-github 29514220+nusu-github@users.noreply.github.com 2023-03-21T09:37:16+09:00 GitHub noreply@github.com 2023-03-21T01:37:16+01:00 Adding missing features of CMakeLists.txt & Refactoring (#131) bd4b46d6ba504b99c936f43fc014529adffb6048 6b6d5b5024faaf82019d08cde5e8a9d69c6ca316 Ben Siraphob bensiraphob@gmail.com 2023-03-20T16:44:30-05:00 Pavol Rusnak pavol@rusnak.io 2023-03-20T22:50:22+01:00 Nix flake: set meta.mainProgram to llama 6b6d5b5024faaf82019d08cde5e8a9d69c6ca316 a791a68b613b162c88a83f5f0225223bc167c762 Qingyou Meng meng.qingyou@gmail.com 2023-03-21T03:33:10+08:00 GitHub noreply@github.com 2023-03-20T19:33:10Z Fixed tokenizer.model not found error when model dir is symlink (#325) a791a68b613b162c88a83f5f0225223bc167c762 0f1b21cb90ac6b84a9af70cafb8e13b5389e3b32 Mack Straight eiz@users.noreply.github.com 2023-03-20T12:26:01-07:00 GitHub noreply@github.com 2023-03-20T19:26:01Z move file magic/version to header, print expected version (#319) 0f1b21cb90ac6b84a9af70cafb8e13b5389e3b32 074bea2eb1f1349a0118239c4152914aecaa1be4 Bernat Vadell hounter.caza@gmail.com 2023-03-20T18:05:20+01:00 GitHub noreply@github.com 2023-03-20T18:05:20+01:00 Docker - Fix publish docker image in GitHub Registry (#235) 074bea2eb1f1349a0118239c4152914aecaa1be4 5cb63e2493c49bc2c3b9b355696e8dc26cdd0380 Mack Straight eiz@users.noreply.github.com 2023-03-20T03:17:23-07:00 GitHub noreply@github.com 2023-03-20T03:17:23-07:00 sentencepiece bpe compatible tokenizer (#252) 5cb63e2493c49bc2c3b9b355696e8dc26cdd0380 da5303c1ea68aa19db829c634f1e10d08d409680 Stephan Walter stephan@walter.name 2023-03-20T08:24:11Z GitHub noreply@github.com 2023-03-20T09:24:11+01:00 Add tqdm to Python requirements (#293) da5303c1ea68aa19db829c634f1e10d08d409680 4545539d718cf88f4c3a76669b8ac2e26cd8a1e5 cocktailpeanut 121128867+cocktailpeanut@users.noreply.github.com 2023-03-19T17:44:20-04:00 GitHub noreply@github.com 2023-03-19T23:44:20+02:00 bugfix: default should not be interactive (#304) 4545539d718cf88f4c3a76669b8ac2e26cd8a1e5 edeba283665591f2f726024a92efe4b0b40434b3 Georgi Gerganov ggerganov@gmail.com 2023-03-19T21:58:51+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-19T21:58:51+02:00 Rename script edeba283665591f2f726024a92efe4b0b40434b3 5c19c70ba631a8f5d54feb6634e0eea178911a84 Georgi Gerganov ggerganov@gmail.com 2023-03-19T21:57:28+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-19T21:57:48+02:00 Add temporary helper script for Alpaca chat 5c19c70ba631a8f5d54feb6634e0eea178911a84 24568371ae0d7caf85164abe4753f36a7dba0288 Rickey Bowers Jr bitRAKE@gmail.com 2023-03-19T13:44:30-06:00 GitHub noreply@github.com 2023-03-19T19:44:30Z fix coloring of last `n_batch` of prompt, and refactor line input (#221) 24568371ae0d7caf85164abe4753f36a7dba0288 7392f1cd2cef4dfed41f4db7c4160ab86c0dfcd9 tjohnman tjohnman@users.noreply.github.com 2023-03-19T20:33:06+01:00 GitHub noreply@github.com 2023-03-19T21:33:06+02:00 Support for multiple reverse prompts. (#299) 7392f1cd2cef4dfed41f4db7c4160ab86c0dfcd9 ad5fd5b60cfdfbfb22b0f2bc9e9f6c9692768f8d Suaj Carrot 72162667+SuajCarrot@users.noreply.github.com 2023-03-19T12:38:44-06:00 GitHub noreply@github.com 2023-03-19T20:38:44+02:00 Improved quantize script (#222) ad5fd5b60cfdfbfb22b0f2bc9e9f6c9692768f8d 368d0c8a9ebae16a20e1c8971b21ee888bdefad5 tjohnman tjohnman@users.noreply.github.com 2023-03-19T19:36:19+01:00 GitHub noreply@github.com 2023-03-19T20:36:19+02:00 Make prompt randomization optional. (#300) 368d0c8a9ebae16a20e1c8971b21ee888bdefad5 50fae10d0339f2bd639f69dd679c0201d939a265 tjohnman tjohnman@users.noreply.github.com 2023-03-19T19:31:17+01:00 GitHub noreply@github.com 2023-03-19T20:31:17+02:00 Respect the maximum number of tokens in interactive. (#298) 50fae10d0339f2bd639f69dd679c0201d939a265 084e2f0ec081c929343d44b09df07ae87cd1ed32 slaren 2141330+slaren@users.noreply.github.com 2023-03-19T19:22:48+01:00 GitHub noreply@github.com 2023-03-19T20:22:48+02:00 Add --ignore-eos parameter (#181) 084e2f0ec081c929343d44b09df07ae87cd1ed32 0b366e735729327476ec31da02de3c9c9771ddfb Qingyou Meng meng.qingyou@gmail.com 2023-03-20T02:10:00+08:00 GitHub noreply@github.com 2023-03-19T20:10:00+02:00 interactive mode: print '\n' in sigint_handler, this flush stdout thus ensure color reset. (#283) 0b366e735729327476ec31da02de3c9c9771ddfb 160bfb217da5038ccbd74438f9f16a16012d7866 Erik Scholz Green-Sky@users.noreply.github.com 2023-03-19T18:57:00+01:00 GitHub noreply@github.com 2023-03-19T19:57:00+02:00 Command line switch to use F16 for memory_k and memory_v (refactor of #154) (#294) 160bfb217da5038ccbd74438f9f16a16012d7866 c494ed5b94b429d3d73721235e78c9f5fa6e5652 Georgi Gerganov ggerganov@gmail.com 2023-03-19T19:51:55+02:00 GitHub noreply@github.com 2023-03-19T19:51:55+02:00 Update hot topics to mention Alpaca support c494ed5b94b429d3d73721235e78c9f5fa6e5652 c1c7026b470ced0b8a6c67e968c04bb47864def1 Georgi Gerganov ggerganov@gmail.com 2023-03-19T19:46:32+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-19T19:46:32+02:00 Fix off-by-one bug (#115) c1c7026b470ced0b8a6c67e968c04bb47864def1 467b149761cd63248b00d6ffb204d50a4cbb451a Georgi Gerganov ggerganov@gmail.com 2023-03-19T19:33:18+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-19T19:33:18+02:00 Fix python stuff (#109) 467b149761cd63248b00d6ffb204d50a4cbb451a 70f01cb8632f73b5cf70428608b89cd3c0775d23 qunash anzoria@gmail.com 2023-03-19T20:17:39+03:00 GitHub noreply@github.com 2023-03-19T19:17:39+02:00 Refactoring `convert-pth-to-ggml.py`: more concise and readable (#109) 70f01cb8632f73b5cf70428608b89cd3c0775d23 a4e63b73dfa1894387926cc8072b5f36deebf0a5 Georgi Gerganov ggerganov@gmail.com 2023-03-19T19:04:44+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-19T19:05:04+02:00 Drop trailing new line from file prompts (#80) a4e63b73dfa1894387926cc8072b5f36deebf0a5 9e1707218a24ff758c7b623594f8c0ce5e12eb6c Georgi Gerganov ggerganov@gmail.com 2023-03-19T18:49:50+02:00 GitHub noreply@github.com 2023-03-19T18:49:50+02:00 Add instruction for using Alpaca (#240) 9e1707218a24ff758c7b623594f8c0ce5e12eb6c 22213a17b56336bbea384a572a9484ce208c0333 Georgi Gerganov ggerganov@gmail.com 2023-03-19T18:37:02+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-19T18:37:02+02:00 Add "--instruct" argument for usage with Alpaca (#240) 22213a17b56336bbea384a572a9484ce208c0333 d7def1a7524f712e5ebb7cd02bab0f13aa56a7f9 Georgi Gerganov ggerganov@gmail.com 2023-03-19T17:30:00+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-19T17:30:00+02:00 Change RMSNorm eps to 1e-6 (#173) d7def1a7524f712e5ebb7cd02bab0f13aa56a7f9 6f61c18ec9a30416e21ed5abfb1321bdb14979be Ronsor ronsor@ronsor.pw 2023-03-18T17:10:47-07:00 GitHub noreply@github.com 2023-03-18T20:10:47-04:00 Warn user if a context size greater than 2048 tokens is specified (#274) 6f61c18ec9a30416e21ed5abfb1321bdb14979be 1e5a6d088d0f3a967c6e86298a756daec9e8df12 Pavol Rusnak pavol@rusnak.io 2023-03-18T22:39:46+01:00 Pavol Rusnak pavol@rusnak.io 2023-03-18T23:18:04+01:00 Fix typo in readme 1e5a6d088d0f3a967c6e86298a756daec9e8df12 554b54152145c30618bac171efb712cf4a7d1e96 Pavol Rusnak pavol@rusnak.io 2023-03-18T22:20:04+01:00 Pavol Rusnak pavol@rusnak.io 2023-03-18T22:25:35+01:00 Add note about Python 3.11 to readme 554b54152145c30618bac171efb712cf4a7d1e96 d3f202d57b694376cef6f381a6b6901825c3f6d9 Pavol Rusnak pavol@rusnak.io 2023-03-18T21:58:46+01:00 Pavol Rusnak pavol@rusnak.io 2023-03-18T22:25:35+01:00 Add memory/disk requirements to readme d3f202d57b694376cef6f381a6b6901825c3f6d9 e03e359730c127f888fcf00e93375771bc0a3500 Alex Nguyen tiendung@users.noreply.github.com 2023-03-18T20:51:49+07:00 GitHub noreply@github.com 2023-03-18T13:51:49Z Remove unused code since n_vocab is model.hparams.n_vocab (#262) e03e359730c127f888fcf00e93375771bc0a3500 a81d0c2a171a4446e6a21a3ec74a0c0768d71184 Justin Suess justin.suess@westpoint.edu 2023-03-18T07:44:09-04:00 GitHub noreply@github.com 2023-03-18T11:44:09Z fixed warning with std::ignore about unused function result (#151) a81d0c2a171a4446e6a21a3ec74a0c0768d71184 b2de7f18dfbb93463eeb5b4392117bbe82d5bd1b Gary Linscott glinscott@gmail.com 2023-03-18T04:17:19-07:00 GitHub noreply@github.com 2023-03-18T11:17:19Z Fix n^2 loop in tokenization (#254) b2de7f18dfbb93463eeb5b4392117bbe82d5bd1b a29274789309029fd88a9465e6d0832d4632272b anzz1 anzz1@live.com 2023-03-18T09:27:12+02:00 GitHub noreply@github.com 2023-03-18T09:27:12+02:00 CI Improvements (#230) a29274789309029fd88a9465e6d0832d4632272b c9f670a17755311aa28c411f5c7f3c8c05434770 Niklas Korz niklas@niklaskorz.de 2023-03-17T23:03:48+01:00 GitHub noreply@github.com 2023-03-17T23:03:48+01:00 Nix flake (#40) c9f670a17755311aa28c411f5c7f3c8c05434770 4f546091102a418ffdc6230f872ac56e5cedb835 thement 40525767+thement@users.noreply.github.com 2023-03-17T21:05:58+01:00 GitHub noreply@github.com 2023-03-17T21:05:58+01:00 Implement non-greedy tokenizer that tries to maximize token lengths (#242) 4f546091102a418ffdc6230f872ac56e5cedb835 e81b9c81c101f64531ef0fa1ee6b77d562635652 Georgi Gerganov ggerganov@gmail.com 2023-03-17T21:46:46+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-17T21:46:46+02:00 Default to 4 threads (#243) e81b9c81c101f64531ef0fa1ee6b77d562635652 367946c668757532deed929e1d78673c6ac6bcb8 Georgi Gerganov ggerganov@gmail.com 2023-03-17T20:30:04+02:00 GitHub noreply@github.com 2023-03-17T20:30:04+02:00 Update Contributing section 367946c668757532deed929e1d78673c6ac6bcb8 6b0df5ccf360fe5c015f6607f0375bfc6849005e Stephan Walter stephan@walter.name 2023-03-17T17:47:35Z GitHub noreply@github.com 2023-03-17T19:47:35+02:00 Don't tell users to use a bad number of threads (#243) 6b0df5ccf360fe5c015f6607f0375bfc6849005e 2af23d30434a677c6416812eea52ccc0af65119c mmyjona jonathan.gonse@gmail.com 2023-03-18T00:38:24+08:00 GitHub noreply@github.com 2023-03-17T13:38:24-03:00 add ptread link to fix cmake build under linux (#114) 2af23d30434a677c6416812eea52ccc0af65119c 904d2a8d6acd667c9633138d45a361d40fbf76d0 Bernat Vadell hounter.caza@gmail.com 2023-03-17T10:47:06+01:00 GitHub noreply@github.com 2023-03-17T10:47:06+01:00 🚀 Dockerize llamacpp (#132) 904d2a8d6acd667c9633138d45a361d40fbf76d0 721311070e31464ac12bef9a4444093eb3eaebf7 Matvey Soloviev blackhole89@gmail.com 2023-03-17T05:48:39+01:00 GitHub noreply@github.com 2023-03-17T06:48:39+02:00 Q4_1 quantization (#193) 721311070e31464ac12bef9a4444093eb3eaebf7 ac15de789547e5a6e93df552e787379b3a23ef26 Georgi Gerganov ggerganov@gmail.com 2023-03-16T15:00:09+02:00 GitHub noreply@github.com 2023-03-16T15:00:09+02:00 Update README.md ac15de789547e5a6e93df552e787379b3a23ef26 273abc47ff9dd899b3c4f58acd19d4649e90d6b4 Georgi Gerganov ggerganov@gmail.com 2023-03-16T08:55:13+02:00 GitHub noreply@github.com 2023-03-16T08:55:13+02:00 Expand "Contributing" section 273abc47ff9dd899b3c4f58acd19d4649e90d6b4 9b4a15b17d8395eb075379b140fcd0b0283f4ef6 Georgi Gerganov ggerganov@gmail.com 2023-03-16T07:12:12+02:00 GitHub noreply@github.com 2023-03-16T07:12:12+02:00 Update hot topics - RMSnorm 9b4a15b17d8395eb075379b140fcd0b0283f4ef6 6eac39ba953acaeec396cea2969dbf413907e2ec Nebula infinitewormhole@gmail.com 2023-03-15T19:29:25-04:00 GitHub noreply@github.com 2023-03-15T19:29:25-04:00 Fix RMS norm in GGML (#191) 6eac39ba953acaeec396cea2969dbf413907e2ec 27944c4206a49bbe003021a2610bacaa3044e619 hoangmit hoangmit@users.noreply.github.com 2023-03-15T18:41:38-04:00 GitHub noreply@github.com 2023-03-16T00:41:38+02:00 Add RMS norm and use it (#187) 27944c4206a49bbe003021a2610bacaa3044e619 2d15d6c9a959749f954d4fbbf44d711e19c5bdff moritzbrantner 31051084+moritzbrantner@users.noreply.github.com 2023-03-15T21:35:25+01:00 GitHub noreply@github.com 2023-03-15T22:35:25+02:00 fixed typo (#178) 2d15d6c9a959749f954d4fbbf44d711e19c5bdff 2d64715ad475f192a4004a52d134c67ccb6f44ad Rickey Bowers Jr bitRAKE@gmail.com 2023-03-15T13:56:24-06:00 GitHub noreply@github.com 2023-03-15T21:56:24+02:00 add SIGINT support for _WIN32 environments (#120) 2d64715ad475f192a4004a52d134c67ccb6f44ad 16b2c61a22f828ea77d9f084ca871c63bc5cc283 Justin Suess justin.suess@westpoint.edu 2023-03-15T15:42:40-04:00 GitHub noreply@github.com 2023-03-15T21:42:40+02:00 added ctx_size parameter (#148) 16b2c61a22f828ea77d9f084ca871c63bc5cc283 977295c700a2952c18400026d57467077dcd1a20 Justin Suess justin.suess@westpoint.edu 2023-03-15T15:39:38-04:00 GitHub noreply@github.com 2023-03-15T21:39:38+02:00 fixed color reset on exit (#149) 977295c700a2952c18400026d57467077dcd1a20 956dfda8ad8cea7961e22e0384bbc315bf79aed2 Musab Gultekin musabgultekin@users.noreply.github.com 2023-03-15T22:39:06+03:00 GitHub noreply@github.com 2023-03-15T21:39:06+02:00 Fix potential licensing issue (#126) 956dfda8ad8cea7961e22e0384bbc315bf79aed2 113e685d18ac4edb20f647fd34b000941556f6a6 Ronsor ronsor@ronsor.pw 2023-03-15T12:37:50-07:00 GitHub noreply@github.com 2023-03-15T21:37:50+02:00 Use `tokenizer.vocab_size()` instead of hardcoding 32000 in convert-pth-to-ggml.py (#142) 113e685d18ac4edb20f647fd34b000941556f6a6 47857e564c218a2c38346d0cdd94314632878fcb hoangmit hoangmit@users.noreply.github.com 2023-03-15T15:05:14-04:00 GitHub noreply@github.com 2023-03-15T21:05:14+02:00 inline -> static inline for "bytesFromNibbles" (#161) 47857e564c218a2c38346d0cdd94314632878fcb 60f819a2b10475055a36415bc489e5b55df2d052 Ronsor ronsor@ronsor.pw 2023-03-14T12:34:37-07:00 GitHub noreply@github.com 2023-03-14T21:34:37+02:00 Don't use vdotq_s32 if it's not available (#139) 60f819a2b10475055a36415bc489e5b55df2d052 97ab2b257897bfe7e2ae72876a3e50ed41b8c7ce Radoslav Gerganov rgerganov@gmail.com 2023-03-14T15:30:08+02:00 GitHub noreply@github.com 2023-03-14T15:30:08+02:00 Add section to README on how to run the project on Android (#130) 97ab2b257897bfe7e2ae72876a3e50ed41b8c7ce 2f700a27381e558a4eb5a3f8fd56757f4c7a417c Georgi Gerganov ggerganov@gmail.com 2023-03-14T09:43:52+02:00 GitHub noreply@github.com 2023-03-14T09:43:52+02:00 Add Misc section + update hot topics + minor fixes 2f700a27381e558a4eb5a3f8fd56757f4c7a417c c09a9cfb06c87d114615c105adda91b0e6273b69 Sebastián A sebastian.aedo29@gmail.com 2023-03-13T17:29:10-03:00 GitHub noreply@github.com 2023-03-13T22:29:10+02:00 Add windows to the CI (#98) c09a9cfb06c87d114615c105adda91b0e6273b69 7ec903d3c162417c11463f14ad5b773a918fb7f1 Georgi Gerganov ggerganov@gmail.com 2023-03-13T21:22:15+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-13T21:22:15+02:00 CMake build in Release by default (#75) 7ec903d3c162417c11463f14ad5b773a918fb7f1 4497ad819c0010a8b19ffeaf8c0428eb7558d3e0 Georgi Gerganov ggerganov@gmail.com 2023-03-13T19:21:51+02:00 GitHub noreply@github.com 2023-03-13T19:21:51+02:00 Update contribution section, hot topics, limitations, etc. 4497ad819c0010a8b19ffeaf8c0428eb7558d3e0 ed6849cc07a8973e5d31947b9df2df2da975ac96 Georgi Gerganov ggerganov@gmail.com 2023-03-13T19:15:08+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-13T19:15:08+02:00 Print system information ed6849cc07a8973e5d31947b9df2df2da975ac96 41be0a3b3d76ee4f254dc81b42bd8ed26ee324e7 Sebastián A sebastian.aedo29@gmail.com 2023-03-13T14:12:33-03:00 GitHub noreply@github.com 2023-03-13T19:12:33+02:00 Initial support for CMake (#75) 41be0a3b3d76ee4f254dc81b42bd8ed26ee324e7 671d5cac15241b495006f56482bf2d6967dca91f Thomas Klausner wiz@gatalith.at 2023-03-13T17:40:54+01:00 GitHub noreply@github.com 2023-03-13T18:40:54+02:00 Add NetBSD support. (#90) 671d5cac15241b495006f56482bf2d6967dca91f 84d9015c4a91ab586ba65d5bd31a8482baf46ba1 Pavol Rusnak pavol@rusnak.io 2023-03-13T17:39:56+01:00 GitHub noreply@github.com 2023-03-13T18:39:56+02:00 Use fprintf for diagnostic output (#48) 84d9015c4a91ab586ba65d5bd31a8482baf46ba1 63fd76fbb06f9b723ca11505352387a3148b1814 Georgi Gerganov ggerganov@gmail.com 2023-03-13T18:36:44+02:00 GitHub noreply@github.com 2023-03-13T18:36:44+02:00 Use vdotq_s32 to improve performance (#67) 63fd76fbb06f9b723ca11505352387a3148b1814 2a20f48efad692a8c2744f10c673bbdbe0c751b7 uint256_t konndennsa@gmail.com 2023-03-14T01:33:43+09:00 GitHub noreply@github.com 2023-03-13T18:33:43+02:00 Reduce model loading time (#43) 2a20f48efad692a8c2744f10c673bbdbe0c751b7 d1f224712d78ab2cbb78777acfeb6739f660eb96 Val Kharitonov mail@kharvd.com 2023-03-13T12:24:18-04:00 GitHub noreply@github.com 2023-03-13T18:24:18+02:00 Fix UTF-8 handling (including colors) (#79) d1f224712d78ab2cbb78777acfeb6739f660eb96 1808ee0500ea674b4bc2911acd0489ee5cbcef87 Pavol Rusnak pavol@rusnak.io 2023-03-13T17:15:20+01:00 GitHub noreply@github.com 2023-03-13T18:15:20+02:00 Add quantize script for batch quantization (#92) 1808ee0500ea674b4bc2911acd0489ee5cbcef87 a169bb889cfe7b77a798f04fbc573e67ccb4316a Georgi Gerganov ggerganov@gmail.com 2023-03-13T09:42:26+02:00 GitHub noreply@github.com 2023-03-13T09:42:26+02:00 Add initial contribution guidelines a169bb889cfe7b77a798f04fbc573e67ccb4316a 460c48254098b28d422382a2bbff6a0b3d7f7e17 Matvey Soloviev blackhole89@gmail.com 2023-03-13T04:08:01+01:00 Matvey Soloviev blackhole89@gmail.com 2023-03-13T04:08:01+01:00 Gate signal support on being on a unixoid system. (#74) 460c48254098b28d422382a2bbff6a0b3d7f7e17 c80e2a8f2adeda202cbffe76ef800f134e51f03f Matvey Soloviev blackhole89@gmail.com 2023-03-13T00:35:51+01:00 Matvey Soloviev blackhole89@gmail.com 2023-03-13T01:04:41+01:00 Fix token count accounting c80e2a8f2adeda202cbffe76ef800f134e51f03f 54a0e66ea0ed3248e6c95a070a2da0bf5c6d4817 Georgi Gerganov ggerganov@gmail.com 2023-03-13T01:28:08+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-13T01:28:08+02:00 Revert "10% performance boost on ARM" 54a0e66ea0ed3248e6c95a070a2da0bf5c6d4817 543c57e991a23121c666561c2837faa09c4a78ca Georgi Gerganov ggerganov@gmail.com 2023-03-13T01:21:03+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-13T01:21:03+02:00 Check for vdotq_s32 availability 543c57e991a23121c666561c2837faa09c4a78ca 113a9e83ebc0f788f861394437087bf3ca0e019b Georgi Gerganov ggerganov@gmail.com 2023-03-13T01:05:24+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-13T01:05:24+02:00 Ammend to previous commit - forgot to update non-QRDMX branch 113a9e83ebc0f788f861394437087bf3ca0e019b 404fac0d623c9eea74ad7a9347da69e33f10984e Georgi Gerganov ggerganov@gmail.com 2023-03-13T00:56:10+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-13T00:56:10+02:00 10% performance boost on ARM 404fac0d623c9eea74ad7a9347da69e33f10984e 1a0a74300f35ad4868715d684d0bc0effdaa9d31 Matvey Soloviev blackhole89@gmail.com 2023-03-12T23:07:34+01:00 GitHub noreply@github.com 2023-03-13T00:07:34+02:00 Fix color getting reset before prompt output done (#65) 1a0a74300f35ad4868715d684d0bc0effdaa9d31 96ea727f4780620b60c1897b538654931411a3fd Georgi Gerganov ggerganov@gmail.com 2023-03-12T23:39:01+02:00 GitHub noreply@github.com 2023-03-12T23:39:01+02:00 Update README.md 96ea727f4780620b60c1897b538654931411a3fd 966195483549e201cff062096a848d9e9833a1a6 Matvey Soloviev blackhole89@gmail.com 2023-03-12T22:13:28+01:00 GitHub noreply@github.com 2023-03-12T23:13:28+02:00 Add interactive mode (#61) 966195483549e201cff062096a848d9e9833a1a6 f385f8dee83d1baf59896b2eb09f1524dc9cde45 Marc Köhlbrugge subscriptions@marckohlbrugge.com 2023-03-13T03:30:08+07:00 GitHub noreply@github.com 2023-03-12T22:30:08+02:00 Fix typo in README (#45) f385f8dee83d1baf59896b2eb09f1524dc9cde45 02f0c6fe7f9b7be24c7d339aed016e54a92388ea Ben Garney bengarney@users.noreply.github.com 2023-03-12T13:28:36-07:00 GitHub noreply@github.com 2023-03-12T22:28:36+02:00 Allow using prompt files (#59) 02f0c6fe7f9b7be24c7d339aed016e54a92388ea eb062bb012c4e131818dd757a6d3a757fdee3961 beiller beiller@gmail.com 2023-03-12T16:23:15-04:00 GitHub noreply@github.com 2023-03-12T22:23:15+02:00 Add back top_k (#56) eb062bb012c4e131818dd757a6d3a757fdee3961 7027a97837c351e0a7bc48db2027af368de382db Sebastián A sebastian.aedo29@gmail.com 2023-03-12T17:15:00-03:00 GitHub noreply@github.com 2023-03-12T22:15:00+02:00 Windows fixes (#31) 7027a97837c351e0a7bc48db2027af368de382db 2d555e5b42922cda6dfc0c3ff54df7b1ee4d0ff4 Georgi Gerganov ggerganov@gmail.com 2023-03-12T22:09:26+02:00 GitHub noreply@github.com 2023-03-12T22:09:26+02:00 Update README.md 2d555e5b42922cda6dfc0c3ff54df7b1ee4d0ff4 7c9e54e55e4106f84688245fb15207f6df917e12 Georgi Gerganov ggerganov@gmail.com 2023-03-12T22:08:24+02:00 GitHub noreply@github.com 2023-03-12T22:08:24+02:00 Add CI (#60) 7c9e54e55e4106f84688245fb15207f6df917e12 b9bd1d014113b7498f04ad4d28e6021d5f4cddad Georgi Gerganov ggerganov@gmail.com 2023-03-12T20:59:01+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-12T20:59:01+02:00 Revert "weights_only" arg - this causing more trouble than help b9bd1d014113b7498f04ad4d28e6021d5f4cddad 129c7d1ea886e52ac1b87ff6184310bab3158806 Oleksandr Nikitin oleksandr@tvori.info 2023-03-12T14:16:33+02:00 GitHub noreply@github.com 2023-03-12T14:16:33+02:00 python/pytorch compat notes (#44) 129c7d1ea886e52ac1b87ff6184310bab3158806 702fddf5c5c3c1377e169ba9ecdfed4cb16c268b beiller beiller@gmail.com 2023-03-12T05:27:42-04:00 GitHub noreply@github.com 2023-03-12T11:27:42+02:00 Add repetition penalty (#20) 702fddf5c5c3c1377e169ba9ecdfed4cb16c268b 7d86e25bf648eb369a3a8388bf239b6b19f7a789 Georgi Gerganov ggerganov@gmail.com 2023-03-12T09:03:25+02:00 GitHub noreply@github.com 2023-03-12T09:03:25+02:00 Clarify meaning of hacking 7d86e25bf648eb369a3a8388bf239b6b19f7a789 a93120236f99e13d77e4b278e47ffcaad4a899e4 Georgi Gerganov ggerganov@gmail.com 2023-03-12T08:41:54+02:00 GitHub noreply@github.com 2023-03-12T08:41:54+02:00 README: add "Supported platforms" + update hot topics a93120236f99e13d77e4b278e47ffcaad4a899e4 6a9a67f0bee5eed67cf8bf03f74f77619da40d3f deepdiffuser 112834445+deepdiffuser@users.noreply.github.com 2023-03-11T22:36:35-08:00 GitHub noreply@github.com 2023-03-12T08:36:35+02:00 use weights_only in conversion script (#32) 6a9a67f0bee5eed67cf8bf03f74f77619da40d3f da1a4ff01f42d058cfa59806dd5679c0fe5a8604 Pavol Rusnak pavol@rusnak.io 2023-03-12T07:36:03+01:00 GitHub noreply@github.com 2023-03-12T08:36:03+02:00 Add LICENSE (#21) da1a4ff01f42d058cfa59806dd5679c0fe5a8604 6b2cb6302ffaf8264e33af1dc52e3ea54003e690 Georgi Gerganov ggerganov@gmail.com 2023-03-12T01:26:32+02:00 GitHub noreply@github.com 2023-03-12T01:26:32+02:00 Update README.md 6b2cb6302ffaf8264e33af1dc52e3ea54003e690 4235e3d5b3f4a0e6844f6291322ebb42181345c9 Juraj Bednar juraj@bednar.io 2023-03-11T18:32:20+01:00 GitHub noreply@github.com 2023-03-11T19:32:20+02:00 Fix a typo in model name (#16) 4235e3d5b3f4a0e6844f6291322ebb42181345c9 f1eaff4721153a5a5094fd1bd8cbdae7a3c079cc Georgi Gerganov ggerganov@gmail.com 2023-03-11T18:10:18+02:00 GitHub noreply@github.com 2023-03-11T18:10:18+02:00 Update README.md f1eaff4721153a5a5094fd1bd8cbdae7a3c079cc a9e58529ea507ac15cd2df4c39d1b9613d6acb6e Georgi Gerganov ggerganov@gmail.com 2023-03-11T17:58:18+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-11T18:04:25+02:00 Add AVX2 support for x86 architectures thanks to @Const-me ! a9e58529ea507ac15cd2df4c39d1b9613d6acb6e 7d9ed7b25fe17db3fc8848b5116d14682864ce8e Georgi Gerganov ggerganov@gmail.com 2023-03-11T17:40:14+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-11T17:40:14+02:00 Fix un-initialized FP16 tables on x86 (#15, #2) 7d9ed7b25fe17db3fc8848b5116d14682864ce8e 0c6803321c818f3f2da4a0693d20128b0f79ad28 Georgi Gerganov ggerganov@gmail.com 2023-03-11T12:44:21+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-11T12:45:01+02:00 Bump memory buffer 0c6803321c818f3f2da4a0693d20128b0f79ad28 f60fa9e50afce35e7ebe1fedf34d4a9327353927 Georgi Gerganov ggerganov@gmail.com 2023-03-11T12:31:21+02:00 GitHub noreply@github.com 2023-03-11T12:31:21+02:00 Update README.md f60fa9e50afce35e7ebe1fedf34d4a9327353927 7211862c943273fc8ce4b7fdf4c04f9821b7b591 Georgi Gerganov ggerganov@gmail.com 2023-03-11T12:26:46+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-11T12:27:02+02:00 .gitignore models/ 7211862c943273fc8ce4b7fdf4c04f9821b7b591 a5c5ae2f545d0d3338f5b2a7840457e35d5bccc1 Georgi Gerganov ggerganov@gmail.com 2023-03-11T12:26:16+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-11T12:27:02+02:00 Update Makefile var + add comment a5c5ae2f545d0d3338f5b2a7840457e35d5bccc1 ea977e85ecda7b983f0e7b1db20b509998ddc889 Georgi Gerganov ggerganov@gmail.com 2023-03-11T11:34:25+02:00 GitHub noreply@github.com 2023-03-11T11:34:25+02:00 Update README.md ea977e85ecda7b983f0e7b1db20b509998ddc889 007a8f6f459c6eb56678fdee4c09219ddb85b640 Georgi Gerganov ggerganov@gmail.com 2023-03-11T11:34:11+02:00 GitHub noreply@github.com 2023-03-11T11:34:11+02:00 Update README.md 007a8f6f459c6eb56678fdee4c09219ddb85b640 5f2f970d51a04b783799bc92fd1d006408269f26 Georgi Gerganov ggerganov@gmail.com 2023-03-11T10:47:09+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-11T11:28:30+02:00 Support all LLaMA models + change Q4_0 quantization storage 5f2f970d51a04b783799bc92fd1d006408269f26 73c6ed5e8784a20f89d51b1703a09bc690c68227 Simon Willison swillison@gmail.com 2023-03-10T21:47:26-08:00 GitHub noreply@github.com 2023-03-11T07:47:26+02:00 Include Python dependencies in README (#6) 73c6ed5e8784a20f89d51b1703a09bc690c68227 01eeed8fb1437978603a8523c0b8ea2f6280f5d7 Georgi Gerganov ggerganov@gmail.com 2023-03-11T01:30:47+02:00 GitHub noreply@github.com 2023-03-11T01:30:47+02:00 Update README.md 01eeed8fb1437978603a8523c0b8ea2f6280f5d7 6da2df34ee40301d9ecb126968ec4c0c6195f26d Georgi Gerganov ggerganov@gmail.com 2023-03-11T01:22:58+02:00 GitHub noreply@github.com 2023-03-11T01:22:58+02:00 Update README.md 6da2df34ee40301d9ecb126968ec4c0c6195f26d 9dcf4dba459ba6482a7a5aced22645a387ec6991 Georgi Gerganov ggerganov@gmail.com 2023-03-11T01:18:10+02:00 GitHub noreply@github.com 2023-03-11T01:18:10+02:00 Update README.md 9dcf4dba459ba6482a7a5aced22645a387ec6991 920a7fe2d94cc7e4fed0e88db830b674c91865c5 Jean-Michaël Celerier jeanmichael.celerier+github@gmail.com 2023-03-10T18:04:06-05:00 GitHub noreply@github.com 2023-03-11T01:04:06+02:00 Add missing headers for memcpy and assert (#3) 920a7fe2d94cc7e4fed0e88db830b674c91865c5 3a57ee59de53c2a9d2c3a2c643b609ce07a58a16 Georgi Gerganov ggerganov@gmail.com 2023-03-11T00:55:22+02:00 GitHub noreply@github.com 2023-03-11T00:55:22+02:00 Update README.md 3a57ee59de53c2a9d2c3a2c643b609ce07a58a16 b85028522d6e924473159ba0da3543fc174d2ded Georgi Gerganov ggerganov@gmail.com 2023-03-11T00:51:46+02:00 GitHub noreply@github.com 2023-03-11T00:51:46+02:00 Update README.md b85028522d6e924473159ba0da3543fc174d2ded 8a01f565ff78cc6c0c5a9fa402787a2f179f2d78 Georgi Gerganov ggerganov@gmail.com 2023-03-11T00:09:19+02:00 GitHub noreply@github.com 2023-03-11T00:09:19+02:00 Update README.md 8a01f565ff78cc6c0c5a9fa402787a2f179f2d78 70bc0b8b15b98dca23b28f0c8f5e34b27e424cda Georgi Gerganov ggerganov@gmail.com 2023-03-10T23:53:11+02:00 GitHub noreply@github.com 2023-03-10T23:53:11+02:00 Update README.md 70bc0b8b15b98dca23b28f0c8f5e34b27e424cda 18ebda34d67c05f4f5584a9209e7efb949f5fd56 Georgi Gerganov ggerganov@gmail.com 2023-03-10T23:46:39+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-10T23:46:57+02:00 Fix a bug in the rope calculation 18ebda34d67c05f4f5584a9209e7efb949f5fd56 319cdb3e1ffe263cf5b08249c9559e011396c1de Georgi Gerganov ggerganov@gmail.com 2023-03-10T21:52:27+02:00 GitHub noreply@github.com 2023-03-10T21:52:27+02:00 Update README.md 319cdb3e1ffe263cf5b08249c9559e011396c1de 775328064e69db1ebd7e19ccb59d2a7fa6142470 Georgi Gerganov ggerganov@gmail.com 2023-03-10T21:50:46+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-10T21:50:46+02:00 Final touches 775328064e69db1ebd7e19ccb59d2a7fa6142470 26c084662903ddaca19bef982831bfb0856e8257 Georgi Gerganov ggerganov@gmail.com 2023-03-10T21:47:46+02:00 GitHub noreply@github.com 2023-03-10T21:47:46+02:00 Create README.md 26c084662903ddaca19bef982831bfb0856e8257 Georgi Gerganov ggerganov@gmail.com 2023-03-10T20:40:58+02:00 Georgi Gerganov ggerganov@gmail.com 2023-03-10T20:56:40+02:00 Initial release