cogito/calculet-npu-research-archive
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d2677d2205d50aab55fdbd1cbe559a6584989433
calculet-npu-research-archive/history/llama.cpp/topics/02-concurrency-parallel-async.tsv
T

108 KiB
Raw Blame History

1f131bf1908b8c5dd1f49d7ae7f616506fc47166633d96dae28e17208ef61a71dba40cda3c04f135a2026-05-27T09:38:51+08:00wangbomengfix patch of qwen2.5vl,n_patch of minicpm = 64,fix n_ctx_slot,comment out M-ROPE
22716130010c58f1cdd9d94f0e2f8e047bc92dc66b35bda124ccd4ed581dd6088d3ffa4931c2809b62026-05-18T08:40:54+08:00Yunzhe Jiafix n_ctx_slot error by adding runtime capacity loading for cal-llm
3ecf01a17651cd7b1e8b85fd6075e83831463ee2399d2078e89305035d87d966cee7987c7d50b39252026-04-09T09:47:55+08:00Yunzhe Jiafix n_parallel problem
41f5accb8d0056e6099cd5b772b1cb787dd590a132759ccdb4adc8568add4316780d5e675519b07752025-11-04T05:04:59ZNoahFix garbled output with REPACK at high thread counts (#16956)
5bcfa87622ae46be6345a8e3dfdbdc5ba5414042ba2054e3a8ff0da3978a4acc18c349ff58554d3362025-11-03T00:41:08+01:00Sascha Rogmannfeat(webui): improve LaTeX rendering with currency detection (#16508)
6cd5e3b57541ecc52421130742f4d89acbcf77cd487c9efc3b297b8a498716b1db3d061842e6fc85b2025-11-02T18:14:04+02:00Georgi Gerganovserver : support unified cache across slots (#16736)
72f68ce7cfd20e9e7098514bf730e5389b7bba908e4a71599e5846110159955dec0008eb4aa24222b2025-11-01T19:49:51+01:00Pascalwebui: auto-refresh /props on inference start to resync model metadata (#16784)
8d3dc9dd898be805c23a408cc36daed5b3bf29221bea04522ff1a0d8559ccfd353aa018dcfbb608cc2025-11-01T06:13:26+01:00Oliver SimonsCUDA: Remove unneded bias/gate dims in fused mmvq (#16858)
90de0a01576772032008a689afc4d7c80685074c4e58d585604bbbbbc24f8149effe444621b5191c62025-10-31T21:20:47+01:00Piotr Wilkin (ilintar)model : Minimax M2 (#16831)
10b52edd25586fabb70f0c21b274473b307cf14499517b7170e1a4d733583c4b07c5b7a49acc05911c2025-10-30T18:42:57+02:00Georgi Gerganovserver : remove n_past (#16818)
118b11deea4663f29d3e042ce1056ba643264cd5f1b9ce94017729465895402cbcfffb51fa926c15e32025-10-30T04:34:15+01:00Oliver SimonsHide latency of bias and gate-loading (#16847)
12b9ce94017729465895402cbcfffb51fa926c15e33464bdac37027c5e9661621fc75ffcef3c19c6ef2025-10-29T15:13:10-05:00Jeff Bolzvulkan: Fuse rope+set_rows (#16769)
13bcf5bda6f5df559565d11d7c8e8295c1159a85ec3eb2be1ca5f37480aeb16102970d9e65f43347fe2025-10-29T14:39:03+01:00Ruben OrtlamVulkan MMQ Integer Dot Refactor and K-Quant support (#16536)
143eb2be1ca5f37480aeb16102970d9e65f43347fee41bcce8f0b53032a1fed275cd253e931c041cf62025-10-29T06:29:12-07:00Max KrasnyanskyHexagon Op queue & dispatch optimizations (#16820)
15338074c383c81366320d176d83b94b0a567ee0c2851553ea6b24cb39fd5fd188b437d777cb411de82025-10-29T08:14:39+02:00YaelLogicsycl: add RMS_NORM_BACK operation support (#16808)
1685a7d8677bf2200981e52f744a21d5267964ffcfa8ca18b4b815a2abdbecb958ee5f4c542d69aac72025-10-28T20:19:44+02:00Georgi Gerganovmemory : remove KV cache size padding (#16812)
171c1409e13169d0ced4b1b4d39fb5b268b75250917a0e900e3615fa46c074a7fdf900b47d3c0a1c7e2025-10-28T03:51:41-07:00Sam Malayekembedding: add raw option for --embd-output-format (#16541)
183479efd112b3910d2d008ad08fe4cb4895605903463bbf20bfbe0da10ac58984d4d62bed5a49362a2025-10-28T10:54:53+08:00Chenguang LiCANN: Improve device ID handling and aclnnArange checks (#16752)
19ad8d36beffd791db10c94eb9e964afb891e3ca55c053e18a66dd95dc340aa61317877c2a41d4e3cf2025-10-28T03:50:33+02:00tamarPalsycl: add SSM_CONV operation support (#16800)
205a4ff43e7dd049e35942bc3d12361dab2f15554410640e31aab0819f31c1e1f2d008b019ee7372322025-10-27T13:51:28-07:00Diego Devesallama : disable pipeline parallelism if compute buffer allocation fails (#16748)
2155945d2ef51b93821d4b6f4a9b994393344a90db0bcb40b48c6fc6f17ba9672625e526ab2574344b2025-10-25T03:39:37+08:00leejetggml: fix CUDA grid launch condition for large block_nums.y in binbcast (#16742)
22fe6a9882acf5c02f96624ed8f80144100d7006cb061f0eff02dc9a82f7bd850db3bd70b8a0b5e87a2025-10-23T17:07:31+05:30Prajwal B MehendarkarManually link -lbsd to resolve flock symbol on AIX (#16610)
239de9672adb0f4ca4e39483ac3ffed52b3f70a55d63d2fc46e17a06be5b4b5823a5ada088317f1f0a2025-10-22T20:05:15-05:00Matthew Michelsycl: use async memory allocation to fix crashes during graph recording (#16644)
24d8eaa26e4d9228df3aa46a930db60c8eaab67c1b9285325ce0174631c3cd6121d56084adc4ef2d8f2025-10-22T12:01:22+02:00Aclytests : fix test-thread-safety when compiling with multiple backends (#16699)
256de8ed75196c7cd98c1f34bbf3a7452451ba8ac284bf3c677857279037adf67cdcfd89eaa4ca92812025-10-21T01:21:12+03:00YehuditEsycl : add PAD_REFLECT_D1 operator support (#16145)
2613f2cfad4170c096c51a02c24a6a158cb47f148006332e28672356b964d6dfc2ba4657e20581cd432025-10-20T12:41:13+02:00Aleksander GrygierEnable per-conversation loading states to allow having parallel conversations (#16327)
2706332e28672356b964d6dfc2ba4657e20581cd4372d53e6c4decee8b339e49aed8cc0e234b9639dc2025-10-20T16:27:09+08:00takuya kodamallama-batch: fix build fails with `-Werror=missing-braces` (#16614)
28342c728d031d50673feded797520a44127d73379ababae7e1ec3e9cfdab0322ee55ea3389e82a4d52025-10-17T18:01:23+08:00muggle-stackggml : fix SpaceMit IME array out-of-bounds in task assignment (#16629)
299ad4f1931ee0f3b41d9355245ef744786aaae0aa79967ec596c0dacfd2251b085a57e79df292b1cc2025-10-17T02:33:58-04:00Ilia Ilmermetal : add `CONV_TRANSPOSE_2D` (#16542)
30683fa6ba4ed3a23b939d3e11e6dc860bd47a0ccfb22572e97dc51757d3ebe917a5a283385010ec682025-10-16T16:28:41+02:00Pascalfix: added a normalization step for MathJax-style \[\] and \(\) delimiters (#16599)
316f5d924637a15abedb111cbbffd7da5f31c81855adc9b60f190c1016a09f439862fa1cbb302262ac2025-10-16T01:11:33-04:00takasurazeemcommon : Update the docs on -t --threads (#16236)
32adc9b60f190c1016a09f439862fa1cbb302262acee50ee1eadff58777ae746827b04de7ba0befc552025-10-16T13:10:32+08:00takuya kodamaggml-cpu: replace putenv with setenv for const-correctness (#16573)
337ea15bb64c81e3813eb0babf9a57e1bc5697f5699c7185dd28416cf67f5e3b268381f311b5e3da562025-10-14T07:51:36-05:00Jeff Bolzvulkan: Improve build time for MSVC (#16545)
341fb9504eb744969a990bfe4cfcf1d3d7a479541c3f750f8d760ab5a61491e6a9409072dfeee4b4d72025-10-13T10:55:32+02:00Pascalfix: add remark plugin to render raw HTML as literal text (#16505)
35f9bc66c3ebcfddb5f09e4b21253623caeb8e414aa31cf36ad946a13b3a646bf0dadf2a481e89f9442025-10-13T08:52:22+08:00hipuddingCANN: Update several operators to support FP16 data format (#16251)
36d00cbea63c671cd85a57adaa50abf60b3b87d86f8328fd4bae76fc027f8ca0e9a05acd3788dabe3f2025-10-09T18:54:51+03:00Georgi Gerganovserver : host-memory prompt caching (#16391)
37e08db4259521de493b7aeb49dadf29ebd1ee966a12bbc3fa50b6df03318a4451c9a2210200a0b28d2025-10-09T08:39:18+02:00Saba Fallahmodel: EmbeddingGemma Adding Support for SentenceTransformers Dense Modules (#16367)
3812bbc3fa50b6df03318a4451c9a2210200a0b28d9d0882840e6c3fb62965d03af0e22880ea90e0122025-10-08T22:18:41+02:00Pascalrefactor: centralize CoT parsing in backend for streaming mode (#16394)
39b2c08c9ec4cd59ed88c1ebdd94f109af5ccc978e7fdd16b432d247121fe5fe7b21f8805f85266c852025-10-08T10:57:53+03:00Georgi Gerganovmetal : mark FA blocks (#16372)
4074b8fc17f92ada295a648e3c5eb28f46bca7d892aeaf8a36f06b5810f5ae4bbefe26edb33925cf5e2025-10-07T13:48:56-07:00Reese Levineggml webgpu: profiling, CI updates, reworking of command submission (#16452)
411d6092fc72f4d10f4486ac95edfd414bc08b62b88ae32dc9ecd7aeebf3a5b43557e0552a0a04cd4f2025-10-07T08:22:35+03:00Georgi Gerganovtests : add -INF blocks to the KQ mask in the FA tests (#16380)
42ca71fb9b368e3db96e028f80c4c9df6b6b370edd35266573b968e1c947b367782fb4b3eddbb4f3c02025-10-05T06:57:47-06:00Gabe Goodhartmodel : Granite docling + Idefics3 preprocessing (SmolVLM) (#16206)
4335266573b968e1c947b367782fb4b3eddbb4f3c086df2c9ae4f2f1ee63d2558a9dc797b98524639b2025-10-04T20:59:31-07:00Reese Levineggml webgpu: actually add softmax, fix rms_norm offset (#16400)
4486df2c9ae4f2f1ee63d2558a9dc797b98524639bf39283960b58a92ecc0c72567711318b20e22b552025-10-04T20:04:27ZEvevulkan: use a more appropriate amount of threads when generating shaders (#16418)
450e1f83855609d73beaf05d818640b6cfd39d287bad126479c25cf983a0f994a08ba0911cf49ed62b2025-10-03T04:52:46-05:00Jeff Bolzvulkan: Fix FA coopmat1 invalid array indexing (#16365)
46d64c8104f090b27b1f99e8da5995ffcfa6b726e2ef07a4090672a3438d7f64f197795d7dc1c189572025-10-02T20:10:12+02:00Sigbjørn Skjærettest-barrier : do not use more threads than physically available (#16389)
472a9b63383a448ec18c754dfdc6e95cb853940a521104ca1a1c926b832274694a2773a17066982ad02025-10-01T15:54:42+02:00Aleksander GrygierImprove code block color theming (#16325)
48f1eb1cb1eba042b2d583234e80f65e2e225d8995de41f2b7bffab7582944b213ce12b605f8cf6e0f2025-09-30T09:07:20+02:00Charles Xukleidiai : fix work size and threads sync for fp16 (#16246)
495f7e166cbf7b9ca928c7fad990098ef32358ac75d72f5f7ba260b546190338b0b76f2f152581424f2025-09-29T18:49:47+02:00PascalFix thinking blocks with quotes + add handling `[THINK]...[/THINK]` blocks (#16326)
50c0bfc57af421f8fd63c946c13b7666aed82560e275a3a6c2cd0002ba40e2dcc92007bc9fdbc69f1a2025-09-28T00:49:32+08:00Aman GuptaCUDA: mul_mat_id for mmf for bs <= 64 for f16 and bs <= 32 for f32 (#16277)
51624207e676ab5eb3ce7af631902bb45fb73a8359807e8c6d310952f2f5656afc63dab9d7083dcb5c2025-09-27T02:03:33+08:00Aaron Teodevops: add s390x & ppc64le CI (#15925)
529b26511857ac09ae69ab485168fe2d3ee5fb1d6e00217cd41388328c93e9e9644921c4319bb03bcc2025-09-26T18:27:25+08:00Aaron Teoggml-cpu: implement MXFP4 SIMD for s390x (#16193)
53a86a580a6692edd1da076d5422f944c344b4fe5e0f7c69689f4e681948a6005adea9bee9c08ff9032025-09-26T08:56:38+08:00R0CKSTARmusa: upgrade musa sdk to 4.3.0 (#16240)
540f7c69689f4e681948a6005adea9bee9c08ff903835b2b915c52bcabcd688d025eacff9a07b65f522025-09-26T08:56:10+08:00R0CKSTARmusa: fix build warnings (#15611)
55f2a789e33490deb483a2694b066b37e45524bb793a599719673c850647e3bb911ed6d91109bb91d22025-09-24T16:17:49+02:00Aclyggml : split graph allocations according to backend max buffer size (#15815)
563a599719673c850647e3bb911ed6d91109bb91d263b54c81a620981be020184ab99e63a8e50e47cb2025-09-24T13:42:26+02:00Tarek Dakhranmodel : add label for LiquidAI LFM2-2.6B model (#16204)
571eeb523c3e0c7ffbd59469f5463dcbdecba3535e5bb4a3edec297e74b0f7bd4ed5d0fdd12e28d8582025-09-21T08:31:55+02:00Giuseppe Scrivanovulkan: optimize UMA buffer operations and fix driver hangs (#16059)
5869ffd891631befa9e6b485fd646a16dab4f2c007246c0d9c795fb25da8268625d597580155a3672f2025-09-18T23:07:26+02:00Adrien Gallouëtggml-amx : fix ggml_amx_init() on generic Linux (#16049)
59d304f459d8619399eb232b1e3689379306f439d30320ac5264279d74f8ee91bafa6c90e9ab9bbb912025-09-17T13:09:40-07:00Reese LevineGGML WebGPU: Support for ADD, MUL, RMS_NORM, GET_ROWS operators (#16018)
60cd08fc3ecc0264b4414b68af3874a6c689ed60c1cb5bb6cc05119c24e7711ca2956cd0e6d409d3962025-09-17T01:08:02-07:00David Ribeiro Alvescommon : Fix corrupted memory error on json grammar initialization (#16038)
61d5fabe3682de515fd09d6c981f7a0d1b756144558ff206097c2bf3ca1c7aa95f9d6db779fc7bdd682025-09-17T14:33:08+08:00Chenguang LiCANN: Optimize ggml_cann_set_device (#15935)
62f161463a54d9f93d41246286aa4a9569a91d804d84d7b2fca11d1be118ce776f6d72a486c4883b742025-09-13T13:54:28+03:00Georgi Gerganovmetal : allow ops to run concurrently (#15929)
630f0a3c2851134d49955f3c85afbb0b1bb47c3e0733daece86b65607451d0d4378d2d04ba6a20ad552025-09-10T17:52:35+03:00Georgi Gerganovmetal : make the backend async (#15906)
64a972faebed5fdc4a3d2a844d92d476058c02e02d550cf726e133fd0a069d991287fd3a2a3e3e1cbd2025-09-09T14:38:02+08:00Aman GuptaCUDA: Add mul_mat_id support for the mmf kernel (#15767)
65e68aa10d8f3d26fdad5b912540362d79de5460e30a16bf52e6874369cb4fd2bdc4863ef984b688e72025-09-08T13:10:07-05:00Jeff Bolzvulkan: sort graph to allow more parallel execution (#15850)
66b0d52998b962bd2681c34bf52af993af79f178b8f28d4f4ac963f182ea9d0fe9e269f3f5f3782aaf2025-09-08T13:56:51+03:00Georgi Gerganovcuda : fix supports_op condition for get_rows when number of blocks is too large (#15868)
67fd621880f3fd908424e675a41715a2dc760247a24281c7b315f8a904549fe527039b976e08098d1a2025-09-05T17:32:39-06:00Gabe GoodhartaLoRA Support (#15327)
68d1e2adba65208d6de3d7f6f23b00c562ff5bb777c1c354e44c06d259679bb5bb7a8fa9f0b28480e42025-09-04T15:40:44+02:00Daniel Beveniusllama : set n_outputs to 1 to avoid 0 outputs mean-pooling (#15791)
69c1c354e44c06d259679bb5bb7a8fa9f0b28480e4a68d9144262f1d0ef4f6ba7ad4a7e73e977ba78c2025-09-04T20:20:14+08:00Chenguang LiCANN: Refactor ND to NZ workspace to be per-device (#15763)
70661ae31c9c68201577e70278285b349a5a662caf407c23786dd0d3a503e9429eead96e611d3950c92025-09-03T19:59:16+02:00Oliver SimonsCUDA: Optimize `rms_norm_f32` kernel and its fused variants, giving 1-6% perf E2E (#15715)
7102c1813517412f3e00aa6ca7c0273fea64edb49277dee9de97be75b7143a213bc48893e0c0b29af72025-09-01T16:19:07+02:00Ruben OrtlamVulkan: Add Integer Dot Product mul_mat_vec shader for legacy quants (#14903)
720d161f021aa33ec0e90cce96f5d1a889255573274efd5a83163ff383285b3a4c2106feabf5c695572025-08-31T20:11:58+03:00Georgi Gerganovserver : enable /slots by default and make it secure (#15630)
73c37052ab4d6d1ae73c0e90bc6e560cc6409e13115c16b9c87d840e4d5d55fa83c732c6b693346f402025-08-31T02:06:43-05:00Jeff Bolzvulkan: mul_mat_id coopmat2 optimizations (#15546)
74792b44f2ed9668cce7f267ff0ae4950ed9b4a5de81017865ee444cf49ce0136f2be1e41a0270ff912025-08-29T19:25:40+02:00ExtReMLapinserver : add documentation for `parallel_tool_calls` param (#15647)
75e8d99dd0b67f2ecc1e45fca8074a3a18c3e036d2a8bca68f727844e7dcf24a956003b3c2039ea5632025-08-28T18:39:31-06:00Gabe Goodhartnvidia nemotron nano v2 (nemotronh) (#15507)
76c8d0d14e77c3c45df5cbbddde9c2b4c377ec4e7a84ab83cc0b4b7e769451ee48e4c7d1acef91ef252025-08-28T17:09:05+03:00Georgi Gerganovkv-cache : fix find_slot to not search for continuous slot (#15638)
7764387f6e95434b393ac3df285864692b7fd9c4d2d35a1e8c41f747548775225973a99507896a8c612025-08-28T10:56:41+02:00Aleksei Nikiforovgguf-py: byteswapping improvements (#12851)
78a6a58d64785cb458ed9de52f391aa38142d38d640373486dbc0dccbdcb3b5fdd65759d88cec061962025-08-26T21:05:25+05:30shalinib-ibmllamafile: PowerPC Sgemm Optimization (#15558)
790115cfb7c1dec0753aecb34f3af4338bf9de8eeb3a617cb18121a85eed17ac5585788810c5d9c1e0 0d5a470223fc90b6b6807921d68011ff06ae7f9e2025-08-26T15:19:56+08:00Yunzhe JiaMerge branch 'master' into dev
805eff6ec9b1220b599a43b594b1110487ab6aca08dfd9b5f6c7586c88588f06a644c131bec071a0a12025-08-25T17:23:40+02:00Johannes GäßlerCUDA: MoE helper in device code, better tile sizes (#15525)
81611f419cff11e4952228162a1c44cb35dff2274ab1afcab804e3281867a5471fbd701e32eb32e5122025-08-23T13:16:17-05:00Jeff Bolzvulkan: optimize rms_norm, and allow the work to spread across multiple SMs (#15281)
82b55f06e1aa67fb10e89f53e31bbccf37eb2678ea0a9b43e507a359ca392c037cf341f55137ad0b692025-08-23T14:58:57+08:00R0CKSTARvulkan.Dockerfile: install vulkan SDK using tarball (#15282)
83330c3d2d21b55bca5517db7d2eea2ea8f131df4ae92734d51bcb82cc35f0a6b5a14928f0036b2c902025-08-23T01:31:54-05:00Jeff Bolzvulkan: optimize mul_mat_id loading row ids into shared memory (#15427)
848ad038c0fdc719ced9fbf921a02cbae9ad79287f5682a3745f2b653dcb855d5766d8edc318fb33362025-08-21T11:06:05+08:00R0CKSTARmusa: add GGML_UNUSED_VARS (#15446)
85a094f381432d92c4bf92d2d6167284316ba73a62fb22dd07a639e81c7415e30b146f545f1a2f2caf2025-08-20T10:17:37+08:00R0CKSTARmusa: fix build warnings (#15258)
8667f09a3a27db443f9870aac87e163dba0d08131e6424594c56f4dbd0573455d89a0d89a0ac093d132025-08-19T18:33:47+08:00R0CKSTARmusa: handle __hgt2_mask, available starting from MUSA SDK rc4.3.0 (#15413)
8700f35d509e5367bf19ccaf4b4adddc38db4811c66028bf74351d35a06bd98498624f8c2f029f7d1a2025-08-13T11:09:39+03:00Georgi Gerganovggml : repack block_iq4_nlx8 (#14904)
886028bf74351d35a06bd98498624f8c2f029f7d1abc5182272c373267352bc689e5fca276934bea2d2025-08-13T10:04:46+02:00Oliver SimonsCUDA: Optimize `reduce_rows_f32` kernel, leading up to 25x perf improvement on kernel-level and 10% perf increase for Gemma3n (#15132)
8925ff6f7659f6a5c47d6a73eada5813f0495331f0be48528b068111304e4a0bb82c028558b5705f052025-08-12T10:02:51+08:00R0CKSTARmusa: fix failures in test-backend-ops for mul_mat_id op (#15236)
90cd3069dfcbeee8e0e96cffb93b0ebb9e595e273a50e81bdf5db563ab57a9a722b08f96fa8a76c9272025-08-11T11:21:19+02:00Daniel Beveniuskv-cache : log (debug) all streams in find_slot (#15176)
9179c1160b073b8148a404f3dd2584be1606dccc6634c9d765bf173c551398f1e7fa4595019bc53bab2025-08-09T13:29:43-05:00David Zhaocuda: refactored ssm_scan and use CUB (#13291)
9234c9d765bf173c551398f1e7fa4595019bc53babe54d41befcc1575f4c898c5ff4ef43970cead75f2025-08-09T20:00:24+08:00Aman GuptaCUDA: add attention sinks for tile and wmma (#15178)
939515c6131aecaccc955fdedcfe16c3e030aaefcbfd1234cb468935ea087d6929b2487926c3afff4b2025-08-05T16:26:38-07:00Reese Levineggml: WebGPU disable SET_ROWS for now (#15078)
94587d0118f50b7e8f4bafbcdd218aefd9da0272e15aa1105da24a8dd1661cea3db0582c9b2c2f54d32025-08-04T08:52:43-07:00Reese Levineggml: WebGPU backend host improvements and style fixing (#14978)
953025b621d12a6931ff5e9775d4f644719980ad91ec0b18802c91badd3ff1388ffd09ee163251bd722025-08-02T17:20:40+08:00R0CKSTARllama-bench: rename DB table name from test to llama_bench (#15003)
96baad94885df512bb24ab01e2b22d1998fce4d00eba42794c9ead96ad52311ba1b23eefcbf3d6f63d2025-08-01T11:50:33+05:30Srihari-mcwggml : Q2k interleaving implementation - x86/x64 SIMD (#14373)
97484b2091ce5017901483b5204c07878f171d1441daf2dd788066b8b239cb7f68210e090c2124c1992025-08-01T08:47:27+08:00R0CKSTARcompare-commits.sh: support both llama-bench and test-backend-ops (#14392)
988a4a85627702b569d7d2810f2de06a4321656e9d11490b36723d511d75fb601995c79b5c363ba3a22025-07-31T19:49:09+08:00Aman GuptaAdd LLaDA 8b Diffusion model (#14771)
9900131d6eaf4df029e1ec84de868c2c59575030071e15bfd42c3938506e0da5939bf7f42780965f012025-07-30T15:12:02+03:00Georgi Gerganovtests : update for LLAMA_SET_ROWS=1 (#14961)
100cd1fce6d4f9c191f1c7429cc96f61281c3b63ffc00fa15fedc79263fa0285e6a3bbb0cfb3e3878a22025-07-28T20:32:15+05:30Akarshan BiswasSYCL: Add set_rows support for quantized types (#14883)
1019b8f3c6c776d77045ee4f7f13fdf7863dfe59e5bc7f3169cd523140a288095f2d79befb20a0b73f42025-07-26T10:36:02+08:00R0CKSTARmusa: fix build warnings (unused variable) (#14869)
102793c0d7f46384001738c337d7afa46b45ae32745ce111d39d666f4a3b6a561ad020f6feb8cc677902025-07-25T10:47:39-06:00Gabe Goodhartmetal: SSM_SCAN performance (#14743)
103c12bbde37258c6d053322d1cedd4a9672109ae583f4fc97f1d745f1d5d3c853949503136d419e6de2025-07-25T01:07:26-07:00Diego Devesasched : fix multiple evaluations of the same graph with pipeline parallelism (#14855)
1043f4fc97f1d745f1d5d3c853949503136d419e6de2df255da3cea108de0ae9b302ffdd31674b6d88d2025-07-25T03:05:37+08:00R0CKSTARmusa: upgrade musa sdk to rc4.2.0 (#14498)
105cb4a63aad6650c2b536a7578403935388cb2920e86f5623d904cfd392fdeb14a143097b4074660f62025-07-24T11:09:57+01:00Alberto Cabrera Pérezsycl: fixed semantics of block offset calculation (#14814)
10648b86c4fdb1b1246d9fe17d2d53e3a1c2a0c324538d3af1b73302377111e88ddd7252576383392862025-07-22T07:45:26+08:00R0CKSTARcuda: remove linking to cublasLt (#14790)
1076c9ee3b17e19dcc82ab93d52ae46fdd0226d4777cd465d823c378853f1f6570eebfb77f69c7e1d392025-07-21T19:03:19+02:00rmatifopencl: add conv2d kernel (#14403)
10890083283ec254fa8d33897746dea229aee401b37d4b91ea7b2da253e1355b503f0fcb7b428ce005d2025-07-19T12:51:22-04:00compiladeimatrix : use GGUF to store importance matrices (#9400)
1092adf8d83acdb9b1bf58db6c9729ac9dc6847a58b021cc28bef4dd7d0bf9c91dbbd0803caa6cb15f22025-07-18T17:33:41+03:00Georgi Gerganovparallel : add option for different RNG seeds (#14757)
110086cf81e88fb75287b71ff19c08a206b7bc2e02fd9b691081c04ec5fb0daa9d2b979f915c142963d2025-07-17T09:22:11+02:00Tarek Dakhranllama : fix parallel processing for lfm2 (#14705)
11121c021745d781edf9c44b4972ef6cbbf53b0ecffb0f0ecc3dce806c68609d375a2b3edc430d8db182025-07-16T08:18:51-07:00Reese Levineggml: Add initial WebGPU backend (#14521)
112225e7a1438f4ea85eaa7b5ef3ab3b266ee4d9c06ab140198211385b85eeeb0abd549a4bbe259e10d2025-07-16T16:35:42+03:00Georgi Gerganovllama : add high-throughput mode (#14363)
113e4841d24d3485ea4af54f8b65a19ec3123f0ff3c538cc77f7f44dfa047dba6a06d90c86dda69cf1d2025-07-16T19:12:22+09:00Shunta Saitollama : fix parallel processing for plamo2 (#14716)
114cbc68be51d88b1d5531643b926a4b359c3cff131bdca38376f7e8dd928defe01ce6a16218a64b0402025-07-15T15:28:53+08:00R0CKSTARcuda: fix build warnings in set-rows.cu (unused variable) (#14687)
1157de5c7cab61d4da4387ed9b216f88b96297bcc2d8eff95544e817704d44bec20f9fc956ce76a33be2025-07-12T21:31:38+08:00Aman GuptaCUDA: add set rows for f32 and f16 (#14551)
116b3ad3a0191994d6c47b2bd389d5c7431526ecd2c98197e5c98388470030d908f355ec5937dcccaaa2025-07-12T05:12:26-05:00Jeff Bolzvulkan: support SET_ROWS (#14587)
1170aedae00e6fb48680324a5ac5da9cba0e35de6b56bdda13981d6c8189b7dc4f9fb8ecb91c21529f82025-07-10T18:20:13-06:00Gabe Goodhartmodel : Granite Four (#13550)
118704bb7a71c01dc07c1478b85f6322bf5dfde1eaf435a6d10d618a015060e45a38c7e9f27f42433162025-07-10T13:59:38+05:30Akarshan BiswasSYCL: Initial set_rows kernel implementation (#14562)
1194a5686da22057867c23bd4a6be941ddc8c51e58598bab638fb28cf95a5a66dd2d51b40d6c8f6d69a2025-07-09T14:59:57-04:00compiladellama : support Jamba hybrid Transformer-Mamba models (#7531)
1206efcd65945a98cf6883cdd9de4c8ccd8c79d219a699f4392a33f57c3352cf8d60bdc53db7ca235e72025-07-08T13:11:42-05:00Jeff Bolzvulkan: optimize flash attention split_k_reduce (#14554)
12168155c66f0e76680f34442247e589a090add22d3e1a7059053a9b8958f2d57a21fc46dbc7fb24f8e2025-07-08T07:58:30+08:00R0CKSTARmusa: fix build warnings (unused variable) (#14561)
122a0374a67e2924f2e845cdc59dd67d9a44065a89cddef99522d1ba74193b7394e803fab8db5c78bae2025-07-05T02:26:04-05:00Jeff Bolzvulkan: Handle updated FA dim2/3 definition (#14518)
123a70c8a0c4b4c1606cd9a0ba889ce61aa886100959067487c4411efb20400103fcccfdd389c80d4282025-07-03T10:53:35+03:00Georgi Gerganovkv-cache : use ggml_set_rows (#14285)
1245d46babdc2d4675d96ebcf23cac098a02f0d30cce17991c466ac835b2c71bd813c1ca7ff8dd97b942025-07-02T13:10:24-04:00compiladellama : initial Mamba-2 support (#9126)
12555a1c5a5fdefef808e95aabd3d5563af1068cc8012a81af45f0dbbab24bd819a15f57c03ceb1be902025-07-02T20:34:24+08:00Aman GuptaCUDA: add softmax broadcast (#14475)
126a0535ffa0d35fccfec3e1a0a3bfc9dbb6054d7c0bd9c981d7226107f18deb8344c3301450311bb8b2025-06-29T11:04:10+02:00Sigbjørn Skjæretggml : implement REGLU/GEGLU/SWIGLU ops (#14158)
127bd9c981d7226107f18deb8344c3301450311bb8b27208bf657cfe7262791df473927225e48efe4822025-06-29T02:43:36-05:00Jeff Bolzvulkan: Add fusion support for RMS_NORM+MUL (#14366)
12800d5282c7f2a0bb05bb315fb81ca3b0f42cf9f07566c16fcce44876a167c37f159085afe6f84b28c2025-06-28T10:17:09-05:00Jeff Bolzvulkan: lock accesses of pinned_memory vector (#14333)
129b25e92774e2fa4ee3820e458d5cf43f40190f8d26609507a910aa7437aaa53fd999447de3947d9982025-06-28T17:35:41+08:00Xinpeng Doufix async_mode bug (#14432)
1305783ae43599400b723b5da0569c1f848419ff3c7bf5bcd0b857db420235e03639f0a5f218a7f8cf82025-06-26T15:50:15+03:00Georgi Gerganovmetal : batch rows copy in a single threadgroup (#14384)
1318f52c6f8abb2ed9ad967cc0bfb90acb073011193d2f325130c4ed77a3dcff2c23a926587fb08e2cc2025-06-26T15:10:45+08:00Yunzhe JiaSquashed commit of the following:
132716301d1b03c31875ec3b24526c48c8b1bd0fd8c60ef23d6c14d325d83eae5752e5de39ad268e9b02025-06-26T12:11:59+08:00R0CKSTARmusa: enable fp16 mma (all) and cublas on qy2 (#13842)
13373e53dc834c0a2336cd104473af6897197b9627762af464227dafa1c55e0535bcb24346326748f462025-06-24T11:46:25-07:00lhezopencl: ref count `ggml_backend_opencl_context` and refactor profiling (#14254)
134fa4a9f2a1ccda2573189a9d4995bdf0bceb41156238005c2dc67426cf678baa2d54c8817016932882025-06-22T22:16:26+01:00Ed Addarioquantize : handle user-defined pruning of whole layers (blocks) (#13037)
135bb16041caef45cd4348cd6f84906b5dfec7a1f6a58cba76a9aab728717509d62b67c13afd8dc227a2025-06-21T08:17:12+02:00Markus TavenrathAdd support for VK_EXT_debug_utils to add labels to Vulkan objects. (#13792)
13667ae5312e255ae97852a4a216e2245580bfafd72692e3cdd0a069ab56411b64506a67537d767683e2025-06-21T08:04:18+03:00Georgi Gerganovmetal : fix thread-safety (#14300)
1378308f98c7fb778e54bf75538f5234d8bd20915e96369be07359d03723f38c0a4a014ff0f698a07382025-06-20T15:07:21+02:00Nicolò Scipionesycl: add usage of enqueue_functions extension (#14244)
138e28c1b93fd7d3f8faf9551d962e8a65fe2122e38d27b3ca1758dfb1718e333d497ef4b68ad109bc22025-06-20T04:57:36-07:00Diego Devesacuda : synchronize graph capture and cublas handle destruction (#14288)
139edc4a29effe716956fdfd2bc5b9cba2a6d8492f8ed3290ab34493fd3d2e0f925f816a401da4f2dfd2025-06-19T00:08:14-05:00Gabe Goodhartmemory : Hybrid recurrent cache (#13979)
1408d947136546773f6410756f37fcc5d3e65b8135d50d2227953ca9024f04255b4f116d06fcc0db74c2025-06-19T01:10:26+08:00Aaron Teodocs: add s390x build documentation (#14264)
141fe9d60e74a6cb71bcaed2029377bfa2872b4abb0e434e69183fd9e1031f4445002083178c331a28b2025-06-17T17:48:08+08:00R0CKSTARmusa: fix build warning (unused variable) (#14231)
1426adc3c3ebc029af058ac950a8e2a825fdf18ecc60dbcabde8c006d5cf781ca0fe071c41559572a722025-06-16T08:11:43-07:00Diego Devesallama : add thread safety test (#14035)
143c89c2d1ab94b11845240b7d3313c87691ea18d883555b3004ba7687be3d734acade52a3345758aa42025-06-16T00:21:08-06:00Jeff Bolzvulkan: mutex around vkQueueSubmit (#14127)
144c311ac664d68d10781a3e7b9f02d9d9520837d80b9912ac570de8945ae9383c9ca8291027bf287dd2025-06-15T10:08:58+03:00Georgi Gerganovcparams : rename LLAMA_MAX_PARALLEL_SEQUENCES to LLAMA_MAX_SEQ (#14188)
145c61285e7396c8e526fe7794c19e8d4f1c99bfc5109cf2c7c655c90e53e100f29b830a788bab0653d2025-06-13T08:45:37+01:00Ewan CrawfordSYCL: Bump oneMath commit (#14152)
146bd248d4dc7265437e1918dc53ae3f49a0c592e5f7781e5fe99f2a4fc1c8af0a8488eedac4644cb722025-06-11T09:48:52-05:00Jeff Bolzvulkan: Better thread-safety for command pools/buffers (#14116)
147dad5c44398b78467943ed0a603ea427fe9f6fc6255f6b9fa6563f6ae49113f9abdc980c12348cc1c2025-06-10T18:20:14-04:00compiladekv-cache : avoid modifying recurrent cells when setting inputs (#13834)
148e21d2d4ae26d6c5daf1602a1061aa4f8e722ca57dc0623fddb661926e0998538895155e4f081ff092025-06-09T19:47:39+08:00Xinpeng DouCANN: Simplify the environment variable setting(#13104)
149dc0623fddb661926e0998538895155e4f081ff0987d34b381d5868e75586210ec17b5ef5deddc2762025-06-09T18:01:17+08:00R0CKSTARwebui: fix sidebar being covered by main content (#14082)
150228f34c9ceefa3ea4f4d6933edd858121e8106cb0974ad7a7cd4bca846b15c484ff3be890135a52c2025-06-07T18:58:20+05:30Akarshan BiswasSYCL: Implement few same quantized type copy kernels (#13739)
151a7b8d35f780ab3e7639ae5345442fb1ad10f01636eba72b71c677ce9aae33c422a6e67008137987a2025-05-29T13:29:50+03:00Georgi Gerganovsync : whisper.cpp (ggml/1250)
152fedf034a98378059274e4243d2a370a243335e738726392d3d927fe3e504868d3548d694496ee37e2025-05-27T20:58:46-04:00Daniel Tangggml : Print backtrace on uncaught C++ exceptions (ggml/1232)
153c04621711a893cbd09cff6c927cb005bc6749e360fc16b42e8793364918e830c234c1f3caec29dae2025-06-01T11:42:16+03:00Georgi Gerganovparallel : fix n_junk == 0 (#13952)
154053b1539c02617eff744f89525ee57497c3c1fbeb3a89c3d9e34c28c5be70d8b687a84775746d4a02025-05-31T15:39:19-07:00Max Krasnyanskythreading: support for GGML_SCHED_PRIO_LOW, update thread info on Windows to avoid throttling (#12995)
15512d0188c0dc6146ffde6d277a93f232ccbe699f8eb3949938e82a128855bc0676220bb2ce6e4228d2025-05-31T10:24:04+03:00Georgi Gerganovkv-cache : refactor + add llama_memory_state_i (#13746)
156dd665cc9d4b378626cde11ea0c4c91f514fdc994df0c0c7d02f951dc639d48fd536f79200460ac832025-05-30T19:38:07+03:00Georgi Gerganovparallel : increase the variability of the prompt lengths (#13927)
157952f3953c1b61cc70e79e536c42ddce6a5ea5ea781713121ee56755ba4a147d1a1e3fa248ec31a662025-05-27T04:05:18-07:00Diego Devesaggml : allow CUDA graphs when using pipeline parallelism (#13814)
1586f180b915c9ed9ec0c240b5dcd64644988fb5e8203f582ae8fccecff225c30a2802461b44761e8222025-05-26T21:10:36+05:30Akarshan BiswasSYCL: Add non contiguous support in RMS_NORM and NORM kernels (#13611)
159de2ef53a4b2c0d703749a309d19fe68fd8f1b9acc508256db2de2b032e19c8ed833f4683c827c9a12025-05-25T16:34:36+03:00Georgi Gerganovkv-cache : rework kv_cell (#13706)
160ffd0eae60b7642e942c8245b89642527e36099e6b775345d788ac16260e7eef49e11fe57ee5677f72025-05-24T11:46:19+02:00Johannes GäßlerCUDA: fix race condition in FA vector kernels (#13742)
161d394a9aedc50a13b7f6373416f7c1ccabfe79c326b56a64690a318fcabcd7739ac7e314d44785ea82025-05-22T13:54:43+02:00Nicolò Scipionesycl : Remove waits from function calls (#13702)
1626b56a64690a318fcabcd7739ac7e314d44785ea8a4e8912dfd4604be1e39bc86ba4c0b02969967ef2025-05-22T09:24:09+01:00Ewan CrawfordSYCL: Avoid using with SYCL-Graph for unsupported nodes (#13587)
163a4e8912dfd4604be1e39bc86ba4c0b02969967efedbf42edfdabb9cea72ae12137570cf48f5d80762025-05-22T02:21:45+03:00Henry Linjamäkiopencl: Add support for multiple devices (#12622)
164c76532e7ba128bb097bf6836bf0f5592e1b56b762aa777d86d3f7bb80b93e226f1c25e47825f6a832025-05-21T19:40:35+03:00antichristHaterconvert : add qwen2vl support for unsloth merges (#13686)
16533983057d0f578aca74ba15eccc3de9c267a5ff6fb1cab201c6c4cd36731f100df74eece2f4706fa2025-05-21T09:58:49+08:00R0CKSTARmusa: Upgrade MUSA SDK version to rc4.0.1 and use mudnn::Unary::IDENTITY op to accelerate D2D memory copy (#13647)
166e298d2fbd082a52c0f6ed02729f94e9bf630cf17f0adb80bf7c2c0d80abb04f4533b5513622d99642025-05-20T08:05:46+03:00Georgi Gerganovkv-cache : add SWA support (#13194)
1678b5e19aea6ce9fe445259866392437323404144060aea028b575ab54e0dfbb31db641bb93d2ae8c42025-05-18T18:30:13-07:00Diego Devesaggml : fix apple OS check in ggml_print_backtrace (ggml/1229)
16860aea028b575ab54e0dfbb31db641bb93d2ae8c49c55e5c5c24990dd17fd6c8f4f2159052d2b06f12025-05-17T19:06:26-04:00Daniel Tangggml : Fix missing backtrace on Linux (ggml/1228)
169518329b2d4434d0683c30b741b4f4cf5734b5f992f5a4e1e09567e09be8b84a5f976334de9539d172025-05-17T12:58:55+03:00Georgi Gerganovparallel : add option for non-shared and larger prompts (#13598)
17009232370fc6426aa5dd9be01a8271b9c28f5af3a7474e00b34629e9cd8b06bc87ad935584ea30f8e2025-05-11T16:20:39+02:00Sigbjørn Skjæretscripts : exit compare-llama-bench.py gracefully when there's nothing to compare (#13451)
1710208355f42bdab88a08507ead4a6302790a08323d2a4ef05c60506ee48e7375eb36f2257de7ab0d22025-05-10T22:22:48+02:00Johannes GäßlerCUDA: fix race conditions FlashAttention kernels (#13438)
17233eff4024084d1f0c8441b79f7208a52fad7985817512a94d636c4b6c1332370acb3e5af3ca709182025-05-09T19:29:37+02:00Xuan-Son Nguyen server : vision support via libmtmd (#12898)
1730527771dd80bd18479dfaaa0a98be297fc3592bf2189fd3b6327a1d17893694125da8edcf74a64682025-05-09T17:25:50+08:00R0CKSTARllama-run: add support for downloading models from ModelScope (#13370)
1741f73301b63668d61b5f3109489050a27dc3f65be4773d7a02ffdb05ba9e673ff21ce95351836e33a2025-05-07T15:48:23+08:00R0CKSTARcuda : remove nrows_x in mul_mat_q_process_tile (#13325)
17593c4e23905987949b714b21ae918ff6bfb55fe368afbd968182909cf93fb15959fc867b6dd3adb532025-05-04T14:16:39+02:00Johannes GäßlerCUDA: fix race condition in MMQ stream-k fixup (#13299)
1768afbd968182909cf93fb15959fc867b6dd3adb538ae5ebcf859b05a2ea3bbd930133a2fe4a89ed3c2025-05-04T13:58:38+02:00Johannes GäßlerCUDA: fix race condition in MMQ ids_dst (#13294)
1778efbdadc616fa717c369059b9b388160958d886cf057808ffadce213f54c1884ab5096e60140f3582025-05-01T17:32:11-04:00Justin Santa Barbararpc : avoid uninitialized memory in serialize_tensor (#13210)
1783e168bede4d27b35656ab8026015b87659ecbec2ceda28ef8e310a8dee60bf275077a3eedae8e36c2025-04-30T16:56:24+02:00Xuan-Son Nguyenconvert : improve model arch handling (#13122)
17943ddab6eeeaab5a04fe5a364af0bafb0e4d350651831f538f720d1d99fba146f24f0a8e970838cc42025-04-28T21:00:20+03:00Ville Vesilehtofix(rpc): Improve input validation and error handling (#13069)
1805fa9e63be82225fb3249c76f39ddda3e5bdec0a3a4c340f974f9b7ac0c1aae897aabaa54549a97e52025-04-28T12:18:59+02:00Xuan-Son Nguyenclip : refactor set input for cgraph + fix qwen2.5vl input (#13136)
181a4c340f974f9b7ac0c1aae897aabaa54549a97e5d0a417f3c7a5a22ef05b3b76d91dbe1d3362bf0c2025-04-28T15:03:25+05:30Akarshan BiswasSYCL: Add all missing unary kernels (#13074)
182f0dd6a1926cdb2f4183a937deee40db26ef8f1da69699be48a6b94570773532850667f1591dc5bbe2025-04-28T15:33:28+08:00R0CKSTARmusa: fix typo in cc control (#13144)
183e291450b7602d7a36239e4ceeece37625f83837359e991c23cc44cb5fb657e7b9358cac21fb798282025-04-27T19:22:49+08:00R0CKSTARmusa: fix build warning (#13129)
184ca2bb89eac2097ab4620448737e58af8452e444b2d451c80590b9ac250322769ac13d3b4870dbcf72025-04-27T16:10:34+08:00HimariOclip : Add Qwen2.5VL support (#12402)
1852cca6c01e46d2fc1124d15730273ed2acdad1016658987cfc9d752dca7758987390d5fb1a7a0a54a2025-04-23T10:32:49+03:00Radoslav Gerganovrpc : add command line option for number of threads for the CPU backend (#13060)
1867a395f67a7a02bb361d944b816d6e933889e28e1971f245b3b5f3f55991bb779cb541b00f82eea1d2025-04-17T20:34:16+08:00hipuddingCANN: Add support for async operator submission (#12864)
187b0c75ac9f93322b45e3766e149417334b4fd1ed9d6d2c2ab8c8865784ba9fef37f2b2de3f2134d332025-04-15T10:04:24+08:00Xinpeng DouCANN: Optimize CANN buffer pool memory management (#12875)
1888ac9f5d765f2b1b7f821873618c0cff68ca59bc812e9158f25cb47e97ca9b8083e1ec7415f2622602025-04-11T15:26:17+08:00R0CKSTARci : Replace freediskspace to free_disk_space in docker.yml (#12861)
18964eda5deb9859e87a020e56bab5d2f9ca956f1defe5b78c89670b2f37ecb216306bed3e677b49d9f2025-04-10T17:24:44+02:00Xuan-Son Nguyenconvert : ability to lazy-load safetensors remotely without downloading to disk (#12820)
190d9a63b2f2e91cdcb0eda211b7f49fadcdea0f6648ed71242f464dc0a3fb3cffcfe064e55bdec72f92025-04-09T17:22:30+08:00R0CKSTARmusa: enable freediskspace for docker image build (#12839)
1910090950f679475c5ecaac2f7bca5049cca96492b7ecd780b1a1d5214b8d04c25ebfc194d310816ed2025-04-09T00:25:08-05:00Jeff Bolzvulkan: In coopmat2 mmq, load q4_k/q5_k scales through shared memory (#12833)
19278a1ba0a4f2bfed5b8b8e312592143d22e5316982dabf759e7c8c827d38cdd18d0792070e6a4f4e12025-04-08T18:37:06+02:00Xuan-Son Nguyenserver : fix thread.join() on exit (#12831)
193916c83bfe7f8b08ada609c3b8e583cf5301e594b0c74b04376b0b9efc096480fe10f866afc8d7c1c2025-04-06T21:23:54+08:00R0CKSTARmusa: fix compilation warnings in mp_22/31 (#12780)
1945f696e88e0eb490c8028421d3c5419df9dcf5385193c3e03a63ccda3ac3d6a2999e41e6d1414fe232025-04-03T19:51:35+08:00R0CKSTARsync : minja (inclusionAI/Ling) and update tests (#12699)
195a10b36c91a091f4606710fba4e9327fd71e0e73883a88bd6affbe148a622ac730952ac5b8b5859792025-04-02T14:32:59+03:00Georgi Gerganovllama : refactor kv cache guard (#12695)
196a6f32f0b3437ac79827ffb55521cb839343324ab2bb3597e426ce092c3e10ae0bdd876963765e6ed2025-04-01T19:12:53+08:00R0CKSTARFix clang warning in gguf_check_reserved_keys (#12686)
197c80a7759dab10657b9b6c3e87eef988a133b9b6a250d7953e829ff0e16074510fef0e7cec696461b2025-03-31T18:40:56+02:00Daniel Beveniusvocab : add special infill tokens for CodeLlama (#11850)
1986c02a032fa21d69e881ef9a5c94ba28ebaf1d749f52d59d771dc231fc2ac39adacf157ddefc977302025-03-31T14:55:24+05:30Akarshan BiswasSYCL: Remove misleading ggml_sycl_op_flatten function (#12387)
199492d7f1ff77e116e44962b832cc3db24cfc46d24d3f1f0acfbf8aaafd2ce494309a10a7cc92042c82025-03-30T16:59:38+08:00R0CKSTARmusa: fix all warnings, re-enable `-DLLAMA_FATAL_WARNINGS=ON` in ci and update doc (#12611)
200c7b43ab60855f752ae79937fb93d561bc30b69a424feaec05792b972d5ff3e2b12d9237ebd50d1ac2025-03-27T12:21:47+05:30amritahs-ibmllamafile : ppc64le MMA implementation for Q4_0. (#12489)
201f17a3bb4e8b0aa24c0f86636d234aca7dc2cfa01bd40678df768ab189b26b8b03e3de4062e3b71a32025-03-27T07:16:00+05:30Akarshan BiswasSYCL: implement memset ggml backend buffer interface (#12580)
202fd7855f8f522ab26681b25ae506ff99c654e2dd553af4dba425768fd507ce6b45873cfbb3df2295f2025-03-26T15:09:48+08:00R0CKSTARdoc: [MUSA] minor changes (#12583)
2033cd3a395323fa9cdf6ecfa1fea290bf228d4e8562d77d88e70d017cd82c3f1a4517e3102e2028ac42025-03-25T15:45:08+08:00R0CKSTARci: [MUSA] add CI and update doc (#12562)
2047ea75035b67f44c22ed7039967f718011fd35ce5c54f6b7988b63714b87b0390e01a1e69aee79a122025-03-24T18:28:34+08:00R0CKSTARCUDA: Fix clang warnings (#12540)
205fac63a3d786b2a0f97876c30add02cb525a9648eeddfb438502bd5d1014d63a812e9b6d03d326f8c2025-03-22T17:11:37+08:00R0CKSTARmusa: refine compute capability (#12493)
2063d82dbcbce2c677fc35fbf99574ccd107d95a1f8732b5fbf5e7f9cf069942f0c5850ee959ef321ba2025-03-20T17:05:34+05:30Srihari-mcwggml : block interleaving support for Q4_K quantization for x86 AVX2 architecture (#12332)
207517b5ddbf002b91fd6d6daf5d8db8c88a0173039a9b59288e222f39fc0311dc66944ed5a86c815fa2025-03-20T01:22:06+05:30Gaurav GargCUDA: Improve flash decoding kernel GPU occupancy for BS=1 case (#12183)
208d84635b1b085d54d6a21924e6171688d6e3dfb4675422e8bc42646005be0754f7aa438b97a5e777e2025-03-18T12:54:55-07:00lhezopencl: improve profiling (#12442)
209bb115d2bf7ed2cdd7dccd7ae74cc9cfe4b0adb7129fff308c704c1c752cdb5153361e545e2bac09d2025-03-19T02:28:26+08:00R0CKSTARmusa: override warp_size of musa device to 32 (#12445)
2107dfad387e3f6ac98d383ded2d175eb59736a399360c902926c928f9c2cd6390ce411876f92feeaf32025-03-18T07:27:50+08:00Molly Sophiallama: Add support for RWKV v7 architecture (#12412)
2119f2250ba722738ec0e6ab684636268a79160c854774973b8f3d5e375b0b74d58638eeb1817e950a82025-03-14T16:41:20ZEric CurtinAdd CLI arg to llama-run to adjust the number of threads used (#12370)
21210f2e81809bbb69ecfe64fc8b4686285f84b0c07ba7654380a3c7c1b5ae154bea19134a3a9417a1e2025-03-11T20:16:03+01:00uvosCUDA/HIP: refractor mmqv to unify the calculation of nwarps and rows per block between host and device code. (#12177)
213251364549fe4a78d4e2e66f1adfaf2bd53041d2c8acdacb3ea00697477eb019efbb6fc183371055c2025-03-11T01:18:25+08:00R0CKSTARmusa: support new arch mp_31 and update doc (#12296)
21405e6f5aad0a4bb48fb2775f2a78505540fdbc47d673cfef9aa8daad09966208909f346eb996628a42025-02-28T13:06:12+05:30Prashant Vithuleggml: aarch64: implement SVE kernels for q2_k_q8_k vector dot (#12064)
215f777a73e18c218848bca0748581c043987348a5daf7747c95ae71a5db4184947f837179e82c70b772025-02-23T13:14:32ZEric CurtinSome llama-run cleanups (#11973)
2160b3863ff9576872855b0265da2cab2ce7e25c4d9ee02ad02c56ff36a5edd22d8617ab3f9546ce7fe2025-02-21T15:46:23+08:00BodhiMUSA: support ARM64 and enable dp4a .etc (#11843)
217c5d91a74006c49376590ef2b67420bc7ce2063974806498bf15ef767de3776b00856e56c373dd1b12025-02-20T14:06:51+01:00Charles Xuggml-cpu: Add CPU backend support for KleidiAI library (#11390)
21873e2ed3ce3492d3ed70193dd09ae8aa44779651df7b1116af102bcac450c1a522e9c59db241c67672025-02-17T14:03:24+01:00Johannes GäßlerCUDA: use async data loading for FlashAttention (#11894)
219c48f630d1c1942fce08aa7cb18a53ace443cd611bd6e55bfd3f3af3e5705cf87a10e5178cef7c3c12025-02-13T14:46:59+01:00Daniel Beveniusllama : add --completion-bash option (#11846)
220bd6e55bfd3f3af3e5705cf87a10e5178cef7c3c1c7f460ab882065ec5696e3d51e24dbf67b5392872025-02-13T20:28:18+08:00R0CKSTARmusa: bump MUSA SDK version to rc3.1.1 (#11822)
221e4376270d971cff7992bdb6c5412a739195b14593e693197724c31d53a9b69018c2f1bd0b93ebab22025-02-13T01:25:34-05:00Oleksandr Kuvshynovllama.cpp: fix warning message (#11839)
222a394039db004c6ee00098250d160b5aa018c2314be3bbd62153820ff6d358c817360927f429105c42025-02-13T01:02:38+01:00Diego Devesaggml-cpu : add chunking support to mul_mat_id (#11666)
223748ee9fe9312acb8755ee4bf46fd9de2e6a45f29198b1ec611a2c551ea40e5b9c0b862f37555a4cc2025-02-12T13:57:33ZRichardggml : fix multi-threaded clamp_f32 (#11824)
2244d3465c5aeca8be29cac77f1535c35f4fb274ecad80be897acdca45f8f7ea2e52c930b1d0e738a142025-02-08T15:30:53+01:00Karol Kontnyggml: Fix data race in ggml threadpool (#11736)
2252fb3c32a1634488a5265d1304ab37628eeb5480d9ab42dc722ad19a12af80f38a06474d498f96da32025-02-06T17:32:29+01:00Xuan-Son Nguyenserver : (webui) migrate project to ReactJS with typescript (#11688)
2263ec9fd4b77b6aca03a3c2bf678eae3f9517d69043962fc1a7956dd0afacfbce10fd1a3ffd3ad857e2025-02-05T02:18:38+08:00fxzjshmHIP: force max threads per block to be 1024 (#11621)
227a83f528688324a21484a97af1d1be5e1bc8d4c8eb1bcd309fc8ac929cbd4a6207b3a19886bda031f2025-01-31T14:15:25ZOlivier Chafik`tool-call`: fix llama 3.x and functionary 3.2, play nice w/ pydantic_ai package, update readme (#11539)
2284314e56c4f8c5091f45732f39bd94c0c6c323798496e5bf46bab757d05e18227cb4e17055ae42f422025-01-30T11:05:00+01:00Daniel Beveniusserver : use lambda instead of std::bind (#11507)
229e0449763a4f335cca374254a72892141f41eaa59eb7cf15a808d4d7a71eef89cc6a9b96fe82989dc2025-01-30T05:48:14+01:00Daniel Beveniusserver : update json snippets in README.md [no ci] (#11492)
2301a0e87d29152cb9d4ce13b4ad64b0382c9ba1ab6d2e518e9b4231077013ffb0e5c4cc24a0c4c1b7d2025-01-23T11:59:08-08:00William Tambelliniggml : add option to not print stack on abort (ggml/1081)
231d2e518e9b4231077013ffb0e5c4cc24a0c4c1b7db636228c0ad0db95bf73008094c6145a05615cf62025-01-17T21:29:08+09:00issixxggml-cpu : fix ggml_graph_compute_thread did not terminate on abort. (ggml/1065)
2325245729e3317064959faefc5e5cbc63f4e9cfbea6152129d05870cb38162c422c6ba80434e021e9f2025-01-23T01:01:17-06:00Jeff Bolzvulkan: fix diag_mask_inf (#11323)
2336171c9d25820ccf676b243c172868819d882848fe28245f35f2faaf249dd352998b3693a8cc28c512025-01-21T13:18:51ZOlivier ChafikAdd Jinja template support (#11016)
234adc5dd92e8aea98f5e7ac84f6e1bc15de35130b5f11cfdfd7fe29436fce512d934c2ff6b94bd89d22025-01-15T19:50:13ZEvevulkan: scale caching for k quants + misc fixes (#11081)
2356369f867a410416239d9f20ec27c2b1d6a9fee5247182dd03fe04a4ffda5d7f4c8a109ae0056cf562025-01-06T10:28:17+01:00Daniel Beveniusllama : rename missed batch params/vars to ubatch (#10059)
2360da5d860266c6928b8c9408efbd264ae59fedda6a45433ba209ee0b33d02c7dc4c31f29894ad83a62025-01-02T15:05:18+01:00Xuan Son Nguyenserver : allow using LoRA adapters per-request (#10994)
237a813badbbdf0d38705f249df7a0c99af5cdee678fdd21889123bec62b1db3b2fc22b5a4abab321742024-12-29T03:16:34-06:00Jeff Bolzvulkan: im2col and matmul optimizations for stable diffusion (#10942)
238227d7c5a7f4cc7734fde8a4ef4382d613486d3c87b1ec53f56bb72c49e4c8434157895f94d3709c22024-12-17T09:52:09+01:00Xuan Son Nguyenserver : (UI) fix missing async generator on safari (#10857)
2397b1ec53f56bb72c49e4c8434157895f94d3709c2160bc039c862c10b9938269a90ddb09d794a7b0d2024-12-17T05:52:55ZEvevulkan: bugfixes for small subgroup size systems + llvmpipe test (#10809)
2405478bbcd173e7027af7689493c7421719f5c43dfb5ae1ddff93403300fc79c7ab5ee73b8cfbb34572024-12-15T05:55:54-06:00Vinesh Janarthananserver: (UI) add syntax highlighting and latex math rendering (#10808)
241a76c56fa1a3d27467eb97468d8c3b2fe1243b61ac27ac678dd393af0da9b8acf10266e760c8a09122024-12-13T12:23:52-08:00lhezIntroducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (#10693)
24211e07fd63bac1cb642380a7a3eac03fd8703e9484601a8bb6784d2ab8b4b605354b51979fbeea1d32024-12-13T18:23:50+01:00Corentin REGALfix: graceful shutdown for Docker images (#10815)
24383ed24a97b500ccdb32b90b94e6f9621ad8db79ed583cd03f663701858ba152a334cbb467fabe3422024-12-13T12:12:15+05:30Akarshan BiswasSYCL: Reduce most of the compiler warnings (#10748)
2443573fa8e7b7f0865638b52b4e9b4d2006f0558a2d9c3ba2b7749c00df477599aa141a98b4521aa2c2024-12-07T20:21:09+01:00Xuan Son Nguyenserver : (refactor) no more json in server_task input (#10691)
245ce4a7b849388b67d45ad420bdd82d5efcd55647a19d8762ab61df8286367588a80b9c7db4cb568db2024-12-07T18:02:05+02:00Georgi Gerganovserver : various fixes (#10704)
246e9e661bd59364e5d4fce035834b6cadcadf8c2efefb6ae963031709fc331e6e48cc4606ac8f9c3a72024-12-03T21:11:43+08:00mahorozteCUDA: remove unnecessary warp reduce in FA (ggml/1032)
247efb6ae963031709fc331e6e48cc4606ac8f9c3a7667d70d1704dfa6977505f5d01d4638669b90dce2024-12-02T19:27:24+01:00PABfeat: add `GGML_UNARY_OP_ARGMAX` Metal kernel (ggml/1019)
2480533e7fb3842a523f64dc533bd7bd7147ec2c63a7cc2d2c88908fc92b97b28acafb82f7d6e425b852024-11-30T07:00:02ZEvevulkan: Dynamic subgroup size support for Q6_K mat_vec (#10536)
2496c595676899013102fdb0aa4b06a49954300c94a890719311b6535e572f15965c6d7ec4ac2537f602024-11-28T19:17:49+01:00Xuan Son Nguyenserver : (tests) don't use thread for capturing stdout/stderr, bump openai client library (#10568)
250249cd93da3df9c8fa78869b0522526d1625aca91904109ed0d97c9b656a5e8bf612925f739bb81662024-11-27T00:00:41+08:00R0CKSTARmtgpu: Add MUSA_DOCKER_ARCH in Dockerfiles && update cmake and make (#10516)
25145abe0f74ee281aea6e5283c1e738061256cfcae0bbd2262a3263f37385297b30de37941836e57f72024-11-26T16:20:18+01:00Xuan Son Nguyenserver : replace behave with pytest (#10416)
25284e1c33cde9e0a7aafcda2d4f21ba51c300482d7811872a59daefb25fc0c4326bcb6d8ae893c2f7c2024-11-26T13:36:40+02:00Georgi Gerganovserver : fix parallel speculative decoding (#10513)
2539ca2e677626fce759d5d95c407c03677b9c87a265931c1f233c616083d64e41a228249d58e039aa52024-11-25T16:31:38+02:00Georgi Gerganovserver : add speculative decoding support (#10455)
254cce5a9007572c6e9fa522296b77571d2e5071357dc39012cbaf8752fabecaeb60af78ccdd1dfb73b2024-11-24T18:03:25+02:00Georgi Gerganovflake.lock: Update (#10470)
2552eb76b2a5e4ea395b971a419c95b473ab6f253e49b75f03cd2ec9cc482084049d87a0f08f9f015172024-11-18T16:08:20+02:00Georgi Gerganovflake.lock: Update (#10346)
256cf32a9b93ad859ea592c31785b6bd3b4b2121463a43178299c2f74f14bcfc659bfd9fd32d931d1f42024-11-17T11:23:01+02:00Georgi Gerganovmetal : refactor kernel args into structs (#10238)
257f0204a0ec70d50ca60e07bc0096ec1d6508ab0c757f8355b29a8c7dfcd1fb6094758ad85644f85352024-11-15T19:47:25+08:00R0CKSTARci: build test musa with cmake (#10298)
2589901068ac78838745e604fffb4601d315a610456231f9360d94446cd083b6b116f63991b1328c4842024-11-15T05:48:49-04:00Xuan Son Nguyenserver : (web UI) add copy button for code block, fix api key (#10242)
259ae8de6d50a09d49545e0afab2e50cc4acfb280e24a8ccb37ad9c9027cbcfd5548c19cdffe48d51972024-11-14T18:04:35+01:00Diego Devesaggml : build backends as libraries (#10256)
260fb4a0ec0833c71cff5a1a367ba375447ce6106eb5ea926dad7f62ebccff7b24784bd1e01a06d13ae2024-11-13T20:00:35+02:00Michael Podvitskiyllama : propagate the results of `graph_compute` (#9525)
26154ef9cfc726a799e6f454ac22c4815d037716edab0cefea58a20020754b7431e3c725625274372a52024-11-11T11:13:51-06:00Jeff Bolzvulkan: Throttle the number of shader compiles during the build step. (#10222)
2624b3a9212b602be3d4e2e3ca26efd796cef13c55e505f33274d60676320216b662a97672a76ec600e2024-11-10T21:45:25+02:00Georgi Gerganovflake.lock: Update (#10243)
2633bcd40b3c593d14261fb2abfabad3c0fb5b9e3185c333e014059122245c318e7ed4ec27d1085573c2024-11-07T18:19:10+11:00Zhiyuan LiOptimize RWKV6 Operator Naming and Implement Multi-core CPU/ SYCL Acceleration (#10133)
264b11f9ba9b8ce319f04b88afe40d264e6b7f4ba4694d8cb8be13b7c4d04eeca5a2b956b9148e6f2222024-11-06T13:29:01+02:00Georgi Gerganovserver : remove hack for extra parallel slot (#10187)
2659e0ecfb697d297355e43c20559d29bcc71beb0c36a066b9978533e2ab9890b7f4f8c0262d91798b32024-11-04T16:33:29+01:00Xuan Son Nguyenserver : clarify /slots endpoint, add is_processing (#10162)
2661839f69130151ceeac4d01c0ef8964e1fb43bba69830b6923b61f1e652a35afeac77aa5f886dad092024-11-03T15:14:15+02:00Georgi Gerganovflake.lock: Update (#10146)
26742cadc74bda60afafb45b71b1a39d150ede0ed4d45950415ed985830c59bf42cf9c9216b20cf08ef2024-11-02T16:34:56Zsasha0552server : fix slot selection by lru (#10126)
268d865d1478cd4e403f82d793c2afcd0f943412f051804adb0cfee4811eaf633741503d683a46e4c772024-11-01T13:33:14Zsasha0552server : fix smart selection of available slot (#10120)
26907028f9d74d895da2ca4a1956624e3f07e04e620524afeec9dad7d765ce91f5cf30c73703867cb472024-10-28T17:41:24+02:00Georgi Gerganovflake.lock: Update (#10063)
270524afeec9dad7d765ce91f5cf30c73703867cb478125e6cbfcf2b3b9066e4d923aca9295526730f52024-10-28T17:02:48+08:00R0CKSTARmusa: workaround for Guilty Lockup in cleaning src0 (#10042)
2718c60a8a46261ffb92b6d23a78acfac2fcb6fe2339e4a2563eadf34e9432d248224d4f43e8495e8fe2024-10-23T14:34:00-04:00bssrdfincrease cuda_cpy block size (ggml/996)
272bc5ba007b2c83ac95875e68724dabfc12159fc61958367bf530d943a902afa1ce1c342476098576b2024-10-25T10:13:46+03:00Georgi Gerganovserver : check that the prompt fits in the slot's context (#10030)
273958367bf530d943a902afa1ce1c342476098576b40f2555797f97314de749873cdc29dc102be66e22024-10-24T21:51:22+02:00Xuan Son Nguyenserver : refactor slot input data, move tokenizer to HTTP thread (#10023)
274873279b1592e433c4d9eb5065091cc98473c7beec8c07d658a6cefc5a50cfdf6be7d7265036123032024-10-20T00:22:59Zgithub-actions[bot]flake.lock: Update
27560ce97c9d809f4b040e90b597468b839df5728d08901755ba328643c9ab071c20e1939ea52951a0e2024-10-18T13:34:36+08:00Ma Mingfeiadd amx kernel for gemm (#8998)
276fbc98b748e7b075e327bcf13237057f647678049dcdd535302fc9702a4709be25f56540d65163a442024-10-15T15:54:55+05:00MaggotHATEsampling : add XTC sampler (#9742)
27792be9f12164f18ce845a5bab60cefa5f7fec6836edc265661cd707327297b6ec4d83423c43cb50a52024-10-13T06:11:26+03:00Georgi Gerganovflake.lock: Update (#9870)
2781bde94dd024b632f98428f4bf2ce48329513077995c76e8e92ecc93f784b185eafae36a0e7ad2fa32024-10-12T16:06:31+03:00Georgi Gerganovserver : remove self-extend features (#9860)
27911ac9800aff532715a5bc7991062c68ba3472e6e943d20b4111c746bcd9dbc7e4771de313b08b50c2024-10-12T08:21:51+03:00Georgi Gerganovllama : improve infill support and special token detection (#9798)
280943d20b4111c746bcd9dbc7e4771de313b08b50c96776405a17034dcfd53d3ddf5d142d34bdbb6572024-10-12T13:09:53+08:00R0CKSTARmusa : update doc (#9856)
281cf8e0a3bb9c0e93e371773b282054cdbbb231038c7499c557cc1efafaf0a6bc12963c398262997032024-10-11T02:10:37+08:00R0CKSTARmusa: add docker image support (#9685)
2826279dac039ddeb6d5ebd125a6274fd3c37a77ba8d5ac8cf2f2e30459489e6721a17d15b92a0c42a62024-10-07T19:35:42+03:00Georgi Gerganovflake.lock: Update (#9753)
28396b69121033d2b6b951d1b6b1b43f8b4f97dac99d5cb86844f26f600c48bf3643738ea68138f961d2024-10-07T13:26:31+01:00Paul Tsochantarismetal : single allocation of encode_async block (#9747)
284ace4f4be37abed4801fbd54a94cf38a7ae4624168277a817f18967581b02b2248989d773e8e999982024-09-30T17:48:49+03:00Georgi Gerganovflake.lock: Update (#9680)
285641002fba8d2a0c0269027e23d2ef58e905460280de8b203f1d31cf5ee0d2a3560a0ad78d44f4d4c2024-09-29T11:50:17-05:00Jeff Bolzvulkan : multithread pipeline creation (ggml/963)
286544f409b4bd8fc98a3e87820f0ac934e00402de76084bfb261b03f812de2255b05b6b5bb8d1c71712024-09-26T08:59:42+02:00Salvatore Mesoracavulkan : argsort barriers must be under uniform control flow (ggml/951)
2876084bfb261b03f812de2255b05b6b5bb8d1c7171faac0bae265449fd988c57bf894018edc36fbe1e2024-09-24T13:23:59+03:00Georgi Gerganovggml : fix GGML_MAX_N_THREADS + improve formatting (ggml/969)
2881b2f992cd2cff0b69e5abe78bb8888d51ed19d67739842703e32cd43443c45e0b4f6647cc4e6b3d62024-09-28T14:32:46+02:00slarentest-backend-ops : use flops for some performance tests (#9657)
289739842703e32cd43443c45e0b4f6647cc4e6b3d66102037bbb55521880ae78a6ee6c2a0c00c901df2024-09-28T15:13:21+03:00Georgi Gerganovllama : add comment about thread-safety [no ci] (#9449)
2907691654c68aa5316108f881136c59f815ccb6809ea9c32be71b91b42ecc538bd902e93cbb5fb36cb2024-09-26T09:27:40+08:00R0CKSTARmtgpu: enable VMM (#9597)
2913d6bf6919f7b10726421779cd344f2da05421c68904837e0cb2f5f01bf5d5901b7aa57a026860ae42024-09-25T01:06:52-06:00Gabe Goodhartllama : add IBM Granite MoE architecture (#9438)
292c087b6f11d3385f4293b6841ebfb755063479490116efee0eef09d8c3c4c60b52fa01b56ddeb432c2024-09-23T21:18:48-07:00Max Krasnyanskythreads: fix msvc build without openmp (#9615)
293f0c7b5edf82aa200656fd88c11ae3a805d7130bf1d48e98e4f3316bd2f6b187d288c7b6cb88d5cb32024-09-23T11:42:43-07:00Max Krasnyanskythreads: improve ggml_barrier scaling with large number of threads (#9598)
294f3979df762b75a2b1c0f622a2cd15d1bc60f037f1e7b9299c6ccb5bbc55d3db7cfa9b51f3ab09b592024-09-23T18:43:40+03:00Georgi Gerganovflake.lock: Update (#9586)
295c35e586ea57221844442c65a1172498c54971cb0912c331d3dba3a079815844208dc36164baa8cc72024-09-22T22:55:49+08:00R0CKSTARmusa: enable building fat binaries, enable unified memory, and disable Flash Attention on QY1 (MTT S80) (#9526)
296424c5d00a9b97dd5559635872db9b57f87c23b02a6809c6a2e8163cba296d4cc0c5cd4bbc80736382024-09-20T19:04:44+03:00Johannes Gäßlerggml/examples: add backend support for numerical optimization (ggml/949)
297a6809c6a2e8163cba296d4cc0c5cd4bbc80736385cb12f68395a5ec00b357e408883ce124a11dcdb2024-09-08T11:10:43+03:00Georgi Gerganovexamples : add null threadpool args where needed (ggml/0)
29864c6af3195c3cd4aa3328a1282d29cd2635c34c90d2f22e45c3c3b6f8222acb6284d0c8c93443ba12024-09-18T19:13:08+02:00slarenggml : fix n_threads_cur initialization with one thread (#9538)
2998b836ae731bbb2c5640bc47df5b0a78ffcb129cb8344ef58f8cdb8ebd6faf4463ca32ae91c374c812024-09-17T09:35:38-04:00Bert Wagnerarg : add env variable for parallel (#9513)
3000226613853133c081b55bb892a41bb5eacc0bc94503147a9f9d195d6a14e7c998df23b6eb61f2bae2024-09-17T01:19:46-07:00Max Krasnyanskythreadpool : skip polling for unused threads (#9461)
3015c3d0f1824714e9a97fc9b06e046eefcb6ecc7210aadac10c7dd704f8285ddf5a63d6f764cb340aa2024-09-16T06:48:24ZEveggml : IQ4_NL sgemm + Q4_0 AVX optimization (#9422)
30290a2fff0e7f80c6fea3fc6cf9a7b482744f3f1646262d13e0b2da91f230129a93a996609a2f5a2f22024-09-16T05:14:23+03:00Georgi Gerganovflake.lock: Update (#9488)
303befaf1197fa447f61714de041828852a270659d2feff4aa8461da7c432d144c11da4802e41fef3cf2024-09-14T09:50:12+02:00Daniel Beveniusllama : make cell_id const in inp_s_mask block (#9470)
3045af118efdaf1098798a06b24fd8a557760e99631d2b496bff4f353a6429f8e833448f071bd237ba72024-09-11T10:22:40+02:00Johannes GäßlerCUDA: fix --split-mode row race condition (#9413)
305b34e02348064c2f0cef1f89b44d9bee4eb15b9e751b603863627c4074e77b7e556e18ece86bdf9a32024-09-11T09:46:55+08:00R0CKSTARmusa: remove Clang builtins mapping (#9421)
306cb9c933eb2a0d2b514556bdcb934b56dfe5d67716cd4e034442f71718563e600070c2b6fc389e1002024-09-11T01:46:59+03:00Georgi Gerganovflake.lock: Update (#9360)
3075fac4d57643b1de8e9ab746f14d2fc4e319ae0c25fb5e24811cb01d48b482c15a974bfbd9f433e1d2024-09-09T21:07:18+05:30Prashant Vithuleggml : vector length agnostic SVE support (#9290)
308efe6a83e3046a94cda38c8be5a7801eadc21d78dfbb7fcffbcfc50009c275e75982b1603a6cb6b802024-08-28T10:23:02+02:00Salvatore Mesoracaggml : fix cont with transposed tensors when one dimension is 1 (ggml/934)
3099b2c24c0993487d3b34a873980e292da571481f3134bc38ecf3e2c5460581badce289a1ffa6804532024-09-06T23:21:29+02:00Xuan Son Nguyenserver : simplify state machine for slot (#9283)
3104a1411b4f17b6569dc0a067e5914dcc0f738b3708ebe8ddebd68526757c631cd019de009697c63c22024-09-06T14:06:04+02:00Xuan Son Nguyenserver : fix missing lock (#9334)
3117605ae7daf31c02211bcfec2f46635ef6ec4b98a8962422b1c6f9b8b15f5aeaea42600bcc2d441772024-09-04T02:36:43+03:00Georgi Gerganovflake.lock: Update (#9261)
312f771d064a95d212ddadea452ad0cc1e42b2c3db36e7d133a5f9409dd257fad90d7f320721b07a1b22024-09-02T08:25:30-07:00yuri@FreeBSDggml : add pthread includes on FreeBSD (#9258)
3136e7d133a5f9409dd257fad90d7f320721b07a1b2b60074f1c26d8354053a952844ef3f7ebefd88032024-09-02T17:11:51+02:00Xuan Son Nguyenserver : refactor multitask handling (#9274)
3148f1d81a0b6f50b9bad72db0b6fcd299ad9ecd48ca47667cff41f5a198eb791974e0afcc1cddd32292024-09-01T22:38:17+08:00Molly Sophiallama : support RWKV v6 models (#8980)
31542c76d1358021ccdbe8ba89109c143dd7ae166df9f7d4bcf5c27d37b0c7da82eeaf9c1499510554b2024-08-29T19:20:53-04:00Faisal ZaghloulThreadpool: take 2 (#8672)
3161d1ccce67613674c75c9c7e3fa4c1e24e428ba489fe94ccac92693d4ae1bc283ff0574e8b3f4e7652024-08-29T07:28:14+03:00Georgi Gerganovflake.lock: Update (#9162)
317a1631e53f6763e17da522ba219b030d8932900bdfc54ef0d1c138133a01933296d50a36a1ab647352024-08-21T17:58:11-04:00compiladellama : simplify Mamba with advanced batch splits (#8526)
3188455340b874aa90d5f70104c99ed2778d9e31c362f3c1466ff46a2413b0e363a5005c46538186ee62024-08-21T09:32:58+02:00Daniel Beveniusllama : std::move llm_bigram_bpe from work_queue (#9062)
319554b049068de24201d19dde2fa83e35389d4585d2339a0be1c8e31fcf4531427183b94f2ef019e562024-08-18T17:43:32+03:00Georgi Gerganovflake.lock: Update (#9068)
3208b3befc0e2ed8fb18b903735831496b8b0c80949d565bb2fd5a2a58b9924a7a34e77a87c78c521372024-08-16T17:19:05+02:00Xuan Son Nguyenserver : refactor middleware and /health endpoint (#9056)
3215fd89a70ead34d1a17015ddecad05aaa2490ca4698a532d474c73d3494a5353024cb6a4fbbabbb352024-08-14T18:32:53+02:000cc4mVulkan Optimizations and Fixes (#8959)
32298a532d474c73d3494a5353024cb6a4fbbabbb3543bdd3ce188cd247bda7c1bd1ad01fa64e5666902024-08-14T02:51:02-04:00compiladeserver : fix segfault on long system prompt (#8987)
3238cd1bcfd3fc9f2b5cbafd7fb7581b3278acec25fa21c6fd45032a20180e026773582d21294c856192024-08-11T16:58:58+03:00Georgi Gerganovflake.lock: Update (#8979)
324272e3bd95e620d285b2fb9faaa7b6b1b8edbbf3a45a55b91aa87958a75d691be64b070266d4fbb942024-08-09T18:24:30+03:00Georgi Gerganovmake : fix llava obj file race (#8946)
325be55695eff44784a141a863f273661a6bce63dfc0478174d5959b66096ae6609fcb0df14cab66b512024-08-07T13:29:02+02:00slarenggml-backend : fix async copy from CPU (#8897)
326db20f50cf4710c46e3a996919a26858cae8c80edefda90c93a62274ec0b0bfa80c4eee4bdb6966d02024-08-06T17:21:47+04:00Jaeden Amerocmake : Link vulkan-shaders-gen with pthreads (#8835)
327064cdc265fb63590c7c8f04a609d36ef200d55a75587e57a76630651752031223cc7024cb32cf3082024-08-05T07:52:55+02:000cc4mvulkan : fix Qantized Mat-Vec Mul on AMD GPUs for ncols < 64 (#8855)
328ecf6b7f23e664afd7ff856ec39034240ce438daa01aae2b4975b57a265ce8194928fd87f2d71027e2024-08-04T03:55:03-07:00Brian Cunniebatched-bench : handle empty `-npl` (#8839)
3294b77ea95f56a4c49bc995f08eac62a6416875ccc76614f352e94d25659306d9e97321f204e5de0d32024-08-04T05:53:20+03:00Georgi Gerganovflake.lock: Update (#8847)
330b7a08fd5e0e7c898c68d1743066ea495202d96087a11eb3a260915aee16101808f291a244e2facc72024-08-01T12:53:46-04:00Alex O'ConnellBuild: Only include execinfo.h on linux systems that support it (#8783)
331ed9d2854c9de4ae1f448334294e61167b04bec2a398ede5efeb07b9adf9fbda7ea63f630d476a7922024-07-31T15:51:06-04:00Clint HerronBuild: Fix potential race condition (#8781)
3327c27a19b2eb91bb0f43c7f7aec0386cec2dddc33140074bb8647df41840d6f32f4409fa8959bcf9f2024-07-30T23:40:18+09:00l3utterflyadded android implementation of ggml_print_backtrace_symbols (#8751)
333140074bb8647df41840d6f32f4409fa8959bcf9f6e2b6000e5fe808954a7dcef8225b5b7f2c1b9e92024-07-30T15:58:57+03:00Georgi Gerganovflake.lock: Update (#8729)
334439b3fc75a8deb42899ac47a8f52aae75e0339fe0832de723695ab400316a6c49b9f712380e3a7312024-07-29T20:56:12+08:00R0CKSTARcuda : organize vendor-specific headers into vendors directory (#8746)
3356eeaeba126ff701f3e8f79f246805b70237099724730faca618ff9cee0780580145e3cbe86f248762024-07-28T22:32:44+02:00Johannes Gäßlercmake: use 1 more thread for non-ggml in CI (#8740)
3364c676c85e59ef8f771f3a129e6eb217552139231e54c35e4fb5777c76316a50671640e6e144c95382024-07-28T00:42:05-04:00compiladellama : refactor session file management (#8699)
337e54c35e4fb5777c76316a50671640e6e144c95385e2727fe0321c38d1664d26173c654fa1801dc5f2024-07-28T07:41:25+08:00R0CKSTARfeat: Support Moore Threads GPU (#8383)
33845f2c19cc57286eead7b232ce8028273a817aa4d22f281aa16f44d8f6ec2c180a0685ff27e04e7142024-07-21T16:45:10+03:00Georgi Gerganovflake.lock: Update (#8610)
33922f281aa16f44d8f6ec2c180a0685ff27e04e714328884f4219c0228673cf1870ac63987fb4f9fd02024-07-20T22:09:17-04:00M-Aexamples : Rewrite pydantic_models_to_grammar_examples.py (#8493)
34069c487f4ed57bb4d4514a1b7ff12608d5a8e7ef007283b1a90e1320aae4762c7e03c8790439102522024-07-20T22:25:26+02:00Johannes GäßlerCUDA: MMQ code deduplication + iquant support (#8495)
34187e397d00bdcedd5cbf6dfda06a7b0f30246272857b1d4f9eb6f2c139b31ea79626d954b261e10512024-07-19T17:17:27+02:00slarenggml : fix quant dot product with odd number of blocks (#8549)
3427acfd4e8d55082c1b597dfc3ffe04fb5d530c6dc97bdd26eee11fe109dec00de75690ceef61c03f22024-07-15T23:13:10-04:00compiladeconvert_hf : faster lazy safetensors (#8482)
343f17f39ff9cb71fa7b0eda0c649bb0ce9b2d8a6b89104bc20edf47f74dc49379b7d61d0c6e85a48822024-07-15T08:04:56-04:00M-Aserver: update README.md with llama-server --help output [no ci] (#8472)
344aaab2419eaa17ab3aa38f4ba49c7eea406999e9973cf442e7bc9baca7b3e213b261551812f1676c92024-07-14T18:54:02+03:00Georgi Gerganovflake.lock: Update (#8475)
345278d0e18469aacf505be18ce790a63c7cc31be26dd07a123b79f9bd9e8a4ba0447427b3083e9347a2024-07-10T20:08:17-04:00Clint HerronInitialize default slot sampling parameters from the global context. (#8418)
3460f1a39f3439825acf7e3a1663566d410be15217083321c6958acf20747d288031174cc1bf8816e332024-07-10T07:14:51-05:00Dibakar Gopeggml : add AArch64 optimized GEMV and GEMM Q4 kernels (#5780)
3477fdb6f73e35605c8dbc39e9f19cd9ed84dbc87f2a130eccef42b75a84da270411cefeed45c153e302024-07-09T01:36:38+03:00Georgi Gerganovflake.lock: Update (#8342)
348470939d483d1c89b7292f78bac1fd27c42c171ce6f0dbf6ab087bcd286fb78560099ca04583167352024-07-08T03:26:53-04:00Kevin Wangcommon : preallocate sampling token data vector (#8363)
349cb4d86c4d723af87d3d7e3177e9485f20039138486e7299ef5dff0f388922dc6fcbce009e99d80052024-07-07T11:10:38+02:00Bjarke Viksøeserver: Retrieve prompt template in /props (#8337)
350213701b51a17175d0d326b566efc03f30ec7fbe6be20e7f49d9e5c6d9e8d9b4871eeba3df7a1639d2024-07-05T19:01:35+02:00jaime-m-pDetokenizer fixes (#8039)
351d0a7145ba99ed3a8bc3145aa785b5c86ffe650209ef07800622e4c371605f9419864d15667c3558f2024-07-01T02:09:34+03:00Georgi Gerganovflake.lock: Update (#8218)
352ab3679112d4c49a215a3d31550a7720b202e901597877eb10bd8e7f8023420b5b5300bcbdadd62dc2024-06-27T18:37:29+03:00Georgi Gerganovflake.lock: Update (#8071)
3539b31a40c6ddabe552875b811d7127aa039ca9703c70d117c37cc7876e775d1e2722208a50c52edb32024-06-27T01:50:09+02:00Daniel Beveniusclip : suppress unused variable warnings (#8105)
354e6bf007744eb06336a231ef39cf08146dd16d2ce84631fe1504de40427dc4b4cdac92fa7ebf659552024-06-25T21:07:28+02:00Daniel Beveniusllama : return nullptr from llama_grammar_init (#8093)
3553791ad219323389106dc3fd80814eb5bbb7b80def702a90e245499283d6de0b287701c723cda2a872024-06-25T16:57:35+05:30HanishKVCSimpleChat v3.1: Boolean chat request options in Settings UI, cache_prompt (#7950)
35695f57bb5d5b18ef0beb2702a0d6c06e46804075ce112b610a1a75cb7fa8351e1a933e2e7a755a5ce2024-06-24T03:07:59+02:00slarenggml : remove ggml_task_type and GGML_PERF (#8017)
357e112b610a1a75cb7fa8351e1a933e2e7a755a5ce6a2f298bd784403c5c33eebb822217ec5d9b55902024-06-24T02:27:57+08:00Eddie-Wangllama : add support for BitnetForCausalLM (#7931)
358b6b9a8e606da7764bd3649c2ea574979c85abd4345c0e2e4c1268c2d7c8c45536f15e3c9a731ecdc2024-06-23T13:14:45+02:00slarenfix CI failures (#8066)
359ba5899315283eb1df3902363daf79bdc5eefe426a7854743c5e6216a6178824a603aa9479728ddd52024-06-19T23:57:10Zsasha0552server : fix smart slot selection (#8020)
3609c77ec1d74874ee22bdef8f110e8e8d41389abf2a04a953cab583f0229e7b4b506346e3e9a85c8d02024-06-19T15:04:15+02:00slarenggml : synchronize threads using barriers (#7993)
3616a2f0b3474d479bda4ac2ee7cfd5dcdcf0be1f7921be9cab94e0b5b53cb6edeeebf8c8c799baad032024-06-17T16:10:15+02:00Markus TavenrathImplement non-mapped async IO for CUDA on Windows. (#7896)
362bc6c457fa35f6791e9a2bb61108e7d49e8fc98bd52399254b3bceda279b4ea9111a983e32310166e2024-06-16T19:16:21+03:00Georgi Gerganovflake.lock: Update (#7951)
363cddaf028adc738b5a7ecc60809cb78e0ba0f97c1c8a82194a888f68f259e0d0fa96f3332ac7c5cb62024-06-16T14:50:12+03:00Georgi Gerganovggml : fix handling of zero blocks in IQ quants (#7955)
3640c7b3595b9e5ad2355818e259f06b0dc3f0065b37b2f4a7d193ef2475259bbe7656fcccfab4b12172024-06-15T18:53:40+02:00Xuan Son NguyenAdd `cvector-generator` example (#7514)
36576d66ee0be91e2bec93206e821ee1db8d023cff566ef1ceedf983773c8ceb4d925285d41d4e50e2a2024-06-14T18:41:49+02:00Johannes GäßlerCUDA: faster q2_K, q3_K MMQ + int8 tensor cores (#7921)
366f578b86b2123d0f92afbaa98a031df4d4464e5821c641e6aac5c18b964e7b32d9dbbb4bf5301d0d72024-06-13T03:11:35+02:00slarenmove BLAS to a separate backend (#6210)
36710ceba354a3b152ff425e9fa97f9caaef99a46b1e95beeb1fc4621826ddd616776dbdf717366bf5c2024-06-10T02:04:50+03:00Georgi Gerganovflake.lock: Update (#7838)
3687a16ce7db2a74a223f0f3b9cee66d4539c5bce8fda799b41891e34aac86ce4e173f9c4c0afd4fab32024-06-08T07:50:31Zsasha0552server : smart slot selection using Longest Common Prefix (#7728)
36927615f5ab21060d96953c9c1e223051ab2188f577027b27d765db95d4ac6b569d976e387a87158812024-06-07T05:15:07-07:00intelmattcmake : fix BUILD_SHARED_LIBS=ON build (#7784)
370ee459f40f65810a810151b24eba5b8bd174ceffef83351f9a62a6262f1fc3d08f320033089cddfb52024-06-06T19:19:59+03:00Georgi Gerganovserver : fix --threads-http arg (#7801)
371ad675e1c67a05b16e4e12abe30dbecfc808e7b7ea143c04375828b1f72eb1a326115791b63e793452024-06-06T06:08:52-07:00Clint HerronAdded support for . (any character) token in grammar engine. (#6467)
3729973e81c5ccf4f31b3980f5aa73f5cfea8699860c90dbe026b456a233f8f0fbe752212e6a0503ca22024-06-04T23:40:49-07:00arch-btwreadme : remove -ins (#7759)
3736d1616944d9efd342ed2a4fd318722adfc9febcdbde7cd3cd949c1a85d3a199498ac98e78039d46f2024-06-04T10:01:09+03:00Georgi Gerganovggml : prevent builds with -ffinite-math-only (#7726)
374a5735e4426b19a3ebd0c653ad8ac01420458ee950b832d53ba0ffcc759c8d62ede3772dd62321f8e2024-06-04T00:14:15+09:00Masaya, Katoggml : use OpenMP as a thread pool (#7606)
3756f28a333c1e3fdfdc7b4f9d0367f2b41a9b7e9d4549279d8049d78620a2b081e26edb654f83c3bbd2024-06-03T15:49:30+08:00zhangkaihuollama : MiniCPM support tied embeddings (#7664)
3761669810d7c2446af8425aa54ff6611bf6f14646c7c4e5b7eae26581869e782015d9deca947c349972024-06-03T00:13:12+03:00Georgi Gerganovflake.lock: Update (#7686)
3772ac95c9d5678d05e253691fb1f26471675bff5ad750f60c03e4d3f53fa51910551ce87a3d508d2d72024-06-01T21:50:18+05:30HanishKVCSimpleChat: Simple histogram/repeatMatching driven garbageTrimming, Settings UI, Streaming mode, OpenAi Compat (Model, Authorization Bearer), Save/Restore session, Auto Settings UI (#7548)
378972b555ab935705f3437abd5909a5c46852811f63854c9d07f67de7f8cd6d86117bfaef47549b05a2024-05-30T09:52:39+02:00Johannes GäßlerREADME: explain parallel build [no ci] (#7618)
37987bdf2a199acd62e19814d7a4d0500a04a7f09f300281b7be32462754618c42ed93f95743af466272024-05-29T13:36:39+02:00slarenggml : use atomic_flag for critical section (#7598)
380ee3dff6b8e39bb8c1cdea1782a7b95ef0118f970edc29433fa08b4e5aeb67649a29fc7713af13d042024-05-28T17:07:05+02:00fairydreamingAdd support for DeepseekV2ForCausalLM (#7519)
381c41767154eb82aa3fe7568fc816c3402b78eae9474b239b3d5f067470d7ef5e26e2e059720572e322024-05-28T00:41:14-04:00Nathan EpsteinMarkdownish code block fix (#7571)
382dff451cfa1f297348751ce6b538670e1ae9a7d5bd298382ad977ec89c8de7b57459b9d7965d2c2722024-05-26T18:54:56+03:00Georgi Gerganovflake.lock: Update (#7540)
383d041d2ceaaf50e058622d92921b3e680ffa4e9e727891f6db03de6e3fd5941983838c29bef2533522024-05-24T18:59:06+03:00Georgi Gerganovflake.lock: Update (#7232)
384fbca2f27fc7fa9aa4a8ad0357478fdb9084729080df0aa8e43c3378975269a51f9b876c8692e70da2024-05-24T14:31:13+02:00fairydreamingAdd support for ArcticForCausalLM (#7020)
3853015851c5ac7334fb544a23a70a284c117b8704455ac3b7aeaf52f19786ed96e885d89521fc0f6c82024-05-23T14:29:26+02:00Daniel Beveniusllama : add getters for n_threads/n_threads_batch (#7464)
3861e374365d170b7f692fd7753c145e21bc14486c8197ff91462dd05bb9a3be03578114abf0c3555362024-05-22T23:23:21+05:30HanishKVCSimpleChat: a simple and dumb web front end for testing /chat/completions and /completions end points and try chat (#7350)
3873bc10cb485dd7efa4da6c64e73ad0c9e2bfe08216bf9b66fa3f263ca2175dcb5f6d0a658581e1dfb2024-05-20T15:10:03+03:00Georgi Gerganovserver : fix temperature + disable some tests (#7409)
388cb42c294279bc4a0a4e926a7b5a5568049f12fa7d233b507cd19fcc2d8d8963ecc6a3eb7a33f2ecc2024-05-18T11:10:47+02:00Johannes Gäßlerserver: correct --threads documentation [no ci] (#7362)
389d273c1402b25086fd91aef2467ac13f2e49fa0ea27b040691cbe45314147c2745e891a38e9c048d42024-05-17T15:11:45+03:00Aarni Koskelapy : convert-hf-to-gguf-update improvements (#7340)
390e1b40ac3b94824d761b5e26ea1bc5692706029d9dc020985b8755dd6aa93a2f002f43c3ede808cce2024-05-15T12:59:12-05:00kunnisggml : use dynamic thread scheduling for matrix multiplication (#6915)
391f308ea705974dff62a1fe5367d776ad9d5109239c3c88f296a72432edb697ac8026dbf2ec18f2b212024-05-13T11:01:07+03:00Georgi Gerganovmetal : tune soft_max number of threads (whisper/0)
392541600201e6480f54ae09e58d16b154d4b4b331defc8f767c8c8c749a245dd96ad4e2f37c164b54c2024-05-14T09:33:42+02:00slarenllama : disable pipeline parallelism with nkvo (#7265)
393988631335a20d06497f58be0b8ba13adb4323a22f99e1e456eaf69cc38c1982a2693ce41c0f897ef2024-05-11T04:13:02-04:00Steve Grubbserver : free llama_batch on exit (#7212)
394bc4bba364fb96d908f2698e908648df5e6f55e02c12452c7aec8a02264afc00196a13caa591a13ac2024-05-08T21:55:49+01:00agray3Introduction of CUDA Graphs to LLama.cpp (#6766)
395b3a995b416e13ae3123a117a743e11d0ede0ca4cbcdee0daa7c5e8e086b719e5eb4073b00df70e012024-05-06T18:36:06+03:00Georgi Gerganovflake.lock: Update (#7079)
3969c67c2773d4b706cf71d70ecf4aa180b62501960952d03dbead16e4dbdd1d3458486340673cc24652024-04-30T12:16:08+03:00Georgi Gerganovggml : add Flash Attention (#5021)
397b8c1476e44cc1f3a1811613f65251cf7790676365539e6fdd1b97e0c37c54d34df67f334fc344a262024-04-29T14:40:14-04:00Clint HerronExtending grammar integration tests (#6644)
3986e472f58e40cd4acf6023e15c75a2700535c5f0b4dba7e8114d84241c842b986e008af8b88d1a0192024-04-28T00:18:27Zgithub-actions[bot]flake.lock: Update
3997d641c26ac73956a54b204cabefe85c7648236785790c8dac1a4f0aa80b4efee3b962d8c04c829e82024-04-26T09:26:59+02:00Pierrick Hymbertci: fix concurrency for pull_request_target (#6917)
40028103f4832e301a9c84d44ff0df9d75d46ab6c76c0d1b3e03e27634ac2871761f5033cf9324d472d2024-04-24T11:08:36+02:00Johannes GäßlerServer: fix seed for multiple slots (#6835)
401e9b4a1bf68c18beff4e33f23ea62c1245b2969155cf5e7d490dfdd2e70bface2d35dfd14aa44b4fb2024-04-21T00:17:47Zgithub-actions[bot]flake.lock: Update
4028cc91dc63c0df397d644a581b2cbeea74eb51ae0dbceec87c0221ec952e69448df6a71f1372a74872024-04-16T14:55:30-04:00Justine Tunneyggml : add llamafile sgemm (#6414)
403f184dd920852d6d372b754f871ee06cfe6f977ad422c2aff1c9735853c9d8f5162104e41a364adc42024-04-14T16:55:30+03:00Georgi Gerganovflake.lock: Update (#6669)
404b804b1ef77351d2a11be945462c6c251710476cb8228b66dbc16290c5cbd70e80ab47c068e2569d82024-04-11T14:51:07+02:00Pierrick Hymberteval-callback: Example how to use eval callback for debugging (#6576)
4055dc9dd7152dedc6046b646855585bd070c91e8c8e11a8999b5690f810c2c99c14347f0834e68c5242024-04-09T09:16:13+01:00Carolinabananallama : add Command R Plus support (#6491)
406beea6e1b16e783a0886e78dec01002a8c00db24d87fb5b4234d4b9c56ac94cf7aa229c8fd7defdb02024-04-08T20:43:30+08:00Jan Boonllama : save and restore kv cache for single seq id (#6341)
407b909236c0bf0b6e872af95df9490492ecec310ace0717e751e12af13f4eedaae8bbbd608e40d7e542024-04-07T21:25:30+03:00Georgi Gerganovflake.lock: Update (#6517)
4088120efee1d9931b514aeb5a047209d576f23286ca74401f0e5ebb15fa4d8b6619d1baa6ea91791232024-04-04T16:59:04+02:00Pierrick Hymbertci: bench fix concurrency for workflow trigger dispatch with sha1 (#6478)
4097a2c92637ae265654a68f62e6a7610b358255d3f4bcd6b959ca3991084ad1d8464caf2a734e29b1d2024-04-04T11:57:58+02:00Pierrick Hymbertci: bench: add more ftype, fix triggers and bot comment (#6466)
41008a0c0206075556e82aca0feafad530dcc5f142652604860f93063ef98863921da697576af1c76652024-04-03T15:07:05+02:00slarenggml : mul_mat_id use the same tensor for all the experts (#6387)
411f87f7b898651339fe173ddf016ca826163e899d833a52448061cfd2ea44da9e6cb30b2ec22e2f6d02024-04-01T19:05:57+03:00Georgi Gerganovflake.lock: Update (#6402)
412b75c38166cf0c66793a32d5c3d6cb69929dd083dbfe7dafc9cf96b9a09ead347fed9a547930fc6312024-03-29T08:15:00+01:00Pedro Cuencaconvert : allow conversion of Mistral HF models (#6144)
4136902cb7f2e3479f364ee177118200fb7e4e9fc92d2d8f389960a106b66313ff1621bdb1aaaaaa2852024-03-28T16:50:48+08:00Eric Zhangserver : stop gracefully on SIGTERM (#6348)
414557410b8f06380560155ac7fcb8316d71ddc983755c1b2a3bbd470e9e2a3a0618b92cf64a885f8062024-03-26T10:46:41-04:00compiladellama : greatly reduce output buffer memory usage (#6122)
41555c1b2a3bbd470e9e2a3a0618b92cf64a885f806e097633f63fdd26d492844f7eff056e4083fd9eb2024-03-26T15:21:27+01:00KawrakowIQ1_M: 1.75 bpw quantization (#6302)
41643139cc528909c3de8c144ed174e09a1f7912a802f34b865b62b1d2b5eb8a27885e4de220deeacbd2024-03-25T17:22:27+02:00Georgi Gerganovflake.lock: Update (#6266)
417f482bb2e4920e544651fb832f2e0bcb4d2ff69ab1997577d5e121568ae39f538021733ccd4278c232024-03-23T18:07:00+01:00Pierrick Hymbertcommon: llama_load_model_from_url split support (#6192)
4181997577d5e121568ae39f538021733ccd4278c23476b0251b27fb64c575507024a671e639d6755942024-03-23T18:00:38+01:00Pierrick Hymbertserver: docs: `--threads` and `--threads`, `--ubatch-size`, `--log-disable` (#6254)
419ee804f6223777019cf921e0d99cc24669313ab9880bd33bc2c4be352697dc8473339f25e1085d1172024-03-23T02:15:06+09:00Minsoo Cheongci: apply concurrency limit for github workflows (#6243)
420be07a03217376c53b1d95e5f658adbbbb2831555d0a71233fbf8ade8ef06ad8e6b81d1d7b254895f2024-03-22T06:41:24+08:00Jan Boonserver : update readme doc from `slot_id` to `id_slot` (#6213)
4212d15886bb092c3b780c676b5cc57ff3337af9c83d199ca79f279e84ebe27caafe0aa59c461d889692024-03-17T06:37:44Zgithub-actions[bot]flake.lock: Update
422dc0f6125487dcfbff913360f9d877bc0ccf6aa57c47cf414efafb8f60596edc7edb5a2d68065e9922024-03-18T01:12:22+08:00GainLeeggml:fix finding transfer queue family index error (#6094)
42312247f4c69a173b9482f68aaa174ec37fc909ccf4e9a7f7f7fb6acbddd1462909c8d696e38edbfcc2024-03-15T16:41:22-04:00Andrew Canisllama : add Command-R support (#6033)
424f30ea47a87ed4446ad55adb265755dc9102956a2d8fd0ccf6ac8b07791ffd1575eed436930854ae32024-03-13T18:54:21+01:00slarenllama : add pipeline parallelism support (#6017)
4258030da7afea2d89f997aeadbd14183d399a017b9184215e783dfad76aded2c68244c327a5c507df52024-03-12T14:27:20+02:00Georgi Gerganovggml : reuse quantum structs across backends (#5943)
42605b06210c954491cf0f12034b0a62bd4d69ce78b83796e62bc9f6caae6228168e359890f51e60fee2024-03-11T17:49:47+02:00Georgi Gerganovllama : more consistent names of count variables (#5994)
427caa106d4e05a0ab94225c220b81f9e2cd522339b3202361c5b1ba15e695b31209567ef42c22c5c322024-03-11T10:56:41+01:00Xuan Son NguyenServer: format error to json (#5961)
428be858f620508385ad12d0e5e862010e666ca729cef3ced26a3817d92890b97b83acaeb018ade02d02024-03-11T07:51:49+01:00KawrakowBetter 1.5 bit quantization (#5971)
429fa8a809a9119411bc9c3f00026550d0343f4d9b7bcebd7dbf62fd7b293d5ed089023e4e733269c712024-03-10T18:17:47+01:00Pierrick Hymbertserver: ci: windows build and tests (#5968)
430c78541479cf835dd9eb568ccd9a2083198a7203d621e86b331f8b0e71f79fd82a4ae1cd54c3e43962024-03-10T16:43:08+02:00Georgi Gerganovnix: update flake.lock (#5969)
431d894f352bf433157232dc8dc54eacd50014e898e098dbaab449f5309a54871ba7e5acef72ae696de2024-03-09T19:55:54+01:00slarenperplexity : support using multiple sequences to allow larger batch sizes (#5946)
432c2101a2e909ac7c08976d414e64e96c90ee5fa9e515f7d0d4fce41c752fc253acf30707c3be2531e2024-03-08T17:31:00-05:00compiladellama : support Mamba Selective State Space Models (#5328)
4332002bc96bf2cbf5ab981a17d7e994d817c9801f5ceca1aef0738b57951cd12c603c3477e75312dec2024-03-07T11:41:53+02:00Georgi Gerganovserver : refactor (#5882)
43421b08674331e1ea1b599f17c5ca91f0ed173be316a87ac3a52668e117d97bcea07b529c93188b3032024-03-05T16:08:35+08:00Neo Zhang Jianyu[SYCL] fix mul_mat fault in CI/unit-test (#5862)
43567be2ce1015d070b3b2cd488bcb041eefb61de72231ae28f078c3148d097b301f2145f1e3e816cc12024-03-03T14:26:18+01:00slarencuda : fix data race in soft max (#5853)
4368ef969afcec1645d2d9c3ab1fc82263bba968989fa974646e1a2024fc7dc9e6f27cf1f2f5d4a37632024-03-03T08:48:36+01:00Pierrick Hymbertserver : init http requests thread pool with --parallel if set (#5836)
437fa974646e1a2024fc7dc9e6f27cf1f2f5d4a37639731134296af3a6839cd682e51d9c2109a871de52024-03-03T06:11:31+02:00Georgi Gerganovflake.lock: Update (#5842)
438bbde6eb2561153aabbdfac5001c690fe00cad639ef2cd694c4155fbf25bae61c5178c47eb3676dba2024-03-02T17:00:51+02:00Kawrakowggml : IQ3_S improvements (#5829)
439c29af7e2252d288f2ea58a7d437c1cb7c0abf16038d16b142624bdd7c41d9955752b7f7b59c5e0482024-03-02T01:00:46+05:30Sourab Mangrulkarllama : add StarCoder2 support (#5795)
4405cb02b4a012bb16c6c699c0c62c05ffa653eee0f6ea0f010ff6967034528d9e0b8330b9b0f0b7c132024-03-01T10:08:08+01:00Pierrick Hymbertserver: allow to override threads server pool with --threads-http (#5794)
4415f706718566e3a5147916dc381f3b99de0ffad47a693bea1e6762a17b78b6ddf4611e54136941ea22024-02-24T11:27:36-05:00UEXTM.comIntroduce backend GUIDs (ggml/743)
442a693bea1e6762a17b78b6ddf4611e54136941ea2adcb12a9bad87bc96f2f158c95892b3d04aa7ffb2024-02-28T09:55:37+01:00Xuan Son Nguyenserver : hit Ctrl+C twice to exit (#5734)
4437c4263d4261d6ee6f0539d53eb9e1b4d120ba8afcb49e0f8c906e5da49e9f6d64a57742a9a241c6a2024-02-28T10:37:02+02:00Kawrakowggml : make i-quants work with super-blocks of 64 (CPU,Metal) (#5760)
4440becb22ac05b6542bd9d5f2235691aa1d3d4d307c24a2a6e6005e5d424301525a42ba45a4a362d302024-02-27T16:34:24+02:00KawrakowIQ4_XS: a 4.25 bpw quantization (#5747)
445c39373398803c669056304090050fe3f44b41bf9e3965cf35aac00d4e24998c8a3d0093ae1d98bd32024-02-25T00:17:11Zgithub-actions[bot]flake.lock: Update
446930b1780269a69948d106e2d1b838ab7661f679ad52d7819b8ced70c642a88a59da8c78208dc58ec2024-02-25T13:50:32+01:00Pierrick Hymbertserver: logs - unified format and --log-format option (#5700)
447d52d7819b8ced70c642a88a59da8c78208dc58ec12894088170f62e4cad4f8d6a3043c185b414bab2024-02-25T13:49:43+01:00Pierrick Hymbertserver: concurrency fix + monitoring - add /metrics prometheus compatible endpoint (#5708)
4489e359a4f47c1b2dceb99e29706c9f7403d32ab5e4c4cb30736582cacb1a164a9d4bc8e17b1014be72024-02-24T19:16:04+01:00Pierrick Hymbertserver: continue to update other slots on embedding concurrent request (#5699)
4494c4cb30736582cacb1a164a9d4bc8e17b1014be7525213d2f5da1eaf4b922b6b792cb52b2c6133682024-02-24T16:23:52+02:00KawrakowIQ3_S: a much better alternative to Q3_K (#5676)
450525213d2f5da1eaf4b922b6b792cb52b2c613368fd43d66f46ee3b5345fb8a74a252d86ccd34a4092024-02-24T12:28:55+01:00Pierrick Hymbertserver: init functional tests (#5566)
4511ecea255ebb70750b52688393f37a63606b90e3fa00a35cef93e057eace8351a667d14d152a91ebc2024-02-21T15:47:48+01:00Pierrick Hymbertserver: health: fix race condition on slots data using tasks queue (#5634)
452a14679cc30c785e75d38028bae6ec39c6209ddef6560bed3f066c876682464762cad90f1e28e3f1b2024-02-21T11:39:52+02:00KawrakowIQ4_NL: 4-bit non-linear quants with blocks of 32 (#5590)
453c0a8c6db371cb3e4379900867b948879f5842201b9111bd209c7b11b0592450a6ed2e0ca545b2c842024-02-20T08:48:19+01:00Pierrick Hymbertserver : health endpoint configurable failure on no slot (#5594)
454f0d1fafc029a056cd765bdae58dcaa12312e9879a0c2dad9d43456c677e205c6240a5f8afb0121ac2024-02-18T23:38:32-08:00bmwlggml : android and old glibc NUMA incompatibility bugfixes (#5557)
455c145f8a132b2fe1d1e65987faddbd9a40bef7a12689a091bbe0537ee9abff3e15a1d74f5f35611652024-02-18T18:39:57+01:00Pierrick Hymbertserver : slots monitoring endpoint (#5550)
456e75c6279d1c8e7abb82a331f5de7124eed402de236376abe05a12a8cb3af548a4af9b8d0e2e695972024-02-18T17:31:28+01:00Pierrick Hymbertserver : enhanced health endpoint (#5548)
45766c1968f7a2e895675425e875b6589f1233a1b521dcc3fde004787e6fc4d84c9de0bb34cd2901a3e2024-02-18T08:23:16-08:00Daniel Hiltgenserver : graceful server shutdown (#5244)
458c8e0d7efeb7634ecc2e9832e879ab9fca4510e718f1be0d42f23016cb6819dbae01126699c4bd9bc2024-02-18T00:17:07Zgithub-actions[bot]flake.lock: Update
459f486f6e1e5e9d01603d9325ab3e05f1edb362a9560ed04cf82dc91ade725dd7ad53f0ee81f76eccf2024-02-16T01:31:07-08:00bmwlggml : add numa options (#5377)
4600d4177126b0556e202efb85bf3f768be810764007930a8a6e89a04c77c51e3ae5dc1cd8e845b6b8f2024-02-15T09:01:57+01:00Elbiosllava : fix memory management bug (#5491)
461cf45252a7cfcb998bade46a886e20477cecc538a03bf161eb6dea6400ee49c6dc6b69bdcfa9fd3fc2024-02-13T15:14:22+02:00Georgi Gerganovtests : multi-thread the tokenizer tests (#5474)
46297a336507ed9b971d72262bec7e2b8b7016a054ac88c74f967028ae3d5ebade40ae586d20a961abc2024-02-11T00:17:31Zgithub-actions[bot]flake.lock: Update
463f026f8120f97090d34a52b3dc023c82e0ede3f7dcd9aea63b577a83def84dbd6dcd90a6fa02af7452024-02-10T02:53:28-08:00Ian Bullmetal : use autoreleasepool to avoid memory leaks (#5437)
4644b7b38bef5addbd31f453871d79647fbae6bec8ae00d2a62dd1441e3b089570ec06d05c18800d3682024-02-10T05:30:19+11:00Neuman Vongvulkan: Set limit for task concurrency (#5427)
465e5ca3937c685d6e012ac4db40555d6ec100ff03ce4124c24775f2cb5b3d7acc93bf9dc5471c172ef2024-02-09T10:48:06ZPaul Tsochantarisllama : do not cap thread count when MoE on CPU (#5419)
4664ffc7a17d4e80c5f3f905139cb570ed9b6934fcb906cff55c2848fda091d888a1585915ec0c9ea9e2024-02-06T08:16:23ZNiall Coatesserver : various fixes for the prompt field in /completion (#5300)
4676fdfa2ecc684000a25a4ad91823bc82a6652b645a2d60c9158435ae9a6f14632f07f1acf7a3becef2024-02-05T10:46:06+02:00Kawrakowiq2_xxs: tune quantization (#5320)
4689392ebd49ea5ae236a55b47cbf6a13247e8a3b8c5ed26e1fc9fab4ce96ecf2d84183fe45bdcab0d42024-02-04T00:17:24Zgithub-actions[bot]flake.lock: Update
469e920ed393d989ed35625ddaf182ebb52cda07fcd52bb63c7082c859c3f1dfc527227e6a95b299c7c2024-02-03T18:15:00+01:000cc4mVulkan Intel Fixes, Optimizations and Debugging Flags (#5301)
47015606309a05ccf7fadbaad5538cb7c32acb1e06bb2b9f025e7821e78bd501d75d01838c26de07a572024-01-31T21:10:15+08:00JidongZhang-THUllava : add MobileVLM support (#5132)
471dabcc5b471348e4ae03ddacc41e19ad75fb2f041f8e9140cb46eebaa867e1184a9946e4840eec7722024-01-31T13:43:03+01:00slarenggml : limit n_threads to the max n_tasks (#5238)
4722307523d322af762ae06648b29ec5a9eb1c730320f648573dde61c510560f68244f70ece7e60d8c12024-01-28T18:03:59+01:000cc4mggml : add Vulkan backend (#2059)
4730f648573dde61c510560f68244f70ece7e60d8c1b764b8f1d079ba44d912801ce6d29bd0d94d51cf2024-01-28T21:26:23+05:30Abhilash Majumderggml : add unified SYCL backend for Intel GPUs (#2690)
474b764b8f1d079ba44d912801ce6d29bd0d94d51cf9241c3a2ace544aef708334e54bbdddb90208ee82024-01-28T16:54:54+02:00Georgi Gerganovflake.lock: Update (#5162)
475b2b2bf988c098851b4f3831f0cf38394bff75121af4980bfedfd8df43b9e4cd1442895e85fee37bc2024-01-28T09:35:14+01:00Johannes GäßlerTests for min_p, sampling queue (#5147)
4767032f4f6349c17a8352f9f93f7d2122f45469e595f1925a8cef81eb9b372faaae34b0dd76d5361d42024-01-26T11:17:59-06:00snadampalggml : update softmax n_task calculation (#5126)
47748c857aa10aea73210a4a72da3f1a6f99269e75d413e7b0559f922bd4de5e9eec548829d111651b12024-01-26T13:42:20+01:00Xuan Son Nguyenserver : refactored the task processing logic (#5065)
4785eaf9964fc797d4585c214db32a463d557f3ed33d292f4f2047963f558dd516f1baaa71793e9acf22024-01-26T05:06:22+09:00l3utterflyllama : dynamic temperature sampling (#4972)
479bf63d695b804b1c995c7ae4427a8a86936ea6d251387ea21178f9f154944013d4dd9764b54c69deb2024-01-22T03:17:05-07:00Michael Hueschennix: add cc to devShell LD_LIBRARY_PATH
480780e24a22eb595b705cbe8284771e9ceff1c4dd23ce7e8f8e7ccfce07e5947ac5f1f3f4628cf68ea2024-01-22T21:15:08+08:00Reinforce-IIggml : parallelize FP32 conversion when using BLAS (#5045)
481f7276f7500f7ea588836dd1fc6f126334c51787815bceec2d73d4166340b46b27677c45ac1b4cdad2024-01-13T17:10:19ZSomeone Sergeworkflows: nix-ci: rebuild on flake.lock updates
4827dcbe39d36b76389f6c5cd3b151928472b7e22ff726c0fa9a2da976e9c5d5c51e185d9dd453fc9e52024-01-21T14:42:44+02:00KawrakowAdd ability to evauate multiple choice tasks (#5047)
483942c0107a7301434c0a5e7da46bc4cf2393aa556b43ebde3b0ccbc42d9dd782b32e2fd8eb35b43b52024-01-21T05:17:27+02:00Georgi Gerganovflake.lock: Update (#5054)
4847051aacfac0057fa5fac9ea46c55bffc3892d8102b3b999cacc7ad1207c32fbdf3479a19c06e1a342024-01-19T11:39:11+02:00Kawrakowwinogrande: evaluate log-probs in parallel (#5036)
48596d7f56d2918ffde1995dbb32392571deb76d7fc2d5419d08ab1131623e6a1d554607b7663435e872024-01-18T21:12:15+01:00slarenllama : fix mlock with no-mmap with Metal (#5025)
4863e945cc1e9c06d2001031360e4e303e9548fb02cad19812cda4062c9f154ef16315df41fbe6a770a2024-01-18T19:18:21+02:00KawrakowHellaSwag: speed up by parallelizing log-prob evaluation (#5020)
487ad19812cda4062c9f154ef16315df41fbe6a770a682986a08eb5cb04865d2e713449f17304d266d82024-01-18T15:33:01+02:00Georgi Gerganovperplexity : faster HellaSwag via batching (#5017)
4884feb4b33eeb1756e46084a4db9230b279af1a480959ef0c0df725c013c7f712eaa7790b8e38a8e202024-01-16T18:41:42+01:00Maximilian Winterexamples : add complete parallel function calling example (#4974)
489c37b3474e61d609d43cccc3bde5d559e80e4f5d1158f8c9e21302114bac3c646f80ea85b52ffa0bd2024-01-16T19:13:54+02:00Georgi Gerganovflake.lock: update flake-parts, flake-parts/nixpkgs-lib, and nixpkgs (#4920)
490158f8c9e21302114bac3c646f80ea85b52ffa0bd862f5e41ab1fdf12d6f59455aad3f5dd8258f8052024-01-16T17:05:19ZPaul Tsochantarismetal : localized logic in `ggml_metal_graph_compute` (#4924)
4913a48d558a69c88ac17efcaa5900cd9eb19596ac47c8d3abd1a17c28fc56b1a4814bc4b29f91d74542024-01-16T14:41:27+01:00Alex Azarovmetal : replace loop of dispatch_async with dispatch_apply (#4934)
492e0324285a569d0583cf2f4a07a2402221ee25f583e5ca7931c68152e4ec18d126e9c832dd84914c82024-01-16T12:04:32+01:00stduhpfspeculative : threading options (#4959)
493bb0c1392479398f9aba86d9ec98db0b95ede6e6d9408cfdad6b1c090a7e1419d4434edc260b7e47e2024-01-14T13:26:53+02:00Georgi Gerganovllama : check LLAMA_TRACE env for extra logging (#4929)
494df845cc982e7e2ea7b9900e29d55b15338faa78d6b48ed089377330cdb362970a51c1c89b6d857a82024-01-13T17:29:43+01:00David Friehsllama : minimize size used for state save/load (#4820)
495356327feb3f66980ab687040495d722696d98970ee8243adaa9a9f51ff449213383874e49efe368f2024-01-13T09:20:46-05:00Ziad Ben Hadj-Alouaneserver : fix deadlock that occurs in multi-prompt scenarios (#4905)
496e7e4df031b9e29d4b55a4e0b0295187f6b213db1584d674be622fbf1578694ada6e62eebedbfd3772024-01-12T20:07:38+01:00slarenllama : ggml-backend integration (#4766)
497eab67950068e4b125007d027232c47d2a5831cd0d8d90aa343c22fe01429d3540e47ded87e9dcb9d2024-01-11T12:41:39-05:00Behnam Mserver : add `LOG_INFO` when model is successfully loaded (#4881)
4987a9f75c38b5e62fe27b8a5a3ed823b4a3714024b5c1980d8d4c4e0c0af77359f81cc44d90b3f250b2024-01-11T02:12:05-05:00Behnam Mserver : update readme to document the new `/health` endpoint (#4866)
499cd108e641dbdedd8c5641c4cec1762f751f3813657d016ba2d46a6e22517a31a75cebb48f9e234b62024-01-10T14:56:05-05:00Behnam Mserver : add a `/health` endpoint (#4860)
500f2eb19bd8bc9f5730d6e05d7a52a9e19bf5ac099f3f62f0d835d559e80714bbeb05d03125574e3dd2024-01-03T03:43:19-05:00Justin Parkerserver : throw an error when `slot unavailable` (#4741)
5015d7002d4372ebf107cfaf46fcd90df27b204f33026f3071d714f0b27ad7f021a46a66a10854802582024-01-02T04:38:15-06:00minarchistserver : add --override-kv parameter (#4710)
502edd1ab7bc34c10a780ee7f9a4499f7689cdad36d198ed7ebfc89b8f2b35a8b1655d57bfb57530c1a2023-12-31T17:42:22ZSomeone Sergeflake.lock: update
50368eccbdc5b56f2a2450f9a8463f9934388cafabf97bbca6e8522d18041fcde6c3d0907a52ce364462023-12-29T06:42:26-08:00Philip Taronflake.nix : rewrite (#4605)
504441f51dca004debf8b275f1bdc08e0f1af7fd8f838b3de4658292582a8941a2be5c77b40ce6ac0f22023-12-29T19:23:27+09:00Tamotsu Takahashici : build with CLBlast + ggml-opencl use GGML_API (whisper/1576)
505dc68f0054cd279cddddb0cae0c9ef4f9cbaa512ade8e496437c59e7d1cc84109e3e49a3478aee25a2023-12-26T21:23:59+01:00slarencuda : fix vmm pool with multi GPU (#4620)
506d232aca5a73b290e218a2e48b91023d5e994203f31f27758faf4a4bd08101a57c7ec3a473f771f862023-12-21T21:07:46+01:00slarenllama : initial ggml-backend integration (#4520)
507880e352277fc017df4d5794f0c21c44e1eae2b8466f35a2f48e1965a13835a523e677223dbf148be2023-12-21T18:07:34+01:00howlgerpy : open merges file as 'utf-8' (#4566)
508799a1cb13b0b1b560ab0ceff485caed68faa8f1ffecac45658a99eddc4d6e36ba0310ca8f87a77f02023-12-13T13:04:25+01:00slarenllama : add Mixtral support (#4406)
509da5eaef1f34d0a1f584cd4a092e7691ea46a9d915f6e0c0dff1e7a89331e6b25eca9a9fd713240692023-12-06T09:08:17+01:00stduhpfspeculative : support `--color` (#4343)
51052c8bc3cf312e1caf02d37bfb9d9d865cbe33594e4b76bbe316ee50fb17d9ac29e654c0edf830eba2023-12-05T15:05:51+05:00MaggotHATEsampling : custom samplers order (#4285)
51123b5e12eb5a76489b4c3ee22213a081da68b1809d208995c6da66f252d4054c1c5a90eb8ccb7a2f72023-12-04T17:04:21+01:00Daniel Beveniussimple : update error message for KV cache check (#4324)
512880f57973b8e0091d0f9f50eb5ab4cd4e31582ca8d6d9f033b8101f929e445cf45b39e1557ca79342023-12-01T18:42:11+02:00Georgi Gerganovllama : fix integer overflow during quantization (#4284)
5138d6d9f033b8101f929e445cf45b39e1557ca7934ef47ec18da469423c276b683dd9b5741cee7023e2023-12-01T10:41:56+01:00Daniel Beveniuspy : add requirements file for convert-hf-to-gguf.py (#4277)
514ef47ec18da469423c276b683dd9b5741cee7023e1d144112c0fbbb4ecc07dbcf4f05a380148bd6de2023-12-01T10:51:24+02:00Georgi Gerganovggml : add ggml_soft_max_ext (#4256)
515f43f09366dfd018e4568e23a232aaa8c4f7cfc78d2809a3ba2780e00fce5a6149a7eda09f1c0e9062023-11-30T17:25:04-05:00Ziad Ben Hadj-Alouaneserver : add single-client multi-prompt support (#4232)
516e2bd725f4b39bc5c6234858d158e01248f5ab5bd1f5cd83275fabb43f2ae92c30033b384a3eb37b42023-11-30T20:50:40Zrhjdvsgsgkspy : fix oai proxy (#3972)
5178406b0924bf323f37d536dee8b8165c1f3d9d11db38a16dfcff88d547f78f52d1bea31b84a05aff72023-11-28T10:32:03+02:00Georgi Gerganovggml : re-enable BLAS for CPU when src0 != F32 + remove redundant full offload checks in llama.cpp (#4240)
5189d5949f04b1f8b76184818abbd99938c2020803fff8238f71d56245f4a6dbea693f4ebd81263464d2023-11-23T12:34:20+01:00Daniel Beveniusexamples : fix typo in parallel example doc comment (#4181)
519f23c0359a32871947169a044eb1dc4dbffd0f40540a34fe8d034bd484efd79ccbb95059ca6308dcb2023-11-20T11:35:47+01:00Galunidci : add flake8 to github actions (python linting) (#4129)
520532dd74e38c29e16ea1cfc4e7eedb4f2fab3f3cde86fc56f7521ca4b18d1d9939e82abd40c2f1c012023-11-11T22:04:58-08:00Richard KissFix some documentation typos/grammar mistakes (#4032)
52148ade94538fa509465d71023e49d07aab0ec8cd5f28af0d81aa1010afa5de74cf627dcb04bea31572023-11-05T08:12:13+01:00slarencuda : revert CUDA pool stuff (#3944)
522d6069051de7165a4e06662c89257f5d2905bb1564ff1046d75e64f0e556d8dcd930ea25c23eb8b182023-11-02T18:10:39+01:00Oleksii Maryshchenkocuda : use CUDA memory pool with async memory allocation/deallocation when available (#3903)
5230e40806c1cb3bdf9955ed807ffbe212be85b4c67a2758d08e44ce3624d233af4d23c6843e2e735b52023-11-01T14:42:01-03:00bandoticommon : allow caller to handle help/argument exceptions (#3715)
524ff3bad83e29e3009010cbc923bebd769055eaa7f82a6646e0221216c41edcdf99f5a44bb051391f52023-10-28T16:41:07+02:00Erik Scholzflake : update flake.lock for newer transformers version + provide extra dev shell (#3797)
52534b2a5e1ee4fe6295fb4420eb91131d743694c656961c4bd0b5176e10ab03b35394f1e9eab7617922023-10-26T22:53:37+03:00Georgi Gerganovserver : do not release slot on image input (#3798)
526ad939626577cd25b462e8026cc543efb715284721717521cdb976a2219888b0e5cba36e210eee9df2023-10-24T16:10:43-04:00cebtenzzreserver : add parameter -tb N, --threads-batch N (#3584) (#3768)
5271717521cdb976a2219888b0e5cba36e210eee9dfb2f7e04bd312eaf97eee0523aa09d950d585626b2023-10-24T23:08:20+03:00Georgi Gerganovserver : do not block system prompt update (#3767)
528438c2ca83045a00ef244093d27e9ed41a8cb4ea99e70cc03229df19ca2d28ce23cc817198f8972782023-10-22T22:53:08+03:00Georgi Gerganovserver : parallel decoding and multimodal (#3677)
5290e89203b517c95ec6675eda75d200a60d1e8921dc67fe68e417f766970fb1feaf2e66458aa24116a2023-10-18T16:21:57+03:00Georgi Gerganovspeculative : add tree-based sampling example (#3624)
530c67fe68e417f766970fb1feaf2e66458aa24116a1117d06607d2d885640ac501f05f0aae5494e2c52023-10-18T07:21:48-05:00Jhen-Jie Hongmetal : implement q5_0 and q5_1 kernels (#3648)
5312a4bcbacead886996f175f33479d1d874a3e577f424b6381c4daeed62e6600e0402e72f39845b58d2023-10-13T12:33:16+02:00Daniel Beveniusllama : remove n_threads from llama_decode_internal (#3614)
532a8bdd65525ae86dea905e9866ad369b53e30ac1470c29da118cdb02bfcbd0376c32b5b2236e48e482023-10-11T15:42:22-04:00Michael Coppolaserver : add parameter -tb N, --threads-batch N (#3584)
533a1202a31ed8c35705bd09fe91c3e7410c619bd7094e502dfb79430870b42b8e8ee132b4aaa93e4a82023-10-08T12:21:19+02:00Johannes Rudolphk-quants : fix comments about block sizing (#3499)
5349ca79d5cbbc8d43f2bff951404b6a40ff1ee37880c731ca4039ccff86ffab90eaae4ca98037c44962023-10-06T10:10:13-06:00Kerfufflekv cache slot search improvements (#3493)
535a8777ad84e00cda0399e827cdf971e2c3fab1da297af49fa395df77e4c18af0e1655b2fee67c96862023-10-06T14:16:38+01:00pudepiedjparallel : add option to load external prompt file (#3416)
536e2583cbc29cd7d6d1403f338842c07dfc0467e6ce8b8d32e8663ffc55a02c9721af3a5190382cbb02023-10-05T15:57:03+04:00shibe2CLBlast: Fix handling of on-device tensor data
5377f1a0fe709ea1a861da2f3759f58a28bf8953c1216bc66d9479edd5ee12ec734973554d4493c5dfa2023-09-29T03:51:52+08:00Qu Zongfuggml : release the requested thread pool resource (#3292)
53816bc66d9479edd5ee12ec734973554d4493c5dfa0512d66670de3f650c579519833c085014b0f2002023-09-28T21:42:38+02:00slarenllama.cpp : split llama_context_params into model and context params (#3301)
5390512d66670de3f650c579519833c085014b0f2000e76a8992c8200237bbc6471a53fb8796b3872f72023-09-28T19:31:04ZEveci : multithreaded builds (#3311)
5400e76a8992c8200237bbc6471a53fb8796b3872f72db94d98eda56982d80238840b0652b4137a2a842023-09-28T20:40:11+02:00xaedestrain : finetune LORA (#2632)
541ec893798b7a2a803466cc8f063051499ec3d96f745855b3f1c7bdd0320aa632334d0b3e8965c26c42023-09-28T19:04:36+03:00Georgi Gerganovllama : custom attention mask + parallel decoding + no context swaps (#3228)
5428185710a80531e9ee0c0cb99d3a9c9af1019ab677eb41179edc56083ef4eb2df7967ac9ff38b34fb2023-09-21T10:43:53+02:00Johannes GäßlerCUDA: use only 1 thread if fully offloaded (#2915)
543a51b68765799e75e17c7622f376bfbeb66f1bd7076164fe2e65c058e9ee2c3afd0ad6b182ca57e252023-09-15T11:09:24+03:00Georgi Gerganovmetal : relax conditions on fast matrix multiplication kernel (#3168)
5444c8643dd6ea1a163bc5979cb69c1e7ab0975bc9335f73049af6c676a106a5a990a819ae0bc3fcd7d2023-09-15T00:32:10+08:00jameswu2014feature : support Baichuan serial models (#3009)
545cb6c44c5e045709b6bb5cc9bb8c9be107c771a78a21baeb12202a9020b48c53beaaf4b355228e8ba2023-09-08T14:09:21+02:00Przemysław Pawełczykbuild : do not use _GNU_SOURCE gratuitously (#2035)
546be8c9c245bd129ebabb80e0a7a8dd7daeb4d30afbe6beeb8d75294552c4918fce06d7b84eebf3d792023-09-07T15:45:01+02:00Kawrakowmetal : parallel RoPE on Metal (#3024)
547d59bd97065cd7ded6c4ecab54b1d5e0b1b11e31835938ee3b0c16f1fbbf240dae21e0228864b938c2023-09-05T09:55:33+02:00KawrakowGuard against all weights in a super-block being zero (#3010)
54835938ee3b0c16f1fbbf240dae21e0228864b938c921772104ba2219bfdc2b2980d05ebc0aa0c92a42023-09-05T10:46:39+03:00Georgi Gerganovllama : update logic for number of threads when using BLAS
549e8d91589258f9204397a7ac5f9b3c857835c98f8bce1fef328941499dc0acb76cc7fd7ac90449c2f2023-09-01T11:15:57+03:00Kawrakowmetal: somewhat faster f16 x f32 matrix multiply kernel (#2951)
5508341a25957b319a03d4a811176cd5ad7f2b0fbd4849408957c687cde4ab32c147107f643fc55130b2023-08-30T08:29:32+02:00staviqmain : log file (#2748)
55144c117f41ee01c5ac8fb86bba041f08d8b87b46d43033b7bb4858da4f591715b3babdf906c9b7cbc2023-08-28T21:51:47+02:00xaedestrain : mem usage and other improvements (#2439)
55292b1bbd2ec43c82ec0530ba3c8758846c5790c75dd0dc366dab10e8df28d3924e7f313b5c695e9082023-08-28T13:23:55+02:00Johannes GäßlerCUDA: fix RoPE asserts, block sizes (#2833)
553f55538c3ccba9b926846ef862fa830cea08c433eebcee207b6058b7f695bb5c203ad87b1066a97902023-08-28T10:59:08+03:00Georgi Gerganovmetal : fix memory leak (#2762)
554c10704d01e21e3dbe4d6ca1026ebff85349dd239230d46c723edf5999752e4cb67fd94edb19ef9c72023-08-27T18:55:41+03:00Georgi Gerganovllama : fix MPI threads (close #2827)
555789c8c945a2814e1487e18e68823d9926e3b1454c1ac54b77aaba10d029084d152be786102010eb22023-08-27T09:03:27+02:00slarenci : add LoRA test to CI (#2650)
556730d9c681e339b76407659344e5a2cd50af7d7d5c7d92e6dfec3f54849f3a0ba373054d29f321ea22023-08-26T14:13:36-06:00Kerfuffleconvert.py : advanced option (#2753)
55738b16dfca6e5032e6cfb90c1653bf1ba4cf647b48f8c28e89cb9531211783da697d6e7c445e2af1d2023-08-24T12:27:25-04:00Shouzheng Liumetal : bug-fix when enable ggml-alloc (#2757)
558cf658adc832badaaa2ca119fe86070e5a830f8f6a192860cfec89a38d59a943623bf595b1fe4495b2023-08-23T23:08:04+03:00Georgi Gerganovllm : add Falcon support (#2717)
5596381d4e110bd0ec02843a60bbeb8b6fc37a9ace9dadbed99e65252d79f81101a392d0d6497b86caa2023-08-21T23:07:43+03:00Georgi Gerganovgguf : new file format with flexible meta data (beta) (#2398)
560a872a2b28eaefc8d464eaa535c94deeb501666f90919a0f73d95cfb93a1646a1d1741a0615fe2c5e2023-08-17T03:35:53-04:00Shouzheng Liuggml-alloc : fix discrepency between measure&eval (#2639)
561fc8ef549e50087762a0b4f901cd74b2defcc6ae3bf83bff6742c0f1795b4c18695a13a34ac7adf622023-08-16T16:08:28-04:00Shouzheng Liumetal : enable ggml-alloc (#2627)
5629ca4abed893685692f90413e4d43153af12342d9e59fcb2bc129881f4a269fee748fb38bce0a64de2023-08-10T13:11:36-07:00DannyDaemonicHandle `ENABLE_VIRTUAL_TERMINAL_PROCESSING` more gracefully on earlier versions of Windows.
56393356bdb7a324a8f6570f99d02af392cd4c4579660baff7c8584ec369e53469cad5f92e102b1efe42023-08-07T14:25:58+03:00Georgi Gerganovggml : mul mat tweaks (#2372)
564f6f9896ac3d2ff207e18f87dab85d126ceef523634a14b28ff7f3c98730339bacee035091b2a812a2023-08-07T10:52:57+03:00Georgi Gerganovmetal : fix out-of-bounds access + inc concurrency nodes (#2416)
5655f631c26794b6371fcf2660e8d0c53494a5575f7415e99fec27be5a2e4283f1937afd17eb33fbd662023-08-04T06:37:24-05:00Stephen NicholsFixing race condition in server and partial stream handling in frontend. (#2391)
5661a941869cbef8e9cc351a6c6987e4ae3b0f021f7b5472ea0ada081a6e1c06998ebbc9a24aa2cd4a42023-07-27T11:00:54+03:00Georgi Gerganovmetal : disable graph concurrency optimization due to bug (#2413)
5671aa18ef994a6a2b531434eb13251ef48e56d345b9a08eaf3c4010962d0126e9e5bfbe9af64b2ac902023-07-25T08:00:19-04:00Shouzheng Liumetal : concurrently dispatch commands (#2358)
56857921ca6db53e08eb90010fba99add85be28b5a13602ac4255fd4cd589821346290b464a64b955d12023-07-23T21:33:02+08:00wzycommon : n_threads == -1 uses std::thread::hardware_concurrency() (#2347)
569e76d630df17e235e6b9ef416c45996765d2e36fb1d0824b2476e7fda09751a0235c9e571b76d6f2c2023-07-23T15:09:47+03:00Georgi Gerganovllama : grouped-query attention + LLaMAv2 70B support (#2276)
570417a85a0010519224cf154eb85d383ffeafeeead294f424554c1599784ac9962462fc39ace92d8a52023-07-20T06:32:22-04:00Shouzheng Liumetal: minor q4 optimization and reduce code size (#2248)
571a7e20edf2266169ccd97a4eb949a593d628fbd641d656d6360359cfdaaf5d64ed9690047b600dbcb2023-07-07T21:23:57+03:00Georgi Gerganovci : switch threads to 1 (#2138)
57272421402834141df6cbdcf595fe46dbd11874dce3e08ae99ceb143d67f9273fda47541e9d98ff23f2023-07-07T18:36:37+03:00Georgi Gerganovggml : remove sched_yield() call in ggml_graph_compute_thread() (#2134)
5737ee76e45afae7f9a7a53e93393accfb5b36684e1acc111caf93fc6681450924df9f99679c384c59e2023-07-04T10:05:27-04:00Tobias LütkeSimple webchat for server (#1998)
5749d23589d638dc74577d5ff880e6d4248b795f12e0be54f75a6c3e9a09ea71bdfcdabf9a996a0549b2023-06-27T19:06:33+02:00Erik Scholzfix pthreads setaffinity usage on android (#2020)
575b853d456018b10820686362af41b2f2f75f1eec69225baef71407d799a6f7f563b77fd7f827914162023-06-26T13:57:59-04:00zrmggml : add NUMA support (#1556)
5766769e944c727c63612dcafbef52009d21ae00fffcbebf61ca7584e9709265395f0127ae7fc0f18822023-06-26T19:43:07+03:00Kawrakowk-quants : support for super-block size of 64 (#2001)
5772c9380dd2f77e41149340f3ecb09764d793b16db051e1b0e6a6e3aee7d989b47760980e6fda5861c2023-06-17T19:15:02+02:00Johannes GäßlerOnly one CUDA stream per device for async compute (#1898)
578794db3e7b982fee37e3995db9c3a216a57ff65e35ddf7ea1fb42bac21026de2f77e0f9c069b922342023-06-17T07:53:04-04:00Randall FitzgeraldServer Example Refactor and Improvements (#1570)
5794bfcc855abdb2c9fcc3c5a84747974521909fa416b8312e7979b852f6b6ac9d29cd51fda16c179482023-06-15T20:29:48+03:00Georgi Gerganovmetal : parallel command buffer encoding (#1860)
580e32089b2c20b1b87b22912f4a8b93fe01647d5b92347e45e7bdb09c9a7d74b2c0bc86c2b65f0c3432023-06-13T21:04:40+02:00xaedestrain : improved training-from-scratch example (#1652)
58174a6d922f12ccfe16b0c265f43be8978c6f25e98e4caa8da59c1c97dc23fa336f4d726984a20560f2023-06-12T22:39:21+03:00KawrakowMetal implementation for all k_quants (#1807)
5824f0154b0bad775ac4651bf73b5c216eb43c45cdcef3171d16241c18581d4d08374f0b9e396ade6b72023-06-10T01:59:17-06:00Kerfufflellama : support requantizing models instead of only allowing quantization from 16/32bit (#1691)
583245fc3c37da5ac5963f9f11a9f4f2ac08d96afc672ff5282bf0388c60821f504c4c8cc2b1f491aa62023-06-09T10:39:59+03:00Kawrakowmetal : faster q4_0 (#1775)
58417366df842e358768c0df7024484fffecfc7865b44f906e8537fcec965e312d621c80556d6aa9bec2023-06-06T21:33:23+02:00Johannes GäßlerMulti GPU support, CUDA refactor, CUDA scratch buffer (#1703)
585d5b111f53d14972669eb52055f9df2567663ad8b2d43387dafe9c60f15f57aa23ee0b37864b98b322023-06-07T01:00:01+08:00LostRuinsClblast fixes + enhancements to save VRAM and offload more layers (#1675)
586ecb217db4fcfa3880300ad08531a5fb6bb142d45dcb2ed48268e421baf25adc00d602dad0f4155642023-06-04T23:34:30+03:00Georgi Gerganovllama : Metal inference (#1642)
587dcb2ed48268e421baf25adc00d602dad0f415564d8bd0013e8768aaa3dc9cfc1ff01499419d5348e2023-06-04T08:12:05+02:000cc4mOpenCL: Fix duplication of layers in VRAM and RAM, add GPU mul kernel (#1653)
5881fcdcc28b119a6608774d52de905931bd5f8a43dac7876ac20124a15a44fd6317721ff1aa25388062023-05-25T23:07:29+02:00Johannes Gäßlercuda : performance optimizations (#1530)
58979b2d5b69d80be0bf29312fb9a95854876b0a8a513c351ad7292c5b5ab35db25c7a4f993e75d9cfd2023-05-14T17:55:02+02:00xaedesggml : alternative fix for race condition bug in non-inplace ggml_compute_forward_diag_mask_f32 (#1454)
59013c351ad7292c5b5ab35db25c7a4f993e75d9cfd60f8c361ca26328ef8523dfb08077fe2f10344902023-05-14T18:22:50+03:00Georgi Gerganovggml : various fixes (#1450)
59166841fdb0eaf0cc210757cc7f683d0f4eebadc21905d87b70aa189623d500a28602d7a3a755a47692023-05-13T16:48:03+03:00Georgi Gerganovggml : multi-thread mul and diag_mask ops (#1428)
592905d87b70aa189623d500a28602d7a3a755a4769f954edda935a70a14cf0cc45ecc7fe7d60cf3e4b2023-05-13T15:38:36+02:00Johannes Gäßlerggml : GPU-accelerated token generation (#1412)
593f954edda935a70a14cf0cc45ecc7fe7d60cf3e4bf048af0230ad5381bb20b9e3c1467ec6fc7debdf2023-05-13T14:56:40+02:00xaedesggml : implement backward pass for llama + small training-llama-from-scratch example (#1360)
594b9fd7eee57df101d4a3e3eabc9fd6c2cb13c9ca1b608b55a3ea8e4760c617418538465449175bdb82023-05-12T00:23:08+03:00Georgi Gerganovggml : remove bit shuffling (#1405)
59558b367c2d757c0ea12aec672382462b42204c724ea3a0ad6b6b5ca4693b94acd4cb32e2803f66fae2023-05-01T18:11:07+02:00slarencuBLAS: refactor and optimize f16 mat mul performance (#1259)
596ea3a0ad6b6b5ca4693b94acd4cb32e2803f66fae2bdc09646d8c6cb74a6f573e9081586b4b83b9d12023-05-01T08:58:51-04:00xloemllama : update stubs for systems without mmap and mlock (#1266)
597a5d30b1f53677cb50791fec41c43e9327434730376a884920aa1d2fc0dc7a7ac12dfc5ec5816377c2023-04-30T14:41:35-04:00jon-chuangcommon : better default number of threads (#934)
5987296c961d9303010a2b98379f738da2a8a55aa1b78ec543733d10a1629f984fd0302fdaa4e87fe662023-04-28T16:57:16+02:000cc4mggml : add CLBlast support (#1164)
5997a32fcb3b29f4db8aed8a85dc58eb958fb118153dd0eabc049fb1efc631cab8eb0a646808d704e182023-04-25T23:40:51+03:00Georgi Gerganovggml : add Q8_0 quantization format (rename the old one to Q8_1) (ARM NEON) (#1179)
60050cb666b8a2e35a49b08c0f6bc81138c8f6f2ac125d7abbd1f73582b7e0fdc422a936e8541c0780b2023-04-21T21:59:17+02:00slarenImprove cuBLAS performance by using a memory pool (#1094)
601018f2279f5fe3ef743bd8254b23ea8f0efae7e739411288271ab548216902a029f42a0a38ebcedb72023-04-22T02:27:06+08:00源文雨cmake : link threads publicly to ggml (#1042)
6021bfc153e2f35ddd9d64b084e8d1a5e6fa57ad1c93d59769c3bb7e72c915646ddb1e239b1face19f52023-04-21T17:18:26+02:00Kawrakowggml : a faster version for Q4_1 x Q8_0 dot products (#1083)
60338de86a7114c97ecf3644e3a60159f1ed893e1b0e0305ead3a072db9c08b35c9600c49273b38a4b52023-04-20T19:42:27+02:00Kawrakowllama : multi-threaded quantization (#1075)
604f7d05095b404b5500b4a702ea16f67fc22446e49884e7d7a2bfd7325b107442d6758983f5886ed3d2023-04-19T20:20:14+02:00KawrakowQ4_2 quantization with rmse-optimized scale and quants (#1062)
60566674012389f9537140044290f8517bc4a5e0b7477a73403ca8eaced2590559d0f9cebd2b3649d322023-04-19T00:53:24+02:00slarenMulti-threaded ggml_cpy (#1035)
60677a73403ca8eaced2590559d0f9cebd2b3649d3250a8a2af97cb92e53e7a3195aa201c3d87da54152023-04-18T23:54:57+03:00Georgi Gerganovggml : add new Q4_2 quantization (ARM only) (#1046)
607723dac55fa2ba7adc6e3fc8609781d1ad03789060f07cacb05f49704d35a39aa27cfd4b419eb6f8d2023-04-14T00:03:03-07:00comexpy : new conversion script (#545)
608a3a2a0eda8828b60436e9f69d9ac2c1060d03e7ad990e3fffc5b0f5448e90a16c79a4f2675100af02023-04-13T18:36:40+03:00Georgi Gerganovggml : add GGML_DEFAULT_N_THREADS
60995ea26f6e92d620a5437f576b80868aee7f808d682d146df9b43cf677e0dbce20b03cf864958a0cc2023-04-13T14:46:23+02:00SebastianApelbenchmark : add tool for timing q4_0 matrix multiplication (#653)
6102663d2c6784ad7b77998c6874df25648d597f74ba0caa34b162449b5c13b8d604573053300ff54a12023-04-11T06:19:54-07:00comexWindows fixes (#890)
611f963b63afa0e057cfb9eba4d88407c6a0850a0d8aaf3b23debc1fe1a06733c8c6468fb84233cc44f2023-04-08T12:24:37-07:00comexRewrite loading code to try to satisfy everyone:
612eeaa7b0492fc79baab8bb1fe195d6c87159f2bd3986b6ce9f99503c51ec5afd8a10baa32359434c62023-04-05T22:11:03+03:00Georgi Gerganovggml : multi-thread ggml_rope() (~3-4 times faster on M1) (#781)
6136f23ba5ee235cbcb1eedd63b98422dd8d4392a7878ca9838ee36660a776e97e3391b6fb5dcaacf7f2023-03-30T01:53:36-07:00Justine TunneyEnsure --mlock works properly with mmap() support
61478ca9838ee36660a776e97e3391b6fb5dcaacf7fa017390358cdb23fffb30988dc84bb190d0403ca2023-03-29T13:51:37-07:00Justine TunneyMake loading weights 10-100x faster
61541318d708ed196ff727dce14d263a64b23c7333da6956b25a1c783e5e96fe06c9c00438f846ef0472023-03-29T18:10:07+02:00Maël Kerbirioullama : use the same threshold for OpenBLAS and ggml thread limiting (#577)
616c1f885067c61191a07a1aedf684168dda62f3f71e0670260fb50a882b37074112b1881fb0820cf772023-03-28T15:56:03ZStephan Walterggml : introduce structs for the q4 data blocks (#356)
617ecbe466a364876927994e2f1ec14f4d82301d201502a400192013d3e95ed87b777e8fa3bec45713c2023-03-25T19:47:21+02:00Georgi GerganovRetire the ggml_mul_mat() branch for transposed src0 (#500)
6184640eff23d341a0273587800e17ff4a378132d60ab77d7631211b299cb734bea6ad1f743241541502023-03-25T17:03:10+02:00Georgi GerganovDon't interefe with BLAS for large prompts by running only 1 thread
619563cdc391dde140f1084d1012234e8e6f57f881f8d4a855c241ecb0f3ddc03447fe56002ebf27a372023-03-24T08:19:05-07:00comexSupport calling mlock() on loaded model data on Linux and macOS (#453)
620483bab2e3d4a868fe679d8bb32827d2a4df214dc404e1da38ec8025707031a8027da14dc1590f9522023-03-23T23:22:01+02:00Georgi GerganovAvoid the transposed X branch in the Z = X * Y matrix multiplication (#439)
621ae44e23ee36c02da0e37ab508a4b473ace724f8e928480ef5b7b03d7a07e98286aebe3d8b24457d92023-03-22T07:47:15+02:00Georgi GerganovWhen seed <= 0 - use the clock to generate one
6222e664f1ff413995506c9a54f3a8d5b8c64e37a918cf9f34eddc124d4ab28f4d2fe8e99d574510bde2023-03-21T07:35:42-07:00Casey PrimozicAdd initial AVX512 support for dot product on Linux (#320)
6237392f1cd2cef4dfed41f4db7c4160ab86c0dfcd9ad5fd5b60cfdfbfb22b0f2bc9e9f6c9692768f8d2023-03-19T12:38:44-06:00Suaj CarrotImproved quantize script (#222)
6244f546091102a418ffdc6230f872ac56e5cedb835e81b9c81c101f64531ef0fa1ee6b77d5626356522023-03-17T21:46:46+02:00Georgi GerganovDefault to 4 threads (#243)
625367946c668757532deed929e1d78673c6ac6bcb86b0df5ccf360fe5c015f6607f0375bfc6849005e2023-03-17T17:47:35ZStephan WalterDon't tell users to use a bad number of threads (#243)
Reference in New Issue View Git Blame Copy Permalink