cogito/calculet-npu-research-archive
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
d2677d2205d50aab55fdbd1cbe559a6584989433
calculet-npu-research-archive/history/llama.cpp/topics/07-prefill-decode-onetoken.tsv
T

96 KiB
Raw Blame History

1c931ef3163940227c9b6291e04216245cce21429a43741244a646d3f8fa3ff01bb9b7d059223d0a52026-05-18T17:34:37+08:00wangbomengcalrt: fix prefill_by_ids.fix create_buf
2eeeef3f6d6a5aad9296323ca15d9d929745b8932431ad6f1c787f1b47e9e7be14b2e1a7d8565485c2026-04-28T16:16:19+08:00wangbomengadd unknown time info: prefill and decode
399962021f2dd1994dbbf90d5712f0ca2633f20f89af6682ef101c6d006c743c703cf150ac25f466f2026-04-17T14:48:33+08:00wangbomengorigin/dev-otcalrt: add onetoken slice and untile_prefill
4a206460e610aef995e36e50b33e8bd41fe477541d8b2aaa00049978cebac15041f960d75b552f97b2026-04-14T10:23:43+08:00wangbomengcalrt: support mixture of prefill_by_ids and prefill_by_embeds
5b8153b6b8f244b223c9f4c2940ae1f212634519e82842cfc3fa3d5c004c4540aeff8a81f38509bee2026-04-09T15:07:02+08:00wangbomengserver: fix limit tokens to max_seq_len
682842cfc3fa3d5c004c4540aeff8a81f38509bee622a22991089c6debd5f4b57738f3df3a0bda8b52026-04-09T15:06:26+08:00wangbomengserver: fix limit tokens to max_seq_len
71e8a16cd0e3ad3c9a0f8746852a0a38c6e52322e893e52bda0fee99bbda1521ea733a760bc4201f12026-04-01T10:14:22+08:00wangbomengcalrt: fix slice and add infer/copy time
8e4eaab700333490e80ebefc8d023f6c1adfcc312aeacc23a883400db0ebffb50e23355ada054da662026-03-25T09:03:55+08:00wangbomengcalrt: prefil to prefill
9aeacc23a883400db0ebffb50e23355ada054da66cbaa7492663630132adeddccd9fec5e44ffa33362026-03-25T08:49:22+08:00wangbomengcalrt: add slice only on decode
103c08ebf0e442cd730ddffd959ec676f9caf31c82613bc2b9a9d7af92efbe42d876ffcdb244f7a00e2026-03-24T02:43:03+08:00wangbomengcalrt: comment LOG_ERROR in untile_decode_cpy
1116cbf81a731f5839a2f6b0eb9d8539826353748b8b4e17d990c23025148c4abadefe1010a0dd07342026-03-23T09:38:21+08:00wangbomengcalrt: fix model_name of untile_cpy
12b2c45d8ddefb7ec09435231c8ec5b736fabf9c91c6ea3f0d2468e3b8eab5de80bacee7fe4c9dd3d72026-03-09T15:03:26+08:00Bomeng Wangclart: past_kv_len = n_tokens
13583c387efaf7bc030574e554088de79d09a27fbd0edc14a60a2d5596f71d728c655e3523da924d892026-02-27T02:24:56+08:00wangbomengcaserver: lim generated tokens to n_ctx_per_seq
140edc14a60a2d5596f71d728c655e3523da924d89aa0a17d719326a63e0d6fea60aa0ced44e7abc332026-02-27T01:57:39+08:00wangbomengcaserver: limit generated tokens to n_ctx_per_seq
1584f2c0b5f334a7ab77347c6779b6027a951d0ff00bd95719fed2f0247c8d0199dd897d24157dfae62026-02-09T01:46:36+08:00wangbomengadd bf16_to_fp32 in untile_decode/prefill_cpy
169328f21378275f2354a19c3af907332b216ab49249fea70a20f550bfdb59d0ffe041b54919c224472026-01-28T15:32:36+08:00Yunzhe Jiaadd prefill-only mode
175d2387931528fbe04aa8d66de935147efb65ca4de6285a748657add8d974b78ca920c5b41753ee122025-12-08T16:40:34+08:00Yunzhe Jiaadd timestamp for one decode process
18e492f5dc8cec17529f53ef2bbdf7b502107f81482e05afd22b3d77c7013b11eee88e88b17188f21a2025-11-04T14:09:59+08:00Yunzhe Jiaredo untile logic
19070ff4d5356083d60b807bb34d36b31c3653a29ebf7b0c9725ed0c406c5debaea022ec72584306342025-11-03T11:11:18+01:00Xuan-Son Nguyenmtmd: add --image-min/max-tokens (#16921)
20cd5e3b57541ecc52421130742f4d89acbcf77cd487c9efc3b297b8a498716b1db3d061842e6fc85b2025-11-02T18:14:04+02:00Georgi Gerganovserver : support unified cache across slots (#16736)
21cf659bbb8ef9eb048e5153a27cf787fd83c05560d8b860a219c2415faac8cc0e50b48b4aa11e3b642025-11-01T15:51:36+01:00Xuan-Son Nguyenmtmd: refactor preprocessing + support max/min pixels (#16878)
22b52edd25586fabb70f0c21b274473b307cf14499517b7170e1a4d733583c4b07c5b7a49acc05911c2025-10-30T18:42:57+02:00Georgi Gerganovserver : remove n_past (#16818)
235a91109a5d7dab5d7adc40bedb397ede99a705b1f8f071faddf32ea09f4234edb6e809b380a9ee262025-10-24T12:02:02+02:00Daniel Beveniusmodel-conversion : add trust_remote_code for orig model run [no ci] (#16751)
2484bf3c677857279037adf67cdcfd89eaa4ca9281c9c1972e2c2cc6a771fcc145bfa138700179f9612025-10-20T21:38:20+02:00Sigbjørn Skjæretmodel : add BailingMoeV2 support (#16063)
2517304cbcc1dd24de7741cbe57925d58e90a98ac13e3cb19f6449f9168a128eaeae01f8f41b049acc2025-10-15T16:53:12+03:00Georgi Gerganovserver : fix img token logs (#16595)
26bc07349a7f87ba6eb31ed4b0ea9d9a7352185213e60f241eacec42d3bd7c9edd37d236ebf35132a82025-10-14T08:48:50+03:00Georgi Gerganovserver : dynamic token limit for prompt cache (#16560)
27477a66b03501cf3bd067f8968b77ca4d053ff1bde60f01d941bc5b7fae62dd57fee4cec76ec0ea6e2025-10-11T11:33:41+03:00amirai21convert : correctly handle LLaMA tokenizer for Jamba (#16470)
2881086cd6a3ca1252f0dc0f938171648399179c5368ee98ae181a5c83a5cc6261daeee69a1f588c152025-10-10T17:17:31+03:00Georgi Gerganovvocab : mark EOT token for Granite models (#16499)
291faa13a1187051af66b0fd9f0d6effe4c77f0b3e1deee0f8d494981c32597dca8b5f8696d399b0f22025-10-09T22:54:57+02:00Pascalwebui: updated the chat service to only include max_tokens in the req… (#16489)
30c08002a1988348403a5fd59b1fa3de3a10a6f92f3a002afafa8e06555f11aed88b02d055d8a166f32025-10-06T10:59:40-06:00Gabe Goodhartchat : Granite Docling stopping (#16438)
31ca71fb9b368e3db96e028f80c4c9df6b6b370edd35266573b968e1c947b367782fb4b3eddbb4f3c02025-10-05T06:57:47-06:00Gabe Goodhartmodel : Granite docling + Idefics3 preprocessing (SmolVLM) (#16206)
3277233277c912d495d1069543ca540c21a2f9e5b4e308efda8e54e3f07578937a45a5d83624eb79702025-10-03T11:30:39+02:00Aleksander GrygierCapture model name only after first token (streaming) or completed request (#16405)
33e74c92e84236b2bab3f3c77bee4ead94928be360b2ba81dbe07b6dbea9c96b13346c66973dede32c2025-09-30T16:24:36-04:00Bartowskimodel : support GLM 4.6 (make a few NextN/MTP tensors not required) (#16359)
342f61c0f5bf8a620ca4c3872408803ab38cfb96133ffd0fae473c954bb3e67526b31262048fb508d42025-09-29T12:33:12+05:30Vinkalllama-cli: prevent spurious assistant token (#16202)
35624207e676ab5eb3ce7af631902bb45fb73a8359807e8c6d310952f2f5656afc63dab9d7083dcb5c2025-09-27T02:03:33+08:00Aaron Teodevops: add s390x & ppc64le CI (#15925)
36077c94d0caf87fbd3cf3288dbb5c0fd9670294cfaa3ee0eb0b80efca126cedf9bcb4fb5864b46ce32025-09-25T22:35:05+08:00Aman GuptaCUDA: add a fused top-K MoE kernel (#16130)
37aa3ee0eb0b80efca126cedf9bcb4fb5864b46ce3d0991da39d3c39b3980c24cdfccb9a4c95a468702025-09-25T12:02:36+02:00Daniel Beveniusmodel-conversion : add embedding prompt file support (#15871)
38351f3da39c85f59d581fc184f09283da7f099a3b3ecb2f671a2f49d56357f99d135a94e8417591782025-09-23T01:57:46+08:00Haiyue Wangclang-tidy : disable warning about performance enum size (#16127)
39745cbcf2fe1eb88f8db615ac622f0b944d924ad61cbd80f8cf80a817715b1ccc5680fe2a3c5172c82025-09-17T15:30:55+08:00Jie Fu (傅杰)llama-quant : fix the verification of attention layers for encoder-decoder models (#16023)
401cbd80f8cf80a817715b1ccc5680fe2a3c5172c885286f354813056f6c835046c0acfa3bf6ba94322025-09-17T15:29:00+08:00Jie Fu (傅杰)examples : support encoder-decoder models in the simple example (#16002)
41c0389dba43d50695f9d3f57dd1f1a14cbefc100c00681dfc16ba4cebb9c7fbd2cf2656e06a0692a42025-09-11T15:59:37+08:00hipuddingCANN: Disable acl_graph for prefill stage (#15933)
424f658855fa8f2e42b7ed9a5b298fa39a2e39b0966ab397e12ba8e9f776341cdae68f7ffb2f8d2cde2025-09-11T02:51:51+08:00Jie Fu (傅杰)llama : support T5 models with unequal number of encoder-decoder layers (#15909)
4370cd37dbbebdb7a2f84f08207f18eabb0b291a55acc1b008cfd95e63d5f99a370dbffb98e5a99d2c2025-09-09T06:06:52+02:00Daniel Beveniusrequirements : update transformers/torch for Embedding Gemma (#15828)
4488021565f08e0b7c4e07ac089a15ec16fae9166c56920f56651908d5cce7a310dabf54ac4f6fbb7f2025-09-08T10:59:48-04:00Jessechat : Deepseek V3.1 reasoning and tool calling support (OpenAI Style) (#15533)
4556920f56651908d5cce7a310dabf54ac4f6fbb7fb0d52998b962bd2681c34bf52af993af79f178b82025-09-08T21:50:05+07:00Xuan-Son Nguyenserver : bring back timings_per_token (#15879)
46fd621880f3fd908424e675a41715a2dc760247a24281c7b315f8a904549fe527039b976e08098d1a2025-09-05T17:32:39-06:00Gabe GoodhartaLoRA Support (#15327)
475fac79cbc77b6d12c9feb5f34fc63586b35fd561408ff524b40baf4f51a81d42a9828200dd4fcb6b2025-09-05T14:31:24-06:00Gabe GoodhartThinking model disabled assistant prefill (#15404)
483dc7397a2799bdc07bccf637ab7ae5a1e786d1a4e92d53b29e393fc4c0f9f1f7c3fe651be8d36faa2025-09-01T08:57:00+08:00hipuddingCANN: fix RoPE cache issue on multi-device (#15629)
49e8d99dd0b67f2ecc1e45fca8074a3a18c3e036d2a8bca68f727844e7dcf24a956003b3c2039ea5632025-08-28T18:39:31-06:00Gabe Goodhartnvidia nemotron nano v2 (nemotronh) (#15507)
5084ab83cc0b4b7e769451ee48e4c7d1acef91ef2555042b3692cb1467c9ee15c62c4a9fbf180f89e32025-08-28T15:49:50+02:00Sigbjørn Skjæretmodel : jina-embeddings-v3 support (#13693)
511e7489745a74996fc36e8fd05b73aa16bc184e0c1cf123a343ab7ca5586aacb9e0a1d2de7fe33be42025-08-27T17:21:41+08:00Chenguang LiCANN: refactor mask handling and improve performance in FA (#15561)
52a6a58d64785cb458ed9de52f391aa38142d38d640373486dbc0dccbdcb3b5fdd65759d88cec061962025-08-26T21:05:25+05:30shalinib-ibmllamafile: PowerPC Sgemm Optimization (#15558)
5362cef26ac5b6b7acb635d3dc963813b43952dc2b8f5afa94c4f929da71f560db7c9f38ef6a783d952025-08-26T16:12:29+02:00Daniel Beveniusmodel-conversion : add qat-q4 quantization targets (#15588)
5479a546220c719e6a70627b243a478ab8d84dc9e185cc1ae998e4898d9fa992cb9b8620338cee97bf2025-08-26T12:54:19+02:00Xuan-Son Nguyenmtmd : support Kimi VL model (#15458)
550115cfb7c1dec0753aecb34f3af4338bf9de8eeb3a617cb18121a85eed17ac5585788810c5d9c1e0 0d5a470223fc90b6b6807921d68011ff06ae7f9e2025-08-26T15:19:56+08:00Yunzhe JiaMerge branch 'master' into dev
566b64f74b55628e4193f4fb00313f07dbd85565280d5a470223fc90b6b6807921d68011ff06ae7f9e2025-08-25T13:56:43+03:00Georgi Gerganovbatched-bench : fix unified KV cache handling + pp timing (#15562)
574afb0a746f22abaa545b3ebdb76a400d7da3a713e288693669cf9d0a71e2f2b8bd57305f063402572025-08-22T08:10:14Z65aserver : Support multimodal completion and embeddings prompts in JSON format (#15108)
58e288693669cf9d0a71e2f2b8bd57305f06340257a0f98dd604d34826eb5ea2560d1e23fe726921df2025-08-22T09:29:08+02:00Tarek Dakhranreadme : model : mtdm : lfm2 improvements (#15476)
59657b8a77bd01854f99d37a47318fa24f2e7e298fec5ab1a36c11dd3efcf4ec8d1ac89a13a8117bc32025-08-20T14:26:01+02:00Daniel Beveniuschat: handle gpt-oss return/end token inconsistency (#15421)
60f0d3c7405c323784a60f14ddddfbac3f7404d417f08c4c0d8d0cb6caaf8b7ad316039232b9fa059c2025-08-19T08:45:12+03:00Georgi Gerganovbatched-bench : use rand tokens (#15398)
61f08c4c0d8d0cb6caaf8b7ad316039232b9fa059c6d7f1117e3e3285d0c5c11b5ebb0439e279200822025-08-18T22:53:52+02:00Xuan-Son Nguyenmtmd : clean up clip_n_output_tokens (#15391)
625e6229a8409ac786e62cb133d09f1679a9aec13ee2c1bfff5305c661ac53e9d57cb732ff626a22422025-08-15T19:50:52+02:00Daniel Beveniuscommon : fix double bos, use common_chat_templates for add_bos and add_eos (#15326)
63b204a5a234f4cf0b3f449476da639a5510c6d157646944cfa8961afd914dd6637739b3cda9a72e112025-08-14T09:23:11-05:00Aldehir Rojasgpt-oss: implement harmony parsing (#15181)
643a617cb18121a85eed17ac5585788810c5d9c1e0c42293848dd9f03eba9a6c1b85a600b398f065412025-08-13T10:58:59+08:00Yunzhe Jiaadd calrt_decode
651ebbaddff2a44b0599df659175d3274bd5bbeb81a3a7874272e5a060079658eb5cca4617b7f990622025-08-11T10:21:24+02:00Daniel Beveniusperplexity : update comments/error msg to use decode [no ci] (#15227)
6699acbc9921b119aa7ed929eb5780a66a8f06e6d97ad67ba9fe2b909e271dd31b99c5fce3aba358992025-08-08T00:20:40+08:00RunningLeonllama : Support intern-s1 (#14875)
671cbf442f50a154aac0bf35d7b8b3436a2cdfd6f5ba67d6c03fed3193987a4ef13050e6e2a4451df42025-07-24T15:50:19+08:00Yunzhe Jia1.sync with calrt prefill,decode,kv_update op 2.add hw pattern untile function
68fd1234cb468935ea087d6929b2487926c3afff4bf324a3b715d5c1081c110ce459f8a8486fb1ee892025-08-05T22:10:36+03:00Georgi Gerganovllama : add gpt-oss (#15091)
69ef0144c087b33e5b8da42d529ac71aaf05cb49df2721257e3e2c4c944ac8a08221113ee7cb503f1b2025-08-05T04:29:25+10:00Sammodel: support GLM 4.5 family of models (#14939)
7097366dc6abdd0bdc74260bd3c42bd06f0feb742883bc2f288c0e08e676d9beca9c4669197e9205932025-08-03T12:38:18-07:00Csaba Kecskemetivocab : JetBrains Mellum pre-tokenizer (#15045)
7183bc2f288c0e08e676d9beca9c4669197e9205936c7a441161080551ce8a52ba32563b62950671922025-08-03T09:56:25-05:00Gabriel Larsonmodel : add text-only support for Kimi-VL (and find special tokens in text_config) (#15051)
722bf3fbf0b54f97aef2b388b76d222789e1c170f1711d5e6fe66eb6cd7a10d71cec4567321848be082025-08-02T14:39:01+02:00Sigbjørn Skjæretci : check that pre-tokenizer hashes are up-to-date (#15032)
73711d5e6fe66eb6cd7a10d71cec4567321848be08f738989dcb9ccbe468c945553eafbeef7b8696752025-08-02T05:51:02-05:00Douglas Hanleyconvert : fix Qwen3-Embedding pre-tokenizer hash (#15030)
74f906275537d14c8fc7c6976d944233771fd6672ca9f7541ec25c4c8547daf5ff48700ad2836e2b7d2025-08-02T10:12:41+02:00Johannes Gäßlerserver: enable token array inputs for OAI API (#15001)
750f5ccd6fd1a1f709010312933db0316867cc30b61c872f71fb8a25589efa3ee9b6bf8b517cb8caa42025-08-01T21:31:12+08:00stevenkuangmodel : add hunyuan dense (#14878)
7694933c8c2eeaa9a7983e3f6c08af76bd86724094c1dacaa99b4ead6edbac928cd2da59436573f6b02025-07-31T05:25:23-07:00g2mtserver : implement universal assisted decoding (#12635)
778a4a85627702b569d7d2810f2de06a4321656e9d11490b36723d511d75fb601995c79b5c363ba3a22025-07-31T19:49:09+08:00Aman GuptaAdd LLaDA 8b Diffusion model (#14771)
7892b8810ec7aa6d778bc287cc918443cf67b962e200131d6eaf4df029e1ec84de868c2c59575030072025-07-30T15:46:13+02:00Johannes GäßlerCUDA: skip masked KV slices for all FA kernels (#14924)
7900fa15fedc79263fa0285e6a3bbb0cfb3e3878a2946b1f685909c8c9c044f145bce819c02f327eaa2025-07-28T15:01:48+02:00Xuan-Son Nguyenmtmd : add support for Voxtral (#14862)
80ca0ef2dddb022cb1337d775cd05cd27d7808aff489d1029559bd2968f76db854f9f113d73e34527c2025-07-27T12:10:51+02:00Daniel Beveniusllama : clarify comment about pp and tg graphs [no ci] (#14895)
81793c0d7f46384001738c337d7afa46b45ae32745ce111d39d666f4a3b6a561ad020f6feb8cc677902025-07-25T10:47:39-06:00Gabe Goodhartmetal: SSM_SCAN performance (#14743)
82b4efd77f8ab407836ca73a5176f041650c5b24112be60cbc2707359241c2784f9d2e30d8fc7cdabb2025-07-21T09:24:51+02:00IsaacDynamoserver : add parse_special option to /tokenize endpoint (#14783)
8309651d09ffc1e941bd1be23163abf5495c416547349ea79fcebc75b0c55bf61594a47736966d4f952025-07-18T06:25:54+02:00Nexes the Eldergraph : Pass the graph placeholder message in debug mode (#14748)
84cb887f1bc1001c92f7b4a595b9014f3a454a07abd6fb3f6b49b27ef1c0f4cf5128e041f7e7dc03af2025-07-17T23:15:32+02:00Piotr Wilkin (ilintar)model: add Ernie 4.5 MoE support (#14658)
8519e5943d9e976b59ccd5a0a87ae3d2ff3da44390496957e1cbcb522abc63aa18521036e40efce9852025-07-16T23:17:43+02:00Sigbjørn Skjæretconvert : make hf token optional (#14717)
864b91d6f71f14040979bcdb7b6729b3bca93ec1c1cf91f217f1c8b98b6db8e4ba6b480f017f81d2062025-07-16T08:52:04+02:00Sigbjørn Skjæretconvert : only check for tokenizer folder if we need it (#14704)
8768e37a61a7b24863f67541db65b4f6195962a268cbc68be51d88b1d5531643b926a4b359c3cff1312025-07-16T01:11:42+09:00Shunta Saitomodel : add PLaMo-2 support (#14560)
88576c82eda210ca0111c04f5256bf77897a4d4cc40aedae00e6fb48680324a5ac5da9cba0e35de6b52025-07-11T16:36:04+09:00Dowonvocab : add midm-2.0 model pre-tokenizer (#14626)
890aedae00e6fb48680324a5ac5da9cba0e35de6b56bdda13981d6c8189b7dc4f9fb8ecb91c21529f82025-07-10T18:20:13-06:00Gabe Goodhartmodel : Granite Four (#13550)
904bb625b713fd9b294b4f7af87eaa752b710c7cc111ee0fea2a24da1d3206eeba8fc52b759d9dfb242025-07-10T13:41:00-04:00Ryan MangenoSmoldocling support (#14597)
918f22dc0a53338c629c1ef8fa878d8e39bfe627c953903ae6fa5f1caf187889c839cdd1ad25da40182025-07-08T10:24:06+02:00Xuan-Son Nguyenmodel : add hunyuan moe (#14425)
92ddef99522d1ba74193b7394e803fab8db5c78bae668168814601d2aef6161ce49ab186bc74177ae72025-07-05T09:17:14+02:00Sigbjørn Skjæretserver : fix assistant prefilling when content is an array (#14360)
935d46babdc2d4675d96ebcf23cac098a02f0d30cce17991c466ac835b2c71bd813c1ca7ff8dd97b942025-07-02T13:10:24-04:00compiladellama : initial Mamba-2 support (#9126)
94caf5681fcb47dfe9bafee94ef9aa8f669ac986c783790b0e7e09ab17238b16452a33053a71dbdfad2025-06-29T20:02:53+02:00matteoserver : support jinja extra template kwargs (Qwen3 enable_thinking feature), from command line and from client (#13196)
958f52c6f8abb2ed9ad967cc0bfb90acb073011193d2f325130c4ed77a3dcff2c23a926587fb08e2cc2025-06-26T15:10:45+08:00Yunzhe JiaSquashed commit of the following:
9666aba7aca9a245d71f1ddf02c7a97223529752a8f1f5e82df6222dcbaca6396c0de44df259a1694f2025-06-23T01:28:06+08:00Ruikai Pengrun : avoid double tokenization (#14327)
97f1f5e82df6222dcbaca6396c0de44df259a1694faf3373f1adfca56119f3e4de0e6a0a8df8edf3d92025-06-22T20:10:07+03:00Georgi Gerganovexamples : fix is_first logic for tokenization (#14329)
98aa0ef5c578eef4c2adc7be1282f21bab5f3e8d26bb16041caef45cd4348cd6f84906b5dfec7a1f6a2025-06-21T18:12:05+02:00Sigbjørn Skjæretgguf-py : fix Qwen3-Embedding eos token (#14314)
99dd6e6d0b6a4bbe3ebfc931d1eb14db2f2b1d70af8308f98c7fb778e54bf75538f5234d8bd20915e92025-06-20T22:13:06+08:00Ruikai Pengvocab : prevent tokenizer overflow (#14301)
10088fc854b4bd2e3caf10e705e6afcbbca136f0a3ce28c1b93fd7d3f8faf9551d962e8a65fe2122e382025-06-20T14:04:09+02:00Sigbjørn Skjæretllama : improve sep token handling (#14272)
101812939a9e90f99d1bd5bb1bc6b99d12600671d504c9fdfbe1580a66fd7d77c77418ce2c606a29fdd2025-06-20T10:50:27+03:00Georgi Gerganovmodel : more uniform output id handling (#14275)
102d2f325130c4ed77a3dcff2c23a926587fb08e2cc6391ee7ffd97e723dfbdcc32f001e1b95a63cba62025-06-17T11:53:33+08:00Yunzhe Jiacalrt: add input_token in graph; input_token will be copy to calrt_in_buffer
103f6e1a7aa8787b5c00acba6370cb70a0beff48b1ec3ee46fab49a765d2e32e171e9ed7a5fa121dd9c2025-06-12T11:50:01+03:00Georgi Gerganovcontext : simplify output counting logic during decode (#14142)
104a20b2b05bce6622c585459ebf46f142f113d021c2e89f76b7af2c0b827be785e445f2e2b3e52e1ca2025-06-12T02:56:04-04:00compiladecontext : round n_tokens to next multiple of n_seqs when reserving (#14140)
105cc66a7f78f95dcb5208420f4dd1abc3fc6aec0ccbd248d4dc7265437e1918dc53ae3f49a0c592e5f2025-06-11T17:16:32+02:00Sigbjørn Skjærettests : add test-tokenizers-repo (#14017)
106dad5c44398b78467943ed0a603ea427fe9f6fc6255f6b9fa6563f6ae49113f9abdc980c12348cc1c2025-06-10T18:20:14-04:00compiladekv-cache : avoid modifying recurrent cells when setting inputs (#13834)
1079f47fa5792bae5312615439e68dbf1826913f7ac9e31bec4fd53634c9e5b04650488a09a055f5dab2025-06-05T09:29:18+02:00Sigbjørn Skjæretvocab : warn about missing mask token (#14022)
1089e31bec4fd53634c9e5b04650488a09a055f5dab5a8ae3053ced350ed300ba91600519fcad1c6ba72025-06-05T09:06:29+03:00Georgi Gerganovcontext : fix pos_min initialization upon error decode (#14008)
1095e1c3aed4074480f63e914d6c44c93536ed1452ac496fe0b1da28618dd17bda8b0a6bf50045540802025-06-01T18:07:21+02:00Sigbjørn Skjæretconvert : fix nomic-bert-moe mask token (#13757)
11012d0188c0dc6146ffde6d277a93f232ccbe699f8eb3949938e82a128855bc0676220bb2ce6e4228d2025-05-31T10:24:04+03:00Georgi Gerganovkv-cache : refactor + add llama_memory_state_i (#13746)
111db38704f0133be7832123495fa8fc2601ea999d407e4351ce663a7802b31f92fd7bc0e555c2044b62025-05-30T14:50:43+02:00Sigbjørn Skjæretconvert : fix rwkv bos/eos token (#13844)
11207e4351ce663a7802b31f92fd7bc0e555c2044b6291f2b6913c7ef8350dbf0e77da38f7af131a08e2025-05-30T12:24:37+02:00Xuan-Son Nguyenconvert : allow partial update to the chkhsh pre-tokenizer list (#13847)
113bc583e3c63c04a11d287c108ea9e6a515ead042372b090da2c50e540143fd312a2f9aa5f151e61362025-05-27T14:06:10+02:00Xuan-Son Nguyenmtmd : support Qwen 2.5 Omni (input audio+vision, no audio output) (#13784)
114f9cd68398baf2ba8af4725ca9ed00bef205e67064f81b33e324b1669b282eb81104e4a1131be7dce2025-05-27T12:07:52+03:00Georgi Gerganovsampling : make sure samplers return at least 1 token (#13822)
11579c137f77677b3c8ee3c60a7da033721b938399a22229314fc46b2f741bb21b12cde71f6c6a60b522025-05-26T14:03:54+03:00Georgi Gerganovexamples : allow extracting embeddings from decoder contexts (#13797)
116aa50ba462f63759e1731227f20f5009cfc2a0f16de2ef53a4b2c0d703749a309d19fe68fd8f1b9ac2025-05-25T16:22:29+02:00Sigbjørn Skjærettests : improve UGM tokenizer test coverage (#13773)
117c3a2624339187e89c4f65fd72a5fe7103968b5adffd0eae60b7642e942c8245b89642527e36099e62025-05-24T12:29:09+02:00Sigbjørn Skjæretvocab : fix ugm tokenizer precision (#13743)
1188a1d206f1d2b4e45918b589f3165b4be232f7ba8797990c4bca0dca5be295c63e3fb2800dc0a69c22025-05-22T22:21:07+03:00Georgi Gerganovtts : fix n_ubatch + make WavTokenizer cache-less (#13713)
119b7a17463ec190aeee7b9077c606c910fb4688b84be0239693c1530a18496086331fc18d8a9adbad12025-05-21T00:55:30+08:00l3utterflymtmd-helper : bug fix to token batching in mtmd (#13650)
120e298d2fbd082a52c0f6ed02729f94e9bf630cf17f0adb80bf7c2c0d80abb04f4533b5513622d99642025-05-20T08:05:46+03:00Georgi Gerganovkv-cache : add SWA support (#13194)
12192ecdcc06a4c405a415bcaa0cb772bc560aa23b1f71f40a2847d4c9f57b86cd206e0a27b2bfb6d1c2025-05-19T13:04:14+02:00Xuan-Son Nguyenmtmd : add vision support for llama 4 (#13282)
1226a2bc8bfb7cd502e5ebc72e36c97a6f848c21c2ce3a7cf6c5bf6a0a24217f88607b06e4405a2b5d92025-05-17T17:59:48-04:00Isaac McFadyenserver : added --no-prefill-assistant flag (#13608)
123b2838049ccf50859cea4c390e81405c2fb01e820aa48e373f256df9608395ee6881e7010840d62022025-05-15T05:57:02+03:00Georgi Gerganovbench : handle decode errors (#13548)
124aa48e373f256df9608395ee6881e7010840d6202e3a9421b78da5c810d812e6348a405f5acc39f342025-05-15T02:39:51+01:00Olivier Chafik`server`: inject date_string in llama 3.x template + fix date for firefunction v2 (#12802)
125360a9c98e13d35f322b4c5b1309aab0cc90ed82b09d13d94fb169093095416ac6323551c37b753392025-05-14T13:35:07+02:00Xuan-Son Nguyenserver : fix cache_tokens bug with no cache_prompt (#13533)
12643dfd741a5da77982e0a94d1e522a2481dfb914db064a51a4e7246fa43a3307f58e59f65e6be170a2025-05-10T17:19:52+02:00Sigbjørn Skjæretllguidance : set tokenizer slices to default (#13424)
12733eff4024084d1f0c8441b79f7208a52fad7985817512a94d636c4b6c1332370acb3e5af3ca709182025-05-09T19:29:37+02:00Xuan-Son Nguyen server : vision support via libmtmd (#12898)
128efb8b47eda78ea8ae570d4fece3953aae499289e0527771dd80bd18479dfaaa0a98be297fc3592bf2025-05-09T05:53:58-04:00Bartowskiimatrix : Add --parse-special for enabling parsing of special tokens in imatrix calculation (#13389)
129f05a6d71a0f3dbf0730b56a1abbad41c0f42e63dee01d71e585f4a17ae83ec55d77acfdcf0bfa7982025-05-08T14:25:39-04:00Matt Claytonmtmd : Expose helper_decode_image_chunk (#13366)
1300ccc1213549e39ef4c1affb1bf5f49651ef4ce486562e5a4d6c58326dcd79002ea396d4141f1b18e2025-05-08T22:03:53+09:00welixmtmd : fix the calculation of n_tokens for smolvlm (#13381)
1316562e5a4d6c58326dcd79002ea396d4141f1b18e51fb96b1ff2e1cc98b2492a012b7d93531a6a9a82025-05-08T14:28:33+03:00Georgi Gerganovcontext : allow cache-less context for embeddings (#13108)
13227aa2595321c4d9cc4086a8e67bdea204b8309b09fdfcdaeddd1ef57c6d041b89cd8fb7048a0f0282025-05-04T23:43:42+02:00Xuan-Son Nguyenmtmd : add C public API (#13184)
13336667c8edcded08063ed51c7d57e9e086bbfc9033bf785f3efa89ed28294fbf73054558a2b034bfb2025-05-03T20:07:54+02:00Xuan-Son Nguyenclip : revert the change of BOI/EOI token for GLM-edge (⚠️ breaking change) (#13259)
1343f3769ba76061a511f02f2a48da2ad2d93fce5112f567611c0234bbca0a4009762acb47b568660952025-05-02T22:23:12+05:30shalinib-ibmggml : Enable MMA for BF16 in llamafile_sgemm (#13148)
1352f567611c0234bbca0a4009762acb47b568660957d2123484e6ba8fcd90fff8c01661b9dbbefdc9d2025-05-02T11:42:30-04:00Jared Van Bortelllama-model : support Qwen2 embedding models and pooling_mode_lasttoken (#13245)
136e2e1ddb93a01ce282e304431b37e60b3cddb6114d9d398f84f96d16c308d4976f5e90222ecc2a4922025-04-29T20:33:10+02:00matteoserver : Prefilling assistant message in openai compatible API (#13174)
13700e3e5a194e88e604e7c91391b9e90332888fd72e98b3692be4cd8fbbd9a56fbacc2f2bf0bf26a682025-04-29T11:47:04+02:00Xuan-Son Nguyenmtmd : add qwen2vl and qwen2.5vl (#13141)
1385f5e39e1ba5dbea814e41f2a15e035d749a520bceaea3253244dc4bbe07f6cd81325847ccc6cf93e2025-04-28T15:52:15-04:00ATmodel : Nomic Embed Text V2 with Mixture-of-Experts (MoE) architecture (#12466)
1394e87962e34a4b257ec374c4baf6b1568554b81a9fb0471d1753824e75474c24f82fbdd54c94dceda2025-04-28T16:12:56+02:00Xuan-Son Nguyenmtmd : fix glm-edge redundant token count (#13139)
140d2b2031e5f11b826dcc718138642f147a20096655fa9e63be82225fb3249c76f39ddda3e5bdec0a32025-04-28T14:20:56+02:00Xuan-Son Nguyenllama : (mrope) allow using normal 1D position for text token (#13138)
141ced44be34290fab450f8344efa047d8a08e723b4e291450b7602d7a36239e4ceeece37625f8383732025-04-27T21:57:32+02:00matteollama-chat : fix wrong template in GLM4-0414 (#13140)
142553a5c3a9fdf771be2101bc3529937963f81745713be08daf992c89d5169518229b3740041c0f4192025-04-25T10:08:08+03:00Radoslav Gerganovrpc : do not wait for response when sending RPC_CMD_SET_TENSOR (#12943)
143ecda2ec4b347031a9b8a89ee2efc664ce63f599ceb1776b15a32d832f1266deeeab75b9d255c58492025-04-23T20:21:59+02:00Xuan-Son Nguyenmtmd : Support Pixtral 12B (#13065)
144b9154ecff93ff54dc554411eb844a2a654be49f22db9ba1464f3de0aceb2b5289963e69fc369cb662025-04-18T10:04:51+02:00Xuan-Son Nguyenmtmd : add methods to access `mtmd_image_tokens` (#12906)
145971f245b3b5f3f55991bb779cb541b00f82eea1d12b17501e6015ffe568ac54fdf08e6580833bf1b2025-04-17T01:37:05-07:00Mikko Juolallama : recognize IBM Granite 3.3 FIM tokens (#12988)
1468b9cc7cdd8a0dcf0176c60c755322c95b596529964eda5deb9859e87a020e56bab5d2f9ca956f1de2025-04-10T22:57:16+02:00Xuan-Son Nguyenllava : introduce libmtmd (#12849)
14764eda5deb9859e87a020e56bab5d2f9ca956f1defe5b78c89670b2f37ecb216306bed3e677b49d9f2025-04-10T17:24:44+02:00Xuan-Son Nguyenconvert : ability to lazy-load safetensors remotely without downloading to disk (#12820)
1480090950f679475c5ecaac2f7bca5049cca96492b7ecd780b1a1d5214b8d04c25ebfc194d310816ed2025-04-09T00:25:08-05:00Jeff Bolzvulkan: In coopmat2 mmq, load q4_k/q5_k scales through shared memory (#12833)
1491466621e738779eefe1bb672e17dc55d63d166bb82974011f312057b446c27267105bd7ad38105992025-04-07T23:06:44+02:00Xuan-Son Nguyenllama : Support llama 4 text-only (#12791)
150a10b36c91a091f4606710fba4e9327fd71e0e73883a88bd6affbe148a622ac730952ac5b8b5859792025-04-02T14:32:59+03:00Georgi Gerganovllama : refactor kv cache guard (#12695)
151c80a7759dab10657b9b6c3e87eef988a133b9b6a250d7953e829ff0e16074510fef0e7cec696461b2025-03-31T18:40:56+02:00Daniel Beveniusvocab : add special infill tokens for CodeLlama (#11850)
152403fbacbbcd3650abcc53e84f5e6bf88e51dd6f5a8a1f3356786cbf8bcc3422e3c8737fc33b453e72025-03-31T16:36:25+02:00Sigbjørn Skjæretconvert : Qwerky : use lora_rank_tokenshift and lora_rank_decay if present (#12667)
15352de2e594979be52f0da92601747aa44a13e50e42c3f8b850a4a6cff0f5dda2135c03fc81d33ed8b2025-03-31T10:20:30+02:00marcoStocchitts : remove printfs (#12640)
1545d01670266859444366e4f333ade5e0e5e2ae63def03229ff423dd1991f4f44ef1352f03334d86eb2025-03-28T01:05:44-07:00Benson Wongserver : include speculative decoding stats when timings_per_token is enabled (#12603)
15513731766db91ec927c1b61bf502ac7a9be2b11b9ab6ab8f809bd514d88e02a63869b4d619f13fa862025-03-28T13:13:22+05:30amritahs-ibmllamafile : ppc64le GEMV forwarding for FP32. (#12594)
156c7b43ab60855f752ae79937fb93d561bc30b69a424feaec05792b972d5ff3e2b12d9237ebd50d1ac2025-03-27T12:21:47+05:30amritahs-ibmllamafile : ppc64le MMA implementation for Q4_0. (#12489)
15700d53800e00bb22a26bf710fa6bd1150e412cc1d7ea75035b67f44c22ed7039967f718011fd35ce52025-03-24T06:47:24-04:00compiladellama-vocab : add SuperBPE pre-tokenizer (#12532)
158732b5fbf5e7f9cf069942f0c5850ee959ef321ba568013d0cd3d5add37c376b3d5e959809b711fc72025-03-20T02:36:37-04:00Bartowskiconvert : avoid calls to tokenizer.added_tokens_decoder (#12473)
1592048b5913d51beab82dfe29955f9008130b936c0f08f4b3187b691bb08a8884ed39ebaa94e9567072025-03-13T06:10:05-04:00Ishaan Gandhiserver : fix crash when using verbose output with input tokens that are not in printable range (#12178) (#12338)
1602b3a25c212f8c4d8a49cec23e03343a7719d51c98352cdc87b207a735d34c431a36c425728cb45862025-03-10T09:44:42ZOlivier Chafik`sampler`: fixes trigger tokens + lazy grammars (fix typo cast from token to string) (#12291)
16106a92a193a07afe445929607be9d5e4d033956fba057897ad4e48e3df0354256e0cc80dadcb575952025-03-05T15:25:45+08:00Clauszyserver : fix cache reuse logic (#12161)
1627b69003af74924ac04d97313c2e57c45ba56b616ece9745bb8079b9f4af45df29b67ad0c6e50584d2025-03-03T11:42:45+01:00Xuan-Son Nguyenwebui : add ?m=... and ?q=... params (#12148)
16334a846b5847a18d133b360074f1fb485b2632b2d7a2c913e66353362d7f28d612fd3c9d51a831eda2025-02-24T13:47:07-08:00lhezopencl: fix for small models (#11950)
164af7747c95ae71a5db4184947f837179e82c70b77a28e0d5eb18c18e6a4598286158f427269b1444e2025-02-23T05:39:24+08:00Aaron Teoggml-cpu: Support s390x SIMD Instruction Set (#12019)
16536c258ee921dbb5c96bdc57c0872e4a9a129bef6f3e64859edb0d55d4223ead78672597cd1a218df2025-02-22T22:28:28+08:00Ting Loullava: build clip image from pixels (#11999)
166abd4d0bc4f1a9a0e429bc8ee0d5ece2a394a0a399626d9351a6dfb665400d9fccbda876a0a96ef672025-02-19T13:29:42+02:00Georgi Gerganovspeculative : update default params (#11954)
16763e489c025d61c7ca5ec06c5d10f36e2b76aaa1d63ac12856303108ee46635e6c9e751f81415ee642025-02-18T18:03:23ZOlivier Chafiktool-call: refactor common chat / tool-call api (+ tests / fixes) (#11900)
1683e693197724c31d53a9b69018c2f1bd0b93ebab2a394039db004c6ee00098250d160b5aa018c23142025-02-13T07:07:51+01:00Daniel Beveniusllama : update llama_decode_internal ref [no ci] (#11840)
169a18f481f99962638092d6f1c98b1d34d3e3256deb9ab0a4d0b2ed19effec130921d05fb5c30b68c52025-02-11T14:06:45+01:00Daniel Beveniusserver : use common_token_to_piece instead of common_detokenize (#11740)
1702d219b389e8c8c40bce547b08c8aa7add60fde1f333820d7491cd31c707a340ff23b984a84e401542025-02-07T08:55:47-05:00Christian Fillionvocab : ignore invalid UTF-8 input in the BPE tokenizer (#11729)
171c3db0480bb820e120249014b380e1f4badf2a1c1d774ab3acc4fee41fbed6dbfc192b57d5f79f34b2025-02-06T01:55:25+01:00Matvey Solovievreadme : add link to Autopen under UIs (#11684)
172cde383323959544abe10a4d79e1d3e1ee479933cb3451785aca16fbf4214eeba7e4612676cdf8ccb2025-02-03T23:49:27ZOlivier Chafik`tool-call`: allow `--chat-template chatml` w/ `--jinja`, default to chatml upon parsing issue, avoid double bos (#11616)
173ff227703d6d6e1888bdc7af6138514092ffcdb960cec062a638700495673f5494d200b74340538be2025-02-01T23:55:32-08:00Michał Moskalsampling : support for llguidance grammars (#10224)
1740cec062a638700495673f5494d200b74340538be53debe6f3c9cca87e9520a83ee8c14d88977afa42025-02-02T15:48:46+08:00piDackllama : add support for GLM-Edge and GLM-Edge-V series models (#10573)
175ffd0821c57edc7e5d04338ab0c6b1461198df15f4314e56c4f8c5091f45732f39bd94c0c6c3237982025-01-30T11:10:59+01:00mgroeber9110vocab : correctly identify LF token for GPT-2 style BPE tokenizer (#11496)
176df984e014714cba4c99ef894b20b51cbcef31b16acd38efee316f3a5ed2e6afcbc5814807c3470532025-01-27T12:07:12+01:00Johannes Gäßlerllama: refactor llama_decode_impl (#11381)
1776171c9d25820ccf676b243c172868819d882848fe28245f35f2faaf249dd352998b3693a8cc28c512025-01-21T13:18:51ZOlivier ChafikAdd Jinja template support (#11016)
1786390a998bfc63241fde8509022b0768a71bf20bb44e18ef93995f3040660750b527e5becf85899d02025-01-18T18:20:57+08:00LostRuins Concedotts : add guide tokens support (#11186)
1793edfa7d3753c29e44b964c0ff424d2ea8d5fdee6667d72846c06b2cf4f7c8a4265e210991a49706b2025-01-17T20:57:56+08:00codezjxllama.android: add field formatChat to control whether to parse special tokens when send message (#11270)
1808f70fc3d1b1d3c17b61842330dd106d391cc12271244cdcf14900dd199907b13f25d9c91a507f5782025-01-13T13:38:20+01:00Daniel Beveniusllama : remove 'd' from bad special token log (#11212)
18108f10f69c38288e9e8bb1f933af63a3fc9013d40afa8a9ec9b520137bbd1ca6838cda93ee39baf202025-01-12T12:15:53+02:00Georgi Gerganovllama : remove notion of CLS token (#11064)
182afa8a9ec9b520137bbd1ca6838cda93ee39baf20c05e8c9934f94fde49bc1bc9dc51eed2826051502025-01-12T11:32:42+02:00Georgi Gerganovllama : add `llama_vocab`, functions -> methods, naming (#11110)
1832739a71e4b88474833b64aa974ca4515574fd3c4ba8a1f9c5b675459c55a83e3f97f10df3a66c7882025-01-11T05:50:33+01:00Daniel Beveniusconvert : sort print supported models [no ci] (#11179)
184ff3fcabc727b2dd0c477d23a258217b27cc639fbc3f9d25706ac84297067aeaa662c1f1af42ed4432025-01-10T11:30:53+01:00Daniel Beveniusconvert : add --print-supported-models option (#11172)
1858cef75c743ba13ebbd6d380c531200c768a8b8aa0d52a69e4bf0d6181beec7853307bdcdeec9905b2025-01-08T16:24:19+05:30amritahs-ibmllamafile : ppc64le MMA INT8 implementation (#10912)
1863e6e7a6bc2c4b980a0cf0fcb5cb3b79a965b5f14ae2f606bb598b287f5fb69c9fdfc98b86598c6cc2025-01-06T10:54:25+02:00Georgi Gerganovtokenize : escape the prompt (#11058)
187727368c60f2ebf2d6a7473a4a9f80957ab063a8e5047dd3546951dea3d65c02257d06c46c86623382025-01-06T10:52:15+02:00Georgi Gerganovllama : use LLAMA_TOKEN_NULL (#11062)
1889394bbd484f802ce80d2858033583af3ef700d25f922a9c542ee117550a168395c63ea79261f5c992025-01-04T21:06:11+01:00fairydreamingllama : Add support for DeepSeek V3 (#11049)
1892f0ee84b9b02d2a98742308026f060ebdc2423f10da5d860266c6928b8c9408efbd264ae59fedda62025-01-02T18:06:12+01:00Pierrick Hymbertserver: bench: minor fixes (#10765)
19016cdce7b68218959e0658e2f95b4572573d5008ed79d8f39b4da6deca4aea8bf130c6034c482b3202024-12-28T15:08:54ZAlexey Parfenovserver : fix token duplication when streaming with stop strings (#10997)
19130caac3a68a54de8396b21e20ba972554c58723060cfa728e27c28537657d4e627ed432508eb95372024-12-24T09:44:20+02:00Georgi Gerganovllama : the WPM vocabs use the CLS token as BOS (#10930)
192b92a14a841fb4dfaf27b29d982ec8ba5289a3bff6f0c9e034bb398915a6617ee4acc62adb87d387d2024-12-23T08:35:44+08:00Yun Doullama : support InfiniAI Megrez 3b (#10893)
1937ae33a616f44ecc081f3dcb589be20962d1d4a92ebdee9478ca7ba65497b9b96f7457698c6ee51152024-12-23T01:09:58+03:00Billel Mokeddemllama : add Falcon3 support (#10883)
19457bb2c40cd94c5a09f5210ed8264cc93b21c4b7ea3c33b1dce2d4f25040b75f66629104bd1e401282024-12-19T15:40:08+01:00Xuan Son Nguyenserver : fix logprobs, make it OAI-compatible (#10783)
1952fffc52b50992ac5bc64db19d33c39cbc06f52cf7585edbdebd02861e0994dae67c9338731fb3fc52024-12-19T06:04:51-07:00Sukriti Sharmallama : fix Roberta embeddings (#10856)
1960bf2d10c5514ff61b99897a4a5054f846e384e1e7bbb5acf125d1d2840cac7d31b9aaa72210dd5ec2024-12-18T19:27:21+02:00Georgi Gerganovtts : add OuteTTS support (#10784)
197152610eda91217ac409342cd976d05f5114ad39f0e70ba686e6c717a0aa41d88284e2a392c2bd0cd2024-12-18T13:01:41+02:00Georgi Gerganovserver : output embeddings for all tokens when pooling = none (#10861)
198644fd71b44c4cdbfc6482fbf0353d289c3bc29e64ddd199f6f6b980e0a7ed9f9b44efeae2fbdf5c42024-12-16T12:31:14+02:00Georgi Gerganovsampling : refactor + optimize penalties sampler (#10803)
199ba1cb19cdd0d92e012e0f6e009e0620f854b6afd56eea0781cbd2608ed8ff524955495569b9264be2024-12-14T20:43:46+08:00HimariOllama : add Qwen2VL support + multimodal RoPE (#10361)
2008fc393f246c550d2481e53323a47644a94e8d01fec450d3bbf9fdb3cd06b27c00c684fd1861cb0cf2024-11-09T15:06:54+08:00haopengscripts : fix pattern and get n_tokens in one go (#10221)
2012319126a70b541f8670225a04a38202bbdccbedb3bcd40b3c593d14261fb2abfabad3c0fb5b9e3182024-11-07T02:02:08-06:00snadampalfix q4_0_8_8 format for corrupted tokens issue (#10198)
202d865d1478cd4e403f82d793c2afcd0f943412f051804adb0cfee4811eaf633741503d683a46e4c772024-11-01T13:33:14Zsasha0552server : fix smart selection of available slot (#10120)
203958367bf530d943a902afa1ce1c342476098576b40f2555797f97314de749873cdc29dc102be66e22024-10-24T21:51:22+02:00Xuan Son Nguyenserver : refactor slot input data, move tokenizer to HTTP thread (#10023)
204c8c07d658a6cefc5a50cfdf6be7d72650361230319d900a7565b8f6b0a708836a57d26966cb9efe22024-10-22T16:59:02+02:00Xuan Son Nguyenllama : fix empty batch causing llama_batch_allocr to crash (#9966)
205c421ac072d46172ab18924e1e8be53680b54ed3b4ff7fe1fb36b04ddd158b2de881c348c5f0ff5e42024-10-22T13:08:41+02:00Xuan Son Nguyenlora : warn user if new token is added in the adapter (#9948)
2064ff7fe1fb36b04ddd158b2de881c348c5f0ff5e46b8447352df3d662b56280c8fc38d7f0928857872024-10-22T18:33:37+08:00Molly Sophiallama : add chat template for RWKV-World + fix EOT (#9968)
207cda0e4b648dde8fac162b3430b14a99597d3d74fafd9909a6481402844aecefa8a8908afdd7f52f12024-10-18T23:18:01+02:00Xuan Son Nguyenllama : remove all_pos_0, all_pos_1, all_seq_id from llama_batch (#9745)
2086f55bccbb8835d42147add4ee48807450f5ff53517bb9280807cfbb6611b853aa1ef05114bd9efe92024-10-18T01:41:51+02:00Daniel Beveniusllama : rename batch_all to batch (#8881)
20999bd4ac28c32cd17c0e337ff5601393b033dc5fc3752217ed5a6a11864682fbf009bcb36afffd6bc2024-10-17T22:32:47+03:00Georgi Gerganovllama : infill sampling handle very long tokens (#9924)
2109e041024481f6b249ab8918e18b9477f873b5a5edbf18e4de9e7aa496871f1555f9f0c8d845671082024-10-16T19:34:28+02:00Daniel Beveniusllama : suppress conversion from 'size_t' to 'int' (#9046)
211fbc98b748e7b075e327bcf13237057f647678049dcdd535302fc9702a4709be25f56540d65163a442024-10-15T15:54:55+05:00MaggotHATEsampling : add XTC sampler (#9742)
21211ac9800aff532715a5bc7991062c68ba3472e6e943d20b4111c746bcd9dbc7e4771de313b08b50c2024-10-12T08:21:51+03:00Georgi Gerganovllama : improve infill support and special token detection (#9798)
2138c475b97b8ba7d678d4c9904b1161bd8811a9b4458b16695e146628481c6b9b8a3b101c0c9bac00f2024-10-05T15:55:04+03:00Georgi Gerganovrerank : use [SEP] token instead of [BOS] (#9737)
214e3c355ba654d4164c1c09e5d0dcacecb8b214af8841713e1e487bdb82fd106a52ad998c5f87b59e92024-10-03T10:22:15-04:00compiladeconvert : handle tokenizer merges format from transformers 4.45 (#9696)
2151927378bcce20ba72b6c89d5977b854a4bcaeb5d6f1d9d71f4c568778a7637ff6582e6f6ba5fb9d32024-10-01T02:31:36-04:00compiladeconvert : refactor rope_freqs generation (#9396)
216faac0bae265449fd988c57bf894018edc36fbe1ef99d3f8367174f7aba73c07fd87de687d4a0ece12024-09-29T05:25:00-07:00matiaslincommon : ensure llama_batch size does not exceed max size (#9668)
217f4d2b8846a6b34419ff9e9491aee6cd95e444bfc1b2f992cd2cff0b69e5abe78bb8888d51ed19d672024-09-28T17:42:03+03:00Georgi Gerganovllama : add reranking support (#9510)
2186102037bbb55521880ae78a6ee6c2a0c00c901df9a913110cf471a8287ac06c43cbe307d3cf6df992024-09-28T20:10:58+08:00Zhenwei Jinvocab : refactor tokenizer to reduce init overhead (#9449)
2199a913110cf471a8287ac06c43cbe307d3cf6df9943bcdd9703ec19af7d2a519640b5ed6f4aac3d532024-09-28T12:08:43Znopperlllama : add support for Chameleon (#8543)
22031ac5834fe75c296476658a124c06c84772aa641cea1486ecf34a1c7e014a9e290eb458f5a11f8762024-09-24T10:16:06+03:00Georgi Gerganovllama : keep track of all EOG tokens in the vocab (#9609)
2216443ddd98576a9da904ef9f07df4e4398bb6a01a8a308354f6520df6bea851b435bd8054ee5617b42024-09-18T13:42:36+02:00Daniel Beveniusllama : use reserve/emplace_back in sampler_sample (#9534)
2228344ef58f8cdb8ebd6faf4463ca32ae91c374c810226613853133c081b55bb892a41bb5eacc0bc942024-09-17T12:18:22+02:00Michael Podvitskiyllama : fix n_vocab init for 'no_vocab' case (#9511)
2230abc6a2c25272d5cf01384dda8ee8bfec4ba8745bd35cb0ae357185c173345f10dc89a4ff925fc252024-09-13T09:53:38+03:00Georgi Gerganovllama : llama_perf + option to disable timings during decode (#9355)
22478203641fee3b1f82abaff0c7f667e1b4a286390e6b7801bd189d102d901d3e72035611a25456ef12024-09-12T22:30:11+02:00Mathijs Henquetserver : Add option to return token pieces in /tokenize endpoint (#9108)
225c837981bba7cf6839b69d32b25552ce685936b143c26a1644dacfa6b5d58af550210524efd7b93fc2024-09-12T20:28:20+09:00daminhopy : add Phi-1.5/Phi-2 tokenizer (#9361)
22639f852f44039b058fdd0611ee127c6efa7ba4a042b00fa799773cc75d53b841c03d21d7468a1e3a12024-09-12T19:25:16+08:00Molly Sophiapy : add special tokens in hf_converter for RWKV v6 (#9428)
2271b28061400eb9832603c9f1dfbec4d339a8490a28db003a19d7055b5bd248ce2afff9324e5b8da952024-09-11T17:52:13+02:00slarenllama : skip token bounds check when evaluating embeddings (#9437)
22883008b7cfe90ad89d0c0ed2c2424fd75edc25ac10b4ac75772b744bb0a0d674927587621d10578842024-09-10T09:03:21+02:00Daniel Beveniusllama : update llm_build_copy_mask_state comment [no ci] (#9385)
229eae597182cb61bbde0b26e7cec5999d28b9327fe00b02bb249406ec0123757a67a5b714c3f33a6992024-09-08T12:41:51+02:00slarenllama : sanitize tokens in the upper bound (#9359)
230fbb7fcffbcfc50009c275e75982b1603a6cb6b80a5b5d9a1014248f385939e6739e9db9de7147e552024-09-07T22:51:00-07:00Kevin Gibbonsllama : set attrs of mislabelled EOT/EOM tokens (#9348)
231faf69d4237c9ae4d7f572b4674d1002463e8acd3e536426ded3fb4a8cd13626e53508cd92928d6c22024-09-08T00:33:13+03:00Georgi Gerganovllama : sanitize invalid tokens (#9357)
2329b2c24c0993487d3b34a873980e292da571481f3134bc38ecf3e2c5460581badce289a1ffa6804532024-09-06T23:21:29+02:00Xuan Son Nguyenserver : simplify state machine for slot (#9283)
2339bc6db28d011d47a5f318dc4aebbe7927fac462932b2ec88bc44b086f3807c739daf28a1613abde12024-09-05T21:48:47-04:00compiladeggml-quants : ternary packing for TriLMs and BitNet b1.58 (#8151)
2349c1ba557335c3cab46807e6478eb7d17a63361f1c6d4cb46559b359d2682cf2a002e7fe01bb7a7672024-09-02T16:51:01+05:30Tusharbuild(nix): Package gguf-py (#5664)
2358f1d81a0b6f50b9bad72db0b6fcd299ad9ecd48ca47667cff41f5a198eb791974e0afcc1cddd32292024-09-01T22:38:17+08:00Molly Sophiallama : support RWKV v6 models (#8980)
236c679e0cb5c378bfb63643c44a453345ba8b90126fb487bb5671b37267f35ff43fe8849ddccd925cd2024-08-16T15:35:18+09:00Minsoo Cheongllama : add EXAONE model support (#9025)
2374af8420afba39de4968adf2695ce12fd42422a136bda7ce6c3a9284fcbb70c1ace4107db8eb63e5c2024-08-15T15:23:23+08:00Zhenwei Jincommon : remove duplicate function llama_should_add_bos_token (#8778)
2386bda7ce6c3a9284fcbb70c1ace4107db8eb63e5cd5492f0525fa533817a67e93a4bde9d71d81cf582024-08-15T10:17:12+03:00Esko Toivonenllama : add pre-tokenizer regexes for BLOOM and gpt3-finnish (#8850)
2395ef07e25ac39e62297a67208c5bcced50835a2dd4134999e01f31256b15342b41c4de9e2477c4a6c2024-08-12T10:21:50+03:00Georgi Gerganovserver : handle models with missing EOS token (#8997)
2407c3f55c10051c634546247387c5c359c9d499360911b437f228e75aa3d235acec21bfddd23ecce2f2024-08-10T11:43:26+02:00fairydreamingAdd support for encoder-only T5 models (#8900)
241cdd1889de62a7140c8c016405ec917464bde8bd3c21a896405de4cdf4207eb8130555ceaac0ab1102024-08-06T02:20:54-05:00Douglas Hanleyconvert : add support for XLMRoberta embedding models (#8658)
242d3f0c7166adfa952237e0f437a5344362d8256d4e31a4f679779220312c165b0f5994c680a610e382024-08-05T09:38:01+02:00fairydreamingStop the generation when <|eom_id|> token is encountered - needed for Llama 3.1 tool call support (#8858)
243978ba3d83d17b10fdf9807006048432b5b3769fcecf6b7f23e664afd7ff856ec39034240ce438daa2024-08-04T18:16:23ZardforkServer: Don't ignore llama.cpp params (#8754)
2449d03d085dd6cb275c078690bb64073b9b043e95fbfb4c74981f0a40d757b450b596a9fe4ca983d262024-07-27T12:45:02+02:00Daniel Beveniuscommon : add --no-warmup option for main/llama-cli (#8712)
245938943cdbf4dd79005a394d732bc226f9e34e0ff751fcfc6c33ea5f43cadd4d976f8fb176871df5e2024-07-23T13:10:17+03:00Georgi Gerganovllama : move vocab, grammar and sampling into separate files (#8508)
246d94c6e0ccbd29ee1ba4f44e9caa8682ad94df9fa566daa5a5b38018b2727950bbd280239adb981b62024-07-22T10:43:01-04:00Jason Stillermanllama : add support for SmolLm pre-tokenizer (#8609)
2476f11a83e4e7700fdf353ed4a29599cb662c792f6e093dd238282a980d248db0786063f8174400f702024-07-22T13:33:22+03:00Georgi Gerganovllama : allow overrides for tokenizer flags (#8614)
248e093dd238282a980d248db0786063f8174400f7050e05353e88d50b644688caa91f5955e8bdb9eb92024-07-22T13:32:49+03:00Georgi Gerganovtests : re-enable tokenizer tests (#8611)
249628154492a0b2dcc954a3af99e5c267097568eae04bab6b7da0ed2b0a57174a57784d602aabb6c102024-07-22T16:02:09+08:00Jan Boonserver : update doc to clarify n_keep when there is bos token (#8619)
250c69c63039cd75f8f33f253ab7485d82a8b4cd40369c487f4ed57bb4d4514a1b7ff12608d5a8e7ef02024-07-20T21:53:01-04:00compiladeconvert_hf : fix Gemma v1 conversion (#8597)
251940362224d20e35f13aa5fd34a0d937ae57bdf7d69b9945b44c3057ec17cb556994cd36060455d442024-07-20T09:43:51-04:00Michael Coppolallama : add support for Tekken pre-tokenizer (#8579)
25269b9945b44c3057ec17cb556994cd36060455d44c3776cacabce2ee35f172fb72be7a519752125fa2024-07-20T09:09:37-04:00Huifeng Oullama.swiftui: fix end of generation bug (#8268)
253fa79495bb4897953a75607addd9d2cdd2ec6322217eb6aa8a992cda37ee65cf848d9289bd6cad8602024-07-13T23:35:10-04:00compiladellama : fix pre-tokenization of non-special added tokens (#8228)
254c3ebcfa148e867a68e78fd5c4f0c23e8f84c788b8a4441ea1a2564578134404f31158c318e9c0bf32024-07-12T03:14:12-05:00Douglas Hanleyserver : ensure batches are either all embed or all completion (#8420)
2555aefbce27a66473d4b1263ba3f7bdd3d1424597571c1121d11f1437be9421fd0cbaa011b9ef490982024-07-12T10:06:33+02:00Jiří Podivínconvert : remove fsep token from GPTRefactForCausalLM (#8237)
2569a55ffe6fbf7f19c865f8f277ca7cd585cc0b0947a221b672e49dfae459b1af27210ba3f2b5419b62024-07-11T03:41:48-04:00compiladetokenize : add --no-parse-special option (#8423)
257470939d483d1c89b7292f78bac1fd27c42c171ce6f0dbf6ab087bcd286fb78560099ca04583167352024-07-08T03:26:53-04:00Kevin Wangcommon : preallocate sampling token data vector (#8363)
2586f0dbf6ab087bcd286fb78560099ca0458316735ffd00797d81ef7db1528b9e10adbdc333ade64952024-07-08T09:34:35+03:00Georgi Gerganovinfill : assert prefix/suffix tokens + remove old space logic (#8351)
2593fd62a6b1c9ca7b7c0093e984cc9c133c6f2726da8db2a9ce64cd4417f6a312ab61858f17f0f85842024-07-07T15:04:39-04:00compiladepy : type-check all Python scripts with Pyright (#8341)
260905942abdba5ba0b28a1b0805e51e4f818c54bc9b5040086d436e7345e4fa33a5b9558060c75603f2024-07-07T20:52:10+08:00toyerllama : support glm3 and glm4 (#8031)
26186e7299ef5dff0f388922dc6fcbce009e99d800560d83a0149849e9217e4b8ae26e277a41aea906e2024-07-06T15:32:04-05:00Derrick T. Woolworthadded support for Authorization Bearer tokens when downloading model (#8307)
262213701b51a17175d0d326b566efc03f30ec7fbe6be20e7f49d9e5c6d9e8d9b4871eeba3df7a1639d2024-07-05T19:01:35+02:00jaime-m-pDetokenizer fixes (#8039)
2636f63d646c1a06a6e09f721009a2676864ae04e3151d2ebadbbf365b894f3888361df42dbacb12b7a2024-07-04T18:38:58+02:00Daniel Beveniustokenize : add --show-count (token) option (#8299)
264807b0c49ff7071094f97ebc3a0a8e2b9e274f503f8c4c0738d72d2162736edd72dd5db8b269adca12024-07-04T15:46:11+02:00fairydreamingInference support for T5 and FLAN-T5 model families (#5763)
26520fc3804bfb727074bc270b6eacb60af8d0bf7d4f619024764e72261f14d7c31d892b8fb976603b42024-07-04T10:41:03+03:00Georgi Gerganovconvert : fix gemma v1 tokenizer convert (#8248)
2665fac350b9cc49d0446fc291b9c4ad53666c77591cb5fad4c6c2cbef92e9b8b63449e1cb7664e48462024-07-02T01:07:23+02:00Xuan Son NguyenFix gemma2 tokenizer convert (#8244)
267e57dc62057d41211ac018056c19c02cd544694dfa27aa50ab7e07fe46aae619076b6e31d5663e9142024-06-28T00:00:43-04:00pcullitonllama: Add support for Gemma2ForCausalLM (#8156)
268f675b20a3b7f878bf3be766b9a737e2c8321ff0d911e35bb8bb2fd1c7d3f40f27e96ff432eae7e142024-06-27T11:58:54+03:00kustaayaAdded support for Viking pre-tokenizer (#8135)
269911e35bb8bb2fd1c7d3f40f27e96ff432eae7e14ac146628e47451c531a3c7e62e6a973a2bb467a02024-06-27T09:46:41+02:00Sigbjørn Skjæretllama : fix CodeLlama FIM token checks (#8144)
2706777c544bdd8c5d9de3220d6e2557957bbbf2a4f163d50adaf8897d8b734d701ff332de6be63d4842024-06-26T01:45:58+01:00Olivier Chafik`json`: fix additionalProperties, allow space after enum/const (#7840)
2716fcbf6823553efabe52ed83e3c2a3329aa3387d1e6bf007744eb06336a231ef39cf08146dd16d2ce2024-06-25T21:14:35+02:00fairydreamingllama : implement Unigram tokenizer needed by T5 and FLAN-T5 model families (#5763)
272d62e4aaa02540c89be8b59426340b909d02bbc9e9a590c82262dd518137f85406e65e452fdf2aca32024-06-24T14:13:39+02:00fairydreaminggguf-py : fix tensor groups for encoder-decoder models in gguf-dump.py (#8090)
273de0d6a68ac99f307fe889c48e21124bc3b7ca29a95f57bb5d5b18ef0beb2702a0d6c06e46804075c2024-06-24T07:06:05+02:00fairydreaminggguf-py, convert-hf : model conversion support for T5 and FLAN-T5 model variants (#5763)
274a927b0f3dd9a86ee042cd2bdcc8c9da4a855926b80ea089d771f0c2d97afa8bead80ded412f600d72024-06-21T08:51:28+03:00Georgi Gerganovllama : optimize long word tokenization with WPM (#8034)
27580ea089d771f0c2d97afa8bead80ded412f600d70e64591e8290037db6412665a56354b789a0597e2024-06-21T00:38:22-05:00Douglas Hanleyllama : allow pooled embeddings on any model (#7477)
27637bef8943312d91183ff06d8f1214082a17344a591c188d6c296bd3384f2a02a83b71187aa3d18b32024-06-18T18:40:52+02:00jaime-m-ptokenizer : BPE fixes (#7530)
27791c188d6c296bd3384f2a02a83b71187aa3d18b384f6de17f6a8602e7ff7f7c7bda36a73f510a2dd2024-06-18T14:19:45+02:00Sigbjørn SkjæretOnly use FIM middle token if it exists (#7648)
2780c7b3595b9e5ad2355818e259f06b0dc3f0065b37b2f4a7d193ef2475259bbe7656fcccfab4b12172024-06-15T18:53:40+02:00Xuan Son NguyenAdd `cvector-generator` example (#7514)
2796fcd1331efbfbb89c8c96eba2321bb7b4d0c40e441b9260f18eb7f325c952006ac46afc1d0d8ad2f2024-06-14T12:20:04+02:00Sigbjørn Skjæretllama : more checks before assuming FIM tokens (#7644)
28041b9260f18eb7f325c952006ac46afc1d0d8ad2f172c8256840ffd882ab9992ecedbb587d9b21f152024-06-14T13:16:49+03:00Elaineconvert : add Poro-34B-chat tokenizer support (#7713)
281ad675e1c67a05b16e4e12abe30dbecfc808e7b7ea143c04375828b1f72eb1a326115791b63e793452024-06-06T06:08:52-07:00Clint HerronAdded support for . (any character) token in grammar engine. (#6467)
282c90dbe026b456a233f8f0fbe752212e6a0503ca2b90dc566c1c615289b05b50d61680f23744a21e72024-06-05T01:26:14+02:00jaime-m-pFix per token atrributes bits (#7749)
2833b38d48609280aa5f8ab7ea135a4351b2a5ee2406d1616944d9efd342ed2a4fd318722adfc9febcd2024-06-04T09:17:17+02:00jaime-m-pPer token attributes (#7685)
284549279d8049d78620a2b081e26edb654f83c3bbd9e405b6e2ecb888e860f7b92720b4809e21b39152024-06-03T08:34:43+03:00Georgi Gerganovllama : avoid double token-to-piece cache (#7654)
2852e666832e6ac78194edf030bd1c295e21bdb022c2ac95c9d5678d05e253691fb1f26471675bff5ad2024-06-01T21:31:48+02:00Yazan Agha-Schraderserver : new UI (#7633)
2862ac95c9d5678d05e253691fb1f26471675bff5ad750f60c03e4d3f53fa51910551ce87a3d508d2d72024-06-01T21:50:18+05:30HanishKVCSimpleChat: Simple histogram/repeatMatching driven garbageTrimming, Settings UI, Streaming mode, OpenAi Compat (Model, Authorization Bearer), Save/Restore session, Auto Settings UI (#7548)
2875921b8f089d3b7bda86aac5a66825df6a6c106035dcdf946764fae49a8e2a90bf2f0960bde1c44e82024-05-30T19:01:41+03:00Georgi Gerganovllama : cache llama_token_to_piece (#7587)
2889c4c9cc83f7297a10bb3b2af54a22ac154fd5b2059b0d077662fab430446b3119fa142f3291c45b22024-05-30T13:40:00+02:00GalunidMove convert.py to examples/convert-legacy-llama.py (#7430)
289eb57fee51f7b4d78039f003249873c2eb46f12f655d62262a99cd8bc28a1492975791fe433c8cc0f2024-05-30T02:10:40+02:00Galunidgguf-py : Add tokenizer.ggml.pre to gguf-new-metadata.py (#7627)
29002c1ecad07f0e2d2febe8196271bcc64bdc9c0066bd12ce409f949012935b7d1b15a21ffa473a5652024-05-28T21:46:34+02:00jaime-m-pTokenizer WPM fixes (#7500)
2915442939fcc5e6ae41abf40612a95fd71377e487e56411a950f255b523a9edd684fd16327524743992024-05-28T20:49:49+02:00Giuseppe Scrivanollama : support small Granite models (#7481)
292edc29433fa08b4e5aeb67649a29fc7713af13d048b99e2aa66ba39e4e1114effea6ef7430881eca42024-05-28T15:04:09+03:00Georgi Gerganovtests : fix test-tokenizer-0.sh
293852aafb163d32d5bad63c10bc323a02c28fec59d0136966dafb452601c23f30395878d5a65ddc5592024-05-28T01:40:47+02:00Djip007update HIP_UMA #7399 (#7414)
294d298382ad977ec89c8de7b57459b9d7965d2c27232a28217f475119926c603341e8273b26932b56a2024-05-27T00:10:17+10:00Brianmain: replace --no-special with --special (#7534)
295b9adcbbf92fc7096bee23fe61496d25652ebf7659588f196b1d7b21bdff013fcf958c249576b26192024-05-26T06:26:34+05:30HanishKVCSimpleChat Completion Mode flexibility and cleanup, Settings gMe, Optional sliding window (#7480)
29600c63907931bb08a0ed2b7e38cf44dd290143cb9faa0e6979a11dcb731e9d778ad42ceaa0302015e2024-05-25T05:04:03-04:00Justine Tunneymain : don't print special tokens with --grammar (#6923)
29757684331fc2d685f7d1f5775af0b9e47d1829833b83bab15a5d2a1e7807d09613a9b34309d86cfaa2024-05-24T18:14:42-07:00Mikko JuolaMake tokenize CLI tool have nicer command line arguments. (#6188)
298fbca2f27fc7fa9aa4a8ad0357478fdb9084729080df0aa8e43c3378975269a51f9b876c8692e70da2024-05-24T14:31:13+02:00fairydreamingAdd support for ArcticForCausalLM (#7020)
2993015851c5ac7334fb544a23a70a284c117b8704455ac3b7aeaf52f19786ed96e885d89521fc0f6c82024-05-23T14:29:26+02:00Daniel Beveniusllama : add getters for n_threads/n_threads_batch (#7464)
300c3f8d583560b4f261fa21c976793e538c60cd66c11474e756de3f56b760986e73086d40e787e52f82024-05-21T19:53:48+03:00Georgi Gerganovtests : test-tokenizer-0.sh print more info (#7402)
301d7e852c1bc8e85bf62a6f1aede08cd2de723404a917dc8cfa67a72fb7c8bf7392270da3bf4833af42024-05-21T14:39:48+02:00jaime-m-pTokenizer SPM fixes for phi-3 and llama-spm (bugfix) (#7425)
302917dc8cfa67a72fb7c8bf7392270da3bf4833af4fabf30b4c4fca32e116009527180c252919ca9222024-05-20T20:15:57+02:00jaime-m-pTokenizer SPM fixes for phi-3 and llama-spm (#7375)
3031b01f06db0cff5f5f600bb754fc39fde565ed56a41858392e17abead21735309bf17cb55183d8c312024-05-19T16:26:02+02:00Johannes Gäßlerserver: add test for token probs (#7347)
3046aade19ee74b896c59929676629340b36be3e22cab33f7a338593f6cf1ae98b10b6f8684f63bd72c2024-05-19T14:46:46+02:00Anas AhouziAdd StableLM2 pre-tokenizer (#7349)
3050f98acfac6cc561dc57586bfff778405e42b576bca57e0f35e33f714b9a6c2c4482b87bfe059c8192024-05-18T10:04:55+02:00Steffen Röckerllama : add support for larger Granite Code Models (20B, 34B) (#7324)
30629c60d8cddcfd14fa8a6bf023a6c4eb8692c76ba359cbe3f46c90ce6f5151005e411b8fb74f8139e2024-05-17T09:59:57+02:00Johannes Gäßlertokenization: add warning for double BOS (#7332)
307dc020985b8755dd6aa93a2f002f43c3ede808cce344f9126cc0d15891fde9472fe40b8572628ad7d2024-05-15T14:44:49+01:00agray3Avoid unnecessarily disabling CUDA graphs (#7302)
3089aa672490c848e45eaa704a554e0f1f6df995fc8b1f8af1886e8187db6bb2a9b87cfc1c0f175f6292024-05-13T10:35:14+02:00Joan Fontanalsllama : rename jina tokenizers to v2 (#7249)
309f99e1e456eaf69cc38c1982a2693ce41c0f897ef5ae3426b0b64672991563d4c28b2018b9f9614672024-05-11T16:12:06+08:00Haoxiang Feillama : lookup word in vocab before doing BPE merges (#7193)
3105ae3426b0b64672991563d4c28b2018b9f961467b83cc3f5b303ff30c52874b2d5864dc6385ebf9f2024-05-11T10:11:28+02:00Johannes Gäßlerserver: fix reported top tokens for temperature 0 (#7203)
311b83cc3f5b303ff30c52874b2d5864dc6385ebf9f9cb317f77e53067f7a138cc89ef7657148eae8e62024-05-11T09:46:09+02:00Joan Fontanalsllama : add Jina Embeddings architecture (#6826)
312f89fe2732c5709f6e86d5f4aee2e6d2a561f2eb2d11afd665241c1b3910ab5f040d0216403019d872024-05-10T15:51:58+05:30HanishKVCMain+: optionally allow special tokens from user in interactive mode (#7097)
31343248e559472556f368988575d9fba906b3eb139a743d76a01f23038b2c85af1e9048ee836767b442024-05-09T15:30:44+02:00jaime-m-pllama3 custom regex split (#6965)
314f31ec120bc36c6270e4948e6a065a7c4cfa0c404fd9f92b154850014146f61717cd292a59a5cee5a2024-05-09T14:13:05+02:00GalunidAdd warning if token is invalid (#7173)
315fd9f92b154850014146f61717cd292a59a5cee5a22842164bcae3251b81ad9e497a16ef66833cb9e2024-05-09T13:03:29+02:00Daniel Beveniusllama : update llama_timings.n_p_eval setting (#7160)
31622842164bcae3251b81ad9e497a16ef66833cb9e47345248827f426038098d016ed11975021b49192024-05-09T12:56:00+02:00Sigbjørn Skjæretgguf-py : add special token modification capability (#7166)
317f98eb31c517c95960df1d0abc48002787f145f3bbc4bba364fb96d908f2698e908648df5e6f55e022024-05-08T18:16:38-04:00compiladeconvert-hf : save memory with lazy evaluation (#7075)
318911b3900dded9a1cfe0f0e41b82c7a29baf3a217ad211edef5db1f1fb955874b7ca6a67bd0c887082024-05-08T14:27:58+02:00Johanserver : add_special option for tokenize endpoint (#7059)
319229ffff872f8ad0d21c997d18ee7a23692ae60a01fd9c1741d864d01cd7ec6d67227b92d7bfabf222024-05-08T20:06:43+08:00Ren Xuanchengllama : add BPE pre-tokenization for Qwen2 (#7114)
3204cd621c26de2095cd7c4464bdec5fe2e696ef3f37e0b6a7b3ba94ff624dc27c1e0e735fded8819b82024-05-08T06:43:23-04:00DAN™convert : add BPE pre-tokenization for DBRX (#7132)
321af0a5b616359809ce886ea433acedebb39b12969b6aa6702030320a3d5fbc2508307af0d7c947e402024-05-07T23:07:58+02:00Johannes Gäßlerserver: fix incorrectly reported token probabilities (#7125)
3223af34c1d1b0da47f85b95f60922abeded1cb5d3304976db7a819fcf8bfefbfc09a3344210b79dd272024-05-07T19:51:31+02:00RhinoDevelmain : update log text (EOS to EOG) (#7104)
323889bdd76866ea31a7625ec2dcea63ff469f3e9816fbd43221167bf96112f899daf22c127b282cbcf2024-05-05T01:19:30-04:00DAN™command-r : add BPE pre-tokenization (#7063)
32403fb8a002df2e96104f9e06de9c78d2a8ed91e9292139b90af4841d7fd060b526bdd443b621770ff2024-05-04T12:06:40+03:00maor-psIf first token generated from the server is the stop word the server will crash (#7038)
32592139b90af4841d7fd060b526bdd443b621770ffa2ac89d6efb41b535778bfeaecaae8fe295b6ed32024-05-04T08:32:32+03:00Georgi Gerganovtests : add test-tokenizer-0.sh + fix some tokenizers (#7036)
326f4ab2a41476600a98067a9474ea8f9e6db41bcfa3f167476b11efa7ab08f6cacdeb8cab0935c12492024-04-29T16:58:41+03:00Georgi Gerganovllama : fix BPE pre-tokenization (#6920)
3277f5ff558eed0f732af8f25c2ab0645610bdec80c9e4e077ec50fde6049b128662c72d37a3c28e34b2024-04-26T12:15:30+02:00Pierrick Hymbertserver: stop generation at `n_ctx_train` if `n_predict` is not set (#6638)
32837246b1031b1680c0dcaf20aef736d6b446203fa28103f4832e301a9c84d44ff0df9d75d46ab6c762024-04-24T05:15:29-05:00Kyle Mistelecommon : revert showing control tokens by default for server (#6860)
3293fec68be4e9577fc53158366d3b3af039c17bb1fc8297c6af5693555652c40b95974b95d49d2674d2024-04-24T15:16:21+08:00Junyang Linconvert : add support of codeqwen due to tokenizer (#6707)
330c8297c6af5693555652c40b95974b95d49d2674d4e96a812b3ce7322a29a3008db2ed73d9087b1762024-04-24T15:00:37+08:00liuwei-gitllama : add phi3 support (#6852)
3318960fe86ae075c846c5df8848230d1904ba8877fc0956b09ba845a7cd787d5580d7c8b96e80f40f52024-04-22T15:41:11+03:00Georgi Gerganovllama : fix typo in <|im_end|> token text (#6745)
33240f74e4d739e9250431cf339ae7588b28d8d0663b9cc76d87e3d7ae5900f19d4fe8f8976d0a358882024-04-21T18:36:45+03:00Georgi Gerganovllama : add option to render special/control tokens (#6807)
3337dbdba5690ca61b3ee8c92cfac8e7e251042e787c1386c936e9fbc38eb2816c711ab28f13355708e2024-04-21T15:03:39+02:00Wouterllama : add llama-3 chat template (#6751)
334b97bc3966e852adb626c90be64fd48282800f504b8109bc0139f15a5b321909f47510b89dca47ffc2024-04-21T13:50:41+02:00Pedro Cuencallama : support Llama 3 HF conversion (#6745)
33503c0946d73c63ea73e1d85015b7088298443d438e11b2e6e1e18522ca7cf129600875a0f6fb9307d2024-04-18T13:49:01+02:00Sigbjørn Skjæretconvert : support models with multiple chat templates (#6588)
3368cc91dc63c0df397d644a581b2cbeea74eb51ae0dbceec87c0221ec952e69448df6a71f1372a74872024-04-16T14:55:30-04:00Justine Tunneyggml : add llamafile sgemm (#6414)
3374fbd8098e63670c6ae11a8adc350f5ba191cfda37593639ce335e8d7f89aa9a54d616951f273af602024-04-16T08:13:13+02:00Daniel Beveniusgguf : add special tokens metadata for FIM/Infill (#6689)
3381b67731e184e27a465b8c5476061294a4af668eac4a3a4ff47d62d2503ddf9bd91b58c21f04fe3c32024-04-09T13:44:08-04:00Jared Van BortelBERT tokenizer fixes (#6498)
3395dc9dd7152dedc6046b646855585bd070c91e8c8e11a8999b5690f810c2c99c14347f0834e68c5242024-04-09T09:16:13+01:00Carolinabananallama : add Command R Plus support (#6491)
34075cd4c77292034ecec587ecb401366f57338f7c0a8bd14d55717754a1f48313a846a2b16fa998ad22024-04-06T05:40:47+02:00Pierrick Hymbertci: bench: support sse and fix prompt processing time / server: add tokens usage in stream OAI response (#6495)
341db214fa578e00b01e0884fc2725c9349608bdab51ff4d9f3d683f02ef8a12e04bfac84300c44bd3a2024-04-03T21:12:52+05:30Abhishek Gopinath KMissing tokenizer.model error during gguf conversion (#6443)
3421ff4d9f3d683f02ef8a12e04bfac84300c44bd3a076b08649ecc3b0e1c0709c2a086a63eddd1bf322024-04-03T23:24:31+08:00kaizauAdd OpenChat, Alpaca, Vicuna chat templates (#6397)
343057400a3fd457f4f214684eeb171444663b47a23b75c38166cf0c66793a32d5c3d6cb69929dd083d2024-03-29T08:23:22+01:00Daniel Beveniusllama : remove redundant reshape in build_kv_store (#6369)
344b75c38166cf0c66793a32d5c3d6cb69929dd083dbfe7dafc9cf96b9a09ead347fed9a547930fc6312024-03-29T08:15:00+01:00Pedro Cuencaconvert : allow conversion of Mistral HF models (#6144)
345557410b8f06380560155ac7fcb8316d71ddc983755c1b2a3bbd470e9e2a3a0618b92cf64a885f8062024-03-26T10:46:41-04:00compiladellama : greatly reduce output buffer memory usage (#6122)
346e097633f63fdd26d492844f7eff056e4083fd9ebd25b1c31b07c3675443a55a828dd58cfef5a241c2024-03-26T13:32:19+01:00Pedro Cuencaconvert-hf : fix exception in sentencepiece with added tokens (#6320)
34794d1b3b4119209efcdd08df0dceaecbd1fe7f85c95562175f83a49755ff6fd3bad09409417c8e6f92024-03-23T18:48:02-04:00Jared Van Borteluse _wfopen instead of fopen on Windows (#6248)
3481d0331c12a2f2a6296b471232bd4e66fbf06e6a1dba1af612926cbd4ebe2d876277af1e3305177e02024-03-22T19:47:14+01:00Kawrakowquantize: options for output and token embedding tensors qtype (#6239)
349bd60d82d0cc8b6852ec535495a5042dbdf05de246c0b287748327741b113d7d6018b68c63039b1c52024-03-20T01:33:49-04:00Jared Van Bortelserver tests : more pythonic process management; fix bare `except:` (#6146)
350b0bc9f4a9da7c19f4779106ea83b23feca7475664755afd1cbd40d93c017e5b98c39796f523453142024-03-15T09:22:24+01:00slarenllama-bench : use random tokens to improve accuracy with mixtral (#6069)
351044ec4b2a567f649459ccd20af2f387c784faa5177178eedc83d49f31bf757d8e12315d76460be782024-03-14T15:14:14+02:00Georgi Gerganovembedding : add EOS token if not present (#899)
352f30ea47a87ed4446ad55adb265755dc9102956a2d8fd0ccf6ac8b07791ffd1575eed436930854ae32024-03-13T18:54:21+01:00slarenllama : add pipeline parallelism support (#6017)
3537ab7b733bb48250b2df26c12b00256ef42c76932d9f65c97c3dc3aa6fa27470b8c6e69b437ec1a272024-03-11T04:03:17+08:00Deanandroid : fix utf8 decoding error (#5935)
354621e86b331f8b0e71f79fd82a4ae1cd54c3e439677d1ac7e00bf049b9f2bba1b5a310a78318c49c42024-03-09T23:41:49+01:00Pierrick Hymbertserver: benchmark: chat/completions scenario and other llm servers comparison (#5941)
3550db32beaf09d90b8959d3d0cc493ed1e456853538a3012a4ad08112bb3dc3f1399afec4e93780c442024-03-09T11:16:53ZAlexey Parfenovserver : fix passing prompt as tokens (#5955)
356c2101a2e909ac7c08976d414e64e96c90ee5fa9e515f7d0d4fce41c752fc253acf30707c3be2531e2024-03-08T17:31:00-05:00compiladellama : support Mamba Selective State Space Models (#5328)
357515f7d0d4fce41c752fc253acf30707c3be2531e76e868821a94072fbc87cb1fcca291694319eae82024-03-08T10:53:37-05:00compiladellama : fix quantization of shared token_embd (#5944)
35876e868821a94072fbc87cb1fcca291694319eae8e457fb3540e0aaec47cfde0abf784c213f9216ee2024-03-08T12:25:04+01:00Pierrick Hymbertserver: metrics: add llamacpp:prompt_seconds_total and llamacpp:tokens_predicted_seconds_total, reset bucket only on /metrics. Fix values cast to int. Add Process-Start-Time-Unix header. (#5937)
359af37fd8b30e37ccbffdd82e6f48559e2fb7ce7dd581ed5c4fe3a8909aaa8313633ac443f471ba7552024-03-08T12:40:02+02:00Georgi Gerganovserver : fix EOS token detection with disabled cache (#5938)
3602002bc96bf2cbf5ab981a17d7e994d817c9801f5ceca1aef0738b57951cd12c603c3477e75312dec2024-03-07T11:41:53+02:00Georgi Gerganovserver : refactor (#5882)
36121b08674331e1ea1b599f17c5ca91f0ed173be316a87ac3a52668e117d97bcea07b529c93188b3032024-03-05T16:08:35+08:00Neo Zhang Jianyu[SYCL] fix mul_mat fault in CI/unit-test (#5862)
36229ae62d2ae163e2b68aa0ad3bf2ab4636de0c957e0843afe1b37890b631bc7d3d2da2ed36c862b912024-03-04T22:31:20+02:00Georgi Gerganovllama : fix embeddings (#5796)
3635a51cc1bb4592f0d71f9af89cd08b11a066ba44767be2ce1015d070b3b2cd488bcb041eefb61de722024-03-04T02:57:20-05:00DAN™main : support special tokens as reverse/anti prompt (#5847)
3649731134296af3a6839cd682e51d9c2109a871de54a6e2d6142ab815c964924896891e9ab3e0506322024-03-02T22:00:14+01:00Pierrick Hymbertserver: tests: passkey challenge / self-extend with context shift demo (#5832)
3654d4d2366fc9c54d4a275065cfe9299c6cf7c5b78c7a0ad8ec9ebb5ddb1c1c80c82f2ee041c525d472024-03-02T12:27:26-05:00Jared Van Bortelconvert : automatically fall back to HfVocab if tokenizer.model doesn't exist (#5821)
366177628bfd85565070916ad66a5ac4071ee0527d86c4416868df2e5455da7d20547f62bcf9735ba8e2024-02-28T02:51:11-06:00Douglas Hanleyllama : improve BERT tokenization (#5740)
367f7625019c51ca437a5840576d92362cfa710e4a2abbabc5e51d0d4656b438aec10b7fae9479ef37d2024-02-25T13:43:50-05:00compiladeserver : fix crash when system prompt is bigger than batch size (#5714)
368525213d2f5da1eaf4b922b6b792cb52b2c613368fd43d66f46ee3b5345fb8a74a252d86ccd34a4092024-02-24T12:28:55+01:00Pierrick Hymbertserver: init functional tests (#5566)
36915499eb94227401bdc8875da6eb85c15d37068f796633eeca1265ed03e57230de54032041c58f9cd2024-02-22T17:05:23-05:00Jared Van Bortelmpt : do not duplicate token_embd.weight on disk (#5670)
37096633eeca1265ed03e57230de54032041c58f9cd847eedbdb2d1ebf14ef56eb507d4b4b9755109082024-02-22T23:23:46+02:00Georgi Gerganovgemma : use more bits for the token_embd.weight tensor (#5650)
37173122473ffd73030146276dbb85da7c8021a3ee40d4177126b0556e202efb85bf3f768be810764002024-02-15T14:14:37+01:00Michaël de Vriesfix(gguf-py): special tokens are no longer skipped when add_<token>_token is set to false (#5487)
372c4e6dd59e45ef7b14f7763fb073b517395dc176c037259be689353081e7bae3c1ab4ab18e7fbe8c92024-02-13T18:18:16+02:00Aarni Koskelallama : allow raw byte in SPM vocabs; don't crash on nl 404 (#5478)
373cf45252a7cfcb998bade46a886e20477cecc538a03bf161eb6dea6400ee49c6dc6b69bdcfa9fd3fc2024-02-13T15:14:22+02:00Georgi Gerganovtests : multi-thread the tokenizer tests (#5474)
37449cc1f7d67de2da99f3ac185f9ff1319b7bf35f899b8b43d7b185a6483f28cf798a2d968b2e16ca72024-02-13T13:01:29+02:00Georgi Gerganovbert : add tests + fix quantization (#5475)
3752891c8aa9af17f4ff636ff3868bc34ff72b56e2597a336507ed9b971d72262bec7e2b8b7016a054a2024-02-11T10:21:38-06:00Douglas HanleyAdd support for BERT embedding models (#5423)
376684780141a08200ec98eba3e982dbafd1d0b500085910c5b30f6e268321be8df044f5528a6efac522024-02-11T13:38:14ZAlexey Parfenovserver : allow to specify tokens as strings in logit_bias (#5003)
3770ef46da632c32faa1a538e5dc180994e8bbb46e1ee1628bdfea8b0079fed0140ac2f00ef1b465b572024-02-07T02:17:25-06:00Xiao-Yong Jinllava-cli : always tokenize special tokens (#5382)
378316c7faf7740fa98ea68f1445f4505810f706b9ef3e2b4fa3f81a410ecb7dec929c259ef8d8dbb7d2024-02-07T14:15:56+08:00runfuturellama : add MiniCPM support (#5346)
379906cff55c2848fda091d888a1585915ec0c9ea9e098f6d737b65134cf220d12b9b706e8cfc5e46102024-02-06T07:47:22+02:00Georgi Gerganovpy : handle byte tokens in `get_token_type` (#5341)
380e1e721094d8169636d55f68efe37f222cd3f0677128dcbd3c9c4b12f42b560a4430427d7b28286282024-02-01T23:20:13-08:00Ian Bullllama : fix memory leak in llama_batch_free (#5252)
381e6f291d15844398f8326940fe5ad7f2e02b5aa564003be0e5feef320f3707786f22722b73cff93562024-01-30T20:17:30+02:00Georgi Gerganovserver : fix context shift (#5195)
382e76627bcce9f77adb6034cb127b7ec93d4287b69fbe7dfa53caff0a7e830b676e6e949917a5c71b42024-01-29T18:24:19+09:00Sang-Kil Parkpy : improve BPE tokenizer support (#5189)
3839241c3a2ace544aef708334e54bbdddb90208ee8b2b2bf988c098851b4f3831f0cf38394bff751212024-01-28T09:59:49+01:00Johannes GäßlerApply min_p to unsorted tokens (#5115)
38444879ee885f48ecf4675dd216b373dce0a6f36909ecdd12e95aee20d6dfaf5f5a0f0ce5ac1fb27472024-01-23T15:17:20+02:00KawrakowAdditional KL-divergence statistics (#5081)
385d6bd4d46ddb6926087c11e0f6633ab1c81da58c3152d9d05e097e35f1cac21262946d57faec7542a2024-01-22T06:21:52-05:00compiladellama : support StableLM 2 1.6B (#5052)
3868b20858e5e9c44b99b4b31ae9c40b8f20d01d94f57e2a7a52a819883f40dada8a2edc24ecf48186b2024-01-19T10:45:06+02:00Georgi Gerganovperplexity : faster Winogrande via batching (#5024)
387ad19812cda4062c9f154ef16315df41fbe6a770a682986a08eb5cb04865d2e713449f17304d266d82024-01-18T15:33:01+02:00Georgi Gerganovperplexity : faster HellaSwag via batching (#5017)
3884f4bf35f46600441dec2f941e667291eeb9a18d82b3a665d3917edf393761a24c4835447894df74a2024-01-17T15:45:03+02:00Georgi Gerganovpy : fix missing added_tokens_dict for SPM and BPE vocabs (#4971)
389cec8a4847062fbd76253e3b085683f39d91e80d3334a835a1ccc8106a5fa355683a965efb1bfa24b2024-01-16T18:54:24+01:00Daniel Beveniusfinetune : add training data file to log message (#4979)
390f172de03f11465dc6c5a0fc3a22f8ec254c6832c2d57de525541247132e354f561ff48775fba5d852024-01-13T18:47:38+02:00Georgi Gerganovllama : fix detokenization of non-special added-tokens (#4916)
391df845cc982e7e2ea7b9900e29d55b15338faa78d6b48ed089377330cdb362970a51c1c89b6d857a82024-01-13T17:29:43+01:00David Friehsllama : minimize size used for state save/load (#4820)
392722d33f34ec74c6f7046109f936d0928ffe171bcc30b1ef39aeba497a943416d2897d69fee055b962024-01-14T00:09:08+08:00Yann Folletmain : add parameter --no-display-prompt (#4541)
393ee8243adaa9a9f51ff449213383874e49efe368f15ebe59210e7fd9817ff67f51fa1a5ee2d0042942024-01-13T14:16:11Zmakomkserver : fix crash with multimodal models without BOS token (#4904)
3943ca63b4538dfc78aaec88cd2c3e3f8417c1924e3b0377875488b33f7114138687d828da1de61775d2024-01-11T22:43:05+02:00Georgi Gerganovmain : disable token count by default (#4874)
395d8d90aa343c22fe01429d3540e47ded87e9dcb9d43f76bf1c362c067fce46bb8dcda0b64af8a95332024-01-11T17:22:34ZSomeoneci: nix-flake-update: new token with pr permissions (#4879)
39643f76bf1c362c067fce46bb8dcda0b64af8a95332f043328e3116724d15b915b5c6078e2df860a692024-01-11T16:14:52Zpudepiedjmain : print total token count and tokens consumed so far (#4874)
3976efb8eb30e7025b168f3fda3ff83b9b386428ad636e5a08b203542dca53cca4eaf172c5dc4bbc9912024-01-09T13:46:46-05:00Austinconvert.py : fix vanilla LLaMA model conversion (#4818)
398128de3585b0f58b1e562733448fc00109f23a95d8c5833031857c9e9ada61948bae894ab9c785f862024-01-09T05:02:05-05:00Behnam Mserver : update readme about token probs (#4777)
399dd5ae06405c5565b99889bdb3f168f4351252cfb668b31fc7d86245435ad6574e0e1126e734049e22024-01-08T16:02:32+01:00KawrakowSOTA 2-bit quants (#4773)
400c75ca5d96f902564cbbbdd7f5cade80d53c288bb96e80dabc6e73ff68b09b68947b1fc25883c50942024-01-06T16:12:59+01:00Daniel Illescas Romerollama.swiftui : use correct pointer for llama_token_eos (#4797)
4010ef3ca2ac62016c0c545de1c89dc2e3e130f4a99540938f8904707dd74cb3be18495f853b312e72f2024-01-02T15:48:49ZPhil Hserver : add token counts to html footer (#4738)
4020040d42eeb237197054cc7790df5776eacfa608e83e633c27efdf0eb0ba54249e784b0ea760b10072024-01-02T06:15:16-08:00Marcus Dunnllama : replace all API facing `int`'s with `int32_t` (#4577)
40365e5f6dadbba4b496bba27f573e473c66b446496ea5497df5d138c83b2b0ca70aefdc4b1175c10012023-12-28T11:20:00-08:00Justine TunneyFix OpenAI server sampling w.r.t. temp and seed (#4668)
404f56d6077d0c37e6606ac0a4fa3169de70593acfedc68f0054cd279cddddb0cae0c9ef4f9cbaa512a2023-12-27T17:37:25+09:00wonjun JangAdd byte token type when tokenizer.model is not exists (#4641)
4057082d24cec35e9ce9147535a2224dfc67ee0a78cba661751322a7c201fd3bef71af077c5aebfaa2a2023-12-22T17:05:56+01:00LeonEricssonlookup : add prompt lookup decoding example (#4484)
406880e352277fc017df4d5794f0c21c44e1eae2b8466f35a2f48e1965a13835a523e677223dbf148be2023-12-21T18:07:34+01:00howlgerpy : open merges file as 'utf-8' (#4566)
407b9e74f9bca5fdf7d0a22ed25e7a9626335fdfa483c04bf6da89eaf4c7d317e0518f0687dfcbf2de72023-12-18T17:27:47ZEbey Abrahamllama : add phi-2 + fix NeoX rope + ggml_mul_mat_set_prec (#4490)
4082994f0c5a2e8c96955b422dedc93ec2595d16b82b1306c439490c7fa4ec33594500d980d1e9e15e62023-12-17T19:39:02-05:00Jared Van Borteldecode : fix logits_valid for legacy API (#4516)
409f7f468a97dceec2f8fe8b1ed7a2091083446ebc7919c40660fd27157b391b5832d2a577d5afef4cb2023-12-17T10:45:46-05:00Jared Van Bortelgguf-py : fail fast on nonsensical special token IDs (#4489)
410873637afc7924f435ac44c067630a28e82eefa7b0353a1840134b24b07ab61fd4490192f28c4db6b2023-12-14T17:09:34+09:00wonjun Jangconvert : support loading vocab from fast tokenizer config (#3633)
411948ff137ec37f1ec74c02905917fa0afc9b975144d98d9a65665eee3838cef936641f640e3f5b6492023-12-13T23:57:15+04:00shibe2server : fix handling of characters that span multiple tokens when streaming (#4446)
412799a1cb13b0b1b560ab0ceff485caed68faa8f1ffecac45658a99eddc4d6e36ba0310ca8f87a77f02023-12-13T13:04:25+01:00slarenllama : add Mixtral support (#4406)
4136391817cd19a4507c6c941a1fd08756268662b2dd9d4cfef64ea416dd66632173787d03ffb180cc72023-12-12T04:25:57-05:00crasmllama : document logits_all deprecation (#4418)
414e18f7345a300920e234f732077bda660cc6cda9cfe680e3d1080a765e5d3150ffd7bab189742898d2023-12-09T16:29:27-05:00Xiang (Kevin) Ligrammar : revert the replacement of llama_token_to_piece with id_to_token (#4396)
4155f6e0c0dff1e7a89331e6b25eca9a9fd713240695aa365d88fdb8fdd430ef3fc141c7a5fd37c35022023-12-05T10:55:12-10:00Marcus Dunngrammar : pre-computed pieces + reserve mem + less string copies (#4330)
41623b5e12eb5a76489b4c3ee22213a081da68b1809d208995c6da66f252d4054c1c5a90eb8ccb7a2f72023-12-04T17:04:21+01:00Daniel Beveniussimple : update error message for KV cache check (#4324)
4175c9f90cba1cc6b0a2a7d19ee5dcb73cad6331d304fa44e84adb4c78e1885694cc3513982d4af2b082023-12-04T22:43:45+09:00Miwa / Ensanswift : fix prompt tokenization logic (#4321)
418b220222a64ce760bfbec9c770f11db3ec6a6abb6511f52c334e37033f9c9de07b98fca4abc9470bd2023-12-02T03:19:45+09:00Miwa / Ensanswift : fix token_to_piece implementation (#4278)
419b18c66ca6eee4fe0465cff5042daf05005dc9ab2f4d973cecb7368c985720ba9100ae6abba14806d2023-11-30T22:43:08+01:00Daniel Beveniusllama : fix alignment of general.name in print meta (#4254)
4208406b0924bf323f37d536dee8b8165c1f3d9d11db38a16dfcff88d547f78f52d1bea31b84a05aff72023-11-28T10:32:03+02:00Georgi Gerganovggml : re-enable BLAS for CPU when src0 != F32 + remove redundant full offload checks in llama.cpp (#4240)
421922754a8d60080e956891f6cee1fb03aa48d57c622da05536ff4ad963080773bef1fb839fdab95d32023-11-26T20:33:07+02:00Georgi Gerganovlookahead : add example for lookahead decoding (#4207)
422f837c3a992b2b6146936cb120871a8cf9d0e38573014b5415d08e3dff961da6eea835b9760a701b82023-11-25T08:58:23-08:00Marcus Dunnllama : grammar `reserve` space in `decode_utf8` (#4210)
423af19d3573481d409b3c4e55494810eb1f65a9aaee9c13ff78114af6fc6a4f27cc8dcdda0f3d389fb2023-11-25T11:29:06+02:00Georgi Gerganovserver : OAI API compatibility (#4198)
4242568a4bf548d7392e9c78c008b33b4c11d53fe95b35f3d0def3efde92ed465d92a267430d957e87d2023-11-24T18:25:10+09:00eastrivermain.swift : fix eos checking (#4197)
42540a34fe8d034bd484efd79ccbb95059ca6308dcbdae06c06e5c6232ae2be4d567dd5101e1e96c8142023-11-20T03:50:04-06:00Branden Butlerspeculative : fix prompt tokenization in speculative example (#4025)
42635985acffaab1eb4ffcbc22c86063bc630f24a89e937066420b79a757bf80e9836eb12b88420a2182023-11-19T18:50:49+02:00Georgi Gerganovgitignore : tokenize
42728a2e6e7d476717881be6eb9e2d3331342cec57b0b5c3b04572a05f80163a365070fb377a837ac272023-11-18T14:48:17-07:00Kerfuffletokenize example: Respect normal add BOS token behavior (#4126)
4285ad387e994dde77a47ec547a4a65f7611dc325f42fa02b4b3d86182381311c98b75065ee1b7c29302023-11-17T18:01:38+02:00Georgi Gerganovtokenize : fix trailing whitespace
4292fa02b4b3d86182381311c98b75065ee1b7c29302ab0707acbf3a3ca9e5bc5959c7920c22eba22572023-11-17T17:36:44+02:00zakkorexamples : add tokenize (#4039)
43091f6499393d2d999331fbfdba47a7f8b9f913f0d8da46278e1a57107591653275f8e03a281de94f02023-11-16T19:14:37-07:00KerfuffleRespect tokenizer.ggml.add_bos_token value when tokenizing (#4040)
431a6fc554e268634494f33b0de76f9dde650dd292f1cf2850d52bb027aa215e039ed9a0c61beeef8d32023-11-15T11:34:47-05:00Jared Van Bortelllama : restore prefix space in llama tokenizer (#4081)
43234b0a082074b073eb14c2bd93c0c070e20ddcd164a4fd3eefad5bd17ab6bcd8e2181b4f62eae76cf2023-11-10T22:04:50-07:00Kerfufflegguf-py: Refactor and allow reading/modifying existing GGUF files (#3981)
433238657db2364cfb728c694470a4a81702afea76007178c98e1b61a5e2af39d347add12e7eb9e08e12023-10-31T14:44:49-05:00kalomazesamplers : Min-P sampler implementation [alternative to Top P/Top K] (#3841)
4346e08281e588bbba1a5d180290a94a43f167f3a1a2046eb4345e62c4575b3cdc0115a51db89f3fb702023-10-29T11:31:40-06:00KerfuffleExtend llama_kv_cache_seq_rm to allow matching any sequence (#3843)
4358a2f2fea2914aaa3f4b2f82800c7de15f15bdb09bd6d9e205982b34e0ba2c3b22bbf31a1ef1a1bb52023-10-28T15:25:15+03:00Georgi Gerganovconvert : ignore tokens if their IDs are within [0, vocab_size) (#3831)
436ee1a0ec9cb367ba41d138134795cbbbe93d2bf1c177461104b454163473dced2a5038f4e016cdb7e2023-10-28T14:23:11+03:00Georgi Gerganovllama : add option for greedy sampling with probs (#3813)
437daab3d7f45832e10773c99f3484b0d5b14d86c0c469c9addef75893e6be12edda852d12e840bf0642023-10-24T09:17:17+02:00GalunidAdd more tokenizer tests (#3742)
43869a6735087c3634963c642fd69f0851ac479cd785be6c803fa5378f62a1590f3ad8c6b64c7c0c2ce2023-10-23T21:46:00+02:00GalunidUpdate special token handling in conversion scripts for gpt2 derived tokenizers (#3746)
4395be6c803fa5378f62a1590f3ad8c6b64c7c0c2ce6336701c9378c23c85d1c0e464b663ca2bbb8e602023-10-23T12:40:03-07:00Marcus Dunnllama : remove token functions with `context` args in favor of `model` (#3720)
440438c2ca83045a00ef244093d27e9ed41a8cb4ea99e70cc03229df19ca2d28ce23cc817198f8972782023-10-22T22:53:08+03:00Georgi Gerganovserver : parallel decoding and multimodal (#3677)
4419e70cc03229df19ca2d28ce23cc817198f8972785a42a5f8e8a86da9ac88008d748cf232a83aa0e12023-10-22T21:21:42+02:00goerchAdd test for MPT tokenization (#3728)
442a5e7dbd6141128bfa3c40a19c2945a181df625d3d3956aea53369455008159cc405ed4c4969766922023-10-22T12:14:56-06:00Kerfufflellama : validate special token ids are in range when loading GGUF model (#3635)
443d3956aea53369455008159cc405ed4c49697669222c69a27945e7acf9690dd3210d316f22182751c2023-10-22T20:09:51+02:00vvhg1main : escape prompt for cfg_negative_prompt and consecutive inputs in main with interactive (#3623)
44440e5ce054f4c4fa555e4510ea5f760bb29185332a5e8c1d8c71f01d98ae2ec63a57c118664f9764d2023-10-11T21:30:06+04:00shibe2CLBlast: Fix temporary buffer size for f16 conversion (wsize)
4451a159553f921a9209fed8c714494e57b3649f232281ef73c258cc1eebec8a64264240432d5878c4b2023-10-17T17:11:01+02:00staviqtokenizer : special token handling (#3538)
446940efa95fec0b8a98c226a889d2ad839dfeeae0d11bff290458f12f020b588792707f76ec658a27a2023-10-16T23:58:00+03:00Georgi Gerganovllava : fix tokenization to not add bos between image embeddings and user prompt (#3645)
4472a4bcbacead886996f175f33479d1d874a3e577f424b6381c4daeed62e6600e0402e72f39845b58d2023-10-13T12:33:16+02:00Daniel Beveniusllama : remove n_threads from llama_decode_internal (#3614)
448233fc1c69f6f415f35363e18a755f9610e89161bc5b49360d0d9e49f32e05a9116e90bd0b39a282d2023-10-10T18:59:52+02:00goerchMinor improvements in GPT2 tokenizer (#3567)
449f5f9121de140eff558f13b5c5e78a3a3b6b9437711ea5c7d96f2c28e1c99659e08ec0a44574056e22023-10-10T09:50:23+02:00Jan Ploskillm : add MPT support (#3417)
45011ea5c7d96f2c28e1c99659e08ec0a44574056e295bd60a0a69f57e9a2ff1269667ea484a1a9bb402023-10-10T09:31:21+02:00vvhg1infill. : fix tokenization (#3508)
4513a716b4dae545c3db307594fbc509a95d3e21b6e1faaae8c2bdc4a21302e367e0754c3fe74a8113e2023-10-07T06:57:01+02:00goerchFix for #3454 (#3455)
4529ca79d5cbbc8d43f2bff951404b6a40ff1ee37880c731ca4039ccff86ffab90eaae4ca98037c44962023-10-06T10:10:13-06:00Kerfufflekv cache slot search improvements (#3493)
45397af49fa395df77e4c18af0e1655b2fee67c968616820a5a0d885113f21021ce934f0b0027b9d69a2023-10-06T07:44:24-05:00Jhen-Jie Hongserver : reuse llama_sample_token common util (#3494)
454e8b8d32e8663ffc55a02c9721af3a5190382cbb08f3a642ec1d878b2d0a0d15e3a4277f522790d4c2023-10-05T09:02:55-05:00Jhen-Jie Hongserver : fix incorrect num_tokens_predicted (#3480)
455ac2219fef34eb5b713c286c34c6e4162c39c8f3b48be797ffbd80b062f55778e09e97180eb25d2ab2023-10-03T21:04:01+03:00Georgi Gerganovllama : fix session saving/loading (#3400)
456ff5a3f0c09dfa0a8e0bf76d1748df5c6dee0e8ff1c84003c08027f5d3a4cb876f51d6b6224a34d0e2023-10-03T09:16:26+02:00goerchWork on the BPE tokenizer (#3252)
45716bc66d9479edd5ee12ec734973554d4493c5dfa0512d66670de3f650c579519833c085014b0f2002023-09-28T21:42:38+02:00slarenllama.cpp : split llama_context_params into model and context params (#3301)
4580e76a8992c8200237bbc6471a53fb8796b3872f72db94d98eda56982d80238840b0652b4137a2a842023-09-28T20:40:11+02:00xaedestrain : finetune LORA (#2632)
459ecf90b1a5114034bc0939b3968f549fe4d63cf6d2619109ad57d7a75388a9cce51e5da645410d92e2023-09-28T14:30:15-04:00Cebtenzzregguf : make token scores and types optional (#3347)
460ec893798b7a2a803466cc8f063051499ec3d96f745855b3f1c7bdd0320aa632334d0b3e8965c26c42023-09-28T19:04:36+03:00Georgi Gerganovllama : custom attention mask + parallel decoding + no context swaps (#3228)
461b08e75baea294e366628b898e85c0bd359b58115e6616cf0db2b63189fc34d0076f654af9adecdf82023-09-16T13:41:33+02:00goerchFixing the last deviations from sentencepiece indicated by test-tokenizer-1 (#3170)
46235f73049af6c676a106a5a990a819ae0bc3fcd7d71ca2fad7d6c0ef95ef9944fb3a1a843e481f3142023-09-14T09:14:44-07:00Leng Yuespeculative : add heuristic algorithm (#3006)
46371ca2fad7d6c0ef95ef9944fb3a1a843e481f3141b6c650d16048d6427dd502a9627e728372658442023-09-13T15:19:44+02:00goerchwhisper : tokenizer fix + re-enable tokenizer test for LLaMa (#3096)
46415b67a66c2f2d6032415b28a699b5131962318f1be8c9c245bd129ebabb80e0a7a8dd7daeb4d30af2023-09-07T15:52:34+02:00slarenllama-bench : use two tokens in the warmup run for prompt evals (#3059)
465c4f496648c1e32efeb714200e7eae7fc7cfbb223fec2fb19e4229aac58c98171c46e77144b99f8a32023-09-07T15:49:09+03:00Georgi Gerganovmetal : fix kernel_norm (fixes Falcon on Metal) (#3057)
466921772104ba2219bfdc2b2980d05ebc0aa0c92a42ba85c8609309a59d49c45ab43c31800b7ba141c2023-09-05T08:46:17+03:00Georgi Gerganovspeculative : add grammar support (#2991)
46737301347767d555d0a66c043ce4ef6ead8e61c55d9151e6f570eb20bfd54427bd8a337d9b1a080182023-09-03T19:18:09+09:00opparcollama : fix bpe tokenize from byte (#2889)
468afc43d5f82588d2ed71ea104e8262f5e5da139806460f758dbd472653296044d36bed8c4554988f52023-09-03T11:48:49+03:00Aloncov : add Code Coverage and codecov.io integration (#2928)
469cff7b0bf07cb46e1ad4fd199f6bdeb538925c8c4340af42f09a80e32f4998857b4f0543e411245252023-09-02T23:52:13-06:00Kerfuffleconvert.py : BPE fixes (#2938)
470950929442070874d45561d2a4b68b010457767de35092fb54712d032860f3976a6fc1ae1f84a4a282023-08-30T12:42:51+03:00alonfarajmake : add test and update CI (#2897)
471dc07dc492ef9640bbb82904d7c7679f7bdcf6d76ad9ddcff6ef322db5cf13785bd7c856b610d242e2023-08-30T02:25:50-06:00Kerfuffleconvert : various script cleanups/fixes + merges and special token handling (#2842)
4728341a25957b319a03d4a811176cd5ad7f2b0fbd4849408957c687cde4ab32c147107f643fc55130b2023-08-30T08:29:32+02:00staviqmain : log file (#2748)
473fa3582f509a2715e80a473e79f88dcd1ebff44c2e37e69dcc3d52f21222a63cafed2a71b3f6b53c62023-08-29T23:55:45+03:00KawrakowTell users attmepting to run perplexity with too few tokens to use more (#2882)
474e37e69dcc3d52f21222a63cafed2a71b3f6b53c653885d7256909ec3e2176cdc2477f3986c15ec692023-08-29T23:55:03+03:00Kawrakow10X faster BPE tokenizer (#2876)
47544c117f41ee01c5ac8fb86bba041f08d8b87b46d43033b7bb4858da4f591715b3babdf906c9b7cbc2023-08-28T21:51:47+02:00xaedestrain : mem usage and other improvements (#2439)
476230d46c723edf5999752e4cb67fd94edb19ef9c7463173a6c0ff353055eb90665794884c888c790f2023-08-27T15:13:31+01:00Olivier Chafikexamples : update llama2.c converter to read vocab and write models in GGUF format (#2751)
477463173a6c0ff353055eb90665794884c888c790feaa13a48ff4136f01c1cdb79cacd61b67ec530952023-08-27T16:50:33+03:00Kawrakowllama : speedup tokenization (#2831)
478edd4c1481708fcd788b0e423268304fd26e2b1251591e2e590762011b43b10a9b6e04f13f98f2aa52023-08-27T14:19:19+03:00Georgi Gerganovllama : more tokenizer fixes (#2810)
479c1ac54b77aaba10d029084d152be786102010eb2730d9c681e339b76407659344e5a2cd50af7d7d52023-08-26T16:11:45-07:00Bruce MacDonaldserver : add `/detokenize` endpoint (#2802)
48072f895c923ba98b8f2af294440206f35915c050150526f37eba0b28336700890242ff282b949cd832023-08-26T14:12:56-04:00Dr. Tom Murphy VII Ph.Dmain : fix bug (penalize_nl=false doesn't work) + suppress warning on mingw (#1528)
4812ba83c8685177faea3399db9564f9c52df75c366bae5c5f679e043371bc2b4dffff8d4964d6cb9532023-08-26T13:45:53+02:00klosaxFix spm whitespaces (#2806)
482c82742ac9cd96fd34aa961978805c1d8a361d58928b2c996ca0ab90a5669946084f13443ec98e2412023-08-25T11:18:48-04:00Matt Pulverllama : add llama_beam_search() (#2267)
48329674ab4e847fcaba60cc6558f0d46d5f74ae2795439a0ab57c16b556ffa91a0953df5e46b1e7fb42023-08-25T18:32:45+08:00Jhen-Jie Hongserver : display token probabilities in the UI (#2489)
4842e5f70a25fc4576e9ed78603fe493eb7702c37a3d0f77b1353fc820d1ff1e6b87bc6bedde315938d2023-08-24T14:49:30-07:00Marcus DunnAdded `enum` to `llama_token_get_type` return type (#2774)
485cf658adc832badaaa2ca119fe86070e5a830f8f6a192860cfec89a38d59a943623bf595b1fe4495b2023-08-23T23:08:04+03:00Georgi Gerganovllm : add Falcon support (#2717)
48695385241a91a616788a3bb76d12c9b7b2379ca2d335acd2ffd7b04501c6d8773ab9fcee6e7bf86392023-08-23T20:33:05+01:00Olivier Chafikexamples : restore the functionality to import llama2.c models (#2685)
4875290c38e6e9b66ee2b543e560e301c1a1a90929ccc34dbda9681418a2b18382446b90cdcec398d822023-08-23T16:46:03+02:00klosaxmain : insert bos if no tokens (#2727)
488b8ad1b66b23f9b2e6e4531e9a62753323036a556f5fe98d11bdf9e7797bcfb05c0c3601ffc4b9d262023-08-23T02:12:12-05:00Xiao-Yong Jinserver : allow json array in prompt or content for direct token input (#2306)
489777f42ba18b29f25c71ff8de3ecf97b8017304c046ef5b5fcf4c366e1fb27726b6394adbbf8fd0ea2023-08-22T17:39:39-06:00KerfuffleImprove handling of special tokens in GGML to GGUF converter (#2725)
49046ef5b5fcf4c366e1fb27726b6394adbbf8fd0eac63bb1d16a70c03440671b76954bb767513cead82023-08-22T23:10:42+02:00goerchllama : fix whitespace escaping in tokenizer (#2724)
4916381d4e110bd0ec02843a60bbeb8b6fc37a9ace9dadbed99e65252d79f81101a392d0d6497b86caa2023-08-21T23:07:43+03:00Georgi Gerganovgguf : new file format with flexible meta data (beta) (#2398)
492cb1c0727bd59803b439b6a3af121c99e6393ff3d9e232f0234073358e7031c1b8d7aa45020469a3b2023-08-21T11:11:31+03:00KawrakowHellaSwag: split token evaluation into batches if needed (#2681)
493220d9318647a8ce127dbf7c9de5400455f41e7d881844fbcfd93a162b7aeaea9e4f2ab1358f7f97e2023-08-02T16:21:11+08:00ldwangreadme : add Aquila-7B model series to supported models (#2487)
494ee1b497c985f61d6ec519c39fcfed78a3c6f1d06d73b8d48b45d6e2c0ae9bb8c39623c4024adc2752023-07-29T02:10:05+08:00eric8607242llama : support more diverse tokenizers? (#2420)
495fce48caf9a6b9930eee9e2a5971428cdff403ba8875086bdb95ce1f3294439811536533199e3b5792023-07-25T21:22:09+08:00ldwangconvert.py : support bpe tokenizer (#2228)
49684e09a7d8bc4ab6d658b5cd81295ac0add60be782f9cf974a066ac0e03fbb235d834b01b0164d7432023-07-23T23:58:10-04:00Evan Jonesllama : add grammar-based sampling (#1773)
497355c80f49e32b0b15c0a457f3bad380e57f5b9ac83a00ce69bef9124c0702424a012ea799128b77d2023-07-23T06:16:48-05:00AustinMrozexamples : simplify vim plugin (#2327)
498f257fd255044decffad93dee2502875ce66ad80c7ee76e45afae7f9a7a53e93393accfb5b36684e12023-07-05T03:06:12+09:00jwj7140Add an API example using server.cpp similar to OAI. (#2009)
499cc45a7feb8412e84ff292207621412fffc0d3d5155dbb915cc2a95048f56e667b09dfad38d8404212023-07-04T02:43:55+08:00Howard SuFix crash of test-tokenizer-0 under Debug build (#2064)
500d7d2e6a0f0c74f7a570dae384dfff371ac744d2a46088f72318981341a2d646f12f6eee6aec06d652023-07-03T05:38:44+08:00WangHaoranRobinserver: add option to output probabilities for completion (#1962)
501ca7c3f4da5d144d4cd1dd44903552e6ba49b8ec8b97ca431db35ec96a339a721acb1219c1dd78bed2023-06-19T18:14:09+03:00Kawrakowcuda : faster k-quants on older GPUs (#1930)
502b24c3049d96557c24782e4d32feaae65f47277af0ede372a51fd8160688e01b587582666c14e94e52023-06-18T17:41:26+02:00Johannes GäßlerAdded tokens per second to info prints (#1928)
503794db3e7b982fee37e3995db9c3a216a57ff65e35ddf7ea1fb42bac21026de2f77e0f9c069b922342023-06-17T07:53:04-04:00Randall FitzgeraldServer Example Refactor and Improvements (#1570)
504e32089b2c20b1b87b22912f4a8b93fe01647d5b92347e45e7bdb09c9a7d74b2c0bc86c2b65f0c3432023-06-13T21:04:40+02:00xaedestrain : improved training-from-scratch example (#1652)
50574a6d922f12ccfe16b0c265f43be8978c6f25e98e4caa8da59c1c97dc23fa336f4d726984a20560f2023-06-12T22:39:21+03:00KawrakowMetal implementation for all k_quants (#1807)
506fa84c4b3e80199a5683438f062009c031a06c4fa12b063f0ecf280e98028e444fc492ee6222cdcdc2023-06-11T08:19:17-06:00KerfuffleFix issue where interactive mode crashes when input exceeds ctx size (#1789)
507e9b66ee9829039d4ab54550d6222e42a0b31e52a4f0154b0bad775ac4651bf73b5c216eb43c45cdc2023-06-10T11:28:11+03:00Kawrakowmetal : add Q4_1 implementation (#1785)
50872ff5282bf0388c60821f504c4c8cc2b1f491aa60bf7cf1b296fc9fca05411b37afdf08a531487d22023-06-08T22:28:21+03:00Kawrakowmetal : add Q2_K implementation (#1762)
5090f291e1f65c1d68201e71ce99c89562a36686b6d8fc8179919a11738910db07a800f2b176f8adf092023-06-08T19:46:22+03:00Kawrakowmetal : Q6_K implementation (#1752)
5104161bdc04debb70bf5f275492b4d89fd9330087c0035858273ebe0694926bf4414d279f3e1cd109d2023-06-08T10:08:23+03:00Kawrakowmetal : add Q4_K implementation (#1733)
51199009e72f8072fa552eb02efee436be596c71cdd5220a991a5e92bddad9542267ab445a2c033681c2023-06-05T22:56:18+03:00Kawrakowggml : add SOTA 2,3,4,5,6 bit k-quantizations (#1684)
5122e6cd4b02549e343bef3768e6b946f999c82e8237e4ea5beff567f53be92f75f9089e6f11fa5dabd2023-05-22T23:33:24+02:000cc4mOpenCL Token Generation Acceleration (#1459)
513affc76edfdefa7b326f526e463cc65ff13fcfb92ea600071cb005267e9e8f2629c1e406dd5fde0832023-05-20T14:19:28+02:00Johannes Gäßlercuda : loading models directly into VRAM, norm calculation on GPU, broadcasting for ggml_mul (#1483)
5147694b52b9a206b93d59139c3c7c9b55da0f5aa5979e3efb0e97b65b6cc72cd9ee970fa8189ad79a42023-05-19T10:24:59-07:00Jason McCartneymain : make reverse prompt option act as a stop token in non-interactive mode (#1032)
515905d87b70aa189623d500a28602d7a3a755a4769f954edda935a70a14cf0cc45ecc7fe7d60cf3e4b2023-05-13T15:38:36+02:00Johannes Gäßlerggml : GPU-accelerated token generation (#1412)
516f954edda935a70a14cf0cc45ecc7fe7d60cf3e4bf048af0230ad5381bb20b9e3c1467ec6fc7debdf2023-05-13T14:56:40+02:00xaedesggml : implement backward pass for llama + small training-llama-from-scratch example (#1360)
517b9fd7eee57df101d4a3e3eabc9fd6c2cb13c9ca1b608b55a3ea8e4760c617418538465449175bdb82023-05-12T00:23:08+03:00Georgi Gerganovggml : remove bit shuffling (#1405)
518b608b55a3ea8e4760c617418538465449175bdb8cf348a60e0af3905acd1d297cb064b918265b7ac2023-05-11T10:10:19-05:00CRD716prompts : model agnostic DAN (#1304)
519f9a6364912fd0463fddfdbc9ef9f79fdc281570d95078cc554fe03d4512363c7e4dec963f0047c722023-05-08T17:41:54+03:00Georgi Gerganovllama : require first token to be BOS (#1303)
520018f2279f5fe3ef743bd8254b23ea8f0efae7e739411288271ab548216902a029f42a0a38ebcedb72023-04-22T02:27:06+08:00源文雨cmake : link threads publicly to ggml (#1042)
5219411288271ab548216902a029f42a0a38ebcedb78687c1f2581d059cd5b6a9502f89bd343566062a2023-04-21T11:18:09-07:00Alex Klinkhamermain : evaluate tokens in batches after swapping context (#1014)
5221bfc153e2f35ddd9d64b084e8d1a5e6fa57ad1c93d59769c3bb7e72c915646ddb1e239b1face19f52023-04-21T17:18:26+02:00Kawrakowggml : a faster version for Q4_1 x Q8_0 dot products (#1083)
523884e7d7a2bfd7325b107442d6758983f5886ed3d7cd5c4a3e9106151d48f328bb3c94c298a211f182023-04-19T20:10:08+03:00Georgi Gerganovggml : use 8-bit precision for Q4_1 intermediate results (#1047)
524723dac55fa2ba7adc6e3fc8609781d1ad03789060f07cacb05f49704d35a39aa27cfd4b419eb6f8d2023-04-14T00:03:03-07:00comexpy : new conversion script (#545)
525e986f94829bae0b9e66b326acbbba179931c84f1c0bb1d3ce21005ab21d686626ba87261a6e3a6602023-04-02T12:23:04+02:00Christian FalchAdded api for getting/setting the kv_cache (#685)
526cbef542879962fdc491656cd0c8cadd65a5f13569733104be5389ebb1ff05095eca2a70280cd875a2023-03-29T21:31:24+02:00Pavol Rusnakpy : cleanup the code
5272e17dfd80a473099dacc0f41c9146d233c6a597220a1a4e09c522a80e2a0db51643d25fa383260652023-03-23T15:22:47-05:00rabidcopyReplace EOS with newline to prevent context/memory being flushed by EOS in interactive mode (#333)
5288eea5ae0e5f31238a97c79ea9103c27647380e3793208cfb929c2323e5d2ac6bf354e278040e70ed2023-03-23T12:26:19+02:00anzz1Delete SHA256SUMS for now (#416)
529305ba6f0e6daa3796aad9dd18053a1945dd4cc584122dffff958cd137175b58f1f27c0913528d7ba2023-03-22T18:16:35+01:00tjohnmanDon't force immediate interactive without `-i` (#354)
53056e659a0b271436e24813a801640d015e7b0532840ea807a972ec7b5a426f034ebfa593b5e7a06ed2023-03-22T17:09:38+01:00Erik Scholzfix perplexity after c-api refactor (#390)
531f5a77a629bd0f37ae1696747633ab42a5530ec15da0e9fe90ccf6e73597eb19dd0cfc0a28363fb3b2023-03-22T07:32:36+02:00Georgi GerganovIntroduce C-style API (#370)
532353ec251a42491f5192c48561da4b444ef67f23c89d5d90f3b6d25f134da7a8e252c3432bffcf6742023-03-21T14:21:50-03:00Fabio R. SluzalaWe could use std::unordered_map over std::map (#305)
533486ae645fd3eda8b9d7413d5ff34fb65a3e337fb3ab3e6582f7320c2b6568c892fdfc8215caf7e6c2023-03-21T09:27:42-07:00Gary LinscottCompute perplexity over prompt (#270)
5346a612959e1b6c37b68b6b141329751a2902b1030d5f56a5e5a0069329a81f96460221e7afb1daddc2023-03-21T17:05:06+01:00tjohnmanCheck for reverse prompt by characters instead of tokens (#292) (#330)
535d5f56a5e5a0069329a81f96460221e7afb1daddc3bfa3b43b7319b71853bfc7d3cf4e9767c24bbc82023-03-21T17:04:43+01:00tjohnmanCheck for reverse prompt by characters instead of tokens (#292) (#330)
536e0ffc861fae5ac8b40ce973f822d03db02929d368f644a0a859938c787d329d27f98e03c58d7df272023-03-21T08:34:49-07:00Kevin KwokUpdate IPFS links to quantized alpaca with new tokenizer format (#352)
537eb34620aeceaf9d9df7fcb19acc17ad41b9f60f82e664f1ff413995506c9a54f3a8d5b8c64e37a912023-03-21T17:29:41+02:00Georgi GerganovAdd tokenizer test + revert to C++11 (#355)
5386b6d5b5024faaf82019d08cde5e8a9d69c6ca316a791a68b613b162c88a83f5f0225223bc167c7622023-03-21T03:33:10+08:00Qingyou MengFixed tokenizer.model not found error when model dir is symlink (#325)
5390f1b21cb90ac6b84a9af70cafb8e13b5389e3b32074bea2eb1f1349a0118239c4152914aecaa1be42023-03-20T18:05:20+01:00Bernat VadellDocker - Fix publish docker image in GitHub Registry (#235)
540074bea2eb1f1349a0118239c4152914aecaa1be45cb63e2493c49bc2c3b9b355696e8dc26cdd03802023-03-20T03:17:23-07:00Mack Straightsentencepiece bpe compatible tokenizer (#252)
541368d0c8a9ebae16a20e1c8971b21ee888bdefad550fae10d0339f2bd639f69dd679c0201d939a2652023-03-19T19:31:17+01:00tjohnmanRespect the maximum number of tokens in interactive. (#298)
542d7def1a7524f712e5ebb7cd02bab0f13aa56a7f96f61c18ec9a30416e21ed5abfb1321bdb14979be2023-03-18T17:10:47-07:00RonsorWarn user if a context size greater than 2048 tokens is specified (#274)
543a81d0c2a171a4446e6a21a3ec74a0c0768d71184b2de7f18dfbb93463eeb5b4392117bbe82d5bd1b2023-03-18T04:17:19-07:00Gary LinscottFix n^2 loop in tokenization (#254)
544c9f670a17755311aa28c411f5c7f3c8c054347704f546091102a418ffdc6230f872ac56e5cedb8352023-03-17T21:05:58+01:00thementImplement non-greedy tokenizer that tries to maximize token lengths (#242)
545956dfda8ad8cea7961e22e0384bbc315bf79aed2113e685d18ac4edb20f647fd34b000941556f6a62023-03-15T12:37:50-07:00RonsorUse `tokenizer.vocab_size()` instead of hardcoding 32000 in convert-pth-to-ggml.py (#142)
546460c48254098b28d422382a2bbff6a0b3d7f7e17c80e2a8f2adeda202cbffe76ef800f134e51f03f2023-03-13T00:35:51+01:00Matvey SolovievFix token count accounting
Reference in New Issue View Git Blame Copy Permalink