Looks like no quantized options with llama.cpp? https://github.com/ggerganov/lla... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		xrd on June 5, 2023 \| parent \| context \| favorite \| on: Open-Source Data Collection Platform for LLM Fine-... Looks like no quantized options with llama.cpp? https://github.com/ggerganov/llama.cpp/issues/1602

dvilasuero on June 5, 2023 [–]

We're very much looking forward to seeing Falcon-40B support on llama.cpp. For production use cases, this is also highly relevant: https://huggingface.co/blog/sagemaker-huggingface-llm

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact