Jais-2
Hello thanks for your great contributions
I tried to create the gguf format for this repo model
inceptionai/Jais-2-8B-Chat
in gguf-my-repo space but it failed could you please create a gguf quantized format for this model i found that you you've already converted the older version
It should also fail because both gguf-my-repo and mradermacher have the latest llama.cpp, I think
The Jais2ForCausalLM architecture is unfortunately not currently supported by llama.cpp nor do I see any users requestion support for it to be implemented or any contributors working on implementing this architecture. Because it this it is unfortunately not possible for use to provide GGUFs for this model.
It should also fail because both gguf-my-repo and mradermacher have the latest llama.cpp, I think
Thank you so much for responding to user requests. Doing so is highly appreciated as my time is usually very limited and so it sometimes can take quite a while until I find times to answer. Team mradermacher uses my llama.cpp fork I maintain under https://github.com/nicoboss/llama.cpp. It is latest llama.cpp but with quality-of-life improvements added such as a dryrun option to test models without actually loading them, and a source option to not impact precision when converting from SafeTensors to GGUF. We always manually investigate every error and when possible, try to find a workaround but this is unfortunately often not possible. Issues we can often fix are missing files such as missing "Missing importance matrix for tensor XXX.weight in a very low-bit quantization", "BPE pre-tokenizer was not recognized" (we only do so if requested), "No such file or directory: 'XXX/preprocessor_config.json' (a bit hit or miss if we find one that works).
I can't even see a PR for llama.cpp to add Jais
@nicoboss
Thanks, I want to check the fork, I also have to write my own code for quality-of-life improvements