Really Long Context LLMs

The following Yi models are fine tuned – using open source datasets – into a chat format.

For the best performance on long context, the 34B size model is recommended. The quality of responses from the 6B model degrades around 10k of context.

• Chat fine-tuned models:
  • Yi 200k chat context: 34B (purchase here) and 6B (purchase here).
• Function-calling Fine-tunes:
  • Function calling Yi 200k chat context: 34B (purchase here) and 6B (purchase here).

Video Tutorials

Really Long Context Language Models: