Skip to content

Fine-tuning and Retrieval (RAG)

Features:

  • GitHub Repo access to the content below.
  • Ability to post Issues.

Repo Content = all you need to fine-tune LLMs:

  • Multi-GPU Distributed Training:
    • with Unsloth.
    • with Transformers: including Distributed Data Parallel and Fully Sharded Data Parallel (FSDP). [Purchase standalone scripts here].
  • Custom Compute Metrics (evaluate benchmarks on the fly during training)
  • Fine-tuning and Evaluation Notebooks (incl. for OpenAI OSS and Qwen3):
    • vLLM + Transformers
    • vLLM + Unsloth
    • Transformers (for evals and fine-tuning)
  • Advanced Data Prep and Visualisation Techniques for Fine-tuning LLMs
  • Reinforcement learning:
    • GRPO
    • SFT with rejection sampling and verifiable backtracking
  • Distilling cheaper/faster models from stronger models using OpenAI
  • Synthetic Data Generation and Fine-tuning:
    • Automated data generation from documents
    • Synthetic Questions and Synthetic Answers
    • Fine-tuning using OpenAI, Unsloth or Transformers
  • Distillation (model pruning, distillation of a student model from a teacher, instruction fine-tuning)
  • Full Fine-tuning with Fewer GPUs (GaLore, subspace descent, optimizer selection)
  • Mastering Retrieval: BM25, similarity, embedding fine-tuning, LLM re-rankers, cross-encoder re-rankers. [Purchase standalone scripts here].
  • Wikipedia dataset creation and unsupervised fine-tuning. [Purchase standalone scripts here].
  • ReFT (and few parameter LoRA) fine-tuning.
  • Unsloth fine-tuning, including rank stabilised LoRA (rsLoRA) and LoRA aware Quantization (loftQ).
  • Combined preference and supervised fine-tuning (ORPO) scripts [Purchase individually, including evaluation instructions, here].
  • Fine-tuning for memorization, data prep and fine-tuning scripts [Purchase these individually here]
  • Mamba fine-tuning
  • Chat-fine tuning [Purchase the individual script here]
  • Fine-tune models for extended context length [Purchase the individual script here]
  • Fine-tuning for Function Calling and Structure Responses v3 script [Purchase the individual script here] (for use with the function_calling_v3 dataset).
  • Direct Preference Optimization (DPO) Notebook [Purchase the individual script here]
  • Supervised Fine-Tuning Notebook [Purchase the individual script and data preparation scripts here]
  • Unsupervised Fine-Tuning Notebook [Purchase the individual script here]
  • Quantization Scripts for AWQ, GGUF and GPTQ [Purchase the individual script here]
  • Scripts to generate and use Embeddings [Purchase the individual script here]
  • Q&A Dataset Generation Script
  • LLM Comparison Script (compare models on code generation, passkey retrieval, website generation and reverse responses). [Purchase the individual script here]

DECPRECATED SCRIPTS:

  • Fine-tuning for Function Calling and Structure Responses v2 script (i.e. for the Trelis/function_calling_extended dataset). Purchase here.

Notes:

  • Price for new buyers increases periodically as new content is added.
  • This repo is for fine-tuning scripts. Datasets are not included.
  • For the purchase of scripts above, please post bug reports/issues on this web page. For those who have purchased repo access, please post an Issue in that GitHub repo. Priority is given to those who have purchased repo access.

Purchase Options

Fine-tuning Repo Access (Join 561+ members)

LIFETIME access, plus support via Github Issues, for one individual:

For TEAM access (i.e. repo used by multiple engineers), kindly post a comment below. It will be submitted privately and I’ll respond by email.

Trelis Multi-Repo Bundle

LIFETIME access to all SEVEN Trelis Repos (Voice, Vision, Fine-tuning (LLMs), Inference, Evals and Time-Series, Robotics) + support via Github Issues PLUS Trelis Discord Access for one individual.

Want lifetime access for a team? Purchase lifetime access here.

Testimonials from Youtube Comments

“Awesome content. Thank you! Purchased your gated repo and learning loads from it.”

“Thank you! It’s a great investment :)”

“I had Whisper & CoLab running a few months ago, but it broke. Your video and notebooks showed me why, and taught me several new tricks! Keep it up please.”

Repo Screenshots

Frequently Asked Questions

Q: Can I get a formal invoice with my tax number so I can reimburse this expense?

Yes, purchases come with an invoice and receipt of payment. You can include your tax/VAT number (not applicable for personal purchases or those using the LEARN discount). If you require modifications to the invoice, just respond to your email receipt.

Q: Is support included in my repo purchase?

Yes, if you purchase repo access, you will be able to post Issues in the corresponding Github repo.

Q: Can I buy individual scripts for one video?

Yes, if you only wish to purchase scripts for one video, you can post a comment and I will add a purchase link. Purchases of individual scripts do not provide GitHub repo access or the ability to post issues to get support.

Q: Is there a student discount available?

Yes. Use LEARN2025 at checkout if EITHER of the following applies:
a) You are currently a registered student at an academic institution.
b) You will use these materials only for non-commercial purposes.

After your purchase, reply to your receipt with a quick note indicating whether a or b applies. In the case of a, kindly provide a link to your LinkedIn page. Purchases using a company email/tax number/Github are not eligible for this discount. This discount applies to repos or the repo bundle, but not to individual scripts.

Video Tutorials

Train an LLM From Scratch with Karpathy’s Nanochat

Multi GPU Training with Unsloth

Fine tuning with Custom Compute Metrics

One-Click GPU Setup for Fine-tuning

GPT OSS

Fine tune Open Weight Models with Unsloth or Transformers

Data Prep and Visualisation for Fine-tuning LLMs

Verifiable Backtracking

SFT vs GRPO

How does GRPO work?

Reinforcement Learning for LLMs in 2025

Advanced Embedding Models and Techniques for RAG

OpenAI Fine-tuning vs Distillation

Synthetic Data Generation and Fine-tuning

Distillation of Transformer Models

Full fine-tuning on fewer GPUs

Mastering Retrieval – BM25, similarity, fine-tuned embeddings, and re-rankers

Top Ten Fine-tuning Tips

Wikipedia Dataset Generation and Fine-tuning

Few parameter ReFT and LoRA

Full fine-tuning vs (Q)LoRA

Multi-GPU Fine-tuning with DDP and FSDP

Combined Preference and Supervised Fine-tuning with ORPO

Fine-tuning LLMs for Memorisation

Advanced Fine Tuning Scripts Overview

Long Context Summarization:

Embeddings:

Supervised Fine-Tuning:

Unsupervised Fine-Tuning:

Preparing Fine-tuning Datasets:

Quantization:

Quantization:

106 thoughts on “Fine-tuning and Retrieval (RAG)”

    1. Ronan | Trelis

      Howdy Josh, I just added you there yesterday evening. Right now I manually add anyone who buys – within 24 hours of purchase. I’m working to automate the process. Cheers, Ronan

    1. Ronan | Trelis

      Adeel, thanks for writing here and by email. Your email went to spam but I have fished it out and responded.

      Your question related to doing training on a Mac M1 or M2. While the same high level approaches can be used, the scripts are quite different as the best way is to use Llama.cpp – which is written in C, not python. I have just added some notes to this ADVANCED fine tuning repo to help people get started, but detailled scripts for fine-tuning on a Mac are not included.

  5. YAOQUAN FU

    hi，I’ve purchased the supervised learning script, but it doesn’t seem to have the QA-generated python files you show in the video. Will you open the corresponding github permissions for me?

  7. Jesús

    Hello, I would like you to contact me, I would like to make a purchase, I want my assistant to be able to call functions, greetings

      1. Madi

        Do you at least provide an example of how the training dataset should be structured? The function calling notebook is completely unusable otherwise

    1. Ronan | Trelis

      The scripts are for causal language models (e.g. gpt type). M2M would use different loading and often dataset formats. If you’re only doing M2M models, you might learn something from the repo, but I can’t strongly recommend it.

      1. Bronislav

        Hi,
        thank you for the feedback. No, I just don’t need m2m only, so I’ll buy a repo. I am a beginner and everything is explained beautifully in the videos. So looking forward to it – great job! Do you have any plans to expand your scripts repo with m2m soon? Alternatively, is it possible to order the creation of such a script for m2m fine-tuning separately? If so and you are open to it, where can I contact you? Thank you and have a nice day.

        1. Ronan | Trelis

          Currently I don’t have plans to do M2M, it’s not something I’ve had as much demand for.

          And yes, you’ll see my email with the purchase receipt so feel free to respond there if there are issues.

  13. Yaoquan Fu

    Hello, when I use QLoRA_Fine_Tuning_SUPERVISED.ipynb, I use my own data set and the model is set to: model_id = “meta-llama/Llama-2-7b-chat-hf”. But I get error:
    RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

    Can you provide some solutions?
    I have invited you to visit my colab.

    Thank you for your help

    1. Ronan | Trelis

      Howdy, since you purchased repo access, you can post there by creating an issue. That’s my preferred way to respond as it helps all others who have purchased the repo.

      Please include a code snippet for replication in your post. Thanks

    2. Tony

      I have this issue as well. Can I get some help ? I bought the training script and I’ve been trying to run the script locally and in collab with a 2b model. I either get the error above when using bnb config or an cuda out of memory error without bnb. I’m not understanding the issue. Can I see the repo?

      1. Ronan | Trelis

        Hi Tony, mostly likely this is because in Colab it’s an older GPU that doesn’t support bf16.

        Where you see bf16 in the script, you’ll need to swap to fp16. Also, you’ll need to comment out anywhere mentioning flash attention.

        BTW, this should be explained towards the top of the script.

        If this doesn’t solve the issue, please let me know a) what exact script you are running, b) if you made any changes, and c) exactly where you are getting the error.

        1. Tony

          Yes, I’m not using flash attention and I’m using fp16 in the trainer.

          I’m using the fucntion calling v3 script. Only changes I’ve made are implementation of my own data

          Is there a way I’d be able to share the script with you so you can see what I’m working with

          1. Ronan | Trelis

            Howdy again. So the issue here is likely that, when using BNB, one needs to ensure that “prepare_model_for_kbit_training” is commented in. That should be described in the scripts.

  15. Zack

    Hi Trelis,

    I bought the train script and function-calling-v3 dataset. When training on the runpod, how much container disk and volume disk do I need to train a 34B size model? And when choosing the model, the orginal one shoule be chosen, not the GPTQ version, am I right?

    btw, your youtube clip https://www.youtube.com/watch?v=hHn_cV5WUDI is very helpful.

    Best,
    Zack

    1. Ronan | Trelis

      To train 34B in full precision (bf16) on runpod, I find I need two A6000s or two A100s. I usually put 500 GB of container and disk, just to have loads of room, and that doesn’t add much cost.

      Yes, if are choosing, use the base, not GPTQ. GPTQ does not allow LoRA adapters to be merged.

  17. Zack

    One more question, If I want to use TheBloke/SUS-Chat-34B-GPTQ to train the function-calling version, could I use this GPTQ version as the base model? In your youtube, you mentioned quantization_config, does it relate to this? And normalling, how long it will take to train a 34B modle?

    Thanks,
    Zack

    1. Ronan | Trelis

      Yes, you can train a GPTQ model with LoRA. However, the adapters can be merged, which is very inconvenient because it means you have to apply an adapter every time you load the model PLUS inferencing a model with an adapter is slower if they aren’t merged. Hence, why everyone trains the base model.

  19. Zack

    I bought Fine-tuning for Function Calling and Structure Responses v3 script already, if I buy this advanced one which includs the function-calling script, could I get the previous payment refund?

    Thanks,
    Zack

  21. Zack

    I’ve used the fine tune script to train the SUSTech/SUS-Chat-34B model on runpod, but failed. Do you have a email or someway to connect to get the help? Another thing is that, in the script there is no way to see the training process, if you can add one, that would be great!

  23. Zack

    Errors as below:
    —————————————————————————
    NotImplementedError Traceback (most recent call last)
    Cell In[34], line 1
    —-> 1 trainer = CustomTrainer(
    2 model=model,
    3 train_dataset=train_dataset,
    4 eval_dataset=validation_dataset, #turn on the eval dataset if you want to use evaluation functionality versus the test dataset (not provided in this script)
    5 args=transformers.TrainingArguments(
    6 # max_steps=1,
    7 num_train_epochs=1, #stronger models typically only need 1 epoch. Check if the validation loss is still dropping, that means you can train more.
    8 per_device_train_batch_size=1,
    9 gradient_accumulation_steps=1,
    10 evaluation_strategy=”steps”,
    11 max_grad_norm=1,
    12 warmup_ratio=0.1,
    13 eval_steps=0.2,
    14 learning_rate=1e-4,
    15 # fp16=True,
    16 bf16=True,
    17 logging_steps=1,
    18 output_dir=”outputs”,
    19 # optim=”paged_adamw_8bit”, #for training in 4bit (quantized)
    20 optim=”adamw_torch”, #for training in full fp16/bf16 precision
    21 lr_scheduler_type=’constant’,
    22 ),
    23 data_collator=data_collator,
    24 # data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),
    25 )
    26 model.config.use_cache = False # silence the warnings. Please re-enable for inference!

    File /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:481, in Trainer.__init__(self, model, args, data_collator, train_dataset, eval_dataset, tokenizer, model_init, compute_metrics, callbacks, optimizers, preprocess_logits_for_metrics)
    476 # Bnb Quantized models doesn’t support `.to` operation.
    477 if (
    478 self.place_model_on_device
    479 and not getattr(model, “quantization_method”, None) == QuantizationMethod.BITS_AND_BYTES
    480 ):
    –> 481 self._move_model_to_device(model, args.device)
    483 # Force n_gpu to 1 to avoid DataParallel as MP will manage the GPUs
    484 if self.is_model_parallel:

    File /usr/local/lib/python3.10/dist-packages/transformers/trainer.py:716, in Trainer._move_model_to_device(self, model, device)
    715 def _move_model_to_device(self, model, device):
    –> 716 model = model.to(device)
    717 # Moving a model to an XLA device disconnects the tied weights, so we have to retie them.
    718 if self.args.parallel_mode == ParallelMode.TPU and hasattr(model, “tie_weights”):

    File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1160, in Module.to(self, *args, **kwargs)
    1156 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None,
    1157 non_blocking, memory_format=convert_to_format)
    1158 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
    -> 1160 return self._apply(convert)

    File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:810, in Module._apply(self, fn, recurse)
    808 if recurse:
    809 for module in self.children():
    –> 810 module._apply(fn)
    812 def compute_should_use_set_data(tensor, tensor_applied):
    813 if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    814 # If the new tensor has compatible tensor type as the existing tensor,
    815 # the current behavior is to change the tensor in-place using `.data =`,
    (…)
    820 # global flag to let the user control whether they want the future
    821 # behavior of overwriting the existing tensor or not.

    File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:810, in Module._apply(self, fn, recurse)
    808 if recurse:
    809 for module in self.children():
    –> 810 module._apply(fn)
    812 def compute_should_use_set_data(tensor, tensor_applied):
    813 if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    814 # If the new tensor has compatible tensor type as the existing tensor,
    815 # the current behavior is to change the tensor in-place using `.data =`,
    (…)
    820 # global flag to let the user control whether they want the future
    821 # behavior of overwriting the existing tensor or not.

    [… skipping similar frames: Module._apply at line 810 (5 times)]

    File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:810, in Module._apply(self, fn, recurse)
    808 if recurse:
    809 for module in self.children():
    –> 810 module._apply(fn)
    812 def compute_should_use_set_data(tensor, tensor_applied):
    813 if torch._has_compatible_shallow_copy_type(tensor, tensor_applied):
    814 # If the new tensor has compatible tensor type as the existing tensor,
    815 # the current behavior is to change the tensor in-place using `.data =`,
    (…)
    820 # global flag to let the user control whether they want the future
    821 # behavior of overwriting the existing tensor or not.

    File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:833, in Module._apply(self, fn, recurse)
    829 # Tensors stored in modules are graph leaves, and we don’t want to
    830 # track autograd history of `param_applied`, so we have to use
    831 # `with torch.no_grad():`
    832 with torch.no_grad():
    –> 833 param_applied = fn(param)
    834 should_use_set_data = compute_should_use_set_data(param, param_applied)
    835 if should_use_set_data:

    File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1158, in Module.to..convert(t)
    1155 if convert_to_format is not None and t.dim() in (4, 5):
    1156 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None,
    1157 non_blocking, memory_format=convert_to_format)
    -> 1158 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)

    NotImplementedError: Cannot copy out of meta tensor; no data!

    1. Ronan | Trelis

      I would need to see a minimum reproducible piece of code so I can reproduce the error myself. Best to do this in the ADVANCED-fine-tuning repo under issues, if you buy that.

  25. Mike D

    I purchased access to the Advanced-fine-tuning scripts. I am looking at QLoRA_Fine_Tuning_function_calling_Zephyr.ipynb on the function-calling branch. I cannot run the notebook as it is trying to load dataset “Trelis/function_calling_extended” (which is a separate paid product). How can I run the notebook? Is it possible to get a sample dataset, even if only with a few rows of usable data, so that I can train a LoRA using my Windows based RTX 4090 system?

  27. Vineet

    Hello There,

    I would like to know – if we purchase the access to the Full Repo today; will we also get the access to the upcoming Codes/Scripts that you add to this repo ?

    Thanks.

      1. Vineet

        Hello Ronan,

        Thanks for the reply.

        When I click on the Purchase Full Repo Access button on the Advanced Fine-tuning Scripts page; it takes to the Stripe Checkout Page. However, on Stripe Checkout Page – it shows as “Advanced Inference and Server Setup Repo” as the description (on the left side)

        I want to purchase the full repo access for “Advanced Fine-tuning Scripts”.

        Ronan, is the above link correct or do I need to use some other link for purchasing the Advanced Fine Tuning Scripts. Please confirm the same and I will go ahead with the purchase.

        Thanks Ronan.

          1. Vineet

            Thanks Ronan for the clarification and the resolution.

            I have made the purchase and have received the access to the Repo.

            Way to go !

  29. Ben

    Hiya

    Got access to your repo, great stuff. I’m a viewer of your videos and enjoy the content. I would be interested in a video of some Dataset creation and what you have had to put into those.

    Good man

    Hoops

    1. Ronan | Trelis

      Great stuff, any specific type of dataset in particular? There’s one video on dataset creation on my channel, and there’s some coverage also in the Embeddings vs Fine-tuning three part series.

      1. Sandeep Srinivasa

        i want to add to this. super interested in dataset creation – especially how to think about it, how to do it, how to test it, iteratively, etc
        there is absolutely no tutorial out there that covers this topic exhaustively.

        super happy to pay for it.

  31. Andreas Stylianou

    Hi,
    I have purchased advanced fine tuning scripts..will the finetuning dataset v3 be enough to interact with a postgres database .Functions are working with gpt function calling.If i finetune mixtral will i have the same results?

    1. Ronan | Trelis

      Howdy, I don’t know for sure as I haven’t tested postgres options. My reccs for you:
      1. Train Mixtral as you said (OR buy the out of the box fine-tuned Mixtral function calling model from mart.trelis.com)
      2. Train OpenChat 3.5 Or DeepSeek Coder 33B (again, out of the box versions are available from mart.trelis.com if you prefer). These models are stronger for function calling than Mixtral.

      If the v3 dataset doesn’t work sufficiently well for postgres queries (although I think there’s a reasonable chance they will), you can append some postgres examples to the v3 dataset when you train.

  33. igloo

    Hi , your youtube videos are really good.

    I wanted to ask if the fine tuning repo also contain all the content of the inference repo or only the future inference scripts as well. I know it is a dummy question because all the fine tuning scripts will contain inference code as well when the model is fine tuned.

    Also, It would help me great deal if you can you provide me with a promo code for some good discount. Thanks

    1. Ronan | Trelis

      Howdy!

      The fine-tuning repo is focused on fine-tuning. Yes, most scripts do a have a generation function because that is needed to test performance.

      If you want scripts to do more complex inference (speed tests, api setup, server setup, long context, concurrent requests), then there is a different repo/product available called Advanced Inference.

      Re discounts, buying lifetime access to a repo includes access to future scripts I upload. Meanwhile, as new scripts are added, the price of each repo increases for new buyers of lifetime access. So, while there isn’t a discount offered in percentage or money terms, there is a benefit to those who buy while the product is young.

    1. Ronan | Trelis

      oooh, nice, advanced paper! I need to read it in full.

      Can you give an example when you say “generate synthetic data using the function calling approach”? Do you mean get the LLM to call a function with some text as the arguments to a function call, and then the function formats that info into a table. If so, then I say yes!

      1. v

        actually simpler ask – function calling is based on the llm being able to format structured data according to a strict schema.

        so i just ask the llm to generate synthetic data according to a strict schema (and maybe strict rules – “dont make this column zero” for e.g.)

          1. VN

            let me try:
            1. https://github.com/sdv-dev/SDV -> tool to generate synthetic tabular data
            2. https://arxiv.org/pdf/2210.06280.pdf -> paper on how to fine-tune LLMs to generate synthetic tabular data
            3. Aha moment! – tabular data is same as function calling. “Function calling is a feature that was largely popularized by OpenAI that enables LLMs to extract structured data from unstructured text”
            4. this means that i can probably do something like “hey OpenAI, generate 10000 simulated function calls of the form fn(int, string, float) where int should always be greater than 50”

            1. Ronan | Trelis

              The short answer is yes you can use prompting to get structured data from an LLM. Some work better than others. Function call is a specific type of request for structured data because you only want that structured data when it is appropriate to call a function.

              Most data extraction requests are easier than this because you are explicitly asking for data to be extracted (there’s no uncertainty of whether to call a function or not). So, you don’t necessarily need a function calling model if extracting data (especially structured data). On this topic, I’ll have a video out soon on data extraction.

    3. Sandeep

      hi
      am interested in this. just quick question – are your finetuning scripts compatible with axolotl and deepspeed ? cos thats generally what everyone has been using these days.

      1. Ronan | Trelis

        I’m using the basic transformers library – (and unsupervised and supervised fine tuning have options to speed up with Unsloth). I think adding Deepspeed is a small step, but it’s not there. Axotyl isn’t there either. I’ll add them to my research list.

        1. Sandeep Srinivasa

          so primarily the reason why i was asking for deepspeed and axolotl is due to compatibility with kubernetes.
          most of us have kubernetes clusters *already*. And it becomes tablestakes. asking for single machines for training becomes one of those problematic things in most teams that are already on k8s.

    5. Sandeep

      what Full Repo access contain.
      is it contain all script.
      what about dataset if required that.

      apart from that Excellent work by you.

    9. Mann

      Hi,
      I hope that my message finds you well.
      Your videos are very informative and thank you for sharing your knowledge.
      I would like to ask if there is any student discount to buy this repository, It is expensive for students like me.

      Best regards,
      Manpreet

    11. wootz

      hi,
      first basic question – should we buy the function calling dataset separately (https://huggingface.co/datasets/Trelis/function_calling_v3) ? or is it included in these scripts package ?

      second genuine question (not trolling) – how would you compare your scripts to something like this https://twitter.com/NexusflowX/status/1732041385455624256 ?

      why im asking this question – my interest is to finetune for my usecase for my functions (and possibly perform at par with gpt4)

      1. Ronan | Trelis

        howdy! appreciate the questions.

        Yes, the function calling dataset is separate to any of the repos (e.g. ADVANCED-fine-tuning or ADVANCED-inference)

        When comparing to Nexus Raven, I suppose you mean comparing with some of the Trelis function calling models (as opposed to the scripts)? I haven’t dug in on Nexus Raven, but my guess is that the OpenChat function calling model is in the ballpark, it’s very strong – see the function calling video. The trelis function calling models are easy to use with runpod one-click templates and the ADVANCED-inference repo, and the YouTube video can be a helpful guide. Probably Nexus Raven has got a different set of compatibilities/toolboxes that make it easy to use there.

        Re your usecase. I would say start and see what performance you get with open source models. Then try out OpenChat function calling if you want to push performance. Lastly, try fine-tuning with a dataset like trelis function calling and maybe the associated training scripts (or one you build yourself).

    13. Linh

      I’m interested in fine tuning with a system prompt. Is your script for the Q&A fine tuning easily adaptable to my use case?

      1. Ronan | Trelis

        Hard to know exactly without knowing your use case, but the script will generate a Q&A dataset from your data source – that is pretty generalizable, and you can of course tweak the prompt.

    17. Kiran

      Hello ,
      I want to purchase the supervised repo, and does it contain the Q&A data generation script?
      I am a student, and the cost is a bit expensive for me. If there is a student discount, where can I apply that?

      1. Ronan | Trelis

        Yes, this contains the Q&A data generation script. If you are a student, you can use the STUDENT discount code. After purchasing, please respond to your receipt with a link to your LinkedIn page so I can confirm student status. Thanks.

    19. Gyozo

      Hello Ronan Trelis,

      I came across your website offering scripts for AI training, and I’m reaching out for your guidance in navigating through the scripts you have developed. My objective is to train a Large Language Model (LLM) on a specialized topic, specifically “dog training,” in a language that is not well-supported by smaller models and even poorly by larger ones, which in my case is Hungarian. I possess digitized books for this purpose but require assistance in preparing the text, converting it into a chat format, and ultimately teaching it to the model. My aim is to automate this process as much as possible, which is why I am interested in subscribing to your scripts.

      Could you please recommend which scripts I should purchase to make this project feasible? I would appreciate it if you could respond via email, but I will also keep an eye on your website for a reply.

      Thank you for your assistance.

      Best regards,
      Győző

      1. Ronan | Trelis

        Best option is the ADVANCED-fine-tuning repo. I suggest doing a combination of unsupervised (with the books as input data) followed by some supervised fine-tuning with some Q&A that you can generate synthetically using the repo guidance.

    21. Dean Kamali

      I’ve developed a streamlined backend API for managing tasks, where some actions depend on others.

      Example routes:

      /workspace/{workspace_name}
      /workspace/{workspace_name}/stack/{stack_name}/{action}
      /workspace/{workspace_name}/stack/{stack_name}/layer/{layer_name}/{action}

      Example Request: create stack1 in workspace1

      I need to create the workspace first, and then create the stack.

      What LLM is best for this? and does advanced repo access give me access to associated training scripts?

      1. Ronan | Trelis

        This looks like a case of structured fine-tuning. So you would follow the tutorials/youtube vid for that. You can see trelis.com/function-calling for options on buying the script OR the advanced fine tuning repo, which has that script and others.

    23. Priya

      Hi, I purchased the fine tuning repository few minutes ago. Billed with name Priya from India . can you please grant access asap.

      Thanks

      1. Ronan | Trelis

        Here’s how to find access after purchasing:

        1. You should have received an email from GitHub.
        2. Alternatively, go to your GitHub activity page (log in and click the github icon to the top left) – you’ll see the invite there.

    25. Vlad

      I just bought the multi-gpu pack. I have some one question though:
      How I can generate the chat template for the meta-llama/Meta-Llama-3-8B?

      1. Ronan | Trelis

        Nice!

        You can find it in the tokenizer_config.json file in the Llama-3-8B repo, scroll right down to the bottom. BTW, if you bought repo access, you can create GitHub issues there to get support :)

    27. Vlad

      Thank you for your quick answer.
      “You can find it in the tokenizer_config.json file in the Llama-3-8B repo”
      What I am looking for is for creating the jinja file type.
      “if you bought repo access, you can create GitHub issues there to get support 🙂”
      I do not think so. I only bought the “Multi-GPU Distributed Training, including Distributed Data Parallel and Fully Sharded Data Parallel (FSDP) – standalone scripts.”

      1. Ronan | Trelis

        Ah, no problem.

        To create a jinja file, you can copy the tokenizer chat template (from the Llama 3 repo tokenizer file) and copy paste it into a blank file that you can save (or rename) as xxxx.jinja . Is that what you’re asking?

    29. Vlad

      Hah, never payed attention! Saw now that only the instruct models have this chat template in the tokenizer_config.json
      Thank you.

    30. Daniel Kleissl

      Hi Ronan,
      From what I saw from your video tutorials on YouTube the scripts seem to help getting started with AI as a Developer. I also wanted to ask if a Team License for the multi-repo is possible. :)

      Regards,
      Daniel

    32. KK

      Greetings Ronan,
      I purchased Advanced Fine Tuning + Bundle but I seem to have limited access to ADVANCED-fine-tuning repo.
      I see only .gitignore & Readme files and followed by Readme Text – ‘Supported Branch List’ & ‘ Fine -Tuning Notebook Setup instructions’ .
      Am following your embeddings & finetuning video but unable to search / view scripts for Fine Tuning in the Repositeries
      Please help

      1. Ronan | Trelis

        Howdy KK, no problem.

        The scripts are in dedicated branches. If you look at the readme in the main branch, you’ll see a list of all the branches and what they contain. Check out that list and then just navigate to the branch you are interested in.

    34. Abdullah MOHAMMED

      hello ,

      i got invitation but however it’s expired ,
      can you please send again , i bought the access already , thanks

      1. Ronan | Trelis

        Do you have a specific model in mind you would like supported? Newer models that are supported by huggingface transformers are typically supported, and if there are issues, members can post an issue and I’ll help out.

    37. abdül

      hello Ronan,
      i have a project about extracting information from resumes(pdf, word etc)
      what do you suggest, if you have opinion about it would be nice
      thank you already now.

      1. Ronan | Trelis

        Howdy Abdul, you can take a look at the latest synthetic data video (see the latest vid on this page). That allows you to extract from pdfs, but you can also adjust that with python-docx to extract from word.

        If you want a different approach, consider using ColPali (I’ll do a vid soon probably).

    41. Ram

      Hi Ronan
      Planning to buy the repos (life-time). Would these notebooks and then the fine-tuned models work on M3-Ultra. Understand that training might require cloud-GPUs. Any changes to ensure that they run on M3s (Mac Minis)?

      1. Ronan | Trelis

        Howdy Ramesh, training on Macs isn’t standard, although it is – in theory – supported by transformers if you set the device to mps. One key issue is that longer contexts tend to be very slow, and so you pretty quickly find it easier to use cloud. To be clear, I haven’t validated operation on Mac, but on M1/2/3/4 type chips there is some chance things might work.

    45. Ashish

      Hi Ronan,
      I have purchased the bundle repo. I can not find the “ADVANCED Training Notebook for Structured Responses (PAID)” . Requesting for providing the github link.

    49. Ashish

      Hi Ronan,
      When we add as special token, model embedding dimension needs to be resized. Apart from finetuning target modules in lora config, do we need to finetune embed_token and lm_head. Please help.

    Comments are closed.