+1
日增长
-4
周增长
-6
月增长
18,883
stars
2,163
forks
151
watchers
130
prs
471
issues
0
releases
README
🦙🌲🤏 Alpaca-LoRA
- 🤗 Try the pretrained model out here, courtesy of a GPU grant from Huggingface!
- Users have created a Discord server for discussion and support here
- 4/14: Chansung Park's GPT4-Alpaca adapters: https://github.com/tloen/alpaca-lora/issues/340
This repository contains code for reproducing the Stanford Alpaca results using low-rank adaptation (LoRA).
We provide an Instruct model of similar quality to text-davinci-003 that can run on a Raspberry Pi (for research),
and the code is easily extended to the 13b, 30b, and 65b models.
In addition to the training code, which runs within hours on a single RTX 4090,
we publish a script for downloading and inference on the foundation model and LoRA,
as well as the resulting LoRA weights themselves.
To fine-tune cheaply and efficiently, we use Hugging Face's PEFT
as well as Tim Dettmers' bitsandbytes.
Without hyperparameter tuning, the LoRA model produces outputs comparable to the Stanford Alpaca model. (Please see the outputs included below.) Further tuning might be able to achieve better performance; I invite interested users to give it a try and report their results.
Local Setup
1. Install dependencies
bash
pip install -r requirements.txt
1. If bitsandbytes doesn't work, install it from source. Windows users can follow these instructions.
Training (finetune.py)
This file contains a straightforward application of PEFT to the LLaMA model,
as well as some code related to prompt construction and tokenization.
PRs adapting this code to support larger models are always welcome.
Example usage:
bash
python finetune.py \
--base_model 'decapoda-research/llama-7b-hf' \
--data_path 'yahma/alpaca-cleaned' \
--output_dir './lora-alpaca'
We can also tweak our hyperparameters:
bash
python finetune.py \
--base_model 'decapoda-research/llama-7b-hf' \
--data_path 'yahma/alpaca-cleaned' \
--output_dir './lora-alpaca' \
--batch_size 128 \
--micro_batch_size 4 \
--num_epochs 3 \
--learning_rate 1e-4 \
--cutoff_len 512 \
--val_set_size 2000 \
--lora_r 8 \
--lora_alpha 16 \
--lora_dropout 0.05 \
--lora_target_modules '[q_proj,v_proj]' \
--train_on_inputs \
--group_by_length
Inference (generate.py)
This file reads the foundation model from the Hugging Face model hub and the LoRA weights from tloen/alpaca-lora-7b, and runs a Gradio interface for inference on a specified input. Users should treat this as example code for the use of the model, and modify it as needed.
Example usage:
bash
python generate.py \
--load_8bit \
--base_model 'decapoda-research/llama-7b-hf' \
--lora_weights 'tloen/alpaca-lora-7b'
Official weights
The most recent "official" Alpaca-LoRA adapter available at tloen/alpaca-lora-7b was trained on March 26 with the following command:
bash
python finetune.py \
--base_model='decapoda-research/llama-7b-hf' \
--num_epochs=10 \
--cutoff_len=512 \
--group_by_length \
Top issues
#279What are the eos_token_id and bos_token_id37 comments · 2023-04-05#87sharing Alpaca-LoRA chatbot (Gradio app)31 comments · 2023-03-20#170[Bug] Training on 13B causes loss to be 0, while 7B works fine29 comments · 2023-03-25#28Anyone try fine-tuning 13B model?28 comments · 2023-03-16#174cuBLAS API failed with status 15 - Error27 comments · 2023-03-26#45How to finetune model with a new knowledge?24 comments · 2023-03-17#368NotImplementedError: Cannot copy out of meta tensor; no data!23 comments · 2023-04-19#354GPU training requiring at least 9G of vram?23 comments · 2023-04-17
Discussions
#614safetensors_rust.SafetensorError: Error while deserializing header: InvalidHeaderDeserialization0 comments · 2024-01-10#613Why using DataCollatorForSeq2Seq instead of DataCollatorForCausalLM0 comments · 2023-12-31#592Fine tune Llama2 with Lora for foreign language0 comments · 2023-10-17#582Idea to improve train loss curves (thx!)0 comments · 2023-09-12#580Ideas to improve the speed of model0 comments · 2023-09-01#569Is there any script I can use to combine two LoRAs?0 comments · 2023-08-17#528How do I train Alpaca on a different dataset1 comments · 2023-06-22#504What setting is suitable for RTX3090?0 comments · 2023-06-08