01bb07dba8
FocalLoRA, Should-It-Be-Executed-Or-Processed, and topicattack were nested git repos (with an inner FocalLoRA/data/FocalLoRA/.git as well). Drop their .git history and track the contents directly in this repo instead of as submodules/gitlinks. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1.7 KiB
1.7 KiB
Don’t Forget the Enjoin: FocalLoRA for Instruction Hierarchical Alignment in Large Language Models
framework
Data Generation
python dataGeneration.py
The second type of data is obtained by reversing the system with the user instruction.
Attention Visualization
visualization_attention.py is used to visualize the attention heatmaps before and after fine-tuning the model. An example script is:
python visualization_attention.py \
--model_path "/home/user/models/Meta-Llama-3.1-8B-Instruct/" \
--lora_path "/home/user/LoraAdapter_set/llama3_loraAdapter3_0.3/" \
--json_file "json/test.json" \
--cuda 0\
--important_file "outputs/case_outputs/important_heads.json" \
--output_path "./attention_visualization/lora_llama_case"
To reproduce the conflict-vs-normal case study described in the paper, run the helper script:
./visualize.sh
This script constructs the greenhouse-effect prompts (English-only system, optional French-only user instruction), and renders attention maps for both the base model and the fine-tuned LoRA adapter under attention_visualization/base_model and attention_visualization/finetuned.
Model fine-tuning
python _tuning.py \
--model_path "/home/user/models/Meta-Llama-3.1-8B-Instruct/" \
--json_path "data/language_instruction.json" \
--output_dir "LoraAdapter_set/llama3_loraAdapter3_0.5" \
--topk 10 \
--epochs 10 \
--lr 2e-4 \
--lambda_focus 1 \
--tune_path tuneData
Model Output
python GetAS.py \
--json_path data/case_instruction.json\
--model_path "/home/user/models/Meta-Llama-3.1-8B-Instruct/" \
--lora_path "" \
--output_dir "results/llama" \
--cuda 1