27 lines
1.2 KiB
Markdown
27 lines
1.2 KiB
Markdown
# Sorted Code Layout
|
|
|
|
This directory is a cleaned copy of the paper pipeline code.
|
|
|
|
## Stages
|
|
|
|
- `1_raw_dataset/`: upstream/raw dataset copies.
|
|
- `2-1_head_identification_preprocess/`: scripts and converted JSONL files for head-identification datasets.
|
|
- `2-2_head_identification/`: attention collection, head scoring, head sorting, visualization, and threshold sweeps.
|
|
- `3-1_model_training_preprocess/`: training-data generation scripts and output directory.
|
|
- `3-2_model_training/`: LoRA training scripts and output directory.
|
|
- `4_model_evaluation/`: model evaluation and summary scripts.
|
|
- `sharedlibs/`: shared Python helpers used by multiple stages.
|
|
|
|
## Path Rules
|
|
|
|
Scripts under each stage use relative paths:
|
|
|
|
- raw dataset inputs point to `../1_raw_dataset/...`
|
|
- head-identification inputs point to `../2-1_head_identification_preprocess/...`
|
|
- head-scoring outputs point to `../2-2_head_identification/head_scoring/...`
|
|
- generated training data points to `../3-1_model_training_preprocess/model_training/...`
|
|
- LoRA outputs point to `../3-2_model_training/lora/...`
|
|
- local model checkpoints point to `../../models/...`
|
|
|
|
Each runnable stage has a `lib` symlink to `../sharedlibs/lib` so existing `from lib...` imports continue to work.
|