0f90602339
Pipeline (stages 1 -> 4-1) can now be run in order from each stage folder.
Stage scripts:
- 2-1: make SEP/FocalLora prep portable (derive paths from __file__ instead of
hardcoded /home/hujk/...) and add prepare_head_ident_dataset.sh runner.
Verified the SEP converter reproduces the committed jsonl byte-for-byte.
- 2-2: unify the four Ident_IH_ALL_1-4_<model>.sh scripts (modernise llama to
conda hook + $ROOT/models; add the missing FocalLora step to qwen3-4b/8b so
focallora.json gets generated for them too).
- 2-3: default TARGETS now covers the three curves from the README
(all_roc_inst_0.1, user_roc_inst_0.1, focallora).
- 3-2: add combos/ with 24 scripts (4 models x {pbs,nts,nts_wam} x {squad,tri}),
head ranking pinned to all_roc_inst_0.1, TOPK overridable.
- 4-1: add eval_single.sh driver + combos/ with 24 cross-eval wrappers
(squad-trained -> tri-eval and vice versa), reusing the --eval-only path.
Eval semantics:
- Judge ASR before UTIL: a response carrying the injected answer now counts as
attacked even when it also contains the correct answer. This changes the
metric, so old training_log.csv rows are not comparable.
- Add --dev-holdout: reserve the last N source rows as a dev slice; training
drops them and the in-training quick eval uses only them. Previously the
quick eval silently defaulted to the squad evaluation set, which contradicted
the README and self-contaminated squad-trained runs.
- train_attn_kl_clean.sh now passes --eval-data-path/--eval-topicattack-path.
- Add --eval-step0 to log an untuned-baseline row before any weight update.
Housekeeping:
- Quarantine superseded entry points under legacy/ (2-2 single-step wrappers,
3-2 old _tuning.fix.* wrappers, 3-1 auxiliary), each with a README.
- Fix .gitignore: the model_score rule was anchored at the repo root and never
matched Codes/..., so ~26GB of intermediates had been staged. Now excludes
*.pkl (~25GB), heads_sorted_eval/ (~690MB), outputs_lora/ checkpoints
(~3.2GB) and pycache. heads_sorted/ and head_scoring_combined.json are kept
deliberately: they are small and are the HEAD_PATH inputs stage 3-2 needs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
65 lines
2.6 KiB
Bash
Executable File
65 lines
2.6 KiB
Bash
Executable File
#!/usr/bin/env sh
|
|
set -eu
|
|
|
|
export CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES:-0}
|
|
export PYTORCH_ALLOC_CONF=${PYTORCH_ALLOC_CONF:-expandable_segments:True}
|
|
|
|
CONDA_BIN=${CONDA_BIN:-}
|
|
if [ -z "$CONDA_BIN" ]; then
|
|
if command -v conda >/dev/null 2>&1; then
|
|
CONDA_BIN=$(command -v conda)
|
|
elif [ -x /opt/miniconda/bin/conda ]; then
|
|
CONDA_BIN=/opt/miniconda/bin/conda
|
|
fi
|
|
fi
|
|
|
|
if [ -n "$CONDA_BIN" ]; then
|
|
eval "$("$CONDA_BIN" shell.bash hook)"
|
|
conda activate focallora4
|
|
fi
|
|
|
|
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
|
|
BASE=$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)
|
|
ROOT=$(CDPATH= cd -- "$BASE/.." && pwd)
|
|
export EVAL_MMLU=1
|
|
export EVAL_TOPICATTACK=1
|
|
MODEL_PATH=${MODEL_PATH:-$ROOT/models/Llama-3.1-8B-Instruct}
|
|
DATA_PATH=${DATA_PATH:-$BASE/3-1_model_training_data_gen/single_turn/tri_native_tool_response_only.json}
|
|
HEAD_PATH=${HEAD_PATH:-$BASE/2-2_head_identification_scoring/model_score/sep_Llama-3.1-8B-Instruct/heads_sorted/all_roc_inst_0.1.json}
|
|
OUTPUT_DIR=${OUTPUT_DIR:-$SCRIPT_DIR/outputs_lora/attn_kl_clean}
|
|
|
|
python "$SCRIPT_DIR/train_attn_kl_clean.py" \
|
|
--model-path "$MODEL_PATH" \
|
|
--data-path "$DATA_PATH" \
|
|
--head-path "$HEAD_PATH" \
|
|
--output-dir "$OUTPUT_DIR" \
|
|
--topk "${TOPK:-18.75p}" \
|
|
--epochs "${EPOCHS:-3}" \
|
|
--batch-size "${BATCH_SIZE:-4}" \
|
|
--max-train-steps "${MAX_TRAIN_STEPS:--1}" \
|
|
--repeat-single-sample "${REPEAT_SINGLE_SAMPLE:-1}" \
|
|
--max-len "${MAX_LEN:-50000}" \
|
|
--lr "${LR:-1e-4}" \
|
|
--attn-chunk-size "${ATTN_CHUNK_SIZE:-4096}" \
|
|
--save-steps "${SAVE_STEPS:-100}" \
|
|
--log-interval "${LOG_INTERVAL:-100}" \
|
|
--log-csv "${LOG_CSV:-}" \
|
|
--eval-output "${EVAL_OUTPUT:-}" \
|
|
--mmlu-split "${MMLU_SPLIT:-dev}" \
|
|
--mmlu-size "${MMLU_SIZE:-256}" \
|
|
--mmlu-max-new-tokens "${MMLU_MAX_NEW_TOKENS:-16}" \
|
|
--eval-size "${EVAL_SIZE:-24}" \
|
|
--dev-holdout "${DEV_HOLDOUT:-0}" \
|
|
--eval-data-path "${EVAL_DATA_PATH:-$BASE/1_raw_dataset/topicattack/data/crafted_instruction_data_squad_injection_qa.json}" \
|
|
--eval-topicattack-path "${EVAL_TOPIC_PATH:-$BASE/1_raw_dataset/topicattack/data/crafted_instruction_data_squad_conversation_attack_complete.json}" \
|
|
--eval-config "${EVAL_CONFIG:-native_tool_response_only}" \
|
|
--eval-attacks "${EVAL_ATTACKS:-none,naive,ignore,escape_separation,completion_realcmb,conv_attack}" \
|
|
--eval-attack-side "${EVAL_ATTACK_SIDE:-end}" \
|
|
--eval-batch-size "${EVAL_BATCH_SIZE:-4}" \
|
|
--eval-max-new-tokens "${EVAL_MAX_NEW_TOKENS:-256}" \
|
|
${GRADIENT_CHECKPOINTING:+--gradient-checkpointing} \
|
|
${EVAL_AFTER_TRAIN:+--eval-after-train} \
|
|
${EVAL_STEP0:+--eval-step0} \
|
|
${EVAL_MMLU:+--eval-mmlu} \
|
|
${EVAL_TOPICATTACK:+--eval-topicattack}
|