Skip to content

fix: correct LoRA initialization and forward pass under tensor parallelism#150

Open
chen2021673 wants to merge 4 commits into
masterfrom
lora_ddp_loss
Open

fix: correct LoRA initialization and forward pass under tensor parallelism#150
chen2021673 wants to merge 4 commits into
masterfrom
lora_ddp_loss

Use TP collectives for LoRA A init

adbc82d
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar