My App
Notes

LoRA的基本概念

LoRA的基本概念

从SVD开始

在正式介绍LoR之前, 先看一个SVD的例子

import torch
import numpy as np
_ = torch.manual_seed(0)
d, k = 10, 10
 
# This way we can generate a rank-deficient matrix
W_rank = 2
W = torch.randn(d,W_rank) @ torch.randn(W_rank,k)
print(W)

On this page