Derivare funzioni di matrici
f:R→Rf′(x)=h→0limhf(x+h)−f(x)∈R
f:\mathbb{R}\to\mathbb{R}\\
f'(x) = \lim_{h\to 0}\frac{f(x+h)-f(x)}{h}\in\mathbb{R}
LIVELLO 1
f:Rn→R∇f(x)=[∂x1f(x),...,∂xnf(x)]∈Rn∂xif(x)=h→0limhf(x1,...,xi−1,xi+h,xi+1,...,xn)−f(x)
f:\mathbb{R}^n\to\mathbb{R}\\
\nabla f(x) = [\partial_{x_1}f(x),...,\partial_{x_n}f(x)]\in\mathbb{R}^n\\
\partial_{x_i} f(x) = \lim_{h\to 0}\frac{f(x_1,...,x_{i-1},x_{i}+h,x_{i+1},...,x_n)-f(x)}{h}
LIVELLO 2
f:Rn→Rmf(x)=[f1(x)⋯fm(x)]Jf(x)=∇f1(x)⋮∇fm(x)∈Rm×n
f:\mathbb{R}^n\to\mathbb{R}^m\\
f(x) = \begin{bmatrix} f_1(x) & \cdots & f_m(x)\end{bmatrix}\\
\,\\
Jf(x) = \begin{bmatrix} \nabla f_1(x)\\ \vdots \\ \nabla f_m(x)\end{bmatrix}\in \mathbb{R}^{m\times n}
LIVELLO 3
f:Rm×n→Resempio : f(A)=21i=1∑mj=1∑nAij,A=[a:1a:2⋯a:n]∈Rm×n,a:=vec(A)=a:1a:2⋮a:n∈Rmn
f:\mathbb{R}^{m\times n}\to\mathbb{R}\\
\text{esempio : }f(A)=\frac{1}{2}\sum_{i=1}^m\sum_{j=1}^n A_{ij},\\
\,\\
A = \begin{bmatrix} a_{:1} & a_{:2} & \cdots & a_{:n} \end{bmatrix}\in\mathbb{R}^{m\times n},\\
a:=\mathrm{vec}(A)=\begin{bmatrix} a_{:1} \\ a_{:2} \\ \vdots \\ a_{:n}\end{bmatrix}\in\mathbb{R}^{mn}
LIVELLO 4
f:Rm×n→Resempio : f(A)=21i=1∑mj=1∑nAij,a:=vec(A)=a:1a:2⋮a:n∈Rmn∇af(A)=[11...1]∈Rmn
f:\mathbb{R}^{m\times n}\to\mathbb{R}\\
\text{esempio : }f(A)=\frac{1}{2}\sum_{i=1}^m\sum_{j=1}^n A_{ij},\\
a:=\mathrm{vec}(A)=\begin{bmatrix} a_{:1} \\ a_{:2} \\ \vdots \\ a_{:n}\end{bmatrix}\in\mathbb{R}^{mn}\\
\,\\
\nabla_a f(A) = \begin{bmatrix} 1 & 1 & ... & 1 \end{bmatrix}\in\mathbb{R}^{mn}
LIVELLO 4
LIVELLO 5
f:Rm×n→Rm×nesempio : f(A)=UAV,U∈Rm×m,V∈Rn×n,a:=vec(A)=a:1a:2⋮a:n∈Rmnvec(f(A))=(VT⊗U)a∈Rmn∂a∂vec(f(A))=VT⊗U∈Rmn×mn.
f:\mathbb{R}^{m\times n}\to\mathbb{R}^{m\times n}\\
\text{esempio : }f(A)=UAV,\,\,U\in\mathbb{R}^{m\times m},\,V\in\mathbb{R}^{n\times n},\\
a:=\mathrm{vec}(A)=\begin{bmatrix} a_{:1} \\ a_{:2} \\ \vdots \\ a_{:n}\end{bmatrix}\in\mathbb{R}^{mn}\\
\mathrm{vec}(f(A))=(V^T\otimes U)a \in\mathbb{R}^{mn}
\,\\
\frac{\partial\mathrm{vec}(f(A))}{\partial a}=V^T\otimes U\in\mathbb{R}^{mn\times mn}.
ESEMPIO
x∈Rn,σ(x):=[sin(x1),...,sin(xn)]F(x)=vTσ(Ax+b)∈R,v∈Rm,A∈Rm×n,b∈RmL(x)=(F(x)−y)2
x\in\mathbb{R}^n,\sigma(x):=[\sin(x_1),...,\sin(x_n)]\\
F(x)=v^T\sigma(Ax+b)\in\mathbb{R},\,\,v\in\mathbb{R}^m,A\in\mathbb{R}^{m\times n},\,b\in\mathbb{R}^m\\
L(x)=(F(x)-y)^2
x
x
σ(Ax+b)
\sigma(Ax+b)
F(x)
F(x)
y
y
L(x)
L(x)
ESEMPIO
x∈Rn,σ(x):=[sin(x1),...,sin(xn)]F(x)=vTσ(Ax+b)∈R,v∈Rm,A∈Rm×n,b∈RmL(x)=(F(x)−y)2
x\in\mathbb{R}^n,\sigma(x):=[\sin(x_1),...,\sin(x_n)]\\
F(x)=v^T\sigma(Ax+b)\in\mathbb{R},\,\,v\in\mathbb{R}^m,A\in\mathbb{R}^{m\times n},\,b\in\mathbb{R}^m\\
L(x)=(F(x)-y)^2
∂vec(A)∂L(x)=?
\frac{\partial L(x)}{\partial \mathrm{vec}(A)}=?
∂vec(A)∂L(x)=2(z−y)∣z=F(x)∂vec(A)∂F(x)=2(F(x)−y)∂vec(A)∂F(x)
\frac{\partial L(x)}{\partial \mathrm{vec}(A)} = 2(z-y)\vert_{z=F(x)}\frac{\partial F(x)}{\partial \mathrm{vec}(A)}
=2(F(x)-y)\frac{\partial F(x)}{\partial \mathrm{vec}(A)}
ESEMPIO
x∈Rn,σ(x):=[sin(x1),...,sin(xn)]F(x)=vTσ(Ax+b)∈R,v∈Rm,A∈Rm×n,b∈RmL(x)=(F(x)−y)2
x\in\mathbb{R}^n,\sigma(x):=[\sin(x_1),...,\sin(x_n)]\\
F(x)=v^T\sigma(Ax+b)\in\mathbb{R},\,\,v\in\mathbb{R}^m,A\in\mathbb{R}^{m\times n},\,b\in\mathbb{R}^m\\
L(x)=(F(x)-y)^2
∂vec(A)∂L(x)=2(z−y)∣z=F(x)∂vec(A)∂F(x)=2(F(x)−y)∂vec(A)∂F(x)
\frac{\partial L(x)}{\partial \mathrm{vec}(A)} = 2(z-y)\vert_{z=F(x)}\frac{\partial F(x)}{\partial \mathrm{vec}(A)}
=2(F(x)-y)\frac{\partial F(x)}{\partial \mathrm{vec}(A)}
∂vec(A)∂F(x)=vT∂vec(A)∂σ(Ax+b))=vTdiag(σ′(Ax+b))∂vec(A)∂(Ax+b)
\frac{\partial F(x)}{\partial \mathrm{vec}(A)}=v^T\frac{\partial \sigma(Ax+b))}{\partial \mathrm{vec}(A)}\\
\,\\
=v^T\mathrm{diag}(\sigma'(Ax+b))\frac{\partial (Ax+b)}{\partial \mathrm{vec}(A)}
ESEMPIO
x∈Rn,σ(x):=[sin(x1),...,sin(xn)]F(x)=vTσ(Ax+b)∈R,v∈Rm,A∈Rm×n,b∈RmL(x)=(F(x)−y)2
x\in\mathbb{R}^n,\sigma(x):=[\sin(x_1),...,\sin(x_n)]\\
F(x)=v^T\sigma(Ax+b)\in\mathbb{R},\,\,v\in\mathbb{R}^m,A\in\mathbb{R}^{m\times n},\,b\in\mathbb{R}^m\\
L(x)=(F(x)-y)^2
∂vec(A)∂L(x)=2(z−y)∣z=F(x)∂vec(A)∂F(x)=2(F(x)−y)∂vec(A)∂F(x)
\frac{\partial L(x)}{\partial \mathrm{vec}(A)} = 2(z-y)\vert_{z=F(x)}\frac{\partial F(x)}{\partial \mathrm{vec}(A)}
=2(F(x)-y)\frac{\partial F(x)}{\partial \mathrm{vec}(A)}
∂vec(A)∂F(x)=vTdiag(σ′(Ax+b))∂vec(A)∂(Ax+b)
\frac{\partial F(x)}{\partial \mathrm{vec}(A)}=v^T\mathrm{diag}(\sigma'(Ax+b))\frac{\partial (Ax+b)}{\partial \mathrm{vec}(A)}
Ax=(xT⊗Im)vec(A),(xT⊗Im)∈Rm×mn
Ax = (x^T\otimes I_m)\mathrm{vec}(A),\,\,(x^T\otimes I_m)\in\mathbb{R}^{m\times mn}
∂vec(A)∂(Ax+b)=xT⊗Im
\frac{\partial (Ax+b)}{\partial \mathrm{vec}(A)} = x^T\otimes I_m
ESEMPIO
x∈Rn,σ(x):=[sin(x1),...,sin(xn)]F(x)=vTσ(Ax+b)∈R,v∈Rm,A∈Rm×n,b∈RmL(x)=(F(x)−y)2
x\in\mathbb{R}^n,\sigma(x):=[\sin(x_1),...,\sin(x_n)]\\
F(x)=v^T\sigma(Ax+b)\in\mathbb{R},\,\,v\in\mathbb{R}^m,A\in\mathbb{R}^{m\times n},\,b\in\mathbb{R}^m\\
L(x)=(F(x)-y)^2
∂vec(A)∂L(x)=2(F(x)−y)[vTdiag(σ′(Ax+b))(xT⊗Im)]
\frac{\partial L(x)}{\partial \mathrm{vec}(A)} = 2(F(x)-y)\left[v^T\mathrm{diag}(\sigma'(Ax+b))(x^T\otimes I_m)\right]
ESEMPIO
Derivare funzioni di matrici