NLPNRA (nonlinear programming – Newton Raphson Algorithm) call in IML

CALL NLPNRA( rc, xr, "fun", x0 <,opt, blc, tc, par, "ptit", "grd", "hes">);

rc: positive value indicate optimization succed, negative value indicate failure.
xr, returned parameters.

An every clear and simple introduction on the Househoder method for QR decomposition

https://www5.in.tum.de/lehre/vorlesungen/konkr_math/SS_10/vorl/v4_Householder.pdf

pull oneself up by one's bootstraps

 the name of bootstraps method comes from this phrase

Hermite and Smith forms

http://www.math.ubc.ca/~cass/siegel/Smith.pdf

https://en.wikipedia.org/wiki/Hermite_normal_form

SAS

HERMITE Function

HERMITE (matrix) ;

The HERMITE function uses elementary row operations to compute the Hermite normal form of a matrix. For square matrices this normal form is upper triangular and idempotent.

If the argument is square and nonsingular, the result is the identity matrix. In general the result satisfies the following four conditions (Graybill, 1969):

• It is upper triangular.
• It has only values of 0 and 1 on the diagonal.
• If a row has a 0 on the diagonal, then every element in that row is 0.
• If a row has a 1 on the diagonal, then every off-diagonal element is 0 in the column in which the 1 appears.

The following statements compute an example from Graybill (1969):

a = {3  6  9,
     1  2  5,
     2  4 10};
h = hermite(a);
print h;

http://support.sas.com/documentation/cdl/en/imlug/65547/HTML/default/viewer.htm#imlug_langref_sect260.htm

If the argument is a square matrix, then the Hermite normal form can be transformed into the row-echelon form by rearranging rows in which all values are 0.

The trace of the Hermite matirx is the rank.

Row vector is a function

A row vector is a function takes a column vector  as input and produce a scalar

LOESS/LOWESS Algorithm

https://www.weisang.com/en/documentation/loessandlowessalgorithm_en/

Lagrange Multipliers

We can connect Lagrange to Ridge and Lasso Regression.

http://www.math.harvard.edu/archive/21a_spring_09/PDF/11-08-Lagrange-Multipliers.pdf

Matrix Calculus from http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/intro.html