Bayesian regression compare to other regressions

https://stats.stackexchange.com/questions/252577/bayes-regression-how-is-it-done-in-comparison-to-standard-regression

Review some Monte Carlo theorems

1.
Suppose the random variable U has a uniform (0,1) distribution.
Let F be a continuous distribution function. Then the random variable X = F^(-1)(U)
has distribution function F.

Note: F(x)=(x-a)/(b-a), for uniform random variable at [a,b] interval. Here F(u)=u.

2.Monte Carlo Integration

generate x1, x2,...,xn from uniform(a,b), then compute Yi = (b - a)g(Xi). Then mean Y is a consistent estimate of the integral

Note: 1. definite integral is a number.

3. Accept-Reject Generation Algorithm

Difference between indefinite and definite integrals.

 Indefinite integrals are functions while definite integrals are numbers. This is quite useful when we calculate Bayesian estimator

EXPECTATION AND THE INDICATOR FUNCTION

Any distribution that has an inverse can be simulated from Uni(0,1)

Proof:

Highest density region

https://stats.stackexchange.com/a/148452/61705

Different likelihoods

Maximum Likelihood

Find β and θ that maximizes L(β, θ|data).

Partial Likelihood

If we can write the likelihood function as:

L(β, θ|data) = L1(β|data) L2(θ|data)

Then we simply maximize L1(β|data).

Profile Likelihood

If we can express θ as a function of β then we replace θ with the corresponding function.

Say, θ = g(β). Then, we maximize:

L(β, g(β)|data)

Marginal Likelihood

We integrate out θ from the likelihood equation by exploiting the fact that we can identify the probability distribution of θ conditional on β.