Linear Regression as Conditional Random Field
Classical linear regression simply uses a linear Gaussian model (CRF).

Gibbs Sampling in Pairwise Markov Networks
Discrete pairwise Markov networks are pretty straightforward to work with analytically. Let's use the canonical overcomplete representation, where the sufficient statistics functions are indicator functions …

Joint vs. Marginal Expectation
They're pretty much the same.