PATHWISE COORDINATE OPTIMIZATION

We consider "one-at-a-time" coordinate-wise descent algorithms for a class of convex optimization problems. An algorithm of this kind has been proposed for the L-1-penalized regression (lasso) in the literature, but it seems to have been largely ignored. Indeed. it seems (hat coordinate-wise algorithms are not often Used in convex optimization. We show that this algorithm is very competitive with the well-known LARS (or homotopy) procedure in large lasso problems, and that it call be applied to related methods such as the garotte and elastic net. It turns out that coordinate-wise descent does not work in the "Fused lasso." however. so we derive a generalized algorithm that yields the solution in much less time that a standard convex optimizer. Finally. we generalize the procedure to the two-dimensional fused lasso, and demonstrate its performance oil some image smoothing problems.