Learning on differential manifolds may involve the optimization of a function of many parameters. In this letter, we deal with Riemannian-gradient-based optimization on a Lie group, namely, the group of unitary unimodular matrices SU(3). In this special case, subalgebras of the associated Lie algebra su(3) may be individuated by computing pair-wise Gell-Mann matrices commutators. Subalgebras generate subgroups of a Lie group, as well as manifold foliation. We show that the Riemannian gradient may be projected over tangent structures to foliation, giving rise to foliation gradients. Exponentiations of foliation gradients may be computed in closed forms, which closely resemble Rodriguez forms for the special orthogonal group SO(3). We thus compare optimization by Riemannian gradient and foliation gradients.