Conservation of angular momentum doesn't seem correct to me. I think this because radiation is present everywhere, no space is completely void, and no object perfect. Thus all objects will have uneven radiation pressure. My question is there anything in space that is not increasing or decreasing in rotation or is fixed in space? Is the rate in which an object spins is determined by surrounding celestial bodies? Or in other words any object in space depending on its position in the galaxy will slow or increase to an ideal rotation?
Angular momentum does not change unless a torque acts upon a body. So your question is just whether the torque on a body could ever be exactly zero. The answer is of course that it can be arbitrarily small but we can never be sure it is exactly zero.
More interesting questions would have been "does the CMB exert a torque on an object?", or "is it possible to define a state of zero absolute angular momentum?".