I don't understand and I haven't understood for a while. How is it that all the galaxies in the universe are moving, how they are moving and how fast?
There are several sources of movement for galaxies.
Galaxies may rotate. Galaxies are formed essentially from matter collapsing under gravity into "clumps". This process isn't uniform due to gravity from other clumps of matter and so it is very rare that all the particles in a single clump "fall" directly into it's centre. This creates rotation (Angular Momentum) and so the eventual galaxy will rotate. Note that some galaxies show very little overall rotation, with the individual stars orbiting the centre of the galaxy in a multiple directions.
Galaxies interact with each other. Most galaxies reside within a group or cluster of galaxies, our galaxy is within a group called the Local Group with Andromeda and several ($\geq$52) dwarf galaxies. These dwarf galaxies are usually part of a "satellite-subsystem" where they orbit a large galaxy under gravity. These orbits are affected by the other local galaxies and occasionally some systems can be "expelled" due to the gravitational forces happening to speed the body up past the velocity required to escape the system.
The major galaxies in a group or cluster interact with each other, orbiting a centre of gravity. This is due to the same mechanism as galaxy rotation. This is usually a very dense area of galaxies, or a massive Brightest Cluster Galaxy (BCG). In the local group, the two main galaxies (our Milky Way and the Andromeda Galaxy) are getting closer together and will collide in approximately 4 billion years.
The universe is expanding. The universe is expanding and so galaxies appear to be moving away from us in every direction. We see this as redshift, a change in the wavelength of light as it reaches us due to the difference in speed between the origin an us. This is not really anything to do with galaxies moving, and more to do with the fabric of the universe. Whether this will continue, or whether the universe will stop expanding and collapse is a current active area of research.