A "Celestron Powerseeker 114AZ" Google turns up old reviews at Amazon.com and .ca. Most are somewhat critical of the mount.
In all honesty that is to be expected at this level. Many amateur astronomers are unhappy with the amount of wiggle in mounts worth many thousands of dollars. A little tweaking might help just don't use much more than finger ...
I am currently using a Celestron Astromaster 130 EQ and I also have a similar type of caps.
Starting from left to right, the first two are caps used to cover eyepiece holder in case the lens is not connected with it, and the last one is a cap for the lens. So I think they might have given you one extra cap as a lens holder.