If gravity is just matter traveling through curved spacetime, then wouldn’t something with no acceleration just act like it’s in outer space (just floating)?
You have already answered your question, yes gravity is an effect of Spacetime curvature, it is the curvature of both space and time component that fully explains gravity. For a bit more understanding, remember that everything that has mass (or energy) curves spacetime, the time is the thing that causes acceleration, see the answers to this question to get a clear view on time dilation. For a more intuitive understanding consider the following graphs:

In the above graph where there is not curvature of spacetime and you are at rest, the axis of both time and space/ distance will be perpendicular, So if your are at rest, you will be staying at rest according to Newton's first law.
Now if there is a curvature in spacetime, there time dilation takes place, which will change the angle between your time axis and your space axis, this will give:

Now, you are not moving and not external force is acting on you, but since the angle between your time axis and space axis s changed, you will see that while you move in time you also move through space, this is the concept of gravitational acceleration according to general relativity. And the graph I showed here does not show you the acceleration, to see acceleration, just see that the angle between the two axis changes at every instant, which will give you a curved graph.