My point of view is that first there was the Big Bang singularity, and then the period of inflation which resulted in the observable Universe becoming many times bigger. But was the Universe expanding between the Big Bang singularity and the beginning of inflation?
This is likely unanswerable. Inflation was in part a resolution to a fine tuning problem: without it, it seemed we needed the early universe in a very specific and precisely balanced state to get to where it is today, and there was no solid scientific way to explain why things were so perfectly arranged (other than to simply assert that they were).
With inflation, the state of the universe before the inflationary epoch is fairly irrelevant. Mostly all it needs is that the region that inflates into what we will know as the observable universe had enough time to achieve thermal equilibrium (which we need to explain why the universe looks pretty much the same in all directions, exactly as if things well outside of light speed communication had nonetheless achieved thermal equilibrium at some point). Curvature and such get "smeared out" by the inflation to give us the mostly flat and homogeneous universe we see today.
On a pedantic note, some researchers consider inflation and the big bang to be the same thing. At least in the sense that talking about "before inflation" is scientifically meaningless, so if the Big Bang is the (scientific) beginning then we might as well take it to be inflation.
Inflation has started with the bigbang. In fact for anything to take place you need an expansion right away. If you had a static universe just after the bigbang, it couldn't have formed subatomic particles or atoms, without any increase in size. Inflation was always happening and will always happen. The process is behaving (in terms of duration) like radioactive material, that means that inflation slowy decays wth a halflife of about 10^-32 seconds. I hope this answer suffices.