Why do the planets in the Solar system stay in the same orbital plane?

An earlier question addressed why all planets formed in the same orbital plane, but how is this angle maintained? What prevents the planets from taking on a different orbital plane?

-
Your latest edit asks a different question than has been asked previously. I have made some more edits to bring your question more in line with your recent focus and reopened your question. –  called2voyage Dec 12 '13 at 14:52

the Angular Momentum Conservation Law states that, for any moving body, its angular momentum does not change unless you exercise an external force different from the central force.

For an orbiting body like a planet, this means that Sun's gravity, being the central force, does not modify Angular Momentum, but any other external force will do.

Examples of external forces are collisions or the forces made by Jupiter on another planet, or by Neptune on Pluto.

After the Solar System was formed, these external forces are quite small, and thus does not change greatly the Angular Momentum of any major body. But you can see how passing near a body can alter a comet's orbit.

Moreover, the external forces made by bodies that are in the same plane as an orbiting body does modify the value of its Angular Momentum, but not the direction. This causes that the orbiting body changes its orbit but can not not change planes.

So if you add small forces from objects in the same plane, you end up with no changes to planes.

-

Angular momentum conservation

To put it in more mathematical terms, you can play with the energy and the angular momentum of a bunch of particles orbinting a central mass $M$, given by

$$E = \sum_i m_i \left(\frac{1}{2}v_i^2 - \frac{GM}{r_i}\right),$$

for the energy and

$${\bf I} = \sum_i m_i {\bf r}_i \times {\bf v_i},$$

for the angular momentum. Now, let's try to extremize the energy for a given angular momentum, keeping in mind that the system has to conserve angular momentum, and that collisions between the particles can reduce the energy. One good way to do it is to use Lagrange multiplier

$$\delta E - \lambda\cdot\delta {\bf I} = \sum_i\left[\delta {\bf v}_i \cdot \left({\bf v}_i - \lambda \cdot {\bf r}_i \right) + \delta {\bf r}_i \cdot \left( \frac{GM}{r_i^3} + \lambda \times {\bf v}_i\right)\right],$$

that requires

$$\lambda\cdot{\bf r}_i = 0, \qquad {\bf v}_i = \lambda \times {\bf r}_i, \qquad \lambda^2 = \frac{GM}{r_i^3},$$

that means that all orbits are coplanar and circular.

Is this true in general?

That's the principle. Note, however, that all the planetary systems do not always stay in an orbital plane. Such systems can be explained by Lidov-Kozai oscillations, typically trigger by "high-excentricity migration" of hot Jupiters (Fabrycky, 2012). As far as we know now, we can say that:

• our Solar System is flat!
• planetary systems observed by Kepler are mostly flat (there is kind of an observational bias, due to the transit method);
• planetary systems observed by radial-velocity method are more or less flat (with an mean angle between 10 and 20°);
• planetary systems with hot Jupiters are not flat in general.

More dirty details:

There is an excellent talk by Scott Tremaine, given at ESO last year you could watch online.

-