The book by Dvorak et al. 2006 "Chaos and Stability in Planetary Systems" is a good place to start (as with many titles by Dvorak), but reviewing and studying recent(ish) published work about the topic can greatly enhance understanding of the topic. For example:
Malhotra et al. 2011, Chaos and stability of the solar system, PNAS (full text online) - this article explores the notion that there may be a significant element of chaos in the planetary orbits. A small snippet from the abstract:
Over the last two decades, there has come about a recognition that chaotic dynamics is pervasive in the solar system. We now understand that the orbits of small members of the solar system—asteroids, comets, and interplanetary dust—are chaotic and undergo large changes on geological time scales. Are the major planets' orbits also chaotic? The answer is not straightforward, and the subtleties have prompted new questions.
In a similar vein, the article: Barnes and Quinn, 2004, The (In)stability of Planetary Systems, The Astrophysical Journal (full PDF online),
present results of numerical simulations that examine the dynamical stability of known planetary systems, a
star with two or more planets.