In many articles they speak about 'mean motion resonance', what is the difference with a classic orbital resonance?
I'm not really sure about a "classic orbital resonance" but at least in Murray & Dermott: Solar System Dynamics the concept of a $p+q:p$ orbit-orbit resonance / orbital resonance / coorbital resonance is introduced first by the integer relation arising from the periods of the bodies orbiting the pericentre of the system. This first idea is "loose" as it can be discussed. I would think that there's no difference between a mean motion resonance and an orbital resonance.
If you have any recommandation of books/articles about this topic, I would be glad to hear it.
The same applies to Binney & Tremaine: Galactic Dynamics. These are good references to look at if interested in motion and mean motion resonances.
How can we define that two exoplanets are in resonance? Scientific articles are using the ratio of the orbital periods being related by an integer ratio, but in a book (exoplanet handbook-Michael Perryman) I saw "mean motion resonance: definition is via the librating or oscillating behaviour of some linear combination of resonant arguments".
When two planets are near to be commensurable (also known as a "nominal resonance", even when this does not necessarily imply a capture in resonance) they could be captured in resonance if the migration process is slow enough. This can be seen from level curves of the Resonance Hamiltonian $K$
$$K = -3 \Delta R + R^2 - 2 \sqrt{2R} \cos{r}$$
corresponding to the "Second Fundamental Model of Resonance" (being the Simple Pendulum the First), that arises from simplifications when treating the 3-body problem. Here ($r$,$R$) are canonical variables and $\Delta$ is a dimensionless resonance distance that can be equivalently defined depending on the authors (e.g., see Chiang & Choksi, 2020, eqn. 1 or Terquem & Papaloizou, 2019, eqn. 44). A complete and enlightening aclaration to this respect is given at Petrovich, Malhotra, Tremaine: Planets Near Mean-Motion Resonances, 2013. Paraphrasing a little:
Generally, a planet is said to be "in resonance" if some dynamically significant critical argument librates - this is consistent with Murray & Dermott SSD Chapt. 8, 2000 - (...) This definition has several shortcomings (...): (1) not consistent (...) as planets can be resonant at any period ratio; (2) by canonical transformations one can change the critical argument from libration to circulation so the definition is coordinate-dependent; (3) interaction of nearby resonances arising from the degenerate frequencies in the Kepler problem can cause a critical argument to jump between libration and circulation at irregular intervals.
(1) can be explained from the Hamiltonian K in the article above:
Here $x$ and $y$ are canonical coordinates: momentum and coordinate, respectively, following $(x,y)= \sqrt{2R}(\cos{r}, \sin{r})$. The interesting fact to note is that from Liouville's Theorem the phase-space volume (area in this case) must be conserved then there are "adiabatic invariants" in the problem: if you start a system @ some conditions, given values of $R,r$ for fixed $K$, $\Delta = \Delta(t)$ will vary over time and this implies that a circulating solution could then start to librate, from where a capture could arise.
What is the difference between this or the ratio of periods to define the resonance?
Then $\Delta$ could keep changing over time (e.g., growing for a convergent migration) but a pair of planets could still be in a resonant motion, after passing by a commensurability. Altough I think this question requieres of someone who's more experienced on the subject, a thought about it is that: if one defines an orbital resonance from the period ratios only, there might be troubles justifying why it is a resonant behavior, e.g., if there's no capture (when the resonance is not stable).
By the way, these articles might also be useful:
