This review detailing the state of the art of searching for intermediate BH candidates, e.g. $m \sim 10^2 - 10^5$ M$_{\odot}$, from electromagnetic observations "for the few hundreds of nearby IMBH candidates found in dwarf galaxies, globular clusters, and ultra-luminous X-ray sources, as well as the possible discovery of a few seed BHs at high redshift."
You might find my answer to the question of Why did astronomers believe most or all stellar black holes had masses no greater than 15 solar masses? helpful.
Why is their mere existence being debated on?
Intermediate mass BHs in active galactic nuclei (AGN) can be difficult to definitively discover, whereas stellar mass BHs in X-ray binaries and supermassive BHs in AGN are comparatively easier to probe, i.e. they typically do not have as strong of a gravitational pull to effect the trajectories of stars and other cosmic material which would produce strong X-ray signature.
The future is bright though! Just like with stellar mass and supermassive BHs, gravitational wave observations are expected to revolutionize the field of intermediate BHs, too. The space based gravitational wave observatory LISA, which will be online in a decade or so (hopefully), is suspected to observe intermediate mass BHs, although one should withhold being too optimistic since this is uncertain. And the coming future of multiband gravitational wave astronomy - low frequency covered by LISA, high frequency covered by LIGO, and the frequencies in between covered by a DECIhertz detector - could be a boon for observing intermediate mass BHs.
In fact, there is a binary black hole merger event detected by LIGO/Virgo, dubbed GW190521, where the resultant post-merger BH had a mass of ~150 M$_{\odot}$, which is the first direct observation of an intermediate mass BH, e.g., m is within $10^2 - 10^5$ M$_{\odot}$ to high confidence, and thus the first conclusive experimental evidence that the intermediate mass range can be populated by mergers of high stellar-mass BHs. How the high stellar-mass BBH system originated is speculative currently: it could have originated from hierarchical mergers of stellar mass progenitor BBHs in a dense stellar cluster, or as a high-mass isolated stellar binary or a stellar binary in an accretion disk, or more exotic possibilities...