I'm afraid this is not straightforward
The amplitude of the gravitational wave strain signal from a merging compact binary (neutrons star or black hole) is
$$h \sim 10^{-22} \left(\frac{M}{2.8M_{\odot}}\right)^{5/3}\left(\frac{0.01{\rm s}}{P}\right)^{2/3}\left(\frac{100 {\rm Mpc}}{d}\right),$$
where $M$ is the total mass of the system in solar masses, $P$ is the instantaneous orbital period in seconds and $d$ is the distance in 100s of Mpc. $h \sim 10^{-22}$ is a reasonable number for the sensitivity of LIGO to gravitational wave strain where it is most sensitive (at frequencies of 30-300 Hz).
So you can see that to increase the observability you can increase the mass, decrease the period or decrease the distance.
But here are the complications. LIGO is only sensitive between about 30-300 Hz and the GW frequencies are twice the orbital frequency. Thus you cannot shorten the period to something very small because it would fall outside the LIGO frequency range and you also cannot increase the mass to something too much bigger than the black holes that have been already seen because they merge before they can attain high enough orbital frequencies to be seen. (The frequency at merger is $\propto M^{-1}$).
A further complication is that the evolution of the signals is more rapid at lower masses. That is - the rate of change of frequency and amplitude increase rapidly with total mass. That is why the recent neutron star merger was detectable for 100s by LIGO, whereas the more massive black hole mergers could only be seen for about 1 second. But what this means is that you have fewer cycles of the black hole signal that can be "added up" to improve the signal to noise, which means that higher mass sources are less detectable than a simple application of the formula I gave above would suggest. A further complication is that there is a geometric factor depending on how the source and detectors are orientated with respect to each other.
OK, these are complications, but the formula can still be used as an approximation. So if we take the GW170817 signal, the total mass was about $2.8M_{\odot}$, the source was at 40 Mpc, so at frequencies of 200 Hz (corresponding to a period of 0.01 s) you might have expected a strain signal of about $3\times 10^{-22}$. This gave a very readily detectable signal. The discovery paper (Abbot et al. 2017) says the "horizon" for detection was approximately 218 Mpc for LIGO-Livingston and 107 Mic for LIGO-Hanford. As the source was much closer than these numbers then it is unsurprising that the detection was strong.
Taking the formula above and a fixed orbital period of 0.01 s, we can see that the horizon distance will scale as $\sim M^{5/3}$. So a $10 M_{\odot} + 10 M_{\odot}$ black hole binary might be seen out to $218 \times (20/2.8)^{5/3} = 5.7$ Gpc (this will be an overestimate by a factor of a few because of the issue of the rapidity of the evolution towards merger that I discussed above).
A more through and technical discussion can be read here, although this is a couple of years out of date and LIGO's reach has been extended by about a factor of five since these calculations were done.