In that formula, $f_1$ and $f_2$ are not the flux of two stars in a binary system. They are the flux from two different stars. In the typical use of that formula $f_2$ is the amount of light coming from a reference star with known magnitude $m_2$, and $f_1$ is the amount of light measured from your star that you are studying. Knowing $m_2$, $f_1$ and $f_2$ you can calculate $m_1$.
If you are observing a binary system, and you don't know either $m_1$ or $m_2$, you can't use the formula.
However, if you know that the two stars in the binary system have the same brightness, and you know the brightness of the system you can work out the brightness of each. Suppose the system has a magnitude of 5.0 Then this means that relative to a magnitude 0 star the total flux from the system (f) is given by
$$ 5.0 - 0 = 2.5 \log(f/F)$$
where F is the flux from a magnitude 0 star, ie $f=F/10^{5.0/2.5} = 0.01F$
As the flux from each star is half the flux from the system, the flux of each star in the binary is $f_* = 0.005F$
And the magnitude of each star is $m_* - 0 = 2.5 \log (f_*/F)$
That is $m_* = 2.5\log(0.005) = 5.75$
That is a close binary in which each star has magnitude 5.75 would appear as a single magnitude 5.0 star.
More generally if the binary has magnitude $m$, and the two stars are assumed to be equally bright with magnitude $m_*$, then:
$$m_*= 2.5\log (10^{m/2.5}/2) = m + 2.5 \log(2) = m+0.75$$