As other users mentioned, the most relevant part is when you divide by 0, you get different infinities depending on if you're dividing by +0.0 or -0.0. But even beyond that: the binary representation of floating point numbers [1] necessarily means that there are two values for zero with distinct bit patterns.
Floating point numbers have a dedicated sign bit that specifies the sign, which means that you can flip the sign of any floating point number and get a float with a different bit pattern (and opposite sign). That means that you get necessarily get both +0.0 and -0.0, and they have different internal representation in bits.
This is one of the major advantages of two's complement notation for representing integers: it doesn't have a dedicated sign bit in the same way, so you only get one representation for 0. You can still check the sign by looking at the top bit, but if you flip it for the number 0, you don't get -0 (which doesn't really exist), you get -128 (for 8 bit signed integers).
I hadn't thought about the infinities part. If they had the same behavior in all operations involving other numbers, then I could see how they could nonetheless be the same from a C++ type perspective, but given they behave differently, C++ certainly couldn't consider them the same.
If you keep infinities and NaNs out of a subsystem, then float and double are totally ordered within that subsystem. So, it is a choice. I would not hesitate on that basis.
Of more moment is that floating point variables often represent imprecise values, such that e.g. 1.41 could represent a value that should equal another transcribed as 1.39. The type system will not help you there. You are obliged to code in tolerance the hard way.
