That's true, with the send-to-IP option, you are sending to whoever answers that IP. Sending to a bitcoin address doesn't have that problem.
The plan is to implement an IP + bitcoin address option that would have the benefits of both. It would still use a different address for each transaction, but the receiver would sign the one-time-use address with the given bitcoin address to prove it belongs to the intended receiver.