It really shouldn't make that much of a difference, in those terms.
That its not on the TV, yes is an inconvenience, but shouldnt change about how you see the caller ID. The info is still sent. And 90% of the phones will have caller ID on the phone itself. It will still be visible there, and can still choose to not answer when looking at the phone.