My wife and I have the same argument every time we go out for dinner. We've been having it for 27 years. She says it's rude to leave the table to use the facilities while there's food in front of you. I say if ya gotta go, ya gotta go. Is there some sort of rule I haven't heard of? Because, aside from my wife, I've never heard anybody else complain.
If you've got to go, I suppose you do, though I can't help wondering why, after 27 years, you haven't just learned to squeeze in a good tinkle immediately after ordering your food. (We are just talking about tinkling here, correct?)
As to whether there's a rule: No, there is not. But if you leave the table with a plate of food in front of you, the food gets cold and your wife has to eat alone. Plus, with apologies to the overactive-bladder set, there's something just kind of weird about taking four bites of food and then heading to the loo.
Listen to your wife, bub. She's a good and patient woman. And maybe lay off the pre-dinner Super Big Gulps, too.