"If you don't like it, you can get out"

As long as the citizens of a nation have the freedom to choose to leave the country, and avoid doing business within the country, then the taxes and regulations made are not theft.

In this short sentence, your fictional interlocutor is falsely excusing the rank moral perversities that his favorite laws order.

The first thing to note is that, with that "argument", you can defend every perversity known to man.  That alone should point out the rank falsehood of the "argument".

The second thing is that the excuse "if you don't like it, giiiiiiiit outttt" is so trite, that it's been used for every single perversity that mankind has ever inflicted on itself.  You name it, the excuse has been used: war, conscription, robbery, extortion, slavery, assault, Mafia protection rackets, rape.  Heck, even South Park has mocked it openly.

Now here's what's going on when someone tries to deny your moral objections using this excuse:  when you present to him the fact that his system of ideas is morally perverted, then (instead of addressing that perversity) he changes the subject to the red herring "you can giiiit outttt".

It's a red herring -- a distraction tactic -- because the fact that you could avoid a violent threat by fleeing, doesn't change the moral nature of the threat.  This is a particularly potent red herring, too:

1. The statement sneaks into the conversation the false idea that "by not leaving, you're consenting to the aggression".  This is, of course, a manipulative lie too -- your interlocutor is telling you "since you have a choice to avoid being a victim, then you're to blame if you're victimized".  It's called "blame the victim".
2. The statement also relieves your interlocutor from having to morally evaluate the moral corruption he wants to victimize you with.  As long as you're distracted by the red herring, he does not need to address your moral objections.
3. The tactic is also a blatant lie, since the vast majority of people actually don't have the false choice he presents, for a variety of reasons.

The "get out" excuse is very popular and easy to identify, since it's the adult version of the exact same excuse that bad, punitive parents give to their children when they're victimizing their children: "Ah, you didn't eat your vegetables, now I'm going to punish you by not letting you watch TV, you see, you had a choice, if you had eaten your vegetables, you would be watching TV right now".  It's a sad testament to morality that your interlocutor's style of rank manipulation (like most moral perversities) begins at home.

It is because of all the above that I can safely conclude: any person using this excuse is a moral coward, who knows he can't defend his proposals on the basis of ethics, so he resorts to manipulative lies.

In short: a wannabe robber telling people "You could avoid robbery by not walking down my alley" is neither a saint, nor has he turned robbery into charity.  He's just a manipulative robber, and nothing more.