Do egoist anarchists believe in government?

From what I've heard they believe in "whatever a man has the might to do, he has the right to do." Since man clearly has the might to govern, as he does, does this mean he has the right to? It makes sense, it's simply a bit contradictory to the typical anarchist view.