Why does punk music attract neo nazis and anarchists ?

Is it because punk is some kind of extreme music. If that was the case, why didn't heavy metal attract most of these groups ?