Constraining the nature of high frequency peakers. II. Polarization properties