The VIRMOS deep imaging survey. IV. Near-infrared observations