The VIMOS VLT Deep Survey. The Assembly History of the Stellar Mass in Galaxies: from the Young to the Old Universe