Improved measurement of η/η′ mixing in B0(s) → J/ψ η(′) decays

Branching fraction ratios between the decays B0(s) → J/ψ η(′) are measured using proton-proton collision data collected by the LHCb experiment at centre-of-mass energies of 7, 8 and 13 TeV, corresponding to an integrated luminosity of 9 fb^−1. The measured ratios of these branching fractions are BF(B0 → J/ψ η′)/BF(B0 → J/ψ η) = 0.48 ± 0.06 ± 0.02 ± 0.01 and BF(B0s → J/ψ η′)/BF(B0s → J/ψ η) = 0.80 ± 0.02 ± 0.02 ± 0.01, where the uncertainties are statistical, systematic and related to the precision of the η(′) branching fractions, respectively. They are used to constrain the η/η′ mixing angle, ϕ_P, and to probe the presence of a possible glueball component in the η′ meson, described by the gluonic mixing angle ϕ_G. The obtained results are ϕ_P = (41.6{−1.2}{+1.0})º and ϕ_G = (28.1{−4.0}{+3.9})º, where the uncertainties are statistically dominated. While the value of ϕ_P is compatible with existing experimental determinations and theoretical calculations, the angle ϕ_G differs from zero by more than four standard deviations, which points to a substantial glueball component in the η′ meson and/or unexpectedly large contributions from gluon-mediated processes in these decays. The absolute branching fractions are also measured relative to that of the well-established B0s → J/ψ ϕ decay, which serves as the normalisation channel. These results supersede the previous LHCb measurements and are the most precise to date.