We use high-resolution maps of the molecular interstellar medium (ISM) in the centres of 86 nearby galaxies from the millimetre-Wave Interferometric Survey of Dark Object Masses (WISDOM) and Physics at High Angular Resolution in Nearby GalaxieS (PHANGS) surveys to investigate the physical mechanisms setting the morphology of the ISM at molecular cloud scales. We show that early-type galaxies tend to have smooth, regular molecular gas morphologies, while the ISM in spiral galaxy bulges is much more asymmetric and clumpy when observed at the same spatial scales. We quantify these differences using non-parametric morphology measures (Asymmetry, Smoothness, and Gini), and compare these measurements with those extracted from idealized galaxy simulations. We show that the morphology of the molecular ISM changes systematically as a function of various large-scale galaxy parameters, including galaxy morphological type, stellar mass, stellar velocity dispersion, effective stellar mass surface density, molecular gas surface density, star formation efficiency, and the presence of a bar. We perform a statistical analysis to determine which of these correlated parameters best predicts the morphology of the ISM. We find the effective stellar mass surface (or volume) density to be the strongest predictor of the morphology of the molecular gas, while star formation and bars maybe be important secondary drivers. We find that gas self-gravity is not the dominant process shaping the morphology of the molecular gas in galaxy centres. Instead effects caused by the depth of the potential well, such as shear, suppression of stellar spiral density waves, and/or inflow, affect the ability of the gas to fragment.