Urban scaling and Zipf’s law are two fundamental paradigms for the science of cities. These laws have mostly been investigated independently and are often perceived as disassociated matters. Here we present a large scale investigation about the connection between these two laws using population and GDP data from almost five thousand consistently-defined cities in 96 countries. We empirically demonstrate that both laws are tied to each other and derive an expression relating the urban scaling and Zipf exponents. This expression captures the average tendency of the empirical relation between both exponents, and simulations yield very similar results to the real data after accounting for random variations. We find that while the vast majority of countries exhibit increasing returns to scale of urban GDP, this effect is less pronounced in countries with fewer small cities and more metropolises (small Zipf exponent) than in countries with a more uneven number of small and large cities (large Zipf exponent). Our research puts forward the idea that urban scaling does not solely emerge from intra-city processes, as population distribution and scaling of urban GDP are correlated to each other.