The topics of language and subaltern social groups appear throughout Antonio Gramsci's Prison Notebooks. Although Gramsci often associates the problem of political fragmentation among subaltern groups with issues concerning language and common sense, there are only a few notes where he explicitly connects his overlapping analyses of language and subalternity. We build on the few places in the literature on Gramsci that focus on how he relates common sense to the questions of language or subalternity. By explicitly tracing out these relations, we hope to bring into relief the direct connections between subalternity and language by showing how the concepts overlap with respect to Gramsci's analyses of common sense, intellectuals, philosophy, folklore, and hegemony. We argue that, for Gramsci, fragmentation of any social group's 'common sense', worldview and language is a political detriment, impeding effective political organisation to counter exploitation but that such fragmentation cannot be overcome by the imposition of a 'rational' or 'logical' worldview. Instead, what is required is a deep engagement with the fragments that make up subaltern historical, social, economic and political conditions. In our view, Gramsci provides an alternative both to the celebration of fragmentation fashionable in liberal multiculturalism and uncritical postmodernism, as well as other attempts of overcoming it through recourse to some external, transcendental or imposed worldview. This is fully in keeping with, and further elucidates Gramsci's understanding of the importance of effective 'democratic centralism' of the leadership of the party in relation to the rank and file and the popular masses.