Grossman’s health demand model shows that residents’ health is affected by many factors, such as medical and health services and language ability. Since the territorial and cultural differences are very huge in China, the amount of dialect is also very large. This article uses the 2012 CFPS survey data to examine the impact of local dialect ability on health and the heterogeneity in health returns. The results show that dialect ability has a significant positive effect on self-rated health status, and a unit increase in dialect ability increases the probability of respondents reporting their health as very healthy by 0.45–0.6%. In the analysis, we have addressed the cross-group comparability problem, omission variable problem and reverse causality. We find that the health returns to dialect are much higher than that to mandarin, and unobservable factors bias the estimation a little. Due to the reporting heterogeneity, the health returns to dialect are slightly overvalued. The reason is that individuals use different thresholds when mapping their true underlying level of health to the response scale available in the survey question. In addition, we also examine the main mechanism by which dialect affects individual health condition: the dialect capability can help to increase personal utility and satisfaction on work, life, and even health care, thereby improving their health. Specifically, dialect can improve the efficiency of job search, expand employment opportunities, and raise labor productivity and wage levels finally. In terms of life, dialect will expand social circles and social networks, promote family harmony, and increase life satisfaction. In terms of health care, dialect can influence the preference of access and choice of medical services, the participation rate of medical insurance, and the quality of medical services. For example, people who can speak dialect can acquire higher quality medical services through various channels under established cost constraints. The sub-sample results show that the health returns to dialect are very different. The health returns in Cantonese, Jianghuai and Wu dialect areas are higher, while that in Gan, Min and Xiang dialect zones are relatively small, and North Mandarin, Western Mandarin have no impact. The main reason is that the dialect is a relative important communication tool in Cantonese, Jianghuai and Wu area since the people in these areas are not familiar with Mandarin, so the effect of dialect on health condition is remarkable. The paper contributes to the literature on the effect of language on health, and these results above also have some strong policy implications for the debate on the protection of linguistic and cultural diversity, and the immigrants who are not familiar with local dialect but want to be integrated into local society.