Named after the world famous Huangshan Mountain (Yellow Mountian), the city of Huangshan has a history of more than 5,000 years. Its two old counties, She County and Yi County, were established just over 2,000 years ago. 

In the period of Yin and Shang which is apart 3,000 to 4,000 years from now, the first people called Shanyue lived here. The Shanyue was one of the 100 Yues. In the period of Chunqiu and Zhanguo, it belonged to Wu firstly, belonged to Yue after Wu died out, belonged to Chu after Yue died out. After the Qin Shi Huang unified six countries practiced the County System, it belonged to the ground for Huiji County.

Huangshan City was formerly called Xin'an Shire in the Jin Dynasty (265-420AD) when the system of shire and county took shape. Its name was then changed to Huizhou until the Song Dynasty (960-1279). At that time, the city was noted for its Hui business, Hui architecture, Hui carving, the Xin’an school of painting, Hui cuisine and Hui opera. In 1121 A.D., Anzhou was changed for Huizhou. In 1667 when setting up the province, took the first word of Anqing and Huizhou as the name of the province. 

On October 1, 1949, after the founding of the People's Republic of China, Huizhou is still belong to the area of south Anhui. Tunxi City was allotted to the Huizhou particularly office's leadership in 1955. In January 1956, Huizhou  Prefecture merged in Wuhu Prefecture. Tunxi City was changed to be governed directly by the province in 1962. Tunxi City was canceled in 1963, merged in Xiuning County. The Huizhou specialty area changed to call the Huizhou region in 1970. 

In November 1987, Huangshan City was formally established including 3 districts (Tunxi, Huizhou, and Huangshan Districts), 4 counties (She, Yi, Qimen and Xiuning Counties) as well as the Huangshan Scenery Area, with a total area of 9,807 square kilometers.