Called Gey ("the City") by its inhabitants, Harar was founded between the 7th and the 11th century (according to different sources)[citation needed] and emerged as the center of Islamic culture and religion in the Horn of Africa.

It was part of the Adal Sultanate (at times a vassal of Ethiopia) of which it became the capital in 1520 under Abu Bekr ibn Muhammad. From Harar, Ahmad ibn Ibrihim al-Ghazi, also known as "Gragn the Left-handed," launched a war of conquest in the sixteenth century that extended its territory and even threatened the existence of the Christian Ethiopian Empire. His successor, Emir Nur ibn Mujahid, encircled the city with a wall, 4 meters high and with five gates. This wall, called Jugol, is still intact, and is a symbol of the town to the inhabitants.

The sixteenth century was the Golden Age of Harar. The local culture flourished, and many poets lived and wrote there. It also became known for coffee, weaving, basketry and bookbinding. The rulers of Harar also struck its own currency, the earliest possible issues bearing a date that may be read as AH 615 (= AD 1218/19); but definitely by AD 1789 the first coins were issued, and more were issued into the nineteenth century.

The city managed to maintain its independence until 1875, when it was conquered by Egypt. During this period, Arthur Rimbaud lived in the city - his former house now a museum. Ten years later, it regained its independence, but this lasted only two years until 6 January 1887 when the Battle of Chelenqo led to Harar's incorporation into the Negus Menelik II of Ethiopia's growing Empire based in Shewa.

Harar lost some of its commercial importance with the creation of the Addis Ababa - Djibouti Railway, initially intended to run via the city but diverted north of the mountains between Harar and the Awash River to save money. As a result of this, Dire Dawa was founded in 1902 as New Harar.

In 1995, the city and its environs became an Ethiopian region (or kilil) in its own right. A pipeline to carry water to the city from Dire Dawa is currently under construction.