The name Abilympics is derived from Ability and Olympics. In 1972, the Japan Association for Employment of the Disabled, under the Ministry of Labour, held the Olympics of Abilities. The aim was to improve the vocational abilities of disabled persons, and to help them reach their highest potential. In 1981, almost a decade later, the First International Abilympics was held in Tokyo to commemorate the United Nations? International Year of Disabled Persons. The success of this event was instrumental in the continuation of the International Abilympics, which are now held every four years. The second International Abilympics were held in Bogota, Colombia in 1985. Here, a proposal was made by the former President of Rehabilitation International, Sir Harry Fang, to establish an International Abilympic Federation (IAF) to enable the holding of International Abilympics on a continuing basis.

Under the dynamic leadership of Sir Harry Fang, a new category of events - leisure and living skills-  was introduced in the Third International Abilympics held in Hong Kong in 1991. Since then, International Abilympics have been held in Australia (1995) and Prague, The Czech Republic (2000), New Delhi, India  (2003), Shizuoka, Japan  (2007). The Eighth International Abilympics is scheduled to take place in Seoul, South Korea in 2011.