The origin of the Imperial Household Agency can be traced back to the provisions on the government structure in the Taiho Code, which was enacted in 701 under the reign of Emperor Monmu.
In 1889 the Constitution of the Empire of Japan and the Imperial House Law were simultaneously promulgated. The former established the constitutional monarchy while the latter dealt with the internal matters of the Imperial House. They were both supreme rules in their respective spheres and established was the principle that internal matters of the Imperial House were decided by the Imperial House itself without any procedures in the Diet. The Imperial Household Ministry, which was independent of the Cabinet, was to deal with the Imperial House's internal matters and it was confirmed in an Imperial Order in 1908 that the Imperial Household Minister, the Head of the Ministry, assisted the Emperor in all matters concerning the Imperial House.
On 3 May 1947 when the Constitution of Japan came into force, the Imperial Household Ministry became the Imperial Household Office, and was placed under the Prime Minister of Japan. The staff numbers were greatly reduced from an immediate post-war figure of over 6,200 to less than 1,500 and the organization was slimmed.
On 1 June 1949 when the Law Establishing the Prime Minister's Office came into force, the Imperial Household Office became the Imperial Household Agency, an external agency of the Prime Minister's Office.
As a result of the Central Government Reform which took effect on 6 January 2001 the Imperial Household Agency has been placed under the Cabinet Office. The Agency is headed by the Grand Steward and he is assisted by the Vice-Grand Steward. The main organization of the Agency comprises the Grand Steward's Secretariat, Board of the Chamberlains, Crown Prince's Household, Board of the Ceremonies, Archives and Mausolea Department, Maintenance and Works Department and Kyoto Office.