The Royal Malaysian Customs Department was formed wholly as a Federal Department when all the Federated Malay States, Unfederated Malay Stated and the Straits Settlement joined together and was known as Federation of Malaya. However the history of Malaysia's revenue / taxation began much earlier than this. Malaysia's vision became a reality when the Royal Malaysian Customs Department expanded covering Sabah and Sarawak, then conferred as the status of "Royal" on 29.10.1963.

With the expansion of the department's role towards the collection of national revenue through industries and inland revenue, a two floor building was built in 1977, adjacent to the Customs warehouse where all the staff including from the Internal Tax Division and the Personnel and Finance Division were stationed. When the Ipoh Cargo Terminal (ICT) at the St John Road was formed, the Internal Tax Division and the Personnel and Finance Division staff were transferred to the new office at the Wisma Maju UMNO locted at Sultan Idris Shah Road.

The staff started moving in December 1988, but only occupied the office at Wisma Maju UMNO until February 1990, after which they started to shift to the Seri Kinta building at Sultan Idris Shah Road. Both these places were rented by the department. In 1992, the department bought a 2.2 acre piece of land which had 2 separate lots. There were 5 separate buildings on this piece of land. They were the Zaman Motor's showroom, an EON workshop, a B-Trak Sdn. Bhd. workshop, Wira Motor's workshop and a Borneo Company office. Four of these buildings were then renovated and were known as the Ipoh Customs Complex which was occupied by the staff from 30.6.1994.

In July 2001, the state customs personnel moved into the new Wisma Kastam which was completed in May 2001. The building was officially opened by HRH the Sultan of Perak on June 11, 2002.