In 1585, Sadatsugu Tsutsui built a castle on the plateau where Heirakuji Temple had been located. In 1608, after Sadatsugu's territory was confiscated by Ieyasu Tokugawa, Takatora Todo, who was said to be a master of castle construction, entered Iga. He renovated and expanded the castle to focus on fortifying the western side of it so that it could counter a battle against the Osaka and Toyotomi forces. At that time, a five-story castle tower was built, but it collapsed due to a great storm in 1612. After that, turrets, a main gate, and a palace were built, but the castle tower was never rebuilt.
The current castle keep is a reconstruction one built by Katsu Kawasaki in 1935 with many of his supporters. It was named "Iga Cultural and Industrial Castle" in the hope that it would become a center of culture and industry in Iga. The elegant appearance of the three-story wooden grand castle keep and the two-story small castle keep is familiar to the citizens of Iga.
Inside, there is an exhibition room for materials such as a black lacquer helmet in the shape of a Chinese crown that Takatora Todo received from Hideyoshi Toyotomi, as well as Iga ware masterpieces, armor, samurai furniture, and Basho's hat for traveling. One of the highlights is the 30m high stone wall, which is the tallest in Japan along with Osaka Castle, and has been used for location of movies and TV dramas.