First construction was in the early 1200s, with many later additions and fortifications in the 13th, 14th and 16th centuries, eventually being abandoned in the 17th century.
In the late 1200s or early 1300s Kilbrannan Chapel (St Brendan's Chapel) was constructed on the nearby shoreline. There are a number of ancient burial slabs in the surrounding graveyard.