The site of Iron age, Roman and Saxon forts - the one remaining Roman lighthouse which was once 80 feet high still stands next to the Saxon church which was built in the year 1000. The other lighthouse was built on the Western Heights but has since been built over. The Roman Portus Dubris Fort built in AD130 and rebuilt in the 3rd third century as part of the Saxon Shore defences is now built over underneath the town.

The location of the first Norman fortifications of 1066 is thought to be close to the lighthouse. The walls were built during the 11th and 12th centuries with 41 towers and the massive 95 feet high rectangular keep and the surrounding fortifications, the inner bailey, were built in 1172-77. During the 1220s the main entrance was rebuilt on the west side, the Constable's Gateway, after the original northern entrance had been undermined during a siege. Further gun batteries were added in the 15th and 18th centuries as well as the underground tunnels. It is probably the strongest castle in England and one of the most important, as it has been in constant use for over 900 years. The nuclear protection bunkers in the tunnels deep under the castle were only decommissioned in 1984.

English Heritage, open all year.