Photo Andrew Tivenan

The first castle was probably built after the Norman invasion, and later redone in stone in the mid 12th century. After rebelling against the King the owner, John Giffard, was executed in 1322 and Brimpsfield was destroyed. The moat is still visible, but no stonework can now be seen.