Dovery Manor dates from 1450 and was built on the site of a much earlier building built on land assigned to Isabella, the widow of Sir Simon Roges. In the time of Henry VIII it is believed to have been occupied by Walter Popham and may have continued to serve as the dower house to Court Place, which was the only other grand building in the village at that time. Court Place was destroyed by fire in the 1800s, it had occupied an elevated and central position on the road to Porlock Weir.

Dovery Manor would have been a prestigious dwelling, partially made of Welsh stone, with handsome oak beamed ceilings, richly traceried windows and stone fireplaces. It is a fine example of a small 15th C manor house