Simon Howard, who ran Castle Howard for 30 years, has been charged with child sex offences.
Mr Howard left Castle Howard in 2014.
A statement released by the family says he strenuously denies the allegations.
He faces two charges when he appears at Scarborough Magistrates Court on Monday (15 February).
The statement said: “Following an initial interview under police caution in July 2018, the Hon Simon Howard of Malton, North Yorkshire, has been charged under the Sexual Offences Act 1956 and is due to appear at Scarborough Magistrates Court on Monday, February 15.
“The Crown Prosecution Service’s charges of indecent assault and incitement to commit an act of gross indecency result from a single alleged incident.
“The incident is said to have taken place in early 1984 in Simon Howard’s home when he was married to his first wife.
“Simon Howard’s whole family are shocked by these allegations and are fully supportive of SH’s confidence that he is innocent of all charges – which he denies in the strongest possible terms.”
In a statement, the estate said: “Simon Howard is no longer involved with Castle Howard, having stepped down as a director and shareholder five years ago.”
A North Yorkshire Police spokesperson said: “A 65 year old man from the Malton area has been charged with two counts of child sex offences.
“The man is due to appear at Scarborough Magistrates Court on Monday February, 15.”
One of Yorkshire’s leading visitor attractions, Castle Howard is also globally famous as the backdrop to series such as Netflix hit Bridgerton and adaptations of the Evelyn Waugh novel Brideshead Revisited.