At the end of the 10th century, Viscount Guy 1st of Limoges wed Emma, the daughter of the Viscount of Ségur. The Ségur viscouncy was bound to the Limoges viscouncy, and became one of its major strongholds. The Viscounts of Limoges lived in Ségur quite regularly and invited families of knights such as the Pérusses and the Hélies to settle down there.
In the end of the 13th century, by marriage, the Vicouncy of Limoges fell into the hands of the Brittany Dukes’ family, who passed it on to younger family members. The Breton War of Succession caused much unrest. In the end of the 14th century, in the middle of the Hundred Years War, the fortress of Ségur ended up being confiscated by the King of France, Charles the 6th. In the end, it was handed back to the new Viscount, Jean de Blois (also known as Jean de L’Aigle), who decided to settle down in Ségur and who had many works carried out here.