Supervisors/Overseers were sometimes appointed in early wills.
I asked Dr Matthew Tompkins about this. His reply was that:
A supervisor's (or overseer's) role was to make sure the executor implemented the will according to its terms, and to take him to court if he did not, so he protected the interests of all beneficiaries under the will, though no doubt his oversight was particularly important to minors and other beneficiaries less able to stand up for themselves.
In many regions in the early modern period it was absolutely standard to appoint overseers as well as an executor, so in those areas no particular significance can be read into their appointment in individual cases. But in areas where it was unusual then perhaps the appointment of overseers indicates some unease in the testator's mind about the chosen executor(s).
Amy Erickson's Women and Property in Early Modern England has an interesting discussion of this on pp. 159-61, with some statistics. 75% of male testators in Sussex between 1579 and 1682 appointed overseers, but only 4-5% in parts of Yorkshire in the 17C did so.
Note that this use of the term 'overseer' should not be confused with the position of 'Overseer of the Poor'. Entirely different beasties.
I will include examples from Copeland wills as I work on them.
Will of Christopher Pearson of Branthwaite in Dean, 1591