Where a matter comes before an Appeal Board in which a member of the Appeal Board is directly concerned, the member shall notify his interest to the Minister and shall temporarily cease to act, and the Minister shall, in the prescribed manner, appoint a temporary substitute member of the Appeal Board.