Witnesses Left the Church
If they really saw the plates, why would most of the Book of Mormon witnesses leave the church?
Each had his own reasons, but the fact is that they continually reaffirmed their testimonies regarding having seen the plates and (in the case of the three witnesses) the angel. None of them left because he no longer believed in the Book of Mormon. Two of the ones who left later returned to the Church. Three of them never left.