"Nothing appears more plausible at first sight, nor more ill-founded upon close inspection, than a scheme...of continuing the Chief Magistrate in office for a certain time and then excluding him from it, either for a limited period or forever after." - Alexander Hamilton defending the Founding Fathers' decision not to include a presidential term limit.
The U.S. was founded on, among other things, an ideal of popular sovereignty that has only grown more inclusive as time has gone on, but the restriction on presidential term limits is a significant outlier to this general progression. Not only do term limits restrict the re-elected President to an ineffective "lame duck term," but the prospect of a future election would hold presidents more accountable and reduce the potential for abuse.