The penny can't buy items like it used to. Today, vending machines, toll booths, and other machines and businesses don't accept it. In addition, it costs about 2 cents to make one penny.
Other countries have removed the penny from their currency and rounding up sales by five-cent increments, and have seen no loss in currency being used. It may be money now, but it is also a burden.
And for those worried about Lincoln being removed from the money, remember he still is on the $5 bill.