The declining value of the U.S. Dollar over the last 150 years has rendered the value of a penny too low for practical use. The last coin that we discontinued was (at the time) worth more than a dime is today.
In any transaction that involves a physical penny, the value of the penny is smaller than the value of the effort taken to deal with the penny. The penny only makes commerce harder to transact; this is a cost that we all pay, but no one gets to collect, it hurts the economy.
The value of the penny is so low that it costs about 2 cents to make. We are literally making the metal less valuable to make pennies.
The only winner here is the zinc and copper industry. We shouldn't hurt the rest of the economy in order to help out the zinc lobby.
Canada, Australia, and New Zealand did it.