In the official response, it was stated that this issue is innocuous because "context matters" and that there is a place for religion in public.
This phrase originally came into existence specifically to spite the perceived "atheism" of the soviets. It was distinctly designed to separate and alienate "nonbelievers." How this phrase symbolizes community harmony is unclear.
It is not advocated that religion be eliminated from the public sphere, but rather that it should be eliminated from government issued/supported currency. It may not be an endorsement of Christianity, but it is an endorsement of theism. How this qualifies as recognizing or protecting secular Americans is unclear.
Also, the President's "personal feelings" are irrelevant to the issue. We prefer justifications and reason.