Reuters has an article about the increasing popularity of the $2 bill here.  The time is definitely ripe to withdraw the $1 bill.  To increase the political palatibility of this move, the penny should also be withdrawn from circulation.