Of course, these doomsday scenarios are to be expected in election year politics and the president himself makes similarly unsettling predictions about what a Romney presidency would bring. Meanwhile, White House aides say climate change, energy reform and an immigration overhaul are on the top of the president's list for term No. 2.

But one issue in particular has some conservatives especially wary: the Supreme Court. With three justices well into their 70s, it's likely there will be at least one retirement in the next four years on the high court, possibly from the conservative bloc.

"This is what many Republican thought-leaders are most concerned about," said Lewis. "They have lifetime appointments. It's the most powerful way Obama can leave his mark."