(CNN) -- U.S. Defense Secretary Chuck Hagel tried to smooth over the latest dispute with Afghan President Hamid Karzai on Sunday, telling Karzai that any talks with the Taliban have to be led "by the Afghans."