(CNN) -- With two wars weighing on him and troops on the line, Defense Secretary Robert Gates recognizes the gravity of a job he doesn't particularly enjoy.
"Being the secretary of war in a time of war is a very painful thing," Robert Gates says.
"How can you like a job when you go to Walter Reed or Bethesda [military hospitals] and you know you sent those young men and women in harm's way? Every single person in combat today, I sent there, and I never forget that for a second. So, no, I don't enjoy my job," Gates told CBS in an interview that aired on "60 Minutes."
Gates, the only holdover from the Bush administration, has helped reshape the conflicts on the ground in Iraq and Afghanistan.
Bush nominated Gates as the nation's 22nd secretary of defense in December 2006 to replace Donald Rumsfeld, one of the architects of the Iraq War. The president said the country needed a "fresh perspective" on Iraq.
Gates was a member of the Iraq Study Group that Bush appointed earlier to recommend a change of course in the Iraq war. He'd also served as deputy national security adviser to Bush's father during the Gulf War in 1991, and was the youngest person ever to lead the CIA.
In 2005, he turned down Bush's offer to be the director of national intelligence, opting to stay on board as the president of Texas A&M University. In accepting the offer to lead the Department of Defense, Gates said, "I love Texas A&M deeply, but I love our country more."
His position at the university was the best job he'd had, Gates said, despite serving in six presidential administrations.
As secretary of defense under Bush, Gates earned a reputation of fixing problems at the Pentagon -- he was the anti-Rumsfeld, and his time was to be short.
Asked last summer if he would continue serving in another administration, Gates said, "I learned a long time ago never to say never, so my answer is: The circumstances under which I would do that are inconceivable to me."
But in December, Gates reluctantly agreed to remain on board. President Obama expressed a desire for continuity at the Pentagon, given the ongoing conflicts abroad.
"I spent a long time hoping the question would never be popped," Gates admitted the day after Obama announced his nomination.
Comparing the challenges Gates faces with those of his predecessors, Michael O'Hanlon, a national security expert at the Brookings Institution, said he would begin "with the word 'different,' not 'greater.'"
"I think every generation and every political period has a potential tendency toward a little bit of narcissism to think that we are living through the most momentous times and the most challenging times," he said, noting that the institution of the secretary of defense has only been around since 1947.
While the landscape is certainly difficult, O'Hanlon said the current situation is slightly better than when Gates was sworn in.
Gates took on the job facing daunting responsibilities: overseeing a military with more than 100,000 soldiers in Iraq, battling an insurgency and growing sectarian violence. At the time, O'Hanlon said, "we were on the verge of losing the Iraq War, and it was a very divisive time politically in the country."
Two-and-a-half years into the job, Gates' focus has evolved as Obama aims to withdraw most U.S. troops from Iraq by the end of August 2010. But with escalating violence in Afghanistan and Pakistan, Gates has been forced to make some bold moves under Obama's watch.
He just fired the top commander in Afghanistan, and he's proposed what he calls a "reform budget" that shifts funds to support weapons the U.S. military is currently using and away from research and weapons that may be used in future conflicts.
The budget proposal upset lawmakers who are concerned about preserving valuable defense contracts for their districts and states. "This is a reform budget, reflecting lessons learned in Iraq and Afghanistan," Gates said last month. "There's no question that a lot of these decisions will be controversial."
O'Hanlon said one could have a good debate over whether Gates' announcement came at the right time, given the economic crisis, but on the other hand, "there's also an argument as long as he is in that position as secretary, he should take the opportunity to press for what he believes in."
This weekend, Gates was surprisingly candid when asked about his current role.
"The truth of the matter is being secretary of war in a time of war is a very painful thing. And it's not a job anybody should like," Gates said.
O'Hanlon said Gates' attitude toward his obligations is multifaceted.
"One, he genuinely doesn't love it that much and he doesn't have enough of an ego problem ... to just inherently enjoy being the secretary of defense so he can strut around town to the most important cocktail parties. There's none of that in Bob Gates."
Secondly, O'Hanlon noted, when a person says he doesn't really care about having power, it can help how he is viewed. And lastly, "he is getting a little tired," O'Hanlon said, noting it's his third year on the job.
"It's probably about a 90-hour-a-week a job. And obviously there are no unimportant decisions you make in that position," he said.
Gates, 65, said he keeps close track of those who have died or been wounded under his command.
"Since the hour I was sworn in as secretary of defense, 1,327 American men and women in uniform have been killed in combat in Iraq and Afghanistan; 10,443 have been wounded. Each of them is in my thoughts and prayers every day," he said.
As the U.S. increases its troop presence in Afghanistan, it's likely those numbers will continue to rise. Obama has pledged to send another 21,000 troops to Afghanistan.
Pulling no punches, Gates told CBS he doesn't know how long troops will need to stay there.
Asked what it would take to be able to draw them down in four years, Gates said, "You're asking me to make up a fairy story. I mean, I don't know what it would take."
"What it would take is the Afghan army growing and doing its job well. It would take the effectiveness of our own strategy and our own forces. It would take bringing better governance to the country. It would take a lot of different things to have a finite time when we can say we're out of here. And I don't believe in those stories. I've been around too long."