This is not Clark Gregg's first rodeo.

The actor has played Agent Phil Coulson, right hand man to Samuel L. Jackson's Nick Fury, in two "Iron Man" movies, as well as "Thor."

Even so, he was in awe of the scope of this summer's first potential blockbuster, "The Avengers."

"I thought 'Iron Man' and 'Thor' were as big as they could get but this surpassed all of them," he told CNN.

Tom Hiddleston, who reprises his "Thor" role as Loki, spoke to us back in October about the changes with his villainous character in the two films.

"He's definitively more menacing. A lot more," he said.

"Loki, in 'Thor,' is a lost prince, and there's a degree of vulnerability and confusion about his identity. In 'The Avengers,' he knows exactly who he is. He's fully self-possessed, and he's here with a particular mission. There's something about Loki that's been expanded. He is an enormously powerful being. He's the god of mischief, and between the end of 'Thor' and the beginning of 'Avengers,' he's evolved. It is as if he's been on three years' worth of military training."

Hiddleston lived up to that on the set.

"There was one day where I managed to slightly mischievously steal Captain America's shield and Thor's hammer. I was parading around the subterranean tunnel with the shield -- and one of the producers saw me and was like, 'What are you doing with those?'"

With a cast including Robert Downey, Jr., Chris Evans, Scarlett Johansson, Chris Hemsworth, Mark Ruffalo and more, Clark Gregg admitted that he was "surprised" they were able to pull off the movie so well.

"Ensemble movies are tough to pull off because it's hard to thread it all together to where you care about the whole thing," he said. "You spend some time with each character and not enough time to make it all about them. On top of it, to throw in the science fiction elements of this world and the identification of the threat, and then the way these people operate together as a team."

But "Avengers," he added, "came together visually beyond my wildest imagination. It's one of my favorite moments I've ever had."

When not on the set in Cleveland, Ohio, and Albuquerque, New Mexico, the group got along splendidly, according to Hiddleston.

"You finish up at work and it's like, 'Does anyone want to have a beer or something?" And we had some fun houses," he recalled. "Chris Hemsworth had a good table tennis [set]. I have to say that Loki beats the crap out of both Thor and Captain America at table tennis."

And then there was the night "where Chris [Hemsworth] sent a round robin text message saying, 'Avengers assemble!' We ended up at a bar in Albuquerque, and it's just the place where everyone goes to hang out on a Saturday night. What was quite interesting was your regular Albuquerque bar-goer sort of looking around going, 'Is that Jeremy Renner doing a lunge on the dance floor? I think it is.'"

But on camera, with so many characters in the movie, one has to be the stand-in to ground things for the audience. To some extent, that falls to Gregg's Coulson, who grew up as a fan of Evans' Captain America. But Gregg said one of the Avengers also takes that role. "The eyes of this movie really seem to come from Steve Rogers [a.k.a. Captain America] and his emergence in this new century."

As for SHIELD, the organization Coulson has worked for over four movies, Gregg said, "You see the purpose of SHIELD and whether it has flaws in its execution, as well as its own moral shortcomings."

Loki, of course, has no use for morals, and Hiddleston pondered the idea of his own group of supervillains: "A little bit of help from Darth Vader if I might want to do this. Hans Gruber from 'Die Hard.' Maybe Scar from 'The Lion King.' Robert Patrick from 'Terminator 2' -- the T-1000. Probably Schwarzenegger from [the first] 'Terminator.' Iago, absolutely [from Shakespeare's 'Othello']. That's a pretty awesome group of people."