Yeah, and I'm sure SHIELD would've been willing to throw up their hands and let Thor take care of him at pretty much any point in the movie before the wormhole opened, rather than risk having him on their planet any longer. Really, I think that's what makes Loki so tragic to me — the only thing he has to do to stop his evil shit is tell Thor he's tired and doesn't want to fight him any more, and Thor would immediately take him back and do anything to protect him. But he doesn't, because he's too resentful and jealous and stubborn.