I think our working theory right now is that a combination of the taking control of others (and constantly maintaining control over them) and accessing that power to speak to the Chitauri was particularly draining, but we can't really be sure about it. I figure a loooot of things to do with Loki will be revealed in Thor 2, because (to me at least) it's very, very clear that we didn't even get a sliver of the whole story there.