You don't need to remember all the stuff in the previous books--there are enough gentle but unobtrusive reminders.

And I recommend the graphic novels, too, if you're into those kinds of things in general. They're great! (The world can always use more Peter Grant, Beverly Brook, and Thomas Nightingale.)