I do not have the energy to go into it fully, but you're not actually wrong. IMO he is low-level problematic and people are acting like he is high-key problematic. Most of the shit he's accused of is either laughing at something shitty someone else said and not discouraging them from it, or else badly executed commentary.

Pretty sure he's said a few problematic things himself, too, but his main sin seems to be a willingness to laugh along with the "joke" regardless of how racist/sexist/whateverist the "joke" actually is.