If he's affecting your and others' ability to preform well in class then it becomes an academic issue and it's well within your right to tell your concern to the professor. Tell them this guy is making the working environment hostile and ineffective. Though I suggest getting at least part of the rest of the group to back you up about with the professor.