I noticed this issue specifically on this talk.
Here's how I found it:
1) As a user, make a comment with a rating on a talk that has no comments or ratings.
2) As a(nother?) user, report the comment.
3) As an admin, remove the reported comment.
4) As a user, view the talk again; you'll see the removed comment's rating is persisted as the talk's average rating.
I've attached a screenshot of what this looks like, where the talk has no comments (and thus no ratings) after the first added comment was removed, yet the average rating reflects that comment's rating.