Implementing a suggestion from Tom Pfeifer, chairs can now mark completed reviews as inactive. Such reviews are not tallied for computing score averages and are not returned to the author in the decision email. They are shown on the paper page with a light gray background. Marking reviews as inactive can be useful when the review is not considered appropriate or helpful, e.g., because the reviewer scores the paper based on the belief that it falls outside the conference scope or the review is considered to show a lack of technical judgement or professionalism.
You can mark a review as inactive by unsetting the checkbox "Review is included in score computation and sent to author" when editing the review status (e.g., by clicking on "Completed".)