Regulators typically treat frequent violators more harshly. When does such harsh treatment maximize overall compliance? We consider the role of two factors: responsiveness to penalties and costs of sanctions. A novel insight is that maintaining a credible threat of sanction against infrequent violators is relatively cheap because that threat seldom needs to be backed up. In a Clean Water Act application, the marginal sanction deters ten times as many violations when directed at infrequent violators. On net, this is due to a sanction cost effect, not because infrequent violators are marginally more responsive to the threat of punishment.