Lack of Discretion

Category: When Analytics Works

Humans often use discretion when applying the rules. "Organizational actors establish and re-negotiate trust under messy and uncertain analytic conditions" (Passi and Jackson, 2018) In the case of learning analytics, Zeide (2019) writes that a human instructor might overlook a student's error "if she notices, for example, that the student clearly has a bad cold." By contrast, "Tools that collect information, particularly based on online interactions, don't always grasp the nuances."

The impact of a lack of discretion is magnified by uncertainties in the data that might be recognized by a human but overlooked by the machine. Passi and Jackson (2018) "describe how four common tensions in corporate data science work – (un)equivocal numbers, (counter)intuitive knowledge, (in)credible data, and (in)scrutable models –raise problems of trust, and show the practices of skepticism, assessment, and credibility by which organizational actors establish and re-negotiate trust under uncertain analytic conditions: work that is simultaneously calculative and collaborative."

By contrast, it could be argued that it is humans that lack the discretion shown by machines. The assessments made by AIs may be much more fine-grained than those made by humans. An analytics engine given student health data may well more consistently and reliably make allowances than a teacher could. It could avoid the sort of discretion that introduces biases and unfairness in grading, evaluations, and interactions based on knowledge of irrelevant information about the students (Malouff & Thorsteinsson, 2016).

Examples and Articles

Proceduralizing control and discretion: Human oversight in artificial intelligence policy
"Human oversight provides an attractive, easily implementable and observable procedural safeguard. However, without awareness of its inherent limitations, human oversight is in danger of becoming a value in itself, an empty procedural shell." Direct Link

Do you have another example of Lack of Discretion? Suggest it here

Force:yes