ArXiv Preprint
In fields such as medicine and drug discovery, the ultimate goal of a
classification is not to guess a class, but to choose the optimal course of
action among a set of possible ones, usually not in one-one correspondence with
the set of classes. This decision-theoretic problem requires sensible
probabilities for the classes. Probabilities conditional on the features are
computationally almost impossible to find in many important cases. The main
idea of the present work is to calculate probabilities conditional not on the
features, but on the trained classifier's output. This calculation is cheap,
needs to be made only once, and provides an output-to-probability "transducer"
that can be applied to all future outputs of the classifier. In conjunction
with problem-dependent utilities, the probabilities of the transducer allow us
to find the optimal choice among the classes or among a set of more general
decisions, by means of expected-utility maximization. This idea is demonstrated
in a simplified drug-discovery problem with a highly imbalanced dataset. The
transducer and utility maximization together always lead to improved results,
sometimes close to theoretical maximum, for all sets of problem-dependent
utilities. The one-time-only calculation of the transducer also provides,
automatically: (i) a quantification of the uncertainty about the transducer
itself; (ii) the expected utility of the augmented algorithm (including its
uncertainty), which can be used for algorithm selection; (iii) the possibility
of using the algorithm in a "generative mode", useful if the training dataset
is biased.
K. Dyrland, A. S. Lundervold, P. G. L. Porta Mana
2023-02-21
