ArXiv Preprint
It is well established in neuroscience that color vision plays an essential
part in the human visual perception system. Meanwhile, many novel designs for
computer vision inspired by human vision have achieved success in a wide range
of tasks and applications. Nonetheless, how color differences affect machine
vision has not been well explored. Our work tries to bridge this gap between
the human color vision aspect of visual recognition and that of the machine. To
achieve this, we curate two datasets: CIFAR10-F and CIFAR100-F, which are based
on the foreground colors of the popular CIFAR datasets. Together with CIFAR10-B
and CIFAR100-B, the existing counterpart datasets with information on the
background colors of CIFAR test sets, we assign each image based on its color
contrast level per its foreground and background color labels and use this as a
proxy to study how color contrast affects machine vision. We first conduct a
proof-of-concept study, showing the effect of color difference and validate our
datasets. Furthermore, on a broader level, an important characteristic of human
vision is its robustness against ambient changes; therefore, drawing
inspirations from ophthalmology and the robustness literature, we analogize
contrast sensitivity from the human visual aspect to machine vision and
complement the current robustness study using corrupted images with our
CIFAR-CoCo datasets. In summary, motivated by neuroscience and equipped with
the datasets we curate, we devise a new framework in two dimensions to perform
extensive analyses on the effect of color contrast and corrupted images: (1)
model architecture, (2) model size, to measure the perception ability of
machine vision beyond total accuracy. We also explore how task complexity and
data augmentation play a role in this setup. Our results call attention to new
evaluation approaches for human-like machine perception.
Ming-Chang Chiu, Yingfei Wang, Derrick Eui Gyu Kim, Pin-Yu Chen, Xuezhe Ma
2022-12-16
