ArXiv Preprint
Generative models are designed to address the data scarcity problem. Even
with the exploding amount of data, due to computational advancements, some
applications (e.g., health care, weather forecast, fault detection) still
suffer from data insufficiency, especially in the time-series domain. Thus
generative models are essential and powerful tools, but they still lack a
consensual approach for quality assessment. Such deficiency hinders the
confident application of modern implicit generative models on time-series data.
Inspired by assessment methods on the image domain, we introduce the
InceptionTime Score (ITS) and the Frechet InceptionTime Distance (FITD) to
gauge the qualitative performance of class conditional generative models on the
time-series domain. We conduct extensive experiments on 80 different datasets
to study the discriminative capabilities of proposed metrics alongside two
existing evaluation metrics: Train on Synthetic Test on Real (TSTR) and Train
on Real Test on Synthetic (TRTS). Extensive evaluation reveals that the
proposed assessment method, i.e., ITS and FITD in combination with TSTR, can
accurately assess class-conditional generative model performance.
Alireza Koochali, Maria Walch, Sankrutyayan Thota, Peter Schichtel, Andreas Dengel, Sheraz Ahmed
2022-10-14
