Receive a weekly summary and discussion of the top papers of the week by leading researchers in the field.

General General

Efficient Actor-Critic Reinforcement Learning with Embodiment of Muscle Tone for Posture Stabilization of the Human Arm.

In Neural computation

This letter proposes a new idea to improve learning efficiency in reinforcement learning (RL) with the actor-critic method used as a muscle controller for posture stabilization of the human arm. Actor-critic RL (ACRL) is used for simulations to realize posture controls in humans or robots using muscle tension control. However, it requires very high computational costs to acquire a better muscle control policy for desirable postures. For efficient ACRL, we focused on embodiment that is supposed to potentially achieve efficient controls in research fields of artificial intelligence or robotics. According to the neurophysiology of motion control obtained from experimental studies using animals or humans, the pedunculopontine tegmental nucleus (PPTn) induces muscle tone suppression, and the midbrain locomotor region (MLR) induces muscle tone promotion. PPTn and MLR modulate the activation levels of mutually antagonizing muscles such as flexors and extensors in a process through which control signals are translated from the substantia nigra reticulata to the brain stem. Therefore, we hypothesized that the PPTn and MLR could control muscle tone, that is, the maximum values of activation levels of mutually antagonizing muscles using different sigmoidal functions for each muscle; then we introduced antagonism function models (AFMs) of PPTn and MLR for individual muscles, incorporating the hypothesis into the process to determine the activation level of each muscle based on the output of the actor in ACRL. ACRL with AFMs representing the embodiment of muscle tone successfully achieved posture stabilization in five joint motions of the right arm of a human adult male under gravity in predetermined target angles at an earlier period of learning than the learning methods without AFMs. The results obtained from this study suggest that the introduction of embodiment of muscle tone can enhance learning efficiency in posture stabilization disorders of humans or humanoid robots.

Iwamoto Masami, Kato Daichi

2020-Oct-20

General General

Resonator Networks, 1: An Efficient Solution for Factoring High-Dimensional, Distributed Representations of Data Structures.

In Neural computation

The ability to encode and manipulate data structures with distributed neural representations could qualitatively enhance the capabilities of traditional neural networks by supporting rule-based symbolic reasoning, a central property of cognition. Here we show how this may be accomplished within the framework of vector symbolic architectures (VSA) (Plate, 1991; Gayler, 1998; Kanerva, 1996), whereby data structures are encoded by combining high-dimensional vectors with operations that together form an algebra on the space of distributed representations. In particular, we propose an efficient solution to a hard combinatorial search problem that arises when decoding elements of a VSA data structure: the factorization of products of multiple code vectors. Our proposed algorithm, called a resonator network, is a new type of recurrent neural network that interleaves VSA multiplication operations and pattern completion. We show in two examples-parsing of a tree-like data structure and parsing of a visual scene-how the factorization problem arises and how the resonator network can solve it. More broadly, resonator networks open the possibility of applying VSAs to myriad artificial intelligence problems in real-world domains. The companion paper in this issue ("Resonator Networks, 2: Factorization Performance and Capacity Compared to Optimization-Based Methods" by Kent, Frady, Sommer, and Olshausen) presents a rigorous analysis and evaluation of the performance of resonator networks, showing it outperforms alternative approaches.

Frady E Paxon, Kent Spencer J, Olshausen Bruno A, Sommer Friedrich T

2020-Oct-20

General General

Flexible Working Memory through Selective Gating and Attentional Tagging.

In Neural computation

Working memory is essential: it serves to guide intelligent behavior of humans and nonhuman primates when task-relevant stimuli are no longer present to the senses. Moreover, complex tasks often require that multiple working memory representations can be flexibly and independently maintained, prioritized, and updated according to changing task demands. Thus far, neural network models of working memory have been unable to offer an integrative account of how such control mechanisms can be acquired in a biologically plausible manner. Here, we present WorkMATe, a neural network architecture that models cognitive control over working memory content and learns the appropriate control operations needed to solve complex working memory tasks. Key components of the model include a gated memory circuit that is controlled by internal actions, encoding sensory information through untrained connections, and a neural circuit that matches sensory inputs to memory content. The network is trained by means of a biologically plausible reinforcement learning rule that relies on attentional feedback and reward prediction errors to guide synaptic updates. We demonstrate that the model successfully acquires policies to solve classical working memory tasks, such as delayed recognition and delayed pro-saccade/anti-saccade tasks. In addition, the model solves much more complex tasks, including the hierarchical 12-AX task or the ABAB ordered recognition task, both of which demand an agent to independently store and updated multiple items separately in memory. Furthermore, the control strategies that the model acquires for these tasks subsequently generalize to new task contexts with novel stimuli, thus bringing symbolic production rule qualities to a neural network architecture. As such, WorkMATe provides a new solution for the neural implementation of flexible memory control.

Kruijne Wouter, Bohte Sander M, Roelfsema Pieter R, Olivers Christian N L

2020-Oct-20

General General

Selecting the best machine learning algorithm to support the diagnosis of Non-Alcoholic Fatty Liver Disease: A meta learner study.

In PloS one ; h5-index 176.0

BACKGROUND & AIMS : Liver ultrasound scan (US) use in diagnosing Non-Alcoholic Fatty Liver Disease (NAFLD) causes costs and waiting lists overloads. We aimed to compare various Machine learning algorithms with a Meta learner approach to find the best of these as a predictor of NAFLD.

METHODS : The study included 2970 subjects, 2920 constituting the training set and 50, randomly selected, used in the test phase, performing cross-validation. The best predictors were combined to create three models: 1) FLI plus GLUCOSE plus SEX plus AGE, 2) AVI plus GLUCOSE plus GGT plus SEX plus AGE, 3) BRI plus GLUCOSE plus GGT plus SEX plus AGE. Eight machine learning algorithms were trained with the predictors of each of the three models created. For these algorithms, the percent accuracy, variance and percent weight were compared.

RESULTS : The SVM algorithm performed better with all models. Model 1 had 68% accuracy, with 1% variance and an algorithm weight of 27.35; Model 2 had 68% accuracy, with 1% variance and an algorithm weight of 33.62 and Model 3 had 77% accuracy, with 1% variance and an algorithm weight of 34.70. Model 2 was the most performing, composed of AVI plus GLUCOSE plus GGT plus SEX plus AGE, despite a lower percentage of accuracy.

CONCLUSION : A Machine Learning approach can support NAFLD diagnosis and reduce health costs. The SVM algorithm is easy to apply and the necessary parameters are easily retrieved in databases.

Sorino Paolo, Caruso Maria Gabriella, Misciagna Giovanni, Bonfiglio Caterina, Campanella Angelo, Mirizzi Antonella, Franco Isabella, Bianco Antonella, Buongiorno Claudia, Liuzzi Rosalba, Cisternino Anna Maria, Notarnicola Maria, Chiloiro Marisa, Pascoschi Giovanni, Osella Alberto Rubén

2020

Surgery Surgery

Automated Detection of Spinal Schwannomas Utilizing Deep Learning Based on Object Detection from MRI.

In Spine ; h5-index 57.0

STUDY DESIGN : A retrospective analysis of magnetic resonance imaging (MRI) was conducted.

OBJECTIVE : This study aims to develop an automated system for the detection of spinal schwannoma, by employing deep learning based on object detection from MRI. The performance of the proposed system was verified to compare the performances of spine surgeons.

SUMMARY OF BACKGROUND DATA : Several MRI scans were conducted for the diagnoses of patients suspected to suffer from spinal diseases. Typically, spinal diseases do not involve tumors on the spinal cord, although a few tumors may exist at the unexpectable level or without symptom by chance. It is difficult to recognize these tumors; in some cases, these tumors may be overlooked. Hence, a deep learning approach based on object detection can minimize the probability of overlooking these tumors.

METHODS : Data from 50 patients with spinal schwannoma who had undergone MRI were retrospectively reviewed. Sagittal T1 and T2 weighted magnetic resonance imaging (T1WI and T2WI) were used in the object detection training and for validation. You Only Look Once version3 was used to develop the object detection system, and its accuracy was calculated. The performance of the proposed system was compared to that of two doctors.

RESULTS : The accuracies of the proposed object detection based on T1W1, T2W1 and both T1W1 and T2W1 were 80.3%, 91.0%, and 93.5%, respectively. The accuracies of the doctors were 90.2% and 89.3%.

CONCLUSIONS : Automated object detection of spinal schwannoma was achieved. The proposed system yielded a high accuracy that was comparable to that of the doctors.

LEVEL OF EVIDENCE : 4.

Ito Sadayuki, Ando Kei, Kobayashi Kazuyoshi, Nakashima Hiroaki, Oda Masahiro, Machino Masaaki, Kanbara Shunsuke, Inoue Taro, Yamaguchi Hidetoshi, Koshimizu Hiroyuki, Mori Kensaku, Ishiguro Naoki, Imagama Shiro

2020-Oct-19

Surgery Surgery

Machine learning identifies two autophagy-related genes as markers of recurrence in colorectal cancer.

In The Journal of international medical research

OBJECTIVE : Colorectal cancer (CRC) is the most common cancer worldwide. Patient outcomes following recurrence of CRC are very poor. Therefore, identifying the risk of CRC recurrence at an early stage would improve patient care. Accumulating evidence shows that autophagy plays an active role in tumorigenesis, recurrence, and metastasis.

METHODS : We used machine learning algorithms and two regression models, univariable Cox proportion and least absolute shrinkage and selection operator (LASSO), to identify 26 autophagy-related genes (ARGs) related to CRC recurrence.

RESULTS : By functional annotation, these ARGs were shown to be enriched in necroptosis and apoptosis pathways. Protein-protein interactions identified SQSTM1, CASP8, HSP80AB1, FADD, and MAPK9 as core genes in CRC autophagy. Of 26 ARGs, BAX and PARP1 were regarded as having the most significant predictive ability of CRC recurrence, with prediction accuracy of 71.1%.

CONCLUSION : These results shed light on prediction of CRC recurrence by ARGs. Stratification of patients into recurrence risk groups by testing ARGs would be a valuable tool for early detection of CRC recurrence.

Wu Jianping, Liu Sulai, Chen Xiaoming, Xu Hongfei, Tang Yaoping

2020-Oct

Colorectal cancer, autophagy, autophagy-related gene, machine learning, recurrence, regression