TY - JOUR

T1 - A construction for circulant type dropout designs

AU - Chisaki, Shoko

AU - Fuji-Hara, Ryoh

AU - Miyamoto, Nobuko

N1 - Funding Information:
The authors would like to thank the anonymous reviewers for their helpful comments and suggestions to improve the readability of the paper. This work was supported in part by JSPS KAKENHI Grant Number JP19K11866.
Publisher Copyright:
© 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

PY - 2021

Y1 - 2021

N2 - Dropout is used in deep learning to prevent overlearning. It is a method of learning by invalidating nodes randomly for each layer in the multi-layer neural network. Let V1, V2, … , Vn be mutually disjoint node sets (layers). A multi-layer neural network can be regarded as a union of the complete bipartite graphs K|Vi|,|Vi+1| on two consecutive node sets Vi and Vi+1 for i= 1 , 2 , … , n- 1. The dropout method deletes a random sample of activations (nodes) to zero during the training process. A random sample of nodes also causes irregular frequencies of dropout edges. A dropout design is a combinatorial design on dropout nodes from each layer which balances frequencies of selected edges. The block set of a dropout design is B={{C1|C2|…|Cn}|Ci⊆Vi,Ci≠∅,1≤i≤n} having a balancing condition in consecutive t sub-blocks Ci, Ci+1, … , Ci+t-1, see [3]. If | Vi| and | Ci| are constants for 1 ≤ i≤ n, then the dropout design is called uniform. If a uniform dropout design satisfies the circulant property, then the design can be extended to a design with as many layers as you need. In this paper, we describe a construction for uniform dropout designs of circulant type by using affine geometries.

AB - Dropout is used in deep learning to prevent overlearning. It is a method of learning by invalidating nodes randomly for each layer in the multi-layer neural network. Let V1, V2, … , Vn be mutually disjoint node sets (layers). A multi-layer neural network can be regarded as a union of the complete bipartite graphs K|Vi|,|Vi+1| on two consecutive node sets Vi and Vi+1 for i= 1 , 2 , … , n- 1. The dropout method deletes a random sample of activations (nodes) to zero during the training process. A random sample of nodes also causes irregular frequencies of dropout edges. A dropout design is a combinatorial design on dropout nodes from each layer which balances frequencies of selected edges. The block set of a dropout design is B={{C1|C2|…|Cn}|Ci⊆Vi,Ci≠∅,1≤i≤n} having a balancing condition in consecutive t sub-blocks Ci, Ci+1, … , Ci+t-1, see [3]. If | Vi| and | Ci| are constants for 1 ≤ i≤ n, then the dropout design is called uniform. If a uniform dropout design satisfies the circulant property, then the design can be extended to a design with as many layers as you need. In this paper, we describe a construction for uniform dropout designs of circulant type by using affine geometries.

KW - Affine geometry

KW - Deep learning

KW - Dropout

KW - Dropout design

KW - Split block design

UR - http://www.scopus.com/inward/record.url?scp=85107070244&partnerID=8YFLogxK

U2 - 10.1007/s10623-021-00890-8

DO - 10.1007/s10623-021-00890-8

M3 - Article

AN - SCOPUS:85107070244

JO - Designs, Codes, and Cryptography

JF - Designs, Codes, and Cryptography

SN - 0925-1022

ER -