ã¯ããã« ããã«ã¡ã¯ãData Strategyæå±ã®å²¡ã§ããã°ã«ãŒãäŒç€ŸBASE BANKã§åæ/ã¢ããªã³ã°ãªã©ãå
ŒåããŠããŸãã ããã¹ãããŒã¿ãç¹åŸŽéã«ãã€äžåè¡¡ããŒã¿åé¡åé¡ãDNNã§è§£ããããªã£ãéãäžèšã®è«æãåèã«ããã®ã§ãã®å
容ã玹ä»ããŸãã https://users.cs.fiu.edu/~chens/PDF/ISM15.pdf äžåè¡¡ããŒã¿åé¡åé¡ã£ãŠãªã«ïŒ äœãããã®ã«ããŽãªãæ©æ¢°åŠç¿ãªã©ã§åé¡äºæž¬ããããšããéãã«ããŽãªããšã®ããŒã¿ä»¶æ°ã«åãããããç¹ã«æ£äŸã®ããŒã¿ã極端ã«å°ãªãã±ãŒã¹ã§äºæž¬ç²ŸåºŠãäžããã«ããããšããåé¡ããã®ããã«åŒãã§ããŸãã äŸ: äžæ£æ±ºæžãšæ£åžžãªæ³šæãäžæ£ååãšå¥å
šãªååãããæ£è
ãšæ£åžžãªæ£è
æ®éã¯ã©ããã£ãŠå¯ŸåŠããã®ïŒ ããŒã¹ãšãªãã¢ãããŒãã¯äžèš3ã€ã«ãŸãšããããŸãã ã¢ãããŒã å
容 ãã¡ãªãã ã¢ã³ããŒãµã³ããªã³ã° 倿°æŽŸããŒã¿ãã©ã³ãã ã«æžãããŠå°æ°æŽŸããŒã¿ãšåäžã«ãã 倿°æŽŸããŒã¿ã®å€ããæšãŠããããæ
å ±æå€±ãçããã ãªãŒããŒãµã³ããªã³ã° å°æ°æŽŸãå¢ããã倿°æŽŸããŒã¿ãšåäžã«ãã äŸ: SMOTE, ADASYN å°æ°æŽŸããŒã¿ã®æ°Žå¢ãã«ãªããããéåŠç¿ãæžå¿µããã æå€±é¢æ°ã®ã«ã¹ã¿ãã€ãº æå€±é¢æ°ã«å¯ŸããŠå€æ°æŽŸããŒã¿ã®ã³ã¹ããå°æ°æŽŸããŒã¿ãšã®å²åã«å¿ããŠå²ãåŒããªã© ããŸããã¡ãªããã«ã€ããŠæèšãããŠãã±ãŒã¹ã¯å°ãªããšæããŸããããããéåŠç¿ã«ãªããããšããå°è±¡ å人çã«ã¯ãã¢ã³ããŒãµã³ããªã³ã°ã®æ
å ±æå€±ãšãã匱ç¹ãã«ããŒããŠãã under sampling + bagging ããã䜿çšããŠããŸããDNNã§ãããšäŒŒããããªã¢ãããŒããã§ããªãã調ã¹ãŠãããšãããåé ã®è«æãçºèŠããŸããã ãããããã§under samplingããã¢ãããŒã ä»å玹ä»ããè«æã§ã¯ããããããäœææã«ã©ãã«ããšã®ãµã³ãã«ãµã€ãºãåãããæ¹æ³ãææ¡ããŠããŸãã ãã®åŠçãããŒã¯äžå³ã®ããã«ãŸãšããããŸãã ãŸãã倿°æŽŸããŒã¿ïŒPåïŒã®ã¿ãããããµã€ãº(ããã§ã¯Nå)åã«åå²ããŠãããããäœæããŸããããã§ããããã1ã€ãããã®ãµã€ãºã¯ïŒP / NïŒåã§ãã ç¶ããŠå°æ°æŽŸããŒã¿ïŒQåïŒããïŒP / NïŒåãéè€ããªãããã«ã©ã³ãã ãµã³ããªã³ã°ããæåã«äœã£ã1åç®ã®ãããããã®ããŒã¿ã«æ··ããŸãããã®ãµã³ããªã³ã°ãç¶ããŠ2,3,4...Nåç®ã®ãããããããšã«ç¹°ãè¿ããã©ãã«ã®æ¯çãåçãªNåã®ããããããäœãããŸãã æ³šæç¹ãšããŠãããããã1åãããã«å¯ŸããŠå°æ°æŽŸããŒã¿ïŒQåïŒããïŒP / NïŒåãéžã³åºããšãã¯é埩å
æœåºãšãªãããããããããåäœã§èŠãã°ããŒã¿ã®éè€ã¯ãããŸããããã ããå¥ã®ããããããäœããšãã«ã¯åãããã«QåããïŒP / NïŒåéžã³åºãããšã«ãªãã®ã§ããããå
šäœãšããŠã¯å°æ°æŽŸããŒã¿ã®éè€ãèš±ãã€ã€ãµã³ããªã³ã°ãããŠããŸãã å®è£
äŸ PyTorchã§ã®å®è£
äŸã瀺ããŸããå³ç€ºãããããããäœæéšåã ãç°¡ç¥åããã³ãŒãã ãšäžèšã®ããã«ãªããŸãã class BinaryBalancedSampler : def __init__ (self, features, labels, n_samples): self.features = features self.labels = labels label_counts = np.bincount(labels) major_label = label_counts.argmax() minor_label = label_counts.argmin() self.major_indices = np.where(labels == major_label)[ 0 ] self.minor_indices = np.where(labels == minor_label)[ 0 ] np.random.shuffle(self.major_indices) np.random.shuffle(self.minor_indices) self.used_indices = 0 self.count = 0 self.n_samples = n_samples self.batch_size = self.n_samples * 2 def __iter__ (self): self.count = 0 while self.count + self.batch_size < len (self.major_indices): # 倿°æŽŸããŒã¿(major_indices)ããã¯é çªã«éžã³åºã # å°æ°æŽŸããŒã¿(minor_indices)ããã¯ã©ã³ãã ã«éžã³åºãæäœãç¹°ãè¿ã indices = self.major_indices[self.used_indices:self.used_indices + self.n_samples].tolist()\ + np.random.choice(self.minor_indices, self.n_samples, replace= False ).tolist() yield torch.tensor(self.features[indices]), torch.tensor(self.labels[indices]) self.used_indices += self.n_samples self.count += self.n_samples * 2 ã³ã¡ã³ãã§ãè§ŠããããŠããâã®ç®æã®ã³ãŒããä»åã®èãšãªãæäœã§ãã indices = self.major_indices[self.used_indices:self.used_indices + self.n_samples].tolist()\ + np.random.choice(self.minor_indices, self.n_samples, replace= False ).tolist() 倿°æŽŸããŒã¿ã®ã€ã³ããã¯ã¹(major_indices)ããã¯é çªã«éžã³åºããå°æ°æŽŸããŒã¿ã®ã€ã³ããã¯ã¹(minor_indices)ããã¯ã©ã³ãã ã«éžã³åºãæäœãããŠããŸããããããã®ã©ãã«ãã n_samples ãã€åãåºããŠçµåããŠããŸãã ãã®æäœã¯åŸã§ã€ãã¬ãŒã¿ãšããŠçšããã®ã§ã def __iter__(self) ã§å®çŸ©ããŠããŸãã yield ã®åºåã¯ããšãã°äžèšã®ããã«ãªããŸãã test_iter = BinaryBalancedSampler(features=train_features, labels=train_labels, n_samples= 50 ) for i in test_iter: feature, label = i print (feature.shape) print (label.shape) print (np.bincount(label)) break > torch.Size([ 100 , 29 ]) > torch.Size([ 100 ]) > [ 50 50 ] 1ã€ãã¬ãŒã·ã§ã³ãããã«ç¹åŸŽéããŒã¿ãšå¯Ÿå¿ããã©ãã«ãè¿ããŠããŸããããã§ã¯ n_samples=50 ãšããã®ã§2ã©ãã«åã§100åã®ããŒã¿ãçæãããŠããŸããã©ãã«ã®ååžã [50 50] ãšåäžã«ãªã£ãŠããã®ã§ãæ³å®éãã®ããããããäœæã§ããŠãããã§ãã å®éš äžåè¡¡ããŒã¿ãæ®éã«åŠç¿ãããæãšãåäžã«ãªã£ããããããã§åŠç¿ãããå Žåã®ç²ŸåºŠã®å·®ãèŠãŠã¿ãŸãã åŠç¿çµæã®å
šå®¹ã¯ ãã¡ãã®git ã«èŒããã®ã§ãããã§ã¯ç°¡åã«èª¬æããŠãããŸãã ååŠç ãµã³ãã«ããŒã¿ã»ããã«ã¯tensorflowã®äžèšãã¥ãŒããªã¢ã«ãšåããã®ãçšããŸããã https://www.tensorflow.org/tutorials/structured_data/imbalanced_data ãã¡ããäžåè¡¡ããŒã¿ãã©ã®ããã«åé¡ããããšããå
容ã«ãªã£ãŠããŸãã import pandas as pd from IPython.display import display raw_df = pd.read_csv( 'https://storage.googleapis.com/download.tensorflow.org/data/creditcard.csv' ) display(raw_df.head()) èªã¿èŸŒãã ããŒã¿ã¯äžèšã®ãããªäžèº«ã«ãªã£ãŠããŠã Class åãåé¡å¯Ÿè±¡ã®ã©ãã«ã§ãã ãã®ã©ãã«ã®ååžãåã£ãŠããããšã確èªããŸãã import numpy as np neg, pos = np.bincount(raw_df[ 'Class' ]) total = neg + pos print ( 'Examples: \n Total: {} \n Positive: {} ({:.2f}% of total) \n ' .format( total, pos, 100 * pos / total)) > Examples: > Total: 284807 > Positive: 492 ( 0.17 % of total) 1ã®ã©ãã«ãå
šäœã®0.17%ãããªããšããåãå
·åã§ãããªãã®äžåè¡¡ããŒã¿ã«ãªã£ãŠããŸãã ç¶ããŠãã¥ãŒããªã¢ã«ãšåãããã«ååŠçããèšç·ŽããŒã¿ãšãã¹ãããŒã¿ã«åå²ããŸãã from sklearn.model_selection import train_test_split # Cleaning cleaned_df = raw_df.copy() # `Time` ã«ã©ã ã¯äžèŠ cleaned_df.pop( 'Time' ) # `Amount` ã«ã©ã ã¯æ°å€ã®ã¬ã³ãžãåºãããã®ã§å¯Ÿæ°å eps = 0.001 cleaned_df[ 'LogAmount' ] = np.log(cleaned_df.pop( 'Amount' ) + eps) # split train and test. train_df, test_df = train_test_split(cleaned_df, test_size= 0.2 , random_state= 0 ) # split label and feature train_labels = np.array(train_df.pop( 'Class' )) test_labels = np.array(test_df.pop( 'Class' )) train_features = np.array(train_df) test_features = np.array(test_df) 䜿çšãããããã¯ãŒã¯ã¯ããªãã·ã³ãã«ã«ããŸããã import torch import torch.nn as nn import torch.nn.functional as F class SimpleNet (nn.Module): def __init__ (self): super (SimpleNet, self).__init__() self.fc1 = nn.Linear( 29 , 32 ) self.fc2 = nn.Linear( 32 , 16 ) self.fc3 = nn.Linear( 16 , 1 ) def forward (self, x): x = F.relu(self.fc1(x)) x = F.relu(self.fc2(x)) x = torch.sigmoid(self.fc3(x)) return x å
¥åãããç¹åŸŽéã29ãŠããããæçµçã«äºæž¬ããã©ãã«ã¯0 or 1ãªã®ã§åºåã¯1ãŠãããã«ããŠããŸãã ãã®ä»ãæå€±é¢æ°ãšãªããã£ãã€ã¶ãŒã¯äžèšãçšããŸããã import torch.optim as optim criterion = nn.BCELoss() # BCELoss: Binary Crossentropy optimizer = optim.Adam(simple_net.parameters(), lr= 1e-3 ) æ®éã«åŠç¿ãããçµæ åŠç¿éšåã®ã³ãŒãã¯äžèšã®ããã«ãªã£ãŠããŸãã PyTorchç¹æã® Dataset, DataLoader ã䜿ã£ãŠãŠèŠæ
£ããªãéšåããããããããŸããããåçŽã«ããããããµã€ãºã¯1000ããšããã¯æ°ã¯2ã§èšç·Žãããã ãã®æ®æ®µéãã®åŠçã§ããèªã¿é£ã°ããŠæ§ããŸããã from torch.utils.data import Dataset, DataLoader class MyDataset (Dataset): def __init__ (self, feature, label): self.feature = feature self.label = label self.data_num = feature.shape[ 0 ] def __len__ (self): return self.data_num def __getitem__ (self, idx): out_feature = self.feature[idx] out_label = self.label[idx] return out_feature, out_label BATCH_SIZE = 1000 trainset = MyDataset(feature=train_features, label=train_labels) trainloader = DataLoader(trainset, batch_size=BATCH_SIZE, shuffle= True ) testset = MyDataset(feature=test_features, label=test_labels) testloader = DataLoader(testset, shuffle= False ) # training for epoch in range ( 2 ): running_loss = 0.0 for i, data in enumerate (trainloader, 0 ): inputs, labels = data # zero the parameter gradients optimizer.zero_grad() # forward + backward + optimize outputs = simple_net(inputs.float()) loss = criterion(outputs, labels.view(- 1 , 1 ).float()) loss.backward() optimizer.step() # print statistics running_loss += loss.item() if i % 100 == 99 : # print every 100 mini-batches print ( '[%d, %5d] loss: %.3f' % (epoch + 1 , i + 1 , running_loss / 100 )) running_loss = 0.0 print ( 'Finished Training' ) 次ã«åŠç¿ãçµãããããã¯ãŒã¯ã§ãã¹ãããŒã¿ã®ã©ãã«ãäºæ³ãããŸãã pred = [] Y = [] for x, y in testloader: with torch.no_grad(): output = simple_net(x.float()) pred += [ 1 if output > 0.5 else 0 ] Y += [ int (l) for l in y] ãã¹ãããŒã¿ã®äºæž¬çµæãconfusion matrixã§å¯èŠåãããšäžèšã®ããã«ãªããŸããã å
šããŒã¿ãã©ãã«0ïŒå€æ°æŽŸïŒãšäºæž¬ããŠããŠãããŸãå
·åã«äžåè¡¡ããŒã¿ã®çœ ã«ããã£ãŠããŸãã ãããããã§under samplingããŠäºæž¬ããçµæ å®è£
äŸã®ã³ãŒãã§æžãã BinaryBalancedSampler ãçšããŠåŠç¿ãããŸãã å°æ°æŽŸã©ãã«ã®ãµã€ãºãèæ
®ããŠãããã§ã¯ããããããµã€ãºã¯100ããšããã¯æ°ã¯2ã«ããŠããŸãã balanced_net = SimpleNet() balanced_loader = BinaryBalancedSampler(features=train_features, labels=train_labels, n_samples= 50 ) criterion = nn.BCELoss() # BCELoss: Binary Crossentropy optimizer = optim.Adam(balanced_net.parameters(), lr= 1e-3 ) # training for epoch in range ( 2 ): running_loss = 0.0 for i, data in enumerate (balanced_loader, 0 ): inputs, labels = data # zero the parameter gradients optimizer.zero_grad() # forward + backward + optimize outputs = balanced_net(inputs.float()) loss = criterion(outputs, labels.view(- 1 , 1 ).float()) loss.backward() optimizer.step() # print statistics running_loss += loss.item() if i % 1000 == 999 : # print every 1000 mini-batches print ( '[%d, %5d] loss: %.3f' % (epoch + 1 , i + 1 , running_loss / 1000 )) running_loss = 0.0 print ( 'Finished Training' ) ãã®ãããã¯ãŒã¯ã§ãã¹ãããŒã¿ãäºæž¬ããããšäžèšã®ããã«ãªããŸããã 倿°æŽŸã©ãã«ã®å·»ã蟌ã¿ãå°ãªãïŒ187 ãµã³ãã«ïŒãå°æ°æŽŸã©ãã«ã®9å²åŒ±ïŒ88ãµã³ãã«ïŒãåœãŠãããšãã§ããŸããã å®éšã®ãŸãšã ãµã³ãã«ã«äœ¿ã£ãäžåè¡¡ã©ãã«ã®ããŒã¿ã»ããã«ã€ããŠã¯ããããããã§under samplingããã¢ãããŒãã¯æå¹ããã§ãã ææ³ ãã®æ¹æ³ã¯ãããããããšã«ã¯åäžãªããŒã¿ã»ãããåŸãããŸããããããå
šäœã§èŠãã°å°æ°æŽŸããŒã¿ãè€æ°åããã¯ã¢ããããã®ã§ãããæå³ãªãŒããŒãµã³ããªã³ã°ãšãªãéåŠç¿ã®å¿é
ããããŸããearly stoppingãªã©ãçµã¿åãããå¿
èŠãããããã§ãã ïŒããšçºè¡šã2015幎ãšããå€ã...ïŒ ãšã¯ããå®è£
ã¯ãšãŠãç°¡åãªãããå®åã§ãµã¯ããšè©Šããç¹ãå人çã«æ°ã«å
¥ã£ãŠããŸãã