{"cells":[{"metadata":{},"cell_type":"markdown","source":"Reference:https://www.kaggle.com/humananalog/binary-image-classifier-training-demo"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import os, sys, random\nimport numpy as np\nimport pandas as pd\nimport cv2\n#파이토치임포트\nimport torch\n#파이토치 인공신경망 모델의 재료들을 담고 있는 모듈 임포트\nimport torch.nn as nn\n#위의 nn모듈을 함수화 한 모듈 임포트\nimport torch.nn.functional as F\n\nfrom tqdm.notebook import tqdm\n\n%matplotlib inline\nimport matplotlib.pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#torch.cuda.is_available():현재상태에서 cuda를 사용할 수 있는지여부\n#cuda를 사용할 수 있으면  \"cuda:0\"을 아니면 \"cpu\" 반환하여 torch.device에 설정한 후 \"gpu\"라는 변수에 저장 해 놓음\ngpu = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\ngpu","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"이 변수 'gpu'는  나중에 텐서와 가중치에 대한 연산을 CPU와 GPU 중 어디에서 실행할지 결정할 때 사용됨, 아무 것도 안하면 \"CPU\""},{"metadata":{"trusted":true},"cell_type":"code","source":"image_size = 224\nbatch_size = 64","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"batch_size는 모델이 한 번 사용할 때 처리하는 파일의 개수"},{"metadata":{"_cell_guid":"","_uuid":"","trusted":true},"cell_type":"code","source":"crops_dir = \"../input/faces-155/\"\n\nmetadata_df = pd.read_csv(\"../input/deepfakefaces/metadata.csv\")\nmetadata_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(metadata_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"위 얼굴 데이터 중 진짜와 가짜 개수를 출력하자"},{"metadata":{"trusted":true},"cell_type":"code","source":"len(metadata_df[metadata_df.label == \"REAL\"]), len(metadata_df[metadata_df.label == \"FAKE\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"numpy의 randomchoice명령어 사용하여 crops_dir에 있는 파일명 랜덤으로 선택하여 폴더경로와 합쳐서 경로를 변수로 저장"},{"metadata":{"trusted":true},"cell_type":"code","source":"img_path = os.path.join(crops_dir, np.random.choice(os.listdir(crops_dir)))\nplt.imshow(cv2.imread(img_path)[..., ::-1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## The dataset and data loaders"},{"metadata":{},"cell_type":"markdown","source":"Like most other torchvision models, the model we're using (ResNeXt50) requires that input images are normalized using mean and stddev. For making plots, we also define an \"unnormalize\" transform that can take a normalized image and turn it back into regular pixels.\n해석:지금여기서 사용한 모델도,이미지 파일 정규화가 필요하지만 unnormalize(역정규화)도 이미지 표시 및 이전 이미지로 돌아가려면 필요해서 정의한다고 함."},{"metadata":{},"cell_type":"markdown","source":"배열과 리스트를 텐서 자료형으로 변환¶(출처:https://datascienceschool.net/view-notebook/4f3606fd839f4320a4120a56eec1e228/)\n* 리스트를 텐서 자료형으로 바꾸러면 torch.tensor() 또는 torch.as_tensor(), torch.from_numpy() 명령을 사용한다.\n* \n* torch.tensor(): 값 복사(value copy)를 사용하여 새로운 텐서 자료형 인스턴스를 생성한다.\n* torch.as_tensor(): 리스트나 ndarray 객체를 받는다. 값 참조(refernce)를 사용하여 텐서 자료형 뷰(view)를 만든다.\n* torch.from_numpy(): ndarray 객체를 받는다. 값 참조(refernce)를 사용하여 텐서 자료형 뷰(view)를 만든다."},{"metadata":{"trusted":true},"cell_type":"code","source":"#정규화를 위해 torchvision.transform에서 정규화 모듈임포트\nfrom torchvision.transforms import Normalize\n\n#역정규화 클래스 선언\nclass Unnormalize:\n    \"\"\"Converts an image tensor that was previously Normalize'd\n    back to an image with pixels in the range [0, 1].\"\"\"\n    \n    #생성자 정의(mean(평균),std(분산)):__init__(생성자):객체가 생성될 때 자동으로 호출되는 메서드\n    def __init__(self, mean, std):\n        self.mean = mean\n        self.std = std\n\n    def __call__(self, tensor):\n        #view함수:텐서의 원소개수를 유지하면서 모양을 바꾼다.\n        mean = torch.as_tensor(self.mean, dtype=tensor.dtype, device=tensor.device).view(3, 1, 1)\n        std = torch.as_tensor(self.std, dtype=tensor.dtype, device=tensor.device).view(3, 1, 1)\n        return torch.clamp(tensor*std + mean, 0., 1.)\n\n#우리가 사용하는 모델에서 요구하는 각 채널의 시퀀스의 분산과 평규니다\nmean = [0.485, 0.456, 0.406]\nstd = [0.229, 0.224, 0.225]\nnormalize_transform = Normalize(mean, std)\nunnormalize_transform = Unnormalize(mean, std)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"During training, we'll apply data augmentation. In this kernel we just do random horizontal flips, but you can add other image transformations here too, such as rotation, zooming, etc. It's possible to use torchvision transforms for this, or a library such as [imgaug](https://www.github.com/aleju/imgaug), but I rolled my own using OpenCV2."},{"metadata":{},"cell_type":"markdown","source":"데이터확장을 위해 수평으로 뒤집는다. 여기에서는 OpenCV2를 사용하지만 ,torchvision transform 사용가능하다."},{"metadata":{"trusted":true},"cell_type":"code","source":"def random_hflip(img, p=0.5):\n    \"\"\"Random horizontal flip.\"\"\"\n    if random.random() < p:\n        return cv2.flip(img, 1)\n    else:\n        return img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Some helper code for loading a training image and its label:"},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_image_and_label(filename, cls, crops_dir, image_size, augment):\n    #해당되는 파일의 이미지를 텐서값으로 변환, 그 라벨 값을 가져온다.\n    #해당경로에 있는 이미지파일 읽어옴\n    img = cv2.imread(os.path.join(crops_dir, filename))\n    #openCV컬러를 BGR로 저장하는데 matplotlib등에서는 RGB로 저장하므로, BGR->RGB로 바꾸는 함수\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    \n    #확장이 True ,이미지를 수평으로 뒤집는다.\n    if augment: \n        img = random_hflip(img)\n     #이미지파일을 (224,224)사이즈로 조절한다. \n    img = cv2.resize(img, (image_size, image_size))\n    #img파일 파이토치 텐서로 변환하고,차원을(원래이미지파일차원인덱스를 넣음)조절함(괄호안대로 ).이미지는 0~255까지픽셀값이 있으므로 255로 나누어 정규화\n    img = torch.tensor(img).permute((2, 0, 1)).float().div(255)\n    #앞서 정의 한 평균과 분산을 가지고 이미지 값을 정규화함.\n    img = normalize_transform(img)\n#cls(라벨)값이 \"fake\"면 1이고, 아니면 0(real)이다.\n    target = 1 if cls == \"FAKE\" else 0\n    return img, target","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"[OpenCV] BGR 사진을 RGB 사진으로 변환하기 (cvtColor, 파이썬)\n\n\n\n파이썬에서 OpenCV를 사용해서 사진을 matplotlib 으로 화면에 출력하는 방법입니다. 컬러 사진을 OpenCV에서는 BGR 순서로 저장하는데 matplotlib에서는 RGB 순서로 저장합니다. 따라서 BGR을 RGB로 바꾸어 주어야만 사진이 제대로 표시됩니다.\n\n\n\n출처: https://crmn.tistory.com/49 [크롬망간이 글 쓰는 공간]"},{"metadata":{},"cell_type":"markdown","source":"It's always smart to test that the code actually works. The following cell should return a normalized PyTorch tensor of shape (3, 224, 224) and the target 1 (for fake).\n\nNote that this dataset has 155x155 images but our model needs at least 224x224, so we resize them."},{"metadata":{"trusted":true},"cell_type":"code","source":"img, target = load_image_and_label(\"aabuyfvwrh.jpg\", \"FAKE\", crops_dir, 224, augment=True)\nimg.shape, target","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To plot the image, we need to unnormalize it and also permute it from (3, 224, 224) to (224, 224, 3). "},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(unnormalize_transform(img).permute((1, 2, 0)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To use the PyTorch data loader, we need to create a `Dataset` object.\n\nBecause of the class imbalance (many more fakes than real videos), we're using a dataset that samples a given number of REAL faces and the same number of FAKE faces, so it's always 50-50.\n:'Dataset'객체를 활용하여 진짜와 가짜 데이터를 동일한 비율로 맞추자"},{"metadata":{"trusted":true},"cell_type":"code","source":"from torch.utils.data import Dataset\n\nclass VideoDataset(Dataset):\n    \"\"\"Face crops dataset.\n\n    Arguments:\n        crops_dir: 이미지자료가 있는 폴더,\n        df: 데이터프레임(메타데이터가 있는)\n        split: 훈련을 한다면 데이터 확장\n        image_size: 사이즈조절한사이즈\n        sample_size: evenly samples this many videos from the REAL\n            and FAKE subfolders (None = use all videos)\n        seed: 랜덤으로 선택하는 샘플링 상태를 저장하는 숫자,\n    \"\"\"\n    def __init__(self, crops_dir, df, split, image_size, sample_size=None, seed=None):\n        self.crops_dir = crops_dir\n        self.split = split\n        self.image_size = image_size\n        \n        if sample_size is not None:#샘플링할 개수가 있다면\n            real_df = df[df[\"label\"] == \"REAL\"]\n            fake_df = df[df[\"label\"] == \"FAKE\"]\n            #sample_size와, 진짜와 가짜데이터의 길이중 가장 작은 값을 반환한다.\n            sample_size = np.min(np.array([sample_size, len(real_df), len(fake_df)]))\n            print(\"%s: sampling %d from %d real videos\" % (split, sample_size, len(real_df)))\n            print(\"%s: sampling %d from %d fake videos\" % (split, sample_size, len(fake_df)))\n            real_df = real_df.sample(sample_size, random_state=seed)\n            fake_df = fake_df.sample(sample_size, random_state=seed)\n            self.df = pd.concat([real_df, fake_df])\n        else:\n            self.df = df\n\n        num_real = len(self.df[self.df[\"label\"] == \"REAL\"])\n        num_fake = len(self.df[self.df[\"label\"] == \"FAKE\"])\n        print(\"%s dataset has %d real videos, %d fake videos\" % (split, num_real, num_fake))\n\n    def __getitem__(self, index):\n        row = self.df.iloc[index]\n        filename = row[\"videoname\"][:-4] + \".jpg\"\n        cls = row[\"label\"]\n        return load_image_and_label(filename, cls, self.crops_dir, \n                                    self.image_size, self.split == \"train\")\n    def __len__(self):\n        return len(self.df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's test that the dataset actually works..."},{"metadata":{"trusted":true},"cell_type":"code","source":"dataset = VideoDataset(crops_dir, metadata_df, \"val\", image_size, sample_size=1000, seed=1234)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(unnormalize_transform(dataset[0][0]).permute(1, 2, 0))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del dataset","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Split up the data into train / validation. There are many different ways to do this. For this kernel, we're going to just grab a percentage of the REAL faces as well as their corresponding FAKEs. This way, a real video and all the fakes that are derived from it will be either completely in the training set or completely in the validation set. (해석:데이터를 학습 / 검증으로 분할합니다. 이를 수행하는 방법에는 여러 가지가 있습니다. 이 커널에서는 REAL면의 비율과 해당 FAKE를 가져옵니다. 이런 식으로 실제 비디오와 그로부터 파생 된 모든 가짜는 훈련 세트 또는 검증 세트에 완전히 들어갑니다.)\n\n* (This is still not ideal because the same person may appear in many different videos. Ideally we want a person to be either in train or in val, but not in both. But it will do for now.)해석:같은 사람이 여러 다른 비디오에 나타날 수 있기 때문에 여전히 이상적이지 않습니다. 이상적으로 우리는 사람이 train 또는 val에 있고 둘 다에 있지 않기를 원합니다. 그러나 지금은 그렇게 될 것입니다."},{"metadata":{"trusted":true},"cell_type":"code","source":"def make_splits(crops_dir, metadata_df, frac):\n    # Make a validation split. Sample a percentage of the real videos, \n    # and also grab the corresponding fake videos.\n    real_rows = metadata_df[metadata_df[\"label\"] == \"REAL\"]\n    real_df = real_rows.sample(frac=frac, random_state=666)\n    #metadata.csv파일에서 \"original\"열에서 real_df의 \"videoname\"이 안에 있으면 가짜로 보고\"fake_df\"변수에 저장\n    fake_df = metadata_df[metadata_df[\"original\"].isin(real_df[\"videoname\"])]\n    #진짜와 가짜 합치기\n    val_df = pd.concat([real_df, fake_df])\n\n    # The training split is the remaining videos.\n    train_df = metadata_df.loc[~metadata_df.index.isin(val_df.index)]\n\n    return train_df, val_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Sanity check:"},{"metadata":{},"cell_type":"markdown","source":"출처(참조):https://wikidocs.net/21050(03_가정 설정문(assert))\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df, val_df = make_splits(crops_dir, metadata_df, frac=0.05)\n#assert(가정문:가로안의 값이 오류가나면 aseertatin Error 발생)\n#assert()의 조건을 보증한다\n#train데이터의 길이와 val데이터의 길이의 앞은 전체 데이터의 길이와 동일하다는 것을 보증\nassert(len(train_df) + len(val_df) == len(metadata_df))\n#train데이터의 \"videoname\"안에 val데이터의 \"videoname\"이 없다는 것을 보증한다.\nassert(len(train_df[train_df[\"videoname\"].isin(val_df[\"videoname\"])]) == 0)\n\ndel train_df, val_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* Use all of the above building blocks to create `DataLoader` objects. Note that we use only a portion of the full amount of training data, for speed reasons. If you have more patience, increase the `sample_size`.\n* 해석:위의 모든 빌딩 블록을 사용하여 'DataLoader'객체를 만듭니다. 속도상의 이유로 전체 교육 데이터의 일부만 사용합니다. 인내심이 더 있으면`sample_size`를 늘리십시오."},{"metadata":{},"cell_type":"markdown","source":"참조:https://jybaek.tistory.com/799(DataLoader num_workers에 대한 고찰)"},{"metadata":{"trusted":true},"cell_type":"code","source":"#데이터셋으로부터 유의미한 데이터를 뽑아오는 것을 데이터로더라고 함,이 모듈은 임포트해 batch_size지정하면 한 번에 batch_size만큼 불러올 수 있다.\nfrom torch.utils.data import DataLoader\n\ndef create_data_loaders(crops_dir, metadata_df, image_size, batch_size, num_workers):\n    #트레인데이터와 발리데이션 데이터를 나눔\n    train_df, val_df = make_splits(crops_dir, metadata_df, frac=0.05)\n    \n    #위에서 정의한 VideoDataset()함수를 이용해 train_dataset을 만듦\n    train_dataset = VideoDataset(crops_dir, train_df, \"train\", image_size, sample_size=10000)\n    #torch.utils.data에서 제공하는 DataLoader를 사용하여 train_dataset에서 한 번에 batch_size만큼 자료를 가져온다.\n    #DataLoader 속성 값: num_workers:데이터프로세싱에 할당하는 cpu코어개수,pin_memory=True이면 메모리에 샘플을 할당하여 데이터 전송속도를 올릴 수 있다.\n    train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True, \n                              num_workers=num_workers, pin_memory=True)\n\n    val_dataset = VideoDataset(crops_dir, val_df, \"val\", image_size, sample_size=500, seed=1234)\n    val_loader = DataLoader(val_dataset, batch_size=batch_size, shuffle=False, \n                            num_workers=num_workers, pin_memory=True)\n\n    return train_loader, val_loader","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_loader, val_loader = create_data_loaders(crops_dir, metadata_df, image_size, \n                                               batch_size, num_workers=2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* And, as usual, a check that it works... The `train_loader` should give a different set of examples each time you run it (because `shuffle=True`), while the `val_loader` always returns the examples in the same order.\n* 해석:그리고 평소와 같이 작동하는지 확인하십시오 ...`train_loader`는 실행할 때마다 다른 예제 세트를 제공해야합니다 (`shuffle = True` 때문에)`val_loader`는 항상 동일한 예제를 반환합니다."},{"metadata":{"trusted":true},"cell_type":"code","source":"#train_loader기에서 iter함수로 차례로 읽어들인것을 next함수로 반환한다\nX, y = next(iter(train_loader))\nplt.imshow(unnormalize_transform(X[0]).permute(1, 2, 0))\nprint(y[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X, y = next(iter(val_loader))\nplt.imshow(unnormalize_transform(X[0]).permute(1, 2, 0))\nprint(y[0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Helper code for training"},{"metadata":{},"cell_type":"markdown","source":"Evaluation function for running the model on the validation set:"},{"metadata":{"trusted":true},"cell_type":"code","source":"def evaluate(net, data_loader, device, silent=False):\n    net.train(False)\n\n    bce_loss = 0\n    total_examples = 0\n   #with tqdm(,desc=\"\") as pbar:상태바 나타내는 함수\n    with tqdm(total=len(data_loader), desc=\"Evaluation\", leave=False, disable=silent) as pbar:\n        for batch_idx, data in enumerate(data_loader):#데이터로더에 있는 값을 인덱스와 함께 펼침\n            with torch.no_grad():#평가할때 기울기 자동계산안한다.\n                batch_size = data[0].shape[0]\n                #자료의 데이터값과 결과값을 해당 메모리에 보냄\n                x = data[0].to(device)\n                y_true = data[1].to(device).float()\n                \n#모델 net(x)의 결과값(즉,예측값)을 squeeze()함수 사용하여 차원수를 축소한다.\n                y_pred = net(x)\n                y_pred = y_pred.squeeze()\n               #예측값과 실제값의 오차를 구하여 스칼라(.item())값으로 구한값에 batch_size값을 구한 값을 한 배치마다 더함.\n                bce_loss += F.binary_cross_entropy_with_logits(y_pred, y_true).item() * batch_size\n            #총자료개수는 한배치가 돌아갈때마다 누적해서 총자료개수가 됨.\n            total_examples += batch_size\n            #프로그래스바 수정\n            pbar.update()\n    #오차의 총합을 총 개수로 나누면 오차평균이 나옴.\n    bce_loss /= total_examples\n\n    if silent:\n        return bce_loss\n    else:\n        print(\"BCE: %.4f\" % (bce_loss))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Simple training loop. I prefer to write those myself from scratch each time, because then you can tweak it to do whatever you like."},{"metadata":{"trusted":true},"cell_type":"code","source":"def fit(epochs):\n    #전역변수 선언\n    global history, iteration, epochs_done, lr\n    #tqdm 함수와 상태바 설정\n    with tqdm(total=len(train_loader), leave=False) as pbar:\n        #에포크 수만큼 for문 돌림\n        for epoch in range(epochs):\n            #상태창을 초기화\n            pbar.reset()\n            #상태창 출력값 설정\n            pbar.set_description(\"Epoch %d\" % (epochs_done + 1))\n            #로스값과 자료개수 초기화\n            bce_loss = 0\n            total_examples = 0\n            \n            #모델의 훈련을 활성화\n            net.train(True)\n            #훈련데이터로더의 자료를 인덱스 값과 펼친다\n            for batch_idx, data in enumerate(train_loader):\n                #batch_size는 데이터의 첫행의 모양의 첫번째\n                batch_size = data[0].shape[0]\n                #자료값과 실제값을 메모리에 보냄\n                x = data[0].to(gpu)\n                y_true = data[1].to(gpu).float()\n                #경사하강법 기울기(가중치)값 초기화한다.(학습을 시작해야하므로)\n                optimizer.zero_grad()\n                #예측값을 squeeze()함수 사용하여 차원을 축소한다.\n                y_pred = net(x)\n                y_pred = y_pred.squeeze()\n                #오차값:실제값과 예측값의 오차를 binary_cross_entropy를 사용하여 구한다\n                loss = F.binary_cross_entropy_with_logits(y_pred, y_true)\n                #역전파:loss의 기울기의 반대방향으로 이동하여 기울기 개선시킨다.\n                loss.backward()\n                #위의 손실함수가 역전파하는동한 경사하강법으로 기울기 최적화시킨다.\n                optimizer.step()\n                #batch_bce에 loss의 스칼라 값을 저장한다\n                batch_bce = loss.item()\n                #for문이 한번 돌때마다 1batch의 loss총합(batch_bce * batch_size)을 bce_loss에 더해 회전이 끝나면 loss의 총합을 구할 수 있다.\n                bce_loss += batch_bce * batch_size\n                history[\"train_bce\"].append(batch_bce)\n                #총자료개수는 한번돌때마다 batch_size를 더해서 총 자료개수 구함\n                total_examples += batch_size\n                #반복회수 1씩증가\n                iteration += 1\n                #프로세스바 update됨\n                pbar.update()\n            #1epoch돌릴때마다 오차의 총합에 총 파일개수를 나누어 평균오차를 구함\n            bce_loss /= total_examples\n            epochs_done += 1\n\n            print(\"Epoch: %3d, train BCE: %.4f\" % (epochs_done, bce_loss))\n            #validation loss는 evaluate함수 사용하여 함께 구함\n            val_bce_loss = evaluate(net, val_loader, device=gpu, silent=True)\n            history[\"val_bce\"].append(val_bce_loss)\n            \n            print(\"              val BCE: %.4f\" % (val_bce_loss))\n            \n            \n            \n\n            # TODO: can do LR annealing here\n            # TODO: can save checkpoint here\n            #학습률 조정\n            scheduler.step()\n            #에포크마다 모델저장하기\n            torch.save(net.state_dict(), \"epoch:{} val_bce:{:.4f}.pth\".format(epochs_done,val_bce_loss))\n            \n\n            print(\"\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"참조:자동미분(https://wikidocs.net/60754"},{"metadata":{},"cell_type":"markdown","source":"## The model"},{"metadata":{},"cell_type":"markdown","source":"Need to load pretrained ImageNet weights into the model.\n\nYou can get these weights from `https://download.pytorch.org/models/resnext50_32x4d-7cdf4587.pth`, or from [this dataset](https://www.kaggle.com/tony92151/pretrained-pytorch) by Kaggler [tonyguo](https://www.kaggle.com/tony92151)."},{"metadata":{"trusted":true},"cell_type":"code","source":"checkpoint = torch.load(\"../input/externaldata/pretrained-pytorch/resnext50_32x4d-7cdf4587.pth\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import torchvision.models as models\n#torchvision.models에 있는 resnet모델에서 ResNet class 상속\nclass MyResNeXt(models.resnet.ResNet):\n    \n    def __init__(self, training=True):\n        #ResNet 생성자 끌어옴.\n        super(MyResNeXt, self).__init__(block=models.resnet.Bottleneck,\n                                        layers=[3, 4, 6, 3], \n                                        groups=32, \n                                        width_per_group=4)\n\n        self.load_state_dict(checkpoint)\n\n        # Override the existing FC layer with a new one.\n        self.fc = nn.Linear(2048, 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"net = MyResNeXt().to(gpu)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del checkpoint","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Test the model on a small batch to see what its output shape is:"},{"metadata":{"trusted":true},"cell_type":"code","source":"out = net(torch.zeros((10, 3, image_size, image_size)).to(gpu))\nout.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Freeze the early layers of the model:\n**layer4이전 층은 학습시키지 않는다.(미세조정,전이학습)\n**"},{"metadata":{"trusted":true},"cell_type":"code","source":"#모델과 파라미터 이름을 입력하면,입력한 파라미터이전 파라미터는 학습 안함(전이학습)\ndef freeze_until(net, param_name):\n    found_name = False\n    \n    for name, params in net.named_parameters():\n        \n        if name == param_name:\n           \n            found_name = True\n    \n        params.requires_grad = found_name","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"[k for k,v in net.named_parameters()]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"freeze_until(net, \"layer4.0.conv1.weight\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"These are the layers we will train:훈련가능한 파라미터 이름 출력"},{"metadata":{"trusted":true},"cell_type":"code","source":"[k for k,v in net.named_parameters() if v.requires_grad]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Before we train, let's run the model on the validation set. This should give a logloss of about 0.6931."},{"metadata":{"trusted":true},"cell_type":"code","source":"evaluate(net, val_loader, device=gpu)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Training"},{"metadata":{"trusted":true},"cell_type":"code","source":"lr = 0.1\n\nwd=0.000\n\nhistory = { \"train_bce\": [], \"val_bce\": [] }\niteration = 0\nepochs_done = 0\n#경사하강법 adam,weight_decay=L2규제\noptimizer = torch.optim.Adam(net.parameters(), lr=lr, weight_decay=wd)\n#학습률 조절기:stept_size=5(5회전할때마다),gamma=0.1 학습률에 0.1을 곱해준다.\nscheduler=torch.optim.lr_scheduler.StepLR(optimizer,step_size=5,gamma=0.1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"At this point you can load the model from the previous checkpoint. If you do, also make sure to restore the optimizer state! Something like this:\n\n```python\ncheckpoint = torch.load(\"model-checkpoint.pth\")\nnet.load_state_dict(checkpoint)\n\ncheckpoint = torch.load(\"optimizer-checkpoint.pth\")\noptimizer.load_state_dict(checkpoint)\n```"},{"metadata":{},"cell_type":"markdown","source":"Let's start training!"},{"metadata":{"trusted":true},"cell_type":"code","source":"fit(10)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"freeze_until(net, \"layer3.0.conv1.weight\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fit(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#위의 스케줄러 설정으로 대체한다\n# def set_lr(optimizer, lr):\n#     for param_group in optimizer.param_groups:\n#         param_group[\"lr\"] = lr","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#위의 스케줄러 설정으로 대체한다.\n# lr /= 10\n# set_lr(optimizer, lr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Plot training progress. It's nicer to use something like TensorBoard for this, but a simple plot also works. ;-)"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.plot(history[\"train_bce\"])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.plot(history[\"val_bce\"])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**All done!** You can now use this checkpoint in the [inference kernel](https://www.kaggle.com/humananalog/inference-demo)."},{"metadata":{"trusted":true},"cell_type":"code","source":"evaluate(net, val_loader, device=gpu, silent=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# torch.save(net.state_dict(), \"checkpoint.pth\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":""}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.6"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"09e9a18ff5b845b283677c263c011941":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"1dd0ee6d24a947b78efb275a3c0a6108":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"Evaluation: 100%","description_tooltip":null,"layout":"IPY_MODEL_9ffc3fdb423b41a090c2b6a4f35e5935","max":16,"min":0,"orientation":"horizontal","style":"IPY_MODEL_28d7220d3dff4ab5b4916bd9d40a1bc5","value":16}},"28d7220d3dff4ab5b4916bd9d40a1bc5":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"2d1cd44b25d44438a29b38c2c3e09f3e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_30cdfc08ee49486ba313e38f96a31b4c","IPY_MODEL_5de9e98309454a829444c8a7569d60f7"],"layout":"IPY_MODEL_c853445fb2684409906685c1ff2224ff"}},"2ef07498f9aa43a499614d5422f45032":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_3b65a690448e41f8b2acf3ddc6dff3bf","placeholder":"​","style":"IPY_MODEL_e72f1a5dfc3f401eb18cfe5d113d86d9","value":" 313/313 [01:15&lt;00:00,  5.06it/s]"}},"30cdfc08ee49486ba313e38f96a31b4c":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"Epoch 5: 100%","description_tooltip":null,"layout":"IPY_MODEL_09e9a18ff5b845b283677c263c011941","max":313,"min":0,"orientation":"horizontal","style":"IPY_MODEL_a0bb15965c2447c58dca32abe4bb5fa7","value":313}},"3b65a690448e41f8b2acf3ddc6dff3bf":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"4010ae9eb32f4006a17c2594f2474004":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"5de9e98309454a829444c8a7569d60f7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_cd34feac7a74414190aef7c074d08cd9","placeholder":"​","style":"IPY_MODEL_ebe147d3cba242a78952a88ef76edd17","value":" 313/313 [01:14&lt;00:00,  5.15it/s]"}},"78d0b713e3c64950b7c227faf83e7086":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"8967e0f4ce744ce6b0bf08fbb2f8b70e":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"8eef86e1ffcb433d9891c21697c1bb51":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"IntProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"IntProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"","description":"Epoch 10: 100%","description_tooltip":null,"layout":"IPY_MODEL_d36e152f7e8e48cd9262daa7dfb05c9c","max":313,"min":0,"orientation":"horizontal","style":"IPY_MODEL_78d0b713e3c64950b7c227faf83e7086","value":313}},"9ffc3fdb423b41a090c2b6a4f35e5935":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"a0bb15965c2447c58dca32abe4bb5fa7":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":"initial"}},"a2234b7ac1f24734a491760baa8edf67":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c04eee7e5cf441f8919dc51b6bef8007":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c853445fb2684409906685c1ff2224ff":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"cafe5b82724949a29449bcf954fdb209":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c04eee7e5cf441f8919dc51b6bef8007","placeholder":"​","style":"IPY_MODEL_8967e0f4ce744ce6b0bf08fbb2f8b70e","value":" 16/16 [00:04&lt;00:00,  4.73it/s]"}},"cd32ff90db7d45a6b09ddcc5ea349501":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_8eef86e1ffcb433d9891c21697c1bb51","IPY_MODEL_2ef07498f9aa43a499614d5422f45032"],"layout":"IPY_MODEL_a2234b7ac1f24734a491760baa8edf67"}},"cd34feac7a74414190aef7c074d08cd9":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d36e152f7e8e48cd9262daa7dfb05c9c":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"d6177027cd1642c897c817de87979342":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_1dd0ee6d24a947b78efb275a3c0a6108","IPY_MODEL_cafe5b82724949a29449bcf954fdb209"],"layout":"IPY_MODEL_4010ae9eb32f4006a17c2594f2474004"}},"e72f1a5dfc3f401eb18cfe5d113d86d9":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ebe147d3cba242a78952a88ef76edd17":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}}},"version_major":2,"version_minor":0}}},"nbformat":4,"nbformat_minor":4}