{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":5048,"databundleVersionId":868335,"sourceType":"competition"}],"dockerImageVersionId":30746,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Distracted driver detection for traffic safety\n\nMany accidents are the results of driver distractions.\n\nIn order to enforce safety, we will use AI, specifically computer </br>\nvision techniques to automatically detect driver distractions.\n\nThis system can be used to automatically penalize distracted behaviors while</br>\ndriving to enforce safety.\n\nWe will proceed in 3 steps:\n\n- Data preparation and visualization\n- Model training\n- Model usage or pseudo deployment","metadata":{}},{"cell_type":"code","source":"from fastai.vision.all import *\nimport timm\nimport pandas as pd","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:11:57.897993Z","iopub.execute_input":"2024-08-20T15:11:57.898385Z","iopub.status.idle":"2024-08-20T15:11:57.905976Z","shell.execute_reply.started":"2024-08-20T15:11:57.898352Z","shell.execute_reply":"2024-08-20T15:11:57.905048Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# I. Data Preparation\n\nThis part is about exploring and getting a basic understanding of the data.","metadata":{}},{"cell_type":"markdown","source":"## 1. Folder organization","metadata":{}},{"cell_type":"code","source":"! tree -L 2 /kaggle/input/state-farm-distracted-driver-detection/","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:11:57.907754Z","iopub.execute_input":"2024-08-20T15:11:57.908066Z","iopub.status.idle":"2024-08-20T15:11:58.952125Z","shell.execute_reply.started":"2024-08-20T15:11:57.908040Z","shell.execute_reply":"2024-08-20T15:11:58.950866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"! tree -L 1 /kaggle/input/state-farm-distracted-driver-detection/imgs/train","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:11:58.953858Z","iopub.execute_input":"2024-08-20T15:11:58.954167Z","iopub.status.idle":"2024-08-20T15:11:59.993808Z","shell.execute_reply.started":"2024-08-20T15:11:58.954140Z","shell.execute_reply":"2024-08-20T15:11:59.992493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"! ls /kaggle/input/state-farm-distracted-driver-detection/imgs/train/c0 | head  -n 5","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:11:59.995632Z","iopub.execute_input":"2024-08-20T15:11:59.995978Z","iopub.status.idle":"2024-08-20T15:12:01.056674Z","shell.execute_reply.started":"2024-08-20T15:11:59.995949Z","shell.execute_reply":"2024-08-20T15:12:01.055474Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_path = '/kaggle/input/state-farm-distracted-driver-detection/'\nfiles = get_image_files(f'{data_path}/imgs/train')\nfiles[:3]","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:01.060663Z","iopub.execute_input":"2024-08-20T15:12:01.061010Z","iopub.status.idle":"2024-08-20T15:12:12.809378Z","shell.execute_reply.started":"2024-08-20T15:12:01.060982Z","shell.execute_reply":"2024-08-20T15:12:12.808424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('class:', str(files[0]).split('/')[-2])\nImage.open(files[0])","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:12.810805Z","iopub.execute_input":"2024-08-20T15:12:12.811455Z","iopub.status.idle":"2024-08-20T15:12:12.941627Z","shell.execute_reply.started":"2024-08-20T15:12:12.811420Z","shell.execute_reply":"2024-08-20T15:12:12.940675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Re-organize the dataframe\n\nHere we want to modify the provided dataframe such that it will embed all information for next steps.\n\n\nAmong them are:\n- the relative path to the file from a reference path.\n- Using the full class name instead of the encoded ones: c0, c1, etc...","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(f'{data_path}/driver_imgs_list.csv')\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:12.942805Z","iopub.execute_input":"2024-08-20T15:12:12.943087Z","iopub.status.idle":"2024-08-20T15:12:12.977980Z","shell.execute_reply.started":"2024-08-20T15:12:12.943061Z","shell.execute_reply":"2024-08-20T15:12:12.977097Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- **img**: image file name.\n- **subject**: unique ID to identify the driver as a single driver might have many images.\n- **classname**: the category of distraction.\nFrtom the description of the dataset:\n\n\n    c0: safe driving\n    c1: texting - right\n    c2: talking on the phone - right\n    c3: texting - left\n    c4: talking on the phone - left\n    c5: operating the radio\n    c6: drinking\n    c7: reaching behind\n    c8: hair and makeup\n    c9: talking to passenger\n","metadata":{}},{"cell_type":"code","source":"df.loc[:, 'img'] = df.classname +'/' + df.img\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:12.979034Z","iopub.execute_input":"2024-08-20T15:12:12.979289Z","iopub.status.idle":"2024-08-20T15:12:12.997681Z","shell.execute_reply.started":"2024-08-20T15:12:12.979267Z","shell.execute_reply":"2024-08-20T15:12:12.996759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"full_name = {\n    'c0': \"safe_driving\",\n    'c1':\" texting_right\",\n    'c2': \"talking_on_the_phone_right\",\n    'c3': \"texting_left\",\n    'c4': \"talking_on_the_phone_left\",\n    'c5': \"operating_the_radio\",\n    'c6': \"drinking\",\n    'c7': \"reaching_behind\",\n    'c8': \"hair_makeup\",\n    'c9': \"talking_to_passenger\"\n}\n\ndf.loc[:, 'classname'] = df.classname.map(full_name)\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:12.998807Z","iopub.execute_input":"2024-08-20T15:12:12.999087Z","iopub.status.idle":"2024-08-20T15:12:13.017342Z","shell.execute_reply.started":"2024-08-20T15:12:12.999063Z","shell.execute_reply":"2024-08-20T15:12:13.016432Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 3. Splitting the data\n\nOne of the most important part of developing a model.</br>\nA lot of bias, ethics problems we hear these days have some of their</br>\nroot causes in the splitting that underlies the sampling process.\n\nHere we have a subject information on top of the class. Thus our splitting must take into account</br>\nthe class and the subject. We should balance the data per class and make sure that images</br>\nfrom one subject are not scattered between train and validation (data split leakage).","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import StratifiedGroupKFold\nsgkf = StratifiedGroupKFold(n_splits=4, shuffle=True, random_state=10)\nfold_iter = sgkf.split(df.img, df.classname, df.subject)\nfor train_index, valid_index in fold_iter:\n    break","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:13.018679Z","iopub.execute_input":"2024-08-20T15:12:13.019141Z","iopub.status.idle":"2024-08-20T15:12:13.121621Z","shell.execute_reply.started":"2024-08-20T15:12:13.019098Z","shell.execute_reply":"2024-08-20T15:12:13.120755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test data split leakage ","metadata":{}},{"cell_type":"code","source":"set(df.loc[train_index].subject.unique()) & set(df.loc[valid_index].subject.unique())","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:13.122983Z","iopub.execute_input":"2024-08-20T15:12:13.123311Z","iopub.status.idle":"2024-08-20T15:12:13.136498Z","shell.execute_reply.started":"2024-08-20T15:12:13.123283Z","shell.execute_reply":"2024-08-20T15:12:13.135365Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Check the class balance ","metadata":{}},{"cell_type":"code","source":"df.loc[train_index].classname.value_counts().plot(kind='bar')","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:13.137598Z","iopub.execute_input":"2024-08-20T15:12:13.137936Z","iopub.status.idle":"2024-08-20T15:12:13.456048Z","shell.execute_reply.started":"2024-08-20T15:12:13.137909Z","shell.execute_reply":"2024-08-20T15:12:13.455092Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.loc[valid_index].classname.value_counts().plot(kind='bar')","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:13.457300Z","iopub.execute_input":"2024-08-20T15:12:13.457663Z","iopub.status.idle":"2024-08-20T15:12:13.711045Z","shell.execute_reply.started":"2024-08-20T15:12:13.457629Z","shell.execute_reply":"2024-08-20T15:12:13.710137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Update the dataframe with the split \n\nRandom state setting is not always safe.</br>\nPersonally I have experienced times where after few months of work the</br>\nsplit has changed. One way to avoid that is to encode it directly in a dataframe.</br>\nThus the split is now saved as part of the data itself.","metadata":{}},{"cell_type":"code","source":"df['valid'] = 0\ndf.loc[valid_index, 'valid'] = 1\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:13.715030Z","iopub.execute_input":"2024-08-20T15:12:13.715338Z","iopub.status.idle":"2024-08-20T15:12:13.728460Z","shell.execute_reply.started":"2024-08-20T15:12:13.715311Z","shell.execute_reply":"2024-08-20T15:12:13.727431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Create a dataloader and visualize the data","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_df(df, \n                               f'{data_path}/imgs/train/',\n                               fn_col='img',\n                               label_col='classname',\n                               valid_col='valid',\n                               bs=8\n                              )","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:13.730356Z","iopub.execute_input":"2024-08-20T15:12:13.730717Z","iopub.status.idle":"2024-08-20T15:12:14.986294Z","shell.execute_reply.started":"2024-08-20T15:12:13.730670Z","shell.execute_reply":"2024-08-20T15:12:14.985449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dls.show_batch()","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:12:14.987640Z","iopub.execute_input":"2024-08-20T15:12:14.988095Z","iopub.status.idle":"2024-08-20T15:12:16.800497Z","shell.execute_reply.started":"2024-08-20T15:12:14.988060Z","shell.execute_reply":"2024-08-20T15:12:16.799625Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# II. Model Development\n\n## 1. Model Training","metadata":{}},{"cell_type":"code","source":"model = resnet34\nlearn = vision_learner( dls, model,\n                       model_dir='/kaggle/working/',\n                       metrics=accuracy,\n                      pretrained=True)","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:39:09.790209Z","iopub.execute_input":"2024-08-20T15:39:09.790983Z","iopub.status.idle":"2024-08-20T15:39:10.303046Z","shell.execute_reply.started":"2024-08-20T15:39:09.790946Z","shell.execute_reply":"2024-08-20T15:39:10.302202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.lr_find()","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:39:14.771463Z","iopub.execute_input":"2024-08-20T15:39:14.771844Z","iopub.status.idle":"2024-08-20T15:39:25.104825Z","shell.execute_reply.started":"2024-08-20T15:39:14.771812Z","shell.execute_reply":"2024-08-20T15:39:25.103736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.fine_tune(3, 0.0052)","metadata":{"execution":{"iopub.status.busy":"2024-08-20T15:39:54.493807Z","iopub.execute_input":"2024-08-20T15:39:54.494758Z","iopub.status.idle":"2024-08-20T16:00:02.648254Z","shell.execute_reply.started":"2024-08-20T15:39:54.494713Z","shell.execute_reply":"2024-08-20T16:00:02.647203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Model Analysis","metadata":{}},{"cell_type":"code","source":"learn.recorder.plot_loss()","metadata":{"execution":{"iopub.status.busy":"2024-08-20T16:07:07.000393Z","iopub.execute_input":"2024-08-20T16:07:07.000875Z","iopub.status.idle":"2024-08-20T16:07:07.577165Z","shell.execute_reply.started":"2024-08-20T16:07:07.000839Z","shell.execute_reply":"2024-08-20T16:07:07.576204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"interp = ClassificationInterpretation.from_learner(learn)\ninterp.plot_confusion_matrix()","metadata":{"execution":{"iopub.status.busy":"2024-08-20T16:07:12.517736Z","iopub.execute_input":"2024-08-20T16:07:12.518130Z","iopub.status.idle":"2024-08-20T16:08:28.575918Z","shell.execute_reply.started":"2024-08-20T16:07:12.518099Z","shell.execute_reply":"2024-08-20T16:08:28.574904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"learn.show_results(max_n=20)","metadata":{"execution":{"iopub.status.busy":"2024-08-20T16:10:13.614203Z","iopub.execute_input":"2024-08-20T16:10:13.615091Z","iopub.status.idle":"2024-08-20T16:10:15.076647Z","shell.execute_reply.started":"2024-08-20T16:10:13.615056Z","shell.execute_reply":"2024-08-20T16:10:15.075675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# III. Model Usage","metadata":{}},{"cell_type":"code","source":"test_files = get_image_files(f'{data_path}/imgs/test/')","metadata":{"execution":{"iopub.status.busy":"2024-08-20T16:10:40.788134Z","iopub.execute_input":"2024-08-20T16:10:40.788830Z","iopub.status.idle":"2024-08-20T16:12:02.894777Z","shell.execute_reply.started":"2024-08-20T16:10:40.788800Z","shell.execute_reply":"2024-08-20T16:12:02.893762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_files","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(3):\n    image = PILImage.create(str(test_files[i]))\n    display(image.to_thumb(224,224))\n","metadata":{"execution":{"iopub.status.busy":"2024-08-20T16:18:45.350775Z","iopub.execute_input":"2024-08-20T16:18:45.351466Z","iopub.status.idle":"2024-08-20T16:18:45.441425Z","shell.execute_reply.started":"2024-08-20T16:18:45.351435Z","shell.execute_reply":"2024-08-20T16:18:45.440490Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(3):\n    image = PILImage.create(str(test_files[i]))\n    prediction,_,_ = learn.predict(image)\n    print(f\"driver behavior : {prediction}.\")\n    display(image.to_thumb(400,400))\n\n","metadata":{"execution":{"iopub.status.busy":"2024-08-20T16:22:08.477009Z","iopub.execute_input":"2024-08-20T16:22:08.477658Z","iopub.status.idle":"2024-08-20T16:22:08.783502Z","shell.execute_reply.started":"2024-08-20T16:22:08.477625Z","shell.execute_reply":"2024-08-20T16:22:08.782643Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}