{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"### This notebook uses MIRNet model from 🤗 Model Hub to enhance Low Light Images using Keras\n\n![](https://img.youtube.com/vi/JRdWOAqQaUc/hqdefault.jpg)\n\n### Objective\n\nThe objective of the notebook is to showcase how simple and easy it is to use a pre-trained Keras model from Hugging Face and build a Deep Learning powered Tool ","metadata":{}},{"cell_type":"markdown","source":"Associated YouTube Tutorial -https://www.youtube.com/watch?v=JRdWOAqQaUc","metadata":{}},{"cell_type":"markdown","source":"Downloading and installing required libraries","metadata":{}},{"cell_type":"code","source":"! pip install keras","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:50:46.601684Z","iopub.execute_input":"2022-03-03T13:50:46.602215Z","iopub.status.idle":"2022-03-03T13:50:57.655582Z","shell.execute_reply.started":"2022-03-03T13:50:46.602178Z","shell.execute_reply":"2022-03-03T13:50:57.654741Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The pre-trained model wil be downloaded from Hugging Face Model Hub, hence we're installing `huggingface_hub`","metadata":{}},{"cell_type":"code","source":"! pip install huggingface_hub","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:50:57.659139Z","iopub.execute_input":"2022-03-03T13:50:57.659442Z","iopub.status.idle":"2022-03-03T13:51:06.677341Z","shell.execute_reply.started":"2022-03-03T13:50:57.659407Z","shell.execute_reply":"2022-03-03T13:51:06.676206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"These are the other two libraries used in this code but it's available with Kaggle Notebooks without Installations","metadata":{}},{"cell_type":"code","source":"#numpy\n#Pillow","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:51:06.679059Z","iopub.execute_input":"2022-03-03T13:51:06.679333Z","iopub.status.idle":"2022-03-03T13:51:06.683815Z","shell.execute_reply.started":"2022-03-03T13:51:06.679303Z","shell.execute_reply":"2022-03-03T13:51:06.68282Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Loading required libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np # array manipulation\nfrom huggingface_hub import from_pretrained_keras # download the model\nimport keras # deep learning\nfrom PIL import Image # Image processing","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:51:06.685322Z","iopub.execute_input":"2022-03-03T13:51:06.685858Z","iopub.status.idle":"2022-03-03T13:51:14.07723Z","shell.execute_reply.started":"2022-03-03T13:51:06.685824Z","shell.execute_reply":"2022-03-03T13:51:14.076484Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#from PIL import Image # Image processing","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:51:14.080126Z","iopub.execute_input":"2022-03-03T13:51:14.081046Z","iopub.status.idle":"2022-03-03T13:51:14.085501Z","shell.execute_reply.started":"2022-03-03T13:51:14.081Z","shell.execute_reply":"2022-03-03T13:51:14.084581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = from_pretrained_keras(\"keras-io/lowlight-enhance-mirnet\", compile=False)","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:51:14.087322Z","iopub.execute_input":"2022-03-03T13:51:14.088284Z","iopub.status.idle":"2022-03-03T13:52:23.895381Z","shell.execute_reply.started":"2022-03-03T13:51:14.088246Z","shell.execute_reply":"2022-03-03T13:52:23.894307Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#! wget","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:23.897105Z","iopub.execute_input":"2022-03-03T13:52:23.897354Z","iopub.status.idle":"2022-03-03T13:52:23.90163Z","shell.execute_reply.started":"2022-03-03T13:52:23.897326Z","shell.execute_reply":"2022-03-03T13:52:23.899968Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If you happen to read PNG images, it's possible they are read as `RGBA` with a transparency channel.Hence, while reading/opening the input image, we are converting it to `RGB` (removing the transparency channel).\n","metadata":{}},{"cell_type":"code","source":"#005e53b1b6aada 00354cd9244e28\n\nlow_light_img = Image.open('../input/happy-whale-and-dolphin/train_images/00354cd9244e28.jpg').convert('RGB')","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:23.903318Z","iopub.execute_input":"2022-03-03T13:52:23.903577Z","iopub.status.idle":"2022-03-03T13:52:23.969633Z","shell.execute_reply.started":"2022-03-03T13:52:23.903548Z","shell.execute_reply":"2022-03-03T13:52:23.968729Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"low_light_img","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:23.971325Z","iopub.execute_input":"2022-03-03T13:52:23.971963Z","iopub.status.idle":"2022-03-03T13:52:26.456485Z","shell.execute_reply.started":"2022-03-03T13:52:23.971919Z","shell.execute_reply":"2022-03-03T13:52:26.455807Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"There are lot of different methods to resize the image, this is one of the simpler methods but I'd strongly encourage you to check other options here - https://pillow.readthedocs.io/en/stable/handbook/concepts.html#concept-filters","metadata":{}},{"cell_type":"code","source":"low_light_img = low_light_img.resize((256,256),Image.NEAREST)","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.457762Z","iopub.execute_input":"2022-03-03T13:52:26.458089Z","iopub.status.idle":"2022-03-03T13:52:26.462261Z","shell.execute_reply.started":"2022-03-03T13:52:26.458061Z","shell.execute_reply":"2022-03-03T13:52:26.46123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"low_light_img","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.463567Z","iopub.execute_input":"2022-03-03T13:52:26.463995Z","iopub.status.idle":"2022-03-03T13:52:26.508509Z","shell.execute_reply.started":"2022-03-03T13:52:26.463946Z","shell.execute_reply":"2022-03-03T13:52:26.507811Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image = keras.preprocessing.image.img_to_array(low_light_img)","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.509503Z","iopub.execute_input":"2022-03-03T13:52:26.510513Z","iopub.status.idle":"2022-03-03T13:52:26.515159Z","shell.execute_reply.started":"2022-03-03T13:52:26.510465Z","shell.execute_reply":"2022-03-03T13:52:26.514394Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.516961Z","iopub.execute_input":"2022-03-03T13:52:26.517565Z","iopub.status.idle":"2022-03-03T13:52:26.530899Z","shell.execute_reply.started":"2022-03-03T13:52:26.517522Z","shell.execute_reply":"2022-03-03T13:52:26.529891Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image = image.astype('float32') / 255.0","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.533904Z","iopub.execute_input":"2022-03-03T13:52:26.534754Z","iopub.status.idle":"2022-03-03T13:52:26.541587Z","shell.execute_reply.started":"2022-03-03T13:52:26.534693Z","shell.execute_reply":"2022-03-03T13:52:26.540866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.542921Z","iopub.execute_input":"2022-03-03T13:52:26.543466Z","iopub.status.idle":"2022-03-03T13:52:26.55278Z","shell.execute_reply.started":"2022-03-03T13:52:26.543426Z","shell.execute_reply":"2022-03-03T13:52:26.552075Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image = np.expand_dims(image, axis = 0)","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.55419Z","iopub.execute_input":"2022-03-03T13:52:26.554581Z","iopub.status.idle":"2022-03-03T13:52:26.563058Z","shell.execute_reply.started":"2022-03-03T13:52:26.554544Z","shell.execute_reply":"2022-03-03T13:52:26.562063Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.566062Z","iopub.execute_input":"2022-03-03T13:52:26.566465Z","iopub.status.idle":"2022-03-03T13:52:26.576533Z","shell.execute_reply.started":"2022-03-03T13:52:26.566431Z","shell.execute_reply":"2022-03-03T13:52:26.575624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"output = model.predict(image) # model inference to enhance the low light pics","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:26.578095Z","iopub.execute_input":"2022-03-03T13:52:26.57865Z","iopub.status.idle":"2022-03-03T13:52:41.693886Z","shell.execute_reply.started":"2022-03-03T13:52:26.578468Z","shell.execute_reply":"2022-03-03T13:52:41.692771Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"output_image = output[0] * 255.0","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.695453Z","iopub.execute_input":"2022-03-03T13:52:41.695754Z","iopub.status.idle":"2022-03-03T13:52:41.700818Z","shell.execute_reply.started":"2022-03-03T13:52:41.695698Z","shell.execute_reply":"2022-03-03T13:52:41.699905Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"output_image.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.702454Z","iopub.execute_input":"2022-03-03T13:52:41.702983Z","iopub.status.idle":"2022-03-03T13:52:41.714299Z","shell.execute_reply.started":"2022-03-03T13:52:41.702932Z","shell.execute_reply":"2022-03-03T13:52:41.713408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"output_image = output_image.clip(0,255)","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.7157Z","iopub.execute_input":"2022-03-03T13:52:41.71622Z","iopub.status.idle":"2022-03-03T13:52:41.727281Z","shell.execute_reply.started":"2022-03-03T13:52:41.71617Z","shell.execute_reply":"2022-03-03T13:52:41.726446Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"output_image.shape","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.728875Z","iopub.execute_input":"2022-03-03T13:52:41.729361Z","iopub.status.idle":"2022-03-03T13:52:41.74489Z","shell.execute_reply.started":"2022-03-03T13:52:41.729311Z","shell.execute_reply":"2022-03-03T13:52:41.743928Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#output_image","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.74613Z","iopub.execute_input":"2022-03-03T13:52:41.746736Z","iopub.status.idle":"2022-03-03T13:52:41.762266Z","shell.execute_reply.started":"2022-03-03T13:52:41.746649Z","shell.execute_reply":"2022-03-03T13:52:41.761246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"output_image = output_image.reshape((np.shape(output_image)[0],np.shape(output_image)[1],3))","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.763619Z","iopub.execute_input":"2022-03-03T13:52:41.76432Z","iopub.status.idle":"2022-03-03T13:52:41.770312Z","shell.execute_reply.started":"2022-03-03T13:52:41.764268Z","shell.execute_reply":"2022-03-03T13:52:41.76963Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"output_image","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.772174Z","iopub.execute_input":"2022-03-03T13:52:41.772694Z","iopub.status.idle":"2022-03-03T13:52:41.789487Z","shell.execute_reply.started":"2022-03-03T13:52:41.772627Z","shell.execute_reply":"2022-03-03T13:52:41.788817Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"output_image = np.uint32(output_image)","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.790977Z","iopub.execute_input":"2022-03-03T13:52:41.792036Z","iopub.status.idle":"2022-03-03T13:52:41.797551Z","shell.execute_reply.started":"2022-03-03T13:52:41.791988Z","shell.execute_reply":"2022-03-03T13:52:41.796874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#output_image","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.799664Z","iopub.execute_input":"2022-03-03T13:52:41.800172Z","iopub.status.idle":"2022-03-03T13:52:41.812968Z","shell.execute_reply.started":"2022-03-03T13:52:41.800137Z","shell.execute_reply":"2022-03-03T13:52:41.812081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Final Image","metadata":{}},{"cell_type":"code","source":"Image.fromarray(output_image.astype('uint8'),'RGB')","metadata":{"execution":{"iopub.status.busy":"2022-03-03T13:52:41.815956Z","iopub.execute_input":"2022-03-03T13:52:41.816535Z","iopub.status.idle":"2022-03-03T13:52:41.846424Z","shell.execute_reply.started":"2022-03-03T13:52:41.816502Z","shell.execute_reply":"2022-03-03T13:52:41.845293Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The end","metadata":{}},{"cell_type":"markdown","source":"# Resources\n\n* [Low Light Image Enhancement in Python & Keras | Pixel-like NightMode using Deep Learning - Tutorial\n](https://www.youtube.com/watch?v=JRdWOAqQaUc)\n\n* [https://huggingface.co/keras-io/lowlight-enhance-mirnet](MIRnet on Hugging Face)\n\n* [Mirnet Keras Documentation](https://keras.io/examples/vision/mirnet/)","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}