Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
1 result
Compare
user avatar
First commit
Matthieu Muller authored
26bf702d
History
user avatar 26bf702d
History
Name Last commit Last update
images
output
readme_imgs
src
.gitignore
README.md
main.ipynb
requirements.txt
README.md

Image analysis course SICOM 3A - Project

1. Introduction

As a final exam, this project aims to evaluate 3A SICOM students.

Students will work individually, and each student must submit its work (report and code) on Chamilo by Tuesday, June 6.

2. Presentation

Many commercial RGB cameras use a technology called Color Filters Array (CFA). This is an array of red, green or blue filters overlaid over the sensors, typically arranged in periodic patterns. The incident light is consequently filtered by each filter, before being captured by the sensor. The usual image acquisition process uses a predetermined CFA pattern to assign a color to each pixel of the sensor. Here are the two configurations that are explored in this project:

alt text

Because of the filters the pixels on the sensor will only acquire one color (being red, green or blue). This leads to a raw acquisition that is a gray-scale image. See example bellow:

alt text

The goal of this project is to perform demosaicking: recover all the missing colors for each pixel. This way we will recover the full RGB image.

We propose you to reconstruct 4 images (you can find them in the images folder of this project). These images are from the open dataset of the National Gallery of Art, USA, which can be found here. To reconstruct these images we provide you the forward operator, modeling the effect of a CFA camera. This operation is described in the file src/forward_operator.py.

3. How to proceed?

All of the images to reconstruct have the same size (1024x1024) and are RGB. The patches to use are all same sized and squared (32x32) and are RGB too. To achieve this project you have to:

  • a. Extract all possible pieces (32x32 patches) from the image to reconstruct, let's call a piece from the image to reconstruct a query (q in the figure below).
  • b. Find for each query the patch (p in the figure below) that best fit the query with the help of an image processing based measure that you will build (feel free to search on the internet!). The patches from the database must be used only one time.
  • c. Reconstruct the image with the found patches.

alt text

4. Some hints to start

We want you to find a metric that allows to find the patch that best fit a given query. Remember how to compute the distance between two images (BE2, BE3).

You can look further and find among these references some other metrics that you can implement to have better results:

The time of computations might be very long depending of your machine. To verify that your method works, try to work with a few patches number (look at the source code and the provided documentation with the help() function).

Don't forget: images are RGB! The three channels are important!

5. Project repository

The project has the following organization:

image_processing_project/
├─.gitignore                   # Ignore this
├─images/                      # All images you need to reconstruct
│ ├─img_1.png
│ ├─img_2.png
│ ├─img_3.png
│ └─img_4.png
├─main.ipynb                   # The notebook to run all of your computations
├─output/                      # The output folder where you can save your reconstruction (ignore .gitkeep)
│ └─.gitkeep
├─patches.npy                  # All of the patches from ImageNet saved in one file
├─readme.md                    # ReadMe, contains all information about the project
├─readme_imgs/                 # Images for the ReadMe, ignore it
│ ├─img0.jpg
│ └─img1.png
├─requirements.txt             # Requirement file for packages installation
└─src/                         # All of the source files for the project, your source file must appear here!
  ├─demo_reconstructions.py
  ├─reconstruct.py
  └─utils.py

6. Instructions:

Each group will submit its work in Chamilo, inside the work Image_processing_projects_2023. It must contain:

  • A report as a pdf file with the name name1_name2_name3.pdf.
  • Your code as a archive (all of the provided project folder named image_processing_folder except the patches.npy file).

The code and the report must be written in English.

6.1. The report:

Your report must be a pdf file written in English. In this file you are asked to explain:

  1. The problem statement, show us that you've understood the project.
  2. The solution that you've chosen, explaining the theory.
  3. With tools that you've built show us that your solution is relevant and working (hint: maybe try to reconstruct an image with a patch database built with the image itself...).
  4. Results, give us your results of the images img_1, img_2, img_3, img_4.
  5. Conclusion, take a step back about your results. What can be improved?

6.2. The code:

  • Your code must compile and be operational.
  • In the notebook, feel free to add cells to show us how your code works and to show us the tools you've developped to evaluate your work.
  • Comment your code when needed.

7. Upload the whole project on jupyter hub

Jupyter hub allows to upload only files and not a folder organization. You can upload the project in one step by uploading a .zip version of the project on jupyter hub and unzip it with the python code below (you can write this code in a temporary notebook in jupyter hub):

import zipfile as zf

files = zf.ZipFile("image_processing_project-master.zip", 'r')
files.extractall('.') # Will extract the project at the root of jupyter hub
files.close()

If the package zipfile is not available install it with the command:
!pip install zipfile36 (run it in a notebook cell).

8. References

9. Supervisors