README.md

# Image analysis course SICOM 3A - Project

## 1. Introduction

As a final exam, this project aims to evaluate 3A SICOM students. 

Students will work individually, and each student must submit its work (report and code) on Chamilo by **Tuesday, June 6**.

## 2. Presentation

Many commercial RGB cameras use a technology called Color Filters Array (CFA). This is an array of red, green or blue filters overlaid over the sensors, typically arranged in periodic patterns. The incident light is consequently filtered by each filter, before being captured by the sensor. The usual image acquisition process uses a predetermined CFA pattern to assign a color to each pixel of the sensor. Here are the two configurations that are explored in this project:

![alt text](readme_imgs/patterns.png)

Because of the filters the pixels on the sensor will only acquire one color (being red, green or blue). This leads to a raw acquisition that is a gray-scale image. Example:

![alt text](readme_imgs/example.png)

The goal of this project is to perform *demosaicking*: recover all the missing colors for each pixel. This way we will recover the full RGB image.

We propose you to reconstruct 4 images (you can find them in the `images` folder of this project). These images are from the open dataset of the **National Gallery of Art**, USA, which can be found [here](https://github.com/NationalGalleryOfArt/opendata). To reconstruct these images we provide you the forward operator, modeling the effect of a CFA camera (either Bayer of Quad Bayer pattern). This operation is described in the file `src/forward_operator.py`.

## 3. How to proceed?

All of the images to reconstruct have the same size (1024x1024) and are RGB. You have to use methods seen during this semester to recover a fully colored RGB image using **only** the raw acquisition (gray-scale image) and the forward operator.

## 4. Some hints to start

We provide in `src/demo_reconstruct` a basic interpolation image which can help you to start. You can also find in the academic litterature techniques (interpolation, inverse problem, machine learning, etc) that solves demosaicking.

*The time of computations might be very long depending of your machine. To verify that your method works, try to work with smaller test images.*

## 5. Project repository

The project has the following organization: 

~~~text
sicom_image_analysis_project/
├─.gitignore                   # Git's ignore file
├─images/                      # All images you need to reconstruct
│ ├─img_1.png
│ ├─img_2.png
│ ├─img_3.png
│ └─img_4.png
├─main.ipynb                   # The notebook to run all of your computations
├─output/                      # The output folder where you can save your reconstruction (ignore .gitkeep)
│ └─.gitkeep
├─README.md                    # Readme, contains all information about the project
├─readme_imgs/                 # Images for the Readme, ignore it
│ ├─example.png
│ └─patterns.png
├─requirements.txt             # Requirement file for packages installation
└─src/                         # All of the source files for the project, your source file must appear here!
  ├─demo_reconstructions.py    # Example of demosaicking | **Do not modify**
  ├─reconstruct.py             # File containing the main reconstruction fonction
  ├─checks.py                  # File containing some sanity checks | **Do not modify**
  ├─forward_model.py           # File containing the CFA operator | **Do not modify**
  └─utils.py                   # Some utilities | **Do not modify**
  ~~~

You are expected to write your own code in new files of `src`, and call it in the `src/reconstruct.py` file.

## 6. Instructions: 

Each student will submit its work in Chamilo, inside the work **Image_analysis_projects_2023**. It must contain:

- A report **as a pdf file** with the name **name.pdf**.
- Your code as a archive with the name **name.zip**.

The code and the report must be **written in English**.

### 6.1. The report:

Your report **must be a pdf file** written in English. In this file you are asked to explain:

1. The problem statement, show us that you've understood the project.
2. The solution that you've chosen, explaining the theory.
3. With tools that **you've built** show us that your solution is relevant and working
4. Results, give us your results of the images `img_1`, `img_2`, `img_3`, `img_4`.
5. Conclusion, take a step back about your results. What can be improved?

### 6.2. The code:

- Your code must be operational.
- In the notebook, feel free to add cells to show us how your code works and to show us the tools you've developped to evaluate your work.
- Comment your code when needed.

## 7. Supervisors
- Mauro Dalla Mura: mauro.dalla-mura@gipsa-lab.grenoble-inp.fr
- Matthieu Muller: matthieu.muller@gipsa-lab.grenoble-inp.fr