@@ -18,15 +18,15 @@ Because of the filters the pixels on the sensor will only acquire one color (bei
The goal of this project is to perform *demosaicking*: recover all the missing colors for each pixel. This way we will recover the full RGB image.
We propose you to reconstruct 4 images (you can find them in the `images` folder of this project). These images are from the open dataset of the **National Gallery of Art**, USA, which can be found [here](https://github.com/NationalGalleryOfArt/opendata). To reconstruct these images we provide you the forward operator, modeling the effect of a CFA camera with 2 different CFA patterns:either Bayer of Quad Bayer pattern. This operation is described in `src/forward_operator.py`.
We propose you to reconstruct 4 images (you can find them in the `images` folder of this project). These images are from the open dataset of the **National Gallery of Art**, USA, which can be found [here](https://github.com/NationalGalleryOfArt/opendata). To reconstruct these images we provide you the forward operator, modeling the effect of a CFA camera with 2 different CFA patterns:either Bayer of Quad Bayer pattern. This operation is described in `src/forward_operator.py`.
## 3. How to proceed?
All of the images to reconstruct have the same size (1024x1024) and are RGB. You have to use methods seen during this semester to recover a fully colored RGB image using **only** the raw acquisition (gray-scale image) and the forward operator.
All the images to reconstruct have the same size (1024x1024) and are RGB. You have to use methods seen during this semester to recover a fully colored RGB image using **only** the raw acquisition (gray-scale image) and the forward operator (but you are not forced to use it). Of course using directly the ground truth (original image) is forbiden, it can only be used to compute metrics (SSIM, PSNR).
## 4. Some hints to start
We provide in `src/methods/baseline` a basic interpolation image which can help you to start. These methods are described in the PhD Thesis _Model Based Signal Processing Techniques for Nonconventional Optical Imaging Systems_ and the user's manual _Pyxalis Image Viewer_ (see references). You can also find in the academic litterature techniques (interpolation, inverse problem, machine learning, etc) that solves demosaicking.
We provide in `src/methods/baseline` a basic interpolation image which can help you to start. These methods are described in the PhD Thesis _Model Based Signal Processing Techniques for Nonconventional Optical Imaging Systems_ and the user's manual _Pyxalis Image Viewer_ (see references). You can also find in the academic litterature techniques (interpolation, inverse problem, machine learning, etc) that solves problems close to this one.
*The time of computations might be very long depending of your machine. To verify that your method works, try to work with smaller test images.*
...
...
@@ -44,12 +44,11 @@ sicom_image_analysis_project/
├─readme_imgs/ # Images for the Readme
├─requirements.txt # Requirement file for packages installation
└─src/ # All of the source files for the project
├─reconstruct.py # File containing the main reconstruction fonction
├─checks.py # File containing some sanity checks
├─forward_model.py # File containing the CFA operator
├─utils.py # Some utilities
└─methods/
├─baseline/ # Example of demosaicking
├─baseline/ # Example of reconstruction
└─template/ # Template of your project (to be copied)
~~~
...
...
@@ -57,13 +56,21 @@ sicom_image_analysis_project/
Each student will fork this Git repository on its own account. You will then work in your own version of this repository.
To submit your work you just need to create a merge request in Gitlab (see [here](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html#when-you-work-in-a-fork) for a detailed explanation). This merge request will **only encompass the changes made to `src/methods/your_name`, without `main.ipynb`**. Because the merge request will only create `src/methods/your_name` there will not be any conflicts between each student's project.
Along with your code you must submit a report **as a pdf file** with the name **name.pdf** inside the folder `src/methods/your_name`.
The code and the report must be **written in English**.
### 6.1. The report:
### 6.1. The code:
- You should first **copy** the folder `src/methods/template` and rename it with your name. It is in this folder that you will work, **nothing else should be modified** apart from `main.ipynb` for experimenting with the code.
- You will add as many files you want in this folder but the only interface to run your code is `run_reconstruction` in `src/methods/your_name`. You can modify this function as you please, as long as it takes in argument the image to reconstruct (`y`), the name of the CFA (`cfa`) and returns a demosaicked image.
- Have a look in `src/methods/baseline`, your projet should work in the same way.
- You can use all the functions defined in the project, even the ones that you should not modify (`utils.py`, `forward_model.py`, etc).
- Your code must be operational through `run_reconstruction`, as we will test it on new and private images.
- The notebook provides a working bench. It should **not be included in the merge request**, it is just a work document for you.
- Comment your code when needed.
### 6.2. The report:
Your report **must be a pdf file** written in English and should not be longer than 5 pages. In this file you are asked to explain:
...
...
@@ -73,15 +80,9 @@ Your report **must be a pdf file** written in English and should not be longer t
- Results, give us your results of the images `img_1`, `img_2`, `img_3`, `img_4`.
- Conclusion, take a step back about your results. What can be improved?
### 6.2. The code:
### 6.3. Submission:
- You should first **copy** the folder `src/methods/template` and rename it with your name. It is in this folder that you will work, **nothing else should be modified** apart from `main.ipynb` for experimenting with the code.
- You will add as many files you want in this folder but the only interface to run your code is `run_reconstruction` in `src/methods/your_name`. You can modify this function as you please, as long as it takes in argument the image to reconstruct (`y`), the name of the CFA (`cfa`) and returns a demosaicked image.
- Have a look in `src/methods/baseline`, your projet should work in the same way.
- You can use all the functions defined in the project, even the ones that you should not modify (`utils.py`, `forward_model.py`, etc).
- Your code must be operational through `run_reconstruction`, as we will test it on new and private images.
- The notebook provides a working bench. It should **not be included in the merge request**, it is just a work document for you.
- Comment your code when needed.
To submit your work you just need to create a merge request in Gitlab (see [here](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html#when-you-work-in-a-fork) for a detailed explanation). This merge request will **only encompass the changes made to `src/methods/your_name`, without `main.ipynb`**. Because the merge request will only create `src/methods/your_name` there will not be any conflicts between each student's project.
