README.md 16.8 KB
Newer Older
Antoine Saget's avatar
Antoine Saget committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Log - OAR Dashboard (react-admin)

Alexis Rollin
Antoine Saget

## Week 1

### Goals

- Get familiar with **react** and **react-admin**
- Get familiar with the subject :
  - reading report
  - basic knowledge of oar
- Start to write a simple **react-admin** example.
Antoine Saget's avatar
Antoine Saget committed
15

Antoine Saget's avatar
Antoine Saget committed
16
## Job done
Antoine Saget's avatar
Antoine Saget committed
17
18
19
20
21

- Reading the report
- Basic knowledge of oar and our mission
- Beginning of the `react-admin` tutorial

Antoine Saget's avatar
Antoine Saget committed
22
## To do
Antoine Saget's avatar
Antoine Saget committed
23
24
25
26
27

- Continue the tutorial
- Get familiar with the `react-admin` documentation
  - How to link with the oar API ?
  - How to make an authentication system ?
Antoine Saget's avatar
Antoine Saget committed
28
- Reading the internship report
Antoine Saget's avatar
Antoine Saget committed
29
30
31
32
33

Reading :

- <https://material-ui.com/styles/basics/>
- <https://marmelab.com/react-admin/Readme.html>
Antoine Saget's avatar
Antoine Saget committed
34
35
36
37
38
39
40
41
42
43
44
45


## Week 2

### Done

- Finished the **react-admin** tutorial
- Installed **oardocker** and run a simple example + access the api

### Summary

**react-admin** will need a `dataProvider` to glue with the **Oar API**. From this `dataprovider` we can use the framework which at first glance seems very intuitive and powerfull. The **Oar Server** will run in a docker container but will be accessible from the outside which is very convenient. The `oardocker` installation guide can be found [here](oardocker_install.md).
46
47
48
49
50
51
52
53
54
55
56
57
58

## Week 3

### Done

- For now, both of us code everything to make sure we understand. Next week we'll try to start a real git and work in collaboration.
- Study the **Oar API** and play with it : get resources, job, etc <http://oar.imag.fr/docs/latest/user/api.html>. The API seems simple and easy to use. `GET /resources/` and `GET /jobs/` are our starting points for this week.
- Fetch data throught **react-admin** using a custom prototype `dataProvider` :

![First prototype ever](prototype_1.png)

### Summary

Antoine Saget's avatar
Antoine Saget committed
59
Implementing a simple Data Provider for the Oar API to visualize the jobs and resources.
60
The Oar API is very simple. Everything regarding sorting, filtering, etc. must be done once data is fetched and not during the call to the API. Next Week we should **make a better example** (choose the right fields that matters) and **look into authentication and jobs posting** as well as **looking into the Angular code**.
61
62
63
64
65
66

## Week 4

### Done

- Trying some job POSTS with curl and postman : `curl -i -X POST $OARAPI/jobs -d '{"command":"sleep 6000"}' -H'Content-Type:application/json'` with `OARAPI=http://docker:docker@localhost:46668/oarapi-priv`
Antoine Saget's avatar
Merge    
Antoine Saget committed
67
68
- Fixing the compilation errors `"Could not find a declaration file for module XYZ"` by adding the key value pair `"noImplicitAny": false"` in `tsconfig.json`
- Setting up the `oar-dashboard` repo with typescript and hotreloading. Note that we already are in React-admin 3.2 which is the latest version. This was a real challenge, here is the final command list to setup the project correctly :
69
70
71
72
73
74
75
76
77
```bash
yarn create react-app oar-dashboard --template typescript
cd oar-dashboard
yarn add react-admin prop-types react-hot-loader
```
```json
// tsconfig.json
"compilerOption" : {
    // ... ,
Antoine Saget's avatar
Merge    
Antoine Saget committed
78
    "noImplicitAny": false // Maybe removing it in the future would be better ?
79
80
81
82
83
84
85
86
87
88
89
90
91
92
}
```
```tsx
// src/App.tsx
import { hot } from 'react-hot-loader';
// ...
export default hot(module)(App);
```
To install and run the project from the git repo :
```bash
git clone git@gricad-gitlab.univ-grenoble-alpes.fr:Projets-INFO4/19-20/23/oar-dashboard.git
yarn install
yarn start
```
Antoine Saget's avatar
Antoine Saget committed
93
94
95
96
97
98
99
100
101
102
103
help from : <https://dev.to/cronokirby/react-typescript-parcel-setting-up-hot-module-reloading-4f3f>

## Week 5

### Done

- **Hot reload**. It turned out that hotreloading was working from the beginning but that the problem came from a maximum file watching capacity that we were not aware of. In the end, increasing this maximum capacity solve the problem : `echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
` <https://stackoverflow.com/questions/16748737/grunt-watch-error-waiting-fatal-error-watch-enospc>
- **Types**. `react-admin` doesn't include Typescript types yet. In order to use some of them and benefit from using Typescript instead of Javascript we copied the `types.ts` file from the source code `react-admin` in `types/react-admin/index.d.ts`.  
We added `"typeRoots": [ "./types", "./node_modules/@types"],` in the `tsconfig.json` compilerOptions.   
Finally, we just have to `import * as ra from '../types/react-admin';` to use types in any source file. Here are some explenations regarding this topic : <https://basarat.gitbook.io/typescript/type-system/types>
Antoine Saget's avatar
Antoine Saget committed
104

Antoine Saget's avatar
typo    
Antoine Saget committed
105
## Week 6
Antoine Saget's avatar
Antoine Saget committed
106

107
### Done
Antoine Saget's avatar
Antoine Saget committed
108

109
#### Data Provider
Antoine Saget's avatar
Antoine Saget committed
110

111
Now that our project is completly setup we can finally start to implement "real" features. The dataProvider need to be as clean and complete as possible as it will be used by the react-admin application all the time.
Antoine Saget's avatar
Antoine Saget committed
112
113
114

The oar-api does not provide sorting, filtering, etc. so we do it afterwards. For example : `data: json.items.slice((page - 1) * perPage, page * perPage -1)`.

115
116
117
So far we implemented every `GET` method. We made a better viewer for both jobs et ressources. We added filters and sorting.

Next steps are better implementing `POST` methods as well as refining the ListViewers.
Alexis Rollin's avatar
Alexis Rollin committed
118
119
120

#### Authentication Provider

Alexis Rollin's avatar
Alexis Rollin committed
121
122
123
124
125
126
127
128
129
We also began the authentication aspect of the application. It consists in implementing an authentication provider, composed of the login and logout functions as well as error (401 and 403) handling functions, linked to oar.
We started with a simple prototype: the login page is displayed when an unauthenticated user tries to access any page of our application.
Please note that in the future, unauthenticated user will have access to some features of the application (which don't require to be authenticated on oar, like viewing all the resources).
The goal of this prototype is to find a way to establish an authentication to oar through the login page and get rid of the defaut authentication pop-up.
We found a way to stimulate oar thanks to a "whoami" request:
- through the login page, we retrieve the credentials (login and password)
- then, we are requesting the "whoami" page on the private API (oarapi-priv), with the credentials in the headers
- if the credentials are correct, we are redirected to the homepage of the application, and the authorized features are accessible
- else, we have to handle a 401 error
Alexis Rollin's avatar
Alexis Rollin committed
130
As I'm writing this, we still have to handle properly the 401 error.
131
132
133

## Week 7

134
### Data Provider
135
136
137
138

Using the API pagination and filtering capabilites when we can (still some work to do on the most complicated methods : getList and getManyReferences).
In the case that filter has to be done afterwards, pagination is a problem that need to be adressed (or ignored).

139
#### TODO
140

141
- getList / getManyReferences for jobs and resources
142
143
- create, update, delete methods

Alexis Rollin's avatar
Alexis Rollin committed
144
145
146
147
148
### Auth Provider

An authentication pop-up was still being displayed after having tried to login on the login page with wrong credentials. We discovered that no 401 error code was thrown before the displaying of a the pop up,
which means that this error is already handled somewhere.

Alexis Rollin's avatar
Alexis Rollin committed
149
150
151
152
### TODO

Continue the digging. If the previous teams were able to solve this problem, there are no reasons that we couldn't solve it too.

Alexis Rollin's avatar
Alexis Rollin committed
153
154
155
156
157
158
159
160
## Week 8

### Auth Provider

According to multiple issues (like this one: https://stackoverflow.com/questions/9859627/how-to-prevent-browser-to-invoke-basic-auth-popup-and-handle-401-error-using-jqu), the response header WWW-Authenticate is responsible for triggering the pop up.
So a 401 error is thrown, but the pop up appears before we can handle it because of the WWW-Authenticate header.
Additionnally, React Admin automatically redirects to the '/login' page if a 401 is encountered. It basically means that, if we figure out how to quiet the WWW-Authenticate header, the 401 will reach our application and will be correctly handled.

161
## Data Provider
Alexis Rollin's avatar
Alexis Rollin committed
162

163
164
165
The OAR API doesn't support the same filters for `resources` and `jobs` so the distinction has been made in the dataProvider to benefits from the API's capabilites has much as possible. For example, filtering by ID is now possible for `jobs` using the API.
Some code refactoring has been done as well.

166
### TODO
167
168
169

- create, update, delete methods ! Important as thoses methods as been delayed for several weeks now.
- thorought testing of GET methods
170
171
172
173
174
175


## Week 9

## Data Provider

Antoine Saget's avatar
Antoine Saget committed
176
177
178
179
- Create, update, and delete methods had been implemented as well as a simple edit form to change a ressource state.
- Documenting myself on how to layout component on a page and what are the available components (react-admin components and material-ui components).
- Making an elaborate create forms for job (taking inspiration from last years project + oarsub command + ~~`GET /jobs/form`~~ *doesn't work*). For know the form is still a prototype and doesn't have any "logic" to it, this is the next step.

Antoine Saget's avatar
Antoine Saget committed
180
![Job form prototype](prototype_job_form.png)
181
182
183

### TODO

Antoine Saget's avatar
Antoine Saget committed
184
- The form inputs need to be correctly passed to the dataProvider (or the dadaProvider need to be modified accordingly, one or the other).
185
186
- To edit a jobs, the OAR API use several entries, such as `/checkpoint`, `/holds` or `/resumptions` that we are not using in the dataProvider yet.
- Link between jobs and resources could be made. To put it simple, we could click on the ressources associated to a job to get informations about it.
Alexis Rollin's avatar
Alexis Rollin committed
187
188
189

## Auth Provider

190
191
Updating oar3 has been harder than expected because it brought some new problems.
After running "oardocker start -n 3", the API links shown aren't working, like if the oar server hasn't been launched.
Alexis Rollin's avatar
Alexis Rollin committed
192
193
194
Luckily for me, I found in the bash_history file the exact commands that I used to install oar3 from scratch, and I was able to re-install it correctly.
This afternoon felt a little bit like a loss of time but this step was still necessary.

Alexis Rollin's avatar
Alexis Rollin committed
195
196
197
198
199
Edit of 24/03: the login mechanism is now working : to access the "sensitive" resources of the API, the dataProvider will have to use the authRequest() instead of httpClient.
authRequest checks if the user is correctly authenticated and, if not, redirect to the login page. The login page is using the 'authentication' API feature to check the validity of the credentials without triggering the browser pop up.
If no 400 code is returned, it means that the credentials are correct and thus can be stored is localStorage in order to navigate freely through the session.
Current implementation seems to work well as accessing to the jobs doesn't trigger login and creating a job does.

Alexis Rollin's avatar
Alexis Rollin committed
200
201
202
203
204
205
### TODO

- test the new API function, authentication
- set up a 'is_connected' flag
- handle the redirect to login page
- close the issue
Alexis Rollin's avatar
Alexis Rollin committed
206
207
208
209
210
211
Edit of 24/03: all the points above have been done. New list:

- implement a Login button when disconnected
- override the logout button to appear only when connected (interesting links: https://github.com/marmelab/react-admin/issues/3033 and https://marmelab.com/react-admin/Authentication.html#adding-a-logout-button)
- displaying useful session infos on the dashboard: am I connected ? As who ?
- run further test, I am not sure that the application isn't starting on the login page
212
213
214
215
216
217
218
219
220
221
222

## Week 10

## Data Provider

- Job form finished and working. *A question remains: the processing of the form should be done in the dataProvider or in a "process" method called when submitting the form before the call to the dataProvider? At first, the "process" method seams to be the right solution. But when I thought of the future work: better dataGrid for jobs and ressources I found out that the dataProvider might actually be the right solution (will detail why when sure of my decision).*

### TODO
- To edit a jobs, the OAR API use several entries, such as `/checkpoint`, `/holds` or `/resumptions` that we are not using in the dataProvider yet.
- Link between jobs and resources could be made. To put it simple, we could click on the ressources associated to a job to get informations about it.
- Side view for ressource and jobs details like in the react-admin demo.
Antoine Saget's avatar
Merge    
Antoine Saget committed
223

Alexis Rollin's avatar
Alexis Rollin committed
224
225
## Auth Provider
- Created a custom Logout button the default one, but there are still some fixes to do.
Alexis Rollin's avatar
Alexis Rollin committed
226
- Created a custom Login page overriding the default one in order to disable the flag 'login_redirection' when arrived to the login page.
Antoine Saget's avatar
Merge    
Antoine Saget committed
227

Alexis Rollin's avatar
Alexis Rollin committed
228
229
230
231
### TODO
- The custom login page needs to be fixed : the theme and the 'submit' button have disappeared, a connection must be missing somewhere
- Fix the problem of redirecting to jobs even if I manually wants to go to the login page
- Turn the custom logout button in login button when disconnected
Antoine Saget's avatar
Merge    
Antoine Saget committed
232
233
- Possibly security weaknesses : the pop up is still appearing in very specific cases -> understand where is the hole. Note to myself : this weakness may be linked to the custom logout button

234
## Week 11
Antoine Saget's avatar
Merge    
Antoine Saget committed
235

236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
## React-admin limitation

react-admin passes special proporties to it's children that are not compatible with usual material/html components.
Thus, a simple code to use grid in order to organise data in two column is not possible :
```javascript
<Show title=" " {...props}>
    <SimpleShowLayout>
        <Grid
            container
            justify="center"
            className={classes.root}
            >
            <Grid item xs={5} >
                <TextField label="CPU" source="cpu" />
                <TextField label="Core" source="core" />
            </Grid>
            <Grid item xs={5}>
                <TextField label="Host" source="host" />
                <DateField label="Last job date" source="last_job_date" />
            </Grid>
        </Grid>
    </SimpleShowLayout>
</Show>
```

- [Github issue on the matter](https://github.com/marmelab/react-admin/issues/3154#ref-pullrequest-507330980)
- [Stackoverflow related post](https://stackoverflow.com/questions/53278585/custom-layout-in-simpleform-component-on-react-admin)

264
265
## Data Provider

266
- An attempt was made to put jobs details as a sidebar. Inspiration and code taken from the demo. This was unsuccessfull and would require more time to implement correctly.
267
- The TODO list have been re-read from the beginning and only the most import features have been kept.
268
269
270
271
272
273
- `updateMany` method for the dataProvider
- Bulk edit actions for ressources (changing the state of multiple ressources at one)
- Details view of ressources expanding from the dataGrid for easier navigation (replacing the old mechanisn that changed page when opening details)

![Bulk actions for resources](bulk_actions_resources.png)
![Expand details for ressources](expand_detail_resources.png)
274
275
276
277

## TODO

- Cleaning in preparation for a merge
278
- Improving datagrids and details show views for jobs (ressources done).
279
280
- To edit a jobs, the OAR API use several entries, such as`/holds` and `/resumptions` that we are not using in the dataProvider yet
- Prepare a setup file for teacher to try our code from home
281
- Clicking a bulk action button should deselect everything
Alexis Rollin's avatar
Alexis Rollin committed
282
283
284
285

## Auth Provider

- The logout button has been turned into a login button when disconnected
Alexis Rollin's avatar
Alexis Rollin committed
286
287

![Login button](login_button.png)
Alexis Rollin's avatar
Alexis Rollin committed
288
![Logout button](logout_button.png)
Alexis Rollin's avatar
Alexis Rollin committed
289

Alexis Rollin's avatar
Alexis Rollin committed
290
291
292
293
294
295
296
- The problem of the pop up still appearing in specific cases has also been fixed : it was due to a bad headers format in authRequester
- There still are some issues with the custom login page, but maybe creating a custom login page isn't necessary to update the 'login_redirection' flag. A hook may be an option to consider.

## TODO

- Try to implement a login hook
- Merge with the Data Provider branch ?
Alexis Rollin's avatar
Alexis Rollin committed
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313

## Week 12

## Auth Provider

- I read about react hooks but sadly it didn't quite fit the way our project is structured. Nevertheless, I managed to make the loginPage work, by also customizing the component triggering compilation errors.
That's why we need the file loginForm.tsx, which is almost the same as the default one in node_modules/ra-ui-materialui/src/auth : the only difference is a necessary cast.
This screen might be more useful for me than for a potential reader of this README, but it's always useful to keep somewhere how an error was solved last time.

![Cast in loginForm](necessary_cast_in_loginForm.png)

Maybe some new difficulties/bugs will appear in the future, but for now I think the next step is to merge with Antoine's Data Provider.

## TODO

- Merging !
- Once merged, testing the features only reachable by 'oar' and not 'docker'. They will probably be some fixes to do because this part was hard to correctly test without the data provider.
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329

## Week 13

## Merge

- The merge of the two main feature : the dataProvider and the authProvider has been made.
- Two issues has been found :
  - a wrong error message is displayed when trying to login : a logout method is called even if we are not logged to anything, this trigger the error.
  - the refresh spinner on the top right corner doesn't stop spinning after a "force login". Refreshing the whole page solve the problem.

## TODO

- Solve the issues
- Setup section
- Report
- Video
Alexis Rollin's avatar
Alexis Rollin committed
330
331
332
333
334
335

Edit:
We tried to solve the issues but without success. Nevertheless, these issues are minor as they don't disturb the functioning of the application.
Thus we prefered moving on to the writing of the report and making a video demo.
The demo can be found [here](https://www.youtube.com/watch?v=CKCQPf8wR6A&feature=youtu.be&fbclid=IwAR3SO3aBCAtC9VguHGwuqu-f0xmFbzTOMwkWamzTvLT3zptUAfMH6Hs30Es).