Vous avez re莽u un message "Your GitLab account has been locked ..." ? Pas d'inqui茅tude : lisez cet article https://docs.gricad-pages.univ-grenoble-alpes.fr/help/unlock/

Commit 62c05765 authored by Alexis Brenon's avatar Alexis Brenon
Browse files

馃攢 Merge branch 'documentation'

parents affdac28 5424d478
doc/
public/
outputs/
# Vim swap files
......@@ -6,3 +7,6 @@ outputs/
# TREPL history file
/.history
# Local config
*.local
pages:
image: abaez/luarocks
stage: deploy
script:
- luarocks install penlight
- curl -L https://github.com/AlexisBRENON/LDoc/tarball/master > master.tgz
- tar xzf master.tgz
- lua *-LDoc*/ldoc.lua .
- mv -v -f public/topics/README.md.html public/index.html
- cp -v -u assets/ldoc/bootstrap-4-navbar.* public
artifacts:
paths:
- public
# Context Repr茅sentation
Auteur: Alexis BRENON, Date: 2016-07-11
# ARCADES
Author: Alexis BRENON, Date: 2017-12-05
## Id茅e
## What is it?
Utiliser les donn茅es issues de capteurs ainsi que des connaissances a priori sur la topologie d'un appartement pour repr茅senter graphiquement les valeurs des capteurs.
ARCADES stands for Adaptive Reinforced Context-Aware Deep Decision System.
It is, at the same time, a proof a concept of using Deep Q-Network for decision making in smart-home, and a library implementing deep reinforcement learning.
Cette repr茅sentation graphique sera par la suite trait茅e gr芒ce 脿 un r茅seau de neuronnes pour en d茅duire un contexte. La premi猫re application sera de d茅terminer l'activit茅 de l'utilisateur dans l'habitat comme ce fut le cas lors du projet SweetHome.
For more informations, let's refer to these publications:
## Organisation du projet
* Brenon (2017). [Mod猫le profond pour la prise de d茅cision adaptative en habitat intelligent](https://www.dropbox.com/s/xtx2ujlozbm6xvi/Alexis_BRENON_PhD.pdf)
* Mnih et al. (2015). [Human-level control through deep reinforcement learning](https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf)
L'ensemble du projet est versionn茅 en utilisant le gestionnaire Git. L'architecture globale se base sur plusieurs branches qui n'ont pas pour vocations 脿 锚tre fusionn茅es *in fine* :
* `master` : la branche principale qui ne contiendra que cet unique README
* `data` : une branche contenant les scripts n茅cessaires 脿 la mise en forme des donn茅es d'entr茅es
* `assets` : une branche contenant les fichiers _binaires_ n茅cessaires pour l'exp茅rimentation
* `src` : la branche contenant les scripts/sources permettant de r茅aliser l'exp茅rimentation
* `xp/*` : des branches pour chaque exp茅rimentation 茅tant la fusion des toutes les branches pr茅c茅dentes
* `article/*` : des branches issues des exp茅rimentations pour la r茅daction d'articles 脿 propos des r茅sultats
## How to use it?
## D茅pendances
This repository is composed of different branches.
The `master` branch contains most of the source code:
Ce projet repose sur un ensemble de d茅pendances qui sont :
* [Torch](http://www.torch.ch) : Framework pour les r茅seaux de neurones
* [hash](https://github.com/torch/hash) : Fonctions de hachage (distribu茅 par Torch)
* [Penlight](http://stevedonovan.github.io/Penlight/api/index.html) : Biblioth猫que d'utilitaires pour Lua (d茅pendance de Torch)
* [lualogging](https://github.com/Neopallium/lualogging) : Biblioth猫que de log pour Lua
* [libglib2.0-0](https://developer.gnome.org/platform-overview/stable/tech-glib.html), [libcairo2](https://cairographics.org/) et [librsvg2-2](https://wiki.gnome.org/Projects/LibRsvg) : Biblioth猫ques C pour la conversion SVG --> matriciel
* The `arcades` folder contains the code of the library.
* The `assets` folder contains some binary files used by the code.
* The `data` folder contains some real data.
* The `public` folder contains the generated documentation (if any).
* More folders will come with different tools like `tsne` or `deepdream`.
* Many scripts, detailed below.
The `xp/*` branches are related to an experiment.
Most of the time they are closed with a tag (and if not, they should) allowing reproducibility of the experiments.
At the root of the repository, you can find some scripts (shell or Lua ones).
Here is a brief description of each:
* `install_dependencies.sh` try to install all dependencies (listed in **Dependencies**) required
* `main.lua` load ARCADES and launch an experiment given its arguments. _ARCADES聽should be updated to rely on a configuration file (JSON or so) to avoid these tricky parsing and instantiations._
* `run.lua` is a wrapper, which builds a well formed arguments list and finally calls `main.lua`.
* `expe_template.lua` is a template file used to create experiments. It defines parameters and then call `run.lua`.
* `debug.lua` is an instance of the `expe_template`, used to... Debug?
* `test.lua` is an instance of the `expe_template` used to test code (short experiment).
* `cross-val.sh` run multiple experiment and changing training and testing sets each time.
* `cross-val-results.sh` looks for experiment results in a directory and build a resulting metrics file.
Finally, the `config.ld` file is the configuration file of the [LDoc](http://stevedonovan.github.io/ldoc/) utility used to build the documentation.
## Dependencies
This project rely on dependencies installed by the `install_dependencies.sh` script.
Some are Lua libraries, such as:
* [`torch`](http://www.torch.ch): The neural network framework.
* [`hash`](https://github.com/torch/hash): hashing functions (distributed by Torch).
Others are C libraries that you have to install yourself:
* [libglib2.0-0](https://developer.gnome.org/platform-overview/stable/tech-glib.html): GLib objects management.
* [libcairo2](https://cairographics.org/): graphics rendering (using GLib objects).
* [librsvg2-2](https://wiki.gnome.org/Projects/LibRsvg): SVG rendering.
# Context Representation - `src`
Auteur : Alexis BRENON, Date : 2016-08-24
Ce dossier contient l'ensemble du code source n茅cessaire 脿 l'exp茅rimentation. Le code source est organis茅 selon la hierarchie suivante :
* `main.lua` : point d'entr茅e de l'exp茅rimentation (`th src/main.lua -h`) ;
* `agent/` : D茅finitions de diff茅rents agents utilisant des techniques diff茅rentes pour la prise de d茅cision ;
* `environment/` : D茅finitions d'environnements sur lesquels un agent peut agir et qui lui retourne un feed-back (r茅compense, 茅tat, etc.) ;
* `experiment/` : D茅finitions de classes qui font int茅ragir un agent avec un environnement pour mesurer les performances ;
* `network/` : D茅finitions de diff茅rents r茅seaux de neurones utilis茅s dans les agents ;
* `utils/` : Modules utilitaires.
L'ensemble du code source est document茅 et la documentation peut 锚tre g茅n茅r茅e avec l'utilitaire [LDoc](http://stevedonovan.github.io/ldoc/).
--- Inferred graphic domus.
-- @classmod environment.smarthome.sweethome.GraphicalAnnotatedSweetHome
-- @alias class
-- @author Alexis BRENON <alexis.brenon@imag.fr>
local torch = require('torch')
......
--- Sensor graphic domus.
-- @classmod environment.smarthome.sweethome.GraphicalSensorSweetHome
-- @alias class
-- @author Alexis BRENON <alexis.brenon@imag.fr>
local torch = require('torch')
......
-- Base file of the arcades project
--- Base of any ARCADES related object.
-- ArcadesComponent is a base class for any
-- ARCADES class, providing default behavior for logging,
-- string representation, dumping and loading.
-- @classmod Arcades_Components
-- @alias class
-- @author Alexis BRENON <alexis.brenon@imag.fr>
local torch = require('torch')
......@@ -9,6 +15,30 @@ local Logger = require('arcades.utils.Logger')
local arcades = require('arcades.utils.package_loader')(...)
local class = torch.class('ArcadesComponent', arcades)
--- Fields
-- @section fields
--- Table (shallow copy) of arguments used for instanciation
-- @tfield table self._args
--- Identifier name of the object
-- @tfield string self._name
--- List of attributes that does not need to be dumped
-- @tfield table self._no_dump_list
--- Object's logger
-- @see utils.Logger
-- @tfield utils.Logger self._logger
---
-- @section end
--- Object constructor.
--
-- Instanciate a new ARCADES object.
-- @tparam[opt={}] table args Table of attributes/arguments to pass to the object
-- @tparam[opt={}] table dump Dump of an object to re-instanciate
function class:__init(args, dump)
args = args or {}; dump = dump or {}
self._args = tablex.copy(args)
......@@ -22,17 +52,8 @@ function class:__init(args, dump)
self._logger:debug("Instanciating a new object")
end
function class:dump(cycles)
cycles = cycles or {}
if self._logger then self._logger:debug("Dumping...") end
cycles[torch.pointer(self)] = true
local args_dump = class._dump(self._args, cycles)
local dump = class._dump(self, cycles)
dump.classname = torch.typename(self)
dump._args = args_dump
return dump
end
--- Get a string representation of the object.
-- @treturn string A string representing the object
function class:__tostring__()
local dump = self:dump()
result = string.format(
......@@ -43,50 +64,49 @@ function class:__tostring__()
return result
end
function class._dump(o, cycles)
if not o then return nil end
--- Public Methods
-- @section public-methods
--- Dump an object.
--
-- Build a serializable table containing enough information to instanciate
-- and initialize a new object at the same state than `self`.
-- @tparam[opt={}] table cycles Already dumped fields to avoid loops
-- @treturn table A serializable table
function class:dump(cycles)
cycles = cycles or {}
local dump = {}
for k, v in pairs(o) do
if (
type(v) ~= "function" and -- You cannot safely save a function
string.sub(k, 1, 1) ~= "_" and -- Don't dump private fields
(not (o._no_dump_list and o._no_dump_list[k])) -- Do not dump attributes listed as undumpable
) then
if (
(torch.typename(v) or torch.type(v) == "table") and
cycles[torch.pointer(v)]
) then
dump[k] = string.format(
"cycle<%s>",
v._name or "table("..torch.pointer(v)..")"
)
elseif ( -- Call dump function on Torch object if available
torch.typename(v) and
v.dump
) then
dump[k] = v:dump(cycles)
elseif (torch.type(v) == "table") then -- Recurcively dump tables
cycles[v] = true
dump[k] = class._dump(v, cycles)
else
dump[k] = v
end
end
end
if self._logger then self._logger:debug("Dumping...") end
cycles[torch.pointer(self)] = true
local args_dump = class._dump(self._args, cycles) -- First dump the instanciation arguments
local dump = class._dump(self, cycles) -- Then dump local attributes
dump.classname = torch.typename(self) -- Used to identify the class from the dump
dump._args = args_dump
return dump
end
--- Load an object from a dump.
--
-- Given a @{dump}, find the associated class, instanciate an object and initialize it.
-- @tparam table dump Dump to restore
-- @return A newly instanciated object
function class.load(dump)
--- Find a class path given a class name.
--
-- Recurcively look for a class named `classname` in loaded packages/modules.
-- @tparam string classname Name of the class to find
-- @tparam[opt=_G] package Base environment/package to search
-- @treturn function Class constructor
-- @error nil
-- @todo Dump classpath to avoid this expensive search
local function find_class_by_name(classname, package)
package = package or _G
for k, v in pairs(package) do
for k, v in pairs(package) do -- look for each exported name
if k == classname then
return v
elseif (
string.lower(string.sub(k, 1, 1)) == string.sub(k, 1, 1) and -- packages are lower cased (while classes are upper cased)
type(v) == "table"
) then
) then -- search sub packages
local constructor = find_class_by_name(classname, v)
if constructor then return constructor end
end
......@@ -94,6 +114,11 @@ function class.load(dump)
return nil
end
--- Recurcively load a dump.
--
-- Instanciate any object referenced in `dump` before actually loading it.
-- @tparam table dump Dump to load
-- @return Newly instantiated object
local function recursive_load(dump)
local class = find_class_by_name(dump.classname, arcades)
if not class then
......@@ -115,8 +140,58 @@ function class.load(dump)
return class(dump._args, dump)
end
arcades.load_all()
arcades.load_all() -- Import all packages of the arcades library
return recursive_load(dump)
end
--- Static Methods
-- @section static-methods
--- Dump an object.
--
-- Dump any object, avoiding cycles.
-- The dump will contain any attribute excepted:
--
-- * functions
-- * attributes with name starting with `_` (underscore)
-- * attributes with name present in @{_no_dump_list}
--
-- If possible, tables and objects will be recurcively dumped.
-- @param[opt] o Object to dump
-- @tparam[opt={}] table cycles Already dumped tables/objects
-- @treturn table A serializable table or `nil`
function class._dump(o, cycles)
if not o then return nil end
cycles = cycles or {}
local dump = {}
for k, v in pairs(o) do
if (
type(v) ~= "function" and -- You cannot safely save a function
string.sub(k, 1, 1) ~= "_" and -- Don't dump private fields
(not (o._no_dump_list and o._no_dump_list[k])) -- Do not dump attributes listed as undumpable
) then
if (
(torch.typename(v) or torch.type(v) == "table") and
cycles[torch.pointer(v)]
) then
dump[k] = string.format(
"cycle<%s>",
v._name or "table("..torch.pointer(v)..")"
)
elseif ( -- Call dump function on Torch object if available
torch.typename(v) and
v.dump
) then
dump[k] = v:dump(cycles)
elseif (torch.type(v) == "table") then -- Recurcively dump tables
cycles[v] = true
dump[k] = class._dump(v, cycles)
else
dump[k] = v
end
end
end
return dump
end
return arcades
--- A preprocessing network to downsample a given image.
-- This preprocessing network will scale the image and convert it to Y color space
-- @classmod network.Downsample
-- @alias class
-- @author Alexis BRENON <alexis.brenon@imag.fr>
local nn = require('nn')
......
--- A convolutional neural network for image processing.
-- @classmod network.Convnet
-- @alias class
-- @author Alexis BRENON <alexis.brenon@imag.fr>
local nn = require('nn')
......
--- Package containing different neural network architectures.
--- A module to build neural network from a descrption.
--
-- List of the classes in this package:
-- <ul>
-- <li>@{network.DebugNetwork|DebugNetwork}</li>
-- <li>@{network.Downsample|Downsample}</li>
-- <li>@{network.Convnet|Convnet}</li>
-- </ul>
-- An example of description can be found in this file: @{Inference.lua}
-- @module network
-- @alias package
-- @alias module
-- @author Alexis BRENON <alexis.brenon@imag.fr>
local module = require('arcades.utils.package_loader')(...)
local torch = require('torch')
local nn = require('nn')
function module.create_network(args)
local net = nn.Sequential()
......
--- Add some useful functions to lualogging module.
-- @module utils.logging
--- A logger class.
--
-- A class that can be used by any object to instanciate dedicated logger.
-- See documentation of Python logging facility to see what I tried to
-- achieve.
--
-- This class also contains a @{main_logger|default/main logger} as a
-- static field for quick and dirty logging.
-- @classmod utils.Logger
-- @alias class
-- @author Alexis BRENON <alexis.brenon@imag.fr>
local torch = require('torch')
......@@ -7,6 +15,28 @@ local torch = require('torch')
local module = {}
local class = torch.class('Logger', module)
--- Fields
-- @section field
--- Name of the logger to identify it.
-- @tfield string name
--- Function used to format log message.
-- @tfield function formatter
--- Output stream used by the logger.
-- @tfield io.file_descripton output
--- Current logging level.
-- @tfield integer level_value
--- @section end
--- Class constructor.
-- @tparam string name @{name|Name} of the logger
-- @param[opt=default_level_value] level Logging level (string or integer) to use
-- @tparam[opt=timestamp_output] function formatter Function used to format log message
-- @tparam[opt=default_output] io.file_descriptor output Stream to use as logger output
function class:__init(name, level, formatter, output)
self.name = name
self.formatter = formatter or self.timestamp_output
......@@ -14,6 +44,15 @@ function class:__init(name, level, formatter, output)
self:set_level(level or class.default_level_value)
end
--- Public Methods
-- @section public-methods
--- Change logging level.
--
-- Change the current logging level to the given `level` witch can be passed as
-- @{level_values|integer} or @{level_strings|string} values.
-- @param level Logging level to use
-- @return `self`
function class:set_level(level)
local int_level, str_level = self.get_levels(level)
self.level_value = int_level
......@@ -21,10 +60,38 @@ function class:set_level(level)
return self
end
--- Get logging level.
--
-- Return current logging level as
-- @{level_values|integer} and @{level_strings|string} values.
-- @treturn integer Current logging integer value
-- @treturn string Current logging string value
function class:get_level()
return self.get_levels(self.level_value)
end
--- Change the output.
-- @tparam[opt=nil] string filename Log to the given file or `io.stdout` if `nil`
function class:set_output(filename)
self:info("Updating log output: " .. filename or "stdout")
if filename then
self.output = io.open(filename, "a")
else
self.output = io.output
end
end
--- Log a message.
--
-- Generate a log message with the given `level`, and print it if current logging level allows it.
--
-- A function is also defined for each log level, using a lower cased version of @{level_strings} values.
-- @usage logger:log("DEBUG", ...)
-- -- equivalent to
-- logger:debug(...)
-- @param level Message level as @{level_values|integer} or @{level_strings|string} value.
-- @param ... Data to format and log. See @_get_log_message.
-- @return self
function class:log(level, ...)
local int_level, _ = self.get_levels(level)
if int_level >= self.level_value then
......@@ -35,6 +102,19 @@ function class:log(level, ...)
return self
end
--- Print a progressing wheel.
-- @warn This function is only available for the @{main_logger}.
-- @tparam[opt] integer step Step number to display
-- @function progress
--- Private Methods
-- @section private-methods
--- Build and format a log message.
-- @tparam integer int_level Integer value of the level of the message
-- @tparam string str_format Base string to format
-- @param ... Values used for formatting
-- @treturn string Formatted log message
function class:_get_log_message(int_level, str_format, ...)
local msg = str_format
local str_level = self._get_level_string(int_level)
......@@ -48,20 +128,36 @@ function class:_get_log_message(int_level, str_format, ...)
return self:_format(str_level, msg)
end
--- Add logger informations to log message.
-- @tparam string str_level String value of the message level
-- @tparam string msg User log message
-- @treturn string Logger formatted message
function class:_format(str_level, msg)
return self:formatter(str_level, msg)
end
--- Change the output of the logging facilities.
-- @tparam[opt=nil] string filename Log to the given file or stdout if <code>nil</code>
function class:set_output(filename)
self:info("Updating log output: " .. filename or "stdout")
if filename then
self.output = io.open(filename, "a")
else
self.output = io.output
end
end
--- Static Fields
-- @section static-fields
--- Available log levels.
--
-- String representations of the log levels.
-- @table level_strings
-- @tfield string DEBUG
-- @tfield string INFO
-- @tfield string WARN
-- @tfield string ERROR
-- @tfield string FATAL
--- Available log levels.
--
-- Integer representations of the log levels.
-- @table level_values
-- @tfield integer 10
-- @tfield integer 20
-- @tfield integer 30
-- @tfield integer 40
-- @tfield integer 50
class.level_strings = {"DEBUG", "INFO", "WARN", "ERROR", "FATAL"}
class.level_values = { 10, 20, 30, 40, 50 }
......@@ -72,10 +168,32 @@ for i, _ in ipairs(class.level_strings) do
end
end
--- Default output.
--
-- Output used by default by new logger instances.
-- @tfield io.file_descriptor default_output `io.stdout`
class.default_output = io.stdout
--- Default log level.
--
-- Log level used by default by new logger instances.
-- @tfield integer default_level_value 20
class.default_level_value = 20
-- Static functions
--- Default logger.
--
-- A pre-instanciated logger for quick and dirty logging.
-- @tfield Logger main_logger
--- Static Functions
-- @section static-functions
--- Standard log formatter function.
--
-- The default standard log formatter function, adding timestamp, message level, file and line of issuing code.
-- @tparam Logger self The logger object
-- @tparam string level @{level_strings|String} representation of message level
-- @tparam string msg User log message
-- @treturn string Logger formatted message
function class.timestamp_output(self, level, msg)
local result = ""
if class.need_lf then result = result .. "\n" end
......@@ -91,6 +209,9 @@ function class.timestamp_output(self, level, msg)
return result
end
--- Set default log level.
-- @param level Level value as @{level_values|integer} or @{level_strings|string} value.
-- @see default_level_value
function class.set_default_level(level)
local int_level, str_level = class.get_levels(level)
class.main_logger:debug("Updating default level to: " .. str_level)
......@@ -98,6 +219,8 @@ function class.set_default_level(level)
class.main_logger:set_level(int_level)
end
--- Set default output.
-- @tparam[opt=nil] string f Name of the file to use as output or `nil` to set `io.stdout`
function class.set_default_output(f)
class.main_logger:info("Updating log output: " .. f or "stdout")
if f then
......@@ -108,9 +231,9 @@ function class.set_default_output(f)
end
--- Return a string representation of a integer debug level (similar to Python ones).
-- @tparam int int_level Debug level as an integer between 0 an 50
-- @treturn string The name of the debug level suitable for lualogging
--- Get string representation of an integer log level.
-- @tparam integer int_level Log level as an @{level_values|integer}
-- @treturn string The @{level_strings|string} representation of the log level
function class._get_level_string(int_level)
for i, v in ipairs(class.level_values) do
if int_level <= v then
......@@ -120,6 +243,9 @@ function class._get_level_string(int_level)
return class.level_strings[1] -- Emergency case
end
--- Get integer representation of a string log level.
-- @tparam string str_level The @{level_strings|string} representation of the log level
-- @treturn integer Log level as an @{level_values|integer}
function class._get_level_value(str_level)
for i, v in ipairs(class.level_strings) do
if str_level == v then
......@@ -129,6 +255,10 @@ function class._get_level_value(str_level)
return class.level_values[1] -- Emergency case
end
--- Get representations of the given level.
-- @param level Log level as @{level_values|integer} or @{level_strings|string} value.
-- @treturn integer Log level as @{level_values|integer}
-- @treturn string Log level as @{level_strings|string}