Commit 62ee4875 authored by EXT Arnaud Clère's avatar EXT Arnaud Clère
Browse files

Documented design.md and a few other parts

parent d6a4f926
......@@ -744,6 +744,7 @@ public:
#include <QtCore/qmetatype.h>
#include <QtCore/qmetaobject.h>
// For QBIND_GADGET_WITH_METAOBJECT below
template<class T, class TResult>
TResult qbind(Val<TResult>&& v, T* t) {
auto rw = v->mode();
......
......@@ -253,8 +253,8 @@ protected:
With the addition of a << shortcut method, MyTextWriter can mimic QDebug and replace it in existing code:
```cpp
// 1.33333 3.14159 ascii characters are common in QDebug false QColor(ARGB 1, 0.176471, 0, 0.729412)
QDebug (&s ) << 1.333333333333f << PI << ascii << false << color ;
TextWriter(&ba) << 1.333333333333f << PI << ascii << false << color ;
QDebug (&s ) << 1.333333333333f << PI << ascii << false << color ;
MyTextWriter(&ba) << 1.333333333333f << PI << ascii << false << color ;
// [ 1.33333 3.14159 ascii characters are common in QDebug false [ RGB:[ 45 0 186] base:255]
```
......@@ -265,7 +265,7 @@ QJsonReader(&baIn).bind(QCborWriter(&baOut).value());
// baOut = 0x bf656e616d65739f644a6f686e63446f65ff66686569676874fa3fe0000063616765206670686f6e65739f6b2b343420313233343536376b2b34342032333435363738ff68636f6d6d656e747360686368696c6472656e9fffff
```
Last but not least, providing in advance some metadata allows binding deep C++ data structures in custom ways for use with multi-dimensional Q...View classes:
Last but not least, providing in advance some `meta` data allows binding deep C++ data structures in custom ways for use with multi-dimensional Q...View classes:
```cpp
QStandardItemModel tree, table, matrix;
QModelWriter<>(& tree).meta(qmChildren,"children" ).bind(persons);
......@@ -273,6 +273,14 @@ QModelWriter<>(& table).meta(qmColumns ,"names,age,children").bind(persons);
QModelWriter<>(&matrix).meta(qmSizes ,"4,3" ).bind(transform);
```
![Q...View](qstandardmodel.PNG)
What meta does is guide the way QModelWriter will translate nested `data` structures into the [QAbstractItemModel](https://doc.qt.io/Qt-5/qabstractitemmodel.html#details).
By convention:
- `qmChildren` defines a named item corresponding to a sequence of data with the same type and same [parent](https://doc.qt.io/Qt-5/qabstractitemmodel.html#parent)
- `qmColumns` defines the ordered set of named items that should be bound to their respective [columns](https://doc.qt.io/Qt-5/qmodelindex.html#column)
- `qmSizes` defines the bounds of successive dimensions of a N-dimensional row- or column-wise array (where N=2 to account for Q...View limitations)
- `qmName` allows naming data items for, e.g. XML root element and sequence items
- `qmType` provides type names for, e.g. XML attributes, CBOR tags
- `qmDataStreamVersion` allows binding specifically for QDataStream compatibility
## Benchmark
......@@ -280,8 +288,8 @@ TBD
## Conclusion
QBind is a (de)serialization mechanism that can be used on top of Qt, enabling convenient and efficient data transfers
among a lot of Qt data types including Json, Cbor, QVariant..., QModel... QMetaObject without loss, replacing **a lot** of
QBind is a (de)serialization mechanism that can be used on top of Qt, enabling convenient and efficient data transfers among
a lot of Qt data types and formats including Json, Cbor, QVariant..., QModel... QMetaObject without loss, replacing **a lot** of
format-specific code with a few `T::bind()` or `QBind<T>::bind()` methods (where T can be defined by user, Qt, or a third-part).
It can reuse QDebug and QDataStream << and >> operators, although this requires more visible templates and loses the ability
to switch format at runtime.
......
# Design
The common data model is formally described by the following recursive automaton:
The QBind traversal is formally described by the following recursive automaton:
```mermaid
graph LR
subgraph Value
subgraph Val
i((start))--"null()" --> x((end))
i((start))--"bind#lt;T>()"--> x((end))
i((start))--"sequence()" --> Sequence
i((start))--"record()" --> Record
Sequence --"out()" --> x((end))
Record --"out()" --> x((end))
Sequence --"item()" --> vs["Value#lt;Sequence>"]
Record --"item(name)" --> vr["Value#lt;Record>"]
i((start))--"sequence()" --> Seq
i((start))--"record()" --> Rec
Seq --"out()" --> x((end))
Rec --"out()" --> x((end))
Seq --"item()" --> vs["Val#lt;Seq>"]
Rec --"item(name)" --> vr["Val#lt;Rec>"]
end
```
- Boxes (nested) represent possible states when traversing the data, the automaton is always in a single valid state
- Edges represent legal state transitions that translate to specific data format read/write actions
- `bind<T>()` calls are forwarded to the actual `IBind` or generic `QBind` depending on `BindSupport<T>`:
- BindNative : **IBind**
- BindGeneric : **QBind**
## Well-formedness guarantees with minimum performance overhead
- Edges represent possible state transitions that translate to specific data format read/write actions
The automaton is implemented as follows:
- `Cursor` instances are, non-owning, almost-unique, pointers to an `IBind` interface which implementations translate data traversal
into specific data format read/write actions
- `Val<_>`, `Rec<_>` and `Seq<_>` types implement possible states where _ denotes the type of outer state
These types expose public methods restricted to the legal state transitions, that return the type representing the destination
state with a valid `Cursor` instance
- The initial state is `Val<Cursor>` and the final state is `Cursor`
- returning a `Rec<_>` or `Seq<_>` type automatically convert to the top-level `Cursor` type invoking as much `out()` transitions as required
- State types only expose public methods corresponding to possible transitions, that return the destination state type
- The initial state type is `Val<Cursor>` and the final state is `Cursor`
*For instance: `Val<Cursor>::null()` returns a `Cursor`*
- Returning a `Rec<_>` or `Seq<_>` type automatically convert to the final `Cursor` type invoking as much `out()` transitions as required
- `Cursor` is a non-owning pointer to an `IBind` interface which implementations translate data traversal into specific
data format read/write actions
- `Cursor` instance is moved from the start state type to the end state type only for successful transitions, allowing to test
alternatives before proceeding with the traversal
- Transitions may fail for various reasons specific to `IBind` implementations:
- builders may not be able to allocate new items
- readers may read data not matching the expected transition
- ...
- In case of unsuccessfull transition the returned state type receives a null `Cursor` that transparently bypasses calls to `IBind`
- `bind<T>()` calls are forwarded to the actual `IBind` or generic `QBind` depending on `BindSupport<T>`:
- BindNative : **IBind** interface method
- BindGeneric : **QBind** template specialization for T
- Every `bind<T>()` starts from a Val<Cursor> which is an un *unsafe* Cursor copy wrt well-formedness
*These `unsafeItem()` copies are protected from incorrect use*
## C++ types extensibility
QBind is a functor templated on T type receiving a Value and T reference (either lvalue or rvalue reference) and returning the Cursor.
Template specializations can be defined for any T and optionally refined for specific Cur<TImpl> with different sets of BindNative types.
A default QBind specialization attempts to call `T::bind(...)` to conveniently bind `T* this` without having to understand template syntax,
rvalue or forwarding references. Such custom implementations are facilitated by the fluent interface below.
## Convenient fluent interface
A convenient side-effect of encoding the QBind traversal in the type system is that smart C++ editors offer QBind-aware code completion
to the fluent interface making it similar to using an advanced XML editor for typing XML tags. Say, after typing `Value(myImpl).` the
editor will propose to either `bind(myData.item)`, or to construct a `sequence()`, `record()` or `null()` value.
## Well-formedness guarantees
Thanks to this design, the compiler will make sure that the only possibility to return a Cursor from a `Val<Cursor>` is to traverse
the data without backtracking, calling only and all necessary IBind virtual methods. Since `Cursor`, `Val`, `Rec` and `Seq` have no
data member other than outer types and `IBind*`, calling their methods can be optimized and replaced with just the following
operations:
the data without backtracking, calling only and all necessary IBind virtual methods.
The addition of default and optional values take into account most data schema evolutions in a purely declarative fluent interface without
having to test schema versions and the like. The benefit is that it is not possible to introduce bugs using just the fluent interface.
The downside is that writing loops with the fluent interface is unnatural as one must never forget to follow the valid Cursor.
For instance:
```cpp
auto seq(v.sequence());
for (auto&& t : ts) {
seq = seq.bind(t); // do not forget to reassign seq, or subsequent items will be `bind` to the moved-from Cursor and automatically ignored
}
```
## Write performance
Since `Cursor`, `Val`, `Rec` and `Seq` have no data member other than outer types and `IBind*`, calling their methods can be
optimized and replaced with just the following operations:
1. test the IBind pointer validity [^1]
2. call the IBind virtual method corresponding to the legal transitions
2. call the IBind virtual method corresponding to the possible transitions
3. return the resulting Cursor, Val, Rec or Seq with a valid or invalid Cursor depending on IBind method success or failure
[^1]: Experiments to use constexpr to bypass this step for writers that always return true did not seem to improve performance.
`QBind<T>` can define up to 3 bind() overloads to efficiently and conveniently handle lvalue references, const lvalue references, and
rvalue references depending on T characteristics (which of copy/move is 1/possible and 2/efficient).
Compared to manually calling non-virtual, format-specific implementations, the overhead of always testing the validity of IBind*
and calling virtual methods is around 20% in our benchmark, with a maximum of 100% for trivial format-specific implementations
like copying a single char to a pre-allocated buffer. This performance cost is usually dwarfed by unoptimized code and the ability to
select a data format that performs well for the data at hand. For instance, choosing Cbor with a mixture of boolean, character
literals and numbers makes this overhead negligible.
like copying a single char to a pre-allocated buffer.
This performance cost is usually dwarfed by unoptimized code and the ability to select a data format that performs well for the data
at hand. For instance, choosing Cbor with a mixture of boolean, character literals and numbers makes this overhead negligible.
Other than that, write performance depends on several factors:
- An explicit QUtf8String type allows handling item names much more efficiently than using utf16 QString, while still being
distinguishable from QByteArray binary data
- In the end, directly using QByteArray buffers instead of using QIODevice can amount to ~ 2x better write performance
- IBind implementations need to use efficient data structures from storing state. For instance, using an optimized std::vector<bool>
to memorize opened JSON object/array can usually be stored in a single byte and avoid memory allocations, resulting in ~ 10x better
write performance compared to QVector<bool>
A convenient side-effect of encoding the common data model in the type system is that smart C++ editors offer data-model-aware code
completion to this fluent interface. Say, after typing `Value(myImpl).` the editor will propose to either `bind(myData.item)`, or to
construct a `sequence()`, `record()` or `null()` value.
## Read robustness
## Format extensibility
Performance is not so important for Read. But compared to manually calling non-virtual, format-specific implementations, QBind
enforces well-formedness checks necessary to reliably read data coming from unknown sources (IBind implementations being responsible
for low-level checks).
All errors are reported as `const char*` literals to the `IBind` implementations that will decide what to do with them:
- ignore the details and set a global error status enumeration is usually appropriate to IWriter implementations
- storing all read mismatches is usually more appropriate to world-facing IReader implementations
Standardizing error literals allows efficient reporting and analysis while ensuring that various libraries can define new ones independantly.
## Data types extensibility
IBind is an abstract base class for translating fluent interface calls to format-specific read or write actions.
The fluent interface guarantees that IBind virtual methods are always called at appropriate times, so IBind implementations do not
have to check well-formedness. It defines a set of BindNative types and default textual representations and encodings for non-textual
native types simplifying again the implementations (see TextWriter example).
IWriter and IReader provide partial IBind implementations simplifying the work of implementors and offering default textual representations.
*NB:* BindNative types could be extended by specifying BindSupport<TImpl,T> trait but then, (de)serialization code must be specialized
for each TImpl. For instance, a QBind<QColor,QDataStream> may be implemented differently from QBind<QColor,IBind> but QBind<QColor>
example shows that meta() can also be used to avoid specialized serialization code that breaks W5 requirement. If meta() is deemed
enough, the BindSupport trait and TImpl template parameters can be removed.
## Types extensibility
QBind is a functor templated on T type receiving a Value and T reference (either lvalue or rvalue reference) and returning the Cursor.
Template specializations can be defined for any T and optionally refined for specific Cur<TImpl> with different sets of BindNative types.
A default QBind specialization attempts to call `T::bind(...)` to conveniently bind `T* this` without having to understand template syntax,
rvalue or forwarding references.
## IBind implementations
IWriter and IReader provide partial IBind implementations simplifying the work of implementors and offering default textual representations.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment