This page serves as the feature specification. The feature will be included in an upcoming release.
In the relational model, data are structured: Entities represented in one table share the same set of attributes. This introduces clarity and enables predictable queries. In datasets with many classes of entities with different sets of attributes, relational designs result in large numbers of tables. Sometimes, it is conventient to deviate from the relational data model and to allow different sets of attributes in each tuple. It can already be accomplished by storing structures of dicts inside blob fields. However blob fields are opaque to queries: their contents cannot be used in searches.
DataJoint introduce the datatype
composite to allow storing objects with their own collections of of attributes.
For example, the following Table Definition has a composite attribute
# Recording rec : int # recording id ---- filepath : varchar(255) # filepath to the recording data stimulus : composite # parameters of the stimulus
To insert data into a composite attribute, use a
struct in MATLAB or a
dict in Python.
insert(test.Recording, struct('rec', 1, 'filepath', '/my/data/path', 'stimulus', struct('contrast', 0.3, 'luminance', 3.5)))
test.Recording().insert1(dict(rec=1, filepath='/my/data/path', 'stimulus'=dict(contrast=0.3, luminance=3.5)))
The main reason to have composite attributes is to allow using them in restrictions. This is done by using the
For example, the following restriction selects all recordings for which the stimulus is defined and exceeds 0.5.
Recording() & 'stimlus.comtrast > 0.5'
The composite datatype relies on JSON support by the database server. Latest versions of MySQL and MariaDB provide JSON support but Amazon’s own Aurora DB does not. We hope to use Aurora DB and hope that it will soon add JSON support.