The Adapt Universal Storage mechanism supports a hierarchy of applications and each one contains one or more Adapt data structures(dAtas).
A dAta is a hierarchy of dAta items and each one may be a simple item or a multiple item consisting of a set of sub-items.
Each item may exist once or may be repeated any number of times from zero upwards. There are a number of standard types of simple items, e.g. numeric, logical, raw text, link, recursive etc. and there may be additional types local to specific dAtas.
Link items provide a safe and flexible mechanism for interconnections between dAtas. Recursive items refer back to a higher level item allowing tree-type structures to be easily implemented.
Any dAta item may have a number of different logics associated with it which are invoked in certain situations to perform the functions associated with the dAta (see Adapt Logic Structures (Logics). In fact, the combination of dAtas and logics provides the total functionality of the system.
dAta Adaption
The process of developing Adapt applications involves interactively viewing dAtas and adding, modifying or deleting items and the associated logics as required. This is in contrast to conventional development technologies which rely on some form of offline programming involving source files.
As Adapt possesses Total Design Knowledge all previous design decisions may be viewed and altered as required. Similarly, various analyses of the effects of possible changes may be presented to the designer to ensure that the ramifications are understood. Also, for items that are calculated by logics then the relevant calculations can be shown and the conditions under which they apply displayed.
Efficiency Issues
dAtas are able to accommodate all types of information and are used to hold not only all application data but also ALL the data within the Adapt system at ALL times. This is an extremely important feature in that dAtas replace all the conventional forms of data storage with a single mechanism and for this reason, it is important that data values may be able to be accessed with very little software overhead and also to be stored in an efficient manner.
Efficient Access
The primary objective of any data storage method is the safety and integrity of the data values stored. With a conventional database system where values are accessed from applications, there is generally a large overhead due to the database software needing to locate the values to be accessed, validate the request and preserve the integrity of the data. The database has to ensure that a badly written program is not able to inadvertently corrupt its contents.
Adapt is able to ensure total data integrity and prevent any corruption because all the code that accesses the dAtas is automatically generated by the system from the information within the logics. This means that the code will only perform appropriate actions on the data so that the time-consuming checks required by databases are not needed and this can result in very high performance. Checks are applied when the designer makes changes to logics to ensure that only valid operations are defined and this means that such checks need not be made during the operation of the application.
The fact that all the code that accesses a dAta is automatically generated by Adapt is very important as it can be trusted to not do anything invalid. This means that the application code may simply be passed the pointers to data items it requires to access and can then directly change their values via the pointers as required. Because the code 'knows' the type and layout of the data items then it will always use or change them in an appropriate way.
In the event that the designer decides to change some aspect of a dAta such as adding/removing or changing the characteristics of a dAta item then all relevant code portions relating to those items will be automatically regenerated so that inconsistencies never arise.
Also, it is often the case that when a particular data value changes then other values may need to be recalculated or some action performed and the code that makes the change automatically includes calls to the relevant logic code.
Efficient Storage
As dAtas are able to store any type of information it is important to ensure that it is stored as efficiently as is reasonably possible. Even though large memories are available, at a very low cost these days, efficient data storage means that information loads faster from external memory and from the cloud. In addition, with such vast amounts of data held in cloud servers then if the data is held in a very compact form the number of servers needed is minimised.
Simple Items
For single data items Adapt analyses how they are used by the owning application in order to decide how much space each one needs. A simple example is the storage of numeric values. The designer only specifies that a particular item is numeric but is not required to decide how much space is needed to store it or how it is to be stored (e.g. as an integer, scaled integer or floating-point etc.). Adapt makes an initial assumption and then as the item is assigned values in expressions or displayed in images it may revise its previous decision. For example, a value may only be assigned integers within a specific range or may be displayed in images with a specific number of decimal places. Adapt takes all these factors into account to decide the most efficient size for the data item in terms of bits/bytes and the numeric form to be used.
Very often numeric values will only need a small number of bits of memory whereas a conventional system would automatically use 32 or 64 bits. If future designer changes make it clear that more bits are needed for a specific item, then this will be adjusted automatically and the relevant code regenerated. This is a good example of the importance of the distinction between designer and implementation decisions; the designer specifies the type of an item and how it is used and the system then decides how it is to be implemented based upon the facts available. If the designer subsequently changes how the item is used then the system may decide it needs to be implemented differently and will then make all the necessary changes transparently.
This automatic process is in contrast to the situation with 3GL-based conventional software where such a change would require all references to the item in the source code to be manually assessed/changed and then to be recompiled and the application re-built and re-tested.
Repeated Items
Any dAta item may be repeated any number of times as it is important to not impose arbitrary limitations upon the design of applications. dAta items may either be simple ones (such as characters or numbers) or multiple ones consisting of a number of sub-items.
Conventional databases often treat some simple data items as a special case; for example, repeated characters (known as strings) are generally limited to a specific size to simplify the implementation of the database. This forces the designer to choose a maximum size for strings and thus restricts the options for future changes. To minimise these limitations the designer tends to choose a large value for the size even though the typical string used will be very much shorter. This wastes a large amount of storage space and can make the storage efficiency low.
For multiple items, a conventional database will use a table held in a number of memory blocks. Very often such a table will be empty or hold only a small number of entries and this is wasteful as it always requires at least one memory block.
The Universal Storage mechanism stores information very efficiently by utilising the specially designed flexible list system. Essentially every repeated dAta item has a separate list consisting of any number of elements. Each element may hold a number of sub-lists (each corresponding to a lower level dAta item).
Larger sub-lists will occupy one or more blocks of memory but smaller ones are embedded in the higher-level element so that almost no memory is wasted. Often small lists may be totally empty or just hold a few small elements and the use of the list embedding technique ensure that the overhead is often just 2 or 3 bytes, rather than almost a whole block.
The decision about which list to embed or not is made within the flexible list system and is transparent to the rest of the Adapt. As dAtas are edited and elements are created or deleted then the layout of each block is analysed dynamically to decide on the most efficient organisation in terms of lists to embed and not embed.