Using MultiValue Databases

MultiValue is a type of NoSQL and multidimensional database, previously considered synonymous with PICK, a database originally developed as the Pick operating system. MutliValue databases give you a tremendous amount of flexibility in terms of how you store your data which means you are able to utilise incredibly efficient data storage structures. These efficiencies of speed and flexibility should follow through to a business app.
Evoke supports most MultiValue databases and allows users to continue to use and benefit from the data structures and efficiences that they have incorporated by using a MultiValue Database. This section reviews how MultiValue, SubValue and remote/local key data, commonly used in MultiValue databases, is supported in Evoke and also how existing and new custom MultiValue DataBASIC can be used in an Evoke app/app design. Details of how to set up your MultiValue database in Evoke are seperate to these specific MultiValue use instructions.
Using MultiValue, SubValue and Remote/Local key data structures
MultiValued and SubValued data
The most effective way to illustrate this use is by using an Example. When using a MultiValue Database a typical example of using MultiValued data would be an ORDER file with MultiValued Data. Each item in the Order file might have attributes that relate to the order e.g. date, time, etc. but also each ORDER LINE may be held as MultiValued data within the item and DELIVERY information may be further held as SubValued data within each ORDER LINE. There is an example of this exact data structure in the Sales Order Processing App Design that is included in all Evoke accounts.
To understand how Evoke handles MultiValued data you should consider that all related MultiValued attributes are "pivoted". This means that the data is considered as the ORDER item (record) having multiple ORDER LINE records within it and each ORDER LINE record having multiple DELIVERY records within it. Each record is represented in Evoke as an Entity.
The data in the database (the raw data) for this type of record would look like the data on the left.

An Order file/object therefore has three (3) Entities in an Evoke App Design - the SALESORDER entity, which has related Entities of SALESORDERLINE(s), which in turn has related Entities of SALESDELIVERY(s).
The image on the right shows an example of the the Entity structure for the SALESORDER Entity.


These three (3) Entities now need to be mapped in the Evoke Data Mappings.
All three (3) Entities are now to be associated with the SAME Repository Object Name (in the example image on the left this is the SALESORDER file in the Database). Local Data attributes (simple standalone fields) are mapped to their attribute position and Related Data has each of the attribute positions, in Evoke data Mappings, mapped (as associated multivalue data) to their respective attribute positions.

However, within the Entity representing the MultiValued (or SubValued) data (in the example on the left this is the SalesOrderLine Entity) the Attribute positions revert to starting from 1 through n and relate to the position of the data within the MultiValued Associated Group (now defined by the Entity SalesOrderLine).
If the MultiValued data is an independant standalone list of values i.e. just one field that contains MultiValued data (and you wish to use/show this list in your app UI) then the data mappings would be as in the example in the image on the right.
This example is also available in the Sales Order Processing App Design that is included in all Evoke Accounts. In the Sales Order Processing App Design you will also see examples of MultiValued and SubValued data being Populated into DataSources through Selections and Related data retrieval as is explained further below.

Local and Remote Key association
The second way that MultiValue Databases often associate data records together is via Local and remote keys. In this case a record may hold a single or multivalued list of keys linking the record to one or more records in another file.
Local Keys mean that this record holds the single or multiple keys into another file and Remote Keys means the keys may be found in records in another file. Remote Keys require a small amount of additional CRUD code to be added by you as explained in the link.
Set each Property/Attribute of the related Data to have an Entity Key Storage, in Data Mappings, to be either Local Key or Remote Key (see image left).
The Entity definition (see image right) will define which file the local key relates to or where the remote keys should be looked for.
In the example shown, the Organization Entity has a Property of "Products". This in turn is defined as Remote key storage in the Data Mappings. The Product Entity is defined, in Data Mappings, as relating the Database object (file) 'ProductFile' by having this name in the 'Repository object name'.
A Local Key Definition would mean that the Organisation File holds, in the Attribute position '4' (Products), one or more item IDs into the ProductFile file that will be used to read/populate the identified ProductFile records, when a RetrieveRelatedData Click action or EntityPropertiesToLoad (buttons and selections) action is performed, into the Organization file DataSource.
A Remote Key Definition (as shown above left) would mean that the ProductFile records hold a single Item ID (in the Attribute position '4') that may relate to the Item ID of this Organization File record. Remote key storage means that the item ID of this entity (item) is held in one or more items in another file. This will be used to identify the ProductFile records to be read/populated, when a RetrieveRelatedData Click action or EntityPropertiesToLoad (buttons and selections) action is performed, into the Organization file DataSource.


Populating Related Data

When you populate a DataSource with a Selection (button, click action, page action, etc) then any Related Data, that you wish to access/use needs to be populated (read from the database) seperately. Please refer to the Retrieve Related Data section of Populating a DataSource to see how to use the RetrieveRelatedData click actions or 'Entity Properties to Load' options on buttons and widgets.


Repository CRUD

Repository CRUD (Create, Read, Update, Delete) code is required for MultiValue Databases to provide access to the database and create the Selections that you set up against your Entities.

The Evoke app generator will create the initial content of these server-side CRUD routines and install the CRUD routines into a file called EVOKE.CRUD.BP, please also review the instructions for creating this program file.

New CRUD code will be generated by Evoke whenever you check the "Generate Repository CRUD" check box, against any or each Entity in the Generate Web App area.

The naming and content of these CRUD routines is based on the Entities you have created in your App Design. Each data entity will be represented by 2 routines called EV.CRUD.xxx and EV.CRUD.xxx.CUSTOM, where xxx represents the name of the data entity.
The EV.CRUD.xxx routine is "owned" by the app generator, that is, you should not amend its content. Conversely, the EV.CRUD.xxx.CUSTOM routine (referred to hereafter as the "custom CRUD routine") is "owned" by you, and will only be modified by the generator once - i.e. when it is first created by the generator.

Whenever you regenerate a CRUD routine you must also compile and catalogue the routine in your database. If you are using a .NET data connector to your database (such as mv.NET) that uses connection pooling then you must remember to STOP any active sessions to your database, in the .NET data connector, whenever you compile and catalogue new CRUD code. This is so that the new byte-code is re-read into the data connector.


Embedded MultiValued Data Related Data CRUD Code

As explained above Embedded MultiValues, Embedded SubValues, etc are described in separate Entity structures. This means that separate CRUD code will be generated for these. Each EV.CRUD.XXXX.CUSTOM item for each Entity that represents either Embedded MultiValues or EmbeddedSubValues, will NEED to be edited by you. The image on the right shows a typical EV.CRUD.XXXX.CUSTOM item (click on the image to see a larger version of the image).

You will see, between lines 10 and 15 the following:
*
* For embedded entities uncomment the 2 lines below
*
* ActionCompleted = TRUE
* RETURN
*
You must remove the "*" on the ActionCompleted and Return lines, save the item, and then compile/re-compile and catalogue, to indicate that this related to MultiValued/SubValued data.


Embedded SubValued Data Related Data CRUD Code

Subvalued attributes or groups need to be treated slightly differently. In the CRUD Basic code (Evoke.Crud.BP) locate the EV.CRUD.XXXX.CUSTOM item where XXXX is the Entity name of the parent entity of the sub-valued attribute/group (the sub-valued entity).
The image on the right shows a typical EV.CRUD.XXXX.CUSTOM item (click on the image to see a larger version of the image) for the parent entity of the subvalued entity.

You will see, between lines 10 and 15 that the lines remain commented.
However, ActionCompleted = TRUE has been placed after each CASE statement that is not being used for the subvalues.
You now need to OPEN the file that contains the multivalues and subvalues. In the example on the right this is done on lines 17 to 23.

If you want to READ the subvalues (i.e. RetrieveRelatedData, this is the most common requirement for a subvalue or a subvalued associated group) then you need to add some new custom BASIC code in the READ CASE statement (see lines 32 to 43). This code will vary based on each group of subvalues. An example is provided below. For create, update, delete, etc you will need to add similar basic code into the appropriate case statement:

SALESORDERID = FIELD(DataIn,'.',1)       * the key of the record/item that contains the subvalues
LINEPOS = FIELD(DataIn,'.',2)          * the position of the multivalues (single or group) that you want to retrieve subvalues for
IF LEN(SALESORDERID) > 0 THEN       * check that you have the key of the record/item that contains the subvalues
  READ SALESORDERDATA FROM SALESORDERLINEFILE, SALESORDERID THEN        * read the item from the file (SALESORDERLINEFILE) into SALESORDERDATA using the key (SALESORDERID)
     SALESORDERLINEDATA = ''
    FOR X = 4 TO 12       * loop around the attributes in the multivalued group (in this case they are attributes 4 to 12)
      SALESORDERLINEDATA = SALESORDERDATA       * Create the data to return
    NEXT X
    DataOut = SALESORDERID:AM:SALESORDERLINEDATA
    ActionCompleted = TRUE
  END
END

Save the item, and then compile/re-compile and catalogue.


Using Embedded MultiValue Data in the App Design UI

If you have followed the information above, you have created your MultiValue Entities, Mapped your data to your Database as Related Data and created your Repository CRUD code. Now you need to use the data in the User Interface of your App Design.

The example screen on the right is once again from the Sales Order Processing app design included in all Evoke accounts. It shows the display of the ORDERS and ORDERLINES that were used in the example above.

A Selection was run against the top segment on this screen (Customer Name), this populated ORDERS into the DataSource called Pending. This selection is identified in the Template associated with this segment and the "Pending" DataSource populated, in this specific case, via the "Actions on Option Select" option in the Template.

The Related Data (in this case MultiValued ORDERLINE data) was retrieved via either a) use of the RetrieveRelatedData ClickAction or b) the Selection Widget EntityPropertiesToLoad option (in the Editor specific section of the Template definition). If you look in the Sales Order Processing App Design you will see option b was used.

The selected Data Grid shown on the image has a DataSource (identified and filled in by using the DataSource Build button) of "Pending.current.OrderLines" - which means that the information shown in the grid will be the MultiValued data (defined as ORDERLINES) associated with whichever ORDER is selected in the grid above.







Using Existing and new MultiValue DataBasic in your app
Most multivalue applications will include a lot of custom coded data basic logic, which is often the powerhouse of multivalue database applications. A business app needs to hook into, enhance and embrace this data basic logic in order to maintain the original investment in creating this custom code. And then go on to enhance and embrace the existing business application.
One of the main purposes of the generated App Design's .NET code is to invoke the relevant DataBASIC subroutines residing on your MultiValue database server. The specific server-side routine that is called and the data that is passed to it depends entirely upon the type of repository related action that has occurred within the App UI. Evoke can create the initial content of these server-side routines (known as CRUD routines - Create, Read, Update and Delete) and you are then able to add your own code as necessary in order to implement the required database file access logic.

At this point it is necessary to create this initial code and full instructions are available in the CRUD creation for multivalue databases user guide/help pages.
The EV.CRUD.xxx.CUSTOM item, for each Entity, allows you to, where relevant/necessary, override the code, in the EV.CRUD.xxx, which can be viewed as the default implementation of each of the possible CRUD actions, by incorporating your own DataBASIC code.

The default implementation of each CRUD action within the EV.CRUD.xxx subroutine is based on the data entity mapping definition set up within the app generator. This mapping data, in essence, defines a logical structure for the "blob" of entity data that passes between .NET and DataBASIC. The knowledge of this structure by both .NET and DataBASIC is obviously crucial because it is the only way in which these 2 environments can successfully synchronize their individual elements of data exchange logic.

Thus, the default implementation of each CRUD action within DataBASIC is based on the assumption that each entity maps onto a single database file, and that the structure of this file is exactly the same as the app generator's mapping definition blob described above. This may be true for some entities, but probably not true for many others. In this latter case you will need to override the default implementation of CRUD actions with your own code within the CUSTOM version of the CRUD routines.


Sometimes it is useful to look at the generated CRUD routine in order to look at an example of how some of the calling signature arguments can/should be used.
A simple example of customising the Evoke generated CRUD code is provided here however, the BlueFinity support team will be happy to assist further.
If you are working with the Custom Basic code extensions AND Evoke integration with the Basic code, the manual (downloadable by clicking on the image on the left) further explains the CRUD Basic parameters ControlIN, DataIn, DataOut etc. and has proved very useful.