Functions import

Functions import algorithm

Function definition file

Definition file must be compatible with export format mentioned here: Function File system structure

File names and extensions

The name of the file, excluding the extension, is the last segment of the function's code. The extension determines the type of the function:

• .js is the extension for Rhino functions
• .groovy is the extension for Groovy functions.

The letter case in the extension does not matter. For example, .groovy, .gRooVy and .GROOVY are valid extensions for Groovy function.

Files with different extensions are not imported. A warning message will be produced if such a file is in the folder where valid files with the definition of function are expected.

Validation

Some basic validations are done on the initial comment:

• existence and format of the comment
• existence and format of the definition of tags (categories) and arguments
• existence and content of the definition of required ctx argument
• whether the names of the function arguments do not repeat

The body of the function is not validated and is imported as provided.

File system directories

The whole file system structure must be compatible with the export format mentioned here: Function File system structure. If any additional files/directories are present in Hyperon directories:

• @functions
• @profiles

they are ignored during import.

Order of processing

The directory tree under the @functions/ folder is processed using a depth-first tree search algorithm. The algorithm will not start processing any sibling directory or another directory up the file tree until the processing of the current directory and all its subdirectories are finished. If the directory is supposed to contain files with the definition of functions, these files are processed after processing all subdirectories.

Files in a directory are grouped by base file name (i.e. file name without extension). Each such group is associated with a unique function code. Within each group, files are processed one by one, until encountering file with valid extension and executing successful import. The remaining files in the group are not processed. A warning message is produced in such cases.

There are no other rules determining the order of processing. Any observed regularity should not be relied upon, because such behavior is not guaranteed and may change in future Hyperon versions. In particular:

• the order of processing directories is not guaranteed. For example, it is not guaranteed that the @global/ directory will be processed before or after the @profiles/ directory.
• the order of processing files is not guaranteed. If there is more than one file with correct function definition and the same base name inside one directory, it is not determined, which file will be imported.

IMPORTANT

If the snapshot ZIP contains two functions with the same code but attached to different regions, the result will depend on the order of processing the import, which is not guaranteed. If the regions are in the same profile, the second import will remove the function created by the first import. If the regions are in different profiles, the second import will not execute, with status ERROR. In such cases, the snapshot file does not contain a valid structure of functions and should be corrected.

Individual import status

The table below summarizes the statuses of importing a function:

Example situations of unsuccessful import of function from a single file:

• the file had an incorrect extension,
• there was a file reading error,
• initial comment with the definition of function arguments and attached categories was not correct, because
• the comment was missing,
• the content of the comment was not a valid Toml string,
• list of attached categories was missing or had a wrong format,
• there were categories in the list which did not exist in the system
• list of function arguments was missing or had a wrong format,
• the definition of required 'ctx' argument was missing or incorrect,
• the list of function arguments contained more than one entry with the same name,
• the region, version of the region or profile of the function did not exist in the system,
• function with the same code already existed in the system but was attached to a different profile.

If the individual import result is not OK, the import result contains a message with a description of the issue. In case the message is not sufficient to establish the source of the problem, Hyperon logs can contain additional information. 

Algorithm for the analysis of function changes

(What happens if Hyperon database already contains some functions)

General rules for dealing with a situation where the database already contains a function with the same code are:

1. if the two functions are identical, the import is skipped, else:
2. if it is possible to create an imported function without changing the existing version, it is done, else
3. existing versions are substituted by imported function.

IMPORTANT

Functions are considered identical if they have identical code, region version (or are both globals), tags, arguments (including names, types, and order of arguments) and body. For the purpose of comparison, the body is left- and right-trimmed from whitespace characters.

The existence of the function is established from the perspective of the current user, i.e. taking into account all changes introduced in the current user's work session. If the import is done through the REST endpoint, the current user is the user authenticated in the POST import request.

Functions from one snapshot are imported one-by-one. From the perspective of importing individual functions, the state of the database already includes changes introduced by preceding imports from the same snapshot import operation.

Summary of operations in case of existing function with the same code:

‍

IMPORTANT

If the snapshot ZIP contains two functions with the same code but attached to different regions, the result will depend on the order of processing the import, which is not guaranteed. If the regions are in the same profile, the second import will remove the function created by the first import. If the regions are in different profiles, the second import will not execute, with status ERROR. In such cases, the snapshot file does not contain a valid structure of functions and should be corrected.

Importing functions with HTTP request

Executing import

To see how to execute import, please go to REST import endpoint

Import result

The response body of the request contains the results of the import in the JSON format.

{
    "importStatus": "ERROR",
    "jobResults": {
        "FUNCTION": {
            "jobStatus": "ERROR",
            "totalElementsCount": 3,
            "invalidElementsCount": 1,
            "skippedElementsCount": 1,
            "importResults": [
                {
                    "action": "IMPORT",
                    "functionCode": "premium.total",
                    "profileCode": "PROF2",
                    "regionCode": "REG1",
                    "version": "1",
                    "path": "/@functions/@profiles/PROF2/@regions/REG1/1/premium/total.js",
                    "message": "Attempt to attach function: premium.total to more than one profile: PROF1, PROF2",
                    "status": "ERROR"
                },
                {
                    "action": "IMPORT",
                    "functionCode": "coverage.isActive",
                    "path": "/@functions/@global/coverage/isActive.js",
                    "message": "No changes, file skipped",
                    "status": "SKIP"
                },
                {
                    "action": "IMPORT",
                    "functionCode": "coverage.isAvailable",
                    "path": "/@functions/@global/coverage/isAvailable.js",
                    "status": "OK"
                }
            ]
        }
    }
}

Results of importing functions are included in field jobResults.FUNCTION and have the following entries:

• jobStatus - overall status of the function's import job. Possible values are:
• SKIP - if there were no functions to import or all imports were skipped for reason other than error,
• OK - if not all imports were skipped and imports not skipped  were completed correctly,
• WARNING - if there was at least one individual function import which was completed but with warnings, and the remaining imports were completed correctly without warning or skipped for reason other than error,
• ERROR - if there were some imports not completed because of an error.
• totalElementsCount - counter of all executed imports, including those skipped, executed with error or warning
• invalidElementsCount - counter of all imports which ended with the status ERROR or WARNING,
• skippedElementsCount - counter of all imports which ended with status SKIP,
• importResults - an array of individual import results.

Individual import result

Individual import result has the following fields:

• status - status of the individual import. possible values are:
• OK - import executed correctly without any warnings
• WARNING - import was not executed because all files had incorrect extension or import was completed but there were some warnings due to unusual situation
• SKIP - import was not executed for reason other than error, for example, because identical function already existed
• ERROR - import was not completed due to an error
• action - performed action, which is IMPORT
• functionCode - code of the imported function
• message - the optional field presents only if the import status is not OK. It contains the message which should help establish why the status is not OK.
• profileCode - the optional field presents only for functions attached to region. It contains the profile code.
• regionCode - the optional field presents only for functions attached to region. It contains the region code.
• path - a path within the ZIP snapshot file to the file(s) with a definition of the imported function. If there are multiple files with different extension, a wildcard is used (e.g. "/@functions/@global/coverage/isAvailable.*").
• version - the optional field presents only for functions attached to the region. It contains the code of the region version.

Known limitations - importing functions from snapshot

National characters in profile, region or version names

Using characters outside the standard US-ASCII set is discouraged, due to the limitations associated with the encoding of paths in ZIP files. 

In Hyperon Studio function codes are validated against containing such characters. However, there is no such validation for the names of profiles, regions or region versions.

Mechanism of function snapshot import assumes the paths in the ZIP file are UTF-8 encoded. However, in practice, this encoding depends on the tool used to create the ZIP file, operating system and national settings. In particular, the standard "compressed folder" utility in Windows produces files with the different encoding of path names. The information about the encoding of paths is not included in ZIP file. These paths cannot be read by snapshot importer, so such files or directories will not be processed. The information about encountering unreadable path during import of snapshot function will be logged to Hyperon log, but without the path (since it is unreadable).

Names differing by letter case

It is not possible to import or export two functions attached to the same region version (or both global) with the first different name segments differing by letter case. For example, it is not possible to have both:

• ‍motor.premium.annual
• ‍motor.Premium.monthly

since the first different segment of the name is premium, and the difference is only by letter case: P vs. p. 

This limitation is associated with the limitation on the ZIP file system - it is not possible to have in one directory two files or sub-directories with names differing only by letter case (in the example above, we would have to have two sub-directories: premium/ and Premium/ in the same folder).

Functions in more than one profile

Currently, Hyperon does not support functions with the same code in multiple profiles. Although it is possible to create such functions, it is strongly discouraged, because it may lead to errors and inconsistencies. The mechanism of snapshot import will prevent such situations, by skipping import of a function attached to a region if a function with the same name, but attached to a region in a different profile, already exists in the system. The existence of such function is determined from the perspective of the user performing the import: all changes made in the current user's session are taken into account.

In particular, if the snapshot ZIP file contains the definition of two functions with the same code, but in different profiles, the second processed file will not be imported, because the function already exists in the current user's session, but in a different profile.

Autocommit for functions

Snapshot import of functions during session publishing will not detect conflicts with other user's session that was published.