Using Document Transformations

Overview#

The LAMP Platform supports Document Transformations using the JSONata query language developed by IBM. The following documentation is extracted from the JSONata documentation website.

The following methods corresponding to API calls have been added to the JSONata engine:

$LAMP.ActivityEvent.list(participant_id: string, origin: string | null, from: number | null, to: number | null, limit: number | null) => ActivityEvent[]
$LAMP.ActivityEvent.create(participant_id: string, activity_events: ActivityEvent | ActivityEvent[]) => void
$LAMP.Activity.list(study_id: string, ignore_binary: boolean = true, sibling: boolean = false) => Activity[]
$LAMP.Activity.create(activity_id: string, activity: Activity) => void
$LAMP.Activity.get(activity_id: string) => Activity
$LAMP.Activity.set(activity_id: string, activity: Activity | null) => void
$LAMP.ActivitySpec.list() => ActivitySpec[]
$LAMP.ActivitySpec.create(activity_spec: ActivitySpec) => void
$LAMP.ActivitySpec.get(activity_spec_id: string) => ActivitySpec
$LAMP.ActivitySpec.set(activity_spec_id: string, activity_spec: ActivitySpec | null) => void
$LAMP.Credential.list(type_id: string | null) => Credential[]
$LAMP.Credential.create(type_id: string | null, credential: Credential) => void
$LAMP.Credential.get(type_id: string | null, access_key: string) => Credential
$LAMP.Credential.set(type_id: string | null, access_key: string, credential: Credential | null) => void
$LAMP.Participant.list(study_id: string, sibling: boolean = false) => Participant[]
$LAMP.Participant.create(study_id: string, participant: Participant) => void
$LAMP.Participant.get(participant_id: string) => Participant
$LAMP.Participant.set(participant_id: string, participant: Participant) => void
$LAMP.Researcher.list() => Researcher[]
$LAMP.Researcher.create(researcher: Researcher) => void
$LAMP.Researcher.get(researcher_id: string) => Researcher
$LAMP.Researcher.set(researcher_id: string, researcher: Researcher | null) => void
$LAMP.SensorEvent.list(participant_id: string, origin: string | null, from: number | null, to: number | null, limit: number | null) => SensorEvent[]
$LAMP.SensorEvent.create(participant_id: string, sensor_events: SensorEvent | SensorEvent[]) => void
$LAMP.Sensor.list(study_id: string, ignore_binary: boolean = true, sibling: boolean = false) => Sensor[]
$LAMP.Sensor.create(study_id: string, sensor: Sensor) => void
$LAMP.Sensor.get(sensor_id: string) => Sensor
$LAMP.Sensor.set(sensor_id: string, sensor: Sensor | null) => void
$LAMP.SensorSpec.list() => SensorSpec[]
$LAMP.SensorSpec.create(sensor_spec: SensorSpec) => void
$LAMP.SensorSpec.get(sensor_spec_id: string) => SensorSpec
$LAMP.SensorSpec.set(sensor_spec_id: string, sensor_spec: SensorSpec | null) => void
$LAMP.Study.list(researcher_id: string) => Study[]
$LAMP.Study.create(researcher_id: string, study: Study) => void
$LAMP.Study.get(study_id: string) => Study
$LAMP.Study.set(study_id: string, study: Study | null) => void
$LAMP.Type.parent(type_id: string) => string
$LAMP.Tag.list(type_id: string) => string[]
$LAMP.Tag.get(type_id: string, attachment_key: string) => any
$LAMP.Tag.set(type_id: string, attachment_key: string, target: string, attachment_value: string) => void

Introduction#

JSONata is a lightweight query and transformation language for JSON data. Inspired by the 'location path' semantics of XPath 3.1, it allows sophisticated queries to be expressed in a compact and intuitive notation. A rich complement of built in operators and functions is provided for manipulating and combining extracted data, and the results of queries can be formatted into any JSON output structure using familiar JSON object and array syntax. Coupled with the facility to create user defined functions, advanced expressions can be built to tackle any JSON query and transformation task. Try it out here.

Simple Queries#

To support the extraction of values from a JSON structure, a location path syntax is defined. In common with XPath, this will select all possible values in the document that match the specified location path. The two structural constructs of JSON are objects and arrays.

Navigating JSON Objects#

A JSON object is an associative array (a.k.a map or hash). The location path syntax to navigate into an arbitrarily deeply nested structure of JSON objects comprises the field names separated by dot '.' delimiters. The expression returns the JSON value referenced after navigating to the last step in the location path. If during the navigation of the location path, a field is not found, then the expression returns nothing (represented by Javascript undefined). No errors are thrown as a result of non-existing data in the input document.

The following sample JSON document is used by examples throughout this guide, unless otherwise indicated:

{
"FirstName": "Fred",
"Surname": "Smith",
"Age": 28,
"Address": {
"Street": "Hursley Park",
"City": "Winchester",
"Postcode": "SO21 2JN"
},
"Phone": [
{
"type": "home",
"number": "0203 544 1234"
},
{
"type": "office",
"number": "01962 001234"
},
{
"type": "office",
"number": "01962 001235"
},
{
"type": "mobile",
"number": "077 7700 1234"
}
],
"Email": [
{
"type": "work",
"address": ["fred.smith@my-work.com", "fsmith@my-work.com"]
},
{
"type": "home",
"address": ["freddy@my-social.com", "frederic.smith@very-serious.com"]
}
],
"Other": {
"Over 18 ?": true,
"Misc": null,
"Alternative.Address": {
"Street": "Brick Lane",
"City": "London",
"Postcode": "E1 6RF"
}
}
}

Examples#

Here are some example expressions and their results when applied to the above JSON document:

  • Returns a JSON string ("double quoted")

    Query

    Surname

    Result

    "Smith"
  • Returns a JSON number

    Query

    Age

    Result

    28
  • Field references are separated by '.'

    Query

    Address.City

    Result

    "Winchester"
  • Matched the path and returns the null value

    Query

    Other.Misc

    Result

    null
  • Path not found. Returns nothing (i.e. Javascript undefined) Query

    Other.Nothing

    Result

  • Field references containing whitespace or reserved tokens can be enclosed in backticks

    Query

    Other.`Over 18 ?`

    Result

    true

Navigating JSON Arrays#

JSON arrays are used when an ordered collection of values is required. Each value in the array is associated with an index (position) rather than a name, so in order to address individual values in an array, extra syntax is required to specify the index. This is done using square brackets after the field name of the array. If the square brackets contains a number, or an expression that evaluates to a number, then the number represents the index of the value to select. Indexes are zero offset, i.e. the first value in an array arr is arr[0]. If the number is not an integer, then it is rounded down to an integer. If the expression in square brackets is non-numeric, or is an expression that doesn't evaluate to a number, then it is treated as a predicate.

Negative indexes count from the end of the array, for example, arr[-1] will select the last value, arr[-2] the second to last, etc. If an index is specified that exceeds the size of the array, then nothing is selected.

If no index is specified for an array (i.e. no square brackets after the field reference), then the whole array is selected. If the array contains objects, and the location path selects fields within these objects, then each object within the array will be queried for selection.

Examples#

  • Returns the first item (an object)

    Query

    Phone[0]

    Result

    { "type": "home", "number": "0203 544 1234" }
  • Returns the second item

    Query

    Phone[1]

    Result

    { "type": "office", "number": "01962 001234" }
  • Returns the last item

    Query

    Phone[-1]

    Result

    { "type": "mobile", "number": "077 7700 1234" }
  • Negative indexed count from the end

    Query

    Phone[-2]

    Result

    { "type": "office", "number": "01962 001235" }
  • Doesn't exist - returns nothing Query

    Phone[8]

    Result

  • Selects the number field in the first item

    Query

    Phone[0].number

    Result

    "0203 544 1234"
  • No index is given to Phone so it selects all of them (the whole array), then it selects all the number fields for each of them

    Query

    Phone.number

    Result

    [ "0203 544 1234", "01962 001234", "01962 001235", "077 7700 1234" ]
  • Might expect it to just return the first number, but it returns the first number of each of the items selected by Phone

    Query

    Phone.number[0]

    Result

    [ "0203 544 1234", "01962 001234", "01962 001235", "077 7700 1234" ]
  • Applies the index to the array returned by Phone.number. One use of parentheses.

    Query

    (Phone.number)[0]

    Result

    "0203 544 1234"
  • Returns a range of items by creating an array of indexes

    Query

    Phone[[0..1]]

    Result

    [
    { "type": "home", "number": "0203 544 1234" },
    { "type": "office", "number": "01962 001234" }
    ]

Result sequences#

Sequence flattening#

Empty sequences and singleton sequences#

Top level arrays, nested arrays and array flattening#

Consider the JSON document:

[
{ "ref": [ 1,2 ] },
{ "ref": [ 3,4 ] }
]

At the top level, we have an array rather than an object. If we want to select the first object in this top level array, we don't have a field name to append the [0] to. We can't use [0] on its own because that clashes with the array constructor syntax. However, we can use the context reference $ to refer to the start of the document as follows:

  • $ at the start of an expression refers to the entire input document

    Query

    $[0]

    Result

    { "ref": [ 1,2 ] }
  • .ref here returns the entire internal array

    Query

    $[0].ref

    Result

    [ 1,2 ]
  • returns element on first position of the internal array

    Query

    $[0].ref[0]

    Result

    1
  • Despite the structure of the nested array, the resultant selection is flattened into a single flat array. The original nested structure of the input arrays is lost. See Array constructors for how to maintain the original structure in the results.

    Query

    $.ref

    Result

    [ 1, 2, 3, 4 ]

Query refinement using predicate expressions#

Predicates#

At any step in a location path, the selected items can be filtered using a predicate - [expr] where expr evaluates to a Boolean value. Each item in the selection is tested against the expression, if it evaluates to true, then the item is kept; if false, it is removed from the selection. The expression is evaluated relative to the current (context) item being tested, so if the predicate expression performs navigation, then it is relative to this context item.

Examples:#

  • Select the Phone items that have a type field that equals "mobile".

    Query

    Phone[type='mobile']

    Result

    { "type": "mobile", "number": "077 7700 1234" }
  • Select the mobile phone number

    Query

    Phone[type='mobile'].number

    Result

    "077 7700 1234"
  • Select the office phone numbers - there are two of them!

    Query

    Phone[type='office'].number

    Result

    [ "01962 001234", "01962 001235" ]

Singleton array and value equivalence#

Within a JSONata expression or subexpression, any value (which is not itself an array) and an array containing just that value are deemed to be equivalent. This allows the language to be composable such that location paths that extract a single value from and object and location paths that extract multiple values from arrays can both be used as inputs to other expressions without needing to use different syntax for the two forms.

Consider the following examples:

  • Address.City returns the single value "Winchester"
  • Phone[0].number matches a single value, and returns that value "0203 544 1234"
  • Phone[type='home'].number likewise matches the single value "0203 544 1234"
  • Phone[type='office'].number matches two values, so returns an array [ "01962 001234", "01962 001235" ]

When processing the return value of a JSONata expression, it might be desirable to have the results in a consistent format regardless of how many values were matched. In the first two expressions above, it is clear that each expression is addressing a single value in the structure and it makes sense to return just that value. In the last two expressions, however, it is not immediately obvious how many values will be matched, and it is not helpful if the host language has to process the results in different ways depending on what gets returned.

If this is a concern, then the expression can be modified to make it return an array even if only a single value is matched. This is done by adding empty square brackets [] to a step within the location path. The examples above can be re-written to always return an array as follows:

  • Address[].City returns [ "Winchester"]
  • Phone[0][].number returns [ "0203 544 1234" ]
  • Phone[][type='home'].number returns [ "0203 544 1234" ]
  • Phone[type='office'].number[] returns [ "01962 001234", "01962 001235" ]

Note that the [] can be placed either side of the predicates and on any step in the path expression

Wildcards#

Use of * instead of field name to select all fields in an object

Examples#

  • Select the values of all the fields of Address

    Query

    Address.*

    Result

    [ "Hursley Park", "Winchester", "SO21 2JN" ]
  • Select the Postcode value of any child object

    Query

    *.Postcode

    Result

    "SO21 2JN"

Navigate arbitrary depths#

Descendant wildcard ** instead of * will traverse all descendants (multi-level wildcard).

Examples#

  • Select all Postcode values, regardless of how deeply nested they are in the structure

    Query

    **.Postcode

    Result

    [ "SO21 2JN", "E1 6RF" ]

Manipulating data with functions and expressions#

String expressions#

Path expressions that point to a string value will return that value.

String literals can also be created by enclosing the sequence of characters in quotes. Either double quotes " or single quotes ' can be used, provided the same quote type is used for the start and end of the string literal. Single quote characters may be included within a double quoted string and vice versa withought escaping. Characters within the string literal may be escaped using the same format as JSON strings.

Strings can be combined using the concatenation operator &. This is an infix operator and will join the two strings returned by the expressions either side of it. This is the only operator that will attempt to typecast its operands to the expected (string) type.

Examples

  • Concatenate FirstName followed by space followed by Surname

    Query

    FirstName & ' ' & Surname

    Result

    "Fred Smith"
  • Concatenates the Street and City from the Address object with a comma separator. Note the use of parentheses

    Query

    Address.(Street & ', ' & City)

    Result

    "Hursley Park, Winchester"
  • Casts the operands to strings, if necessary

    Query

    5&0&true

    Result

    "50true"

Numeric expressions#

Path expressions that point to a number value will return that value.

Numeric literals can also be created using the same syntax as JSON numbers.

Numbers can be combined using the usual mathematical operators to produce a resulting number. Supported operators:

  • + addition
  • - subtraction
  • * multiplication
  • / division
  • % remainder (modulo)

Examples

Consider the following JSON document:

{
"Numbers": [1, 2.4, 3.5, 10, 20.9, 30]
}
ExpressionOutputComments
Numbers[0] + Numbers[1]3.4Adding 2 prices
Numbers[0] - Numbers[4]-19.9Subtraction
Numbers[0] * Numbers[5]30Multiplying price by quantity
Numbers[0] / Numbers[4]0.04784688995215Division
Numbers[2] % Numbers[5]3.5Modulo operator

Comparison expressions#

Often used in predicates, for comparison of two values. Returns Boolean true or false. Supported operators:

  • = equals
  • != not equals
  • < less than
  • <= less than or equal
  • > greater than
  • >= greater than or equal
  • in value is contained in an array

Examples

ExpressionOutputComments
Numbers[0] = Numbers[5]falseEquality
Numbers[0] != Numbers[4]trueInequality
Numbers[1] < Numbers[5]trueLess than
Numbers[1] <= Numbers[5]trueLess than or equal
Numbers[2] > Numbers[4]falseGreater than
Numbers[2] >= Numbers[4]falseGreater than or equal
"01962 001234" in Phone.numbertrueValue is contained in

Boolean expressions#

Used to combine Boolean results, often to support more sophisticated predicate expressions. Supported operators:

  • and
  • or

Note that not is supported as a function, not an operator.

Examples

ExpressionOutputComments
(Numbers[2] != 0) and (Numbers[5] != Numbers[1])trueand operator
(Numbers[2] != 0) or (Numbers[5] = Numbers[1])trueor operator

Building result structures#

So far, we have discovered how to extract values from a JSON document, and how to manipulate the data using numeric, string and other operators. It is useful to be able to specify how this processed data is presented in the output.

Array constructors#

As previously observed, when a location path matches multiple values in the input document, these values are returned as an array. The values might be objects or arrays, and as such will have their own structure, but the matched values themselves are at the top level in the resultant array.

It is possible to build extra structure into the resultant array by specifying the construction of arrays (or objects) within the location path expression. At any point in a location path where a field reference is expected, a pair of square brackets [] can be inserted to specify that the results of the expression within those brackets should be contained within a new array in the output. Commas are used to separate multiple expressions within the array constructor.

Array constructors can also be used within location paths for making multiple selections without the broad brush use of wildcards.

Examples

  • The four email addresses are returned in a flat array.

    Query

    Email.address

    Result

    [
    "fred.smith@my-work.com",
    "fsmith@my-work.com",
    "freddy@my-social.com",
    "frederic.smith@very-serious.com"
    ]
  • Each email object generates an array of addresses.

    Query

    Email.[address]

    Result

    [
    [ "fred.smith@my-work.com", "fsmith@my-work.com" ],
    [ "freddy@my-social.com", "frederic.smith@very-serious.com" ]
    ]
  • Selects the City value of both Address and Alternative.Address objects.

    Query

    [Address, Other.`Alternative.Address`].City

    Result

    [ "Winchester", "London" ]

Object constructors#

In a similar manner to the way arrays can be constructed, JSON objects can also be constructed in the output. At any point in a location path where a field reference is expected, a pair of braces {} containing key/value pairs separated by commas, with each key and value separated by a colon: {key1: value2, key2:value2}. The keys and values can either be literals or can be expressions. The key must either be a string or an expression that evaluates to a string.

When an object constructor follows an expression that selects multiple values, the object constructor will create a single object that contains a key/value pair for each of those context values. If an array of objects is required (one for each context value), then the object constructor should immediately follow the dot '.' operator.

Examples

  • Produces an array of objects (one for each phone).

    Query

    Phone.{type: number}

    Result

    [
    { "home": "0203 544 1234" },
    { "office": "01962 001234" },
    { "office": "01962 001235" },
    { "mobile": "077 7700 1234" }
    ]
  • Combines the key/value pairs into a single object. See Grouping using object key expression for more details.

    Query

    Phone{type: number}

    Result

    {
    "home": "0203 544 1234",
    "office": [
    "01962 001234",
    "01962 001235"
    ],
    "mobile": "077 7700 1234"
    }
  • Combines the key/value pairs into a single object. In this case, for consistency, all numbers are grouped into arrays. See Singleton array and value equivalence for more details.

    Query

    Phone{type: number[]}

    Result

    {
    "home": [
    "0203 544 1234"
    ],
    "office": [
    "01962 001234",
    "01962 001235"
    ],
    "mobile": [
    "077 7700 1234"
    ]
    }

JSON literals#

The array and object constructors use the standard JSON syntax for JSON arrays and JSON objects. In addition to this values of the other JSON data types can be entered into an expression using their native JSON syntax:

  • strings - "hello world"
  • numbers - 34.5
  • Booleans - true or false
  • nulls - null
  • objects - {"key1": "value1", "key2": "value2"}
  • arrays - ["value1", "value2"]

JSONata is a superset of JSON. This means that any valid JSON document is also a valid JSONata expression. This property allows you to use a JSON document as a template for the desired output, and then replace parts of it with expressions to insert data into the output from the input document.

Query composition#

In JSONata, everything is an expression. An expression comprises values, functions and operators which, when evaluated, produces a resulting value. Functions and operators are applied to values which themselves can be the results of evaluating sub-expressions. In that way, the language is fully composable to any level of complexity.

Parenthesized expressions and blocks#

Used to override the operator precedence rules. E.g.

  • (5 + 3) * 4

Used to compute complex expressions on a context value

  • Product.(Price * Quantity) - both Price and Quantity are properties of the Product object

Used to support 'code blocks' - multiple expressions, separated by semicolons

  • (expr1; expr2; expr3)

Each expression in the block is evaluated in sequential order; the result of the last expression is returned from the block.

Sorting, Grouping and Aggregation#

Sorting#

Arrays contain an ordered collection of values. If you need to re-order the values, then the array must be sorted. In JSONata, there are two ways of sorting an array:

  1. Using the $sort() function.

  2. Using the order-by operator.

The order-by operator is a convenient syntax that can used directly in a path expression to sort the result sequences in ascending or descending order. The $sort() function requires more syntax to be written, but is more flexible and supports custom comparator functions.

Grouping#

The JSONata object constructor syntax allows you to specify an expression for the key in any key/value pair (the value can obviously be an expression too). The key expression must evaluate to a string since this is a restriction on JSON objects. The key and value expressions are evaluated for each item in the input context (see processing model). The result of each key/value expression pair is inserted into the resulting JSON object.

If the evaluation of any key expression results in a key that is already in the result object, then the result of its associated value expression will be grouped with the value(s) already associated with that key. Note that the value expressions are not evaluated until all of the grouping has been performed. This allows for aggregation expressions to be evaluated over the collection of items for each group.

Examples

  • Group all of the product sales by name, with the price of each item in each group

    Query

    Account.Order.Product{`Product Name`: Price}

    Result

    {
    "Bowler Hat": [ 34.45, 34.45 ],
    "Trilby hat": 21.67,
    "Cloak": 107.99
    }
  • Group all of the product sales by name, with the price and the quantity of each item in each group

    Query

    Account.Order.Product {
    `Product Name`: {"Price": Price, "Qty": Quantity}
    }

    Result

    {
    "Bowler Hat": {
    "Price": [ 34.45, 34.45 ],
    "Qty": [ 2, 4 ]
    },
    "Trilby hat": { "Price": 21.67, "Qty": 1 },
    "Cloak": { "Price": 107.99, "Qty": 1 }
    }

Note in the above example, the value expression grouped all of the prices together and all of the quantities together into separate arrays. This is because the context value is the sequence of all grouped Products and the Price expression will select all prices from all products. If you want to collect the price and quantity into individual objects, then you need to evaluate the object constructor for each product in the context sequence. The following example shows this.

  • Explicit use of $.{ ... } to create an object for each item in the group.

    Query

    Account.Order.Product {
    `Product Name`: $.{"Price": Price, "Qty": Quantity}
    }

    Result

    {
    "Bowler Hat": [
    { "Price": 34.45, "Qty": 2 },
    { "Price": 34.45, "Qty": 4 }
    ],
    "Trilby hat": { "Price": 21.67, "Qty": 1 },
    "Cloak": { "Price": 107.99, "Qty": 1 }
    }
  • Multiply the Price by the Quantity for each product in each group

    Query

    Account.Order.Product{`Product Name`: $.(Price*Quantity)}

    Result

    {
    "Bowler Hat": [ 68.9, 137.8 ],
    "Trilby hat": 21.67,
    "Cloak": 107.99
    }
  • The total aggregated value in each group

    Query

    Account.Order.Product{`Product Name`: $sum($.(Price*Quantity))}

    Result

    {
    "Bowler Hat": 206.7,
    "Trilby hat": 21.67,
    "Cloak": 107.99
    }

Aggregation#

Often queries are just required to return aggregated results from a set of matching values. A number of aggregation functions are available which return a single aggregated value when applied to an array of values.

Examples

  • Total price of each product in each order

    Query

    $sum(Account.Order.Product.Price)

    Result

    198.56
  • More likely want to add up the total of the price times quantity for each order

    Query

    $sum(Account.Order.Product.(Price*Quantity))

    Result

    336.36

Other numeric aggregation functions are available (i.e. average, min, max) and an aggregator for strings. It is also possible to write complex custom aggregators using the $reduce() higher-order function.

The JSONata processing model#

The JSONata type system#

JSONata is a superset of JSON and the JSONata type system is a superset of the JSON data types. In common with all functional programming languages, the function is also a first-class data type. The following data types are supported by JSONata:

  • string
  • number
  • Boolean
  • null
  • object
  • array
  • function

All but the last one are in common with JSON.

Sequences#

JSONata has been designed foremost as a query language, whereby a path expression can select zero, one or more than one values from the JSON document. These values, each of which can be of any of the types listed above, are returned as a result sequence. During the evaluation of expressions, which involves the results of subexpressions being combined or becoming the context inputs to other subexpressions, the sequences are subject to the process of sequence flattening.

The sequence flattening rules are as follows:

  1. An empty sequence is a sequence with no values and is considered to be 'nothing' or 'no match'. It won't appear in the output of any expression. If it is associated with an object property (key/value) pair in a result object, then that object will not have that property.

  2. A singleton sequence is a sequence containing a single value. It is considered equivalent to that value itself, and the output from any expression, or sub-expression will be that value without any surrounding structure.

  3. A sequence containing more than one value is represented in the output as a JSON array. This is still internally flagged as a sequence and subject to the next rule. Note that if an expression matches an array from the input JSON, or a JSON array is explicitly constructed in the query using the array constructor, then this remains an array of values rather than a sequence of values and will not be subject to the sequence flattening rules. However, if this array becomes the context of a subsequent expression, then the result of that will be a sequence.

  4. If a sequence contains one or more (sub-)sequences, then the values from the sub-sequence are pulled up to the level of the outer sequence. A result sequence will never contain child sequences (they are flattened).

JSONata path processing#

The JSONata path expression is a declarative functional language.

Functional because it is based on the map/filter/reduce programming paradigm as supported by popular functional programming languages through the use of higher-order functions.

Declarative because these higher-order functions are exposed through a lightweight syntax which lets the user focus on the intention of the query (declaration) rather than the programming constructs that control their evaluation.

A path expression is a sequence of one or more of the following functional stages:

StageSyntaxAction
Mapseq.exprEvaluates the RHS expression in the context of each item in the input sequence. Flattens results into result sequence.
Filterseq[expr]Filter results from previous stage by applying predicate expression between brackets to each item.
Sortseq^(expr)Sorts (re-orders) the input sequence according to the criteria in parentheses.
Indexseq#$varBinds a named variable to the current context position (zero offset) in the sequence.
Joinseq@$varBinds a named variable to the current current context item in the sequence. Can only be used directly following a map stage.
Reduceseq{ expr:expr, expr:expr ...}Group and aggregate the input sequence to a single result object as defined by the name/value expressions. Can only appear as the final stage in a path expression.

In the above table:

  • In the 'Syntax' column, 'seq' refers to the input sequence for the current stage, which is the result sequence from the previous stage.
  • The 'Action' column gives a brief outline of the stage's behavior; fuller details are in the Path Operators reference page.
  • The relative precedence of each operator affects the scope of its influence on the input sequence. Specifically,
    • The Filter operator binds tighter than the Map operator. This means, for example, that books.authors[0] will select the all of the first authors from each book rather than the first author from all of the books.
    • The Sort (order-by) operator has the lowest precedence, meaning that the full path to the left of it will be evaluated, and its result sequence will be sorted.
    • This operator precedence can be overridden by using parentheses. For example, (books.authors)[0] will select the the first author from all of the books (single value). Note, however, that parentheses also define a scope frame for variables, so any variables that have been bound within the parentheses block including those bound by the @ and # operators will go out of scope at the end of the parens block.
  • The variables bound by the @ and # operators go out of scope at the end of the path expression.
    • The Reduce stage, if used, will terminate the current path expression. Although a Map operator can immediately follow this, it will be interpreted as the start of a new path expression, meaning that any previously bound context or index variables will be out of scope.

Functional programming constructs#

So far, we have introduced all the parts of the language that allow us to extract data from an input JSON document, combine the data using string and numeric operators, and format the structure of the output JSON document. What follows are the parts that turn this into a Turing complete, functional programming language.

Comments#

JSONata expressions can be interleaved with comments using 'C' style comment delimeters. For example,

/* Long-winded expressions might need some explanation */
(
$pi := 3.1415926535897932384626;
/* JSONata is not known for its graphics support! */
$plot := function($x) {(
$floor := $string ~> $substringBefore(?, '.') ~> $number;
$index := $floor(($x + 1) * 20 + 0.5);
$join([0..$index].('.')) & 'O' & $join([$index..40].('.'))
)};
/* Factorial is the product of the integers 1..n */
$product := function($a, $b) { $a * $b };
$factorial := function($n) { $n = 0 ? 1 : $reduce([1..$n], $product) };
$sin := function($x){ /* define sine in terms of cosine */
$cos($x - $pi/2)
};
$cos := function($x){ /* Derive cosine by expanding Maclaurin series */
$x > $pi ? $cos($x - 2 * $pi) : $x < -$pi ? $cos($x + 2 * $pi) :
$sum([0..12].($power(-1, $) * $power($x, 2*$) / $factorial(2*$)))
};
[0..24].$sin($*$pi/12).$plot($)
)

Produces this, if you're interested!

Conditional logic#

If/then/else constructs can be written using the ternary operator "? :".

predicate ? expr1 : expr2

The expression predicate is evaluated. If its effective boolean value (see definition) is true then expr1 is evaluated and returned, otherwise expr2 is evaluated and returned.

Examples

TBD

Variables#

Any name that starts with a dollar '$' is a variable. A variable is a named reference to a value. The value can be one of any type in the language's type system.

Built-in variables#

  • $ The variable with no name refers to the context value at any point in the input JSON hierarchy. Examples
  • $$ The root of the input JSON. Only needed if you need to break out of the current context to temporarily navigate down a different path. E.g. for cross-referencing or joining data. Examples
  • Native (built-in) functions. See function library.

Variable binding#

Values (of any type in the type system) can be bound to variables

$var_name := "value"

The stored value can be later referenced using the expression $var_name.

The scope of a variable is limited to the 'block' in which it was bound. E.g.

Invoice.(
$p := Product.Price;
$q := Product.Quantity;
$p * $q
)

Returns Price multiplied by Quantity for the Product in the Invoice.

Functions#

The function is a first-class type, and can be stored in a variable just like any other data type. A library of built-in functions is provided (link) and assigned to variables in the global scope. For example, $uppercase contains a function which, when invoked with a string argument, str, will return a string with all the characters in str changed to uppercase.

Invoking a function#

A function is invoked by following its reference (or definition) by parentheses containing a comma delimited sequence of arguments.

Examples

  • $uppercase("Hello") returns the string "HELLO".
  • $substring("hello world", 0, 5) returns the string "hello"
  • $sum([1,2,3]) returns the number 6

Defining a function#

Anonymous (lambda) functions can be defined using the following syntax:

function($l, $w, $h){ $l * $w * $h }

and can be invoked using

function($l, $w, $h){ $l * $w * $h }(10, 10, 5) which returns 500

The function can also be assigned to a variable for future use (within the block)

(
$volume := function($l, $w, $h){ $l * $w * $h };
$volume(10, 10, 5);
)

Function signatures#

Functions can be defined with an optional signature which specifies the parameter types of the function. If supplied, the evaluation engine will validate the arguments passed to the function before it is invoked. A dynamic error is thown if the argument list does not match the signature.

A function signature is a string of the form <params:return>. params is a sequence of type symbols, each one representing an input argument's type. return is a single type symbol representing the return value type.

Type symbols work as follows:

Simple types:

  • b - Boolean
  • n - number
  • s - string
  • l - null

Complex types:

  • a - array
  • o - object
  • f - function

Union types:

  • (sao) - string, array or object
  • (o) - same as o
  • u - equivalent to (bnsl) i.e. Boolean, number, string or null
  • j - any JSON type. Equivalent to (bnsloa) i.e. Boolean, number, string, null, object or array, but not function
  • x - any type. Equivalent to (bnsloaf)

Parametrised types:

  • a<s> - array of strings
  • a<x> - array of values of any type

Some examples of signatures of built-in JSONata functions:

  • $count has signature <a:n>; it accepts an array and returns a number.
  • $append has signature <aa:a>; it accepts two arrays and returns an array.
  • $sum has signature <a<n>:n>; it accepts an array of numbers and returns a number.
  • $reduce has signature <fa<j>:j>; it accepts a reducer function f and an a<j> (array of JSON objects) and returns a JSON object.

Each type symbol may also have options applied.

  • + - one or more arguments of this type
    • E.g. $zip has signature <a+>; it accepts one array, or two arrays, or three arrays, or...
  • ? - optional argument
    • E.g. $join has signature <a<s>s?:s>; it accepts an array of strings and an optional joiner string which defaults to the empty string. It returns a string.
  • - - if this argument is missing, use the context value ("focus").
    • E.g. $length has signature <s-:n>; it can be called as $length(OrderID) (one argument) but equivalently as OrderID.$length().

Recursive functions#

Functions that have been assigned to variables can invoke themselves using that variable reference. This allows recursive functions to be defined. Eg.

Query

(
$factorial:= function($x){ $x <= 1 ? 1 : $x * $factorial($x-1) };
$factorial(4)
)

Result

24

Note that it is actually possible to write a recursive function using purely anonymous functions (i.e. nothing gets assigned to variables). This is done using the Y-combinator which might be an interesting diversion for those interested in functional programming.

Tail call optimization (Tail recursion)#

A recursive function adds a new frame to the call stack each time it invokes itself. This can eventually lead to stack exhaustion if the function recuses beyond a certain limit. Consider the classic recursive implementation of the factorial function

(
$factorial := function($x) {
$x <= 1 ? 1 : $x * $factorial($x-1)
};
$factorial(170)
)

This function works by pushing the number onto the stack, then when the stack unwinds, multiplying it by the result of the factorial of the number minus one. Written in this way, the JSONata evaluator has no choice but to use the call stack to store the intermediate results. Given a large enough number, the call stack will overflow.

This is a recognised problem with functional programming and the solution is to rewrite the function slightly to avoid the need for the stack to store the itermediate result. The following implementation of factorial achieves this

(
$factorial := function($x){(
$iter := function($x, $acc) {
$x <= 1 ? $acc : $iter($x - 1, $x * $acc)
};
$iter($x, 1)
)};
$factorial(170)
)

Here, the multiplication is done before the function invokes itself and the intermediate result is carried in the second parameter $acc (accumulator). The invocation of itself is the last thing that the function does. This is known as a 'tail call', and when the JSONata parser spots this, it internally rewrites the recursion as a simple loop. Thus it can run indefinitely without growing the call stack. Functions written in this way are said to be tail recursive.

Higher order functions#

A function, being a first-class data type, can be passed as a parameter to another function, or returned from a function. Functions that process other functions are known as higher order functions. Consider the following example:

(
$twice := function($f) { function($x){ $f($f($x)) } };
$add3 := function($y){ $y + 3 };
$add6 := $twice($add3);
$add6(7)
)
  • The function stored in variable $twice is a higher order function. It takes a parameter $f which is a function, and returns a function which takes a parameter $x which, when invoked, applies the function $f twice to $x.
  • $add3 stores a function that adds 3 to its argument. Neither $twice or $add3 have been invoked yet.
  • $twice is invoked by passing the function add3 as its argument. This returns a function that applies $add3 twice to its argument. This returned function is not invoked yet, but rather assigned to the variable add6.
  • Finally the function in $add6 is invoked with the argument 7, resulting in 3 being added to it twice. It returns 13.

Functions are closures#

When a lambda function is defined, the evaluation engine takes a snapshot of the environment and stores it with the function body definition. The environment comprises the context item (i.e. the current value in the location path) together with the current in-scope variable bindings. When the lambda function is later invoked, it is done so in that stored environment rather than the current environment at invocation time. This property is known as lexical scoping and is a fundamental property of closures.

Consider the following example:

Account.(
$AccName := function() { $.'Account Name' };
Order[OrderID = 'order104'].Product.{
'Account': $AccName(),
'SKU-' & $string(ProductID): $.'Product Name'
}
)

When the function is created, the context item (referred to by '$') is the value of Account. Later, when the function is invoked, the context item has moved down the structure to the value of each Product item. However, the function body is invoked in the environment that was stored when it was defined, so its context item is the value of Account. This is a somewhat contrived example, you wouldn't really need a function to do this. The expression produces the following result:

{
"Account": "Firefly",
"SKU-858383": "Bowler Hat",
"SKU-345664": "Cloak"
}

Partial function application#

Functions can partially applied by invoking the function with one or more (but not all) arguments replaced by a question mark ? placeholder. The result of this is another function whose arity (number of parameters) is reduced by the number of arguments supplied to the original function. This returned function can be treated like any other newly defined function, e.g. bound to a variable, passed to a higher-order function, etc.

Examples

  • Create a function to return the first five characters of a string by partially applying the $substring function

    Query

    (
    $first5 := $substring(?, 0, 5);
    $first5("Hello, World")
    )

    Result

    "Hello"
  • Partially applied function can be further partially applied

    Query

    (
    $firstN := $substring(?, 0, ?);
    $first5 := $firstN(?, 5);
    $first5("Hello, World")
    )

    Result

    "Hello"

Function chaining#

Function chaining can be used in two ways:

  1. To avoid lots of nesting when multiple functions are applied to a value

  2. As a higher-order construct for defining new functions by combining existing functions

Invocation chaining#

value ~> $funcA ~> $funcB\ is equivalent to\ $funcB($funcA(value))

Examples

  • Customer.Email ~> $substringAfter("@") ~> $substringBefore(".") ~> $uppercase()

Function composition#

Function composition is the application of one function to another function to produce a third function.

$funcC := $funcA ~> $funcB\ is equivalent to\ $funcC := function($arg) { $funcB($funcA($arg)) }

Examples

  • Create a new function by chaining two existing functions Query
    (
    $normalize := $uppercase ~> $trim;
    $normalize(" Some Words ")
    )
    Result
    "SOME WORDS"

Functions as first class values#

Function composition can be combined with partial function application to produce a very compact syntax for defining new functions.

Examples

  • Create a new function by chaining two partially evaluated functions Query
    (
    $first5Capitalized := $substring(?, 0, 5) ~> $uppercase(?);
    $first5Capitalized(Address.City)
    )
    Result
    "WINCH"

Advanced example - The Y-combinator#

There is no need to read this section - it will do nothing for your sanity or ability to manipulate JSON data.

Earlier we learned how to write a recursive function to calculate the factorial of a number and hinted that this could be done without naming any functions. We can take higher-order functions to the extreme and write the following:

位($f) { 位($x) { $x($x) }( 位($g) { $f( (位($a) {$g($g)($a)}))})}(位($f) { 位($n) { $n < 2 ? 1 : $n * $f($n - 1) } })(6)

which produces the result 720. The Greek lambda (位) symbol can be used in place of the word function which, if you can find it on your keyboard, will save screen space and please the fans of lambda calculus.

The first part of this above expression is an implementation of the Y-combinator in this language. We could assign it to a variable and apply it to other recursive anonymous functions:

(
$Y := 位($f) { 位($x) { $x($x) }( 位($g) { $f( (位($a) {$g($g)($a)}))})};
[1,2,3,4,5,6,7,8,9] . $Y(位($f) { 位($n) { $n <= 1 ? $n : $f($n-1) + $f($n-2) } }) ($)
)

to produce the Fibonacci series [ 1, 1, 2, 3, 5, 8, 13, 21, 34 ].

But we don't need to do any of this. Far more sensible to use named functions:

(
$fib := 位($n) { $n <= 1 ? $n : $fib($n-1) + $fib($n-2) };
[1,2,3,4,5,6,7,8,9] . $fib($)
)

Using Regular Expressions#

The regular expression is a syntax for matching and extracting parts of a string. JSONata provides first class support for regular expressions surrounded by the familiar slash delimeters found in many scripting languages.

/regex/flags

where:

  • regex - the regular expression

  • flags - optionally either or both of:

    • i - ignore case
    • m - multiline match

    Functions which use regular expressions#

A number of functions are available that take a regular expression as a parameter

Examples

Regular expressions in query predicates#

Regexes are often used in query predicates (filter expressions) when selecting objects that contain a matching string property. For this, a short cut notation can be used as follows:

path.to.object[stringProperty ~> /regex/]

The ~> is the chain operator, and its use here implies that the result of /regex/ is a function. We'll see below that this is in fact the case.

Examples

Account.Order.Product[`Product Name` ~> /hat/i ]

will match all products that have 'hat' in their name.

Generic matchers#

The JSONata type system is based on the JSON type system, with the addition of the function type. In order to accommodate the regex as a standalone expression, the syntax /regex/ evaluates to a function. Think of the /regex/ syntax as a higher-order function that results in a 'matching function' when evaluated. The regex between the slashes is the parameter to this HOF and the function that gets returned, when applied to its string parameter, will return a structure that contains details of parts of the string that have been matched. If nothing is matched, then it returns the empty sequence (i.e. JavaScript undefined).

Example

$matcher := /[a-z]*an[a-z]*/i

Evaluation of the regex returns a function, and this has been bound to a variable $matcher. Later, the $matcher function is invoked on a string:

$matcher('A man, a plan, a canal, Panama!')

This returns the following JSONata object (JSON, but also with a function property):

{
"match": "man",
"start": 2,
"end": 4,
"groups": [],
"next": "<native function>#0"
}

This contains information of the first matching substring within this famous palindrome, specifically:

  • match - the substring within the original string that matches the regex
  • start - the starting position (zero offset) of the matching substring within the original string
  • end - the endinging position of the matching substring within the original string
  • groups - if capturing groups are used in the regex, then this array contains a string for the text captured by each group
  • next() - when invoked, will return details of the second occurrence of any matching substring (and so on).

In this example, invoking next() will return:

{
"match": "canal",
"start": 17,
"end": 22,
"groups": [],
"next": "<native function>#0"
}

and so on, until it eventually returns the empty sequence.

Writing a custom matcher#

We've learned that the regex syntax is just a function generator, and the signature and return structure of the generated 'matcher' function is well defined. The four regex-aware functions ($match, $contain, $split, $replace) simply invoke this function as part of their implementation. Apart from that, they have no awareness that these matcher functions were generated by the regex syntax.

So it's possible to write any user-defined matcher function, provided it conforms to this contract. This can be done as a JSONata lambda function or (more likely) as an extension function. It can then be passed to these four 'regex-aware' functions and they will search using the custom matcher rather than a regex.

Date/Time processing#

The 'evaluation time鈥 - $now()#

There are two functions that return the 'current' date/time timestamp:

  1. $now() returns the timestamp in an ISO 8601 formatted string.
  2. $millis() returns the same timestamp as the number of milliseconds since midnight on 1st January 1970 UTC (the Unix epoch).

The timestamp is captured at the start of the expression evaluation, and that same timestamp value is returned for every occurrence of $now() and $millis() in the same expression for the duration of the evaluation.

Example

  • The timestamp will be the same for all invocations within an expression Query
    {
    "invoiceTime": $now(),
    "total": $sum(Account.Order.Product.(Price * Quantity)),
    "closingTime": $now()
    }
    Result
    {
    "invoiceTime": "2018-12-10T13:49:51.141Z",
    "total": 336.36,
    "closingTime": "2018-12-10T13:49:51.141Z"
    }

JSON and ISO 8601#

JSON does not have a built-in type for date/time values. The general consensus is to store the date/time value as a string in ISO 8601 format.

Example

{
"myDateTime": "2018-12-10T13:45:00.000Z"
}

JSONata follows this convention and provides functions for formatting and parsing ISO 8601 formatted timestamps (toMillis() and fromMillis())

Support for other date/time formats#

Since there is no standard for date/time format in JSON, it is entirely possible that the JSON data you are working with will have date/time values formatted in other ways. JSONata supports the highly versatile picture string notation from the XPath/XQuery fn:format-dateTime() specification for both the formatting and parsing of a wide variety of date/time formats.

See toMillis() and fromMillis() for details.

Examples

  • The date 12/10/2018 in US format and 10/12/2018 in European format both refer to the same day.

    Query

    $toMillis('10/12/2018', '[D]/[M]/[Y]') ~> $fromMillis('[M]/[D]/[Y]')

    Result

    "12/10/2018"
  • More verbose format.

    Query

    $toMillis('10/12/2018', '[D]/[M]/[Y]')
    ~> $fromMillis('[FNn], [D1o] [MNn] [YI]')

    Result

    "Monday, 10th December MMXVIII"

Path Operators#

The path operators underpin the declarative nature of the map/filter/reduce processing model in JSONata.

. (Map)#

The dot operator is one of the fundamental building blocks in JSONata expressions. It implements the 'for each' or 'map' function that is common in many functional languages.

The dot operator performs the following logic:

  • The expression on the LHS is evaluated to produce an array of values.
    • If it evaluates to a single value, that is treated as equivalent to an array containing that single value
    • If it evaluates to nothing (no match or empty array), then the result of the operator expression is nothing
  • For each value in the LHS array in turn:
    • The value is known as the context and is used as the basis for any relative path expression on the RHS. It is also accessible in the RHS expression using the $ symbol.
    • The RHS expression is evaluated to produce a value or array of values (or nothing). These values are appended to a combined array of results for the operator as a whole.
  • The combined result of the operator is returned.

This operator is left associative meaning that the expression a.b.c.d is evaluated like ((a.b).c).d; i.e. left to right

Examples

  • Address.City => "Winchester"
  • Phone.number => [ "0203 544 1234", "01962 001234", "01962 001235", "077 7700 1234" ]
  • Account.Order.Product.(Price * Quantity) => [ 68.9, 21.67, 137.8, 107.99 ]
  • Account.Order.OrderID.$uppercase() => [ "ORDER103", "ORDER104"]

[ ... ] (Filter)#

The filter operator (a.k.a predicate) is used to select only the items in the input sequence that satisfy the predicate expression contained between the square brackets.

If the predicate expression is an integer, or an expression that evaluates to an integer, then the item at that position (zero offset) in the input sequence is the only item selected for the result sequence. If the number is non-integer, then it is rounded down to the nearest integer.

If the predicate expression is an array of integers, or an expression that evaluates to an array of integers, then the items at those positions (zero offset) in the input sequence is the only item selected for the result sequence.

If the predicate expression evaluates to any other value, then it is cast to a Boolean as if using the $boolean() function. If this evaluates to true, then the item is retained in the result sequence. Otherwise it is rejected.

See Navigating JSON Arrays and Predicates for more details and examples.

^( ... ) (Order-by)#

The order-by operator is used to sort an array of values into ascending or descending order according to one or more expressions defined within the parentheses.

By default, the array will be sorted into ascending order. For example:

Account.Order.Product^(Price)

sorts all of the products into order of increasing price (Price is a numeric field in the Product object).

To sort in descending order, the sort expression must be preceded by the > symbol. For example:

Account.Order.Product^(>Price)

sorts all of the products into order of decreasing price. The < symbol can be used to explicitly indicate ascending order, although that is the default behaviour.

Secondary (and more) sort expressions can be specified by separating them with commas (,). The secondary expression will be used to determine order if the primary expression ranks two values the same. For example,

Account.Order.Product^(>Price, <Quantity)

orders the products primarily by decreasing price, but for products of the same price, by increasing quantity.

The sort expression(s) can be any valid JSONata expression that evaluates to a number or a string. If it evaluates to a string then the array is sorted in order of unicode codepoint.

Examples

  • Account.Order.Product^(Price * Quantity) => Increasing order of price times quantity.
  • student[type='fulltime']^(DoB).name => The names of all full time students sorted by date of birth (the DoB value is an ISO 8601 date format)

{ ... } (Reduce)#

The reduce operator can be used as the last step in a path expression to group and aggregate its input sequence into a single object. The key/value pairs between the curly braces determine the groupings (by evaluating the key expression) and the aggregated values for each group. See Grouping and Aggregation for more details.

* (Wildcard)#

This wildcard selects the values of all the properties of the context object. It can be used in a path expression in place of a property name, but it cannot be combined with other characters like a glob pattern. The order of these values in the result sequence is implementation dependent. See Wildcards for examples.

** (Descendants)#

This wildcard recursively selects the values of all the properties of the context object, and the properties of any objects contained within these values as it descends the hierarchy. See Navigate arbitrary depths.

% (Parent)#

This will select the 'parent' of the current context value. Here, we define 'parent' to be the enclosing object which has the property representing the context value.

This is the only operation which searches 'backwards' in the input data structure. It is implemented by static analysis of the expression at compile time and can only be used within expressions that navigate through that target parent value in the first place. If, for any reason, the parent location cannot be determined, then a static error (S0217) is thrown.

Example

Account.Order.Product.{
'Product': `Product Name`,
'Order': %.OrderID,
'Account': %.%.`Account Name`
}

This returns an array of objects for each product in each order in each account. Information from the enclosing Order and Account objects can be accessed using the parent operator. The repeated combination of %.%. is used to access the grandparent and higher ancestors.

# (Positional variable binding)#

This can be used to determine at which position in the sequence the current context item is. It can be used following any map, filter or order-by stage in the path. The variable is available for use within subsequent stages of the path (e.g. within filter predicates) and goes out of scope at the end of the path expression.

Example

library.books#$i['Kernighan' in authors].{
'title': title,
'index': $i
}

This returns an array of objects for each book in the library where Kernighan is one of the authors. Each object contains the book's title and its position within the books array before it was filtered.

@ (Context variable binding)#

This is used to bind the current context item ($) to a named variable. It can only be used directly following a map stage, not a filter or order-by stage. The variable binding remains in scope for the remainder of the path expression.

Because the current context has now been explicitly bound to a named variable, this context will be carried forward to be the context of the next stage in the path. For example, in this snippet of a path, library.loans@$l.books, the loans array is a property of the library object and each loan will, in turn, be bound to the variable $l. The books array, which is also a property of the library object, will then be selected.

This operator can be used to perform data joins within a path because of its ability to do cross-referencing across objects.

Example

library.loans@$l.books@$b[$l.isbn=$b.isbn].{
'title': $b.title,
'customer': $l.customer
}

This performs an 'inner join' between objects in the loans array and objects in the books array where the ISBNs match between the structures.

Block expressions can be used to widen the scope of the data cross-referencing as shown in this example:

(library.loans)@$l.(catalog.books)@$b[$l.isbn=$b.isbn].{
'title': $b.title,
'customer': $l.customer
}

Numeric Operators#

+ (Addition)#

The addition operator adds the operands to produce the numerical sum. It is an error if either operand is not a number.

Example

5 + 2 => 7

- (Substraction/Negation)#

The subtraction operator subtracts the RHS value from the LHS value to produce the numerical difference It is an error if either operand is not a number.

It can also be used in its unary form to negate a number

Examples

  • 5 - 2 => 3
  • - 42 => -42

* (Multiplication)#

The multiplication operator multiplies the operands to produce the numerical product. It is an error if either operand is not a number.

Example

5 * 2 => 10

/ (Division)#

The division operator divides the RHS into the LHS to produce the numerical quotient. It is an error if either operand is not a number.

Example

5 / 2 => 2.5

% (Modulo)#

The modulo operator divides the RHS into the LHS using whole number division to produce a whole number quotient and a remainder. This operator returns the remainder. It is an error if either operand is not a number.

Example

5 % 2 => 1

.. (Range)#

The sequence generation operator is used to create an array of monotonically increasing integer start with the number on the LHS and ending with the number on the RHS. It is an error if either operand does not evaluate to an integer. The sequence generator can only be used within an array constructor [].

Examples

  • [1..5] => [1, 2, 3, 4, 5]
  • [1..3, 7..9] => [1, 2, 3, 7, 8, 9]
  • [1..$count(Items)].("Item " & $) => ["Item 1","Item 2","Item 3"]
  • [1..5].($*$) => [1, 4, 9, 16, 25]

Comparison Operators#

= (Equals)#

The equality operator returns Boolean true if both operands are the same (type and value). Arrays and objects are checked for deep equality. Arrays must have the same values in the same order. Objects must have the same key/value pairs (order is not relevant). Otherwise it returns false.

Examples

  • 1+1 = 2 => true
  • "Hello" = "World" => false

!= (Not equals)#

The inequality operator returns Boolean false if both operands are the same (type and value, deep equality). Otherwise it returns true.

Examples

  • 1+1 != 3 => true
  • "Hello" != "World" => true

> (Greater than)#

The 'greater than' operator returns Boolean true if the LHS is numerically greater than the RHS. Otherwise it returns false.

Examples

  • 22 / 7 > 3 => true
  • 5 > 5 => false

< (Less than)#

The 'less than' operator returns Boolean true if the LHS is numerically less than the RHS. Otherwise it returns false.

Examples

  • 22 / 7 < 3 => false
  • 5 < 5 => false

>= (Greater than or equals)#

The 'greater than or equals' operator returns Boolean true if the LHS is numerically greater than or equal to the RHS. Otherwise it returns false.

Examples

  • 22 / 7 >= 3 => true
  • 5 >= 5 => true

<= (Less than or equals)#

The 'less than or equals' operator returns Boolean true if the LHS is numerically less than or equal to the RHS. Otherwise it returns false.

Examples

  • 22 / 7 <= 3 => false
  • 5 <= 5 => true

in (Inclusion)#

The array (sequence) inclusion operator returns Boolean true if the value of the LHS is included in the array of values on the RHS. Otherwise it returns false. If the RHS is a single value, then it is treated as a singleton array.

Examples

  • "world" in ["hello", "world"] => true
  • "hello" in "hello" => true
  • library.books["Aho" in authors].title

Boolean Operators#

and (Boolean AND)#

The 'and' operator returns Boolean true if both operands evaluate to true. If either or both operands is not a Boolean type, then they are first cast to a Boolean using the rules of the $boolean function.

Example

library.books["Aho" in authors and price < 50].title

or (Boolean OR)#

The 'or' operator returns Boolean true if either operand evaluates to true. If either or both operands is not a Boolean type, then they are first cast to a Boolean using the rules of the $boolean function.

Example

library.books[price < 10 or section="diy"].title

Please note that Boolean 'NOT' is a function, not an operator.

Other Operators#

& (Concatenation)#

The string concatenation operator is used to join the string values of the operands into a single resultant string. If either or both of the operands are not strings, then they are first cast to string using the rules of the $string function.

Example

"Hello" & "World" => "HelloWorld"

? : (Conditional)#

The conditional ternary operator is used to evaluate one of two alternative expressions based on the result of a predicate (test) condition. The operator takes the form:

<test_expr> ? <expr_T> : <expr_F>

The <test_expr> expression is first evaluated. If it evaluates to Boolean true, then the operator returns the result of evaluating the <expr_T> expression. Otherwise it returns the result of evaluating the <expr_F> expression. If <test_expr> evaluates to a non-Boolean value, then the value is first cast to Boolean using the rules of the $boolean function.

Example

Price < 50 ? "Cheap" : "Expensive"

:= (Variable binding)#

The variable binding operator is used to bind the value of the RHS to the variable name defined on the LHS. The variable binding is scoped to the current block and any nested blocks. It is an error if the LHS is not a $ followed by a valid variable name.

Examples

  • $five := 5
  • $square := function($n) { $n * $n }

~> (Chain)#

The function chaining operator is used in the situations where multiple nested functions need to be applied to a value, while making it easy to read. The value on the LHS is evaluated, then passed into the function on the RHS as its first argument. If the function has any other arguments, then these are passed to the function in parenthesis as usual. It is an error if the RHS is not a function, or an expression that evaluates to a function.

Examples

$uppercase($substringBefore($substringAfter(Customer.Email, "@"), "."))

and

$sum(Account.Order.Product.(Price * Quantity))

can be more clearly written:

Customer.Email ~> $substringAfter("@") ~> $substringBefore(".") ~> $uppercase()

and

Account.Order.Product.(Price * Quantity) ~> $sum()

This operator can also be used in a more abstract form to define new functions based on a combination of existing functions. In this form, there is no value passed in on the LHS of the first function in the chain.

For example, the expression

(
$uppertrim := $trim ~> $uppercase;
$uppertrim(" Hello World ")
)

=> "HELLO WORLD"

creates a new function $uppertrim that performs $trim followed by $uppercase.

... ~> | ... | ... | (Transform)#

The object transform operator is used to modify a copy of an object structure using a pattern/action syntax to target specific modifications while keeping the rest of the structure unchanged.

The syntax has the following structure:

head ~> | location | update [, delete] |

where

  • head evaluates to the object that is to be copied and transformed
  • location evaluates to the part(s) within the copied object that are to be updated. The location expression is evaluated relative to the result of head. The result of evaluating location must be an object or array of objects.
  • update evaluates to an object that is merged into the object matched by each location. update is evaluated relative to the result of location and if location matched multiple objects, then the update gets evaluated for each one of these. The result of (each) update is merged into the result of location.
  • delete (optional) evaluates to a string or an array of strings. Each string is the name of the name/value pair in each object matched by location that is to be removed from the resultant object.

The ~> operator is the operator for function chaining and passes the value on the left hand side to the function on the right hand side as its first argument. The expression on the right hand side must evaluate to a function, hence the |...|...| syntax generates a function with one argument.

Example:

| Account.Order.Product | {'Price': Price * 1.2} |

defines a transform that will return a deep copy the object passed to it, but with the Product object modified such that its Price property has had its value increased by 20%. The first part of the expression is the path location that specifies all of the objects within the overall object to change, and the second part defines an object that will get merged into the object(s) matched by the first part. The merging semantics is the same as that of the $merge() function.

This transform definition syntax creates a JSONata function which you can either assign to a variable and use multiple times, or invoke inline. Example:

payload ~> |Account.Order.Product|{'Price': Price * 1.2}|

or:

$increasePrice := |Account.Order.Product|{'Price': Price * 1.2}|

This also has the benefit that multiple transforms can be chained together for more complex transformations.

In common with $merge(), multiple changes (inserts or updates) can be made to an object. Example:

|Account.Order.Product|{'Price': Price * 1.2, 'Total': Price * Quantity}|

Note that the Total will be calculated using the original price, not the modified one (JSONata is declarative not imperative).

Properties can also be removed from objects. This is done using the optional delete clause which specifies the name(s) of the properties to delete. Example:

$ ~> |Account.Order.Product|{'Total': Price * Quantity}, ['Price', 'Quantity']|

This copies the input, but for each Product it inserts a Total and removes the Price and Quantity properties.

String functions#

$string()#

Signature: $string(arg, prettify)

Casts the arg parameter to a string using the following casting rules

  • Strings are unchanged
  • Functions are converted to an empty string
  • Numeric infinity and NaN throw an error because they cannot be represented as a JSON number
  • All other values are converted to a JSON string using the JSON.stringify function

If arg is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of arg.

If prettify is true, then "prettified" JSON is produced. i.e One line per field and lines will be indented based on the field depth.

Examples

  • $string(5) => "5"
  • [1..5].$string() => ["1", "2", "3", "4", "5"]

$length()#

Signature: $length(str)

Returns the number of characters in the string str. If str is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of str. An error is thrown if str is not a string.

Examples

  • $length("Hello World") => 11

$substring()#

Signature: $substring(str, start[, length])

Returns a string containing the characters in the first parameter str starting at position start (zero-offset). If str is not specified (i.e. this function is invoked with only the numeric argument(s)), then the context value is used as the value of str. An error is thrown if str is not a string.

If length is specified, then the substring will contain maximum length characters.

If start is negative then it indicates the number of characters from the end of str. See substr for full definition.

Examples

  • $substring("Hello World", 3) => "lo World"
  • $substring("Hello World", 3, 5) => "lo Wo"
  • $substring("Hello World", -4) => "orld"
  • $substring("Hello World", -4, 2) => "or"

$substringBefore()#

Signature: $substringBefore(str, chars)

Returns the substring before the first occurrence of the character sequence chars in str. If str is not specified (i.e. this function is invoked with only one argument), then the context value is used as the value of str. If str does not contain chars, then it returns str. An error is thrown if str and chars are not strings.

Examples

  • $substringBefore("Hello World", " ") => "Hello"

$substringAfter()#

Signature: $substringAfter(str, chars)

Returns the substring after the first occurrence of the character sequence chars in str. If str is not specified (i.e. this function is invoked with only one argument), then the context value is used as the value of str. If str does not contain chars, then it returns str. An error is thrown if str and chars are not strings.

Examples

  • $substringAfter("Hello World", " ") => "World"

$uppercase()#

Signature: $uppercase(str)

Returns a string with all the characters of str converted to uppercase. If str is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of str. An error is thrown if str is not a string.

Examples

  • $uppercase("Hello World") => "HELLO WORLD"

$lowercase()#

Signature: $lowercase(str)

Returns a string with all the characters of str converted to lowercase. If str is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of str. An error is thrown if str is not a string.

Examples

  • $lowercase("Hello World") => "hello world"

$trim()#

Signature: $trim(str)

Normalizes and trims all whitespace characters in str by applying the following steps:

  • All tabs, carriage returns, and line feeds are replaced with spaces.
  • Contiguous sequences of spaces are reduced to a single space.
  • Trailing and leading spaces are removed.

If str is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of str. An error is thrown if str is not a string.

Examples

  • $trim(" Hello \n World ") => "Hello World"

$pad()#

Signature: $pad(str, width [, char])

Returns a copy of the string str with extra padding, if necessary, so that its total number of characters is at least the absolute value of the width parameter. If width is a positive number, then the string is padded to the right; if negative, it is padded to the left. The optional char argument specifies the padding character(s) to use. If not specified, it defaults to the space character.

Examples

  • $pad("foo", 5) => "foo "
  • $pad("foo", -5) => " foo"
  • $pad("foo", -5, "#") => "##foo"
  • $formatBase(35, 2) ~> $pad(-8, '0') => "00100011"

$contains()#

Signature: $contains(str, pattern)

Returns true if str is matched by pattern, otherwise it returns false. If str is not specified (i.e. this function is invoked with one argument), then the context value is used as the value of str.

The pattern parameter can either be a string or a regular expression (regex). If it is a string, the function returns true if the characters within pattern are contained contiguously within str. If it is a regex, the function will return true if the regex matches the contents of str.

Examples

  • $contains("abracadabra", "bra") => true
  • $contains("abracadabra", /a.*a/) => true
  • $contains("abracadabra", /ar.*a/) => false
  • $contains("Hello World", /wo/) => false
  • $contains("Hello World", /wo/i) => true
  • Phone[$contains(number, /^077/)] => { "type": "mobile", "number": "077 7700 1234" }

$split()#

Signature: $split(str, separator [, limit])

Splits the str parameter into an array of substrings. If str is not specified, then the context value is used as the value of str. It is an error if str is not a string.

The separator parameter can either be a string or a regular expression (regex). If it is a string, it specifies the characters within str about which it should be split. If it is the empty string, str will be split into an array of single characters. If it is a regex, it splits the string around any sequence of characters that match the regex.

The optional limit parameter is a number that specifies the maximum number of substrings to include in the resultant array. Any additional substrings are discarded. If limit is not specified, then str is fully split with no limit to the size of the resultant array. It is an error if limit is not a non-negative number.

Examples

  • $split("so many words", " ") => [ "so", "many", "words" ]
  • $split("so many words", " ", 2) => [ "so", "many" ]
  • $split("too much, punctuation. hard; to read", /[ ,.;]+/) => ["too", "much", "punctuation", "hard", "to", "read"]

$join()#

Signature: $join(array[, separator])

Joins an array of component strings into a single concatenated string with each component string separated by the optional separator parameter.

It is an error if the input array contains an item which isn't a string.

If separator is not specified, then it is assumed to be the empty string, i.e. no separator between the component strings. It is an error if separator is not a string.

Examples

  • $join(['a','b','c']) => "abc"
  • $split("too much, punctuation. hard; to read", /[ ,.;]+/, 3) ~> $join(', ') => "too, much, punctuation"

$match()#

Signature: $match(str, pattern [, limit])

Applies the str string to the pattern regular expression and returns an array of objects, with each object containing information about each occurrence of a match withing str.

The object contains the following fields:

  • match - the substring that was matched by the regex.
  • index - the offset (starting at zero) within str of this match.
  • groups - if the regex contains capturing groups (parentheses), this contains an array of strings representing each captured group.

If str is not specified, then the context value is used as the value of str. It is an error if str is not a string.

Examples

$match("ababbabbcc",/a(b+)/) =>

[
{
"match": "ab",
"index": 0,
"groups": ["b"]
},
{
"match": "abb",
"index": 2,
"groups": ["bb"]
},
{
"match": "abb",
"index": 5,
"groups": ["bb" ]
}
]

$replace()#

Signature: $replace(str, pattern, replacement [, limit])

Finds occurrences of pattern within str and replaces them with replacement.

If str is not specified, then the context value is used as the value of str. It is an error if str is not a string.

The pattern parameter can either be a string or a regular expression (regex). If it is a string, it specifies the substring(s) within str which should be replaced. If it is a regex, its is used to find .

The replacement parameter can either be a string or a function. If it is a string, it specifies the sequence of characters that replace the substring(s) that are matched by pattern. If pattern is a regex, then the replacement string can refer to the characters that were matched by the regex as well as any of the captured groups using a $ followed by a number N:

  • If N = 0, then it is replaced by substring matched by the regex as a whole.
  • If N > 0, then it is replaced by the substring captured by the Nth parenthesised group in the regex.
  • If N is greater than the number of captured groups, then it is replaced by the empty string.
  • A literal $ character must be written as $$ in the replacement string

If the replacement parameter is a function, then it is invoked for each match occurrence of the pattern regex. The replacement function must take a single parameter which will be the object structure of a regex match as described in the $match function; and must return a string.

The optional limit parameter, is a number that specifies the maximum number of replacements to make before stopping. The remainder of the input beyond this limit will be copied to the output unchanged.

Examples

Query

$replace("John Smith and John Jones", "John", "Mr")

Result

"Mr Smith and Mr Jones"

Query

$replace("John Smith and John Jones", "John", "Mr", 1)

Result

"Mr Smith and John Jones"

Query

$replace("abracadabra", /a.*?a/, "*")

Result

"*c*bra"

Query

$replace("John Smith", /(\w+)\s(\w+)/, "$2, $1")

Result

"Smith, John"

Query

$replace("265USD", /([0-9]+)USD/, "$$$1")

Result

"$265"

Query

(
$convert := function($m) {
($number($m.groups[0]) - 32) * 5/9 & "C"
};
$replace("temperature = 68F today", /(\d+)F/, $convert)
)

Result

"temperature = 20C today"

$eval()#

Signature: $eval(expr [, context])

Parses and evaluates the string expr which contains literal JSON or a JSONata expression using the current context as the context for evaluation.

Examples

Query

$eval("[1,2,3]")

Result

[1, 2, 3]

Query

$eval('[1,$string(2),3]')

Result

[1,"2",3]

Optionally override the context by specifying the second parameter

$base64encode()#

Signature: $base64encode()

Converts an ASCII string to a base 64 representation. Each each character in the string is treated as a byte of binary data. This requires that all characters in the string are in the 0x00 to 0xFF range, which includes all characters in URI encoded strings. Unicode characters outside of that range are not supported.

Examples

  • $base64encode("myuser:mypass") => "bXl1c2VyOm15cGFzcw=="

$base64decode()#

Signature: $base64decode()

Converts base 64 encoded bytes to a string, using a UTF-8 Unicode codepage.

Examples

  • $base64decode("bXl1c2VyOm15cGFzcw==") => "myuser:mypass"

$encodeUrlComponent()#

Signature: $encodeUrlComponent(str)

Encodes a Uniform Resource Locator (URL) component by replacing each instance of certain characters by one, two, three, or four escape sequences representing the UTF-8 encoding of the character.

Examples

  • $encodeUrlComponent("?x=test") => "%3Fx%3Dtest"

$encodeUrl()#

Signature: $encodeUrl(str)

Encodes a Uniform Resource Locator (URL) by replacing each instance of certain characters by one, two, three, or four escape sequences representing the UTF-8 encoding of the character.

Examples

  • $encodeUrl("https://mozilla.org/?x=褕械谢谢褘") => "https://mozilla.org/?x=%D1%88%D0%B5%D0%BB%D0%BB%D1%8B"

$decodeUrlComponent()#

Signature: $decodeUrlComponent(str)

Decodes a Uniform Resource Locator (URL) component previously created by encodeUrlComponent.

Examples

  • $decodeUrlComponent("%3Fx%3Dtest") => "?x=test"

$decodeUrl()#

Signature: $decodeUrl(str)

Decodes a Uniform Resource Locator (URL) previously created by encodeUrl.

Examples

  • $decodeUrl("https://mozilla.org/?x=%D1%88%D0%B5%D0%BB%D0%BB%D1%8B") => "https://mozilla.org/?x=褕械谢谢褘"

Numeric functions#

$number()#

Signature: $number(arg)

Casts the arg parameter to a number using the following casting rules

  • Numbers are unchanged
  • Strings that contain a sequence of characters that represent a legal JSON number are converted to that number
  • Boolean true casts to 1, Boolean false casts to 0
  • All other values cause an error to be thrown.

If arg is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of arg.

Examples

  • $number("5") => 5
  • ["1", "2", "3", "4", "5"].$number() => [1, 2, 3, 4, 5]

$abs()#

Signature: $abs(number)

Returns the absolute value of the number parameter, i.e. if the number is negative, it returns the positive value.

If number is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of number.

Examples

  • $abs(5) => 5
  • $abs(-5) => -5

$floor()#

Signature: $floor(number)

Returns the value of number rounded down to the nearest integer that is smaller or equal to number.

If number is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of number.

Examples

  • $floor(5) => 5
  • $floor(5.3) => 5
  • $floor(5.8) => 5
  • $floor(-5.3) => -6

$ceil()#

Signature: $ceil(number)

Returns the value of number rounded up to the nearest integer that is greater than or equal to number.

If number is not specified (i.e. this function is invoked with no arguments), then the context value is used as the value of number.

Examples

  • $ceil(5) => 5
  • $ceil(5.3) => 6
  • $ceil(5.8) => 6
  • $ceil(-5.3) => -5

$round()#

Signature: $round(number [, precision])

Returns the value of the number parameter rounded to the number of decimal places specified by the optional precision parameter.

The precision parameter (which must be an integer) species the number of decimal places to be present in the rounded number. If precision is not specified then it defaults to the value 0 and the number is rounded to the nearest integer. If precision is negative, then its value specifies which column to round to on the left side of the decimal place

This function uses the Round half to even strategy to decide which way to round numbers that fall exactly between two candidates at the specified precision. This strategy is commonly used in financial calculations and is the default rounding mode in IEEE 754.

Examples

  • $round(123.456) => 123
  • $round(123.456, 2) => 123.46
  • $round(123.456, -1) => 120
  • $round(123.456, -2) => 100
  • $round(11.5) => 12
  • $round(12.5) => 12
  • $round(125, -1) => 120

$power()#

Signature: $power(base, exponent)

Returns the value of base raised to the power of exponent (baseexponent).

If base is not specified (i.e. this function is invoked with one argument), then the context value is used as the value of base.

An error is thrown if the values of base and exponent lead to a value that cannot be represented as a JSON number (e.g. Infinity, complex numbers).

Examples

  • $power(2, 8) => 256
  • $power(2, 0.5) => 1.414213562373
  • $power(2, -2) => 0.25

$sqrt()#

Signature: $sqrt(number)

Returns the square root of the value of the number parameter.

If number is not specified (i.e. this function is invoked with one argument), then the context value is used as the value of number.

An error is thrown if the value of number is negative.

Examples

  • $sqrt(4) => 2
  • $sqrt(2) => 1.414213562373

$random()#

Signature: $random()

Returns a pseudo random number greater than or equal to zero and less than one (0 鈮 n < 1)

Examples

  • $random() => 0.7973541067127
  • $random() => 0.4029142127028
  • $random() => 0.6558078550072

$formatNumber()#

Signature: $formatNumber(number, picture [, options])

Casts the number to a string and formats it to a decimal representation as specified by the picture string.

The behaviour of this function is consistent with the XPath/XQuery function fn:format-number as defined in the XPath F&O 3.1 specification. The picture string parameter defines how the number is formatted and has the same syntax as fn:format-number.

The optional third argument options is used to override the default locale specific formatting characters such as the decimal separator. If supplied, this argument must be an object containing name/value pairs specified in the decimal format section of the XPath F&O 3.1 specification.

Examples

  • $formatNumber(12345.6, '#,###.00') => "12,345.60"
  • $formatNumber(1234.5678, "00.000e0") => "12.346e2"
  • $formatNumber(34.555, "#0.00;(#0.00)") => "34.56"
  • $formatNumber(-34.555, "#0.00;(#0.00)") => "(34.56)"
  • $formatNumber(0.14, "01%") => "14%"
  • $formatNumber(0.14, "###pm", {"per-mille": "pm"}) => "140pm"
  • $formatNumber(1234.5678, "鈶犫憼.鈶犫憼鈶爀鈶", {"zero-digit": "\u245f"}) => "鈶犫憽.鈶⑩懀鈶鈶"

$formatBase()#

Signature: $formatBase(number [, radix])

Casts the number to a string and formats it to an integer represented in the number base specified by the radix argument. If radix is not specified, then it defaults to base 10. radix can be between 2 and 36, otherwise an error is thrown.

Examples

  • $formatBase(100, 2) => "1100100"
  • $formatBase(2555, 16) => "9fb"

$formatInteger()#

Signature: $formatInteger(number, picture)

Casts the number to a string and formats it to an integer representation as specified by the picture string.

The behaviour of this function is consistent with the two-argument version of the XPath/XQuery function fn:format-integer as defined in the XPath F&O 3.1 specification. The picture string parameter defines how the number is formatted and has the same syntax as fn:format-integer.

Examples

  • $formatInteger(2789, 'w') => "two thousand, seven hundred and eighty-nine"
  • $formatInteger(1999, 'I') => "MCMXCIX"

$parseInteger()#

Signature: $parseInteger(string, picture)

Parses the contents of the string parameter to an integer (as a JSON number) using the format specified by the picture string. The picture string parameter has the same format as $formatInteger. Although the XPath specification does not have an equivalent function for parsing integers, this capability has been added to JSONata.

Examples

  • $parseInteger("twelve thousand, four hundred and seventy-six", 'w') => 12476
  • $parseInteger('12,345,678', '#,##0') => 12345678

Numeric aggregation functions#

$sum()#

Signature: $sum(array)

Returns the arithmetic sum of an array of numbers. It is an error if the input array contains an item which isn't a number.

Example

  • $sum([5,1,3,7,4]) => 20

$max()#

Signature: $max(array)

Returns the maximum number in an array of numbers. It is an error if the input array contains an item which isn't a number.

Example

  • $max([5,1,3,7,4]) => 7

$min()#

Signature: $min(array)

Returns the minimum number in an array of numbers. It is an error if the input array contains an item which isn't a number.

Example

  • $min([5,1,3,7,4]) => 1

$average()#

Signature: $average(array)

Returns the mean value of an array of numbers. It is an error if the input array contains an item which isn't a number.

Example

  • $average([5,1,3,7,4]) => 4

Boolean functions#

$boolean()#

Signature: $boolean(arg)

Casts the argument to a Boolean using the following rules:

Argument typeResult
Booleanunchanged
string: emptyfalse
string: non-emptytrue
number: 0false
number: non-zerotrue
nullfalse
array: emptyfalse
array: contains a member that casts to truetrue
array: all members cast to falsefalse
object: emptyfalse
object: non-emptytrue
functionfalse

$not()#

Signature: $not(arg)

Returns Boolean NOT on the argument. arg is first cast to a boolean

$exists()#

Signature: $exists(arg)

Returns Boolean true if the arg expression evaluates to a value, or false if the expression does not match anything (e.g. a path to a non-existent field reference).

Array Functions#

$count()#

Signature: $count(array)

Returns the number of items in the array parameter. If the array parameter is not an array, but rather a value of another JSON type, then the parameter is treated as a singleton array containing that value, and this function returns 1.

If array is not specified, then the context value is used as the value of array.

Examples

  • $count([1,2,3,1]) => 4
  • $count("hello") => 1

$append()#

Signature: $append(array1, array2)

Returns an array containing the values in array1 followed by the values in array2. If either parameter is not an array, then it is treated as a singleton array containing that value.

Examples

  • $append([1,2,3], [4,5,6]) => [1,2,3,4,5,6]
  • $append([1,2,3], 4) => [1,2,3,4]
  • $append("Hello", "World") => ["Hello", "World"]

$sort()#

Signature: $sort(array [, function])

Returns an array containing all the values in the array parameter, but sorted into order. If no function parameter is supplied, then the array parameter must contain only numbers or only strings, and they will be sorted in order of increasing number, or increasing unicode codepoint respectively.

If a comparator function is supplied, then is must be a function that takes two parameters:

function(left, right)

This function gets invoked by the sorting algorithm to compare two values left and right. If the value of left should be placed after the value of right in the desired sort order, then the function must return Boolean true to indicate a swap. Otherwise it must return false.

Example

$sort(Account.Order.Product, function($l, $r) {
$l.Description.Weight > $r.Description.Weight
})

This sorts the products in order of increasing weight.

The sorting algorithm is stable which means that values within the original array which are the same according to the comparator function will remain in the original order in the sorted array.

$reverse()#

Signature: $reverse(array)

Returns an array containing all the values from the array parameter, but in reverse order.

Examples

  • $reverse(["Hello", "World"]) => ["World", "Hello"]
  • [1..5] ~> $reverse() => [5, 4, 3, 2, 1]

$shuffle()#

Signature: $shuffle(array)

Returns an array containing all the values from the array parameter, but shuffled into random order.

Examples

  • $shuffle([1..9]) => [6, 8, 2, 3, 9, 5, 1, 4, 7]

$distinct()#

Signature $distinct(array)

Returns an array containing all the values from the array parameter, but with any duplicates removed. Values are tested for deep equality as if by using the equality operator.

Examples

  • $distinct([1,2,3,3,4,3,5]) => [1, 2, 3, 4, 5]
  • $distinct(Account.Order.Product.Description.Colour) => [ "Purple", "Orange", "Black" ]

$zip()#

Signature: $zip(array1, ...)

Returns a convolved (zipped) array containing grouped arrays of values from the array1 ... arrayN arguments from index 0, 1, 2, etc.

This function accepts a variable number of arguments. The length of the returned array is equal to the length of the shortest array in the arguments.

Examples

  • $zip([1,2,3], [4,5,6]) => [[1,4] ,[2,5], [3,6]]
  • $zip([1,2,3],[4,5],[7,8,9]) => [[1,4,7], [2,5,8]]

Object functions#

$keys()#

Signature: $keys(object)

Returns an array containing the keys in the object. If the argument is an array of objects, then the array returned contains a de-duplicated list of all the keys in all of the objects.

$lookup()#

Signature: $lookup(object, key)

Returns the value associated with key in object. If the first argument is an array of objects, then all of the objects in the array are searched, and the values associated with all occurrences of key are returned.

$spread()#

Signature: $spread(object)

Splits an object containing key/value pairs into an array of objects, each of which has a single key/value pair from the input object. If the parameter is an array of objects, then the resultant array contains an object for every key/value pair in every object in the supplied array.

$merge()#

Signature: $merge(array<object>)

Merges an array of objects into a single object containing all the key/value pairs from each of the objects in the input array. If any of the input objects contain the same key, then the returned object will contain the value of the last one in the array. It is an error if the input array contains an item that is not an object.

$sift()#

Signature: $sift(object, function)

See definition under 'Higher-order functions'

$each()#

Signature: $each(object, function)

Returns an array containing the values return by the function when applied to each key/value pair in the object.

The function parameter will get invoked with two arguments:

function(value, name)

where the value parameter is the value of each name/value pair in the object and name is its name. The name parameter is optional.

Examples

$each(Address, function($v, $k) {$k & ": " & $v})

=>

[
"Street: Hursley Park",
"City: Winchester",
"Postcode: SO21 2JN"
]

$error()#

Signature:$error(message)

Deliberately throws an error with an optional message

$assert()#

Signature:$assert(condition, message)

If condition is true, the function returns undefined. If the condition is false, an exception is thrown with the message as the message of the exception.

$type()#

Signature:$type(value)

Evaluates the type of value and returns one of the following strings:

  • "null"
  • "number"
  • "string"
  • "boolean"
  • "array"
  • "object"
  • "function" Returns (non-string) undefined when value is undefined.

Date/Time functions#

$now()#

Signature: $now([picture [, timezone]])

Generates a UTC timestamp in ISO 8601 compatible format and returns it as a string. All invocations of $now() within an evaluation of an expression will all return the same timestamp value.

If the optional picture and timezone parameters are supplied, then the current timestamp is formatted as described by the $fromMillis() function.

Examples

  • $now() => "2017-05-15T15:12:59.152Z"

$millis()#

Signature: $millis()

Returns the number of milliseconds since the Unix Epoch (1 January, 1970 UTC) as a number. All invocations of $millis() within an evaluation of an expression will all return the same value.

Examples

  • $millis() => 1502700297574

$fromMillis()#

Signature: $fromMillis(number [, picture [, timezone]])

Convert the number representing milliseconds since the Unix Epoch (1 January, 1970 UTC) to a formatted string representation of the timestamp as specified by the picture string.

If the optional picture parameter is omitted, then the timestamp is formatted in the ISO 8601 format.

If the optional picture string is supplied, then the timestamp is formatted occording to the representation specified in that string. The behaviour of this function is consistent with the two-argument version of the XPath/XQuery function fn:format-dateTime as defined in the XPath F&O 3.1 specification. The picture string parameter defines how the timestamp is formatted and has the same syntax as fn:format-dateTime.

If the optional timezone string is supplied, then the formatted timestamp will be in that timezone. The timezone string should be in the format "卤HHMM", where 卤 is either the plus or minus sign and HHMM is the offset in hours and minutes from UTC. Positive offset for timezones east of UTC, negative offset for timezones west of UTC.

Examples

  • $fromMillis(1510067557121) => "2017-11-07T15:12:37.121Z"
  • $fromMillis(1510067557121, '[M01]/[D01]/[Y0001] [h#1]:[m01][P]') => "11/07/2017 3:12pm"
  • $fromMillis(1510067557121, '[H01]:[m01]:[s01] [z]', '-0500') => "10:12:37 GMT-05:00"

$toMillis()#

Signature: $toMillis(timestamp [, picture])

Convert a timestamp string to the number of milliseconds since the Unix Epoch (1 January, 1970 UTC) as a number.

If the optional picture string is not specified, then the format of the timestamp is assumed to be ISO 8601. An error is thrown if the string is not in the correct format.

If the picture string is specified, then the format is assumed to be described by this picture string using the same syntax as the XPath/XQuery function fn:format-dateTime, defined in the XPath F&O 3.1 specification.

Examples

  • $toMillis("2017-11-07T15:07:54.972Z") => 1510067274972

Higher order functions#

$map()#

Signature: $map(array, function)

Returns an array containing the results of applying the function parameter to each value in the array parameter.

The function that is supplied as the second parameter must have the following signature:

function(value [, index [, array]])

Each value in the input array is passed in as the first parameter in the supplied function. The index (position) of that value in the input array is passed in as the second parameter, if specified. The whole input array is passed in as the third parameter, if specified.

Examples

  • $map([1..5], $string) => ["1", "2", "3", "4", "5"]

With user-defined (lambda) function:

$map(Email.address, function($v, $i, $a) {
'Item ' & ($i+1) & ' of ' & $count($a) & ': ' & $v
})

evaluates to:

[
"Item 1 of 4: fred.smith@my-work.com",
"Item 2 of 4: fsmith@my-work.com",
"Item 3 of 4: freddy@my-social.com",
"Item 4 of 4: frederic.smith@very-serious.com"
]

$filter()#

Signature: $filter(array, function)

Returns an array containing only the values in the array parameter that satisfy the function predicate (i.e. function returns Boolean true when passed the value).

The function that is supplied as the second parameter must have the following signature:

function(value [, index [, array]])

Each value in the input array is passed in as the first parameter in the supplied function. The index (position) of that value in the input array is passed in as the second parameter, if specified. The whole input array is passed in as the third parameter, if specified.

Example The following expression returns all the products whose price is higher than average:

$filter(Account.Order.Product, function($v, $i, $a) {
$v.Price > $average($a.Price)
})

$single()#

Signature: $single(array, function)

Returns the one and only one value in the array parameter that satisfy the function predicate (i.e. function returns Boolean true when passed the value). Throws an exception if the number of matching values is not exactly one.

The function that is supplied as the second parameter must have the following signature:

function(value [, index [, array]])

Each value in the input array is passed in as the first parameter in the supplied function. The index (position) of that value in the input array is passed in as the second parameter, if specified. The whole input array is passed in as the third parameter, if specified.

Example The following expression the product in the order whose SKU is "0406654608":

$single(Account.Order.Product, function($v, $i, $a) {
$v.SKU = "0406654608"
})

$reduce()#

Signature: $reduce(array, function [, init])

Returns an aggregated value derived from applying the function parameter successively to each value in array in combination with the result of the previous application of the function.

The function must accept at least two arguments, and behaves like an infix operator between each value within the array. The signature of this supplied function must be of the form:

myfunc($accumulator, $value[, $index[, $array]])

Example

(
$product := function($i, $j){$i * $j};
$reduce([1..5], $product)
)

This multiplies all the values together in the array [1..5] to return 120.

If the optional init parameter is supplied, then that value is used as the initial value in the aggregation (fold) process. If not supplied, the initial value is the first value in the array parameter.

$sift()#

Signature: $sift(object, function)

Returns an object that contains only the key/value pairs from the object parameter that satisfy the predicate function passed in as the second parameter.

If object is not specified, then the context value is used as the value of object. It is an error if object is not an object.

The function that is supplied as the second parameter must have the following signature:

function(value [, key [, object]])

Each value in the input object is passed in as the first parameter in the supplied function. The key (property name) of that value in the input object is passed in as the second parameter, if specified. The whole input object is passed in as the third parameter, if specified.

Example

Account.Order.Product.$sift(function($v, $k) {$k ~> /^Product/})

This sifts each of the Product objects such that they only contain the fields whose keys start with the string "Product" (using a regex). This example returns:

[
{
"Product Name": "Bowler Hat",
"ProductID": 858383
},
{
"Product Name": "Trilby hat",
"ProductID": 858236
},
{
"Product Name": "Bowler Hat",
"ProductID": 858383
},
{
"ProductID": 345664,
"Product Name": "Cloak"
}
]
Last updated on by Aditya Vaidyam