This guide will describe how we can use a range of different operators within our transformations to change the context of the data which we are transforming. The default context is within the payload data, but this can be changed instead to data created previously in the transformation.
Note that a sample payload has been included here at the bottom of the page for you to use in exploring the transformations included here.
Current Node – @
Throughout the guides and tutorials on this site, you may see references to the @ symbol. This is defined by JMESPath as the ‘current node’. The current node is the point in the payload that our transformation is currently referring to. To illustrate this, consider the sample data at the end of this guide. Here we can use the current node to assign the entire payload to a single data point:
{
"Data": @
}This results in:
{
"Data": {
"Globals": {
...
},
"SalesData": [
{
"Date": "4/1/21",
"Region": "Denmark",
"SalesRep": "S. Ehlers",
"Product": "Template",
"Units": "99",
"PricePerUnit": "52.99",
"TotalCost": "5246.01"
},
...
]
}
}Evaluating data using the current node
We can also use the current node operator to evaluate our data. In our sample data, SalesData is an array of 11 different objects, each of which includes a TotalCost value. The following simple transformation allows us to create a single object to contain both our minimum and maximum values. As SalesData has been defined in the first line as the context for the object, we can simply use the current node operator to refer to this.
{
min_max: SalesData.{
max: max_by(@, &TotalCost),
min: min_by(@, &TotalCost)
}
}The following more complex transformation shows an example of the current node being referred to a number of times to produce a result. In this case, the result is essentially the same as in the first example but includes the addition of an index key onto each object within SalesData to allow for a unique identifier to be added.
{
data: sort_by(SalesData[*] | map(&merge(@,{__index: __index}), @), &TotalCost).{
min: @[0],
max: @[-1]
}
}Note the use of the pipe expression on the first line. You can find an explanation of this here. The current node is used in four different instances:
- As part of the map function that allows us to apply a transformation to each object in the array.
- As part of the merge function which is the function being applied to each object and adds the index value.
- As part of the definition for max and min. In this case, sort the array by
TotalCostthen select the first and last objects.
The $ operator
Sometimes it is convenient to refer to already transformed data rather than repeating the same transformation. The $ operator allows us to be able to do this. In the below example, you can see the $ operator being used to subtract the previously transformed minimum value from the maximum value.
{
data: sort_by(SalesData[*] | map(&merge(@,{__index: __index}), @), &TotalCost).{
min: @[0],
max: @[-1]
},
totalCostSpread: subtract($.data."max".TotalCost, $.data."min".TotalCost)
}NOTE, the $ operator will only work on data points that have previously been defined in the transformation. For example, if totalCostSpread was defined before data, then this transformation would fail. Aside from this restriction, all other transformations can be applied to data as normal.
Parent
It is sometimes necessary to include data from multiple nodes that exist on the same or higher levels in the payload. There are a number of ways to achieve this depending on how the data is structured. One way is to use the parent operator. In the example below we extend the min_max object we created earlier to include the date from our Globals data point. We use parent here because the min_max object refers to the SalesData array which is on the same level in the data as Globals.
{
min_max: SalesData.{
max: max_by(@, &TotalCost),
min: min_by(@, &TotalCost),
reportDate: parent(@).Globals.Date
}
}The result here is:
{
"min_max": {
"max": {
"Date": "7/21/21",
"Region": "Ireland",
"SalesRep": "J. Hansen",
"Product": "Template",
"Units": "79",
"PricePerUnit": "86.99",
"TotalCost": "6872.21"
},
"min": {
"Date": "4/28/21",
"Region": "Germany",
"SalesRep": "E. McKenna",
"Product": "Template",
"Units": "63",
"PricePerUnit": "49.99",
"TotalCost": "3149.37"
},
"reportDate": "01/02/2020"
}
}NOTE that parent can be used to go back as far in the data structure as required by nesting it. For example parent(parent(parent(@))) will allow you to go back three levels within the payload. Similarly, parent(parent(parent($))) will allow you to go back three levels within already transformed data.
The pipe (“|”) operator
The pipe operator is JMESPath functionality that allows us to make a transformation, then use the result of that transformation as the basis of a new transformation. The basic concept is that whatever is the result of the left-hand side of the pipe operator becomes the data used for the transformation on the right-hand side of the pipe operator and can be referred to using the @ operator. More information on the pipe can be found on the JMESPath site here. In the example below, we are using the operator in the first line of our transformation to set the SalesData array as our current node (left side of the operator). This is then referred to in the map and merge functions (right side of the operator).
{
data: sort_by(SalesData[*]|map(&merge(@,{__index: __index}), @), &TotalCost).{
min: @[0],
max: @[-1]
}
}Sample Data
{
"Globals": {
"Date": "01/02/2020",
"Region": "Global"
},
"SalesData": [
{
"Date": "4/1/21",
"Region": "Denmark",
"SalesRep": "S. Ehlers",
"Product": "Template",
"Units": "99",
"PricePerUnit": "52.99",
"TotalCost": "5246.01"
},
{
"Date": "2/7/21",
"Region": "Germany",
"SalesRep": "M. Priesler",
"Product": "Documotor",
"Units": "66",
"PricePerUnit": "50.99",
"TotalCost": "3365.34"
},
{
"Date": "2/24/21",
"Region": "Germany",
"SalesRep": "E. McKenna",
"Product": "Template",
"Units": "51",
"PricePerUnit": "82.99",
"TotalCost": "4232.49"
},
{
"Date": "3/20/21",
"Region": "Germany",
"SalesRep": "M. Rafn",
"Product": "Addin",
"Units": "60",
"PricePerUnit": "74.99",
"TotalCost": "4499.4"
},
{
"Date": "5/5/21",
"Region": "Ireland",
"SalesRep": "B. Sigurdsson",
"Product": "Template",
"Units": "82",
"PricePerUnit": "68.99",
"TotalCost": "5657.18"
},
{
"Date": "6/27/21",
"Region": "Denmark",
"SalesRep": "S. Ehlers",
"Product": "Documotor",
"Units": "95",
"PricePerUnit": "45.99",
"TotalCost": "4369.05"
},
{
"Date": "1/19/21",
"Region": "Germany",
"SalesRep": "M. Mortensen",
"Product": "Template",
"Units": "64",
"PricePerUnit": "86.99",
"TotalCost": "5567.36"
},
{
"Date": "4/28/21",
"Region": "Germany",
"SalesRep": "E. McKenna",
"Product": "Template",
"Units": "63",
"PricePerUnit": "49.99",
"TotalCost": "3149.37"
},
{
"Date": "7/21/21",
"Region": "Ireland",
"SalesRep": "J. Hansen",
"Product": "Template",
"Units": "79",
"PricePerUnit": "86.99",
"TotalCost": "6872.21"
},
{
"Date": "2/8/21",
"Region": "Denmark",
"SalesRep": "S. Ehlers",
"Product": "Documotor",
"Units": "77",
"PricePerUnit": "62.99",
"TotalCost": "4850.23"
},
{
"Date": "1/17/21",
"Region": "Germany",
"SalesRep": "K. Vagsheyg",
"Product": "Template",
"Units": "55",
"PricePerUnit": "93.99",
"TotalCost": "5169.45"
}
]
}