Data transformation is a crucial component of an integration. In the Azure cloud, you have several ways to transform a JSON message into another one to match the target format. Here, I will provide you with a very powerful alternative I have been using for a while, the Workmaze JUST.Net library, available under the MIT license, created by Neeraj Salpekar.
This .NET library can robustly map, transform, shape, and process one JSON file into another using a powerful transformer language. It has support for direct mapping, conditions, loops, math-statistical functions, conversions, custom transformation methods, XML-CSV conversions, and many more. I know there is Azure Data Factory these days, and I know the Liquid Templates functionality in Logic Apps, but these all have their limitations, cost concerns, readability and trackability problems, and at the time of writing, nearly all are in their ‘preview’ versions, sometimes not running stably. So, using JUST.net is still a preferred choice, especially for low-medium scale integrations.
Example usage
I have created a sample Azure function and a web page using JUST.net transformations in my GitHub repository here: https://github.com/sertanyaman/json-tojson-just-transformations-demo. To be able to do the transformation using JUST.net, you need two JSON documents. One is your source JSON (input), and the other is a JUST-formatted transformer file. After downloading my repository, run the test website I included:
For the source use this JSON:
{
"menu": {
"popup": {
"menuitem": [{
"value": "Open",
"onclick": "OpenDoc()"
}, {
"value": "Close",
"onclick": "CloseDoc()"
}
],
"submenuitem": "CloseSession()"
}
}
}
For the transformer use this one:
{
"result": {
"Open": "#valueof($.menu.popup.menuitem[?(@.value=='Open')].onclick)",
"Close": "#valueof($.menu.popup.menuitem[?(@.value=='Close')].onclick)"
}
}
Click the Transform button, and select pretty print from below, you will see the result as:
The syntax of the transformer file is easy to get used to. #valueof() returns you the value of a path in the source JSON. $ is the root node of your source JSON, $.menu.popup.menuitem pointing to the array within the menuitem node. Then with ‘?’ operator, you query the array elements using @ to point to the current element. Then you give the onclick node as a parameter for the #valueOf function, which will return the string value of that node.
It is quite convenient that you use another JSON document as a transformer; you can keep these transformer documents in your version control system, publish them to a shared space in your CI/CD pipelines, like an Azure storage, and keep track of every change per your application versions.
To use it in your own .NET projects, just include the correct library from the NuGet packages. There are 3 ones now, 2 being the old ones. The one I highlighted is the good one, and the others I crossed out:
To do the transformation, you just run the following line. Remember to give the transformer parameter as a string (at the time of writing, it is):
..................
result = new JsonTransformer().Transform(transformatorString, inputJson);
..................
Advanced transformations
JUST.net can perform many other complex format transformations, like looping arrays and shaping them into another type of document. Here is an example of an array transformation:
{
"tree": {
"branch": {
"leaf": "green",
"flower": "red",
"bird": "crow",
"extra": {
"twig": "birdnest"
}
},
"ladder": {
"wood": "treehouse"
}
},
"numbers": [1, 2, 3, 4],
"arrayobjects": [{
"country": {
"name": "norway",
"language": "norsk"
}
}, {
"country": {
"name": "UK",
"language": "english"
}
}, {
"country": {
"name": "Sweden",
"language": "swedish"
}
}
]
}
And the transformer:
{
"iteration": {
"#loop($.numbers)": {
"CurrentValue": "#currentvalue()",
"CurrentIndex": "#currentindex()",
"IsLast": "#ifcondition(#currentindex(),#lastindex(),yes,no)",
"LastValue": "#lastvalue()"
}
},
"iteration2": {
"#loop($.arrayobjects)": {
"CurrentValue": "#currentvalueatpath($.country.name)",
"CurrentIndex": "#currentindex()",
"IsLast": "#ifcondition(#currentindex(),#lastindex(),yes,no)",
"LastValue": "#lastvalueatpath($.country.language)"
}
}
,
"number_index": {
"#loop($.spell_numbers)": {
"#eval(#currentindex())": "#currentvalueatpath(#concat($.,#currentproperty()))"
}
},
"othervalue": "othervalue"
}
And it can create a flattened and shaped JSON result from this complex array-based source, as shown below:
There are many more complex transformations JUST.net can do, just check their wiki page here to get an idea and decide if it suits you: https://github.com/WorkMaze/JUST.net/blob/master/README.md
Using it in Azure
I have included an Azure function in my repository that receives source and transformer JSON in its body and returns the result. You can publish it to Azure either in an app plan or in a container as a microservice, and call it from your other apps, containers, or microservices to perform JSON transformations with it. It is also possible to call it from Logic Apps instead of the default Liquid Templates feature if you hit one of its limitations, or are not happy with the readability and preview version issues. Run the Azure Function I have included in my project:
Using Postman or another tool, enter the HTTP body as given below, including your source and transformer JSON in “input” and “transformator” parameters, and send:
{
"input": {
"menu": {
"popup": {
"menuitem": [
{
"value": "Open",
"onclick": "OpenDoc()"
},
{
"value": "Close",
"onclick": "CloseDoc()"
}
],
"submenuitem": "CloseSession()"
}
}
},
"transformator": {
"result": {
"Open": "#valueof($.menu.popup.menuitem[?(@.value=='Open')].onclick)",
"Close": "#valueof($.menu.popup.menuitem[?(@.value=='Close')].onclick)"
}
}
}
That’s it. I hope by introducing you to the JUST library, I have given you a very powerful alternative to Azure standard transformation tools, and you can use it in your own integrations with the samples I have provided. Enjoy, and if you have any questions, you can comment below.
