Many of the existing Atrium Jobs are not working Post Upgrade - 23.3.04 Pentaho 9

Issue 1: The jobs would be corrupted in case jobs are opened and modified in the spoon client Pentaho 6

Lookup step not processing any data

Solution: Open the existing job in Pentaho 9 make a dummy change and save the job

Run from the local spoon (The latest client is similar to the AR server version eg 23.3.Patch 4 the client is Pentaho 9 ) and check if the job works fine.

If successful, the job can be run from DMT and or from Atrium Integrator (CMDB UI)

Note: Always use spoon clients based on the AR server version.

Issue 2: After the Pentaho upgrade to the 9.x version SYNTAX of the JSON input is updated.

2025/02/07 13:00:46 - Json Input.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2024-11-21 10.31.25 by buildadm) : Error initializing step [Json Input]
2025/02/07 13:00:46 - Select values 3.0 - Released server socket on port 0
2025/02/07 13:00:46 - Json Input.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2024-11-21 10.31.25 by buildadm) : com.jayway.jsonpath.InvalidPathException: Character '.' on position 3 is not valid.

Solution:

For a JSON payload in Pentaho 6 mapping that works is shown in the screenshot.

{ "data": [ { "result2": "{\"@odata.context\":\"https://graph.microsoft.com/v1.0/$metadata#users(businessPhones,displayName,givenName,jobTitle,mail,mobilePhone,officeLocation,preferredLanguage,surname,userPrincipalName,id,department,employeeId,manager,manager())\",\"value\":[{\"businessPhones\":[],\"displayName\":\"Exchange Online-ApplicationAccount\",\"givenName\":null,\"jobTitle\":null,\"mail\":null,\"mobilePhone\":null,\"officeLocation\":null,\"preferredLanguage\":null,\"surname\":null,\"userPrincipalName\":\"user@domain.com\",\"id\":\"2e1f108c-b9d2-4954-95fb-4ec4c06ee131\",\"department\":null,\"employeeId\":null}]}" } ] }

2 JSON input steps are needed

JSON Input Step 1

JSON Input step 2 where resulttext is passed as the input

In Pentaho 9 below changes are needed.

In some cases this mapping still throws error

We can not find any data with path [$.value[*].JobTitle]! as the data in the JSON payload is null.

After REST Client Get URL step and Set Variable step include a step to handle Null value

https://docs.hitachivantara.com/r/en-us/pentaho-data-integration-and-analytics/9.3.x/mk-95pdia003/pdi-transformation-steps/json-input

Issue 3: The Add Checksum step does not work with the below error

2025/02/03 09:51:52 - Calc Checksum.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2025-01-01 10.41.04 by buildadm) : Error running stepCannot invoke "Object.toString()" because the return value of "org.pentaho.di.core.row.ValueMetaInterface.getNativeDataType(Object)" is null

Solution:

As a workaround add this step If the field value is null -> Set empty string

Defect CMDB-2155 Fixed in 25.1.01 version

Issue 4:  Token generation does not work via REST Client, POST step when content type is set as "form-data

while running POST call via RESTAPI to generate token

Azure AD IDIADA2" 2025/01/28 09:37:55 - Write ok 2.0 - org.pentaho.di.core.exception.KettleException:
2025/01/28 09:37:55 - Write ok 2.0 - Can not result from [https://login.microsoftonline.com/f1f4e2c6-4e87-4d61-b715-063974ea537f/oauth2/v2.0/token]
2025/01/28 09:37:55 - Write ok 2.0 - Error parsing media type 'form-data'

Solution:

Step 1. In Generate rows step set Header as application/x-www-form-urlencoded 

Step 2. Remove Content type field mapping 

Step 3. Pass this Header variable as a field under REST Client step under Header and remove content type. 

Issue 5: Insert/Update fails with below error

2025/02/03 09:51:52 - Sync People to SSP - Transformation detected one or more steps with errors.
2025/02/03 09:51:52 - Insert/Update SSP.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2025-01-01 10.41.04 by buildadm) : Unable to commit connection(s) :org.pentaho.di.core.exception.KettleDatabaseException:
2025/02/03 09:51:52 - Insert/Update SSP.0 - Error comitting connection
2025/02/03 09:51:52 - Sync People to SSP - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2025-01-01 10.41.04 by buildadm) : Errors detected!

Solution:

Workaround is not available 

Defect CMDB-1861 and Fixed in 23.3.04HF4

Issue 6:  DMT job load step shows ARERR 8760 Cannot establish a network connection to the AR System Plug-In server. platformplugin:9999

There is no solution identified yet

Issue 7: The job fails with below error 

Unable to run job Azure AtriumJob. The Save Data (Step) has an error. An error occurred reading a transformation from the repository2025/02/11 14:02:57 - Save Data (2) - Error while decoding trans attribute definition2025/02/11 14:02:57 - Save Data (2) - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2024-11-21 10.31.25 by buildadm) : org.pentaho.di.core.exception.KettleException:

Solution: 

Open the existing job in Pentaho 9 make a dummy change and save the job

Issue 8: BMC Helix CMDB- kitchen.sh: cannot execute the spoon jobs through command 

Please refer article 000435403 for workaround.

Issue 9

When JSON structure has fields with different count below error can be seen while extracting the fields using JSON input step

2025/08/15 12:52:09 - JSON input 3.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2025-03-20 12.35.16 by buildadm) : 
2025/08/15 12:52:09 - JSON input 3.0 - The data structure is not the same inside the resource! We found 8 values for json path [$.var[*].field[*].filed1], which is different that the number returned for path [$.var[*].public_ip_address] (4 values). We MUST have the same number of values for all paths.
2025/08/15 12:52:09 - JSON input 3.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2025-03-20 12.35.16 by buildadm) : org.pentaho.di.trans.steps.jsoninput.exception.JsonInputException: 

Solution:

For this multiple JSON step can be included to retrieve fields from similar count

Wherever error is there exclude those fields to another JSON input step

1. Get a JSON file and add a JSON input step in the transformation
2. Add the file from File tab

2. Go to Fields and click on Select fields with this automatically all paths will be selected.

3. Click on Preview and the fields mapping can be seen. In case of any errors this preview will not work

4. JSON file with multiple field paths

When Previewed

025/08/15 13:46:35 - /BMC SAMPLES/Transformation 1.ktr : Transformation 1 - Dispatching started for transformation [/BMC SAMPLES/Transformation 1.ktr : Transformation 1]
2025/08/15 13:46:35 - JSON input 3 2.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2025-03-20 12.35.16 by buildadm) : 
2025/08/15 13:46:35 - JSON input 3 2.0 - The data structure is not the same inside the resource! We found 4 values for json path [$.projects[*].tasks[*].task_id], which is different that the number returned for path [$.projects[*].owner.progress] (2 values). We MUST have the same number of values for all paths.

For this add 2 different JSON step for the fields that are erroring out

JSON Input Step 1

Select the fields other than the shown in error above eg: tasks and tasks id

Next JSON Input step select other fields

Key points

• The repeated field here is progress.

• Paths:

  1. $.projects[].owner.progress → progress at the owner level

  2. $.projects[].tasks[].progress → progress at the task level

• Different values for each occurrence to clearly distinguish them.

Issue 10

Atrium jobs are not working with the below error.

2025/07/17 15:41:30 - HTTP Post.0 - Response time (milliseconds): [240] for https://customer.com/oauth/token
2025/07/17 15:41:30 - HTTP Post.0 - The response code is 401
2025/07/17 15:41:30 - HTTP Post.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2025-05-13 06.24.53 by buildadm) : Because of an error, this step can't continue: 
2025/07/17 15:41:30 - HTTP Post.0 - Can not result from https://customer.com/oauth/token
2025/07/17 15:41:30 - HTTP Post.0 - 
2025/07/17 15:41:30 - HTTP Post.0 - You need to be authenticated to access the resource at https://customer.com/oauth/token.
2025/07/17 15:41:30 - HTTP Post.0 - ERROR (version 9.2.0.0-290, build 9.2.0.0-290 from 2025-05-13 06.24.53 by buildadm) : org.pentaho.di.core.exception.KettleException: 
2025/07/17 15:41:30 - HTTP Post.0 - Can not result from https://customer.com/oauth/token
2025/07/17 15:41:30 - HTTP Post.0 - 
2025/07/17 15:41:30 - HTTP Post.0 - You need to be authenticated to access the resource at https://customer.com/oauth/token.

Solution: 

Query Parameters in "HTTP Post" are no longer appended to the API URL but working fine when its explicitly appended on the Request URL field 

Pentaho's official v9 documentation does not explicitly state that enabling "HTTP Post" step will disable automatic appending of Query Parameters defined in the Fields tab. However, this behavior has been confirmed through empirical testing by users in the community

 https://docs.hitachivantara.com/r/en-us/pentaho-data-integration-and-analytics/9.5.x/mk-95pdia003/pdi-transformation-steps

https://pentaho-public.atlassian.net/browse/PDI-18911 defect filed with Pentaho 9

See also BEST FAQ on BMC CMDB Suite - Atrium Integrator