How do I troubleshoot DMT / UDM validate issues during a job load
Remedy IT Service Management Suite
Remedy ITSM Foundation
Remedy IT Service Management Suite
Remedy ITSM Foundation
Remedy IT Service Management Suite
Remedy ITSM Foundation
This knowledge article may contain information that does not apply to version 21.05 or later which runs in a container environment. Please refer to Article Number 000385088 for more information about troubleshooting BMC products in containers.
This article is also available in Spanish version000168589here
The following guide will be separated in sections outlined below
When you create and run a dataload job for the first time, the system uses the data from a job template to create new records on the target form or forms.
Data updates
The Data Management supports data updates only for CTM:LoadPeople form.
If you try to update the data on the forms for which data update is not supported:
For certain forms, the system overwrites the data and replaces the earlier field values with the values provided in the update. If you want to update data on such forms, you must provide values for all the form fields irrespective of whether you want to update those fields.
For certain forms listed in table, the data is only created and not updated. These forms are marked by a Create Only flag, which is displayed in the Steps panel of a job record when the Validate and Promote steps are created for a job.
For a list of what forms support create only and not updates see the following documentation
Validate Process
When you decide to run a dataload job, the data load is automatically validated. The validation of data occurs before the data is promoted.
Each Validate step represents the validation of data for one staging form.
Validating foundation fields for transactional records against staging data will only be possible when both are contained within the same job and are for the same company. If separate jobs are created for foundation and related transactional data, then the foundation data must be promoted first, and validations must occur against the target ITSM forms.
All validate step error details will be available in the following forms
The load form when you navigate via the 'View Data' button. Double Click on a validate step and it will show you the staging form Used. If you get an error on this validate, open the staging form and you can view the errors found in Error_code and Error_message
Examples:
CTM:LoadPeopleOrganization
RKM:LoadKnowledgeArticleManager
CTM:LoadPeople
COM:LoadCompany
AST:LoadAttributes
CAI:Events form
The Error Management console - Errors found on staging forms are flagged. For details about fixing errors, see Managing data errors.
ARerror.log - This log file lists errors in running validation and promotion.
**You can also find the load forms by looking at the AROutput step of the transaction job with Pentaho/Spoon. If you open the Output step you will see the Form Name that the job uses to load data.
When in Pentaho / Spoon click the folder icon with the Arrow. Locate the job "Transformation" and open it up for the job you are running
For example AST Computer System loads to AST:LoadAttributes
Example:
In the below example I tried to load a How To Knowledge article and filled out the Site incorrectly to trigger a fail. Since the site is not a site in the environment it did not finish correctly. How would someone troubleshoot this failure?
First I double click or select 'View' on the validate error and identify the form
Copy the staging form name and navigate to the form
Enter the job ID in the Job ID field and look for Error Flag marked as yes in the column Error Flag. Notice the error message invalid site. Knowing this I would go back and investigate my spreadsheet for incorrect site names
CAI:Events form shows the server target to be a different server name
Modification of the ar.conf/ar.cfg is necessary
Change Server-Name: ServerB to Server-Name: ServerA ( Where Data Load is trying to Validate/Promote)
Verify that the CAI:Application Registry Form has a record with Registry Name = BMC Remedy Data Management.
On Connection tab, set Server to ServerA ( Where Data Load is trying to Validate/Promote) if incorrect
Delete all DLD records from the CAI:Events form.
If there are TMS related records and if the task group templates are not being used in Change, delete them as well. Otherwise, backup these records and then remove them.If there are other records like SRM, back them up and remove them.
The CAI:Events form should be clean
Wait Before Execution” on the Validate step does not get removed, and it stays In Progress on the validate step.
If this is a CI load, the reconciliation job has to be complete. Please verify any Recon job times to make sure they are not set at a certain time to run. Typically this means it is waiting on reconciliation
Validate (and Promote) step remain in a 'Queued' status while overall UDM Job completes
Make sure that the "Max Entries Returned By GetList" setting on the AR Server you are connecting to when starting the job is equal to, or higher than the count returned by the following query:
SELECT COUNT(*) FROM DMT_SYS_SequencingEngine WHERE Parent_Job_GUID = 'DO NOT REMOVE'
When starting a Job run, Active Link "DMT:DJB:RUNJOB_100`!" triggers a Push to the 'DMT:SYS:SequencingEngine' form to build the required sequencing records. The "Max Entries Returned By GetList" setting will determine the maximum number of records that will be targeted by such a Push field action (therefore if it is set too low, this can cause this build process to fail)