YAML Builder

This topic describes the YAML Builder tool for Migration Vault.

The YAML Builder creates the mapping files (YAMLs) used for study migrations. With this tool, users can auto-generate a baseline set of mapping files for each Form in their EDC Study Design. The YAML Builder is based on the latest YAML version (for example, 21.2.0). Casebooks and derived data are not supported.

Generating Files

To generate files with the YAML Builder:

1. From the Projects tab, hover next to the project name to show the Actions () menu.
2. From the Actions menu, click YAML Builder.
3. Select the Target Vault. This is the EDC vault where the study design is retrieved.
4. Enter the Study Name. This is the Name and the Study Environment for which you want to create mapping files (for example, “Cholecap_PROD” or “Deetoza_DEV1”).
5. Select the Data Source (In-Form, Legacy, or Rave).
6. Enter the Casebook Version.
7. Click Execute.

A progression banner displays at the top of the application during YAML file generation. After the generation is complete, you’ll receive in-app and email notifications. If the generation is successful, files are added to the Attachments section of the project’s Details page.

Job Failure

If the generation job is unsuccessful, information regarding the failure is in Admin > Operations > Job Status > History. To find the most recent job under the History section, locate the YAML Builder title and sort by Started Time. You can also download the job’s log file by hovering next to the Job ID, and selecting Actions () menu > Download Log.

Modifying YAML Files

You can manually modify and re-upload YAML files when needed. To make changes, download the YAML file in the Mapping Configurations section of the project’s Details page. Make the appropriate changes and upload the file to the same Mapping Configurations section.

For instructions on uploading, read Creating Mapping Configurations.

Understanding the Mapping Process

All YAMLS are created based on a specific legacy system. The YAML Builder is designed to work with various systems, such as Rave™ and InForm™. Each system contains its own mapping properties and use cases.

External ID

An object’s external ID (from the Study Design) is used to map a Veeva object to a legacy object. external IDs default to the veevaDef name and do not allow for item-type suffixes. This applies to Events, Event Groups, Forms, Item Groups, and Items.

Excluded Objects

Labels and headers are not generally included in the source data and are excluded when creating YAMLs. Objects with any of the following properties are not included or mapped by the YAML Builder:

• Item Type = Header; Read Only; Status
• Data Type = Label

Header files

Whenever the YAML Builder is executed, two default header files are created. Header files are based on Event Type. If Event Type equals “Log”, the filename “logheader.yaml” is used. If Event Type does not equal “Log”, the filename “header.yaml” is used.

Transformation Files

As part of the YAML generation process, Migration Vault creates default Transformation YAML files (transformations.yaml) to use in conjunction with generated mapping files. These files contain system defaults for multiple legacy platforms, such as Rave and InForm.

The YAML Builder automatically applies the following default transformers during the YAML generation process:

• DateFormatter
• RemovePrefixT
• UnknownDate
• YesNoBoolean
• MedicalCoding
• DateTimeFormatter
• LabsDateTimeFormatter
• VisitDateFormatter
• DoNotMigrateUnknownTime
• TextItemCharacterSplit1500
• TextItemCharacterSplit500
• formStatusTransformer
• VisitMethod

If a transformer is not needed, it won’t be used during the migration process and will have no impact on the study.

Read Transformation YAML Files for code examples.

Selectors

Default transformers contain selectors that are used to streamline migration tasks:

• formStatus: Sets the status of migrated forms to Submitted. This does not apply to repaired forms.
• visitMethod: Applies global transformations to all Events with Visit Methods, converting source values into a Veeva-compatible format.
• Visit Date: Converts Event Dates & Item types into the standard Veeva format.

Mapping & Study Design Structure

Migration Vault identifies the structure of all study Forms in your EDC Study Design. This includes Items, Item Groups, Events, Event Groups, and Log Events, and forms containing lab data.

Item Groups: Repeating & Non-Repeating

For Forms with repeating Item Groups, the YAML Builder generates a single mapping file for each repeating Item Group definition retrieved from the target study in EDC. For example, if a Form has seven repeating Item Groups, the YAML Builder creates seven Item Group YAML files.

The single mapping file includes a source file for each Item Group, and a default naming structure of “form name_name of item group”, based on the Form’s definition name in Studio.

For Forms without repeating Item Groups, the YAML Builder generates a single mapping file for each form definition retrieved from the target study in EDC. Any non-repeating Item Groups appearing on the same Form are generated in a separate YAML file.

Log Events

Log Events are events that are not tied to a specific site visit and can span across multiple Events. The YAML Builder generates mapping files for Forms that exist under Log Events (both repeating and non-repeating).

Progressive Display

Migration Vault provides mapping configurations for Forms with controlling Items & Item Groups for progressive display.

After identifying controlling Items, the YAML Builder creates mapping configurations containing the following:

• columnName of the Item that progressively displays
• veevaDef of the Item that progressively displays
• columnName of the controlling Item
• columnValue that “triggers” the progressive display
  • Note: If multiple values trigger a display, they’re listed with commas. For example, columnValue: [“N”, “NA”]

The mapping object’s columnName/columnValue is based on the external ID assigned to the object (as present in the Study Design).

After identifying controlling Item Groups, the YAML Builder creates mapping configurations containing the following:

• veevadef for the Item Group that progressively displays
• Mappings for controlling Items including:
  • the columnName of the controlling Item
  • the columnValue that “triggers” the progressive display of the Item Group
• Mappings for the Item Groups items

Events & Event Groups

The YAML Builder creates mappings for Events and Event Groups. The mappings can be found in the eventGroups section of the Forms YAML file. They consist of the following:

• Events section header
• Forms section header
• Event Group definition (and if applicable, the corresponding columnValue for the source data’s Event Group)
• The source data’s columnValue for the Event, and its corresponding Event definition

Single & Multiple Event Groups

The YAML Builder also creates mappings for Event Groups with single and multiple Events. The number of Events dictates the default columnValue. If an Event Group has one Event, the YAML Builder inserts the Event’s veevaDef as the Event Group’s columnValue. For example, “ev_LOG”. If an Event Group has more than one Event, the YAML Builder inserts all of the Events’ veevaDef values as the Event Group’s columnValue (separated by a comma). For example, “ev_LOG1”, “ev_LOG2”. If the Event Group does not contain any Events, the system does not specify a columnValue.

Generating Event & Event Group YAMLs

For InForm™, Rave™, and EDC study migrations, the YAML Builder provides a baseline set of YAMLs for Events and Event Groups. These files can easily be used for any Load by adjusting the YAML’s file mapping.

The YAML Builder generates two default YAML files for Event data:

• events.yaml: For all scheduled EDC Events
• events_unscheduled.yaml: For all unscheduled Events

The placeholder filename, placeholder.csv, is used for InForm™ and Rave™ source data files. Users must replace this with the actual filename after generation. EDC source data files receive the SYS_EVT.csv filename.

Default header file and target mappings are inserted into each file:

Default header file mappings

• header.yaml: For all scheduled EDC Events
• logheader.yaml: For all unscheduled Events

Target mappings

• CASEBOOK
• EVENT_GROUP
• EVENT

For each Event Group, the YAML Builder inserts the external ID of the Event: columnValue. It also inserts Event mappings for each Event within an Event Group and assigns default columnNames for Events. Default columnNames and format mappings are not required for Event Dates.

Did Not Occur Events

The YAML Builder generates the dno_event.yaml template for importing Did Not Occur (DNO) Events. This template is applicable to the following source data types:

• InForm™(Legacy)
• InForm™(Standard)
• EDC

The placeholder filename, placeholder.csv, is used for InForm™ (legacy and standard) and Rave™ source data files. Users must replace this with the actual filename after generation. EDC source data files receive the SYS_EVT.csv filename.

Default header file, target, and placeholder mappings are inserted into each file.

Default header file mapping:

• header.yaml: Default

Target mappings:

• CASEBOOK
• EVENT_GROUP
• DID_NOT_OCCUR_EVENT

Placeholder mappings

The YAML Builder inserts placeholder mappings for events.events and eventGroups.eventGroups. Each receives placeholder values for their columnValue and veevaDef attributes.

For InForm™ and Rave™ source data files, the YAML Builder inserts placeholder mappings for the Did Not Occur columnName and a default NULL value for columnValue. For EDC source data files, the YAML Builder inserts a default STATUS value for columnName and a default Did Not Occur value for columnValue.

For Rave™ files, a placeholder mapping is also added for the Did Not Occur Change Reason.

Med Coding

For studies with Med Coding enabled, the YAML Builder includes mappings whenever possible. If the YAML Builder cannot determine the appropriate mapping value, a placeholder is used instead. Dictionary releases are used to determine which placeholder to insert:

• The Drug Code placeholder is used for the WHODrug dictionary
• The LLT Code placeholder is used for the MedDRA dictionary
• The ATC Code placeholder is used when the Study has Code with ATCs enabled.

Each Form can only have one dictionary release version, and all three codes (Drug, LLT, ATC) will never be used for the same item.

Lab Data

Migration Vault automates the YAML mapping process for lab data originating in Rave™.

The YAML Builder generates YAML files for study designs containing lab Forms, and auto-names each file. A labheader.yaml file is assigned to each file and contains default column settings, sequence number options, and targets.

Lab forms are identified when an Item Group contains lab_header__v or lab__v.

Event Groups, Events, & Forms

The YAML file includes a nested hierarchy of Event Groups, Events, and Forms. The external ID is used to determine the columnValue mappings for each.

The mappings are as follows:

• Event Group: Its veevaDef and corresponding columnValue in the source data.
• Event: Its veevaDef and corresponding columnValue in the source data.
• Form: The form and lab header name, and mapping for its Items.

The following are example mappings for Event Groups, Events, and Forms:

eventGroups:
- map:
    columnValue: "Unscheduled"
    veevaDef: "Unscheduled"
  events:
  - map:
      columnValue: "Unscheduled"
      veevaDef: "Unscheduled"
    forms:
    - include: "lb_hema_01_001"
components:
- name: "lb_hema_01_001"
  forms:
  - map:
      columnValue: "lb_hema_01_001"
      veevaDef: "lb_hema_01_001"

In the Form YAML file, if the LBHEADER variable and lab Collection DateTime are present, the LabsDateTimeFormatter transformer is applied to the Collection DateTime.

For example:

items:
- map:
    columnName: "SaveTS"
    veevaDef: "LBDTC"
  transformers:
		- { name: LabsDateTimeFormatter }

The YAML Builder creates a Clinical Significance transformer to ensure lab data is compatible with EDC. This transforms source values into a value that the system can validate and process. The source values are source-specific and typically include Clinically Significant and Not Clinically Significant values.

The following is a Clinical Significance transformer YAML file example:

 - dataType: TEXT
        name: LabClinicalSignificance
        default: false
		transformations:
          - { type: SEARCH_AND_REPLACE, pattern: "^1$", replace: "Not Clinically Significant" }
          - { type: SEARCH_AND_REPLACE, pattern: "^3$", replace: "Clinically Significant" }

Lab Panels

Forms containing lab header data also include a lab panel. In EDC, there can only be one lab panel per Form.

The YAML Builder maps the following variables for lab panels:

• veevaDef: The lab panel’s veevaDef.
• columnValue: The lab panel name in the source data, based on the external ID of the lab panel’s veevaDef.

    - map:
	    columnValue: "Hematology"
		veevaDef: "Hematology"

A lab panel can have up to 35 analytes, and up to 5 or 6 fields per analyte. The YAML Builder adds mappings for the analyte name and its lab result.

The following variables are mapped for each analyte:

• columnValue: The analyte name, based on the external ID of the lab result’s veevaDef.
• veevadef: The lab result field’s veevaDef. This is a system-set field, identified by the LBORRES prefix.
• columnName: The columnName associated with the plugin (i.e., AnalyteValue) that contains the lab result.