BI-2779 - Observation levels data migration

[nickpalladino]

Description

Story: BI-2779

• Migrate observation level data from additionalinfo to new dynamic observation levels schema
• Remove observation level data from additionalinfo and unused global levels

NOTE: We should probably wipe qa-test before deploying this.

Dependencies

• develop bi-api/bi-web

Testing

See JIRA card for instructions on replicating prod data locally.

Spot check different cases

Prior global level

Some of the existing data had observation unit levels pointing to a global level. One example is plot in the program with program_id 6dd2c9c9-d1ec-4004-a627-ccf65655c502.

GET http://localhost:8083/brapi/v2/observationunits?programDbId=6dd2c9c9-d1ec-4004-a627-ccf65655c502

Will see this prior to migration (levelNameDbId will be system specific):

"observationLevel": {
    "levelNameDbId": "5707a749-43aa-419a-9dbe-03c493b1ab80",
    "levelName": "plot",
    "levelOrder": 6,
    "levelCode": "68 [SWPCLT-5]"
},

After migration (levelNameDbId will be system specific):

"observationLevel": {
    "levelNameDbId": "4eb8eeb4-a1b5-4133-93f5-cbc1a115d6c7",
    "levelName": "plot",
    "levelOrder": 0,
    "programDbId": "6dd2c9c9-d1ec-4004-a627-ccf65655c502",
    "programName": "Sweetpotato-Charleston (SWPCLT)",
    "levelCode": "68 [SWPCLT-5]"
},

Prior null level

Some prior existing data had null observation unit levels. This was for a period of time where bi-api was submitting levels that were not supported by the brapi server like fish. One example is program with program_id f2ac1d4d-afd3-43c1-a617-0ed14f1129c4.

GET http://localhost:8083/brapi/v2/observationunits?programDbId=f2ac1d4d-afd3-43c1-a617-0ed14f1129c4

Will see this prior to migration:

"observationLevel": null,

After migration (levelNameDbId will be system specific):

"observationLevel": {
    "levelNameDbId": "949a82ed-bf6e-45f4-a22f-6d2ee76ffffc",
    "levelName": "fish",
    "levelOrder": 0,
    "programDbId": "f2ac1d4d-afd3-43c1-a617-0ed14f1129c4",
    "programName": "HOTFIX-BI-2053-TEST (ZYA)",
    "levelCode": "Omy_sentinel_YC23_00639 [ZYA-1]"
},

Prior different level

Some prior existing data wrote plot as the observation unit observation level but stored a different custom level like tree in the additionalinfo to get around brapi server errors. This was done for a period of time by bi-api. One example is program with program_id 6df60fd5-bb07-465c-b7cd-fd94bea4f615.

GET http://localhost:8083/brapi/v2/observationunits?programDbId=6df60fd5-bb07-465c-b7cd-fd94bea4f615

Will see this prior to migration (levelNameDbId will be system specific):

"observationLevel": {
    "levelNameDbId": "5707a749-43aa-419a-9dbe-03c493b1ab80",
    "levelName": "plot",
    "levelOrder": 6,
    "levelCode": "5_2 [PECSOM-1]"
},

After migration (levelNameDbId will be system specific):

"observationLevel": {
    "levelNameDbId": "f1946767-0df1-4fbc-a64d-1b7bb96eab6b",
    "levelName": "tree",
    "levelOrder": 0,
    "programDbId": "6df60fd5-bb07-465c-b7cd-fd94bea4f615",
    "programName": "Pecan-Somerville (PECSOM)",
    "levelCode": "5_2 [PECSOM-1]"
},

Checklist:

• I have performed a self-review of my own code
• I have tested my code and ensured it meets the acceptance criteria of the story
• I have commented my code, particularly in hard-to-understand areas
• I have either updated the source of truth or arranged for update with product owner if needed https://breedinginsight.atlassian.net/wiki/spaces/BI/pages/1559953409/Source+of+Truth