#106 ResultDB UpdaterV2 Mapping added for component-version.test.complete.yaml
Closed 3 years ago by draval. Opened 3 years ago by draval.
fedora-ci/ draval/messages master  into  master

file modified
+35 -3
@@ -1,4 +1,3 @@ 

- 

  CI Messages

  ===========

  
@@ -38,6 +37,8 @@ 

      make convert

      ./scripts/validate.py schemas/brew-build.test.complete.json message.json

  

+ **Note**: Release tag 1.x.x onward all messages will be validated against scheme before updating the ResultsDB,

+   and invalid messges will not be processed.

  

  Examples

  --------
@@ -113,9 +114,7 @@ 

  All data should be stored in the body of the message. Additional

  fields can be added to the messages as required by the senders,

  although it is advised to document them here.

- 

  Syntax requirements for field names:

- 

  * Use only characters `[a-z0-9_]`

  * No spaces are allowed in the name of the field

  * For separation of multi-word strings in field names use the
@@ -128,6 +127,38 @@ 

    e.g. recipients: ["mvadkert", "optak"]. In case of an empty field,

    this needs to be and empty list: []

  

+ ResultsDB-Updater V2 Mappings

+ ------------------------------

+ All the messages with version:1.x.x onward will be Processed by ResultsDB-Updater V2 logic.

+ Mappings are stored in `/mappings` directory and its tightly coupled with schemas.

+ These mappings are used to transform the messages into ResultsDB format.

+ 

+ Syntax requirements for `/mappings` directory:

+ 

+  * Only .yaml files supported for `/mappings` directory.

+  * File name should match with `/schemas` file name.

+  * Mapping file format/syntex should be driven from ResultsDB [Attributes][resultsDB]

+ 

+ Headers & Body Supported for `/mappings` files:

+ 

+ Headers will contain pre-processed metadata.

+ i.e. UMB = /topic/VirtualTopic.eng.ci.<ciNamespace>.<ciArtifact>.<ciEvent>.<ciStatus>

+ 

+  * headers.ciNamespace

+  * headers.ciArtifact  

+  * headers.ciEvent

+  * headers.ciStatus

+  * headers.ciFullEvent : value as ciEvent.ciStatus

+  * headers.ciComputedStatus : Computes status value based on ciStatus

+  * headers.uuid : uuid generated for individual message

+  * Body will be root elment to access all message content.

+    e.g. body.test.result will resolve to test.result value.

+  * Inorder to evalute the header and body expressions it shold be enclosed with "${}".

+     e.g.  "${body.test.result}"

+  * status vs computed-status mapping as follows:

+  

+  error >> ERROR , queued >> QUEUED , running >> RUNNING

+  pass >> PASSED , failed >> FAILED , failure >> FAILED

  

  Licensing

  ---------
@@ -148,3 +179,4 @@ 

  [js]: http://json-schema.org/

  [fedmsg]: https://fedmsg2.readthedocs.io/en/latest/topics.html

  [CC-BY]: http://creativecommons.org/licenses/by/4.0/

+ [resultsDB]: https://resultsdb20.docs.apiary.io/#reference/0/result-groups/create-new-result

@@ -0,0 +1,23 @@ 

+ # Resultsdb Updater V2 Mapping, Supported for release Tag 1.x.x onwards.

+ 

+ outcome: "${body.test.result}"

+ testcase:

+     name: "${body.test.namespace}-${body.test.type}-${body.test.category}"

+     ref_url: "${body.run.url}"

+ groups:

+     - uuid: "${headers.uuid}"

+       url: "${body.run.url}"

+ ref_url: "${body.artifact.source}"

+ data:

+     item:

+         - "${body.artifact.component}-${body.artifact.version}"

+     log:

+         - "${body.run.log}"

+     type:

+         - "${body.artifact.type}"

+     component:

+         - "${body.artifact.component}"

+     version:

+         - "${body.artifact.version}"

+     category:

+         - "${body.test.category}"

As a part of ResultDB Updater V2 initiative, we have added Required mapping in the "component-version.test.complete.yaml".

Nice! Can the mapping be in a separate directory?

could somebody provide more context on what is this about pls?

@lholecek
I guess we can create a new schema directory for V2 lets say SchemaV2 which will include new mapping + existing schema in single file only.

Please let me know if that make sense.

@mvadkert
This Documnet should help you to get more insights about ResultDB Updater V2.

Doc: https://docs.google.com/document/d/1X3tUBKQlYPVbRaAMUiRXcSxZzzsNFQtGaL21Po1EsnM/edit

@draval thanks :) seems I already saw that document, cool :)

seems we will need some more documentation somewhere, seems body is not defined anywhere in the standard? Same goes for header.

seems we will need some more documentation somewhere, seems body is not defined anywhere in the standard? Same goes for header.

@mvadkert
yup it will be moved to confluence as proper documentation and we should be able to address format and stuff there.
Its already in our to-do list.

I guess we can create a new schema directory for V2 lets say SchemaV2 which will include new mapping + existing schema in single file only.

I meant it would be better to keep the mapping and schema separate. E.g. store the mapping in resultsdb-mappings/component-version.test.complete.yaml. I'm not a fan of long yaml files and it would be easier to see if a message type has a mapping.

seems we will need some more documentation somewhere, seems body is not defined anywhere in the standard? Same goes for header.

I suggest adding a quick description in README file in this repo.

Metadata Update from @lholecek:
- Request assigned

3 years ago

1 new commit added

  • ResultDB UpdaterV2 Mapping Directory added along with component-version.test.complete.yaml
3 years ago

:+1: to the suggestion to add a brief note to the README about the mappings and what they are for.

The mapping itself looks good to me.

@ralph
Yup we will create separate PR to update README.
If no other changes required can we please proceed with merging the PR and it my require new Tag: 1.x.x version.

As resultDB UpdaterV2 will only use version = 1.x.x release tag.
cc: @taziz

Oh - let's add the README to this PR so the doc and the initial mapping are a part of the same change, please.

4 new commits added

  • Update README.md
  • Update README.md
  • Update README.md
  • Added ResultsDB updater V2 mapping & header information in README, Modified the Mappings .yml file as per new changes
3 years ago

Pull-Request has been closed by draval

3 years ago