PR#107: ResultDB UpdaterV2 Mapping Directory added along with component-version.test.complete.yaml & README - fedora-ci/messages

fedora-ci / messages

#107 ResultDB UpdaterV2 Mapping Directory added along with component-version.test.complete.yaml & README

Merged 3 years ago by mvadkert. Opened 3 years ago by draval.

fedora-ci/ draval/messages master into master

Moved resultsdb mapping files under /mappings/results Directory

DivyangRaval • 3 years ago

99dc926

Addressed Review Comments for README File

DivyangRaval • 3 years ago

6f3fea8

ResultDB UpdaterV2 Mapping Directory added along with component-version.test.complete.yaml & README

DivyangRaval • 3 years ago

7b72118

README.md

file modified

+35 -3

		`@@ -1,4 +1,3 @@`
		`-`
		`CI Messages`
		`===========`

		`@@ -38,6 +37,8 @@`
		`make convert`
		`./scripts/validate.py schemas/brew-build.test.complete.json message.json`

		`+ Note: Release tag 1.x.x onward all messages will be validated against scheme before updating the ResultsDB,`
		`+ and invalid messges will not be processed.`

		`Examples`
		`--------`
		`@@ -113,9 +114,7 @@`
		`All data should be stored in the body of the message. Additional`
		`fields can be added to the messages as required by the senders,`
		`although it is advised to document them here.`
		`-`
		`Syntax requirements for field names:`
		`-`
		* Use only characters `[a-z0-9_]`
		`* No spaces are allowed in the name of the field`
		`* For separation of multi-word strings in field names use the`
		`@@ -128,6 +127,38 @@`
		`e.g. recipients: ["mvadkert", "optak"]. In case of an empty field,`
		`this needs to be and empty list: []`

		`+ ResultsDB-Updater V2 Mappings`
		`+ ------------------------------`
		`+ All the messages with version:1.x.x onward will be processed by ResultsDB-Updater V2 logic.`
		+ Mappings are stored in the `/mappings/results` directory and are tightly coupled with the schemas.
		`+ These mappings are used to transform the messages into the ResultsDB format.`
		`+`
		+ Syntax requirements for `/mappings/results` directory:
		`+`
		+ * Only .yaml files supported for `/mappings/results` directory.
		+ * Filename should match with `/schemas` filename.
		`+ * Mapping file format/syntax should be driven from the ResultsDB [Attributes][resultsDB]`
		`+`
		+ Headers & Body supported for `/mappings/results/*` files:
		`+`
		`+ Headers will contain pre-processed metadata.`
		`+ i.e. UMB = /topic/VirtualTopic.eng.ci.<ciNamespace>.<ciArtifact>.<ciEvent>.<ciStatus>`
		`+`
		`+ * headers.ciNamespace`
		`+ * headers.ciArtifact`
		`+ * headers.ciEvent`
		`+ * headers.ciStatus`
		`+ * headers.ciFullEvent : value as ciEvent.ciStatus`
		`+ * headers.ciComputedStatus : Computed status value based on ciStatus`
		`+ * headers.uuid : uuid generated for individual message`
		`+ * Body will be the root element to access all the message content.`
		`+ e.g. body.test.result will resolve to test.result value.`
		`+ * In order to evaluate the header and body expressions it should be enclosed with "${}".`
		`+ e.g. "${body.test.result}"`
		`+ * status vs computed-status mapping as follows:`
		`+`
		`+ error >> ERROR , queued >> QUEUED , running >> RUNNING`
		`+ pass >> PASSED , failed >> FAILED , failure >> FAILED`

		`Licensing`
		`---------`
		`@@ -148,3 +179,4 @@`
		`[js]: http://json-schema.org/`
		`[fedmsg]: https://fedmsg2.readthedocs.io/en/latest/topics.html`
		`[CC-BY]: http://creativecommons.org/licenses/by/4.0/`
		`+ [resultsDB]: https://resultsdb20.docs.apiary.io/#reference/0/result-groups/create-new-result`

mappings/results/component-version.test.complete.yaml

file added

+23

		`@@ -0,0 +1,23 @@`
		`+ # Resultsdb Updater V2 Mapping, Supported for release Tag 1.x.x onwards.`
		`+`
		`+ outcome: "${body.test.result}"`
		`+ testcase:`
		`+ name: "${body.test.namespace}-${body.test.type}-${body.test.category}"`
		`+ ref_url: "${body.run.url}"`
		`+ groups:`
		`+ - uuid: "${headers.uuid}"`
		`+ url: "${body.run.url}"`
		`+ ref_url: "${body.artifact.source}"`
		`+ data:`
		`+ item:`
		`+ - "${body.artifact.component}-${body.artifact.version}"`
		`+ log:`
		`+ - "${body.run.log}"`
		`+ type:`
		`+ - "${body.artifact.type}"`
		`+ component:`
		`+ - "${body.artifact.component}"`
		`+ version:`
		`+ - "${body.artifact.version}"`
		`+ category:`
		`+ - "${body.test.category}"`

draval commented 3 years ago

As a part of ResultDB Updater V2 initiative,
-We have added new directory "/mappings" which will store all mapping file against "/schemas " file.
-Added Mapping file for " component-version.test.complete.yaml" in /mappings directory.
-Updated README file with ReasultsDB-Updater V2 mapping related info & headers.

draval commented 3 years ago

@lholecek @ralph
I have closed previous PR & created new as I was running in README syntax issues not sure why same markdown text didn't work here, but finally I managed to fix all issues.

mvadkert commented on line 11 of README.md 3 years ago

Maybe this should be a warning instead? :)

type: messges

mvadkert commented on line 32 of README.md 3 years ago

Mappings ... its ... seems wrong, Mappings .. are ?

mvadkert commented on line 33 of README.md 3 years ago

Some link to ResultsDB format would be nice

mvadkert commented on line 39 of README.md 3 years ago

typo: syntex

mvadkert commented on line 55 of README.md 3 years ago

typos: Inorder -> In order, shold -> should

Edited 3 years ago by mvadkert

draval commented 3 years ago

Maybe this should be a warning instead? :)

type: messges

@mvadkert It can not be warning as the new resultsDB Updater logic depends on message validation only as its generic.
however logs should be able to provide the details why validation failed , which can be further use to correct the schema/message.
@taziz feel free to add if you think otherwise.

draval commented 3 years ago

Some link to ResultsDB format would be nice

39 of README.md contains the link of attributes which can be use as format reference for the mapping.

please let me know if you think otherwise

1 new commit added

Addressed Review Comments for README File

3 years ago

draval commented 3 years ago

@mvadkert
Addressed all review comments in above commit.

cc: @ralph @taziz

ralph commented 3 years ago

One last request - (sorry we did not raise it sooner) - can you move the mappings into one more folder deep so that it is clear that the mappings are specifically for resultsdb. In theory, someone might want to store mappings for a different service here in addition to resultsdb.

Put them in mappings/results/*.

1 new commit added