PR#87: Add message for group test results - fedora-ci/messages

fedora-ci / messages

#87 Add message for group test results

Merged 4 years ago by mvadkert. Opened 4 years ago by lholecek.

fedora-ci/ lholecek/messages add-group-tests into master

Bump version to 0.2.5

Lukas Holecek • 4 years ago

cd3504b

Fix type for rpm-build-group

Lukas Holecek • 4 years ago

c8ea954

Add message for group test results

Lukas Holecek • 4 years ago

68c20bb

examples/brew-build-group.build.complete.json

file modified

+2 -2

		`@@ -8,7 +8,7 @@`
		`"email": "baseos-ci@somewhere.com"`
		`},`
		`"artifact": {`
		`- "type": "rpm-build-group",`
		`+ "type": "brew-build-group",`
		`"id": "selinux-update-345",`
		`"repository": "https://some.url/repo",`
		`"builds":`
		`@@ -31,5 +31,5 @@`
		`}]`
		`},`
		`"generated_at": "2018-05-10 08:58:31.222602",`
		`- "version": "0.2.2"`
		`+ "version": "0.2.5"`
		`}`

examples/brew-build-group.build.error.json

file modified

+2 -2

		`@@ -8,7 +8,7 @@`
		`"email": "baseos-ci@somewhere.com"`
		`},`
		`"artifact": {`
		`- "type": "rpm-build-group",`
		`+ "type": "brew-build-group",`
		`"id": "selinux-update-345",`
		`"repository": "https://some.url/repo",`
		`"builds":`
		`@@ -35,5 +35,5 @@`
		`"issue_url": "http://sentry.somewhere.com/baseos/production/issues/1149/"`
		`},`
		`"generated_at": "2018-05-10 08:58:31.222602",`
		`- "version": "0.2.2"`
		`+ "version": "0.2.5"`
		`}`

examples/brew-build-group.test.complete.json

file added

+66

		`@@ -0,0 +1,66 @@`
		`+ {`
		`+ "contact": {`
		`+ "name": "BaseOS CI",`
		`+ "team": "BaseOS",`
		`+ "url": "https://somewhere.com",`
		`+ "docs": "https://somewhere.com/user-documentation",`
		`+ "irc": "#baseosci",`
		`+ "email": "baseos-ci@somewhere.com"`
		`+ },`
		`+ "run": {`
		`+ "url": "https://somewhere.com/job/ci-openstack/4794",`
		`+ "log": "https://somewhere.com/job/ci-openstack/4794/console",`
		`+ "log_raw": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "log_stream": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "debug": "https://somewhere.com/job/ci-openstack/4794/artifacts/debug.txt",`
		`+ "rebuild": "https://somewhere.com/job/ci-openstack/4794/rebuild/parametrized"`
		`+ },`
		`+ "artifact": {`
		`+ "type": "brew-build-group",`
		`+ "id": "selinux-update-345",`
		`+ "repository": "https://some.url/repo",`
		`+ "builds":`
		`+ [{`
		`+ "type": "brew-build",`
		`+ "id": 14546276,`
		`+ "issuer": "plautrba",`
		`+ "component": "libselinux",`
		`+ "nvr": "libselinux-2.8-6.el7.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"`
		`+ }, {`
		`+ "type": "brew-build",`
		`+ "id": 14546277,`
		`+ "issuer": "plautrba",`
		`+ "component": "libsepol",`
		`+ "nvr": "libsepol-2.8-3.el7.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"`
		`+ }]`
		`+ },`
		`+ "pipeline": {`
		`+ "id": "ac11dcddf99a",`
		`+ "name": "ci-openstack"`
		`+ },`
		`+ "test": {`
		`+ "category": "functional",`
		`+ "label": [ "fast", "aarch64" ],`
		`+ "namespace": "baseos-qe.baseos-ci",`
		`+ "note": "Some notes.",`
		`+ "result": "failed",`
		`+ "type": "tier1",`
		`+ "xunit": "https://somewhere.com/job/ci-openstack/4794/artifacts/results.xml"`
		`+ },`
		`+ "system":`
		`+ [{`
		`+ "os": "Fedora-Cloud-Base-28",`
		`+ "provider": "openstack",`
		`+ "architecture": "x86_64",`
		`+ "label": "master"`
		`+ }],`
		`+ "notification": {`
		`+ "recipients": [ "ovasik", "mvadkert" ]`
		`+ },`
		`+ "generated_at": "2018-05-10 08:58:31.222602",`
		`+ "version": "0.2.5"`
		`+ }`

examples/brew-build-group.test.error.json

file added

+63

		`@@ -0,0 +1,63 @@`
		`+ {`
		`+ "contact": {`
		`+ "name": "BaseOS CI",`
		`+ "team": "BaseOS",`
		`+ "url": "https://somewhere.com",`
		`+ "docs": "https://somewhere.com/user-documentation",`
		`+ "irc": "#baseosci",`
		`+ "email": "baseos-ci@somewhere.com"`
		`+ },`
		`+ "run": {`
		`+ "url": "https://somewhere.com/job/ci-openstack/4794",`
		`+ "log": "https://somewhere.com/job/ci-openstack/4794/console",`
		`+ "log_raw": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "log_stream": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "debug": "https://somewhere.com/job/ci-openstack/4794/artifacts/debug.txt",`
		`+ "rebuild": "https://somewhere.com/job/ci-openstack/4794/rebuild/parametrized"`
		`+ },`
		`+ "artifact": {`
		`+ "type": "brew-build-group",`
		`+ "id": "selinux-update-345",`
		`+ "repository": "https://some.url/repo",`
		`+ "builds":`
		`+ [{`
		`+ "type": "brew-build",`
		`+ "id": 14546276,`
		`+ "issuer": "plautrba",`
		`+ "component": "libselinux",`
		`+ "nvr": "libselinux-2.8-6.el7.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"`
		`+ }, {`
		`+ "type": "brew-build",`
		`+ "id": 14546277,`
		`+ "issuer": "plautrba",`
		`+ "component": "libsepol",`
		`+ "nvr": "libsepol-2.8-3.el7.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"`
		`+ }]`
		`+ },`
		`+ "pipeline": {`
		`+ "id": "ac11dcddf99a",`
		`+ "name": "ci-openstack"`
		`+ },`
		`+ "test": {`
		`+ "category": "functional",`
		`+ "label": [ "fast", "aarch64" ],`
		`+ "namespace": "baseos-qe.baseos-ci",`
		`+ "note": "Some notes.",`
		`+ "result": "failed",`
		`+ "type": "tier1",`
		`+ "xunit": "https://somewhere.com/job/ci-openstack/4794/artifacts/results.xml"`
		`+ },`
		`+ "error": {`
		`+ "reason": "CI pipeline aborted",`
		`+ "issue_url": "http://sentry.somewhere.com/baseos/production/issues/1149/"`
		`+ },`
		`+ "notification": {`
		`+ "recipients": [ "ovasik", "mvadkert" ]`
		`+ },`
		`+ "generated_at": "2018-05-10 08:58:31.222602",`
		`+ "version": "0.2.5"`
		`+ }`

examples/brew-build-group.test.queued.json

file added

+58

		`@@ -0,0 +1,58 @@`
		`+ {`
		`+ "contact": {`
		`+ "name": "BaseOS CI",`
		`+ "team": "BaseOS",`
		`+ "url": "https://somewhere.com",`
		`+ "docs": "https://somewhere.com/user-documentation",`
		`+ "irc": "#baseosci",`
		`+ "email": "baseos-ci@somewhere.com"`
		`+ },`
		`+ "run": {`
		`+ "url": "https://somewhere.com/job/ci-openstack/4794",`
		`+ "log": "https://somewhere.com/job/ci-openstack/4794/console",`
		`+ "log_raw": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "log_stream": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "debug": "https://somewhere.com/job/ci-openstack/4794/artifacts/debug.txt",`
		`+ "rebuild": "https://somewhere.com/job/ci-openstack/4794/rebuild/parametrized"`
		`+ },`
		`+ "artifact": {`
		`+ "type": "brew-build-group",`
		`+ "id": "selinux-update-345",`
		`+ "repository": "https://some.url/repo",`
		`+ "builds":`
		`+ [{`
		`+ "type": "brew-build",`
		`+ "id": 14546276,`
		`+ "issuer": "plautrba",`
		`+ "component": "libselinux",`
		`+ "nvr": "libselinux-2.8-6.el7.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"`
		`+ }, {`
		`+ "type": "brew-build",`
		`+ "id": 14546277,`
		`+ "issuer": "plautrba",`
		`+ "component": "libsepol",`
		`+ "nvr": "libsepol-2.8-3.el7.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"`
		`+ }]`
		`+ },`
		`+ "pipeline": {`
		`+ "id": "ac11dcddf99a",`
		`+ "name": "ci-openstack"`
		`+ },`
		`+ "test": {`
		`+ "category": "functional",`
		`+ "label": [ "fast", "aarch64" ],`
		`+ "namespace": "baseos-qe.baseos-ci",`
		`+ "note": "Some notes.",`
		`+ "type": "tier1",`
		`+ "xunit": "https://somewhere.com/job/ci-openstack/4794/artifacts/results.xml"`
		`+ },`
		`+ "notification": {`
		`+ "recipients": [ "ovasik", "mvadkert" ]`
		`+ },`
		`+ "generated_at": "2018-05-10 08:58:31.222602",`
		`+ "version": "0.2.5"`
		`+ }`

examples/brew-build-group.test.running.json

file added

+55

		`@@ -0,0 +1,55 @@`
		`+ {`
		`+ "contact": {`
		`+ "name": "BaseOS CI",`
		`+ "team": "BaseOS",`
		`+ "url": "https://somewhere.com",`
		`+ "docs": "https://somewhere.com/user-documentation",`
		`+ "irc": "#baseosci",`
		`+ "email": "baseos-ci@somewhere.com"`
		`+ },`
		`+ "run": {`
		`+ "url": "https://somewhere.com/job/ci-openstack/4794",`
		`+ "log": "https://somewhere.com/job/ci-openstack/4794/console",`
		`+ "log_raw": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "log_stream": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "debug": "https://somewhere.com/job/ci-openstack/4794/artifacts/debug.txt",`
		`+ "rebuild": "https://somewhere.com/job/ci-openstack/4794/rebuild/parametrized"`
		`+ },`
		`+ "artifact": {`
		`+ "type": "brew-build-group",`
		`+ "id": "selinux-update-345",`
		`+ "repository": "https://some.url/repo",`
		`+ "builds":`
		`+ [{`
		`+ "type": "brew-build",`
		`+ "id": 14546276,`
		`+ "issuer": "plautrba",`
		`+ "component": "libselinux",`
		`+ "nvr": "libselinux-2.8-6.el7.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"`
		`+ }, {`
		`+ "type": "brew-build",`
		`+ "id": 14546277,`
		`+ "issuer": "plautrba",`
		`+ "component": "libsepol",`
		`+ "nvr": "libsepol-2.8-3.el7.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"`
		`+ }]`
		`+ },`
		`+ "pipeline": {`
		`+ "id": "ac11dcddf99a",`
		`+ "name": "ci-openstack"`
		`+ },`
		`+ "test": {`
		`+ "category": "functional",`
		`+ "label": [ "fast", "aarch64" ],`
		`+ "namespace": "baseos-qe.baseos-ci",`
		`+ "note": "Some notes.",`
		`+ "type": "tier1",`
		`+ "xunit": "https://somewhere.com/job/ci-openstack/4794/artifacts/results.xml"`
		`+ },`
		`+ "generated_at": "2018-05-10 08:58:31.222602",`
		`+ "version": "0.2.5"`
		`+ }`

examples/koji-build-group.build.complete.json

file modified

+2 -2

		`@@ -8,7 +8,7 @@`
		`"email": "baseos-ci@somewhere.com"`
		`},`
		`"artifact": {`
		`- "type": "rpm-build-group",`
		`+ "type": "koji-build-group",`
		`"id": "selinux-update-345",`
		`"repository": "https://some.url/repo",`
		`"builds":`
		`@@ -31,5 +31,5 @@`
		`}]`
		`},`
		`"generated_at": "2018-05-10 08:58:31.222602",`
		`- "version": "0.2.2"`
		`+ "version": "0.2.5"`
		`}`

examples/koji-build-group.build.error.json

file modified

+2 -2

		`@@ -8,7 +8,7 @@`
		`"email": "baseos-ci@somewhere.com"`
		`},`
		`"artifact": {`
		`- "type": "rpm-build-group",`
		`+ "type": "koji-build-group",`
		`"id": "selinux-update-345",`
		`"repository": "https://some.url/repo",`
		`"builds":`
		`@@ -35,5 +35,5 @@`
		`"issue_url": "http://sentry.somewhere.com/baseos/production/issues/1149/"`
		`},`
		`"generated_at": "2018-05-10 08:58:31.222602",`
		`- "version": "0.2.2"`
		`+ "version": "0.2.5"`
		`}`

examples/koji-build-group.test.complete.json

file added

+66

		`@@ -0,0 +1,66 @@`
		`+ {`
		`+ "contact": {`
		`+ "name": "BaseOS CI",`
		`+ "team": "BaseOS",`
		`+ "url": "https://somewhere.com",`
		`+ "docs": "https://somewhere.com/user-documentation",`
		`+ "irc": "#baseosci",`
		`+ "email": "baseos-ci@somewhere.com"`
		`+ },`
		`+ "run": {`
		`+ "url": "https://somewhere.com/job/ci-openstack/4794",`
		`+ "log": "https://somewhere.com/job/ci-openstack/4794/console",`
		`+ "log_raw": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "log_stream": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "debug": "https://somewhere.com/job/ci-openstack/4794/artifacts/debug.txt",`
		`+ "rebuild": "https://somewhere.com/job/ci-openstack/4794/rebuild/parametrized"`
		`+ },`
		`+ "artifact": {`
		`+ "type": "koji-build-group",`
		`+ "id": "selinux-update-345",`
		`+ "repository": "https://some.url/repo",`
		`+ "builds":`
		`+ [{`
		`+ "type": "koji-build",`
		`+ "id": 14546276,`
		`+ "issuer": "plautrba",`
		`+ "component": "libselinux",`
		`+ "nvr": "libselinux-2.8-6.fc29.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"`
		`+ }, {`
		`+ "type": "koji-build",`
		`+ "id": 14546277,`
		`+ "issuer": "plautrba",`
		`+ "component": "libsepol",`
		`+ "nvr": "libsepol-2.8-3.fc29.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"`
		`+ }]`
		`+ },`
		`+ "pipeline": {`
		`+ "id": "ac11dcddf99a",`
		`+ "name": "ci-openstack"`
		`+ },`
		`+ "test": {`
		`+ "category": "functional",`
		`+ "label": [ "fast", "aarch64" ],`
		`+ "namespace": "baseos-qe.baseos-ci",`
		`+ "note": "Some notes.",`
		`+ "result": "failed",`
		`+ "type": "tier1",`
		`+ "xunit": "https://somewhere.com/job/ci-openstack/4794/artifacts/results.xml"`
		`+ },`
		`+ "system":`
		`+ [{`
		`+ "os": "Fedora-Cloud-Base-28",`
		`+ "provider": "openstack",`
		`+ "architecture": "x86_64",`
		`+ "label": "master"`
		`+ }],`
		`+ "notification": {`
		`+ "recipients": [ "ovasik", "mvadkert" ]`
		`+ },`
		`+ "generated_at": "2018-05-10 08:58:31.222602",`
		`+ "version": "0.2.5"`
		`+ }`

examples/koji-build-group.test.error.json

file added

+63

		`@@ -0,0 +1,63 @@`
		`+ {`
		`+ "contact": {`
		`+ "name": "BaseOS CI",`
		`+ "team": "BaseOS",`
		`+ "url": "https://somewhere.com",`
		`+ "docs": "https://somewhere.com/user-documentation",`
		`+ "irc": "#baseosci",`
		`+ "email": "baseos-ci@somewhere.com"`
		`+ },`
		`+ "run": {`
		`+ "url": "https://somewhere.com/job/ci-openstack/4794",`
		`+ "log": "https://somewhere.com/job/ci-openstack/4794/console",`
		`+ "log_raw": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "log_stream": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "debug": "https://somewhere.com/job/ci-openstack/4794/artifacts/debug.txt",`
		`+ "rebuild": "https://somewhere.com/job/ci-openstack/4794/rebuild/parametrized"`
		`+ },`
		`+ "artifact": {`
		`+ "type": "koji-build-group",`
		`+ "id": "selinux-update-345",`
		`+ "repository": "https://some.url/repo",`
		`+ "builds":`
		`+ [{`
		`+ "type": "koji-build",`
		`+ "id": 14546276,`
		`+ "issuer": "plautrba",`
		`+ "component": "libselinux",`
		`+ "nvr": "libselinux-2.8-6.fc29.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"`
		`+ }, {`
		`+ "type": "koji-build",`
		`+ "id": 14546277,`
		`+ "issuer": "plautrba",`
		`+ "component": "libsepol",`
		`+ "nvr": "libsepol-2.8-3.fc29.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"`
		`+ }]`
		`+ },`
		`+ "pipeline": {`
		`+ "id": "ac11dcddf99a",`
		`+ "name": "ci-openstack"`
		`+ },`
		`+ "test": {`
		`+ "category": "functional",`
		`+ "label": [ "fast", "aarch64" ],`
		`+ "namespace": "baseos-qe.baseos-ci",`
		`+ "note": "Some notes.",`
		`+ "result": "failed",`
		`+ "type": "tier1",`
		`+ "xunit": "https://somewhere.com/job/ci-openstack/4794/artifacts/results.xml"`
		`+ },`
		`+ "error": {`
		`+ "reason": "CI pipeline aborted",`
		`+ "issue_url": "http://sentry.somewhere.com/baseos/production/issues/1149/"`
		`+ },`
		`+ "notification": {`
		`+ "recipients": [ "ovasik", "mvadkert" ]`
		`+ },`
		`+ "generated_at": "2018-05-10 08:58:31.222602",`
		`+ "version": "0.2.5"`
		`+ }`

examples/koji-build-group.test.queued.json

file added

+58

		`@@ -0,0 +1,58 @@`
		`+ {`
		`+ "contact": {`
		`+ "name": "BaseOS CI",`
		`+ "team": "BaseOS",`
		`+ "url": "https://somewhere.com",`
		`+ "docs": "https://somewhere.com/user-documentation",`
		`+ "irc": "#baseosci",`
		`+ "email": "baseos-ci@somewhere.com"`
		`+ },`
		`+ "run": {`
		`+ "url": "https://somewhere.com/job/ci-openstack/4794",`
		`+ "log": "https://somewhere.com/job/ci-openstack/4794/console",`
		`+ "log_raw": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "log_stream": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "debug": "https://somewhere.com/job/ci-openstack/4794/artifacts/debug.txt",`
		`+ "rebuild": "https://somewhere.com/job/ci-openstack/4794/rebuild/parametrized"`
		`+ },`
		`+ "artifact": {`
		`+ "type": "koji-build-group",`
		`+ "id": "selinux-update-345",`
		`+ "repository": "https://some.url/repo",`
		`+ "builds":`
		`+ [{`
		`+ "type": "koji-build",`
		`+ "id": 14546276,`
		`+ "issuer": "plautrba",`
		`+ "component": "libselinux",`
		`+ "nvr": "libselinux-2.8-6.fc29.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"`
		`+ }, {`
		`+ "type": "koji-build",`
		`+ "id": 14546277,`
		`+ "issuer": "plautrba",`
		`+ "component": "libsepol",`
		`+ "nvr": "libsepol-2.8-3.fc29.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"`
		`+ }]`
		`+ },`
		`+ "pipeline": {`
		`+ "id": "ac11dcddf99a",`
		`+ "name": "ci-openstack"`
		`+ },`
		`+ "test": {`
		`+ "category": "functional",`
		`+ "label": [ "fast", "aarch64" ],`
		`+ "namespace": "baseos-qe.baseos-ci",`
		`+ "note": "Some notes.",`
		`+ "type": "tier1",`
		`+ "xunit": "https://somewhere.com/job/ci-openstack/4794/artifacts/results.xml"`
		`+ },`
		`+ "notification": {`
		`+ "recipients": [ "ovasik", "mvadkert" ]`
		`+ },`
		`+ "generated_at": "2018-05-10 08:58:31.222602",`
		`+ "version": "0.2.5"`
		`+ }`

examples/koji-build-group.test.running.json

file added

+55

		`@@ -0,0 +1,55 @@`
		`+ {`
		`+ "contact": {`
		`+ "name": "BaseOS CI",`
		`+ "team": "BaseOS",`
		`+ "url": "https://somewhere.com",`
		`+ "docs": "https://somewhere.com/user-documentation",`
		`+ "irc": "#baseosci",`
		`+ "email": "baseos-ci@somewhere.com"`
		`+ },`
		`+ "run": {`
		`+ "url": "https://somewhere.com/job/ci-openstack/4794",`
		`+ "log": "https://somewhere.com/job/ci-openstack/4794/console",`
		`+ "log_raw": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "log_stream": "https://somewhere.com/job/ci-openstack/4794/consoleText",`
		`+ "debug": "https://somewhere.com/job/ci-openstack/4794/artifacts/debug.txt",`
		`+ "rebuild": "https://somewhere.com/job/ci-openstack/4794/rebuild/parametrized"`
		`+ },`
		`+ "artifact": {`
		`+ "type": "koji-build-group",`
		`+ "id": "selinux-update-345",`
		`+ "repository": "https://some.url/repo",`
		`+ "builds":`
		`+ [{`
		`+ "type": "koji-build",`
		`+ "id": 14546276,`
		`+ "issuer": "plautrba",`
		`+ "component": "libselinux",`
		`+ "nvr": "libselinux-2.8-6.fc29.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"`
		`+ }, {`
		`+ "type": "koji-build",`
		`+ "id": 14546277,`
		`+ "issuer": "plautrba",`
		`+ "component": "libsepol",`
		`+ "nvr": "libsepol-2.8-3.fc29.x86_64",`
		`+ "scratch": false,`
		`+ "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"`
		`+ }]`
		`+ },`
		`+ "pipeline": {`
		`+ "id": "ac11dcddf99a",`
		`+ "name": "ci-openstack"`
		`+ },`
		`+ "test": {`
		`+ "category": "functional",`
		`+ "label": [ "fast", "aarch64" ],`
		`+ "namespace": "baseos-qe.baseos-ci",`
		`+ "note": "Some notes.",`
		`+ "type": "tier1",`
		`+ "xunit": "https://somewhere.com/job/ci-openstack/4794/artifacts/results.xml"`
		`+ },`
		`+ "generated_at": "2018-05-10 08:58:31.222602",`
		`+ "version": "0.2.5"`
		`+ }`

schemas/brew-build-group.test.complete.yaml

file added

+44

		`@@ -0,0 +1,44 @@`
		`+ $id: brew-build-group.test.complete`
		`+ $schema: http://json-schema.org/draft-07/schema#`
		`+`
		`+ description:`
		`+ Testing of the group (side-tag with its associated builds) has`
		`+ been completed.`
		`+`
		`+ This is a mandatory message.`
		`+`
		`+ properties:`
		`+ contact:`
		`+ $ref: contact.json`
		`+ run:`
		`+ $ref: run.json`
		`+ artifact:`
		`+ $ref: rpm-build-group.json`
		`+ pipeline:`
		`+ $ref: pipeline.json`
		`+ test:`
		`+ allOf:`
		`+ - $ref: test-common.json`
		`+ - $ref: test-complete.json`
		`+ notification:`
		`+ $ref: notification.json`
		`+ system:`
		`+ type: array`
		`+ items:`
		`+ $ref: system.json`
		`+ generated_at:`
		`+ $ref: common.json#properties/generated_at`
		`+ version:`
		`+ $ref: common.json#properties/version`
		`+`
		`+ required:`
		`+ - contact`
		`+ - run`
		`+ - artifact`
		`+ - pipeline`
		`+ - test`
		`+ - system`
		`+ - generated_at`
		`+ - version`
		`+`
		`+ type: object`

schemas/brew-build-group.test.error.yaml

file added

+42

		`@@ -0,0 +1,42 @@`
		`+ $id: brew-build-group.test.error`
		`+ $schema: http://json-schema.org/draft-07/schema#`
		`+`
		`+ description:`
		`+ Testing has aborted, it was not finished, e.g. because of infrastructure`
		`+ error, CI system error, etc. Note that a test failure is not an error.`
		`+`
		`+ Test failures should be exposed under the <namespace>.group.test.complete topic.`
		`+`
		`+ This is a mandatory message.`
		`+`
		`+ properties:`
		`+ contact:`
		`+ $ref: contact.json`
		`+ run:`
		`+ $ref: run.json`
		`+ artifact:`
		`+ $ref: rpm-build-group.json`
		`+ pipeline:`
		`+ $ref: pipeline.json`
		`+ test:`
		`+ $ref: test-common.json`
		`+ error:`
		`+ $ref: error.json`
		`+ notification:`
		`+ $ref: notification.json`
		`+ generated_at:`
		`+ $ref: common.json#properties/generated_at`
		`+ version:`
		`+ $ref: common.json#properties/version`
		`+`
		`+ required:`
		`+ - contact`
		`+ - run`
		`+ - artifact`
		`+ - pipeline`
		`+ - test`
		`+ - error`
		`+ - generated_at`
		`+ - version`
		`+`
		`+ type: object`

schemas/brew-build-group.test.queued.yaml

file added

+34

		`@@ -0,0 +1,34 @@`
		`+ $id: brew-build-group.test.queued`
		`+ $schema: http://json-schema.org/draft-07/schema#`
		`+`
		`+ description:`
		`+ Testing has been queued, but not yet started.`
		`+`
		`+ This is an optional message.`
		`+`
		`+ properties:`
		`+ contact:`
		`+ $ref: contact.json`
		`+ run:`
		`+ $ref: run.json`
		`+ artifact:`
		`+ $ref: rpm-build-group.json`
		`+ pipeline:`
		`+ $ref: pipeline.json`
		`+ test:`
		`+ $ref: test-common.json`
		`+ generated_at:`
		`+ $ref: common.json#properties/generated_at`
		`+ version:`
		`+ $ref: common.json#properties/version`
		`+`
		`+ required:`
		`+ - contact`
		`+ - run`
		`+ - artifact`
		`+ - pipeline`
		`+ - test`
		`+ - generated_at`
		`+ - version`
		`+`
		`+ type: object`

schemas/brew-build-group.test.running.yaml

file added

+34

		`@@ -0,0 +1,34 @@`
		`+ $id: brew-build-group.test.running`
		`+ $schema: http://json-schema.org/draft-07/schema#`
		`+`
		`+ description:`
		`+ Testing has been started and is in progress.`
		`+`
		`+ This is an optional message.`
		`+`
		`+ properties:`
		`+ contact:`
		`+ $ref: contact.json`
		`+ run:`
		`+ $ref: run.json`
		`+ artifact:`
		`+ $ref: rpm-build-group.json`
		`+ pipeline:`
		`+ $ref: pipeline.json`
		`+ test:`
		`+ $ref: test-common.json`
		`+ generated_at:`
		`+ $ref: common.json#properties/generated_at`
		`+ version:`
		`+ $ref: common.json#properties/version`
		`+`
		`+ required:`
		`+ - contact`
		`+ - run`
		`+ - artifact`
		`+ - pipeline`
		`+ - test`
		`+ - generated_at`
		`+ - version`
		`+`
		`+ type: object`

schemas/common.yaml

file modified

+2 -2

		`@@ -20,9 +20,9 @@`
		`version:`
		`description:`
		`Version of the specification. Following semantic versioning`
		`- (https://semver.org/). Current version is 0.2.4.`
		`+ (https://semver.org/). Current version is 0.2.5.`
		`examples:`
		`- - 0.2.4`
		`+ - 0.2.5`
		`type: string`

		`type: object`

schemas/koji-build-group.test.complete.yaml

file added

+44

		`@@ -0,0 +1,44 @@`
		`+ $id: koji-build-group.test.complete`
		`+ $schema: http://json-schema.org/draft-07/schema#`
		`+`
		`+ description:`
		`+ Testing of the group (side-tag with its associated builds) has`
		`+ been completed.`
		`+`
		`+ This is a mandatory message.`
		`+`
		`+ properties:`
		`+ contact:`
		`+ $ref: contact.json`
		`+ run:`
		`+ $ref: run.json`
		`+ artifact:`
		`+ $ref: rpm-build-group.json`
		`+ pipeline:`
		`+ $ref: pipeline.json`
		`+ test:`
		`+ allOf:`
		`+ - $ref: test-common.json`
		`+ - $ref: test-complete.json`
		`+ notification:`
		`+ $ref: notification.json`
		`+ system:`
		`+ type: array`
		`+ items:`
		`+ $ref: system.json`
		`+ generated_at:`
		`+ $ref: common.json#properties/generated_at`
		`+ version:`
		`+ $ref: common.json#properties/version`
		`+`
		`+ required:`
		`+ - contact`
		`+ - run`
		`+ - artifact`
		`+ - pipeline`
		`+ - test`
		`+ - system`
		`+ - generated_at`
		`+ - version`
		`+`
		`+ type: object`

schemas/koji-build-group.test.error.yaml

file added

+42

		`@@ -0,0 +1,42 @@`
		`+ $id: koji-build-group.test.error`
		`+ $schema: http://json-schema.org/draft-07/schema#`
		`+`
		`+ description:`
		`+ Testing has aborted, it was not finished, e.g. because of infrastructure`
		`+ error, CI system error, etc. Note that a test failure is not an error.`
		`+`
		`+ Test failures should be exposed under the <namespace>.group.test.complete topic.`
		`+`
		`+ This is a mandatory message.`
		`+`
		`+ properties:`
		`+ contact:`
		`+ $ref: contact.json`
		`+ run:`
		`+ $ref: run.json`
		`+ artifact:`
		`+ $ref: rpm-build-group.json`
		`+ pipeline:`
		`+ $ref: pipeline.json`
		`+ test:`
		`+ $ref: test-common.json`
		`+ error:`
		`+ $ref: error.json`
		`+ notification:`
		`+ $ref: notification.json`
		`+ generated_at:`
		`+ $ref: common.json#properties/generated_at`
		`+ version:`
		`+ $ref: common.json#properties/version`
		`+`
		`+ required:`
		`+ - contact`
		`+ - run`
		`+ - artifact`
		`+ - pipeline`
		`+ - test`
		`+ - error`
		`+ - generated_at`
		`+ - version`
		`+`
		`+ type: object`

schemas/koji-build-group.test.queued.yaml

file added

+34

		`@@ -0,0 +1,34 @@`
		`+ $id: koji-build-group.test.queued`
		`+ $schema: http://json-schema.org/draft-07/schema#`
		`+`
		`+ description:`
		`+ Testing has been queued, but not yet started.`
		`+`
		`+ This is an optional message.`
		`+`
		`+ properties:`
		`+ contact:`
		`+ $ref: contact.json`
		`+ run:`
		`+ $ref: run.json`
		`+ artifact:`
		`+ $ref: rpm-build-group.json`
		`+ pipeline:`
		`+ $ref: pipeline.json`
		`+ test:`
		`+ $ref: test-common.json`
		`+ generated_at:`
		`+ $ref: common.json#properties/generated_at`
		`+ version:`
		`+ $ref: common.json#properties/version`
		`+`
		`+ required:`
		`+ - contact`
		`+ - run`
		`+ - artifact`
		`+ - pipeline`
		`+ - test`
		`+ - generated_at`
		`+ - version`
		`+`
		`+ type: object`

schemas/koji-build-group.test.running.yaml

file added

+34

		`@@ -0,0 +1,34 @@`
		`+ $id: koji-build-group.test.running`
		`+ $schema: http://json-schema.org/draft-07/schema#`
		`+`
		`+ description:`
		`+ Testing has been started and is in progress.`
		`+`
		`+ This is an optional message.`
		`+`
		`+ properties:`
		`+ contact:`
		`+ $ref: contact.json`
		`+ run:`
		`+ $ref: run.json`
		`+ artifact:`
		`+ $ref: rpm-build-group.json`
		`+ pipeline:`
		`+ $ref: pipeline.json`
		`+ test:`
		`+ $ref: test-common.json`
		`+ generated_at:`
		`+ $ref: common.json#properties/generated_at`
		`+ version:`
		`+ $ref: common.json#properties/version`
		`+`
		`+ required:`
		`+ - contact`
		`+ - run`
		`+ - artifact`
		`+ - pipeline`
		`+ - test`
		`+ - generated_at`
		`+ - version`
		`+`
		`+ type: object`

schemas/rpm-build-group.yaml

file modified

+3 -2

		`@@ -13,9 +13,10 @@`
		`type: string`
		`type:`
		`description:`
		`- Artifact type, in this case 'rpm-build-group'.`
		`+ Artifact type.`
		`enum:`
		`- - rpm-build-group`
		`+ - brew-build-group`
		`+ - koji-build-group`
		`type: string`
		`builds:`
		`description:`

lholecek commented 4 years ago

JIRA: FACTORY-5348
Signed-off-by: Lukas Holecek hluk@email.cz

lholecek commented 4 years ago

@pingou, @cverna, @lucarval, @gnaponie This is draft for the group/side-tag test result messages. Please review.

mvadkert commented 4 years ago

@lholecek could we name it to the koji-build-group and brew-build-group instead? as we have for build messages?

mvadkert commented on line 24 of examples/group.test.complete.json 4 years ago

Interesting, why put results per artifact here? Seems weird to me. There should be only one result per the whole group which will be by resultsdb-updater replicated for all builds??

lholecek commented 4 years ago

@mvadkert Can you suggest other format for the items? Maybe following?

    "items": [
      {
        "result": "failed",
        "artifact": {
          "type": "brew-build",
          "id": 14546276,
          "issuer": "plautrba",
          "component": "libselinux",
          "nvr": "libselinux-2.8-6.el7.x86_64",
          "scratch": false,
          "source": "git+https://src.fedoraproject.org/rpms/libselinux.git?#5e0ae23a"
        }
      }, {
        "result": "passed",
        "artifact": {
          "type": "brew-build",
          "id": 14546277,
          "issuer": "plautrba",
          "component": "libsepol",
          "nvr": "libsepol-2.8-3.el7.x86_64",
          "scratch": false,
          "source": "git+https://src.fedoraproject.org/rpms/libsepol.git?#5e0ae23a"
        }
      }
    ]

mvadkert commented 4 years ago

There should be no result filed in artifacts dict in my POW. I believe CI systems will test the sidetag as whole and report on the whole sidetag with one result.

@bgoncalv @bookwar are we agreed on this?

gnaponie commented 4 years ago

@lholecek could we name it to the koji-build-group and brew-build-group instead? as we have for build messages?

This way, everytime you have a "new-type-group" you would need to specify it. Don't you think is more flexible to just have "group"?

mvadkert commented 4 years ago

@gnaponie The reason why we have different artifact names for build system instances is to mitigate clashing their results. Note that we also test fedora bits internally and report on them to CI dashboard. If we would use for both brew builds ad koji builds the same artifact name (i.e. build), they would be perceived as related, but they are not, they are artifacts from different buiild systems. We take the same approach also for modules (redhat-module) and build groups.

bgoncalv commented 4 years ago

@mvadkert This seems to require big change in the pipeline to support it. I thought for group build the pipeline would trigger the test execution for each build in the group and publish each result individually without need to wait for all tests to complete and publish a single aggregated message.

mvadkert commented 4 years ago

@bgoncalv yes, changes might be needed unfortunately. We can talk about it later. There are more things about this. This change is just the first step of the three. The scenario you are describing is something to solve later (in step 2 or step 3) afaik. It is expected that this still needs to work also later.

lholecek commented 4 years ago

@mvadkert This seems to require big change in the pipeline to support it. I thought for group build the pipeline would trigger the test execution for each build in the group and publish each result individually without need to wait for all tests to complete and publish a single aggregated message.

@bgoncalv yes, changes might be needed unfortunately. We can talk about it later. There are more things about this. This change is just the first step of the three. The scenario you are describing is something to solve later (in step 2 or step 3) afaik. It is expected that this still needs to work also later.

Hmm, but this means that ResultsDB-Updater won't need to create separate test results for items in the group (for the new group message it will receive) since it will have already received and created the separate test results. We planned to work on this after we know the message format.

So would it be enough for ResultsDB-Updater to create test results just for the group and not for each item in the group?

gnaponie commented 4 years ago

@mvadkert Yeah I thought we agreed on having one message with inside the information about all the tests. And then from that message we would create:
one result for each test
one result for the group

If we don't save a result for each test we cannot cover @bookwar's usecase (and possible future usecases) I believe.
Beside the usecases, that was out "team" decision when we discussed internally. We want to give full flexibility.
Is providing this kind of info in the msg a problem from your side?

rebased onto 096d6425eea045d32c65eb3f10670196ca5111c1

4 years ago

lholecek commented 4 years ago

I've updated the schema so that there is single test result for whole group and list of items in the group are just RPM builds.

lucarval commented 4 years ago

I thought we had sorted this out, but it looks like there's still some confusion.

The first goal we're trying to achieve is to introduce the ability to provide test results for a given group. For instance, Koji builds foo-1.0-1 and bar-2.0-1 have been tested together for test X, here's the result. AFAIK this is the only thing we're targeting right now.

It looks like rpm-build-group does exactly this. However, the problem with rpm-build-group is that its id is described as: "The package group is identified by assigned side-tag." This assumes Koji tags are immutable, which is not correct. Removing, or adding, an artifact, e.g. koji-build, to the group should change the group identifier. The product-build.yaml definition does this. Can we just use that?

As a means of flexibility, we talked about enhancing resultsdb-updater as @gnaponie mentioned. Upon receiving a result for a group, create the result for it, but also create additional results for each item in the group.

Apologies if I'm missing something obvious here.

mvadkert commented 4 years ago

I thought we had sorted this out, but it looks like there's still some confusion.

The first goal we're trying to achieve is to introduce the ability to provide test results for a given group. For instance, Koji builds foo-1.0-1 and bar-2.0-1 have been tested together for test X, here's the result. AFAIK this is the only thing we're targeting right now.

:thumbsup: was my understanding also

It looks like rpm-build-group does exactly this. However, the problem with rpm-build-group is that its id is described as: "The package group is identified by assigned side-tag." This assumes Koji tags are immutable, which is not correct. Removing, or adding, an artifact, e.g. koji-build, to the group should change the group identifier. The product-build.yaml definition does this. Can we just use that?

I believe the "sidetag" artifact is still the correct one, but we need a better ID. In product build w we use a sum of hashes of builds in the sidetag. That should be unique. Another possibility is to use brew/koji events IDs afaik.

As a means of flexibility, we talked about enhancing resultsdb-updater as @gnaponie mentioned. Upon receiving a result for a group, create the result for it, but also create additional results for each item in the group.

Ack, right, we would basically copy the results for the sidetag to all the builds in the sidetag. This was also my understanding.

Apologies if I'm missing something obvious here.

It is completely fine, we need to agree and be in sync :)

mvadkert commented 4 years ago

@lholecek @gnaponie I would not go with a group like artifact, this is too generic. We do not want to mix together groups of different artifacts. Or we would need to come up with a way how to distinguish between them with a new field in body.

Here is an example of build group build events we use now in production:
https://url.corp.redhat.com/brew-build-group-build-complete

Edited 4 years ago by mvadkert

mvadkert commented 4 years ago

@gnaponie

@mvadkert Yeah I thought we agreed on having one message with inside the information about > all the tests. And then from that message we would create:
one result for each test
one result for the group

That is true, one message, but the result is just for the whole group. This one group result will be saved as test result to all artifacts. There are no "sub" results for each artifact.

rebased onto b3028ba083ab475cbcddd8dac0673ea904f379af

4 years ago

rebased onto 68c20bb

4 years ago

lholecek commented 4 years ago

Updated. Please review.

gnaponie commented on line 23 of examples/brew-build-group.test.complete.json 4 years ago

Is this needed at this point?
There's no harm in keeping it though.

gnaponie commented on line 20 of examples/brew-build-group.test.complete.json 4 years ago

The file name doesn't match the type in here.

gnaponie commented on line 20 of examples/koji-build-group.test.queued.json 4 years ago

Also here.. shouldn't this be "koji-build-group"?

lholecek commented on line 20 of examples/brew-build-group.test.complete.json 4 years ago

This file is basically the same as examples/brew-build-group.build.complete.json with added fields common for *.test.* messages.

lholecek commented on line 23 of examples/brew-build-group.test.complete.json 4 years ago

I think it's needed, so Greenwave can list the builds in decision (at least with verbose flag).

lholecek commented on line 20 of examples/koji-build-group.test.queued.json 4 years ago

From schema:

    artifact:
        $ref: rpm-build-group.json

gnaponie commented on line 20 of examples/brew-build-group.test.complete.json 4 years ago

Yeah, but if the type in here is "rpm-build-group" shouldn't the file name be "rpm-build-group.test.complete.json"?

lholecek commented on line 20 of examples/brew-build-group.test.complete.json 4 years ago

Yeah, but if the type in here is "rpm-build-group" shouldn't the file name be "rpm-build-group.test.complete.json"?

The request was to be able to distinguish between brew-build-group and koji-build-group messages (these could differ in future) -- this is consistent with the *-build-group.build.* messages. But this was confusing for me too since the koji and brew messages are basically the same.

mvadkert commented 4 years ago

yeah, I think giulia is write, schema name is generalized so it can be used for both artifact types, koji-build-group and brew-build-group

mvadkert commented 4 years ago

I believe this is wrong:
https://pagure.io/fedora-ci/messages/blob/master/f/examples/brew-build-group.build.complete.json#_11

also ...

mvadkert commented 4 years ago

@lholecek can you also correct those build messages? the artifact names should be:

koji-build-group
brew-build-group

Sorry :(

Looking what we use in production ...

mvadkert commented 4 years ago

@psss the names of the artifacts are brew-build-group and koji-build-group, not rpm-build-group, that is just the generic schema used for bot artfiact types

mvadkert commented 4 years ago

I see it wrong even in production, fixing all over the place

1 new commit added

Fix type for rpm-build-group

4 years ago

lholecek commented 4 years ago

the names of the artifacts are brew-build-group and koji-build-group, not rpm-build-group, that is just the generic schema used for bot artfiact types

Fixed in additional commit.

gnaponie commented 4 years ago

+1 \o/

mvadkert commented 4 years ago

@lholecek one last stupid nitpick, can you set the version to 0.2.5 and also increment schemas/common.yml to 0.2.5 pls?

mvadkert commented 4 years ago

and LGTM :thumbsup:

1 new commit added