#414 Describe the API v1 and v2 separately. Describe the messaging API.
Merged 4 years ago by jkaluza. Opened 4 years ago by jkaluza.
jkaluza/freshmaker docs-update2  into  master

file removed
-16
@@ -1,16 +0,0 @@ 

- ===============

- Freshmaker APIs

- ===============

- 

- .. automodule:: freshmaker

- 

- 

- .. _http-api:

- 

- HTTP REST API

- =============

- 

- .. autoflask:: freshmaker:app

-     :undoc-static:

-     :modules: freshmaker.views

-     :order: path

file added
+312
@@ -0,0 +1,312 @@ 

+ ========================

+ API version 1

+ ========================

+ 

+ .. _event_json_api_1:

+ 

+ Event JSON representation

+ ====================================

+ 

+ The Freshmaker Event is always represented in the API request as JSON, for example:

+ 

+ .. sourcecode:: none

+ 

+     {

+         "builds": [],

+         "depending_events": [],

+         "depends_on_events": [],

+         "dry_run": false,

+         "event_type_id": 10,

+         "id": 18730,

+         "message_id": "message_123",

+         "requested_rebuilds": [],

+         "requester": null,

+         "requester_metadata": {},

+         "search_key": "44637",

+         "state": 3,

+         "state_name": "COMPLETE",

+         "state_reason": "4 images rebuilt.",

+         "time_created": "2019-08-01T07:00:46Z",

+         "time_done": "2019-08-01T07:03:12Z",

+         "url": "/api/1/events/18730"

+     }

+ 

+ The fields used in the Freshmaker Event JSON have following meaning:

+ 

+ .. _event_builds:

+ 

+ *builds* - ``(list of Freshmaker Artifact build JSONs)``

+     List of artifact builds which are part of this Freshmaker event.

+ 

+ .. _event_depending_events:

+ 

+ *depending_events* - ``(list of numbers)``

+     List of IDs of other Freshmaker events which are depending on this Freshmaker event.

+ 

+ .. _event_depends_on_events:

+ 

+ *depends_on_events* - ``(list of numbers)``

+     List of IDs of other Freshmaker events which this event depends on.

+ 

+ .. _event_dry_run:

+ 

+ *dry_run* - ``(boolean)``

+     When set to ``true`` the event is handled in dry run mode. In this mode, no real builds are submitted to build system. Instead, all the builds are automatically moved to ``COMPLETE`` state.

+ 

+ .. _event_event_type_id:

+ 

+ *event_type_id* - ``(number)``

+     The ID of the type of this Event. Full list of Event types can listed using the ``/api/1/event-types/`` REST API. The most important event types are:

+ 

+     - 10 (``ErrataAdvisoryStateChangedEvent``) - Event triggered by message, rebuilding all artifacts (currently just container images) as result of advisory release.

+     - 13 (``ManualRebuildWithAdvisoryEvent``) - Event triggered manually, rebuilding all artifacts (currently just container images) as result of advisory release.

+ 

+ .. _event_id:

+ 

+ *id* - ``(number)``

+     The ID of Freshmaker event.

+ 

+ .. _event_message_id:

+ 

+ *message_id* - ``(string)``

+     The ID of message (fedmsg or AMQP) which triggered the Event.

+ 

+ .. _event_requested_rebuilds:

+ 

+ *requested_rebuilds* - ``(list of strings)``

+     List of artifacts which should be rebuilt in this Freshmaker Event. Filled in only for manual rebuilds which requested particular artifacts to be rebuilt. Currently, it is always list of container images NVRs.

+ 

+ .. _event_requester:

+ 

+ *requester* - ``(string or null)``

+     The Kerberos username of requester of this Freshmaker Event. Set to string only for manual rebuilds, otherwise ``null``.

+ 

+ .. _event_requester_metadata:

+ 

+ *requester_metadata* - ``(JSON object)``

+     Additional metadata set by requester when submitting the manual rebuild. It can be used to track the context of manual rebuild.

+ 

+ .. _event_search_key:

+     

+ *search_key* - ``(string)``

+     The key identifying source of the Event. For each Event type, there is a different way how the key is generated:

+ 

+     - Event type 10 (``ErrataAdvisoryStateChangedEvent``) - The ID of advisory which triggered the Event.

+     - Event type 13 (``ManualRebuildWithAdvisoryEvent``) - The ID of advisory which triggered the Event.

+ 

+ .. _event_state:

+ 

+ *state* - ``(number)``

+     Number defining the state the Event is currently in:

+ 

+     - 0 (``INITIALIZED``) - Event is initialized and Freshmaker is now searching for artifacts to build.

+     - 1 (``BUILDING``) - Some artifacts to build have been found and Freshmaker is building them.

+     - 2 (``COMPLETE``) - All artifacts have been build. Note that this does not mean all the builds were successfull - just one successfull artifact build is enough to mark the Event as ``COMPLETE``.

+     - 3 (``FAILED``) - There was some major error while handling the Event or all the artifact builds failed to build.

+     - 4 (``SKIPPED``) - No artifacts have been found to build or the Event is not allowed by the Freshmaker's configuration.

+     - 5 (``CANCELED``) - Event has been manually cancelled using the REST API.

+ 

+ .. _event_state_name:

+ 

+ *state_name* - ``(string)``

+     Name of the state the Event is currently in. See ``state`` for more info.

+ 

+ .. _event_state_reason:

+ 

+ *state_reason* - ``(string)``

+     Sentence describing the current state.

+ 

+ .. _event_time_created:

+ 

+ *time_created* - ``(datetime)``

+     The date and time on which the Event has been initialized (moved to ``INITIALIZED`` state).

+ 

+ .. _event_time_done:

+ 

+ *time_done* - ``(datetime)``

+     The date and time on which the Event has been moved to ``FAILED``, ``COMPLETE``, or ``CANCELED`` state.

+ 

+ 

+ .. _build_json_api_1:

+ 

+ Artifact Build JSON representation

+ ==================================

+ 

+ The Freshmaker Artifact Build is always represented in the API request as JSON, for example:

+ 

+ .. sourcecode:: none

+ 

+     {

+         "build_args": {

+             ...

+         },

+         "build_id": 22429387,

+         "dep_on": "fedora-30-container",

+         "dep_on_id": 21657,

+         "event_id": 16730,

+         "id": 21837,

+         "name": "httpd-container",

+         "odcs_composes": [],

+         "original_nvr": "httpd-container-2.4-1",

+         "rebuild_reason": "directly_affected",

+         "rebuilt_nvr": "httpd-container-2.4-1.1561731291",

+         "state": 1,

+         "state_name": "DONE",

+         "state_reason": "Built successfully.",

+         "time_completed": "2019-06-29T03:28:56Z",

+         "time_submitted": "2019-06-28T14:14:51Z",

+         "type": 1,

+         "type_name": "IMAGE",

+         "url": "/api/1/builds/21837"

+     }

+ 

+ .. _build_build_args:

+ 

+ *build_args* - ``(JSON object)``

+     JSON object containing arguments passed to build system to build this artifact.

+     

+     .. WARNING::

+         The content of this JSON object is not part of the Freshmaker REST API and can change at any time.

+ 

+     Commonly used ``build_args`` are:

+ 

+     - ``arches`` - white-space separated list of architecture the Artifact is built against.

+     - ``branch`` - name of the branch from which the Artifact's source code is taken.

+     - ``commit`` - commit hash in source repository from which the Artifact's source code is taken.

+     - ``target`` - Koji target in which the Artifact is built.

+     - ``retry_count`` - number describes how many times was Artifact build retried.

+ 

+ .. _build_build_id:

+ 

+ *build_id* - ``(number)``

+     The ID of Artifact Build.

+ 

+ .. _build_dep_on:

+ 

+ *dep_on* - ``(string)``

+     The :ref:`name<build_name>`. of the Artifact build this Artifact build depends on.

+ 

+ .. _build_dep_on_id:

+ 

+ *dep_on_id* - ``(number)``

+     The ID of the Artifact build this Artifact build depends on.

+ 

+ .. _build_event_id:

+ 

+ *event_id* - ``(number)``

+     The ID of the Event this Artifact Build is part of.

+ 

+ .. _build_name:

+ 

+ *name* - ``(string)``

+     The name of this Artifact Build.

+ 

+ .. _build_odcs_composes:

+ 

+ *odcs_composes* - ``(list of numbers)``

+     List of ODCS composes the Freshmaker directly generated and used while building this Artifact build in the build system.

+ 

+ .. _build_original_nvr:

+ 

+ *original_nvr* - ``(string)``

+     The original (before the rebuild) NVR of Artifact build.

+ 

+ .. _build_rebuild_reason:

+ 

+ *rebuild_reason* - ``(string)``

+     The reason why this artifact is included in the Event. Can be one of:

+ 

+     - ``directly_affected`` - The Artifact build is directly affected by the Event (for example affected by the CVE) and is whitelisted by Freshmaker's configuration.

+     - ``dependency`` - The Artifact build is included in the Event just because it is dependency of ``directly_affected`` Artifact build.

+ 

+ .. _build_rebuilt_nvr:

+ 

+ *rebuilt_nvr* - ``(string)``

+     The NVR of Artifact build built by Freshmaker.

+ 

+ .. _build_state:

+ 

+ *state* - ``(number)``

+     Number defining the state the Artifact build is currently in:

+ 

+     - 0 (``BUILD``) - Artifact build is currently being built in build system.

+     - 1 (``DONE``) - Artifact build has been built successfully.

+     - 2 (``FAILED``) - Artifact build failed to be build.

+     - 3 (``CANCELED``) - Artifact build has been cancelled manually.

+     - 4 (``PLANNED``) - Artifact build is planned to be build.

+ 

+ .. _build_state_name:

+ 

+ *state_name* - ``(string)``

+     Name of the state the Artifact build is currently in. See :ref:`state<build_state>` for more info.

+ 

+ .. _build_state_reason:

+ 

+ *state_reason* - ``(string)``

+     Sentence describing the current state.

+ 

+ .. _build_time_completed:

+ 

+ *time_completed* - ``(datetime)``

+     The date and time on which the Artifact Build has been completed.

+ 

+ .. _build_time_submitted:

+ 

+ *time_submitted* - ``(datetime)``

+     The date and time on which the Artifact build has been moved to ``PLANNED`` state.

+ 

+ .. _build_type:

+ 

+ *type* - ``(number)``

+     Type of the Artifact build:

+ 

+     - 0 (``RPM``) - Artifact build is an RPM package.

+     - 1 (``IMAGE``) - Artifact build is a container image.

+     - 2 (``MODULE``) - Artifact build is a module.

+     - 3 (``IMAGE_REPOSITORY``) - Artifact build is a container image repository.

+ 

+ .. _build_type_name:

+ 

+ *type__name* - ``(string)``

+     Name of the type of Artifact build. See :ref:`type<build_type>` for more info.

+     

+ 

+ .. _pagination_api_1:

+ 

+ REST API pagination

+ ===================

+ 

+ When multiple objects are returned by the Freshmaker REST API, they are wrapped in the following JSON which allows pagination:

+ 

+ .. sourcecode:: none

+ 

+     {

+         "items": [

+             {JSON_OBJECT},

+             ...

+         ],

+         "meta": {

+             "first": "http://freshmaker.localhost/api/1/events/?per_page=10&page=1",

+             "last": "http://freshmaker.localhost/api/1/events/?per_page=10&page=14890",

+             "next": "http://freshmaker.localhost/api/1/events/?per_page=10&page=2",

+             "page": 1,

+             "pages": 14890,

+             "per_page": 10,

+             "prev": null,

+             "total": 148898

+         }

+     }

+ 

+ The ``items`` list contains the objects JSONs. The ``meta`` dict contains metadata about pagination. It is possible to use ``per_page`` argument to set the number of objects showed per single page and ``page`` to choose the page to show.

+ 

+ 

+ HTTP REST API

+ =============

+ 

+ .. automodule:: freshmaker

+ 

+ .. autoflask:: freshmaker:app

+     :undoc-static:

+     :endpoints: event_types_list, event_type, build_types_list, build_type, build_states_list, build_state, events_list, event, builds_list, build, manual_trigger, about, verify_image, verify_image_repository

+     :modules: freshmaker.views

+     :order: path

file added
+83
@@ -0,0 +1,83 @@ 

+ ========================

+ API version 2

+ ========================

+ 

+ The Freshmaker API version 2 is mostly same as the API version 1. This document therefore describes only the differences between these two API versions.

+ 

+ .. _event_json_api_2:

+ 

+ Event JSON representation

+ ====================================

+ 

+ The Freshmaker Event is always represented in the API request as JSON, for example:

+ 

+ .. sourcecode:: none

+ 

+     {

+         "builds_summary": {

+             "total": 0

+         },

+         "depending_events": [],

+         "depends_on_events": [],

+         "dry_run": false,

+         "event_type_id": 10,

+         "id": 18730,

+         "message_id": "message_123",

+         "requested_rebuilds": [],

+         "requester": null,

+         "requester_metadata": {},

+         "search_key": "44637",

+         "state": 3,

+         "state_name": "COMPLETE",

+         "state_reason": "4 images rebuilt.",

+         "time_created": "2019-08-01T07:00:46Z",

+         "time_done": "2019-08-01T07:03:12Z",

+         "url": "/api/1/events/18730"

+     }

+ 

+ The meaning of almost all the fields is the same as in :ref:`event_json_api_1` using the API version 1.

+ 

+ There are following differences between API version 1 and API version 2:

+ 

+ - The ``builds`` field is replaced with ``builds_summary``.

+ 

+ .. _event_builds_summary:

+ 

+ *builds_summary* - ``(JSON)``

+     JSON object showing the summary of artifacts builds included in the Event grouped by their ``state_name``. For example:

+ 

+     .. sourcecode:: none

+ 

+         "builds_summary": {

+             "total": 10,

+             "DONE": 7,

+             "FAILED": 3

+         }

+ 

+ 

+ .. _build_json_api_2:

+ 

+ Artifact Build JSON representation

+ ==================================

+ 

+ The JSON representation of Artifact Build is exactly the same as in :ref:`build_json_api_1` using the API version 1.

+ 

+ 

+ .. _pagination_api_2:

+ 

+ REST API pagination

+ ===================

+ 

+ The pagination works exactly the same way as in :ref:`pagination_api_1` using the API version 1.

+ 

+ 

+ HTTP REST API

+ =============

+ 

+ .. automodule:: freshmaker

+ 

+ .. autoflask:: freshmaker:app

+     :undoc-static:

+     :endpoints: event_types_list_v2, event_type_v2, build_types_list_v2, build_type_v2, build_states_list_v2, build_state_v2, events_list_v2, event_v2, builds_list_v2, build_v2, manual_trigger_v2, about_v2, verify_image_v2, verify_image_repository_v2

+     :modules: freshmaker.views

+     :order: path

file modified
+3 -1
@@ -15,7 +15,9 @@ 

     :caption: Contents:

  

     about

-    api

+    api_v1

+    api_v2

+    messaging_api

  

  Indices and tables

  ==================

@@ -0,0 +1,20 @@ 

+ ========================

+ Messaging API

+ ========================

+ 

+ Freshmaker also sends AMQP or Fedmsg messages when events or builds change its state.

+ 

+ ``event.state.changed``

+ =======================

+ 

+ This message is sent on every :ref:`Freshmaker Event<event_json_api_1>`'s :ref:`state<event_state>` change. The message contains :ref:`Event JSON representation as defined in API version 1<event_json_api_1>`.

+ 

+ ``event.state.changed.min``

+ ===========================

+ 

+ This message is sent on every :ref:`Freshmaker Event<event_json_api_2>`'s :ref:`state<event_state>` change. The message contains :ref:`Event JSON representation as defined in API version 2<event_json_api_2>`.

+ 

+ ``build.state.changed``

+ =======================

+ 

+ This message is sent on every :ref:`Artifact Build<build_json_api_1>`'s :ref:`state<build_state>` change. The message contains :ref:`Artifact Build<build_json_api_1>`.

file modified
+21 -73
@@ -224,36 +224,31 @@ 

  

      @freshmaker_event_api_latency.time()

      def get(self, id):

-         """ Returns the list of Freshmaker events.

+         """ Returns Freshmaker Events.

  

-         Whend ``id`` is set, only the Freshmaker Event defined by that ID is

+         If ``id`` is set, only the Freshmaker Event defined by that ID is

          returned.

  

-         **Sample response**:

+         :query string message_id: Return only events with this :ref:`message_id<event_message_id>`.

+         :query string search_key: Return only events with this :ref:`search_key<event_search_key>`.

+         :query number event_type_id: Return only events with this :ref:`event_type_id<event_event_type_id>`.

+         :query number/string state: Return only events int this :ref:`state<event_state>`.

+         :query bool show_full_json: When ``True``, the returned Freshmaker Event JSON objects

+             contains all the fields described in the

+             :ref:`Freshmaker Event representation for API version 1<event_json_api_1>`.

  

-         .. sourcecode:: none

+             When ``False``, the returned Freshmaker Event JSON objects are in the

+             :ref:`Freshmaker Event representation for API version 2<event_json_api_2>` format.

  

-             {

-                 "builds": [],

-                 "depending_events": [],

-                 "depends_on_events": [],

-                 "dry_run": null,

-                 "event_type_id": 8,

-                 "id": 1000,

-                 "message_id": "ID:message-1",

-                 "requested_rebuilds": [],

-                 "requester": null,

-                 "requester_metadata": {},

-                 "search_key": "32460",

-                 "state": 4,

-                 "state_name": "SKIPPED",

-                 "state_reason": "Event skipped",

-                 "time_created": "2018-03-06T15:27:49Z",

-                 "time_done": null,

-                 "url": "/api/1/events/1000"

-             }

+             Default value for API version 1 is ``True``, for API version 2 is ``False``.

+ 

+         :query string order_by: Order the events by the given field. If ``-`` prefix is used,

+             the order will be descending. The default value is ``-id``. Available fields are:

  

-         :statuscode 200: Current events are returned.

+             - :ref:`id<event_id>`

+             - :ref:`message_id<event_message_id>`

+ 

+         :statuscode 200: Requested events are returned.

          :statuscode 404: Freshmaker event not found.

          """

          # Boolean that is set to false if builds should not
@@ -297,7 +292,7 @@ 

          Manage Freshmaker event defined by ID. The request must be

          :mimetype:`application/json`.

  

-         Returns the cancelled Freshmaker event.

+         Returns the cancelled Freshmaker event as JSON.

  

          **Sample request**:

  
@@ -311,30 +306,6 @@ 

                  "action": "cancel"

              }

  

-         **Sample response**:

- 

-         .. sourcecode:: none

- 

-             {

-                 "builds": [],

-                 "depending_events": [],

-                 "depends_on_events": [],

-                 "dry_run": null,

-                 "event_type_id": 8,

-                 "id": 1000,

-                 "message_id": "ID:message-1",

-                 "requested_rebuilds": [],

-                 "requester": null,

-                 "requester_metadata": {},

-                 "search_key": "32460",

-                 "state": 4,

-                 "state_name": "SKIPPED",

-                 "state_reason": "Event skipped",

-                 "time_created": "2018-03-06T15:27:49Z",

-                 "time_done": null,

-                 "url": "/api/1/events/1000"

-             }

- 

          :jsonparam string action: Action to do with an Event. Currently only "cancel"

              is supported.

          :statuscode 200: Cancelled event is returned.
@@ -404,7 +375,7 @@ 

          Trigger manual Freshmaker rebuild. The request must be

          :mimetype:`application/json`.

  

-         Returns the newly created Freshmaker event.

+         Returns the newly created Freshmaker event as JSON.

  

          **Sample request**:

  
@@ -418,29 +389,6 @@ 

                  "errata_id": 12345

              }

  

-         **Sample response**:

- 

-         .. sourcecode:: none

- 

-             {

-                 "builds": [],

-                 "depending_events": [],

-                 "depends_on_events": [],

-                 "dry_run": null,

-                 "event_type_id": 8,

-                 "id": 1000,

-                 "message_id": "ID:message-1",

-                 "requested_rebuilds": [],

-                 "requester": null,

-                 "requester_metadata": {},

-                 "search_key": "32460",

-                 "state": 4,

-                 "state_name": "SKIPPED",

-                 "state_reason": "Event skipped",

-                 "time_created": "2018-03-06T15:27:49Z",

-                 "time_done": null,

-                 "url": "/api/1/events/1000"

-             }

  

          :jsonparam string errata_id: The ID of Errata advisory to rebuild

              artifacts for.

no initial comment

Is this any kind of JSON? Or does it have to be at least a JSON object, {}?

It should be at least {}. I will fix it.

For my own understanding...
Since Freshmaker is not done determining which artifacts to build, we could think of this as "initializing" instead, right?

Is this after or before Freshmaker has determined which artifacts to build?

Do you think it makes sense to track attempt attribute to indicate how many time the build has been retried?

I'm curious, what exactly is an IMAGE_REPOSITORY artifact?

Yeah, "initializing" would probably be better name, but "initialized" is what we already use in the API.

Before, it's when it is moved to initialized state, which is the first state when event is created in db.

:thumbsup: This is great! Thanks for doing this.

rebased onto 778b7e9

4 years ago

Pull-Request has been merged by jkaluza

4 years ago