This sounds more like a initiative to me: https://docs.fedoraproject.org/en-US/cpe/initiatives/
But we have one similar initiative this quarter, so we will see if this could be part of it.

Oh, also: this is the current fedmsg service: https://pagure.io/discourse2fedmsg. Patrick had it deployed in a testing instance but it is no longer running.

Also: we would want to be able to distinguish between different discourse instances. I'm not sure if the current code does this.

I would be nice to port the code to fedora-messaging before we deploy it to prod. It'll simplify the ansible playbooks.

One question I should answer (with your help!) is: what events do we want to translate to fedmsg. Here's the options

My thinking is that new topics and posts being created is obviously useful, and "solved" could be used for badges and other things (at least if it carries the name of the person who posted the solution -- otherwise less so). The discourse badges are separate from Fedora Badges, and I don't know what to do about that. Possibly completely disable them, but one thing we could do is use this to echo them instead.

Group stuff we don't need because that's gonna come from the SSO, and other stuff is basically management blah blah blah (categories, etc.)

Maybe "likes" is good -- likes are not anonymous on the site, so it's not revealing anything to broadcast them. Could be useful and indicates reading/lurking activity.

And I don't think user events is really particularly useful except for stalking, which... is a non-goal.

One question I should answer (with your help!) is: what events do we want to translate to fedmsg. Here's the options

My thinking is that new topics and posts being created is obviously useful, and "solved" could be used for badges and other things (at least if it carries the name of the person who posted the solution -- otherwise less so). The discourse badges are separate from Fedora Badges, and I don't know what to do about that. Possibly completely disable them, but one thing we could do is use this to echo them instead.

Group stuff we don't need because that's gonna come from the SSO, and other stuff is basically management blah blah blah (categories, etc.)

Maybe "likes" is good -- likes are not anonymous on the site, so it's not revealing anything to broadcast them. Could be useful and indicates reading/lurking activity.

And I don't think user events is really particularly useful except for stalking, which... is a non-goal.

These all look good to me. I think the only thing we could possibly use User events for is aggregate data to track overall activity on the site but I'm not sure if this is available elsewhere

These all look good to me. I think the only thing we could possibly use User events for is aggregate data to track overall activity on the site but I'm not sure if this is available elsewhere

Yeah, I'm a little torn. The site does have its own reporting tools, but there's something amazing about having all of this in the Fedora message stream.

On the balance, though, I think that for those purposes, measuring topics, posts, and likes gives us the most useful view into meaningful activity anyway.

Also, I should say this directly in case it's unclear -- we'll want to distinguish Ask Fedora and Discussion posts in the messages. That's probably inherent, but just to be sure.

Also, I should say this directly in case it's unclear -- we'll want to distinguish Ask Fedora and Discussion posts in the messages. That's probably inherent, but just to be sure.

I spend some time today to look at what needs to be done with the code.

The code itself is pretty simple, it's just waiting till anything arrives to the /webhook endpoint, verifies if it's an event from discourse and then sends the JSON payload as fedmsg message with topic taken from X-Discourse-Event-Type.

Here is the list of things, that needs to be done:

• Port over to fedora messaging

• Create fedora messaging schema (This could be tricky because we don't know what could be in the payload, but maybe after looking at the discourse documentation this will be clear)

• Create unit tests (at least some simple tests would be nice, maybe adding format and lint checker would be a nice bonus)

• Update README.md (current one is too brief)

• [Nice to have] CI

I added a PR with tox and some basics tests https://pagure.io/discourse2fedmsg/pull-request/3

I created new issues on the discourse2fedmsg tracker based on the list I in previous comment.
You can see them on discourse2fedmsg tracker.

https://pagure.io/discourse2fedmsg/pull-request/3 is now merged

ok, i have started trying to get a functional development environment set up so i can work on this -- looking at setting up a discoruse instance locally as a start. From there, we can try to get it ported to fedora messaging.

Okay, an update:

• moved the source of this project to https://github.com/fedora-infra/discourse2fedmsg as all the other 2fedmsg / fedoramessage projects are on github, and it makes sense to have it there
• used @abompard 's flask cookiecutter to populte the project, and migrate the code over. This means we now have all the standard checks, test infra, and other stuff working
• created a fully functional dev envrionemnt using vagrant, (including discourse for easy testing)
• ported to fedora messaging -- no schemas, it just takes the messages that discourse POSTs to it, and pushes them as the body of the fedora message (and generates the topic based on the discourse topic headers in the request)

next steps are to
figure out if we can (or want schemas for this)
get deployed on staging so @mattdm can hook the discourse instances up to the staging instance to see if it does what he needs it to do

Okay also have a few questions on how we should structure the messages -- opened an issue here:

https://github.com/fedora-infra/discourse2fedmsg/issues/9

discourse2fedmsg is now running in staging! (thanks @abompard and @mattdm )

and is sending messages from ask.fp.o and discussion.fp.o to the fedora messaging queue, and showing up in datagrepper here:

https://apps.stg.fedoraproject.org/datagrepper2/v2/search?rows_per_page=1&delta=127800&category=discourse

also, the separate schema definition package is available (and used by discourse2fedmsg) here:

https://github.com/fedora-infra/discourse2fedmsg-messages

and on pypi here:

https://pypi.org/project/discourse2fedmsg-messages/

ok, so next steps -- going to email the infra list about descourse2fedmsg in staging, and then maybe deploy to prod next week?

any thoughts @mattdm @abompard ?

Yay Ryan!

1. Do you still need admin privs? We have room on Ask because we're on the Business+ plan now, but for Discussion I would need to kick one of the existing-but-less-active moderators in order to grant the power.

2. I added "Likes" to the event to trigger on. I think we can do something with Fedora Badges around that, and I think it's an important measure of engagement. Is this covered under the current schema or does it need something more?

3. "Likes" does not currently include other reactions. See https://meta.discourse.org/t/webhook-for-reactions/209197 for upstream request. (Right now, an informal one, but I can also make it a support request to add some weight.) If that's implemented, will that just start working or will it need an update here?

In any case, I'm ready for it to go to prod when you are!

Yay Ryan!

1. Do you still need admin privs? We have room on Ask because we're on the Business+ plan now, but for Discussion I would need to kick one of the existing-but-less-active moderators in order to grant the power.

No, not anymore -- the messages seem to be coming through now -- we just need to ensure that they are returning 200 OK on the discourse end-- which they seem to be.

1. I added "Likes" to the event to trigger on. I think we can do something with Fedora Badges around that, and I think it's an important measure of engagement. Is this covered under the current schema or does it need something more?

2. "Likes" does not currently include other reactions. See https://meta.discourse.org/t/webhook-for-reactions/209197 for upstream request. (Right now, an informal one, but I can also make it a support request to add some weight.) If that's implemented, will that just start working or will it need an update here?
   

the schema that we made here was super broad -- basically it will forward any message that discourse sends it to the fedora messaging bus -- so if you turn on more messages in the webhooks settings UI in discourse, those will show up on the bus.

As for the topics, we are using the x-discourse headers that it sends, so in the case of:
diuscourse sending a message with:

• X-Discourse-Event-Type: topic
• X-Discourse-Event: topic_created

the topic of the fedora message will be discourse.topic.topic_created

In any case, I'm ready for it to go to prod when you are!

awesome. will work with @abompard to try to make this happen this week

cheers,
ryanlerch

Cool, thanks Ryan! I have confirmed that all of the requests in the recent log are status code 200. Some of them do seem to take a while, though — up to 3.3 seconds.