Setup Synapse + Riot + freenode IRC bridge on Openshift to have as much communication methods unified as possible
When do you need this? (YYYY/MM/DD) Completing this in 2019 would be great
When is this no longer needed or useful? (YYYY/MM/DD) Once a better idea than federated messaging servers appears we should evaluate it and switch to it
If we cannot complete your request, what is the impact?
Communication in various channels stay fragmented.
Matrix is an open standard for interoperable, decentralised, real-time communication over IP. It can be used to power Instant Messaging, VoIP/WebRTC signalling, Internet of Things communication - or anywhere you need a standard HTTP API for publishing and subscribing to data whilst tracking the conversation history.
I have some experience with running Synapse (server) + Riot (web client) + IRC bridge + Mautrix-telegram (to bridge telegram) on Openshift, so I could help with setting, monitoring and maintaining this.
Benefits of this approach: * Federation Various communities may setup other matrix servers similar to email, so users from, say, regional Fedora communities may run their own servers on other domains and communicate with other Matrix users
Bridges Matrix, similar to XMPP, has many bridges to other communication methods - IRC, Telegram, Slack etc. Server admin needs to run additional app and setup integration. Matrix has a concept of 'ghost' users, so bridged users would appear as 'User (Telegram)' or 'User (IRC)' as if they were connected to the same server.
Privacy
All messages and data is stored in a Postgres DB. Matrix uses SSL everywhere to communicate between services
Matrix supports various authentication methods - most useful would be LDAP, IIUC we could use Fedora single-sign on system to log users in.
In general for these proposals we need to know: How much resources is needed - CPU - Memory - Disk - Database? (type and size per year) Who is going to take care of it when it breaks? Who is going to update it when things need updating? Who is going to write documentation for people to know how a service is supposed to work? Who is the person who will answer user questions? How is it managed? [This seems to be done via openshift and ansible]
How much resources is needed CPU
2 vCPU is sufficient for operation
Memory
The server usually takes less than 1 GB
Disk
Except the DB it would store only user uploaded pictures. Usually a few GBs is sufficient
Database? (type and size per year)
Postgresql. Size depends on the amount of messages stored there. A test instance inside Red Hat with ~15 active users in ~300 rooms has generated 50Gb DB in 6 months, so I guess the estimate of 100 GB per year would be realistic. The history can be purged for selected rooms until a specified timestamp, so the space can be reclaimed.
Matrix server can be sharded using workers, so we might be able to scale the amount of workers with the load
Who is going to take care of it when it breaks? Who is going to update it when things need updating? Who is going to write documentation for people to know how a service is supposed to work? Who is the person who will answer user questions?
The answer for all four questions would be 'Fedora Matrix instance group', which so far consists of only me. Breaks are complicated topic, but Matrix developers are usually responsive we could use their help. Updates have been so far as simple as 'bump version number in Openshift's ImageStream' Docs and answering would ideally be handled by upstream - see https://matrix.org/docs/guides/faq and https://github.com/matrix-org/matrix-doc.
How is it managed? [This seems to be done via openshift and ansible]
Right, Ansible bit to generate necessary configs, handle certs and upload YAMLs to Openshift. Upstream k8s repo - https://github.com/ananace/matrix-synapse
Matrix server for Fedora is a very good idea, but can Synapse be integrated with FAS accounts?
Public user registration on this Matrix server should be disabled and new user accounts can be created only using FAS accounts.
i'm interested in having something like this. I think it can be the foundation for our future communications in Fedora. I have openshift experience so i might be able to help @vrutkovs if he needs it.
I'm not too familiar with Fedora's FAS, but I think its feasible. LDAP and SAML2 are supported, but we might need an input from Synapse developers on this.
That can be done, yes
Matrix server for Fedora is a very good idea, but can Synapse be integrated with FAS accounts? I'm not too familiar with Fedora's FAS, but I think its feasible. LDAP and SAML2 are supported, but we might need an input from Synapse developers on this.
If we go ahead with any of this, we'd want to use OpenID Connect or SAML2. The LDAP server is an internal implementation detail that should not be used by any applications.
It seems we want SAML, but I'm not sure how would that work with desktop clients
Users can use FAS to register a new Matrix account with it's own password to use with desktop clients.
So, what advantage does this give us over asking people to use matrix.org and their freenode bridge?
I guess it would provide a fedoraproject.org home server, but that seems pretty cosmetic for providing another service.
Additionally there may be legal issues to consider here since we would be storing messages and possibly storing encrypted messages.
A few matrix-specific features are - encrypted chats, embedding a page as a widget (e.g. jitsi session), secure file sharing, threaded chats, mobile clients support and tight control over the data
matrix.org server has terrible response time.
That definitely needs to be approved by legal. Here's how Matrix tackled GDPR - https://matrix.org/blog/2018/05/08/gdpr-compliance-in-matrix/
I recently discovered a Golang tool called Matterbridge that looks cool. It is (basically) a multi-platform proxy server that bridges several different chat protocols together:
I'm using it for a community outside of Fedora and wrote an Ansible role for deploying it. I've had reliable use out of it, but I've only deployed it for a week. I'm also only using it for IRC <=> Slack. I haven't tried the other options.
Also, the project development community is alive and well. They are responsive and helpful. I reported a bug with the Slack bridge, and it was fixed with a new git-tagged version in ~48 hours.
Metadata Update from @bowlofeggs: - Issue priority set to: Waiting on Assignee (was: Needs Review)
I recently discovered a Golang tool called Matterbridge that looks cool
Looks interesting, afaik there are native bridges in Matrix for (almost) all of these protocols.
Recently we've been experimenting with internal Matrix setup, few notes: Matrix gives a fairly large set of metrics, helps with identifying bottlenecks Postgres is king there, stable setup and regular backups are important * Old messages can be purged for particular rooms
I could take care of setting up the test/stage server given the certificates (required for federation) and a VM is provided
I'd love to give this a try on Communishift instance - could someone create a namespace for me play with?
An excellent idea! I've added you to provisioners there, so you should be able to login and create your project. :)
I guess perhaps we should just close this and down the road if things prove really popular or we have some need to move it in more supported we can do so then?
Current state:
#welcome:matrix.apps.os.fedorainfracloud.org
TODO:
I wonder if I should keep updating this ticket or move it elsewhere
:heart:
I'm following along!
At this point I'm blocked on SSO - synapse doesn't seem to support OIDC, just SAML2. If anyone has ideas / prs / proxies - holler
The rest - adding bridges and setting up metrics most importantly - is trivial
synapse doesn't seem to support OIDC, just SAML2
Is this just an unimplemented feature? Should we open a feature request somewhere or does one already exist we can follow?
Is this just an unimplemented feature?
Yes, its basically https://github.com/matrix-org/synapse/issues/2376 if I understood correctly
Might be nice for you to setup a dedicated pagure project to the install? Then you can manage things as you like... ie, file tickets, setup deps, people can file tickets there if they run into problems with it, etc.
Right, so SSO login issue has been tackled, so lets keep track of missing features and issues of the stage instance at https://pagure.io/matrix-communishift/issues
I'll close this ticket - and once the "test" instance reaches maturity we'll open a new one to upgrade it to production
Metadata Update from @vrutkovs: - Issue close_status updated to: Fixed - Issue status updated to: Closed (was: Open)
Right, so SSO login issue has been tackled,
It's fixed? Did you fix it?
so lets keep track of missing features and issues of the stage instance at https://pagure.io/matrix-communishift/issues
Watching!
Yes, its a bit ugly right now (additional proxy is deployed which transforms OpenID Connect flow to CAS), but it works
Matrix Synapse 1.14 will include native OpenID Connect support, so we can use that when it releases, or we can backport the feature to Fedora's matrix-synapse package.
matrix-synapse
Login to comment on this ticket.