#7399 Run Matrix Server and bridge existing communication methods
Opened 20 days ago by vrutkovs. Modified 3 days ago

  • Describe what you need us to do:

Setup Synapse + Riot + freenode IRC bridge on Openshift to have as much communication methods unified as possible

  • When do you need this? (YYYY/MM/DD)
    Completing this in 2019 would be great

  • When is this no longer needed or useful? (YYYY/MM/DD)
    Once a better idea than federated messaging servers appears we should evaluate it and switch to it

  • If we cannot complete your request, what is the impact?

Communication in various channels stay fragmented.


Matrix is an open standard for interoperable, decentralised, real-time communication over IP. It can be used to power Instant Messaging, VoIP/WebRTC signalling, Internet of Things communication - or anywhere you need a standard HTTP API for publishing and subscribing to data whilst tracking the conversation history.

I have some experience with running Synapse (server) + Riot (web client) + IRC bridge + Mautrix-telegram (to bridge telegram) on Openshift, so I could help with setting, monitoring and maintaining this.

Benefits of this approach:
* Federation
Various communities may setup other matrix servers similar to email, so users from, say, regional Fedora communities may run their own servers on other domains and communicate with other Matrix users

  • Bridges
    Matrix, similar to XMPP, has many bridges to other communication methods - IRC, Telegram, Slack etc. Server admin needs to run additional app and setup integration. Matrix has a concept of 'ghost' users, so bridged users would appear as 'User (Telegram)' or 'User (IRC)' as if they were connected to the same server.

  • Privacy

All messages and data is stored in a Postgres DB. Matrix uses SSL everywhere to communicate between services

  • Clients
    There are various clients for Matrix protocol - defacto standard Riot (Web/Electron and iOS/Android), Fractal (GTK+Rust), Nheko (Qt)

Matrix supports various authentication methods - most useful would be LDAP, IIUC we could use Fedora single-sign on system to log users in.


In general for these proposals we need to know:
How much resources is needed
- CPU
- Memory
- Disk
- Database? (type and size per year)
Who is going to take care of it when it breaks?
Who is going to update it when things need updating?
Who is going to write documentation for people to know how a service is supposed to work?
Who is the person who will answer user questions?
How is it managed? [This seems to be done via openshift and ansible]

How much resources is needed
CPU

2 vCPU is sufficient for operation

Memory

The server usually takes less than 1 GB

Disk

Except the DB it would store only user uploaded pictures. Usually a few GBs is sufficient

Database? (type and size per year)

Postgresql. Size depends on the amount of messages stored there. A test instance inside Red Hat with ~15 active users in ~300 rooms has generated 50Gb DB in 6 months, so I guess the estimate of 100 GB per year would be realistic. The history can be purged for selected rooms until a specified timestamp, so the space can be reclaimed.

Matrix server can be sharded using workers, so we might be able to scale the amount of workers with the load

Who is going to take care of it when it breaks?
Who is going to update it when things need updating?
Who is going to write documentation for people to know how a service is supposed to work?
Who is the person who will answer user questions?

The answer for all four questions would be 'Fedora Matrix instance group', which so far consists of only me.
Breaks are complicated topic, but Matrix developers are usually responsive we could use their help.
Updates have been so far as simple as 'bump version number in Openshift's ImageStream'
Docs and answering would ideally be handled by upstream - see https://matrix.org/docs/guides/faq and https://github.com/matrix-org/matrix-doc.

How is it managed? [This seems to be done via openshift and ansible]

Right, Ansible bit to generate necessary configs, handle certs and upload YAMLs to Openshift. Upstream k8s repo - https://github.com/ananace/matrix-synapse

Matrix server for Fedora is a very good idea, but can Synapse be integrated with FAS accounts?

Public user registration on this Matrix server should be disabled and new user accounts can be created only using FAS accounts.

i'm interested in having something like this. I think it can be the foundation for our future communications in Fedora. I have openshift experience so i might be able to help @vrutkovs if he needs it.

Matrix server for Fedora is a very good idea, but can Synapse be integrated with FAS accounts?

I'm not too familiar with Fedora's FAS, but I think its feasible. LDAP and SAML2 are supported, but we might need an input from Synapse developers on this.

Public user registration on this Matrix server should be disabled and new user accounts can be created only using FAS accounts.

That can be done, yes

Matrix server for Fedora is a very good idea, but can Synapse be integrated with FAS accounts?

I'm not too familiar with Fedora's FAS, but I think its feasible. LDAP and SAML2 are supported, but we might need an input from Synapse developers on this.

If we go ahead with any of this, we'd want to use OpenID Connect or SAML2.
The LDAP server is an internal implementation detail that should not be used by any applications.

It seems we want SAML, but I'm not sure how would that work with desktop clients

It seems we want SAML, but I'm not sure how would that work with desktop clients

Users can use FAS to register a new Matrix account with it's own password to use with desktop clients.

So, what advantage does this give us over asking people to use matrix.org and their freenode bridge?

I guess it would provide a fedoraproject.org home server, but that seems pretty cosmetic for providing another service.

Additionally there may be legal issues to consider here since we would be storing messages and possibly storing encrypted messages.

So, what advantage does this give us over asking people to use matrix.org and their freenode bridge?

A few matrix-specific features are - encrypted chats, embedding a page as a widget (e.g. jitsi session), secure file sharing, threaded chats, mobile clients support and tight control over the data

I guess it would provide a fedoraproject.org home server, but that seems pretty cosmetic for providing another service.

matrix.org server has terrible response time.

Additionally there may be legal issues to consider here since we would be storing messages and possibly storing encrypted messages.

That definitely needs to be approved by legal. Here's how Matrix tackled GDPR - https://matrix.org/blog/2018/05/08/gdpr-compliance-in-matrix/

I recently discovered a Golang tool called Matterbridge that looks cool. It is (basically) a multi-platform proxy server that bridges several different chat protocols together:

  • Mattermost
  • IRC
  • XMPP
  • Gitter
  • Slack
  • Discord
  • Telegram
  • Hipchat
  • Rocket.chat
  • Matrix
  • Steam
  • Twitch
  • Zulip

I'm using it for a community outside of Fedora and wrote an Ansible role for deploying it. I've had reliable use out of it, but I've only deployed it for a week. I'm also only using it for IRC <=> Slack. I haven't tried the other options.

Also, the project development community is alive and well. They are responsive and helpful. I reported a bug with the Slack bridge, and it was fixed with a new git-tagged version in ~48 hours.

Metadata Update from @bowlofeggs:
- Issue priority set to: Waiting on Assignee (was: Needs Review)

3 days ago

Login to comment on this ticket.

Metadata