Commit graph

1037 commits

Author SHA1 Message Date
Simon Ser ec2c0685dd Add WHO cache
This adds a new field to upstreams, members, which is a casemapped map
of upstream users known to the soju. The upstream users known to soju
are: self, any monitored user, and any user with whom we share a
channel.

The information stored for each upstream user corresponds to the info
that can be returned by a WHO/WHOX command.

We build the upstream user information both incrementally, capturing
information contained in JOIN and AWAY messages; and with the bulk user
information contained in WHO replies we receive.

This lets us build a user cache that can then be used to return
synthetic WHO responses to later WHO requests by downstreams.

This is useful because some networks (eg Libera) heavily throttle WHO
commands, and without this cache, any downstream connecting would send 1
WHO command per channel, so possibly more than a dozen WHO commands,
which soju then forwarded to the upstream as WHO commands.

With this cache most WHO commands can be cached and avoid sending
WHO commands to the upstream.

In order to cache the "flags" field, we synthetize the field from user
info we get from incremental messages: away status (H/G) and bot status
(B). This could result in incorrect values for proprietary user fields.
Support for the server-operator status (*) is also not supported.

Of note is that it is difficult to obtain a user "connected server"
field incrementally, so clients that want to maximize their WHO cache
hit ratio can use WHOX to only request fields they need, and in
particular not include the server field flag.

Co-authored-by: delthas <delthas@dille.cc>
2022-12-01 15:50:33 +01:00
delthas ac578823dc Fix clearing webpush targets after any MARKREAD
Previously, we would clear webpush targets after any MARKREAD.

Consider the following scenario (ignore any typos, this is crafted by
hand):

    <<< @time=2020-01-01T00:00:00Z PRIVMSG #foo :hi mark!
    <<< @time=2020-01-02T00:00:00Z PRIVMSG #foo :hi again mark!
    >>> MARKREAD #foo timestamp=2020-01-01T00:00:00Z
    >>> MARKREAD #foo timestamp=2020-01-02T00:00:00Z

The push target was previously cleared on the first MARKREAD, which
means that the second MARKREAD was never broadcast to Firebase, and all
devices would keep the "hi again mark!" notification indefinitely.

This changes the webpush target map so that we store a timestamp of the
last highlight we sent. We only clear the push target when sending a
MARKREAD that is at or after the last message.
2022-11-28 17:55:19 +01:00
delthas 897c21dbb4 Truncate message times to the second when using the FS message store
The FS message store truncates message times to the second.

This means that a message sent out as 2020-01-01T00:00:00.123Z could be
sent later as part of a CHATHISTORY batch as 2020-01-01T00:00:00.000Z,
which could cause issues in clients.

One such issue is a client sending a MARKREAD for
2020-01-01T00:00:00.000Z, with another client considering the
2020-01-01T00:00:00.123Z message it has as unread.

This fixes the issue by truncating all message times to the second when
using the FS message store.
2022-11-28 17:54:13 +01:00
Sandra Snan 87b2d32682 Be more clear that -connect-command needs quotes 2022-11-28 15:10:39 +01:00
Simon Ser 330d5d794c Upgrade dependencies 2022-11-22 14:55:52 +01:00
Simon Ser 106d40dcd4 Upgrade to gopkg.in/irc.v4 2022-11-14 12:06:58 +01:00
Simon Ser a0e9c10bc4 contrib/clients: reference read_marker.py for Weechat 2022-11-11 11:52:24 +01:00
Simon Ser ea97b1a9bd Use ratified extended-monitor cap name for needAllDownstreamCaps
Fixes: 6ad3dcc396 ("Use ratified extended-monitor cap name")
2022-11-06 20:39:04 +01:00
Simon Ser 6ad3dcc396 Use ratified extended-monitor cap name
References: https://github.com/ircv3/ircv3-specifications/pull/508
2022-11-06 20:36:33 +01:00
Simon Ser 90be9a8ab9 downstream: stop sending HTTP OPTIONS request on WEBPUSH REGISTER
We were sending a test notification later anyways. Let's just do
that to check that the endpoint accepts our messages.
2022-09-30 12:20:07 +02:00
zsrv 926dcb37ac Make the auto-away functionality configurable 2022-09-26 19:52:07 +02:00
Łukasz Margiela d605d64d1d Add build tag for modernc/sqlite driver 2022-09-26 19:31:43 +02:00
Simon Ser c3ab11de4e downstream: drop downstreamConn.unmarshalEntity{,Network} 2022-09-26 16:57:05 +02:00
Simon Ser dde4ee9518 config: drop multi-upstream-mode 2022-09-26 16:57:05 +02:00
Simon Ser 085f5c17a4 doc: drop multi-upstream mode bits 2022-09-26 16:57:05 +02:00
Simon Ser f646dc9ff2 irc: drop needMarshaling from applyChannelModes return values
Not used anymore.
2022-09-26 16:57:05 +02:00
Simon Ser a02a06de0d upstream: use forEachDownstreamByID to forward RPL_AWAY
It's a reply to a command. If possible, avoid broadcasting it to
all connected clients.
2022-09-26 16:57:05 +02:00
Simon Ser b78d8be490 upstream: remove heuristic to marshal nick in errors 2022-09-26 16:57:05 +02:00
Simon Ser 8a2f544806 downstream: drop downstreamConn.marshalEntity
This is a no-op.
2022-09-26 16:57:05 +02:00
Simon Ser 31957a9ac4 downstream: drop downstreamConn.marshalMessage
It's a no-op.
2022-09-26 16:57:05 +02:00
Simon Ser 25257b8892 downstream: drop downstreamConn.marshalUserPrefix
This is now a no-op.
2022-09-26 16:57:05 +02:00
Simon Ser 00919e9412 downstream: drop downstreamConn.unmarshalText
It's unused.
2022-09-26 16:57:05 +02:00
Simon Ser eba7aa38d2 downstream: drop downstreamConn.isMultiUpstream
This is now always false.
2022-09-26 16:57:05 +02:00
Simon Ser 1ea3a19403 downstream: refuse to enable multi-upstream mode
References: https://todo.sr.ht/~emersion/soju/125
2022-09-26 16:57:04 +02:00
Simon Ser 027a89a0b0 server: log when waiting for user goroutines on shutdown
Useful for debugging.
2022-09-18 21:43:31 +02:00
Simon Ser d354c73933 server: close queued up connections on shutdown
Closes: https://todo.sr.ht/~emersion/soju/204
2022-09-18 21:37:45 +02:00
Simon Ser c5f6a41d6c downstream: remove outdated Web Push TODO
We already do that.
2022-09-16 18:58:42 +02:00
Simon Ser bb868eae82 downstream: fix CHATHISTORY LATEST without a bound
Fixes: 5e56cc30c5 ("downstream: fix inverted range in CHATHISTORY LATEST with a timestamp")
2022-09-16 18:55:31 +02:00
Simon Ser 5b10348957 downstream: don't echo back SASL mechanism
Some clients will queue up multiple AUTHENTICATE commands without
waiting for a reply to avoid some roundtrips. However that means
the traffic looks like so:

    AUTHENTICATE <mechanism>
    AUTHENTICATE <base64 blob containing credentials>

soju will fail the first command, and will behave as if no SASL
authentication was in progress when interpreting the second one.
This means we'll echo back the security-sensitive base64 blob to
the client in the error message, which is definitely not great.

Stop doing that.
2022-09-12 21:42:03 +02:00
Simon Ser 9cdbee3b1b Upgrade dependencies 2022-09-11 15:48:24 +02:00
Simon Ser a9949c2e95 database/sqlite: make optional
SQLite requires cgo, for some use cases this is undesirable.
2022-09-11 13:57:00 +02:00
Simon Ser d27880e03e config: use structs to group DB/MsgAuth 2022-09-11 13:50:34 +02:00
Ember Sawady aee65cd269 Fix labeled-response without downstream echo-message
Previously, receiving labeled responses to messages sent from a
downstream without echo-message would fail, because soju would filter
out the responses under the assumption that it was an echoed message.
Only do this filtering when msg.Prefix.Name != uc.nick in order to avoid
this issue.
2022-09-06 15:15:34 +02:00
Simon Ser f4af7975d3 Pass-through ISUPPORT LINELEN
Can be used by the server to indicate support for a greater message
size limit.
2022-08-28 18:53:41 +02:00
Simon Ser 6ba63aec4f go fmt 2022-08-23 09:53:11 +02:00
Ember Sawady 57584c08ed Fix NICK on upstreams supporting MONITOR
Previously, uc.network.Network.Nick wasn't successfully updated on
downstream NICK. This would cause soju to immediately switch back to the
old nick when the upstream supported MONITOR, so long as the network had
a nick configured as of initialization.

In addition, stop monitoring our desired nick once we've successfully
switched to it once, in order to not immediately undo server-induced
nick changes.
2022-08-23 09:39:24 +02:00
Petr Ročkai 5e56cc30c5 downstream: fix inverted range in CHATHISTORY LATEST with a timestamp 2022-08-17 19:43:01 +02:00
Simon Ser 6fa8a1f30a downstream: add hard limit on Web Push subscriptions
Each subscription comes with the cost of a HTTP request when
broadcasting a message.
2022-08-17 17:08:23 +02:00
Simon Ser cffdbc16b4 downstream: break findWebPushSubscription in two functions
We'll use that to count the number of existing subscriptions in
the next commit.
2022-08-17 17:04:11 +02:00
Simon Ser 65f0b2367e Broadcast Web Push subscriptions in a new goroutine 2022-08-17 16:09:12 +02:00
Simon Ser 05a382ef16 Downgrade Web Push urgency for unimportant messages
For instance TAGMSG and MARKREAD aren't really high urgency.
2022-08-17 16:04:11 +02:00
Simon Ser 1ee5dc062d upstream: add timeout for pending commands
References: https://todo.sr.ht/~emersion/soju/194
2022-08-17 15:45:05 +02:00
Jeff Martin a662091a9d server: fix panic stacktrace print formatting
This change uses string formatting for the output of debug.Stack. The
output of debug.Stack is []byte, which isn't human readable.
2022-08-08 15:27:04 +02:00
Simon Ser 0ef8b7d4c8 downstream: re-format doc comment with Go 1.19 style
We can use lists now.
2022-08-05 18:37:32 +02:00
Reto Brunner ac01ac0143 Use output given to NewLogger
Previously the output param was ignored
2022-08-02 10:11:25 +02:00
Simon Ser ce19f76673 database/postgres: fix inverted networkID check in ListWebPushSubscriptions
Same as 02844ca9df ("database/postgres: fix inverted networkID
check in StoreWebPushSubscription").
2022-07-27 09:48:52 +02:00
Simon Ser 02844ca9df database/postgres: fix inverted networkID check in StoreWebPushSubscription
Presented without comment.
2022-07-25 19:32:09 +02:00
Simon Ser 8849669d61 database/postgres: set max open conns
Open at most 25 connections. Should fix errors such as
"pq: sorry, too many clients already".
2022-07-25 19:00:14 +02:00
Simon Ser d0868722cd database/postgres: make WebPushSubscription.user NOT NULL
Also see 6734464b72 ("database/postgres: fix inverted network and
user in StoreWebPushSubscription").
2022-07-25 18:55:22 +02:00
Simon Ser 6734464b72 database/postgres: fix inverted network and user in StoreWebPushSubscription 2022-07-25 18:49:21 +02:00