Development/ModeSync
From the makers of InspIRCd.
Protocol modifications to fix MODExMODE and MODExJOIN channel desyncs.
Contents |
Problem
Consider two servers: A and B, and two users AA and BB - one on each server, both opped in #test. The lag between servers A and B is one second. Initially, there is no channel limit set.
- At time 0, AA sends MODE #test +l 5, and BB sends MODE #test +l 6.
- At time 1, AA sees :BB MODE #test +l 6, and BB sees :AA MODE #test +l 5.
The servers will continue to disagree about the value of the mode until it is changed again.
It is also possible to desync a binary mode:
- At time 0, AA sends MODE #test +m
- At time 0, AA sends MODE #test -m
- At time 0, BB sends MODE #test +m
- At time 1, AA sees :BB MODE #test +m
- At time 1, BB sees :AA MODE #test -m (he might also see the +m)
The servers will continue to disagree about the value until someone changes it.
Incomplete solutions
- Alphabetically comparing mode parameters
- This will prevent changing a mode to a smaller alphabetical value, which is a major usability flaw
- Resync on other events such as join
- This is what inspircd 1.2 does
- Causes unexpected bouncing of modes, observed as annoying by users on live networks (where bots adjust channel +l)
Solution 1
Add a mode sequence to the channel structure. Changes to protocol:
:UID DMODE #channel <timestamp> <sequence> <mode and parameters>
Used for mode changes only. Allows server source for /SAMODE and similar.
:UID DTOPIC #channel <timestamp> <topicts> <setter> <sequence> :<topic>
Topic change. Channel timestamp is required to prevent splitride topic changes. While technically the setter field is not required here, it is included for easier compatibility with services. This replaces TOPIC in s2s communication.
:UID DJOIN #channel <timestamp>
Used for joins to an existing channel. If this creates a channel on a server (acccidental create), send a RESYNC to the source server.
:SID FJOIN #channel <timestamp> <sequence> +<simple modes and parameters> :<nicks (optional)> :SID FMODE #channel <timestamp> <list modes and parameters> :SID FTOPIC #channel <timestamp> <topicts> <setter> :<topic>
Used during netburst.
:SID RESYNC <destination SID> #channel
Used to request a mode re-sync for a channel. The destination server must broadcast an FMODE and FTOPIC containing the current mode list for the channel (including simple modes and list modes).
Sequence format and use
Increment the mode sequence and send the new value with any outgoing DMODE or DTOPIC.
When receiving a DMODE, apply the mode as usual. If the mode sequence is strictly greater than the current mode sequence for the channel, update the mode sequence and finish. If the mode sequence is less than or equal to the current mode sequence, there has been a possible race. After applying the mode, send an FMODE containing your current values for all changed modes.
On receipt of an FJOIN or FMODE, apply netburst rules for mode merging. Take the larger mode sequence as the channel mode sequence (unless channel timestamp is conflicting; in that case, take the winner's).
Same rules apply for topic changes; FTOPIC takes the newer topic timestamp or if equal, the greater (by strcmp) topic.
Example
- Initial state: sequence number = 16, modes +ntl 5
- Time 0
- User AA sends MODE #foo +l 6
- Server A sends :AA DMODE #foo 1234567890 17 +l 6
- User BB sends MODE #foo +l 7
- Server B sends :BB DMODE #foo 1234567890 17 +l 7
- Time 1
- Server A gets :BB DMODE #foo 1234567890 17 +l 7
- Server A notices race. Applies mode change, sends :A FMODE #foo 1234567890 +l 7
- Server B gets :AA DMODE #foo 1234567890 17 +l 6
- Server B notices race. Applies mode change, sends :B FMODE #foo 1234567890 +l 6
- Time 2
- Server A gets :B FMODE #foo 1234567890 +l 6 - discards, loses the merge
- Server B gets :A FMODE #foo 1234567890 +l 7 - applies, wins the merge
The servers agree on a limit of 7.
Overflow
As declared, the integer in the sequence number may overflow the bounds of an integer. This is obviously a problem as the comparisons will be incorrect. To address this, the integer component shall always be printed as a 16-bit unsigned number (uint16_t), and comparisons shall be done by subtracting the two unsigned integers and casting the result to a signed integer (int16_t). This allows up to 32768 mode changes to happen during a netsplit and still result in a correct sequence comparison.
Third-party Compatability
Because the sequence number calculation may be regarded as too complex by some third-party authors, it is permissible to send a "*" in place of the sequence number for a channel. The first inspircd server to receive this message should replace the "*" with a proper mode sequence - for DMODE, incrementing and using their own SID; and for FJOIN/FMODE, using the stored mode sequence number.
This could result in the third-party servers having a desynced view of the mode changes. However, mode changes by third-party servers often fall under one of the following categories:
- noncritical changes that can be ignored or desynced without issue (i.e. Defender setting +R on a channel in the presense of a joinflood)
- persistant changes (i.e. ChanServ forcing a mode value) which will automatically resync when the mode change is bounced for the last time.
If a third-party server only makes mode changes of this type, it can safely ignore the mode sequence information.
Solution 2
Similar to solution 1, but maintain a separate global-order timestamp for each mode letter. This removes the requirement for the RESYNC action, at the expense of more complexity and memory use. A single global-order timestamp can still be used in sending the modes; it must only be larger than any individual mode timestamp.
Global-order Sequence Numbers
The sequence field is a combination of an integer and an SID, separated by a colon. For example, 17:437 or 2832:00A. When comparing two sequences, the integer portion is compared first, and only if the integers match, the SID is compared alphanumerically.
For example: 3:977 < 4:234 < 4:977 < 14:00A < 14:862.
When preparing a DMODE to send, take the current mode sequence, increment the integer by 1, and append your own SID. Store the value as the new mode sequence for the channel.
