Merge pull request #2721 from ClearlyClaire/glitch-soc/merge-upstream

Merge upstream changes up to 7f808ff6e9
This commit is contained in:
Claire 2024-05-30 16:00:17 +02:00 committed by GitHub
commit 2a34c9a01e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 194 additions and 18 deletions

View file

@ -2,6 +2,61 @@
All notable changes to this project will be documented in this file.
## [4.2.9] - 2024-05-30
### Security
- Update dependencies
- Fix private mention filtering ([GHSA-5fq7-3p3j-9vrf](https://github.com/mastodon/mastodon/security/advisories/GHSA-5fq7-3p3j-9vrf))
- Fix password change endpoint not being rate-limited ([GHSA-q3rg-xx5v-4mxh](https://github.com/mastodon/mastodon/security/advisories/GHSA-q3rg-xx5v-4mxh))
- Add hardening around rate-limit bypass ([GHSA-c2r5-cfqr-c553](https://github.com/mastodon/mastodon/security/advisories/GHSA-c2r5-cfqr-c553))
### Added
- Add rate-limit on OAuth application registration ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/30316))
- Add fallback redirection when getting a webfinger query `WEB_DOMAIN@WEB_DOMAIN` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/28592))
- Add `digest` attribute to `Admin::DomainBlock` entity in REST API ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/29092))
### Removed
- Remove superfluous application-level caching in some controllers ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29862))
- Remove aggressive OAuth application vacuuming ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/30316))
### Fixed
- Fix leaking Elasticsearch connections in Sidekiq processes ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30450))
- Fix language of remote posts not being recognized when using unusual casing ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30403))
- Fix off-by-one in `tootctl media` commands ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30306))
- Fix removal of allowed domains (in `LIMITED_FEDERATION_MODE`) not being recorded in the audit log ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/30125))
- Fix not being able to block a subdomain of an already-blocked domain through the API ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30119))
- Fix `Idempotency-Key` being ignored when scheduling a post ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/30084))
- Fix crash when supplying the `FFMPEG_BINARY` environment variable ([timothyjrogers](https://github.com/mastodon/mastodon/pull/30022))
- Fix improper email address validation ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29838))
- Fix results/query in `api/v1/featured_tags/suggestions` ([mjankowski](https://github.com/mastodon/mastodon/pull/29597))
- Fix unblocking internationalized domain names under certain conditions ([tribela](https://github.com/mastodon/mastodon/pull/29530))
- Fix admin account created by `mastodon:setup` not being auto-approved ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29379))
- Fix reference to non-existent var in CLI maintenance command ([mjankowski](https://github.com/mastodon/mastodon/pull/28363))
## [4.2.8] - 2024-02-23
### Added
- Add hourly task to automatically require approval for new registrations in the absence of moderators ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29318), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/29355))
In order to prevent future abandoned Mastodon servers from being used for spam, harassment and other malicious activity, Mastodon will now automatically switch new user registrations to require moderator approval whenever they are left open and no activity (including non-moderation actions from apps) from any logged-in user with permission to access moderation reports has been detected in a full week.
When this happens, users with the permission to change server settings will receive an email notification.
This feature is disabled when `EMAIL_DOMAIN_ALLOWLIST` is used, and can also be disabled with `DISABLE_AUTOMATIC_SWITCHING_TO_APPROVED_REGISTRATIONS=true`.
### Changed
- Change registrations to be closed by default on new installations ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29280))
If you are running a server and never changed your registrations mode from the default, updating will automatically close your registrations.
Simply re-enable them through the administration interface or using `tootctl settings registrations open` if you want to enable them again.
### Fixed
- Fix processing of remote ActivityPub actors making use of `Link` objects as `Image` `url` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29335))
- Fix link verifications when page size exceeds 1MB ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/29358))
## [4.2.7] - 2024-02-16
### Fixed

View file

@ -147,6 +147,9 @@ class NotifyService < BaseService
end
def statuses_that_mention_sender
# This queries private mentions from the recipient to the sender up in the thread.
# This allows up to 100 messages that do not match in the thread, allowing conversations
# involving multiple people.
Status.count_by_sql([<<-SQL.squish, id: @notification.target_status.in_reply_to_id, recipient_id: @recipient.id, sender_id: @sender.id, depth_limit: 100])
WITH RECURSIVE ancestors(id, in_reply_to_id, mention_id, path, depth) AS (
SELECT s.id, s.in_reply_to_id, m.id, ARRAY[s.id], 0
@ -154,16 +157,17 @@ class NotifyService < BaseService
LEFT JOIN mentions m ON m.silent = FALSE AND m.account_id = :sender_id AND m.status_id = s.id
WHERE s.id = :id
UNION ALL
SELECT s.id, s.in_reply_to_id, m.id, st.path || s.id, st.depth + 1
FROM ancestors st
JOIN statuses s ON s.id = st.in_reply_to_id
LEFT JOIN mentions m ON m.silent = FALSE AND m.account_id = :sender_id AND m.status_id = s.id
WHERE st.mention_id IS NULL AND NOT s.id = ANY(path) AND st.depth < :depth_limit
SELECT s.id, s.in_reply_to_id, m.id, ancestors.path || s.id, ancestors.depth + 1
FROM ancestors
JOIN statuses s ON s.id = ancestors.in_reply_to_id
/* early exit if we already have a mention matching our requirements */
LEFT JOIN mentions m ON m.silent = FALSE AND m.account_id = :sender_id AND m.status_id = s.id AND s.account_id = :recipient_id
WHERE ancestors.mention_id IS NULL AND NOT s.id = ANY(path) AND ancestors.depth < :depth_limit
)
SELECT COUNT(*)
FROM ancestors st
JOIN statuses s ON s.id = st.id
WHERE st.mention_id IS NOT NULL AND s.visibility = 3
FROM ancestors
JOIN statuses s ON s.id = ancestors.id
WHERE ancestors.mention_id IS NOT NULL AND s.account_id = :recipient_id AND s.visibility = 3
SQL
end
end

View file

@ -48,6 +48,7 @@ require_relative '../lib/chewy/strategy/bypass_with_warning'
require_relative '../lib/webpacker/manifest_extensions'
require_relative '../lib/webpacker/helper_extensions'
require_relative '../lib/rails/engine_extensions'
require_relative '../lib/action_dispatch/remote_ip_extensions'
require_relative '../lib/active_record/database_tasks_extensions'
require_relative '../lib/active_record/batches'
require_relative '../lib/simple_navigation/item_extensions'

View file

@ -37,6 +37,10 @@ class Rack::Attack
authenticated_token&.id
end
def warden_user_id
@env['warden']&.user&.id
end
def unauthenticated?
!authenticated_user_id
end
@ -58,10 +62,6 @@ class Rack::Attack
end
end
Rack::Attack.safelist('allow from localhost') do |req|
req.remote_ip == '127.0.0.1' || req.remote_ip == '::1'
end
Rack::Attack.blocklist('deny from blocklist') do |req|
IpBlock.blocked?(req.remote_ip)
end
@ -141,6 +141,10 @@ class Rack::Attack
req.session[:attempt_user_id] || req.params.dig('user', 'email').presence if req.post? && req.path_matches?('/auth/sign_in')
end
throttle('throttle_password_change/account', limit: 10, period: 10.minutes) do |req|
req.warden_user_id if req.put? || (req.patch? && req.path_matches?('/auth'))
end
self.throttled_responder = lambda do |request|
now = Time.now.utc
match_data = request.env['rack.attack.match_data']

View file

@ -55,7 +55,7 @@ services:
web:
build: .
image: ghcr.io/mastodon/mastodon:v4.2.7
image: ghcr.io/mastodon/mastodon:v4.2.9
restart: always
env_file: .env.production
command: bundle exec puma -C config/puma.rb
@ -76,7 +76,7 @@ services:
streaming:
build: .
image: ghcr.io/mastodon/mastodon:v4.2.7
image: ghcr.io/mastodon/mastodon:v4.2.9
restart: always
env_file: .env.production
command: node ./streaming
@ -94,7 +94,7 @@ services:
sidekiq:
build: .
image: ghcr.io/mastodon/mastodon:v4.2.7
image: ghcr.io/mastodon/mastodon:v4.2.9
restart: always
env_file: .env.production
command: bundle exec sidekiq

View file

@ -0,0 +1,72 @@
# frozen_string_literal: true
# Mastodon is not made to be directly accessed without a reverse proxy.
# This monkey-patch prevents remote IP address spoofing when being accessed
# directly.
#
# See PR: https://github.com/rails/rails/pull/51610
# In addition to the PR above, it also raises an error if a request with
# `X-Forwarded-For` or `Client-Ip` comes directly from a client without
# going through a trusted proxy.
# rubocop:disable all -- This is a mostly vendored file
module ActionDispatch
class RemoteIp
module GetIpExtensions
def calculate_ip
# Set by the Rack web server, this is a single value.
remote_addr = ips_from(@req.remote_addr).last
# Could be a CSV list and/or repeated headers that were concatenated.
client_ips = ips_from(@req.client_ip).reverse!
forwarded_ips = ips_from(@req.x_forwarded_for).reverse!
# `Client-Ip` and `X-Forwarded-For` should not, generally, both be set. If they
# are both set, it means that either:
#
# 1) This request passed through two proxies with incompatible IP header
# conventions.
#
# 2) The client passed one of `Client-Ip` or `X-Forwarded-For`
# (whichever the proxy servers weren't using) themselves.
#
# Either way, there is no way for us to determine which header is the right one
# after the fact. Since we have no idea, if we are concerned about IP spoofing
# we need to give up and explode. (If you're not concerned about IP spoofing you
# can turn the `ip_spoofing_check` option off.)
should_check_ip = @check_ip && client_ips.last && forwarded_ips.last
if should_check_ip && !forwarded_ips.include?(client_ips.last)
# We don't know which came from the proxy, and which from the user
raise IpSpoofAttackError, "IP spoofing attack?! " \
"HTTP_CLIENT_IP=#{@req.client_ip.inspect} " \
"HTTP_X_FORWARDED_FOR=#{@req.x_forwarded_for.inspect}"
end
# NOTE: Mastodon addition to make sure we don't get requests from a non-trusted client
if @check_ip && (forwarded_ips.last || client_ips.last) && !@proxies.any? { |proxy| proxy === remote_addr }
raise IpSpoofAttackError, "IP spoofing attack?! client #{remote_addr} is not a trusted proxy " \
"HTTP_CLIENT_IP=#{@req.client_ip.inspect} " \
"HTTP_X_FORWARDED_FOR=#{@req.x_forwarded_for.inspect}"
end
# We assume these things about the IP headers:
#
# - X-Forwarded-For will be a list of IPs, one per proxy, or blank
# - Client-Ip is propagated from the outermost proxy, or is blank
# - REMOTE_ADDR will be the IP that made the request to Rack
ips = forwarded_ips + client_ips
ips.compact!
# If every single IP option is in the trusted list, return the IP that's
# furthest away
filter_proxies([remote_addr] + ips).first || ips.last || remote_addr
end
end
end
end
ActionDispatch::RemoteIp::GetIp.prepend(ActionDispatch::RemoteIp::GetIpExtensions)
# rubocop:enable all

View file

@ -17,7 +17,7 @@ module Mastodon
end
def default_prerelease
'alpha.3'
'alpha.4'
end
def prerelease

View file

@ -56,7 +56,7 @@ describe Rack::Attack, type: :request do
end
def throttle_count
described_class.cache.read("#{counter_prefix}:#{throttle}:#{remote_ip}") || 0
described_class.cache.read("#{counter_prefix}:#{throttle}:#{discriminator}") || 0
end
def counter_prefix
@ -64,11 +64,12 @@ describe Rack::Attack, type: :request do
end
def increment_counter
described_class.cache.count("#{throttle}:#{remote_ip}", period)
described_class.cache.count("#{throttle}:#{discriminator}", period)
end
end
let(:remote_ip) { '1.2.3.5' }
let(:discriminator) { remote_ip }
describe 'throttle excessive sign-up requests by IP address' do
context 'when accessed through the website' do
@ -149,4 +150,30 @@ describe Rack::Attack, type: :request do
it_behaves_like 'throttled endpoint'
end
describe 'throttle excessive password change requests by account' do
let(:user) { Fabricate(:user, email: 'user@host.example') }
let(:throttle) { 'throttle_password_change/account' }
let(:limit) { 10 }
let(:period) { 10.minutes }
let(:request) { -> { put path, headers: { 'REMOTE_ADDR' => remote_ip } } }
let(:path) { '/auth' }
let(:discriminator) { user.id }
before do
sign_in user, scope: :user
# Unfortunately, devise's `sign_in` helper causes the `session` to be
# loaded in the next request regardless of whether it's actually accessed
# by the client code.
#
# So, we make an extra query to clear issue a session cookie instead.
#
# A less resource-intensive way to deal with that would be to generate the
# session cookie manually, but this seems pretty involved.
get '/'
end
it_behaves_like 'throttled endpoint'
end
end

View file

@ -309,6 +309,19 @@ RSpec.describe NotifyService do
expect(subject.filter?).to be false
end
end
context 'when the sender is mentioned in an unrelated message chain' do
before do
original_status = Fabricate(:status, visibility: :direct)
intermediary_status = Fabricate(:status, visibility: :direct, thread: original_status)
notification.target_status.update(thread: intermediary_status)
Fabricate(:mention, status: original_status, account: notification.from_account)
end
it 'returns true' do
expect(subject.filter?).to be true
end
end
end
end
end