* Fix attachments getting processed despite failing content-type validation
* Add a restrictive ImageMagick security policy tailored for Mastodon
* Fix misdetection of MP3 files with large cover art
* Reject unprocessable audio/video files instead of keeping them unchanged
Conflicts:
- `app/models/user_settings.rb`:
Upstream added a constraint on a setting textually close
to glitch-soc-only settings.
Applied upstream's change.
- `lib/sanitize_ext/sanitize_config.rb`:
Upstream added support for the `translate` attribute on a few elements,
where glitch-soc had a different set of allowed elements and attributes.
Extended glitch-soc's allowed attributes with `translate` as upstream did.
- `spec/validators/status_length_validator_spec.rb`:
Upstream refactored to use RSpec's `instance_double` instead of `double`,
but glitch-soc had changes to tests due to configurable max toot chars.
Applied upstream's changes while keeping tests against configurable max
toot chars.
Conflicts:
- `app/views/settings/profiles/show.html.haml`:
Upstream redesigned the settings page, where glitch-soc had changes because of
the ability to set some custom limits.
Went with upstream's design while keeping our custom limits.
- `yarn.lock`:
Upstream updated dependencies textually close to a glitch-soc-only dependency.
Updated the dependnencies as well.
Conflicts:
- `package.json`:
Upstream changed various script definitions in lines surrounding the one for
`i18n:extract`, which had glitch-soc-specific changes.
Updated the scripts as upstream did, while keeping our changes to
`i18n:extract`.
Conflicts:
- `app/controllers/auth/confirmations_controller.rb`:
Upstream merged our captcha code, but there are some
conflicts due to glitch-soc's theming system.
- `app/views/admin/settings/registrations/show.html.haml`:
Upstream merged our captcha code, but there are some
conflicts due to glitch-soc's theming system.
Additional changes:
- `Gemfile`:
Upstream added hcaptcha dependency in another place in the file.
- `config/settings.yml`:
Upstream added the `captcha_enabled` setting in another place in the file.
Conflicts:
- `config/webpack/generateLocalePacks.js`:
A dependency update changed how functions are imported.
Also, some linting fixes not applicable to glitch-soc.
Conflicts:
- `.github/dependabot.yml`:
Upstream made changes, but we had removed it.
Discarded upstream changes.
- `.rubocop_todo.yml`:
Upstream regenerated the file, we had some glitch-soc-specific ignores.
- `app/models/account_statuses_filter.rb`:
Minor upstream code style change where glitch-soc had slightly different code
due to handling of local-only posts.
Updated to match upstream's code style.
- `app/models/status.rb`:
Upstream moved ActiveRecord callback definitions, glitch-soc had an extra one.
Moved the definitions as upstream did.
- `app/services/backup_service.rb`:
Upstream rewrote a lot of the backup service, glitch-soc had changes because
of exporting local-only posts.
Took upstream changes and added back code to deal with local-only posts.
- `config/routes.rb`:
Upstream split the file into different files, while glitch-soc had a few
extra routes.
Extra routes added to `config/routes/settings.rb`, `config/routes/api.rb`
and `config/routes/admin.rb`
- `db/schema.rb`:
Upstream has new migrations, while glitch-soc had an extra migration.
Updated the expected serial number to match upstream's.
- `lib/mastodon/version.rb`:
Upstream added support to set version tags from environment variables, while
glitch-soc has an extra `+glitch` tag.
Changed the code to support upstream's feature but prepending a `+glitch`.
- `spec/lib/activitypub/activity/create_spec.rb`:
Minor code style change upstream, while glitch-soc has extra tests due to
`directMessage` handling.
Applied upstream's changes while keeping glitch-soc's extra tests.
- `spec/models/concerns/account_interactions_spec.rb`:
Minor code style change upstream, while glitch-soc has extra tests.
Applied upstream's changes while keeping glitch-soc's extra tests.
Conflicts:
- `app/javascript/styles/mastodon/forms.scss`:
Conflict because we ran eslint autofix on upstream files.
- `config/initializers/content_security_policy.rb`:
Code style changes but we have a different version.
Kept our version.
- `streaming/index.js`:
Upstream fixed a typo close to glitch-soc-only code.
Applied upstream's changes.
Conflicts:
- `.github/dependabot.yml`:
Updated upstream, removed in glitch-soc to disable noise.
Kept removed.
- `CODE_OF_CONDUCT.md`:
Upstream updated to a new version of the covenant, but I have not read it
yet, so kept unchanged.
- `Gemfile.lock`:
Not a real conflict, one upstream dependency updated textually too close to
the glitch-soc only `hcaptcha` dependency.
Applied upstream changes.
- `app/controllers/admin/base_controller.rb`:
Minor conflict due to glitch-soc's theming system.
Applied upstream changes.
- `app/controllers/application_controller.rb`:
Minor conflict due to glitch-soc's theming system.
Applied upstream changes.
- `app/controllers/disputes/base_controller.rb`:
Minor conflict due to glitch-soc's theming system.
Applied upstream changes.
- `app/controllers/relationships_controller.rb`:
Minor conflict due to glitch-soc's theming system.
Applied upstream changes.
- `app/controllers/statuses_cleanup_controller.rb`:
Minor conflict due to glitch-soc's theming system.
Applied upstream changes.
- `app/helpers/application_helper.rb`:
Minor conflict due to glitch-soc's theming system.
Applied upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.jsx`:
Upstream added a highlight animation for onboarding, while we changed the
max character limit.
Applied our local changes on top of upstream's new version.
- `app/views/layouts/application.html.haml`:
Minor conflict due to glitch-soc's theming system.
Applied upstream changes.
- `stylelint.config.js`:
Upstream added ignore paths, glitch-soc had extra ignore paths.
Added the same paths as upstream.
Conflicts:
- `app/controllers/auth/setup_controller.rb`:
Upstream removed a method close to a glitch-soc theming-related method.
Removed the method like upstream did.
* Run rubocop --autocorrect on app/, config/ and lib/, also manually fix some remaining style issues
* Run rubocop --autocorrect-all on db/
* Run rubocop --autocorrect-all on `spec/` and fix remaining issues
Conflicts:
- `README.md`:
Upstream added a link to the roadmap, but we have a completely different README.
Kept ours.
- `app/models/media_attachment.rb`:
Upstream upped media attachment limits.
Updated the default according to upstream's.
- `db/migrate/20180831171112_create_bookmarks.rb`:
Upstream changed the migration compatibility level.
Did so too.
- `config/initializers/content_security_policy.rb`:
Upstream refactored this file but we have a different version.
Kept our version.
- `app/controllers/settings/preferences_controller.rb`:
Upstream completely refactored user settings storage, and glitch-soc has a
different set of settings.
The file does not directly references individual settings anymore.
Applied upstream changes.
- `app/lib/user_settings_decorator.rb`:
Upstream completely refactored user settings storage, and glitch-soc has a
different set of settings.
The file got removed entirely.
Removed it as well.
- `app/models/user.rb`:
Upstream completely refactored user settings storage, and glitch-soc has a
different set of settings.
References to individual settings have been removed from the file.
Removed them as well.
- `app/views/settings/preferences/appearance/show.html.haml`:
Upstream completely refactored user settings storage, and glitch-soc has a
different set of settings.
Applied upstream's changes and ported ours back.
- `app/views/settings/preferences/notifications/show.html.haml`:
Upstream completely refactored user settings storage, and glitch-soc has a
different set of settings.
Applied upstream's changes and ported ours back.
- `app/views/settings/preferences/other/show.html.haml`:
Upstream completely refactored user settings storage, and glitch-soc has a
different set of settings.
Applied upstream's changes and ported ours back.
- `config/settings.yml`:
Upstream completely refactored user settings storage, and glitch-soc has a
different set of settings.
In particular, upstream removed user-specific and unused settings.
Did the same in glitch-soc.
- `spec/controllers/application_controller_spec.rb`:
Conflicts due to glitch-soc's theming system.
Mostly kept our version, as upstream messed up the tests.
Conflicts:
- `app/models/status.rb`:
Upstream added lines close to a glitch-soc only line, not a real conflict.
Applied upstream's changes (added hooks) while keeping glitch-soc's changes
(`local_only` scope).
- `config/environments/production.rb`:
Upstream removed a header, while we have glitch-soc specific ones.
Removed the header removed upstream.
Conflicts:
- `README.md`:
Upstream changed their README, we have our own.
Kept ours.
- `app/helpers/application_helper.rb`:
Minor code style fix upstream, on a line that is different in glitch-soc
due to the different theming system.
Applied the code style fix to our own code.
- `app/views/settings/preferences/appearance/show.html.haml`:
Code style fix on a line next to lines exclusive to glitch-soc.
Applied upstream changes.
- `yarn.lock`:
Upstream updated a dependency textually close to a glitch-soc-only
dependency.
Updated the dependency like upstream did.
Conflicts:
- `.github/workflows/build-image.yml`:
Upstream switched to pushing to both DockerHub and GitHub Container
Repository, while glitch-soc was already pushing to the latter only.
Updated our configuration to be slightly more consistent with upstream's
naming and styling, but kept our behavior.
- `Gemfile.lock`:
Updated dependencies textually too close to glitch-soc only hcaptcha
dependency.
Updated dependencies as upstream did.
- `README.md`:
Upstream updated its README, but we have a completely different one.
Kept our README, though it probably should be reworked at some point.
- `app/views/auth/sessions/two_factor.html.haml`:
Minor style fix upstream that's on a line glitch-soc removed because
of its different theming system.
Kept our file as is.
- `spec/controllers/health_controller_spec.rb`:
This file apparently did not exist upstream, upstream created it with
different contents but it is functionally the same.
Switched to upstream's version of the file.
- `spec/presenters/instance_presenter_spec.rb`:
Upstream changed the specs around `GITHUB_REPOSITORY`, while glitch-soc
had its own code because it's a fork and does not have the same default
source URL.
Took upstream's change, but with glitch-soc's repo as the default case.
- `yarn.lock`:
Upstream dependencies textually too close to a glitch-soc only one.
Updated dependencies as upstream did.
Conflicts:
- `README.md`:
Upstream README has been changed, but we have a completely different one.
Kept our `README.md`.
- `lib/sanitize_ext/sanitize_config.rb`:
Upstream added support for more incoming HTML tags (a large subset of what
glitch-soc accepts).
Change the code style to match upstream's but otherwise do not change our
code.
- `spec/lib/sanitize_config_spec.rb`:
Upstream added support for more incoming HTML tags (a large subset of what
glitch-soc accepts).
Kept our version, since the tests are mostly glitch-soc's, except for cases
which are purposefuly different.
Conflicts:
- `.prettierignore`:
Upstream added a line at the end of the file, while glitch-soc had its own
extra lines.
Took upstream's change.
- `CONTRIBUTING.md`:
We have our custom CONTRIBUTING.md quoting upstream. Upstream made changes.
Ported upstream changes.
- `app/controllers/application_controller.rb`:
Upstream made code style changes in a method that is entirely replaced
in glitch-soc.
Ignored the change.
- `app/models/account.rb`:
Code style changes textually close to glitch-soc-specific changes.
Ported upstream changes.
- `lib/sanitize_ext/sanitize_config.rb`:
Upstream code style changes.
Ignored them.
Conflicts:
- `.github/dependabot.yml`:
Upstream made changes while we have dropped this file.
Keep the file deleted.
- `.prettierignore`:
Upstream made changes at the end of the file, where we
had our extra lines.
Just moved our extra lines back at the end.
- `app/serializers/initial_state_serializer.rb`:
Upstream code style changes.
Applied them.
- `app/services/backup_service.rb`:
Upstream code style changes.
Applied them.
Conflicts:
- `README.md`:
Minor upstream change, our README is completely different.
Kept ours.
- `lib/tasks/assets.rake`:
glitch-soc has extra code to deal with its theming system,
upstream changed a line that exists in glitch-soc.
Applied upstream changes.
Conflicts:
- `config/i18n-tasks.yml`:
Upstream added new ignored strings, glitch-soc has extra ignored strings
because of the theming system.
Added upstream's changes.
* Validate internal actor
* Use “internal.actor” by default for the server actor username
* Fix instance actor username on the fly if it includes ':'
* Change actor name from internal.actor to mastodon.internal
* Do not offer to mark statuses as sensitive if there is no undeleted status with media attachments
* Fix crash when marking statuses as sensitive while some statuses are deleted
Fixes#21910
* Fix multiple strikes being created for a single report when selecting “Mark as sensitive”
* Add tests
Conflicts:
- `app/models/status.rb`:
Minor upstream refactor moved hook definitions around,
and glitch-soc has an extra `before_create`.
Moved the `before_create` accordingly.
- `app/services/batched_remove_status_service.rb`:
Minor upstream refactor changed a block in which glitch-soc
had one extra call to handle direct timelines.
Adapted changes to keep glitch-soc's extra call.
* fix(status): remove send usage for private unlink_from_conversations
- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service
* fix(feeds_vacuum): replace find_in_batches with in_batches
because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
* Make autosuggest for mentions return followed accounts first
This makes it so that (when elasticsearch is disabled) when a user types '@foo' in the compose box, they are first going to get accounts they follow ordered by the ranking algorithm, and then second they will get accounts they do not follow, also ordered by the ranking algorithm.
This makes behavior more consistent with user expectation and also with results when elasticsearch is enabled.
* Fix ranking order to correct direction
* One more fixup per @gargron suggestion
* Tweak to ranking to no longer include following modifier
Conflicts:
- `README.md`:
Upstream updated its README, while we have a completely different one.
Kept our README.
- `app/controllers/concerns/web_app_controller_concern.rb`:
Conflict because of glitch-soc's theming system.
Additionally, glitch-soc has different behavior regarding moved accounts.
Ported some of the changes, but kept our overall behavior.
- `app/javascript/packs/admin.js`:
Code changes actually applied to `app/javascript/core/admin.js`
Conflicts:
- `README.md`:
Discarded upstream changes: we have our own README
- `app/controllers/follower_accounts_controller.rb`:
Port upstream's minor refactoring
* Clear sessions on password change
* Rename User::clear_sessions to revoke_access for a clearer meaning
* Add reset paassword controller test
* Use User.find instead of User.find_for_authentication for reset password test
* Use redirect and render for better test meaning in reset password
Co-authored-by: Effy Elden <effy@effy.space>
Conflicts:
- `app/models/concerns/domain_materializable.rb`:
Fixed a code style issue upstream in a PR that got merged in glitch-soc
earlier.
Changed the code to match upstream's.
* Fix trying to fetch posts from other users when fetching featured posts
* Rate-limit discovery of new subdomains
* Put a limit on recursively discovering new accounts
* Fix trying to fetch posts from other users when fetching featured posts
* Rate-limit discovery of new subdomains
* Put a limit on recursively discovering new accounts
* refactor(statuses_vacuum): remove dead code - unused
Method is not called inside class and private.
Clean up dead code.
* refactor(statuses_vacuum): make retention_period present test explicit
This private method only hides functionality.
It is best practice to be as explicit as possible.
* refactor(statuses_vacuum): improve query performance
- fix statuses_scope having sub-select for Account.remote scope by
`joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
because it is inexplicit, bad practice and even slower than normal
`.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
because in_batches delivers a full blown active record query result,
in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send
* refactor(status): simplify unlink_from_conversations
- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?
* refactor(status): searchable_by way less sub selects
These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.
This is how it was:
```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
Status Load (1.6ms) SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1 [["LIMIT", 1]]
Mention Load (1.5ms) SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
QUERY PLAN
------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.15..23.08 rows=1 width=41)
-> Seq Scan on accounts (cost=0.00..10.90 rows=1 width=8)
Filter: (domain IS NULL)
-> Index Scan using index_mentions_on_account_id_and_status_id on mentions (cost=0.15..8.17 rows=1 width=41)
Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
Filter: (NOT silent)
(6 rows)
```
This is how it is with this change:
```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
Status Load (1.7ms) SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1 [["LIMIT", 1]]
Mention Load (0.7ms) SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
QUERY PLAN
------------------------------------------------------------------------------------------------------------------
Nested Loop (cost=0.15..23.08 rows=1 width=41)
-> Seq Scan on accounts (cost=0.00..10.90 rows=1 width=8)
Filter: (domain IS NULL)
-> Index Scan using index_mentions_on_account_id_and_status_id on mentions (cost=0.15..8.17 rows=1 width=41)
Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
Filter: (NOT silent)
(6 rows)
```
When a poll is edited, we reset the poll and remove all previous
votes. However, prior to this commit, the voter count on the poll
was not reset. This leads to incorrect percentages being shown in
poll results.
Fixes#21696
* Don't allow URLs that contain non-normalized paths to be verified
This stops things like https://example.com/otheruser/../realuser where
"/otheruser" appears to be the verified URL, but the actual URL being
verified is "/realuser" due to the "/../".
Also fix a test to use 'https', so it is testing the right thing, now
that since #20304 https is required.
* missing do
Conflicts:
- `.github/workflows/build-image.yml`:
Upstream changed how docker images were built, including how
they were cached.
I don't know much about it, so applied upstream's changes.
- `app/controllers/admin/domain_blocks_controller.rb`:
The feature, that was in glitch-soc, got backported upstream.
It also had a few fixes upstream, so those have been ported!
- `app/javascript/packs/admin.js`:
Glitch-soc changes have been backported upstream. As a result,
some code from `app/javascript/core/admin.js` got added upstream.
Kept our version since our shared Javascript already has that feature.
- `app/models/user.rb`:
Upstream added something to distinguish unusable and unusable-because-moved
accounts, while glitch-soc considers moved accounts usable.
Took upstream's code for `functional_or_moved?` and made `functional?`
call it.
- `app/views/statuses/_simple_status.html.haml`:
Upstream cleaned up code style a bit, on a line that we had custom changes
for.
Applied upstream's change while keeping our change.
- `config/initializers/content_security_policy.rb`:
Upstream adopted one CSP directive we already had.
The conflict is because of our files being structurally different, but the
change itself was already part of glitch-soc.
Kept our version.
Conflicts:
- `app/views/admin/announcements/edit.html.haml`:
Upstream change too close to theming-related glitch-soc change.
Ported upstream changes.
- `app/views/admin/announcements/new.html.haml`
Upstream change too close to theming-related glitch-soc change.
Ported upstream changes.
Conflicts:
- `app/models/account.rb`:
Conflict because we (glitch-soc) have disabled trending of posts without
review.
Discarded that upstream change.
- `app/views/admin/settings/discovery/show.html.haml`:
Just an extra setting in glitch-soc.
Kept that extra setting.
If `Nokogiri::HTML(value).at_xpath('//body')` fails to find the `body` element, it will return `nil`. We can guard against that with an early return. Avoids calling `children` on `Nilclass` in those cases.
Conflicts:
- `app/models/custom_emoji.rb`:
Not a real conflict, just upstream changing a line too close to
a glitch-soc-specific validation.
Applied upstream changes.
- `app/models/public_feed.rb`:
Not a real conflict, just upstream changing a line too close to
a glitch-soc-specific parameter documentation.
Applied upstream changes.
- Change verification to happen in `default` queue
- Change verification worker to only be queued if there's something to do
- Add `link` tags from metadata fields to page header of profiles
* Fix followers count not being updated when migrating follows
Fixes#19900
* Fix validation error in SynchronizeFeaturedTagsCollectionWorker
Also saves remote user's chosen case for hashtags
* Limit remote featured tags before validation
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
glitch-soc change because of having changed the default number of
available poll options.
Applied upstream's changes while keeping glitch-soc's default number of
poll options.
- `public/oops.png`:
We had a minor graphics change, probably not worth diverging from upstream.
Took upstream version.
