* Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService
ActivityPub::FetchRemoteAccountService is kept as a wrapper for when the actor is
specifically required to be an Account
* Refactor SignatureVerification to allow non-Account actors
* fixup! Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService
* Refactor ActivityPub::FetchRemoteKeyService to potentially return non-Account actors
* Refactor inbound ActivityPub payload processing to accept non-Account actors
* Refactor inbound ActivityPub processing to accept activities relayed through non-Account
* Refactor how Account key URIs are built
* Refactor Request and drop unused key_id_format parameter
* Rename ActivityPub::Dereferencer `signature_account` to `signature_actor`
* Add a more descriptive PrivateNetworkAddressError exception class
* Remove unnecessary exception class to rescue clause
* Remove unnecessary include to JsonLdHelper
* Give more neutral error message when too many webfinger redirects
* Remove unnecessary guard condition
* Rework how “ActivityPub::FetchRemoteAccountService” handles errors
Add “suppress_errors” keyword argument to avoid raising errors in
ActivityPub::FetchRemoteAccountService#call (default/previous behavior).
* Rework how “ActivityPub::FetchRemoteKeyService” handles errors
Add “suppress_errors” keyword argument to avoid raising errors in
ActivityPub::FetchRemoteKeyService#call (default/previous behavior).
* Fix Webfinger::RedirectError not being a subclass of Webfinger::Error
* Add suppress_errors option to ResolveAccountService
Defaults to true (to preserve previous behavior). If set to false,
errors will be raised instead of caught, allowing the caller to be
informed of what went wrong.
* Return more precise error when failing to fetch account signing AP payloads
* Add tests
* Fixes
* Refactor error handling a bit
* Fix various issues
* Add specific error when provided Digest is not 256 bits of base64-encoded data
* Please CodeClimate
* Improve webfinger error reporting
- `.env.production.sample`:
Our sample config file is very different from upstream since it is much more
complete. Upstream added documentation for a few env variables.
Copied the new variables/documentation from upstream.
- `app/lib/feed_manager.rb`:
Upstream added a timeline type (hashtags), while glitch-soc already had an
extra one (direct messages). Not really a conflict but textually close
changes.
Ported upstream's changes.
- `app/models/custom_emoji.rb`:
Upstream upped the custom emoji size limit, while glitch-soc had configurable
limits.
Upped the default limits accordingly.
- `streaming/index.js`:
Upstream reworked how hastags were normalized. Minor conflict due to
glitch-soc's handling of instance-local posts.
Ported upstream's changes.
Conflicts:
- `app/controllers/admin/base_controller.rb`:
Minor conflict caused by glitch-soc's theming system.
- `app/javascript/mastodon/initial_state.js`:
Minor conflict caused by glitch-soc making use of max_toot_chars.
- `app/models/form/admin_settings.rb`:
Minor conflict caused by glitch-soc's theming system.
- `app/models/trends.rb`:
Minor conflict caused by glitch-soc having more granular
notification settings for trends.
- `app/views/admin/accounts/index.html.haml`:
Minor conflict caused by glitch-soc's theming system.
- `app/views/admin/instances/show.html.haml`:
Minor conflict caused by glitch-soc's theming system.
- `app/views/layouts/application.html.haml`:
Minor conflict caused by glitch-soc's theming system.
- `app/views/settings/preferences/notifications/show.html.haml`:
Minor conflict caused by glitch-soc having more granular
notification settings for trends.
- `config/navigation.rb`:
Minor conflict caused by glitch-soc having additional
navigation items for the theming system while upstream
slightly changed every line.
Conflicts:
- `.github/workflows/build-image.yml`:
Fix erroneous deletion in a previous merge.
- `Gemfile`:
Conflict caused by glitch-soc-only hCaptcha dependency
- `app/controllers/auth/sessions_controller.rb`:
Minor conflict due to glitch-soc's theming system.
- `app/controllers/filters_controller.rb`:
Minor conflict due to glitch-soc's theming system.
- `app/serializers/rest/status_serializer.rb`:
Minor conflict due to glitch-soc having an extra `local_only` property
* Add model for custom filter keywords
* Use CustomFilterKeyword internally
Does not change the API
* Fix /filters/edit and /filters/new
* Add migration tests
* Remove whole_word column from custom_filters (covered by custom_filter_keywords)
* Redesign /filters
Instead of a list, present a card that displays more information and handles
multiple keywords per filter.
* Redesign /filters/new and /filters/edit to add and remove keywords
This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.
* Add /api/v2/filters to edit filter with multiple keywords
Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
`keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`
API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
`keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
`keywords_attributes` can also be passed to edit, delete or add keywords in
one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword
* Change from `irreversible` boolean to `action` enum
* Remove irrelevent `irreversible_must_be_within_context` check
* Fix /filters/new and /filters/edit with update for filter_action
* Fix Rubocop/Codeclimate complaining about task names
* Refactor FeedManager#phrase_filtered?
This moves regexp building and filter caching to the `CustomFilter` class.
This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.
* Perform server-side filtering and output result in REST API
* Fix numerous filters_changed events being sent when editing multiple keywords at once
* Add some tests
* Use the new API in the WebUI
- use client-side logic for filters we have fetched rules for.
This is so that filter changes can be retroactively applied without
reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
(e.g. network error, or initial timeline loading)
* Minor optimizations and refactoring
* Perform server-side filtering on the streaming server
* Change the wording of filter action labels
* Fix issues pointed out by linter
* Change design of “Show anyway” link in accordence to review comments
* Drop “irreversible” filtering behavior
* Move /api/v2/filter_keywords to /api/v1/filters/keywords
* Rename `filter_results` attribute to `filtered`
* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer
* Fix systemChannelId value in streaming server
* Simplify code by removing client-side filtering code
The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
* Remove duplicate in_chosen_languages definition
* Use `DEFAULT_FIELDS_SIZE` instead of `MAX_FIELDS` to reduce code differences with upstream
* Remove duplicate annotation
* Fix incorrect cross-flavor imports
* Remove deprecated `hide_network` setting (replaced by account column)
* Remove unused KeywordMutesHelper
* Remove trailing spaces
* Remove commit_hash from InstancePresenter, as it has been unused since mid-2017
Conflicts:
- `app/services/remove_status_service.rb`:
Conflict due to glitch-soc having extra code for a proper direct visibility
timeline, in a part of the code upstream refactored.
Restored glitch-soc's extra code in the refactored bit.
Conflicts:
- `package.json`:
Not really a conflict, upstream updated a dependency textually adjacent to a
glitch-soc-only one.
Updated the dependency as upstream did.
* Change RSS feeds
- Use date and time for titles instead of ellipsized text
- Use full content in body, even when there is a content warning
- Use media extensions
* Change feed icons and add width and height attributes to custom emojis
* Fix custom emoji animate on hover breaking
* Fix tests
Conflicts:
- `app/javascript/packs/admin.js`:
Conflicts due to glitch-soc's theming system.
Upstream changes have been ported to `app/javascript/core/admin.js`
- `app/models/trends/statuses.rb`:
Minor conflict due to glitch-soc's option to allow CWed toots in trends.
Ported upstream changes.
Conflicts:
- `package.json`:
Not a real conflict, upstream dependency updated textually too close to a
glitch-soc-only dependency.
Updated the upstream dependency.
* Fix PeerTube videos appearing with an erroneous “Edited at” marker
PeerTube videos have an `updated` field equal to `published`.
When processing an incoming activity that has the same value for `updated` and
`published`, assume this doesn't represent an actual edit.
* Please CodeClimate
* Fix error responses in `from` search prefix (addresses mastodon/mastodon#17941)
Using unsupported prefixes now reports a 422; searching for posts from an
account the instance is not aware of reports a 404. TODO: The UI for this
on the front end is abysmal.
Searching `from:username@domain` now succeeds when `domain` is the local
domain; searching `from:@username(@domain)?` now works as expected.
* Remove unused methods on new Error classes as they are not being used
Currently when `raise`d there are error messages being supplied, but
this is not actually being used. The associated `raise`s have been
edited accordingly.
* Remove needless comments
* Satisfy rubocop
* Try fixing tests being unable to find AccountFindingConcern methods
* Satisfy rubocop
* Simplify `from` prefix logic
This incorporates @ClearlyClaire's suggestion (see
https://github.com/mastodon/mastodon/pull/17963#pullrequestreview-933986737).
Accepctable account strings in `from:` clauses are more lenient than
before this commit; for example, `from:@user@example.org@asnteo +cat`
will not error, and return posts by @user@example.org containing the
word "cat". This is more consistent with how Mastodon matches mentions
in statuses. In addition, `from` clauses will not be checked for
syntatically invalid usernames or domain names, simply 404ing when
`Account.find_remote!` raises ActiveRecord::NotFound.
New code for this PR that is no longer used has been removed.
Conflicts:
- `Gemfile.lock`:
Not a real conflict, just an upstream dependency udpated
textually too close to a glitch-soc-only dependency.
Updated dependencies like upstream.
- `app/controllers/settings/preferences_controller.rb`:
Upstream added settings where we had extra glitch-soc-specific settings.
Added upstream's new settings.
- `app/models/user.rb`:
Upstream added settings where we had extra glitch-soc-specific settings.
Added upstream's new settings.
- `config/i18n-tasks.yml`:
Not a real conflict, just a new upstream line too textually close to
a glitch-soc-only line.
Ported upstream's change.
* Change e-mail notifications to only be sent when recipient is offline
Change the default for follow and mention notifications back on
* Add preference to always send e-mail notifications
* Change wording
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
Upstream removed this file, while glitch-soc had changes to deal with
its theming system.
Removed the file like upstream did.
Conflicts:
- `app/lib/formatter.rb`:
Upstream completely refactored the formatting code and removed that file,
while glitch-soc had code for Markdown and HTML toots.
Took upstream code, glitch-soc changes will be re-implemented on top of the
refactored classes in a later commit.
- `app/models/status.rb`:
Upstream refactored status edit handling and moved code to
`app/models/concerns/status_snapshot_concern.rb`.
Applied glitch-soc's changes to that file.
- `app/serializers/activitypub/note_serializer.rb`:
Not really a conflict, just a line added too close to one modified by
glitch-soc.
Applied upstream changes while keeping the glitch-soc-modified one.
- `app/services/update_status_service.rb`:
Not really a conflict, upstream modified a line adjacent to one added by
glitch-soc.
Applied upstream changes while keeping the glitch-soc line.
- `app/views/statuses/_simple_status.html.haml`:
Upstream refactored formatting, glitch-soc changed the markup slightly.
Applied upstream changes.
- `spec/lib/formatter_spec.rb`:
Upstream completely refactored the formatting code and removed that file,
while glitch-soc had code for Markdown and HTML toots.
Took upstream code, glitch-soc changes will be re-implemented on top of the
refactored classes in a later commit.
* Fix edits with no actual changes being allowed locally
* Fix edits with no actual changes being allowed through ActivityPub
* Fix false positive changes caused by description processing in model
* Fix not recording poll expiration update
* Fix test
* Revert changes to ProcessStatusUpdateService
* Various fixes and improvements
* Fix code style issues
* Various changes and improvements
* Add guard clause
Conflicts:
- `app/helpers/accounts_helper.rb`:
Upstream fixed an issue that glitch-soc did not have, in a place with
glitch-soc changes to hide the number of followers.
Kept glitch-soc's code.
Conflicts:
- `app/models/status.rb`:
Upstream updated media and edit-related code textually close to glitch-soc
additions (local-only and content-type).
Ported upstream changes.
- `app/models/status_edit.rb`:
Upstream changes textually close to glitch-soc additions (content-type).
Ported upstream changes.
- `app/serializers/activitypub/note_serializer.rb`:
Upstream changed how media attachments are handled. Not really a conflict,
but textually close to glitch-soc additions (directMessage attribute).
Ported upstream changes.
- `app/services/remove_status_service.rb`:
Upstream changed how media attachments are handled. Not really a conflict,
but textually close to glitch-soc additions (DM timeline).
Ported upstream changes.
- `app/services/update_status_service.rb`:
Upstream fixed an issue with language selection. Not really a conflict,
but textually close to glitch-soc additions (content-type).
Ported upstream changes.
- `db/schema.rb`:
Upstream added columns to the `status_edits` table, the conflict is because
of an additional column (`content-type`) in glitch-soc.
Ported upstream changes.
- `package.json`:
Upstream dependency (express) textually adjacent to a glitch-soc-specific one
(favico.js) got updated.
Updated it as well.
* Change how changes to media attachments are stored for edits
Fix not being able to re-order media attachments
* Fix not broadcasting updates when polls/media is changed through ActivityPub
* Various fixes and improvements
* Update app/models/report.rb
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Add tracking of media attachment description changes
* Change poll in status edit to have a structure closer to the real one
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Change design of federation pages in admin UI
* Fix query performance in instance media attachments measure
* Fix reblogs being included in instance languages dimension
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
Conflicts due to us having more user settings and upstream dropping
`hide_network` (to replace it with an account attribute, properly migrated).
Dropped `hide_network` like upstream.
- `app/lib/user_settings_decorator.rb`:
Conflicts due to us having more user settings and upstream dropping
`hide_network` (to replace it with an account attribute, properly migrated).
Dropped `hide_network` like upstream.
- `app/models/status.rb`:
Conflict because of slight change in how glitch-soc handles the scope to
filter out local-only posts for anonymous viewers.
Took upstream's changes and re-applied glitch-soc's change.
- `app/models/user.rb`:
Conflicts due to us having more user settings and upstream dropping
`hide_network` (to replace it with an account attribute, properly migrated).
Dropped `hide_network` like upstream.
- `app/views/directories/index.html.haml`:
Conflict because upstream redesigned that page while glitch-soc had a minor
change to support hiding the number of followers.
Ported glitch-soc's change on top of upstream's redesign.
Additional changes:
- `app/models/account_statuses_filter.rb`:
See change to `app/models/status.rb`.
* Add `/api/v1/accounts/familiar_followers` to REST API
* Change hide network preference to be stored consistently for local and remote accounts
* Add dummy classes to migration
* Apply suggestions from code review
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/models/account.rb`:
Not a real conflict, just upstream getting rid of unused constants too close
to glitch-soc-specific contents.
Removed unused constants like upstream did.
- `app/models/trends.rb`:
Conflict because glitch-soc disabled email notifications for trending links.
Upstream has refactored this quite a bit and added trending posts.
Took upstream code, but disabling the extra trending stuff will come in
another commit.
- `app/views/admin/trends/links/index.html.haml`:
Conflict due to glitch-soc's theming system.
Ported upstream changes accordingly.
* Add trending statuses
* Fix dangling items with stale scores in localized sets
* Various fixes and improvements
- Change approve_all/reject_all to approve_accounts/reject_accounts
- Change Trends::Query methods to not mutate the original query
- Change Trends::Query#skip to offset
- Change follow recommendations to be refreshed in a transaction
* Add tests for trending statuses filtering behaviour
* Fix not applying filtering scope in controller
Conflicts:
- `.github/workflows/build-image.yml`:
Upstream changed the workflow a bit.
Conflict comes from us pushing to ghcr while upstream pushes to dockerhub.
Ported the upstream changes while still pushing to ghcr.
Conflicts:
- `app/models/media_attachment.rb`:
Upstream moved hardcoded values around, while in glitch-soc they are
configurable.
Moved them like upstream, but keeping glitch-soc's ability to configure
them through env vars.
Video files with variable framerates are converted to constant framerate videos
and the output framerate picked by ffmpeg is based on the original file's
container framerate (which can be different from the average framerate).
This means that an input video with variable framerate with about 30 frames per
second on average, but a maximum of 120 fps will be converted to a constant 120
fps file, which won't be processed by other Mastodon servers.
This commit changes it so that input files with VFR and a maximum framerate
above the framerate threshold are converted to VFR files with the maximum frame
rate enforced.
Conflicts:
- `db/schema.rb`:
Conflict due to glitch-soc adding the `content_type` column on status edits
and thus having a different schema version number.
Solved by taking upstream's schema version number, as it is higher than
glitch-soc's.
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
Upstream renamed some helper functions that were used in a part of the
settings page which glitch-soc slightly changed the layout of.
Ported the change.
* Fix structured data parsing from links choking on bad data
- Fix og:url meta tag being prioritized over canonical link tag
- Fix structured data parsing choking on commented-out CDATA declarations
- Fix HTML entities in title, description, provider_name, author_name
- Change structured data parsing to attempt every JSON-LD script tag
* Remove unnecessary slash escapes from CDATA regex pattern
* Fix Sidekiq warnings about JSON serialization
This occurs on every symbol argument we pass, and every symbol key in hashes,
because Sidekiq expects strings instead.
See https://github.com/mperham/sidekiq/pull/5071
We do not need to change how workers parse their arguments because this has
not changed and we were already converting to symbols adequately or using
`with_indifferent_access`.
* Set Sidekiq to raise on unsafe arguments in test mode
In order to more easily catch issues that would produce warnings in production
code.
Conflicts:
- `app/lib/activitypub/activity/create.rb`:
Upstream refactored how `Create` activities are handled and how values are
extracted from `Create`d objects. This conflicted with how glitch-soc
supported the `directMessage` flag to explicitly distinguish between
limited and direct messages.
Ported glitch-soc's changes to latest upstream changes.
- `app/services/fan_out_on_write_service.rb`:
Upstream largely refactored that file and changed some of the logic.
This conflicted with glitch-soc's handling of the direct timeline and
the options to allow replies and boosts in public feeds.
Ported those glitch-soc changes on top of latest upstream changes.
- `app/services/process_mentions_service.rb`:
Upstream refactored to move mention-related ActivityPub deliveries to
`ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check
to not send local-only toots to remote mentioned users.
Took upstream's version, as the check is not needed anymore, since it is
performed at the `ActivityPub::DeliveryWorker` call site already.
- `app/workers/feed_insert_worker.rb`:
Upstream added support for `update` toot events, while glitch-soc had
support for an extra timeline support, `direct`.
Ported upstream changes and extended them to the `direct` timeline.
Additional changes:
- `app/lib/activitypub/parser/status_parser.rb`:
Added code to handle the `directMessage` flag and take it into account
to compute visibility.
- `app/lib/feed_manager.rb`:
Extended upstream's support of `update` toot events to glitch-soc's
`direct` timeline.
* Add support for editing for published statuses
* Fix references to stripped-out code
* Various fixes and improvements
* Further fixes and improvements
* Fix updates being potentially sent to unauthorized recipients
* Various fixes and improvements
* Fix wrong words in test
* Fix notifying accounts that were tagged but were not in the audience
* Fix mistake
Conflicts:
- `app/controllers/accounts_controller.rb`:
Upstream introduced support for private pinned toots, but glitch-soc's query
was a bit different as it filtered out local-only toots.
Used upstream's query, while adding local-only filtering back.
- `app/controllers/activitypub/collections_controller.rb`:
Same thing with regards to local-only posts.
- `app/validators/status_pin_validator.rb`:
Not a real conflict, but the line below was different in glitch-soc due to
the configurable pinned toots limit.
Conflicts:
- `README.md`:
We have completely different contents. Kept our version.
- `package.json`:
Not a real conflict, just an upstream dependency udpated
textually too close to a glitch-soc-only dependency.
Updated dependencies like upstream.
- `streaming/index.js`:
Conflict due to code style changes on parts that were
modified in glitch-soc to handle local-only toots.
Changed style according to upstream.
Conflicts:
- `.env.production.sample`:
Copied upstream changes.
- `app/controllers/settings/identity_proofs_controller.rb`:
Minor conflict due to glitch-soc's extra “enable_keybase” setting.
Upstream removed keybase support altogether, so did the same.
- `app/controllers/well_known/keybase_proof_config_controller.rb`:
Minor conflict due to glitch-soc's extra “enable_keybase” setting.
Upstream removed keybase support altogether, so did the same.
- `lib/mastodon/statuses_cli.rb`:
Minor conflict due to an optimization that wasn't shared between
the two versions. Copied upstream's version.
Conflicts:
- `app/views/admin/tags/index.html.haml`:
Removed upstream while it had changes in glitch-soc to accomodate for the
theming system.
Additional changes to accomodate for the theming system:
- `app/views/admin/trends/links/preview_card_providers/index.html.haml`
- `app/views/admin/trends/links/index.html.haml`
- `app/views/admin/trends/tags/index.html.haml`
- `app/views/admin/tags/show.html.haml`
* Add trending links
* Add overriding specific links trendability
* Add link type to preview cards and only trend articles
Change trends review notifications from being sent every 5 minutes to being sent every 2 hours
Change threshold from 5 unique accounts to 15 unique accounts
* Fix tests
For some reason, some misconfigured servers return an empty document when
queried over webfinger. Since an empty document does not lead to a parse
error, the error is not caught properly and triggers uncaught exceptions
later on.
This PR fixes that by immediately erroring out with `Webfinger::Error` on
getting an empty response.
Conflicts:
- `app/controllers/admin/dashboard_controller.rb`:
Upstream completely redesigned the admin dashboard.
glitch-soc tracked extra features, but that list is
gone.
Followed upstram.
- `app/views/admin/dashboard/index.html.haml`
Upstream completely redesigned the admin dashboard.
glitch-soc tracked extra features, but that list is
gone.
Followed upstram.
Conflicts:
- `app/models/custom_emoji.rb`:
Slight refactor upstream, next to a line that was different in glitch-soc
because of our local configurable limits on custom emoji size.
Ported upstream changes.
- `yarn.lock`:
Not really a conflict, upstream dependency textually too close to a
glitch-soc-only dependency.
Updated upstream dependency as upstream.
* Switch from unmaintained paperclip to kt-paperclip
* Drop some compatibility monkey-patches not required by kt-paperclip
* Drop media spoof check monkey-patching
It's broken with kt-paperclip and hopefully it won't be needed anymore
* Fix regression introduced by paperclip 6.1.0
* Do not rely on pathname to call FastImage
* Add test for ogg vorbis file with cover art
* Add audio/vorbis to the accepted content-types
This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…
* Restore missing for_as_default method
* Refactor Attachmentable concern and delay Paperclip's content-type spoof check
Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.
* Please CodeClimate
* Add audio/vorbis to the unreliable set
It doesn't correspond to a file format and thus has no extension associated.
Conflicts:
- `streaming/index.js`:
Filtering code for streaming notifications has been refactored upstream, but
glitch-soc had similar code for local-only toots in the same places.
Ported upstream changes, but did not refactor local-only filtering.
Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream-updated dependency (redis) textually too
close to glitch-soc-only dependecy.
Updated redis gem like upstream did.
The auto-linking code basically rewrote the whole string escaping non-ascii
characters in an inefficient way, and building a full character offset map
between the unescaped and escaped texts before sending the contents to
TwitterText's extractor.
Instead of doing that, this commit changes the TwitterText regexps to include
valid IRI characters in addition to valid URI characters.
* Fix Delete and Create-related locks expiring too fast
Fixes#16238
By default, RedisLock expires after 10 seconds, which may not be enough to
process statuses, especially when those have attached media files.
This commit extends those 10 seconds to 15 minutes, which should be plenty
enough to handle any status, while being short enough to not waste many
sidekiq job retries in the exceedingly rare case in which a sidekiq process
would crash when processing a `Create` or `Delete`.
* Fix other RedisLock autorelease durations
Fixes#15645
- things that only perform a few simple database queries (e.g. finding and
saving a record) have been left unchanged, so they'll still use the default
10s duration
- things that perform significantly more complex database queries have been
changed to a 5 minutes timeout
- things that perform multiple HTTP queries have been changed to a 15 minutes
timeout
- `app/views/statuses/_simple_status.html.haml`:
Small markup change in glitch-soc, on a line that has been modified by
upstream. Ported upstream changes.
If a status with a hashtag becomes very popular, it stands to
reason that the hashtag should have a chance at trending
Fix no stats being recorded for hashtags that are not allowed
to trend, and stop ignoring bots
Remove references to hashtags in profile directory from the code
and the admin UI
* Fix media processing getting stuck on too much stdin/stderr
See thoughtbot/terrapin#5
* Remove dependency on paperclip-av-transcoder gem
* Remove dependency on streamio-ffmpeg gem
* Disable stdin on ffmpeg process
* Add tests
* Ensure deleted statuses are marked as such
* Save some redis memory by not storing URIs in delete_upon_arrival values
* Avoid possible race condition when processing incoming Deletes
* Avoid potential duplicate Delete forwards
* Lower lock durations to reduce issues in case of hard crash of the Rails process
* Check for `lock.aquired?` and improve comment
* Refactor RedisLock usage in app/lib/activitypub
* Fix using incorrect or non-existent sender for relaying Deletes
Conflicts:
- `README.md`:
Upstream updated copyright year, we don't mention it so kept our version.
- `app/controllers/admin/dashboard_controller.rb`:
Not really a conflict, upstream change (removing the spam checker) too close
to glitch-soc changes. Ported upstream changes.
- `app/models/form/admin_settings.rb`:
Same.
- `app/services/remove_status_service.rb`:
Same.
- `app/views/admin/settings/edit.html.haml`:
Same.
- `config/settings.yml`:
Same.
- `config/environments/production.rb`:
Not a real conflict, upstream added a default HTTP header, but we have
extra headers in glitch-soc.
Added the header.
* Update devise-two-factor to unreleased fork for Rails 6 support
Update tests to match new `rotp` version.
* Update nsa gem to unreleased fork for Rails 6 support
* Update rails to 6.1.3 and rails-i18n to 6.0
* Update to unreleased fork of pluck_each for Ruby 6 support
* Run "rails app:update"
* Add missing ActiveStorage config file
* Use config.ssl_options instead of removed ApplicationController#force_ssl
Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.
* Fix nonce directives by removing Rails 5 specific monkey-patching
* Fix fixture_file_upload deprecation warning
* Fix yield-based test failing with Rails 6
* Use Rails 6's index_with when possible
* Use ActiveRecord::Cache::Store#delete_multi from Rails 6
This will yield better performances when deleting an account
* Disable Rails 6.1's automatic preload link headers
Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.
In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.
* Switch to Rails 6.0 default config
* Switch to Rails 6.1 default config
* Do not include autoload paths in the load path
* Prepare Mastodon for zeitwerk autoloader (Rails 6)
Add inflections and rename/move a few classes.
In particular, app/lib/exceptions.rb and app/lib/sanitize_config.rb
were manually loaded while still in autoload paths.
* Add inflection for Url → URL
* Fix misuse of foreign_type
* Fix use of removed "add_template_helper"
* Use response.media_type instead of response.content_type in tests
* Fix CSV export controller test on Rails 6
Rails 6 sets a "filename*" field in the Content-Disposition header to
explicitly encode the filename as UTF-8.
This changes checks the first part of the Content-Disposition header so
it matches in both Rails 5 and Rails 6.
* Fix emoji formatting with Rails 6
* Make emoji output more idiomatic and robust
* Switch from redis-rails gem to built-in Rails redis cache storage
Conflicts:
- `app/validators/status_length_validator.rb`:
Upstream changes too close to glitch-soc MAX_CHARS changes, but not a real
conflict.
Applied upstream changes.
- `package.json`:
glitch-soc-only dependency textually too close to a dependency updated
upstream, not a real conflict.
Applied upstream changes.
* Update twitter-text from 1.14 to 3.1.0
* Disable emoji parsing
* Properly depend on twitter-text for url detection
* Fix some URLs being wrongly detected client-side
* Add test for server-side validation of non-autolinkable URLs
* Fix server-side status length counting
Conflicts:
- `app/javascript/styles/mastodon/modal.scss`:
For some reason we changed the file loading path in glitch-soc,
but now upstream has completely changed how the logo is loaded.
Applied upstream changes.
* Fix URI of repeat follow requests not being recorded
In case we receive a “repeat” or “duplicate” follow request, we automatically
fast-forward the accept with the latest received Activity `id`, but we don't
record it.
In general, a “repeat” or “duplicate” follow request may happen if for some
reason (e.g. inconsistent handling of Block or Undo Accept activities, an
instance being brought back up from the dead, etc.) the local instance thought
the remote actor were following them while the remote actor thought otherwise.
In those cases, the remote instance does not know about the older Follow
activity `id`, so keeping that record serves no purpose, but knowing the most
recent one is useful if the remote implementation at some point refers to it
by `id` without inlining it.
* Add tests
Unlike locally-issued blocks, they weren't clearing follow
relationships in both directions, follow requests or notifications.
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/models/public_feed.rb`:
Upstream refactored a bit, glitch-soc had specific code for local-only
statuses.
Updated glitch-soc's specific code accordingly.
Conflicts:
- `app/lib/feed_manager.rb`:
Not a real conflict, glitch-soc-only DM-related method
too close to changed upstream stuff.
Ported upstream changes.
- `app/services/batched_remove_status_service.rb`:
Additional logic in glitch-soc to clear DMs from timelines.
Ported upstream changes and fixed the DM TL clearing logic.
- `app/workers/scheduler/feed_cleanup_scheduler.rb`:
Additional code in glitch-soc to clear DM timelines.
Ported upstream changes.
* Delete status records by batches of 50
* Do not precompute values that are only used once
* Do not generate redis events for removal of public toots older than two weeks
* Filter reported toots a priori for polls and status deletion
* Do not process reblogs when cleaning up public timelines
As in Mastodon proper, reblogs don't appear in public TLs
* Clean the deleted account's own feed in one go
* Refactor Account#clean_feed_manager and List#clean_feed_manager
* Delete instead of destroy a few more associations
* Fix preloading
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Conflicts:
- `app/models/form/admin_settings.rb`:
New setting added upstream. Ported it.
- `app/views/statuses/_simple_status.html.haml`:
Upstream removed RTL classes. Did the same.
- `config/settings.yml`:
New setting added upstream. Ported it.
Conflicts:
- `app/services/remove_status_service.rb`:
Conflict caused by us having a distinc Direct timeline.
Ported upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.js`:
Conflict between glitch-soc's variable character limit and upstream
refactoring that part of the code.
Ported upstream changes.
Extract logic for determining ActivityPub inboxes to send deletes
to to its own class and explicitly include the person the status
replied to (even if not mentioned), people who favourited it, and
people who replied to it (though that one is still not recursive)
Conflicts:
- `.github/ISSUE_TEMPLATE/bug_report.md`:
Upstream added the `bug` label to bug reports.
Did the same.
- `app/services/fan_out_on_write_service.rb`:
Upstream put DMs back into timelines, glitch-soc was already doing it.
Ignored upstream changes.
Nginx can be configured to bypass proxy cache when a special header
is in the request. If the response is cacheable, it will replace
the cache for that request. Proxy caching of media files is
desirable when using object storage as a way of minimizing bandwidth
costs, but has the drawback of leaving deleted media files for
a configured amount of cache time. A cache buster can make those
media files immediately unavailable. This especially makes sense
when suspending and unsuspending an account.
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
Conflict caused because of glitch-soc's theming system.
Took upstream's new code and applied the theming system changes on top
of it.
- `app/controllers/concerns/two_factor_authentication_concern.rb`:
Conflict caused because of glitch-soc's theming system.
Took upstream's new code and applied the theming system changes on top
of it.
When failing to fetch the target account, the ProcessingWorker fails
as expected, but since it hasn't cleared the `move_in_progress` flag,
the next attempt at processing skips the `Move` activity altogether.
This commit changes it to clear the flag when encountering any
unexpected error on fetching the target account. This is likely to
occur because, of, e.g., a timeout, when many instances query the
same actor at the same time.
Conflicts:
- `app/controllers/follower_accounts_controller.rb`:
Conflict due to upstream changing suspension logic while
glitch-soc has an extra option to hide followers count.
Ported upstream changes.
Conflicts:
- `.github/dependabot.yml`:
Updated upstream, we deleted it to not be flooded by Depandabot.
Kept deleted.
- `Gemfile.lock`:
Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
Upstream updated the serializer to support timed mutes, while
glitch-soc added a custom API ages ago to get information that
is already available elsewhere.
Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
🤷
- `app/models/user.rb`:
New user setting added upstream while we have glitch-soc-specific
user settings. Added upstream's user setting.
- `config/settings.yml`:
Upstream added a new user setting close to a user setting we had
changed the defaults for. Added the new upstream setting.
- `package.json`:
Upstream dependency updated “too close” to a glitch-soc-specific
dependency. No real conflict. Updated the dependency.
* Add support for followers synchronization on the receiving end
Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.
* Add tests for followers synchronization on the receiving end
* Add support for follower synchronization on the sender's end
* Add tests for the sending end
* Switch from AS attributes to HTTP header
Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
receiving instance (where “receiving instance” is determined by accounts
sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor
Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.
* Add environment variable to disable followers synchronization
Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.
The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
netloc as a managed account must be managed by the same Mastodon instance
as well.
As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.
Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.
* Only set follower synchronization header on non-public statuses
This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)
* Change how ActivityPub::SynchronizeFollowersService handles follow requests
If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.
* Integrate review feeback
- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes
* Only select required fields when computing followers_hash
* Use actor URI rather than webfinger domain in synchronization endpoint
* Change hash computation to be a XOR of individual hashes
Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.
* Marginally improve followers_hash computation speed
* Further improve hash computation performances by using pluck_each
There are edge cases where requests to certain hosts timeout when
using the vanilla HTTP.rb gem, which the goldfinger gem uses. Now
that we no longer need to support OStatus servers, webfinger logic
is so simple that there is no point encapsulating it in a gem, so
we can just use our own Request class. With that, we benefit from
more robust timeout code and IPv4/IPv6 resolution.
Fix#14091
Conflicts:
- `Gemfile.lock`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
Upstream changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's theming system.
Ported upstream changes.
- `app/models/user.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc not preventing moved accounts from logging
in.
Ported upstream changes while keeping the ability for moved accounts to log
in.
- `app/policies/status_policy.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's ability to hide followers count.
Ported upstream changes.
- `app/services/process_mentions_service.rb`:
Upstream refactored and changed the logic surrounding suspended accounts.
Minor conflict due to glitch-soc's local-only toots.
Ported upstream changes.
- `package.json`:
Not a real conflict, upstream updated dependencies that were too close to
glitch-soc-only ones in the file.
* Add bell button
Fix#4890
* Remove duplicate type from post-deployment migration
* Fix legacy class type mappings
* Improve query performance with better index
* Fix validation
* Remove redundant index from notifications
Conflicts:
- app/controllers/api/v1/timelines/public_controller.rb
- app/lib/feed_manager.rb
- app/models/status.rb
- app/services/precompute_feed_service.rb
- app/workers/feed_insert_worker.rb
- spec/models/status_spec.rb
All conflicts are due to upstream refactoring feed management and us having
local-only toots on top of that. Rewrote local-only toots management for
upstream's changes.
* Check for and record reblog info atomically
Instead of using ZREVRANK to determine whether a reblog is a new reblog or not,
use ZADD's NX option to perform the check/addition option atomically.
* Replace ZREVRANK call with ZSCORE key which is more efficient
* Make tests a bit stricter
* Fix off-by-one
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
Conflict caused because we have additional code to make sure pinned
local-only toots don't get rendered on the ActivityPub endpoints.
Ported upstream changes.
Conflicts:
- `config/webpack/shared.js`:
Upstream has changed how Tesseract.js gets included and dropped a dependency.
The conflict is caused by glitch-soc having different code due to its
theming system.
Ported upstream changes.
- `lib/mastodon/version.rb`:
Upstream refactor/code style change in a place we replaced upstream's
repo URL with ours.
Ported upstram changes, keeping our repo URL.
- `yarn.lock`:
Upstream dropped dependencies, one of which was textually too close to
a glitch-soc-specific dependency. Not a real conflict.
* Add database support for list show-reply preferences
* Add backend support to read and update list-specific show_replies settings
* Add basic UI to set list replies setting
* Add specs for list replies policy
* Switch "cycling" reply policy link to a set of radio inputs
* Capitalize replies_policy strings
* Change radio button design to be consistent with that of the directory explorer
Conflicts:
- `app/controllers/accounts_controller.rb`:
Upstream change too close to a glitch-soc change related to
instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
Minor conflict due to glitch-soc's handling of Direct Messages,
merged upstream changes.
- `yarn.lock`:
Not really a conflict, caused by glitch-soc-only dependencies
being textually too close to updated upstream dependencies.
Merged upstream changes.
Follow-up to #14359
In the case of limited toots, the receiver may not be explicitly part of the
audience. If a specific user's inbox URI was specified, it makes sense to
dereference the toot from the corresponding user, instead of trying to find
someone in the explicit audience.
Conflicts:
- `app/javascript/mastodon/actions/compose.js`:
Not a “real” conflict, but change too close to a change we made to
fix the vanilla WebUI locally pushing authored local-only toots in the
public TL view.
* Fix not handling Undo on some activity types when they aren't inlined
When receiving an Undo for a non-inlined activity, try looking it up in
database using the URI. The queries are ad-hoc because we don't have a global
index of object URIs, and not all activity types are stored in database with
an index on their URI.
Announces are just statuses, and have an index on URIs, so this check can
be done efficiently.
Accepts cannot be handled at all because we don't record their URI at any
point.
Follows don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.
Likes don't have an index on URI, they have an index on the issuing account,
but the number of favs per account may be very high, so I decided not to
handle that.
Blocks don't have an index on URI, but they have an index on the issuing
account, which should make such queries largely manageable.
In all cases, if an Undo could not be handled properly, we call `delete_later!`
because that does not require us to know more than the URI of the undone
property.
* Add tests
* Make newer blocks overwrite older ones
Allows re-synchronizing block info by re-blocking and un-blocking again
when the original Undo Block has been lost.
