Commit graph

1779 commits

Author SHA1 Message Date
Eugen Rochko
c4d2c39a75
Fix being able to report otherwise inaccessible statuses (#18528) 2022-05-26 22:08:02 +02:00
Eugen Rochko
3e0e7a1cfb
Fix follower and other counters being able to go negative (#18517) 2022-05-26 20:32:48 +02:00
Claire
f13d910f3e Merge branch 'main' into glitch-soc/merge-upstream 2022-05-26 18:34:27 +02:00
Eugen Rochko
088dc0ec5a
Fix regression in tootctl search deploy caused by unloaded attribute (#18514) 2022-05-26 18:05:47 +02:00
Claire
440eb71310
Change unapproved and unconfirmed account to not be accessible in the REST API (#17530)
* Change unapproved and unconfirmed account to not be accessible in the REST API

* Change Account#searchable? to reject unconfirmed and unapproved users

* Disable search for unapproved and unconfirmed users in Account.search_for

* Disable search for unapproved and unconfirmed users in Account.advanced_search_for

* Remove unconfirmed and unapproved accounts from Account.searchable scope

* Prevent mentions to unapproved/unconfirmed accounts

* Fix some old tests for Account.advanced_search_for

* Add some Account.advanced_search_for tests for existing behaviors

* Add some tests for Account.search_for

* Add Account.advanced_search_for tests unconfirmed and unapproved accounts

* Add Account.searchable tests

* Fix Account.without_unapproved scope potentially messing with previously-applied scopes

* Allow lookup of unconfirmed/unapproved accounts through /api/v1/accounts/lookup

This is so that the API can still be used to check whether an username is free
to use.
2022-05-26 15:50:33 +02:00
Claire
25dda3061e
Fix unnecessary query on status creation (#17901) 2022-05-26 00:20:30 +02:00
Claire
3fd2aadb23
Various code cleanup (#1782)
* Remove duplicate in_chosen_languages definition

* Use `DEFAULT_FIELDS_SIZE` instead of `MAX_FIELDS` to reduce code differences with upstream

* Remove duplicate annotation

* Fix incorrect cross-flavor imports

* Remove deprecated `hide_network` setting (replaced by account column)

* Remove unused KeywordMutesHelper

* Remove trailing spaces

* Remove commit_hash from InstancePresenter, as it has been unused since mid-2017
2022-05-25 12:27:11 +02:00
Claire
22111914bf Merge branch 'main' into glitch-soc/merge-upstream 2022-05-24 13:59:21 +02:00
Claire
e5997a1956
Fix warning an account outside of a report closing all reports for that account (#18387)
* Fix warning an account outside of a report closing all reports for that account

* Make it clear what actions solve other reports

* Revert "Make it clear what actions solve other reports"

This reverts commit ad006de821f72e75480701298d13f0945b509059.
2022-05-23 20:38:29 +02:00
Eugen Rochko
a9b64b24d6
Change algorithm of tootctl search deploy to improve performance (#18463) 2022-05-22 22:16:43 +02:00
Claire
4415005de7 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/compose_form.js`:
  Conflict caused because of minor code style change upstream, while glitch-soc
  has different code for handling variable maximum chars.
  Ported the change.
- `app/serializers/initial_state_serializer.rb`:
  Conflict because glitch-soc had two extra attributes where upstream added
  one.
  Added upstream's attribute.
2022-05-16 22:29:05 +02:00
Eugen Rochko
6c699b1723
Fix preferred posting language returning unusable value in REST API (#18428) 2022-05-16 19:13:36 +02:00
Claire
b91196f4b7
Add confirmation page when importing blocked domains (#1773)
* Move glitch-soc-specific strings to glitch-soc-specific locale files

* Add confirmation page when importing blocked domains
2022-05-16 18:26:49 +02:00
Claire
040b7d37a4 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/services/remove_status_service.rb`:
  Conflict due to glitch-soc having extra code for a proper direct visibility
  timeline, in a part of the code upstream refactored.
  Restored glitch-soc's extra code in the refactored bit.
2022-05-16 09:42:32 +02:00
Levi Bard
94e98864e3
Allow import/export of instance-level domain blocks/allows (#1754)
* Allow import/export of instance-level domain blocks/allows.
Fixes #15095

* Pacify circleci

* Address simple code review feedback

* Add headers to exported CSV

* Extract common import/export functionality to
AdminExportControllerConcern

* Add additional fields to instance-blocked domain export

* Address review feedback

* Split instance domain block/allow import/export into separate pages/controllers

* Address code review feedback

* Pacify DeepSource

* Work around Paperclip::HasAttachmentFile for Rails 6

* Fix deprecated API warning in export tests

* Remove after_commit workaround
2022-05-16 09:29:01 +02:00
Claire
aa08399e6f
Fix setting for trending tags and trending links notifications being swapped (#1771) 2022-05-13 22:57:47 +02:00
Eugen Rochko
6cf57c6765
Refactor how Redis locks are created (#18400)
* Refactor how Redis locks are created

* Fix autorelease duration on account deletion lock
2022-05-13 00:02:35 +02:00
Claire
22080786df Merge branch 'main' into glitch-soc/merge-upstream 2022-05-08 21:00:49 +02:00
Claire
a53003c6f1
Fix account warnings not being recorded in audit log (#18338)
* Fix account warnings not being recorded in audit log

Fixes #18334

* Only record warnings if they are not associated to another action
2022-05-06 21:40:20 +02:00
Claire
392ca0472a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/admin.js`:
  Conflicts due to glitch-soc's theming system.
  Upstream changes have been ported to `app/javascript/core/admin.js`
- `app/models/trends/statuses.rb`:
  Minor conflict due to glitch-soc's option to allow CWed toots in trends.
  Ported upstream changes.
2022-05-01 16:52:27 +02:00
Claire
6e4d932da5
Fix possible crash when a post references an invalid media attachment (#18211) 2022-05-01 00:55:26 +02:00
Eugen Rochko
f6d35ed57d
Remove IP matching from e-mail domain blocks (#18190)
Clear out e-mail domain blocks created from automatically resolved DNS records
2022-04-29 23:27:03 +02:00
Eugen Rochko
7b0fe4aef9
Fix opening and closing Redis connections instead of using a pool (#18171)
* Fix opening and closing Redis connections instead of using a pool

* Fix Redis connections not being returned to the pool in CLI commands
2022-04-29 22:43:07 +02:00
Eugen Rochko
6476f7e4da
Change trending statuses to only show one status from each account (#18181)
Calculate trends in temporary sets to avoid having to manage items
that go below the decay threshold while not having any moments
where a half-processed set is accessible to end-users
2022-04-29 22:42:42 +02:00
Eugen Rochko
6726d2933a
Change half-life of trending status scores from 6 hours to 2 hours (#18182) 2022-04-29 11:46:59 +02:00
Claire
f23f784f18 Merge branch 'main' into glitch-soc/merge-upstream 2022-04-28 18:16:42 +02:00
Eugen Rochko
3917353645
Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Claire
fc1dd81ea5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not a real conflict, upstream dependency updated textually too close to a
  glitch-soc-only dependency.
  Updated the upstream dependency.
2022-04-27 08:48:21 +02:00
Claire
33cd80d69c
Fix instance actor being incorrectly created when running migrations (#18109)
* Add migration test about instance actor key

* Fix old migration

* Work around incorrect database state
2022-04-26 21:22:09 +02:00
Claire
002d2729fb Merge branch 'main' into glitch-soc/merge-upstream 2022-04-25 17:09:32 +02:00
Jeong Arm
ea38327816
Let votes statuses are also searchable (#18070) 2022-04-23 21:47:27 +02:00
Claire
5c808ee0de Revert support from loading Home timeline from database
Unfortunately, the database query could turn out very inefficient and I did not
manage to find a way to improve that. Furthermore, there were still behavior
inconsistencies between fetching the timeline from Redis and fetching it from
Postgres.
2022-04-19 15:06:38 +02:00
Claire
ec4a8d81b1 Revert DM support in HomeFeed#from_database
Fixes #1746

Queries could get prohibitively expensive.
2022-04-16 21:28:46 +02:00
Claire
5db3a14388
Fix loading Home TL from database not respecting min_id and not including DMs (#1744)
* Rework tests

* Add tests

* Fix HomeFeed#get with min_id fetching from database

* Minor code cleanup and optimizations

* Add tests

* Take DMs into account when fetching home TL from database

* Fix not listing own DMs in Home timeline

* Add tests

* Please CodeClimate
2022-04-14 17:44:21 +02:00
Jeong Arm
d353bb5ee3
Implement infinity home timeline (#1610)
* Implement infinity home timeline

* Fix test for infinite home timeline

* Fix infinity home timeline with min_id

* Fix infinite home timeline duplicated statuses

* Codeclimate for infinite home timeline

* Refactor code as reviewed

* Fix redis sufficient check

* Fix typo on variable name
2022-04-12 17:52:14 +02:00
Claire
f9fbff8539 Merge branch 'main' into glitch-soc/merge-upstream 2022-04-11 16:16:33 +02:00
Claire
dc50347247
Fix crash in alias settings page (#18004) 2022-04-09 20:11:06 +02:00
Claire
ff73d43b35 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, just an upstream dependency udpated
  textually too close to a glitch-soc-only dependency.
  Updated dependencies like upstream.
- `app/controllers/settings/preferences_controller.rb`:
  Upstream added settings where we had extra glitch-soc-specific settings.
  Added upstream's new settings.
- `app/models/user.rb`:
  Upstream added settings where we had extra glitch-soc-specific settings.
  Added upstream's new settings.
- `config/i18n-tasks.yml`:
  Not a real conflict, just a new upstream line too textually close to
  a glitch-soc-only line.
  Ported upstream's change.
2022-04-08 19:53:32 +02:00
Eugen Rochko
68273a7c6d
Fix dangling language-specific trends (#17997)
- Change score half-life for trending statuses from 2 to 6 hours
- Change score threshold for trimming old items from 1 to 0.3
2022-04-08 19:35:31 +02:00
Eugen Rochko
8e20e16cf0
Change e-mail notifications to only be sent when recipient is offline (#17984)
* Change e-mail notifications to only be sent when recipient is offline

Change the default for follow and mention notifications back on

* Add preference to always send e-mail notifications

* Change wording
2022-04-08 18:03:31 +02:00
Eugen Rochko
fd9a9b07c2
Fix trends returning less results per page when filtered in REST API (#17996)
- Change filtering and pagination to occur in SQL instead of Redis
- Change rank/score displayed on trends in admin UI to be locale-specific
2022-04-08 17:10:53 +02:00
Eugen Rochko
465ee7792f
Fix pagination header on empty trends responses in REST API (#17986) 2022-04-07 18:06:15 +02:00
Claire
b368c75029 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
  Upstream removed this file, while glitch-soc had changes to deal with
  its theming system.
  Removed the file like upstream did.
2022-04-06 21:10:23 +02:00
Eugen Rochko
6221b36b27
Remove sign-in token authentication, instead send e-mail about new sign-in (#17970) 2022-04-06 20:58:12 +02:00
Claire
6bdec0aec9 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-31 10:36:23 +02:00
Claire
ef196c913c
Fix error MethodError in Chewy::Strategy::Sidekiq::Worker (#17912)
Also refactor a bit to reduce code duplication.
2022-03-31 00:49:24 +02:00
Claire
fc5dd0c538 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-30 15:02:56 +02:00
Eugen Rochko
5554ff2a1d
Fix being able to bypass e-mail restrictions (#17909) 2022-03-30 14:45:52 +02:00
Claire
51a297e128 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-30 13:40:10 +02:00
Claire
894956e20c
Fix /api/v1/admin/accounts (#17887)
* Fix /api/v1/admin/accounts

Compatibility was broken since #17009 which changed the underlying filter class
without changing the controller.

This commits restore support for the old parameters.

* Add /api/v2/admin/accounts with the new parameters

* Add tests

* Add missing filter for `silenced` status

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-03-28 23:57:38 +02:00
Claire
aaa9ec340b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/formatter.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
- `app/models/status.rb`:
  Upstream refactored status edit handling and moved code to
  `app/models/concerns/status_snapshot_concern.rb`.
  Applied glitch-soc's changes to that file.
- `app/serializers/activitypub/note_serializer.rb`:
  Not really a conflict, just a line added too close to one modified by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc-modified one.
- `app/services/update_status_service.rb`:
  Not really a conflict, upstream modified a line adjacent to one added by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc line.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream refactored formatting, glitch-soc changed the markup slightly.
  Applied upstream changes.
- `spec/lib/formatter_spec.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
2022-03-26 19:18:55 +01:00
Eugen Rochko
2dd30804b6
Change how unconfirmed accounts are displayed in admin UI (#17874)
Fix #17815
2022-03-26 02:53:13 +01:00
Eugen Rochko
71f2b95106
Fix edits with no actual changes being allowed (#17843)
* Fix edits with no actual changes being allowed locally

* Fix edits with no actual changes being allowed through ActivityPub

* Fix false positive changes caused by description processing in model

* Fix not recording poll expiration update

* Fix test

* Revert changes to ProcessStatusUpdateService

* Various fixes and improvements

* Fix code style issues

* Various changes and improvements

* Add guard clause
2022-03-26 00:38:44 +01:00
Eugen Rochko
e3a2203061
Add offset pagination to trends in REST API (#17872) 2022-03-26 00:26:50 +01:00
Claire
c79a03b319 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream changed their README, we have a completely different one.
  Kept our README.
2022-03-15 20:45:27 +01:00
Eugen Rochko
a794117679
Fix individually approved/rejected statuses/links showing as pending review (#17787) 2022-03-15 07:51:55 +01:00
Eugen Rochko
e6ffbfb5e7
Add types param to GET /api/v1/notifications in REST API (#17767)
* Add `types` param to `GET /api/v1/notifications` in REST API

* Improve tests
2022-03-15 04:11:29 +01:00
Eugen Rochko
eebafe24a8
Fix statuses not being referenced in strike when category is spam (#17786) 2022-03-15 04:11:13 +01:00
Claire
a945182b56 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-14 16:44:35 +01:00
Eugen Rochko
5db1f377ea
Fix nil error when viewing suspended domain in admin UI (#17765) 2022-03-14 05:27:37 +01:00
Claire
bb9a6fb5c2 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-12 11:51:25 +01:00
Claire
642528f455
Update fix-duplicates maintenance task (#17731)
* Update fix-duplicates task to 2022_02_10_153119

Also add support for Appeal to AccountMerging#merge_with!

* Update fix-duplicates task to 2022_03_07_094650

* Update fix-duplicates task to 2022_03_09_213005

* Update fix-duplicates task to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060626

* Update fix-duplicates script to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060706

* Update fix-duplicates task to 2022_03_10_060959

* Silence CodeClimate
2022-03-12 08:33:11 +01:00
Eugen Rochko
ee8d4dd056
Fix moderator account leak in status edit history (#17746) 2022-03-12 08:23:03 +01:00
Jeong Arm
18e27e0488
Enable domain purge button on suspended domains too (#17741)
Related: #17209
2022-03-11 04:58:09 +01:00
Claire
24e83246f9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/status.rb`:
  Upstream updated media and edit-related code textually close to glitch-soc
  additions (local-only and content-type).
  Ported upstream changes.
- `app/models/status_edit.rb`:
  Upstream changes textually close to glitch-soc additions (content-type).
  Ported upstream changes.
- `app/serializers/activitypub/note_serializer.rb`:
  Upstream changed how media attachments are handled. Not really a conflict,
  but textually close to glitch-soc additions (directMessage attribute).
  Ported upstream changes.
- `app/services/remove_status_service.rb`:
  Upstream changed how media attachments are handled. Not really a conflict,
  but textually close to glitch-soc additions (DM timeline).
  Ported upstream changes.
- `app/services/update_status_service.rb`:
  Upstream fixed an issue with language selection. Not really a conflict,
  but textually close to glitch-soc additions (content-type).
  Ported upstream changes.
- `db/schema.rb`:
  Upstream added columns to the `status_edits` table, the conflict is because
  of an additional column (`content-type`) in glitch-soc.
  Ported upstream changes.
- `package.json`:
  Upstream dependency (express) textually adjacent to a glitch-soc-specific one
  (favico.js) got updated.
  Updated it as well.
2022-03-10 09:52:45 +01:00
Takeshi Umeda
d7fab238a8
Fix a type error in domain_block policies (#17735) 2022-03-10 04:10:20 +01:00
Eugen Rochko
9f2791eb64
Add polls and media attachments to edit comparison modal in web UI (#17727) 2022-03-09 21:15:24 +01:00
Claire
2a56a890da
Fix rare race condition when rebloged status is deleted (#17693)
* Fix rare race condition when rebloged status is deleted

* Use INSERT INTO … SELECT
2022-03-09 20:49:14 +01:00
Eugen Rochko
b2cd34474b
Add rate limit for editing (#17728) 2022-03-09 20:06:51 +01:00
chandrn7
a6ed6845c9
Allow login through OpenID Connect (#16221)
* added OpenID Connect as an SSO option

* minor fixes

* added comments, removed an option that shouldn't be set

* fixed Gemfile.lock

* added newline to end of Gemfile.lock

* removed tab from Gemfile.lock

* remove chomp

* codeclimate changes and small name change to make function's purpose clearer

* codeclimate fix

* added SSO buttons to /about page

* minor refactor

* minor style change

* removed spurious change

* removed unecessary conditional from ensure_valid_username and added support for auth.info.name in user_params_from_auth

* minor changes
2022-03-09 12:07:35 +01:00
Eugen Rochko
d17fb70131
Change how changes to media attachments are stored for edits (#17696)
* Change how changes to media attachments are stored for edits

Fix not being able to re-order media attachments

* Fix not broadcasting updates when polls/media is changed through ActivityPub

* Various fixes and improvements

* Update app/models/report.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add tracking of media attachment description changes

* Change poll in status edit to have a structure closer to the real one

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-09 09:06:17 +01:00
Eugen Rochko
bd53dd5210
Change design of federation pages in admin UI (#17704)
* Change design of federation pages in admin UI

* Fix query performance in instance media attachments measure

* Fix reblogs being included in instance languages dimension
2022-03-09 08:52:32 +01:00
Eugen Rochko
318d34d528
Fix data integrity of featured tags (#17712) 2022-03-09 08:51:12 +01:00
Claire
1fbd1fa5c4 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/lib/user_settings_decorator.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/models/status.rb`:
  Conflict because of slight change in how glitch-soc handles the scope to
  filter out local-only posts for anonymous viewers.
  Took upstream's changes and re-applied glitch-soc's change.
- `app/models/user.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/views/directories/index.html.haml`:
  Conflict because upstream redesigned that page while glitch-soc had a minor
  change to support hiding the number of followers.
  Ported glitch-soc's change on top of upstream's redesign.

Additional changes:
- `app/models/account_statuses_filter.rb`:
  See change to `app/models/status.rb`.
2022-03-08 20:22:54 +01:00
Eugen Rochko
8f6c67bfde
Fix performance of account timelines (#17709)
* Fix performance of account timelines

* Various fixes and improvements

* Fix duplicate results being returned

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Fix grouping for pinned statuses scope

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-08 09:14:39 +01:00
Eugen Rochko
292c75aa31
Fix nil error when submitting report without rule_ids (#17713) 2022-03-07 11:21:27 +01:00
Eugen Rochko
edf09ec747
Add /api/v1/accounts/familiar_followers to REST API (#17700)
* Add `/api/v1/accounts/familiar_followers` to REST API

* Change hide network preference to be stored consistently for local and remote accounts

* Add dummy classes to migration

* Apply suggestions from code review

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-07 09:36:47 +01:00
Claire
d9c3109bb6 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-04 01:07:49 +01:00
Claire
02dbc59c91
Add option to automatically delete toots after one week (#17691)
Fixes #17658
2022-03-03 16:18:58 +01:00
Claire
425a6c90c4 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-02 18:02:48 +01:00
Eugen Rochko
25d3dc4373
Add ability to mark statuses as sensitive from reports in admin UI (#17668)
* Add ability to mark statuses as sensitive from reports in admin UI

* Allow mark as sensitive action on statuses with preview cards
2022-03-01 22:20:29 +01:00
Claire
14919fe11e
Change old moderation strikes to be displayed in a separate page (#17566)
* Change old moderation strikes to be displayed in a separate page

Fixes #17552

This changes the moderation strikes displayed on `/auth/edit` to be those from
the past 3 months, and make all moderation strikes targeting the current user
available in `/disputes`.

* Add short description of what the strikes page is for

* Move link to list of strikes to “Account status” instead of navigation item

* Normalize i18n file

* Fix layout and styling of strikes link

* Revert highlights_on regexp

* Reintroduce account status summary

- this way, “Account status” is never empty
- account status is not necessarily bound to strikes, or recent strikes
2022-03-01 19:37:47 +01:00
Claire
756f1b6615 Add option (on by default) to allow toots with content warnings to trend 2022-02-26 10:46:26 +01:00
Claire
c6df6686af Disable notifications for trending links and trending statuses by default 2022-02-26 10:46:26 +01:00
Claire
be493b6c0d Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/account.rb`:
  Not a real conflict, just upstream getting rid of unused constants too close
  to glitch-soc-specific contents.
  Removed unused constants like upstream did.
- `app/models/trends.rb`:
  Conflict because glitch-soc disabled email notifications for trending links.
  Upstream has refactored this quite a bit and added trending posts.
  Took upstream code, but disabling the extra trending stuff will come in
  another commit.
- `app/views/admin/trends/links/index.html.haml`:
  Conflict due to glitch-soc's theming system.
  Ported upstream changes accordingly.
2022-02-26 09:29:23 +01:00
Eugen Rochko
27965ce5ed
Add trending statuses (#17431)
* Add trending statuses

* Fix dangling items with stale scores in localized sets

* Various fixes and improvements

- Change approve_all/reject_all to approve_accounts/reject_accounts
- Change Trends::Query methods to not mutate the original query
- Change Trends::Query#skip to offset
- Change follow recommendations to be refreshed in a transaction

* Add tests for trending statuses filtering behaviour

* Fix not applying filtering scope in controller
2022-02-25 00:34:14 +01:00
Eugen Rochko
a29a982eaa
Change e-mail domain blocks to block IPs dynamically (#17635)
* Change e-mail domain blocks to block IPs dynamically

* Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

* Update app/workers/scheduler/email_domain_block_refresh_scheduler.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-24 17:28:23 +01:00
Claire
3d25fe0426 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream changed the workflow a bit.
  Conflict comes from us pushing to ghcr while upstream pushes to dockerhub.
  Ported the upstream changes while still pushing to ghcr.
2022-02-23 18:13:12 +01:00
Eugen Rochko
7b816eb5ae
Add notifications for new sign-ups (#16953) 2022-02-23 16:45:22 +01:00
Claire
954e052b07 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/media_attachment.rb`:
  Upstream moved hardcoded values around, while in glitch-soc they are
  configurable.
  Moved them like upstream, but keeping glitch-soc's ability to configure
  them through env vars.
2022-02-22 18:06:29 +01:00
Claire
166f6e4b50
Fix some media attachments being converted with too high framerates (#17619)
Video files with variable framerates are converted to constant framerate videos
and the output framerate picked by ffmpeg is based on the original file's
container framerate (which can be different from the average framerate).

This means that an input video with variable framerate with about 30 frames per
second on average, but a maximum of 120 fps will be converted to a constant 120
fps file, which won't be processed by other Mastodon servers.

This commit changes it so that input files with VFR and a maximum framerate
above the framerate threshold are converted to VFR files with the maximum frame
rate enforced.
2022-02-22 17:11:22 +01:00
Claire
f224237862 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `db/schema.rb`:
  Conflict due to glitch-soc adding the `content_type` column on status edits
  and thus having a different schema version number.
  Solved by taking upstream's schema version number, as it is higher than
  glitch-soc's.
2022-02-17 10:58:44 +01:00
Claire
ac99f586bb
Fix issues when attempting to appeal an old strike (#17554)
* Display an error when an appeal could not be submitted

* Do not offer users to appeal old strikes

* Fix 500 error when trying to appeal a strike that is too old

* Avoid using an extra translatable string
2022-02-16 22:29:48 +01:00
Eugen Rochko
564efd0651
Add appeals (#17364)
* Add appeals

* Add ability to reject appeals and ability to browse pending appeals in admin UI

* Add strikes to account page in settings

* Various fixes and improvements

- Add separate notification setting for appeals, separate from reports
- Fix style of links in report/strike header
- Change approving an appeal to not restore statuses (due to federation complexities)
- Change style of successfully appealed strikes on account settings page
- Change account settings page to only show unappealed or recently appealed strikes

* Change appealed_at to overruled_at

* Fix missing method error
2022-02-14 21:27:53 +01:00
Jeong Arm
d39df35441
Fix admin statuses page order with media (#17538) 2022-02-14 16:07:04 +01:00
Eugen Rochko
6240466866
Fix duplicate accounts when searching by IP range in admin UI (#17524) 2022-02-13 01:58:26 +01:00
Claire
d9379f5331 Merge branch 'main' into glitch-soc/merge-upstream 2022-02-12 11:12:41 +01:00
Eugen Rochko
8f03b7a2fb
Add notifications when a reblogged status has been updated (#17404)
* Add notifications when a reblogged status has been updated

* Change wording to say "edit" instead of "update" and add missing controls

* Replace previous update notifications with the most up-to-date one
2022-02-11 22:20:19 +01:00
Claire
51cad4f7fb Merge branch 'main' into glitch-soc/merge-upstream 2022-02-11 21:53:20 +01:00
Eugen Rochko
d0fcf07436
Change actions in reports to require only one click (#17487) 2022-02-11 21:51:57 +01:00
Eugen Rochko
a27729ee48
Change reblogs to not count towards hashtag trends anymore (#17501) 2022-02-11 21:16:23 +01:00
Claire
abd113167b Add ability to change content-type when editing a toot
Content-type defaults to edited toot's content-type to avoid surprising
behaviors when using clients that do not support this feature.
2022-02-10 19:10:14 +01:00
Claire
f1a6f9062e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
  Upstream moved things around in a place where glitch-soc had support for
  an extra parameter (`content_type`).
  Follow upstream but reintroduce `content_type`.
2022-02-10 19:09:27 +01:00
Claire
63854bee6c
Fix poll votes not being properly reset on poll change (#17498)
* Fix poll votes not being properly reset on poll change

* Fix and add tests

* Fix poll update handling when the number of options changes
2022-02-10 14:26:54 +01:00
Eugen Rochko
63002cde03
Add editing for published statuses (#17320)
* Add editing for published statuses

* Fix change of multiple-choice boolean in poll not resetting votes

* Remove the ability to update existing media attachments for now
2022-02-10 00:15:30 +01:00
Eugen Rochko
2f8159baad
Add category and rule_ids params to POST /api/v1/reports (#17492) 2022-02-10 00:10:16 +01:00
Claire
d90da7d080 Add content_type to status_edits 2022-02-09 18:02:31 +01:00
Claire
322e907e04 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/settings/preferences/appearance/show.html.haml`:
  Upstream renamed some helper functions that were used in a part of the
  settings page which glitch-soc slightly changed the layout of.
  Ported the change.
2022-02-09 17:28:33 +01:00
Eugen Rochko
fd3a45e348
Add edit history to web UI (#17390)
* Add edit history to web UI

* Change history reducer to store items per status

* Fix missing loading prop
2022-02-09 01:17:07 +01:00
Eugen Rochko
2adcad04ff
Fix error in suggestions API due to typo (#17486)
Regression from #17479
2022-02-08 22:23:04 +01:00
Claire
692963d43b Merge branch 'main' into glitch-soc/merge-upstream 2022-02-08 18:23:53 +01:00
Eugen Rochko
b6d7726ecb
Remove language detection through cld3 (#17478)
* Remove language detection through cld3

* Update app/helpers/languages_helper.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-08 02:41:17 +01:00
Eugen Rochko
35850f8195
Fix localization of cold-start follow recommendations (#17479) 2022-02-08 01:53:49 +01:00
Claire
098f2bc1e1 Merge branch 'main' into glitch-soc/merge-upstream 2022-02-01 20:59:28 +01:00
Claire
987d88ea56
Fix requiring an extra restart after recent post-deployment migrations (#17422)
Follow-up to #16409
2022-02-01 20:57:39 +01:00
Claire
b2915613fb Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Upstream-updated lib textually too close to glitch-soc-only dep.
  Updated like upstream.
2022-01-28 08:58:32 +01:00
Claire
03d59340da
Fix Sidekiq warnings about JSON serialization (#17381)
* Fix Sidekiq warnings about JSON serialization

This occurs on every symbol argument we pass, and every symbol key in hashes,
because Sidekiq expects strings instead.

See https://github.com/mperham/sidekiq/pull/5071

We do not need to change how workers parse their arguments because this has
not changed and we were already converting to symbols adequately or using
`with_indifferent_access`.

* Set Sidekiq to raise on unsafe arguments in test mode

In order to more easily catch issues that would produce warnings in production
code.
2022-01-28 00:43:56 +01:00
Claire
b7cf3941b3 Change CAPTCHA handling to be only on email verification
This simplifies the implementation considerably, and while not providing
ideal UX, it's the most flexible approach.
2022-01-25 23:56:57 +01:00
Claire
0fb907441c Add ability to set hCaptcha either on registration form or on e-mail validation
Upshot of CAPTCHA on e-mail validation is it does not need to break the in-band
registration API.
2022-01-25 23:09:48 +01:00
Claire
1b493c9fee Add optional hCaptcha support
Fixes #1649

This requires setting `HCAPTCHA_SECRET_KEY` and `HCAPTCHA_SITE_KEY`, then
enabling the admin setting at
`/admin/settings/edit#form_admin_settings_captcha_enabled`

Subsequently, a hCaptcha widget will be displayed on `/about` and
`/auth/sign_up` unless:
- the user is already signed-up already
- the user has used an invite link
- the user has already solved the captcha (and registration failed for another
  reason)

The Content-Security-Policy headers are altered automatically to allow the
third-party hCaptcha scripts on `/about` and `/auth/sign_up` following the same
rules as above.
2022-01-24 21:22:13 +01:00
Claire
61ef81c548 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `spec/models/status_spec.rb`:
  Upstream added tests too close to glitch-soc-specific tests.
  Kept both tests.
2022-01-23 18:24:01 +01:00
Claire
0a120d86d2
Fix error-prone SQL queries (#15828)
* Fix error-prone SQL queries in Account search

While this code seems to not present an actual vulnerability, one could
easily be introduced by mistake due to how the query is built.

This PR parameterises the `to_tsquery` input to make the query more robust.

* Harden code for Status#tagged_with_all and Status#tagged_with_none

Those two scopes aren't used in a way that could be vulnerable to an SQL
injection, but keeping them unchanged might be a hazard.

* Remove unneeded spaces surrounding tsquery term

* Please CodeClimate

* Move advanced_search_for SQL template to its own function

This avoids one level of indentation while making clearer that the SQL template
isn't build from all the dynamic parameters of advanced_search_for.

* Add tests covering tagged_with, tagged_with_all and tagged_with_none

* Rewrite tagged_with_none to avoid multiple joins and make it more robust

* Remove obsolete brakeman warnings

* Revert "Remove unneeded spaces surrounding tsquery term"

The two queries are not strictly equivalent.

This reverts commit 86f16c537e06c6ba4a8b250f25dcce9f049023ff.
2022-01-23 18:10:10 +01:00
Claire
8a07ecd377
Remove leftover database columns from Devise::Models::Rememberable (#17191)
* Remove leftover database columns from Devise::Models::Rememberable

* Update fix-duplication maintenance script

* Improve errors/warnings in the fix-duplicates maintenance script
2022-01-23 15:46:30 +01:00
Claire
fe89554a54 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/activitypub/activity/create.rb`:
  Upstream refactored how `Create` activities are handled and how values are
  extracted from `Create`d objects. This conflicted with how glitch-soc
  supported the `directMessage` flag to explicitly distinguish between
  limited and direct messages.
  Ported glitch-soc's changes to latest upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Upstream largely refactored that file and changed some of the logic.
  This conflicted with glitch-soc's handling of the direct timeline and
  the options to allow replies and boosts in public feeds.
  Ported those glitch-soc changes on top of latest upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored to move mention-related ActivityPub deliveries to
  `ActivityPub::DeliveryWorker`, while glitch-soc contained an extra check
  to not send local-only toots to remote mentioned users.
  Took upstream's version, as the check is not needed anymore, since it is
  performed at the `ActivityPub::DeliveryWorker` call site already.
- `app/workers/feed_insert_worker.rb`:
  Upstream added support for `update` toot events, while glitch-soc had
  support for an extra timeline support, `direct`.
  Ported upstream changes and extended them to the `direct` timeline.

Additional changes:
- `app/lib/activitypub/parser/status_parser.rb`:
  Added code to handle the `directMessage` flag and take it into account
  to compute visibility.
- `app/lib/feed_manager.rb`:
  Extended upstream's support of `update` toot events to glitch-soc's
  `direct` timeline.
2022-01-19 23:52:48 +01:00
Eugen Rochko
1060666c58
Add support for editing for published statuses (#16697)
* Add support for editing for published statuses

* Fix references to stripped-out code

* Various fixes and improvements

* Further fixes and improvements

* Fix updates being potentially sent to unauthorized recipients

* Various fixes and improvements

* Fix wrong words in test

* Fix notifying accounts that were tagged but were not in the audience

* Fix mistake
2022-01-19 22:37:27 +01:00
Claire
b3bf32a21e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/reports/show.html.haml`:
  Conflicts due to glitch-soc's theming system.
2022-01-17 10:45:25 +01:00
Eugen Rochko
14f436c457
Add notifications for statuses deleted by moderators (#17204) 2022-01-17 09:41:33 +01:00
Claire
b61c3ddff8 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream added some text, but our README is completely different.
  Kept our README unchanged.
2022-01-16 18:11:20 +01:00
Eugen Rochko
8e84ebf0cb
Remove IP tracking columns from users table (#16409) 2022-01-16 13:23:50 +01:00
Claire
09c61e1376 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  We have completely different contents. Kept our version.
- `package.json`:
  Not a real conflict, just an upstream dependency udpated
  textually too close to a glitch-soc-only dependency.
  Updated dependencies like upstream.
- `streaming/index.js`:
  Conflict due to code style changes on parts that were
  modified in glitch-soc to handle local-only toots.
  Changed style according to upstream.
2021-12-27 15:16:18 +01:00
Jeong Arm
720e8ab0f5
Fix duplicate record on admin/accounts when searching with IP (#17150) 2021-12-21 00:17:14 +01:00
Claire
d4f315f304 Merge branch 'main' into glitch-soc/merge-upstream 2021-12-18 11:44:54 +01:00
Claire
76761d5fc0
Add ability for admins to delete canonical email blocks (#16644)
* Add admin option to remove canonical email blocks from a deleted account

* Add tootctl canonical_email_blocks to inspect and remove canonical email blocks
2021-12-17 23:02:14 +01:00
Claire
7f803c41e2
Add ability to purge undeliverable domains from admin interface (#16686)
* Add ability to purge undeliverable domains from admin interface

* Add tests
2021-12-17 23:01:21 +01:00
Claire
b2526316f5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/pending_accounts/index.html.haml`:
  Removed upstream, while it had glitch-soc-specific changes to accomodate
  for glitch-soc's theming system.
  Removed the file.

Additional changes:
- `app/views/admin/accounts/index.html.haml':
  Accomodate for glitch-soc's theming system.
2021-12-16 16:20:44 +01:00
Eugen Rochko
fe45184b36
Change trending hashtags threshold back from 15 to 5 (#17122) 2021-12-13 05:32:29 +01:00
Eugen Rochko
0fb9536d38
Add batch suspend for accounts in admin UI (#17009) 2021-12-05 21:48:39 +01:00
Claire
9b861d56a9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Copied upstream changes.
- `app/controllers/settings/identity_proofs_controller.rb`:
  Minor conflict due to glitch-soc's extra “enable_keybase” setting.
  Upstream removed keybase support altogether, so did the same.
- `app/controllers/well_known/keybase_proof_config_controller.rb`:
  Minor conflict due to glitch-soc's extra “enable_keybase” setting.
  Upstream removed keybase support altogether, so did the same.
- `lib/mastodon/statuses_cli.rb`:
  Minor conflict due to an optimization that wasn't shared between
  the two versions. Copied upstream's version.
2021-11-26 22:53:55 +01:00
Eugen Rochko
7de0ee7aba
Remove Keybase integration (#17045) 2021-11-26 05:58:18 +01:00
Claire
3304df0ab5 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-26 01:15:32 +01:00
Claire
afd71867ba Revert "Fix trends admin page crashing"
This reverts commit 5f10e64330.
2021-11-26 01:15:29 +01:00
Eugen Rochko
12b3ff6c6d
Fix error on trending hashtags/links pages in admin UI due to missing constant (#17044) 2021-11-26 01:12:39 +01:00
Claire
5f10e64330 Fix trends admin page crashing 2021-11-26 00:50:13 +01:00
Claire
0ccaedc92b Disable trending links review request emails 2021-11-25 23:55:05 +01:00
Claire
443ec4f8ba Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/tags/index.html.haml`:
  Removed upstream while it had changes in glitch-soc to accomodate for the
  theming system.

Additional changes to accomodate for the theming system:
- `app/views/admin/trends/links/preview_card_providers/index.html.haml`
- `app/views/admin/trends/links/index.html.haml`
- `app/views/admin/trends/tags/index.html.haml`
- `app/views/admin/tags/show.html.haml`
2021-11-25 23:50:35 +01:00
Claire
534b0a2704 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-25 18:42:18 +01:00
Eugen Rochko
6e50134a42
Add trending links (#16917)
* Add trending links

* Add overriding specific links trendability

* Add link type to preview cards and only trend articles

Change trends review notifications from being sent every 5 minutes to being sent every 2 hours

Change threshold from 5 unique accounts to 15 unique accounts

* Fix tests
2021-11-25 13:07:38 +01:00
Claire
02a87431cf
Fix error when suspending user with an already-existing canonical email block (#17036)
* Fix error when suspending user with an already-existing canonical email block

Fixes #17033

While attempting to create a `CanonicalEmailBlock` with an existing hash would
raise an `ActiveRecord::RecordNotUnique` error, this being done within a
transaction would cancel the whole transaction. For this reason, checking for
uniqueness in Rails would query the database within the transaction and avoid
invalidating the whole transaction for this reason.

A race condition is still possible, where multiple accounts sharing a canonical
email would be blocked in concurrent transactions, in which only one would
succeed, but that is way less likely to happen that the current issue, and can
always be retried after the first failure, unlike the current situation.

* Add tests
2021-11-24 17:41:03 +01:00
Claire
2b4c7beb73 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-19 18:51:41 +01:00
Takeshi Umeda
3419d3ec84
Bump chewy from 5.2.0 to 7.2.3 (supports Elasticsearch 7.x) (#16915)
* Bump chewy from 5.2.0 to 7.2.2

* fix style (codeclimate)

* fix style

* fix style

* Bump chewy from 7.2.2 to 7.2.3
2021-11-18 22:02:08 +01:00
Claire
4bb6b1a1e7 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-06 00:15:38 +01:00
Claire
6da135a493
Fix reviving revoked sessions and invalidating login (#16943)
Up until now, we have used Devise's Rememberable mechanism to re-log users
after the end of their browser sessions. This mechanism relies on a signed
cookie containing a token. That token was stored on the user's record,
meaning it was shared across all logged in browsers, meaning truly revoking
a browser's ability to auto-log-in involves revoking the token itself, and
revoking access from *all* logged-in browsers.

We had a session mechanism that dynamically checks whether a user's session
has been disabled, and would log out the user if so. However, this would only
clear a session being actively used, and a new one could be respawned with
the `remember_user_token` cookie.

In practice, this caused two issues:
- sessions could be revived after being closed from /auth/edit (security issue)
- auto-log-in would be disabled for *all* browsers after logging out from one
  of them

This PR removes the `remember_token` mechanism and treats the `_session_id`
cookie/token as a browser-specific `remember_token`, fixing both issues.
2021-11-06 00:13:58 +01:00
Claire
87085a5152
Fix AccountNote not having a maximum length (#16942) 2021-11-06 00:12:25 +01:00
Claire
989c67d29d
Fix handling announcements with links (#16941)
Broken since #15827
2021-11-05 21:14:35 +01:00
Claire
e855df149b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Upstream added new configuration options, uncommented by default.
  Commented them.
- `Gemfile.lock`:
  Upstream updated dependencies textually close to glitch-soc-specific
  dependencies.
  Updated those upstream dependencies.
2021-10-25 20:28:51 +02:00
Takeshi Umeda
17f4e457b3
Add remove from followers api (#16864)
* Add followed_by? to account_interactions

* Add RemoveFromFollowersService

* Fix AccountBatch to use RemoveFromFollowersService

* Add remove from followers API
2021-10-18 12:02:35 +02:00
Claire
b2eaf5562d Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream added some documentation in the README, but our README is pretty
  different and barebone.
  Kept our README.
2021-10-14 22:14:26 +02:00
Claire
694c073d1f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/admin/dashboard_controller.rb`:
  Upstream completely redesigned the admin dashboard.
  glitch-soc tracked extra features, but that list is
  gone.
  Followed upstram.
- `app/views/admin/dashboard/index.html.haml`
  Upstream completely redesigned the admin dashboard.
  glitch-soc tracked extra features, but that list is
  gone.
  Followed upstram.
2021-10-14 21:44:57 +02:00
Jeong Arm
f4081d1564
Allow keeping only 1 boosts/favs on auto deleting posts (#16653)
* Allow keeping 1 boosts/favs on auto deleting posts

* Fix tests
2021-10-14 21:11:14 +02:00
Claire
9b978872cc
Fix 404 error when filtering admin action logs by non-existent target account (#16643)
Currently, there is no way for an admin to delete an account record. However,
should that happen in the future, or should an admin mistype an account id,
`/admin/action_logs?target_account_id=<non-existent-account-id>` currently
returns a 404 instead of an empty list.

With this commit, the aforementioned page returns a correct list instead
(potentially non-empty if the account has been manually deleted)
2021-10-14 20:53:11 +02:00
Claire
216570ad98
Fix scheduled statuses decreasing statuses counts (#16791)
* Add tests

* Fix scheduled statuses decreasing statuses counts

Fixes #16774
2021-10-14 19:59:21 +02:00
Claire
1665b84285 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not really a conflict, just a glitch-soc-only dependency textually too close
  to an updated upstream one.
- `yarn.lock`:
  Not really a conflict, just a glitch-soc-only dependency textually too close
  to an updated upstream one.
2021-10-13 15:50:24 +02:00
Claire
5159ba26e4
Fix error when rendering public pages with media attachments (#16763)
* Add tests

* Fix error when rendering public pages with media attachments

* Add tests

* Fix tests

* Please CodeClimate
2021-10-13 15:27:19 +02:00
Claire
079ed6c26d Merge branch 'main' into glitch-soc/merge-upstream 2021-10-06 16:46:32 +02:00
Claire
84ceebe1c4
Fix media attachment size validation not correctly accounting for file type (#16819)
* Fix media attachment size validation not correctly accounting for file type

Fixes a regression introduced in #16724 caused by the fact that kt-paperclip
now correctly runs validations before processing, meaning that file size
verification could not rely on our before_post_processing hook.

Moved the `before_post_processing` hooks to `before_validate` to make sure
the media attachment type is set correctly before the file gets validated.

* Add tests
2021-10-06 14:49:32 +01:00
Claire
29b5c5f483 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/custom_emoji.rb`:
  Slight refactor upstream, next to a line that was different in glitch-soc
  because of our local configurable limits on custom emoji size.
  Ported upstream changes.
- `yarn.lock`:
  Not really a conflict, upstream dependency textually too close to a
  glitch-soc-only dependency.
  Updated upstream dependency as upstream.
2021-09-30 00:01:20 +02:00
Claire
fc3ae1343d
Switch from unmaintained paperclip to kt-paperclip (#16724)
* Switch from unmaintained paperclip to kt-paperclip

* Drop some compatibility monkey-patches not required by kt-paperclip

* Drop media spoof check monkey-patching

It's broken with kt-paperclip and hopefully it won't be needed anymore

* Fix regression introduced by paperclip 6.1.0

* Do not rely on pathname to call FastImage

* Add test for ogg vorbis file with cover art

* Add audio/vorbis to the accepted content-types

This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…

* Restore missing for_as_default method

* Refactor Attachmentable concern and delay Paperclip's content-type spoof check

Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.

* Please CodeClimate

* Add audio/vorbis to the unreliable set

It doesn't correspond to a file format and thus has no extension associated.
2021-09-29 23:52:36 +02:00
Claire
f1e6884d6b Merge branch 'main' into glitch-soc/merge-upstream 2021-09-14 13:50:30 +02:00
Claire
db57bff11d
Stop setting a shortcode to newly-created media attachments (#16730)
* Stop setting a shortcode to newly-created media attachments

The WebUI has stopped using the “short media URL” in ages. This isn't used
anywhere except for mail notifications.

Deprecating it would allow us to eventually get rid of at least a database
column and corruption-prone index, as well as a controller.

* Fix tests
2021-09-13 18:59:37 +02:00
Claire
f9185c72a9 Merge branch 'main' into glitch-soc/merge-upstream 2021-09-02 10:52:09 +02:00
Claire
2b18f7a943
Fix processing mentions to domains with non-ascii TLDs (#16689)
Fixes #16602
2021-09-01 22:06:40 +02:00
Claire
4e2a8c9b38 Merge branch 'main' into glitch-soc/merge-upstream 2021-08-20 13:01:50 +02:00
Claire
5efb1ff337
Fix followers synchronization mechanism not working when URI has empty path (#16510)
* Fix followers synchronization mechanism not working when URI has empty path

To my knowledge, there is no current implementation on the fediverse
that can use bare domains (e.g., actor is at https://example.org instead of
something like https://example.org/actor) that also plans to support the
followers synchronization mechanism. However, Mastodon's current implementation
would exclude such accounts from followers list.

Also adds tests and rename them to reflect the proper method names.

* Move url prefix regexp to its own constant
2021-08-11 17:48:42 +02:00
Claire
47ea9b2d67 Merge branch 'main' into glitch-soc/merge-upstream 2021-08-11 16:41:59 +02:00
Claire
13b08610a0
Fix crash when encountering invalid account fields (#16598)
* Add test

* Fix crash when encountering invalid account fields
2021-08-11 16:40:55 +02:00
Claire
8ec4be4233 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream-updated dependency (redis) textually too
  close to glitch-soc-only dependecy.
  Updated redis gem like upstream did.
2021-08-09 23:25:49 +02:00
Claire
4ac78e2a06
Add feature to automatically delete old toots (#16529)
* Add account statuses cleanup policy model

* Record last inspected toot to delete to speed up successive calls to statuses_to_delete

* Add service to cleanup a given account's statuses within a budget

* Add worker to go through account policies and delete old toots

* Fix last inspected status id logic

All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.

* Add tests

* Refactor scheduler and add tests

* Add user interface

* Add support for discriminating based on boosts/favs

* Add UI support for min_reblogs and min_favs, rework UI

* Address first round of review comments

* Replace Snowflake#id_at_start with with_random parameter

* Add tests

* Add tests for StatusesCleanupController

* Rework settings page

* Adjust load-avoiding mechanisms

* Please CodeClimate
2021-08-09 23:11:50 +02:00
Claire
763ab0c7eb
Fix owned account notes not being deleted when an account is deleted (#16579)
* Add account_notes relationship

* Add tests

* Fix owned account notes not being deleted when an account is deleted

* Add post-migration to clean up orphaned account notes
2021-08-08 15:29:57 +02:00
Claire
d1c87be8cf Merge branch 'main' into glitch-soc/merge-upstream 2021-07-17 17:45:35 +02:00
Claire
30ce6e395c
Fix user email address being banned on self-deletion (#16503)
* Add tests

* Fix user email address being banned on self-deletion

Fixes #16498
2021-07-14 05:35:49 +02:00
Claire
cdf20f4367 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/serializers/rest/instance_serializer.rb`:
  Upstream changed the fields returned by /api/v1/instance by adding a
  `configuration` field holding a lot of useful information making our
  `max_toot_chars` and `poll_limits` fields obsolete.
  Keeping those around for now for compatibility.
- `app/validators/status_length_validator.rb`:
  No real conflict, just URL_PLACEHOLDER_CHARS introduced too close to
  MAX_CHARS which is defined differently in glitch-soc.
  Ported upstream changes.
2021-07-11 21:41:23 +02:00
Eugen Rochko
7095c80373
Add ability to filter reports by origin of target account (#16487) 2021-07-11 11:01:38 +02:00
Claire
3160e050a9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/helpers/accounts_helper.rb`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
- `app/views/accounts/_header.html.haml`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
- `app/views/directories/index.html.haml`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
2021-07-08 16:17:19 +02:00
Eugen Rochko
771c9d4ba8
Add ability to skip sign-in token authentication for specific users (#16427)
Remove "active within last two weeks" exception for sign in token requirement

Change admin reset password to lock access until the password is reset
2021-07-08 05:31:28 +02:00
Claire
8569126c2e
Fix upgrade path from 3.4.0 (#16465)
3.4.1 dropped account_stats.lock_version, but in a way breaking the usual
upgrade path by requiring services to be reloaded after the post-migrations.

Indeed, `self.locking_column = nil` was not enough for Rails to ignore the
`lock_version` column when preparing statements on application load, resulting
in some ActiveRecord queries (typically those involving
`includes(:account_stat)`) erroring out with:

  ActiveRecord::StatementInvalid (PG::UndefinedColumn: ERROR:  column account_stats.lock_version does not exist
2021-07-07 21:13:30 +02:00
Claire
1e2097afd4 Merge branch 'main' into glitch-soc/merge-upstream 2021-07-02 16:12:11 +02:00
Eugen Rochko
38b3419950
Fix not being able to suspend accounts that already have a canonical e-mail block (#16455) 2021-07-02 12:03:15 +02:00
Claire
37fd9670a5 Merge branch 'main' into glitch-soc/merge-upstream 2021-06-23 10:46:24 +02:00
Eugen Rochko
d174d12c83
Add authentication history (#16408) 2021-06-21 17:07:30 +02:00
Claire
0157caacef Merge branch 'main' into glitch-soc/merge-upstream 2021-06-03 20:47:02 +02:00
Claire
526332c545
Fix account deletion sometimes failing because of optimistic locks (#16317)
* Fix account deletion sometimes failing because of optimistic locks

In some rare occasions[1], deleting accounts would fail with a
`StaleObjectError` exception.

Indeed, account deletion manually sets the `AccountStat` values without
handling cases where the optimistic locking on `AccountStat` would fail.

To my knowledge, with the rewrite of account counters in #15913, the
`DeleteAccountService` is now the only place that changes the counters in
a way that is not atomic.

Since in this specific case, we do not care about the previous values of the
account counters, it appears we don't need locking at all for this table
anymore.

[1]: https://discourse.joinmastodon.org/t/account-cant-be-deleted/3602

* Bump MAX_SUPPORTED_VERSION in maintenance script
2021-06-02 17:41:25 +02:00
Claire
69737b703a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`: our READMEs are entirely different and
  upstream's has been updated. Kept our README.
2021-05-17 10:30:34 +02:00
Claire
76064e6608
Update fix-duplicates maintenance script to support latest migrations (#16231)
* Update maintenance script to support latest database migrations

* Update Account#merge_with!
2021-05-12 23:19:44 +02:00
Claire
dc58d02192 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-12 11:03:05 +02:00
Takeshi Umeda
c403c3695b
Fix to be able to redownload avatar and header (#16190)
* Fix to reset if header and avatar download fails

* Add RedownloadAvatarWorker and RedownloadHeaderWorker
2021-05-11 14:19:22 +02:00
Claire
87c3a0d0b3 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-09 10:41:28 +02:00
Claire
f9c2d1b9a1
Fix FollowRecommendationsScheduler failing because of unpopulated views (#16189)
Since #16173, `account_summaries` and `follow_recommendations` are not
populated at creation time, which causes concurrent refresh to fail.

As we currently only use those materialized views right after explicitly
refreshing them, this commit changes refreshes to not be performed
concurrently. This will fix the issue and ensure the refresh completes
faster while using less resources.
2021-05-09 10:39:29 +02:00
Claire
fbf097f7e1 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-08 00:01:47 +02:00
Eugen Rochko
1294f9ee4f
Remove PubSubHubbub-related columns from accounts table (#16170) 2021-05-07 19:32:58 +02:00
Claire
50b430d9a2 Merge branch 'main' into glitch-soc/merge-upstream
- `app/views/statuses/_simple_status.html.haml`:
  Small markup change in glitch-soc, on a line that has been modified by
  upstream. Ported upstream changes.
2021-05-07 18:21:59 +02:00