Commit graph

1015 commits

Author SHA1 Message Date
Holger
39b489ba4c
fix: s3_force_single_request not parsed (#17922) 2022-04-01 23:56:23 +02:00
Claire
fc5dd0c538 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-30 15:02:56 +02:00
Eugen Rochko
8c7223f4ea
Bump version to 3.5.0 (#17911) 2022-03-30 14:52:37 +02:00
Claire
61cefbebf7 Add advanced text formatting back into glitch-soc 2022-03-28 22:21:37 +02:00
Claire
aaa9ec340b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/formatter.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
- `app/models/status.rb`:
  Upstream refactored status edit handling and moved code to
  `app/models/concerns/status_snapshot_concern.rb`.
  Applied glitch-soc's changes to that file.
- `app/serializers/activitypub/note_serializer.rb`:
  Not really a conflict, just a line added too close to one modified by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc-modified one.
- `app/services/update_status_service.rb`:
  Not really a conflict, upstream modified a line adjacent to one added by
  glitch-soc.
  Applied upstream changes while keeping the glitch-soc line.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream refactored formatting, glitch-soc changed the markup slightly.
  Applied upstream changes.
- `spec/lib/formatter_spec.rb`:
  Upstream completely refactored the formatting code and removed that file,
  while glitch-soc had code for Markdown and HTML toots.
  Took upstream code, glitch-soc changes will be re-implemented on top of the
  refactored classes in a later commit.
2022-03-26 19:18:55 +01:00
Eugen Rochko
d7d049aab7
Bump version to 3.5.0rc3 (#17876) 2022-03-26 04:29:36 +01:00
Eugen Rochko
07f8b4d1b1
Bump version to 3.5.0rc2 (#17855) 2022-03-26 02:54:11 +01:00
Claire
59864ff495 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Upstream modified it, but we deleted it in glitch-soc.
  Keep it deleted.
2022-03-22 17:00:38 +01:00
Claire
3afd59df0f
Fix tootctl email_domain_blocks add (#17842)
Fixes #17831
2022-03-21 19:10:09 +01:00
Claire
4e3c68e025 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/helpers/accounts_helper.rb`:
  Upstream fixed an issue that glitch-soc did not have, in a place with
  glitch-soc changes to hide the number of followers.
  Kept glitch-soc's code.
2022-03-17 19:23:39 +01:00
Claire
b07906bdb0
Fix wrong language code for Kurdish languages (#17812) 2022-03-17 01:37:03 +01:00
Claire
c79a03b319 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream changed their README, we have a completely different one.
  Kept our README.
2022-03-15 20:45:27 +01:00
Eugen Rochko
4bdce2c513
Bump version to 3.5.0rc1 (#17618)
* Bump version to 3.5.0rc1

* Various fixes and improvements

* Update AUTHORS.md

* Various fixes and improvements

* Update README.md
2022-03-15 08:16:45 +01:00
Claire
bb9a6fb5c2 Merge branch 'main' into glitch-soc/merge-upstream 2022-03-12 11:51:25 +01:00
Claire
642528f455
Update fix-duplicates maintenance task (#17731)
* Update fix-duplicates task to 2022_02_10_153119

Also add support for Appeal to AccountMerging#merge_with!

* Update fix-duplicates task to 2022_03_07_094650

* Update fix-duplicates task to 2022_03_09_213005

* Update fix-duplicates task to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060626

* Update fix-duplicates script to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060706

* Update fix-duplicates task to 2022_03_10_060959

* Silence CodeClimate
2022-03-12 08:33:11 +01:00
Eugen Rochko
75e33fd08f
Fix null values being included in some indexes (#17711)
* Fix null values being included in some indexes

* Update lib/mastodon/migration_helpers.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add documentation link to corruption error message

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-12 08:12:57 +01:00
Claire
1fbd1fa5c4 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/settings/preferences_controller.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/lib/user_settings_decorator.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/models/status.rb`:
  Conflict because of slight change in how glitch-soc handles the scope to
  filter out local-only posts for anonymous viewers.
  Took upstream's changes and re-applied glitch-soc's change.
- `app/models/user.rb`:
  Conflicts due to us having more user settings and upstream dropping
  `hide_network` (to replace it with an account attribute, properly migrated).
  Dropped `hide_network` like upstream.
- `app/views/directories/index.html.haml`:
  Conflict because upstream redesigned that page while glitch-soc had a minor
  change to support hiding the number of followers.
  Ported glitch-soc's change on top of upstream's redesign.

Additional changes:
- `app/models/account_statuses_filter.rb`:
  See change to `app/models/status.rb`.
2022-03-08 20:22:54 +01:00
Claire
61ae6b3535
Add more migration tests (#17710)
* Add migration tests for hide_network settings migration

* Add tests about suspended/suspended_at

* Add more tests regarding the results of migrations

* Fix migration test regarding stale conflicting remote account

* Add migration tests about AccountConversation
2022-03-07 23:40:55 +01:00
Rens Groothuijsen
c439e13e12
Enable importing GIF emojis in CLI (#17706) 2022-03-06 23:41:44 +01:00
Josh Soref
b5329e0035
Spelling (#17705)
* spelling: account

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: affiliated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: appearance

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: autosuggest

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: cacheable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: component

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: conversations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: domain.example

Clarify what's distinct and use RFC friendly domain space.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: environment

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeds

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: functional

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: inefficiency

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: not

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: notifications

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: occurring

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: position

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: progress

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: promotable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: reblogging

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: repetitive

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: resolve

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: saturated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: similar

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: strategies

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: success

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: targeting

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnails

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unauthorized

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unsensitizes

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: validations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: various

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
2022-03-06 22:51:40 +01:00
Claire
3d25fe0426 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream changed the workflow a bit.
  Conflict comes from us pushing to ghcr while upstream pushes to dockerhub.
  Ported the upstream changes while still pushing to ghcr.
2022-02-23 18:13:12 +01:00
Jeong Arm
3e12abc1fe
Calculate max_id without random vector (#17623) 2022-02-23 16:44:59 +01:00
Claire
954e052b07 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/media_attachment.rb`:
  Upstream moved hardcoded values around, while in glitch-soc they are
  configurable.
  Moved them like upstream, but keeping glitch-soc's ability to configure
  them through env vars.
2022-02-22 18:06:29 +01:00
Claire
166f6e4b50
Fix some media attachments being converted with too high framerates (#17619)
Video files with variable framerates are converted to constant framerate videos
and the output framerate picked by ffmpeg is based on the original file's
container framerate (which can be different from the average framerate).

This means that an input video with variable framerate with about 30 frames per
second on average, but a maximum of 120 fps will be converted to a constant 120
fps file, which won't be processed by other Mastodon servers.

This commit changes it so that input files with VFR and a maximum framerate
above the framerate threshold are converted to VFR files with the maximum frame
rate enforced.
2022-02-22 17:11:22 +01:00
Claire
692963d43b Merge branch 'main' into glitch-soc/merge-upstream 2022-02-08 18:23:53 +01:00
Eugen Rochko
b6d7726ecb
Remove language detection through cld3 (#17478)
* Remove language detection through cld3

* Update app/helpers/languages_helper.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2022-02-08 02:41:17 +01:00
Eugen Rochko
3413f1c44b
Forward-port version bump to 3.4.6 (#17434) 2022-02-03 14:21:38 +01:00
Claire
2beb0a7af5 Bump version to 3.4.6 2022-02-03 12:12:27 +01:00
Claire
098f2bc1e1 Merge branch 'main' into glitch-soc/merge-upstream 2022-02-01 20:59:28 +01:00
Claire
54581d43e7
Bump version to 3.4.5 (#17402) 2022-01-31 21:27:40 +01:00
Claire
2fcf652fff Merge branch 'main' into glitch-soc/merge-upstream 2022-01-31 10:42:17 +01:00
Claire
a0e06c3c3e
Add more advanced migration tests (#17393)
- populate the database with some data when testing migrations
- try both one-step and two-step migrations (`SKIP_POST_DEPLOYMENT_MIGRATIONS`)
2022-01-30 23:50:08 +01:00
Claire
c6b291afc3
Change index corruption warning to be a little less scary (#17395) 2022-01-30 23:49:52 +01:00
Claire
a99adeaad3
Fix edge case in migration helpers that caused crash because of PostgreSQL quirks (#17398) 2022-01-30 22:34:54 +01:00
Claire
61ef81c548 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `spec/models/status_spec.rb`:
  Upstream added tests too close to glitch-soc-specific tests.
  Kept both tests.
2022-01-23 18:24:01 +01:00
Claire
8a07ecd377
Remove leftover database columns from Devise::Models::Rememberable (#17191)
* Remove leftover database columns from Devise::Models::Rememberable

* Update fix-duplication maintenance script

* Improve errors/warnings in the fix-duplicates maintenance script
2022-01-23 15:46:30 +01:00
Claire
1e8c885e5a
Change mastodon:webpush:generate_vapid_key task to not require functional env (#17338)
Fixes #17297
2022-01-20 14:51:23 +01:00
Claire
09c61e1376 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  We have completely different contents. Kept our version.
- `package.json`:
  Not a real conflict, just an upstream dependency udpated
  textually too close to a glitch-soc-only dependency.
  Updated dependencies like upstream.
- `streaming/index.js`:
  Conflict due to code style changes on parts that were
  modified in glitch-soc to handle local-only toots.
  Changed style according to upstream.
2021-12-27 15:16:18 +01:00
Eugen Rochko
fe71548844
Fix warnings on Rails boot (#16946) 2021-12-27 00:47:20 +01:00
Claire
d4f315f304 Merge branch 'main' into glitch-soc/merge-upstream 2021-12-18 11:44:54 +01:00
Claire
76761d5fc0
Add ability for admins to delete canonical email blocks (#16644)
* Add admin option to remove canonical email blocks from a deleted account

* Add tootctl canonical_email_blocks to inspect and remove canonical email blocks
2021-12-17 23:02:14 +01:00
Claire
b2526316f5 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/pending_accounts/index.html.haml`:
  Removed upstream, while it had glitch-soc-specific changes to accomodate
  for glitch-soc's theming system.
  Removed the file.

Additional changes:
- `app/views/admin/accounts/index.html.haml':
  Accomodate for glitch-soc's theming system.
2021-12-16 16:20:44 +01:00
Takeshi Umeda
6c8c031bcd
Add remove orphans to tootctl statuses remove (#17067)
* Add remove orphans to tootctl statuses remove

* Add REINDEX and change option from vacuum to compression-database

* Changed to extract the deletion target of conversations to a temporary table

* Support progress bar and exceptions when media remove

* Add continue option

* Fix compression to compress

* Remove skip_remove_orphans
2021-12-12 06:09:14 +01:00
Claire
9b861d56a9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Copied upstream changes.
- `app/controllers/settings/identity_proofs_controller.rb`:
  Minor conflict due to glitch-soc's extra “enable_keybase” setting.
  Upstream removed keybase support altogether, so did the same.
- `app/controllers/well_known/keybase_proof_config_controller.rb`:
  Minor conflict due to glitch-soc's extra “enable_keybase” setting.
  Upstream removed keybase support altogether, so did the same.
- `lib/mastodon/statuses_cli.rb`:
  Minor conflict due to an optimization that wasn't shared between
  the two versions. Copied upstream's version.
2021-11-26 22:53:55 +01:00
Takeshi Umeda
0ac7efdc50
Fix performance of tootctl statuses remove (#17052)
* Fix performance of tootctl statuses remove

* Fix model class
2021-11-26 22:08:47 +01:00
Takeshi Umeda
06631fdc53
Fix ElasticSearch to Elasticsearch (#17050) 2021-11-26 08:30:02 +01:00
OSAMU SATO
53aca8aecf
Add batch_size option to bin/tootctl search deploy (#17049) 2021-11-26 08:29:53 +01:00
Claire
443ec4f8ba Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/tags/index.html.haml`:
  Removed upstream while it had changes in glitch-soc to accomodate for the
  theming system.

Additional changes to accomodate for the theming system:
- `app/views/admin/trends/links/preview_card_providers/index.html.haml`
- `app/views/admin/trends/links/index.html.haml`
- `app/views/admin/trends/tags/index.html.haml`
- `app/views/admin/tags/show.html.haml`
2021-11-25 23:50:35 +01:00
Eugen Rochko
6e50134a42
Add trending links (#16917)
* Add trending links

* Add overriding specific links trendability

* Add link type to preview cards and only trend articles

Change trends review notifications from being sent every 5 minutes to being sent every 2 hours

Change threshold from 5 unique accounts to 15 unique accounts

* Fix tests
2021-11-25 13:07:38 +01:00
Claire
2b4c7beb73 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-19 18:51:41 +01:00
Takeshi Umeda
3419d3ec84
Bump chewy from 5.2.0 to 7.2.3 (supports Elasticsearch 7.x) (#16915)
* Bump chewy from 5.2.0 to 7.2.2

* fix style (codeclimate)

* fix style

* fix style

* Bump chewy from 7.2.2 to 7.2.3
2021-11-18 22:02:08 +01:00
Claire
48f8658d34
Fix upload of remote media with OpenStack Swift sometimes failing (#16998)
Under certain conditions, files fetched from remotes trigger an error when
being uploaded using OpenStack Swift. This is because in some cases, the
remote server will not return a content-length, so our ResponseWithLimitAdapter
will hold a `nil` value for `#size`, which will lead to an invalid value
for the Content-Length header of the Swift API call.

This commit fixes that by taking the size from the actually-downloaded file
size rather than the upstream-provided Content-Length header value.
2021-11-16 21:36:28 +01:00
Claire
2cda8be805 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-13 09:40:13 +01:00
Claire
18b885ee3a
Fix "bundle exec rails mastodon:setup" crashing in some circumstances (#16976)
Fix regression from #16896
2021-11-11 14:00:30 +01:00
Eugen Rochko
2251db42ec
Forward port version bumps to 3.4.2 and 3.4.3 (#16945)
* Bump version to 3.4.2

* Bump version to 3.4.3
2021-11-06 05:32:14 +01:00
Claire
4bb6b1a1e7 Merge branch 'main' into glitch-soc/merge-upstream 2021-11-06 00:15:38 +01:00
Jeong Arm
884c60002e
Skip blocked domains media on tootctl media refresh (#16914) 2021-10-28 19:30:44 +02:00
Claire
e855df149b Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Upstream added new configuration options, uncommented by default.
  Commented them.
- `Gemfile.lock`:
  Upstream updated dependencies textually close to glitch-soc-specific
  dependencies.
  Updated those upstream dependencies.
2021-10-25 20:28:51 +02:00
Claire
5ba46952af
Fix mastodon:setup to take dotenv/docker-compose differences into account (#16896)
In order to work around https://github.com/mastodon/mastodon/issues/16895,
add a warning to .env.production.sample, and change the mastodon:setup rake
task to:
- output a warning if a variable will be interpreted differently by dotenv
  and docker-compose
- ensure the printed config is compatible with docker-compose
2021-10-25 16:34:15 +02:00
Claire
6ba8bc45cb
Add S3_FORCE_SINGLE_REQUEST env var to work around S3 compatibility issues (#16866)
Fixes #16822
2021-10-18 18:29:04 +02:00
Claire
b2eaf5562d Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream added some documentation in the README, but our README is pretty
  different and barebone.
  Kept our README.
2021-10-14 22:14:26 +02:00
Claire
694c073d1f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/admin/dashboard_controller.rb`:
  Upstream completely redesigned the admin dashboard.
  glitch-soc tracked extra features, but that list is
  gone.
  Followed upstram.
- `app/views/admin/dashboard/index.html.haml`
  Upstream completely redesigned the admin dashboard.
  glitch-soc tracked extra features, but that list is
  gone.
  Followed upstram.
2021-10-14 21:44:57 +02:00
Jeong Arm
3f5f4273b3
Add optional domain restrict to tootctl accounts cull (#16511)
* Add optional domain restrict to accounts cull

* Use "unless" - codeclimate
2021-10-14 21:09:56 +02:00
Claire
a8ef6d24d8
Fix tootctl accounts cull not excluding domains on timeouts and certificate issues (#16433)
Fixes #16410
2021-10-14 21:08:37 +02:00
Claire
959f7fc580
Fix tootctl self-destruct not sending Delete activities for recently-suspended accounts (#16688)
* Do not block existing users' emails on self-destruct

That is wasteful and unintuitive

* Do not close registrations when running tootctl self-destruct with --dry-run

* Close registrations on self-destruct regardless of known remote accounts

* Fix tootctl self-destruct not sending Deletes for recently-suspended accounts

* Suspend local users even if no remote account is known

* Do not show scary confirmation text if ran with --dry-run
2021-10-14 19:59:28 +02:00
Claire
29b5c5f483 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/custom_emoji.rb`:
  Slight refactor upstream, next to a line that was different in glitch-soc
  because of our local configurable limits on custom emoji size.
  Ported upstream changes.
- `yarn.lock`:
  Not really a conflict, upstream dependency textually too close to a
  glitch-soc-only dependency.
  Updated upstream dependency as upstream.
2021-09-30 00:01:20 +02:00
Claire
fc3ae1343d
Switch from unmaintained paperclip to kt-paperclip (#16724)
* Switch from unmaintained paperclip to kt-paperclip

* Drop some compatibility monkey-patches not required by kt-paperclip

* Drop media spoof check monkey-patching

It's broken with kt-paperclip and hopefully it won't be needed anymore

* Fix regression introduced by paperclip 6.1.0

* Do not rely on pathname to call FastImage

* Add test for ogg vorbis file with cover art

* Add audio/vorbis to the accepted content-types

This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…

* Restore missing for_as_default method

* Refactor Attachmentable concern and delay Paperclip's content-type spoof check

Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.

* Please CodeClimate

* Add audio/vorbis to the unreliable set

It doesn't correspond to a file format and thus has no extension associated.
2021-09-29 23:52:36 +02:00
Claire
8ec4be4233 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream-updated dependency (redis) textually too
  close to glitch-soc-only dependecy.
  Updated redis gem like upstream did.
2021-08-09 23:25:49 +02:00
Claire
4ac78e2a06
Add feature to automatically delete old toots (#16529)
* Add account statuses cleanup policy model

* Record last inspected toot to delete to speed up successive calls to statuses_to_delete

* Add service to cleanup a given account's statuses within a budget

* Add worker to go through account policies and delete old toots

* Fix last inspected status id logic

All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.

* Add tests

* Refactor scheduler and add tests

* Add user interface

* Add support for discriminating based on boosts/favs

* Add UI support for min_reblogs and min_favs, rework UI

* Address first round of review comments

* Replace Snowflake#id_at_start with with_random parameter

* Add tests

* Add tests for StatusesCleanupController

* Rework settings page

* Adjust load-avoiding mechanisms

* Please CodeClimate
2021-08-09 23:11:50 +02:00
Takeshi Umeda
818e0b314f
Fix unsupported video error message handling (#16581) 2021-08-08 15:28:57 +02:00
Claire
fc500a6062 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `CONTRIBUTING.md`:
  Upstream file has changed and we're quoting it.
  Ported the changes.
- `README.md`:
  Upstream file has changed but we have a completely different one.
  Kept our version.
- `lib/mastodon/version.rb`:
  Upstream has changed from `tootsuite/mastodon` to `mastodon/mastodon`,
  but we're still `glitch-soc/mastodon`.
  Kept our version.
- `spec/presenters/instance_presenter_spec.rb`:
  Upstream has changed from `tootsuite/mastodon` to `mastodon/mastodon`,
  but we're still `glitch-soc/mastodon`.
  Kept our version.
2021-07-13 17:57:15 +02:00
Claire
5a1e072517
Change references to tootsuite/mastodon to mastodon/mastodon (#16491)
* Change references to tootsuite/mastodon to mastodon/mastodon

* Remove obsolete test fixture

* Replace occurrences of tootsuite/mastodon with mastodon/mastodon in CHANGELOG

And a few other places
2021-07-13 15:46:20 +02:00
Claire
3160e050a9 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/helpers/accounts_helper.rb`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
- `app/views/accounts/_header.html.haml`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
- `app/views/directories/index.html.haml`:
  Conflict due to upstream changing how followers count is displayed while we
  have an option to hide followers count.
  Ported upstream change.
2021-07-08 16:17:19 +02:00
Eugen Rochko
771c9d4ba8
Add ability to skip sign-in token authentication for specific users (#16427)
Remove "active within last two weeks" exception for sign in token requirement

Change admin reset password to lock access until the password is reset
2021-07-08 05:31:28 +02:00
Claire
2e0eac71dd
Add --by-uri option to tootctl domains purge (#16434)
Fixes #16410
2021-07-07 21:17:00 +02:00
Claire
0157caacef Merge branch 'main' into glitch-soc/merge-upstream 2021-06-03 20:47:02 +02:00
Claire
f6088922c0
Update emoji codepoint mappings to v13.1 (#16352) 2021-06-03 16:08:07 +02:00
Eugen Rochko
d6486c969f
Bump version to 3.4.1 (#16350) 2021-06-03 04:26:02 +02:00
Claire
11d3c065a5
Fix migration script not being able to run if it fails midway (#16312)
* Fix migration script not being able to run if it fails midway

* Fix old migration script

* Fix old migration script

* Refactor CorruptionError
2021-06-02 19:15:17 +02:00
Claire
526332c545
Fix account deletion sometimes failing because of optimistic locks (#16317)
* Fix account deletion sometimes failing because of optimistic locks

In some rare occasions[1], deleting accounts would fail with a
`StaleObjectError` exception.

Indeed, account deletion manually sets the `AccountStat` values without
handling cases where the optimistic locking on `AccountStat` would fail.

To my knowledge, with the rewrite of account counters in #15913, the
`DeleteAccountService` is now the only place that changes the counters in
a way that is not atomic.

Since in this specific case, we do not care about the previous values of the
account counters, it appears we don't need locking at all for this table
anymore.

[1]: https://discourse.joinmastodon.org/t/account-cant-be-deleted/3602

* Bump MAX_SUPPORTED_VERSION in maintenance script
2021-06-02 17:41:25 +02:00
Claire
abf4c2ab21
Fix tootctl search deploy on Ruby 3 (#16346)
Fixes #16344
2021-06-01 17:31:49 +02:00
Claire
69737b703a Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`: our READMEs are entirely different and
  upstream's has been updated. Kept our README.
2021-05-17 10:30:34 +02:00
Eugen Rochko
4c7efdba40
Bump version to 3.4.0 (#16239) 2021-05-16 23:55:07 +02:00
Eugen Rochko
d862728ae1
Add more checks to repo:check_locales_files (#16249) 2021-05-16 23:54:46 +02:00
Claire
76064e6608
Update fix-duplicates maintenance script to support latest migrations (#16231)
* Update maintenance script to support latest database migrations

* Update Account#merge_with!
2021-05-12 23:19:44 +02:00
Claire
dc58d02192 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-12 11:03:05 +02:00
rinsuki
5ed5f62705
Fix animated GIF generates animated thumbnail (#16216) 2021-05-11 19:15:11 +02:00
Eugen Rochko
7bd2b54a46
Bump version to 3.4.0rc2 (#16206) 2021-05-10 22:23:48 +02:00
Claire
87c3a0d0b3 Merge branch 'main' into glitch-soc/merge-upstream 2021-05-09 10:41:28 +02:00
Takeshi Umeda
9b18914c35
Add a Redis environment variable for sidekiq (#16188) 2021-05-09 10:40:17 +02:00
Eugen Rochko
d1442a06c3
Bump version to 3.4.0rc1 (#16053) 2021-05-08 17:15:06 +02:00
Claire
50b430d9a2 Merge branch 'main' into glitch-soc/merge-upstream
- `app/views/statuses/_simple_status.html.haml`:
  Small markup change in glitch-soc, on a line that has been modified by
  upstream. Ported upstream changes.
2021-05-07 18:21:59 +02:00
Claire
a5f91a11d0
Fix older migrations on Ruby 3 (#16174) 2021-05-07 15:56:45 +02:00
Claire
566fc90913
Add Ruby 3.0 support (#16046)
* Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0

Also improve the Terrapin monkey-patch for the stderr/stdout issue.

* Fix keyword argument handling throughout the codebase

* Monkey-patch Paperclip to fix keyword arguments handling in validators

* Change validation_extensions to please CodeClimate

* Bump microformats from 4.2.1 to 4.3.1

* Allow Ruby 3.0

* Add Ruby 3.0 test target to CircleCI

* Add test for admin dashboard warnings

* Fix admin dashboard warnings on Ruby 3.0
2021-05-06 14:22:54 +02:00
Eugen Rochko
6d9ad30bf8
Fix media redownload worker retrying on unexpected response codes (#16111) 2021-05-05 23:46:59 +02:00
Eugen Rochko
036556d350
Fix media processing getting stuck on too much stdin/stderr (#16136)
* Fix media processing getting stuck on too much stdin/stderr

See thoughtbot/terrapin#5

* Remove dependency on paperclip-av-transcoder gem

* Remove dependency on streamio-ffmpeg gem

* Disable stdin on ffmpeg process
2021-05-05 19:44:01 +02:00
Takeshi Umeda
25345c90ff
Fix how to change connection pool for rails 6 (#16158)
* Fix how to change connection pool for rails 6

* Fix to match the code elsewhere
2021-05-05 04:41:10 +02:00
Takeshi Umeda
422df9d670
Fix cache redis not being used (#16131) 2021-04-29 15:43:49 +02:00
Ikko Ashimine
0bc909687a
Fix typo in db.rake (#16126)
occured -> occurred
2021-04-27 16:25:24 +02:00
Claire
e2a2bc9021 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream updated copyright year, we don't mention it so kept our version.
- `app/controllers/admin/dashboard_controller.rb`:
  Not really a conflict, upstream change (removing the spam checker) too close
  to glitch-soc changes. Ported upstream changes.
- `app/models/form/admin_settings.rb`:
  Same.
- `app/services/remove_status_service.rb`:
  Same.
- `app/views/admin/settings/edit.html.haml`:
  Same.
- `config/settings.yml`:
  Same.
- `config/environments/production.rb`:
  Not a real conflict, upstream added a default HTTP header, but we have
  extra headers in glitch-soc.
  Added the header.
2021-04-20 12:17:14 +02:00
Claire
e78d06eecf
Add border to 🚲 emoji (#16035) 2021-04-13 23:43:51 +02:00
Eugen Rochko
ad61265268
Remove dependency on pluck_each gem (#16012) 2021-04-12 03:35:58 +02:00
Claire
c901ae77d4 Merge branch 'main' into glitch-soc/merge-upstream 2021-04-05 14:31:07 +02:00
abcang
ddabbbf5a6
Fix DB connection pool settings in CLI (#15983) 2021-03-31 17:46:17 +02:00
Claire
15efa32cca Merge branch 'main' into glitch-soc/merge-upstream 2021-03-31 10:34:29 +02:00
Stanislas
d33351af3c
tootctl emoji import: case insensitive duplicate check (#15738) 2021-03-24 10:55:16 +01:00
Claire
f60c99a8fb Merge branch 'main' into glitch-soc/merge-upstream 2021-03-24 10:53:50 +01:00
Claire
cbd0ee1d07
Update Mastodon to Rails 6.1 (#15910)
* Update devise-two-factor to unreleased fork for Rails 6 support

Update tests to match new `rotp` version.

* Update nsa gem to unreleased fork for Rails 6 support

* Update rails to 6.1.3 and rails-i18n to 6.0

* Update to unreleased fork of pluck_each for Ruby 6 support

* Run "rails app:update"

* Add missing ActiveStorage config file

* Use config.ssl_options instead of removed ApplicationController#force_ssl

Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.

* Fix nonce directives by removing Rails 5 specific monkey-patching

* Fix fixture_file_upload deprecation warning

* Fix yield-based test failing with Rails 6

* Use Rails 6's index_with when possible

* Use ActiveRecord::Cache::Store#delete_multi from Rails 6

This will yield better performances when deleting an account

* Disable Rails 6.1's automatic preload link headers

Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.

In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.

* Switch to Rails 6.0 default config

* Switch to Rails 6.1 default config

* Do not include autoload paths in the load path
2021-03-24 10:44:31 +01:00
Claire
82556834cf
Change mastodon:setup to not call assets:precompile in docker (#13942)
It appears assets are built during image build, and they shouldn't need
to be rebuilt, since we now have reproducible builds.
2021-03-24 10:37:24 +01:00
Claire
3ad6ef72cb Merge branch 'main' into glitch-soc/merge-upstream 2021-03-19 13:57:15 +01:00
Claire
741d0952b1
Improve account counters handling (#15913)
* Improve account counters handling

* Use ActiveRecord::Base::sanitize_sql to pass values instead of interpolating them

Keep using string interpolation for `key` as it is safe and using
“ActiveRecord::Base::sanitize_sql_hash_for_assignment” would require stitching
bits of SQL in a way that is not more easily checked for safety.

* Add migration hook to catch PostgreSQL versions earlier than 9.5
2021-03-19 13:14:57 +01:00
Claire
c31c95ffe4
Remove MySQL-specific code from Mastodon::MigrationHelpers (#15924)
Mastodon::MigrationHelpers has been forked from Gitlab a long time ago, but
Mastodon has never supported using a MySQL database.

Removing MySQL support from Mastodon::MigrationHelpers makes it a little easier
to maintain. In particular, it removes code that would need updating with
Rails 6.
2021-03-19 13:14:40 +01:00
Claire
82caed594c
Change deduplication order of tootctl maintenance fix-duplicates (#15923)
Hopefully fixes #15922

Also update support up to latest database schema version
2021-03-19 11:07:56 +01:00
Claire
b358229834
Further preparation for Rails 6 (#15916)
* Use ActiveRecord::Result#to_ary instead of deprecated to_hash

They do the same thing, and to_hash has been removed from Rails 6.1

* Explicitly name polymorphic indexes to workaround a bug in Rails 6.1

cf. https://github.com/rails/rails/issues/41693

* Fix incorrect usage of “foreign_key” in migration script

* Use `ActiveModel::Errors#delete` instead of deprecated clear method

* Fix link headers tests on Rails 6.1

Rails 6.1 adds values to the Link header by default, thus it is not a
LinkHeader object anymore. Fix the test to parse the Link header instead
of assuming it is a LinkHeader.
2021-03-19 02:45:34 +01:00
Claire
9aaaa96d2f
Use more robust hook for loading timestamp_id function into database (#15919) 2021-03-19 02:43:13 +01:00
Claire
a4dcaef53b
Prepare Mastodon for zeitwerk autoloader (#15917)
* Prepare Mastodon for zeitwerk autoloader (Rails 6)

Add inflections and rename/move a few classes.

In particular, app/lib/exceptions.rb and app/lib/sanitize_config.rb
were manually loaded while still in autoload paths.

* Add inflection for Url → URL
2021-03-19 02:42:43 +01:00
Claire
43eff898a0
Prepare Mastodon for Rails 6 (#15911)
* Fix misuse of foreign_type

* Fix use of removed "add_template_helper"

* Use response.media_type instead of response.content_type in tests

* Fix CSV export controller test on Rails 6

Rails 6 sets a "filename*" field in the Content-Disposition header to
explicitly encode the filename as UTF-8.

This changes checks the first part of the Content-Disposition header so
it matches in both Rails 5 and Rails 6.

* Fix emoji formatting with Rails 6

* Make emoji output more idiomatic and robust

* Switch from redis-rails gem to built-in Rails redis cache storage
2021-03-17 10:09:55 +01:00
Claire
d8fdbb054e Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/validators/status_length_validator.rb`:
  Upstream changes too close to glitch-soc MAX_CHARS changes, but not a real
  conflict.
  Applied upstream changes.
- `package.json`:
  glitch-soc-only dependency textually too close to a dependency updated
  upstream, not a real conflict.
  Applied upstream changes.
2021-03-02 12:06:58 +01:00
Claire
a8139ab016
Add borders to 📱 and 📲 emojis (#15794) 2021-03-01 21:22:54 +01:00
Claire
679642e26c Merge branch 'main' into glitch-soc/merge-upstream 2021-02-22 15:23:46 +01:00
Justin Tracey
c9e8e1739c
replace all instances of "ends_with?" with "end_with?" (#15745)
The "ends_with?" method is just a Rails alias of Ruby's "end_with?" method.
Using the latter makes the code less brittle.
2021-02-19 09:56:14 +01:00
Justin Tracey
3f8523130d
use host instead of headers to make Rack happy (#15741)
"headers" is provided by Rails, Rack can't rely on it
2021-02-16 15:28:17 +01:00
Cecylia Bocovich
3447bd2f80
Monkey patch Rack::Session to send secure cookies to onions (#15725) 2021-02-14 00:10:52 +01:00
Claire
5e11f3a6e1 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/styles/mastodon/modal.scss`:
  For some reason we changed the file loading path in glitch-soc,
  but now upstream has completely changed how the logo is loaded.
  Applied upstream changes.
2021-02-12 10:28:32 +01:00
Claire
21fb3f3684
Drop dependency on secure_headers, fix response headers (#15712)
* Drop dependency on secure_headers, use always_write_cookie instead

* Fix cookies in Tor Hidden Services by moving configuration to application.rb

* Instead of setting always_write_cookie at boot, monkey-patch ActionDispatch
2021-02-11 23:47:05 +01:00
Claire
6feaad33f2
Fix obsolete digitalocean.rake file breaking rake tasks (#15618) 2021-02-11 02:11:30 +01:00
Claire
4d40685850 Merge branch 'main' into glitch-soc/merge-upstream 2021-02-03 17:02:48 +01:00
Shubhendra Singh Chauhan
c8d11b8bdb
Fixed code quality issues (#15541)
* Added .deepsource.toml

* Removed bad use of `alias`

* Fixed operand order in the binary expression

* Prefixed unused method arguments with an underscore

* Replaced the old OpenSSL algorithmic constants with the newer strings initializers.

* Removed unnecessary UTF-8 encoding comment
2021-01-31 21:26:09 +01:00
ThibG
13d5b81579
Fix “tootctl accounts unfollow” (#15639)
Fixes #15635

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-29 18:38:56 +01:00
Claire
b81710c02c Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `CONTRIBUTING.md`:
  Not a real conflict, glitch-soc quotes the upstream file, which has been
  changed. Update the quote.
2021-01-26 14:01:30 +01:00
luigi
eb51e43fb4
Optimize some regex matching (#15528)
* Use Regex#match?

* Replace =~ too

* Avoid to call match? from Nil

* Keep value of Regexp.last_match
2021-01-22 10:09:08 +01:00
Claire
e202314a75 Merge branch 'master' into glitch-soc/merge-upstream 2021-01-18 13:57:12 +01:00
ThibG
54d4e5252b
Use Rails' index_by where it makes sense (#15542)
* Use Rails' index_by where it makes sense

* Fix tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-12 09:27:38 +01:00
Claire
33d30632fb Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/models/public_feed.rb`:
  Upstream refactored a bit, glitch-soc had specific code for local-only
  statuses.
  Updated glitch-soc's specific code accordingly.
2021-01-11 11:55:42 +01:00
luigi
087ed84367
Optimize map { ... }.compact calls (#15513)
* Optimize map { ... }.compact

using Enumerable#filter_map, supported since Ruby 2.7

* Add poyfill for Enumerable#filter_map
2021-01-10 00:32:01 +01:00
ThibG
9395143126
Fix maintenance script not re-indexing some indexes on textual values (#15515)
* Fix maintenance script not re-indexing some indexes on textual values

Fixes #15475

* Refresh instance view at the end of the maintenance script run

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-09 03:51:32 +01:00
abcang
efffdd3778
Fix rubocop config and warnings (#15503)
* disable NewCops

* update TargetRubyVersion

* Fix Lint/MissingSuper for ActiveModelSerializers::Model

* Fix Lint/MissingSuper for feed

* Fix Lint/FloatComparison

* Do not use instance variables
2021-01-07 09:40:55 +01:00
Claire
90528f43bc Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `config/webpack/configuration.js`:
  Upstream updated the `js-yaml` dependency, which changed how to call it.
  Those changes conflicted because that code is pretty different in glitch-soc
  which has to deal with its more complex theming system.
  Proceeded to the same compatibility changes in glitch-soc's code.
- `package.json` and `yarn.lock`:
  Not really a conflict, just glitch-soc-specific dependencies textually too
  close to some dependencies updated upstream.
2021-01-06 18:08:06 +01:00
Sophie Parker
c7262d12b2
Improve Emoji import (fix #15429) (#15430)
* Improve Emoji import

Skip macOS '._' shadow files in tar archive to speed up import

* Fix codeclimate format issue with whitespace

* Update lib/mastodon/emoji_cli.rb

suggestions from Gargron to improve comment

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Update emoji_cli.rb

Remove extraneous comment (macOS-specific comment now with correct line)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2020-12-30 23:19:55 +01:00
Eugen Rochko
633d175146
Bump version to 3.3.0 (#15433) 2020-12-27 06:00:55 +01:00
ThibG
f1f96ebf02
Fix being able to import more than allowed number of follows (#15384)
* Fix being able to import more than allowed number of follows

Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.

This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).

* Catch validation errors to not re-queue failed follows

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-26 23:52:46 +01:00
Claire
537afa00f3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/lib/feed_manager.rb`:
  Not a real conflict, glitch-soc-only DM-related method
  too close to changed upstream stuff.
  Ported upstream changes.
- `app/services/batched_remove_status_service.rb`:
  Additional logic in glitch-soc to clear DMs from timelines.
  Ported upstream changes and fixed the DM TL clearing logic.
- `app/workers/scheduler/feed_cleanup_scheduler.rb`:
  Additional code in glitch-soc to clear DM timelines.
  Ported upstream changes.
2020-12-23 01:47:45 +01:00
Eugen Rochko
444b21b55f
Bump version to 3.3.0rc3 (#15411) 2020-12-23 01:36:13 +01:00
ThibG
1cf2c3a810
Fix external user creation failing when invite request text is required (#15405)
* Fix external user creation failing when invite request text is required

Also fixes tootctl-based user creation.

* Add test about invites when invite request text is otherwise required

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-22 17:14:32 +01:00
Eugen Rochko
9915d11c0d
Fix unnecessary queries when batch-removing statuses, 100x faster (#15387) 2020-12-22 17:13:55 +01:00
Claire
ec3e2b8f72 Merge branch 'master' into glitch-soc/merge-upstream 2020-12-19 14:52:09 +01:00
ThibG
c4e860277d
Fix "tootctl accounts fix-duplicates" (#15373)
- `pluck_each` cannot be used this way with `group`
- typo

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-19 13:34:16 +01:00
Claire
f9d000ebae Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/services/resolve_url_service.rb`:
  The private toot search by URL hack has been revamped upstream.
  Took upstream's version.
2020-12-19 00:55:12 +01:00
Eugen Rochko
c6598b17d9
Bump version to 3.3.0rc2 (#15365) 2020-12-19 00:28:38 +01:00
ThibG
a60d9335d8
Fix resolving accounts sometimes creating duplicate records for a given AP id (#15364)
* Fix ResolveAccountService accepting mismatching acct: URI

* Set attributes that should be updated regardless of suspension

* Fix key fetching

* Automatically merge remote accounts with duplicate `uri`

* Add tests

* Add "tootctl accounts fix-duplicates"

Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and
merge them under the canonical `acct:` URI.

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 23:26:26 +01:00
ThibG
75944a2f2d
Bump max supported schema version in maintenance script (#15359)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 09:46:59 +01:00
Claire
e4f8679eae Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/models/form/admin_settings.rb`:
  New setting added upstream. Ported it.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream removed RTL classes. Did the same.
- `config/settings.yml`:
  New setting added upstream. Ported it.
2020-12-15 14:27:06 +01:00
ThibG
8485c436d5
Bump max supported schema version in maintenance script (#15349)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-15 12:57:24 +01:00
Eugen Rochko
1045549f85
Add stoplight for object storage failures, return HTTP 503 (#13043) 2020-12-15 12:55:29 +01:00
Eugen Rochko
216b85b053
Fix performance on instances list in admin UI (#15282)
- Reduce duplicate queries
- Remove n+1 queries
- Add accounts count to detailed view
- Add separate action log entry for updating existing domain blocks
2020-12-14 09:06:34 +01:00
Claire
f8d867bac4 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/services/remove_status_service.rb`:
  Conflict caused by us having a distinc Direct timeline.
  Ported upstream changes.
- `app/javascript/mastodon/features/compose/components/compose_form.js`:
  Conflict between glitch-soc's variable character limit and upstream
  refactoring that part of the code.
  Ported upstream changes.
2020-12-05 17:33:37 +01:00
ThibG
fbe7519243
Fix account merging in maintenance script (#15264)
Also include AccountNote and AccountDeletionRequest to the list of classes
needing to be reassigned the merged account.
2020-12-04 02:23:51 +01:00
ThibG
fbff89be03
Fix tootctl ip_blocks add failing if not given a comment (#15263)
Fixes #15261
2020-12-02 21:19:41 +01:00
Eugen Rochko
8e0b98556f
Bump version to 3.3.0rc1 (#15230) 2020-12-01 06:38:05 +01:00
Eugen Rochko
e7f1439b97
Bump version to 3.2.1 (#15227) 2020-11-27 18:29:22 +01:00
Eugen Rochko
ec0edf74d6
Fix post-migration warning about index corruption being too scary (#15223) 2020-11-27 15:41:43 +01:00
Thibaut Girka
e71b16ae2b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  glitch-soc-only dependency textually too close to updated upstream
  dependencies. Updated to upsteam dependencies.
2020-11-25 15:31:45 +01:00
Eugen Rochko
f844386809
Add tootctl accounts merge (#15201)
* Add `tootctl accounts merge`

* Update lib/mastodon/accounts_cli.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-11-23 17:50:16 +01:00
Thibaut Girka
48f0f3ffee Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.github/ISSUE_TEMPLATE/bug_report.md`:
  Upstream added the `bug` label to bug reports.
  Did the same.
- `app/services/fan_out_on_write_service.rb`:
  Upstream put DMs back into timelines, glitch-soc was already doing it.
  Ignored upstream changes.
2020-11-20 13:27:48 +01:00
ThibG
1242e57c27
Deal with collation-related index corruption (#14860)
* Add tootctl maintenance fix-duplicates

This tool goes through the database to detect and fix duplicates.
This operation is very slow and may cause data loss (of data that would be
inaccessible without intervention because of the existing index corruptions).
It tries its best to make sensible decisions, and asks the user in some cases.

* Add warning message in db:migrate hook

* Clear Rails cache after being done with database deduplication

Avoids followers hash cache being incorrect, among other things
2020-11-19 17:37:49 +01:00
Thibaut Girka
0437d70628 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/follower_accounts_controller.rb`:
  Conflict due to upstream changing suspension logic while
  glitch-soc has an extra option to hide followers count.
  Ported upstream changes.
2020-11-08 14:20:35 +01:00
Eugen Rochko
3134691948
Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
Thibaut Girka
412218af2e Merge branch 'master' into glitch-soc/merge-upstream 2020-11-07 18:23:02 +01:00
Eugen Rochko
9b1f2a4b61
Add subresource integrity for JS and CSS assets (#15096)
Fix #2744
2020-11-06 11:56:31 +01:00
Thibaut Girka
ec49aa8175 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Updated upstream, we deleted it to not be flooded by Depandabot.
  Kept deleted.
- `Gemfile.lock`:
  Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
  Upstream updated the serializer to support timed mutes, while
  glitch-soc added a custom API ages ago to get information that
  is already available elsewhere.
  Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
  Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
  Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
  🤷
- `app/models/user.rb`:
  New user setting added upstream while we have glitch-soc-specific
  user settings. Added upstream's user setting.
- `config/settings.yml`:
  Upstream added a new user setting close to a user setting we had
  changed the defaults for. Added the new upstream setting.
- `package.json`:
  Upstream dependency updated “too close” to a glitch-soc-specific
  dependency. No real conflict. Updated the dependency.
2020-10-21 19:10:50 +02:00
Eugen Rochko
5e1364c448
Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Thibaut Girka
a7aedebc31 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
  Upstream changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/models/user.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc not preventing moved accounts from logging
  in.
  Ported upstream changes while keeping the ability for moved accounts to log
  in.
- `app/policies/status_policy.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's ability  to hide followers count.
  Ported upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `package.json`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
2020-09-28 14:13:30 +02:00
ThibG
5bbc9a4f78
Fix downloading remote media files when server returns empty filename (#14867)
Fixes #14817
2020-09-27 22:00:43 +02:00
Eugen Rochko
ed099d8bdc
Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
Eugen Rochko
e6b272e5c9
Change REST API to return empty data for suspended accounts (#14765) 2020-09-11 15:16:29 +02:00
ThibG
91eecd1b3c
Add border around 🕺 emoji (#14769)
Fixes #14768
2020-09-10 19:08:03 +02:00
Thibaut Girka
e5f934ddf0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
  Conflict caused because we have additional code to make sure pinned
  local-only toots don't get rendered on the ActivityPub endpoints.
  Ported upstream changes.
2020-09-07 09:21:38 +02:00
Y.Yamashiro
abc91e5fa6
Change S3 ACL in "tootctl media remove-orphans --fix-permissions" from fixed value to environment file's value. (#14715) 2020-09-02 00:17:58 +02:00
Thibaut Girka
3fefb7909b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `config/webpack/shared.js`:
  Upstream has changed how Tesseract.js gets included and dropped a dependency.
  The conflict is caused by glitch-soc having different code due to its
  theming system.
  Ported upstream changes.
- `lib/mastodon/version.rb`:
  Upstream refactor/code style change in a place we replaced upstream's
  repo URL with ours.
  Ported upstram changes, keeping our repo URL.
- `yarn.lock`:
  Upstream dropped dependencies, one of which was textually too close to
  a glitch-soc-specific dependency. Not a real conflict.
2020-09-01 17:02:18 +02:00
Eugen Rochko
71085987e8
Add --fix-permissions option to tootctl media remove-orphans (#14383) 2020-09-01 03:33:21 +02:00
dependabot[bot]
8972e5f7f6
Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-09-01 03:04:00 +02:00
ThibG
f0b6ddd979
Add border around 🐞 emoji (#14712) 2020-08-31 23:30:27 +02:00
Thibaut Girka
8c3c27bf06 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/accounts_controller.rb`:
  Upstream change too close to a glitch-soc change related to
  instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Minor conflict due to glitch-soc's handling of Direct Messages,
  merged upstream changes.
- `yarn.lock`:
  Not really a conflict, caused by glitch-soc-only dependencies
  being textually too close to updated upstream dependencies.
  Merged upstream changes.
2020-08-30 16:13:08 +02:00
Takeshi Umeda
175cd4f8ef
Fix an error when file_file_size is nil in tootctl media remove (#14657) 2020-08-24 18:09:46 +02:00
Thibaut Girka
da62e350e0 Merge branch 'master' into glitch-soc/merge-upstream 2020-08-13 22:17:29 +02:00
ThibG
8d217d7231
Improve email address validation (#14565)
* Increase DNS timeout from 1 second to 5 seconds for MX check

1 seconds is rather short when using a recursive DNS resolver which
hasn't got a cached result already available. Use 5 seconds instead,
which is the timeout value we use for outgoing HTTP queries.

* Add more precise error messages for invalid e-mail addresses
2020-08-12 12:40:25 +02:00
Thibaut Girka
78fa15d08f Merge branch 'master' into glitch-soc/master
Conflicts:
- `streaming/index.js`:
  Upstream entirely refactored it.
  Ported our changes to upstream's refactor. Hopefuly.
2020-08-11 19:19:27 +02:00
ThibG
decc5b9a78
Fix tootctl media commands not handling snowflake ids for media_attachments (#14536) 2020-08-10 01:51:06 +02:00
Thibaut Girka
b578d70957 Merge branch 'master' into glitch-soc/merge-upstream 2020-08-02 22:29:17 +02:00
ThibG
0ebc8d8626
Fix thumbnail color extraction (#14464)
* Fix contrast calculation for thumbnail color extraction

Luminance calculation was using 0-255 RGB values instead of 0-1 sRGB values,
leading to incorrectly-computed contrast values.

Since we use ColorDiff already, just use its XYZ colorspace conversion code
to get the value.

* Require at least 3:1 contrast for both accent and foreground colors

* Lower required contrast for the accent color
2020-08-02 18:47:44 +02:00
Thibaut Girka
23cd5f2c15 Merge branch 'master' into glitch-soc/merge-upstream 2020-08-02 14:30:28 +02:00
ThibG
a1412491b7
Change content-type to be always computed from file data (#14452)
* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test
2020-08-02 11:21:10 +02:00
Thibaut Girka
2cfa142961 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `yarn.lock`:
  No real conflict, just a glitch-soc-only dependency textually too close to
  one updated upstream. Ported upstream changes.
2020-08-01 20:24:23 +02:00
Eugen Rochko
93dd413a47
Bump version to 3.2.0 (#14395) 2020-07-27 00:07:26 +02:00
Eugen Rochko
d1a6f8d5a8
Bump version to 3.2.0rc2 (#14382) 2020-07-24 18:47:25 +02:00
Thibaut Girka
b9f351d845 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/actions/compose.js`:
  Not a “real” conflict, but change too close to a change we made to
  fix the vanilla WebUI locally pushing authored local-only toots in the
  public TL view.
2020-07-22 13:38:17 +02:00
Takeshi Umeda
101485a41f
Fix mimetype returning nil (#14356) 2020-07-19 13:09:47 +02:00
Thibaut Girka
1c6c40d17c Merge branch 'master' into glitch-soc/merge-upstream 2020-07-16 14:43:50 +02:00
Eugen Rochko
6f798eb574
Bump version to 3.2.0rc1 (#14312) 2020-07-15 21:07:27 +02:00
Thibaut Girka
3f60b096b5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `config/routes.rb`:
  Upstream disabled E2EE routes, which we did earlier, but slightly
  differently. Took upstream's version.
2020-07-15 15:32:40 +02:00
Eugen Rochko
4abe3be321
Change tootctl search deploy algorithm (#14300) 2020-07-14 18:10:35 +02:00
Eugen Rochko
96e89d1ef4
Bump version to 3.1.5 (#14284) 2020-07-10 21:07:30 +02:00
Thibaut Girka
de938023ab Merge branch 'master' into glitch-soc/merge-upstream 2020-07-08 15:23:41 +02:00
ThibG
701e5b9a19
Fix ogg vorbis files with a cover art not being correctly processed (#14255) 2020-07-07 16:32:14 +02:00
Thibaut Girka
2d8be0a6e1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Upstream changed it completely.
  Changed ours to merge upstream's new structure, but
  keeping most of the information.
2020-07-05 19:35:56 +02:00
Eugen Rochko
99f3a55540
Add color extraction for audio thumbnails (#14209) 2020-07-05 18:28:25 +02:00
Eugen Rochko
534da9ba23
Fix audio uploads without embedded image (#14203) 2020-07-03 03:05:32 +02:00
Thibaut Girka
45deca65b3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/public.js`:
  Conflict because part of that file has been split to
  `app/javascript/core/settings.js`. Ported those changes
  there.
2020-07-01 19:23:14 +02:00
Eugen Rochko
6d23d40420
Change Redis#exists calls to Redis#exists? to avoid deprecation warning (#14191) 2020-07-01 19:05:21 +02:00
Eugen Rochko
7aaf2b44ec
Fix remote files not using Content-Type header, streaming (#14184) 2020-06-30 23:58:02 +02:00
Thibaut Girka
9c3b9d5ab9 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Upstream deleted it, I decided to keep it.
- `package.json`:
  Upstream updated a dependency textually too close to wavesurfer.js
  which has been deleted from upstream but is kept in glitch-soc for now.
2020-06-30 23:00:20 +02:00
Eugen Rochko
64aac30733
Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
Eugen Rochko
8c04e37b03
Remove the terms blacklist and whitelist from UX (#14149)
Localization strings:

- "Whitelist mode" -> "Limited federation mode"
- "Blacklist e-mail domain" -> "Block e-mail domain"
- "Whitelist domain" -> "Allow domain for federation"

...And so on

Environment variables (backwards-compatible):

- `WHITELIST_MODE` -> `LIMITED_FEDERATION_MODE`
- `EMAIL_DOMAIN_BLACKLIST` -> `EMAIL_DOMAIN_DENYLIST`
- `EMAIL_DOMAIN_WHITELIST` -> `EMAIL_DOMAIN_ALLOWLIST`

tootctl:

- `tootctl domains purge --whitelist-mode` -> `tootctl domains purge --limited-federation-mode`

Removed badly maintained and no longer relevant .env.production.sample file
2020-06-27 20:20:11 +02:00
Eugen Rochko
4662afe075
Fix help text around tootctl email_domain_blocks (#14147) 2020-06-26 21:28:40 +02:00
Thibaut Girka
a01674fb2b Merge branch 'master' into glitch-soc/merge-upstream 2020-06-26 13:02:14 +02:00
tateisu
370e38ee15
Add tootctl email-domain-blocks (#13589)
* Add tootctl email_domains (block|unblock)

* fix codeclimate issues.

* fix codeclimate issues.

* fix codeclimate issues.

* add list subcommand, remove log_action.

* fix codeclimate issues.

* filter duplicate hostnames,ips before block

* rebase from currnet master branch.
rename email_domains_cli.rb to email_domain_blocks_cli.rb .
rename Mastodon::EmailDomainsCLI to Mastodon::EmailDomainBlocksCLI .
rename command email_domains to email-domain-blocks . (Thor recognizes both of - and _ )
rename subcommand block to add .
rename subcommand unblock to remove .
change the color in list subcommand to while for domain or cyan for childlen.
don't use include() in list subcommand.
suppress console output about succeeded entry.
add console output about count of processed/skipped.
remove capitalization in subcommand description.
remove long_desc in subcommand 'remove'.
remove duplicate where in subcommand 'remove'.

* fix codeclimate issue.
2020-06-25 12:17:10 +02:00
Thibaut Girka
12c8ac9e14 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
  Conflict due to glitch-soc having to take care of local-only
  pinned toots in that controller.
  Took upstream's changes and restored the local-only special
  handling.
- `app/controllers/auth/sessions_controller.rb`:
  Minor conflicts due to the theming system, applied upstream
  changes, adapted the following two files for glitch-soc's
  theming system:
  - `app/controllers/concerns/sign_in_token_authentication_concern.rb`
  - `app/controllers/concerns/two_factor_authentication_concern.rb`
- `app/services/backup_service.rb`:
  Minor conflict due to glitch-soc having to handle local-only
  toots specially. Applied upstream changes and restored
  the local-only special handling.
- `app/views/admin/custom_emojis/index.html.haml`:
  Minor conflict due to the theming system.
- `package.json`:
  Upstream dependency updated, too close to a glitch-soc-only
  dependency in the file.
- `yarn.lock`:
  Upstream dependency updated, too close to a glitch-soc-only
  dependency in the file.
2020-06-09 10:39:20 +02:00
leo60228
e0f55f374c
Add emojis:generate_borders Rake task (#13773)
* Add emojis:generate_borders Rake task

* Address review

* Border all dark emoji

* Combine stroke with filter to reduce artifacting

* Cleanup Camera with Flash

* Add stroke-linejoin="round"

The previous filter and tweaks were effectively a poor imitation of it.
There are no artifacts for any dark emoji now!

* Set stroke-width using property

This fixes old versions of Firefox.

* Store emoji in string instead of array

* Use separate arguments for each path segment

* Remove "background: black;"
2020-06-09 00:12:20 +02:00
ThibG
346d9b3d2e
Add blurhash to preview cards (#13984)
Fixes #13001
2020-06-05 23:10:41 +02:00
ThibG
175fd5b8ba
Fix mastodon:setup choking on env variables containing '%' (#13940)
Fix a regression introduced in #13928, caused by TTY::Command building
shell commands by chaining string substitutions.

Ditch TTY::Command and use system instead (both do shell out).
2020-06-03 20:18:52 +02:00
ThibG
9b7e3b4774
Fix mastodon:setup not running properly when some environment variables are set (#13928)
When using one of the docker-compose containers, mastodon:setup will use the
existing .env.production rather than the generated one during the setup steps.

This is because dotenv does not overwrite env variables that are alreayd
defined, and the docker-compose.yml file loads the environment variables
from .env.production.
2020-06-02 13:57:21 +02:00
Thibaut Girka
27b5143dc4 Merge branch 'master' into glitch-soc/merge-upstream 2020-05-20 18:22:42 +02:00
Stanislas
ce87469d0a
Fix tootctl upgrade storage-schema S3 ACL (#13768) 2020-05-17 17:27:36 +02:00
Eugen Rochko
199bbbcb9f
Fix tootctl media remove-orphans choking on unknown files in storage (#13765)
Fix #13762

Catch tootctl interrupt to prevent confusing stacktrace
2020-05-15 18:41:27 +02:00
Eugen Rochko
2b91a3dac0
Fix tootctl upgrade storage-schema misbehaving (#13761)
- Fix not moving original files of custom emojis
- Fix command failing to move any files with S3 storage
- Fix command marking records as upgraded when move failed

Fix #13594
2020-05-15 17:15:24 +02:00
ThibG
927f9ea499
Fix GifReader exceptions (#13760) 2020-05-15 11:38:12 +02:00
Thibaut Girka
f94aebafe5 Merge branch 'master' into glitch-soc/merge-upstream 2020-05-14 18:49:10 +02:00
Eugen Rochko
06225dee94
Bump version to 3.1.4 (#13750) 2020-05-14 07:46:11 +02:00
Takeshi Umeda
cbfb2c0e71
Fix tootctl media refresh (#13751) 2020-05-14 07:45:52 +02:00
Thibaut Girka
4a70792b4a Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, just a glitch-soc-only dependency too close to a
  dependency that got updated upstream. Updated as well.
- `app/models/status.rb`:
  Not a real conflict, just a change too close to glitch-soc-changed code
  for optionally showing boosts in public timelines.
  Applied upstream changes.
- `app/views/layouts/application.html.haml`:
  Upstream a new, static CSS file, conflict due to glitch-soc's theming
  system, include the file regardless of the theme.
- `config/initializers/content_security_policy.rb`:
  Upstream dropped 'unsafe-inline' from the 'style-src' directive, but
  both files are very different. Removed 'unsafe-inline' as well.
2020-05-10 16:19:56 +02:00
ThibG
34756cc4e0
Fix "tootctl media remove-orphans" crashing on “Import” files (#13685)
* Fix "tootctl media remove-orphans" crashing on “Import” files

* Also remove empty directories when removing orphaned media
2020-05-09 21:06:55 +02:00
Hanage999
ff72c0472f
Fix tootctl upgrade storage-schema failing to delete empty directories (#13593) 2020-05-04 13:51:34 +02:00
Thibaut Girka
a22e6a3683 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/statuses_controller.rb`:
  Upstream disabled the embed controller for reblogs.
  Not a real conflict, but glitch-soc has an extra line to deal
  with its theming system.
  Ported upstream changes.
- `app/javascript/packs/public.js`:
  Upstream made changes to get rid of most inline CSS, this changes
  javascript for public pages, which in glitch are split between
  different files. Ported those changes.
- `app/models/status.rb`:
  Upstream changed the block check in `Status#permitted_for` to
  include domain-block checks. Not a real conflict with glitch-soc,
  but our scope is slightly different, as our scope for
  unauthenticated access do not include instance-local toots.
  Ported upstream changes.
- `app/serializers/rest/instance_serializer.rb`:
  Not a real conflict, upstream added a new field to the instance
  serializer, the conflict is one line above since we added more of
  that.
  Ported upstream changes.
- `app/views/settings/profiles/show.html.haml`:
  Upstream got rid of most inline CSS and moved hidden elements
  to data attributes in the process, in fields were we have
  different values.
  Ported upstream changes while keeping our glitch-specific
  values.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream got rid of inline CSS on an HAML line we treat
  differently, stripping empty text nodes.
  Ported upstream changes to the style attribute, keeping
  the empty text node stripping behavior.
2020-05-03 21:23:49 +02:00
Lerk
157850577f
Add tootctl emoji export (#13534)
* add emoji export command to cli

* fix codeclimate issues

* add error when no matching category was found

* add other suggestions

* exit 1 when no matching category is found

* changes according to suggestions

* 👀

* RubyNein

Y u always autoformat :c
2020-04-27 22:17:49 +02:00
Eugen Rochko
c3ca3801f2
Add separate cache directory for non-local uploads (#12821) 2020-04-26 23:29:08 +02:00
Thibaut Girka
63dc7cfa90 Merge branch 'master' into glitch-soc/merge-upstream 2020-04-20 16:45:40 +02:00
D Anzorge
29f5353f8b
Update Twemoji to 12.1.5 (#13021)
* Update Twemoji to 12.1.5

SVGs from Twemoji repo https://github.com/twitter/twemoji

* Update emoji_map.json to Unicode 12
2020-04-19 23:38:31 +02:00
Thibaut Girka
fdd0beefa7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  We have a different README than upstream, discarded upstream changes
  and kept ours.
2020-04-12 16:50:27 +02:00
Eugen Rochko
c99bc5d51b
Fix regression in tootctl media remove-orphans (#13405)
Fix #13401
2020-04-06 14:11:47 +02:00
Eugen Rochko
0773a512ac
Fix trying to delete already deleted file when post-processing (#13406)
Fix #13403
2020-04-06 14:11:22 +02:00
Thibaut Girka
9101254d0a Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/public.js`:
  Upstream modified code that we split between multiple files due
  to glitch-soc's theming system.
  Ported those changes.
2020-04-05 18:32:06 +02:00
Eugen Rochko
a889756dd5
Bump version to 3.1.3 (#13389) 2020-04-05 06:23:46 +02:00
Thibaut Girka
f3eff922a3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
  Upstream bumped poll option character limit, but we already had
  a higher one, kept ours.
- `app/validators/poll_validator.rb`:
  Upstream bumped poll option character limit, but we already had
  a higher one, kept ours.
- `config/initializers/content_security_policy.rb`:
  Upstream added a rule, the way we compute ours is different, but
  that added rule has been ported.
- `package.json`:
  No real conflict, dependency update. Performed the same update.
- `yarn.lock`:
  No real conflict, dependency update. Performed the same update.
2020-04-02 20:32:00 +02:00
Eugen Rochko
a9a063c0e9
Fix tootctl media remove-orphans ignoring PAPERCLIP_ROOT_PATH (#13375)
Fix #13371
2020-04-02 05:28:51 +02:00
Thibaut Girka
7b435fd9bf Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/services/post_status_service.rb`:
  CW/sensitive logic fixed upstream, but different in glitch-soc.
  Ported the changes accordingly.
2020-03-27 21:54:44 +01:00
ThibG
6c79b7237e
Fix Paperclip using deprecated URI.escape function (#13320)
Monkey-patch Paperclip to perform URL escaping in a slightly more
appropriate way, and get rid of runtime deprecation warnings.
2020-03-26 15:09:16 +01:00
Eugen Rochko
0c8945e5ff
Change tootctl media remove-orphans to work for all classes (#13316)
Change `tootctl media lookup` to not use an interactive prompt
2020-03-26 01:56:41 +01:00
Thibaut Girka
9abb227250 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Our README.md files are completely different. Discarded upstream changes.
- `app/javascript/core/admin.js`:
  Updating rails-ujs, no real conflict, but a comment to close to changed
  code. Various glitch-soc-only files have been updated to match those changes,
  though.
- `package.json`:
  No real conflict, just an additional dependency in glitch-soc that was too
  close to something updated upstream. Took upstream's changes.
2020-03-22 16:10:44 +01:00
ThibG
2c6099125d
Fix videos with unsupported colorspace not being transcoded (#13242) 2020-03-10 11:58:40 +01:00
ThibG
abd8394880
Fix MP4 (H264 + AAC) video files being needlessly re-encoded (#13239) 2020-03-09 23:15:59 +01:00
Eugen Rochko
9660aa4543
Change local media attachments to perform heavy processing asynchronously (#13210)
Fix #9106
2020-03-08 23:56:18 +01:00
Thibaut Girka
c790ecb14d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/api/v1/statuses_controller.rb`:
  Conflict due to upstream adding a new parameter (with_rate_limit),
  too close to glitch-soc's own additional parameter (content_type).
  Added upstream's parameter.
- `app/services/post_status_service.rb`:
  Conflict due to upstream adding a new parameter (rate_limit),
  too close to glitch-soc's own additional parameter (content_type).
  Added upstream's parameter.
- `app/views/settings/preferences/appearance/show.html.haml`:
  Conflict due to us not exposing theme settings here (as we have
  a different flavour/skin menu).
  Took upstream change, while still not exposing theme settings.
- `config/webpack/shared.js`:
  Coding style fixes for a part we have rewritten.
  Discarded upstream changes.
2020-03-08 19:38:53 +01:00
tateisu
4e524218d3
Add --skip-media-remove option to tootctl statuses remove (#13080)
* Add skip_media_remove option to tootctl statuses remove

* Add skip_media_remove option to tootctl statuses remove

Co-authored-by: tateisu <tateisu@juggler.jp>
2020-03-08 16:01:07 +01:00
Thibaut Girka
427ef4948a Merge branch 'master' into glitch-soc/merge-upstream 2020-02-29 12:20:54 +01:00
ThibG
ce17cea221
Fix installation failing when Redis password contains special characters (#13156)
* Add support for special characters in Redis passwords

Fixes #13154

* Refactor
2020-02-29 03:00:43 +01:00
Thibaut Girka
51d326f852 Merge branch 'master' into glitch-soc/merge-upstream 2020-02-27 12:37:15 +01:00
Eugen Rochko
aa8568b7f2
Bump version to 3.1.2 (#13162) 2020-02-27 12:34:07 +01:00
Thibaut Girka
85933bc9ff Merge branch 'master' into glitch-soc/merge-upstream 2020-02-25 14:28:13 +01:00
ThibG
ca7ee9f1cf
Add --reset-password option to tootctl accounts modify (#13126) 2020-02-22 01:29:14 +01:00
Thibaut Girka
e037002401 Merge branch 'master' into glitch-soc/merge-upstream 2020-02-19 23:04:18 +01:00
Marcin Cieślak
bd4bbba98a
Do not suggest to login into the GitLab database (#13084)
As pointed out on Discourse:

https://discourse.joinmastodon.org/t/obscure-wtf-error-message-running-migrations-for-3-1/2524

The message the administrators were getting was telling them
to log in to the "GitLab database" and also mentions MySQL
2020-02-15 12:52:57 +01:00
Eugen Rochko
b752666e3f
Bump version to 3.1.1 (#13069) 2020-02-10 02:55:40 +01:00
Thibaut Girka
dae5e446fe Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile`:
  We updated httplog in a separate commit.
  Took upstream's change which updated it further.
- `Gemfile.lock`:
  We updated httplog in a separate commit.
  Took upstream's change which updated it further.
- `app/lib/sanitize_config.rb`:
  Upstream added better unsupported link stripping,
  while we had different sanitizing configs.
  Took only upstream's link stripping code.
- `config/locales/simple_form.pl.yml`:
  Strings unused in glitch-soc had been removed from
  glitch-soc, reintroduced them even if they are not
  useful, to reduce the risk of later merge conflicts.
2020-02-09 12:15:55 +01:00
Eugen Rochko
d2bcef7f6f Bump version to 3.1.0 (#13053) 2020-02-09 00:02:08 +01:00
Thibaut Girka
369201a425 Merge branch 'master' into glitch-soc/merge-upstream 2020-02-03 09:22:58 +01:00
Eugen Rochko
60e160a02d
Bump version to 3.1.0rc2 (#12999) 2020-01-28 20:56:32 +01:00
Thibaut Girka
9adeaf2bfc Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/statuses_controller.rb`:
  Minor conflict due to theming system
2020-01-24 14:37:06 +01:00
Eugen Rochko
4bae4e972d
Bump version to 3.1.0rc1 (#12932) 2020-01-24 01:14:51 +01:00
Eugen Rochko
f52c988e12
Add announcements (#12662)
* Add announcements

Fix #11006

* Add reactions to announcements

* Add admin UI for announcements

* Add unit tests

* Fix issues

- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"

* Fix scheduler unpublishing announcements before they are due

* Fix filter params not being passed to announcements filter
2020-01-23 22:00:13 +01:00
Jeong Arm
26988be57d Close registration before self-destruct (#12877) 2020-01-21 18:56:18 +01:00
Thibaut Girka
980c004f06 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  No real conflict, glitch-soc-only dependency (redcarpet) too close to an
  upstream one (rdf-normalize)
- `README.md`:
  we have different READMEs, discarded upstream's changes
- `app/views/admin/custom_emojis/index.html.haml`:
  No real conflict, different context because of glitch-soc theming
- `lib/mastodon/statuses_cli.rb`:
  Upstream added code to keep bookmarked statuses, we were already doing so
  with slightly different code. Discarded upstream's changes.
- `package.json`:
  No real conflict, glitch-soc-only dependency (favico.js) too close to
  an upstream one
2020-01-12 15:57:34 +01:00
ThibG
10f1450bba Fix ruby 2.7 warnings about keyword parameters (#12824) 2020-01-11 21:38:02 +01:00
Gomasy
3b08535463 Keep statuses bookmarked by local users in tootctl statuses remove (#12818) 2020-01-11 06:41:37 +01:00
ThibG
817d4a9372 Add --remote-only option to emoji purge (#12810)
Fixes #12804
2020-01-10 00:10:17 +01:00
Thibaut Girka
ab5ff638f2 Merge branch 'master' into glitch-soc/master
Conflicts:
- `config/locales/en.yml`
  No real conflict, upstream added a translatable string “too close” to
  one specific to glitch-soc
- `lib/mastodon/statuses_cli.rb`
  Fixes made upstream, while changed in glitch-soc to keep bookmarked statuses
- `package.json`
  No real conflict, additional dependency in glitch-soc
2019-12-19 13:52:54 +01:00
ThibG
7fe45e7c03 Changes to tootctl statuses remove (#11267)
* Fix comments in statuses_cli

The comment for the part of query keeping statuses which have replies was
incorrect, explaining the part of the query kept favourited statuses instead

* Keep statuses favourited by local users in tootctl statuses remove

* Do delete old replies that are not direct interactions with local users

* Skip reblogs of local statuses instead of all reblogs

* Optimize some queries

* Keep old statuses with recent reblogs

* Add option to clear toots from followed accounts too
2019-12-17 13:33:36 +01:00
trwnh
1c10283217 Fix typo in tootctl statuses remove (#12603) 2019-12-12 19:50:08 +01:00
Thibaut Girka
be4849c083 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/application_controller.rb
  Minor conflict due to glitch-soc's theming system
2019-12-12 15:40:06 +01:00
Eugen Rochko
d7bcc0930c
Fix error handling in tootctl media remove-orphans (#12571) 2019-12-09 04:26:00 +01:00
Eugen Rochko
f3d232381d
Add tootctl media remove-orphans (#12568) 2019-12-08 15:37:12 +01:00
Thibaut Girka
abcba5c198 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- package.json
  Not really a conflict, caused by an additional dependency in glitch-soc.
- yarn.lock
  Not really a conflict, caused by an additional dependency in glitch-soc.
2019-12-07 12:28:59 +01:00
Yamagishi Kazutoshi
6be16d02cb Update ESLint and RuboCop in Code Climate (#12534) 2019-12-02 18:25:43 +01:00
Thibaut Girka
0ae7d84273 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `package.json`
2019-11-06 14:38:28 +01:00
Nathaniel Suchy
23ed9303b8 Add tootctl media lookup command (#12283)
* Add a lookup tool to the media cli

* Improved lookup logic

* Clarified wording in the output

* Code style changes

* Code style changes

* Code style changes

* Code style changes

* Add error handling code incase an attachment isn't found

* Code style changes

* Code style changes

* Make requested changes

* Fix styling issues

* Handle other media types

* Remove an inadvertently added log

* Make requested changes

* Make the code safe no matter what the path, S3 or not

* Code style changes

* Code style changes

* Replace select method with Ruby Enumerable grep method
2019-11-04 12:55:20 +01:00
Thibaut Girka
61631f4751 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
- app/javascript/styles/mastodon/components.scss
  conflicts caused by image URLs being different
- app/models/status.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
- app/views/statuses/_simple_status.html.haml
- config/locales/en.yml
  some strings were changed upstream
- spec/models/status_spec.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
2019-10-10 17:26:08 +02:00
Eugen Rochko
45eccaf8c5
Fix preloaded JSON-LD context for identity not being used (#12138)
Regression from #11316
2019-10-10 06:48:53 +02:00
Eugen Rochko
e6d111f38b
Bump version to 3.0.1 (#12116) 2019-10-09 07:36:57 +02:00
Eugen Rochko
4a98e77d0e
Change tootctl media refresh to skip already downloaded attachments (#12118) 2019-10-08 05:59:10 +02:00
Eugen Rochko
0aaa3afc2d
Add tootctl media usage command (#12115) 2019-10-07 20:04:56 +02:00
Eugen Rochko
6c9b4f6b77
Fix tootctl not allocating enough database connections for main thread (#12097) 2019-10-07 05:05:02 +02:00
Eugen Rochko
38b6c34e32
Fix issues with tootctl's parallelization and progress reporting (#12093) 2019-10-07 04:24:05 +02:00
Jeong Arm
c4fbfaf0b8 Remove unused option from tootctl accounts cull (#12074) 2019-10-06 04:30:07 +02:00
Thibaut Girka
1bbba0016a Merge tag 'v3.0.0' into glitch-soc/merge-upstream 2019-10-04 19:49:43 +02:00
Eugen Rochko
83d3e7733d
Bump version to 3.0.0 (#12000) 2019-10-03 22:44:22 +02:00
Thibaut Girka
a84aedb7a7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream raised max image size from 8MB to 10MB while our limit is
  configurable. Raised the default to 10MB.
2019-10-03 11:10:12 +02:00
Eugen Rochko
bae268b2f6
Bump version to 3.0.0rc3 (#12063) 2019-10-03 04:13:22 +02:00
Eugen Rochko
ca22a22d7f
Fix performance of GIF re-encoding (#12057)
* Change animated GIF detection to not shell out to ImageMagick

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Change video encoding parameters to limit to 10800 video frames

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Limit GIF image size further

Signed-off-by: Eugen Rochko <eugen@zeonfederated.com>

* Always strip metadata from video files

* Fix code style issues
2019-10-03 01:09:12 +02:00
Takeshi Umeda
0ce0baa9b5 Add parallelization to tootctl search deploy (#12051)
* Add parallel gem

* Modify parallel option in tootctl search deploy

* Add paralell option to tootctl search deploy

* Change 1 to false

* Clean up

* Rename --parallel to --processes
2019-10-02 21:50:43 +02:00
Eugen Rochko
c7adf80827
Change tootctl domains purge to accept multiple domains at once (#12046) 2019-10-02 04:49:53 +02:00
Eugen Rochko
5c42f47617
Fix records not being indexed sometimes (#12024)
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
2019-10-01 01:19:11 +02:00