Commit graph

1307 commits

Author SHA1 Message Date
Jeremy Kescher
004b2fc290
Merge remote-tracking branch 'upstream/main' into develop 2023-01-15 20:38:42 +01:00
Claire
3e63fcd4f0 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/status.rb`:
  Minor upstream refactor moved hook definitions around,
  and glitch-soc has an extra `before_create`.
  Moved the `before_create` accordingly.
- `app/services/batched_remove_status_service.rb`:
  Minor upstream refactor changed a block in which glitch-soc
  had one extra call to handle direct timelines.
  Adapted changes to keep glitch-soc's extra call.
2023-01-12 10:15:46 +01:00
Markus Unterwaditzer
0c689b9d01
fix: allow verification when page size exceeds 1MB (using HTML5 parser) (#22879)
* fix: allow verification when page size exceeds 1MB
Truncates the page after 1MB instead

Closes #15316

* switch to HTML5 parser, fix rubocop errors

* undo rubocop fixes

Co-authored-by: Chris Zubak-Skees <chriszs@gmail.com>
2023-01-11 21:59:13 +01:00
Kaspar V
ae62e5fa53
Fix/remove calling private method with send in model (#22951)
* fix(status): remove send usage for private unlink_from_conversations

- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service

* fix(feeds_vacuum): replace find_in_batches with in_batches

because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
2023-01-11 21:57:24 +01:00
Jeremy Kescher
48373a71e3
Merge remote-tracking branch 'upstream/main' into develop
# Conflicts:
#	Gemfile.lock
#	app/javascript/flavours/glitch/components/status_action_bar.js
#	app/javascript/flavours/glitch/containers/status_container.js
#	app/javascript/flavours/glitch/features/status/components/action_bar.js
#	app/javascript/flavours/glitch/locales/de.json
#	app/javascript/flavours/glitch/locales/fr.json
#	app/javascript/mastodon/components/status_action_bar.js
#	app/javascript/mastodon/features/status/components/action_bar.js
#	config/locales-glitch/de.yml
#	config/locales-glitch/fr.yml
#	config/locales-glitch/simple_form.de.yml
#	config/locales-glitch/simple_form.fr.yml
2023-01-05 23:29:53 +01:00
Claire
6c20dad201 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Upstream updated its README, while we have a completely different one.
  Kept our README.
- `app/controllers/concerns/web_app_controller_concern.rb`:
  Conflict because of glitch-soc's theming system.
  Additionally, glitch-soc has different behavior regarding moved accounts.
  Ported some of the changes, but kept our overall behavior.
- `app/javascript/packs/admin.js`:
  Code changes actually applied to `app/javascript/core/admin.js`
2023-01-05 14:16:25 +01:00
Claire
18fb01ef7c
Fix possible race conditions when suspending/unsuspending accounts (#22363)
* Fix possible race conditions when suspending/unsuspending accounts

* Fix tests

Tests were assuming SuspensionWorker and UnsuspensionWorker would do the
suspending/unsuspending themselves, but this has changed.
2023-01-05 13:47:21 +01:00
Effy Elden
4dc9152b3e
Be more lenient with OEmbed detection and validation (#22533) 2023-01-05 13:36:24 +01:00
Jeremy Kescher
1661561640
Merge branch 'reactions' into develop
# Conflicts:
#	Gemfile.lock
#	app/javascript/flavours/glitch/locales/de.js
#	app/javascript/flavours/glitch/locales/en.js
#	app/javascript/flavours/glitch/locales/es.js
#	app/javascript/flavours/glitch/locales/fr.js
#	app/javascript/flavours/glitch/locales/ja.js
#	app/javascript/flavours/glitch/locales/ko.js
#	app/javascript/flavours/glitch/locales/pl.js
#	app/javascript/flavours/glitch/locales/zh-CN.js
#	app/javascript/mastodon/actions/interactions.js
#	app/javascript/mastodon/locales/fr.json
#	db/migrate/20221124114030_create_status_reactions.rb
2022-12-22 00:18:36 +01:00
Jeremy Kescher
2379efb52f
Merge remote-tracking branch 'upstream/main' into reactions
# Conflicts:
#	app/javascript/flavours/glitch/locales/de.js
#	app/javascript/flavours/glitch/locales/en.js
#	app/javascript/flavours/glitch/locales/fr.js
#	app/javascript/mastodon/actions/interactions.js
#	app/javascript/mastodon/locales/fr.json
#	config/locales-glitch/de.yml
#	config/locales-glitch/fr.yml
#	config/locales-glitch/simple_form.de.yml
#	config/locales-glitch/simple_form.fr.yml
2022-12-21 23:45:28 +01:00
Claire
b248aa4d3e Merge branch 'main' into glitch-soc/merge-upstream 2022-12-21 15:59:39 +01:00
fef
f4faa1b511
fix reaction deletion bug and clean up controller
Turns out the strange error where it would delete
the wrong reaction occurred because I forgot to
pass the emoji name to the query, which resulted
in the database deleting the first reaction it
found.  Also, this removes the unused set_reaction
callback and includes the Authorization module for
the status reactions controller.
2022-12-20 18:27:52 +01:00
fef
89a8c21f80
cleanup backend emoji reaction code 2022-12-20 18:24:20 +01:00
fef
19491d2032
cherry-pick emoji reaction changes 2022-12-20 18:23:55 +01:00
fef
738433e46a
make frontend fetch reaction limit
the maximum number of reactions was previously
hardcoded to 8.  this commit also fixes an
incorrect query in StatusReactionValidator where
it didn't count per-user reactions but the total
amount of different ones.
2022-12-20 18:22:05 +01:00
fef
cc71e734c4
federate emoji reactions
this is kind of experimental, but it should work
in theory.  at least i tested it with a remove
akkoma instance and it didn't crash.
2022-12-20 18:22:05 +01:00
Claire
c1de673060
Fix suspension worker crashing on S3-compatible setups without ACL support (#22487) 2022-12-19 17:55:17 +01:00
Jeremy Kescher
0406f173cf
Merge remote-tracking branch 'reactions/main' into develop
# Conflicts:
#	app/javascript/flavours/glitch/locales/de.js
2022-12-17 11:09:32 +01:00
Jeremy Kescher
a1a1d08890
Merge remote-tracking branch 'upstream/main' into develop
# Conflicts:
#	.github/workflows/build-image.yml
#	Gemfile.lock
2022-12-16 17:53:33 +01:00
Claire
f3a4d57be1 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Discarded upstream changes: we have our own README
- `app/controllers/follower_accounts_controller.rb`:
  Port upstream's minor refactoring
2022-12-15 20:25:25 +01:00
Claire
bf1c7e2122
Ensure exact match is the first result in hashtag searches (#21315)
Fixes #17494
2022-12-15 18:41:40 +01:00
Claire
2644a28cb3
Change remote media files to be downloaded outside of transactions (#21796) 2022-12-15 18:09:48 +01:00
Brian Campbell
2d12948220
Fix idempotency when database writes are slow (#21840)
There is an idempotency key generated by clients when authoring a post,
and stored in Redis, to ensure that if a user or client retries posting
the same status, we don't get a duplicate.

Hachyderm.io has been experiencing some filesystem and database
performance issues, causing database writes to be slow. This can mean
that there are successful posts, but the reverse proxy returns 504
Gateway Timeout before the idempotency status has been updated; users or
clients who retry (such as Tusky which retries automatically, see
tuskyapp/Tusky#2951) can re-try the same post with the same idempotency
key before it has actually been recorded in Redis, leading to duplicate
posts.

To address this issue, move all of the database updates after the
initial transaction that creates the status into the
`postprocess_status!` method, so we can insert the idempotency key
immediately after the status has been created, significantly reducing
the window in which the status could be created but the idempotency key
not yet stored.

Note: this has not yet been tested; I'm submitting this PR for
discussion and to offer to the Hachyderm.io admins to try out to fix the
multiple posting problem.

Co-authored-by: Brian Campbell <brcampbell@beta.team>
2022-12-15 18:08:40 +01:00
fef
935788db14
fix reaction deletion bug and clean up controller
Turns out the strange error where it would delete
the wrong reaction occurred because I forgot to
pass the emoji name to the query, which resulted
in the database deleting the first reaction it
found.  Also, this removes the unused set_reaction
callback and includes the Authorization module for
the status reactions controller.
2022-12-09 23:08:42 +01:00
fef
852e6ef195
cleanup backend emoji reaction code 2022-12-09 23:04:12 +01:00
fef
079b0d15c5
cherry-pick emoji reaction changes 2022-12-09 23:04:12 +01:00
fef
4577711201
make frontend fetch reaction limit
the maximum number of reactions was previously
hardcoded to 8.  this commit also fixes an
incorrect query in StatusReactionValidator where
it didn't count per-user reactions but the total
amount of different ones.
2022-12-09 23:04:12 +01:00
fef
5b30421f3b
federate emoji reactions
this is kind of experimental, but it should work
in theory.  at least i tested it with a remove
akkoma instance and it didn't crash.
2022-12-09 23:04:11 +01:00
Claire
c8849d6cee
Fix unbounded recursion in account discovery (#22025)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-07 00:15:24 +01:00
Jeremy Kescher
8a24bef2e3
Fix Redis warnings regarding version 5.0.0 2022-12-04 22:35:48 +01:00
Claire
fe523a3045
Fix unbounded recursion in account discovery (#1994)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-04 21:23:19 +01:00
Claire
a2e7997592 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Our README is completely different.
  Discarded upstream changes.
2022-11-28 11:33:34 +01:00
Claire
f4f2b062ec
Remove LDSignature on actor Delete activities (#21466)
They are currently not used for anything and represent more than half of the
payload size.
2022-11-27 20:43:24 +01:00
Claire
625216d8e1
Fix attachments of edited statuses not being fetched (#21565)
* Fix attachments of edited statuses not being fetched

* Fix tests
2022-11-27 20:39:05 +01:00
Claire
aec61a703f Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `.github/workflows/build-image.yml`:
  Upstream changed how docker images were built, including how
  they were cached.
  I don't know much about it, so applied upstream's changes.
- `app/controllers/admin/domain_blocks_controller.rb`:
  The feature, that was in glitch-soc, got backported upstream.
  It also had a few fixes upstream, so those have been ported!
- `app/javascript/packs/admin.js`:
  Glitch-soc changes have been backported upstream. As a result,
  some code from `app/javascript/core/admin.js` got added upstream.
  Kept our version since our shared Javascript already has that feature.
- `app/models/user.rb`:
  Upstream added something to distinguish unusable and unusable-because-moved
  accounts, while glitch-soc considers moved accounts usable.
  Took upstream's code for `functional_or_moved?` and made `functional?`
  call it.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream cleaned up code style a bit, on a line that we had custom changes
  for.
  Applied upstream's change while keeping our change.
- `config/initializers/content_security_policy.rb`:
  Upstream adopted one CSP directive we already had.
  The conflict is because of our files being structurally different, but the
  change itself was already part of glitch-soc.
  Kept our version.
2022-11-17 12:01:37 +01:00
Joshua Wood
daf6f3453e
Handle links with no href in VerifyLinkService (#20741)
Before this change, the following error would cause VerifyAccountLinksWorker to fail:

NoMethodError: undefined method `downcase' for nil:NilClass
  [PROJECT_ROOT]/app/services/verify_link_service.rb:31 :in `block in link_back_present?`
2022-11-17 10:59:35 +01:00
Claire
32c70d2f09 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/views/admin/announcements/edit.html.haml`:
  Upstream change too close to theming-related glitch-soc change.
  Ported upstream changes.
- `app/views/admin/announcements/new.html.haml`
  Upstream change too close to theming-related glitch-soc change.
  Ported upstream changes.
2022-11-14 08:35:55 +01:00
Hampton Lintorn-Catlin
147d8bd8fc
Support UTF-8 Characters in Domains During CSV Import (#20592)
* Support UTF-8 Characters in Domains During Import

* Update Changelong
2022-11-14 05:52:13 +01:00
Claire
c118918520 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/models/custom_emoji.rb`:
  Not a real conflict, just upstream changing a line too close to
  a glitch-soc-specific validation.
  Applied upstream changes.
- `app/models/public_feed.rb`:
  Not a real conflict, just upstream changing a line too close to
  a glitch-soc-specific parameter documentation.
  Applied upstream changes.
2022-11-10 09:36:47 +01:00
James Tucker
78a6b871fe
Improve performance by avoiding regex construction (#20215)
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
Claire
a5394980f2
Fix NameError in Webfinger redirect handling in ActivityPub::FetchRemoteActorService (#20260) 2022-11-09 20:10:38 +01:00
Eugen Rochko
e98833748e
Fix being able to spoof link verification (#20217)
- Change verification to happen in `default` queue
- Change verification worker to only be queued if there's something to do
- Add `link` tags from metadata fields to page header of profiles
2022-11-09 08:24:21 +01:00
trwnh
b1a48e05b6
Change Report category to "violation" if rule IDs are provided (#20137)
* Change Report category to "violation" if rule IDs are provided

* Fix LiteralAsCondition

* Add parentheses to conditional statement
2022-11-08 17:28:02 +01:00
Claire
a8c854b3ea Merge branch 'main' into glitch-soc/merge-upstream 2022-11-08 15:44:57 +01:00
Claire
bbf74498f5
Fix validation error in SynchronizeFeaturedTagsCollectionWorker (#20018)
* Fix followers count not being updated when migrating follows

Fixes #19900

* Fix validation error in SynchronizeFeaturedTagsCollectionWorker

Also saves remote user's chosen case for hashtags

* Limit remote featured tags before validation
2022-11-07 22:35:53 +01:00
Claire
0ad919b192 Merge branch 'main' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/mastodon/features/compose/components/poll_form.js`:
  glitch-soc change because of having changed the default number of
  available poll options.
  Applied upstream's changes while keeping glitch-soc's default number of
  poll options.
- `public/oops.png`:
  We had a minor graphics change, probably not worth diverging from upstream.
  Took upstream version.
2022-11-06 09:50:41 +01:00
Eugen Rochko
c4b92b1aee
Fix n+1 query during status removal (#19753) 2022-11-05 00:09:52 +01:00
Claire
c2170991c7
Fix reblogs being discarded after the reblogged status (#19731) 2022-11-04 16:31:44 +01:00
Eugen Rochko
5f9e47be34
Add caching for payload serialization during fan-out (#19642) 2022-11-04 13:21:06 +01:00
Claire
4fb0aae636
Change mentions of blocked users to not be processed (#19725)
Fixes #19698
2022-11-04 13:19:12 +01:00