Commit graph

24 commits

Author SHA1 Message Date
ThibG
9d3c6f1849 Improved remote thread fetching ()
* Fetch up to 5 replies when discovering a new remote status

This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.

* Add some tests for ActivityPub::FetchRepliesWorker

* Add specs for ActivityPub::FetchRepliesService

* Serialize up to 5 public self-replies for ActivityPub notes

* Add specs for ActivityPub::NoteSerializer

* Move exponential backoff logic to a worker concern

* Fetch first page of paginated collections when fetching thread replies

* Add specs for paginated collections in replies

* Move Note replies serialization to a first CollectionPage

The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.

* Use pluck(:uri) instead of map(&:uri) to improve performances

* Fix fetching replies when they are in a CollectionPage
2019-02-28 15:22:21 +01:00
Eugen Rochko
a49d43d112
Add scheduled statuses ()
Fix 
2019-01-05 12:43:28 +01:00
Eugen Rochko
24cafd73a2
Lists ()
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
aschmitz
554c2fd8af Clean up reblog tracking keys, related improvements ()
* Clean up reblog-tracking sets from FeedManager

Builds on , with a few minor optimizations and cleanup of sets
after they are no longer needed.

* Update tests, fix multiply-reblogged case

Previously, we would have lost the fact that a given status was
reblogged if the displayed reblog of it was removed, now we don't.

Also added tests to make sure FeedManager#trim cleans up our reblog
tracking keys, fixed up FeedCleanupScheduler to use the right loop,
and fixed the test for it.
2017-10-17 11:45:06 +02:00
Eugen Rochko
e528114c53 Follow-up to and , removing dead code () 2017-09-26 01:06:27 +02:00
Eugen Rochko
cf615abbf9 Add configuration to disable private status federation over PuSH () 2017-08-24 17:51:32 +02:00
Eugen Rochko
412ea87306 Improve ActivityPub/OStatus compatibility ()
*Note: OStatus URIs are invalid for ActivityPub. But we have them for
as long as we want to keep old OStatus-sourced content and as long as
we remain OStatus-compatible.*

- In Announce handling, if object URI is not a URL, fallback to object URL
- Do not use specialized ThreadResolveWorker, rely on generalized handling
- When serializing notes, if parent's URI is not a URL, use parent's URL
2017-08-19 18:44:48 +02:00
Eugen Rochko
b7370ac8ba ActivityPub delivery ()
* Deliver ActivityPub Like

* Deliver ActivityPub Undo-Like

* Deliver ActivityPub Create/Announce activities

* Deliver ActivityPub creates from mentions

* Deliver ActivityPub Block/Undo-Block

* Deliver ActivityPub Accept/Reject-Follow

* Deliver ActivityPub Undo-Follow

* Deliver ActivityPub Follow

* Deliver ActivityPub Delete activities

Incidentally fix 

* Adjust BatchedRemoveStatusService for ActivityPub

* Add tests for ActivityPub workers

* Add tests for FollowService

* Add tests for FavouriteService, UnfollowService and PostStatusService

* Add tests for ReblogService, BlockService, UnblockService, ProcessMentionsService

* Add tests for AuthorizeFollowService, RejectFollowService, RemoveStatusService

* Add tests for BatchedRemoveStatusService

* Deliver updates to a local account to ActivityPub followers

* Minor adjustments
2017-08-13 00:44:41 +02:00
Eugen Rochko
a549d1ae6b Bump to 1.5.0rc1 () 2017-07-24 16:21:08 +02:00
Eugen Rochko
1fcdaafa6f Fix webfinger retries ()
* Do not raise unretryable exceptions in ResolveRemoteAccountService

* Removed fatal exceptions from ResolveRemoteAccountService

Exceptions that cannot be retried should not be raised. New exception
class for those that can be retried (Mastodon::UnexpectedResponseError)
2017-07-20 01:59:07 +02:00
Eugen Rochko
1618b68bfa HTTP signatures ()
* Add Request class with HTTP signature generator

Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06

* Add HTTP signature verification concern

* Add test for SignatureVerification concern

* Add basic test for Request class

* Make PuSH subscribe/unsubscribe requests use new Request class

Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day

* Make all PuSH workers use new Request class

* Make Salmon sender use new Request class

* Make FetchLinkService use new Request class

* Make FetchAtomService use the new Request class

* Make Remotable use the new Request class

* Make ResolveRemoteAccountService use the new Request class

* Add more tests

* Allow +-30 seconds window for signed request to remain valid

* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
2017-07-14 20:41:49 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
7d8e3721ae Overwrite old statuses with reblogs in PrecomputeFeedService () 2017-06-28 14:50:23 +02:00
Eugen Rochko
e17c2e5da5 Batched remove status service ()
* Make Pubsubhubbub::DistributionWorker handle both single stream entry
arguments, as well as arrays of stream entries

* Add BatchedRemoveStatusService, make SuspendAccountService use it

* Improve method names

* Add test

* Add more tests

* Use PuSH payloads of 100 to have a clear mapping of
1000 input statuses -> 10 PuSH payloads

It was nice while it lasted
2017-06-14 18:01:35 +02:00
Matt Jankowski
ec34ec63b1 Specs for cleanup workers ()
* Add spec files for feed and media cleanup workers

* Add coverage for feed and media cleanup schedulers

* Clean up feed and media cleanup workers
2017-05-22 19:36:21 +02:00
Matt Jankowski
d2e0edd721 Add spec coverage for regeneration worker () 2017-05-19 16:55:09 +02:00
Matt Jankowski
8ec8410651 After remote follow worker specs ()
* Add specs for the after remote follow workers

* Refactor the after remote follow workers
2017-05-18 21:10:41 +02:00
Eugen Rochko
657496b5a9 Do not cancel PuSH subscriptions after encountering "permanent" error… ()
* Do not cancel PuSH subscriptions after encountering "permanent" error response

After talking with MMN about it, turns out some servers/php setups do
return 4xx errors while rebooting, so this anti-feature that was meant
to take load off of the hub is doing more harm than good in terms of
breaking subscriptions

* Update delivery_worker.rb
2017-05-14 03:22:48 +02:00
Matt Jankowski
0d70fe2659 Spec coverage and refactor for pubsub/delivery worker ()
* Framework for delivery worker spec

* Refactor of pubsub delivery worker
2017-05-12 20:35:36 +02:00
Matt Jankowski
60ecfb87ae Coverage for domain block worker () 2017-05-11 22:25:15 +02:00
Matt Jankowski
08949cca41 Pubsub confirmation worker spec ()
* Add specs for valid challenge response from pubsub confirmation worker

* Refactor the pubsub confirmation worker
2017-05-11 00:19:37 +02:00
Matt Jankowski
cc9a6a710f Spec feed insert worker ()
* Spec for feed insert worker when missing records

* more specs!

* Refactor feed insert worker
2017-05-10 20:32:05 +02:00
Matt Jankowski
d142544159 Spec coverage and refactor of digest mailer worker () 2017-05-10 15:45:43 +02:00
Eugen Rochko
3a38322a54 Add spec for Pubsubhubbub::DistributionWorker. PuSH-deliver public items ()
to all subscribers. IDN-normalize callback URLs for subscriptions on insert.
2017-05-10 02:55:43 +02:00
Eugen Rochko
b8e166894b Fix Scheduler::SubscriptionsScheduler ()
* Fix Scheduler::SubscriptionsScheduler, add worker test for it

* Change production log level of Sidekiq to "warn" instead of "info"
2017-05-06 13:05:03 +02:00