Commit graph

1224 commits

Author SHA1 Message Date
Jenkins
efc9ffcb80 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 04:10:16 +00:00
David Yip
9105b0c954
Introduce html2text for extracting plaintext from statuses. #236.
Unlike strip_tags, html2text will preserve text present in other nodes,
e.g. anchor tags:

    [1] pry(main)> str = '<a href="http://www.example.com">A link</a>'
    => "<a href=\"http://www.example.com\">A link</a>"
    [2] pry(main)> Html2Text.convert(str)
    => "[A link](http://www.example.com)"
    [3] pry(main)> include ActionView::Helpers::SanitizeHelper
    => Object
    [4] pry(main)> strip_tags(str)
    => "A link"

Preserving the href of an anchor allows keyword mutes to also match on
URLs, which is something that the frontend regex filter can currently
do.
2018-02-10 10:36:16 -06:00
Eugen Rochko
3ebc0ad4d3
Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
David Yip
a6fb1c58ee
Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	.env.production.sample
	app/controllers/auth/confirmations_controller.rb
	db/schema.rb
2018-02-04 16:36:19 -06:00
Eugen Rochko
38e0133e1b
Make PAM gem optional, allow configuration over environment (#6415) 2018-02-04 15:05:53 +01:00
Eugen Rochko
26f21fd5a0
CAS + SAML authentication feature (#6425)
* Cas authentication feature

* Config

* Remove class_eval + Omniauth initializer

* Codeclimate review

* Codeclimate review 2

* Codeclimate review 3

* Remove uid/email reconciliation

* SAML authentication

* Clean up code

* Improve login form

* Fix code style issues

* Add locales
2018-02-04 05:42:13 +01:00
David Yip
4c1fd9a19c
Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
      app/javascript/styles/mastodon/components.scss
2018-02-02 08:39:52 -06:00
Alexander
04fef7b888 pam authentication (#5303)
* add pam support, without extra column

* bugfixes for pam login

* document options

* fix code style

* fix codestyle

* fix tests

* don't call remember_me without password

* fix codestyle

* improve checks for pam usage (should fix tests)

* fix remember_me part 1

* add remember_token column because :rememberable requires either a password or this column.

* migrate db for remember_token

* move pam_authentication to the right place, fix logic bug in edit.html.haml

* fix tests

* fix pam authentication, improve username lookup, add comment

* valid? is sometimes not honored, return nil instead trying to authenticate with pam

* update devise_pam_authenticatable2 and adjust code. Fixes sideeffects observed in tests

* update devise_pam_authenticatable gem, fixes for codeconventions, fix finding user

* codeconvention fixes

* code convention fixes

* fix idention

* update dependency, explicit conflict check

* fix disabled password updates if in pam mode

* fix check password if password is present, fix templates

* block registration if account is maintained by pam

* Revert "block registration if account is maintained by pam"

This reverts commit 8e7a083d650240b6fac414926744b4b90b435f20.

* fix identation error introduced by rebase

* block usernames maintained by pam

* document pam settings better

* fix code style
2018-02-02 10:18:55 +01:00
Jenkins
0e10667fbe Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-24 19:17:11 +00:00
Eugen Rochko
d966878e87
Update goldfinger, ostatus2 and http.rb versions (#6337) 2018-01-23 14:30:42 +01:00
David Yip
2ca965c704
Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	app/javascript/styles/mastodon/components.scss
	app/javascript/styles/mastodon/modal.scss
2018-01-19 15:22:10 -06:00
Yamagishi Kazutoshi
7233ac07d2 Add support Ruby v2.5.0 (#6097) 2018-01-19 20:53:30 +01:00
David Yip
89a9d629f7
Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
	Gemfile.lock
2018-01-15 22:17:48 -06:00
Eugen Rochko
5276c0a090
HTML e-mails for UserMailer (#6256)
- premailer gem to turn CSS into inline styles automatically
- rework UserMailer templates
- reword UserMailer templates
2018-01-16 03:29:11 +01:00
Jenkins
933840bebf Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-03 04:17:11 +00:00
Patrick Figel
04ecf44c2f Add confirmation step for email changes (#6071)
* Add confirmation step for email changes

This adds a confirmation step for email changes of existing users.
Like the initial account confirmation, a confirmation link is sent
to the new address.

Additionally, a notification is sent to the existing address when
the change is initiated. This message includes instruction to reset
the password immediately or to contact the instance admin if the
change was not initiated by the account owner.

Fixes #3871

* Add review fixes
2018-01-02 16:55:00 +01:00
Jenkins
6b85dba8da Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-12-26 18:17:16 +00:00
beatrix
127bfda521 add ruby-progressbar to gemfile (fixes #6110) (#6111) 2017-12-26 18:43:52 +01:00
beatrix
65d083338d
Merge pull request #263 from KnzkDev/optimize-paperclip
Optimizing paperclip
2017-12-12 09:15:52 -05:00
ncls7615
c95af71da5 optimize paperclip 2017-12-12 12:29:09 +09:00
Yamagishi Kazutoshi
b0db4dad79 Revert fog-aws (ref #5604) (#5934) 2017-12-09 00:47:52 +01:00
Eugen Rochko
42bcbd36b7 Remove rabl dependency (#5894)
* Remove rabl dependency

* Replicate old Oj configuration
2017-12-06 15:04:49 +09:00
Yamagishi Kazutoshi
47b0c61853 Unify file upload to using fog (#5604) 2017-11-07 14:30:31 +01:00
Yamagishi Kazutoshi
a2a4bf4e78 Update dependencies for Ruby (2017-10-30) (#5566)
* Update better_errors to version 2.4.0

* Update binding_of_caller to version 0.7.3

* Update bootsnap to version 1.1.5

* Update browser to version 2.5.2

* Update capistrano to version 3.10.0

* Update capistrano-bundler to version 1.3.0

* Update capistrano-rbenv to version 2.1.2

* Update capybara to version 2.15.4

* Update cld3 to version 3.2.1

* Update fabrication to version 2.18.0

* Update fog-openstack to version 0.1.22

* Update kaminari to version 1.1.1

* Update lograge to version 0.7.1

* Update nokogiri to version 1.8.1

* Update oj to version 3.3.9

* Update ox to version 2.8.1

* Update parallel_tests to version 2.17.0

* Update pkg-config to version 1.2.8

* Update rspec-rails to version 3.7.1

* Update rubocop to version 0.51.0

* Update scss_lint to version 0.55.0

* Update sidekiq to version 5.0.5

* Update sidekiq-scheduler to version 2.1.10

* Update tzinfo-data to version 1.2017.3

* Update webpacker to version 3.0.2

* bundle update
2017-10-31 12:22:32 +01:00
unarist
a5143df303 Add strong_migrations to production dependency (#5234) 2017-10-06 03:24:54 +02:00
Eugen Rochko
35a8cafa35 Replace self-rolled statsd instrumention with localshred/nsa (#5118) 2017-09-29 03:16:44 +02:00
Yamagishi Kazutoshi
901fc48aae Upgrade Webpacker to version 3.0.1 (#5122) 2017-09-27 14:41:54 +02:00
Eugen Rochko
6e83b5e719 Update brakeman to 3.0 and bundler-audit to 0.6 (#5117) 2017-09-27 03:13:09 +02:00
Eugen Rochko
b982d549f4 Add strong_migrations gem to warn when creating unsafe migrations (#5078) 2017-09-25 02:11:14 +02:00
Akihiko Odaki
48d77ea1eb Fix filterable_languages method of SettingsHelper (#4966) 2017-09-16 14:59:41 +02:00
Grey Baker
a6a206ef85 Bump puma from 3.9.1 to 3.10.0 (#4879)
Bumps [puma](https://github.com/puma/puma) from 3.9.1 to 3.10.0.
- [Release notes](https://github.com/puma/puma/releases/tag/v3.10.0)
- [Changelog](https://github.com/puma/puma/blob/master/History.md)
- [Commits](https://github.com/puma/puma/compare/v3.9.1...v3.10.0)
2017-09-11 23:43:19 +02:00
Grey Baker
bbff144004 Bump rails from 5.1.3 to 5.1.4 (#4875)
Bumps [rails](https://github.com/rails/rails) from 5.1.3 to 5.1.4.
- [Commits](https://github.com/rails/rails/compare/v5.1.3...v5.1.4)
2017-09-11 23:03:14 +02:00
Eugen Rochko
1caf11ddcc Fix language filter codes (#4841)
* Fix language filter codes

CLD3 returns BCP-47 language identifier, filter settings expect
identifiers in the ISO 639-1 format. Convert between formats,
and exclude duplicate languages from filter choices (zh-CN->zh)

* Fix zh name
2017-09-08 12:32:22 +02:00
Adam Thurlow
6994664a13 swift-enable the paperclip! 📎 (#2322) 2017-09-05 23:17:06 +02:00
Eugen Rochko
00840f4f2e Add handling of Linked Data Signatures in payloads (#4687)
* Add handling of Linked Data Signatures in payloads

* Add a way to sign JSON, fix canonicalization of signature options

* Fix signatureValue encoding, send out signed JSON when distributing

* Add missing security context
2017-08-26 13:47:38 +02:00
nullkal
54a04e3658 Update charlock_holmes to 0.7.5 (#4620) 2017-08-17 14:46:53 +02:00
Eugen Rochko
399f9f4a4e Update Goldfinger gem to 2.0 (#4286)
- No masking of HTTP::Error and OpenSSL::SSL::SSLError
- No longer accepts non-HTTPS WebFinger endpoints
2017-07-21 13:40:48 +02:00
Eugen Rochko
a390abdefb Use the same emoji data on the frontend and backend (#4284)
* Use the same emoji data on the frontend and backend

* Move emoji.json to repository, add tests

This way you don't need to install node dependencies if you only
want to run Ruby code
2017-07-21 04:27:40 +02:00
Eugen Rochko
8400bee3b1 Refactor ResolveRemoteAccountService (#4258)
* Refactor ResolveRemoteAccountService

* Remove trailing whitespace

* Use redis locks around critical ResolveRemoteAccountService code

* Add test for race condition of lock
2017-07-19 14:44:04 +02:00
ThibG
05cd37097c Optimize uri normalization (#4212)
* Add dependency on idn-ruby to speed up URI normalization

* Use normalized_host instead of normalize.host when applicable

When we are only interested in the normalized host, calling normalized_host
avoids normalizing the other components of the URI as well as creating a
new object
2017-07-15 17:24:35 +02:00
unarist
8d224ad23b Follow renaming of microformats2 gem (#4203) 2017-07-14 19:57:49 +02:00
Eugen Rochko
e2685ccc81 Fix #4149, fix #1199 - Store emojis as unicode (#4189)
- Use unicode when selecting emoji through picker
- Convert shortcodes to unicode when storing text input server-side
- Do not convert shortcodes in JS anymore
2017-07-14 19:47:53 +02:00
Sorin Davidoi
0c7c188c45 Web Push Notifications (#3243)
* feat: Register push subscription

* feat: Notify when mentioned

* feat: Boost, favourite, reply, follow, follow request

* feat: Notification interaction

* feat: Handle change of public key

* feat: Unsubscribe if things go wrong

* feat: Do not send normal notifications if push is enabled

* feat: Focus client if open

* refactor: Move push logic to WebPushSubscription

* feat: Better title and body

* feat: Localize messages

* chore: Fix lint errors

* feat: Settings

* refactor: Lazy load

* fix: Check if push settings exist

* feat: Device-based preferences

* refactor: Simplify logic

* refactor: Pull request feedback

* refactor: Pull request feedback

* refactor: Create /api/web/push_subscriptions endpoint

* feat: Spec PushSubscriptionController

* refactor: WebPushSubscription => Web::PushSubscription

* feat: Spec Web::PushSubscription

* feat: Display first media attachment

* feat: Support direction

* fix: Stuff broken while rebasing

* refactor: Integration with session activations

* refactor: Cleanup

* refactor: Simplify implementation

* feat: Set VAPID keys via environment

* chore: Comments

* fix: Crash when no alerts

* fix: Set VAPID keys in testing environment

* fix: Follow link

* feat: Notification actions

* fix: Delete previous subscription

* chore: Temporary logs

* refactor: Move migration to a later date

* fix: Fetch the correct session activation and misc bugs

* refactor: Move migration to a later date

* fix: Remove follow request (no notifications)

* feat: Send administrator contact to push service

* feat: Set time-to-live

* fix: Do not show sensitive images

* fix: Reducer crash in error handling

* feat: Add badge

* chore: Fix lint error

* fix: Checkbox label overlap

* fix: Check for payload support

* fix: Rename action "type" (crash in latest Chrome)

* feat: Action to expand notification

* fix: Lint errors

* fix: Unescape notification body

* fix: Do not allow boosting if the status is hidden

* feat: Add VAPID keys to the production sample environment

* fix: Strip HTML tags from status

* refactor: Better error messages

* refactor: Handle browser not implementing the VAPID protocol (Samsung Internet)

* fix: Error when target_status is nil

* fix: Handle lack of image

* fix: Delete reference to invalid subscriptions

* feat: Better error handling

* fix: Unescape HTML characters after tags are striped

* refactor: Simpify code

* fix: Modify to work with #4091

* Sort strings alphabetically

* i18n: Updated Polish translation

it annoys me that it's not fully localized :P

* refactor: Use current_session in PushSubscriptionController

* fix: Rebase mistake

* fix: Set cacheName to mastodon

* refactor: Pull request feedback

* refactor: Remove logging statements

* chore(yarn): Fix conflicts with master

* chore(yarn): Copy latest from master

* chore(yarn): Readd offline-plugin

* refactor: Use save! and update!

* refactor: Send notifications async

* fix: Allow retry when push fails

* fix: Save track for failed pushes

* fix: Minify sw.js

* fix: Remove account_id from fabricator
2017-07-13 22:15:32 +02:00
nullkal
007ab330e6 Use charlock_holmes instead of nkf at FetchLinkCardService (#4080)
* Specs for language detection

* Use CharlockHolmes instead of NKF

* Correct mistakes

* Correct style

* Set hint_enc instead of falling back and strip_tags

* Improve specs

* Add dependencies
2017-07-08 22:44:31 +02:00
Eugen Rochko
8b2cad5637 Refactor JSON templates to be generated with ActiveModelSerializers instead of Rabl (#4090) 2017-07-07 04:02:06 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
a37cf9548c Explicitly require MIME::Types (#4083) 2017-07-05 23:58:03 +02:00
Eugen Rochko
f7301bd5b9 Add overview of active sessions (#3929)
* Add overview of active sessions

* Better display of browser/platform name

* Improve how browser information is stored and displayed for sessions overview

* Fix test
2017-06-25 16:54:30 +02:00
Yamagishi Kazutoshi
53e42bf91e Upgrade Webpacker to version 2.0 (#3729) 2017-06-18 02:57:09 +02:00
Matt Jankowski
5be1214c26 Gem version bumps (#3524)
* Update annotate to version 2.7.2

* Update puma to version 3.9.0

* Update aws-sdk to version 2.9.28

* Update bootsnap to version 1.0.0

* Update nio4r to version 2.1.0

* Update nokogumbo to version 1.4.12

* Update oj to version 3.0.11

* Update pkg-config to version 1.2.3

* Update rubocop to version 0.49.1

* Update sidekiq-scheduler to version 2.1.5
2017-06-05 01:10:13 +02:00
takayamaki
3eedad2737 change sidekiq queueing to bulk push (#3536) 2017-06-04 00:11:15 +02:00
Clworld
18fe77084f bump rails version in Gemfile: '~> 5.0.0' -> '~> 5.1.0' (#3523) 2017-06-02 20:11:13 +02:00
Yamagishi Kazutoshi
41fa53253c Keep ENV['LOCAL_HTTPS'] with ApplicationControllerSpec (fix random fail) (#3479)
* Keep ENV['LOCAL_HTTPS'] with ApplicationControllerSpec (fix random fail)

* use climate_control
2017-05-31 17:09:17 +02:00
Jack Jennings
3a2003ba86 Extract authorization policy for viewing statuses (#3150) 2017-05-29 18:22:22 +02:00
Matt Jankowski
5e2c5e95b6 Version bumps for non-rails gems (#3398)
* Update xpath to version 2.1.0

* Update aws-sdk to version 2.9.25

* Update oj to version 3.0.10

* Update brakeman to version 3.6.2

* Update rufus-scheduler to version 3.4.2

* Update doorkeeper to version 4.2.6

* Update rubocop to version 0.49.0
2017-05-29 18:18:06 +02:00
masarakki
4bd327a0c5 fix-rails-minor-version (#3403) 2017-05-29 17:52:26 +02:00
Eugen Rochko
3b4095cf1b Update bootsnap to 0.3.0 (fix xattr.h error) (#3390) 2017-05-28 16:32:29 +02:00
Eugen Rochko
c2f70829d9 Adds bootsnap. Faster boot time (#3176) 2017-05-20 19:42:27 +02:00
Matt Jankowski
5f727f9068 Update rails to version 5.0.3 (#3054)
* Update rails to version 5.0.3

* Update aws-sdk to version 2.9.17

* Update capistrano to version 3.8.1

* Update climate_control to version 0.2.0

* Update debug_inspector to version 0.0.3

* Update et-orbi to version 1.0.4

* Update oj to version 3.0.7

* Update microformats2 to version 2.9.0

* Update lograge to version 0.5.1

* Clean up usage of Microformats2 object to remove deprecation warnings

* Update microformats2 to version 3.0.1
2017-05-14 16:30:14 +02:00
Yamagishi Kazutoshi
a822f7a05a Stop require of i18n-tasks (#2992) 2017-05-11 15:11:59 +02:00
Jack Jennings
c78e8c01a3 Add pessimistic constraints to gem versions (#2980) 2017-05-11 11:27:01 +02:00
Akihiko Odaki
d5cabfe5c6 Use CLD3 (#2949)
Compact Language Detector v3 (CLD3) is the successor of CLD2, which was
used in the previous implementation. CLD3 includes improvements since CLD2,
and supports newer compilers. On the other hand, it has additional
requirements and cld3-ruby, the FFI of CLD3 for Ruby, is still new and may
be still inmature.

Though CLD3 is named after CLD2, it is implemented with a neural network
model, different from the old implementation, which is based on a Naïve
Bayesian classifier.

CLD3 supports newer compilers, such as GCC 6. CLD2 is not compatible with
GCC 6 because it assigns negative values to varibales typed unsigned.
(see internal/cld_generated_cjk_uni_prop_80.cc) The support for GCC 6 and
newer compilers are essential today, when some server operating system
such as Ubuntu Server 16.10 has GCC 6 by default.

On the one hand, CLD3 requires C++11 support. Environments with old
compilers such as Ubuntu Server 14.04 needs to update the system or install
a newer compiler.

CLD3 needs protocol buffers as a new dependency. However,it is not
considered problematic because major server operating systems, CentOS and
Ubuntu Server provide them.

The FFI cld3-ruby was written by me (Akihiko Odaki) for use in Mastodon.
It is still new and may be inmature, but confirmed to pass existing tests.
2017-05-09 19:58:03 +02:00
alpaca-tc
ef226a6f22 Remove rails_12factor (#2901) 2017-05-09 05:15:54 +02:00
Matt Jankowski
aef554d553 Misc gem bumps (#2919)
* Update bundler-audit to version 0.5.0

* Update oj to version 3.0.6

* Update http-form_data to version 1.0.2

* Update parallel to version 1.11.2

* Update rails-i18n to version 5.0.4

* Update rack-protection to version 2.0.0

* Update haml to version 5.0.1

* Update scss_lint to version 0.53.0

* Relax requirement on i18n-tasks

* Remove haml and haml_lint gem requirement
2017-05-08 18:55:42 +02:00
yhirano
48594b18e6 Added haml-lint and fix warnings (#2773)
* add haml_lint to Gemfile

* add .haml-lint.yml

* fix warnings of haml_lint
2017-05-08 03:35:25 +02:00
yhirano
d63c291f86 Enable CodeClimate SCSS Lint checks (#2886)
* add scss_lint to Gemfile

* add .scss-lint.yml

* fix warnings of scss-lint

* chmod -x styles/variables.scss

* Enable CodeClimate SCSS Lint checks
2017-05-07 20:47:31 +02:00
beatrix
5c1f70b5c5 namespace redis usage (#2869)
* add redis-namespace gem

* namespace redis usage

* refactor redis namespace code to be less intrusive

previously : would be prepended to keys when the
REDIS_NAMESPACE env var was not set

now if it is not set the namespacing functions are
not used at all, which should prevent disruptions
when instances update.

* fix redis namespace variable style in streaming js

* remove trailing space

* final redis namespace style fix
2017-05-07 19:42:32 +02:00
Yamagishi Kazutoshi
f000673599 Remove local_time gem, and add similar (#2875) 2017-05-07 15:22:54 +02:00
yhirano
bfbfaf9f9f Enable CodeClimate Brakeman checks (#2861)
* add brakeman to Gemfile

* Enable CodeClimate brakeman checks

* add config/brakeman.ignore
2017-05-07 02:45:33 +02:00
yhirano
87588fa894 Enable CodeClimate Bundler-Audit checks (#2862)
* add bundler-audit to Gemfile

* Enable CodeClimate bundler-audit checks
2017-05-07 02:41:12 +02:00
Eugen Rochko
81584779cb More robust PuSH subscription refreshes (#2799)
* Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron

Fix an issue where / in domain would raise exception in TagManager#normalize_domain

PuSH subscriptions refresh done in a round-robin way to avoid hammering a single
server's hub in sequence. Correct handling of failures/retries through Sidekiq (see
also #2613). Optimize Account#with_followers scope. Also, since subscriptions
are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire
refreshing operation halfway through

Fix #2702 - Correct user agent header on outgoing http requests

* Add test for SubscribeService

* Extract #expiring_accounts into method

* Make mastodon:push:refresh no-op

* Queues are now defined in sidekiq.yml

* Queues are now in sidekiq.yml
2017-05-05 02:23:01 +02:00
Matt Jankowski
f9d398e8fb Remove the react-rails gem (#2801)
* Remove react-rails gem

* Fix broken view spec
2017-05-04 23:45:18 +02:00
Matt Jankowski
2bd132d458 Replace best_in_place editor on admin settings page (#2789)
* Remove best_in_place

* Replace best_in_place usage with rails helpers

* Move admin/settings#index to #edit action

* Remove click_to__edit from i18n
2017-05-04 18:12:44 +02:00
Matt Jankowski
8c5ad23b24 Language improvements, replace whatlanguage with CLD (#2753)
* add failing en specs

* add cld2 gem

* Replace WhatLanguage with CLD
2017-05-03 16:59:31 +02:00
Keiji Matsuzaki
90c00f075a Added support parallel_tests (#2740)
- Added new gem the `parallel_tests`.
- Updated .travis.yml.
  - Improved scripts and environment variable.
2017-05-03 12:20:38 +02:00
Eugen Rochko
f5bf5ebb82 Replace sprockets/browserify with Webpack (#2617)
* Replace browserify with webpack

* Add react-intl-translations-manager

* Do not minify in development, add offline-plugin for ServiceWorker background cache updates

* Adjust tests and dependencies

* Fix production deployments

* Fix tests

* More optimizations

* Improve travis cache for npm stuff

* Re-run travis

* Add back support for custom.scss as before

* Remove offline-plugin and babili

* Fix issue with Immutable.List().unshift(...values) not working as expected

* Make travis load schema instead of running all migrations in sequence

* Fix missing React import in WarningContainer. Optimize rendering performance by using ImmutablePureComponent instead of
React.PureComponent. ImmutablePureComponent uses Immutable.is() to compare props. Replace dynamic callback bindings in
<UI />

* Add react definitions to places that use JSX

* Add Procfile.dev for running rails, webpack and streaming API at the same time
2017-05-03 02:04:16 +02:00
yhirano
298796cc7b annotate models (#2697)
* add annotate to Gemfile

* rails g annotate:install

* configure annotate_models

* add schema info to models

* fix rubocop to add frozen_string_literal
2017-05-02 02:14:47 +02:00
yhirano
8325866c61 Downgrade rubocop 0.48.1 => 0.46.0 (#2628)
* downgrade rubocop 0.48.1 => 0.46.0

* exclude vendor/**/* from rubocop target files

* add frozen_string_literal comment line

* fix percent literal delimited by ( and )

* fix alignment

* remove comment disabling unknown cop
2017-04-30 00:23:45 +02:00
y-temp4
2ca246d7d1 Use the new Ruby 1.9 hash syntax (#2577) 2017-04-28 09:07:16 -04:00
Eugen Rochko
88725d6ce8 OEmbed support for PreviewCard (#2337)
* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix #1681 - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller
2017-04-27 14:42:22 +02:00
Matt Jankowski
452dc6b5fe Use local_time gem for timezone replacement (#2174)
For the 'time ago' value of stream entry simple statuses and the timestamp value
of detailed statuses - output their UTC value and use the browser timezone to
convert locally.
2017-04-21 18:19:59 +02:00
Eugen
e4af4898de Add language detection (#1772)
* Add language detection via WhatLanguage and (de)serialization of it through Atom

* Fix default language in ProcessFeedService

* Re-add newline before 'react-rails' Gem to fix groupings

Fixes Code Climate issue
2017-04-16 20:32:17 +02:00
Eugen
7e2e0d6dcc Fix #1670 - Update OStatus2 gem (#1936) 2017-04-16 19:32:47 +02:00
Akihiko Odaki
0611209141 Relax Ruby version requirement (#1901) 2017-04-16 14:57:30 +02:00
Alex Dunn
865cb39e9b lock capistrano to 3.8.0 (#1890) 2017-04-16 12:52:18 +02:00
luigi
f97272549c Remove unused gem: coffee-rails (#1885) 2017-04-16 04:02:29 +02:00
Hiromi Kai
7b10794afb Add rails-i18n gem (#1837) 2017-04-15 13:32:53 +02:00
Ben Roberts
89707ad0ac add basic microformats tests (#1803)
as suggested, moving to view tests rather than a controller test
replaces https://github.com/tootsuite/mastodon/pull/1786 which i will
close momentary
2017-04-15 02:37:00 +02:00
Chad Pytel
92cd207c50 Introduce capybara and first feature spec (#1801)
This commit introduces Capybara and the first feature spec.

I focused on coverage for log in for the first feature spec because that would
have prevented 624a9a7136 causing #1236.
2017-04-15 01:21:02 +02:00
Bryce Chidester
067eb220c6 Include pkg-config as a gem dependency. (#1717)
The pkg-config gem was necessary in order for nokogiri to find the system
libraries when building on OpenBSD.

Closes #1637

Signed-off-by: Bryce Chidester <bryce@cobryce.com>
2017-04-14 11:15:03 +02:00
maxypy
a6807201d2 Implementing Subresource Integrity (#1729)
* Add sprockets-rails to Gemfile

* Add sprockets-rails to Gemfile.lock

* Update show.html.haml

* Update index.html.haml

* Update admin.html.haml

* Update auth.html.haml

* Update embedded.html.haml

* Update public.html.haml
2017-04-14 11:09:20 +02:00
Matt Jankowski
4ada50985a Pagination improvements (#1445)
* Replace will_paginate with kaminari

* Use #page instead of #paginate in controllers

* Replace will_paginate.page_gap with pagination.truncate in i18n

* Customize kaminari views to match prior styles

* Set kaminari options to match prior behavior

* Replace will_paginate with paginate in views
2017-04-11 01:11:41 +02:00
Eugen
2810013b93 API param to exclude notification types from response (#1341)
* Add exclude_types param to /api/v1/notifications

* Exclude notification types in web UI through exclude_types in the API
2017-04-10 23:45:29 +02:00
Matt Jankowski
64dbde0dbf Version bumps for ruby and misc gems (#1159)
* Update rspec-rails to version 3.5.2

* Update addressable to version 2.5.1

* Update autoprefixer-rails to version 6.7.7.1

* Update bullet to version 5.5.1

* Update domain_name to version 0.5.20170404

* Update letter_opener_web to version 1.3.1

* Upate redis-rails to version 5.0.2

* Update active_record_query_trace to version 1.5.4

* Update capistrano-rails to version 1.2.3

* Update dotenv-rails to version 2.2.0

* Update pg to version 0.20.0

* Update tilt to version 2.0.7

* Update warden to version 1.2.7

* Update tins to version 1.13.2

* Update terminal-table to version 1.7.3

* Update oj to version 2.18.5

* Update simplecov to version 0.14.1

* Update uglifier to version 3.1.13

* Update hashdiff to version 0.3.2

* Update webmock to version 2.3.2

* Update devise to version 4.2.1

* Use ruby version 2.4.1

* Update sass to version 3.4.23

* Update puma to version 3.8.2

* Update will_paginate to version 3.1.5

* Update font-awesome-rails to version 4.7.0.1

* Update fuubar to version 2.2.0

* Update pry-rails to version 0.3.6

* Update simple-navigation to version 4.0.5

* Update rubocop to version 0.48.1

* Update doorkeeper to version 4.2.5

* Update faker to version 1.7.3

* Update aws-sdk to version 2.9.5

* Update fabrication to version 2.16.1

* Update hamlit-rails to version 0.2.0

* Update http to version 2.2.1

* Update httplog to version 0.99.2

* Update sidekiq to version 4.2.10

* Update rspec-sidekiq to version 3.0.0

* Update pghero to version 1.6.4

* Update rack-cors to version 0.4.1

* Update i18n-tasks to version 0.9.13

* Update ruby-oembed to version 0.12.0

* Update jquery-rails to version 4.3.1

* Update simple_form to version 3.4.0

* Update react-rails to version 1.11.0

* Update aws-sdk to version 2.9.6

* Update sidekiq-unique-jobs to version 5.0.0

* Update uglifier to version 3.2.0
2017-04-10 22:47:41 +02:00
David Authier
f0bd439486 Use HTTP Accept-Language to detect locale (#1166)
* Use HTTP Accept-Language to detect locale

* Fix gem order to comply with codeclimate

* Sort gem to comply with rubocop

* I18n.default_locale fallback when there is no accept-language header
2017-04-09 18:40:24 +02:00
Eugen
6d6a429af8 Rewrite Atom generation from stream entries to use Ox instead of Nokogiri (#1124)
* Rewrite Atom generation from stream entries to use Ox instead of Nokogiri::Builder

StreamEntry is now limited to only statuses, which allows some optimization. Removed
extra queries on AccountsController#show. AtomSerializer instead of AtomBuilderHelper
used in AccountsController#show, StreamEntriesController#show, StreamEntryRenderer
and PubSubHubbub::DistributionWorker

PubSubHubbub::DistributionWorker moves n+1 DomainBlock query to PubSubHubbub::DeliveryWorker
instead.

All Salmon slaps that aren't based on StreamEntry still use AtomBuilderHelper and Nokogiri

* All Salmon slaps now use Ox instead of Nokogiri. No touch from status on account
2017-04-07 05:56:56 +02:00
Kurtis Rainbolt-Greene
03adb5d727 Mastodon isn't using jbuilder or sdoc, and it prevents an upgrade to 2.4.0 2017-04-04 21:31:02 -07:00
Samy KACIMI
5af0ecbcd9 alphebatically order test gem group as required by rubocop 2017-04-05 00:52:55 +02:00
Samy KACIMI
79ef756f64 fix rubocop issues 2017-04-05 00:47:17 +02:00
Samy KACIMI
81c76fe375 add more tests to models 2017-04-05 00:29:56 +02:00
Eugen Rochko
38b504b7a7 Remove sidekiq-merger 2017-04-04 14:28:57 +02:00
Eugen Rochko
82aaedec46 Reduce number of items in feeds, optimize regeneration worker slightly,
make regeneration worker unique, (only schedule/execute once at a time)
2017-04-04 13:58:34 +02:00
Eugen Rochko
ce9df2fa82 Optimize filter methods in FeedManager a bit, use redis pipelining on merge/unmerge feed methods,
do not re-create a dynamic class on each feed push call, make sure redis-rb uses hiredis
2017-04-04 13:01:14 +02:00
Wonderfall
92d35c52d9 update Dockerfile 2017-04-03 00:46:44 +02:00
Eugen Rochko
5b12624847 Add proper error page for request timeouts 2017-04-02 19:43:44 +02:00
Eugen Rochko
7637386228 Upgrade Rails to 5.0.2 2017-03-22 15:37:03 +01:00
Eugen Rochko
ad0d82d3ce Make account search blazing fast and rank followers/followees higher in the results 2017-03-17 20:48:14 +01:00
Eugen Rochko
3202bdd744 Fix #555 - Use a better URL parser 2017-02-22 19:35:11 +01:00
Eugen Rochko
9c88d1b99e Speed up capistrano deployments 2017-02-16 02:34:21 +01:00
Eugen Rochko
920ba5fc4e Fix #61 - Add list of blocked users to the UI; clean up failed push notifications API
Try to fix Travis CI setup
2017-02-05 19:18:11 +01:00
Eugen Rochko
a2a85e8549 Add capistrano to development environment 2017-01-30 20:03:17 +01:00
Eugen Rochko
3f075c7794 API for apps to register for push notifications 2017-01-29 01:30:32 +01:00
Eugen Rochko
ba192f12e3 Added optional two-factor authentication 2017-01-27 20:35:16 +01:00
Eugen Rochko
cc5c1e5feb Removed accidental n+1 query from notifications, updated some deps,
improved how "show more" link looks
2017-01-26 14:52:07 +01:00
Eugen Rochko
d4d56b8af0 Fix #365, 1/2 of #408 - replace rails_autolink with URI.regexp, run link_hashtags on simplified_format 2017-01-23 14:45:09 +01:00
Effy Elden
5fef55c11a Set remote dev environments to use letter_opener_web, add default .env.vagrant variables 2017-01-21 20:22:49 +11:00
Eugen Rochko
f0de621e76 Fix #463 - Fetch and display previews of URLs using OpenGraph tags 2017-01-20 01:00:14 +01:00
Eugen Rochko
f051c2e813 Fix statsd null backend not being initialized properly 2017-01-19 09:37:07 +01:00
Eugen Rochko
306eb6e9c9 Add optional StatsD performance tracking 2017-01-18 23:44:29 +01:00
Eugen Rochko
b11fdc3ae3 Migrate from ledermann/rails-settings to rails-settings-cached which allows global settings
with YAML-defined defaults. Add admin page for editing global settings. Add "site_description"
setting that would show as a paragraph on the frontpage
2017-01-12 20:46:24 +01:00
Eugen Rochko
7951e7ffd5 Add ruby version to Gemfile, move devDependencies in package.json to dependencies,
fix bug in process feed service
2017-01-11 15:39:31 +01:00
Eugen Rochko
c128fcee16 Update Rails to 5.0.1 2016-12-21 19:09:59 +01:00
Eugen Rochko
76ec907993 Improved admin UI 2016-12-13 13:42:10 +01:00
Eugen Rochko
5973ca3d11 Upgrade Paperclip to 5, AWS-SDK to 2, do not generate medium/small versions of avatars 2016-11-29 14:20:15 +01:00
Eugen Rochko
cc70f28f19 Adding rack timeout of 30sec, PuSH jobs moved to push queue so they
can be processed separately
2016-11-29 02:07:14 +01:00
Eugen Rochko
8efa081f21 Remove Neo4J 2016-11-24 23:46:27 +01:00
Eugen Rochko
8e34bed7cc Mini Profiler not working well, remove it 2016-11-24 19:59:11 +01:00
Eugen Rochko
fc90d38893 Moving some counter queries out of subqueries in the API 2016-11-22 22:59:54 +01:00
Eugen Rochko
4b5b953d42 Upgrade to latest Rails, which apparently improves ActionCable performance 2016-11-22 22:26:44 +01:00
Eugen Rochko
546c4718e7 Localizations for most server-side strings 2016-11-16 00:55:33 +01:00
Eugen Rochko
fdc17bea58 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko
09218d4c01 Use full-text search for autosuggestions 2016-11-12 14:36:10 +01:00
Eugen Rochko
448ab18a20 Replace setting custom CORS headers with rack-cors, set it on /oauth/token endpoint 2016-11-09 18:06:01 +01:00
Eugen Rochko
d0d799f911 Replace react-skylight with own solution that centers images better 2016-11-07 18:23:36 +01:00
Eugen Rochko
ea4170785a Downgrade aws gem as this one was incompatible with Paperclip 2016-11-06 20:55:57 +01:00
Eugen Rochko
3ab193bc3f Adding optional S3, fail-mastodon 2016-11-06 18:35:46 +01:00
Eugen Rochko
6471a548fe Add pghero for analyzing postgres performance in production 2016-11-04 18:35:04 +01:00
Eugen Rochko
53fb7b1aa8 Replace in-memory cache store with redis store, should decrease memory
footprint of each running process by 128mb
2016-11-02 13:30:52 +01:00
Eugen Rochko
e21a3fe0cd Adding sync of follow relationships to Neo4J, accounts/suggestions API 2016-10-14 23:10:07 +02:00
Eugen Rochko
06016453bd Adding user settings (model and mailer), no form yet 2016-10-07 13:17:56 +02:00
Eugen Rochko
4909bbf415 Add logging for outgoing http requests 2016-10-02 14:58:06 +02:00
Eugen Rochko
57583f3fb8 Improve partial rendering and Haml performance (Haml->Hamlit) 2016-09-24 13:40:42 +02:00
Eugen Rochko
4bec613897 Fix #24 - Thread resolving for remote statuses
This is a big one, so let me enumerate:

Accounts as well as stream entry pages now contain Link headers that
reference the Atom feed and Webfinger URL for the former and Atom entry
for the latter. So you only need to HEAD those resources to get that
information, no need to download and parse HTML <link>s.

ProcessFeedService will now queue ThreadResolveWorker for each remote
status that it cannot find otherwise. Furthermore, entries are now
processed in reverse order (from bottom to top) in case a newer entry
references a chronologically previous one.

ThreadResolveWorker uses FetchRemoteStatusService to obtain a status
and attach the child status it was queued for to it.

FetchRemoteStatusService looks up the URL, first with a HEAD, tests
if it's an Atom feed, in which case it processes it directly. Next
for Link headers to the Atom feed, in which case that is fetched
and processed. Lastly if it's HTML, it is checked for <link>s to the Atom
feed, and if such is found, that is fetched and processed. The account for
the status is derived from author/name attribute in the XML and the hostname
in the URL (domain). FollowRemoteAccountService and ProcessFeedService
are used.

This means that potentially threads are resolved recursively until a dead-end
is encountered, however it is performed asynchronously over background jobs,
so it should be ok.
2016-09-21 01:50:31 +02:00
Eugen Rochko
af375b5bac Update sidekiq, remove sinatra dependency 2016-09-18 13:40:30 +02:00
Eugen Rochko
ce29624c6d Fixing image upload limits, allowing webm, merge/unmerge events trigger
timeline reload in UI, other small fixes
2016-09-12 18:22:43 +02:00
Eugen Rochko
85d89b472d Optimized n+1 queries in accounts Atom and HTML views
Added stack trace for SQL queries in development
Removed badly thought out accounts/lookup API
2016-09-08 20:36:01 +02:00
Eugen Rochko
0f4bc56719 Improving production logs, removing n+1 on media attachments in atom,
adding attachments display to static views
2016-09-08 01:24:26 +02:00
Eugen Rochko
bc0692d75b Removing mini-profiler that doesn't work, formatting timelines a bit better 2016-08-24 19:23:37 +02:00
Eugen Rochko
49520d6e62 Adding React.js, Redux, revamping dashboard 2016-08-24 17:56:44 +02:00
Eugen Rochko
6deb9f966e Live timelines using ActionCable 2016-08-18 15:49:51 +02:00
Eugen Rochko
10ba09f546 Upgrade to Rails 5.0.0.1 2016-08-17 17:58:00 +02:00
Eugen Rochko
85b00d19b8 Moving Salmon notifications to background processing, fixing mini-profiler
behaviour with Turbolinks enabled, optimizing Rabl for production
2016-03-26 13:42:10 +01:00
Eugen Rochko
318886287b Fixing some stuff for Turbolinks, adding gzip on top, fixing a n+1 query 2016-03-25 15:09:40 +01:00
Eugen Rochko
e24bfbde1a Fixing FanOutOnWriteService, fixing Sidekiq not having enough DB connections
in the pool, adding a throttle of 60rpm per IP, adding mini profiler, adding
admin status to users
2016-03-25 14:12:24 +01:00
Eugen Rochko
42dcb0d4cb Adding Sidekiq for background processing (firstly just of mailers) 2016-03-25 02:50:48 +01:00
Eugen Rochko
f14f462eaf Adding Turbolinks, adding status posting form on homepage 2016-03-21 18:26:47 +01:00
Eugen Rochko
19a259915e Security update 2016-03-21 10:08:19 +01:00
Eugen Rochko
e2b846f630 Adding letter opener for development and Rack::Attack for future rate limiting implementations 2016-03-19 14:57:30 +01:00
Eugen Rochko
c349200761 Removing some unused gems, adding pagination to profiles, fix #10 2016-03-19 12:49:34 +01:00
Eugen Rochko
3441361568 Adding simple_form, adding profile settings, header image 2016-03-12 20:47:22 +01:00
Eugen Rochko
aab9f57e36 Adding config for puma, dashboard layout, fixing some queries 2016-03-12 16:21:53 +01:00
Eugen Rochko
447cfef62d Improving feed queries, switching API to doorkeeper authentication 2016-03-11 16:47:36 +01:00
Eugen Rochko
6c4c84b161 Distrubute statuses as a fan-out-on-write system, with optional precomputing 2016-03-08 20:20:45 +01:00
Eugen Rochko
ab6696e855 Adding doorkeeper, adding a REST API
POST /api/statuses                  Params: status (text contents), in_reply_to_id (optional)
GET  /api/statuses/:id
POST /api/statuses/:id/reblog

GET  /api/accounts/:id
GET  /api/accounts/:id/following
GET  /api/accounts/:id/followers
POST /api/accounts/:id/follow
POST /api/accounts/:id/unfollow

POST /api/follows                  Params: uri (e.g. user@domain)

OAuth authentication is currently disabled, but the API can be used with HTTP Auth.
2016-03-07 12:42:33 +01:00
Eugen Rochko
3824c58853 Adding GNU Public license, adding home timeline, reblog/favourite counters 2016-03-06 17:52:23 +01:00
Eugen Rochko
7e93da3f8d Removing grape and adding devise 2016-03-05 13:12:24 +01:00
Eugen Rochko
0e8f59c16f Refactoring Grape API methods into normal controllers & other things 2016-02-29 19:42:08 +01:00
Eugen Rochko
11ff92c9d7 Adding a test for ReblogService, fixing mentions for remote statuses 2016-02-28 21:22:56 +01:00
Eugen Rochko
2c70f0ecaa Adding paperclip for avatars, fixing design of the public pages 2016-02-28 00:02:59 +01:00
Eugen Rochko
24646d5769 Adding views for the profile and entry pages 2016-02-26 20:48:20 +01:00
Eugen Rochko
f16b31f077 Adding a bunch of tests 2016-02-26 15:28:08 +01:00
Eugen Rochko
71fe24096c Adding a Mention model, test stubs 2016-02-25 00:17:01 +01:00
Eugen Rochko
fa7868675d Send Salmon interactions 2016-02-24 12:57:29 +01:00
Eugen Rochko
709c6685a9 Made some progress 2016-02-22 16:00:20 +01:00
Eugen Rochko
9c4856bdb1 Initial commit 2016-02-20 22:53:20 +01:00