Commit Graph

39 Commits

Author SHA1 Message Date
Julia 5f675201f2
Merge commit from fork
* enhance: Add a few validation fixes from Sharkey

See the original MR on the GitLab instance:
https://activitypub.software/TransFem-org/Sharkey/-/merge_requests/484

Co-Authored-By: Dakkar <dakkar@thenautilus.net>

* fix: primitive 2: acceptance of cross-origin alternate

Co-Authored-By: Laura Hausmann <laura@hausmann.dev>

* fix: primitive 3: validation of non-final url

* fix: primitive 4: missing same-origin identifier validation of collection-wrapped activities

* fix: primitives 5 & 8: reject activities with non
string identifiers

Co-Authored-By: Laura Hausmann <laura@hausmann.dev>

* fix: primitive 6: reject anonymous objects that were fetched by their id

* fix: primitives 9, 10 & 11: http signature validation
doesn't enforce required headers or specify auth header name

Co-Authored-By: Laura Hausmann <laura@hausmann.dev>

* fix: primitive 14: improper validation of outbox, followers, following & shared inbox collections

* fix: code style for primitive 14

* fix: primitive 15: improper same-origin validation for
note uri and url

Co-Authored-By: Laura Hausmann <laura@hausmann.dev>

* fix: primitive 16: improper same-origin validation for user uri and url

* fix: primitive 17: note same-origin identifier validation can be bypassed by wrapping the id in an array

* fix: code style for primitive 17

* fix: check attribution against actor in notes

While this isn't strictly required to fix the exploits at hand, this
mirrors the fix in `ApQuestionService` for GHSA-5h8r-gq97-xv69, as a
preemptive countermeasure.

* fix: primitive 18: `ap/get` bypasses access checks

One might argue that we could make this one actually preform access
checks against the returned activity object, but I feel like that's a
lot more work than just restricting it to administrators, since, to me
at least, it seems more like a debugging tool than anything else.

* fix: primitive 19 & 20: respect blocks and hide more

Ideally, the user property should also be hidden (as leaving it in leaks
information slightly), but given the schema of the note endpoint, I
don't think that would be possible without introducing some kind of
"ghost" user, who is attributed for posts by users who have you blocked.

* fix: primitives 21, 22, and 23: reuse resolver

This also increases the default `recursionLimit` for `Resolver`, as it
theoretically will go higher that it previously would and could possibly
fail on non-malicious collection activities.

* fix: primitives 25-33: proper local instance checks

* revert: fix: primitive 19 & 20

This reverts commit 465a9fe6591de90f78bd3d084e3c01e65dc3cf3c.

---------

Co-authored-by: Dakkar <dakkar@thenautilus.net>
Co-authored-by: Laura Hausmann <laura@hausmann.dev>
Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
2024-11-21 08:20:09 +09:00
syuilo ff47fef572
feat: リモートサーバーのサーバー情報を収集しないオプション (#14634)
* wip

* wip

* Update FetchInstanceMetadataService.ts

* Update FetchInstanceMetadataService.ts

* Update types.ts
2024-10-13 20:22:16 +09:00
かっこかり 9d026975bc
fix(backend/test): #14558 以降e2eテストがたまに失敗する問題を修正 (#14709)
* fix(backend/test): MisskeyIO#727 以降e2eテストがたまに失敗する問題を修正 (MisskeyIO#735)

* ✌️

---------

Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>
2024-10-05 16:20:44 +09:00
syuilo f0d0cd2e50
wip (#14643) 2024-09-28 18:15:32 +09:00
KOBA789 7134d24c1f
perf(backend): Defer instance metadata update (#14558)
* Defer instance metadata update

* Fix last new line

* Fix typo

* Add license notice

* Fix syntax

* Perform deferred jobs on shutdown

* Fix missing async/await

* Fix typo :)

* Update collapsed-queue.ts

---------

Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
2024-09-26 10:25:20 +09:00
syuilo 023fa30280
refactor/perf(backend): provide metadata statically (#14601)
* wip

* Update ReactionService.ts

* Update ApiCallService.ts

* Update timeline.ts

* Update GlobalModule.ts

* Update GlobalModule.ts

* Update NoteEntityService.ts

* wip

* wip

* wip

* Update ApPersonService.ts

* wip

* Update GlobalModule.ts

* Update mock-resolver.ts

* Update RoleService.ts

* Update activitypub.ts

* Update activitypub.ts

* Update activitypub.ts

* Update activitypub.ts

* Update activitypub.ts

* clean up

* Update utils.ts

* Update UtilityService.ts

* Revert "Update utils.ts"

This reverts commit a27d4be764.

* Revert "Update UtilityService.ts"

This reverts commit e5fd9e004c.

* vuwa-

* Revert "vuwa-"

This reverts commit 0c3bd12472.

* Update entry.ts

* Update entry.ts

* Update entry.ts

* Update entry.ts

* Update jest.setup.ts
2024-09-22 12:53:13 +09:00
syuilo 337b42bcb1 revert 5f88d56d96
バグがある(かつすぐに修正できそうにない) & まだレビュー途中で意図せずマージされたため
2024-07-20 21:33:20 +09:00
tamaina 5f88d56d96
perf(federation): Ed25519署名に対応する (#13464)
* 1. ed25519キーペアを発行・Personとして公開鍵を送受信

* validate additionalPublicKeys

* getAuthUserFromApIdはmainを選ぶ

* ✌️

* fix

* signatureAlgorithm

* set publicKeyCache lifetime

* refresh

* httpMessageSignatureAcceptable

* ED25519_SIGNED_ALGORITHM

* ED25519_PUBLIC_KEY_SIGNATURE_ALGORITHM

* remove sign additionalPublicKeys signature requirements

* httpMessageSignaturesSupported

* httpMessageSignaturesImplementationLevel

* httpMessageSignaturesImplementationLevel: '01'

* perf(federation): Use hint for getAuthUserFromApId (#13470)

* Hint for getAuthUserFromApId

* とどのつまりこれでいいのか?

* use @misskey-dev/node-http-message-signatures

* fix

* signedPost, signedGet

* ap-request.tsを復活させる

* remove digest prerender

* fix test?

* fix test

* add httpMessageSignaturesImplementationLevel to FederationInstance

* ManyToOne

* fetchPersonWithRenewal

* exactKey

* ✌️

* use const

* use gen-key-pair fn. from  '@misskey-dev/node-http-message-signatures'

* update node-http-message-signatures

* fix

* @misskey-dev/node-http-message-signatures@0.0.0-alpha.11

* getAuthUserFromApIdでupdatePersonの頻度を増やす

* cacheRaw.date

* use requiredInputs
https://github.com/misskey-dev/misskey/pull/13464#discussion_r1509964359

* update @misskey-dev/node-http-message-signatures

* clean up

* err msg

* fix(backend): fetchInstanceMetadataのLockが永遠に解除されない問題を修正

Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>

* fix httpMessageSignaturesImplementationLevel validation

* fix test

* fix

* comment

* comment

* improve test

* fix

* use Promise.all in genRSAAndEd25519KeyPair

* refreshAndprepareEd25519KeyPair

* refreshAndfindKey

* commetn

* refactor public keys add

* digestプリレンダを復活させる

RFC実装時にどうするか考える

* fix, async

* fix

* !== true

* use save

* Deliver update person when new key generated (not tested)
https://github.com/misskey-dev/misskey/pull/13464#issuecomment-1977049061

* 循環参照で落ちるのを解消?

* fix?

* Revert "fix?"

This reverts commit 0082f6f8e8.

* a

* logger

* log

* change logger

* 秘密鍵の変更は、フラグではなく鍵を引き回すようにする

* addAllKnowingSharedInboxRecipe

* nanka meccha kaeta

* delivre

* キャッシュ有効チェックはロック取得前に行う

* @misskey-dev/node-http-message-signatures@0.0.3

* PrivateKeyPem

* getLocalUserPrivateKey

* fix test

* if

* fix ap-request

* update node-http-message-signatures

* fix type error

* update package

* fix type

* update package

* retry no key

* @misskey-dev/node-http-message-signatures@0.0.8

* fix type error

* log keyid

* logger

* db-resolver

* JSON.stringify

* HTTP Signatureがなかったり使えなかったりしそうな場合にLD Signatureを活用するように

* inbox-delayed use actor if no signature

* ユーザーとキーの同一性チェックはhostの一致にする

* log signature parse err

* save array

* とりあえずtryで囲っておく

* fetchPersonWithRenewalでエラーが起きたら古いデータを返す

* use transactionalEntityManager

* fix spdx

* @misskey-dev/node-http-message-signatures@0.0.10

* add comment

* fix

* publicKeyに配列が入ってもいいようにする
https://github.com/misskey-dev/misskey/pull/13950

* define additionalPublicKeys

* fix

* merge fix

* refreshAndprepareEd25519KeyPair → refreshAndPrepareEd25519KeyPair

* remove gen-key-pair.ts

* defaultMaxListeners = 512

* Revert "defaultMaxListeners = 512"

This reverts commit f2c412c180.

* genRSAAndEd25519KeyPairではキーを直列に生成する?

* maxConcurrency: 8

* maxConcurrency: 16

* maxConcurrency: 8

* Revert "genRSAAndEd25519KeyPairではキーを直列に生成する?"

This reverts commit d0aada55c1.

* maxWorkers: '90%'

* Revert "maxWorkers: '90%'"

This reverts commit 9e0a93f110.

* e2e/timelines.tsで個々のテストに対するtimeoutを削除, maxConcurrency: 32

* better error handling of this.userPublickeysRepository.delete

* better comment

* set result to keypairEntityCache

* deliverJobConcurrency: 16, deliverJobPerSec: 1024, inboxJobConcurrency: 4

* inboxJobPerSec: 64

* delete request.headers['host'];

* fix

* // node-fetch will generate this for us. if we keep 'Host', it won't change with redirects!

* move delete host

* modify comment

* modify comment

* fix correct → collect

* refreshAndfindKey → refreshAndFindKey

* modify comment

* modify attachLdSignature

* getApId, InboxProcessorService

* TODO

* [skip ci] add CHANGELOG

---------

Co-authored-by: MeiMei <30769358+mei23@users.noreply.github.com>
Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>
2024-07-18 01:28:17 +09:00
tamaina 89b27d8587
fix(federation): InboxにきたCreate, AnnounceのobjectがBearcaps urlだった際はスキップするように (#13610)
* fix(federation): AnnounceのobjectがLike出なかったらキューにためない
Fix https://github.com/misskey-dev/misskey/issues/13552

* revert

* better reason handlings

* result

* improve announce handling

* skip bearcaps

* also announce
2024-05-28 14:36:06 +09:00
anatawa12 83a9aa4533
feat: suspend instance improvements (#13861)
* feat(backend): dead instance detection

* feat(backend): suspend type detection

* feat(frontend): show suspend reason on frontend

* feat(backend): resume federation automatically if the server is automatically suspended

* docs(changelog): 配信停止まわりの改善

* lint: fix lint errors

* Update packages/frontend/src/pages/instance-info.vue

* lint: fix lint error

* chore: suspendedState => suspensionState

---------

Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
2024-05-23 15:55:47 +09:00
Daiki Mizukami d2a5bb39e3
Merge pull request from GHSA-2vxv-pv3m-3wvj
* fix: normalize incoming signed activities

* Tweak style

* Update CHANGELOG.md

* Log compacted activity as well

---------

Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
2024-05-01 16:33:58 +09:00
tamaina eb60460d28
enhance: 禁止ワードチェック強化 (#27)
* enhance: 禁止ワードチェック強化
* リモートの禁止ワードチェックを添付ファイルとユーザーを登録する前に行うなど
  Resolve https://github.com/misskey-dev/misskey/issues/13374
* 禁止ワートの対象の見直し

* performActivityで特定のエラーが出た際にDelayedに追加しないように

* use IdentifiableError

* NoteCreateService.checkProhibitedWords

* https://github.com/misskey-dev/misskey-private/pull/27/files#r1507416135

* remove comment
2024-03-01 20:16:32 +09:00
anatawa12 b36e6b1a77
fix: 禁止キーワードを含むノートがDelayed Queueに追加されて再処理される問題 (#13428)
* refactor: use IdentifiableError instead of NoteCreateService.ContainsProhibitedWordsError

* fix: notes with prohibited words are reprocessed with delay

* docs(changelog): 禁止キーワードを含むノートがDelayed Queueに追加されて再処理される問題

* lint: fix lint errors

* fix: rethrowするべきなのにrethrowし忘れていたのを修正
2024-02-22 00:59:59 +09:00
tamaina c1514ce91d (re) update SPDX-FileCopyrightText
Fix  #13290
2024-02-13 15:59:27 +00:00
tamaina 311c2172d7 Revert "update SPDX-FileCopyrightText"
This reverts commit 9b5aeb76d8.
2024-02-13 15:50:11 +00:00
syuilo 9b5aeb76d8 update SPDX-FileCopyrightText 2024-02-12 11:37:45 +09:00
riku6460 24645e3d3d
enhance(backend): ActivityPub 周りで連合先から HTTP 429 Too Many Requests を受け取った際にジョブをリトライするように (#12917)
* enhance(backend): ActivityPub 周りで HTTP 429 Too Many Requests を受け取った際にリトライするように

* add to changelog

---------

Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2024-01-06 09:40:08 +09:00
woxtu 0bddd0ceae
Remove unnecessary nullish coalescing (#12058) 2023-10-18 09:54:18 +09:00
syuilo 053da10e94 refactor(backend): update directory structure for models 2023-09-20 11:33:36 +09:00
syuilo 792622aead
refactor: prefix Mi for all entities (#11719)
* wip

* wip

* wip

* wip

* Update RepositoryModule.ts

* wip

* wip

* wip

* Revert "wip"

This reverts commit c1c13b37d2.
2023-08-16 17:51:28 +09:00
woxtu 8a6791da3f
refactor(backend): Remove unused injections (#11462)
* Remove unused injections

* Remove unused imports
2023-08-05 10:33:00 +09:00
Shun Sakai c2370a1be6
chore: 著作権とライセンスについての情報を各ファイルに追加する (#11348)
* chore: Add the SPDX information to each file

Add copyright and licensing information as defined in version 3.0 of
the REUSE Specification.

* tweak format

---------

Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2023-07-27 14:31:52 +09:00
syuilo fd7b77c542
enhance(backend): migrate bull to bullmq (#10910)
* wip

* wip

* Update QueueService.ts

* wip

* refactor

* ✌️

* fix

* Update QueueStatsService.ts

* refactor

* Update ApNoteService.ts

* Update mock-resolver.ts

* refactor

* Update mock-resolver.ts
2023-05-29 11:54:49 +09:00
Namekuji d28866f71a
enhance: account migration (#10592)
* copy block and mute then create follow and unfollow jobs

* copy block and mute and update lists when detecting an account has moved

* no need to care promise orders

* refactor updating actor and target

* automatically accept if a locked account had accepted an old account

* fix exception format

* prevent the old account from calling some endpoints

* do not unfollow when moving

* adjust following and follower counts

* check movedToUri when receiving a follow request

* skip if no need to adjust

* Revert "disable account migration"

This reverts commit 2321214c98.

* fix translation specifier

* fix checking alsoKnownAs and uri

* fix updating account

* fix refollowing locked account

* decrease followersCount if followed by the old account

* adjust following and followers counts when unfollowing

* fix copying mutings

* prohibit moved account from moving again

* fix move service

* allow app creation after moving

* fix lint

* remove unnecessary field

* fix cache update

* add e2e test

* add e2e test of accepting the new account automatically

* force follow if any error happens

* remove unnecessary joins

* use Array.map instead of for const of

* ユーザーリストの移行は追加のみを行う

* nanka iroiro

* fix misskey-js?

* ✌️

* 移行を行ったアカウントからのフォローリクエストの自動許可を調整

* newUriを外に出す

* newUriを外に出す2

* clean up

* fix newUri

* prevent moving if the destination account has already moved

* set alsoKnownAs via /i/update

* fix database initialization

* add return type

* prohibit updating alsoKnownAs after moving

* skip to add to alsoKnownAs if toUrl is known

* skip adding to the list if it already has

* use Acct.parse instead

* rename error code

* 🎨

* 制限を5から10に緩和

* movedTo(Uri), alsoKnownAsはユーザーidを返すように

* test api res

* fix

* 元アカウントはミュートし続ける

* 🎨

* unfollow

* fix

* getUserUriをUserEntityServiceに

* ?

* job!

* 🎨

* instance => server

* accountMovedShort, forbiddenBecauseYouAreMigrated

* accountMovedShort

* fix test

* import, pin禁止

* 実績を凍結する

* clean up

* ✌️

* change message

* ブロック, フォロー, ミュート, リストのインポートファイルの制限を32MiBに

* Revert "ブロック, フォロー, ミュート, リストのインポートファイルの制限を32MiBに"

This reverts commit 3bd7be35d8.

* validateAlsoKnownAs

* 移行後2時間以内はインポート可能なファイルサイズを拡大

* clean up

* どうせactorをupdatePersonで更新するならupdatePersonしか移行処理を発行しないことにする

* handle error?

* リモートからの移行処理の条件を是正

* log, port

* fix

* fix

* enhance(dev): non-production環境でhttpサーバー間でもユーザー、ノートの連合が可能なように

* refactor (use checkHttps)

* MISSKEY_WEBFINGER_USE_HTTP

* Environment Variable readme

* NEVER USE IN PRODUCTION

* fix punyHost

* fix indent

* fix

* experimental

---------

Co-authored-by: tamaina <tamaina@hotmail.co.jp>
Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2023-04-30 00:09:29 +09:00
syuilo 34492f3c9a enhance(backend): tweak cache of federated instance
#10631
2023-04-22 19:59:08 +09:00
syuilo 31f3f5f0f0 enhance(backend): リモートサーバーのチャート生成を無効にするオプションを追加 2023-03-24 19:08:08 +09:00
Kisaragi 0ac1fc846b
refactor(backend): 必要ないas anyを消去 (#10293) 2023-03-11 08:51:37 +09:00
Kagami Sascha Rosylight 63df2c851e
refactor: remove all unused imports (#9951)
Co-authored-by: tamaina <tamaina@hotmail.co.jp>
2023-02-16 23:09:41 +09:00
syuilo 1b21bad202 refactor 2023-02-13 15:50:22 +09:00
syuilo f34f9f6ea5 refactor: fix types 2023-02-13 15:28:07 +09:00
tamaina 303519a1bd
enhance: Judge instance block by endsWith (#9263)
* TypeScriptでendsWith

* fix

* SQL?

* バ〜カアホ

* Update packages/backend/src/core/UtilityService.ts

Co-authored-by: Acid Chicken (硫酸鶏) <root@acid-chicken.com>

* add comment

* add description

* Update packages/backend/src/core/UtilityService.ts

Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>

* Update packages/backend/src/core/chart/charts/federation.ts

Co-authored-by: Acid Chicken (硫酸鶏) <root@acid-chicken.com>

* remove comment

* fix

* fix?

* add changelog

* ILIKE, ARRAY

Co-authored-by: Acid Chicken (硫酸鶏) <root@acid-chicken.com>
Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2023-01-13 18:21:07 +09:00
syuilo 4cc71d2443 ✌️ 2023-01-03 09:32:36 +09:00
syuilo c8bd139713 drop lastCommunicatedAt of instance 2023-01-03 09:00:42 +09:00
syuilo bbb49457f9 refactor: introduce bindThis decorator to bind this automaticaly 2022-12-04 15:03:09 +09:00
syuilo 22ccb0fa71 refactor 2022-12-04 10:16:03 +09:00
syuilo 174a8b1b3e fixes 2022-09-23 06:21:31 +09:00
syuilo 01d4d55e78 fix import type 2022-09-21 05:33:11 +09:00
syuilo a2eac9fff6 test 2022-09-19 03:11:50 +09:00
syuilo b75184ec8e
なんかもうめっちゃ変えた 2022-09-18 03:27:08 +09:00