Commit Graph

117 Commits

Author SHA1 Message Date
syuilo 0b062f1407
Misskey® Reactions Buffering Technology™ (#14579)
* wip

* wip

* Update ReactionsBufferingService.ts

* Update ReactionsBufferingService.ts

* wip

* wip

* wip

* Update ReactionsBufferingService.ts

* wip

* wip

* wip

* Update NoteEntityService.ts

* wip

* wip

* wip

* wip

* Update CHANGELOG.md
2024-09-20 21:03:53 +09:00
syuilo 337b42bcb1 revert 5f88d56d96
バグがある(かつすぐに修正できそうにない) & まだレビュー途中で意図せずマージされたため
2024-07-20 21:33:20 +09:00
tamaina 5f88d56d96
perf(federation): Ed25519署名に対応する (#13464)
* 1. ed25519キーペアを発行・Personとして公開鍵を送受信

* validate additionalPublicKeys

* getAuthUserFromApIdはmainを選ぶ

* ✌️

* fix

* signatureAlgorithm

* set publicKeyCache lifetime

* refresh

* httpMessageSignatureAcceptable

* ED25519_SIGNED_ALGORITHM

* ED25519_PUBLIC_KEY_SIGNATURE_ALGORITHM

* remove sign additionalPublicKeys signature requirements

* httpMessageSignaturesSupported

* httpMessageSignaturesImplementationLevel

* httpMessageSignaturesImplementationLevel: '01'

* perf(federation): Use hint for getAuthUserFromApId (#13470)

* Hint for getAuthUserFromApId

* とどのつまりこれでいいのか?

* use @misskey-dev/node-http-message-signatures

* fix

* signedPost, signedGet

* ap-request.tsを復活させる

* remove digest prerender

* fix test?

* fix test

* add httpMessageSignaturesImplementationLevel to FederationInstance

* ManyToOne

* fetchPersonWithRenewal

* exactKey

* ✌️

* use const

* use gen-key-pair fn. from  '@misskey-dev/node-http-message-signatures'

* update node-http-message-signatures

* fix

* @misskey-dev/node-http-message-signatures@0.0.0-alpha.11

* getAuthUserFromApIdでupdatePersonの頻度を増やす

* cacheRaw.date

* use requiredInputs
https://github.com/misskey-dev/misskey/pull/13464#discussion_r1509964359

* update @misskey-dev/node-http-message-signatures

* clean up

* err msg

* fix(backend): fetchInstanceMetadataのLockが永遠に解除されない問題を修正

Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>

* fix httpMessageSignaturesImplementationLevel validation

* fix test

* fix

* comment

* comment

* improve test

* fix

* use Promise.all in genRSAAndEd25519KeyPair

* refreshAndprepareEd25519KeyPair

* refreshAndfindKey

* commetn

* refactor public keys add

* digestプリレンダを復活させる

RFC実装時にどうするか考える

* fix, async

* fix

* !== true

* use save

* Deliver update person when new key generated (not tested)
https://github.com/misskey-dev/misskey/pull/13464#issuecomment-1977049061

* 循環参照で落ちるのを解消?

* fix?

* Revert "fix?"

This reverts commit 0082f6f8e8.

* a

* logger

* log

* change logger

* 秘密鍵の変更は、フラグではなく鍵を引き回すようにする

* addAllKnowingSharedInboxRecipe

* nanka meccha kaeta

* delivre

* キャッシュ有効チェックはロック取得前に行う

* @misskey-dev/node-http-message-signatures@0.0.3

* PrivateKeyPem

* getLocalUserPrivateKey

* fix test

* if

* fix ap-request

* update node-http-message-signatures

* fix type error

* update package

* fix type

* update package

* retry no key

* @misskey-dev/node-http-message-signatures@0.0.8

* fix type error

* log keyid

* logger

* db-resolver

* JSON.stringify

* HTTP Signatureがなかったり使えなかったりしそうな場合にLD Signatureを活用するように

* inbox-delayed use actor if no signature

* ユーザーとキーの同一性チェックはhostの一致にする

* log signature parse err

* save array

* とりあえずtryで囲っておく

* fetchPersonWithRenewalでエラーが起きたら古いデータを返す

* use transactionalEntityManager

* fix spdx

* @misskey-dev/node-http-message-signatures@0.0.10

* add comment

* fix

* publicKeyに配列が入ってもいいようにする
https://github.com/misskey-dev/misskey/pull/13950

* define additionalPublicKeys

* fix

* merge fix

* refreshAndprepareEd25519KeyPair → refreshAndPrepareEd25519KeyPair

* remove gen-key-pair.ts

* defaultMaxListeners = 512

* Revert "defaultMaxListeners = 512"

This reverts commit f2c412c180.

* genRSAAndEd25519KeyPairではキーを直列に生成する?

* maxConcurrency: 8

* maxConcurrency: 16

* maxConcurrency: 8

* Revert "genRSAAndEd25519KeyPairではキーを直列に生成する?"

This reverts commit d0aada55c1.

* maxWorkers: '90%'

* Revert "maxWorkers: '90%'"

This reverts commit 9e0a93f110.

* e2e/timelines.tsで個々のテストに対するtimeoutを削除, maxConcurrency: 32

* better error handling of this.userPublickeysRepository.delete

* better comment

* set result to keypairEntityCache

* deliverJobConcurrency: 16, deliverJobPerSec: 1024, inboxJobConcurrency: 4

* inboxJobPerSec: 64

* delete request.headers['host'];

* fix

* // node-fetch will generate this for us. if we keep 'Host', it won't change with redirects!

* move delete host

* modify comment

* modify comment

* fix correct → collect

* refreshAndfindKey → refreshAndFindKey

* modify comment

* modify attachLdSignature

* getApId, InboxProcessorService

* TODO

* [skip ci] add CHANGELOG

---------

Co-authored-by: MeiMei <30769358+mei23@users.noreply.github.com>
Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>
2024-07-18 01:28:17 +09:00
syuilo 80f3cb96b0
feat: sentry integration (#13897)
* wip

* wip

* wip

* wip

* Update CHANGELOG.md

* Update ApiCallService.ts

* Update config.ts
2024-05-28 17:06:33 +09:00
FineArchs e4eaf1220e
Update example.yml (#13551) 2024-03-09 17:55:41 +09:00
Acid Chicken (硫酸鶏) acba96c1d3
feat: license violation protection (#13285)
* spec(frontend): aboutページにリポジトリ・フィードバックのURLを表示させる

Cherry-picked from MisskeyIO#441
Cherry-picked from MisskeyIO#438

* feat: license violation protection

* build: fix typo

* build: fix typo

* fix: farewell to the static type land

* fix: key typo

* fix: import typo

* fix: properly interpret `prominently`

* docs: add disclaimer

* docs: update CHANGELOG

* chore: add gap

---------

Co-authored-by: まっちゃとーにゅ <17376330+u1-liquid@users.noreply.github.com>
Co-authored-by: syuilo <4439005+syuilo@users.noreply.github.com>
2024-02-17 13:34:50 +09:00
woxtu cdac3988b5
fix(backend): Fix typos in job configurations (#13086)
* Fix typos

* Update CHANGELOG
2024-01-28 15:08:45 +09:00
Camilla Ett 06ca63f9c2
Fix(backend): inboxJobPerSecのデフォルト値を16から32に (#12631) 2023-12-13 08:14:34 +09:00
Ryan He e926411812
chore: Add descriptions for "MeiliSearch" and "allowedPrivateNetworks" to example.yml (#12594)
* Update example.yml, add descriptions for some items

Add descriptions for "MeiliSearch" and "allowedPrivateNetworks"

* Update docker_example.yml

Add descriptions for "MeiliSearch" and "allowedPrivateNetworks"
2023-12-07 17:00:34 +09:00
syuilo 6277a5545c
feat: improve tl performance (#11946)
* wip

* wip

* wip

* wip

* wip

* wip

* Update NoteCreateService.ts

* wip

* wip

* wip

* wip

* Update NoteCreateService.ts

* wip

* Update NoteCreateService.ts

* wip

* Update user-notes.ts

* wip

* wip

* wip

* Update NoteCreateService.ts

* wip

* Update timeline.ts

* Update timeline.ts

* Update timeline.ts

* Update timeline.ts

* Update timeline.ts

* wip

* Update timelines.ts

* Update timelines.ts

* Update timelines.ts

* wip

* wip

* wip

* Update timelines.ts

* Update misskey-js.api.md

* Update timelines.ts

* Update timelines.ts

* wip

* wip

* wip

* Update timelines.ts

* wip

* Update timelines.ts

* wip

* test

* Update activitypub.ts

* refactor: UserListJoining -> UserListMembership

* Update NoteCreateService.ts

* wip
2023-10-03 20:26:11 +09:00
CyberRex 5ad0906c89
feat(backend): MasterプロセスのPIDを書き出せるように (#11909) 2023-09-27 09:32:36 +09:00
syuilo ca00a08e6e feat: introduce aidx and make it default
Co-Authored-By: MeiMei <30769358+mei23@users.noreply.github.com>
2023-09-06 18:33:51 +09:00
Essem 3eacbe6b6d
enhance(backend): Add address bind config option (#11725)
* Add address bind config option

* Edit changelog

* forgot to add to types
2023-08-20 13:20:01 +09:00
tamaina 2217a3c549
chore: cacheRemoteFilesのデフォルト値をfalseに (#11398)
* chore: cacheRemoteFilesのデフォルト値をfalseに

* proxyRemoteFiles to default

* update ja-JP.yml

* update ja-JP.yml

* youCanCleanRemoteFilesCache

* changelog

---------

Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2023-08-05 14:03:01 +09:00
tamaina 0a06eb27da
enhance(backend): 設定ファイルでioredisの全てのオプションを指定可能に (#11329)
* enhance(backend): 設定ファイルでioredisの全てのオプションを指定可能に

* yappa kousuru

* fix

* fix?

* fix

* Revert "fix"

This reverts commit 227f19ff3a.

* fix
2023-07-20 19:50:31 +09:00
dogcraft 5dab918999
enhance(backend): add unix socket support (#11275)
* unix socket support

* add changelog

* lint

---------

Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2023-07-17 14:12:02 +09:00
まっちゃとーにゅ 9e330c9e38
feat: MeilisearchにIndexするノートの範囲を設定できるように (#11282) 2023-07-15 09:59:19 +09:00
syuilo 1cc106b8de fix(backend): ひとつのMeilisearchサーバーを複数のMisskeyサーバーで使えない問題を修正 2023-05-11 21:09:29 +09:00
Caipira ae21b75687
fix(backend): Use SSL option for Meilisearch (#10772) 2023-05-06 04:02:34 +09:00
syuilo 5c08f2b93b
feat: Introduce Meilisearch (#10755)
* wip

* wip

* Update SearchService.ts

* Update SearchService.ts

* wip

* wip

* Update SearchService.ts

* Update CHANGELOG.md

* wip

* Update SearchService.ts

* Update docker-compose.yml.example
2023-05-05 08:52:14 +09:00
Namekuji d28866f71a
enhance: account migration (#10592)
* copy block and mute then create follow and unfollow jobs

* copy block and mute and update lists when detecting an account has moved

* no need to care promise orders

* refactor updating actor and target

* automatically accept if a locked account had accepted an old account

* fix exception format

* prevent the old account from calling some endpoints

* do not unfollow when moving

* adjust following and follower counts

* check movedToUri when receiving a follow request

* skip if no need to adjust

* Revert "disable account migration"

This reverts commit 2321214c98.

* fix translation specifier

* fix checking alsoKnownAs and uri

* fix updating account

* fix refollowing locked account

* decrease followersCount if followed by the old account

* adjust following and followers counts when unfollowing

* fix copying mutings

* prohibit moved account from moving again

* fix move service

* allow app creation after moving

* fix lint

* remove unnecessary field

* fix cache update

* add e2e test

* add e2e test of accepting the new account automatically

* force follow if any error happens

* remove unnecessary joins

* use Array.map instead of for const of

* ユーザーリストの移行は追加のみを行う

* nanka iroiro

* fix misskey-js?

* ✌️

* 移行を行ったアカウントからのフォローリクエストの自動許可を調整

* newUriを外に出す

* newUriを外に出す2

* clean up

* fix newUri

* prevent moving if the destination account has already moved

* set alsoKnownAs via /i/update

* fix database initialization

* add return type

* prohibit updating alsoKnownAs after moving

* skip to add to alsoKnownAs if toUrl is known

* skip adding to the list if it already has

* use Acct.parse instead

* rename error code

* 🎨

* 制限を5から10に緩和

* movedTo(Uri), alsoKnownAsはユーザーidを返すように

* test api res

* fix

* 元アカウントはミュートし続ける

* 🎨

* unfollow

* fix

* getUserUriをUserEntityServiceに

* ?

* job!

* 🎨

* instance => server

* accountMovedShort, forbiddenBecauseYouAreMigrated

* accountMovedShort

* fix test

* import, pin禁止

* 実績を凍結する

* clean up

* ✌️

* change message

* ブロック, フォロー, ミュート, リストのインポートファイルの制限を32MiBに

* Revert "ブロック, フォロー, ミュート, リストのインポートファイルの制限を32MiBに"

This reverts commit 3bd7be35d8.

* validateAlsoKnownAs

* 移行後2時間以内はインポート可能なファイルサイズを拡大

* clean up

* どうせactorをupdatePersonで更新するならupdatePersonしか移行処理を発行しないことにする

* handle error?

* リモートからの移行処理の条件を是正

* log, port

* fix

* fix

* enhance(dev): non-production環境でhttpサーバー間でもユーザー、ノートの連合が可能なように

* refactor (use checkHttps)

* MISSKEY_WEBFINGER_USE_HTTP

* Environment Variable readme

* NEVER USE IN PRODUCTION

* fix punyHost

* fix indent

* fix

* experimental

---------

Co-authored-by: tamaina <tamaina@hotmail.co.jp>
Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2023-04-30 00:09:29 +09:00
syuilo 9114c8cb8e feat(backend): support replication of postgresql
Resolve #10205
2023-04-08 15:53:36 +09:00
syuilo 239d3f2dbf feat(backend): ジョブキュー用Redisを別サーバーに分離できるように 2023-04-07 11:27:01 +09:00
syuilo ff6d9d2860 feat(backend): イベント用Redisを別サーバーに分離できるように 2023-04-07 11:20:14 +09:00
syuilo 5b3a07ee9e Revert "Allow configuring the listen host (#9924)"
This reverts commit 3dfe3aa9a4.
2023-02-22 18:00:35 +09:00
ledlamp 3dfe3aa9a4
Allow configuring the listen host (#9924) 2023-02-22 17:51:40 +09:00
tamaina ee03ab8d2c
enhance(server): videoThumbnailGenerator config (#9845)
* enhance(server): videoThumbnailGenerator config

* ✌️

* fix

* 相対url

* サムネイルのproxyRemoteFilesは直接プロキシを指定する

* メディアプロキシ
2023-02-12 09:13:47 +09:00
tamaina 2dfed75402
perf(server): improvement of external mediaProxy (#9787)
* perf(server): improvement of external mediaProxy

* add a comment

* ✌️

* /filesでsharpの処理を行わずリダイレクトする

* fix

* thumbnail => static

* Fix #9788

* add avatar mode

* add url

* fix

* static.webp

* remove encodeURIComponent from media proxy path

* remove existance check
2023-02-04 13:38:51 +09:00
syuilo 825551d64f drop syslog
Close #9774
2023-02-03 15:08:36 +09:00
tamaina 978a9bbb3b
perf(backend): Use undici instead of node-fetch and got (#9459)
* Implement? HttpFetchService

* ✌️

* remove node-fetch

* fix

* refactor

* fix

* gateway timeout

* UndiciFetcherクラスを追加 (仮コミット, ビルドもstartもさせていない)

* fix

* add logger and fix url preview

* fix ip check

* enhance logger and error handling

* fix

* fix

* clean up

* Use custom fetcher for ApRequest / ApResolver

* bypassProxyはproxyBypassHostsに判断を委譲するように

* set maxRedirections (default 3, ApRequest/ApResolver: 0)

* fix comment

* handle error s3 upload

* add debug message

* no return await

* Revert "no return await"

This reverts commit b5b0dc58a3.

* reduce maxSockets

* apResolverのUndiciFetcherを廃止しapRequestのものを使う、 add ap logger

* Revert "apResolverのUndiciFetcherを廃止しapRequestのものを使う、 add ap logger"

This reverts commit 997243915c.

* add logger

* fix

* change logger name

* safe

* デフォルトでUser-Agentを設定
2023-01-12 21:03:02 +09:00
syuilo 3402131b7d chore(server): make signToActivityPubGet true by default
Close #9376
2022-12-21 15:30:06 +09:00
MeiMei 6f8e3fe366
enhance: Redisをioredisに統一してIPv6サポート (#8869)
* Use ioredis, Supports IPv6 host

https://github.com/misskey-dev/misskey/issues/8862

* Fix import

* order

* a

* i

* fix

* flushdb

* family

* CHANGELOG

* redis_version

Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2022-06-24 19:22:19 +09:00
Johann150 26d90cd030
remove HTTPS handling (#8380) 2022-03-08 23:23:18 +09:00
tamaina e314be5b59
Fix avatar/banner proxy (#8346)
* Fix avatar/banner proxy

Co-authored-by: mei23 <m@m544.net>

* use getAvatarUrl

* fix

* join avatar and banner to improve performance

* join

* Update hybrid-timeline.ts

* fix

Co-authored-by: mei23 <m@m544.net>
Co-authored-by: syuilo <Syuilotan@yahoo.co.jp>
2022-02-27 13:59:10 +09:00
MeiMei e21ff916b0
ファイルサイズのハードリミット (#7760)
* maxFileSize

* CHANGELOG
2021-09-04 20:33:14 +09:00
MeiMei e1a8b158e0 Tune download (#2)
* s2-2

* allowedPrivateNetworks

* style

* Proxyの間にあると誤解しそうなのでconfigの記述順を変更

* Fix error handler
2021-09-03 21:13:06 +09:00
MeiMei 85a0f696bc
ActivityPubでリモートのオブジェクトをGETするときのリクエストをHTTP Signatureで署名するオプション (#6731)
* Sign ActivityPub GET

* Fix v12, v12.48.0 UI bug
2020-10-18 01:46:40 +09:00
MeiMei 36b9a0d42f
プロキシの除外ホスト (#6244)
* プロキシの除外ホスト

* オブジェクトストレージとの通信にProxyを使うかを選択できるように

* fix lint

* コメント

Co-authored-by: rinsuki <428rinsuki+git@gmail.com>
2020-04-12 20:32:34 +09:00
MeiMei 5bbd4ae703
ElasticSearchで認証ができるように (#6158) 2020-03-20 14:00:34 +09:00
syuilo f6154dc0af
v12 (#5712)
Co-authored-by: MeiMei <30769358+mei23@users.noreply.github.com>
Co-authored-by: Satsuki Yanagi <17376330+u1-liquid@users.noreply.github.com>
2020-01-30 04:37:25 +09:00
MeiMei 84178ba38a APの流量制限とリトライ期間の変更 (#5734)
* AP rate limit

* AP Job attempts

* fix
2020-01-20 14:14:09 +09:00
MeiMei b0bb5d8dfc
期限切れ/未保存リモートファイルのローカルプロキシ (#5655)
* Media Proxy を実装

* サンプルを追加

* https://github.com/syuilo/misskey/pull/5649#discussion_r359967471 の修正

* https://github.com/syuilo/misskey/pull/5649#discussion_r359967966 の修正

* https://github.com/syuilo/misskey/pull/5649#discussion_r359968219 の修正

* 期限切れ/未保存リモートファイルのローカルプロキシ

* 設定

* 説明

* comment out

* fix

Co-authored-by: 和風ドレッシング <37681609+CookieRamen@users.noreply.github.com>
2019-12-31 17:23:47 +09:00
和風ドレッシング 9bc07c1a1c Media Proxy を実装 (#5649)
* Media Proxy を実装

* サンプルを追加
2019-12-20 01:54:28 +09:00
tamaina c8ab0e9d62 Update example.yml (#5572)
Redisの設定にprefix, dbを追加
2019-11-04 20:48:07 +09:00
rinsuki 795b56f8ab
.config/example.ymlの不要なセミコロンを削除 (#5537) 2019-10-22 19:30:18 +09:00
MeiMei 593c2b9517 Proxy for SMTP (#5371) 2019-09-02 04:42:52 +09:00
syuilo 50abb51ece Syslog support
Resolve #5355
2019-08-30 08:29:46 +09:00
MeiMei 6a0affcec1 Tune worker/job counts (#5346)
* デフォルトのワーカー数を1に

* Tune default job count
2019-08-27 05:33:24 +09:00
syuilo 2c955858a0 Add note 2019-07-03 20:31:52 +09:00
Satsuki Yanagi 1432a7193d 設定でPostgreSQLのクエリー結果のキャッシュを無効できるように (#5046) 2019-06-11 20:49:08 +09:00