コピペコードで快適生活

明日使えるソースを自分のために

wgetがリダイレクトしまくってエラーする

Linux

wgetがリダイレクトしまくってエラーする現象に出くわしたのでメモ。

自分の環境

golang:1.7

どんなエラーでてる？

root@7453a9101b5e:/var/app# wget --max-redirect=1 https://github.com/wp-cli/wp-cli/releases/download/v2.5.0/wp-cli-2.5.0.phar
converted 'https://github.com/wp-cli/wp-cli/releases/download/v2.5.0/wp-cli-2.5.0.phar' (ANSI_X3.4-1968) -> 'https://github.com/wp-cli/wp-cli/releases/download/v2.5.0/wp-cli-2.5.0.phar' (UTF-8)
--2021-12-02 07:31:42--  https://github.com/wp-cli/wp-cli/releases/download/v2.5.0/wp-cli-2.5.0.phar
Resolving github.com (github.com)... 52.69.186.44
Connecting to github.com (github.com)|52.69.186.44|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/2360755/ae27af00-b8c0-11eb-8a65-ec58a9e54400?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211202%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211202T073059Z&X-Amz-Expires=300&X-Amz-Signature=86171b4129a9e57e04e99ccfa77650440c906a3adc2831ed086b4495920c95df&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=2360755&response-content-disposition=attachment%3B%20filename%3Dwp-cli-2.5.0.phar&response-content-type=application%2Foctet-stream [following]
converted 'https://objects.githubusercontent.com/github-production-release-asset-2e65be/2360755/ae27af00-b8c0-11eb-8a65-ec58a9e54400?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211202%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211202T073059Z&X-Amz-Expires=300&X-Amz-Signature=86171b4129a9e57e04e99ccfa77650440c906a3adc2831ed086b4495920c95df&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=2360755&response-content-disposition=attachment%3B%20filename%3Dwp-cli-2.5.0.phar&response-content-type=application%2Foctet-stream' (ANSI_X3.4-1968) -> 'https://objects.githubusercontent.com/github-production-release-asset-2e65be/2360755/ae27af00-b8c0-11eb-8a65-ec58a9e54400?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A/20211202/us-east-1/s3/aws4_request&X-Amz-Date=20211202T073059Z&X-Amz-Expires=300&X-Amz-Signature=86171b4129a9e57e04e99ccfa77650440c906a3adc2831ed086b4495920c95df&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=2360755&response-content-disposition=attachment; filename=wp-cli-2.5.0.phar&response-content-type=application/octet-stream' (UTF-8)
--2021-12-02 07:31:42--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/2360755/ae27af00-b8c0-11eb-8a65-ec58a9e54400?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A/20211202/us-east-1/s3/aws4_request&X-Amz-Date=20211202T073059Z&X-Amz-Expires=300&X-Amz-Signature=86171b4129a9e57e04e99ccfa77650440c906a3adc2831ed086b4495920c95df&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=2360755&response-content-disposition=attachment;%20filename=wp-cli-2.5.0.phar&response-content-type=application/octet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.108.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized

Username/Password Authentication Failed.
--2021-12-02 07:31:42--  https://github.com/wp-cli/wp-cli/releases/download/v2.5.0/wp-cli-2.5.0.phar
Connecting to github.com (github.com)|52.69.186.44|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/2360755/ae27af00-b8c0-11eb-8a65-ec58a9e54400?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211202%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211202T073059Z&X-Amz-Expires=300&X-Amz-Signature=86171b4129a9e57e04e99ccfa77650440c906a3adc2831ed086b4495920c95df&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=2360755&response-content-disposition=attachment%3B%20filename%3Dwp-cli-2.5.0.phar&response-content-type=application%2Foctet-stream [following]
converted 'https://objects.githubusercontent.com/github-production-release-asset-2e65be/2360755/ae27af00-b8c0-11eb-8a65-ec58a9e54400?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211202%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211202T073059Z&X-Amz-Expires=300&X-Amz-Signature=86171b4129a9e57e04e99ccfa77650440c906a3adc2831ed086b4495920c95df&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=2360755&response-content-disposition=attachment%3B%20filename%3Dwp-cli-2.5.0.phar&response-content-type=application%2Foctet-stream' (ANSI_X3.4-1968) -> 'https://objects.githubusercontent.com/github-production-release-asset-2e65be/2360755/ae27af00-b8c0-11eb-8a65-ec58a9e54400?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A/20211202/us-east-1/s3/aws4_request&X-Amz-Date=20211202T073059Z&X-Amz-Expires=300&X-Amz-Signature=86171b4129a9e57e04e99ccfa77650440c906a3adc2831ed086b4495920c95df&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=2360755&response-content-disposition=attachment; filename=wp-cli-2.5.0.phar&response-content-type=application/octet-stream' (UTF-8)
1 redirections exceeded.

似たような現象に出くわしている人がいた

https://github.com/hashicorp/packer/issues/2392

For me it was local encoding issue try wget --local-encoding=UTF-8

だって。

実際やってみた

wget --local-encoding=UTF-8 https://github.com/wp-cli/wp-cli/releases/download/v2.5.0/wp-cli-2.5.0.phar

動いた！

どういうこと？

  • wgetはIRI (ユニコード対応版URI) をサポートしている。
  • IRIのリモートのエンコーディングが UTF-8 、ローカルのエンコーディングが ANSI_X3.4-1968 となっていた。
  • サーバから受け取ったリダイレクト先のアドレスを ANSI_X3.4-1968 から、UTF-8 に変換しようとしていた。
  • この変換で正しくないアドレスが生成され、アクセスしてもエラーする状態となっていた。

ということかな。間違ってるかもしれないけど。

wgetのオプションについて

--no-iri IRI サポートを使わない
--local-encoding=ENC 指定した ENC を IRI のローカルエンコーディングにする
--remote-encoding=ENC 指定した ENC をデフォルトのリモートエンコーディングにする