Sanitizes filenames by removing or replacing potentially problematic characters
Makes filenames safe for cross-platform use
Prevents directory traversal attacks (e.g., "../../../")
After a recent release rustls provides better error messages for
invalid certificates. For example:
```
invalid peer certificate: certificate not valid for name "wrong.host.badssl.com"; certificate is only valid for DnsName("*.badssl.com") or DnsName("badssl.com")
```
The message for expired certificates still isn't too readable but the
error now contains timestamps so we enhance it ourselves:
```
xh: error: error sending request for url (https://expired.badssl.com/)
Caused by:
0: client error (Connect)
1: invalid peer certificate: certificate expired: verification time 1742381579 (UNIX), but certificate is not valid after 1428883199 (313498380 seconds ago)
Certificate not valid after 2015-04-12 23:59:59.0 +00:00:00 (9years 11months 6days 8h 43m 24s ago).
```
Fixes a panic for `xh head https://httpbin.dev/zstd`.
`ZstdDecoder::new()` returns a `Result`. We used to panic on this, but
it needs to be a `Read` error instead, so we can suppress the error
for an empty input the way we do for other decoders.
Our existing approach couldn't handle this, so I ended up refactoring
the system. I think it's cleaner now, though still weird.
We now also preserve the original decoder error instead of
`.to_string()`ing it, or strip it completely if there was an I/O
error. That should improve the error reporting.
If RUST_BACKTRACE=1 was set outside the test runner,
e.g. `RUST_BACKTRACE=1 cargo test`, this propagated to the test binary
and changed error outputs.
Interestingly this only affected `nested_json_type_error`.
Resolves#406.
Note that this commit does not alter the storage format of sessions.
Perhaps requiring a cookie path at all times would be a good idea, but
I haven't done that here, because I don't want to break existing
sessions for users.
Fixes: #400
Enable `rustls`'s `logging` feature to start emitting logs.
Enable the `tracing` crate's `log` feature to hook up the dependencies
that log via that crate.
`hyper` can use `tracing` but it's currently unstable and locked
behind `RUSTFLAGS='--cfg hyper_unstable_tracing'` so we shouldn't use
it yet.
This partially addresses #389.
```console
$ RUST_LOG=trace/ALPN xh https://example.org
[0.495665s DEBUG rustls::client::hs] ALPN protocol is Some(b"h2")
[0.499526s TRACE hyper_util::client::legacy::client] ALPN negotiated h2, updating pool
HTTP/2.0 200 OK
[...]
$ RUST_LOG=rustls xh https://example.org
[0.288085s DEBUG rustls::client::hs] No cached session for DnsName("example.org")
[0.288657s DEBUG rustls::client::hs] Not resuming any session
[0.288767s TRACE rustls::client::hs] Sending ClientHello Message {
version: TLSv1_0,
payload: Handshake {
[...]
[0.698465s DEBUG rustls::client::hs] Using ciphersuite TLS13_AES_256_GCM_SHA384
[0.698508s DEBUG rustls::client::tls13] Not resuming
[0.698530s TRACE rustls::client::client_conn] EarlyData rejected
[0.699267s DEBUG rustls::client::tls13] TLS1.3 encrypted extensions: [Protocols([ProtocolName(6832)])]
[0.699342s DEBUG rustls::client::hs] ALPN protocol is Some(b"h2")
[0.699578s TRACE rustls::client::tls13] Server cert is
CertificateChain([CertificateDer(0x3082076e3082[...]
```
`native-tls` barely has any logging so we don't get much useful info
from there yet.
If the `Content-Disposition` header includes directory
separators (e.g. `/`) then we now only take the base
filename. Including the directories is a vulnerability.
Originally fixed in 028cbb0165af54123a4829162a6a00f46e8dce74 but then
broken again in 330d3f2ed4e1af82ef89fefce2e6e84a8ac66330. This time I
added a regression test.
External changes:
- We now print the actual reason phrase sent by the server instead
of guessing it from the status code. That is, if servers reply with
"200 Wonderful" instead of "200 OK" then we show that. This is
especially useful for status codes that xh doesn't recognize.
- Header values are now decoded as latin1, with the UTF-8 decoding
also shown if applicable.
- A new FAQ file with an entry that explains header value encoding.
Header output now hyperlinks to this entry when relevant and if
supported by the terminal.
Under the hood we now color headers manually. It's still hooked up to
the `.tmTheme` files but not to the `.sublime-syntax` file. This lets
us highlight the latin1 header values differently. In the future we
could use the same approach to optimize JSON highlighting.
I'm unsure about the position of the hyperlink. Currently it's the
text "UTF-8" in `<latin1 value> (UTF-8: <utf-8 value>)`. But that
means it's only shown if the value can be decoded as UTF-8. An
alternative is to turn the latin1 value itself into a hyperlink, but
that's confusing if the value itself is already a URL (which is a
common case for the `Location` header).
I also don't feel that our text is quite distinct enough from the
header value in the default `ansi` theme. Though the hyperlink does
help to set it apart.
Add `env_logger` to be able to print the logs that our libraries
already generate.
Add some logging to the application code. We'll probably want more log
messages, and we might want to upgrade some of them from trace to
debug.
Add a `--debug` flag that automatically enables `env_logger` as well
as backtraces for `anyhow` and panics.
```console
$ xh --debug :
[2024-06-05T13:54:40Z DEBUG xh] xh 0.22.0 -native-tls +rustls
[2024-06-05T13:54:40Z DEBUG xh] Cli {
httpie_compat_mode: false,
[...]
}
[2024-06-05T13:54:40Z DEBUG xh] Complete URL: http://localhost/
[2024-06-05T13:54:40Z DEBUG xh] HTTP method: GET
[2024-06-05T13:54:40Z DEBUG reqwest::connect] starting new connection: http://localhost/
[2024-06-05T13:54:40Z DEBUG hyper_util::client::legacy::connect::dns] resolving host="localhost"
[...]
xh: error: error sending request for url (http://localhost/)
Caused by:
0: client error (Connect)
1: tcp connect error: Connection refused (os error 111)
2: Connection refused (os error 111)
Stack backtrace:
0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.82/src/error.rs:565:25
[...]
```
We could ask users to post `--debug` output when reporting bugs.
