After three and a half years of development of is presented the first stable project release GNU Wget2 , developing a fully recycled version of the program to automate the recursive content load GNU WGET . GNU Wget2 is designed and rewritten from scratch and notable for the removal of the basic functionality of the Web client to the libwget library, which can be applied to the applications. The utility comes under the GPLV3 + license, and the library under lgplv3 +.
Instead of gradual processing of an existing code base, it was decided to remake everything from scratch and to establish a separate WGET2 branch for the embodiments of the restructuring ideas, increasing the functionality and making changes that violate compatibility. With the exception of stopping the support of the FTP protocol and the format of Warc , Wget2 in most situations can act as a transparent replacement of the classic WGET utility.
At the same time, Wget2 has some Docounted Differences in behavior, Provides About 30 additional options and stops support for several dozen options . Including the processing of such options as “–ask-password”, “–Header”, “–ExClude-Directories”, “–ftp *”, “–warc *”, “–Limit rate “,” –relative “and” –unlink “.
From key innovations can be noted:
- Takeaway of functionality in the libwget library.
- Transition to multithreaded architecture.
- The ability to parallel setting multiple connections and loads into several threads. Including, it is possible to parallelize the download of one file with a breakdown to blocks using the “–Chunk-size” option.
- HTTP / 2 protocol support.
- Using the HTTP header if-modified-since To download only changed data.
- Transition to the use of external bandwidth limiters, such as Trickle .
- Support header accept-encoding, data transmission in compressed form and compression algorithms Brotli, ZSTD, LZIP, Gzip, Deflate, Lzma and Bzip2.
- Support TLS 1.3, OCSP Protocol (Online Certificate Status Protocol) To verify reconciled certificates, HTS mechanism (HTTP Strict Transport Security) for forced redirection to HTTPS and HPKP (HTTP Public Key Pinning) to bind certificates.
- Ability to use GNUTLS, WolfSSL and OpenSSL as a backend for TLS.
- Support Quick Opening Mode TCP Connections (TCP Fastopen).
- Built-in support for the size of Metalink .
- Support for internationalized domain names (IDNA2008).