https://nzbget.net/
>Efficient Usenet downloader.
>Download NZBGet
>Windows, Mac, Linux, BSD, Android
>https://nzbget.net/download
(nzbget-21.1-bin-windows-setup.exe / 5.41 MB), installed . . .
(C:\Program Files\NZBGet / 8.12 MB / 50 Files, 3 Folders)
>NZBGet is open source. It's hosted, developed, and maintained on GitHub.
>View the GitHub project https://github.com/nzbget/nzbget
>Documentation https://nzbget.net/documentation
>Introduction
>Feature highlights . . .
>https://nzbget.net/feature-highlights
>NZBGet : Feature Highlights
>Contents
>*High performance
>*Fast deobfuscation
>*Quick par-verification
>*Multicore par-repair
>*Retry failed articles
>*Full-featured API
>*RSS with duplicate check
>High performance
>Sometimes new users wonder why care about performance in a downloader? They say:
> The speed is only limited by the connection, the program only has to write the
> received data into disk.
>Although that might be true when downloading from web- or ftp-servers the binary
>downloads from Usenet work quite differently. The Usenet was designed to transfer
>text messages only. In order to transfer binary files (such as photos or videos)
>a special technique was invented to embed the binary data into text messages.
>This technique involves:
> *splitting of binary files into small pieces - because the message length is
> limited. One message is usually less than 1 MB;
> *converting binary data into text form - because the messages can safely
> transport only basic characters used in texts;
> *adding extra recovery data to repair downloaded files in a case some messages
> get lost - because Usenet doesn't guarantee message delivery.
>From a downloader stand of point this means that the download process isn't that
>straightforward as moving file from location one to location two.
>A lot of computation work has to be done in order to restore original binary
>files from thousands of plain text messages.
>That's why efficiency is important. How does NZBGet care about this?
> *First of all NZBGet is written in C++ language. The program code is compiled
> into native CPU instructions during build process. That provides the best
> possible usage of CPU power.
> *Do care about efficiency - that's a general motto when developing NZBGet: use
> efficient algorithms, avoid unnecessary calculations whenever possible, don't
> load everything into memory.
>All this allows NZBGet to run on systems with as little RAM as 32 MB or less and
>with CPUs running on 200 MHz or less.
>There is another aspect of efficiency however. If the system is powerful enough
>and has a lot of resources - can they be used to improve download and post-
>processing speed? Sure, they can. NZBGet offers a bunch of options to configure
>the program for systems of different leagues, for example:
> *use memory cache to decrease amount of disk operations;
> *use more threads for downloading (connections) or for par-repair;
> *etc., etc. - see Performance tips for details.
>Fast deobfuscation
>Many files posted to Usenet nowadays have obfuscated file names, for example:
> cf8ae6185547f6ca0ad263439f2279fa.01
> cf8ae6185547f6ca0ad263439f2279fa.21
> ...
> cf8ae6185547f6ca0ad263439f2279fa.par2
>Once downloaded such files can not be directly unpacked (unpack will fail). The
>original names of rar-archives must be restored first. The files come with par2-
>set, which hold the recovery information including original file names. To
>restore the file names a recovery process using external tool par2cmdline is
>usually used:
>par2cmdline r cf8ae6185547f6ca0ad263439f2279fa.par2
>During recovery the tool reads all downloaded files first and then restores
>original file names. The process of reading of files can take very long, many
>minutes or even tens of minutes depending on files size and disk speed.
>NZBGet has a unique feature called "fast par-rename", which restores original
>file names within few seconds, even on very slow machines, eliminating the
>need for time consuming par-verification step required when using external tool
>par2cmdline.
>A typical log in NZBGet looks like this:
>INFO Thu Jul 16 2015 01:38:43 Checking renamed files for Debian.7.i686
>INFO Thu Jul 16 2015 01:38:43 Renaming cf8ae6185547f6ca0ad263439f2279fa.50 to Debian.7.i686.r65
>INFO Thu Jul 16 2015 01:38:44 Renaming cf8ae6185547f6ca0ad263439f2279fa.47 to Debian.7.i686.r25
>INFO Thu Jul 16 2015 01:38:44 Renaming cf8ae6185547f6ca0ad263439f2279fa.26 to Debian.7.i686.r40
>...
>INFO Thu Jul 16 2015 01:38:44 Successfully renamed 75 file(s) for Debian.7.i686
>That's a 4GB download renamed in two seconds on a slow Linux machine.
>Rar-rename
>Version 18.0 takes deobfuscation in NZBGet to even higher level by introducing
>new rar-rename feature, which can restore correct files names for multivolume
>rar-archives even if the download doesn't have any par2-files at all, or if the
>files were obfuscated before creating par2-files.
>Quick par-verification
>If one or more Usenet messages were missing during download the resulting files
>are damaged and cannot be unpacked (or used in other way). Luckily, following
>Usenet standards most downloads include recovery data in form of extra files
>with par2-extensions. These files can be used to repair the damaged files using
>par2 recovery algorithm.
>Once again the recovery process is usually done using the already mentioned
>external tool par2cmdline:
> par2cmdline r cf8ae6185547f6ca0ad263439f2279fa.par2
>The repair process consists of two steps:
> 1. verification - the tool reads all files from disk, calculates their
> checksums to determine which files are damaged and need to be repaired;
> 2. repairing - the tool performs repair.
>Both of these steps are time consuming. Typically the first step takes minutes
>or tens of minutes and the second step may take a few minutes or many hours
>depending on the damage (and CPU speed of course).
>For a large download with little damage the first step (verification) may take
>longer than the actual repair. For example on a slow device the verification
>may take an hour with repair taking 5 minutes.
>NZBGet doesn't use external tool par2cmdline. Instead the source code of that
>tool (which is also written in C++ like NZBGet) is integrated into NZBGet.
>NZBGet uses that fact to its advantage:
>Since NZBGet knows exactly which files were downloaded with errors it passes
>that knowledge to the repair module.
>As a result the time consuming verification step is skipped and the repair
>module starts the repair process directly.
>Multicore par-repair
>As mentioned above the repair is usually performed using external tool
>par2cmdline. That tool was originally written many years ago (and hasn't been
>updated since) when mutlicore CPUs did not exist yet. Therefore the tool
>doesn't support multiple cores and the very time consuming repair process can
>use only one CPU core.
>Thanks to independent developers a multicore version of the tool was developed
> - par2cmdline_tbb. That's a great tool but it has one weak point - for
>multicore functionality it uses a program library built by Intel. That library
>works only on CPUs with Intel architecture.
>Many Linux devices are equipped with ARM CPUs with multiple cores: 4 cores are
>usual nowadays and will be more in the future. Such devices cannot use
>par2cmdline_tbb and must use the old par2cmdline tool which supports only one
>core.
>NZBGet has its own implementation of multithreading (multicore) support in the
>par2-module which works on all platforms and all CPUs.
>For example, on a 4-core ARM CPU NZBGet repairs 2-3 times faster than
>par2cmdline. Not just ARM, all other architectures profit two: MIPS, PowerPC
>and, of course, x86.
>Retry failed articles
>If download fails for whatever reason the failed item is put to history from
>where it can be retried in several flavors. In addition to classic "Download
>again from scratch" NZBGet can also retry only failed pieces. This can be
>useful in several cases:
> *if the failure is caused by temporary propagation issues;
> *one or more of your optional news servers were not used during download due
> to connection issues;
> *you have activated another news server.
>In all these cases it's preferably to not discard already downloaded pieces but
>instead retry only the failed ones. The "Retry failed articles" function works
>granularly and redownloads only missing file pieces (articles).
>Full-featured API
>The whole NZBGet functionality is accessible via API. It's not just for third-
>party developers. NZBGet itself uses that API very extensively. The built-in
>web-interface relies fully on the API. That means that every feature or
>function you see in web-interface is also available for third-party developers.
>Other programs often have a built-in special web-server for web-interface, which
>generates web-pages and process requests from web-interface. In addition they
>have API-server. The functions of web-interface and API-servers are often not
>the same.
>In NZBGet there is no such difference. The web-interface consists of a static
>html-page with (a lot of) javascript-code. NZBGet's built-in web-servers only
>serves the static web-application content. After the web-page is loaded into
>browser the javascript-application activates and communicates with NZBGet via
>API to receive the list of downloads, messages, etc. and to perform edit actions.
>RSS with duplicate check
>For whose of you who can't or don't want to use additional programs to automate
>downloads NZBGet offers very powerful RSS support. Example RSS filter:
> Reject: age:>10
> Require: 720p
> # this is comment
> Accept: game of thrones S03E##
> Accept: dexter S08E##
>One big issue you may have when using RSS in other programs is a large amount of
>duplicate downloads. it's because same titles are usually posted to Usenet by
>more than one posters. NZBGet has a powerful duplicate handling to avoid multiple
>downloads of the same title.
>Not only it does that but it also handles failed downloads and automatically
>falls back to other releases of the same title if necessary.
[end quote]