FetchWeb

FetchWeb is a utility to download files from the Web. When it downloads a html file, it downloads also images, stylesheets, and scripts that are used in the html. If you know GNU Wget, its work may be understood as "wget -p -k" .

Invoking

When you select FetchWeb in "Share page" menu of a browser, an URL of the current page is sent to FetchWeb. It starts downloading the html, then downloads images, stylesheets, and scripts that are necessary for the html.

You can invoke FetchWeb also when opening an URL by clicking a link in a mailer, a browser, etc.

When you invoke FetchWeb from a launcher, FetchWeb displays the preferences screen. In this screen, you can set preferences, and also start download by inputting URL directly. Note that you can download not only htmls but also zips, rars etc. so long as its URL starts with "http".

Moreover, by the "Read URLs from a file" button, you can choose a text file and invoke FetchWeb with URLs in it. This is useful when you want to download multiple pages. A text file should consist of a series of URLs, one per line. Lines begin with "#" are ignored as comments.

Conversion of links

After the download is complete, FetchWeb converts some tags in the html to make them suitable for local viewing. Because in some htmls, images and a css are specified by an absolute path on the host.

If you want to download htmls "as is" and don't need conversion, you can stop this function by setting a preference.

Download options

Time-stamping

If "Time-stamping" option is on, FetchWeb skips download a file that already exists in the local storage and newer than the remote one. Note that html and css files will always be downloaded even if the option is on. Because FetchWeb might convert links in the downloaded html and css files, so it have to read the remote files.

Foreign hosts

By default, FetchWeb does not span downloading across hosts. If you want to force download files on a foreign hosts from the target URL, turn on "Span hosts" in the preference.

User-Agent

FetchWeb send a same string as the standard browser as the User-Agent header on a http request. If you want to use a custom string for the User-Agent, for example you need a page for normal PCs instead of a page for mobile phones, you can set it in the preference.

Download

• FetchWeb
• FetchWeb (source)

Changelog

1.5

• Added: a preference for spanning across hosts
• Added: a custom User-Agent string
• Fixed: after 1.4, it doesn't download images etc. when link conversion is off
• Fixed: failure of download might cause infinite loop
• Fixed: failure of link detection in some cases

1.4.1

• Fixed: failed to download a page using single quotes for html attribute values
• Fixed: failed to download a page that has newlines within its tags

1.4

• Performance improvement
• When downloads from multiple URLs, now FetchWeb do not stop until all downloads are finished
• Added: a screen for error messages after processing
• Allow comments in a URL list file
• Added: time-stamping to skip download non-updated files

1.3

• Added: reading URLs from a text file
• Added: invoking FetchWeb by clicking a link
• Fixed: failed to download a page using "data" scheme

1.2

• Scripts that are needed for the page are also downloaded
• <base> tag is also converted for local viewing
• Displays error if the page to download is not found

1.1.3

• Minor fixes

1.1.2

• Fixed: another case of download failure
• Fixed: downloading didn't finish in some pages

1.1.1

• Fixed: failed to download files in some pages
• Fixed: crash on 404 error

1.1

• The first release