Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: node-fetch/node-fetch
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v2.6.0
Choose a base ref
...
head repository: node-fetch/node-fetch
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v2.6.1
Choose a head ref
  • 12 commits
  • 9 files changed
  • 8 contributors

Commits on Aug 9, 2019

  1. Copy the full SHA
    086be6f View commit details

Commits on Sep 7, 2019

  1. feat: Data URI support (#659)

    Adds support for Data URIs using native methods in Node 5.10.0+
    Richienb authored Sep 7, 2019
    Copy the full SHA
    eb3a572 View commit details
  2. docs: Add Discord badge

    Richienb authored Sep 7, 2019
    Copy the full SHA
    1d5778a View commit details

Commits on Sep 16, 2019

  1. Copy the full SHA
    5535c2e View commit details

Commits on Oct 2, 2019

  1. chore: Add funding link

    Richienb authored Oct 2, 2019
    2
    Copy the full SHA
    7b13662 View commit details
  2. Copy the full SHA
    47a24a0 View commit details

Commits on Oct 7, 2019

  1. Copy the full SHA
    6a5d192 View commit details
  2. Copy the full SHA
    244e6f6 View commit details

Commits on Oct 10, 2019

  1. fix: Change error message thrown with redirect mode set to error (#653)

    The original error message does not provide enough information about what went wrong. It simply states a configuration setting.
    uwu-ara authored and Richienb committed Oct 10, 2019
    Copy the full SHA
    1e99050 View commit details

Commits on Oct 21, 2019

  1. Copy the full SHA
    8c197f8 View commit details

Commits on Sep 5, 2020

  1. Honor the size option after following a redirect and revert data ur…

    …i support
    
    Co-authored-by: Richie Bendall <richiebendall@gmail.com>
    xxczaki and Richienb authored Sep 5, 2020
    Copy the full SHA
    2358a6c View commit details
  2. update version number

    Antoni Kepinski committed Sep 5, 2020
    Copy the full SHA
    b5e2e41 View commit details
Showing with 140 additions and 90 deletions.
  1. +12 −0 .github/FUNDING.yml
  2. +6 −0 CHANGELOG.md
  3. +30 −23 README.md
  4. +3 −1 browser.js
  5. +64 −64 package.json
  6. +6 −0 src/body.js
  7. +3 −2 src/index.js
  8. +6 −0 test/server.js
  9. +10 −0 test/test.js
12 changes: 12 additions & 0 deletions .github/FUNDING.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# These are supported funding model platforms

github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: node-fetch # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
custom: # Replace with a single custom sponsorship URL
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -5,6 +5,12 @@ Changelog

# 2.x release

## v2.6.1

**This is an important security release. It is strongly recommended to update as soon as possible.**

- Fix: honor the `size` option after following a redirect.

## v2.6.0

- Enhance: `options.agent`, it now accepts a function that returns custom http(s).Agent instance based on current URL, see readme for more information.
53 changes: 30 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -5,11 +5,14 @@ node-fetch
[![build status][travis-image]][travis-url]
[![coverage status][codecov-image]][codecov-url]
[![install size][install-size-image]][install-size-url]
[![Discord][discord-image]][discord-url]

A light-weight module that brings `window.fetch` to Node.js

(We are looking for [v2 maintainers and collaborators](https://github.com/bitinn/node-fetch/issues/567))

[![Backers][opencollective-image]][opencollective-url]

<!-- TOC -->

- [Motivation](#motivation)
@@ -48,17 +51,17 @@ A light-weight module that brings `window.fetch` to Node.js

## Motivation

Instead of implementing `XMLHttpRequest` in Node.js to run browser-specific [Fetch polyfill](https://github.com/github/fetch), why not go from native `http` to `fetch` API directly? Hence `node-fetch`, minimal code for a `window.fetch` compatible API on Node.js runtime.
Instead of implementing `XMLHttpRequest` in Node.js to run browser-specific [Fetch polyfill](https://github.com/github/fetch), why not go from native `http` to `fetch` API directly? Hence, `node-fetch`, minimal code for a `window.fetch` compatible API on Node.js runtime.

See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorphic-fetch) or Leonardo Quixada's [cross-fetch](https://github.com/lquixada/cross-fetch) for isomorphic usage (exports `node-fetch` for server-side, `whatwg-fetch` for client-side).

## Features

- Stay consistent with `window.fetch` API.
- Make conscious trade-off when following [WHATWG fetch spec][whatwg-fetch] and [stream spec](https://streams.spec.whatwg.org/) implementation details, document known differences.
- Use native promise, but allow substituting it with [insert your favorite promise library].
- Use native Node streams for body, on both request and response.
- Decode content encoding (gzip/deflate) properly, and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically.
- Use native promise but allow substituting it with [insert your favorite promise library].
- Use native Node streams for body on both request and response.
- Decode content encoding (gzip/deflate) properly and convert string output (such as `res.text()` and `res.json()`) to UTF-8 automatically.
- Useful extensions such as timeout, redirect limit, response size limit, [explicit errors](ERROR-HANDLING.md) for troubleshooting.

## Difference from client-side fetch
@@ -72,16 +75,16 @@ See Matt Andrews' [isomorphic-fetch](https://github.com/matthew-andrews/isomorph
Current stable release (`2.x`)

```sh
$ npm install node-fetch --save
$ npm install node-fetch
```

## Loading and configuring the module
We suggest you load the module via `require`, pending the stabalizing of es modules in node:
We suggest you load the module via `require` until the stabilization of ES modules in node:
```js
const fetch = require('node-fetch');
```

If you are using a Promise library other than native, set it through fetch.Promise:
If you are using a Promise library other than native, set it through `fetch.Promise`:
```js
const Bluebird = require('bluebird');

@@ -90,7 +93,7 @@ fetch.Promise = Bluebird;

## Common Usage

NOTE: The documentation below is up-to-date with `2.x` releases, [see `1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide](UPGRADE-GUIDE.md) for the differences.
NOTE: The documentation below is up-to-date with `2.x` releases; see the [`1.x` readme](https://github.com/bitinn/node-fetch/blob/1.x/README.md), [changelog](https://github.com/bitinn/node-fetch/blob/1.x/CHANGELOG.md) and [2.x upgrade guide](UPGRADE-GUIDE.md) for the differences.

#### Plain text or HTML
```js
@@ -146,9 +149,9 @@ fetch('https://httpbin.org/post', { method: 'POST', body: params })
```

#### Handling exceptions
NOTE: 3xx-5xx responses are *NOT* exceptions, and should be handled in `then()`, see the next section.
NOTE: 3xx-5xx responses are *NOT* exceptions and should be handled in `then()`; see the next section for more information.

Adding a catch to the fetch promise chain will catch *all* exceptions, such as errors originating from node core libraries, like network errors, and operational errors which are instances of FetchError. See the [error handling document](ERROR-HANDLING.md) for more details.
Adding a catch to the fetch promise chain will catch *all* exceptions, such as errors originating from node core libraries, network errors and operational errors, which are instances of FetchError. See the [error handling document](ERROR-HANDLING.md) for more details.

```js
fetch('https://domain.invalid/')
@@ -186,7 +189,7 @@ fetch('https://assets-cdn.github.com/images/modules/logos_page/Octocat.png')
```

#### Buffer
If you prefer to cache binary data in full, use buffer(). (NOTE: buffer() is a `node-fetch` only API)
If you prefer to cache binary data in full, use buffer(). (NOTE: `buffer()` is a `node-fetch`-only API)

```js
const fileType = require('file-type');
@@ -211,7 +214,7 @@ fetch('https://github.com/')

#### Extract Set-Cookie Header

Unlike browsers, you can access raw `Set-Cookie` headers manually using `Headers.raw()`, this is a `node-fetch` only API.
Unlike browsers, you can access raw `Set-Cookie` headers manually using `Headers.raw()`. This is a `node-fetch` only API.

```js
fetch(url).then(res => {
@@ -263,11 +266,11 @@ fetch('https://httpbin.org/post', options)

#### Request cancellation with AbortSignal

> NOTE: You may only cancel streamed requests on Node >= v8.0.0
> NOTE: You may cancel streamed requests only on Node >= v8.0.0
You may cancel requests with `AbortController`. A suggested implementation is [`abort-controller`](https://www.npmjs.com/package/abort-controller).

An example of timing out a request after 150ms could be achieved as follows:
An example of timing out a request after 150ms could be achieved as the following:

```js
import AbortController from 'abort-controller';
@@ -308,7 +311,7 @@ See [test cases](https://github.com/bitinn/node-fetch/blob/master/test/test.js)

Perform an HTTP(S) fetch.

`url` should be an absolute url, such as `https://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected promise.
`url` should be an absolute url, such as `https://example.com/`. A path-relative URL (`/file/under/root`) or protocol-relative URL (`//can-be-http-or-https.com/`) will result in a rejected `Promise`.

<a id="fetch-options"></a>
### Options
@@ -350,15 +353,15 @@ Note: when `body` is a `Stream`, `Content-Length` is not set automatically.

##### Custom Agent

The `agent` option allows you to specify networking related options that's out of the scope of Fetch. Including and not limit to:
The `agent` option allows you to specify networking related options which are out of the scope of Fetch, including and not limited to the following:

- Support self-signed certificate
- Use only IPv4 or IPv6
- Custom DNS Lookup

See [`http.Agent`](https://nodejs.org/api/http.html#http_new_agent_options) for more information.

In addition, `agent` option accepts a function that returns http(s).Agent instance given current [URL](https://nodejs.org/api/url.html), this is useful during a redirection chain across HTTP and HTTPS protocol.
In addition, the `agent` option accepts a function that returns `http`(s)`.Agent` instance given current [URL](https://nodejs.org/api/url.html), this is useful during a redirection chain across HTTP and HTTPS protocol.

```js
const httpAgent = new http.Agent({
@@ -432,7 +435,7 @@ The following properties are not implemented in node-fetch at this moment:

<small>*(spec-compliant)*</small>

- `body` A string or [Readable stream][node-readable]
- `body` A `String` or [`Readable` stream][node-readable]
- `options` A [`ResponseInit`][response-init] options dictionary

Constructs a new `Response` object. The constructor is identical to that in the [browser](https://developer.mozilla.org/en-US/docs/Web/API/Response/Response).
@@ -462,7 +465,7 @@ This class allows manipulating and iterating over a set of HTTP headers. All met

- `init` Optional argument to pre-fill the `Headers` object

Construct a new `Headers` object. `init` can be either `null`, a `Headers` object, an key-value map object, or any iterable object.
Construct a new `Headers` object. `init` can be either `null`, a `Headers` object, an key-value map object or any iterable object.

```js
// Example adapted from https://fetch.spec.whatwg.org/#example-headers-class
@@ -503,15 +506,15 @@ The following methods are not yet implemented in node-fetch at this moment:

* Node.js [`Readable` stream][node-readable]

The data encapsulated in the `Body` object. Note that while the [Fetch Standard][whatwg-fetch] requires the property to always be a WHATWG `ReadableStream`, in node-fetch it is a Node.js [`Readable` stream][node-readable].
Data are encapsulated in the `Body` object. Note that while the [Fetch Standard][whatwg-fetch] requires the property to always be a WHATWG `ReadableStream`, in node-fetch it is a Node.js [`Readable` stream][node-readable].

#### body.bodyUsed

<small>*(spec-compliant)*</small>

* `Boolean`

A boolean property for if this body has been consumed. Per spec, a consumed body cannot be used again.
A boolean property for if this body has been consumed. Per the specs, a consumed body cannot be used again.

#### body.arrayBuffer()
#### body.blob()
@@ -538,9 +541,9 @@ Consume the body and return a promise that will resolve to a Buffer.

* Returns: <code>Promise&lt;String&gt;</code>

Identical to `body.text()`, except instead of always converting to UTF-8, encoding sniffing will be performed and text converted to UTF-8, if possible.
Identical to `body.text()`, except instead of always converting to UTF-8, encoding sniffing will be performed and text converted to UTF-8 if possible.

(This API requires an optional dependency on npm package [encoding](https://www.npmjs.com/package/encoding), which you need to install manually. `webpack` users may see [a warning message](https://github.com/bitinn/node-fetch/issues/412#issuecomment-379007792) due to this optional dependency.)
(This API requires an optional dependency of the npm package [encoding](https://www.npmjs.com/package/encoding), which you need to install manually. `webpack` users may see [a warning message](https://github.com/bitinn/node-fetch/issues/412#issuecomment-379007792) due to this optional dependency.)

<a id="class-fetcherror"></a>
### Class: FetchError
@@ -574,6 +577,10 @@ MIT
[codecov-url]: https://codecov.io/gh/bitinn/node-fetch
[install-size-image]: https://flat.badgen.net/packagephobia/install/node-fetch
[install-size-url]: https://packagephobia.now.sh/result?p=node-fetch
[discord-image]: https://img.shields.io/discord/619915844268326952?color=%237289DA&label=Discord&style=flat-square
[discord-url]: https://discord.gg/Zxbndcm
[opencollective-image]: https://opencollective.com/node-fetch/backers.svg
[opencollective-url]: https://opencollective.com/node-fetch
[whatwg-fetch]: https://fetch.spec.whatwg.org/
[response-init]: https://fetch.spec.whatwg.org/#responseinit
[node-readable]: https://nodejs.org/api/stream.html#stream_readable_streams
4 changes: 3 additions & 1 deletion browser.js
Original file line number Diff line number Diff line change
@@ -16,7 +16,9 @@ var global = getGlobal();
module.exports = exports = global.fetch;

// Needed for TypeScript and Webpack.
exports.default = global.fetch.bind(global);
if (global.fetch) {
exports.default = global.fetch.bind(global);
}

exports.Headers = global.Headers;
exports.Request = global.Request;
128 changes: 64 additions & 64 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,66 +1,66 @@
{
"name": "node-fetch",
"version": "2.6.0",
"description": "A light-weight module that brings window.fetch to node.js",
"main": "lib/index",
"browser": "./browser.js",
"module": "lib/index.mjs",
"files": [
"lib/index.js",
"lib/index.mjs",
"lib/index.es.js",
"browser.js"
],
"engines": {
"node": "4.x || >=6.0.0"
},
"scripts": {
"build": "cross-env BABEL_ENV=rollup rollup -c",
"prepare": "npm run build",
"test": "cross-env BABEL_ENV=test mocha --require babel-register --throw-deprecation test/test.js",
"report": "cross-env BABEL_ENV=coverage nyc --reporter lcov --reporter text mocha -R spec test/test.js",
"coverage": "cross-env BABEL_ENV=coverage nyc --reporter json --reporter text mocha -R spec test/test.js && codecov -f coverage/coverage-final.json"
},
"repository": {
"type": "git",
"url": "https://github.com/bitinn/node-fetch.git"
},
"keywords": [
"fetch",
"http",
"promise"
],
"author": "David Frank",
"license": "MIT",
"bugs": {
"url": "https://github.com/bitinn/node-fetch/issues"
},
"homepage": "https://github.com/bitinn/node-fetch",
"devDependencies": {
"@ungap/url-search-params": "^0.1.2",
"abort-controller": "^1.1.0",
"abortcontroller-polyfill": "^1.3.0",
"babel-core": "^6.26.3",
"babel-plugin-istanbul": "^4.1.6",
"babel-preset-env": "^1.6.1",
"babel-register": "^6.16.3",
"chai": "^3.5.0",
"chai-as-promised": "^7.1.1",
"chai-iterator": "^1.1.1",
"chai-string": "~1.3.0",
"codecov": "^3.3.0",
"cross-env": "^5.2.0",
"form-data": "^2.3.3",
"is-builtin-module": "^1.0.0",
"mocha": "^5.0.0",
"nyc": "11.9.0",
"parted": "^0.1.1",
"promise": "^8.0.3",
"resumer": "0.0.0",
"rollup": "^0.63.4",
"rollup-plugin-babel": "^3.0.7",
"string-to-arraybuffer": "^1.0.2",
"whatwg-url": "^5.0.0"
},
"dependencies": {}
"name": "node-fetch",
"version": "2.6.1",
"description": "A light-weight module that brings window.fetch to node.js",
"main": "lib/index",
"browser": "./browser.js",
"module": "lib/index.mjs",
"files": [
"lib/index.js",
"lib/index.mjs",
"lib/index.es.js",
"browser.js"
],
"engines": {
"node": "4.x || >=6.0.0"
},
"scripts": {
"build": "cross-env BABEL_ENV=rollup rollup -c",
"prepare": "npm run build",
"test": "cross-env BABEL_ENV=test mocha --require babel-register --throw-deprecation test/test.js",
"report": "cross-env BABEL_ENV=coverage nyc --reporter lcov --reporter text mocha -R spec test/test.js",
"coverage": "cross-env BABEL_ENV=coverage nyc --reporter json --reporter text mocha -R spec test/test.js && codecov -f coverage/coverage-final.json"
},
"repository": {
"type": "git",
"url": "https://github.com/bitinn/node-fetch.git"
},
"keywords": [
"fetch",
"http",
"promise"
],
"author": "David Frank",
"license": "MIT",
"bugs": {
"url": "https://github.com/bitinn/node-fetch/issues"
},
"homepage": "https://github.com/bitinn/node-fetch",
"devDependencies": {
"@ungap/url-search-params": "^0.1.2",
"abort-controller": "^1.1.0",
"abortcontroller-polyfill": "^1.3.0",
"babel-core": "^6.26.3",
"babel-plugin-istanbul": "^4.1.6",
"babel-preset-env": "^1.6.1",
"babel-register": "^6.16.3",
"chai": "^3.5.0",
"chai-as-promised": "^7.1.1",
"chai-iterator": "^1.1.1",
"chai-string": "~1.3.0",
"codecov": "^3.3.0",
"cross-env": "^5.2.0",
"form-data": "^2.3.3",
"is-builtin-module": "^1.0.0",
"mocha": "^5.0.0",
"nyc": "11.9.0",
"parted": "^0.1.1",
"promise": "^8.0.3",
"resumer": "0.0.0",
"rollup": "^0.63.4",
"rollup-plugin-babel": "^3.0.7",
"string-to-arraybuffer": "^1.0.2",
"whatwg-url": "^5.0.0"
},
"dependencies": {}
}
6 changes: 6 additions & 0 deletions src/body.js
Original file line number Diff line number Diff line change
@@ -306,6 +306,12 @@ function convertBody(buffer, headers) {
// html4
if (!res && str) {
res = /<meta[\s]+?http-equiv=(['"])content-type\1[\s]+?content=(['"])(.+?)\2/i.exec(str);
if (!res) {
res = /<meta[\s]+?content=(['"])(.+?)\1[\s]+?http-equiv=(['"])content-type\3/i.exec(str);
if (res) {
res.pop(); // drop last quote
}
}

if (res) {
res = /charset=(.*)/i.exec(res.pop());
5 changes: 3 additions & 2 deletions src/index.js
Original file line number Diff line number Diff line change
@@ -114,7 +114,7 @@ export default function fetch(url, opts) {
// HTTP fetch step 5.5
switch (request.redirect) {
case 'error':
reject(new FetchError(`redirect mode is set to error: ${request.url}`, 'no-redirect'));
reject(new FetchError(`uri requested responds with a redirect, redirect mode is set to error: ${request.url}`, 'no-redirect'));
finalize();
return;
case 'manual':
@@ -153,7 +153,8 @@ export default function fetch(url, opts) {
method: request.method,
body: request.body,
signal: request.signal,
timeout: request.timeout
timeout: request.timeout,
size: request.size
};

// HTTP-redirect fetch step 9
Loading