如何在不使用第三方库的情况下使用Node.js下载文件?
我不需要什么特别的东西。我只想从给定的URL下载文件,然后将其保存到给定的目录。
如何在不使用第三方库的情况下使用Node.js下载文件?
我不需要什么特别的东西。我只想从给定的URL下载文件,然后将其保存到给定的目录。
当前回答
你可以使用https://github.com/douzi8/ajax-request#download
request.download('http://res.m.ctrip.com/html5/Content/images/57.png',
function(err, res, body) {}
);
其他回答
不要忘记处理错误!下面的代码是基于Augusto Roman的回答。
var http = require('http');
var fs = require('fs');
var download = function(url, dest, cb) {
var file = fs.createWriteStream(dest);
var request = http.get(url, function(response) {
response.pipe(file);
file.on('finish', function() {
file.close(cb); // close() is async, call cb after close completes.
});
}).on('error', function(err) { // Handle errors
fs.unlink(dest); // Delete the file async. (But we don't check the result)
if (cb) cb(err.message);
});
};
现代版本(ES6, Promise, Node 12。X +)适用于https/http。它还支持重定向302和301。我决定不使用第三方库,因为它可以很容易地使用标准Node.js库。
// download.js
import fs from 'fs'
import https from 'https'
import http from 'http'
import { basename } from 'path'
import { URL } from 'url'
const TIMEOUT = 10000
function download (url, dest) {
const uri = new URL(url)
if (!dest) {
dest = basename(uri.pathname)
}
const pkg = url.toLowerCase().startsWith('https:') ? https : http
return new Promise((resolve, reject) => {
const request = pkg.get(uri.href).on('response', (res) => {
if (res.statusCode === 200) {
const file = fs.createWriteStream(dest, { flags: 'wx' })
res
.on('end', () => {
file.end()
// console.log(`${uri.pathname} downloaded to: ${path}`)
resolve()
})
.on('error', (err) => {
file.destroy()
fs.unlink(dest, () => reject(err))
}).pipe(file)
} else if (res.statusCode === 302 || res.statusCode === 301) {
// Recursively follow redirects, only a 200 will resolve.
download(res.headers.location, dest).then(() => resolve())
} else {
reject(new Error(`Download request failed, response status: ${res.statusCode} ${res.statusMessage}`))
}
})
request.setTimeout(TIMEOUT, function () {
request.abort()
reject(new Error(`Request timeout after ${TIMEOUT / 1000.0}s`))
})
})
}
export default download
感谢安德烈·特卡琴科,我修改了他的要点
将其包含在另一个文件中并使用
const download = require('./download.js')
const url = 'https://raw.githubusercontent.com/replace-this-with-your-remote-file'
console.log('Downloading ' + url)
async function run() {
console.log('Downloading file')
try {
await download(url, 'server')
console.log('Download done')
} catch (e) {
console.log('Download failed')
console.log(e.message)
}
}
run()
根据上面的其他答案和一些微妙的问题,下面是我的尝试。
Check the file does not exist before hitting the network by using fs.access. Only create the fs.createWriteStream if you get a 200 OK status code. This reduces the amount of fs.unlink commands required to tidy up temporary file handles. Even on a 200 OK we can still possibly reject due to an EEXIST file already exists (imagine another process created the file whilst we were doing network calls). Recursively call download if you get a 301 Moved Permanently or 302 Found (Moved Temporarily) redirect following the link location provided in the header. The issue with some of the other answers recursively calling download was that they called resolve(download) instead of download(...).then(() => resolve()) so the Promise would return before the download actually finished. This way the nested chain of promises resolve in the correct order. It might seem cool to clean up the temp file asynchronously, but I chose to reject only after that completed too so I know that everything start to finish is done when this promise resolves or rejects.
const https = require('https');
const fs = require('fs');
/**
* Download a resource from `url` to `dest`.
* @param {string} url - Valid URL to attempt download of resource
* @param {string} dest - Valid path to save the file.
* @returns {Promise<void>} - Returns asynchronously when successfully completed download
*/
function download(url, dest) {
return new Promise((resolve, reject) => {
// Check file does not exist yet before hitting network
fs.access(dest, fs.constants.F_OK, (err) => {
if (err === null) reject('File already exists');
const request = https.get(url, response => {
if (response.statusCode === 200) {
const file = fs.createWriteStream(dest, { flags: 'wx' });
file.on('finish', () => resolve());
file.on('error', err => {
file.close();
if (err.code === 'EEXIST') reject('File already exists');
else fs.unlink(dest, () => reject(err.message)); // Delete temp file
});
response.pipe(file);
} else if (response.statusCode === 302 || response.statusCode === 301) {
//Recursively follow redirects, only a 200 will resolve.
download(response.headers.location, dest).then(() => resolve());
} else {
reject(`Server responded with ${response.statusCode}: ${response.statusMessage}`);
}
});
request.on('error', err => {
reject(err.message);
});
});
});
}
编写自己的解决方案,因为现有的不符合我的要求。
包括:
HTTPS下载(http下载时切换包到http) 基于承诺的函数 处理转发路径(状态302) 浏览器头-需要在一些cdn 来自URL的文件名(以及硬编码) 错误处理
打印出来的,更安全。如果你使用的是纯JS(没有Flow,没有TS),可以随意删除类型,或者转换为.d。ts文件
index.js
import httpsDownload from httpsDownload;
httpsDownload('https://example.com/file.zip', './');
httpsDownload.[js|ts]
import https from "https";
import fs from "fs";
import path from "path";
function download(
url: string,
folder?: string,
filename?: string
): Promise<void> {
return new Promise((resolve, reject) => {
const req = https
.request(url, { headers: { "User-Agent": "javascript" } }, (response) => {
if (response.statusCode === 302 && response.headers.location != null) {
download(
buildNextUrl(url, response.headers.location),
folder,
filename
)
.then(resolve)
.catch(reject);
return;
}
const file = fs.createWriteStream(
buildDestinationPath(url, folder, filename)
);
response.pipe(file);
file.on("finish", () => {
file.close();
resolve();
});
})
.on("error", reject);
req.end();
});
}
function buildNextUrl(current: string, next: string) {
const isNextUrlAbsolute = RegExp("^(?:[a-z]+:)?//").test(next);
if (isNextUrlAbsolute) {
return next;
} else {
const currentURL = new URL(current);
const fullHost = `${currentURL.protocol}//${currentURL.hostname}${
currentURL.port ? ":" + currentURL.port : ""
}`;
return `${fullHost}${next}`;
}
}
function buildDestinationPath(url: string, folder?: string, filename?: string) {
return path.join(folder ?? "./", filename ?? generateFilenameFromPath(url));
}
function generateFilenameFromPath(url: string): string {
const urlParts = url.split("/");
return urlParts[urlParts.length - 1] ?? "";
}
export default download;
使用http2模块
我看到了使用http、https和request模块的答案。我想添加一个使用另一个本地NodeJS模块,支持http或https协议:
解决方案
我已经参考了官方的NodeJS API,以及关于这个问题的一些其他答案。下面是我编写的测试,它可以按照预期工作:
import * as fs from 'fs';
import * as _path from 'path';
import * as http2 from 'http2';
/* ... */
async function download( host, query, destination )
{
return new Promise
(
( resolve, reject ) =>
{
// Connect to client:
const client = http2.connect( host );
client.on( 'error', error => reject( error ) );
// Prepare a write stream:
const fullPath = _path.join( fs.realPathSync( '.' ), destination );
const file = fs.createWriteStream( fullPath, { flags: "wx" } );
file.on( 'error', error => reject( error ) );
// Create a request:
const request = client.request( { [':path']: query } );
// On initial response handle non-success (!== 200) status error:
request.on
(
'response',
( headers/*, flags*/ ) =>
{
if( headers[':status'] !== 200 )
{
file.close();
fs.unlink( fullPath, () => {} );
reject( new Error( `Server responded with ${headers[':status']}` ) );
}
}
);
// Set encoding for the payload:
request.setEncoding( 'utf8' );
// Write the payload to file:
request.on( 'data', chunk => file.write( chunk ) );
// Handle ending the request
request.on
(
'end',
() =>
{
file.close();
client.close();
resolve( { result: true } );
}
);
/*
You can use request.setTimeout( 12000, () => {} ) for aborting
after period of inactivity
*/
// Fire off [flush] the request:
request.end();
}
);
}
然后,例如:
/* ... */
let downloaded = await download( 'https://gitlab.com', '/api/v4/...', 'tmp/tmpFile' );
if( downloaded.result )
{
// Success!
}
// ...
外部引用
https://nodejs.org/api/http2.html#http2_client_side_example https://nodejs.org/api/http2.html#http2_clienthttp2session_request_headers_options
编辑信息
解决方案是为typescript编写的,函数是一个类方法——但是没有注意到这一点,如果没有正确使用函数声明,这个解决方案将无法为假定的javascript用户工作,这是我们的贡献者迅速添加的。谢谢!