如何在不使用第三方库的情况下使用Node.js下载文件?
我不需要什么特别的东西。我只想从给定的URL下载文件,然后将其保存到给定的目录。
如何在不使用第三方库的情况下使用Node.js下载文件?
我不需要什么特别的东西。我只想从给定的URL下载文件,然后将其保存到给定的目录。
当前回答
Gfxmonk的答案在回调和file.close()完成之间有一个非常紧张的数据竞赛。File.close()实际上接受一个回调函数,该函数在close完成时被调用。否则,立即使用文件可能会失败(非常罕见!)。
一个完整的解决方案是:
var http = require('http');
var fs = require('fs');
var download = function(url, dest, cb) {
var file = fs.createWriteStream(dest);
var request = http.get(url, function(response) {
response.pipe(file);
file.on('finish', function() {
file.close(cb); // close() is async, call cb after close completes.
});
});
}
如果不等待finish事件,幼稚的脚本可能最终得到一个不完整的文件。如果不通过close调度cb回调,您可能会在访问文件和文件实际准备就绪之间出现竞争。
其他回答
您可以创建一个HTTP GET请求,并将其响应管道到可写文件流:
const http = require('http'); // or 'https' for https:// URLs
const fs = require('fs');
const file = fs.createWriteStream("file.jpg");
const request = http.get("http://i3.ytimg.com/vi/J---aiyznGQ/mqdefault.jpg", function(response) {
response.pipe(file);
// after download completed close filestream
file.on("finish", () => {
file.close();
console.log("Download Completed");
});
});
如果您希望支持在命令行上收集信息(比如指定目标文件或目录或URL),可以使用类似Commander的工具。
更详细的解释在https://sebhastian.com/nodejs-download-file/
var requestModule=require("request");
requestModule(filePath).pipe(fs.createWriteStream('abc.zip'));
说到处理错误,监听请求错误甚至更好。我甚至会通过检查响应代码来验证。这里认为只有200个响应代码成功,但其他代码可能很好。
const fs = require('fs');
const http = require('http');
const download = (url, dest, cb) => {
const file = fs.createWriteStream(dest);
const request = http.get(url, (response) => {
// check if response is success
if (response.statusCode !== 200) {
return cb('Response status was ' + response.statusCode);
}
response.pipe(file);
});
// close() is async, call cb after close completes
file.on('finish', () => file.close(cb));
// check for request error too
request.on('error', (err) => {
fs.unlink(dest, () => cb(err.message)); // delete the (partial) file and then return the error
});
file.on('error', (err) => { // Handle errors
fs.unlink(dest, () => cb(err.message)); // delete the (partial) file and then return the error
});
};
尽管这段代码相对简单,但我建议使用request模块,因为它处理更多http不支持的协议(你好,HTTPS!)。
可以这样做:
const fs = require('fs');
const request = require('request');
const download = (url, dest, cb) => {
const file = fs.createWriteStream(dest);
const sendReq = request.get(url);
// verify response code
sendReq.on('response', (response) => {
if (response.statusCode !== 200) {
return cb('Response status was ' + response.statusCode);
}
sendReq.pipe(file);
});
// close() is async, call cb after close completes
file.on('finish', () => file.close(cb));
// check for request errors
sendReq.on('error', (err) => {
fs.unlink(dest, () => cb(err.message)); // delete the (partial) file and then return the error
});
file.on('error', (err) => { // Handle errors
fs.unlink(dest, () => cb(err.message)); // delete the (partial) file and then return the error
});
};
编辑:
要使它与https兼容,请更改
const http = require('http');
to
const http = require('https');
编写自己的解决方案,因为现有的不符合我的要求。
包括:
HTTPS下载(http下载时切换包到http) 基于承诺的函数 处理转发路径(状态302) 浏览器头-需要在一些cdn 来自URL的文件名(以及硬编码) 错误处理
打印出来的,更安全。如果你使用的是纯JS(没有Flow,没有TS),可以随意删除类型,或者转换为.d。ts文件
index.js
import httpsDownload from httpsDownload;
httpsDownload('https://example.com/file.zip', './');
httpsDownload.[js|ts]
import https from "https";
import fs from "fs";
import path from "path";
function download(
url: string,
folder?: string,
filename?: string
): Promise<void> {
return new Promise((resolve, reject) => {
const req = https
.request(url, { headers: { "User-Agent": "javascript" } }, (response) => {
if (response.statusCode === 302 && response.headers.location != null) {
download(
buildNextUrl(url, response.headers.location),
folder,
filename
)
.then(resolve)
.catch(reject);
return;
}
const file = fs.createWriteStream(
buildDestinationPath(url, folder, filename)
);
response.pipe(file);
file.on("finish", () => {
file.close();
resolve();
});
})
.on("error", reject);
req.end();
});
}
function buildNextUrl(current: string, next: string) {
const isNextUrlAbsolute = RegExp("^(?:[a-z]+:)?//").test(next);
if (isNextUrlAbsolute) {
return next;
} else {
const currentURL = new URL(current);
const fullHost = `${currentURL.protocol}//${currentURL.hostname}${
currentURL.port ? ":" + currentURL.port : ""
}`;
return `${fullHost}${next}`;
}
}
function buildDestinationPath(url: string, folder?: string, filename?: string) {
return path.join(folder ?? "./", filename ?? generateFilenameFromPath(url));
}
function generateFilenameFromPath(url: string): string {
const urlParts = url.split("/");
return urlParts[urlParts.length - 1] ?? "";
}
export default download;
也许node.js已经改变了,但似乎其他解决方案(使用node v8.1.2)有一些问题:
You don't need to call file.close() in the finish event. Per default the fs.createWriteStream is set to autoClose: https://nodejs.org/api/fs.html#fs_fs_createwritestream_path_options file.close() should be called on error. Maybe this is not needed when the file is deleted (unlink()), but normally it is: https://nodejs.org/api/stream.html#stream_readable_pipe_destination_options Temp file is not deleted on statusCode !== 200 fs.unlink() without a callback is deprecated (outputs warning) If dest file exists; it is overridden
下面是一个修改后的解决方案(使用ES6和promises),它可以处理这些问题。
const http = require("http");
const fs = require("fs");
function download(url, dest) {
return new Promise((resolve, reject) => {
const file = fs.createWriteStream(dest, { flags: "wx" });
const request = http.get(url, response => {
if (response.statusCode === 200) {
response.pipe(file);
} else {
file.close();
fs.unlink(dest, () => {}); // Delete temp file
reject(`Server responded with ${response.statusCode}: ${response.statusMessage}`);
}
});
request.on("error", err => {
file.close();
fs.unlink(dest, () => {}); // Delete temp file
reject(err.message);
});
file.on("finish", () => {
resolve();
});
file.on("error", err => {
file.close();
if (err.code === "EEXIST") {
reject("File already exists");
} else {
fs.unlink(dest, () => {}); // Delete temp file
reject(err.message);
}
});
});
}