库建议：NodeJs读取csv文件

作者: 老夫的少女心吖
发布时间: 2024-11-23 03:52:36 (2月前)
转自：

13 条回复

0#
回复此人
木木 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 好像你需要使用一些基于流的解决方案，已经存在这样的库，所以在重新发明自己之前，尝试这个库，其中还包括验证支持。 <a href="https://www.npmjs.org/package/fast-csv" rel="noreferrer"> https://www.npmjs.org/package/fast-csv </A> </p> </DIV>

编辑
1#
回复此人
北极星微控 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 我当前的解决方案使用异步模块来串行执行： </p> <pre> <code> var fs = require('fs'); var parse = require('csv-parse'); var async = require('async'); var inputFile='myfile.csv'; var parser = parse({delimiter: ','}, function (err, data) { async.eachSeries(data, function (line, callback) { // do something with the line doSomething(line).then(function() { // when processing finishes invoke the callback to move to the next one callback(); }); }) }); fs.createReadStream(inputFile).pipe(parser); </code> </pre> </DIV>

编辑
2#
回复此人
与之 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <UL> <LI> 该解决方案使用 <code> csv-parser </code> 代替 <code> csv-parse </code> 用在一些上面的答案。 </LI> <LI> <code> csv-parser </code> 大约两年后来了 <code> csv-parse </code> 。 </LI> <LI> 他们两个都解决了同样的目的，但我个人发现了 <code> csv-parser </code> 更好，因为它很容易通过它处理标题。 </LI> </UL> <P> 首先安装csv-parser： </p> <pre> <code> npm install csv-parser </code> </pre> <P> 所以假设你有一个像这样的csv文件： </p> <pre> <code> NAME, AGE Lionel Messi, 31 Andres Iniesta, 34 </code> </pre> <P> 您可以执行以下所需的操作： </p> <pre> <code> const fs = require('fs'); const csv = require('csv-parser'); fs.createReadStream(inputFilePath) .pipe(csv()) .on('data', function(data){ try { console.log("Name is: "+data.NAME); console.log("Age is: "+data.AGE); //perform the operation } catch(err) { //error handler } }) .on('end',function(){ //some final operation }); </code> </pre> <P> 进一步阅读 <a href="https://www.npmjs.com/package/csv-parser" rel="noreferrer"> 参考 </A> </p> </DIV>

编辑
3#
回复此人
取之 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> npm安装csv </p> <P> 示例CSV文件你需要一个CSV文件来解析，所以要么你已经有一个，或者你可以复制下面的文本并将其粘贴到一个新文件中并调用该文件“mycsv.csv” </p> <pre> <code> ABC, 123, Fudge 532, CWE, ICECREAM 8023, POOP, DOGS 441, CHEESE, CARMEL 221, ABC, HOUSE 1 ABC, 123, Fudge 2 532, CWE, ICECREAM 3 8023, POOP, DOGS 4 441, CHEESE, CARMEL 5 221, ABC, HOUSE </code> </pre> <P> 示例代码读取和解析CSV文件 </p> <P> 创建一个新文件，并将以下代码插入其中。务必仔细阅读幕后发生的事情。 </p> <pre> <code> var csv = require('csv'); // loads the csv module referenced above. var obj = csv(); // gets the csv module to access the required functionality function MyCSV(Fone, Ftwo, Fthree) { this.FieldOne = Fone; this.FieldTwo = Ftwo; this.FieldThree = Fthree; }; // Define the MyCSV object with parameterized constructor, this will be used for storing the data read from the csv into an array of MyCSV. You will need to define each field as shown above. var MyData = []; // MyData array will contain the data from the CSV file and it will be sent to the clients request over HTTP. obj.from.path('../THEPATHINYOURPROJECT/TOTHE/csv_FILE_YOU_WANT_TO_LOAD.csv').to.array(function (data) { for (var index = 0; index < data.length; index++) { MyData.push(new MyCSV(data[index][0], data[index][1], data[index][2])); } console.log(MyData); }); //Reads the CSV file from the path you specify, and the data is stored in the array we specified using callback function. This function iterates through an array and each line from the CSV file will be pushed as a record to another array called MyData , and logs the data into the console to ensure it worked. var http = require('http'); //Load the http module. var server = http.createServer(function (req, resp) { resp.writeHead(200, { 'content-type': 'application/json' }); resp.end(JSON.stringify(MyData)); }); // Create a webserver with a request listener callback. This will write the response header with the content type as json, and end the response by sending the MyData array in JSON format. server.listen(8080); // Tells the webserver to listen on port 8080(obviously this may be whatever port you want.) 1 var csv = require('csv'); 2 // loads the csv module referenced above. 3 鈥媆n4 var obj = csv(); 5 // gets the csv module to access the required functionality 6 鈥媆n7 function MyCSV(Fone, Ftwo, Fthree) { 8 this.FieldOne = Fone; 9 this.FieldTwo = Ftwo; 10 this.FieldThree = Fthree; 11 }; 12 // Define the MyCSV object with parameterized constructor, this will be used for storing the data read from the csv into an array of MyCSV. You will need to define each field as shown above. 13 鈥媆n14 var MyData = []; 15 // MyData array will contain the data from the CSV file and it will be sent to the clients request over HTTP. 16 鈥媆n17 obj.from.path('../THEPATHINYOURPROJECT/TOTHE/csv_FILE_YOU_WANT_TO_LOAD.csv').to.array(function (data) { 18 for (var index = 0; index < data.length; index++) { 19 MyData.push(new MyCSV(data[index][0], data[index][1], data[index][2])); 20 } 21 console.log(MyData); 22 }); 23 //Reads the CSV file from the path you specify, and the data is stored in the array we specified using callback function. This function iterates through an array and each line from the CSV file will be pushed as a record to another array called MyData , and logs the data into the console to ensure it worked. 24 鈥媆n25 var http = require('http'); 26 //Load the http module. 27 鈥媆n28 var server = http.createServer(function (req, resp) { 29 resp.writeHead(200, { 'content-type': 'application/json' }); 30 resp.end(JSON.stringify(MyData)); 31 }); 32 // Create a webserver with a request listener callback. This will write the response header with the content type as json, and end the response by sending the MyData array in JSON format. 33 鈥媆n34 server.listen(8080); 35 // Tells the webserver to listen on port 8080(obviously this may be whatever port you want.) Things to be aware of in your app.js code In lines 7 through 11, we define the function called 'MyCSV' and the field names. If your CSV file has multiple columns make sure you define this correctly to match your file. On line 17 we define the location of the CSV file of which we are loading. Make sure you use the correct path here. </code> </pre> <P> 启动您的应用程序并验证功能打开控制台并键入以下命令： </p> <P> 节点应用 1 节点应用您应该在控制台中看到以下输出： </p> <pre> <code> [ MYCSV { Fieldone: 'ABC', Fieldtwo: '123', Fieldthree: 'Fudge' }, MYCSV { Fieldone: '532', Fieldtwo: 'CWE', Fieldthree: 'ICECREAM' }, MYCSV { Fieldone: '8023', Fieldtwo: 'POOP', Fieldthree: 'DOGS' }, MYCSV { Fieldone: '441', Fieldtwo: 'CHEESE', Fieldthree: 'CARMEL' }, MYCSV { Fieldone: '221', Fieldtwo: 'ABC', Fieldthree: 'HOUSE' }, ] </code> </pre> <P> 1 [MYCSV {Fieldone：'ABC'，Fieldtwo：'123'，Fieldthree：'Fudge'}， 2 MYCSV {Fieldone：'532'，Fieldtwo：'CWE'，Fieldthree：'ICECREAM'}， 3 MYCSV {Fieldone：'8023'，Fieldtwo：'POOP'，Fieldthree：'DOGS'}， 4 MYCSV {Fieldone：'441'，Fieldtwo：'CHEESE'，Fieldthree：'CARMEL'}，五 MYCSV {Fieldone：'221'，Fieldtwo：'ABC'，Fieldthree：'HOUSE'}，] 现在您应该打开Web浏览器并导航到您的服务器。您应该看到它以JSON格式输出数据。 </p> <P> 结论使用node.js及其CSV模块，我们可以快速，轻松地读取和使用存储在服务器上的数据，并根据请求将其提供给客户端 </p> </DIV>

编辑
4#
回复此人
無口君 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 我用这种方式： - </p> <pre> <code> var fs = require('fs'); var parse = require('csv-parse'); var csvData=[]; fs.createReadStream(req.file.path) .pipe(parse({delimiter: ':'})) .on('data', function(csvrow) { console.log(csvrow); //do something with csvrow csvData.push(csvrow); }) .on('end',function() { //do something wiht csvData console.log(csvData); }); </code> </pre> </DIV>

编辑
5#
回复此人
昵称不能为空 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 您正在引用的node-csv项目完全足以完成转换大部分CSV数据的每一行的任务，来自以下文档： <a href="http://csv.adaltas.com/transform/" rel="noreferrer"> http://csv.adaltas.com/transform/ </A> ： </p> <pre> <code> csv() .from('82,Preisner,Zbigniew\n94,Gainsbourg,Serge') .to(console.log) .transform(function(row, index, callback){ process.nextTick(function(){ callback(null, row.reverse()); }); }); </code> </pre> <P> 根据我的经验，我可以说它也是一个相当快速的实现，我一直在处理具有接近10k记录的数据集，并且整个集合的处理时间在合理的几十毫秒级别。 </p> <P> Rearding <EM> jurka </EM> 基于流的解决方案建议：node-csv是基于流的，并遵循Node.js的流API。 </p> </DIV>

编辑
6#
回复此人
昵称为空呵呵 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 该 <a href="https://www.npmjs.com/package/fast-csv" rel="nofollow noreferrer"> 快速CSV </A> npm模块可以从csv文件逐行读取数据。 </p> <P> 这是一个例子： </p> <pre> <code> let csv= require('fast-csv'); var stream = fs.createReadStream("my.csv"); csv .fromStream(stream, {headers : true}) .on("data", function(data){ console.log('I am one line of data', data); }) .on("end", function(){ console.log("done"); }); </code> </pre> </DIV>

编辑
7#
回复此人
VIP | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 为了暂停流媒体 <a href="https://www.npmjs.org/package/fast-csv" rel="noreferrer"> 快速CSV </A> 你可以做到以下几点： </p> <pre> <code> let csvstream = csv.fromPath(filePath, { headers: true }) .on("data", function (row) { csvstream.pause(); // do some heavy work // when done resume the stream csvstream.resume(); }) .on("end", function () { console.log("We are done!") }) .on("error", function (error) { console.log(error) }); </code> </pre> </DIV>

编辑
8#
回复此人
polo | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 尝试逐行npm插件。 </p> <pre> <code> npm install line-by-line --save </code> </pre> </DIV>

编辑
9#
回复此人
Colin | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 您可以使用csv-to-json模块将csv转换为json格式，然后您可以在程序中轻松使用json文件 </p> </DIV>

编辑
10#
回复此人
青阳 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <pre> <code> fs = require('fs'); fs.readFile('FILENAME WITH PATH','utf8', function(err,content){ if(err){ console.log('error occured ' +JSON.stringify(err)); } console.log('Fileconetent are ' + JSON.stringify(content)); }) </code> </pre> </DIV>

编辑
11#
回复此人
尼拉。 | 2019-08-31 10-32

<div class =“post-text”itemprop =“text”> <P> 我需要一个异步csv阅读器，最初尝试@Pransh Tiwari的答案，但无法使用它 <code> await </code> 和 <code> util.promisify() </code> 。最终我遇到了 <a href="https://github.com/Keyang/node-csvtojson" rel="nofollow noreferrer"> 节点csvtojson </A> ，它几乎和csv-parser一样，但是有了promises。以下是csvtojson的示例用法： </p> <pre> <code> const csvToJson = require('csvtojson'); const processRecipients = async () => { const recipients = await csvToJson({ trim:true }).fromFile('./recipients.csv'); // Code executes after recipients are fully loaded. recipients.forEach((recipient) => { console.log(recipient.name, recipient.email); }); }; </code> </pre> </DIV>

编辑

登录后才能参与评论