[運算] 讀取檔案最佳化

[greg2977 (澔呆)]

我的檔案大概有30萬行，貼前面幾行


"TOA5","CR800Series","CR800","3780","CR800.Std.05","CPU:1011109.CR8","59367","Table1"

"TIMESTAMP","RECORD","WS_ms_S_WVT","WindDir_D1_WVT","WS_ms_new_S_WVT","WindDir_new_D1_WVT"
"TS","RN","","Deg","","Deg"
"","","WVc","WVc","WVc","WVc"
"2012-11-09 11:52:30",0,5.28,342,4.541,12.87
"2012-11-09 11:52:36",1,4.475,343.4,4.394,17.41
"2012-11-09 11:52:42",2,3.482,345.5,3.283,20.87
"2012-11-09 11:52:48",3,3.016,355.2,3.43,4.637
"2012-11-09 11:52:54",4,3.746,347.9,3.348,43.72
"2012-11-09 11:53:00",5,5.461,347.2,5.668,14.22
...

我想要寫個程式將去四行檔頭後~把資料讀入並分門別類

我目前的寫法是:

==============================================================================
filename = '1011203.dat';  %%檔名

fid=fopen(filename);
fgetl(fid);fgetl(fid);fgetl(fid);fgetl(fid); %跳過四行檔頭
i=0;

tline = fgetl(fid);
while ischar(tline)
    i=i+1;              %%%計數器
    tline = fgetl(fid);
end

data=zeros(i,12);       %設定檔案大小

fid=fopen(filename);
fgetl(fid);fgetl(fid);fgetl(fid);fgetl(fid); %跳過四行檔頭
i=0;

tline = fgetl(fid);
while ischar(tline)

    tlength=length(tline);
    whereComma=strfind(tline,','); %找逗號

    data(i,1)=str2double(tline(2:5));
    data(i,2)=str2double(tline(7:8));
    data(i,3)=str2double(tline(10:11));
    data(i,4)=str2double(tline(13:14));
    data(i,5)=str2double(tline(16:17));
    data(i,6)=str2double(tline(19:20));
    data(i,7)=str2double(tline(whereComma(1)+1:whereComma(2)-1));
    data(i,8)=str2double(tline(whereComma(2)+1:whereComma(3)-1));
    data(i,9)=str2double(tline(whereComma(3)+1:whereComma(4)-1));
    data(i,10)=str2double(tline(whereComma(4)+1:whereComma(5)-1));
    data(i,11)=str2double(tline(whereComma(5)+1:tlength));
    data(i,11)=str2double(tline(whereComma(5)+1:whereComma(6)-1));
    data(i,12)=str2double(tline(whereComma(6)+1:tlength));

    tline = fgetl(fid);
end
==============================================================================

可是讀檔速度有點慢，問題可能在於讀法太蠢...

不知道版上的高手們能不能點撥一下


--
※ 發信站: 批踢踢實業坊(ptt.cc) 
◆ From: 180.177.112.62
→ mp19990920:若您只是要刪除前4行後的檔案,可用任何的編輯軟體完成
→ mp19990920:若您要把刪除前4行的資料弄進matlab的話,可以用
→ mp19990920:File->"Import Data..."->選擇你的檔案, 之後用matlab
→ mp19990920:的指令把前4行刪掉
→ greg2977:問題是我會需要一次讀入好多筆來做比對與運算
→ greg2977:況且這個動作一段時間就要執行一次
→ greg2977:感謝一樓的推文~讓我知道有如此神速的方法
→ greg2977:程式的美妙之處就在於多久可以找出答案
→ greg2977:我覺得我找出來了= =+
→ mp19990920:不客氣....我看成你出來了 = =|||
→ greg2977:textread(filename,'%s %*d %f %f %f %f %f %f',
→ greg2977:'headerlines',4,'delimiter',',');