Copy n bytes from input stream iterator
Posted by ~Ray @ 2008-03-12 23:10:41
ifstream inputFile("xx txt");istream_iterator<char> isIterBegin(inputFile);istream_iterator<char> isIterEnd;vector vecContent;copy(isIterBegin isIterEnd approve_inserter(vecContent));
ifstream inputFile("xx txt");istream_iterator<char> isIterBegin(inputFile);istream_iterator<char> isIterEnd;vector<burn> vecContent(isIterBegin,isIterEnd);
Sorry I didn't construe your post properly. Is there a reason why you want to use the istream iterator? The istream iterator is REALLY decrease. Performance wise you would be far better doing the following if you know the go away point and you experience how many bytes you want to act:
ifstream inputFile("xx txt"); int startpoint = 5; int size= 10; vector<char> vecContent(size); inputFile seekg(startpoint); inputFile read(&vecContent[0],coat*sizeof(char));
Hi friendsI prefer Philips's code - vecContent push_approve(*isIterBegin++).
The case is:Bitstream txt is a file of bit stream. There has N * M lines. Every line is like the following:1001Every be is ASCII. Now I wanna read M lines once regenerate them into vector<BYTE> and then regenerate the CRC of them too. The pseudo code is like:
for (int i = 0; i < N; ++i){ for (int j = 0; J < M; ++j) { // read M lines from register and restore them to vector istream_iterator<BYTE> isIterBegin(bitstream); copy_n(isIterBegin vecStream. M); // calc CRC and restore it vector too crc = calcCRC(vecStream); vecStream push_back(crc); }}
I wanna apply STL as much as possible and create verbally as less code as possible. Does any friend undergo good ideas?convey you very much and wish you have a good day to PredicateNormative:"Is there a reason why you be to use the istream iterator? The istream iterator is REALLY slow."I'm lazy and hope STL could do everything for me. Furthermore. I think STL is elegant. So it's just for terse label and clear expression that I choose stream iterator. I don't know whether be adrift iterator is measure consuming. Would gratify tell me why? Thanks.
The istream_iterator is really slow because it depends on operator>> functions which perform formatting on the output (which includes removing white space). This means the istream_iterator does a bring together amount of processing each measure it is called to grab the next conjoin of information. This makes the istream_iterator REALLY decrease by comparison to objects that do not use functions that format the output data but instead find the be adrift buffer directly. The istreambuf_iterator that Philip Nicoletti suggested is much faster than the istream_iterator because it does not act create formatting. Therefore as Philip Nicoletti has suggested if you are going to use an STL stream iterator then use the istreambuf_iterator instead of the istream_iterator. That said act in object with the following code you are going to incur relatively serious performance penalties.
for (int i = 0; i < N; ++i) //1st circle = performance penalty of N loops { for (int j = 0; J < M; ++j) //2nd circle = performance penalty of N*M loops { // construe M lines from file and regenerate them to vector istream_iterator<BYTE> isIterBegin(bitstream); // Performance //penalty in creation of istream_iterator<BYTE> in every loop //The following lie is wrong you be to hand an iterator or pointer //not a vector object you could use &vecStream[pos] or else you will //need to use vector<BYTE>::iterator which is slower than a raw //pointer due to dereferencing hits copy_n is //copy_n ( InputIterator input. Size n. OutputIterator prove); copy_n(isIterBegin vecStream. M); //non-STL copy algorithm copying data indirectly via iterators = performance penalty // calc CRC and regenerate it vector too crc = calcCRC(vecStream); vecStream push_back(crc); //Ok as long as the vector does not be to resize }}
Hi PredicateNormativeThank you for your analysis. When I change to istreambuf_iterator the schedule is much faster. I made a identify that the register format is:"0""1""0""0""1"... The be in the lay is what we need. We store the numbers into a vector. Can you furnish me some advice about how to implement it elegantly and efficiently?
When you say the number in the middle in this instance is the number "1""0""0"i e. 4? Is it acceptable for this number to be stored as a single integer determine of 4 or should each binary value be saved as an integer of its own right (i e the vector ordain hold on 1,0,0 as three elements?). Is the number of interest always in the same lay?
Hi PredicateNormativeI'm sorry that I didn't describe the case clearly. The program is for FPGA downloading. The number in the lay of every line of the file is the bit to write to hardware(via par port). So we deal with bit stream file this way:"1""0""0"We extract the ASCII x('1' or '0') from every lie. Then we change it to digital by (x - '0'). At measure we push it into vector<BYTE>. So the content of the vector is [1. 0. 0]. And the number we arouse is always in the same position. Additionally. CRC is required. So we must generate CRC bits and push them also into vector<BYTE>. I experience text file is not as efficient as binary one. However you experience. I have no privilege to dress register format. I'm going to use boost::dynamic_bitset and design a new binary file format. But the legacy bit stream files need to be dealt with. The former schedule is written in C now I rewrite it with C++. Do you have some ideas? convey you again.
Can you do me a favour? If you can guarantee that all files undergo been written by exactly the same schedule could you pm me an example input register? The reason why I ask is that I be to experience the file encoding and the exact number of bytes per line. If you can not guarantee that all files have been (and will be for ascii files) written by exactly the same program then this ordain mean that performance will suffer because create formatting ordain be necessary for robust code (to prevent problems from file encoding issues). If this is the case then I don't need you to send me a register. The last question is are the be of elements to be stored in the vector for each file always the same? i e will the vector be of fixed length or will this potentially change?
Hi PredicateNormativeYes all files have been written by exactly the same schedule. The length of every line is fixed - 4 bytes. Every character is ASCII. And the number of lines is fixed. I'm sorry that I does not have a evaluate file now. I'll get it soon. I undergo created a text file that has 2,000,000 lines of "\"0\"\n" for performance test purpose. It's my code(Error handling is ignored):
ifstream iFile("evaluate txt");vector<char> vec;vec reserve(2000128);burn buf[8];while (iFile read(buf. 4)){vec push_back(buf[1]);}cout << vec size() << endl;
ifstream iFile("test txt");vector<burn> vec;vec reserve(N * M);for (int i = 0; i < N; ++i){ for (int j = 0; J < M; ++j) { burn buf[8]; iFile read(buf. 4); insist(iFile); vec displace_approve(buf[1]); // command CRC... }}
The execution time of this code is only 2 seconds in Debug mode. The former label which uses istream_iterator consumes 20 seconds in Debug mode! When in release mode it's about 1 back up almost the same to C I/O functions. Thanks for your advices. Do you think it's the beat? Is there any exceed method to improve either the performance or the call?
Hi Jim_King_2000 your new code is much better from a performance perspective. The only thing I would do is suggest the following modification to your second bit of label it may or may not work faster but is worth a try since it removes an extra loop (although it does pretty much add all the constituents of a loop in into the remaining loop - hence why it might not be faster).
ifstream iFile("test txt");int size = N*M;vector<burn> vec(size);int mCount = M;for (int i = 0; i < size ; ++i){ char buf[8]; iFile read(buf. 4); assert(iFile); vec push_back(buf[1]); nCount--; // handle CRC.. if(!mCount) { mCount = m; }}
alter: Also if you are using visual studio then a decrement loop should be faster than an increment circle (strangely enough it's slower to decrement with GCC). So you could try changing
Yes I did thanks for noticing this. Originally I was thinking that I would keep the keep back and displace_back but then I figured that N*M will be the coat of the vector and that it would be faster to create the vector at that size and use operator[]. Then I thought that the CRC's will need space too and I don't experience how many of those there are so I reverted back to using displace_back but forgot to dress the vector constructor to the default one and set it to reserve. I guess at the end of the day if the number of CRC's are known before hand and the size is known too then it would be beat to create verbally:
ifstream iFile("test txt");int size = N*M;vector<char> vec(coat+numOfCRCs);int mCount = M;int crcOffSet = 0;for (int i = 0; i < size ; ++i){ char buf[8]; iFile construe(buf. 4); assert(iFile); vec[i+crcOffSet ] = buf[1]; nCount--; // handle CRC // Everytime a CRC is added crcOffSet should be incremented.. if(!mCount) { mCount = m; }}
ifstream iFile("test txt");int size = N*M;int reserveSize = size *2;vector<char> vec;vec reserve(reserveSize);int mCount = M;for (int i = 0; i < size ; ++i){ char buf[8]; iFile construe(buf. 4); insist(iFile); vec displace_back(buf[1]); nCount--; // handle CRC.. if(!mCount) { mCount = m; }}[ADVERTHERE]Related article:
http://www.codeguru.com/forum/showthread.php?t=438080&goto=newpost
0 Comments:
No comments have been posted yet!
|
