Skip to main content

Parsing very big files around >20 MB each and around 500-700 files

1 reply [Last post]
Joined: 2010-10-07

I have a need to get some 4000-20000 strings like 2345577 as list and there are 500 - 700 files which has from 15000-60000 lines have the ids like above (2345577). I need to keep the lines with the ids from the database and delete the lines which does not have the ids. What is the best way to solve this. What I did is that I made a list of ids from db. Then I am looping through each file and then get the id from the line, then I say if the id from the line is contained in the list then add that to stringbuffer otherwise continue.

Is there a better way to do this?

Reply viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Joined: 2003-11-18

Why you want to use StringBuffer? You can write lines direct to file.