Hello everyone,
I am having some difficulty with file handling, I hope someone can help.
Here is the background:
1) I have to read a .dat file, file size varies from 692kb to 109.742kb with the following format of data:
eg :
ID|flag
1|y
2|y
3|y
4|y
4.37777451|y
5.52625317|y
2) i have to extract the data from the file and put them into strings for persisting.
The problem
I can copy the file and read the file.
The trouble I have is with speed.
I have used
1) BufferedReader in = new BufferedReader(new FileReader(copiedFile));
(which gave me an avg time of 25mins to process a 692k file)
2) FileInputStream fis = new FileInputStream(fileName);
(which gave avg time of 12 mins to process)
3) File file = new File(fileName); Scanner scanner = new Scanner(file);
(which gave avg time of 118mins to process)
what is the BEST and FASTEST way to do this?????
I am totally running out of options here. :(
please help.
sample code:
public void meh() throws Exception {
String DATE_FORMAT_NOW = "yyyy/MM/dd hh:mm:ss";
SimpleDateFormat sdf = new SimpleDateFormat(DATE_FORMAT_NOW);
long starttime = System.currentTimeMillis();
Date timeNow = new Date();
String timeS = sdf.format(timeNow);
System.out.println(">> time of exe: " + timeS + " milliseconds: " + starttime);
String fileLoc2 = props.getProperty(PROP_FILENAME);
System.out.println(">>>> File Location: " + fileLoc2);
String output = ">> ";
String delim = "|";
String idno;
String flag;
String line;
int count = 0;
try {
FileInputStream fis = new FileInputStream(fileName);
int cnt3 = 0;
final int BUFSIZE = 1024;
byte buf[] = new byte[BUFSIZE];
int len;
while ((len = fis.read(buf)) != -1) {
for (int i = 0; i < len; i++) {
if (buf[i] == 'n') {
line = new String(buf);
for(StringTokenizer tk = new StringTokenizer(line,delim); tk.hasMoreTokens();){
idno = tk.nextToken();
if(tk.hasMoreTokens()){
flag = tk.nextToken();
}
System.out.println(">>id " + idno + ">>>> flg");
}
cnt3++;
}
}
}
fis.close();
System.out.println("row count " + cnt3);
long elapsedTime = System.currentTimeMillis() - starttime;
System.out.println(">>> elapsed time: " + elapsedTime);
} catch (IOException e) {
System.err.println(e);
}
}
code 2:
public class read2 {
String DATE_FORMAT_NOW = "yyyy/MM/dd hh:mm:ss";
SimpleDateFormat sdf = new SimpleDateFormat(DATE_FORMAT_NOW);
long starttime = System.currentTimeMillis();
Date timeNow = new Date();
String timeS = sdf.format(timeNow);
int count = 0;
void readFile(String fileName) {
System.out.println(">> time of exe: " + timeS + " milliseconds: " + starttime);
try {
Scanner scanner = new Scanner(new File(fileName));
scanner.useDelimiter(System.getProperty("line.separator"));
while (scanner.hasNext()) {
parseLine(scanner.next());
count++;
}
scanner.close();
System.out.println(">> rows: " + count);
long elapsedTime = System.currentTimeMillis() - starttime;
System.out.println(">>> elapsed time: " + elapsedTime);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
private static void parseLine(String line) {
Scanner lineScanner = new Scanner(line);
lineScanner.useDelimiter("|");
String idno = lineScanner.next();
String flag = lineScanner.next();
System.out.println(">> id: " + idno + " >> flag: " + flag);
}
public static void main(String[] args) {
String fileName = "Id.dat";
read2 r = new read2();
r.readFile(fileName);
}
}
