EXCEL CSV FILES Handling

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

If you need a line to be read from a particular value (like a date), you could do one of many things--

-(Hard) create a regular-expression to match dates (in MM/DD/YYYY format) and separate lines once a new date has been analyzed.
-(Easy) create a method that takes a String argument and determines if it is a date by checking for the appropriate characters in a date-format String literal.

--I'd store each line in an ArrayList.

From there I'd probably tokenize each comma separated value and for the particular line then store it in a > Map.

Go to my profile, then code snippets and look at "Search Engine" to get an idea of why you'd do this.

You can easily change the separation from a space to a comma in my code snippet.

From there you can use "p" or "f" keys to locate all Modules that passed or failed.

Since the ones of interest are ones that failed, simply retrieve the ArrayList associated with the key "f", and write the information to a file or do what you want with it.

Hopefully this post was helpful.

Edit: If you want to do this an easier way it may help to invest time in a Swing Application and implement a JTable with the columns and rows of the information then have some kind of iterator to retrieve particular rows of information of interest based on a value of a column/row (basically use a double array or an ArrayList of ArrayList values to store the information relative to the JTable and use actionListeners to update the ArrayLists when an update to the JTable has been made.)

Posting Shark

972 posts since Jun 2008

Reputation Points: 392

Solved Threads: 109

If you need a line to be read from a particular value (like a date), you could do one of many things--

-(Hard) create a regular-expression to match dates (in MM/DD/YYYY format) and separate lines once a new date has been analyzed. -(Easy) create a method that takes a String argument and determines if it is a date by checking for the appropriate characters in a date-format String literal.

--I'd store each line in an ArrayList.

From there I'd probably tokenize each comma separated value and for the particular line then store it in a > Map.

Go to my profile, then code snippets and look at "Search Engine" to get an idea of why you'd do this.

You can easily change the separation from a space to a comma in my code snippet.

From there you can use "p" or "f" keys to locate all Modules that passed or failed.

Since the ones of interest are ones that failed, simply retrieve the ArrayList associated with the key "f", and write the information to a file or do what you want with it.

Hopefully this post was helpful.

Edit: If you want to do this an easier way it may help to invest time in a Swing Application and implement a JTable with the columns and rows of the information then have some kind of iterator to retrieve particular rows of information of interest based on a value of a column/row (basically use a double array or an ArrayList of ArrayList values to store the information relative to the JTable and use actionListeners to update the ArrayLists when an update to the JTable has been made.)

Thank you for the tips...
I'm planning to do this... however i'm rusty again...what i have done so far was to count the number of F =fails that each csv file had... and assign to the name of the file and colect data regarding the date of creation of the file... i'm going to have a look on your sample.. i hope that u don't mind... by the way,,,, can you give a small sample how to

" Since the ones of interest are ones that failed, simply retrieve the ArrayList associated with the key "f", and write the information to a file or do what you want with it."

its not so easy to me... i understand if you wont reply to this request... i'm learning!!

thanks by the tips

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

Ok I'd like to apologize - I didn't realize your values extended so far!

What exactly deems a unit as a failure? I see many "passes" values in each row. What is the condition for a unit to be a "fail?"

Posting Shark

972 posts since Jun 2008

Reputation Points: 392

Solved Threads: 109

You don't need to apologize ... i was not so explicit...

In fact what i need to colect is... lets say that i have a faillure... this failure is based in the cross analysis of the values provided... lets say that i have a faillure that is located in
"HeightAvgResult,HeightAvg,HeightAvgUpFail,HeightAvgLowFail"

F,238.539642,238.000000,84.000000,140.000000

what i need is only the value that is in front of the F.. in this case 238.539642 and what kind of faillure that is related to .. in this case... HeightAvgResult...

So the info will appear:
Module 1 ,IC2,TSOP8,SOP8_1,Height,F,138.539642...

i hope that you can understand what i mean.. thanks once again

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

Show Comments

There are a few ways we can do this.

In my opinion, the best way to do this is create a class that accepts these kind of values and stores them in appropriate data types (the types listed in each column in the first row (Date, time... etc), so it would be wise to use Strings).

For now we'll have 3 classes (maybe more) and have them hold all of the data we need.

When you read the lines from the file you'll store them in the classes the same way I mentioned before.

The classes toString method will be overridden to display all of the information for that particular line of data.

Now this may or may not complicate things but you can write methods that return if a particular column is a fail or not.

Now you can just use an iterator (or array of these classes) where each has different information and when a fail comes up upon invocation of the classes's method and it returns true you can display the toString method of the class that has the information you need.

This is probably the simplest way to do it. I can provide an example but it'll take time to sift through the different value per column.

Posting Shark

972 posts since Jun 2008

Reputation Points: 392

Solved Threads: 109

Thank u for the tips.. i will try to find examples in the Web in order to guid me... because i'm not so good..

the reason i'v choose only the fails was that sometimes i have 1500 files with i don't how many lines... so the data that i need is basically the faillures,,, F, so this will allow me to make my output smaller and provide the correct data regarding the failures...

So... the idea will be if the line has a faillures i will collect the data regarding that line split the line and place the needed data in "Classes" when you mean classe you are refering to a txt file ??
regarding the array... i'm not so good with controling arrays... i suck...i know... but slowly i can get there if well guided.. i want to learn....

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

I've provided a class that should help you with this project. Examine it closely--

public class InspectionResults{

	public static final byte  HEIGHT_AVG_RESULT = 0,
							  HEIGHT_RANGE_RESULT = 1,
							  AREA_AVG_RESULT = 2,
							  AREA_RANGE_RESULT = 3,
							  VOLUME_AVG_RESULT = 4,
							  VOLUME_RANGE_RESULT = 5,
							  HAV_FAILED_FEATURE_RESULT = 6,
							  REG_FAILED_FEATURE_RESULT = 7,
							  BRIDGE_FAILED_FEATURE_RESULT = 8;
	private String retrievedData[];
	private boolean failed[];

	/**
	 * Constructs this InspectionResult with the data stored in the args.
	 * This class expects 44 values within the range of the args.
	 */
	public InspectionResults(String... args){
		retrievedData = args;
		boolean temp[] ={
			((retrievedData[7].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[12].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[15].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[20].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[23].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[28].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[35].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[38].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[41].equalsIgnoreCase("F")) ? true: false)
		};
		failed = temp;
	}

	/**
	 * Returns true if the given value has failed, returns false otherwise.
	 * It's preferred to use the constants defined within this class to get the
	 * desired information, and not regular ints.
	 */
	public boolean hasFailed(byte result) throws Exception{
		if(result >= 0 && result < failed.length)
			return failed[result];
		else{
			throw new Exception("Attempt to access invalid result type! Use the Result Constants to avoid this error!");
		}
	}

	/**
	 * Returns the value next to the specified result.
	 */
	public String getAdjacentValue(byte result) throws Exception{
		if(result >= 0 && result < retrievedData.length - 1)
			return retrievedData[result + 1];
		else throw new Exception("Error! Attempt to access column with either no adjacent value or outside of data-range!");
	}

	/**
	 * Simply returns a String representing the data for each value in this class.
	 */
	@Override
	public String toString(){
		String temp = "";
		for(String element : retrievedData){
			if(element.toString() != retrievedData[retrievedData.length - 1])
				temp += element + ", ";
			else temp += element;
		}
		return temp;
	}
}

If you can manage to retrieve each line of information from the file and store it in a new object of this class you should be able to eliminate a world of trouble.

I took the time to look through which values had a P or F next to it. The Constructor of this class is expecting an Array of Strings, so you will need to use a tokenizer for each line you examine to tokenize a line of value separated by commas.

If you want to turn a line of information into an Array you can use the
String.split(",") command (where String is replaced by the actual value that contains the String-value of the line of information you retrieve), to turn your line into an array of Strings. From there you can create a new object of this class that will handle the comparisons for you (checking if a particular value is P or F) and also enables you to retrieve the Adjacent value (which actually needs to be edited - more constants are needed to get the appropriate adjacent values OR you could just enter the numbers of the column to get the adjacent values which may be annoying but I forgot to implement that).

Hopefully this helps you out.

Posting Shark

972 posts since Jun 2008

Reputation Points: 392

Solved Threads: 109

Uau.. you are fast.. i will give a look and tryout with my program... i have evrything done regarding access the folders read/access the csv files, count the number of ocurrences... store the data in the txt file and also send the data to a DB.... this is what is missing i will try and tomorrow i will give you the feedback.... thank you once again... C u

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

regarding
"so you will need to use a tokenizer for each line you examine to tokenize a line of value separated by commas."

I have that as well...

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

regarding "so you will need to use a tokenizer for each line you examine to tokenize a line of value separated by commas."

I have that as well...

Then all you need to do is store the tokenized values in an array and use that array to create an object of the class listed above (remember, it's expecting an array of Strings of length 44 (from indices 0-43)).

Posting Shark

972 posts since Jun 2008

Reputation Points: 392

Solved Threads: 109

I used this as a reference, it might make the code clearer in case it isn't already--

,SRFF File: D:\SPI Master Program List\200-34-02\200-34-02-B-01-R.SRF ,Panel Name: Panel Description ,Units: Microns ,Date, Time, PanelId, Board, Location, Part, Package, HeightAvgResult, // 7 HeightAvg, HeightAvgUpFail, HeightAvgLowFail, HeightAvgTarget, HeightRangeResult, // 12 HeightRange, HeightRangeMax, AreaAvgResult, // 15 AreaAvg, AreaAvgUpFail, AreaAvgLowFail, AreaAvgTarget, AreaRangeResult, // 20 AreaRange, AreaRangeMax, VolumeAvgResult, // 23 VolumeAvg, VolumeAvgUpFail, VolumeAvgLowFail, VolumeAvgTarget, VolumeRangeResult, // 28 VolumeRange, VolumeRangeMax, XOffset, YOffset, Rotation, Scaling, HAVFailedFeatureResult, // 35 HAVFailedFeatures, HAVFailedFeatureMax, RegFailedFeatureResult, // 38 RegFailedFeatures, RegFailedFeatureMax, BridgeFailedFeatureResult, // 41 BridgeFailedFeatures, BridgeFailedFeatureMax ,07/19/2008, 00:48:55, #280, Module 1 , IC2, TSOP8, SOP8_1, F, // 7 138.539642, 238.000000, 84.000000, 140.000000, P, // 12 9.226400, 140.000000, P, // 15 427747.781250, 729933.140625, 243311.046875, 486622.090000, P, // 20 26717.400391, 486622.093750, P, // 23 60986780.000000, 122628772.800000, 34063548.000000, 68127092.600000, P, // 28 3087930.000000, 68127096.000000, -22.886625, -23.317499, -0.039985, -0.131595, P, // 35 0, 1, P, // 38 0, 1, P, // 41 0, 1 ,07/19/2008,00:48:55,#280,Module 1 ,C38,0402,0402_1_3,P,131.077194,238.000000,84.000000,140.000000,P,12.636800,140.000000,P,265426.750000,470565.468750,156855.156250,313710.320000,P,5929.500000,313710.312500,P,36161180.000000,79054999.200000,21959722.000000,43919444.800000,P,2340260.000000,43919444.000000,-26.636499,-28.285000,-0.201710,-0.635564,P,0,1,P,0,1,P,0,1 ,07/19/2008,00:48:55,#280,Module 1 ,C66,0402,0402_1_3,P,146.934555,238.000000,84.000000,140.000000,P,4.062300,140.000000,P,266860.500000,470565.468750,156855.156250,313710.320000,P,15511.000000,313710.312500,P,40048568.000000,79054999.200000,21959722.000000,43919444.800000,P,235260.000000,43919444.000000,-32.004501,-21.340000,-0.163571,-1.147347,P,0,1,P,0,1,P,0,1

Posting Shark

972 posts since Jun 2008

Reputation Points: 392

Solved Threads: 109

what i have similar to split is...

String st = record;

String []splitSt = st.split(",");

System.out.println("string : " + st );
System.out.println("length :" + splitSt.length );
int i;

//Connecting to the database
//String URL =
//"jdbc:odbc:Driver={Microsoft Access Driver (*.mdb)};DBQ=E:\\public\\Log\\Results\\spimon.mdb;}",
String HOST = "jdbc:odbc:spimon",
DRIVER = "sun.jdbc.odbc.JdbcOdbcDriver";
Class.forName(DRIVER)/*.newInstance()*/;
myConn = DriverManager.getConnection(HOST,"","");
//Connection myConn = DriverManager.getConnection(URL,"","");

//using the database
Statement stat = myConn.createStatement();
PreparedStatement updaterecord;
String query="INSERT INTO logspi01 VALUES(?,?,?,?,?,?,?)"; //tem que ter o mesmo numero de campos da DB
updaterecord = myConn.prepareStatement(query);

for (i = 0; i < splitSt.length; i++ )
{
//updaterecord.setString ((i+1), splitSt[i]); /*or*/

updaterecord.setString((1),splitSt[0]); //machine
updaterecord.setString((2),splitSt[1]); //package
updaterecord.setString((3)," "); //lotnumber
updaterecord.setString((4),splitSt[2]); //Fail Location
updaterecord.setString((5),splitSt[3]); //Fail Qty
updaterecord.setString((6),splitSt[4]); //Date
updaterecord.setString((7),splitSt[5]); //Time
}
updaterecord.executeUpdate();
//int x = updaterecord.executeUpdate();
stat.close();
myConn.close();
//myConn = null;
//System.out.println(" Inserting Data " + x + " row(s)");

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

Here's and updated version of the class I gave you - should allow you to use the constants to retrieve data relative to pass-fail fields more leniently.

public class InspectionResults{

	public static final byte  HEIGHT_AVG_RESULT = 7,
							  HEIGHT_RANGE_RESULT = 12,
							  AREA_AVG_RESULT = 15,
							  AREA_RANGE_RESULT = 20,
							  VOLUME_AVG_RESULT = 23,
							  VOLUME_RANGE_RESULT = 28,
							  HAV_FAILED_FEATURE_RESULT = 35,
							  REG_FAILED_FEATURE_RESULT = 38,
							  BRIDGE_FAILED_FEATURE_RESULT = 41;
	private String retrievedData[];
	private boolean failed[];

	/**
	 * Constructs this InspectionResult with the data stored in the args.
	 * This class expects 44 values within the range of the args.
	 */
	public InspectionResults(String... args){
		retrievedData = args;
		boolean temp[] ={
			((retrievedData[7].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[12].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[15].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[20].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[23].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[28].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[35].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[38].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[41].equalsIgnoreCase("F")) ? true: false)
		};
		failed = temp;
	}

	/**
	 * Returns true if the given value has failed, returns false otherwise.
	 * It's preferred to use the constants defined within this class to get the
	 * desired information, and not regular ints.
	 */
	public boolean hasFailed(byte result) throws Exception{
		switch(result){
			case HEIGHT_AVG_RESULT:
				return failed[0];
			case HEIGHT_RANGE_RESULT:
				return failed[1];
			case AREA_AVG_RESULT:
				return failed[2];
			case AREA_RANGE_RESULT:
				return failed[3];
			case VOLUME_AVG_RESULT:
				return failed[4];
			case VOLUME_RANGE_RESULT:
				return failed[5];
			case HAV_FAILED_FEATURE_RESULT:
				return failed[6];
			case REG_FAILED_FEATURE_RESULT:
				return failed[7];
			case BRIDGE_FAILED_FEATURE_RESULT:
				return failed[8];
			default :
				throw new Exception("Attempt to access invalid result type! Use the Result Constants to avoid this error!");
		}
	}

	/**
	 * Returns the value next to the specified result.
	 */
	public String getAdjacentValue(byte result) throws Exception{
		if(result >= 0 && result < retrievedData.length - 1)
			return retrievedData[result + 1];
		else throw new Exception("Error! Attempt to access column with either no adjacent value or outside of data-range!");
	}

	/**
	 * Simply returns a String representing the date for each value in this class.
	 */
	@Override
	public String toString(){
		String temp = "";
		for(String element : retrievedData){
			if(element.toString() != retrievedData[retrievedData.length - 1])
				temp += element + ", ";
			else temp += element;
		}
		return temp;
	}
}

Posting Shark

972 posts since Jun 2008

Reputation Points: 392

Solved Threads: 109

I was reading your code... i'm really lousy...

thats why i need to know how should i use your code sorry....:( My code is

private class Start extends Task<Void, Void> {
	@Override protected Void doInBackground() throws InterruptedException {
			                         
			try {
							
				////Redirecting outputFrame = new Redirecting(false, false, null,500, 300, JFrame.HIDE_ON_CLOSE);
					
				/*... Initialize array of indentations */
				INDENTS[0] = INDENT_STR;
				for (int i = 1; i < MAX_DEPTH; i++) {
					INDENTS[i] = INDENTS[i-1] + INDENT_STR;
				}

				/*... Initialize read files.... */ 
				File Folder = new File("E:\\public\\Log\\Log");
				File FolderLD = new File("E:\\public\\Log\\LotData");
				File HostFile = new File("E:\\public\\Log\\SPIlog.txt");
				File HostFileLD = new File("E:\\public\\Log\\SPIlotData.txt");
				File LotInput = new File ("z:\\DataLot\\");
				
				File file = null;       
				File[] files = null;       
				FileReader fr = null;       
				BufferedReader br = null;
				FileOutputStream fos = null; 
				String line = " ";
				
				/* backup the file */
				setMessage("backup the SPI Log file");
				//setProgress(k, 0, 2);
				try {
					copyFile(new File("E:\\public\\Log\\SPIlog.txt"),new File("E:\\public\\Log\\Backups\\log.txt"));
				} catch (IOException e1) {
					e1.printStackTrace();
				}
				System.out.println("Backup File SPIlog.txt..........................Done");
				
				File LInput = new File ("z:\\SE300 Outputc");
				
				File HostFileLD01 = new File("E:\\public\\Log\\LotData\\LD01.log");
				File HostFileLD02 = new File("E:\\public\\Log\\LotData\\LD02.log");
				File HostFileLD03 = new File("E:\\public\\Log\\LotData\\LD03.log");
				File HostFileLD04 = new File("E:\\public\\Log\\LotData\\LD04.log");
				File HostFileLD05 = new File("E:\\public\\Log\\LotData\\LD05.log");
				File HostFileLD06 = new File("E:\\public\\Log\\LotData\\LD06.log");
				File HostFileLD07 = new File("E:\\public\\Log\\LotData\\LD07.log");
				File HostFileLD08 = new File("E:\\public\\Log\\LotData\\LD08.log");
				File HostFileLD09 = new File("E:\\public\\Log\\LotData\\LD09.log");
				File HostFileLD010 = new File("E:\\public\\Log\\LotData\\LD10.log");
				File HostFileLD011 = new File("E:\\public\\Log\\LotData\\LD11.log");
				File HostFileLD012 = new File("E:\\public\\Log\\LotData\\LD12.log");
				File HostFileLD013 = new File("E:\\public\\Log\\LotData\\LD13.log");
				File HostFileLD014 = new File("E:\\public\\Log\\LotData\\LD14.log");
				File HostFileLD015 = new File("E:\\public\\Log\\LotData\\LD15.log");
				File HostFileLD016 = new File("E:\\public\\Log\\LotData\\LD16.log");
				File HostFileLD017 = new File("E:\\public\\Log\\LotData\\LD17.log");
				File HostFileLD018 = new File("E:\\public\\Log\\LotData\\LD18.log");
				File HostFileLD019 = new File("E:\\public\\Log\\LotData\\LD19.log");
				File HostFileLD020 = new File("E:\\public\\Log\\LotData\\LD20.log");
				File HostFileLD021 = new File("E:\\public\\Log\\LotData\\LD21.log");
				File HostFileLD022 = new File("E:\\public\\Log\\LotData\\LD22.log");
				File HostFileLD023 = new File("E:\\public\\Log\\LotData\\LD23.log");
				
				File HostFile01 = new File("E:\\public\\Log\\Log\\SPI01.log");
				File HostFile02 = new File("E:\\public\\Log\\Log\\SPI02.log");
				File HostFile03 = new File("E:\\public\\Log\\Log\\SPI03.log");
				File HostFile04 = new File("E:\\public\\Log\\Log\\SPI04.log");
				File HostFile05 = new File("E:\\public\\Log\\Log\\SPI05.log");
				File HostFile06 = new File("E:\\public\\Log\\Log\\SPI06.log");
				File HostFile07 = new File("E:\\public\\Log\\Log\\SPI07.log");
				File HostFile08 = new File("E:\\public\\Log\\Log\\SPI08.log");
				File HostFile09 = new File("E:\\public\\Log\\Log\\SPI09.log");
				File HostFile10 = new File("E:\\public\\Log\\Log\\SPI10.log");
				File HostFile11 = new File("E:\\public\\Log\\Log\\SPI11.log");
				File HostFile12 = new File("E:\\public\\Log\\Log\\SPI12.log");
				File HostFile13= new File("E:\\public\\Log\\Log\\SPI13.log");
				File HostFile14 = new File("E:\\public\\Log\\Log\\SPI14.log");
				File HostFile15 = new File("E:\\public\\Log\\Log\\SPI15.log");
				File HostFile16 = new File("E:\\public\\Log\\Log\\SPI16.log");
				File HostFile17 = new File("E:\\public\\Log\\Log\\SPI17.log");
				File HostFile18= new File("E:\\public\\Log\\Log\\SPI18.log");
				File HostFile19 = new File("E:\\public\\Log\\Log\\SPI19.log");
				File HostFile20 = new File("E:\\public\\Log\\Log\\SPI20.log");
				File HostFile21 = new File("E:\\public\\Log\\Log\\SPI21.log");
				File HostFile22 = new File("E:\\public\\Log\\Log\\SPI22.log");
				File HostFile23 = new File("E:\\public\\Log\\Log\\SPI23.log");
			
				
				/* Creates the Object Log folder*/
				
				File fY = new File("E:\\public\\Log\\");
				/* if it doesn't exists*/
				if(!fY.exists())
					System.out.println("Log Folder was created");
					setMessage("Log Folder was created");
					//setProgress(k, 3, 4);
				fY.mkdir();
				/* if exists*/
				if(fY.exists())
					//System.out.println("Log Folder................................... Done;");
					setMessage("Log Folder................................... Done;");
					//setProgress(k, 5, 6);
				File fYY = new File("E:\\public\\Log\\Log");
				/* if it doesn't exists*/
				if(!fYY.exists())
					//System.out.println("Log SubFolder was created");
					setMessage("Log SubFolder was created");
				//setProgress(k, 7, 8);
				fYY.mkdir();
				/* if exists*/
				if(fYY.exists())
					//System.out.println("Log SubFolder................................... Done;");
					setMessage("Log SubFolder................................... Done;");
				//setProgress(k, 9, 10);
				
				File fxY = new File("E:\\public\\Log\\LotData");
				/* if it doesn't exists*/
				if(!fxY.exists()){
					//System.out.println("LotData SubFolder was created");
					setMessage("LotData SubFolder was created");
					//setProgress(k, 10, 11);
					fxY.mkdir();}
				/* if exists*/
				if(fxY.exists())
					//System.out.println("LotData SubFolder............................... Done;");
					setMessage("LotData SubFolder............................... Done;");
				
				
				/* ---------------------SPI01----------------------- */
				/*Creates the Object file SPI1.log */
				//outputFrame.aTextArea.setText("");
				File f1 = new File("E:\\public\\Log\\Log\\SPI01.log"); 
				if(f1.exists())
					//System.out.println("Log File SPI01..................................... Done;");
					setMessage("Log File SPI01..................................... Done;");
				/* if it doesn't exists */
				if(!f1.exists())
					//System.out.println("The Log file does not exist");
					setMessage("The Log file does not exist");
				/* New file ... exception popup */
				try {
					if(f1.createNewFile())
						//System.out.println("Log file created.................................. Done;");
						setMessage("Log file created.................................. Done;");
				}
				catch (IOException e) {
					//System.out.println("I/O Error");
					setMessage("I/O Error");
				}
				/*Creates the Object file LD01.log */
				File ff1 = new File("E:\\public\\Log\\LotData\\LD01.log"); 
				if(ff1.exists())
					//System.out.println("LotData SPI01..................................... done;");
					setMessage("LotData SPI01..................................... done;");
				/* if it doesn't exists */
				if(!ff1.exists()){
					setMessage("The LotData file does not exist");
					//System.out.println("The LotData file does not exist");
					/* New file ... exception popup */
					try {
						if(ff1.createNewFile())
							//System.out.println("LotData file created.................................. done;");
							setMessage("LotData file created.................................. done;");
					}
					catch (IOException e) {
						//System.out.println("I/O Error");
						setMessage("I/O Error");
					}
				}		

				/* mapping */
				/*   Creates a temporary Network Drive before starting the cycle */
				String mapSPI01 = new String("net use z: \\\\172.21.34.1\\d$ qmy001 /user:172.21.34.1\\administrator /Persistent:No");            
				Runtime rt = Runtime.getRuntime();
				try     {
					Process con = rt.exec(mapSPI01);
					con.waitFor();
					//System.out.println("Drive mapping Spi 01......................... Done");
					setMessage("Drive mapping Spi 01......................... Done");
					}
				catch (IOException e)   {
					System.out.println(e);
					}
				catch (InterruptedException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}
					
					/*Copy LotData contents from d:\DataLot\LotData.csv to the Respective LD.log*/
					try {       
						if (LotInput.isDirectory()) { 
							//fos = new FileOutputStream(HostFileLD01);   
							files = LotInput.listFiles();
							xw = new FileWriter(HostFileLD01);
							
							for (int i = 0; i < files.length; i++) {       
								file = files[i];       
			
								fr = new FileReader(file);       
								br = new BufferedReader(fr); 
								x=1;
			
								while ((line = br.readLine()) != null) {
									//System.out.println(line);
									xw.write(x+","+line + System.getProperty("line.separator")) ;//change line
									xw.flush();
								}
							} 
							
						}  
						       
					}
					catch (FileNotFoundException fnex) {       
						fnex.printStackTrace();       
					} 
					catch (IOException ioex) {       
						ioex.printStackTrace();       
					}
					/*end of copy*/
					
					setMessage("Collecting Data from SPI 01");
					/* Scanning */
					File root = new File("z:\\SE300 Output");
					//File root = new File("K:\\spi15\\SE 300 Output\\CSV");
					if (root != null && root.isDirectory()) {
						try{
							fw = new FileWriter("E:\\public\\Log\\Log\\SPI01.log");
							x=1;
							listRecursively(root, 0);
							fw.close();
							}
					catch(NullPointerException np){
						np.printStackTrace();
					}
					catch(IOException exc){
						exc.printStackTrace();
					}
				}
				else
				{
					//System.out.println("Not a directory: " + root);
					setMessage("Not a directory: " + root);}

				/* Deletes temporary Network Drive before finishing the cycle */
					try
			        {            
			            String[] cmd = new String[3];
			                  
			            cmd[0] = "cmd.exe" ;
			            cmd[1] = "/C" ;
			            cmd[2] = "net use z: /delete /y";
			       
			      
			            Runtime rtoff = Runtime.getRuntime();
			            //System.out.println("Executing " + cmd[0] + " " + cmd[1] 
			            //                  + " " + cmd[2]);
			            setMessage("Executing " + cmd[0] + " " + cmd[1] 
			              			                               + " " + cmd[2]);
			            Process proc = rtoff.exec(cmd);
			           
			            // any error message?
			            StreamGobbler1 errorGobbler = new 
			                StreamGobbler1(proc.getErrorStream(), "ERROR");            
			            
			            // any output?
			            StreamGobbler1 outputGobbler = new 
			                StreamGobbler1(proc.getInputStream(), "OUTPUT");
			                
			            // kick them off
			            errorGobbler.start();
			            outputGobbler.start();
			                                    
			            // any error???
			            int exitVal = proc.waitFor();
			            setMessage("ExitValue: " + exitVal);
			        } catch (Throwable t)
			          {
			            t.printStackTrace();
			          }
//				String mapSPIDEL = new String("net use z: /delete /y");            
//				Runtime rtoff = Runtime.getRuntime();
//				try     {
//					Process con = rtoff.exec(mapSPIDEL);
//					con.waitFor();
//					System.out.println("Network Drive deleted......................... Done");
//				}
//				catch (IOException e)   {
//					System.out.println(e);
//				}

and then goes to the second machine and so on and so on.. i'm using a recursive method to read and access all the csv files...

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

/* .........................Recursive list............................... */
	public  void listRecursively(File fdir, int depth) throws IOException {
		
		/*Transform milliseconds time to gregorian time */
		long datefiles = fdir.lastModified();
		SimpleDateFormat Date = new SimpleDateFormat (" dd/MM/yyyy , HH:mm:ss aaa");
		Date nDate = new Date(datefiles);
		
		String F = ",F,";
		int count = 0;
		
		/*Line counter*/
		try
		{
			RandomAccessFile File = new RandomAccessFile(fdir,"r");
			long lastline=File.length();
			File.close();
			FileReader fileRead = new FileReader(fdir);
			BufferedReader bufferReader = new BufferedReader(fileRead);
			Scanner scan = new Scanner(fdir);
			while(scan.hasNextLine())
			{
				if(scan.nextLine().contains(F))
					count++;
			}
			
			fileRead.close();
			bufferReader.close();
			
//			/*Start Line counter*/
//			LineNumberReader lineRead = new LineNumberReader(fileRead);
//			lineRead.skip(lastline);
//			int countline = lineRead.getLineNumber()-6; //number of default lines = 6
//			fileRead.close();
//			lineRead.close();
//			/*End Line counter*/
			
		/* Output1 */
			if (fdir.getPath().endsWith(".csv") /*&& fdir.lastModified() > HostFile.lastModified()*/)
				System.out.println(INDENTS[depth] + x +", "+fdir.getName() +", "+ count +","+Date.format(nDate));
		/* Output2 */
			if (fdir.getPath().endsWith(".csv") /*&& fdir.lastModified() > HostFile.lastModified()*/)
				fw.write( INDENTS[depth] + x +", "+ fdir.getName() +", "+ count +","+ Date.format(nDate)+ System.getProperty("line.separator"));
			fw.flush();
		//if (fdir.getPath().endsWith(".csv") && (fdir.length()/512 )>= 1 && fdir.length()/512 <= 3)	
		}
		catch(IOException e){
		} 
		if (fdir.isDirectory() && !fdir.isHidden() && depth < MAX_DEPTH) {
			for (File f : fdir.listFiles()){  // Go over each file/subdirectory.
				listRecursively(f, depth+1);
				}}}

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

Hi Alex!

First i would like to say. Thank you very much!

Secondly... i'm lost i don't know how to use your code despite i understand the concept behind! I need to reed more in order to apply your knowledge...

I was stopped for 1,5 years...

Regards,

jorge

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

RandomAccessFile File = new RandomAccessFile(fdir,"r"); long lastline=File.length(); File.close(); FileReader fileRead = new FileReader(fdir); BufferedReader bufferReader = new BufferedReader(fileRead); Scanner scan = new Scanner(fdir); while(scan.hasNextLine()) { if(scan.nextLine().contains(F)) count++; } fileRead.close(); bufferReader.close();

Ok your problem is that you need to read a big line of text and yoou're using a Scanner instead of a Buffered Reader. Buffered Readers, as the name implies, are made for long lines of text and can be size for scanning lines of the appropriate length.

After scanning a line, you may want to do something like parse the first comma (since they are comma separated values and I believe the first value you encounter would be considered a " " or "" due to the comma that exists before the data).

Create a new object of my class immediately after doing so. It would look something like this-- BufferedReader br = new BufferedReader(fileRead/*, int line size*/ ); // set length of line you will be reading via BufferedReader constructor while(br.ready()){ StringBuilder sb = new StringBuilder(br.readLine()); // truncate or replace or remove unnecessary values like first comma and the last InspectionResults ir = new InspectionResults( sb.toString(). split(",")); // perform some action if a particular fail is acknowledged via my class //store information if needed }

By the way, this smells like a sub-module or a part of a company project >_>

Posting Shark

972 posts since Jun 2008

Reputation Points: 392

Solved Threads: 109

Hi again...

I'm completely lost... despite the fact i don't know how to use your code... my head is almost getting nuts... i've found that some records have 32 instead of 45.. therefore i'm using the ... If (linesize =45)...i'm applying your code.. else... i'm using your codebase but adjusted for the record in question

else

Light Poster

45 posts since Apr 2008

Reputation Points: 7

Solved Threads: 0

Here's a test I ran using this file--

INSPECT.txt
,07/19/2008,00:48:55,#280,Module 1 ,IC2,TSOP8,SOP8_1,F,138.539642,238.000000,84.000000,140.000000,P,9.226400,140.000000,P,427747.781250,729933.140625,243311.046875,486622.090000,P,26717.400391,486622.093750,P,60986780.000000,122628772.800000,34063548.000000,68127092.600000,P,3087930.000000,68127096.000000,-22.886625,-23.317499,-0.039985,-0.131595,P,0,1,P,0,1,P,0,1 ,07/19/2008,00:48:55,#280,Module 1 ,C38,0402,0402_1_3,P,131.077194,238.000000,84.000000,140.000000,P,12.636800,140.000000,P,265426.750000,470565.468750,156855.156250,313710.320000,P,5929.500000,313710.312500,P,36161180.000000,79054999.200000,21959722.000000,43919444.800000,P,2340260.000000,43919444.000000,-26.636499,-28.285000,-0.201710,-0.635564,P,0,1,P,0,1,P,0,1 ,07/19/2008,00:48:55,#280,Module 1 ,C66,0402,0402_1_3,P,146.934555,238.000000,84.000000,140.000000,P,4.062300,140.000000,P,266860.500000,470565.468750,156855.156250,313710.320000,P,15511.000000,313710.312500,P,40048568.000000,79054999.200000,21959722.000000,43919444.800000,P,235260.000000,43919444.000000,-32.004501,-21.340000,-0.163571,-1.147347,P,0,1,P,0,1,P,0,1

import java.io.*;
import java.util.Arrays;

public class InspectionResults{

	public static final byte  HEIGHT_AVG_RESULT = 7,
							  HEIGHT_RANGE_RESULT = 12,
							  AREA_AVG_RESULT = 15,
							  AREA_RANGE_RESULT = 20,
							  VOLUME_AVG_RESULT = 23,
							  VOLUME_RANGE_RESULT = 28,
							  HAV_FAILED_FEATURE_RESULT = 35,
							  REG_FAILED_FEATURE_RESULT = 38,
							  BRIDGE_FAILED_FEATURE_RESULT = 41;
	private String retrievedData[];
	private boolean failed[];

	/**
	 * Constructs this InspectionResult with the data stored in the args.
	 * This class expects 44 values within the range of the args.
	 */
	public InspectionResults(String... args){
		retrievedData = args;
		boolean temp[] ={
			((retrievedData[7].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[12].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[15].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[20].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[23].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[28].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[35].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[38].equalsIgnoreCase("F")) ? true: false),
			((retrievedData[41].equalsIgnoreCase("F")) ? true: false)
		};
		failed = temp;
	}

	public static void main(String... args){

		FileReader fr = null;
		BufferedReader br = null;
		try{
			fr = new FileReader(new File("INSPECT.txt"));
			br = new BufferedReader(fr);
		}catch(Exception e){e.printStackTrace();}


		String dwArray[][] ={ {""}, {""}, {""} };

		for(int i = 0; i < dwArray.length; i++){
			String temp[] = null;

			try{ temp = br.readLine().split(",");}catch(Exception f){f.printStackTrace(); System.exit(1);};
			temp = Arrays.<String>copyOfRange(temp, 1, temp.length);
			dwArray[i] = temp;
		}


		InspectionResults ir[] =
		{
			new InspectionResults(dwArray[0]),
			new InspectionResults(dwArray[1]),
			new InspectionResults(dwArray[2])
		};

		System.out.println(ir[0]); // as an example
		spacer(3);

		try{
			System.out.println(ir[0].hasFailed(InspectionResults.HEIGHT_AVG_RESULT));
			System.out.println(ir[0].getAdjacentValue(InspectionResults.HEIGHT_AVG_RESULT));
		}catch(Exception e){
			System.out.println(e);
		}

		try{
			fr.close();
			br.close();
		}catch(Exception e){
		}
	}

	private static void spacer(int lines){
		for(int i = 0; i < lines; i++)
			System.out.println();
	}

	/**
	 * Returns true if the given value has failed, returns false otherwise.
	 * It's preferred to use the constants defined within this class to get the
	 * desired information, and not regular ints.
	 */
	public boolean hasFailed(byte result) throws Exception{
		switch(result){
			case HEIGHT_AVG_RESULT:
				return failed[0];
			case HEIGHT_RANGE_RESULT:
				return failed[1];
			case AREA_AVG_RESULT:
				return failed[2];
			case AREA_RANGE_RESULT:
				return failed[3];
			case VOLUME_AVG_RESULT:
				return failed[4];
			case VOLUME_RANGE_RESULT:
				return failed[5];
			case HAV_FAILED_FEATURE_RESULT:
				return failed[6];
			case REG_FAILED_FEATURE_RESULT:
				return failed[7];
			case BRIDGE_FAILED_FEATURE_RESULT:
				return failed[8];
			default :
				throw new Exception("Attempt to access invalid result type! Use the Result Constants to avoid this error!");
		}
	}

	/**
	 * Returns the value next to the specified result.
	 */
	public String getAdjacentValue(byte result) throws Exception{
		if(result >= 0 && result < retrievedData.length - 1)
			return retrievedData[result + 1];
		else throw new Exception("Error! Attempt to access column with either no adjacent value or outside of data-range!");
	}

	/**
	 * Simply returns a String representing the data for each value in this class.
	 */
	@Override
	public String toString(){
		String temp = "";
		for(String element : retrievedData){
			if(element.toString() != retrievedData[retrievedData.length - 1])
				temp += element + ", ";
			else temp += element;
		}
		return temp;
	}
}