Problems with data access/operations

[Swiftle]

Hello

I built a personal 2D array for my current project. I also store these arrays (called Matrix) in a std::vector. The matrices are 24*181 and currently I sue between 30 and 50 of them. I don't think project details are relevant however I'm getting 3 errors which I have no idea where they might be coming from.

1)After some calculations on the data, the output will feature a random 1.#J / 1.#R or something like this. This is strange because in all my calculations I never divide by zero, plus I'll get 0-2 of those per Matrix; NOTE: this happens only after operations are done on the arrays, not on initialization.

2)In one function I try to determine the best possible Matrix. The function does it's job well but upon exit the program crashes and I have no idea why. I tried debugging with Dev C++ but nothing showed.

3)Finally, when attempting to access a specific value (within range) within a vector,I also get a crash. This is strange because it's always the same value, and judging by the output file there isn't anything wrong with that particular value. On top of that I already used the same value beforehand in another method.

#include "Matrix.hpp"
#include "Constants.hpp"
#include "Functions.hpp"
#include "InvertMatrix.hpp"

#include <boost/numeric/ublas/matrix.hpp>
#include <boost/numeric/ublas/lu.hpp>

#include <iostream>
#include <stdio.h>
#include <time.h>
#include <math.h>


using namespace std;


/******************************************************************************/
/*                           FIRST SIMULTATION                                */
/******************************************************************************/


void getPerDayAvg(vector<Matrix> days, vector<Matrix> &mean, int avgDays) {   
    for(int i = 0; i < TOTAL_DAYS - avgDays; i++) {
        mean.push_back(Matrix());
        for(int hour = 0; hour < HOURS; hour ++) {
            for(int angle = 0; angle < ANGLES; angle ++)  {
                for(int day = i; day < i + avgDays; day++)
                    mean[i].data[hour][angle] += days[day].data[hour][angle];
                mean[i].data[hour][angle] /= avgDays; 
                mean[i].data[hour][angle] = pow(days[avgDays+i].data[hour][angle] - mean[i].data[hour][angle],2);
            } 
        }    
    }
}

void getGlobalAvg(vector<Matrix> &globalMean, vector<Matrix> &mean, int index) {
    globalMean.push_back(Matrix());
	for(int hour = 0; hour < HOURS; hour++) {
		for(int angle = 0; angle < ANGLES; angle++) {
            for(int matrix = 0; matrix < mean.size() ; matrix++)
               globalMean[index-1].data[hour][angle] += mean[matrix].data[hour][angle];
            globalMean[index-1].data[hour][angle] /= mean.size();
        }
    }
    mean.clear();
}

void getMeans(vector<Matrix> dataSample, vector<Matrix> avgPD, vector<Matrix> &avgGlobal){
    for(int i = 1; i <= MAX_DAYS_TAKEN; i++) {
        getPerDayAvg(dataSample, avgPD, i);
        getGlobalAvg(avgGlobal, avgPD, i);
    }
    write(avgGlobal,"constantModel.txt");
}


/******************************************************************************/
/*                          SECOND SIMULTATION                                */
/******************************************************************************/


void linearRegression(vector<Matrix> days, vector<Matrix> &prediction, vector<Matrix> today, int avgDays){
    using namespace boost::numeric::ublas;
    cout << "OK" <<endl;
    cout << days[40].data[1][136]<<endl;
    cout << "notok" <<endl;
    matrix<double> first;
    matrix<double> invert (avgDays, avgDays);
    matrix<double> X (TOTAL_DAYS - avgDays,avgDays);
    matrix<double> Y (TOTAL_DAYS - avgDays, 1);
    matrix<double> B (avgDays, 1);
    
    for(int hour = 0; hour < HOURS; hour++){
        for(int angle = 0; angle < ANGLES; angle++){

            for(int i = avgDays; i < TOTAL_DAYS; i++)
                Y.insert_element(i-avgDays,0,days[i].data[hour][angle]);
            for(int i = 0; i < TOTAL_DAYS - avgDays; i++)
                for(int j = 0; j < avgDays; j++)
                    X.insert_element(i,j,days[i+j].data[hour][angle]);
            
            first = prod(trans(X),X);
            InvertMatrix(first,invert);
            B = prod(invert,trans(X));
            B = prod(B,Y);        

            for(int i = 0; i < avgDays; i++)
                prediction[avgDays-1].data[hour][angle] += B(i,0)*days[TOTAL_DAYS - i].data[hour][angle];

            prediction[avgDays - 1].data[hour][angle] = pow(prediction[avgDays - 1].data[hour][angle] - today[0].data[hour][angle],2);   
        }
    } 
}

/******************************************************************************/
/*                          GENERAL FUNCTIONS                                 */
/******************************************************************************/


void simulation(Matrix average, Matrix m) {
    int cost;
    int currentPosition = 181;
    int bestPosition;
    double currentGain, bestGain;
    
    for(int hour = 0; hour < HOURS; hour++){
        cout << "Hour: "<<hour<<"\t";
        currentPosition = optimalPosition(hour, m);
        currentGain = m.data[hour][currentPosition];
        bestPosition = optimalPosition(hour, average);
        bestGain = average.data[hour][bestPosition];
        cost = abs(currentPosition - bestPosition)*COST;
        if(bestGain < cost && bestGain > currentGain){
            currentPosition = bestPosition;
            cout << "   Moving panel by " << abs(currentPosition - bestPosition) << " degrees." << endl;
        }
        else
            cout << "   Not moving " << endl;
    
    }
} 


int getBest(vector<Matrix> global){    
    vector<int> winner;
    for(int i = 0; i < MAX_DAYS_TAKEN;i++)
        winner.push_back(0);
    double min;
    int minIndex;

    for(int hour = 0; hour < HOURS; hour++) {
        for(int angle = 0; angle < ANGLES; angle++) {
            min = 100.0;
            for(int matrix = 0; matrix < global.size() ; matrix++){
                if((global[matrix].data[hour][angle] < min) and (global[matrix].data[hour][angle] != 0.0)) {
                    min = global[matrix].data[hour][angle];
                    minIndex = matrix;
                }
            }
            if(min != 0.0)
                winner[minIndex]++;
        }
    }
    for(int i = 0; i < winner.size(); i++)
        cout << winner[i] << " ";
    vector<int>::iterator result;
    result = max_element(winner.begin(), winner.end());
    return (int)distance(winner.begin(), result);
}

void getAverage(vector<Matrix> sample, Matrix &avg, int days){
    cout << days<<endl;
    for(int hour = 0; hour < HOURS; hour ++) {
        for(int angle = 0; angle < ANGLES; angle ++)  {
            for(int i = TOTAL_DAYS - days; i < TOTAL_DAYS; i++) 
                avg.data[hour][angle] += sample[i].data[hour][angle];
            avg.data[hour][angle] /= days; 
        }
    }
    write(avg, "average.txt");
}

void init(vector<Matrix> &m, vector<Matrix> &predictions){
    for(int i = 0; i < TOTAL_DAYS; i++) 
        m.push_back(Matrix(i));
    for(int i = 0; i < MAX_DAYS_TAKEN; i++)
        predictions.push_back(Matrix());
    write(m,"base.txt");
} 


int main(){ 
    srand (time(NULL));
    vector<Matrix> __daysSample, __meanPerDays, __globalMean, __predictions, __today;
    int best;
    
    Matrix average = Matrix();
    Matrix today = Matrix(1);
    init(__daysSample,__predictions);
    write(today, "today.txt");
    
    
    
    //getMeans(__daysSample, __meanPerDays, __globalMean);
    //best =  getBest(__globalMean) +1;
    //cout << "days chosen:" << best << endl;
    //getAverage(__daysSample, average,best);  
    //simulation(average,today);
   
    
    for(int i = 1; i <= MAX_DAYS_TAKEN; i++)
        linearRegression(__daysSample,__predictions,__today, i);
    write(__predictions, "result.txt");
    
    //best = getBest(__predictions) + 1;
    //cout << "days chosen:" << best;
    //simulation(best, __today, __predictions);   


    cin.get();
}

I can provide the rest if someone wants to compile. This is the main file. The first error appears in the .txt files. The second appears in main() right after getMeans(...)-->getBest(...)-->ERROR;
The final error appears in the first function simulation(...) @

prediction[avgDays-1].data[hour][angle] += B(i,0)*days[TOTAL_DAYS - i].data[hour][angle];

. Currently I get errors for hour = 1; angle = 136;

[Swiftle]

Maybe someone could tell me something about using large containers that I didn't know about?

[VernonDozier]

The first step in tracking down the problem would be to get more information on point number 3.

3)Finally, when attempting to access a specific value (within range) within a vector,I also get a crash.

If it's truly "within range", I don't see how it can crash. How do you know it's within range? What's the error? If you are using a vector and accessing via the "at" function, I think you should be able to catch an out_of_range error. A quick check of your code makes it appear that you are using [] to access elements, which DOES NOT throw that error. Bottom line is that perhaps you are correct and the element is indeed "within range", but unless you are 100% sure, you need to use the "at" function and catch that potentially thrown error.

[Swiftle]

I'm sure because I use the same data in the first and the second model AND I also write it down in a file. So it goes like this:

1)Data is created
2)Data is stored in a .txt file
3)Data is used in the first simulation

As soon as I try to use it in the second simulation I get a crash. I thought it was caused by mixing boost arrays with my own, but then why does it crash @ [1][136] and not in the beginning? I will try using .at to see if something changes.

[Swiftle]

As expected I made a stupid mistake with ranges... forgetting that an array with size N stops at N-1 -_-.

Now that this problem is fixed I still have #1 and #2 to solve. I was thinking about the first problem, and if I'm not mistaken this the #1.N/R/J errors occur when there is division by zero or when the number gets out of bounds. Is there a way to limit my doubles during calculations (not during output) so they will stop at a certain point after the decimal? (e.g. using only 3 numbers after the decimal)?

[VernonDozier]

>> if I'm not mistaken this the #1.N/R/J errors occur when there is division by zero or when the number gets out of bounds.

I can't remember precisely, but yeah, I believe those # signs will represent something that's "not a number" or "infinity" of some type, so somewhere you are overflowing or ending up with something that's "not a number" (dividing by 0 seems like a good cause for that).

You're using boost already, so this might help in testing.

http://www.boost.org/doc/libs/1_38_0/libs/math/doc/sf_and_dist/html/math_toolkit/special/fpclass.html

You may also be able to use cmath to test. I don't know. Obviously all bets are off when you do arithmetic on something that's not a number or infinity.

As far as how it got there and how to prevent it, it can be from a round-off error, an overflow, a divide by zero, who knows. As far as "stopping at" a certain point in order to maintain some control over a potential round-off error, you can do whatever you like, but I imagine you'll have to write your own round-off routine and do some studying about how doubles are represented internally. I doubt the rounding will be as easy as integer rounding with those nice % and / operators. Boost may have some rounding functions for doubles, and a quick glance through cmath shows a few functions that may or may not help, but it's been my experience that finding the roundoff errors in floating point arithmetic using stuff like matrix multiplication is a tedious process.