forum.alglib.net
http://forum.alglib.net/

mincgoptimize returns zeros
http://forum.alglib.net/viewtopic.php?f=2&t=620
Page 1 of 1

Author:  sjm_majewski [ Sun Sep 30, 2012 9:56 pm ]
Post subject:  mincgoptimize returns zeros

Hello, I am new to Alglib community, and I have run into a problem.

Recentlly I am trying to write my own Machine Learning library in C++. I a, using Eigen3, which is a very good linear algebra library, but doesn't support nonlinear convex optimization, which is really usefull for ML. Thus I try to use Alglib for the task of convex optimization.

I have written the following code:

Code:
void alglibCG(VectorParametrizedModel * vpm) {

    double epsg = 0.000001;
    double epsf = 0;
    double epsx = 0;
    double stpmax = 0.1;
    alglib::ae_int_t maxits = 0;

    alglib::real_1d_array x;
    x.setcontent((*vpm).theta.rows(), (*vpm).theta.data());

    alglib::mincgstate state;
    alglib::mincgreport rep;

    alglib::mincgcreate(x,state);
    alglib::mincgsetcond(state, epsg, epsf, epsx, maxits);
    alglib::mincgsetstpmax(state, stpmax);

    alglib::mincgoptimize(state, alglibCostAndGradient, NULL, vpm);
    alglib::mincgresults(state, x, rep);

    std::cout << x.tostring((*vpm).theta.rows()).c_str() << "\n";
    std::cout << int(rep.terminationtype) << "\n";
   
    for (int i=0; i< (*vpm).theta.rows(); ++i) {
        (*vpm).theta(i) = x[i];
    }
}


A VectorParametrizedModel is an abstraction of a ML model that I want to train.

The alglibCostAndGradient function computes value of the cost function and gradient of the cost function at points x, using the VectorParametrizedModel object that is passed by the (void * ) argument of mincgoptimize.

Code:
void alglibCostAndGradient(const alglib::real_1d_array &x,
        double &func, alglib::real_1d_array &grad, void * ptr) {
    VectorParametrizedModel * vpm = static_cast<VectorParametrizedModel*>(ptr);
    (*vpm).computeCostAndGradient();
    func = (*vpm).cost;
    for (int i=0; i< (*vpm).gradient.rows(); ++i) {
        grad[i] = (*vpm).gradient(i);
    }
}


All of this code compiles.

Now the problem. I have tried to train a LogisticRegression using this functions, and the mincgoptimize function returns x = [0, 0, 0]. Also, rep.terminationtype is 1.

I am also quite confident that gradient and cost functions are correctly implemented, since on itself it outputs the same values as different implementation of Log Regression cost/gradient (which I took from Coursera ML class, so I suppose is correct).

I don't know what I have done wrong, and what terminationtype 1, hence I ask for help.
Also If it is possible, please provide me with a source to decode terminationtype values, for future references.

Thank you for reading through my problems and thank you in adavnce for answering ;)

Author:  sjm_majewski [ Mon Oct 01, 2012 12:17 am ]
Post subject:  Re: mincgoptimize returns zeros

I solved my own problem. There was a stupid bug, still visible in the code above, namely I didn't set theta to x before computing gradient ...

All works now, alglib is a great library, I must say the (void *) parameter of mincgoptimize was very thoughtfull.

I would still love some reference to what different terminationtype values mean.

Cheers!

Page 1 of 1 All times are UTC
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/