ALGLIB forum
It is currently Tue Nov 29, 2022 2:33 pm

All times are UTC

Forum rules

1. This forum can be used for discussion of both ALGLIB-related and general numerical analysis questions
2. This forum is English-only - postings in other languages will be removed.

Post new topic Reply to topic  [ 2 posts ] 
Author Message
 Post subject: mincgoptimize returns zeros
PostPosted: Sun Sep 30, 2012 9:56 pm 

Joined: Sun Sep 30, 2012 9:39 pm
Posts: 2
Hello, I am new to Alglib community, and I have run into a problem.

Recentlly I am trying to write my own Machine Learning library in C++. I a, using Eigen3, which is a very good linear algebra library, but doesn't support nonlinear convex optimization, which is really usefull for ML. Thus I try to use Alglib for the task of convex optimization.

I have written the following code:

void alglibCG(VectorParametrizedModel * vpm) {

    double epsg = 0.000001;
    double epsf = 0;
    double epsx = 0;
    double stpmax = 0.1;
    alglib::ae_int_t maxits = 0;

    alglib::real_1d_array x;
    x.setcontent((*vpm).theta.rows(), (*vpm);

    alglib::mincgstate state;
    alglib::mincgreport rep;

    alglib::mincgsetcond(state, epsg, epsf, epsx, maxits);
    alglib::mincgsetstpmax(state, stpmax);

    alglib::mincgoptimize(state, alglibCostAndGradient, NULL, vpm);
    alglib::mincgresults(state, x, rep);

    std::cout << x.tostring((*vpm).theta.rows()).c_str() << "\n";
    std::cout << int(rep.terminationtype) << "\n";
    for (int i=0; i< (*vpm).theta.rows(); ++i) {
        (*vpm).theta(i) = x[i];

A VectorParametrizedModel is an abstraction of a ML model that I want to train.

The alglibCostAndGradient function computes value of the cost function and gradient of the cost function at points x, using the VectorParametrizedModel object that is passed by the (void * ) argument of mincgoptimize.

void alglibCostAndGradient(const alglib::real_1d_array &x,
        double &func, alglib::real_1d_array &grad, void * ptr) {
    VectorParametrizedModel * vpm = static_cast<VectorParametrizedModel*>(ptr);
    func = (*vpm).cost;
    for (int i=0; i< (*vpm).gradient.rows(); ++i) {
        grad[i] = (*vpm).gradient(i);

All of this code compiles.

Now the problem. I have tried to train a LogisticRegression using this functions, and the mincgoptimize function returns x = [0, 0, 0]. Also, rep.terminationtype is 1.

I am also quite confident that gradient and cost functions are correctly implemented, since on itself it outputs the same values as different implementation of Log Regression cost/gradient (which I took from Coursera ML class, so I suppose is correct).

I don't know what I have done wrong, and what terminationtype 1, hence I ask for help.
Also If it is possible, please provide me with a source to decode terminationtype values, for future references.

Thank you for reading through my problems and thank you in adavnce for answering ;)

 Post subject: Re: mincgoptimize returns zeros
PostPosted: Mon Oct 01, 2012 12:17 am 

Joined: Sun Sep 30, 2012 9:39 pm
Posts: 2
I solved my own problem. There was a stupid bug, still visible in the code above, namely I didn't set theta to x before computing gradient ...

All works now, alglib is a great library, I must say the (void *) parameter of mincgoptimize was very thoughtfull.

I would still love some reference to what different terminationtype values mean.


Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

All times are UTC

Who is online

Users browsing this forum: Bing [Bot] and 2 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group