Hello, I am new to Alglib community, and I have run into a problem.
Recentlly I am trying to write my own Machine Learning library in C++. I a, using Eigen3, which is a very good linear algebra library, but doesn't support nonlinear convex optimization, which is really usefull for ML. Thus I try to use Alglib for the task of convex optimization.
I have written the following code:
Code:
void alglibCG(VectorParametrizedModel * vpm) {
double epsg = 0.000001;
double epsf = 0;
double epsx = 0;
double stpmax = 0.1;
alglib::ae_int_t maxits = 0;
alglib::real_1d_array x;
x.setcontent((*vpm).theta.rows(), (*vpm).theta.data());
alglib::mincgstate state;
alglib::mincgreport rep;
alglib::mincgcreate(x,state);
alglib::mincgsetcond(state, epsg, epsf, epsx, maxits);
alglib::mincgsetstpmax(state, stpmax);
alglib::mincgoptimize(state, alglibCostAndGradient, NULL, vpm);
alglib::mincgresults(state, x, rep);
std::cout << x.tostring((*vpm).theta.rows()).c_str() << "\n";
std::cout << int(rep.terminationtype) << "\n";
for (int i=0; i< (*vpm).theta.rows(); ++i) {
(*vpm).theta(i) = x[i];
}
}
A VectorParametrizedModel is an abstraction of a ML model that I want to train.
The alglibCostAndGradient function computes value of the cost function and gradient of the cost function at points x, using the VectorParametrizedModel object that is passed by the (void * ) argument of mincgoptimize.
Code:
void alglibCostAndGradient(const alglib::real_1d_array &x,
double &func, alglib::real_1d_array &grad, void * ptr) {
VectorParametrizedModel * vpm = static_cast<VectorParametrizedModel*>(ptr);
(*vpm).computeCostAndGradient();
func = (*vpm).cost;
for (int i=0; i< (*vpm).gradient.rows(); ++i) {
grad[i] = (*vpm).gradient(i);
}
}
All of this code compiles.
Now the problem. I have tried to train a LogisticRegression using this functions, and the mincgoptimize function returns x = [0, 0, 0]. Also, rep.terminationtype is 1.
I am also quite confident that gradient and cost functions are correctly implemented, since on itself it outputs the same values as different implementation of Log Regression cost/gradient (which I took from Coursera ML class, so I suppose is correct).
I don't know what I have done wrong, and what terminationtype 1, hence I ask for help.
Also If it is possible, please provide me with a source to decode terminationtype values, for future references.
Thank you for reading through my problems and thank you in adavnce for answering ;)
Login
Register
FAQ
Search
