Hi!
It is called secant updates of the quadratic model.
You can google "levenberg marquardt secant updates" for a detailed explanation, but shortly speaking - after computing step with finite differences Jacobian, the solver tries to perform cheap update of the internally stored Jacobian using information obtained from the step. This update needs just one function vector evaluation, not N, so it is really cheap. On the other hand, it is definitely imprecise, and updated Jacobian quickly becomes irrelevant, so after several steps the solver recomputes Jacobian from scratch anyway.
|