Skip to content

Commit 76e20a0

Browse files
committed
[SPARK-6141][MLlib] Upgrade Breeze from 0.10 to 0.11 to fix convergence bug
LBFGS and OWLQN in Breeze 0.10 has convergence check bug. This is fixed in 0.11, see the description in Breeze project for detail: scalanlp/breeze#373 (comment) Author: Xiangrui Meng <[email protected]> Author: DB Tsai <[email protected]> Author: DB Tsai <[email protected]> Closes apache#4879 from dbtsai/breeze and squashes the following commits: d848f65 [DB Tsai] Merge pull request apache#1 from mengxr/AlpineNow-breeze c2ca6ac [Xiangrui Meng] upgrade to breeze-0.11.1 35c2f26 [Xiangrui Meng] fix LRSuite 397a208 [DB Tsai] upgrade breeze
1 parent d334bfb commit 76e20a0

File tree

2 files changed

+5
-1
lines changed

2 files changed

+5
-1
lines changed

mllib/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@
6363
<dependency>
6464
<groupId>org.scalanlp</groupId>
6565
<artifactId>breeze_${scala.binary.version}</artifactId>
66-
<version>0.10</version>
66+
<version>0.11.1</version>
6767
<exclusions>
6868
<!-- This is included as a compile-scoped dependency by jtransforms, which is
6969
a dependency of breeze. -->

mllib/src/test/scala/org/apache/spark/mllib/classification/LogisticRegressionSuite.scala

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -372,8 +372,12 @@ class LogisticRegressionSuite extends FunSuite with MLlibTestSparkContext with M
372372
testRDD2.cache()
373373
testRDD3.cache()
374374

375+
val numIteration = 10
376+
375377
val lrA = new LogisticRegressionWithLBFGS().setIntercept(true)
378+
lrA.optimizer.setNumIterations(numIteration)
376379
val lrB = new LogisticRegressionWithLBFGS().setIntercept(true).setFeatureScaling(false)
380+
lrB.optimizer.setNumIterations(numIteration)
377381

378382
val modelA1 = lrA.run(testRDD1, initialWeights)
379383
val modelA2 = lrA.run(testRDD2, initialWeights)

0 commit comments

Comments
 (0)