Skip to content

[SPARK-56938][CONNECT][TESTS] Initialize base session in AddArtifactsHandlerSuite#55976

Open
sarutak wants to merge 1 commit into
apache:masterfrom
sarutak:fix-AddArtifactsHandlerSuite
Open

[SPARK-56938][CONNECT][TESTS] Initialize base session in AddArtifactsHandlerSuite#55976
sarutak wants to merge 1 commit into
apache:masterfrom
sarutak:fix-AddArtifactsHandlerSuite

Conversation

@sarutak
Copy link
Copy Markdown
Member

@sarutak sarutak commented May 19, 2026

What changes were proposed in this pull request?

This PR fixes an issue that all tests in AddArtifactsHandlerSuite fails if the suite runs individually.

$ build/sbt 'testOnly org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite'

...

[info] AddArtifactsHandlerSuite:
[info] - single chunk artifact *** FAILED *** (235 milliseconds)
[info]   io.grpc.StatusRuntimeException: INTERNAL: None.get
[info]   at io.grpc.Status.asRuntimeException(Status.java:532)
[info]   at io.grpc.protobuf.StatusProto.toStatusRuntimeException(StatusProto.java:52)
[info]   at org.apache.spark.sql.connect.utils.ErrorUtils$.org$apache$spark$sql$connect$utils$ErrorUtils$$processErrorCommon(ErrorUtils.scala:335)
[info]   at org.apache.spark.sql.connect.utils.ErrorUtils$$anonfun$handleError$1.applyOrElse(ErrorUtils.scala:407)
[info]   at org.apache.spark.sql.connect.utils.ErrorUtils$$anonfun$handleError$1.applyOrElse(ErrorUtils.scala:405)
[info]   at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35)
[info]   at org.apache.spark.sql.connect.service.SparkConnectAddArtifactsHandler.onNext(SparkConnectAddArtifactsHandler.scala:87)
[info]   at org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite.addSingleChunkArtifact(AddArtifactsHandlerSuite.scala:239)
[info]   at org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite.$anonfun$new$1(AddArtifactsHandlerSuite.scala:294)
[info]   at org.scalatest.enablers.Timed$$anon$1.timeoutAfter(Timed.scala:127)
[info]   at org.scalatest.concurrent.TimeLimits$.failAfterImpl(TimeLimits.scala:282)
[info]   at org.scalatest.concurrent.TimeLimits.failAfter(TimeLimits.scala:231)
[info]   at org.scalatest.concurrent.TimeLimits.failAfter$(TimeLimits.scala:230)
[info]   at org.apache.spark.SparkFunSuite.failAfter(SparkFunSuite.scala:30)
[info]   at org.apache.spark.SparkFunSuite.$anonfun$test$2(SparkFunSuite.scala:41)
[info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:226)
[info]   at org.apache.spark.SparkTestSuite.withFixture(SparkTestSuite.scala:175)
[info]   at org.apache.spark.SparkTestSuite.withFixture$(SparkTestSuite.scala:169)
[info]   at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:30)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:224)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:236)
[info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:236)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:218)
[info]   at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:30)
[info]   at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)
Warning: Unable to serialize throwable of type io.grpc.StatusRuntimeException for TestFailed(Ordinal(0, 3),INTERNAL: None.get,AddArtifactsHandlerSuite,org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite,Some(org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite),single chunk artifact,single chunk artifact,Vector(),Vector(),Some(io.grpc.StatusRuntimeException: INTERNAL: None.get),Some(235),Some(IndentedText(- single chunk artifact,single chunk artifact,0)),Some(SeeStackDepthException),Some(org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite),None,pool-1-thread-1-ScalaTest-running-AddArtifactsHandlerSuite,1779160574268), setting it as NotSerializableWrapperException.
[info]   at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)
[info]   at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:30)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
[info]   at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
[info]   at scala.collection.immutable.List.foreach(List.scala:323)
[info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info]   at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
[info]   at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
[info]   at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
[info]   at org.scalatest.Suite.run(Suite.scala:1114)
[info]   at org.scalatest.Suite.run$(Suite.scala:1096)
[info]   at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
[info]   at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
[info]   at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
[info]   at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:30)
[info]   at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
[info]   at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
[info]   at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
[info]   at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:30)
[info]   at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:321)
[info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:517)
[info]   at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:414)
[info]   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[info]   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[info]   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[info]   at java.base/java.lang.Thread.run(Thread.java:840)

...

The reason is this suite doesn't initialize base session before each test.

Why are the changes needed?

Bug fix.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Confirmed all tests in AddArtifactsHandlerSuite passed.

$ build/sbt 'testOnly org.apache.spark.sql.connect.service.AddArtifactsHandlerSuite'

Was this patch authored or co-authored using generative AI tooling?

No.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants