test_retry_non_txn slow with serializable&non-pushable

[tbg]

Adding to @bdarnell's comment in test_retry_non_txn:

# The write/write can't-push test cases take 15 seconds each.
# TODO: why? because the go version uses Store.SetRangeRetryOptions?
#(Methods.Put, data_pb2.SNAPSHOT, False, 1),
#(Methods.Put, data_pb2.SERIALIZABLE, False, 1),
# Read/write conflicts.

I have found that the test cases present still incur a 15sec wait for me, unless I comment the last one:

        test_cases = [
            # Write/write conflicts.
            (Methods.Put, data_pb2.SNAPSHOT, True, 2),
            (Methods.Put, data_pb2.SERIALIZABLE, True, 2),
            # The write/write can't-push test cases take 15 seconds each.
            # TODO: why? because the go version uses Store.SetRangeRetryOptions?
            #(Methods.Put, data_pb2.SNAPSHOT, False, 1),
            #(Methods.Put, data_pb2.SERIALIZABLE, False, 1),
            # Read/write conflicts.
            (Methods.Get, data_pb2.SNAPSHOT, True, 1),
            (Methods.Get, data_pb2.SERIALIZABLE, True, 2),
            (Methods.Get, data_pb2.SNAPSHOT, False, 1),
            #(Methods.Get, data_pb2.SERIALIZABLE, False, 1),
        ]

This test appears to be reliably slow (~19sec for all tests) with the last case, and fast (~4sec for all tests) without it.

[bdarnell]

Yeah, I saw this too. Not sure why I commented out two of the slow cases and left the third. I apparently have about 20 seconds worth of patience for a test run.

[spencerkimball]

Perhaps we can add something to the RequestHeader for retries...?

[bdarnell]

But that's not generally something we want to let the clients control, is it? I was thinking the answer would be to pass in extra flags when starting the docker container.

[spencerkimball]

Yeah, probably not. Having these settable via flags from container sounds OK.

[tbg]

probable root cause: cockroachdb/cockroach#877