"./operations.test -test.v -test.timeout 1h0m0s -remote TestShade: -verbose" - Starting (try 1/5)
2026/07/29 01:53:55 DEBUG : Creating backend with remote "TestShade:rclone-test-duximoj0wami"
2026/07/29 01:53:55 DEBUG : Using config file from "/home/rclone/.rclone.conf"
2026/07/29 01:53:55 DEBUG : Creating new ShadeFS backend with drive: cf193d00-c783-4bab-aa7a-fd84b3700d27
2026/07/29 01:53:55 DEBUG : Creating backend with remote "/tmp/rclone1660533275"
=== RUN   TestDoMultiThreadCopy
--- PASS: TestDoMultiThreadCopy (0.00s)
=== RUN   TestMultithreadCalculateNumChunks
=== RUN   TestMultithreadCalculateNumChunks/{size:1_chunkSize:65536_wantNumChunks:1}
=== RUN   TestMultithreadCalculateNumChunks/{size:1048576_chunkSize:1_wantNumChunks:1048576}
=== RUN   TestMultithreadCalculateNumChunks/{size:1048576_chunkSize:2_wantNumChunks:524288}
=== RUN   TestMultithreadCalculateNumChunks/{size:1048577_chunkSize:2_wantNumChunks:524289}
=== RUN   TestMultithreadCalculateNumChunks/{size:1048575_chunkSize:2_wantNumChunks:524288}
--- PASS: TestMultithreadCalculateNumChunks (0.00s)
    --- PASS: TestMultithreadCalculateNumChunks/{size:1_chunkSize:65536_wantNumChunks:1} (0.00s)
    --- PASS: TestMultithreadCalculateNumChunks/{size:1048576_chunkSize:1_wantNumChunks:1048576} (0.00s)
    --- PASS: TestMultithreadCalculateNumChunks/{size:1048576_chunkSize:2_wantNumChunks:524288} (0.00s)
    --- PASS: TestMultithreadCalculateNumChunks/{size:1048577_chunkSize:2_wantNumChunks:524289} (0.00s)
    --- PASS: TestMultithreadCalculateNumChunks/{size:1048575_chunkSize:2_wantNumChunks:524288} (0.00s)
=== RUN   TestMultithreadCopy
    run.go:198: Remote "Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-duximoj0wami", Local "Local file system at /tmp/rclone1660533275", Modify Window "876000h0m0s"
=== RUN   TestMultithreadCopy/upload=false,size=134217727,streams=2
2026/07/29 01:53:58 DEBUG : test-multithread-copy-false-134217727-2: multipart upload: starting chunk 0 size 64Mi offset 0/128.000Mi
2026/07/29 01:53:58 DEBUG : test-multithread-copy-false-134217727-2: multipart upload: starting chunk 1 size 64.000Mi offset 64Mi/128.000Mi
2026/07/29 01:54:01 DEBUG : pacer: low level retry 1/10 (error EOF)
2026/07/29 01:54:01 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2026/07/29 01:54:01 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:01 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2026/07/29 01:54:02 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:02 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2026/07/29 01:54:02 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:02 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2026/07/29 01:54:02 DEBUG : pacer: low level retry 5/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:02 DEBUG : pacer: Rate limited, increasing sleep to 320ms
2026/07/29 01:54:03 DEBUG : pacer: low level retry 6/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:03 DEBUG : pacer: Rate limited, increasing sleep to 640ms
2026/07/29 01:54:03 DEBUG : pacer: low level retry 7/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:03 DEBUG : pacer: Rate limited, increasing sleep to 1.28s
2026/07/29 01:54:04 DEBUG : pacer: low level retry 8/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:04 DEBUG : pacer: Rate limited, increasing sleep to 2.56s
2026/07/29 01:54:05 DEBUG : pacer: low level retry 9/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:05 DEBUG : pacer: Rate limited, increasing sleep to 5.12s
2026/07/29 01:54:08 DEBUG : pacer: low level retry 10/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:08 DEBUG : pacer: Rate limited, increasing sleep to 10.24s
2026/07/29 01:54:08 DEBUG : test-multithread-copy-false-134217727-2: Cancelling multipart upload
2026/07/29 01:54:13 DEBUG : pacer: Reducing sleep to 5.12s
=== NAME  TestMultithreadCopy
    run.go:299: Retry Put of "test-multithread-copy-false-134217727-2" to Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-duximoj0wami: 1/10 (multipart upload: failed to finalise: failed to complete multipart upload: HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:23 DEBUG : pacer: Reducing sleep to 2.56s
2026/07/29 01:54:23 DEBUG : test-multithread-copy-false-134217727-2: multipart upload: starting chunk 0 size 64Mi offset 0/128.000Mi
2026/07/29 01:54:23 DEBUG : test-multithread-copy-false-134217727-2: multipart upload: starting chunk 1 size 64.000Mi offset 64Mi/128.000Mi
2026/07/29 01:54:28 DEBUG : pacer: Reducing sleep to 1.28s
2026/07/29 01:54:30 DEBUG : pacer: Reducing sleep to 640ms
2026/07/29 01:54:33 DEBUG : pacer: Reducing sleep to 320ms
2026/07/29 01:54:35 DEBUG : pacer: Reducing sleep to 160ms
2026/07/29 01:54:36 DEBUG : pacer: low level retry 1/10 (error EOF)
2026/07/29 01:54:36 DEBUG : pacer: Rate limited, increasing sleep to 320ms
2026/07/29 01:54:36 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:36 DEBUG : pacer: Rate limited, increasing sleep to 640ms
2026/07/29 01:54:37 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:37 DEBUG : pacer: Rate limited, increasing sleep to 1.28s
2026/07/29 01:54:37 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:37 DEBUG : pacer: Rate limited, increasing sleep to 2.56s
2026/07/29 01:54:38 DEBUG : pacer: low level retry 5/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:38 DEBUG : pacer: Rate limited, increasing sleep to 5.12s
2026/07/29 01:54:41 DEBUG : pacer: low level retry 6/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:41 DEBUG : pacer: Rate limited, increasing sleep to 10.24s
2026/07/29 01:54:46 DEBUG : pacer: low level retry 7/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:46 DEBUG : pacer: Rate limited, increasing sleep to 20.48s
2026/07/29 01:54:56 DEBUG : pacer: low level retry 8/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:54:56 DEBUG : pacer: Rate limited, increasing sleep to 40.96s
2026/07/29 01:55:17 DEBUG : pacer: low level retry 9/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:55:17 DEBUG : pacer: Rate limited, increasing sleep to 1m21.92s
2026/07/29 01:55:58 DEBUG : pacer: low level retry 10/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 01:55:58 DEBUG : pacer: Rate limited, increasing sleep to 2m43.84s
2026/07/29 01:55:58 DEBUG : test-multithread-copy-false-134217727-2: Cancelling multipart upload
2026/07/29 01:57:20 DEBUG : pacer: Reducing sleep to 1m21.92s
    run.go:299: Retry Put of "test-multithread-copy-false-134217727-2" to Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-duximoj0wami: 2/10 (multipart upload: failed to finalise: failed to complete multipart upload: HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:00:04 DEBUG : pacer: Reducing sleep to 40.96s
2026/07/29 02:00:04 DEBUG : test-multithread-copy-false-134217727-2: multipart upload: starting chunk 0 size 64Mi offset 0/128.000Mi
2026/07/29 02:00:04 DEBUG : test-multithread-copy-false-134217727-2: multipart upload: starting chunk 1 size 64.000Mi offset 64Mi/128.000Mi
2026/07/29 02:01:25 DEBUG : pacer: Reducing sleep to 20.48s
2026/07/29 02:02:06 DEBUG : pacer: Reducing sleep to 10.24s
2026/07/29 02:02:29 DEBUG : pacer: Reducing sleep to 5.12s
2026/07/29 02:02:41 DEBUG : pacer: Reducing sleep to 2.56s
2026/07/29 02:02:43 DEBUG : pacer: low level retry 1/10 (error EOF)
2026/07/29 02:02:43 DEBUG : pacer: Rate limited, increasing sleep to 5.12s
2026/07/29 02:02:45 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:02:45 DEBUG : pacer: Rate limited, increasing sleep to 10.24s
2026/07/29 02:02:50 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:02:50 DEBUG : pacer: Rate limited, increasing sleep to 20.48s
2026/07/29 02:03:00 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:03:00 DEBUG : pacer: Rate limited, increasing sleep to 40.96s
2026/07/29 02:03:21 DEBUG : pacer: low level retry 5/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:03:21 DEBUG : pacer: Rate limited, increasing sleep to 1m21.92s
2026/07/29 02:04:02 DEBUG : pacer: low level retry 6/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:04:02 DEBUG : pacer: Rate limited, increasing sleep to 2m43.84s
2026/07/29 02:05:24 DEBUG : pacer: low level retry 7/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:05:24 DEBUG : pacer: Rate limited, increasing sleep to 5m0s
2026/07/29 02:08:08 DEBUG : pacer: low level retry 8/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:13:08 DEBUG : pacer: low level retry 9/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:18:08 DEBUG : pacer: low level retry 10/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:18:08 DEBUG : test-multithread-copy-false-134217727-2: Cancelling multipart upload
2026/07/29 02:23:08 DEBUG : pacer: Reducing sleep to 2m30s
    run.go:299: Retry Put of "test-multithread-copy-false-134217727-2" to Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-duximoj0wami: 3/10 (multipart upload: failed to finalise: failed to complete multipart upload: HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:28:08 DEBUG : pacer: Reducing sleep to 1m15s
2026/07/29 02:28:08 DEBUG : test-multithread-copy-false-134217727-2: multipart upload: starting chunk 0 size 64Mi offset 0/128.000Mi
2026/07/29 02:28:08 DEBUG : test-multithread-copy-false-134217727-2: multipart upload: starting chunk 1 size 64.000Mi offset 64Mi/128.000Mi
2026/07/29 02:30:37 DEBUG : pacer: Reducing sleep to 37.5s
2026/07/29 02:31:53 DEBUG : pacer: Reducing sleep to 18.75s
2026/07/29 02:32:33 DEBUG : pacer: Reducing sleep to 9.375s
2026/07/29 02:32:53 DEBUG : pacer: Reducing sleep to 4.6875s
2026/07/29 02:32:59 DEBUG : pacer: low level retry 1/10 (error EOF)
2026/07/29 02:32:59 DEBUG : pacer: Rate limited, increasing sleep to 9.375s
2026/07/29 02:33:03 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:33:03 DEBUG : pacer: Rate limited, increasing sleep to 18.75s
2026/07/29 02:33:12 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:33:12 DEBUG : pacer: Rate limited, increasing sleep to 37.5s
2026/07/29 02:33:31 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:33:31 DEBUG : pacer: Rate limited, increasing sleep to 1m15s
2026/07/29 02:34:09 DEBUG : pacer: low level retry 5/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:34:09 DEBUG : pacer: Rate limited, increasing sleep to 2m30s
2026/07/29 02:35:24 DEBUG : pacer: low level retry 6/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:35:24 DEBUG : pacer: Rate limited, increasing sleep to 5m0s
2026/07/29 02:37:54 DEBUG : pacer: low level retry 7/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:42:54 DEBUG : pacer: Reducing sleep to 2m30s
2026/07/29 02:42:54 DEBUG : test-multithread-copy-false-134217727-2: Cancelling multipart upload
2026/07/29 02:47:54 DEBUG : pacer: Reducing sleep to 1m15s
2026/07/29 02:47:54 DEBUG : Saving config "token" in section "TestShade" of the config file
2026/07/29 02:47:54 DEBUG : Config file has changed externally - reloading
2026/07/29 02:47:54 DEBUG : Saving config "token_expiry" in section "TestShade" of the config file
2026/07/29 02:50:24 DEBUG : pacer: Reducing sleep to 37.5s
    run.go:303: Failed to put "test-multithread-copy-false-134217727-2" to "Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-duximoj0wami": multipart upload: failed to finalise: failed to complete multipart upload: HTTP error 400 (400 Bad Request) returned body: "Invalid token"
=== NAME  TestMultithreadCopy/upload=false,size=134217727,streams=2
    testing.go:1913: test executed panic(nil) or runtime.Goexit: subtest may have called FailNow on a parent test
2026/07/29 02:51:39 DEBUG : pacer: Reducing sleep to 18.75s
2026/07/29 02:52:16 DEBUG : pacer: Reducing sleep to 9.375s
2026/07/29 02:52:35 DEBUG : pacer: Reducing sleep to 4.6875s
--- FAIL: TestMultithreadCopy (3520.05s)
    --- FAIL: TestMultithreadCopy/upload=false,size=134217727,streams=2 (3388.08s)
=== RUN   TestMultithreadCopyAbort
    run.go:198: Remote "Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-duximoj0wami", Local "Local file system at /tmp/rclone1660533275", Modify Window "876000h0m0s"
2026/07/29 02:52:44 DEBUG : pacer: Reducing sleep to 2.34375s
2026/07/29 02:52:49 DEBUG : pacer: Reducing sleep to 1.171875s
2026/07/29 02:52:51 DEBUG : pacer: Reducing sleep to 585.9375ms
2026/07/29 02:52:51 DEBUG : test-multithread-abort: multipart upload: starting chunk 0 size 100 offset 0/100
2026/07/29 02:52:52 DEBUG : pacer: Reducing sleep to 292.96875ms
2026/07/29 02:52:53 DEBUG : pacer: Reducing sleep to 146.484375ms
2026/07/29 02:52:54 DEBUG : pacer: low level retry 1/10 (error EOF)
2026/07/29 02:52:54 DEBUG : pacer: Rate limited, increasing sleep to 292.96875ms
2026/07/29 02:52:55 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:52:55 DEBUG : pacer: Rate limited, increasing sleep to 585.9375ms
2026/07/29 02:52:55 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:52:55 DEBUG : pacer: Rate limited, increasing sleep to 1.171875s
2026/07/29 02:52:55 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:52:55 DEBUG : pacer: Rate limited, increasing sleep to 2.34375s
2026/07/29 02:52:57 DEBUG : pacer: low level retry 5/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:52:57 DEBUG : pacer: Rate limited, increasing sleep to 4.6875s
2026/07/29 02:52:59 DEBUG : pacer: low level retry 6/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:52:59 DEBUG : pacer: Rate limited, increasing sleep to 9.375s
2026/07/29 02:53:04 DEBUG : pacer: low level retry 7/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:53:04 DEBUG : pacer: Rate limited, increasing sleep to 18.75s
2026/07/29 02:53:13 DEBUG : pacer: low level retry 8/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:53:13 DEBUG : pacer: Rate limited, increasing sleep to 37.5s
2026/07/29 02:53:32 DEBUG : pacer: low level retry 9/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error")
2026/07/29 02:53:32 DEBUG : pacer: Rate limited, increasing sleep to 1m15s
panic: test timed out after 1h0m0s
	running tests:
		TestMultithreadCopyAbort (1m20s)

goroutine 430 [running]:
testing.(*M).startAlarm.func1()
	/usr/local/go/src/testing/testing.go:2802 +0x34b
created by time.goFunc
	/usr/local/go/src/time/sleep.go:215 +0x2d

goroutine 1 [chan receive, 2 minutes]:
testing.(*T).Run(0x710a4991208, {0x2b5f5eb?, 0x710a4821ae8?}, 0x2cbce68)
	/usr/local/go/src/testing/testing.go:2109 +0x4e5
testing.runTests.func1(0x710a4991208)
	/usr/local/go/src/testing/testing.go:2585 +0x37
testing.tRunner(0x710a4991208, 0x710a4821c10)
	/usr/local/go/src/testing/testing.go:2036 +0xea
testing.runTests({0x2b5f603, 0x18}, {0x2b9b45f, 0x26}, 0x710a52dc5b8, {0x47b74c0, 0x72, 0x72}, {0xc29278b0d0dc1547, 0x3463ee60776, ...})
	/usr/local/go/src/testing/testing.go:2583 +0x505
testing.(*M).Run(0x710a50eb680)
	/usr/local/go/src/testing/testing.go:2443 +0x6ac
github.com/rclone/rclone/fstest.TestMain(0x710a50eb680)
	/home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:74 +0xa6
github.com/rclone/rclone/fs/operations_test.TestMain(...)
	/home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:61
main.main()
	_testmain.go:276 +0xa6

goroutine 22 [chan receive, 60 minutes]:
github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1()
	/home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27
created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 1
	/home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xa7

goroutine 21 [syscall, 60 minutes]:
os/signal.signal_recv()
	/usr/local/go/src/runtime/sigqueue.go:152 +0x98
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:23 +0x13
created by os/signal.Notify.func1.1 in goroutine 1
	/usr/local/go/src/os/signal/signal.go:152 +0x1f

goroutine 36 [chan receive, 59 minutes]:
github.com/rclone/rclone/lib/atexit.Register.func1.1()
	/home/rclone/go/src/github.com/rclone/rclone/lib/atexit/atexit.go:45 +0x29
created by github.com/rclone/rclone/lib/atexit.Register.func1 in goroutine 30
	/home/rclone/go/src/github.com/rclone/rclone/lib/atexit/atexit.go:44 +0x68

goroutine 411 [chan receive]:
github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall(0x710a4da4240, 0x0)
	/home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:168 +0xa7
github.com/rclone/rclone/lib/pacer.(*Pacer).call(0x710a4da4240, 0x710a48d0190, 0xa)
	/home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:227 +0xb2
github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0x710a4da4240, 0x710a48d0190)
	/home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:247 +0x92
github.com/rclone/rclone/backend/shade.(*shadeChunkWriter).Close(0x710a4e8e0a0, {0x2d051f8, 0x47f4a20})
	/home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:269 +0x42a
github.com/rclone/rclone/lib/multipart.UploadMultipart({0x2d051f8, 0x47f4a20}, {0x2d15a90, 0x710a4da0d20}, {0x2cd84a0, 0x710a51d28a0}, {{0x2cdbb40, 0x710a4c38140}, {0x0, 0x0, ...}})
	/home/rclone/go/src/github.com/rclone/rclone/lib/multipart/multipart.go:124 +0xadf
github.com/rclone/rclone/backend/shade.(*Object).uploadMultipart(0x710a52c2200, {0x2d051f8?, 0x47f4a20?}, {0x2d15a90?, 0x710a4da0d20?}, {0x2cd84a0?, 0x710a51d28a0?}, {0x0, 0x0, 0x0})
	/home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:40 +0xd3
github.com/rclone/rclone/backend/shade.(*Object).Update(0x710a52c2200, {0x2d051f8, 0x47f4a20}, {0x2cd84a0, 0x710a51d28a0}, {0x2d15a90, 0x710a4da0d20}, {0x0, 0x0, 0x0})
	/home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:971 +0x1ac
github.com/rclone/rclone/backend/shade.(*Fs).Put(0x710a4c38140, {0x2d051f8, 0x47f4a20}, {0x2cd84a0, 0x710a51d28a0}, {0x2d15a90, 0x710a4da0d20}, {0x0, 0x0, 0x0})
	/home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:555 +0x108
github.com/rclone/rclone/fstest.(*Run).WriteObjectTo(0x710a4bf8b60, {0x2d051f8, 0x47f4a20}, {0x2d24010, 0x710a4c38140}, {0x2b5772b, 0x16}, {0x710a489a8c0, 0x64}, {0x1dcd64ff, ...}, ...)
	/home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:293 +0x3e8
github.com/rclone/rclone/fstest.(*Run).WriteObject(...)
	/home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:310
github.com/rclone/rclone/fs/operations.TestMultithreadCopyAbort(0x710a52ef208)
	/home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:312 +0x1b3
testing.tRunner(0x710a52ef208, 0x2cbce68)
	/usr/local/go/src/testing/testing.go:2036 +0xea
created by testing.(*T).Run in goroutine 1
	/usr/local/go/src/testing/testing.go:2101 +0x4c5

goroutine 470 [sleep]:
time.Sleep(0x8bb2c9700)
	/usr/local/go/src/runtime/time.go:363 +0x165
github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall.func1(0x710a4e38798?)
	/home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:178 +0x1c
created by github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall in goroutine 411
	/home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:177 +0x17f

goroutine 376 [IO wait]:
internal/poll.runtime_pollWait(0x7ae24d631200, 0x72)
	/usr/local/go/src/runtime/netpoll.go:351 +0x85
internal/poll.(*pollDesc).wait(0x710a4dfc080?, 0x710a4d3b500?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x710a4dfc080, {0x710a4d3b500, 0x1500, 0x1500})
	/usr/local/go/src/internal/poll/fd_unix.go:165 +0x2ae
net.(*netFD).Read(0x710a4dfc080, {0x710a4d3b500?, 0x710a48508f0?, 0x4da937?})
	/usr/local/go/src/net/fd_posix.go:68 +0x25
net.(*conn).Read(0x710a4d10428, {0x710a4d3b500?, 0x0?, 0x0?})
	/usr/local/go/src/net/net.go:196 +0x45
github.com/rclone/rclone/fs/fshttp.(*timeoutConn).Read(0x710a47c82e8, {0x710a4d3b500?, 0x7ae1effc1ae8?, 0x7ae24d6b7bf0?})
	/home/rclone/go/src/github.com/rclone/rclone/fs/fshttp/dialer.go:111 +0x29
crypto/tls.(*atLeastReader).Read(0x710a4bd9470, {0x710a4d3b500?, 0x710a4f2e000?, 0x710a4850a48?})
	/usr/local/go/src/crypto/tls/conn.go:815 +0x3b
bytes.(*Buffer).ReadFrom(0x710a4f149a8, {0x2cda1c0, 0x710a4bd9470})
	/usr/local/go/src/bytes/buffer.go:229 +0x98
crypto/tls.(*Conn).readFromUntil(0x710a4f14708, {0x7ae1eee54e80, 0x710a47c82e8}, 0x710a4850c08?)
	/usr/local/go/src/crypto/tls/conn.go:837 +0xde
crypto/tls.(*Conn).readRecordOrCCS(0x710a4f14708, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:626 +0x3db
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:588
crypto/tls.(*Conn).Read(0x710a4f14708, {0x710a4db6000, 0x1000, 0x710a4850ce0?})
	/usr/local/go/src/crypto/tls/conn.go:1393 +0x145
bufio.(*Reader).Read(0x710a4da4420, {0x710a5096124, 0x9, 0x710a4850d20?})
	/usr/local/go/src/bufio/bufio.go:245 +0x197
io.ReadAtLeast({0x2cd83c0, 0x710a4da4420}, {0x710a5096124, 0x9, 0x9}, 0x9)
	/usr/local/go/src/io/io.go:335 +0x8e
io.ReadFull(...)
	/usr/local/go/src/io/io.go:354
net/http.http2readFrameHeader({0x710a5096124, 0x9, 0x71000000000?}, {0x2cd83c0?, 0x710a4da4420?})
	/usr/local/go/src/net/http/h2_bundle.go:1805 +0x65
net/http.(*http2Framer).ReadFrameHeader(0x710a50960e0)
	/usr/local/go/src/net/http/h2_bundle.go:2071 +0x6b
net/http.(*http2Framer).ReadFrame(0x710a50960e0)
	/usr/local/go/src/net/http/h2_bundle.go:2130 +0x18
net/http.(*http2clientConnReadLoop).run(0x710a4850fa8)
	/usr/local/go/src/net/http/h2_bundle.go:9550 +0xca
net/http.(*http2ClientConn).readLoop(0x710a4c90380)
	/usr/local/go/src/net/http/h2_bundle.go:9419 +0x52
created by net/http.(*http2Transport).newClientConn in goroutine 375
	/usr/local/go/src/net/http/h2_bundle.go:8171 +0xda5
"./operations.test -test.v -test.timeout 1h0m0s -remote TestShade: -verbose" - Finished ERROR in 1h0m0.258884887s (try 1/5): exit status 2: Failed [TestMultithreadCopy/upload=false,size=134217727,streams=2]