"./bisync.test -test.v -test.timeout 1h0m0s -remote TestShade: -verbose" - Starting (try 1/5) 2026/07/25 04:14:53 DEBUG : Creating backend with remote "TestShade:rclone-test-cucayuw5tiwi" 2026/07/25 04:14:53 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2026/07/25 04:14:53 DEBUG : Creating new ShadeFS backend with drive: cf193d00-c783-4bab-aa7a-fd84b3700d27 2026/07/25 04:14:53 DEBUG : Creating backend with remote "/tmp/rclone807032989" === RUN TestLockfileIsExpired_UnreadableWithMaxLock 2026/07/25 04:14:53 INFO : /tmp/TestLockfileIsExpired_UnreadableWithMaxLock1665287426/001/test.lck: Lock file is unreadable (decode error: invalid character 'o' in literal null (expecting 'u')) and --max-lock is set. Treating as expired. --- PASS: TestLockfileIsExpired_UnreadableWithMaxLock (0.00s) === RUN TestLockfileIsExpired_UnreadableWithoutMaxLock 2026/07/25 04:14:53 ERROR : /tmp/TestLockfileIsExpired_UnreadableWithoutMaxLock3184751137/001/test.lck: Lock file exists, but contents are unreadable. (decode error: invalid character 'o' in literal null (expecting 'u')) --- PASS: TestLockfileIsExpired_UnreadableWithoutMaxLock (0.00s) === RUN TestLockfileIsExpired_ValidExpired 2026/07/25 04:14:53 INFO : /tmp/TestLockfileIsExpired_ValidExpired3035109211/001/test.lck: Lock file found, but it expired at 2026-07-25 04:09:53.439367118 +0000 UTC. Will delete it and proceed. --- PASS: TestLockfileIsExpired_ValidExpired (0.00s) === RUN TestLockfileIsExpired_ValidNotExpired 2026/07/25 04:14:53 INFO : /tmp/TestLockfileIsExpired_ValidNotExpired3185169617/001/test.lck: Valid lock file found. Expires at 2026-07-25 04:24:53.443121891 +0000 UTC. (10m0s from now) 2026/07/25 04:14:53 INFO : Lockfile info: { "Session": "test", "PID": "12345", "TimeRenewed": "2026-07-25T04:14:53.443121811Z", "TimeExpires": "2026-07-25T04:24:53.443121891Z" } --- PASS: TestLockfileIsExpired_ValidNotExpired (0.00s) === RUN TestBisyncRemoteLocal 2026/07/25 04:14:53 DEBUG : Creating backend with remote "TestShade:rclone-test-sazenaz1jafi" 2026/07/25 04:14:53 DEBUG : Creating new ShadeFS backend with drive: cf193d00-c783-4bab-aa7a-fd84b3700d27 2026/07/25 04:14:53 NOTICE: remote: TestShade:rclone-test-sazenaz1jafi === RUN TestBisyncRemoteLocal/all_changed 2026/07/25 04:14:53 DEBUG : Creating backend with remote "TestShade:rclone-test-sazenaz1jafi/041453noxepif9" 2026/07/25 04:14:53 DEBUG : Creating new ShadeFS backend with drive: cf193d00-c783-4bab-aa7a-fd84b3700d27 2026/07/25 04:14:53 INFO : path1: Making directory 2026/07/25 04:14:54 DEBUG : Creating backend with remote "TestShade:rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1" 2026/07/25 04:14:54 DEBUG : Creating new ShadeFS backend with drive: cf193d00-c783-4bab-aa7a-fd84b3700d27 2026/07/25 04:14:54 DEBUG : Creating backend with remote "/tmp/041453noxepif9" 2026/07/25 04:14:54 DEBUG : Creating backend with remote "/tmp/041453noxepif9/all_changed/path2" 2026/07/25 04:14:54 DEBUG : Creating backend with remote "/home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/testdata/test_all_changed/initial" 2026/07/25 04:14:54 DEBUG : Creating backend with remote "/tmp/041453noxepif9/initdir/test_all_changed-xefexep7" 2026/07/25 04:14:54 DEBUG : RCLONE_TEST: Need to transfer - File not found at Destination 2026/07/25 04:14:54 DEBUG : file1.copy1.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:54 DEBUG : file1.copy2.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:54 DEBUG : file1.copy3.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:54 DEBUG : file1.copy4.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:54 DEBUG : file1.copy5.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:54 DEBUG : file1.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:54 DEBUG : subdir: Making directory with metadata 2026/07/25 04:14:54 INFO : subdir: Made directory with metadata (mtime=2021-11-02T05:00:07Z) 2026/07/25 04:14:54 DEBUG : Added delayed dir = "subdir", newDst=subdir 2026/07/25 04:14:54 DEBUG : file1.copy1.txt.21b8fa2b.partial: size = 0 OK 2026/07/25 04:14:54 DEBUG : subdir/file20.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:54 DEBUG : RCLONE_TEST.abb09051.partial: size = 109 OK 2026/07/25 04:14:54 DEBUG : file1.copy2.txt.455881d5.partial: size = 0 OK 2026/07/25 04:14:54 DEBUG : Local file system at /tmp/041453noxepif9/initdir/test_all_changed-xefexep7: Waiting for checks to finish 2026/07/25 04:14:54 DEBUG : file1.copy1.txt: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2026/07/25 04:14:54 DEBUG : RCLONE_TEST: md5 = 294d25b294ff26a5243dba914ac3fbf7 OK 2026/07/25 04:14:54 DEBUG : Local file system at /tmp/041453noxepif9/initdir/test_all_changed-xefexep7: Waiting for transfers to finish 2026/07/25 04:14:54 DEBUG : file1.copy2.txt: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2026/07/25 04:14:54 DEBUG : file1.copy1.txt.21b8fa2b.partial: renamed to: file1.copy1.txt 2026/07/25 04:14:54 INFO : file1.copy1.txt: Copied (new) 2026/07/25 04:14:54 DEBUG : file1.copy3.txt.d0285540.partial: size = 0 OK 2026/07/25 04:14:54 DEBUG : RCLONE_TEST.abb09051.partial: renamed to: RCLONE_TEST 2026/07/25 04:14:54 DEBUG : file1.copy3.txt: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2026/07/25 04:14:54 INFO : RCLONE_TEST: Copied (new) 2026/07/25 04:14:54 DEBUG : file1.copy4.txt.8c987629.partial: size = 0 OK 2026/07/25 04:14:54 DEBUG : file1.copy2.txt.455881d5.partial: renamed to: file1.copy2.txt 2026/07/25 04:14:54 INFO : file1.copy2.txt: Copied (new) 2026/07/25 04:14:54 DEBUG : file1.copy4.txt: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2026/07/25 04:14:54 DEBUG : file1.copy5.txt.19e8a2bc.partial: size = 0 OK 2026/07/25 04:14:54 DEBUG : file1.copy3.txt.d0285540.partial: renamed to: file1.copy3.txt 2026/07/25 04:14:54 INFO : file1.copy3.txt: Copied (new) 2026/07/25 04:14:54 DEBUG : file1.copy5.txt: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2026/07/25 04:14:54 DEBUG : file1.copy4.txt.8c987629.partial: renamed to: file1.copy4.txt 2026/07/25 04:14:54 INFO : file1.copy4.txt: Copied (new) 2026/07/25 04:14:54 DEBUG : file1.txt.56d0d699.partial: size = 0 OK 2026/07/25 04:14:54 DEBUG : subdir/file20.txt.1ac3db2f.partial: size = 0 OK 2026/07/25 04:14:54 DEBUG : file1.copy5.txt.19e8a2bc.partial: renamed to: file1.copy5.txt 2026/07/25 04:14:54 INFO : file1.copy5.txt: Copied (new) 2026/07/25 04:14:54 DEBUG : file1.txt: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2026/07/25 04:14:54 DEBUG : subdir/file20.txt: md5 = d41d8cd98f00b204e9800998ecf8427e OK 2026/07/25 04:14:54 DEBUG : file1.txt.56d0d699.partial: renamed to: file1.txt 2026/07/25 04:14:54 DEBUG : subdir/file20.txt.1ac3db2f.partial: renamed to: subdir/file20.txt 2026/07/25 04:14:54 INFO : file1.txt: Copied (new) 2026/07/25 04:14:54 INFO : subdir/file20.txt: Copied (new) 2026/07/25 04:14:54 INFO : subdir: Set directory modification time (using SetModTime) 2026/07/25 04:14:54 DEBUG : Waiting for deletions to finish 2026/07/25 04:14:54 DEBUG : Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1: Error from List call: HTTP error 404 (404 Not Found) returned body: "can't derive ino for path" 2026/07/25 04:14:54 ERROR : error listing: directory not found 2026/07/25 04:14:54 DEBUG : Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1: Error from List call: HTTP error 404 (404 Not Found) returned body: "can't derive ino for path" 2026/07/25 04:14:54 ERROR : Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1: Failed to list "": directory not found 2026/07/25 04:14:54 DEBUG : removing 1 level 0 directories 2026/07/25 04:14:54 INFO : Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1: Removing directory 2026/07/25 04:14:54 ERROR : Failed to rmdir: HTTP error 404 (404 Not Found) returned body: "can't derive ino for path" 2026/07/25 04:14:54 DEBUG : Waiting for deletions to finish 2026/07/25 04:14:54 ERROR : error listing: directory not found 2026/07/25 04:14:54 ERROR : Local file system at /tmp/041453noxepif9/all_changed/path2: Failed to list "": directory not found 2026/07/25 04:14:54 DEBUG : removing 1 level 0 directories 2026/07/25 04:14:54 INFO : Local file system at /tmp/041453noxepif9/all_changed/path2: Removing directory 2026/07/25 04:14:54 ERROR : Failed to rmdir: stat /tmp/041453noxepif9/all_changed/path2: no such file or directory 2026/07/25 04:14:55 DEBUG : Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1: Error from List call: HTTP error 404 (404 Not Found) returned body: "can't derive ino for path" 2026/07/25 04:14:55 ERROR : error listing: directory not found 2026/07/25 04:14:55 ERROR : error listing: directory not found 2026/07/25 04:14:55 NOTICE: checking initFs Local file system at /tmp/041453noxepif9/initdir/test_all_changed-xefexep7 2026/07/25 04:14:55 DEBUG : Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1: Error from List call: HTTP error 404 (404 Not Found) returned body: "can't derive ino for path" 2026/07/25 04:14:55 DEBUG : RCLONE_TEST: Need to transfer - File not found at Destination 2026/07/25 04:14:55 DEBUG : file1.copy1.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:55 DEBUG : file1.copy2.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:55 DEBUG : file1.copy3.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:55 DEBUG : file1.copy4.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:55 DEBUG : file1.copy5.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:55 DEBUG : file1.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:55 INFO : subdir: Making directory 2026/07/25 04:14:56 DEBUG : subdir/file20.txt: Need to transfer - File not found at Destination 2026/07/25 04:14:56 DEBUG : Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1: Waiting for checks to finish 2026/07/25 04:14:56 DEBUG : Shade drive cf193d00-c783-4bab-aa7a-fd84b3700d27 path rclone-test-sazenaz1jafi/041453noxepif9/all_changed/path1: Waiting for transfers to finish 2026/07/25 04:14:56 DEBUG : file1.copy1.txt: multipart upload: starting chunk 0 size 0 offset 0/0 2026/07/25 04:14:56 DEBUG : file1.copy3.txt: multipart upload: starting chunk 0 size 0 offset 0/0 2026/07/25 04:14:56 DEBUG : RCLONE_TEST: multipart upload: starting chunk 0 size 109 offset 0/109 2026/07/25 04:14:56 DEBUG : file1.copy2.txt: multipart upload: starting chunk 0 size 0 offset 0/0 2026/07/25 04:14:56 DEBUG : pacer: low level retry 1/10 (error EOF) 2026/07/25 04:14:56 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2026/07/25 04:14:56 DEBUG : pacer: low level retry 1/10 (error EOF) 2026/07/25 04:14:56 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2026/07/25 04:14:56 DEBUG : pacer: low level retry 1/10 (error EOF) 2026/07/25 04:14:56 DEBUG : pacer: Rate limited, increasing sleep to 80ms 2026/07/25 04:14:56 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:14:56 DEBUG : pacer: Rate limited, increasing sleep to 160ms 2026/07/25 04:14:56 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:14:56 DEBUG : pacer: Rate limited, increasing sleep to 320ms 2026/07/25 04:14:56 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:14:56 DEBUG : pacer: Rate limited, increasing sleep to 640ms 2026/07/25 04:14:57 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:14:57 DEBUG : pacer: Rate limited, increasing sleep to 1.28s 2026/07/25 04:14:57 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:14:57 DEBUG : pacer: Rate limited, increasing sleep to 2.56s 2026/07/25 04:14:57 DEBUG : pacer: low level retry 1/10 (error EOF) 2026/07/25 04:14:57 DEBUG : pacer: Rate limited, increasing sleep to 5.12s 2026/07/25 04:14:58 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:14:58 DEBUG : pacer: Rate limited, increasing sleep to 10.24s 2026/07/25 04:15:03 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:15:03 DEBUG : pacer: Rate limited, increasing sleep to 20.48s 2026/07/25 04:15:13 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:15:13 DEBUG : pacer: Rate limited, increasing sleep to 40.96s 2026/07/25 04:15:33 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:15:33 DEBUG : pacer: Rate limited, increasing sleep to 1m21.92s 2026/07/25 04:16:14 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:16:14 DEBUG : pacer: Rate limited, increasing sleep to 2m43.84s 2026/07/25 04:17:36 DEBUG : pacer: low level retry 5/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:17:36 DEBUG : pacer: Rate limited, increasing sleep to 5m0s 2026/07/25 04:20:20 DEBUG : pacer: low level retry 5/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:25:20 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:30:20 DEBUG : pacer: low level retry 5/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:35:20 DEBUG : pacer: low level retry 6/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:40:20 DEBUG : pacer: low level retry 6/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:45:20 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 04:50:20 DEBUG : pacer: Reducing sleep to 2m30s 2026/07/25 04:50:20 DEBUG : file1.copy2.txt: Cancelling multipart upload 2026/07/25 04:55:20 DEBUG : pacer: Reducing sleep to 1m15s 2026/07/25 04:55:20 DEBUG : file1.copy1.txt: Cancelling multipart upload 2026/07/25 04:57:50 DEBUG : pacer: Reducing sleep to 37.5s 2026/07/25 04:57:50 DEBUG : file1.copy3.txt: Cancelling multipart upload 2026/07/25 04:59:05 DEBUG : pacer: Reducing sleep to 18.75s 2026/07/25 04:59:05 DEBUG : RCLONE_TEST: Cancelling multipart upload 2026/07/25 04:59:43 DEBUG : pacer: Reducing sleep to 9.375s 2026/07/25 04:59:43 DEBUG : Saving config "token" in section "TestShade" of the config file 2026/07/25 04:59:43 DEBUG : Config file has changed externally - reloading 2026/07/25 04:59:43 DEBUG : Saving config "token_expiry" in section "TestShade" of the config file 2026/07/25 05:00:01 DEBUG : pacer: Reducing sleep to 4.6875s 2026/07/25 05:00:01 ERROR : file1.copy2.txt: Failed to copy: multipart upload: failed to finalise: failed to complete multipart upload: HTTP error 400 (400 Bad Request) returned body: "Invalid token" 2026/07/25 05:00:11 DEBUG : pacer: Reducing sleep to 2.34375s 2026/07/25 05:00:11 ERROR : file1.copy1.txt: Failed to copy: multipart upload: failed to finalise: failed to complete multipart upload: HTTP error 400 (400 Bad Request) returned body: "Invalid token" 2026/07/25 05:00:15 DEBUG : pacer: Reducing sleep to 1.171875s 2026/07/25 05:00:15 ERROR : file1.copy3.txt: Failed to copy: multipart upload: failed to finalise: failed to complete multipart upload: HTTP error 400 (400 Bad Request) returned body: "Invalid token" 2026/07/25 05:00:18 DEBUG : pacer: Reducing sleep to 585.9375ms 2026/07/25 05:00:18 ERROR : RCLONE_TEST: Failed to copy: multipart upload: failed to finalise: failed to complete multipart upload: HTTP error 400 (400 Bad Request) returned body: "Invalid token" 2026/07/25 05:00:19 DEBUG : pacer: Reducing sleep to 292.96875ms 2026/07/25 05:00:19 DEBUG : file1.copy4.txt: multipart upload: starting chunk 0 size 0 offset 0/0 2026/07/25 05:00:20 DEBUG : pacer: Reducing sleep to 146.484375ms 2026/07/25 05:00:20 DEBUG : file1.copy5.txt: multipart upload: starting chunk 0 size 0 offset 0/0 2026/07/25 05:00:20 DEBUG : pacer: Reducing sleep to 73.242187ms 2026/07/25 05:00:20 DEBUG : file1.txt: multipart upload: starting chunk 0 size 0 offset 0/0 2026/07/25 05:00:20 DEBUG : pacer: Reducing sleep to 36.621093ms 2026/07/25 05:00:20 DEBUG : subdir/file20.txt: multipart upload: starting chunk 0 size 0 offset 0/0 2026/07/25 05:00:20 DEBUG : pacer: low level retry 1/10 (error EOF) 2026/07/25 05:00:20 DEBUG : pacer: Rate limited, increasing sleep to 73.242186ms 2026/07/25 05:00:21 DEBUG : pacer: low level retry 1/10 (error EOF) 2026/07/25 05:00:21 DEBUG : pacer: Rate limited, increasing sleep to 146.484372ms 2026/07/25 05:00:21 DEBUG : pacer: low level retry 1/10 (error EOF) 2026/07/25 05:00:21 DEBUG : pacer: Rate limited, increasing sleep to 292.968744ms 2026/07/25 05:00:21 DEBUG : pacer: low level retry 1/10 (error EOF) 2026/07/25 05:00:21 DEBUG : pacer: Rate limited, increasing sleep to 585.937488ms 2026/07/25 05:00:21 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:00:21 DEBUG : pacer: Rate limited, increasing sleep to 1.171874976s 2026/07/25 05:00:21 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:00:21 DEBUG : pacer: Rate limited, increasing sleep to 2.343749952s 2026/07/25 05:00:21 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:00:21 DEBUG : pacer: Rate limited, increasing sleep to 4.687499904s 2026/07/25 05:00:22 DEBUG : pacer: low level retry 2/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:00:22 DEBUG : pacer: Rate limited, increasing sleep to 9.374999808s 2026/07/25 05:00:26 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:00:26 DEBUG : pacer: Rate limited, increasing sleep to 18.749999616s 2026/07/25 05:00:36 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:00:36 DEBUG : pacer: Rate limited, increasing sleep to 37.499999232s 2026/07/25 05:00:54 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:00:54 DEBUG : pacer: Rate limited, increasing sleep to 1m14.999998464s 2026/07/25 05:01:32 DEBUG : pacer: low level retry 3/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:01:32 DEBUG : pacer: Rate limited, increasing sleep to 2m29.999996928s 2026/07/25 05:02:47 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:02:47 DEBUG : pacer: Rate limited, increasing sleep to 4m59.999993856s 2026/07/25 05:05:17 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") 2026/07/25 05:05:17 DEBUG : pacer: Rate limited, increasing sleep to 5m0s 2026/07/25 05:10:17 DEBUG : pacer: low level retry 4/10 (error HTTP error 500 (500 Internal Server Error) returned body: "Internal Server Error") panic: test timed out after 1h0m0s running tests: TestBisyncRemoteLocal (1h0m0s) TestBisyncRemoteLocal/all_changed (1h0m0s) goroutine 538 [running]: testing.(*M).startAlarm.func1() /usr/local/go/src/testing/testing.go:2802 +0x34b created by time.goFunc /usr/local/go/src/time/sleep.go:215 +0x2d goroutine 1 [chan receive, 60 minutes]: testing.(*T).Run(0x1b2b1d64908, {0x2b8a419?, 0x1b2b1b456f0?}, 0x2d087a0) /usr/local/go/src/testing/testing.go:2109 +0x4e5 testing.runTests.func1(0x1b2b1d64908) /usr/local/go/src/testing/testing.go:2585 +0x37 testing.tRunner(0x1b2b1d64908, 0x1b2b1b45818) /usr/local/go/src/testing/testing.go:2036 +0xea testing.runTests({0x2b963b8, 0x18}, {0x2bc60df, 0x23}, 0x1b2b25af278, {0x4825280, 0x8, 0x8}, {0xc2912f7359f42d01, 0x34643d98c96, ...}) /usr/local/go/src/testing/testing.go:2583 +0x505 testing.(*M).Run(0x1b2b22ae960) /usr/local/go/src/testing/testing.go:2443 +0x6ac github.com/rclone/rclone/fstest.TestMain(0x1b2b22ae960) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:74 +0xa6 github.com/rclone/rclone/cmd/bisync_test.TestMain(0x1b2b22ae960) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:245 +0x12b main.main() _testmain.go:64 +0xa5 goroutine 52 [syscall, 60 minutes]: os/signal.signal_recv() /usr/local/go/src/runtime/sigqueue.go:152 +0x98 os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 created by os/signal.Notify.func1.1 in goroutine 1 /usr/local/go/src/os/signal/signal.go:152 +0x1f goroutine 53 [chan receive, 60 minutes]: github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1() /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27 created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xa7 goroutine 54 [chan receive, 60 minutes]: github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1() /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27 created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xa7 goroutine 33 [chan receive, 60 minutes]: testing.(*T).Run(0x1b2b25c8008, {0x1b2b20ee125?, 0x2d0f730?}, 0x1b2b231cb40) /usr/local/go/src/testing/testing.go:2109 +0x4e5 github.com/rclone/rclone/cmd/bisync_test.testBisync({0x2d50cd0, 0x1b2b25a9c50}, 0x1b2b25c8008, {0x1b2b25c2600, 0x22}, {0x0, 0x0}) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:400 +0x905 github.com/rclone/rclone/cmd/bisync_test.TestBisyncRemoteLocal(0x1b2b25c8008) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:258 +0x175 testing.tRunner(0x1b2b25c8008, 0x2d087a0) /usr/local/go/src/testing/testing.go:2036 +0xea created by testing.(*T).Run in goroutine 1 /usr/local/go/src/testing/testing.go:2101 +0x4c5 goroutine 8 [chan receive, 60 minutes]: github.com/rclone/rclone/lib/atexit.Register.func1.1() /home/rclone/go/src/github.com/rclone/rclone/lib/atexit/atexit.go:45 +0x29 created by github.com/rclone/rclone/lib/atexit.Register.func1 in goroutine 33 /home/rclone/go/src/github.com/rclone/rclone/lib/atexit/atexit.go:44 +0x68 goroutine 9 [sync.WaitGroup.Wait, 59 minutes]: sync.runtime_SemacquireWaitGroup(0x0?, 0x0?) /usr/local/go/src/runtime/sema.go:114 +0x2e sync.(*WaitGroup).Wait(0x1b2b21d6898) /usr/local/go/src/sync/waitgroup.go:206 +0x85 github.com/rclone/rclone/fs/sync.(*syncCopyMove).stopTransfers(0x1b2b21d6788) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:556 +0x6f github.com/rclone/rclone/fs/sync.(*syncCopyMove).run(0x1b2b21d6788) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:988 +0x34c github.com/rclone/rclone/fs/sync.runSyncCopyMove({0x2d50cd0, 0x1b2b260ab10}, {0x2d70150, 0x1b2b25ab540}, {0x2d6f040, 0x1b2b261e840}, 0x0, 0x0, 0x0, 0x1, ...) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:1379 +0x225 github.com/rclone/rclone/fs/sync.CopyDir(...) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:1390 github.com/rclone/rclone/cmd/bisync_test.(*bisyncTest).runTestCase(0x1b2b1e10a80, {0x2d50cd0, 0x1b2b2723320}, 0x1b2b2334008?, {0x1b2b20ee125?, 0xb?}) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:508 +0xf71 github.com/rclone/rclone/cmd/bisync_test.testBisync.func4(0x1b2b2334008) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:407 +0x265 testing.tRunner(0x1b2b2334008, 0x1b2b231cb40) /usr/local/go/src/testing/testing.go:2036 +0xea created by testing.(*T).Run in goroutine 33 /usr/local/go/src/testing/testing.go:2101 +0x4c5 goroutine 523 [select]: github.com/rclone/rclone/fs/accounting.(*Account).averageLoop(0x1b2b2248b00) /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting.go:222 +0xec created by github.com/rclone/rclone/fs/accounting.newAccountSizeName in goroutine 265 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting.go:120 +0x43f goroutine 528 [sleep, 6 minutes]: time.Sleep(0x45d964b800) /usr/local/go/src/runtime/time.go:363 +0x165 github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall.func1(0x1b2b2008f98?) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:178 +0x1c created by github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall in goroutine 265 /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:177 +0x17f goroutine 521 [select]: github.com/rclone/rclone/fs/accounting.(*Account).averageLoop(0x1b2b2248900) /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting.go:222 +0xec created by github.com/rclone/rclone/fs/accounting.newAccountSizeName in goroutine 266 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting.go:120 +0x43f goroutine 264 [chan receive, 10 minutes]: github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall(0x1b2b25f12c0, 0x0) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:168 +0xa7 github.com/rclone/rclone/lib/pacer.(*Pacer).call(0x1b2b25f12c0, 0x1b2b2618190, 0xa) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:227 +0xb2 github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0x1b2b25f12c0, 0x1b2b2618190) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:247 +0x92 github.com/rclone/rclone/backend/shade.(*shadeChunkWriter).Close(0x1b2b26180a0, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:269 +0x42a github.com/rclone/rclone/lib/multipart.UploadMultipart({0x2d50d08, 0x1b2b21f6be0}, {0x7ff5d0542920, 0x1b2b1bd86c0}, {0x2d24220, 0x1b2b2248a00}, {{0x2d26d80, 0x1b2b25ab540}, {0x1b2b21eb420, 0x1, ...}}) /home/rclone/go/src/github.com/rclone/rclone/lib/multipart/multipart.go:124 +0xadf github.com/rclone/rclone/backend/shade.(*Object).uploadMultipart(0x1b2b242ccc0, {0x2d50d08?, 0x1b2b21f6be0?}, {0x7ff5d0542920?, 0x1b2b1bd86c0?}, {0x2d24220?, 0x1b2b2248a00?}, {0x1b2b21eb420, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:40 +0xd3 github.com/rclone/rclone/backend/shade.(*Object).Update(0x1b2b242ccc0, {0x2d50d08, 0x1b2b21f6be0}, {0x2d24220, 0x1b2b2248a00}, {0x7ff5d0542920, 0x1b2b1bd86c0}, {0x1b2b21eb420, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:971 +0x1ac github.com/rclone/rclone/backend/shade.(*Fs).Put(0x1b2b25ab540, {0x2d50d08, 0x1b2b21f6be0}, {0x2d24220, 0x1b2b2248a00}, {0x7ff5d0542920, 0x1b2b1bd86c0}, {0x1b2b21eb420, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:555 +0x108 github.com/rclone/rclone/fs/operations.(*copy).updateOrPut(0x1b2b217f440, {0x2d50d08, 0x1b2b21f6be0}, {0x2d396c8?, 0x1b2b27544b0?}, {0x1b2b21eb420, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:229 +0x225 github.com/rclone/rclone/fs/operations.(*copy).manualCopy(0x1b2b217f440, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:282 +0x6f6 github.com/rclone/rclone/fs/operations.(*copy).copy(0x1b2b217f440, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:322 +0x170 github.com/rclone/rclone/fs/operations.Copy({0x2d50d08, 0x1b2b21f6be0}, {0x2d70150, 0x1b2b25ab540}, {0x0, 0x0}, {0x1b2b22ce8a0, 0xf}, {0x2d6f0b0, 0x1b2b1bd86c0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:424 +0x4dc github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0x1b2b21d6788, {0x2d50d08, 0x1b2b21f6be0}, 0x1b2b21f6b40, {0x2d70150, 0x1b2b25ab540}, 0x19, 0x1b2b2851fb8?) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:518 +0x1fe created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers in goroutine 9 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:548 +0x7d goroutine 266 [chan receive, 12 minutes]: github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall(0x1b2b25f12c0, 0x0) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:168 +0xa7 github.com/rclone/rclone/lib/pacer.(*Pacer).call(0x1b2b25f12c0, 0x1b2b2618000, 0xa) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:227 +0xb2 github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0x1b2b25f12c0, 0x1b2b2618000) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:247 +0x92 github.com/rclone/rclone/backend/shade.(*shadeChunkWriter).Close(0x1b2b232e140, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:269 +0x42a github.com/rclone/rclone/lib/multipart.UploadMultipart({0x2d50d08, 0x1b2b21f6be0}, {0x7ff5d0542920, 0x1b2b1bd8780}, {0x2d24220, 0x1b2b2248900}, {{0x2d26d80, 0x1b2b25ab540}, {0x1b2b21eb300, 0x1, ...}}) /home/rclone/go/src/github.com/rclone/rclone/lib/multipart/multipart.go:124 +0xadf github.com/rclone/rclone/backend/shade.(*Object).uploadMultipart(0x1b2b242cb80, {0x2d50d08?, 0x1b2b21f6be0?}, {0x7ff5d0542920?, 0x1b2b1bd8780?}, {0x2d24220?, 0x1b2b2248900?}, {0x1b2b21eb300, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:40 +0xd3 github.com/rclone/rclone/backend/shade.(*Object).Update(0x1b2b242cb80, {0x2d50d08, 0x1b2b21f6be0}, {0x2d24220, 0x1b2b2248900}, {0x7ff5d0542920, 0x1b2b1bd8780}, {0x1b2b21eb300, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:971 +0x1ac github.com/rclone/rclone/backend/shade.(*Fs).Put(0x1b2b25ab540, {0x2d50d08, 0x1b2b21f6be0}, {0x2d24220, 0x1b2b2248900}, {0x7ff5d0542920, 0x1b2b1bd8780}, {0x1b2b21eb300, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:555 +0x108 github.com/rclone/rclone/fs/operations.(*copy).updateOrPut(0x1b2b217f290, {0x2d50d08, 0x1b2b21f6be0}, {0x2d396c8?, 0x1b2b27543c0?}, {0x1b2b21eb300, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:229 +0x225 github.com/rclone/rclone/fs/operations.(*copy).manualCopy(0x1b2b217f290, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:282 +0x6f6 github.com/rclone/rclone/fs/operations.(*copy).copy(0x1b2b217f290, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:322 +0x170 github.com/rclone/rclone/fs/operations.Copy({0x2d50d08, 0x1b2b21f6be0}, {0x2d70150, 0x1b2b25ab540}, {0x0, 0x0}, {0x1b2b22ce8e0, 0xf}, {0x2d6f0b0, 0x1b2b1bd8780}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:424 +0x4dc github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0x1b2b21d6788, {0x2d50d08, 0x1b2b21f6be0}, 0x1b2b21f6b40, {0x2d70150, 0x1b2b25ab540}, 0x4b, 0x1b2b2857fb8?) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:518 +0x1fe created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers in goroutine 9 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:548 +0x7d goroutine 265 [chan receive, 6 minutes]: github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall(0x1b2b25f12c0, 0x0) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:168 +0xa7 github.com/rclone/rclone/lib/pacer.(*Pacer).call(0x1b2b25f12c0, 0x1b2b1b96140, 0xa) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:227 +0xb2 github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0x1b2b25f12c0, 0x1b2b1b96140) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:247 +0x92 github.com/rclone/rclone/backend/shade.(*shadeChunkWriter).Close(0x1b2b1b96050, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:269 +0x42a github.com/rclone/rclone/lib/multipart.UploadMultipart({0x2d50d08, 0x1b2b21f6be0}, {0x7ff5d0542920, 0x1b2b1bd8660}, {0x2d24220, 0x1b2b2248b00}, {{0x2d26d80, 0x1b2b25ab540}, {0x1b2b21eb510, 0x1, ...}}) /home/rclone/go/src/github.com/rclone/rclone/lib/multipart/multipart.go:124 +0xadf github.com/rclone/rclone/backend/shade.(*Object).uploadMultipart(0x1b2b242cdc0, {0x2d50d08?, 0x1b2b21f6be0?}, {0x7ff5d0542920?, 0x1b2b1bd8660?}, {0x2d24220?, 0x1b2b2248b00?}, {0x1b2b21eb510, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:40 +0xd3 github.com/rclone/rclone/backend/shade.(*Object).Update(0x1b2b242cdc0, {0x2d50d08, 0x1b2b21f6be0}, {0x2d24220, 0x1b2b2248b00}, {0x7ff5d0542920, 0x1b2b1bd8660}, {0x1b2b21eb510, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:971 +0x1ac github.com/rclone/rclone/backend/shade.(*Fs).Put(0x1b2b25ab540, {0x2d50d08, 0x1b2b21f6be0}, {0x2d24220, 0x1b2b2248b00}, {0x7ff5d0542920, 0x1b2b1bd8660}, {0x1b2b21eb510, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:555 +0x108 github.com/rclone/rclone/fs/operations.(*copy).updateOrPut(0x1b2b217f560, {0x2d50d08, 0x1b2b21f6be0}, {0x2d396c8?, 0x1b2b27545a0?}, {0x1b2b21eb510, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:229 +0x225 github.com/rclone/rclone/fs/operations.(*copy).manualCopy(0x1b2b217f560, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:282 +0x6f6 github.com/rclone/rclone/fs/operations.(*copy).copy(0x1b2b217f560, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:322 +0x170 github.com/rclone/rclone/fs/operations.Copy({0x2d50d08, 0x1b2b21f6be0}, {0x2d70150, 0x1b2b25ab540}, {0x0, 0x0}, {0x1b2b22ce830, 0x9}, {0x2d6f0b0, 0x1b2b1bd8660}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:424 +0x4dc github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0x1b2b21d6788, {0x2d50d08, 0x1b2b21f6be0}, 0x1b2b21f6b40, {0x2d70150, 0x1b2b25ab540}, 0x32, 0x1b2b2850fb8?) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:518 +0x1fe created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers in goroutine 9 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:548 +0x7d goroutine 263 [chan receive, 14 minutes]: github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall(0x1b2b25f12c0, 0x0) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:168 +0xa7 github.com/rclone/rclone/lib/pacer.(*Pacer).call(0x1b2b25f12c0, 0x1b2b21f6190, 0xa) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:227 +0xb2 github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0x1b2b25f12c0, 0x1b2b21f6190) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:247 +0x92 github.com/rclone/rclone/backend/shade.(*shadeChunkWriter).Close(0x1b2b21f60a0, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:269 +0x42a github.com/rclone/rclone/lib/multipart.UploadMultipart({0x2d50d08, 0x1b2b21f6be0}, {0x7ff5d0542920, 0x1b2b25f0540}, {0x2d24220, 0x1b2b2248c00}, {{0x2d26d80, 0x1b2b25ab540}, {0x1b2b21eb640, 0x1, ...}}) /home/rclone/go/src/github.com/rclone/rclone/lib/multipart/multipart.go:124 +0xadf github.com/rclone/rclone/backend/shade.(*Object).uploadMultipart(0x1b2b242cf00, {0x2d50d08?, 0x1b2b21f6be0?}, {0x7ff5d0542920?, 0x1b2b25f0540?}, {0x2d24220?, 0x1b2b2248c00?}, {0x1b2b21eb640, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/upload.go:40 +0xd3 github.com/rclone/rclone/backend/shade.(*Object).Update(0x1b2b242cf00, {0x2d50d08, 0x1b2b21f6be0}, {0x2d24220, 0x1b2b2248c00}, {0x7ff5d0542920, 0x1b2b25f0540}, {0x1b2b21eb640, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:971 +0x1ac github.com/rclone/rclone/backend/shade.(*Fs).Put(0x1b2b25ab540, {0x2d50d08, 0x1b2b21f6be0}, {0x2d24220, 0x1b2b2248c00}, {0x7ff5d0542920, 0x1b2b25f0540}, {0x1b2b21eb640, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/backend/shade/shade.go:555 +0x108 github.com/rclone/rclone/fs/operations.(*copy).updateOrPut(0x1b2b217f830, {0x2d50d08, 0x1b2b21f6be0}, {0x2d396c8?, 0x1b2b2754690?}, {0x1b2b21eb640, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:229 +0x225 github.com/rclone/rclone/fs/operations.(*copy).manualCopy(0x1b2b217f830, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:282 +0x6f6 github.com/rclone/rclone/fs/operations.(*copy).copy(0x1b2b217f830, {0x2d50d08, 0x1b2b21f6be0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:322 +0x170 github.com/rclone/rclone/fs/operations.Copy({0x2d50d08, 0x1b2b21f6be0}, {0x2d70150, 0x1b2b25ab540}, {0x0, 0x0}, {0x1b2b286e0c0, 0x11}, {0x2d6f0b0, 0x1b2b25f0540}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:424 +0x4dc github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0x1b2b21d6788, {0x2d50d08, 0x1b2b21f6be0}, 0x1b2b21f6b40, {0x2d70150, 0x1b2b25ab540}, 0x0, 0x1b2b2509c38?) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:518 +0x1fe created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers in goroutine 9 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync.go:548 +0x7d goroutine 275 [chan receive, 59 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:268 +0x3c created by github.com/rclone/rclone/fs/march.(*March).Run in goroutine 9 /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:266 +0x359 goroutine 132 [select]: github.com/rclone/rclone/fs/accounting.(*StatsInfo).averageLoop(0x1b2b1f91a40, {0x2d50d08, 0x1b2b21f6050}) /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/stats.go:352 +0x145 created by github.com/rclone/rclone/fs/accounting.(*StatsInfo)._startAverageLoop in goroutine 264 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/stats.go:389 +0x11d goroutine 526 [select]: github.com/rclone/rclone/fs/accounting.(*Account).averageLoop(0x1b2b2248c00) /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting.go:222 +0xec created by github.com/rclone/rclone/fs/accounting.newAccountSizeName in goroutine 263 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting.go:120 +0x43f goroutine 522 [select]: github.com/rclone/rclone/fs/accounting.(*Account).averageLoop(0x1b2b2248a00) /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting.go:222 +0xec created by github.com/rclone/rclone/fs/accounting.newAccountSizeName in goroutine 264 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting.go:120 +0x43f "./bisync.test -test.v -test.timeout 1h0m0s -remote TestShade: -verbose" - Finished ERROR in 1h0m0.337927005s (try 1/5): exit status 2: Failed []