"./sync.test -test.v -test.timeout 1h0m0s -remote TestChunkerChunk50bYandex: -verbose -size-limit 1024 -test.run '^(TestCopyRedownload|TestSyncBackupDir|TestSyncConcurrentDelete)$'" - Starting (try 2/5) 2025/07/01 04:49:47 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-lelipal3jeda" 2025/07/01 04:49:47 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/07/01 04:49:47 DEBUG : Creating backend with remote "TestYandex:rclone-test-lelipal3jeda" 2025/07/01 04:49:48 DEBUG : Creating backend with remote "/tmp/rclone201563068" === RUN TestCopyRedownload run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda'", Local "Local file system at /tmp/rclone201563068", Modify Window "1ns" 2025/07/01 04:49:53 INFO : sub dir/hello world.rclone_chunk.001_pkdn77: Moved (server-side) to: sub dir/hello world 2025/07/01 04:49:54 DEBUG : Added delayed dir = "sub dir", newDst= 2025/07/01 04:49:55 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/07/01 04:49:55 DEBUG : Local file system at /tmp/rclone201563068: Waiting for checks to finish 2025/07/01 04:49:55 DEBUG : Local file system at /tmp/rclone201563068: Waiting for transfers to finish 2025/07/01 04:49:56 DEBUG : sub dir/hello world: md5 = 5eb63bbbe01eeed093cb22bb8f5acdc3 OK 2025/07/01 04:49:56 DEBUG : sub dir/hello world.7ddbfcba.partial: renamed to: sub dir/hello world 2025/07/01 04:49:56 INFO : sub dir/hello world: Copied (new) 2025/07/01 04:49:56 INFO : sub dir: Set directory modification time (using DirSetModTime) --- PASS: TestCopyRedownload (20.14s) === RUN TestSyncBackupDir run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda'", Local "Local file system at /tmp/rclone201563068", Modify Window "1ns" 2025/07/01 04:50:13 INFO : dst/one.rclone_chunk.001_pke6do: Moved (server-side) to: dst/one 2025/07/01 04:50:18 INFO : dst/two.rclone_chunk.001_pkebb6: Moved (server-side) to: dst/two 2025/07/01 04:50:23 INFO : dst/three.txt.rclone_chunk.001_pkeg5b: Moved (server-side) to: dst/three.txt 2025/07/01 04:50:24 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-lelipal3jeda/dst" 2025/07/01 04:50:24 DEBUG : Creating backend with remote "TestYandex:rclone-test-lelipal3jeda/dst" 2025/07/01 04:50:27 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-lelipal3jeda/backup" 2025/07/01 04:50:27 DEBUG : Creating backend with remote "TestYandex:rclone-test-lelipal3jeda/backup" 2025/07/01 04:50:28 DEBUG : one: Sizes differ (src 4 vs dst 3) 2025/07/01 04:50:28 DEBUG : two: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:50:28 DEBUG : two: Unchanged skipping 2025/07/01 04:50:28 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda/dst': Waiting for checks to finish 2025/07/01 04:50:28 DEBUG : one: move non-chunked object... 2025/07/01 04:50:29 INFO : one: Moved (server-side) 2025/07/01 04:50:29 INFO : one: Moved (server-side) 2025/07/01 04:50:29 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda/dst': Waiting for transfers to finish 2025/07/01 04:50:29 DEBUG : one: skip slow MD5 on source file, hashing in-transit 2025/07/01 04:50:35 INFO : one.rclone_chunk.001_pkerrz: Moved (server-side) to: one 2025/07/01 04:50:35 DEBUG : one: md5 = c7957179c41f69d44f217a108c7915d8 OK 2025/07/01 04:50:35 INFO : one: Copied (new) 2025/07/01 04:50:35 DEBUG : Waiting for deletions to finish 2025/07/01 04:50:35 DEBUG : three.txt: move non-chunked object... 2025/07/01 04:50:37 INFO : three.txt: Moved (server-side) 2025/07/01 04:50:37 INFO : three.txt: Moved (server-side) 2025/07/01 04:50:37 INFO : three.txt: Moved into backup dir 2025/07/01 04:50:43 INFO : dst/three.txt.rclone_chunk.001_pkf0qe: Moved (server-side) to: dst/three.txt 2025/07/01 04:50:44 DEBUG : one: Sizes differ (src 5 vs dst 4) 2025/07/01 04:50:44 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda/dst': Waiting for checks to finish 2025/07/01 04:50:44 DEBUG : two: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:50:44 DEBUG : two: Unchanged skipping 2025/07/01 04:50:49 INFO : one: Deleted 2025/07/01 04:50:49 DEBUG : one: move non-chunked object... 2025/07/01 04:50:50 INFO : one: Moved (server-side) 2025/07/01 04:50:50 INFO : one: Moved (server-side) 2025/07/01 04:50:50 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda/dst': Waiting for transfers to finish 2025/07/01 04:50:51 DEBUG : one: skip slow MD5 on source file, hashing in-transit 2025/07/01 04:50:56 INFO : one.rclone_chunk.001_pkfdfw: Moved (server-side) to: one 2025/07/01 04:50:56 DEBUG : one: md5 = 0f93e81041f0cab37c37a05ae998b219 OK 2025/07/01 04:50:56 INFO : one: Copied (new) 2025/07/01 04:50:56 DEBUG : Waiting for deletions to finish 2025/07/01 04:51:01 INFO : three.txt: Deleted 2025/07/01 04:51:01 DEBUG : three.txt: move non-chunked object... 2025/07/01 04:51:03 INFO : three.txt: Moved (server-side) 2025/07/01 04:51:03 INFO : three.txt: Moved (server-side) 2025/07/01 04:51:03 INFO : three.txt: Moved into backup dir --- PASS: TestSyncBackupDir (88.42s) === RUN TestSyncConcurrentDelete run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda'", Local "Local file system at /tmp/rclone201563068", Modify Window "1ns" 2025/07/01 04:51:41 INFO : both0.rclone_chunk.001_pkgn89: Moved (server-side) to: both0 2025/07/01 04:51:46 INFO : only0.rclone_chunk.001_pkgrz3: Moved (server-side) to: only0 2025/07/01 04:51:50 INFO : both1.rclone_chunk.001_pkgw9r: Moved (server-side) to: both1 2025/07/01 04:51:55 INFO : only1.rclone_chunk.001_pkh19t: Moved (server-side) to: only1 2025/07/01 04:52:00 INFO : both2.rclone_chunk.001_pkh524: Moved (server-side) to: both2 2025/07/01 04:52:06 INFO : only2.rclone_chunk.001_pkhazj: Moved (server-side) to: only2 2025/07/01 04:52:10 INFO : both3.rclone_chunk.001_pkhg7v: Moved (server-side) to: both3 2025/07/01 04:52:15 INFO : only3.rclone_chunk.001_pkhlyj: Moved (server-side) to: only3 2025/07/01 04:52:20 INFO : both4.rclone_chunk.001_pkhqfu: Moved (server-side) to: both4 2025/07/01 04:52:24 INFO : only4.rclone_chunk.001_pkhuu8: Moved (server-side) to: only4 2025/07/01 04:52:31 INFO : both5.rclone_chunk.001_pkhz6h: Moved (server-side) to: both5 2025/07/01 04:52:35 INFO : only5.rclone_chunk.001_pki540: Moved (server-side) to: only5 2025/07/01 04:52:40 INFO : both6.rclone_chunk.001_pkiaxo: Moved (server-side) to: both6 2025/07/01 04:52:43 DEBUG : pacer: low level retry 1/10 (error [500 - InternalServerError] Internal Server Error (Ошибка сервера.)) 2025/07/01 04:52:43 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/07/01 04:52:43 DEBUG : pacer: Reducing sleep to 15ms 2025/07/01 04:52:44 DEBUG : pacer: Reducing sleep to 11.25ms 2025/07/01 04:52:44 DEBUG : pacer: Reducing sleep to 10ms 2025/07/01 04:52:46 INFO : only6.rclone_chunk.001_pkiejg: Moved (server-side) to: only6 2025/07/01 04:52:51 INFO : both7.rclone_chunk.001_pkikcp: Moved (server-side) to: both7 2025/07/01 04:52:55 INFO : only7.rclone_chunk.001_pkipam: Moved (server-side) to: only7 2025/07/01 04:53:01 INFO : both8.rclone_chunk.001_pkiun8: Moved (server-side) to: both8 2025/07/01 04:53:06 INFO : only8.rclone_chunk.001_pkizq4: Moved (server-side) to: only8 2025/07/01 04:53:11 INFO : both9.rclone_chunk.001_pkj4qv: Moved (server-side) to: both9 2025/07/01 04:53:15 INFO : only9.rclone_chunk.001_pkj9gu: Moved (server-side) to: only9 2025/07/01 04:53:20 INFO : both10.rclone_chunk.001_pkjd7a: Moved (server-side) to: both10 2025/07/01 04:53:25 INFO : only10.rclone_chunk.001_pkji6i: Moved (server-side) to: only10 2025/07/01 04:53:30 INFO : both11.rclone_chunk.001_pkjoej: Moved (server-side) to: both11 2025/07/01 04:53:34 INFO : only11.rclone_chunk.001_pkjszl: Moved (server-side) to: only11 2025/07/01 04:53:39 INFO : both12.rclone_chunk.001_pkjx5g: Moved (server-side) to: both12 2025/07/01 04:53:44 INFO : only12.rclone_chunk.001_pkk14u: Moved (server-side) to: only12 2025/07/01 04:53:48 INFO : both13.rclone_chunk.001_pkk6am: Moved (server-side) to: both13 2025/07/01 04:53:52 INFO : only13.rclone_chunk.001_pkka2f: Moved (server-side) to: only13 2025/07/01 04:53:57 INFO : both14.rclone_chunk.001_pkkfn4: Moved (server-side) to: both14 2025/07/01 04:54:03 INFO : only14.rclone_chunk.001_pkkkz5: Moved (server-side) to: only14 2025/07/01 04:54:07 INFO : both15.rclone_chunk.001_pkkp51: Moved (server-side) to: both15 2025/07/01 04:54:11 INFO : only15.rclone_chunk.001_pkktzr: Moved (server-side) to: only15 2025/07/01 04:54:16 INFO : both16.rclone_chunk.001_pkkyvf: Moved (server-side) to: both16 2025/07/01 04:54:20 INFO : only16.rclone_chunk.001_pkl2z8: Moved (server-side) to: only16 2025/07/01 04:54:26 INFO : both17.rclone_chunk.001_pkl7bt: Moved (server-side) to: both17 2025/07/01 04:54:31 INFO : only17.rclone_chunk.001_pkldy0: Moved (server-side) to: only17 2025/07/01 04:54:36 INFO : both18.rclone_chunk.001_pklhgj: Moved (server-side) to: both18 2025/07/01 04:54:39 DEBUG : pacer: low level retry 1/10 (error [500 - InternalServerError] Internal Server Error (Ошибка сервера.)) 2025/07/01 04:54:39 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/07/01 04:54:42 DEBUG : pacer: Reducing sleep to 15ms 2025/07/01 04:54:42 DEBUG : pacer: Reducing sleep to 11.25ms 2025/07/01 04:54:42 DEBUG : pacer: Reducing sleep to 10ms 2025/07/01 04:54:44 INFO : only18.rclone_chunk.001_pklmfr: Moved (server-side) to: only18 2025/07/01 04:54:50 INFO : both19.rclone_chunk.001_pklury: Moved (server-side) to: both19 2025/07/01 04:54:55 INFO : only19.rclone_chunk.001_pkm13z: Moved (server-side) to: only19 fstest.go:124: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:124 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:129 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:148 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:192 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:335 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:347 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:340 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2745 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2759 Error: Should be true Test: TestSyncConcurrentDelete Messages: both5: Modification time difference too big |-213936h47m23.500000001s| > 1ns (want 2001-02-03 04:05:06.499999999 +0000 UTC vs got 2025-07-01 04:52:30 +0000 UTC) (precision 1ns) 2025/07/01 04:54:56 DEBUG : both0: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both12: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda': Waiting for checks to finish 2025/07/01 04:54:56 DEBUG : both10: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both11: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both0: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both13: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both13: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both14: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both14: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both15: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both15: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both16: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both12: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both17: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both17: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both18: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both18: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both19: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both19: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both1: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both1: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both2: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both2: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both3: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both3: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both4: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both4: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both5: Modification times differ by 213936h47m23.500000001s: 2001-02-03 04:05:06.499999999 +0000 UTC, 2025-07-01 04:52:30 +0000 UTC 2025/07/01 04:54:56 DEBUG : both10: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both6: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both6: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both7: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both7: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both8: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both8: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both9: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/07/01 04:54:56 DEBUG : both9: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both5: md5 = 8ee2027983915ec78acc45027d874316 OK 2025/07/01 04:54:56 DEBUG : both11: Unchanged skipping 2025/07/01 04:54:56 DEBUG : both16: Unchanged skipping 2025/07/01 04:54:56 INFO : both5: Updated modification time in destination 2025/07/01 04:54:56 DEBUG : both5: Unchanged skipping 2025/07/01 04:54:56 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda': Waiting for transfers to finish 2025/07/01 04:54:56 DEBUG : Waiting for deletions to finish 2025/07/01 04:55:00 INFO : only18: Deleted 2025/07/01 04:55:01 INFO : only16: Deleted 2025/07/01 04:55:01 INFO : only2: Deleted 2025/07/01 04:55:01 INFO : only1: Deleted 2025/07/01 04:55:05 INFO : only9: Deleted 2025/07/01 04:55:05 INFO : only0: Deleted 2025/07/01 04:55:06 INFO : only10: Deleted 2025/07/01 04:55:06 INFO : only13: Deleted 2025/07/01 04:55:10 INFO : only4: Deleted 2025/07/01 04:55:10 INFO : only14: Deleted 2025/07/01 04:55:11 INFO : only6: Deleted 2025/07/01 04:55:11 INFO : only7: Deleted 2025/07/01 04:55:15 INFO : only3: Deleted 2025/07/01 04:55:15 INFO : only11: Deleted 2025/07/01 04:55:15 INFO : only8: Deleted 2025/07/01 04:55:16 INFO : only15: Deleted 2025/07/01 04:55:19 INFO : only17: Deleted 2025/07/01 04:55:20 INFO : only5: Deleted 2025/07/01 04:55:21 INFO : only12: Deleted 2025/07/01 04:55:21 INFO : only19: Deleted --- FAIL: TestSyncConcurrentDelete (322.49s) FAIL 2025/07/01 04:56:59 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-lelipal3jeda': Purge remote "./sync.test -test.v -test.timeout 1h0m0s -remote TestChunkerChunk50bYandex: -verbose -size-limit 1024 -test.run '^(TestCopyRedownload|TestSyncBackupDir|TestSyncConcurrentDelete)$'" - Finished ERROR in 7m16.635000473s (try 2/5): exit status 1: Failed [TestSyncConcurrentDelete]