"./sync.test -test.v -test.timeout 1h0m0s -remote TestChunkerChunk50bYandex: -verbose -size-limit 1024 -test.run '^(TestSyncConcurrentDelete|TestSyncMultipleCompareDest)$'" - Starting (try 2/5) 2025/05/19 06:26:55 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-jifozen7pibo" 2025/05/19 06:26:55 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/05/19 06:26:55 DEBUG : Creating backend with remote "TestYandex:rclone-test-jifozen7pibo" 2025/05/19 06:26:55 DEBUG : Creating backend with remote "/tmp/rclone4204519228" === RUN TestSyncMultipleCompareDest run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-jifozen7pibo'", Local "Local file system at /tmp/rclone4204519228", Modify Window "1ns" 2025/05/19 06:27:01 INFO : pre-dest1/1.rclone_chunk.001_i2743s: Moved (server-side) to: pre-dest1/1 2025/05/19 06:27:06 INFO : pre-dest2/2.rclone_chunk.001_i27azv: Moved (server-side) to: pre-dest2/2 2025/05/19 06:27:07 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-jifozen7pibo/dest" 2025/05/19 06:27:07 DEBUG : Creating backend with remote "TestYandex:rclone-test-jifozen7pibo/dest" 2025/05/19 06:27:08 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-jifozen7pibo/pre-dest1" 2025/05/19 06:27:08 DEBUG : Creating backend with remote "TestYandex:rclone-test-jifozen7pibo/pre-dest1" 2025/05/19 06:27:08 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-jifozen7pibo/pre-dest2" 2025/05/19 06:27:08 DEBUG : Creating backend with remote "TestYandex:rclone-test-jifozen7pibo/pre-dest2" 2025/05/19 06:27:09 DEBUG : 1: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:27:09 DEBUG : 1: Destination found in --compare-dest, skipping 2025/05/19 06:27:10 DEBUG : 2: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:27:10 DEBUG : 2: Destination found in --compare-dest, skipping 2025/05/19 06:27:11 DEBUG : 3: Need to transfer - File not found at Destination 2025/05/19 06:27:11 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-jifozen7pibo/dest': Waiting for checks to finish 2025/05/19 06:27:11 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-jifozen7pibo/dest': Waiting for transfers to finish 2025/05/19 06:27:11 DEBUG : 3: skip slow MD5 on source file, hashing in-transit 2025/05/19 06:27:16 INFO : 3.rclone_chunk.001_i27jhr: Moved (server-side) to: 3 2025/05/19 06:27:16 DEBUG : 3: md5 = eccbc87e4b5ce2fe28308fd9f2a7baf3 OK 2025/05/19 06:27:16 INFO : 3: Copied (new) 2025/05/19 06:27:16 DEBUG : Waiting for deletions to finish --- PASS: TestSyncMultipleCompareDest (55.53s) === RUN TestSyncConcurrentDelete run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-jifozen7pibo'", Local "Local file system at /tmp/rclone4204519228", Modify Window "1ns" 2025/05/19 06:27:56 INFO : both0.rclone_chunk.001_i28neh: Moved (server-side) to: both0 2025/05/19 06:28:02 INFO : only0.rclone_chunk.001_i28ty8: Moved (server-side) to: only0 2025/05/19 06:28:09 INFO : both1.rclone_chunk.001_i28znd: Moved (server-side) to: both1 2025/05/19 06:28:14 INFO : only1.rclone_chunk.001_i2950z: Moved (server-side) to: only1 2025/05/19 06:28:21 INFO : both2.rclone_chunk.001_i29alx: Moved (server-side) to: both2 2025/05/19 06:28:26 INFO : only2.rclone_chunk.001_i29heu: Moved (server-side) to: only2 2025/05/19 06:28:32 INFO : both3.rclone_chunk.001_i29nsp: Moved (server-side) to: both3 2025/05/19 06:28:37 INFO : only3.rclone_chunk.001_i29sea: Moved (server-side) to: only3 2025/05/19 06:28:42 INFO : both4.rclone_chunk.001_i29xar: Moved (server-side) to: both4 2025/05/19 06:28:48 INFO : only4.rclone_chunk.001_i2a28r: Moved (server-side) to: only4 2025/05/19 06:28:53 INFO : both5.rclone_chunk.001_i2a87w: Moved (server-side) to: both5 2025/05/19 06:28:58 INFO : only5.rclone_chunk.001_i2ae5g: Moved (server-side) to: only5 2025/05/19 06:29:04 INFO : both6.rclone_chunk.001_i2ai8r: Moved (server-side) to: both6 2025/05/19 06:29:11 INFO : only6.rclone_chunk.001_i2aohm: Moved (server-side) to: only6 2025/05/19 06:29:15 INFO : both7.rclone_chunk.001_i2avwn: Moved (server-side) to: both7 2025/05/19 06:29:21 INFO : only7.rclone_chunk.001_i2azv8: Moved (server-side) to: only7 2025/05/19 06:29:26 INFO : both8.rclone_chunk.001_i2b5pc: Moved (server-side) to: both8 2025/05/19 06:29:30 INFO : only8.rclone_chunk.001_i2bay1: Moved (server-side) to: only8 2025/05/19 06:29:36 INFO : both9.rclone_chunk.001_i2bfkj: Moved (server-side) to: both9 2025/05/19 06:29:42 INFO : only9.rclone_chunk.001_i2bkp6: Moved (server-side) to: only9 2025/05/19 06:29:48 INFO : both10.rclone_chunk.001_i2brns: Moved (server-side) to: both10 2025/05/19 06:29:54 INFO : only10.rclone_chunk.001_i2bwjq: Moved (server-side) to: only10 2025/05/19 06:30:00 INFO : both11.rclone_chunk.001_i2c3ct: Moved (server-side) to: both11 2025/05/19 06:30:09 INFO : only11.rclone_chunk.001_i2c8r8: Moved (server-side) to: only11 2025/05/19 06:30:11 DEBUG : pacer: low level retry 1/10 (error [500 - InternalServerError] Internal Server Error (Ошибка сервера.)) 2025/05/19 06:30:11 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/05/19 06:30:12 DEBUG : pacer: Reducing sleep to 15ms 2025/05/19 06:30:12 DEBUG : pacer: Reducing sleep to 11.25ms 2025/05/19 06:30:12 DEBUG : pacer: Reducing sleep to 10ms 2025/05/19 06:30:14 INFO : both12.rclone_chunk.001_i2ch0j: Moved (server-side) to: both12 2025/05/19 06:30:19 INFO : only12.rclone_chunk.001_i2cmia: Moved (server-side) to: only12 2025/05/19 06:30:25 INFO : both13.rclone_chunk.001_i2cs4s: Moved (server-side) to: both13 2025/05/19 06:30:32 INFO : only13.rclone_chunk.001_i2cyh6: Moved (server-side) to: only13 2025/05/19 06:30:37 INFO : both14.rclone_chunk.001_i2d4kl: Moved (server-side) to: both14 2025/05/19 06:30:43 INFO : only14.rclone_chunk.001_i2daav: Moved (server-side) to: only14 2025/05/19 06:30:48 INFO : both15.rclone_chunk.001_i2dfk4: Moved (server-side) to: both15 2025/05/19 06:30:54 INFO : only15.rclone_chunk.001_i2dk21: Moved (server-side) to: only15 2025/05/19 06:30:59 INFO : both16.rclone_chunk.001_i2dqbp: Moved (server-side) to: both16 2025/05/19 06:31:04 INFO : only16.rclone_chunk.001_i2dv66: Moved (server-side) to: only16 2025/05/19 06:31:09 INFO : both17.rclone_chunk.001_i2e0p8: Moved (server-side) to: both17 2025/05/19 06:31:15 INFO : only17.rclone_chunk.001_i2e5x3: Moved (server-side) to: only17 2025/05/19 06:31:21 INFO : both18.rclone_chunk.001_i2ebtr: Moved (server-side) to: both18 2025/05/19 06:31:25 INFO : only18.rclone_chunk.001_i2ehui: Moved (server-side) to: only18 2025/05/19 06:31:37 INFO : both19.rclone_chunk.001_i2emno: Moved (server-side) to: both19 2025/05/19 06:31:42 INFO : only19.rclone_chunk.001_i2eyy7: Moved (server-side) to: only19 fstest.go:121: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:121 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:126 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:145 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:189 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:302 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:344 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:340 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2744 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2758 Error: Should be true Test: TestSyncConcurrentDelete Messages: only5: Modification time difference too big |-212906h23m51.500000001s| > 1ns (want 2001-02-03 04:05:06.499999999 +0000 UTC vs got 2025-05-19 06:28:58 +0000 UTC) (precision 1ns) 2025/05/19 06:31:44 DEBUG : both0: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both1: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both10: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both11: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both0: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both12: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both12: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both13: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both1: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both14: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both14: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both15: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both15: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both16: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both10: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both17: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both17: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both18: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both18: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both19: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both11: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both2: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both2: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both3: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both13: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both4: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both16: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both5: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both4: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both6: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both6: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both7: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both7: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both8: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both8: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both9: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/05/19 06:31:44 DEBUG : both9: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both19: Unchanged skipping 2025/05/19 06:31:44 DEBUG : both3: Unchanged skipping 2025/05/19 06:31:44 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-jifozen7pibo': Waiting for checks to finish 2025/05/19 06:31:44 DEBUG : both5: Unchanged skipping 2025/05/19 06:31:44 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-jifozen7pibo': Waiting for transfers to finish 2025/05/19 06:31:44 DEBUG : Waiting for deletions to finish 2025/05/19 06:31:50 INFO : only10: Deleted 2025/05/19 06:31:50 INFO : only17: Deleted 2025/05/19 06:31:50 INFO : only16: Deleted 2025/05/19 06:31:50 INFO : only1: Deleted 2025/05/19 06:31:55 INFO : only6: Deleted 2025/05/19 06:31:55 INFO : only19: Deleted 2025/05/19 06:31:55 INFO : only8: Deleted 2025/05/19 06:31:56 INFO : only7: Deleted 2025/05/19 06:32:00 INFO : only9: Deleted 2025/05/19 06:32:00 INFO : only0: Deleted 2025/05/19 06:32:00 INFO : only11: Deleted 2025/05/19 06:32:01 INFO : only14: Deleted 2025/05/19 06:32:04 INFO : only15: Deleted 2025/05/19 06:32:05 INFO : only2: Deleted 2025/05/19 06:32:05 INFO : only3: Deleted 2025/05/19 06:32:06 INFO : only4: Deleted 2025/05/19 06:32:09 INFO : only12: Deleted 2025/05/19 06:32:09 INFO : only18: Deleted 2025/05/19 06:32:11 INFO : only13: Deleted 2025/05/19 06:32:11 INFO : only5: Deleted run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-jifozen7pibo'", Local "Local file system at /tmp/rclone4204519228", Modify Window "1ns" --- FAIL: TestSyncConcurrentDelete (357.33s) FAIL 2025/05/19 06:33:48 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-jifozen7pibo': Purge remote "./sync.test -test.v -test.timeout 1h0m0s -remote TestChunkerChunk50bYandex: -verbose -size-limit 1024 -test.run '^(TestSyncConcurrentDelete|TestSyncMultipleCompareDest)$'" - Finished ERROR in 6m58.353325005s (try 2/5): exit status 1: Failed [TestSyncConcurrentDelete]