"./sync.test -test.v -test.timeout 1h0m0s -remote TestChunkerChunk50bYandex: -verbose -size-limit 1024 -test.run '^(TestServerSideMoveDeleteEmptySourceDirs|TestSyncConcurrentDelete)$'" - Starting (try 3/5) 2025/01/14 02:38:13 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-quwewub0duzu" 2025/01/14 02:38:13 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/01/14 02:38:13 DEBUG : Creating backend with remote "TestYandex:rclone-test-quwewub0duzu" 2025/01/14 02:38:14 DEBUG : Reset feature "ListR" 2025/01/14 02:38:14 DEBUG : Creating backend with remote "/tmp/rclone1473549717" === RUN TestServerSideMoveDeleteEmptySourceDirs run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-quwewub0duzu'", Local "Local file system at /tmp/rclone1473549717", Modify Window "1ns" 2025/01/14 02:38:14 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-pocowir4wufo" 2025/01/14 02:38:14 DEBUG : Creating backend with remote "TestYandex:rclone-test-pocowir4wufo" 2025/01/14 02:38:14 DEBUG : Reset feature "ListR" 2025/01/14 02:38:26 INFO : potato2.rclone_chunk.001_2a8tjk: Moved (server-side) to: potato2.rclone_chunk.001 2025/01/14 02:38:28 INFO : potato2.rclone_chunk.002_2a8tjk: Moved (server-side) to: potato2.rclone_chunk.002 2025/01/14 02:38:39 INFO : empty space.rclone_chunk.001_2a9bjr: Moved (server-side) to: empty space 2025/01/14 02:38:52 INFO : potato3.rclone_chunk.001_2a9isq: Moved (server-side) to: potato3.rclone_chunk.001 2025/01/14 02:38:55 INFO : potato3.rclone_chunk.002_2a9isq: Moved (server-side) to: potato3.rclone_chunk.002 2025/01/14 02:38:58 INFO : tomatoDir: Making directory sync_test.go:1705: Server side move (if possible) Chunked 'TestChunkerChunk50bYandex:rclone-test-quwewub0duzu' -> Chunked 'TestChunkerChunk50bYandex:rclone-test-pocowir4wufo' 2025/01/14 02:39:08 INFO : empty space.rclone_chunk.001_2aa478: Moved (server-side) to: empty space 2025/01/14 02:39:17 INFO : potato3.rclone_chunk.001_2aabks: Moved (server-side) to: potato3.rclone_chunk.001 2025/01/14 02:39:19 INFO : potato3.rclone_chunk.002_2aabks: Moved (server-side) to: potato3.rclone_chunk.002 2025/01/14 02:39:23 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-pocowir4wufo': Using server-side directory move 2025/01/14 02:39:23 INFO : Chunked 'TestChunkerChunk50bYandex:rclone-test-pocowir4wufo': Server side directory move failed - fallback to file moves: can't copy directory - destination already exists 2025/01/14 02:39:24 DEBUG : potato2: Need to transfer - File not found at Destination 2025/01/14 02:39:24 DEBUG : empty space: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:39:24 DEBUG : empty space: Unchanged skipping 2025/01/14 02:39:24 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/01/14 02:39:25 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-pocowir4wufo': Waiting for checks to finish 2025/01/14 02:39:25 DEBUG : potato2: move 2 data chunks... 2025/01/14 02:39:27 INFO : potato2.rclone_chunk.001: Moved (server-side) 2025/01/14 02:39:28 INFO : empty space: Deleted 2025/01/14 02:39:28 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-pocowir4wufo': Waiting for transfers to finish 2025/01/14 02:39:29 INFO : potato2.rclone_chunk.002: Moved (server-side) 2025/01/14 02:39:32 INFO : potato2: Moved (server-side) 2025/01/14 02:39:34 INFO : potato2: Moved (server-side) 2025/01/14 02:39:36 INFO : potato3: Deleted 2025/01/14 02:39:36 DEBUG : potato3: move 2 data chunks... 2025/01/14 02:39:39 INFO : potato3.rclone_chunk.001: Moved (server-side) 2025/01/14 02:39:41 INFO : potato3.rclone_chunk.002: Moved (server-side) 2025/01/14 02:39:43 INFO : potato3: Moved (server-side) 2025/01/14 02:39:46 INFO : potato3: Moved (server-side) 2025/01/14 02:39:46 INFO : tomatoDir: Removing directory 2025/01/14 02:39:51 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-quwewub0duzu': deleted 1 directories 2025/01/14 02:39:54 DEBUG : Creating backend with remote "TestChunkerChunk50bYandex:rclone-test-didefas3nolo" 2025/01/14 02:39:54 DEBUG : Creating backend with remote "TestYandex:rclone-test-didefas3nolo" 2025/01/14 02:39:54 DEBUG : Reset feature "ListR" 2025/01/14 02:39:54 INFO : tomatoDir: Making directory 2025/01/14 02:39:55 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-didefas3nolo': Using server-side directory move 2025/01/14 02:39:58 INFO : Chunked 'TestChunkerChunk50bYandex:rclone-test-didefas3nolo': Server side directory move succeeded 2025/01/14 02:40:01 ERROR : : error listing: directory not found 2025/01/14 02:40:02 ERROR : : error listing: directory not found 2025/01/14 02:40:02 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-didefas3nolo': Purge remote 2025/01/14 02:40:14 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-pocowir4wufo': Purge remote 2025/01/14 02:40:14 NOTICE: purge failed: [404 - DiskNotFoundError] Resource not found. (Не удалось найти запрошенный ресурс.) --- PASS: TestServerSideMoveDeleteEmptySourceDirs (120.66s) === RUN TestSyncConcurrentDelete run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-quwewub0duzu'", Local "Local file system at /tmp/rclone1473549717", Modify Window "1ns" 2025/01/14 02:40:22 INFO : both0.rclone_chunk.001_2ac6te: Moved (server-side) to: both0 2025/01/14 02:40:35 INFO : only0.rclone_chunk.001_2acc73: Moved (server-side) to: only0 2025/01/14 02:40:42 INFO : both1.rclone_chunk.001_2acq8u: Moved (server-side) to: both1 2025/01/14 02:40:49 INFO : only1.rclone_chunk.001_2acx8a: Moved (server-side) to: only1 2025/01/14 02:40:55 INFO : both2.rclone_chunk.001_2ad39c: Moved (server-side) to: both2 2025/01/14 02:41:02 INFO : only2.rclone_chunk.001_2ad92m: Moved (server-side) to: only2 2025/01/14 02:41:11 INFO : both3.rclone_chunk.001_2adi1y: Moved (server-side) to: both3 2025/01/14 02:41:19 INFO : only3.rclone_chunk.001_2adqf8: Moved (server-side) to: only3 2025/01/14 02:41:25 INFO : both4.rclone_chunk.001_2adx8a: Moved (server-side) to: both4 2025/01/14 02:41:32 INFO : only4.rclone_chunk.001_2ae419: Moved (server-side) to: only4 2025/01/14 02:41:38 INFO : both5.rclone_chunk.001_2aeao6: Moved (server-side) to: both5 2025/01/14 02:41:44 INFO : only5.rclone_chunk.001_2aegxr: Moved (server-side) to: only5 2025/01/14 02:41:51 INFO : both6.rclone_chunk.001_2aenb4: Moved (server-side) to: both6 2025/01/14 02:41:57 INFO : only6.rclone_chunk.001_2aeute: Moved (server-side) to: only6 2025/01/14 02:42:04 INFO : both7.rclone_chunk.001_2af0uz: Moved (server-side) to: both7 2025/01/14 02:42:10 INFO : only7.rclone_chunk.001_2af6dd: Moved (server-side) to: only7 2025/01/14 02:42:17 INFO : both8.rclone_chunk.001_2afcmp: Moved (server-side) to: both8 2025/01/14 02:42:24 INFO : only8.rclone_chunk.001_2afj9z: Moved (server-side) to: only8 2025/01/14 02:42:31 INFO : both9.rclone_chunk.001_2afrxo: Moved (server-side) to: both9 2025/01/14 02:42:43 INFO : only9.rclone_chunk.001_2afydn: Moved (server-side) to: only9 2025/01/14 02:42:52 INFO : both10.rclone_chunk.001_2ag9l7: Moved (server-side) to: both10 2025/01/14 02:42:58 INFO : only10.rclone_chunk.001_2agi0z: Moved (server-side) to: only10 2025/01/14 02:43:05 INFO : both11.rclone_chunk.001_2agpd2: Moved (server-side) to: both11 2025/01/14 02:43:13 INFO : only11.rclone_chunk.001_2agwlq: Moved (server-side) to: only11 2025/01/14 02:43:16 DEBUG : pacer: low level retry 1/10 (error [500 - InternalServerError] Internal Server Error (Ошибка сервера.)) 2025/01/14 02:43:16 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/01/14 02:43:18 DEBUG : pacer: Reducing sleep to 15ms 2025/01/14 02:43:18 DEBUG : pacer: Reducing sleep to 11.25ms 2025/01/14 02:43:19 DEBUG : pacer: Reducing sleep to 10ms 2025/01/14 02:43:22 INFO : both12.rclone_chunk.001_2ah36u: Moved (server-side) to: both12 2025/01/14 02:43:28 INFO : only12.rclone_chunk.001_2ahc3h: Moved (server-side) to: only12 2025/01/14 02:43:35 INFO : both13.rclone_chunk.001_2ahi82: Moved (server-side) to: both13 2025/01/14 02:43:42 INFO : only13.rclone_chunk.001_2ahpmv: Moved (server-side) to: only13 2025/01/14 02:43:48 INFO : both14.rclone_chunk.001_2ahwy1: Moved (server-side) to: both14 2025/01/14 02:43:55 INFO : only14.rclone_chunk.001_2ai3ky: Moved (server-side) to: only14 2025/01/14 02:44:02 INFO : both15.rclone_chunk.001_2aiagq: Moved (server-side) to: both15 2025/01/14 02:44:08 INFO : only15.rclone_chunk.001_2aihyr: Moved (server-side) to: only15 2025/01/14 02:44:14 INFO : both16.rclone_chunk.001_2ainxk: Moved (server-side) to: both16 2025/01/14 02:44:22 INFO : only16.rclone_chunk.001_2aisuk: Moved (server-side) to: only16 2025/01/14 02:44:27 INFO : both17.rclone_chunk.001_2aj0o9: Moved (server-side) to: both17 2025/01/14 02:44:38 INFO : only17.rclone_chunk.001_2aj6kg: Moved (server-side) to: only17 2025/01/14 02:44:45 INFO : both18.rclone_chunk.001_2ajh8r: Moved (server-side) to: both18 2025/01/14 02:44:53 INFO : only18.rclone_chunk.001_2ajnsf: Moved (server-side) to: only18 2025/01/14 02:45:01 INFO : both19.rclone_chunk.001_2ajv79: Moved (server-side) to: both19 2025/01/14 02:45:12 INFO : only19.rclone_chunk.001_2ak4qz: Moved (server-side) to: only19 fstest.go:121: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:121 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:126 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:145 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:189 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:302 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:344 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:340 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2744 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2758 Error: Should be true Test: TestSyncConcurrentDelete Messages: both19: Modification time difference too big |-209902h39m54.500000001s| > 1ns (want 2001-02-03 04:05:06.499999999 +0000 UTC vs got 2025-01-14 02:45:01 +0000 UTC) (precision 1ns) 2025/01/14 02:45:14 DEBUG : both1: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both10: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both11: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-quwewub0duzu': Waiting for checks to finish 2025/01/14 02:45:14 DEBUG : both0: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both1: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both12: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both12: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both13: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both13: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both14: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both14: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both15: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both15: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both16: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both16: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both17: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both17: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both18: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both18: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both19: Modification times differ by 209902h39m54.500000001s: 2001-02-03 04:05:06.499999999 +0000 UTC, 2025-01-14 02:45:01 +0000 UTC 2025/01/14 02:45:14 DEBUG : both10: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both2: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both19: md5 = 8ee2027983915ec78acc45027d874316 OK 2025/01/14 02:45:14 DEBUG : both2: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both3: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both3: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both4: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both4: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both5: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both5: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both6: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both11: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both7: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both7: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both8: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both0: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both9: Size and modification time the same (differ by 0s, within tolerance 1ns) 2025/01/14 02:45:14 DEBUG : both9: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both6: Unchanged skipping 2025/01/14 02:45:14 DEBUG : both8: Unchanged skipping 2025/01/14 02:45:15 INFO : both19: Updated modification time in destination 2025/01/14 02:45:15 DEBUG : both19: Unchanged skipping 2025/01/14 02:45:15 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-quwewub0duzu': Waiting for transfers to finish 2025/01/14 02:45:15 DEBUG : Waiting for deletions to finish 2025/01/14 02:45:19 INFO : only7: Deleted 2025/01/14 02:45:19 INFO : only5: Deleted 2025/01/14 02:45:20 INFO : only8: Deleted 2025/01/14 02:45:20 INFO : only0: Deleted 2025/01/14 02:45:23 INFO : only13: Deleted 2025/01/14 02:45:23 INFO : only16: Deleted 2025/01/14 02:45:23 INFO : only19: Deleted 2025/01/14 02:45:23 INFO : only18: Deleted 2025/01/14 02:45:26 INFO : only3: Deleted 2025/01/14 02:45:27 INFO : only6: Deleted 2025/01/14 02:45:27 INFO : only12: Deleted 2025/01/14 02:45:27 INFO : only15: Deleted 2025/01/14 02:45:30 INFO : only17: Deleted 2025/01/14 02:45:30 INFO : only1: Deleted 2025/01/14 02:45:31 INFO : only10: Deleted 2025/01/14 02:45:32 INFO : only9: Deleted 2025/01/14 02:45:34 INFO : only11: Deleted 2025/01/14 02:45:34 INFO : only14: Deleted 2025/01/14 02:45:34 INFO : only2: Deleted 2025/01/14 02:45:35 INFO : only4: Deleted 2025/01/14 02:45:35 INFO : There was nothing to transfer run.go:180: Remote "Chunked 'TestChunkerChunk50bYandex:rclone-test-quwewub0duzu'", Local "Local file system at /tmp/rclone1473549717", Modify Window "1ns" --- FAIL: TestSyncConcurrentDelete (398.04s) FAIL 2025/01/14 02:46:53 DEBUG : Chunked 'TestChunkerChunk50bYandex:rclone-test-quwewub0duzu': Purge remote "./sync.test -test.v -test.timeout 1h0m0s -remote TestChunkerChunk50bYandex: -verbose -size-limit 1024 -test.run '^(TestServerSideMoveDeleteEmptySourceDirs|TestSyncConcurrentDelete)$'" - Finished ERROR in 8m43.446141325s (try 3/5): exit status 1: Failed [TestSyncConcurrentDelete]