"./sync.test -test.v -test.timeout 1h0m0s -remote TestLinkbox: -verbose -test.run '^(TestCopyRedownload|TestMoveOverSelf|TestNothingToTransferWithEmptyDirs|TestNothingToTransferWithoutEmptyDirs|TestServerSideCopy|TestServerSideCopyOverSelf|TestServerSideMove|TestServerSideMoveDeleteEmptySourceDirs|TestServerSideMoveOverSelf|TestServerSideMoveWithFilter|TestSyncAfterChangingFilesSizeOnly|TestSyncAfterRemovingAFileAndAddingAFile|TestSyncAfterRemovingAFileAndAddingAFileSubDirWithErrors|TestSyncCompareDest|TestSyncReplaceDirModTime|TestSyncWithExclude)$'" - Starting (try 2/5) 2025/04/29 07:08:02 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-siganed5faza" 2025/04/29 07:08:02 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/04/29 07:08:03 DEBUG : Creating backend with remote "/tmp/rclone3233252353" === RUN TestServerSideCopy run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:08:10 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-leyosac0vodi" sync_test.go:590: Server side copy (if possible) Linkbox root 'rclone-test-siganed5faza' -> Linkbox root 'rclone-test-leyosac0vodi' 2025/04/29 07:08:12 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/04/29 07:08:12 DEBUG : Linkbox root 'rclone-test-leyosac0vodi': Waiting for checks to finish 2025/04/29 07:08:12 DEBUG : Linkbox root 'rclone-test-leyosac0vodi': Waiting for transfers to finish 2025/04/29 07:08:12 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:594: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:594 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideCopy 2025/04/29 07:08:12 DEBUG : Linkbox root 'rclone-test-leyosac0vodi': Purge remote 2025/04/29 07:08:12 NOTICE: purge failed: directory not found --- FAIL: TestServerSideCopy (13.44s) === RUN TestServerSideCopyOverSelf run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:08:22 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-vijopuz2meyi" sync_test.go:626: Server side copy (if possible) Linkbox root 'rclone-test-siganed5faza' -> Linkbox root 'rclone-test-vijopuz2meyi' 2025/04/29 07:08:24 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/04/29 07:08:24 DEBUG : Linkbox root 'rclone-test-vijopuz2meyi': Waiting for checks to finish 2025/04/29 07:08:24 DEBUG : Linkbox root 'rclone-test-vijopuz2meyi': Waiting for transfers to finish 2025/04/29 07:08:24 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:630: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:630 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideCopyOverSelf 2025/04/29 07:08:24 DEBUG : Linkbox root 'rclone-test-vijopuz2meyi': Purge remote 2025/04/29 07:08:24 NOTICE: purge failed: directory not found --- FAIL: TestServerSideCopyOverSelf (12.17s) === RUN TestMoveOverSelf run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:08:35 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for checks to finish 2025/04/29 07:08:35 DEBUG : sub dir/hello world: Sizes differ (src 17 vs dst 11) 2025/04/29 07:08:35 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for transfers to finish 2025/04/29 07:08:35 DEBUG : sub dir/hello world: Update: removing old file 2025/04/29 07:08:38 INFO : sub dir/hello world: Copied (replaced existing) 2025/04/29 07:08:38 INFO : sub dir/hello world: Deleted run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" --- PASS: TestMoveOverSelf (15.54s) === RUN TestServerSideMoveOverSelf run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:08:49 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-serayeh7jivu" sync_test.go:671: Server side copy (if possible) Linkbox root 'rclone-test-siganed5faza' -> Linkbox root 'rclone-test-serayeh7jivu' 2025/04/29 07:08:51 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/04/29 07:08:51 DEBUG : Linkbox root 'rclone-test-serayeh7jivu': Waiting for checks to finish 2025/04/29 07:08:51 DEBUG : Linkbox root 'rclone-test-serayeh7jivu': Waiting for transfers to finish 2025/04/29 07:08:51 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:675: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:675 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMoveOverSelf 2025/04/29 07:08:51 DEBUG : Linkbox root 'rclone-test-serayeh7jivu': Purge remote 2025/04/29 07:08:51 NOTICE: purge failed: directory not found --- FAIL: TestServerSideMoveOverSelf (11.59s) === RUN TestCopyRedownload run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:09:11 DEBUG : Local file system at /tmp/rclone3233252353: Waiting for checks to finish 2025/04/29 07:09:11 DEBUG : Local file system at /tmp/rclone3233252353: Waiting for transfers to finish run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:09:11 ERROR : : error listing: directory not found fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 2025/04/29 07:09:12 ERROR : : error listing: directory not found fstest.go:292: Sleeping for 2s for list eventual consistency: 2/3 2025/04/29 07:09:14 ERROR : : error listing: directory not found fstest.go:292: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:299: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:299 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:350 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:736 Error: Should be true Test: TestCopyRedownload Messages: listing wrong, want sub dir/hello world (11) got fstest.go:197: Not found "sub dir/hello world" fstest.go:200: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:200 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:304 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:350 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:736 Error: Not equal: expected: 0 actual : 1 Test: TestCopyRedownload Messages: 1 objects not found --- FAIL: TestCopyRedownload (27.66s) === RUN TestSyncAfterChangingFilesSizeOnly run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:09:27 DEBUG : potato: Sizes differ (src 21 vs dst 60) 2025/04/29 07:09:27 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for checks to finish 2025/04/29 07:09:27 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for transfers to finish 2025/04/29 07:09:27 DEBUG : potato: Update: removing old file 2025/04/29 07:09:31 INFO : potato: Copied (replaced existing) 2025/04/29 07:09:31 DEBUG : Waiting for deletions to finish run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" --- PASS: TestSyncAfterChangingFilesSizeOnly (10.50s) === RUN TestSyncAfterRemovingAFileAndAddingAFile run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:286: Sleeping for 6s just to make sure 2025/04/29 07:09:58 DEBUG : empty space: Sizes identical 2025/04/29 07:09:58 DEBUG : potato2: Need to transfer - File not found at Destination 2025/04/29 07:09:58 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for checks to finish 2025/04/29 07:09:58 DEBUG : empty space: Unchanged skipping 2025/04/29 07:09:58 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for transfers to finish 2025/04/29 07:10:00 INFO : potato2: Copied (new) 2025/04/29 07:10:00 DEBUG : Waiting for deletions to finish 2025/04/29 07:10:02 INFO : potato: Deleted run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" run.go:125: removing file "potato2" failed - try 1/3: could not Remove: Linkbox error 500: Post "http://192.168.0.133:9200/zdisk_item/_search": context deadline exceeded --- PASS: TestSyncAfterRemovingAFileAndAddingAFile (37.67s) === RUN TestSyncAfterRemovingAFileAndAddingAFileSubDirWithErrors run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:10:26 INFO : d: Making directory 2025/04/29 07:10:39 DEBUG : a/potato2: Need to transfer - File not found at Destination 2025/04/29 07:10:39 DEBUG : c/non empty space: Sizes identical 2025/04/29 07:10:39 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for checks to finish 2025/04/29 07:10:39 DEBUG : c/non empty space: Unchanged skipping 2025/04/29 07:10:39 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for transfers to finish 2025/04/29 07:10:44 INFO : a/potato2: Copied (new) 2025/04/29 07:10:44 ERROR : Linkbox root 'rclone-test-siganed5faza': not deleting files as there were IO errors 2025/04/29 07:10:44 ERROR : Linkbox root 'rclone-test-siganed5faza': not deleting directories as there were IO errors run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" run.go:125: removing dir "b" failed - try 1/3: directory not empty run.go:125: removing dir "b" failed - try 2/3: directory not empty run.go:125: removing dir "b" failed - try 3/3: directory not empty run.go:128: removing dir "b" failed: directory not empty fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:295: Flushing the directory cache fstest.go:299: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:299 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:173 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1684 Error: Should be true Test: TestSyncAfterRemovingAFileAndAddingAFileSubDirWithErrors Messages: listing wrong, want got b/potato (21) fstest.go:185: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:185 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:302 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:173 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1684 Error: Should be true Test: TestSyncAfterRemovingAFileAndAddingAFileSubDirWithErrors Messages: Unexpected file "b/potato" fstest.go:321: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:321 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:173 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1684 Error: Not equal: expected: []string{} actual : []string{"b"} Diff: --- Expected +++ Actual @@ -1,2 +1,3 @@ -([]string) { +([]string) (len=1) { + (string) (len=1) "b" } Test: TestSyncAfterRemovingAFileAndAddingAFileSubDirWithErrors Messages: directories --- FAIL: TestSyncAfterRemovingAFileAndAddingAFileSubDirWithErrors (72.15s) === RUN TestSyncWithExclude run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:11:30 DEBUG : enormous: Excluded (Size Filter) 2025/04/29 07:11:30 DEBUG : enormous: Excluded 2025/04/29 07:11:30 DEBUG : potato2: Excluded (Size Filter) 2025/04/29 07:11:30 DEBUG : potato2: Excluded 2025/04/29 07:11:30 DEBUG : potato2: Excluded (Size Filter) 2025/04/29 07:11:30 DEBUG : potato2: Excluded 2025/04/29 07:11:30 DEBUG : empty space: Sizes identical 2025/04/29 07:11:30 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for checks to finish 2025/04/29 07:11:30 DEBUG : empty space: Unchanged skipping 2025/04/29 07:11:30 DEBUG : Linkbox root 'rclone-test-siganed5faza': Waiting for transfers to finish 2025/04/29 07:11:30 DEBUG : Waiting for deletions to finish 2025/04/29 07:11:30 INFO : There was nothing to transfer run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:11:31 DEBUG : enormous: Excluded (Size Filter) 2025/04/29 07:11:31 DEBUG : enormous: Excluded 2025/04/29 07:11:31 DEBUG : potato2: Excluded (Size Filter) 2025/04/29 07:11:31 DEBUG : potato2: Excluded 2025/04/29 07:11:31 DEBUG : potato2: Excluded (Size Filter) 2025/04/29 07:11:31 DEBUG : potato2: Excluded 2025/04/29 07:11:31 DEBUG : empty space: Sizes identical 2025/04/29 07:11:31 DEBUG : empty space: Unchanged skipping 2025/04/29 07:11:31 DEBUG : Local file system at /tmp/rclone3233252353: Waiting for checks to finish 2025/04/29 07:11:31 DEBUG : Local file system at /tmp/rclone3233252353: Waiting for transfers to finish 2025/04/29 07:11:31 DEBUG : Waiting for deletions to finish 2025/04/29 07:11:31 INFO : There was nothing to transfer run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" --- PASS: TestSyncWithExclude (12.85s) === RUN TestServerSideMove run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:11:37 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-gatijum7jire" sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-siganed5faza' -> Linkbox root 'rclone-test-gatijum7jire' 2025/04/29 07:11:59 DEBUG : potato2: Need to transfer - File not found at Destination 2025/04/29 07:11:59 DEBUG : empty space: Sizes identical 2025/04/29 07:11:59 DEBUG : empty space: Unchanged skipping 2025/04/29 07:11:59 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/04/29 07:11:59 DEBUG : Linkbox root 'rclone-test-gatijum7jire': Waiting for checks to finish 2025/04/29 07:11:59 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:11:59 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:12:00 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:12:00 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:12:01 INFO : empty space: Deleted 2025/04/29 07:12:01 DEBUG : Linkbox root 'rclone-test-gatijum7jire': Waiting for transfers to finish sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1881 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMove 2025/04/29 07:12:01 DEBUG : Linkbox root 'rclone-test-gatijum7jire': Purge remote --- FAIL: TestServerSideMove (29.25s) === RUN TestServerSideMoveWithFilter run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:12:06 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-huzuwus2xavi" 2025/04/29 07:12:21 DEBUG : empty space: Trying to read object after upload: try again in 100ms (1/10) 2025/04/29 07:12:21 DEBUG : empty space: Trying to read object after upload: try again in 200ms (2/10) 2025/04/29 07:12:21 DEBUG : empty space: Trying to read object after upload: try again in 400ms (3/10) sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-siganed5faza' -> Linkbox root 'rclone-test-huzuwus2xavi' 2025/04/29 07:12:46 DEBUG : empty space: Excluded (Size Filter) 2025/04/29 07:12:46 DEBUG : empty space: Excluded 2025/04/29 07:12:46 DEBUG : empty space: Excluded (Size Filter) 2025/04/29 07:12:46 DEBUG : empty space: Excluded 2025/04/29 07:12:46 DEBUG : potato2: Need to transfer - File not found at Destination 2025/04/29 07:12:46 DEBUG : Linkbox root 'rclone-test-huzuwus2xavi': Waiting for checks to finish 2025/04/29 07:12:46 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/04/29 07:12:46 DEBUG : Linkbox root 'rclone-test-huzuwus2xavi': Waiting for transfers to finish 2025/04/29 07:12:46 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:12:46 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:12:46 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:12:46 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1894 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMoveWithFilter 2025/04/29 07:12:46 DEBUG : Linkbox root 'rclone-test-huzuwus2xavi': Purge remote --- FAIL: TestServerSideMoveWithFilter (46.41s) === RUN TestServerSideMoveDeleteEmptySourceDirs run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:12:52 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-tawagid8zuju" 2025/04/29 07:13:00 INFO : tomatoDir: Making directory sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-siganed5faza' -> Linkbox root 'rclone-test-tawagid8zuju' 2025/04/29 07:13:09 DEBUG : potato2: Need to transfer - File not found at Destination 2025/04/29 07:13:09 DEBUG : empty space: Sizes identical 2025/04/29 07:13:09 DEBUG : empty space: Unchanged skipping 2025/04/29 07:13:09 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/04/29 07:13:09 DEBUG : Linkbox root 'rclone-test-tawagid8zuju': Waiting for checks to finish 2025/04/29 07:13:09 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:13:09 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:13:09 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:13:09 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/29 07:13:10 INFO : empty space: Deleted 2025/04/29 07:13:10 DEBUG : Linkbox root 'rclone-test-tawagid8zuju': Waiting for transfers to finish 2025/04/29 07:13:10 ERROR : Linkbox root 'rclone-test-siganed5faza': not deleting directories as there were IO errors sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1901 Error: Received unexpected error: not deleting directories as there were IO errors Test: TestServerSideMoveDeleteEmptySourceDirs 2025/04/29 07:13:10 DEBUG : Linkbox root 'rclone-test-tawagid8zuju': Purge remote --- FAIL: TestServerSideMoveDeleteEmptySourceDirs (27.02s) === RUN TestSyncCompareDest run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:13:19 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-siganed5faza/dst" 2025/04/29 07:13:31 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-siganed5faza/CompareDest" 2025/04/29 07:13:42 DEBUG : one: Need to transfer - File not found at Destination 2025/04/29 07:13:42 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for checks to finish 2025/04/29 07:13:42 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for transfers to finish 2025/04/29 07:13:46 INFO : one: Copied (new) 2025/04/29 07:13:46 DEBUG : Waiting for deletions to finish 2025/04/29 07:13:47 DEBUG : one: Sizes differ (src 5 vs dst 3) 2025/04/29 07:13:47 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for checks to finish 2025/04/29 07:13:48 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for transfers to finish 2025/04/29 07:13:48 DEBUG : one: Update: removing old file 2025/04/29 07:13:51 INFO : one: Copied (replaced existing) 2025/04/29 07:13:51 DEBUG : Waiting for deletions to finish 2025/04/29 07:13:52 DEBUG : dst/one: Update: removing old file 2025/04/29 07:14:15 DEBUG : one: Sizes differ (src 5 vs dst 3) 2025/04/29 07:14:15 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for checks to finish 2025/04/29 07:14:15 DEBUG : one: Sizes identical 2025/04/29 07:14:15 DEBUG : one: Destination found in --compare-dest, skipping 2025/04/29 07:14:15 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for transfers to finish 2025/04/29 07:14:15 DEBUG : Waiting for deletions to finish 2025/04/29 07:14:15 INFO : There was nothing to transfer 2025/04/29 07:14:20 DEBUG : one: Sizes differ (src 5 vs dst 3) 2025/04/29 07:14:20 DEBUG : two: Sizes identical 2025/04/29 07:14:20 DEBUG : two: Destination found in --compare-dest, skipping 2025/04/29 07:14:20 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for checks to finish 2025/04/29 07:14:20 DEBUG : one: Sizes identical 2025/04/29 07:14:20 DEBUG : one: Destination found in --compare-dest, skipping 2025/04/29 07:14:20 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for transfers to finish 2025/04/29 07:14:20 DEBUG : Waiting for deletions to finish 2025/04/29 07:14:20 INFO : There was nothing to transfer 2025/04/29 07:14:21 DEBUG : one: Sizes differ (src 5 vs dst 3) 2025/04/29 07:14:21 DEBUG : two: Sizes identical 2025/04/29 07:14:21 DEBUG : two: Destination found in --compare-dest, skipping 2025/04/29 07:14:21 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for checks to finish 2025/04/29 07:14:21 DEBUG : one: Sizes identical 2025/04/29 07:14:21 DEBUG : one: Destination found in --compare-dest, skipping 2025/04/29 07:14:21 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for transfers to finish 2025/04/29 07:14:21 DEBUG : Waiting for deletions to finish 2025/04/29 07:14:21 INFO : There was nothing to transfer sync_test.go:2175: No hash on uploaded file so skipping compare timestamp test 2025/04/29 07:14:23 DEBUG : one: Sizes differ (src 5 vs dst 3) 2025/04/29 07:14:23 DEBUG : two: Sizes differ (src 5 vs dst 3) 2025/04/29 07:14:23 DEBUG : two: Need to transfer - File not found at Destination 2025/04/29 07:14:23 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for checks to finish 2025/04/29 07:14:25 INFO : two: Copied (new) 2025/04/29 07:14:34 DEBUG : Linkbox root 'rclone-test-siganed5faza/dst': Waiting for transfers to finish 2025/04/29 07:14:34 DEBUG : one: Update: removing old file 2025/04/29 07:14:37 INFO : one: Copied (replaced existing) 2025/04/29 07:14:37 DEBUG : Waiting for deletions to finish fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:295: Flushing the directory cache fstest.go:299: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:299 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:344 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:340 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2192 Error: Should be true Test: TestSyncCompareDest Messages: listing wrong, want CompareDest/one (5), CompareDest/two (3), dst/one (3), dst/two (5) got CompareDest/one (5), CompareDest/two (3), dst/one (5), dst/two (5) fstest.go:144: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:144 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:189 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:302 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:344 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:340 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2192 Error: Not equal: expected: 3 actual : 5 Test: TestSyncCompareDest Messages: dst/one: size incorrect file=3 vs obj=5 --- FAIL: TestSyncCompareDest (96.62s) === RUN TestSyncReplaceDirModTime run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:14:56 DEBUG : empty_dir: Making directory with metadata 2025/04/29 07:14:56 INFO : empty_dir: Made directory with metadata (mtime=2011-12-25T12:59:59.123456789Z) 2025/04/29 07:14:56 DEBUG : empty_on_remote: Making directory with metadata 2025/04/29 07:14:56 INFO : empty_on_remote: Made directory with metadata (mtime=2011-12-25T12:59:59.123456789Z) 2025/04/29 07:14:56 INFO : empty_on_remote: Making directory 2025/04/29 07:14:58 INFO : empty_on_remote: Made directory with modification time 2011-12-25 12:59:59.123456789 +0000 UTC run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:15:07 INFO : test_dir1: Set directory modification time (using DirSetModTime) 2025/04/29 07:15:07 INFO : test_dir2: Set directory modification time (using DirSetModTime) 2025/04/29 07:15:07 INFO : test_dir2/sub_dir: Set directory modification time (using DirSetModTime) 2025/04/29 07:15:07 INFO : empty_on_remote: Set directory modification time (using DirSetModTime) run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" --- PASS: TestSyncReplaceDirModTime (29.10s) === RUN TestNothingToTransferWithEmptyDirs run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:15:25 INFO : sub dir: Set directory modification time (using DirSetModTime) 2025/04/29 07:15:25 INFO : sub dir: Making directory 2025/04/29 07:15:27 INFO : sub dir: Made directory with modification time 2011-12-30 12:59:59 +0000 UTC run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" --- PASS: TestNothingToTransferWithEmptyDirs (83.75s) === RUN TestNothingToTransferWithoutEmptyDirs run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" 2025/04/29 07:16:49 INFO : sub dir: Set directory modification time (using DirSetModTime) 2025/04/29 07:16:49 INFO : sub dir: Making directory 2025/04/29 07:16:50 INFO : sub dir: Made directory with modification time 2011-12-30 12:59:59 +0000 UTC run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" run.go:180: Remote "Linkbox root 'rclone-test-siganed5faza'", Local "Local file system at /tmp/rclone3233252353", Modify Window "876000h0m0s" fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:295: Flushing the directory cache fstest.go:299: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:299 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:344 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:340 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2895 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2950 Error: Should be true Test: TestNothingToTransferWithoutEmptyDirs Messages: listing wrong, want sub dir/hello world (11), sub dir2/very/very/very/very/very/nested/subdir/hello world (11) got sub dir/hello world (11) fstest.go:197: Not found "sub dir2/very/very/very/very/very/nested/subdir/hello world" fstest.go:200: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:200 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:304 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:344 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:340 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2895 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2950 Error: Not equal: expected: 0 actual : 1 Test: TestNothingToTransferWithoutEmptyDirs Messages: 1 objects not found sync_test.go:2900: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2900 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2950 Error: Should be false Test: TestNothingToTransferWithoutEmptyDirs Messages: expected to find no "Copied" logs, but found one: 2025/04/29 07:18:23 INFO : sub dir2/very/very/very/very/very/nested/subdir/hello world: Copied (new) sync_test.go:2906: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2906 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2950 Error: Should be true Test: TestNothingToTransferWithoutEmptyDirs Messages: expected to find a "There was nothing to transfer" log: 2025/04/29 07:18:23 INFO : sub dir2/very/very/very/very/very/nested/subdir/hello world: Copied (new) sync_test.go:2907: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2907 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2950 Error: Not equal: expected: 0 actual : 1 Test: TestNothingToTransferWithoutEmptyDirs --- FAIL: TestNothingToTransferWithoutEmptyDirs (171.46s) FAIL 2025/04/29 07:19:40 DEBUG : Linkbox root 'rclone-test-siganed5faza': Purge remote "./sync.test -test.v -test.timeout 1h0m0s -remote TestLinkbox: -verbose -test.run '^(TestCopyRedownload|TestMoveOverSelf|TestNothingToTransferWithEmptyDirs|TestNothingToTransferWithoutEmptyDirs|TestServerSideCopy|TestServerSideCopyOverSelf|TestServerSideMove|TestServerSideMoveDeleteEmptySourceDirs|TestServerSideMoveOverSelf|TestServerSideMoveWithFilter|TestSyncAfterChangingFilesSizeOnly|TestSyncAfterRemovingAFileAndAddingAFile|TestSyncAfterRemovingAFileAndAddingAFileSubDirWithErrors|TestSyncCompareDest|TestSyncReplaceDirModTime|TestSyncWithExclude)$'" - Finished ERROR in 11m39.435421109s (try 2/5): exit status 1: Failed [TestServerSideCopy TestServerSideCopyOverSelf TestServerSideMoveOverSelf TestCopyRedownload TestSyncAfterRemovingAFileAndAddingAFileSubDirWithErrors TestServerSideMove TestServerSideMoveWithFilter TestServerSideMoveDeleteEmptySourceDirs TestSyncCompareDest TestNothingToTransferWithoutEmptyDirs]