"./sync.test -test.v -test.timeout 1h0m0s -remote TestLinkbox: -verbose -test.run '^(TestCopyRedownload|TestServerSideCopy|TestServerSideCopyOverSelf|TestServerSideMove|TestServerSideMoveDeleteEmptySourceDirs|TestServerSideMoveOverSelf|TestServerSideMoveWithFilter)$'" - Starting (try 3/5) 2025/05/20 05:13:10 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-rofosef0sayo" 2025/05/20 05:13:10 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/05/20 05:13:11 DEBUG : Creating backend with remote "/tmp/rclone3717522318" === RUN TestServerSideCopy run.go:180: Remote "Linkbox root 'rclone-test-rofosef0sayo'", Local "Local file system at /tmp/rclone3717522318", Modify Window "876000h0m0s" 2025/05/20 05:13:17 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-raniruv8dofa" sync_test.go:590: Server side copy (if possible) Linkbox root 'rclone-test-rofosef0sayo' -> Linkbox root 'rclone-test-raniruv8dofa' 2025/05/20 05:13:18 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/05/20 05:13:18 DEBUG : Linkbox root 'rclone-test-raniruv8dofa': Waiting for checks to finish 2025/05/20 05:13:18 DEBUG : Linkbox root 'rclone-test-raniruv8dofa': Waiting for transfers to finish 2025/05/20 05:13:19 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:594: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:594 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideCopy 2025/05/20 05:13:19 DEBUG : Linkbox root 'rclone-test-raniruv8dofa': Purge remote 2025/05/20 05:13:19 NOTICE: purge failed: directory not found --- FAIL: TestServerSideCopy (11.73s) === RUN TestServerSideCopyOverSelf run.go:180: Remote "Linkbox root 'rclone-test-rofosef0sayo'", Local "Local file system at /tmp/rclone3717522318", Modify Window "876000h0m0s" 2025/05/20 05:13:27 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-rofecib3qupi" sync_test.go:626: Server side copy (if possible) Linkbox root 'rclone-test-rofosef0sayo' -> Linkbox root 'rclone-test-rofecib3qupi' 2025/05/20 05:13:28 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/05/20 05:13:28 DEBUG : Linkbox root 'rclone-test-rofecib3qupi': Waiting for checks to finish 2025/05/20 05:13:28 DEBUG : Linkbox root 'rclone-test-rofecib3qupi': Waiting for transfers to finish 2025/05/20 05:13:29 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:630: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:630 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideCopyOverSelf 2025/05/20 05:13:29 DEBUG : Linkbox root 'rclone-test-rofecib3qupi': Purge remote 2025/05/20 05:13:29 NOTICE: purge failed: directory not found --- FAIL: TestServerSideCopyOverSelf (10.01s) === RUN TestServerSideMoveOverSelf run.go:180: Remote "Linkbox root 'rclone-test-rofosef0sayo'", Local "Local file system at /tmp/rclone3717522318", Modify Window "876000h0m0s" 2025/05/20 05:13:37 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-kicemod9xole" sync_test.go:671: Server side copy (if possible) Linkbox root 'rclone-test-rofosef0sayo' -> Linkbox root 'rclone-test-kicemod9xole' 2025/05/20 05:13:38 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/05/20 05:13:38 DEBUG : Linkbox root 'rclone-test-kicemod9xole': Waiting for checks to finish 2025/05/20 05:13:38 DEBUG : Linkbox root 'rclone-test-kicemod9xole': Waiting for transfers to finish 2025/05/20 05:13:38 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:675: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:675 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMoveOverSelf 2025/05/20 05:13:38 DEBUG : Linkbox root 'rclone-test-kicemod9xole': Purge remote 2025/05/20 05:13:39 NOTICE: purge failed: directory not found --- FAIL: TestServerSideMoveOverSelf (9.97s) === RUN TestCopyRedownload run.go:180: Remote "Linkbox root 'rclone-test-rofosef0sayo'", Local "Local file system at /tmp/rclone3717522318", Modify Window "876000h0m0s" 2025/05/20 05:13:48 DEBUG : Added delayed dir = "sub dir", newDst= 2025/05/20 05:13:48 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/05/20 05:13:48 DEBUG : Local file system at /tmp/rclone3717522318: Waiting for checks to finish 2025/05/20 05:13:48 DEBUG : Local file system at /tmp/rclone3717522318: Waiting for transfers to finish 2025/05/20 05:13:48 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:13:48 ERROR : sub dir: Failed to update directory timestamp or metadata: chtimes /tmp/rclone3717522318/sub dir: no such file or directory sync_test.go:732: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:732 Error: Received unexpected error: failed to set directory modtime: chtimes /tmp/rclone3717522318/sub dir: no such file or directory Test: TestCopyRedownload --- FAIL: TestCopyRedownload (9.23s) === RUN TestServerSideMove run.go:180: Remote "Linkbox root 'rclone-test-rofosef0sayo'", Local "Local file system at /tmp/rclone3717522318", Modify Window "876000h0m0s" 2025/05/20 05:13:52 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-suhokaq4cudo" sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-rofosef0sayo' -> Linkbox root 'rclone-test-suhokaq4cudo' 2025/05/20 05:14:07 DEBUG : potato2: Need to transfer - File not found at Destination 2025/05/20 05:14:07 DEBUG : empty space: Sizes identical 2025/05/20 05:14:07 DEBUG : empty space: Unchanged skipping 2025/05/20 05:14:07 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/05/20 05:14:07 DEBUG : Linkbox root 'rclone-test-suhokaq4cudo': Waiting for checks to finish 2025/05/20 05:14:07 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:14:07 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:14:08 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:14:08 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:14:09 INFO : empty space: Deleted 2025/05/20 05:14:09 DEBUG : Linkbox root 'rclone-test-suhokaq4cudo': Waiting for transfers to finish sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1881 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMove 2025/05/20 05:14:09 DEBUG : Linkbox root 'rclone-test-suhokaq4cudo': Purge remote fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:295: Flushing the directory cache fstest.go:299: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:299 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:173 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1684 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/panic.go:629 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1006 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1881 Error: Should be true Test: TestServerSideMove Messages: listing wrong, want got potato2 (60), potato3 (68) fstest.go:185: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:185 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:302 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:173 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1684 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/panic.go:629 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1006 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1881 Error: Should be true Test: TestServerSideMove Messages: Unexpected file "potato2" fstest.go:185: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:185 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:302 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:173 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1684 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/panic.go:629 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1006 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1881 Error: Should be true Test: TestServerSideMove Messages: Unexpected file "potato3" --- FAIL: TestServerSideMove (29.52s) === RUN TestServerSideMoveWithFilter run.go:180: Remote "Linkbox root 'rclone-test-rofosef0sayo'", Local "Local file system at /tmp/rclone3717522318", Modify Window "876000h0m0s" 2025/05/20 05:14:21 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-kivahot9yofi" 2025/05/20 05:14:21 DEBUG : Config file has changed externally - reloading sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-rofosef0sayo' -> Linkbox root 'rclone-test-kivahot9yofi' 2025/05/20 05:14:37 DEBUG : empty space: Excluded (Size Filter) 2025/05/20 05:14:37 DEBUG : empty space: Excluded 2025/05/20 05:14:37 DEBUG : empty space: Excluded (Size Filter) 2025/05/20 05:14:37 DEBUG : empty space: Excluded 2025/05/20 05:14:37 DEBUG : potato2: Need to transfer - File not found at Destination 2025/05/20 05:14:37 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/05/20 05:14:37 DEBUG : Linkbox root 'rclone-test-kivahot9yofi': Waiting for checks to finish 2025/05/20 05:14:37 DEBUG : Linkbox root 'rclone-test-kivahot9yofi': Waiting for transfers to finish 2025/05/20 05:14:37 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:14:37 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:14:37 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:14:37 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1894 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMoveWithFilter 2025/05/20 05:14:37 DEBUG : Linkbox root 'rclone-test-kivahot9yofi': Purge remote --- FAIL: TestServerSideMoveWithFilter (21.87s) === RUN TestServerSideMoveDeleteEmptySourceDirs run.go:180: Remote "Linkbox root 'rclone-test-rofosef0sayo'", Local "Local file system at /tmp/rclone3717522318", Modify Window "876000h0m0s" 2025/05/20 05:14:43 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-tezeten6doxa" 2025/05/20 05:14:51 INFO : tomatoDir: Making directory sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-rofosef0sayo' -> Linkbox root 'rclone-test-tezeten6doxa' 2025/05/20 05:15:00 DEBUG : potato2: Need to transfer - File not found at Destination 2025/05/20 05:15:00 DEBUG : empty space: Sizes identical 2025/05/20 05:15:00 DEBUG : empty space: Unchanged skipping 2025/05/20 05:15:00 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/05/20 05:15:00 DEBUG : Linkbox root 'rclone-test-tezeten6doxa': Waiting for checks to finish 2025/05/20 05:15:00 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:15:00 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:15:01 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:15:01 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/05/20 05:15:02 INFO : empty space: Deleted 2025/05/20 05:15:02 DEBUG : Linkbox root 'rclone-test-tezeten6doxa': Waiting for transfers to finish 2025/05/20 05:15:02 ERROR : Linkbox root 'rclone-test-rofosef0sayo': not deleting directories as there were IO errors sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1901 Error: Received unexpected error: not deleting directories as there were IO errors Test: TestServerSideMoveDeleteEmptySourceDirs 2025/05/20 05:15:02 DEBUG : Linkbox root 'rclone-test-tezeten6doxa': Purge remote --- FAIL: TestServerSideMoveDeleteEmptySourceDirs (25.01s) FAIL 2025/05/20 05:15:08 DEBUG : Linkbox root 'rclone-test-rofosef0sayo': Purge remote "./sync.test -test.v -test.timeout 1h0m0s -remote TestLinkbox: -verbose -test.run '^(TestCopyRedownload|TestServerSideCopy|TestServerSideCopyOverSelf|TestServerSideMove|TestServerSideMoveDeleteEmptySourceDirs|TestServerSideMoveOverSelf|TestServerSideMoveWithFilter)$'" - Finished ERROR in 1m59.482930321s (try 3/5): exit status 1: Failed [TestServerSideCopy TestServerSideCopyOverSelf TestServerSideMoveOverSelf TestCopyRedownload TestServerSideMove TestServerSideMoveWithFilter TestServerSideMoveDeleteEmptySourceDirs]