"./sync.test -test.v -test.timeout 1h0m0s -remote TestLinkbox: -verbose -test.run '^(TestCopyRedownload|TestServerSideCopy|TestServerSideCopyOverSelf|TestServerSideMove|TestServerSideMoveDeleteEmptySourceDirs|TestServerSideMoveOverSelf|TestServerSideMoveWithFilter)$'" - Starting (try 5/5)
2025/04/20 04:39:31 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-vuhizil3paga"
2025/04/20 04:39:31 DEBUG : Using config file from "/home/rclone/.rclone.conf"
2025/04/20 04:39:32 DEBUG : Creating backend with remote "/tmp/rclone2456787011"
=== RUN   TestServerSideCopy
    run.go:180: Remote "Linkbox root 'rclone-test-vuhizil3paga'", Local "Local file system at /tmp/rclone2456787011", Modify Window "876000h0m0s"
2025/04/20 04:39:39 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-qanomuv8ruzo"
2025/04/20 04:39:39 DEBUG : Config file has changed externally - reloading
    sync_test.go:590: Server side copy (if possible) Linkbox root 'rclone-test-vuhizil3paga' -> Linkbox root 'rclone-test-qanomuv8ruzo'
2025/04/20 04:39:41 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination
2025/04/20 04:39:41 DEBUG : Linkbox root 'rclone-test-qanomuv8ruzo': Waiting for checks to finish
2025/04/20 04:39:41 DEBUG : Linkbox root 'rclone-test-qanomuv8ruzo': Waiting for transfers to finish
2025/04/20 04:39:41 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
    sync_test.go:594: 
        	Error Trace:	/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:594
        	Error:      	Received unexpected error:
        	            	failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
        	Test:       	TestServerSideCopy
2025/04/20 04:39:41 DEBUG : Linkbox root 'rclone-test-qanomuv8ruzo': Purge remote
2025/04/20 04:39:41 NOTICE: purge failed: directory not found
--- FAIL: TestServerSideCopy (14.18s)
=== RUN   TestServerSideCopyOverSelf
    run.go:180: Remote "Linkbox root 'rclone-test-vuhizil3paga'", Local "Local file system at /tmp/rclone2456787011", Modify Window "876000h0m0s"
2025/04/20 04:39:50 DEBUG : sub dir/hello world: Trying to read object after upload: try again in 100ms (1/10)
2025/04/20 04:39:51 DEBUG : sub dir/hello world: Trying to read object after upload: try again in 200ms (2/10)
2025/04/20 04:39:51 DEBUG : sub dir/hello world: Trying to read object after upload: try again in 400ms (3/10)
2025/04/20 04:39:52 DEBUG : sub dir/hello world: Trying to read object after upload: try again in 800ms (4/10)
2025/04/20 04:39:53 DEBUG : sub dir/hello world: Trying to read object after upload: try again in 1.6s (5/10)
    fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3
    fstest.go:295: Flushing the directory cache
    fstest.go:286: Sleeping for 6s just to make sure
2025/04/20 04:40:04 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-ludeguc6piyi"
    sync_test.go:626: Server side copy (if possible) Linkbox root 'rclone-test-vuhizil3paga' -> Linkbox root 'rclone-test-ludeguc6piyi'
2025/04/20 04:40:06 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination
2025/04/20 04:40:06 DEBUG : Linkbox root 'rclone-test-ludeguc6piyi': Waiting for checks to finish
2025/04/20 04:40:06 DEBUG : Linkbox root 'rclone-test-ludeguc6piyi': Waiting for transfers to finish
2025/04/20 04:40:06 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
    sync_test.go:630: 
        	Error Trace:	/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:630
        	Error:      	Received unexpected error:
        	            	failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
        	Test:       	TestServerSideCopyOverSelf
2025/04/20 04:40:06 DEBUG : Linkbox root 'rclone-test-ludeguc6piyi': Purge remote
2025/04/20 04:40:07 NOTICE: purge failed: directory not found
--- FAIL: TestServerSideCopyOverSelf (25.61s)
=== RUN   TestServerSideMoveOverSelf
    run.go:180: Remote "Linkbox root 'rclone-test-vuhizil3paga'", Local "Local file system at /tmp/rclone2456787011", Modify Window "876000h0m0s"
2025/04/20 04:40:18 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-muficow2zecu"
2025/04/20 04:40:18 DEBUG : Config file has changed externally - reloading
    sync_test.go:671: Server side copy (if possible) Linkbox root 'rclone-test-vuhizil3paga' -> Linkbox root 'rclone-test-muficow2zecu'
2025/04/20 04:40:20 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination
2025/04/20 04:40:20 DEBUG : Linkbox root 'rclone-test-muficow2zecu': Waiting for checks to finish
2025/04/20 04:40:20 DEBUG : Linkbox root 'rclone-test-muficow2zecu': Waiting for transfers to finish
2025/04/20 04:40:20 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
    sync_test.go:675: 
        	Error Trace:	/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:675
        	Error:      	Received unexpected error:
        	            	failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
        	Test:       	TestServerSideMoveOverSelf
2025/04/20 04:40:20 DEBUG : Linkbox root 'rclone-test-muficow2zecu': Purge remote
2025/04/20 04:40:20 NOTICE: purge failed: directory not found
--- FAIL: TestServerSideMoveOverSelf (13.90s)
=== RUN   TestCopyRedownload
    run.go:180: Remote "Linkbox root 'rclone-test-vuhizil3paga'", Local "Local file system at /tmp/rclone2456787011", Modify Window "876000h0m0s"
2025/04/20 04:40:32 DEBUG : Added delayed dir = "sub dir", newDst=<nil>
2025/04/20 04:40:32 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination
2025/04/20 04:40:32 DEBUG : Local file system at /tmp/rclone2456787011: Waiting for checks to finish
2025/04/20 04:40:32 DEBUG : Local file system at /tmp/rclone2456787011: Waiting for transfers to finish
2025/04/20 04:40:32 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:40:32 ERROR : sub dir: Failed to update directory timestamp or metadata: chtimes /tmp/rclone2456787011/sub dir: no such file or directory
    sync_test.go:732: 
        	Error Trace:	/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:732
        	Error:      	Received unexpected error:
        	            	failed to set directory modtime: chtimes /tmp/rclone2456787011/sub dir: no such file or directory
        	Test:       	TestCopyRedownload
--- FAIL: TestCopyRedownload (12.13s)
=== RUN   TestServerSideMove
    run.go:180: Remote "Linkbox root 'rclone-test-vuhizil3paga'", Local "Local file system at /tmp/rclone2456787011", Modify Window "876000h0m0s"
2025/04/20 04:40:37 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-lariyes7yapa"
2025/04/20 04:40:37 DEBUG : Config file has changed externally - reloading
    sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-vuhizil3paga' -> Linkbox root 'rclone-test-lariyes7yapa'
2025/04/20 04:40:57 DEBUG : potato2: Need to transfer - File not found at Destination
2025/04/20 04:40:57 DEBUG : empty space: Sizes identical
2025/04/20 04:40:57 DEBUG : empty space: Unchanged skipping
2025/04/20 04:40:57 DEBUG : potato3: Sizes differ (src 68 vs dst 60)
2025/04/20 04:40:57 DEBUG : Linkbox root 'rclone-test-lariyes7yapa': Waiting for checks to finish
2025/04/20 04:40:57 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:40:57 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:40:57 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:40:57 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:40:58 INFO  : empty space: Deleted
2025/04/20 04:40:58 DEBUG : Linkbox root 'rclone-test-lariyes7yapa': Waiting for transfers to finish
    sync_test.go:1716: 
        	Error Trace:	/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716
        	            				/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1881
        	Error:      	Received unexpected error:
        	            	failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
        	Test:       	TestServerSideMove
2025/04/20 04:40:58 DEBUG : Linkbox root 'rclone-test-lariyes7yapa': Purge remote
--- FAIL: TestServerSideMove (26.66s)
=== RUN   TestServerSideMoveWithFilter
    run.go:180: Remote "Linkbox root 'rclone-test-vuhizil3paga'", Local "Local file system at /tmp/rclone2456787011", Modify Window "876000h0m0s"
2025/04/20 04:41:04 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-wikitef0bawu"
    sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-vuhizil3paga' -> Linkbox root 'rclone-test-wikitef0bawu'
2025/04/20 04:41:23 DEBUG : empty space: Excluded (Size Filter)
2025/04/20 04:41:23 DEBUG : empty space: Excluded
2025/04/20 04:41:23 DEBUG : empty space: Excluded (Size Filter)
2025/04/20 04:41:23 DEBUG : empty space: Excluded
2025/04/20 04:41:23 DEBUG : potato2: Need to transfer - File not found at Destination
2025/04/20 04:41:23 DEBUG : potato3: Sizes differ (src 68 vs dst 60)
2025/04/20 04:41:23 DEBUG : Linkbox root 'rclone-test-wikitef0bawu': Waiting for checks to finish
2025/04/20 04:41:23 DEBUG : Linkbox root 'rclone-test-wikitef0bawu': Waiting for transfers to finish
2025/04/20 04:41:23 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:41:23 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:41:23 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:41:23 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
    sync_test.go:1716: 
        	Error Trace:	/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716
        	            				/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1894
        	Error:      	Received unexpected error:
        	            	failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
        	Test:       	TestServerSideMoveWithFilter
2025/04/20 04:41:23 DEBUG : Linkbox root 'rclone-test-wikitef0bawu': Purge remote
--- FAIL: TestServerSideMoveWithFilter (25.80s)
=== RUN   TestServerSideMoveDeleteEmptySourceDirs
    run.go:180: Remote "Linkbox root 'rclone-test-vuhizil3paga'", Local "Local file system at /tmp/rclone2456787011", Modify Window "876000h0m0s"
2025/04/20 04:41:30 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-yumedak7foho"
2025/04/20 04:41:30 DEBUG : Config file has changed externally - reloading
2025/04/20 04:41:40 INFO  : tomatoDir: Making directory
    sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-vuhizil3paga' -> Linkbox root 'rclone-test-yumedak7foho'
2025/04/20 04:41:51 DEBUG : potato2: Need to transfer - File not found at Destination
2025/04/20 04:41:51 DEBUG : empty space: Sizes identical
2025/04/20 04:41:51 DEBUG : empty space: Unchanged skipping
2025/04/20 04:41:51 DEBUG : potato3: Sizes differ (src 68 vs dst 60)
2025/04/20 04:41:52 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:41:52 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:41:52 DEBUG : Linkbox root 'rclone-test-yumedak7foho': Waiting for checks to finish
2025/04/20 04:41:52 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:41:52 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect"
2025/04/20 04:41:53 INFO  : empty space: Deleted
2025/04/20 04:41:53 DEBUG : Linkbox root 'rclone-test-yumedak7foho': Waiting for transfers to finish
2025/04/20 04:41:53 ERROR : Linkbox root 'rclone-test-vuhizil3paga': not deleting directories as there were IO errors
    sync_test.go:1716: 
        	Error Trace:	/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716
        	            				/home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1901
        	Error:      	Received unexpected error:
        	            	not deleting directories as there were IO errors
        	Test:       	TestServerSideMoveDeleteEmptySourceDirs
2025/04/20 04:41:53 DEBUG : Linkbox root 'rclone-test-yumedak7foho': Purge remote
--- FAIL: TestServerSideMoveDeleteEmptySourceDirs (31.38s)
FAIL
2025/04/20 04:42:01 DEBUG : Linkbox root 'rclone-test-vuhizil3paga': Purge remote
"./sync.test -test.v -test.timeout 1h0m0s -remote TestLinkbox: -verbose -test.run '^(TestCopyRedownload|TestServerSideCopy|TestServerSideCopyOverSelf|TestServerSideMove|TestServerSideMoveDeleteEmptySourceDirs|TestServerSideMoveOverSelf|TestServerSideMoveWithFilter)$'" - Finished ERROR in 2m32.417648802s (try 5/5): exit status 1: Failed [TestServerSideCopy TestServerSideCopyOverSelf TestServerSideMoveOverSelf TestCopyRedownload TestServerSideMove TestServerSideMoveWithFilter TestServerSideMoveDeleteEmptySourceDirs]