"./sync.test -test.v -test.timeout 1h0m0s -remote TestLinkbox: -verbose -test.run '^(TestCopyRedownload|TestServerSideCopy|TestServerSideCopyOverSelf|TestServerSideMove|TestServerSideMoveDeleteEmptySourceDirs|TestServerSideMoveOverSelf|TestServerSideMoveWithFilter|TestSyncConcurrentDelete)$'" - Starting (try 2/5) 2025/04/30 06:36:37 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-filiwom3tufe" 2025/04/30 06:36:37 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/04/30 06:36:37 DEBUG : Creating backend with remote "/tmp/rclone3778635316" === RUN TestServerSideCopy run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" 2025/04/30 06:36:42 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-sixoxec0kaqu" sync_test.go:590: Server side copy (if possible) Linkbox root 'rclone-test-filiwom3tufe' -> Linkbox root 'rclone-test-sixoxec0kaqu' 2025/04/30 06:36:43 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/04/30 06:36:43 DEBUG : Linkbox root 'rclone-test-sixoxec0kaqu': Waiting for checks to finish 2025/04/30 06:36:43 DEBUG : Linkbox root 'rclone-test-sixoxec0kaqu': Waiting for transfers to finish 2025/04/30 06:36:43 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:594: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:594 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideCopy 2025/04/30 06:36:43 DEBUG : Linkbox root 'rclone-test-sixoxec0kaqu': Purge remote 2025/04/30 06:36:43 NOTICE: purge failed: directory not found --- FAIL: TestServerSideCopy (9.84s) === RUN TestServerSideCopyOverSelf run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" 2025/04/30 06:36:51 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-nupuhoj0bada" sync_test.go:626: Server side copy (if possible) Linkbox root 'rclone-test-filiwom3tufe' -> Linkbox root 'rclone-test-nupuhoj0bada' 2025/04/30 06:36:52 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/04/30 06:36:52 DEBUG : Linkbox root 'rclone-test-nupuhoj0bada': Waiting for checks to finish 2025/04/30 06:36:52 DEBUG : Linkbox root 'rclone-test-nupuhoj0bada': Waiting for transfers to finish 2025/04/30 06:36:52 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:630: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:630 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideCopyOverSelf 2025/04/30 06:36:52 DEBUG : Linkbox root 'rclone-test-nupuhoj0bada': Purge remote 2025/04/30 06:36:52 NOTICE: purge failed: directory not found --- FAIL: TestServerSideCopyOverSelf (8.38s) === RUN TestServerSideMoveOverSelf run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" 2025/04/30 06:36:59 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-xaturuh4fame" sync_test.go:671: Server side copy (if possible) Linkbox root 'rclone-test-filiwom3tufe' -> Linkbox root 'rclone-test-xaturuh4fame' 2025/04/30 06:37:00 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/04/30 06:37:00 DEBUG : Linkbox root 'rclone-test-xaturuh4fame': Waiting for checks to finish 2025/04/30 06:37:00 DEBUG : Linkbox root 'rclone-test-xaturuh4fame': Waiting for transfers to finish 2025/04/30 06:37:00 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:675: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:675 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMoveOverSelf 2025/04/30 06:37:00 DEBUG : Linkbox root 'rclone-test-xaturuh4fame': Purge remote 2025/04/30 06:37:00 NOTICE: purge failed: directory not found --- FAIL: TestServerSideMoveOverSelf (8.38s) === RUN TestCopyRedownload run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" 2025/04/30 06:37:08 DEBUG : Added delayed dir = "sub dir", newDst= 2025/04/30 06:37:08 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2025/04/30 06:37:08 DEBUG : Local file system at /tmp/rclone3778635316: Waiting for checks to finish 2025/04/30 06:37:08 DEBUG : Local file system at /tmp/rclone3778635316: Waiting for transfers to finish 2025/04/30 06:37:08 ERROR : sub dir/hello world: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:37:08 ERROR : sub dir: Failed to update directory timestamp or metadata: chtimes /tmp/rclone3778635316/sub dir: no such file or directory sync_test.go:732: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:732 Error: Received unexpected error: failed to set directory modtime: chtimes /tmp/rclone3778635316/sub dir: no such file or directory Test: TestCopyRedownload --- FAIL: TestCopyRedownload (7.93s) === RUN TestServerSideMove run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" 2025/04/30 06:37:12 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-vunexim3luxi" 2025/04/30 06:37:12 DEBUG : Config file has changed externally - reloading sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-filiwom3tufe' -> Linkbox root 'rclone-test-vunexim3luxi' 2025/04/30 06:37:24 DEBUG : potato2: Need to transfer - File not found at Destination 2025/04/30 06:37:24 DEBUG : empty space: Sizes identical 2025/04/30 06:37:24 DEBUG : empty space: Unchanged skipping 2025/04/30 06:37:24 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/04/30 06:37:24 DEBUG : Linkbox root 'rclone-test-vunexim3luxi': Waiting for checks to finish 2025/04/30 06:37:24 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:37:24 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:37:24 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:37:24 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:37:25 INFO : empty space: Deleted 2025/04/30 06:37:25 DEBUG : Linkbox root 'rclone-test-vunexim3luxi': Waiting for transfers to finish sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1881 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMove 2025/04/30 06:37:25 DEBUG : Linkbox root 'rclone-test-vunexim3luxi': Purge remote --- FAIL: TestServerSideMove (18.26s) === RUN TestServerSideMoveWithFilter run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" 2025/04/30 06:37:30 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-vuzotob5mayo" sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-filiwom3tufe' -> Linkbox root 'rclone-test-vuzotob5mayo' 2025/04/30 06:37:42 DEBUG : empty space: Excluded (Size Filter) 2025/04/30 06:37:42 DEBUG : empty space: Excluded 2025/04/30 06:37:42 DEBUG : empty space: Excluded (Size Filter) 2025/04/30 06:37:42 DEBUG : empty space: Excluded 2025/04/30 06:37:42 DEBUG : potato2: Need to transfer - File not found at Destination 2025/04/30 06:37:42 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/04/30 06:37:42 DEBUG : Linkbox root 'rclone-test-vuzotob5mayo': Waiting for checks to finish 2025/04/30 06:37:42 DEBUG : Linkbox root 'rclone-test-vuzotob5mayo': Waiting for transfers to finish 2025/04/30 06:37:42 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:37:42 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:37:42 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:37:42 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1894 Error: Received unexpected error: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" Test: TestServerSideMoveWithFilter 2025/04/30 06:37:42 DEBUG : Linkbox root 'rclone-test-vuzotob5mayo': Purge remote --- FAIL: TestServerSideMoveWithFilter (18.30s) === RUN TestServerSideMoveDeleteEmptySourceDirs run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" 2025/04/30 06:37:48 DEBUG : Creating backend with remote "TestLinkbox:rclone-test-zupolow3hepa" 2025/04/30 06:37:54 INFO : tomatoDir: Making directory sync_test.go:1705: Server side move (if possible) Linkbox root 'rclone-test-filiwom3tufe' -> Linkbox root 'rclone-test-zupolow3hepa' 2025/04/30 06:38:02 DEBUG : potato2: Need to transfer - File not found at Destination 2025/04/30 06:38:02 DEBUG : empty space: Sizes identical 2025/04/30 06:38:02 DEBUG : empty space: Unchanged skipping 2025/04/30 06:38:02 DEBUG : potato3: Sizes differ (src 68 vs dst 60) 2025/04/30 06:38:02 DEBUG : Linkbox root 'rclone-test-zupolow3hepa': Waiting for checks to finish 2025/04/30 06:38:03 ERROR : potato2: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:38:03 ERROR : potato2: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:38:03 ERROR : potato3: Failed to copy: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:38:03 ERROR : potato3: Not deleting source as copy failed: failed to open source object: Open failed: HTTP error 403 (403 Forbidden) returned body: "ip incorrect" 2025/04/30 06:38:04 INFO : empty space: Deleted 2025/04/30 06:38:04 DEBUG : Linkbox root 'rclone-test-zupolow3hepa': Waiting for transfers to finish 2025/04/30 06:38:04 ERROR : Linkbox root 'rclone-test-filiwom3tufe': not deleting directories as there were IO errors sync_test.go:1716: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1716 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:1901 Error: Received unexpected error: not deleting directories as there were IO errors Test: TestServerSideMoveDeleteEmptySourceDirs 2025/04/30 06:38:04 DEBUG : Linkbox root 'rclone-test-zupolow3hepa': Purge remote --- FAIL: TestServerSideMoveDeleteEmptySourceDirs (21.63s) === RUN TestSyncConcurrentDelete run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" 2025/04/30 06:39:30 DEBUG : both0: Sizes identical 2025/04/30 06:39:30 DEBUG : both0: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both1: Sizes identical 2025/04/30 06:39:30 DEBUG : both1: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both10: Sizes identical 2025/04/30 06:39:30 DEBUG : both10: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both11: Sizes identical 2025/04/30 06:39:30 DEBUG : both11: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both12: Sizes identical 2025/04/30 06:39:30 DEBUG : both12: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both13: Sizes identical 2025/04/30 06:39:30 DEBUG : both13: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both14: Sizes identical 2025/04/30 06:39:30 DEBUG : both14: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both15: Sizes identical 2025/04/30 06:39:30 DEBUG : both15: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both16: Sizes identical 2025/04/30 06:39:30 DEBUG : both16: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both17: Sizes identical 2025/04/30 06:39:30 DEBUG : both17: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both18: Sizes identical 2025/04/30 06:39:30 DEBUG : both18: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both19: Sizes identical 2025/04/30 06:39:30 DEBUG : both19: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both2: Sizes identical 2025/04/30 06:39:30 DEBUG : both2: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both3: Sizes identical 2025/04/30 06:39:30 DEBUG : both3: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both4: Sizes identical 2025/04/30 06:39:30 DEBUG : both4: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both5: Sizes identical 2025/04/30 06:39:30 DEBUG : both5: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both6: Sizes identical 2025/04/30 06:39:30 DEBUG : both6: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both7: Sizes identical 2025/04/30 06:39:30 DEBUG : both7: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both8: Sizes identical 2025/04/30 06:39:30 DEBUG : both8: Unchanged skipping 2025/04/30 06:39:30 DEBUG : both9: Sizes identical 2025/04/30 06:39:30 DEBUG : both9: Unchanged skipping 2025/04/30 06:39:30 DEBUG : Linkbox root 'rclone-test-filiwom3tufe': Waiting for checks to finish 2025/04/30 06:39:30 DEBUG : Linkbox root 'rclone-test-filiwom3tufe': Waiting for transfers to finish 2025/04/30 06:39:30 DEBUG : Waiting for deletions to finish 2025/04/30 06:39:31 INFO : only2: Deleted 2025/04/30 06:39:32 INFO : only10: Deleted 2025/04/30 06:39:32 INFO : only4: Deleted 2025/04/30 06:39:32 INFO : only7: Deleted 2025/04/30 06:39:33 INFO : only8: Deleted 2025/04/30 06:39:33 INFO : only9: Deleted 2025/04/30 06:39:33 INFO : only0: Deleted 2025/04/30 06:39:33 INFO : only13: Deleted 2025/04/30 06:39:34 INFO : only14: Deleted 2025/04/30 06:39:34 INFO : only16: Deleted 2025/04/30 06:39:34 INFO : only18: Deleted 2025/04/30 06:39:35 INFO : only19: Deleted 2025/04/30 06:39:35 INFO : only6: Deleted 2025/04/30 06:39:36 INFO : only1: Deleted 2025/04/30 06:39:36 INFO : only11: Deleted 2025/04/30 06:39:36 INFO : only12: Deleted 2025/04/30 06:39:37 INFO : only15: Deleted 2025/04/30 06:39:37 INFO : only17: Deleted 2025/04/30 06:39:37 INFO : only3: Deleted 2025/04/30 06:39:37 INFO : only5: Deleted 2025/04/30 06:39:37 INFO : There was nothing to transfer run.go:180: Remote "Linkbox root 'rclone-test-filiwom3tufe'", Local "Local file system at /tmp/rclone3778635316", Modify Window "876000h0m0s" --- PASS: TestSyncConcurrentDelete (114.51s) FAIL 2025/04/30 06:40:04 DEBUG : Linkbox root 'rclone-test-filiwom3tufe': Purge remote "./sync.test -test.v -test.timeout 1h0m0s -remote TestLinkbox: -verbose -test.run '^(TestCopyRedownload|TestServerSideCopy|TestServerSideCopyOverSelf|TestServerSideMove|TestServerSideMoveDeleteEmptySourceDirs|TestServerSideMoveOverSelf|TestServerSideMoveWithFilter|TestSyncConcurrentDelete)$'" - Finished ERROR in 3m29.062014524s (try 2/5): exit status 1: Failed [TestServerSideCopy TestServerSideCopyOverSelf TestServerSideMoveOverSelf TestCopyRedownload TestServerSideMove TestServerSideMoveWithFilter TestServerSideMoveDeleteEmptySourceDirs]