"./sync.test -test.v -test.timeout 1h0m0s -remote TestGoFile: -verbose -test.run '^(TestServerSideCopyOverSelf|TestSyncBackupDirWithSuffix)$'" - Starting (try 2/5) 2024/11/06 02:58:44 DEBUG : Creating backend with remote "TestGoFile:rclone-test-wuqelox0tego" 2024/11/06 02:58:44 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2024/11/06 02:58:45 DEBUG : Creating backend with remote "/tmp/rclone1555494683" === RUN TestServerSideCopyOverSelf run.go:180: Remote "gofile root 'rclone-test-wuqelox0tego'", Local "Local file system at /tmp/rclone1555494683", Modify Window "1s" 2024/11/06 02:58:45 DEBUG : Rate limited, sleep for 5s 2024/11/06 02:58:50 DEBUG : pacer: low level retry 1/10 (error Error "error-rateLimit") 2024/11/06 02:58:50 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2024/11/06 02:58:50 DEBUG : pacer: Reducing sleep to 10ms 2024/11/06 02:58:52 DEBUG : Upload server store1 (eu) responded in 121.040937ms 2024/11/06 02:58:52 DEBUG : Upload server store10 (eu) responded in 124.243913ms 2024/11/06 02:58:52 DEBUG : sub dir/hello world: Using upload server store1 (eu) 2024/11/06 02:58:53 DEBUG : Creating backend with remote "TestGoFile:rclone-test-xijugev3haje" sync_test.go:626: Server side copy (if possible) gofile root 'rclone-test-wuqelox0tego' -> gofile root 'rclone-test-xijugev3haje' 2024/11/06 02:58:54 DEBUG : Added delayed dir = "sub dir", newDst= 2024/11/06 02:58:54 DEBUG : sub dir/hello world: Need to transfer - File not found at Destination 2024/11/06 02:58:54 DEBUG : gofile root 'rclone-test-xijugev3haje': Waiting for checks to finish 2024/11/06 02:58:54 DEBUG : gofile root 'rclone-test-xijugev3haje': Waiting for transfers to finish 2024/11/06 02:58:55 DEBUG : sub dir/hello world: md5 = 5eb63bbbe01eeed093cb22bb8f5acdc3 OK 2024/11/06 02:58:55 INFO : sub dir/hello world: Copied (server-side copy) 2024/11/06 02:58:55 INFO : sub dir: Set directory modification time (using DirSetModTime) run.go:180: Remote "gofile root 'rclone-test-wuqelox0tego'", Local "Local file system at /tmp/rclone1555494683", Modify Window "1s" 2024/11/06 02:58:56 DEBUG : sub dir/hello world: Using upload server store10 (eu) 2024/11/06 02:58:56 DEBUG : sub dir/hello world: Removing old object on successful upload 2024/11/06 02:58:57 DEBUG : Added delayed dir = "sub dir", newDst=sub dir 2024/11/06 02:58:57 DEBUG : sub dir/hello world: Sizes differ (src 17 vs dst 11) 2024/11/06 02:58:57 DEBUG : gofile root 'rclone-test-xijugev3haje': Waiting for checks to finish 2024/11/06 02:58:57 DEBUG : gofile root 'rclone-test-xijugev3haje': Waiting for transfers to finish 2024/11/06 02:58:57 DEBUG : sub dir/hello world: md5 = 00321a507bdb420dbbd4f53c733b1505 OK 2024/11/06 02:58:57 INFO : sub dir/hello world: Copied (server-side copy) 2024/11/06 02:58:58 INFO : sub dir: Set directory modification time (using SetModTime) run.go:180: Remote "gofile root 'rclone-test-wuqelox0tego'", Local "Local file system at /tmp/rclone1555494683", Modify Window "1s" fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Flushing the directory cache 2024/11/06 03:01:15 DEBUG : pacer: low level retry 1/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:01:15 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2024/11/06 03:02:15 DEBUG : pacer: low level retry 2/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:02:15 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2024/11/06 03:03:15 DEBUG : pacer: low level retry 3/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:03:15 DEBUG : pacer: Rate limited, increasing sleep to 80ms 2024/11/06 03:04:15 DEBUG : pacer: low level retry 4/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:04:15 DEBUG : pacer: Rate limited, increasing sleep to 160ms 2024/11/06 03:05:15 DEBUG : pacer: low level retry 5/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:05:15 DEBUG : pacer: Rate limited, increasing sleep to 320ms 2024/11/06 03:06:15 DEBUG : pacer: low level retry 6/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:06:15 DEBUG : pacer: Rate limited, increasing sleep to 640ms 2024/11/06 03:07:15 DEBUG : pacer: low level retry 7/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:07:15 DEBUG : pacer: Rate limited, increasing sleep to 1.28s 2024/11/06 03:08:15 DEBUG : pacer: low level retry 8/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:08:15 DEBUG : pacer: Rate limited, increasing sleep to 2.56s 2024/11/06 03:09:15 DEBUG : pacer: low level retry 9/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:09:15 DEBUG : pacer: Rate limited, increasing sleep to 5.12s 2024/11/06 03:10:15 DEBUG : pacer: low level retry 10/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:10:15 DEBUG : pacer: Rate limited, increasing sleep to 10.24s fstest.go:276: Error listing: couldn't list files: Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout 2024/11/06 03:10:15 DEBUG : gofile root 'rclone-test-xijugev3haje': Purge remote 2024/11/06 03:11:15 DEBUG : pacer: low level retry 1/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:11:15 DEBUG : pacer: Rate limited, increasing sleep to 20s 2024/11/06 03:12:15 DEBUG : pacer: low level retry 2/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:13:15 DEBUG : pacer: low level retry 3/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:14:15 DEBUG : pacer: low level retry 4/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:15:15 DEBUG : pacer: low level retry 5/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:16:15 DEBUG : pacer: low level retry 6/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:17:15 DEBUG : pacer: low level retry 7/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:18:15 DEBUG : pacer: low level retry 8/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:19:15 DEBUG : pacer: low level retry 9/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:20:15 DEBUG : pacer: low level retry 10/10 (error Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:20:15 NOTICE: purge failed: couldn't list files: Get "https://api.gofile.io/contents/3af76dee-601b-4f7f-984c-339db8f7ffd2?contentname=rclone-test-xijugev3haje&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout 2024/11/06 03:21:15 DEBUG : pacer: low level retry 1/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:21:15 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2024/11/06 03:22:15 DEBUG : pacer: low level retry 2/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:22:15 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2024/11/06 03:23:15 DEBUG : pacer: low level retry 3/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:23:15 DEBUG : pacer: Rate limited, increasing sleep to 80ms 2024/11/06 03:24:15 DEBUG : pacer: low level retry 4/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:24:15 DEBUG : pacer: Rate limited, increasing sleep to 160ms 2024/11/06 03:25:15 DEBUG : pacer: low level retry 5/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:25:15 DEBUG : pacer: Rate limited, increasing sleep to 320ms 2024/11/06 03:26:15 DEBUG : pacer: low level retry 6/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:26:15 DEBUG : pacer: Rate limited, increasing sleep to 640ms 2024/11/06 03:27:15 DEBUG : pacer: low level retry 7/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:27:15 DEBUG : pacer: Rate limited, increasing sleep to 1.28s 2024/11/06 03:28:15 DEBUG : pacer: low level retry 8/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:28:15 DEBUG : pacer: Rate limited, increasing sleep to 2.56s 2024/11/06 03:29:15 DEBUG : pacer: low level retry 9/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:29:15 DEBUG : pacer: Rate limited, increasing sleep to 5.12s 2024/11/06 03:30:15 DEBUG : pacer: low level retry 10/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:30:15 DEBUG : pacer: Rate limited, increasing sleep to 10.24s run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1175 /usr/local/go/src/testing/testing.go:1353 /usr/local/go/src/testing/testing.go:1683 /usr/local/go/src/runtime/panic.go:626 /usr/local/go/src/testing/testing.go:1005 /usr/local/go/src/testing/testing.go:1089 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:276 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:641 Error: Received unexpected error: couldn't recursively list files: Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout Test: TestServerSideCopyOverSelf 2024/11/06 03:31:15 DEBUG : pacer: low level retry 1/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:31:15 DEBUG : pacer: Rate limited, increasing sleep to 20s 2024/11/06 03:32:15 DEBUG : pacer: low level retry 2/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:33:15 DEBUG : pacer: low level retry 3/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:34:15 DEBUG : pacer: low level retry 4/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:35:15 DEBUG : pacer: low level retry 5/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:36:15 DEBUG : pacer: low level retry 6/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:37:15 DEBUG : pacer: low level retry 7/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:38:15 DEBUG : pacer: low level retry 8/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:39:15 DEBUG : pacer: low level retry 9/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:40:15 DEBUG : pacer: low level retry 10/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1175 /usr/local/go/src/testing/testing.go:1353 /usr/local/go/src/testing/testing.go:1337 /usr/local/go/src/runtime/panic.go:626 /usr/local/go/src/testing/testing.go:1005 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1175 /usr/local/go/src/testing/testing.go:1353 /usr/local/go/src/testing/testing.go:1683 /usr/local/go/src/runtime/panic.go:626 /usr/local/go/src/testing/testing.go:1005 /usr/local/go/src/testing/testing.go:1089 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:276 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:641 Error: Received unexpected error: couldn't recursively list files: Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout Test: TestServerSideCopyOverSelf 2024/11/06 03:41:15 DEBUG : pacer: low level retry 1/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:42:15 DEBUG : pacer: low level retry 2/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:43:15 DEBUG : pacer: low level retry 3/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:44:15 DEBUG : pacer: low level retry 4/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:45:15 DEBUG : pacer: low level retry 5/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:46:15 DEBUG : pacer: low level retry 6/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:47:15 DEBUG : pacer: low level retry 7/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:48:15 DEBUG : pacer: low level retry 8/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:49:15 DEBUG : pacer: low level retry 9/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:50:15 DEBUG : pacer: low level retry 10/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1175 /usr/local/go/src/testing/testing.go:1353 /usr/local/go/src/testing/testing.go:1337 /usr/local/go/src/runtime/panic.go:626 /usr/local/go/src/testing/testing.go:1005 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1175 /usr/local/go/src/testing/testing.go:1353 /usr/local/go/src/testing/testing.go:1337 /usr/local/go/src/runtime/panic.go:626 /usr/local/go/src/testing/testing.go:1005 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1175 /usr/local/go/src/testing/testing.go:1353 /usr/local/go/src/testing/testing.go:1683 /usr/local/go/src/runtime/panic.go:626 /usr/local/go/src/testing/testing.go:1005 /usr/local/go/src/testing/testing.go:1089 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:276 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:641 Error: Received unexpected error: couldn't recursively list files: Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?maxdepth=4&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout Test: TestServerSideCopyOverSelf --- FAIL: TestServerSideCopyOverSelf (3090.15s) === RUN TestSyncBackupDirWithSuffix run.go:180: Remote "gofile root 'rclone-test-wuqelox0tego'", Local "Local file system at /tmp/rclone1555494683", Modify Window "1s" 2024/11/06 03:51:15 DEBUG : pacer: low level retry 1/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?contentname=dst&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:52:15 DEBUG : pacer: low level retry 2/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?contentname=dst&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:53:15 DEBUG : pacer: low level retry 3/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?contentname=dst&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:54:15 DEBUG : pacer: low level retry 4/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?contentname=dst&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:55:15 DEBUG : pacer: low level retry 5/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?contentname=dst&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:56:15 DEBUG : pacer: low level retry 6/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?contentname=dst&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:57:15 DEBUG : pacer: low level retry 7/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?contentname=dst&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) 2024/11/06 03:58:15 DEBUG : pacer: low level retry 8/10 (error Get "https://api.gofile.io/contents/d2fb3911-2790-4d58-955e-ce72152679c0?contentname=dst&page=1&pageSize=1000": dial tcp 45.112.123.126:443: i/o timeout) panic: test timed out after 1h0m0s running tests: TestSyncBackupDirWithSuffix (8m30s) goroutine 681 [running]: testing.(*M).startAlarm.func1() /usr/local/go/src/testing/testing.go:2366 +0x385 created by time.goFunc /usr/local/go/src/time/sleep.go:177 +0x2d goroutine 1 [chan receive, 9 minutes]: testing.(*T).Run(0xc0000d4000, {0x2350c97?, 0x0?}, 0x24505e0) /usr/local/go/src/testing/testing.go:1750 +0x3ab testing.runTests.func1(0xc0000d4000) /usr/local/go/src/testing/testing.go:2161 +0x37 testing.tRunner(0xc0000d4000, 0xc00088fc00) /usr/local/go/src/testing/testing.go:1689 +0xfb testing.runTests(0xc000538d50, {0x3883f00, 0x5a, 0x5a}, {0x1?, 0xedebcd265?, 0x3898ec0?}) /usr/local/go/src/testing/testing.go:2159 +0x445 testing.(*M).Run(0xc000630b40) /usr/local/go/src/testing/testing.go:2027 +0x68b github.com/rclone/rclone/fstest.TestMain(0xc000630b40) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:69 +0xa5 github.com/rclone/rclone/fs/sync.TestMain(...) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:44 main.main() _testmain.go:227 +0x196 goroutine 21 [select]: go.opencensus.io/stats/view.(*worker).start(0xc00031d580) /home/rclone/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:292 +0x9f created by go.opencensus.io/stats/view.init.0 in goroutine 1 /home/rclone/go/pkg/mod/go.opencensus.io@v0.24.0/stats/view/worker.go:34 +0x8d goroutine 23 [syscall, 60 minutes]: os/signal.signal_recv() /usr/local/go/src/runtime/sigqueue.go:152 +0x29 os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 created by os/signal.Notify.func1.1 in goroutine 1 /usr/local/go/src/os/signal/signal.go:151 +0x1f goroutine 24 [chan receive, 60 minutes]: github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1() /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27 created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xab goroutine 613 [select]: net/http.(*Transport).getConn(0xc0002603c0, 0xc0004f6000, {{}, 0x0, {0xc00042f7a0, 0x5}, {0xc0000dc048, 0x11}, 0x0}) /usr/local/go/src/net/http/transport.go:1406 +0x5a5 net/http.(*Transport).roundTrip(0xc0002603c0, 0xc00036e000) /usr/local/go/src/net/http/transport.go:595 +0x73a net/http.(*Transport).RoundTrip(...) /usr/local/go/src/net/http/roundtrip.go:17 github.com/rclone/rclone/fs/fshttp.(*Transport).RoundTrip(0xc0002e6150, 0xc00036e000) /home/rclone/go/src/github.com/rclone/rclone/fs/fshttp/http.go:352 +0x345 net/http.send(0xc00036e000, {0x26d8b00, 0xc0002e6150}, {0xc0006ca801?, 0x41c385?, 0x0?}) /usr/local/go/src/net/http/client.go:259 +0x5e4 net/http.(*Client).send(0xc0005c3f50, 0xc00036e000, {0x0?, 0x6ca928?, 0x0?}) /usr/local/go/src/net/http/client.go:180 +0x98 net/http.(*Client).do(0xc0005c3f50, 0xc00036e000) /usr/local/go/src/net/http/client.go:724 +0x8dc net/http.(*Client).Do(...) /usr/local/go/src/net/http/client.go:590 github.com/rclone/rclone/lib/rest.(*Client).Call(0xc0000ba0f0, {0x26fc688, 0xc0004cd710}, 0xc0004d6140) /home/rclone/go/src/github.com/rclone/rclone/lib/rest/rest.go:325 +0xcc5 github.com/rclone/rclone/lib/rest.(*Client).callCodec(0xc0000ba0f0, {0x26fc688, 0xc0004cd710}, 0x412530?, {0x0?, 0x0?}, {0x1f1d680, 0xc000780000}, 0x40c7cb?, 0x2451b00, ...) /home/rclone/go/src/github.com/rclone/rclone/lib/rest/rest.go:522 +0x3e5 github.com/rclone/rclone/lib/rest.(*Client).CallJSON(...) /home/rclone/go/src/github.com/rclone/rclone/lib/rest/rest.go:465 github.com/rclone/rclone/backend/gofile.(*Fs).listAll.func1() /home/rclone/go/src/github.com/rclone/rclone/backend/gofile/gofile.go:642 +0x85 github.com/rclone/rclone/fs.pacerInvoker(0x9, 0xa, 0x26d7ba0?) /home/rclone/go/src/github.com/rclone/rclone/fs/pacer.go:88 +0x32 github.com/rclone/rclone/lib/pacer.(*Pacer).call(0xc00057e5a0, 0xc000936400, 0xa) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:197 +0x7c github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0xc00057e5a0, 0xc000936400) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:216 +0x92 github.com/rclone/rclone/backend/gofile.(*Fs).listAll(0xc0001989c0, {0x26fc688, 0xc0004cd710}, {0xc000694540, 0x24}, 0x1, 0x0, {0x231857b, 0x3}, 0xc0006cb618) /home/rclone/go/src/github.com/rclone/rclone/backend/gofile/gofile.go:641 +0x536 github.com/rclone/rclone/backend/gofile.(*Fs).FindLeaf(0x231857b?, {0x26fc688?, 0xc0004cd710?}, {0xc000694540?, 0x3555400?}, {0x231857b?, 0xd0?}) /home/rclone/go/src/github.com/rclone/rclone/backend/gofile/gofile.go:568 +0x85 github.com/rclone/rclone/lib/dircache.(*DirCache)._findDir(0xc000151560, {0x26fc688, 0xc0004cd710}, {0x231857b, 0x3}, 0x0) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:234 +0x112 github.com/rclone/rclone/lib/dircache.(*DirCache).FindDir(0xc000151560, {0x26fc688, 0xc0004cd710}, {0x231857b, 0x3}, 0x0) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:205 +0x152 github.com/rclone/rclone/lib/dircache.(*DirCache).FindPath(0xc000151560, {0x26fc688, 0xc0004cd710}, {0x231857b?, 0xc000ac1850?}, 0x0) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:275 +0xac github.com/rclone/rclone/backend/gofile.(*Fs).readMetaDataForPath(0xc0001989c0, {0x26fc688, 0xc0004cd710}, {0x231857b?, 0x414385?}) /home/rclone/go/src/github.com/rclone/rclone/backend/gofile/gofile.go:230 +0x4f github.com/rclone/rclone/backend/gofile.(*Object).readMetaData(0xc000532870, {0x26fc688?, 0xc0004cd710?}) /home/rclone/go/src/github.com/rclone/rclone/backend/gofile/gofile.go:1434 +0x4e github.com/rclone/rclone/backend/gofile.(*Fs).newObjectWithInfo(0xc0001989c0, {0x26fc688, 0xc0004cd710}, {0x231857b, 0x7}, 0x0) /home/rclone/go/src/github.com/rclone/rclone/backend/gofile/gofile.go:551 +0x9c github.com/rclone/rclone/backend/gofile.(*Fs).NewObject(...) /home/rclone/go/src/github.com/rclone/rclone/backend/gofile/gofile.go:562 github.com/rclone/rclone/backend/gofile.(*Fs).Put(0xc0001989c0, {0x26fc688, 0xc0004cd710}, {0x26d7560, 0xc0004cd950}, {0x2705c60, 0xc00047ec40}, {0x0, 0x0, 0x0}) /home/rclone/go/src/github.com/rclone/rclone/backend/gofile/gofile.go:857 +0x79 github.com/rclone/rclone/fstest.(*Run).WriteObjectTo(0xc00047eaf0, {0x26fc688, 0xc0004cd710}, {0x270f930, 0xc0001989c0}, {0x231857b, 0x7}, {0x23133d8, 0x3}, {0x1dcd64ff, ...}, ...) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:275 +0x418 github.com/rclone/rclone/fstest.(*Run).WriteObject(...) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 github.com/rclone/rclone/fs/sync.testSyncBackupDir(0xc0000d4680, {0x2316434, 0x6}, {0x2313d36, 0x4}, 0x0) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2371 +0x3e8 github.com/rclone/rclone/fs/sync.TestSyncBackupDirWithSuffix(0xc0000d4680?) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:2431 +0x2e testing.tRunner(0xc0000d4680, 0x24505e0) /usr/local/go/src/testing/testing.go:1689 +0xfb created by testing.(*T).Run in goroutine 1 /usr/local/go/src/testing/testing.go:1742 +0x390 goroutine 661 [IO wait]: internal/poll.runtime_pollWait(0x7f3f2836aed0, 0x77) /usr/local/go/src/runtime/netpoll.go:345 +0x85 internal/poll.(*pollDesc).wait(0xc0004da080?, 0x0?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 internal/poll.(*pollDesc).waitWrite(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:93 internal/poll.(*FD).WaitWrite(...) /usr/local/go/src/internal/poll/fd_unix.go:683 net.(*netFD).connect(0xc0004da080, {0x26fc730, 0xc000478000}, {0xc000ac1198?, 0x414145?}, {0x26d77a0?, 0xc000876000?}) /usr/local/go/src/net/fd_unix.go:141 +0x70b net.(*netFD).dial(0xc0004da080, {0x26fc730, 0xc000478000}, {0x2702138?, 0x0?}, {0x2702138, 0xc000a96000}, 0x10?) /usr/local/go/src/net/sock_posix.go:124 +0x3bc net.socket({0x26fc730, 0xc000478000}, {0x231353d, 0x3}, 0x2, 0x1, 0xc00044e040?, 0x0, {0x2702138, 0x0}, ...) /usr/local/go/src/net/sock_posix.go:70 +0x29b net.internetSocket({0x26fc730, 0xc000478000}, {0x231353d, 0x3}, {0x2702138, 0x0}, {0x2702138, 0xc000a96000}, 0x1, 0x0, ...) /usr/local/go/src/net/ipsock_posix.go:154 +0xf8 net.(*sysDialer).doDialTCPProto(0xc0004a0140, {0x26fc730, 0xc000478000}, 0x0, 0xc000a96000, 0x0) /usr/local/go/src/net/tcpsock_posix.go:85 +0xec net.(*sysDialer).doDialTCP(...) /usr/local/go/src/net/tcpsock_posix.go:75 net.(*sysDialer).dialTCP(0x421556?, {0x26fc730?, 0xc000478000?}, 0x414145?, 0x7f3f70635f18?) /usr/local/go/src/net/tcpsock_posix.go:71 +0x65 net.(*sysDialer).dialSingle(0xc0004a0140, {0x26fc730, 0xc000478000}, {0x26e9f18, 0xc000a96000}) /usr/local/go/src/net/dial.go:651 +0x27d net.(*sysDialer).dialSerial(0xc0004a0140, {0x26fc730, 0xc000478000}, {0xc000192050?, 0x1, 0xc000192050?}) /usr/local/go/src/net/dial.go:616 +0x24e net.(*sysDialer).dialParallel(0xc000192010?, {0x26fc730?, 0xc000478000?}, {0xc000192050?, 0xc000478000?, 0x2313f36?}, {0x0?, 0x231353d?, 0xc0000dc090?}) /usr/local/go/src/net/dial.go:517 +0x3b4 net.(*Dialer).DialContext(0xc0001cdb80, {0x26fc688, 0xc0004cd710}, {0x231353d, 0x3}, {0xc0000dc048, 0x11}) /usr/local/go/src/net/dial.go:508 +0x69a github.com/rclone/rclone/fs/fshttp.(*Dialer).DialContext(0xc0001cdb80, {0x26fc688, 0xc0004cd710}, {0x231353d?, 0x60?}, {0xc0000dc048, 0x11}) /home/rclone/go/src/github.com/rclone/rclone/fs/fshttp/dialer.go:62 +0x125 github.com/rclone/rclone/fs/fshttp.NewTransportCustom.func1({0x26fc688, 0xc0004cd710}, {0x231353d, 0x3}, {0xc0000dc048, 0x11}) /home/rclone/go/src/github.com/rclone/rclone/fs/fshttp/http.go:109 +0x65 net/http.(*Transport).dial(0x0?, {0x26fc688?, 0xc0004cd710?}, {0x231353d?, 0x0?}, {0xc0000dc048?, 0x0?}) /usr/local/go/src/net/http/transport.go:1187 +0xd2 net/http.(*Transport).dialConn(0xc0002603c0, {0x26fc688, 0xc0004cd710}, {{}, 0x0, {0xc00042f7a0, 0x5}, {0xc0000dc048, 0x11}, 0x0}) /usr/local/go/src/net/http/transport.go:1647 +0x7e8 net/http.(*Transport).dialConnFor(0xc0002603c0, 0xc00044a0b0) /usr/local/go/src/net/http/transport.go:1484 +0xca created by net/http.(*Transport).queueForDial in goroutine 613 /usr/local/go/src/net/http/transport.go:1449 +0x3c9 goroutine 511 [select]: net.(*netFD).connect.func2() /usr/local/go/src/net/fd_unix.go:118 +0x7a created by net.(*netFD).connect in goroutine 661 /usr/local/go/src/net/fd_unix.go:117 +0x37c "./sync.test -test.v -test.timeout 1h0m0s -remote TestGoFile: -verbose -test.run '^(TestServerSideCopyOverSelf|TestSyncBackupDirWithSuffix)$'" - Finished ERROR in 1h0m1.049870243s (try 2/5): exit status 2: Failed [TestServerSideCopyOverSelf]