"./sync.test -test.v -test.timeout 1h0m0s -remote TestOneDrive: -verbose" - Starting (try 1/5) 2025/08/02 02:16:42 DEBUG : Creating backend with remote "TestOneDrive:rclone-test-difiril9guwa" 2025/08/02 02:16:42 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/08/02 02:16:43 DEBUG : Too many requests. Trying again in 3600 seconds. 2025/08/02 02:16:43 DEBUG : pacer: low level retry 1/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 1h0m0s) 2025/08/02 02:16:43 DEBUG : pacer: Rate limited, increasing sleep to 1h0m0s 2025/08/02 02:16:44 DEBUG : Too many requests. Trying again in 3600 seconds. 2025/08/02 02:16:44 DEBUG : pacer: low level retry 2/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 1h0m0s) 2025/08/02 03:12:51 DEBUG : OneDrive root 'rclone-test-difiril9guwa': Token expired but no uploads in progress - doing nothing 2025/08/02 03:16:43 DEBUG : Config file has changed externally - reloading 2025/08/02 03:16:43 DEBUG : TestOneDrive: Loaded fresh token from config file 2025/08/02 03:16:43 DEBUG : TestOneDrive: Loaded new refresh token from config file 2025/08/02 03:16:44 DEBUG : pacer: Reducing sleep to 45m0s 2025/08/02 04:12:49 DEBUG : OneDrive root 'rclone-test-difiril9guwa': Token expired but no uploads in progress - doing nothing 2025/08/02 04:16:43 DEBUG : Config file has changed externally - reloading 2025/08/02 04:16:43 DEBUG : TestOneDrive: Loaded fresh token from config file 2025/08/02 04:16:43 DEBUG : TestOneDrive: Loaded new refresh token from config file 2025/08/02 04:16:44 DEBUG : pacer: Reducing sleep to 33m45s 2025/08/02 05:01:44 DEBUG : pacer: Reducing sleep to 25m18.75s 2025/08/02 05:01:44 DEBUG : Creating backend with remote "/tmp/rclone25238680" 2025/08/02 05:01:44 DEBUG : Config file has changed externally - reloading === RUN TestPipe --- PASS: TestPipe (0.00s) === RUN TestPipeConcurrent --- PASS: TestPipeConcurrent (0.05s) === RUN TestPipeOrderBy === RUN TestPipeOrderBy/#00 === RUN TestPipeOrderBy/size === RUN TestPipeOrderBy/name === RUN TestPipeOrderBy/modtime === RUN TestPipeOrderBy/size,ascending === RUN TestPipeOrderBy/name,asc === RUN TestPipeOrderBy/modtime,ascending === RUN TestPipeOrderBy/size,descending === RUN TestPipeOrderBy/name,desc === RUN TestPipeOrderBy/modtime,descending === RUN TestPipeOrderBy/size,mixed,50 === RUN TestPipeOrderBy/size,mixed,51 --- PASS: TestPipeOrderBy (0.00s) --- PASS: TestPipeOrderBy/#00 (0.00s) --- PASS: TestPipeOrderBy/size (0.00s) --- PASS: TestPipeOrderBy/name (0.00s) --- PASS: TestPipeOrderBy/modtime (0.00s) --- PASS: TestPipeOrderBy/size,ascending (0.00s) --- PASS: TestPipeOrderBy/name,asc (0.00s) --- PASS: TestPipeOrderBy/modtime,ascending (0.00s) --- PASS: TestPipeOrderBy/size,descending (0.00s) --- PASS: TestPipeOrderBy/name,desc (0.00s) --- PASS: TestPipeOrderBy/modtime,descending (0.00s) --- PASS: TestPipeOrderBy/size,mixed,50 (0.00s) --- PASS: TestPipeOrderBy/size,mixed,51 (0.00s) === RUN TestNewLess === RUN TestNewLess/blankOK === RUN TestNewLess/tooManyParts === RUN TestNewLess/tooManyParts2 === RUN TestNewLess/badMixed === RUN TestNewLess/unknownComparison === RUN TestNewLess/unknownSortDirection === RUN TestNewLess/size === RUN TestNewLess/name === RUN TestNewLess/modtime === RUN TestNewLess/size,ascending === RUN TestNewLess/name,asc === RUN TestNewLess/modtime,ascending === RUN TestNewLess/size,descending === RUN TestNewLess/name,desc === RUN TestNewLess/modtime,descending === RUN TestNewLess/modtime,mixed === RUN TestNewLess/modtime,mixed,30 --- PASS: TestNewLess (0.00s) --- PASS: TestNewLess/blankOK (0.00s) --- PASS: TestNewLess/tooManyParts (0.00s) --- PASS: TestNewLess/tooManyParts2 (0.00s) --- PASS: TestNewLess/badMixed (0.00s) --- PASS: TestNewLess/unknownComparison (0.00s) --- PASS: TestNewLess/unknownSortDirection (0.00s) --- PASS: TestNewLess/size (0.00s) --- PASS: TestNewLess/name (0.00s) --- PASS: TestNewLess/modtime (0.00s) --- PASS: TestNewLess/size,ascending (0.00s) --- PASS: TestNewLess/name,asc (0.00s) --- PASS: TestNewLess/modtime,ascending (0.00s) --- PASS: TestNewLess/size,descending (0.00s) --- PASS: TestNewLess/name,desc (0.00s) --- PASS: TestNewLess/modtime,descending (0.00s) --- PASS: TestNewLess/modtime,mixed (0.00s) --- PASS: TestNewLess/modtime,mixed,30 (0.00s) === RUN TestRcCopy rc_test.go:16: Skipping test on non local remote --- SKIP: TestRcCopy (0.00s) === RUN TestRcMove rc_test.go:16: Skipping test on non local remote --- SKIP: TestRcMove (0.00s) === RUN TestRcSync rc_test.go:16: Skipping test on non local remote --- SKIP: TestRcSync (0.00s) === RUN TestCopyWithDryRun run.go:180: Remote "OneDrive root 'rclone-test-difiril9guwa'", Local "Local file system at /tmp/rclone25238680", Modify Window "1s" 2025/08/02 05:12:49 DEBUG : OneDrive root 'rclone-test-difiril9guwa': Token expired but no uploads in progress - doing nothing 2025/08/02 05:35:28 DEBUG : Config file has changed externally - reloading 2025/08/02 05:35:28 DEBUG : TestOneDrive: Loaded fresh token from config file 2025/08/02 05:35:28 DEBUG : TestOneDrive: Loaded new refresh token from config file 2025/08/02 05:35:29 DEBUG : pacer: Reducing sleep to 18m59.0625s 2025/08/02 06:00:48 DEBUG : Too many requests. Trying again in 64800 seconds. 2025/08/02 06:00:48 DEBUG : pacer: low level retry 1/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 18h0m0s) 2025/08/02 06:00:48 DEBUG : pacer: Rate limited, increasing sleep to 18h0m0s panic: test timed out after 1h0m0s running tests: TestCopyWithDryRun (1h0m0s) goroutine 178 [running]: testing.(*M).startAlarm.func1() /usr/local/go/src/testing/testing.go:2484 +0x394 created by time.goFunc /usr/local/go/src/time/sleep.go:215 +0x2d goroutine 1 [chan receive, 61 minutes]: testing.(*T).Run(0xc000855340, {0x26b18b0?, 0xc000069ad8?}, 0x27cfb60) /usr/local/go/src/testing/testing.go:1859 +0x431 testing.runTests.func1(0xc000855340) /usr/local/go/src/testing/testing.go:2279 +0x37 testing.tRunner(0xc000855340, 0xc000069c18) /usr/local/go/src/testing/testing.go:1792 +0xf4 testing.runTests(0xc0009b2eb8, {0x3e95540, 0x6c, 0x6c}, {0xee01f92b8?, 0x3eab5a0?, 0x3eab5a0?}) /usr/local/go/src/testing/testing.go:2277 +0x4b4 testing.(*M).Run(0xc000463860) /usr/local/go/src/testing/testing.go:2142 +0x64a github.com/rclone/rclone/fstest.TestMain(0xc000463860) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:69 +0xa5 github.com/rclone/rclone/fs/sync.TestMain(...) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:45 main.main() _testmain.go:261 +0xa9 goroutine 35 [syscall, 225 minutes]: os/signal.signal_recv() /usr/local/go/src/runtime/sigqueue.go:152 +0x29 os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 created by os/signal.Notify.func1.1 in goroutine 1 /usr/local/go/src/os/signal/signal.go:152 +0x1f goroutine 36 [chan receive, 225 minutes]: github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1() /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27 created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xa7 goroutine 37 [select, 48 minutes]: github.com/rclone/rclone/lib/oauthutil.(*Renew).renewOnExpiry(0xc000397bc0) /home/rclone/go/src/github.com/rclone/rclone/lib/oauthutil/renew.go:43 +0x86 created by github.com/rclone/rclone/lib/oauthutil.NewRenew in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/lib/oauthutil/renew.go:32 +0xe5 goroutine 151 [chan receive]: github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall(0xc0002dd620) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:161 +0x25 github.com/rclone/rclone/lib/pacer.(*Pacer).call(0xc0002dd620, 0xc0006b01e0, 0xa) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:198 +0x5c github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0xc0002dd620, 0xc0006b01e0) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:218 +0x93 github.com/rclone/rclone/backend/onedrive.(*Fs).CreateDir(0xc0005a9680, {0x2b06130, 0xc0003d1440}, {0xc0009b80c0, 0x25}, {0xc00051a00d, 0x18}) /home/rclone/go/src/github.com/rclone/rclone/backend/onedrive/onedrive.go:1258 +0x2ec github.com/rclone/rclone/lib/dircache.(*DirCache)._findDir(0xc000558090, {0x2b06130, 0xc0003d1440}, {0xc00051a00d, 0x18}, 0x1) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:242 +0x172 github.com/rclone/rclone/lib/dircache.(*DirCache)._findRoot(0xc000558090, {0x2b06130?, 0xc0003d1440?}, 0xa0?) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:305 +0x45 github.com/rclone/rclone/lib/dircache.(*DirCache).FindDir(0xc000558090, {0x2b06130, 0xc0003d1440}, {0x0, 0x0}, 0x1) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:201 +0xcd github.com/rclone/rclone/backend/onedrive.(*Fs).Mkdir(0x26b2948?, {0x2b06130?, 0xc0003d1440?}, {0x0?, 0x1?}) /home/rclone/go/src/github.com/rclone/rclone/backend/onedrive/onedrive.go:1566 +0x2f github.com/rclone/rclone/fstest.(*Run).ForceMkdir(0xc000444d90, {0x2b06130?, 0xc0003d1440?}, {0x2b1c9a0, 0xc0005a9680}) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:235 +0x38 github.com/rclone/rclone/fstest.(*Run).Mkdir(0xc000444d90, {0x2b06130, 0xc0003d1440}, {0x2b1c9a0, 0xc0005a9680}) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:245 +0x71 github.com/rclone/rclone/fs/sync.TestCopyWithDryRun(0xc0007e6380) /home/rclone/go/src/github.com/rclone/rclone/fs/sync/sync_test.go:54 +0x145 testing.tRunner(0xc0007e6380, 0x27cfb60) /usr/local/go/src/testing/testing.go:1792 +0xf4 created by testing.(*T).Run in goroutine 1 /usr/local/go/src/testing/testing.go:1851 +0x413 goroutine 168 [sleep]: time.Sleep(0x109356a6aa0) /usr/local/go/src/runtime/time.go:338 +0x165 github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall.func1(0x0?) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:169 +0x1c created by github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall in goroutine 151 /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:168 +0xef goroutine 79 [IO wait]: internal/poll.runtime_pollWait(0x71fccb4c87f0, 0x72) /usr/local/go/src/runtime/netpoll.go:351 +0x85 internal/poll.(*pollDesc).wait(0xc000525880?, 0xc0005e9300?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc000525880, {0xc0005e9300, 0x1300, 0x1300}) /usr/local/go/src/internal/poll/fd_unix.go:165 +0x27a net.(*netFD).Read(0xc000525880, {0xc0005e9300?, 0x106d?, 0xc0003946e0?}) /usr/local/go/src/net/fd_posix.go:55 +0x25 net.(*conn).Read(0xc000088070, {0xc0005e9300?, 0xc0005e958e?, 0x5?}) /usr/local/go/src/net/net.go:194 +0x45 github.com/rclone/rclone/fs/fshttp.(*timeoutConn).Read(0xc0003324e0, {0xc0005e9300?, 0x71fcca48af58?, 0x71fd129a2a78?}) /home/rclone/go/src/github.com/rclone/rclone/fs/fshttp/dialer.go:111 +0x29 crypto/tls.(*atLeastReader).Read(0xc000333290, {0xc0005e9300?, 0x106d?, 0xc0002ec008?}) /usr/local/go/src/crypto/tls/conn.go:809 +0x3b bytes.(*Buffer).ReadFrom(0xc0001489b8, {0x2ae0b60, 0xc000333290}) /usr/local/go/src/bytes/buffer.go:211 +0x98 crypto/tls.(*Conn).readFromUntil(0xc000148708, {0x71fcc92df098, 0xc0003324e0}, 0x4426d4?) /usr/local/go/src/crypto/tls/conn.go:831 +0xde crypto/tls.(*Conn).readRecordOrCCS(0xc000148708, 0x0) /usr/local/go/src/crypto/tls/conn.go:629 +0x3cf crypto/tls.(*Conn).readRecord(...) /usr/local/go/src/crypto/tls/conn.go:591 crypto/tls.(*Conn).Read(0xc000148708, {0xc000371000, 0x1000, 0xc000095ca0?}) /usr/local/go/src/crypto/tls/conn.go:1385 +0x145 bufio.(*Reader).Read(0xc000a2c960, {0xc0009cc200, 0x9, 0x3e23110?}) /usr/local/go/src/bufio/bufio.go:245 +0x197 io.ReadAtLeast({0x2ade060, 0xc000a2c960}, {0xc0009cc200, 0x9, 0x9}, 0x9) /usr/local/go/src/io/io.go:335 +0x91 io.ReadFull(...) /usr/local/go/src/io/io.go:354 net/http.http2readFrameHeader({0xc0009cc200, 0x9, 0xc0003c8450?}, {0x2ade060?, 0xc000a2c960?}) /usr/local/go/src/net/http/h2_bundle.go:1805 +0x65 net/http.(*http2Framer).ReadFrame(0xc0009cc1c0) /usr/local/go/src/net/http/h2_bundle.go:2072 +0x7d net/http.(*http2clientConnReadLoop).run(0xc000095fa8) /usr/local/go/src/net/http/h2_bundle.go:9933 +0xda net/http.(*http2ClientConn).readLoop(0xc0007e6540) /usr/local/go/src/net/http/h2_bundle.go:9812 +0x79 created by net/http.(*http2Transport).newClientConn in goroutine 78 /usr/local/go/src/net/http/h2_bundle.go:8334 +0xde5 "./sync.test -test.v -test.timeout 1h0m0s -remote TestOneDrive: -verbose" - Finished ERROR in 3h45m1.646571621s (try 1/5): exit status 2: Failed []