"./bisync.test -test.v -test.timeout 1h0m0s -remote TestOneDrive: -verbose" - Starting (try 2/5) 2025/08/26 03:32:50 DEBUG : Creating backend with remote "TestOneDrive:rclone-test-sasuter0civu" 2025/08/26 03:32:50 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/08/26 03:32:50 DEBUG : TestOneDrive: Loaded invalid token from config file - ignoring 2025/08/26 03:32:51 DEBUG : Saving config "token" in section "TestOneDrive" of the config file 2025/08/26 03:32:51 DEBUG : TestOneDrive: Saved new token in config file 2025/08/26 03:32:52 DEBUG : Too many requests. Trying again in 72000 seconds. 2025/08/26 03:32:52 DEBUG : pacer: low level retry 1/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 20h0m0s) 2025/08/26 03:32:52 DEBUG : pacer: Rate limited, increasing sleep to 20h0m0s 2025/08/26 03:32:52 DEBUG : Too many requests. Trying again in 72000 seconds. 2025/08/26 03:32:52 DEBUG : pacer: low level retry 2/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 20h0m0s) 2025/08/26 04:32:50 DEBUG : OneDrive root 'rclone-test-sasuter0civu': Token expired but no uploads in progress - doing nothing 2025/08/26 23:32:52 DEBUG : Config file has changed externally - reloading 2025/08/26 23:32:52 DEBUG : TestOneDrive: Loaded invalid token from config file - ignoring 2025/08/26 23:32:52 DEBUG : TestOneDrive: Loaded new refresh token from config file 2025/08/26 23:32:52 DEBUG : Config file has changed externally - reloading 2025/08/26 23:32:52 DEBUG : Saving config "token" in section "TestOneDrive" of the config file 2025/08/26 23:32:52 DEBUG : TestOneDrive: Saved new token in config file 2025/08/26 23:32:53 DEBUG : Too many requests. Trying again in 3600 seconds. 2025/08/26 23:32:53 DEBUG : pacer: low level retry 3/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 1h0m0s) 2025/08/26 23:32:53 DEBUG : pacer: Rate limited, increasing sleep to 1h0m0s 2025/08/27 00:32:51 DEBUG : OneDrive root 'rclone-test-sasuter0civu': Token expired but no uploads in progress - doing nothing 2025/08/27 19:32:52 DEBUG : Config file has changed externally - reloading 2025/08/27 19:32:52 DEBUG : TestOneDrive: Loaded fresh token from config file 2025/08/27 19:32:52 DEBUG : TestOneDrive: Loaded new refresh token from config file 2025/08/27 19:32:52 DEBUG : Too many requests. Trying again in 3600 seconds. 2025/08/27 19:32:52 DEBUG : pacer: low level retry 4/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 1h0m0s) 2025/08/27 20:06:41 DEBUG : OneDrive root 'rclone-test-sasuter0civu': Token expired but no uploads in progress - doing nothing 2025/08/27 20:32:52 DEBUG : TestOneDrive: Loaded invalid token from config file - ignoring 2025/08/27 20:32:52 DEBUG : Saving config "token" in section "TestOneDrive" of the config file 2025/08/27 20:32:52 DEBUG : TestOneDrive: Saved new token in config file 2025/08/27 20:32:53 DEBUG : Too many requests. Trying again in 10800 seconds. 2025/08/27 20:32:53 DEBUG : pacer: low level retry 5/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 3h0m0s) 2025/08/27 20:32:53 DEBUG : pacer: Rate limited, increasing sleep to 3h0m0s 2025/08/27 21:32:51 DEBUG : OneDrive root 'rclone-test-sasuter0civu': Token expired but no uploads in progress - doing nothing 2025/08/27 21:32:52 DEBUG : TestOneDrive: Loaded invalid token from config file - ignoring 2025/08/27 21:32:52 DEBUG : Saving config "token" in section "TestOneDrive" of the config file 2025/08/27 21:32:52 DEBUG : TestOneDrive: Saved new token in config file 2025/08/27 21:32:52 DEBUG : Too many requests. Trying again in 7201 seconds. 2025/08/27 21:32:52 DEBUG : pacer: low level retry 6/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 2h0m1s) 2025/08/27 21:32:52 DEBUG : pacer: Rate limited, increasing sleep to 2h0m1s 2025/08/27 22:32:51 DEBUG : OneDrive root 'rclone-test-sasuter0civu': Token expired but no uploads in progress - doing nothing 2025/08/28 00:32:52 DEBUG : Config file has changed externally - reloading 2025/08/28 00:32:52 DEBUG : TestOneDrive: Loaded fresh token from config file 2025/08/28 00:32:52 DEBUG : TestOneDrive: Loaded new refresh token from config file 2025/08/28 00:32:52 DEBUG : Too many requests. Trying again in 20985 seconds. 2025/08/28 00:32:52 DEBUG : pacer: low level retry 7/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 5h49m45s) 2025/08/28 00:32:52 DEBUG : pacer: Rate limited, increasing sleep to 5h49m45s 2025/08/28 01:22:36 DEBUG : OneDrive root 'rclone-test-sasuter0civu': Token expired but no uploads in progress - doing nothing 2025/08/28 02:32:53 DEBUG : Config file has changed externally - reloading 2025/08/28 02:32:53 DEBUG : TestOneDrive: Loaded fresh token from config file 2025/08/28 02:32:53 DEBUG : TestOneDrive: Loaded new refresh token from config file 2025/08/28 02:32:53 DEBUG : Too many requests. Trying again in 14400 seconds. 2025/08/28 02:32:53 DEBUG : pacer: low level retry 8/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 4h0m0s) 2025/08/28 02:32:53 DEBUG : pacer: Rate limited, increasing sleep to 4h0m0s 2025/08/28 03:22:37 DEBUG : OneDrive root 'rclone-test-sasuter0civu': Token expired but no uploads in progress - doing nothing 2025/08/28 08:22:38 DEBUG : Config file has changed externally - reloading 2025/08/28 08:22:38 DEBUG : TestOneDrive: Loaded fresh token from config file 2025/08/28 08:22:38 DEBUG : TestOneDrive: Loaded new refresh token from config file 2025/08/28 08:22:38 DEBUG : pacer: Reducing sleep to 3h0m0s 2025/08/28 08:22:38 DEBUG : Creating backend with remote "/tmp/rclone547553832" === RUN TestBisyncRemoteLocal 2025/08/28 08:22:38 DEBUG : Creating backend with remote "TestOneDrive:rclone-test-muqadiz5yexi" 2025/08/28 08:22:40 NOTICE: remote: TestOneDrive:rclone-test-muqadiz5yexi === RUN TestBisyncRemoteLocal/all_changed 2025/08/28 08:22:40 DEBUG : Creating backend with remote "TestOneDrive:rclone-test-muqadiz5yexi/082240so" 2025/08/28 08:22:41 INFO : path1: Making directory 2025/08/28 08:22:42 DEBUG : Too many requests. Trying again in 18000 seconds. 2025/08/28 08:22:42 DEBUG : pacer: low level retry 1/10 (error accessDenied: throttledRequest: Too Many Requests: trying again in 5h0m0s) 2025/08/28 08:22:42 DEBUG : pacer: Rate limited, increasing sleep to 5h0m0s 2025/08/28 08:22:42 DEBUG : pacer: Reducing sleep to 3h45m0s 2025/08/28 08:57:04 DEBUG : OneDrive root 'rclone-test-sasuter0civu': Token expired but no uploads in progress - doing nothing 2025/08/28 08:57:04 DEBUG : OneDrive root 'rclone-test-muqadiz5yexi': Token expired but no uploads in progress - doing nothing panic: test timed out after 1h0m0s running tests: TestBisyncRemoteLocal (1h0m0s) TestBisyncRemoteLocal/all_changed (59m58s) goroutine 222 [running]: testing.(*M).startAlarm.func1() /usr/local/go/src/testing/testing.go:2484 +0x394 created by time.goFunc /usr/local/go/src/time/sleep.go:215 +0x2d goroutine 1 [chan receive, 60 minutes]: testing.(*T).Run(0xc00045b880, {0x27e50d5?, 0xc00006d6e0?}, 0x29109b8) /usr/local/go/src/testing/testing.go:1859 +0x431 testing.runTests.func1(0xc00045b880) /usr/local/go/src/testing/testing.go:2279 +0x37 testing.tRunner(0xc00045b880, 0xc00006d820) /usr/local/go/src/testing/testing.go:1792 +0xf4 testing.runTests(0xc0003dbaa0, {0x40727c0, 0x4, 0x4}, {0xee04208ce?, 0x40b0a40?, 0x40b0a40?}) /usr/local/go/src/testing/testing.go:2277 +0x4b4 testing.(*M).Run(0xc000640a00) /usr/local/go/src/testing/testing.go:2142 +0x64a github.com/rclone/rclone/fstest.TestMain(0xc000640a00) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:69 +0xa5 github.com/rclone/rclone/cmd/bisync_test.TestMain(0xc000640a00) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:242 +0xf2 main.main() _testmain.go:55 +0xa8 goroutine 54 [syscall, 3229 minutes]: os/signal.signal_recv() /usr/local/go/src/runtime/sigqueue.go:152 +0x29 os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 created by os/signal.Notify.func1.1 in goroutine 1 /usr/local/go/src/os/signal/signal.go:152 +0x1f goroutine 55 [chan receive, 3229 minutes]: github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1() /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27 created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xa7 goroutine 56 [chan receive, 3229 minutes]: github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1() /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27 created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xa7 goroutine 57 [select, 26 minutes]: github.com/rclone/rclone/lib/oauthutil.(*Renew).renewOnExpiry(0xc000341240) /home/rclone/go/src/github.com/rclone/rclone/lib/oauthutil/renew.go:43 +0x86 created by github.com/rclone/rclone/lib/oauthutil.NewRenew in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/lib/oauthutil/renew.go:32 +0xe5 goroutine 189 [select, 26 minutes]: github.com/rclone/rclone/lib/oauthutil.(*Renew).renewOnExpiry(0xc000434ec0) /home/rclone/go/src/github.com/rclone/rclone/lib/oauthutil/renew.go:43 +0x86 created by github.com/rclone/rclone/lib/oauthutil.NewRenew in goroutine 188 /home/rclone/go/src/github.com/rclone/rclone/lib/oauthutil/renew.go:32 +0xe5 goroutine 218 [sleep, 60 minutes]: time.Sleep(0x105ef39b2000) /usr/local/go/src/runtime/time.go:338 +0x165 github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall.func1(0xc00045b500?) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:169 +0x1c created by github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall in goroutine 237 /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:168 +0xef goroutine 248 [select]: github.com/rclone/rclone/fs/accounting.(*StatsInfo).averageLoop(0xc00078e000, {0x2c67720, 0xc00068a000}) /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/stats.go:339 +0x145 created by github.com/rclone/rclone/fs/accounting.(*StatsInfo)._startAverageLoop in goroutine 247 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/stats.go:376 +0x11a goroutine 196 [sleep, 60 minutes]: time.Sleep(0xd18c2e28000) /usr/local/go/src/runtime/time.go:338 +0x165 github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall.func1(0x0?) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:169 +0x1c created by github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:168 +0xef goroutine 188 [chan receive, 60 minutes]: testing.(*T).Run(0xc00045ba40, {0xc000a1e505?, 0x2c2cce8?}, 0xc000525a40) /usr/local/go/src/testing/testing.go:1859 +0x431 github.com/rclone/rclone/cmd/bisync_test.testBisync({0x2c676e8, 0xc0005665d0}, 0xc00045ba40, {0xc00055fda0, 0x25}, {0x0, 0x0}) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:387 +0x94f github.com/rclone/rclone/cmd/bisync_test.TestBisyncRemoteLocal(0xc00045ba40) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:255 +0x175 testing.tRunner(0xc00045ba40, 0x29109b8) /usr/local/go/src/testing/testing.go:1792 +0xf4 created by testing.(*T).Run in goroutine 1 /usr/local/go/src/testing/testing.go:1851 +0x413 goroutine 236 [chan receive, 60 minutes]: github.com/rclone/rclone/lib/atexit.Register.func1.1() /home/rclone/go/src/github.com/rclone/rclone/lib/atexit/atexit.go:45 +0x29 created by github.com/rclone/rclone/lib/atexit.Register.func1 in goroutine 188 /home/rclone/go/src/github.com/rclone/rclone/lib/atexit/atexit.go:44 +0x68 goroutine 237 [chan receive, 60 minutes]: github.com/rclone/rclone/lib/pacer.(*Pacer).beginCall(0xc000525d40) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:161 +0x25 github.com/rclone/rclone/lib/pacer.(*Pacer).call(0xc000525d40, 0xc00052b280, 0xa) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:198 +0x5c github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0xc000525d40, 0xc00052b280) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:218 +0x93 github.com/rclone/rclone/backend/onedrive.(*Fs).readMetaDataForPathRelativeToID(0xc0003a2900, {0x2c676e8, 0xc0005665d0}, {0xc0007a0ba0, 0x29}, {0xc00074e086, 0x8}) /home/rclone/go/src/github.com/rclone/rclone/backend/onedrive/onedrive.go:934 +0x2c9 github.com/rclone/rclone/backend/onedrive.(*Fs).FindLeaf(0xc0003a2900, {0x2c676e8, 0xc0005665d0}, {0xc0007a0ba0, 0x29}, {0xc00074e086, 0x8}) /home/rclone/go/src/github.com/rclone/rclone/backend/onedrive/onedrive.go:1232 +0x77 github.com/rclone/rclone/lib/dircache.(*DirCache)._findDir(0xc000191c20, {0x2c676e8, 0xc0005665d0}, {0xc00074e06d, 0x21}, 0x1) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:234 +0x112 github.com/rclone/rclone/lib/dircache.(*DirCache)._findRoot(0xc000191c20, {0x2c676e8?, 0xc0005665d0?}, 0x0?) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:305 +0x45 github.com/rclone/rclone/lib/dircache.(*DirCache).FindDir(0xc000191c20, {0x2c676e8, 0xc0005665d0}, {0x27b2517, 0x5}, 0x1) /home/rclone/go/src/github.com/rclone/rclone/lib/dircache/dircache.go:201 +0xcd github.com/rclone/rclone/backend/onedrive.(*Fs).Mkdir(0x27b2506?, {0x2c676e8?, 0xc0005665d0?}, {0x27b2517?, 0x10?}) /home/rclone/go/src/github.com/rclone/rclone/backend/onedrive/onedrive.go:1566 +0x2f github.com/rclone/rclone/fs/operations.Mkdir({0x2c676e8, 0xc0005665d0}, {0x2c7f010, 0xc0003a2900}, {0x27b2517, 0x5}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations.go:1052 +0x109 github.com/rclone/rclone/cmd/bisync_test.(*bisyncTest).makeTempRemote(0xc0001fd500, {0x2c676e8, 0xc0005665d0}, {0xc00055fda0, 0x25}, {0x27b2517, 0x5}) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:627 +0x466 github.com/rclone/rclone/cmd/bisync_test.(*bisyncTest).runTestCase(0xc0001fd500, {0x2c676e8, 0xc0005665d0}, 0x66b?, {0xc000a1e505?, 0x560200?}) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:421 +0xa5 github.com/rclone/rclone/cmd/bisync_test.testBisync.func4(0xc0001fd340) /home/rclone/go/src/github.com/rclone/rclone/cmd/bisync/bisync_test.go:394 +0x234 testing.tRunner(0xc0001fd340, 0xc000525a40) /usr/local/go/src/testing/testing.go:1792 +0xf4 created by testing.(*T).Run in goroutine 188 /usr/local/go/src/testing/testing.go:1851 +0x413 "./bisync.test -test.v -test.timeout 1h0m0s -remote TestOneDrive: -verbose" - Finished ERROR in 53h49m48.360506944s (try 2/5): exit status 2: Failed []