"./gitannex.test -test.v -test.timeout 1h0m0s -remote TestOneDriveBusiness: -verbose -test.run '^TestGitAnnexFstestBackendCases$/^(CheckPresentAndTransfer|TransferStorePathWithInteriorWhitespace)$'" - Starting (try 2/5) 2025/04/09 05:09:27 DEBUG : Creating backend with remote "TestOneDriveBusiness:rclone-test-dujoduc6raya" 2025/04/09 05:09:27 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/04/09 05:09:29 DEBUG : Creating backend with remote "/tmp/rclone2938444102" === RUN TestGitAnnexFstestBackendCases === RUN TestGitAnnexFstestBackendCases/TransferStorePathWithInteriorWhitespace run.go:180: Remote "OneDrive root 'rclone-test-dujoduc6raya'", Local "Local file system at /tmp/rclone2938444102", Modify Window "1s" 2025/04/09 05:09:29 DEBUG : Creating backend with remote "TestOneDriveBusiness:rclone-test-dujoduc6raya" 2025/04/09 05:09:31 DEBUG : Creating backend with remote "../../rclone2938444102" 2025/04/09 05:09:31 DEBUG : fs cache: renaming cache item "../../rclone2938444102" to be canonical "/tmp/rclone2938444102" 2025/04/09 05:09:32 DEBUG : filename with spaces.txt: Need to transfer - File not found at Destination 2025/04/09 05:09:33 DEBUG : KeyRelative: Starting multipart upload 2025/04/09 05:09:34 DEBUG : KeyRelative: Uploading segment 0/5 size 5 2025/04/09 05:09:37 DEBUG : filename with spaces.txt: quickxor = 4828021398f00400000000000500000000000000 OK 2025/04/09 05:09:37 INFO : filename with spaces.txt: Copied (new) to: KeyRelative 2025/04/09 05:09:38 DEBUG : Too many requests. Trying again in 18 seconds. 2025/04/09 05:09:38 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:09:38 DEBUG : pacer: Rate limited, increasing sleep to 18s 2025/04/09 05:09:39 DEBUG : pacer: Reducing sleep to 13.5s 2025/04/09 05:09:56 DEBUG : Too many requests. Trying again in 27 seconds. 2025/04/09 05:09:56 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:09:56 DEBUG : pacer: Rate limited, increasing sleep to 27s gitannex_test.go:319: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:319 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1051 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1405 Error: Received unexpected error: attempt to read line timed out: context deadline exceeded Test: TestGitAnnexFstestBackendCases/TransferStorePathWithInteriorWhitespace 2025/04/09 05:10:08 DEBUG : Too many requests. Trying again in 24 seconds. 2025/04/09 05:10:08 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:10:08 DEBUG : pacer: Rate limited, increasing sleep to 24s 2025/04/09 05:10:08 DEBUG : Too many requests. Trying again in 15 seconds. 2025/04/09 05:10:08 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:10:08 DEBUG : pacer: Rate limited, increasing sleep to 15s 2025/04/09 05:10:10 DEBUG : Too many requests. Trying again in 27 seconds. 2025/04/09 05:10:10 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:10:32 DEBUG : Too many requests. Trying again in 26 seconds. 2025/04/09 05:10:32 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:10:32 DEBUG : pacer: Rate limited, increasing sleep to 26s 2025/04/09 05:10:37 DEBUG : Too many requests. Trying again in 17 seconds. 2025/04/09 05:10:37 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:10:37 DEBUG : pacer: Rate limited, increasing sleep to 17s 2025/04/09 05:10:47 DEBUG : Too many requests. Trying again in 26 seconds. 2025/04/09 05:10:47 DEBUG : pacer: low level retry 4/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:11:04 DEBUG : Too many requests. Trying again in 20 seconds. 2025/04/09 05:11:04 DEBUG : pacer: low level retry 4/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:11:04 DEBUG : pacer: Rate limited, increasing sleep to 20s 2025/04/09 05:11:13 DEBUG : Too many requests. Trying again in 18 seconds. 2025/04/09 05:11:13 DEBUG : pacer: low level retry 5/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:11:13 DEBUG : pacer: Rate limited, increasing sleep to 18s 2025/04/09 05:11:21 DEBUG : Too many requests. Trying again in 20 seconds. 2025/04/09 05:11:21 DEBUG : pacer: low level retry 5/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:11:39 DEBUG : Too many requests. Trying again in 29 seconds. 2025/04/09 05:11:39 DEBUG : pacer: low level retry 6/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:11:39 DEBUG : pacer: Rate limited, increasing sleep to 29s 2025/04/09 05:11:41 DEBUG : Too many requests. Trying again in 17 seconds. 2025/04/09 05:11:41 DEBUG : pacer: low level retry 6/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:11:41 DEBUG : pacer: Rate limited, increasing sleep to 17s 2025/04/09 05:11:57 DEBUG : Too many requests. Trying again in 23 seconds. 2025/04/09 05:11:57 DEBUG : pacer: low level retry 7/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:11:57 DEBUG : pacer: Rate limited, increasing sleep to 23s 2025/04/09 05:12:01 DEBUG : Too many requests. Trying again in 22 seconds. 2025/04/09 05:12:01 DEBUG : pacer: low level retry 7/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:12:01 DEBUG : pacer: Rate limited, increasing sleep to 22s 2025/04/09 05:12:18 DEBUG : Too many requests. Trying again in 21 seconds. 2025/04/09 05:12:18 DEBUG : pacer: low level retry 8/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:12:18 DEBUG : pacer: Rate limited, increasing sleep to 21s 2025/04/09 05:12:26 DEBUG : Too many requests. Trying again in 18 seconds. 2025/04/09 05:12:26 DEBUG : pacer: low level retry 8/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:12:26 DEBUG : pacer: Rate limited, increasing sleep to 18s 2025/04/09 05:12:40 DEBUG : Too many requests. Trying again in 24 seconds. 2025/04/09 05:12:40 DEBUG : pacer: low level retry 9/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:12:40 DEBUG : pacer: Rate limited, increasing sleep to 24s 2025/04/09 05:12:49 DEBUG : Too many requests. Trying again in 22 seconds. 2025/04/09 05:12:49 DEBUG : pacer: low level retry 9/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:12:49 DEBUG : pacer: Rate limited, increasing sleep to 22s 2025/04/09 05:13:01 DEBUG : Too many requests. Trying again in 26 seconds. 2025/04/09 05:13:01 DEBUG : pacer: low level retry 10/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:13:01 DEBUG : pacer: Rate limited, increasing sleep to 26s 2025/04/09 05:13:07 DEBUG : Too many requests. Trying again in 15 seconds. 2025/04/09 05:13:07 DEBUG : pacer: low level retry 10/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:13:07 DEBUG : pacer: Rate limited, increasing sleep to 15s 2025/04/09 05:13:07 ERROR : : error listing: couldn't list files: serviceNotAvailable: Service unavailable run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1376 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1684 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/panic.go:629 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1006 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:319 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1051 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1405 Error: Received unexpected error: couldn't list files: serviceNotAvailable: Service unavailable Test: TestGitAnnexFstestBackendCases/TransferStorePathWithInteriorWhitespace 2025/04/09 05:13:29 DEBUG : Too many requests. Trying again in 27 seconds. 2025/04/09 05:13:29 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:13:29 DEBUG : pacer: Rate limited, increasing sleep to 27s 2025/04/09 05:13:44 DEBUG : Too many requests. Trying again in 17 seconds. 2025/04/09 05:13:44 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:13:44 DEBUG : pacer: Rate limited, increasing sleep to 17s 2025/04/09 05:14:11 DEBUG : Too many requests. Trying again in 17 seconds. 2025/04/09 05:14:11 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:14:28 DEBUG : Too many requests. Trying again in 23 seconds. 2025/04/09 05:14:28 DEBUG : pacer: low level retry 4/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:14:28 DEBUG : pacer: Rate limited, increasing sleep to 23s 2025/04/09 05:14:45 DEBUG : Too many requests. Trying again in 21 seconds. 2025/04/09 05:14:45 DEBUG : pacer: low level retry 5/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:14:45 DEBUG : pacer: Rate limited, increasing sleep to 21s 2025/04/09 05:15:08 DEBUG : Too many requests. Trying again in 27 seconds. 2025/04/09 05:15:08 DEBUG : pacer: low level retry 6/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:15:08 DEBUG : pacer: Rate limited, increasing sleep to 27s 2025/04/09 05:15:29 DEBUG : Too many requests. Trying again in 26 seconds. 2025/04/09 05:15:29 DEBUG : pacer: low level retry 7/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:15:29 DEBUG : pacer: Rate limited, increasing sleep to 26s 2025/04/09 05:15:56 DEBUG : Too many requests. Trying again in 21 seconds. 2025/04/09 05:15:56 DEBUG : pacer: low level retry 8/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:15:56 DEBUG : pacer: Rate limited, increasing sleep to 21s 2025/04/09 05:16:22 DEBUG : Too many requests. Trying again in 22 seconds. 2025/04/09 05:16:22 DEBUG : pacer: low level retry 9/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:16:22 DEBUG : pacer: Rate limited, increasing sleep to 22s 2025/04/09 05:16:43 DEBUG : Too many requests. Trying again in 23 seconds. 2025/04/09 05:16:43 DEBUG : pacer: low level retry 10/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:16:43 DEBUG : pacer: Rate limited, increasing sleep to 23s 2025/04/09 05:16:43 ERROR : : error listing: couldn't list files: serviceNotAvailable: Service unavailable run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1338 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/panic.go:629 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1006 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1376 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1176 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1354 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1684 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/runtime/panic.go:629 /home/rclone/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.23.0.linux-amd64/src/testing/testing.go:1006 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:319 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1051 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1405 Error: Received unexpected error: couldn't list files: serviceNotAvailable: Service unavailable Test: TestGitAnnexFstestBackendCases/TransferStorePathWithInteriorWhitespace === RUN TestGitAnnexFstestBackendCases/CheckPresentAndTransfer run.go:180: Remote "OneDrive root 'rclone-test-dujoduc6raya'", Local "Local file system at /tmp/rclone2938444102", Modify Window "1s" 2025/04/09 05:16:43 DEBUG : Creating backend with remote "TestOneDriveBusiness:rclone-test-dujoduc6raya" 2025/04/09 05:16:44 DEBUG : Too many requests. Trying again in 19 seconds. 2025/04/09 05:16:44 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:16:44 DEBUG : pacer: Rate limited, increasing sleep to 19s 2025/04/09 05:16:45 DEBUG : Too many requests. Trying again in 18 seconds. 2025/04/09 05:16:45 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:16:45 DEBUG : pacer: Rate limited, increasing sleep to 18s 2025/04/09 05:17:03 DEBUG : Too many requests. Trying again in 19 seconds. 2025/04/09 05:17:03 DEBUG : pacer: low level retry 3/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:17:03 DEBUG : pacer: Rate limited, increasing sleep to 19s gitannex_test.go:319: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:319 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1074 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1405 Error: Received unexpected error: attempt to read line timed out: context deadline exceeded Test: TestGitAnnexFstestBackendCases/CheckPresentAndTransfer 2025/04/09 05:17:14 DEBUG : Too many requests. Trying again in 28 seconds. 2025/04/09 05:17:14 DEBUG : pacer: low level retry 1/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:17:14 DEBUG : pacer: Rate limited, increasing sleep to 28s 2025/04/09 05:17:21 DEBUG : Too many requests. Trying again in 29 seconds. 2025/04/09 05:17:21 DEBUG : pacer: low level retry 4/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:17:21 DEBUG : pacer: Rate limited, increasing sleep to 29s 2025/04/09 05:17:37 DEBUG : Too many requests. Trying again in 28 seconds. 2025/04/09 05:17:37 DEBUG : pacer: low level retry 2/10 (error serviceNotAvailable: Service unavailable) 2025/04/09 05:17:40 DEBUG : pacer: Reducing sleep to 21.75s 2025/04/09 05:18:05 DEBUG : pacer: Reducing sleep to 21s 2025/04/09 05:18:10 DEBUG : pacer: Reducing sleep to 16.3125s 2025/04/09 05:18:31 DEBUG : pacer: Reducing sleep to 12.234375s 2025/04/09 05:18:33 DEBUG : pacer: Reducing sleep to 15.75s 2025/04/09 05:18:48 DEBUG : pacer: Reducing sleep to 9.17578125s 2025/04/09 05:18:54 DEBUG : pacer: Reducing sleep to 11.8125s 2025/04/09 05:19:09 DEBUG : pacer: Reducing sleep to 8.859375s 2025/04/09 05:19:21 DEBUG : pacer: Reducing sleep to 6.64453125s --- FAIL: TestGitAnnexFstestBackendCases (592.04s) --- FAIL: TestGitAnnexFstestBackendCases/TransferStorePathWithInteriorWhitespace (434.20s) --- FAIL: TestGitAnnexFstestBackendCases/CheckPresentAndTransfer (157.84s) FAIL 2025/04/09 05:19:21 DEBUG : OneDrive root 'rclone-test-dujoduc6raya': Purge remote 2025/04/09 05:19:30 DEBUG : pacer: Reducing sleep to 4.983398437s "./gitannex.test -test.v -test.timeout 1h0m0s -remote TestOneDriveBusiness: -verbose -test.run '^TestGitAnnexFstestBackendCases$/^(CheckPresentAndTransfer|TransferStorePathWithInteriorWhitespace)$'" - Finished ERROR in 10m3.214616405s (try 2/5): exit status 1: Failed [TestGitAnnexFstestBackendCases/TransferStorePathWithInteriorWhitespace TestGitAnnexFstestBackendCases/CheckPresentAndTransfer]