"./gitannex.test -test.v -test.timeout 2h0m0s -remote TestDrime: -verbose -test.run '^TestGitAnnexFstestBackendCases$/^Remove$'" - Starting (try 3/5) 2026/02/02 05:22:22 DEBUG : Creating backend with remote "TestDrime:rclone-test-refonoj4laba" 2026/02/02 05:22:22 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2026/02/02 05:22:23 DEBUG : Creating backend with remote "/tmp/rclone3687135515" === RUN TestGitAnnexFstestBackendCases === RUN TestGitAnnexFstestBackendCases/Remove run.go:185: Remote "drime root 'rclone-test-refonoj4laba'", Local "Local file system at /tmp/rclone3687135515", Modify Window "876000h0m0s" 2026/02/02 05:22:23 DEBUG : Creating backend with remote "TestDrime:rclone-test-refonoj4laba" 2026/02/02 05:22:24 DEBUG : Creating backend with remote "/tmp/rclone3687135515" 2026/02/02 05:22:24 DEBUG : file.txt: Need to transfer - File not found at Destination 2026/02/02 05:22:25 DEBUG : SomeKey: size = 5 OK 2026/02/02 05:22:25 INFO : file.txt: Copied (new) to: SomeKey 2026/02/02 05:22:25 ERROR : error listing: directory not found fstest.go:298: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:301: Flushing the directory cache 2026/02/02 05:22:27 ERROR : error listing: directory not found fstest.go:298: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:301: Flushing the directory cache 2026/02/02 05:22:29 ERROR : error listing: directory not found fstest.go:298: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:301: Flushing the directory cache fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:345 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1308 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1446 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestGitAnnexFstestBackendCases/Remove Messages: listing wrong, want SomeKey (5) got fstest.go:203: Not found "SomeKey" fstest.go:206: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:206 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:310 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:345 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1308 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1446 Error: Not equal: expected: 0 actual : 1 Test: TestGitAnnexFstestBackendCases/Remove Messages: 1 objects not found 2026/02/02 05:22:33 INFO : SomeKey: Deleted fstest.go:298: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:301: Flushing the directory cache fstest.go:298: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:301: Flushing the directory cache fstest.go:298: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:301: Flushing the directory cache fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:345 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:272 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1316 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1446 Error: Should be true Test: TestGitAnnexFstestBackendCases/Remove Messages: listing wrong, want got SomeKey (5) fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:345 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:272 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1316 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1446 Error: Should be true Test: TestGitAnnexFstestBackendCases/Remove Messages: Unexpected file "SomeKey" gitannex_test.go:308: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:308 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1319 /home/rclone/go/src/github.com/rclone/rclone/cmd/gitannex/gitannex_test.go:1446 Error: Not equal: expected: "CHECKPRESENT-FAILURE SomeKey\n" actual : "CHECKPRESENT-SUCCESS SomeKey\n" Diff: --- Expected +++ Actual @@ -1,2 +1,2 @@ -CHECKPRESENT-FAILURE SomeKey +CHECKPRESENT-SUCCESS SomeKey Test: TestGitAnnexFstestBackendCases/Remove 2026/02/02 05:22:43 DEBUG : pacer: low level retry 1/10 (error Error "Server Error") 2026/02/02 05:22:43 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2026/02/02 05:22:43 DEBUG : pacer: low level retry 2/10 (error Error "Server Error") 2026/02/02 05:22:43 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2026/02/02 05:22:44 DEBUG : pacer: low level retry 3/10 (error Error "Server Error") 2026/02/02 05:22:44 DEBUG : pacer: Rate limited, increasing sleep to 80ms 2026/02/02 05:22:44 DEBUG : pacer: low level retry 4/10 (error Error "Server Error") 2026/02/02 05:22:44 DEBUG : pacer: Rate limited, increasing sleep to 160ms 2026/02/02 05:22:44 DEBUG : pacer: low level retry 5/10 (error Error "Server Error") 2026/02/02 05:22:44 DEBUG : pacer: Rate limited, increasing sleep to 320ms 2026/02/02 05:22:44 DEBUG : pacer: low level retry 6/10 (error Error "Server Error") 2026/02/02 05:22:44 DEBUG : pacer: Rate limited, increasing sleep to 640ms 2026/02/02 05:22:45 DEBUG : pacer: low level retry 7/10 (error Error "Server Error") 2026/02/02 05:22:45 DEBUG : pacer: Rate limited, increasing sleep to 1.28s 2026/02/02 05:22:45 DEBUG : pacer: low level retry 8/10 (error Error "Server Error") 2026/02/02 05:22:45 DEBUG : pacer: Rate limited, increasing sleep to 2.56s 2026/02/02 05:22:47 DEBUG : pacer: low level retry 9/10 (error Error "Server Error") 2026/02/02 05:22:47 DEBUG : pacer: Rate limited, increasing sleep to 5.12s 2026/02/02 05:22:49 DEBUG : pacer: low level retry 10/10 (error Error "Server Error") 2026/02/02 05:22:49 DEBUG : pacer: Rate limited, increasing sleep to 10.24s run.go:130: removing file "SomeKey" failed - try 1/3: failed to delete item: Error "Server Error" 2026/02/02 05:22:54 DEBUG : pacer: low level retry 1/10 (error Error "Server Error") 2026/02/02 05:22:54 DEBUG : pacer: Rate limited, increasing sleep to 20s 2026/02/02 05:23:05 DEBUG : pacer: low level retry 2/10 (error Error "Server Error") 2026/02/02 05:23:25 DEBUG : pacer: low level retry 3/10 (error Error "Server Error") 2026/02/02 05:23:45 DEBUG : pacer: low level retry 4/10 (error Error "Server Error") 2026/02/02 05:24:05 DEBUG : pacer: low level retry 5/10 (error Error "Server Error") 2026/02/02 05:24:25 DEBUG : pacer: low level retry 6/10 (error Error "Server Error") 2026/02/02 05:24:45 DEBUG : pacer: low level retry 7/10 (error Error "Server Error") 2026/02/02 05:25:05 DEBUG : pacer: low level retry 8/10 (error Error "Server Error") 2026/02/02 05:25:25 DEBUG : pacer: low level retry 9/10 (error Error "Server Error") 2026/02/02 05:25:45 DEBUG : pacer: low level retry 10/10 (error Error "Server Error") run.go:130: removing file "SomeKey" failed - try 2/3: failed to delete item: Error "Server Error" 2026/02/02 05:26:05 DEBUG : pacer: low level retry 1/10 (error Error "Server Error") 2026/02/02 05:26:25 DEBUG : pacer: low level retry 2/10 (error Error "Server Error") 2026/02/02 05:26:45 DEBUG : pacer: low level retry 3/10 (error Error "Server Error") 2026/02/02 05:27:05 DEBUG : pacer: low level retry 4/10 (error Error "Server Error") 2026/02/02 05:27:25 DEBUG : pacer: low level retry 5/10 (error Error "Server Error") 2026/02/02 05:27:45 DEBUG : pacer: low level retry 6/10 (error Error "Server Error") 2026/02/02 05:28:05 DEBUG : pacer: low level retry 7/10 (error Error "Server Error") 2026/02/02 05:28:25 DEBUG : pacer: low level retry 8/10 (error Error "Server Error") 2026/02/02 05:28:45 DEBUG : pacer: low level retry 9/10 (error Error "Server Error") 2026/02/02 05:29:05 DEBUG : pacer: low level retry 10/10 (error Error "Server Error") run.go:130: removing file "SomeKey" failed - try 3/3: failed to delete item: Error "Server Error" run.go:133: removing file "SomeKey" failed: failed to delete item: Error "Server Error" 2026/02/02 05:29:25 DEBUG : pacer: Reducing sleep to 10s 2026/02/02 05:29:45 DEBUG : pacer: Reducing sleep to 5s 2026/02/02 05:29:55 DEBUG : pacer: Reducing sleep to 2.5s --- FAIL: TestGitAnnexFstestBackendCases (452.19s) --- FAIL: TestGitAnnexFstestBackendCases/Remove (452.19s) FAIL 2026/02/02 05:29:55 DEBUG : drime root 'rclone-test-refonoj4laba': Purge remote 2026/02/02 05:30:00 DEBUG : pacer: Reducing sleep to 1.25s "./gitannex.test -test.v -test.timeout 2h0m0s -remote TestDrime: -verbose -test.run '^TestGitAnnexFstestBackendCases$/^Remove$'" - Finished ERROR in 7m37.740124502s (try 3/5): exit status 1: Failed [TestGitAnnexFstestBackendCases/Remove]