"./operations.test -test.v -test.timeout 1h0m0s -remote TestWebdavInfiniteScale: -verbose -test.run '^(TestCheckSumDownload|TestParseSumFile)$|^TestCheckDownload$/^(1|2|3|4|5|6|7)$'" - Starting (try 3/5) 2026/01/05 02:33:41 DEBUG : Creating backend with remote "TestWebdavInfiniteScale:rclone-test-zapumec9folu" 2026/01/05 02:33:41 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2026/01/05 02:33:41 DEBUG : Setting type="webdav" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_TYPE 2026/01/05 02:33:41 DEBUG : Setting url="https://127.0.0.1:28639/dav/spaces/some-admin-user-id-0000-100000000000" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_URL 2026/01/05 02:33:41 DEBUG : Setting vendor="infinitescale" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_VENDOR 2026/01/05 02:33:41 DEBUG : Setting user="admin" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_USER 2026/01/05 02:33:41 DEBUG : Setting pass="wtB0iBNtXFss5AbDF1IEZKHQb7wn" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_PASS 2026/01/05 02:33:41 DEBUG : TestWebdavInfiniteScale: detected overridden config - adding "{7VgIw}" suffix to name 2026/01/05 02:33:41 DEBUG : Setting url="https://127.0.0.1:28639/dav/spaces/some-admin-user-id-0000-100000000000" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_URL 2026/01/05 02:33:41 DEBUG : Setting vendor="infinitescale" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_VENDOR 2026/01/05 02:33:41 DEBUG : Setting user="admin" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_USER 2026/01/05 02:33:41 DEBUG : Setting pass="wtB0iBNtXFss5AbDF1IEZKHQb7wn" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_PASS 2026/01/05 02:33:41 DEBUG : found headers: 2026/01/05 02:33:41 DEBUG : Creating backend with remote "/tmp/rclone2822229241" === RUN TestCheckDownload run.go:185: Remote "webdav root 'rclone-test-zapumec9folu'", Local "Local file system at /tmp/rclone2822229241", Modify Window "1s" 2026/01/05 02:33:41 DEBUG : rutabaga: Update will use the tus protocol to upload 2026/01/05 02:33:41 INFO : rutabaga: Failed to read metadata: object not found 2026/01/05 02:33:41 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 02:33:41 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 8 2026/01/05 02:33:41 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished === RUN TestCheckDownload/1 check_test.go:86: 1: Expecting 0 errors but got 2 check_test.go:89: 1: Total files matching line missing === NAME TestCheckDownload check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:56 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "= rutabaga"} actual : []string{"", "! rutabaga"} Diff: --- Expected +++ Actual @@ -2,3 +2,3 @@ (string) "", - (string) (len=10) "= rutabaga" + (string) (len=10) "! rutabaga" } Test: TestCheckDownload Messages: combined check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:59 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "rutabaga"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,4 +1,2 @@ -([]string) (len=2) { - (string) "", - (string) (len=8) "rutabaga" +([]string) { } Test: TestCheckDownload Messages: match check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:61 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{} actual : []string{"", "rutabaga"} Diff: --- Expected +++ Actual @@ -1,2 +1,4 @@ -([]string) { +([]string) (len=2) { + (string) "", + (string) (len=8) "rutabaga" } Test: TestCheckDownload Messages: error === RUN TestCheckDownload/2 check_test.go:86: 2: Expecting 1 errors but got 3 check_test.go:89: 2: Total files matching line missing === NAME TestCheckDownload check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:56 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "+ potato2", "= rutabaga"} actual : []string{"", "! rutabaga", "+ potato2"} Diff: --- Expected +++ Actual @@ -2,4 +2,4 @@ (string) "", - (string) (len=9) "+ potato2", - (string) (len=10) "= rutabaga" + (string) (len=10) "! rutabaga", + (string) (len=9) "+ potato2" } Test: TestCheckDownload Messages: combined check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:59 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "rutabaga"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,4 +1,2 @@ -([]string) (len=2) { - (string) "", - (string) (len=8) "rutabaga" +([]string) { } Test: TestCheckDownload Messages: match check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:61 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{} actual : []string{"", "rutabaga"} Diff: --- Expected +++ Actual @@ -1,2 +1,4 @@ -([]string) { +([]string) (len=2) { + (string) "", + (string) (len=8) "rutabaga" } Test: TestCheckDownload Messages: error 2026/01/05 02:40:10 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 02:40:10 DEBUG : empty space: Update will use the tus protocol to upload 2026/01/05 02:40:12 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 02:40:12 INFO : empty space: Failed to read metadata: object not found 2026/01/05 02:40:13 DEBUG : pacer: Reducing sleep to 843.75ms 2026/01/05 02:40:13 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 02:40:14 DEBUG : pacer: Reducing sleep to 632.8125ms 2026/01/05 02:40:14 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 1 2026/01/05 02:40:14 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 02:40:15 DEBUG : pacer: Reducing sleep to 474.609375ms 2026/01/05 02:40:16 DEBUG : pacer: Reducing sleep to 355.957031ms === RUN TestCheckDownload/3 check_test.go:86: 3: Expecting 2 errors but got 4 check_test.go:89: 3: Total files matching line missing === NAME TestCheckDownload check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:56 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "+ potato2", "- empty space", "= rutabaga"} actual : []string{"", "! rutabaga", "+ potato2", "- empty space"} Diff: --- Expected +++ Actual @@ -2,5 +2,5 @@ (string) "", + (string) (len=10) "! rutabaga", (string) (len=9) "+ potato2", - (string) (len=13) "- empty space", - (string) (len=10) "= rutabaga" + (string) (len=13) "- empty space" } Test: TestCheckDownload Messages: combined check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:59 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "rutabaga"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,4 +1,2 @@ -([]string) (len=2) { - (string) "", - (string) (len=8) "rutabaga" +([]string) { } Test: TestCheckDownload Messages: match check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:61 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{} actual : []string{"", "rutabaga"} Diff: --- Expected +++ Actual @@ -1,2 +1,4 @@ -([]string) { +([]string) (len=2) { + (string) "", + (string) (len=8) "rutabaga" } Test: TestCheckDownload Messages: error 2026/01/05 02:43:33 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 02:43:33 DEBUG : potato2: Update will use the tus protocol to upload 2026/01/05 02:43:35 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 02:43:35 INFO : potato2: Failed to read metadata: object not found 2026/01/05 02:43:36 DEBUG : pacer: Reducing sleep to 843.75ms 2026/01/05 02:43:36 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 02:43:37 DEBUG : pacer: Reducing sleep to 632.8125ms 2026/01/05 02:43:37 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 60 2026/01/05 02:43:37 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 02:43:38 DEBUG : pacer: Reducing sleep to 474.609375ms 2026/01/05 02:43:39 DEBUG : pacer: Reducing sleep to 355.957031ms === RUN TestCheckDownload/4 check_test.go:86: 4: Expecting 1 errors but got 5 check_test.go:89: 4: Total files matching line missing === NAME TestCheckDownload check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:56 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "- empty space", "= potato2", "= rutabaga"} actual : []string{"", "! potato2", "! rutabaga", "- empty space"} Diff: --- Expected +++ Actual @@ -2,5 +2,5 @@ (string) "", - (string) (len=13) "- empty space", - (string) (len=9) "= potato2", - (string) (len=10) "= rutabaga" + (string) (len=9) "! potato2", + (string) (len=10) "! rutabaga", + (string) (len=13) "- empty space" } Test: TestCheckDownload Messages: combined check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:59 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "potato2", "rutabaga"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,5 +1,2 @@ -([]string) (len=3) { - (string) "", - (string) (len=7) "potato2", - (string) (len=8) "rutabaga" +([]string) { } Test: TestCheckDownload Messages: match check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:61 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{} actual : []string{"", "potato2", "rutabaga"} Diff: --- Expected +++ Actual @@ -1,2 +1,5 @@ -([]string) { +([]string) (len=3) { + (string) "", + (string) (len=7) "potato2", + (string) (len=8) "rutabaga" } Test: TestCheckDownload Messages: error === RUN TestCheckDownload/5 check_test.go:86: 5: Expecting 1 errors but got 5 check_test.go:89: 5: Total files matching line missing === NAME TestCheckDownload check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:56 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "* empty space", "= potato2", "= rutabaga"} actual : []string{"", "! potato2", "! rutabaga", "* empty space"} Diff: --- Expected +++ Actual @@ -2,5 +2,5 @@ (string) "", - (string) (len=13) "* empty space", - (string) (len=9) "= potato2", - (string) (len=10) "= rutabaga" + (string) (len=9) "! potato2", + (string) (len=10) "! rutabaga", + (string) (len=13) "* empty space" } Test: TestCheckDownload Messages: combined check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:59 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "potato2", "rutabaga"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,5 +1,2 @@ -([]string) (len=3) { - (string) "", - (string) (len=7) "potato2", - (string) (len=8) "rutabaga" +([]string) { } Test: TestCheckDownload Messages: match check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:61 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{} actual : []string{"", "potato2", "rutabaga"} Diff: --- Expected +++ Actual @@ -1,2 +1,5 @@ -([]string) { +([]string) (len=3) { + (string) "", + (string) (len=7) "potato2", + (string) (len=8) "rutabaga" } Test: TestCheckDownload Messages: error 2026/01/05 02:56:57 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 02:56:57 DEBUG : remotepotato: Update will use the tus protocol to upload 2026/01/05 02:56:59 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 02:56:59 INFO : remotepotato: Failed to read metadata: object not found 2026/01/05 02:57:01 DEBUG : pacer: Reducing sleep to 843.75ms 2026/01/05 02:57:01 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 02:57:02 DEBUG : pacer: Reducing sleep to 632.8125ms 2026/01/05 02:57:02 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 60 2026/01/05 02:57:02 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 02:57:03 DEBUG : pacer: Reducing sleep to 474.609375ms 2026/01/05 02:57:03 DEBUG : pacer: Reducing sleep to 355.957031ms === RUN TestCheckDownload/6 check_test.go:86: 6: Expecting 2 errors but got 6 check_test.go:89: 6: Total files matching line missing === NAME TestCheckDownload check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:56 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "* empty space", "- remotepotato", "= potato2", "= rutabaga"} actual : []string{"", "! potato2", "! rutabaga", "* empty space", "- remotepotato"} Diff: --- Expected +++ Actual @@ -2,6 +2,6 @@ (string) "", + (string) (len=9) "! potato2", + (string) (len=10) "! rutabaga", (string) (len=13) "* empty space", - (string) (len=14) "- remotepotato", - (string) (len=9) "= potato2", - (string) (len=10) "= rutabaga" + (string) (len=14) "- remotepotato" } Test: TestCheckDownload Messages: combined check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:59 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "potato2", "rutabaga"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,5 +1,2 @@ -([]string) (len=3) { - (string) "", - (string) (len=7) "potato2", - (string) (len=8) "rutabaga" +([]string) { } Test: TestCheckDownload Messages: match check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:61 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{} actual : []string{"", "potato2", "rutabaga"} Diff: --- Expected +++ Actual @@ -1,2 +1,5 @@ -([]string) { +([]string) (len=3) { + (string) "", + (string) (len=7) "potato2", + (string) (len=8) "rutabaga" } Test: TestCheckDownload Messages: error === RUN TestCheckDownload/7 check_test.go:86: 7: Expecting 1 errors but got 5 check_test.go:89: 7: Total files matching line missing === NAME TestCheckDownload check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:56 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "* empty space", "= potato2", "= rutabaga"} actual : []string{"", "! potato2", "! rutabaga", "* empty space"} Diff: --- Expected +++ Actual @@ -2,5 +2,5 @@ (string) "", - (string) (len=13) "* empty space", - (string) (len=9) "= potato2", - (string) (len=10) "= rutabaga" + (string) (len=9) "! potato2", + (string) (len=10) "! rutabaga", + (string) (len=13) "* empty space" } Test: TestCheckDownload Messages: combined check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:59 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{"", "potato2", "rutabaga"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,5 +1,2 @@ -([]string) (len=3) { - (string) "", - (string) (len=7) "potato2", - (string) (len=8) "rutabaga" +([]string) { } Test: TestCheckDownload Messages: match check_test.go:52: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:52 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:61 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:94 Error: Not equal: expected: []string{} actual : []string{"", "potato2", "rutabaga"} Diff: --- Expected +++ Actual @@ -1,2 +1,5 @@ -([]string) { +([]string) (len=3) { + (string) "", + (string) (len=7) "potato2", + (string) (len=8) "rutabaga" } Test: TestCheckDownload Messages: error 2026/01/05 03:10:22 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 03:10:24 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 03:10:25 DEBUG : pacer: Reducing sleep to 843.75ms 2026/01/05 03:10:26 DEBUG : pacer: Reducing sleep to 632.8125ms 2026/01/05 03:10:27 DEBUG : pacer: Reducing sleep to 474.609375ms 2026/01/05 03:10:28 DEBUG : pacer: Reducing sleep to 355.957031ms --- FAIL: TestCheckDownload (2206.95s) --- FAIL: TestCheckDownload/1 (184.86s) --- FAIL: TestCheckDownload/2 (201.55s) --- FAIL: TestCheckDownload/3 (194.73s) --- FAIL: TestCheckDownload/4 (394.84s) --- FAIL: TestCheckDownload/5 (401.62s) --- FAIL: TestCheckDownload/6 (394.86s) --- FAIL: TestCheckDownload/7 (401.62s) === RUN TestParseSumFile run.go:185: Remote "webdav root 'rclone-test-zapumec9folu'", Local "Local file system at /tmp/rclone2822229241", Modify Window "1s" 2026/01/05 03:10:28 DEBUG : pacer: Reducing sleep to 266.967773ms 2026/01/05 03:10:28 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:10:29 DEBUG : pacer: Reducing sleep to 200.225829ms 2026/01/05 03:10:29 INFO : test.sum: Failed to read metadata: object not found 2026/01/05 03:10:29 DEBUG : pacer: Reducing sleep to 150.169371ms 2026/01/05 03:10:29 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:10:29 DEBUG : pacer: Reducing sleep to 112.627028ms 2026/01/05 03:10:29 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 81 2026/01/05 03:10:29 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 03:10:29 DEBUG : pacer: Reducing sleep to 84.470271ms 2026/01/05 03:10:29 DEBUG : pacer: Reducing sleep to 63.352703ms 2026/01/05 03:10:30 DEBUG : pacer: low level retry 1/10 (error 500 Internal Server Error) 2026/01/05 03:10:30 DEBUG : pacer: Rate limited, increasing sleep to 126.705406ms 2026/01/05 03:10:30 DEBUG : pacer: low level retry 2/10 (error 500 Internal Server Error) 2026/01/05 03:10:30 DEBUG : pacer: Rate limited, increasing sleep to 253.410812ms 2026/01/05 03:10:30 DEBUG : pacer: low level retry 3/10 (error 500 Internal Server Error) 2026/01/05 03:10:30 DEBUG : pacer: Rate limited, increasing sleep to 506.821624ms 2026/01/05 03:10:30 DEBUG : pacer: low level retry 4/10 (error 500 Internal Server Error) 2026/01/05 03:10:30 DEBUG : pacer: Rate limited, increasing sleep to 1.013643248s 2026/01/05 03:10:31 DEBUG : pacer: low level retry 5/10 (error 500 Internal Server Error) 2026/01/05 03:10:31 DEBUG : pacer: Rate limited, increasing sleep to 2s 2026/01/05 03:10:31 DEBUG : pacer: low level retry 6/10 (error 500 Internal Server Error) 2026/01/05 03:10:34 DEBUG : pacer: low level retry 7/10 (error 500 Internal Server Error) 2026/01/05 03:10:35 DEBUG : pacer: low level retry 8/10 (error 500 Internal Server Error) 2026/01/05 03:10:37 DEBUG : pacer: low level retry 9/10 (error 500 Internal Server Error) 2026/01/05 03:10:39 DEBUG : pacer: low level retry 10/10 (error 500 Internal Server Error) check_test.go:311: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:311 Error: Received unexpected error: 500 Internal Server Error Test: TestParseSumFile check_test.go:313: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:313 Error: Not equal: expected: 4 actual : 0 Test: TestParseSumFile check_test.go:316: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:316 Error: Not equal: expected: "1" actual : "" Diff: --- Expected +++ Actual @@ -1 +1 @@ -1 + Test: TestParseSumFile check_test.go:316: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:316 Error: Not equal: expected: "2" actual : "" Diff: --- Expected +++ Actual @@ -1 +1 @@ -2 + Test: TestParseSumFile check_test.go:316: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:316 Error: Not equal: expected: "3" actual : "" Diff: --- Expected +++ Actual @@ -1 +1 @@ -3 + Test: TestParseSumFile check_test.go:316: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:316 Error: Not equal: expected: "4" actual : "" Diff: --- Expected +++ Actual @@ -1 +1 @@ -4 + Test: TestParseSumFile 2026/01/05 03:10:41 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 03:10:41 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:10:43 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 03:10:45 DEBUG : pacer: Reducing sleep to 843.75ms 2026/01/05 03:10:45 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:10:46 DEBUG : pacer: Reducing sleep to 632.8125ms 2026/01/05 03:10:46 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 90 2026/01/05 03:10:46 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 03:10:47 DEBUG : pacer: Reducing sleep to 474.609375ms 2026/01/05 03:10:48 DEBUG : pacer: Reducing sleep to 355.957031ms 2026/01/05 03:10:48 DEBUG : pacer: low level retry 1/10 (error 500 Internal Server Error) 2026/01/05 03:10:48 DEBUG : pacer: Rate limited, increasing sleep to 711.914062ms 2026/01/05 03:10:48 DEBUG : pacer: low level retry 2/10 (error 500 Internal Server Error) 2026/01/05 03:10:48 DEBUG : pacer: Rate limited, increasing sleep to 1.423828124s 2026/01/05 03:10:49 DEBUG : pacer: low level retry 3/10 (error 500 Internal Server Error) 2026/01/05 03:10:49 DEBUG : pacer: Rate limited, increasing sleep to 2s 2026/01/05 03:10:51 DEBUG : pacer: low level retry 4/10 (error 500 Internal Server Error) 2026/01/05 03:10:53 DEBUG : pacer: low level retry 5/10 (error 500 Internal Server Error) 2026/01/05 03:10:55 DEBUG : pacer: low level retry 6/10 (error 500 Internal Server Error) 2026/01/05 03:10:57 DEBUG : pacer: low level retry 7/10 (error 500 Internal Server Error) 2026/01/05 03:10:59 DEBUG : pacer: low level retry 8/10 (error 500 Internal Server Error) 2026/01/05 03:11:01 DEBUG : pacer: low level retry 9/10 (error 500 Internal Server Error) 2026/01/05 03:11:03 DEBUG : pacer: low level retry 10/10 (error 500 Internal Server Error) check_test.go:311: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:311 Error: Received unexpected error: 500 Internal Server Error Test: TestParseSumFile check_test.go:313: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:313 Error: Not equal: expected: 4 actual : 0 Test: TestParseSumFile check_test.go:316: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:316 Error: Not equal: expected: "1" actual : "" Diff: --- Expected +++ Actual @@ -1 +1 @@ -1 + Test: TestParseSumFile check_test.go:316: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:316 Error: Not equal: expected: "2" actual : "" Diff: --- Expected +++ Actual @@ -1 +1 @@ -2 + Test: TestParseSumFile check_test.go:316: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:316 Error: Not equal: expected: "3" actual : "" Diff: --- Expected +++ Actual @@ -1 +1 @@ -3 + Test: TestParseSumFile check_test.go:316: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:316 Error: Not equal: expected: "4" actual : "" Diff: --- Expected +++ Actual @@ -1 +1 @@ -4 + Test: TestParseSumFile 2026/01/05 03:11:05 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 03:11:07 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 03:11:08 DEBUG : pacer: Reducing sleep to 843.75ms --- FAIL: TestParseSumFile (40.25s) === RUN TestCheckSumDownload run.go:185: Remote "webdav root 'rclone-test-zapumec9folu'", Local "Local file system at /tmp/rclone2822229241", Modify Window "1s" 2026/01/05 03:11:08 DEBUG : Creating backend with remote "TestWebdavInfiniteScale:rclone-test-zapumec9folu/data" 2026/01/05 03:11:08 DEBUG : Config file has changed externally - reloading 2026/01/05 03:11:08 DEBUG : Setting type="webdav" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_TYPE 2026/01/05 03:11:08 DEBUG : Setting url="https://127.0.0.1:28639/dav/spaces/some-admin-user-id-0000-100000000000" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_URL 2026/01/05 03:11:08 DEBUG : Setting vendor="infinitescale" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_VENDOR 2026/01/05 03:11:08 DEBUG : Setting user="admin" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_USER 2026/01/05 03:11:08 DEBUG : Setting pass="wtB0iBNtXFss5AbDF1IEZKHQb7wn" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_PASS 2026/01/05 03:11:08 DEBUG : TestWebdavInfiniteScale: detected overridden config - adding "{7VgIw}" suffix to name 2026/01/05 03:11:08 DEBUG : Setting url="https://127.0.0.1:28639/dav/spaces/some-admin-user-id-0000-100000000000" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_URL 2026/01/05 03:11:08 DEBUG : Setting vendor="infinitescale" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_VENDOR 2026/01/05 03:11:08 DEBUG : Setting user="admin" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_USER 2026/01/05 03:11:08 DEBUG : Setting pass="wtB0iBNtXFss5AbDF1IEZKHQb7wn" for "TestWebdavInfiniteScale" from environment variable RCLONE_CONFIG_TESTWEBDAVINFINITESCALE_PASS 2026/01/05 03:11:08 DEBUG : found headers: 2026/01/05 03:11:09 DEBUG : pacer: Reducing sleep to 632.8125ms 2026/01/05 03:11:09 DEBUG : data/banana: Update will use the tus protocol to upload 2026/01/05 03:11:10 DEBUG : pacer: Reducing sleep to 474.609375ms 2026/01/05 03:11:10 INFO : data/banana: Failed to read metadata: object not found 2026/01/05 03:11:11 DEBUG : pacer: Reducing sleep to 355.957031ms 2026/01/05 03:11:11 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:11:11 DEBUG : pacer: Reducing sleep to 266.967773ms 2026/01/05 03:11:11 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 13 2026/01/05 03:11:11 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 03:11:11 DEBUG : pacer: Reducing sleep to 200.225829ms 2026/01/05 03:11:12 DEBUG : pacer: Reducing sleep to 150.169371ms 2026/01/05 03:11:12 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:11:12 DEBUG : pacer: Reducing sleep to 112.627028ms 2026/01/05 03:11:12 INFO : test.sum: Failed to read metadata: object not found 2026/01/05 03:11:12 DEBUG : pacer: Reducing sleep to 84.470271ms 2026/01/05 03:11:12 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:11:12 DEBUG : pacer: Reducing sleep to 63.352703ms 2026/01/05 03:11:12 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 41 2026/01/05 03:11:12 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 03:11:12 DEBUG : pacer: Reducing sleep to 47.514527ms 2026/01/05 03:11:12 DEBUG : pacer: Reducing sleep to 35.635895ms 2026/01/05 03:11:13 DEBUG : pacer: Reducing sleep to 26.726921ms === RUN TestCheckSumDownload/subtest1 === NAME TestCheckSumDownload check_test.go:404: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:404 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Received unexpected error: failed to parse sum file: 500 Internal Server Error Test: TestCheckSumDownload Messages: unexpected error in run 1 check_test.go:413: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:413 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: "" does not contain "matching files" Test: TestCheckSumDownload Messages: missing matching files in run 1 check_test.go:415: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:415 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: 1 actual : 0 Test: TestCheckSumDownload Messages: wrong number of checks in run 1 check_test.go:384: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:384 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:417 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: []string{"", "= banana"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,4 +1,2 @@ -([]string) (len=2) { - (string) "", - (string) (len=8) "= banana" +([]string) { } Test: TestCheckSumDownload Messages: wrong combined result in run 1 check_test.go:384: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:384 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:420 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: []string{"", "banana"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,4 +1,2 @@ -([]string) (len=2) { - (string) "", - (string) (len=6) "banana" +([]string) { } Test: TestCheckSumDownload Messages: wrong match result in run 1 2026/01/05 03:11:21 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 03:11:21 DEBUG : data/potato: Update will use the tus protocol to upload 2026/01/05 03:11:23 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 03:11:23 INFO : data/potato: Failed to read metadata: object not found 2026/01/05 03:11:25 DEBUG : pacer: Reducing sleep to 843.75ms 2026/01/05 03:11:25 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:11:26 DEBUG : pacer: Reducing sleep to 632.8125ms 2026/01/05 03:11:26 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 15 2026/01/05 03:11:26 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 03:11:27 DEBUG : pacer: Reducing sleep to 474.609375ms 2026/01/05 03:11:27 DEBUG : pacer: Reducing sleep to 355.957031ms 2026/01/05 03:11:27 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:11:28 DEBUG : pacer: Reducing sleep to 266.967773ms 2026/01/05 03:11:28 DEBUG : pacer: Reducing sleep to 200.225829ms 2026/01/05 03:11:28 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:11:28 DEBUG : pacer: Reducing sleep to 150.169371ms 2026/01/05 03:11:28 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 41 2026/01/05 03:11:28 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 03:11:29 DEBUG : pacer: Reducing sleep to 112.627028ms 2026/01/05 03:11:29 DEBUG : pacer: Reducing sleep to 84.470271ms 2026/01/05 03:11:29 DEBUG : pacer: Reducing sleep to 63.352703ms === RUN TestCheckSumDownload/subtest2 === NAME TestCheckSumDownload check_test.go:409: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:409 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: 1 actual : 0 Test: TestCheckSumDownload Messages: wrong error count in run 2 check_test.go:413: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:413 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: "" does not contain "matching files" Test: TestCheckSumDownload Messages: missing matching files in run 2 check_test.go:415: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:415 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: 2 actual : 0 Test: TestCheckSumDownload Messages: wrong number of checks in run 2 check_test.go:384: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:384 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:417 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: []string{"", "- potato", "= banana"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,5 +1,2 @@ -([]string) (len=3) { - (string) "", - (string) (len=8) "- potato", - (string) (len=8) "= banana" +([]string) { } Test: TestCheckSumDownload Messages: wrong combined result in run 2 check_test.go:384: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:384 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:418 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: []string{"", "potato"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,4 +1,2 @@ -([]string) (len=2) { - (string) "", - (string) (len=6) "potato" +([]string) { } Test: TestCheckSumDownload Messages: wrong missingonsrc result in run 2 check_test.go:384: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:384 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:420 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: []string{"", "banana"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,4 +1,2 @@ -([]string) (len=2) { - (string) "", - (string) (len=6) "banana" +([]string) { } Test: TestCheckSumDownload Messages: wrong match result in run 2 2026/01/05 03:11:40 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 03:11:40 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:11:42 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 03:11:44 DEBUG : pacer: Reducing sleep to 843.75ms 2026/01/05 03:11:44 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:11:45 DEBUG : pacer: Reducing sleep to 632.8125ms 2026/01/05 03:11:45 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 ok, range 0 -> 82 2026/01/05 03:11:45 DEBUG : webdav root 'rclone-test-zapumec9folu': -- Uploaded finished 2026/01/05 03:11:46 DEBUG : pacer: Reducing sleep to 474.609375ms 2026/01/05 03:11:46 DEBUG : pacer: Reducing sleep to 355.957031ms 2026/01/05 03:11:47 DEBUG : pacer: Reducing sleep to 266.967773ms === RUN TestCheckSumDownload/subtest3 === NAME TestCheckSumDownload check_test.go:404: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:404 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Received unexpected error: failed to parse sum file: 500 Internal Server Error Test: TestCheckSumDownload Messages: unexpected error in run 3 check_test.go:413: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:413 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: "" does not contain "matching files" Test: TestCheckSumDownload Messages: missing matching files in run 3 check_test.go:415: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:415 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: 2 actual : 0 Test: TestCheckSumDownload Messages: wrong number of checks in run 3 check_test.go:384: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:384 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:417 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: []string{"", "= banana", "= potato"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,5 +1,2 @@ -([]string) (len=3) { - (string) "", - (string) (len=8) "= banana", - (string) (len=8) "= potato" +([]string) { } Test: TestCheckSumDownload Messages: wrong combined result in run 3 check_test.go:384: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:384 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:420 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:428 Error: Not equal: expected: []string{"", "banana", "potato"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,5 +1,2 @@ -([]string) (len=3) { - (string) "", - (string) (len=6) "banana", - (string) (len=6) "potato" +([]string) { } Test: TestCheckSumDownload Messages: wrong match result in run 3 2026/01/05 03:12:02 DEBUG : pacer: Reducing sleep to 1.5s 2026/01/05 03:12:02 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:04 DEBUG : pacer: Reducing sleep to 1.125s 2026/01/05 03:12:06 DEBUG : pacer: Reducing sleep to 843.75ms 2026/01/05 03:12:06 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:07 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:07 DEBUG : pacer: Rate limited, increasing sleep to 1.6875s 2026/01/05 03:12:07 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:07 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 1/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:09 DEBUG : pacer: Reducing sleep to 1.265625s 2026/01/05 03:12:09 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:11 DEBUG : pacer: Reducing sleep to 949.21875ms 2026/01/05 03:12:12 DEBUG : pacer: Reducing sleep to 711.914062ms 2026/01/05 03:12:12 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:13 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:13 DEBUG : pacer: Rate limited, increasing sleep to 1.423828124s 2026/01/05 03:12:13 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:13 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 2/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:15 DEBUG : pacer: Reducing sleep to 1.067871093s 2026/01/05 03:12:15 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:16 DEBUG : pacer: Reducing sleep to 800.903319ms 2026/01/05 03:12:17 DEBUG : pacer: Reducing sleep to 600.677489ms 2026/01/05 03:12:17 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:18 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:18 DEBUG : pacer: Rate limited, increasing sleep to 1.201354978s 2026/01/05 03:12:18 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:18 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 3/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:20 DEBUG : pacer: Reducing sleep to 901.016233ms 2026/01/05 03:12:20 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:21 DEBUG : pacer: Reducing sleep to 675.762174ms 2026/01/05 03:12:22 DEBUG : pacer: Reducing sleep to 506.82163ms 2026/01/05 03:12:22 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:23 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:23 DEBUG : pacer: Rate limited, increasing sleep to 1.01364326s 2026/01/05 03:12:23 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:23 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 4/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:25 DEBUG : pacer: Reducing sleep to 760.232445ms 2026/01/05 03:12:25 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:26 DEBUG : pacer: Reducing sleep to 570.174333ms 2026/01/05 03:12:27 DEBUG : pacer: Reducing sleep to 427.630749ms 2026/01/05 03:12:27 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:27 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:27 DEBUG : pacer: Rate limited, increasing sleep to 855.261498ms 2026/01/05 03:12:27 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:27 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 5/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:29 DEBUG : pacer: Reducing sleep to 641.446123ms 2026/01/05 03:12:29 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:30 DEBUG : pacer: Reducing sleep to 481.084592ms 2026/01/05 03:12:31 DEBUG : pacer: Reducing sleep to 360.813444ms 2026/01/05 03:12:31 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:31 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:31 DEBUG : pacer: Rate limited, increasing sleep to 721.626888ms 2026/01/05 03:12:31 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:31 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 6/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:33 DEBUG : pacer: Reducing sleep to 541.220166ms 2026/01/05 03:12:33 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:34 DEBUG : pacer: Reducing sleep to 405.915124ms 2026/01/05 03:12:35 DEBUG : pacer: Reducing sleep to 304.436343ms 2026/01/05 03:12:35 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:35 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:35 DEBUG : pacer: Rate limited, increasing sleep to 608.872686ms 2026/01/05 03:12:35 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:35 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 7/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:37 DEBUG : pacer: Reducing sleep to 456.654514ms 2026/01/05 03:12:37 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:38 DEBUG : pacer: Reducing sleep to 342.490885ms 2026/01/05 03:12:38 DEBUG : pacer: Reducing sleep to 256.868163ms 2026/01/05 03:12:38 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:38 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:38 DEBUG : pacer: Rate limited, increasing sleep to 513.736326ms 2026/01/05 03:12:38 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:38 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 8/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:40 DEBUG : pacer: Reducing sleep to 385.302244ms 2026/01/05 03:12:40 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:41 DEBUG : pacer: Reducing sleep to 288.976683ms 2026/01/05 03:12:41 DEBUG : pacer: Reducing sleep to 216.732512ms 2026/01/05 03:12:41 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:41 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:41 DEBUG : pacer: Rate limited, increasing sleep to 433.465024ms 2026/01/05 03:12:41 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:41 DEBUG : test.sum: tus update failed. run.go:286: Retry Put of "test.sum" to webdav root 'rclone-test-zapumec9folu': 9/10 (tus update failed: uploadChunk failed: 500 Internal Server Error) 2026/01/05 03:12:44 DEBUG : pacer: Reducing sleep to 325.098768ms 2026/01/05 03:12:44 DEBUG : test.sum: Update will use the tus protocol to upload 2026/01/05 03:12:44 DEBUG : pacer: Reducing sleep to 243.824076ms 2026/01/05 03:12:44 DEBUG : pacer: Reducing sleep to 182.868057ms 2026/01/05 03:12:44 DEBUG : webdav root 'rclone-test-zapumec9folu': Uploaded starts 2026/01/05 03:12:45 DEBUG : pacer: low level retry 1/1 (error 500 Internal Server Error) 2026/01/05 03:12:45 DEBUG : pacer: Rate limited, increasing sleep to 365.736114ms 2026/01/05 03:12:45 ERROR : webdav root 'rclone-test-zapumec9folu': Uploaded chunk no 1 failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:45 DEBUG : test.sum: tus update failed. run.go:290: Failed to put "test.sum" to "webdav root 'rclone-test-zapumec9folu'": tus update failed: uploadChunk failed: 500 Internal Server Error 2026/01/05 03:12:45 DEBUG : pacer: Reducing sleep to 274.302085ms 2026/01/05 03:12:45 DEBUG : pacer: Reducing sleep to 205.726563ms 2026/01/05 03:12:45 DEBUG : pacer: Reducing sleep to 154.294922ms 2026/01/05 03:12:46 DEBUG : pacer: Reducing sleep to 115.721191ms 2026/01/05 03:12:46 DEBUG : pacer: Reducing sleep to 86.790893ms 2026/01/05 03:12:46 DEBUG : pacer: Reducing sleep to 65.093169ms 2026/01/05 03:12:46 DEBUG : pacer: Reducing sleep to 48.819876ms 2026/01/05 03:12:46 DEBUG : pacer: Reducing sleep to 36.614907ms --- FAIL: TestCheckSumDownload (97.97s) --- PASS: TestCheckSumDownload/subtest1 (6.75s) --- PASS: TestCheckSumDownload/subtest2 (9.17s) --- PASS: TestCheckSumDownload/subtest3 (13.66s) FAIL 2026/01/05 03:12:46 DEBUG : webdav root 'rclone-test-zapumec9folu': Purge remote 2026/01/05 03:12:46 DEBUG : pacer: Reducing sleep to 27.46118ms "./operations.test -test.v -test.timeout 1h0m0s -remote TestWebdavInfiniteScale: -verbose -test.run '^(TestCheckSumDownload|TestParseSumFile)$|^TestCheckDownload$/^(1|2|3|4|5|6|7)$'" - Finished ERROR in 39m5.392636737s (try 3/5): exit status 1: Failed [TestCheckDownload/1 TestCheckDownload/2 TestCheckDownload/3 TestCheckDownload/4 TestCheckDownload/5 TestCheckDownload/6 TestCheckDownload/7 TestParseSumFile TestCheckSumDownload]