"./sync.test -test.v -test.timeout 2h0m0s -remote TestDrime: -verbose -test.run '^TestSyncConcurrentTruncate$'" - Starting (try 2/5)
2026/01/31 05:51:12 DEBUG : Creating backend with remote "TestDrime:rclone-test-gibuted8qeyi"
2026/01/31 05:51:12 DEBUG : Using config file from "/home/rclone/.rclone.conf"
2026/01/31 05:51:14 DEBUG : Creating backend with remote "/tmp/rclone1806836115"
=== RUN TestSyncConcurrentTruncate
run.go:185: Remote "drime root 'rclone-test-gibuted8qeyi'", Local "Local file system at /tmp/rclone1806836115", Modify Window "876000h0m0s"
2026/01/31 06:00:13 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:00:13 DEBUG : pacer: low level retry 1/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:00:13 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2026/01/31 06:00:15 DEBUG : pacer: Reducing sleep to 10ms
2026/01/31 06:05:49 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:05:49 DEBUG : pacer: low level retry 1/1 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:05:49 DEBUG : pacer: Rate limited, increasing sleep to 20ms
run.go:286: Retry Put of "both14" to drime root 'rclone-test-gibuted8qeyi': 1/10 (failed to upload file: Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:05:51 DEBUG : pacer: Reducing sleep to 10ms
2026/01/31 06:10:09 DEBUG : both10: size = 6 OK
2026/01/31 06:10:09 DEBUG : both0: size = 6 OK
2026/01/31 06:10:09 DEBUG : both10: Sizes identical
2026/01/31 06:10:09 DEBUG : both12: size = 6 OK
2026/01/31 06:10:09 DEBUG : both11: size = 6 OK
2026/01/31 06:10:09 DEBUG : drime root 'rclone-test-gibuted8qeyi': Waiting for checks to finish
2026/01/31 06:10:09 DEBUG : both0: Sizes identical
2026/01/31 06:10:09 DEBUG : both11: Sizes identical
2026/01/31 06:10:09 DEBUG : both10: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both12: Sizes identical
2026/01/31 06:10:09 DEBUG : both13: size = 6 OK
2026/01/31 06:10:09 DEBUG : both13: Sizes identical
2026/01/31 06:10:09 DEBUG : both12: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both14: size = 6 OK
2026/01/31 06:10:09 DEBUG : both14: Sizes identical
2026/01/31 06:10:09 DEBUG : both14: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both15: size = 6 OK
2026/01/31 06:10:09 DEBUG : both15: Sizes identical
2026/01/31 06:10:09 DEBUG : both15: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both16: size = 6 OK
2026/01/31 06:10:09 DEBUG : both16: Sizes identical
2026/01/31 06:10:09 DEBUG : both0: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both17: size = 6 OK
2026/01/31 06:10:09 DEBUG : both17: Sizes identical
2026/01/31 06:10:09 DEBUG : both17: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both18: size = 6 OK
2026/01/31 06:10:09 DEBUG : both18: Sizes identical
2026/01/31 06:10:09 DEBUG : both13: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both19: size = 6 OK
2026/01/31 06:10:09 DEBUG : both19: Sizes identical
2026/01/31 06:10:09 DEBUG : both19: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both1: size = 6 OK
2026/01/31 06:10:09 DEBUG : both1: Sizes identical
2026/01/31 06:10:09 DEBUG : both11: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both2: size = 6 OK
2026/01/31 06:10:09 DEBUG : both2: Sizes identical
2026/01/31 06:10:09 DEBUG : both2: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both3: size = 6 OK
2026/01/31 06:10:09 DEBUG : both3: Sizes identical
2026/01/31 06:10:09 DEBUG : both16: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both4: size = 6 OK
2026/01/31 06:10:09 DEBUG : both4: Sizes identical
2026/01/31 06:10:09 DEBUG : both4: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both5: size = 6 OK
2026/01/31 06:10:09 DEBUG : both5: Sizes identical
2026/01/31 06:10:09 DEBUG : both5: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both6: size = 6 OK
2026/01/31 06:10:09 DEBUG : both6: Sizes identical
2026/01/31 06:10:09 DEBUG : both18: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both7: size = 6 OK
2026/01/31 06:10:09 DEBUG : both7: Sizes identical
2026/01/31 06:10:09 DEBUG : both7: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both8: size = 6 OK
2026/01/31 06:10:09 DEBUG : both8: Sizes identical
2026/01/31 06:10:09 DEBUG : both1: Unchanged skipping
2026/01/31 06:10:09 DEBUG : both9: size = 6 OK
2026/01/31 06:10:09 DEBUG : both9: Sizes identical
2026/01/31 06:10:09 DEBUG : both3: Unchanged skipping
2026/01/31 06:10:09 DEBUG : only0: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only0: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only0: Sizes differ
2026/01/31 06:10:09 DEBUG : both6: Unchanged skipping
2026/01/31 06:10:09 DEBUG : only10: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only10: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only10: Sizes differ
2026/01/31 06:10:09 DEBUG : only11: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only11: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only11: Sizes differ
2026/01/31 06:10:09 DEBUG : both8: Unchanged skipping
2026/01/31 06:10:09 DEBUG : only12: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only12: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only12: Sizes differ
2026/01/31 06:10:09 DEBUG : only13: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only13: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only13: Sizes differ
2026/01/31 06:10:09 DEBUG : only14: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only14: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only14: Sizes differ
2026/01/31 06:10:09 DEBUG : both9: Unchanged skipping
2026/01/31 06:10:09 DEBUG : only15: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only15: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only15: Sizes differ
2026/01/31 06:10:09 DEBUG : only16: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only16: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only16: Sizes differ
2026/01/31 06:10:09 DEBUG : only17: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only17: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only17: Sizes differ
2026/01/31 06:10:09 DEBUG : only18: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only18: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only18: Sizes differ
2026/01/31 06:10:09 DEBUG : only19: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only19: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only19: Sizes differ
2026/01/31 06:10:09 DEBUG : only1: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only1: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only1: Sizes differ
2026/01/31 06:10:09 DEBUG : only2: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only2: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only2: Sizes differ
2026/01/31 06:10:09 DEBUG : only3: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only3: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only3: Sizes differ
2026/01/31 06:10:09 DEBUG : only4: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only4: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only4: Sizes differ
2026/01/31 06:10:09 DEBUG : only5: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only5: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only5: Sizes differ
2026/01/31 06:10:09 DEBUG : only6: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only6: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only6: Sizes differ
2026/01/31 06:10:09 DEBUG : only7: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only7: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only7: Sizes differ
2026/01/31 06:10:09 DEBUG : only8: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only8: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only8: Sizes differ
2026/01/31 06:10:09 DEBUG : only9: size = 0 (Local file system at /tmp/rclone1806836115)
2026/01/31 06:10:09 DEBUG : only9: size = 6 (drime root 'rclone-test-gibuted8qeyi')
2026/01/31 06:10:09 DEBUG : only9: Sizes differ
2026/01/31 06:10:09 DEBUG : drime root 'rclone-test-gibuted8qeyi': Waiting for transfers to finish
2026/01/31 06:10:28 DEBUG : only0: Removing old object on successful upload
2026/01/31 06:10:29 DEBUG : only0: size = 0 OK
2026/01/31 06:10:29 INFO : only0: Copied (replaced existing)
2026/01/31 06:10:35 DEBUG : only12: Removing old object on successful upload
2026/01/31 06:10:35 DEBUG : only12: size = 0 OK
2026/01/31 06:10:35 INFO : only12: Copied (replaced existing)
2026/01/31 06:10:36 DEBUG : only10: Removing old object on successful upload
2026/01/31 06:10:36 DEBUG : only13: Removing old object on successful upload
2026/01/31 06:10:36 DEBUG : only10: size = 0 OK
2026/01/31 06:10:36 INFO : only10: Copied (replaced existing)
2026/01/31 06:10:37 DEBUG : only13: size = 0 OK
2026/01/31 06:10:37 INFO : only13: Copied (replaced existing)
2026/01/31 06:10:47 DEBUG : only11: Removing old object on successful upload
2026/01/31 06:10:48 DEBUG : only11: size = 0 OK
2026/01/31 06:10:48 INFO : only11: Copied (replaced existing)
2026/01/31 06:10:51 DEBUG : only19: Removing old object on successful upload
2026/01/31 06:10:51 DEBUG : only19: size = 0 OK
2026/01/31 06:10:51 INFO : only19: Copied (replaced existing)
2026/01/31 06:10:54 DEBUG : only18: Removing old object on successful upload
2026/01/31 06:10:54 DEBUG : only18: size = 0 OK
2026/01/31 06:10:54 INFO : only18: Copied (replaced existing)
2026/01/31 06:10:57 DEBUG : only17: Removing old object on successful upload
2026/01/31 06:10:57 DEBUG : only17: size = 0 OK
2026/01/31 06:10:57 INFO : only17: Copied (replaced existing)
2026/01/31 06:11:09 DEBUG : only14: Removing old object on successful upload
2026/01/31 06:11:09 DEBUG : only14: size = 0 OK
2026/01/31 06:11:09 INFO : only14: Copied (replaced existing)
2026/01/31 06:11:12 DEBUG : only2: Removing old object on successful upload
2026/01/31 06:11:13 DEBUG : only2: size = 0 OK
2026/01/31 06:11:13 INFO : only2: Copied (replaced existing)
2026/01/31 06:11:15 DEBUG : only16: Removing old object on successful upload
2026/01/31 06:11:16 DEBUG : only16: size = 0 OK
2026/01/31 06:11:16 INFO : only16: Copied (replaced existing)
2026/01/31 06:11:20 DEBUG : only15: Removing old object on successful upload
2026/01/31 06:11:21 DEBUG : only15: size = 0 OK
2026/01/31 06:11:21 INFO : only15: Copied (replaced existing)
2026/01/31 06:11:32 DEBUG : only3: Removing old object on successful upload
2026/01/31 06:11:33 DEBUG : only3: size = 0 OK
2026/01/31 06:11:33 INFO : only3: Copied (replaced existing)
2026/01/31 06:11:36 DEBUG : only1: Removing old object on successful upload
2026/01/31 06:11:36 DEBUG : only1: size = 0 OK
2026/01/31 06:11:36 INFO : only1: Copied (replaced existing)
2026/01/31 06:11:38 DEBUG : only8: Removing old object on successful upload
2026/01/31 06:11:38 DEBUG : only8: size = 0 OK
2026/01/31 06:11:38 INFO : only8: Copied (replaced existing)
2026/01/31 06:11:48 DEBUG : only7: Removing old object on successful upload
2026/01/31 06:11:48 DEBUG : only7: size = 0 OK
2026/01/31 06:11:48 INFO : only7: Copied (replaced existing)
2026/01/31 06:11:52 DEBUG : only4: Removing old object on successful upload
2026/01/31 06:11:52 DEBUG : only4: size = 0 OK
2026/01/31 06:11:52 INFO : only4: Copied (replaced existing)
2026/01/31 06:11:59 DEBUG : only5: Removing old object on successful upload
2026/01/31 06:12:04 DEBUG : only5: size = 0 OK
2026/01/31 06:12:04 INFO : only5: Copied (replaced existing)
2026/01/31 06:12:07 DEBUG : only6: Removing old object on successful upload
2026/01/31 06:12:08 DEBUG : only6: size = 0 OK
2026/01/31 06:12:08 INFO : only6: Copied (replaced existing)
2026/01/31 06:12:24 DEBUG : only9: Removing old object on successful upload
2026/01/31 06:12:24 DEBUG : only9: size = 0 OK
2026/01/31 06:12:24 INFO : only9: Copied (replaced existing)
2026/01/31 06:12:24 DEBUG : Waiting for deletions to finish
2026/01/31 06:12:24 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:24 DEBUG : pacer: low level retry 1/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:24 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2026/01/31 06:12:24 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:24 DEBUG : pacer: low level retry 2/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:24 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2026/01/31 06:12:24 DEBUG : pacer: Reducing sleep to 20ms
2026/01/31 06:12:24 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:24 DEBUG : pacer: low level retry 1/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:24 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2026/01/31 06:12:24 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:24 DEBUG : pacer: low level retry 2/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:24 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2026/01/31 06:12:25 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:25 DEBUG : pacer: low level retry 3/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:25 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2026/01/31 06:12:25 DEBUG : pacer: Reducing sleep to 80ms
2026/01/31 06:12:25 DEBUG : pacer: Reducing sleep to 40ms
2026/01/31 06:12:26 DEBUG : pacer: Reducing sleep to 20ms
2026/01/31 06:12:26 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:26 DEBUG : pacer: low level retry 1/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:26 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2026/01/31 06:12:26 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:26 DEBUG : pacer: low level retry 2/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:26 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2026/01/31 06:12:26 DEBUG : pacer: Reducing sleep to 40ms
2026/01/31 06:12:26 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:26 DEBUG : pacer: low level retry 1/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:26 DEBUG : pacer: Rate limited, increasing sleep to 80ms
2026/01/31 06:12:26 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:26 DEBUG : pacer: low level retry 2/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:26 DEBUG : pacer: Rate limited, increasing sleep to 160ms
2026/01/31 06:12:27 DEBUG : pacer: Reducing sleep to 80ms
2026/01/31 06:12:27 DEBUG : pacer: Reducing sleep to 40ms
2026/01/31 06:12:28 DEBUG : pacer: Reducing sleep to 20ms
2026/01/31 06:12:28 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:28 DEBUG : pacer: low level retry 1/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:28 DEBUG : pacer: Rate limited, increasing sleep to 40ms
2026/01/31 06:12:28 DEBUG : pacer: Reducing sleep to 20ms
2026/01/31 06:12:29 DEBUG : pacer: Reducing sleep to 10ms
2026/01/31 06:12:29 DEBUG : Couldn't decode error response: invalid character '<' looking for beginning of value
2026/01/31 06:12:29 DEBUG : pacer: low level retry 1/10 (error Error "500 Internal Server Error (500): \r\n500 Internal Server Error\r\n\r\n500 Internal Server Error
\r\n
nginx\r\n\r\n\r\n")
2026/01/31 06:12:29 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2026/01/31 06:12:29 DEBUG : pacer: Reducing sleep to 10ms
--- PASS: TestSyncConcurrentTruncate (1289.79s)
PASS
2026/01/31 06:12:44 DEBUG : drime root 'rclone-test-gibuted8qeyi': Purge remote
"./sync.test -test.v -test.timeout 2h0m0s -remote TestDrime: -verbose -test.run '^TestSyncConcurrentTruncate$'" - Finished OK in 21m32.43127345s (try 2/5)