"./operations.test -test.v -test.timeout 1h0m0s -remote TestS3MinioEdge: -verbose -test.run '^(TestApplyTransforms|TestCat|TestCheck|TestCheckDownload|TestCheckSizeOnly|TestCheckSum|TestCheckSumDownload|TestCopyFile|TestCopyFileBackupDir|TestCopyFileCompareDest|TestCopyFileCopyDest|TestCopyFileMaxTransfer|TestCopyInplace|TestCopyLongFile|TestCopyLongFileName|TestCopyLongFileNameCollision|TestCount|TestDeduplicateFirst|TestDeduplicateInteractive|TestDeduplicateLargest|TestDeduplicateNewest|TestDeduplicateNewestByHash|TestDeduplicateOldest|TestDeduplicateRename|TestDeduplicateSizeOnly|TestDeduplicateSkip|TestDeduplicateSmallest|TestDelete|TestHashSums|TestListDirSorted|TestListDirSortedFn|TestListJSON|TestLs|TestLsLong|TestLsWithFilesFrom|TestLsd|TestMaxDelete|TestMaxDeleteSize|TestMaxDeleteSizeLargeFile|TestMergeDirs|TestMkdir|TestMultithreadCopy|TestMultithreadCopyAbort|TestParseSumFile|TestReadFile|TestStatJSON)$'" - Starting (try 2/5) 2025/10/26 03:05:49 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-vekokuj7nuni" 2025/10/26 03:05:49 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/10/26 03:05:49 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/26 03:05:49 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:05:49 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:05:49 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:05:49 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:05:49 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/26 03:05:49 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:05:49 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:05:49 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:05:49 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:05:49 DEBUG : Creating backend with remote "/tmp/rclone3995334511" === RUN TestMultithreadCopy run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:07:55 DEBUG : pacer: low level retry 1/2 (error operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:07:55 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/26 03:09:53 DEBUG : pacer: low level retry 2/2 (error operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:09:53 DEBUG : pacer: Rate limited, increasing sleep to 20ms multithread_test.go:137: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:137 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:148 Error: Received unexpected error: failed to prepare upload: operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused Test: TestMultithreadCopy 2025/10/26 03:11:54 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:11:54 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2025/10/26 03:13:41 DEBUG : pacer: low level retry 2/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:13:41 DEBUG : pacer: Rate limited, increasing sleep to 80ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:137 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:148 Error: Received unexpected error: operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused Test: TestMultithreadCopy --- FAIL: TestMultithreadCopy (471.71s) === RUN TestMultithreadCopyAbort run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:15:38 DEBUG : pacer: low level retry 1/2 (error operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:15:38 DEBUG : pacer: Rate limited, increasing sleep to 160ms 2025/10/26 03:17:45 DEBUG : pacer: low level retry 2/2 (error operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:17:45 DEBUG : pacer: Rate limited, increasing sleep to 320ms multithread_test.go:137: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:137 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:293 Error: Received unexpected error: failed to prepare upload: operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused Test: TestMultithreadCopyAbort 2025/10/26 03:19:51 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:19:51 DEBUG : pacer: Rate limited, increasing sleep to 640ms 2025/10/26 03:21:39 DEBUG : pacer: low level retry 2/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:21:39 DEBUG : pacer: Rate limited, increasing sleep to 1.28s run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:137 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:293 Error: Received unexpected error: operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused Test: TestMultithreadCopyAbort --- FAIL: TestMultithreadCopyAbort (478.67s) === RUN TestCheck run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:23:37 DEBUG : pacer: low level retry 1/2 (error operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:23:37 DEBUG : pacer: Rate limited, increasing sleep to 2s 2025/10/26 03:25:31 DEBUG : pacer: low level retry 2/2 (error operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:237: Failed to mkdir "S3 bucket rclone-test-vekokuj7nuni": operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/26 03:27:23 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:27:25 DEBUG : pacer: Reducing sleep to 1.5s run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:237 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:245 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:258 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:303 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:98 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:181 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCheck --- FAIL: TestCheck (345.27s) === RUN TestCheckDownload run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:27:25 DEBUG : pacer: Reducing sleep to 1.125s run.go:237: Failed to mkdir "S3 bucket rclone-test-vekokuj7nuni": operation error S3: CreateBucket, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested 2025/10/26 03:27:26 DEBUG : pacer: Reducing sleep to 843.75ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:237 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:245 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:258 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:303 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:98 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:204 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCheckDownload --- FAIL: TestCheckDownload (1.51s) === RUN TestCheckSizeOnly run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:27:27 DEBUG : pacer: Reducing sleep to 632.8125ms run.go:237: Failed to mkdir "S3 bucket rclone-test-vekokuj7nuni": operation error S3: CreateBucket, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested 2025/10/26 03:27:28 DEBUG : pacer: Reducing sleep to 474.609375ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:237 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:245 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:258 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:303 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:98 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:181 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCheckSizeOnly --- FAIL: TestCheckSizeOnly (1.97s) === RUN TestParseSumFile run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:27:29 DEBUG : pacer: Reducing sleep to 355.957031ms run.go:237: Failed to mkdir "S3 bucket rclone-test-vekokuj7nuni": operation error S3: CreateBucket, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested 2025/10/26 03:27:29 DEBUG : pacer: Reducing sleep to 266.967773ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:237 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:245 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:258 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:308 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestParseSumFile --- FAIL: TestParseSumFile (1.11s) === RUN TestCheckSum run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:27:29 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-vekokuj7nuni/data" 2025/10/26 03:27:29 DEBUG : Config file has changed externally - reloading 2025/10/26 03:27:29 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/26 03:27:29 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:27:29 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:27:29 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:27:29 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:27:29 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/26 03:27:29 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:27:29 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:27:29 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:27:29 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:29:19 DEBUG : pacer: low level retry 1/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/data": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:29:19 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/26 03:31:16 DEBUG : pacer: low level retry 2/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/data": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:31:16 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/10/26 03:31:16 DEBUG : pacer: Reducing sleep to 200.225829ms run.go:237: Failed to mkdir "S3 bucket rclone-test-vekokuj7nuni": operation error S3: CreateBucket, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested 2025/10/26 03:31:16 DEBUG : pacer: Reducing sleep to 150.169371ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:237 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:245 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:258 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:355 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:432 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:537 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCheckSum --- FAIL: TestCheckSum (226.75s) === RUN TestCheckSumDownload run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:31:16 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-vekokuj7nuni/data" 2025/10/26 03:31:16 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/26 03:31:16 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:31:16 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:31:16 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:31:16 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:31:16 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/26 03:31:16 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:31:16 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:31:16 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:31:16 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:33:14 DEBUG : pacer: low level retry 1/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/data": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:33:14 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/26 03:35:12 DEBUG : pacer: low level retry 2/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/data": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:35:12 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/10/26 03:35:12 DEBUG : pacer: Reducing sleep to 112.627028ms run.go:237: Failed to mkdir "S3 bucket rclone-test-vekokuj7nuni": operation error S3: CreateBucket, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested 2025/10/26 03:35:12 DEBUG : pacer: Reducing sleep to 84.470271ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:237 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:245 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:258 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:355 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:432 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:541 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCheckSumDownload --- FAIL: TestCheckSumDownload (236.27s) === RUN TestApplyTransforms 2025/10/26 03:35:12 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-cogirud5cave" 2025/10/26 03:35:12 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/10/26 03:35:12 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/26 03:35:12 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:35:12 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:35:12 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:35:12 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:35:12 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/26 03:35:12 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:35:12 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:35:12 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:35:12 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:35:12 DEBUG : Creating backend with remote "/tmp/rclone1294230581" run.go:180: Remote "S3 bucket rclone-test-cogirud5cave", Local "Local file system at /tmp/rclone1294230581", Modify Window "1ns" 2025/10/26 03:37:19 DEBUG : pacer: low level retry 1/2 (error operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-cogirud5cave": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:37:19 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/26 03:39:07 DEBUG : pacer: low level retry 2/2 (error operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-cogirud5cave": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:39:07 DEBUG : pacer: Rate limited, increasing sleep to 20ms run.go:237: Failed to mkdir "S3 bucket rclone-test-cogirud5cave": operation error S3: CreateBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-cogirud5cave": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/26 03:39:07 DEBUG : S3 bucket rclone-test-cogirud5cave: Purge remote 2025/10/26 03:41:01 DEBUG : pacer: low level retry 1/2 (error operation error S3: GetBucketVersioning, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-cogirud5cave?versioning=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:41:01 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2025/10/26 03:42:52 DEBUG : pacer: low level retry 2/2 (error operation error S3: GetBucketVersioning, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-cogirud5cave?versioning=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:42:52 DEBUG : pacer: Rate limited, increasing sleep to 80ms 2025/10/26 03:42:52 ERROR : S3 bucket rclone-test-cogirud5cave: Failed to read versioning status, assuming unversioned: operation error S3: GetBucketVersioning, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-cogirud5cave?versioning=": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/26 03:42:52 DEBUG : S3 bucket rclone-test-cogirud5cave: bucket is versioned: false 2025/10/26 03:42:52 DEBUG : Waiting for deletions to finish 2025/10/26 03:44:46 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-cogirud5cave?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:44:46 DEBUG : pacer: Rate limited, increasing sleep to 160ms 2025/10/26 03:46:33 DEBUG : pacer: low level retry 2/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-cogirud5cave?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:46:33 DEBUG : pacer: Rate limited, increasing sleep to 320ms 2025/10/26 03:48:27 DEBUG : pacer: low level retry 1/2 (error operation error S3: DeleteBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Delete "http://127.0.0.1:28626/rclone-test-cogirud5cave": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:48:27 DEBUG : pacer: Rate limited, increasing sleep to 640ms 2025/10/26 03:50:30 DEBUG : pacer: low level retry 2/2 (error operation error S3: DeleteBucket, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Delete "http://127.0.0.1:28626/rclone-test-cogirud5cave": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:50:30 DEBUG : pacer: Rate limited, increasing sleep to 1.28s 2025/10/26 03:50:30 NOTICE: purge failed: operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-cogirud5cave?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused --- FAIL: TestApplyTransforms (917.70s) === RUN TestCopyFile run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" fstest.go:298: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:298: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:298: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:108 Error: Should be true Test: TestCopyFile Messages: listing wrong, want file1 (14) got file1 (14), rutabaga (8) fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:108 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFile Messages: Unexpected file "rutabaga" 2025/10/26 03:50:37 DEBUG : pacer: Reducing sleep to 63.352703ms copy_test.go:114: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:114 Error: Received unexpected error: operation error S3: HeadObject, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/sub/file2": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCopyFile 2025/10/26 03:50:37 DEBUG : pacer: Reducing sleep to 47.514527ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:114 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCopyFile --- FAIL: TestCopyFile (7.10s) === RUN TestCopyLongFile run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" copy_test.go:154: Test only runs on local 2025/10/26 03:50:37 DEBUG : pacer: Reducing sleep to 35.635895ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1156 /usr/local/go/src/testing/testing.go:1132 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:154 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCopyLongFile --- FAIL: TestCopyLongFile (0.06s) === RUN TestCopyFileBackupDir run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" fstest.go:298: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:298: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:298: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:180 Error: Should be true Test: TestCopyFileBackupDir Messages: listing wrong, want dst/file1 (14) got dst/file1 (14), file1 (14), rutabaga (8) fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:180 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileBackupDir Messages: Unexpected file "file1" fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:180 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileBackupDir Messages: Unexpected file "rutabaga" 2025/10/26 03:50:44 DEBUG : pacer: Reducing sleep to 26.726921ms run.go:237: Failed to mkdir "S3 bucket rclone-test-vekokuj7nuni": operation error S3: CreateBucket, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-vekokuj7nuni": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested 2025/10/26 03:50:44 DEBUG : pacer: Reducing sleep to 20.04519ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:237 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:245 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:258 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:182 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCopyFileBackupDir --- FAIL: TestCopyFileBackupDir (7.05s) === RUN TestCopyFileCompareDest run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:50:44 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-vekokuj7nuni/dst" 2025/10/26 03:50:44 DEBUG : Config file has changed externally - reloading 2025/10/26 03:50:44 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/26 03:50:44 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:50:44 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:50:44 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:50:44 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:50:44 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/26 03:50:44 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:50:44 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:50:44 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:50:44 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:52:40 DEBUG : pacer: low level retry 1/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/dst": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:52:40 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/26 03:54:34 DEBUG : pacer: low level retry 2/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/dst": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:54:34 DEBUG : pacer: Rate limited, increasing sleep to 20ms fstest.go:298: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:298: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:298: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:204 Error: Should be true Test: TestCopyFileCompareDest Messages: listing wrong, want one (3) got dst/file1 (14), file1 (14), one (3), rutabaga (8) fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:204 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileCompareDest Messages: Unexpected file "file1" fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:204 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileCompareDest Messages: Unexpected file "rutabaga" fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:204 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileCompareDest Messages: Unexpected file "dst/file1" 2025/10/26 03:56:41 DEBUG : pacer: low level retry 1/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/dst/one": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:56:41 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2025/10/26 03:58:35 DEBUG : pacer: low level retry 2/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/dst/one": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 03:58:35 DEBUG : pacer: Rate limited, increasing sleep to 80ms copy_test.go:207: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:207 Error: Received unexpected error: operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/dst/one": dial tcp 127.0.0.1:28626: connect: connection refused Test: TestCopyFileCompareDest 2025/10/26 03:58:35 DEBUG : pacer: Reducing sleep to 15.033892ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:207 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-vekokuj7nuni?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCopyFileCompareDest --- FAIL: TestCopyFileCompareDest (471.01s) === RUN TestCopyFileCopyDest run.go:180: Remote "S3 bucket rclone-test-vekokuj7nuni", Local "Local file system at /tmp/rclone3995334511", Modify Window "1ns" 2025/10/26 03:58:35 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-vekokuj7nuni/dst" 2025/10/26 03:58:35 DEBUG : Config file has changed externally - reloading 2025/10/26 03:58:35 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/26 03:58:35 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:58:35 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:58:35 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:58:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 03:58:35 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/26 03:58:35 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/26 03:58:35 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/26 03:58:35 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/26 03:58:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/26 04:00:24 DEBUG : pacer: low level retry 1/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/dst": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 04:00:24 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/26 04:02:15 DEBUG : pacer: low level retry 2/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/dst": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 04:02:15 DEBUG : pacer: Rate limited, increasing sleep to 20ms fstest.go:298: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:298: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:298: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:287 Error: Should be true Test: TestCopyFileCopyDest Messages: listing wrong, want one (3) got dst/file1 (14), file1 (14), one (3), rutabaga (8) fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:287 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileCopyDest Messages: Unexpected file "file1" fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:287 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileCopyDest Messages: Unexpected file "rutabaga" fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:287 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileCopyDest Messages: Unexpected file "dst/file1" 2025/10/26 04:04:21 DEBUG : pacer: low level retry 1/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-vekokuj7nuni/dst/one": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/26 04:04:21 DEBUG : pacer: Rate limited, increasing sleep to 40ms panic: test timed out after 1h0m0s running tests: TestCopyFileCopyDest (7m14s) goroutine 918 [running]: testing.(*M).startAlarm.func1() /usr/local/go/src/testing/testing.go:2484 +0x394 created by time.goFunc /usr/local/go/src/time/sleep.go:215 +0x2d goroutine 1 [chan receive, 7 minutes]: testing.(*T).Run(0xc000169340, {0x27b18af?, 0xc000807ad8?}, 0x28d0da8) /usr/local/go/src/testing/testing.go:1859 +0x431 testing.runTests.func1(0xc000169340) /usr/local/go/src/testing/testing.go:2279 +0x37 testing.tRunner(0xc000169340, 0xc000807c18) /usr/local/go/src/testing/testing.go:1792 +0xf4 testing.runTests(0xc0002c9e00, {0x401a700, 0x6e, 0x6e}, {0xee08f870d?, 0x4030040?, 0x4030040?}) /usr/local/go/src/testing/testing.go:2277 +0x4b4 testing.(*M).Run(0xc0007a6aa0) /usr/local/go/src/testing/testing.go:2142 +0x64a github.com/rclone/rclone/fstest.TestMain(0xc0007a6aa0) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:69 +0xa5 github.com/rclone/rclone/fs/operations_test.TestMain(...) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:61 main.main() _testmain.go:267 +0xa9 goroutine 775 [select]: github.com/aws/aws-sdk-go-v2/internal/sdk.sleepWithContext({0x2c21ea8, 0xc0004ea450}, 0x2c07a20?) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v1.39.1/internal/sdk/time.go:36 +0xc5 github.com/aws/aws-sdk-go-v2/aws/retry.(*Attempt).handleAttempt(0xc000526de0, {0x2c21ea8, 0xc0004ea450}, {{0x27391e0?, 0xc0004ea270?}}, 0xc0007a80b0, {0x2c03460, 0xc0007a2720}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v1.39.1/aws/retry/middleware.go:276 +0x9c8 github.com/aws/aws-sdk-go-v2/aws/retry.(*Attempt).HandleFinalize(0xc000526de0, {0x2c21ea8, 0xc000527950}, {{0x27391e0?, 0xc000527590?}}, {0x2c03460, 0xc0007a2720}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v1.39.1/aws/retry/middleware.go:121 +0x525 github.com/aws/smithy-go/middleware.decoratedFinalizeHandler.HandleFinalize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_finalize.go:200 github.com/aws/smithy-go/transport/http.(*InterceptBeforeRetryLoop).HandleFinalize(0x0?, {0x2c21ea8, 0xc000527950}, {{0x27391e0?, 0xc000527590?}}, {0x2c03460, 0xc0007a2740}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/transport/http/interceptor_middleware.go:131 +0x10a github.com/aws/smithy-go/middleware.decoratedFinalizeHandler.HandleFinalize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_finalize.go:200 github.com/aws/aws-sdk-go-v2/service/s3.(*spanRetryLoop).HandleFinalize(0x1507025?, {0x2c21ea8, 0xc000527950}, {{0x27391e0?, 0xc000527590?}}, {0x2c03460, 0xc0007a2760}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:693 +0x129 github.com/aws/smithy-go/middleware.decoratedFinalizeHandler.HandleFinalize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_finalize.go:200 github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding.(*DisableGzip).HandleFinalize(0x41821f?, {0x2c21ea8, 0xc000527950}, {{0x27391e0?, 0xc000527590?}}, {0x2c03460, 0xc0007a2780}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding@v1.13.1/accept_encoding_gzip.go:67 +0x15e github.com/aws/smithy-go/middleware.decoratedFinalizeHandler.HandleFinalize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_finalize.go:200 github.com/aws/smithy-go/middleware.(*FinalizeStep).HandleMiddleware(0x2c21ea8?, {0x2c21ea8, 0xc000527950}, {0x27391e0, 0xc000527590}, {0x2c02580, 0xc0007a2160}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_finalize.go:114 +0x22c github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/middleware.go:57 github.com/aws/smithy-go/middleware.buildWrapHandler.HandleBuild(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:184 github.com/aws/aws-sdk-go-v2/service/s3.(*spanBuildRequestEnd).HandleBuild(0x4982a0?, {0x2c21ea8?, 0xc000527950?}, {{0x27391e0?, 0xc000527590?}}, {0x2c034c0, 0xc0006055e0}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:1444 +0x8b github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:200 github.com/aws/aws-sdk-go-v2/aws/middleware.(*RecursionDetection).HandleBuild(0x15026b8?, {0x2c21ea8, 0xc000527950}, {{0x27391e0?, 0xc000527590?}}, {0x2c034e0, 0xc0007a24e0}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v1.39.1/aws/middleware/recursion_detection.go:46 +0x258 github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:200 github.com/aws/aws-sdk-go-v2/internal/middleware.AddTimeOffsetMiddleware.HandleBuild({0x2794366?}, {0x2c21ea8?, 0xc000527770?}, {{0x27391e0?, 0xc000527590?}}, {0x2c034e0?, 0xc0007a2500?}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v1.39.1/internal/middleware/middleware.go:30 +0xb2 github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:200 github.com/aws/aws-sdk-go-v2/aws/middleware.(*RequestUserAgent).HandleBuild(0x0?, {0x2c21ea8, 0xc000527770}, {{0x27391e0, 0xc000527590}}, {0x2c034e0, 0xc0007a2520}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v1.39.1/aws/middleware/user_agent.go:343 +0x8d github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:200 github.com/aws/aws-sdk-go-v2/service/s3.setCredentialSourceMiddleware.HandleBuild({0xc0005fcca8, {{0x4053620, 0x0, 0x0}, {0x0, 0x0}, 0xc0008c1910, 0x0, 0x0, {0x2bfbda0, ...}, ...}}, ...) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:916 +0x148 github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:200 github.com/aws/smithy-go/transport/http.(*ComputeContentLength).HandleBuild(0x824cbcb67b81c5c0?, {0x2c21ea8, 0xc000527770}, {{0x27391e0?, 0xc000527590?}}, {0x2c034e0, 0xc0007a2560}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/transport/http/middleware_content_length.go:49 +0x12a github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:200 github.com/aws/aws-sdk-go-v2/aws/middleware.ClientRequestID.HandleBuild({}, {0x2c21ea8, 0xc000527770}, {{0x27391e0?, 0xc000527590?}}, {0x2c034e0, 0xc0007a2580}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v1.39.1/aws/middleware/middleware.go:42 +0x1ed github.com/aws/smithy-go/middleware.decoratedBuildHandler.HandleBuild(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:200 github.com/aws/smithy-go/middleware.(*BuildStep).HandleMiddleware(0x6?, {0x2c21ea8, 0xc000527770}, {0x27391e0, 0xc000527590}, {0x2c02580, 0xc0007a2180}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_build.go:114 +0x22c github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/middleware.go:57 github.com/aws/smithy-go/middleware.serializeWrapHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:192 github.com/aws/aws-sdk-go-v2/service/s3.(*isExpressUserAgent).HandleSerialize(0xc000708340, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033c0, 0xc0006055c0}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/express_user_agent.go:33 +0xf5 github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3/internal/customizations.(*removeBucketFromPathMiddleware).HandleSerialize(0x22eb720?, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a2340}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/internal/customizations/remove_bucket_middleware.go:26 +0x1fd github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3/internal/customizations.(*updateEndpoint).HandleSerialize(0xc0005fccc0, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a2360}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/internal/customizations/update_endpoint.go:145 +0x52b github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/smithy-go/transport/http.(*InterceptAfterSerialization).HandleSerialize(0xc0005fcd08, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a2380}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/transport/http/interceptor_middleware.go:106 +0x1d0 github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3.(*awsRestxml_serializeOpHeadObject).HandleSerialize(0x4053620?, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a23a0}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/serializers.go:5025 +0x86a github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/smithy-go/transport/http.(*InterceptBeforeSerialization).HandleSerialize(0x22eb720?, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a23c0}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/transport/http/interceptor_middleware.go:79 +0x146 github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3.(*serializeImmutableHostnameBucketMiddleware).HandleSerialize(0xc0006807fd, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a23e0}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/serialize_immutable_hostname_bucket.go:53 +0x3a6 github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3/internal/customizations.(*s3ObjectLambdaEndpoint).HandleSerialize(0xc000716f00, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a2400}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/internal/customizations/s3_object_lambda.go:34 +0x697 github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3/internal/customizations.(*processARNResource).HandleSerialize(0xc000716e40, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a2420}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/internal/customizations/process_arn_resource.go:54 +0x1219 github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3.(*ResolveEndpoint).HandleSerialize(0xc0003755e0, {0x2c21ea8, 0xc000527770}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a2440}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/endpoints.go:89 +0x755 github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3.(*setOperationInputMiddleware).HandleSerialize(0x4053620?, {0x2c21ea8?, 0xc000527680?}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a2460}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:355 +0xb5 github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3.(*putBucketContextMiddleware).HandleSerialize(0x41b074?, {0x2c21ea8, 0xc000527560}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a2480}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/bucket_context.go:26 +0xdc github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/aws-sdk-go-v2/service/s3.(*spanBuildRequestStart).HandleSerialize(0x90?, {0x2c21ea8?, 0xc000527560?}, {{0x23a6580?, 0xc00012f2c0?}, {0x27391e0?, 0xc000527590?}}, {0x2c033e0, 0xc0007a24a0}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:1426 +0xaf github.com/aws/smithy-go/middleware.decoratedSerializeHandler.HandleSerialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:208 github.com/aws/smithy-go/middleware.(*SerializeStep).HandleMiddleware(0xc000605510, {0x2c21ea8, 0xc000527560}, {0x23a6580, 0xc00012f2c0}, {0x2c02580, 0xc0007a21a0}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_serialize.go:122 +0x26f github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/middleware.go:57 github.com/aws/smithy-go/middleware.initializeWrapHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:184 github.com/aws/aws-sdk-go-v2/service/s3.(*spanInitializeEnd).HandleInitialize(0xc000708308?, {0x2c21ea8?, 0xc000527560?}, {{0x23a6580?, 0xc00012f2c0?}}, {0x2c03400, 0xc000605580}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:1409 +0x8b github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:200 github.com/aws/aws-sdk-go-v2/service/s3.(*validateOpHeadObject).HandleInitialize(0xc000708301?, {0x2c21ea8, 0xc000527560}, {{0x23a6580?, 0xc00012f2c0?}}, {0x2c03420, 0xc0007a2200}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/validators.go:1210 +0xc6 github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:200 github.com/aws/smithy-go/middleware.(*setLogger).HandleInitialize(0x0?, {0x2c21ea8?, 0xc0005274a0?}, {{0x23a6580?, 0xc00012f2c0?}}, {0x2c03420, 0xc0007a2240}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/logging.go:45 +0x8c github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:200 github.com/aws/aws-sdk-go-v2/service/internal/s3shared.(*ARNLookup).HandleInitialize(0xc0005274a0?, {0x2c21ea8, 0xc0005274a0}, {{0x23a6580?, 0xc00012f2c0?}}, {0x2c03420, 0xc0007a2280}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/internal/s3shared@v1.19.8/arn_lookup.go:39 +0x262 github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:200 github.com/aws/aws-sdk-go-v2/service/s3.(*legacyEndpointContextSetter).HandleInitialize(0x15026b8?, {0x2c21ea8?, 0xc0005274a0?}, {{0x23a6580?, 0xc00012f2c0?}}, {0x2c03420?, 0xc0007a22a0?}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:418 +0x9e github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:200 github.com/aws/aws-sdk-go-v2/aws/middleware.RegisterServiceMetadata.HandleInitialize({{0x2780140, 0x2}, {0x0, 0x0}, {0x2792291, 0x9}, {0x279559a, 0xa}}, {0x2c21ea8, 0xc000527260}, ...) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2@v1.39.1/aws/middleware/metadata.go:41 +0x1c6 github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:200 github.com/aws/smithy-go/transport/http.(*InterceptExecution).HandleInitialize(0xc000526fc0, {0x2c21ea8, 0xc0005270e0}, {{0x23a6580?, 0xc00012f2c0?}}, {0x2c03420, 0xc0007a22e0}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/transport/http/interceptor_middleware.go:46 +0x1f0 github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:200 github.com/aws/aws-sdk-go-v2/service/s3.(*spanInitializeStart).HandleInitialize(0x41821f?, {0x2c21ea8?, 0xc0005270e0?}, {{0x23a6580?, 0xc00012f2c0?}}, {0x2c03420, 0xc0007a2300}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:1391 +0x85 github.com/aws/smithy-go/middleware.decoratedInitializeHandler.HandleInitialize(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:200 github.com/aws/smithy-go/middleware.(*InitializeStep).HandleMiddleware(0xc000554820?, {0x2c21ea8, 0xc0005270e0}, {0x23a6580, 0xc00012f2c0}, {0x2c02580, 0xc0007a21c0}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/step_initialize.go:114 +0x22c github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/middleware.go:57 github.com/aws/smithy-go/middleware.(*Stack).HandleMiddleware(0x418201?, {0x2c21ea8, 0xc0005270e0}, {0x23a6580, 0xc00012f2c0}, {0x2c02520?, 0xc0007a2100?}) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/stack.go:109 +0x183 github.com/aws/smithy-go/middleware.decoratedHandler.Handle(...) /home/rclone/go/pkg/mod/github.com/aws/smithy-go@v1.23.0/middleware/middleware.go:57 github.com/aws/aws-sdk-go-v2/service/s3.(*Client).invokeOperation(0xc000850708, {0x2c21ea8?, 0xc000602de0?}, {0x279559a, 0xa}, {0x23a6580, 0xc00012f2c0}, {0x0, 0x0, 0x0}, ...) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_client.go:305 +0xb75 github.com/aws/aws-sdk-go-v2/service/s3.(*Client).HeadObject(0xc000637760?, {0x2c21ea8?, 0xc000602de0?}, 0x41b074?, {0x0?, 0xc0006377d8?, 0x41373d?}) /home/rclone/go/pkg/mod/github.com/aws/aws-sdk-go-v2/service/s3@v1.88.2/api_op_HeadObject.go:147 +0x10c github.com/rclone/rclone/backend/s3.(*Fs).headObject.func1() /home/rclone/go/src/github.com/rclone/rclone/backend/s3/s3.go:3633 +0x4e github.com/rclone/rclone/fs.pacerInvoker(0x2, 0x2, 0x2bf9380?) /home/rclone/go/src/github.com/rclone/rclone/fs/pacer.go:86 +0x32 github.com/rclone/rclone/lib/pacer.(*Pacer).call(0xc000717440, 0xc0005729c0, 0x2) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:232 +0xc5 github.com/rclone/rclone/lib/pacer.(*Pacer).Call(0xc000717440, 0xc0005729c0) /home/rclone/go/src/github.com/rclone/rclone/lib/pacer/pacer.go:251 +0x93 github.com/rclone/rclone/backend/s3.(*Fs).headObject(0xc0003fc908, {0x2c21ea8, 0xc000602de0}, 0xc00012f2c0) /home/rclone/go/src/github.com/rclone/rclone/backend/s3/s3.go:3631 +0x1ae github.com/rclone/rclone/backend/s3.(*Object).headObject(0xc00071d3b0, {0x2c21ea8, 0xc000602de0}) /home/rclone/go/src/github.com/rclone/rclone/backend/s3/s3.go:3615 +0x129 github.com/rclone/rclone/backend/s3.(*Object).readMetaData(0xc00071d3b0, {0x2c21ea8?, 0xc000602de0?}) /home/rclone/go/src/github.com/rclone/rclone/backend/s3/s3.go:3655 +0x29 github.com/rclone/rclone/backend/s3.(*Fs).newObjectWithInfo(0xc0003fc908, {0x2c21ea8, 0xc000602de0}, {0x278042f, 0x3}, 0x0, 0x0) /home/rclone/go/src/github.com/rclone/rclone/backend/s3/s3.go:1749 +0x505 github.com/rclone/rclone/backend/s3.(*Fs).NewObject(0xc000637c00?, {0x2c21ea8?, 0xc000602de0?}, {0x278042f?, 0x0?}) /home/rclone/go/src/github.com/rclone/rclone/backend/s3/s3.go:1760 +0x28 github.com/rclone/rclone/fs/operations.moveOrCopyFile({0x2c21ea8, 0xc000602de0}, {0x2c39670, 0xc0003fc908}, {0x2c38800, 0xc0006aa480}, {0x278042f, 0x3}, {0x278042f, 0x3}, ...) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations.go:2029 +0x49b github.com/rclone/rclone/fs/operations.CopyFile(...) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy.go:421 github.com/rclone/rclone/fs/operations_test.TestCopyFileCopyDest(0xc0002ee1c0) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:289 +0x35c testing.tRunner(0xc0002ee1c0, 0x28d0da8) /usr/local/go/src/testing/testing.go:1792 +0xf4 created by testing.(*T).Run in goroutine 1 /usr/local/go/src/testing/testing.go:1851 +0x413 goroutine 7 [syscall, 60 minutes]: os/signal.signal_recv() /usr/local/go/src/runtime/sigqueue.go:152 +0x29 os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 created by os/signal.Notify.func1.1 in goroutine 1 /usr/local/go/src/os/signal/signal.go:152 +0x1f goroutine 8 [chan receive, 60 minutes]: github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1() /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27 created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 1 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xa7 goroutine 585 [select]: github.com/rclone/rclone/fs/accounting.(*StatsInfo).averageLoop(0xc00056e960, {0x2c21ee0, 0xc0002e85f0}) /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/stats.go:352 +0x145 created by github.com/rclone/rclone/fs/accounting.(*StatsInfo)._startAverageLoop in goroutine 584 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/stats.go:389 +0x11a goroutine 394 [chan receive, 32 minutes]: github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler.func1() /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:24 +0x27 created by github.com/rclone/rclone/fs/accounting.(*tokenBucket).startSignalHandler in goroutine 393 /home/rclone/go/src/github.com/rclone/rclone/fs/accounting/accounting_unix.go:21 +0xa7 "./operations.test -test.v -test.timeout 1h0m0s -remote TestS3MinioEdge: -verbose -test.run '^(TestApplyTransforms|TestCat|TestCheck|TestCheckDownload|TestCheckSizeOnly|TestCheckSum|TestCheckSumDownload|TestCopyFile|TestCopyFileBackupDir|TestCopyFileCompareDest|TestCopyFileCopyDest|TestCopyFileMaxTransfer|TestCopyInplace|TestCopyLongFile|TestCopyLongFileName|TestCopyLongFileNameCollision|TestCount|TestDeduplicateFirst|TestDeduplicateInteractive|TestDeduplicateLargest|TestDeduplicateNewest|TestDeduplicateNewestByHash|TestDeduplicateOldest|TestDeduplicateRename|TestDeduplicateSizeOnly|TestDeduplicateSkip|TestDeduplicateSmallest|TestDelete|TestHashSums|TestListDirSorted|TestListDirSortedFn|TestListJSON|TestLs|TestLsLong|TestLsWithFilesFrom|TestLsd|TestMaxDelete|TestMaxDeleteSize|TestMaxDeleteSizeLargeFile|TestMergeDirs|TestMkdir|TestMultithreadCopy|TestMultithreadCopyAbort|TestParseSumFile|TestReadFile|TestStatJSON)$'" - Finished ERROR in 1h0m0.097570399s (try 2/5): exit status 2: Failed [TestMultithreadCopy TestMultithreadCopyAbort TestCheck TestCheckDownload TestCheckSizeOnly TestParseSumFile TestCheckSum TestCheckSumDownload TestApplyTransforms TestCopyFile TestCopyLongFile TestCopyFileBackupDir TestCopyFileCompareDest]