"./operations.test -test.v -test.timeout 1h0m0s -remote TestS3MinioEdge: -verbose -fast-list -test.run '^(TestCheck|TestCheckDownload|TestCheckSizeOnly|TestCheckSum|TestCheckSumDownload|TestCopyFile|TestCopyFileBackupDir|TestMultithreadCopyAbort|TestParseSumFile)$|^TestMultithreadCopy$/^(upload=true,size=10485760,streams=2|upload=true,size=10485761,streams=2)$'" - Starting (try 2/5) 2025/10/25 01:54:22 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-wayupug3totu" 2025/10/25 01:54:22 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2025/10/25 01:54:22 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/25 01:54:22 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/25 01:54:22 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/25 01:54:22 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/25 01:54:22 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/25 01:54:22 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/25 01:54:22 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/25 01:54:22 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 01:54:22 DEBUG : Creating backend with remote "/tmp/rclone1944841182" === RUN TestMultithreadCopy run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" 2025/10/25 01:54:22 INFO : S3 bucket rclone-test-wayupug3totu: Bucket "rclone-test-wayupug3totu" created with ACL "" 2025/10/25 01:54:22 DEBUG : chunksize-probe: open chunk writer: started multipart upload: e4190d40-b980-47d4-b5f2-01cf743394c0 2025/10/25 01:54:22 DEBUG : chunksize-probe: multipart upload "e4190d40-b980-47d4-b5f2-01cf743394c0" aborted === RUN TestMultithreadCopy/upload=true,size=10485760,streams=2 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multi-thread copy: disabling buffering because source is local disk 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: open chunk writer: started multipart upload: 0803cc10-dca7-4a89-9d6c-d51a0f18059d 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 2 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multi-thread copy: number of streams 4 was bigger than number of chunks 2 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: Starting multi-thread copy with 2 chunks of size 5Mi with 2 parallel streams 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multi-thread copy: chunk 2/2 (5242880-10485760) size 5Mi starting 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multi-thread copy: chunk 1/2 (0-5242880) size 5Mi starting 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: Seek from 5242880 to 0 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: Seek from 5242880 to 0 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: Seek from 5242880 to 0 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: Seek from 5242880 to 0 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multipart upload wrote chunk 1 with 5242880 bytes and etag "dcf795a8b0aa570bd57ac8009a55f952" 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multi-thread copy: chunk 1/2 (0-5242880) size 5Mi finished 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multipart upload wrote chunk 2 with 5242880 bytes and etag "77757a6a9d01a469e24e9a27d4008435" 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multi-thread copy: chunk 2/2 (5242880-10485760) size 5Mi finished 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: multipart upload "0803cc10-dca7-4a89-9d6c-d51a0f18059d" finished 2025/10/25 01:54:24 DEBUG : test-multithread-copy-true-10485760-2: Finished multi-thread copy with 2 parts of size 5Mi === RUN TestMultithreadCopy/upload=true,size=10485761,streams=2 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: disabling buffering because source is local disk 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: open chunk writer: started multipart upload: 0dd2fa8c-bb87-4bd1-911b-c1df6d793aa2 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 2 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: number of streams 4 was bigger than number of chunks 3 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: Starting multi-thread copy with 3 chunks of size 5Mi with 3 parallel streams 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: chunk 3/3 (10485760-10485761) size 1 starting 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: chunk 1/3 (0-5242880) size 5Mi starting 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: chunk 2/3 (5242880-10485760) size 5Mi starting 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: Seek from 1 to 0 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: Seek from 1 to 0 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multipart upload wrote chunk 3 with 1 bytes and etag "415290769594460e2e485922904f345d" 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: chunk 3/3 (10485760-10485761) size 1 finished 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: Seek from 5242880 to 0 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: Seek from 5242880 to 0 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: Seek from 5242880 to 0 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: Seek from 5242880 to 0 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multipart upload wrote chunk 1 with 5242880 bytes and etag "d9c97c9165c57653166b06546ceb60a7" 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: chunk 1/3 (0-5242880) size 5Mi finished 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multipart upload wrote chunk 2 with 5242880 bytes and etag "6883a31987309f769bf6b7033b0134ad" 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multi-thread copy: chunk 2/3 (5242880-10485760) size 5Mi finished 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: multipart upload "0dd2fa8c-bb87-4bd1-911b-c1df6d793aa2" finished 2025/10/25 01:54:26 DEBUG : test-multithread-copy-true-10485761-2: Finished multi-thread copy with 3 parts of size 5Mi --- PASS: TestMultithreadCopy (5.80s) --- PASS: TestMultithreadCopy/upload=true,size=10485760,streams=2 (2.86s) --- PASS: TestMultithreadCopy/upload=true,size=10485761,streams=2 (2.93s) === RUN TestMultithreadCopyAbort run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" 2025/10/25 01:54:28 DEBUG : chunksize-probe: open chunk writer: started multipart upload: 5dec6cfe-0929-4894-97a1-fafdee686ba5 2025/10/25 01:54:28 DEBUG : chunksize-probe: multipart upload "5dec6cfe-0929-4894-97a1-fafdee686ba5" aborted 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multi-thread copy: disabling buffering because source is local disk 2025/10/25 01:54:29 DEBUG : test-multithread-abort: open chunk writer: started multipart upload: 8d25eff8-8ff3-420e-a9fe-94ec10c318dc 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 1 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multi-thread copy: number of streams 4 was bigger than number of chunks 3 2025/10/25 01:54:29 DEBUG : test-multithread-abort: Starting multi-thread copy with 3 chunks of size 5Mi with 3 parallel streams 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multi-thread copy: chunk 3/3 (10485760-10485761) size 1 starting 2025/10/25 01:54:29 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multi-thread copy: chunk 1/3 (0-5242880) size 5Mi starting 2025/10/25 01:54:29 DEBUG : Open with options = [RangeOption(0,5242879)] 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multi-thread copy: chunk 2/3 (5242880-10485760) size 5Mi starting 2025/10/25 01:54:29 DEBUG : Open with options = [RangeOption(5242880,10485759)] 2025/10/25 01:54:29 DEBUG : test-multithread-abort: Seek from 5242880 to 0 2025/10/25 01:54:29 DEBUG : Open with options = [RangeOption(0,5242879)] 2025/10/25 01:54:29 DEBUG : test-multithread-abort: Seek from 5242880 to 0 2025/10/25 01:54:29 DEBUG : Open with options = [RangeOption(5242880,10485759)] 2025/10/25 01:54:29 DEBUG : test-multithread-abort: Seek from 5242880 to 0 2025/10/25 01:54:29 DEBUG : Open with options = [RangeOption(0,5242879)] 2025/10/25 01:54:29 DEBUG : test-multithread-abort: Seek from 5242880 to 0 2025/10/25 01:54:29 DEBUG : Open with options = [RangeOption(5242880,10485759)] 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multipart upload wrote chunk 1 with 5242880 bytes and etag "ee4efd7808ecedc58cc02e8cafb349c0" 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multi-thread copy: chunk 1/3 (0-5242880) size 5Mi finished 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multipart upload wrote chunk 2 with 5242880 bytes and etag "60ddda5602a94bc09c8facf1cd66cafa" 2025/10/25 01:54:29 DEBUG : test-multithread-abort: multi-thread copy: chunk 2/3 (5242880-10485760) size 5Mi finished 2025/10/25 01:54:30 DEBUG : Returning error reader 2025/10/25 01:54:30 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:30 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 1/10: BOOM: simulated read failure 2025/10/25 01:54:30 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:31 DEBUG : Returning error reader 2025/10/25 01:54:31 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:31 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 2/10: BOOM: simulated read failure 2025/10/25 01:54:31 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:32 DEBUG : Returning error reader 2025/10/25 01:54:32 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:32 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 3/10: BOOM: simulated read failure 2025/10/25 01:54:32 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:33 DEBUG : Returning error reader 2025/10/25 01:54:33 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:33 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 4/10: BOOM: simulated read failure 2025/10/25 01:54:33 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:34 DEBUG : Returning error reader 2025/10/25 01:54:34 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:34 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 5/10: BOOM: simulated read failure 2025/10/25 01:54:34 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:35 DEBUG : Returning error reader 2025/10/25 01:54:35 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:35 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 6/10: BOOM: simulated read failure 2025/10/25 01:54:35 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:36 DEBUG : Returning error reader 2025/10/25 01:54:36 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:36 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 7/10: BOOM: simulated read failure 2025/10/25 01:54:36 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:37 DEBUG : Returning error reader 2025/10/25 01:54:37 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:37 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 8/10: BOOM: simulated read failure 2025/10/25 01:54:37 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:38 DEBUG : Returning error reader 2025/10/25 01:54:38 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:38 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 9/10: BOOM: simulated read failure 2025/10/25 01:54:38 DEBUG : Open with options = [RangeOption(10485760,10485760)] 2025/10/25 01:54:39 DEBUG : Returning error reader 2025/10/25 01:54:39 DEBUG : BOOM: simulated read failure 2025/10/25 01:54:39 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 10/10: BOOM: simulated read failure 2025/10/25 01:54:39 DEBUG : test-multithread-abort: Reopen failed after offset 0 bytes read: failed to reopen: too many retries 2025/10/25 01:54:39 DEBUG : test-multithread-abort: multi-thread copy: chunk 3/3 failed: multi-thread copy: failed to write chunk: BOOM: simulated read failure 2025/10/25 01:54:39 DEBUG : test-multithread-abort: multi-thread copy: cancelling transfer on exit 2025/10/25 01:56:38 DEBUG : pacer: low level retry 1/2 (error operation error S3: AbortMultipartUpload, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Delete "http://127.0.0.1:28626/rclone-test-wayupug3totu/test-multithread-abort?uploadId=8d25eff8-8ff3-420e-a9fe-94ec10c318dc&x-id=AbortMultipartUpload": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 01:56:38 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/25 01:58:41 DEBUG : pacer: low level retry 2/2 (error operation error S3: AbortMultipartUpload, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Delete "http://127.0.0.1:28626/rclone-test-wayupug3totu/test-multithread-abort?uploadId=8d25eff8-8ff3-420e-a9fe-94ec10c318dc&x-id=AbortMultipartUpload": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 01:58:41 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/10/25 01:58:41 DEBUG : test-multithread-abort: multi-thread copy: abort failed: failed to abort multipart upload "8d25eff8-8ff3-420e-a9fe-94ec10c318dc": operation error S3: AbortMultipartUpload, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Delete "http://127.0.0.1:28626/rclone-test-wayupug3totu/test-multithread-abort?uploadId=8d25eff8-8ff3-420e-a9fe-94ec10c318dc&x-id=AbortMultipartUpload": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:00:34 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:00:34 DEBUG : pacer: Rate limited, increasing sleep to 40ms 2025/10/25 02:02:26 DEBUG : pacer: low level retry 2/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:02:26 DEBUG : pacer: Rate limited, increasing sleep to 80ms fstest.go:282: Error listing: operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:04:31 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:04:31 DEBUG : pacer: Rate limited, increasing sleep to 160ms 2025/10/25 02:06:34 DEBUG : pacer: low level retry 2/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:34 DEBUG : pacer: Rate limited, increasing sleep to 320ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:282 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:340 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/multithread_test.go:328 Error: Received unexpected error: operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused Test: TestMultithreadCopyAbort --- FAIL: TestMultithreadCopyAbort (726.19s) === RUN TestCheck run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" 2025/10/25 02:06:34 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:34 DEBUG : pacer: Rate limited, increasing sleep to 640ms run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 1/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:36 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:36 DEBUG : pacer: Rate limited, increasing sleep to 1.28s run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 2/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:38 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:38 DEBUG : pacer: Rate limited, increasing sleep to 2s run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 3/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:40 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 4/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:42 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 5/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:44 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 6/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:46 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 7/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:48 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 8/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:50 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 9/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:06:52 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:285: Failed to put "rutabaga" to "S3 bucket rclone-test-wayupug3totu": operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:08:46 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:41 DEBUG : pacer: low level retry 2/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:285 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:303 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:98 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:181 Error: Received unexpected error: operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused Test: TestCheck --- FAIL: TestCheck (247.25s) === RUN TestCheckDownload run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" 2025/10/25 02:10:41 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 1/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:43 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 2/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:45 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 3/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:47 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 4/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:49 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 5/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:51 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 6/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:53 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 7/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:55 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 8/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:57 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 9/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:10:59 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:285: Failed to put "rutabaga" to "S3 bucket rclone-test-wayupug3totu": operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:12:54 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:14:52 DEBUG : pacer: low level retry 2/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:285 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:303 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:98 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:204 Error: Received unexpected error: operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused Test: TestCheckDownload --- FAIL: TestCheckDownload (251.28s) === RUN TestCheckSizeOnly run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" 2025/10/25 02:14:52 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 1/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:14:54 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 2/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:14:56 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 3/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:14:58 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 4/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:15:00 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 5/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:15:02 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 6/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:15:04 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 7/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:15:06 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 8/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:15:08 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "rutabaga" to S3 bucket rclone-test-wayupug3totu: 9/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:15:10 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:285: Failed to put "rutabaga" to "S3 bucket rclone-test-wayupug3totu": operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/rutabaga?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:17:13 DEBUG : pacer: low level retry 1/2 (error operation error S3: ListObjectsV2, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:14 DEBUG : pacer: Reducing sleep to 1.5s run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:285 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:303 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:98 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:181 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCheckSizeOnly --- FAIL: TestCheckSizeOnly (141.92s) === RUN TestParseSumFile run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" 2025/10/25 02:17:15 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:15 DEBUG : pacer: Rate limited, increasing sleep to 2s run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 1/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:17 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 2/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:19 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 3/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:21 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 4/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:23 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 5/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:25 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 6/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:27 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 7/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:29 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 8/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:31 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "test.sum" to S3 bucket rclone-test-wayupug3totu: 9/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:17:33 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:285: Failed to put "test.sum" to "S3 bucket rclone-test-wayupug3totu": operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/test.sum?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:17:35 DEBUG : pacer: Reducing sleep to 1.5s run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:285 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:308 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestParseSumFile --- FAIL: TestParseSumFile (20.78s) === RUN TestCheckSum run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" 2025/10/25 02:17:35 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-wayupug3totu/data" 2025/10/25 02:17:35 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/25 02:17:35 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/25 02:17:35 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/25 02:17:35 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:17:35 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/25 02:17:35 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/25 02:17:35 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/25 02:17:35 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/25 02:17:35 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:19:32 DEBUG : pacer: low level retry 1/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-wayupug3totu/data": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:19:32 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/25 02:21:30 DEBUG : pacer: low level retry 2/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-wayupug3totu/data": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:30 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/10/25 02:21:30 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:30 DEBUG : pacer: Rate limited, increasing sleep to 2s run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 1/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:32 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 2/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:34 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 3/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:36 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 4/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:38 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 5/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:40 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 6/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:42 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 7/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:44 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 8/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:46 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 9/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:21:48 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:285: Failed to put "data/banana" to "S3 bucket rclone-test-wayupug3totu": operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:21:50 DEBUG : pacer: Reducing sleep to 1.5s run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:285 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:355 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:432 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:537 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCheckSum --- FAIL: TestCheckSum (255.19s) === RUN TestCheckSumDownload run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" 2025/10/25 02:21:50 DEBUG : Creating backend with remote "TestS3MinioEdge:rclone-test-wayupug3totu/data" 2025/10/25 02:21:50 DEBUG : Config file has changed externally - reloading 2025/10/25 02:21:50 DEBUG : Setting type="s3" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_TYPE 2025/10/25 02:21:50 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/25 02:21:50 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/25 02:21:50 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:21:50 DEBUG : TestS3MinioEdge: detected overridden config - adding "{FbV4e}" suffix to name 2025/10/25 02:21:50 DEBUG : Setting provider="Minio" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_PROVIDER 2025/10/25 02:21:50 DEBUG : Setting access_key_id="rclone" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ACCESS_KEY_ID 2025/10/25 02:21:50 DEBUG : Setting secret_access_key="DeniseOxygenEiffel4" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_SECRET_ACCESS_KEY 2025/10/25 02:21:50 DEBUG : Setting endpoint="http://127.0.0.1:28626/" for "TestS3MinioEdge" from environment variable RCLONE_CONFIG_TESTS3MINIOEDGE_ENDPOINT 2025/10/25 02:23:43 DEBUG : pacer: low level retry 1/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-wayupug3totu/data": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:23:43 DEBUG : pacer: Rate limited, increasing sleep to 10ms 2025/10/25 02:25:44 DEBUG : pacer: low level retry 2/2 (error operation error S3: HeadObject, exceeded maximum number of attempts, 10, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-wayupug3totu/data": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:44 DEBUG : pacer: Rate limited, increasing sleep to 20ms 2025/10/25 02:25:44 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:44 DEBUG : pacer: Rate limited, increasing sleep to 2s run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 1/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:46 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 2/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:48 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 3/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:50 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 4/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:52 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 5/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:54 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 6/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:56 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 7/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:25:58 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 8/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:00 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "data/banana" to S3 bucket rclone-test-wayupug3totu: 9/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:02 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:285: Failed to put "data/banana" to "S3 bucket rclone-test-wayupug3totu": operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/data/banana?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:26:04 DEBUG : pacer: Reducing sleep to 1.5s run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:285 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:355 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:432 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:541 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCheckSumDownload --- FAIL: TestCheckSumDownload (253.82s) === RUN TestCopyFile run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" fstest.go:298: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:298: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:298: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:108 Error: Should be true Test: TestCopyFile Messages: listing wrong, want file1 (14) got file1 (14), rutabaga (8), test-multithread-abort (10485761) fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:108 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFile Messages: Unexpected file "rutabaga" fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:108 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFile Messages: Unexpected file "test-multithread-abort" 2025/10/25 02:26:11 DEBUG : pacer: Reducing sleep to 1.125s copy_test.go:114: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:114 Error: Received unexpected error: operation error S3: HeadObject, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Head "http://127.0.0.1:28626/rclone-test-wayupug3totu/sub/file2": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCopyFile 2025/10/25 02:26:13 DEBUG : pacer: Reducing sleep to 843.75ms run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:114 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCopyFile --- FAIL: TestCopyFile (8.52s) === RUN TestCopyFileBackupDir run.go:180: Remote "S3 bucket rclone-test-wayupug3totu", Local "Local file system at /tmp/rclone1944841182", Modify Window "1ns" fstest.go:298: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:298: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:298: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:305: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:305 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:180 Error: Should be true Test: TestCopyFileBackupDir Messages: listing wrong, want dst/file1 (14) got dst/file1 (14), file1 (14), rutabaga (8), test-multithread-abort (10485761) fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:180 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileBackupDir Messages: Unexpected file "file1" fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:180 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileBackupDir Messages: Unexpected file "rutabaga" fstest.go:191: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:191 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:308 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:338 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:350 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:334 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:180 /usr/local/go/src/runtime/asm_amd64.s:1700 Error: Should be true Test: TestCopyFileBackupDir Messages: Unexpected file "test-multithread-abort" 2025/10/25 02:26:20 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:20 DEBUG : pacer: Rate limited, increasing sleep to 1.6875s run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 1/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:22 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:22 DEBUG : pacer: Rate limited, increasing sleep to 2s run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 2/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:24 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 3/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:26 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 4/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:28 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 5/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:30 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 6/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:32 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 7/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:34 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 8/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:36 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:281: Retry Put of "dst/file1" to S3 bucket rclone-test-wayupug3totu: 9/10 (operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) 2025/10/25 02:26:38 DEBUG : pacer: low level retry 1/1 (error operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused) run.go:285: Failed to put "dst/file1" to "S3 bucket rclone-test-wayupug3totu": operation error S3: PutObject, exceeded maximum number of attempts, 1, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Put "http://127.0.0.1:28626/rclone-test-wayupug3totu/dst/file1?x-id=PutObject": dial tcp 127.0.0.1:28626: connect: connection refused 2025/10/25 02:26:40 DEBUG : pacer: Reducing sleep to 1.5s run.go:164: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:164 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1211 /usr/local/go/src/testing/testing.go:1445 /usr/local/go/src/testing/testing.go:1786 /usr/local/go/src/runtime/panic.go:636 /usr/local/go/src/testing/testing.go:1041 /usr/local/go/src/testing/testing.go:1125 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:285 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:292 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/copy_test.go:182 Error: Received unexpected error: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested Test: TestCopyFileBackupDir --- FAIL: TestCopyFileBackupDir (27.03s) FAIL 2025/10/25 02:26:40 DEBUG : S3 bucket rclone-test-wayupug3totu: Purge remote 2025/10/25 02:26:42 DEBUG : pacer: Reducing sleep to 1.125s 2025/10/25 02:26:42 ERROR : S3 bucket rclone-test-wayupug3totu: Failed to read versioning status, assuming unversioned: operation error S3: GetBucketVersioning, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?versioning=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested 2025/10/25 02:26:42 DEBUG : S3 bucket rclone-test-wayupug3totu: bucket is versioned: false 2025/10/25 02:26:42 DEBUG : Waiting for deletions to finish 2025/10/25 02:26:43 DEBUG : pacer: Reducing sleep to 843.75ms 2025/10/25 02:26:44 DEBUG : pacer: Reducing sleep to 632.8125ms 2025/10/25 02:26:44 NOTICE: purge failed: operation error S3: ListObjectsV2, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "http://127.0.0.1:28626/rclone-test-wayupug3totu?delimiter=&encoding-type=url&list-type=2&max-keys=1000&prefix=": dial tcp 127.0.0.1:28626: connect: connection refused failed to get rate limit token, retry quota exceeded, 0 available, 5 requested "./operations.test -test.v -test.timeout 1h0m0s -remote TestS3MinioEdge: -verbose -fast-list -test.run '^(TestCheck|TestCheckDownload|TestCheckSizeOnly|TestCheckSum|TestCheckSumDownload|TestCopyFile|TestCopyFileBackupDir|TestMultithreadCopyAbort|TestParseSumFile)$|^TestMultithreadCopy$/^(upload=true,size=10485760,streams=2|upload=true,size=10485761,streams=2)$'" - Finished ERROR in 32m22.484087035s (try 2/5): exit status 1: Failed [TestMultithreadCopyAbort TestCheck TestCheckDownload TestCheckSizeOnly TestParseSumFile TestCheckSum TestCheckSumDownload TestCopyFile TestCopyFileBackupDir]