"./operations.test -test.v -test.timeout 1h0m0s -remote TestPcloud: -verbose" - Starting (try 1/5) 2024/11/14 19:02:06 DEBUG : Creating backend with remote "TestPcloud:rclone-test-bimeyaz6lisi" 2024/11/14 19:02:06 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2024/11/14 19:02:14 DEBUG : Creating backend with remote "/tmp/rclone840152747" === RUN TestDoMultiThreadCopy --- PASS: TestDoMultiThreadCopy (0.00s) === RUN TestMultithreadCalculateNumChunks === RUN TestMultithreadCalculateNumChunks/{size:1_chunkSize:65536_wantNumChunks:1} === RUN TestMultithreadCalculateNumChunks/{size:1048576_chunkSize:1_wantNumChunks:1048576} === RUN TestMultithreadCalculateNumChunks/{size:1048576_chunkSize:2_wantNumChunks:524288} === RUN TestMultithreadCalculateNumChunks/{size:1048577_chunkSize:2_wantNumChunks:524289} === RUN TestMultithreadCalculateNumChunks/{size:1048575_chunkSize:2_wantNumChunks:524288} --- PASS: TestMultithreadCalculateNumChunks (0.00s) --- PASS: TestMultithreadCalculateNumChunks/{size:1_chunkSize:65536_wantNumChunks:1} (0.00s) --- PASS: TestMultithreadCalculateNumChunks/{size:1048576_chunkSize:1_wantNumChunks:1048576} (0.00s) --- PASS: TestMultithreadCalculateNumChunks/{size:1048576_chunkSize:2_wantNumChunks:524288} (0.00s) --- PASS: TestMultithreadCalculateNumChunks/{size:1048577_chunkSize:2_wantNumChunks:524289} (0.00s) --- PASS: TestMultithreadCalculateNumChunks/{size:1048575_chunkSize:2_wantNumChunks:524288} (0.00s) === RUN TestMultithreadCopy run.go:180: Remote "pcloud root 'rclone-test-bimeyaz6lisi'", Local "Local file system at /tmp/rclone840152747", Modify Window "1s" === RUN TestMultithreadCopy/upload=false,size=134217727,streams=2 2024/11/14 19:02:25 DEBUG : test-multithread-copy-false-134217727-2: multi-thread copy: disabling buffering because destination uses OpenWriterAt 2024/11/14 19:02:25 DEBUG : test-multithread-copy-false-134217727-2: multi-thread copy: write buffer set to 131072 2024/11/14 19:02:25 DEBUG : test-multithread-copy-false-134217727-2: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 2 2024/11/14 19:02:25 DEBUG : test-multithread-copy-false-134217727-2: multi-thread copy: number of streams 4 was bigger than number of chunks 2 2024/11/14 19:02:25 DEBUG : test-multithread-copy-false-134217727-2: Starting multi-thread copy with 2 chunks of size 64Mi with 2 parallel streams 2024/11/14 19:02:25 DEBUG : test-multithread-copy-false-134217727-2: multi-thread copy: chunk 2/2 (67108864-134217727) size 64.000Mi starting 2024/11/14 19:02:25 DEBUG : test-multithread-copy-false-134217727-2: multi-thread copy: chunk 1/2 (0-67108864) size 64Mi starting 2024/11/14 19:02:26 DEBUG : Couldn't parse Date: from server edef8.pcloud.com: "Thu, 14 Nov 2024 19:02:26 +0000": parsing time "Thu, 14 Nov 2024 19:02:26 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 14 Nov 2024 19:02:26 +0000" as " " 2024/11/14 19:02:26 DEBUG : test-multithread-copy-false-134217727-2: writing chunk 0 2024/11/14 19:02:26 DEBUG : Couldn't parse Date: from server edef8.pcloud.com: "Thu, 14 Nov 2024 19:02:26 +0000": parsing time "Thu, 14 Nov 2024 19:02:26 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 14 Nov 2024 19:02:26 +0000" as " " 2024/11/14 19:02:26 DEBUG : test-multithread-copy-false-134217727-2: writing chunk 1 2024/11/14 19:02:28 DEBUG : test-multithread-copy-false-134217727-2: multi-thread copy: chunk 2/2 (67108864-134217727) size 64.000Mi finished 2024/11/14 19:02:28 DEBUG : test-multithread-copy-false-134217727-2: multi-thread copy: chunk 1/2 (0-67108864) size 64Mi finished 2024/11/14 19:02:28 DEBUG : test-multithread-copy-false-134217727-2: Finished multi-thread copy with 2 parts of size 64Mi === RUN TestMultithreadCopy/upload=false,size=134217728,streams=2 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: multi-thread copy: disabling buffering because destination uses OpenWriterAt 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: multi-thread copy: write buffer set to 131072 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 2 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: multi-thread copy: number of streams 4 was bigger than number of chunks 2 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: Starting multi-thread copy with 2 chunks of size 64Mi with 2 parallel streams 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: multi-thread copy: chunk 2/2 (67108864-134217728) size 64Mi starting 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: multi-thread copy: chunk 1/2 (0-67108864) size 64Mi starting 2024/11/14 19:02:44 DEBUG : Couldn't parse Date: from server edef12.pcloud.com: "Thu, 14 Nov 2024 19:02:44 +0000": parsing time "Thu, 14 Nov 2024 19:02:44 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 14 Nov 2024 19:02:44 +0000" as " " 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: writing chunk 1 2024/11/14 19:02:44 DEBUG : Couldn't parse Date: from server edef12.pcloud.com: "Thu, 14 Nov 2024 19:02:44 +0000": parsing time "Thu, 14 Nov 2024 19:02:44 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 14 Nov 2024 19:02:44 +0000" as " " 2024/11/14 19:02:44 DEBUG : test-multithread-copy-false-134217728-2: writing chunk 0 2024/11/14 19:02:46 DEBUG : test-multithread-copy-false-134217728-2: multi-thread copy: chunk 1/2 (0-67108864) size 64Mi finished 2024/11/14 19:02:46 DEBUG : test-multithread-copy-false-134217728-2: multi-thread copy: chunk 2/2 (67108864-134217728) size 64Mi finished 2024/11/14 19:02:46 DEBUG : test-multithread-copy-false-134217728-2: Finished multi-thread copy with 2 parts of size 64Mi === RUN TestMultithreadCopy/upload=false,size=134217729,streams=2 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: disabling buffering because destination uses OpenWriterAt 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: write buffer set to 131072 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 2 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: number of streams 4 was bigger than number of chunks 3 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: Starting multi-thread copy with 3 chunks of size 64Mi with 3 parallel streams 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: chunk 3/3 (134217728-134217729) size 1 starting 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: chunk 1/3 (0-67108864) size 64Mi starting 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: chunk 2/3 (67108864-134217728) size 64Mi starting 2024/11/14 19:03:25 DEBUG : Couldn't parse Date: from server edef12.pcloud.com: "Thu, 14 Nov 2024 19:03:25 +0000": parsing time "Thu, 14 Nov 2024 19:03:25 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 14 Nov 2024 19:03:25 +0000" as " " 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: writing chunk 2 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: chunk 3/3 (134217728-134217729) size 1 finished 2024/11/14 19:03:25 DEBUG : Couldn't parse Date: from server edef12.pcloud.com: "Thu, 14 Nov 2024 19:03:25 +0000": parsing time "Thu, 14 Nov 2024 19:03:25 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 14 Nov 2024 19:03:25 +0000" as " " 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: writing chunk 0 2024/11/14 19:03:25 DEBUG : Couldn't parse Date: from server edef12.pcloud.com: "Thu, 14 Nov 2024 19:03:25 +0000": parsing time "Thu, 14 Nov 2024 19:03:25 +0000" as "Mon Jan _2 15:04:05 2006": cannot parse ", 14 Nov 2024 19:03:25 +0000" as " " 2024/11/14 19:03:25 DEBUG : test-multithread-copy-false-134217729-2: writing chunk 1 2024/11/14 19:03:27 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: chunk 2/3 (67108864-134217728) size 64Mi finished 2024/11/14 19:03:27 DEBUG : test-multithread-copy-false-134217729-2: multi-thread copy: chunk 1/3 (0-67108864) size 64Mi finished 2024/11/14 19:03:27 DEBUG : test-multithread-copy-false-134217729-2: Finished multi-thread copy with 3 parts of size 64Mi === RUN TestMultithreadCopy/upload=true,size=134217727,streams=2 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: multi-thread copy: disabling buffering because destination uses OpenWriterAt 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: multi-thread copy: write buffer set to 131072 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 2 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: multi-thread copy: number of streams 4 was bigger than number of chunks 2 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: Starting multi-thread copy with 2 chunks of size 64Mi with 2 parallel streams 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: multi-thread copy: chunk 2/2 (67108864-134217727) size 64.000Mi starting 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: multi-thread copy: chunk 1/2 (0-67108864) size 64Mi starting 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: writing chunk 0 2024/11/14 19:03:41 DEBUG : test-multithread-copy-true-134217727-2: writing chunk 1 2024/11/14 19:04:58 DEBUG : test-multithread-copy-true-134217727-2: multi-thread copy: chunk 2/2 (67108864-134217727) size 64.000Mi finished 2024/11/14 19:04:58 DEBUG : test-multithread-copy-true-134217727-2: multi-thread copy: chunk 1/2 (0-67108864) size 64Mi finished 2024/11/14 19:04:58 DEBUG : test-multithread-copy-true-134217727-2: checking file size: try 0/5 2024/11/14 19:04:59 DEBUG : test-multithread-copy-true-134217727-2: checking file size: try 1/5 2024/11/14 19:05:00 DEBUG : test-multithread-copy-true-134217727-2: checking file size: try 2/5 2024/11/14 19:05:00 DEBUG : test-multithread-copy-true-134217727-2: Finished multi-thread copy with 2 parts of size 64Mi === RUN TestMultithreadCopy/upload=true,size=134217728,streams=2 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: multi-thread copy: disabling buffering because destination uses OpenWriterAt 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: multi-thread copy: write buffer set to 131072 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 2 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: multi-thread copy: number of streams 4 was bigger than number of chunks 2 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: Starting multi-thread copy with 2 chunks of size 64Mi with 2 parallel streams 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: multi-thread copy: chunk 2/2 (67108864-134217728) size 64Mi starting 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: multi-thread copy: chunk 1/2 (0-67108864) size 64Mi starting 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: writing chunk 0 2024/11/14 19:05:14 DEBUG : test-multithread-copy-true-134217728-2: writing chunk 1 2024/11/14 19:06:12 DEBUG : test-multithread-copy-true-134217728-2: multi-thread copy: chunk 2/2 (67108864-134217728) size 64Mi finished 2024/11/14 19:06:12 DEBUG : test-multithread-copy-true-134217728-2: multi-thread copy: chunk 1/2 (0-67108864) size 64Mi finished 2024/11/14 19:06:12 DEBUG : test-multithread-copy-true-134217728-2: checking file size: try 0/5 2024/11/14 19:06:13 DEBUG : test-multithread-copy-true-134217728-2: checking file size: try 1/5 2024/11/14 19:06:13 ERROR : test-multithread-copy-true-134217728-2: multi-thread copy: can't set metadata as SetMetadata isn't implemented in: pcloud root 'rclone-test-bimeyaz6lisi' 2024/11/14 19:06:13 DEBUG : test-multithread-copy-true-134217728-2: Finished multi-thread copy with 2 parts of size 64Mi === RUN TestMultithreadCopy/upload=true,size=134217729,streams=2 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: disabling buffering because destination uses OpenWriterAt 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: write buffer set to 131072 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 2 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: number of streams 4 was bigger than number of chunks 3 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: Starting multi-thread copy with 3 chunks of size 64Mi with 3 parallel streams 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: chunk 3/3 (134217728-134217729) size 1 starting 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: chunk 1/3 (0-67108864) size 64Mi starting 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: writing chunk 2 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: writing chunk 0 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: chunk 2/3 (67108864-134217728) size 64Mi starting 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: writing chunk 1 2024/11/14 19:06:27 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: chunk 3/3 (134217728-134217729) size 1 finished 2024/11/14 19:07:28 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: chunk 1/3 (0-67108864) size 64Mi finished 2024/11/14 19:07:28 DEBUG : test-multithread-copy-true-134217729-2: multi-thread copy: chunk 2/3 (67108864-134217728) size 64Mi finished 2024/11/14 19:07:28 DEBUG : test-multithread-copy-true-134217729-2: checking file size: try 0/5 2024/11/14 19:07:29 DEBUG : test-multithread-copy-true-134217729-2: checking file size: try 1/5 2024/11/14 19:07:30 DEBUG : test-multithread-copy-true-134217729-2: checking file size: try 2/5 2024/11/14 19:07:30 DEBUG : test-multithread-copy-true-134217729-2: Finished multi-thread copy with 3 parts of size 64Mi --- PASS: TestMultithreadCopy (321.27s) --- PASS: TestMultithreadCopy/upload=false,size=134217727,streams=2 (18.55s) --- PASS: TestMultithreadCopy/upload=false,size=134217728,streams=2 (17.95s) --- PASS: TestMultithreadCopy/upload=false,size=134217729,streams=2 (41.06s) --- PASS: TestMultithreadCopy/upload=true,size=134217727,streams=2 (93.67s) --- PASS: TestMultithreadCopy/upload=true,size=134217728,streams=2 (72.47s) --- PASS: TestMultithreadCopy/upload=true,size=134217729,streams=2 (77.39s) === RUN TestMultithreadCopyAbort run.go:180: Remote "pcloud root 'rclone-test-bimeyaz6lisi'", Local "Local file system at /tmp/rclone840152747", Modify Window "1s" 2024/11/14 19:07:45 DEBUG : test-multithread-abort: multi-thread copy: disabling buffering because destination uses OpenWriterAt 2024/11/14 19:07:45 DEBUG : test-multithread-abort: multi-thread copy: write buffer set to 131072 2024/11/14 19:07:45 DEBUG : test-multithread-abort: multi-thread copy: using backend concurrency of 4 instead of --multi-thread-streams 1 2024/11/14 19:07:45 DEBUG : test-multithread-abort: multi-thread copy: number of streams 4 was bigger than number of chunks 3 2024/11/14 19:07:45 DEBUG : test-multithread-abort: Starting multi-thread copy with 3 chunks of size 64Mi with 3 parallel streams 2024/11/14 19:07:45 DEBUG : test-multithread-abort: multi-thread copy: chunk 3/3 (134217728-134217729) size 1 starting 2024/11/14 19:07:45 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:07:45 DEBUG : test-multithread-abort: multi-thread copy: chunk 1/3 (0-67108864) size 64Mi starting 2024/11/14 19:07:45 DEBUG : Open with options = [RangeOption(0,67108863)] 2024/11/14 19:07:45 DEBUG : test-multithread-abort: multi-thread copy: chunk 2/3 (67108864-134217728) size 64Mi starting 2024/11/14 19:07:45 DEBUG : Open with options = [RangeOption(67108864,134217727)] 2024/11/14 19:07:45 DEBUG : test-multithread-abort: writing chunk 0 2024/11/14 19:07:45 DEBUG : test-multithread-abort: writing chunk 1 2024/11/14 19:08:59 DEBUG : test-multithread-abort: multi-thread copy: chunk 2/3 (67108864-134217728) size 64Mi finished 2024/11/14 19:08:59 DEBUG : test-multithread-abort: multi-thread copy: chunk 1/3 (0-67108864) size 64Mi finished 2024/11/14 19:08:59 DEBUG : Returning error reader 2024/11/14 19:08:59 DEBUG : test-multithread-abort: writing chunk 2 2024/11/14 19:08:59 DEBUG : BOOM: simulated read failure 2024/11/14 19:08:59 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 1/10: BOOM: simulated read failure 2024/11/14 19:08:59 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:00 DEBUG : Returning error reader 2024/11/14 19:09:00 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:00 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 2/10: BOOM: simulated read failure 2024/11/14 19:09:00 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:01 DEBUG : Returning error reader 2024/11/14 19:09:01 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:01 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 3/10: BOOM: simulated read failure 2024/11/14 19:09:01 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:02 DEBUG : Returning error reader 2024/11/14 19:09:02 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:02 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 4/10: BOOM: simulated read failure 2024/11/14 19:09:02 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:03 DEBUG : Returning error reader 2024/11/14 19:09:03 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:03 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 5/10: BOOM: simulated read failure 2024/11/14 19:09:03 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:04 DEBUG : Returning error reader 2024/11/14 19:09:04 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:04 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 6/10: BOOM: simulated read failure 2024/11/14 19:09:04 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:05 DEBUG : Returning error reader 2024/11/14 19:09:05 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:05 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 7/10: BOOM: simulated read failure 2024/11/14 19:09:05 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:06 DEBUG : Returning error reader 2024/11/14 19:09:06 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:06 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 8/10: BOOM: simulated read failure 2024/11/14 19:09:06 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:07 DEBUG : Returning error reader 2024/11/14 19:09:07 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:07 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 9/10: BOOM: simulated read failure 2024/11/14 19:09:07 DEBUG : Open with options = [RangeOption(134217728,134217728)] 2024/11/14 19:09:08 DEBUG : Returning error reader 2024/11/14 19:09:08 DEBUG : BOOM: simulated read failure 2024/11/14 19:09:08 DEBUG : test-multithread-abort: Reopening on read failure after offset 0 bytes: retry 10/10: BOOM: simulated read failure 2024/11/14 19:09:08 DEBUG : test-multithread-abort: Reopen failed after offset 0 bytes read: failed to reopen: too many retries 2024/11/14 19:09:08 DEBUG : test-multithread-abort: multi-thread copy: chunk 3/3 failed: multi-thread copy: failed to write chunk: BOOM: simulated read failure 2024/11/14 19:09:08 DEBUG : test-multithread-abort: multi-thread copy: cancelling transfer on exit 2024/11/14 19:09:08 DEBUG : test-multithread-abort: checking file size: try 0/5 2024/11/14 19:09:09 DEBUG : test-multithread-abort: checking file size: try 1/5 2024/11/14 19:09:11 DEBUG : test-multithread-abort: checking file size: try 2/5 2024/11/14 19:09:12 DEBUG : test-multithread-abort: checking file size: try 3/5 2024/11/14 19:09:13 DEBUG : test-multithread-abort: checking file size: try 4/5 2024/11/14 19:09:14 ERROR : test-multithread-abort: multi-thread copy: failed to close file before aborting: incorrect size after upload: got 134217728, want 134217729 fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:286: Sleeping for 6s just to make sure --- PASS: TestMultithreadCopyAbort (106.72s) === RUN TestSizeDiffers --- PASS: TestSizeDiffers (0.00s) === RUN TestReOpen === RUN TestReOpen/Normal === RUN TestReOpen/Normal/Basics 2024/11/14 19:09:22 DEBUG : potato: Seek from 10 to 0 === RUN TestReOpen/Normal/ErrorAtStart === RUN TestReOpen/Normal/WithErrors 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 1/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 2/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 6 bytes: retry 3/10: test error === RUN TestReOpen/Normal/TooManyErrors 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 1/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 2/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 6 bytes: retry 3/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopen failed after offset 6 bytes read: failed to reopen: too many retries === RUN TestReOpen/Normal/Seek 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 0/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 1/10: test error 2024/11/14 19:09:22 DEBUG : potato: Seek from 5 to 2 === RUN TestReOpen/Normal/AccountRead === RUN TestReOpen/Normal/AccountReadDelay 2024/11/14 19:09:22 DEBUG : potato: Seek from 10 to 0 2024/11/14 19:09:22 DEBUG : potato: Seek from 10 to 0 2024/11/14 19:09:22 DEBUG : potato: Seek from 10 to 0 === RUN TestReOpen/Normal/AccountReadError === RUN TestReOpen/WithRangeOption === RUN TestReOpen/WithRangeOption/Basics 2024/11/14 19:09:22 DEBUG : potato: Seek from 7 to 0 === RUN TestReOpen/WithRangeOption/ErrorAtStart === RUN TestReOpen/WithRangeOption/WithErrors 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 1/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 2/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 6 bytes: retry 3/10: test error === RUN TestReOpen/WithRangeOption/TooManyErrors 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 1/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 2/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 6 bytes: retry 3/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopen failed after offset 6 bytes read: failed to reopen: too many retries === RUN TestReOpen/WithRangeOption/Seek 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 0/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 1/10: test error 2024/11/14 19:09:22 DEBUG : potato: Seek from 5 to 2 2024/11/14 19:09:22 DEBUG : potato: Seek from 7 to 4 === RUN TestReOpen/WithRangeOption/AccountRead === RUN TestReOpen/WithRangeOption/AccountReadDelay 2024/11/14 19:09:22 DEBUG : potato: Seek from 7 to 0 2024/11/14 19:09:22 DEBUG : potato: Seek from 7 to 0 2024/11/14 19:09:22 DEBUG : potato: Seek from 7 to 0 === RUN TestReOpen/WithRangeOption/AccountReadError === RUN TestReOpen/WithSeekOption === RUN TestReOpen/WithSeekOption/Basics 2024/11/14 19:09:22 DEBUG : potato: Seek from 8 to 0 === RUN TestReOpen/WithSeekOption/ErrorAtStart === RUN TestReOpen/WithSeekOption/WithErrors 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 1/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 2/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 6 bytes: retry 3/10: test error === RUN TestReOpen/WithSeekOption/TooManyErrors 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 1/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 2/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 6 bytes: retry 3/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopen failed after offset 6 bytes read: failed to reopen: too many retries === RUN TestReOpen/WithSeekOption/Seek 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 0/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 1/10: test error 2024/11/14 19:09:22 DEBUG : potato: Seek from 5 to 2 2024/11/14 19:09:22 DEBUG : potato: Seek from 7 to 5 === RUN TestReOpen/WithSeekOption/AccountRead === RUN TestReOpen/WithSeekOption/AccountReadDelay 2024/11/14 19:09:22 DEBUG : potato: Seek from 8 to 0 2024/11/14 19:09:22 DEBUG : potato: Seek from 8 to 0 2024/11/14 19:09:22 DEBUG : potato: Seek from 8 to 0 === RUN TestReOpen/WithSeekOption/AccountReadError === RUN TestReOpen/UnknownSize === RUN TestReOpen/UnknownSize/Basics 2024/11/14 19:09:22 DEBUG : potato: Seek from 9 to 0 === RUN TestReOpen/UnknownSize/ErrorAtStart === RUN TestReOpen/UnknownSize/WithErrors 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 1/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 2/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 6 bytes: retry 3/10: test error === RUN TestReOpen/UnknownSize/TooManyErrors 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 1/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 2/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 6 bytes: retry 3/3: test error 2024/11/14 19:09:22 DEBUG : potato: Reopen failed after offset 6 bytes read: failed to reopen: too many retries === RUN TestReOpen/UnknownSize/Seek 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 2 bytes: retry 0/10: test error 2024/11/14 19:09:22 DEBUG : potato: Reopening on read failure after offset 3 bytes: retry 1/10: test error 2024/11/14 19:09:22 DEBUG : potato: Seek from 5 to 2 2024/11/14 19:09:22 DEBUG : potato: Seek from 7 to 6 === RUN TestReOpen/UnknownSize/AccountRead === RUN TestReOpen/UnknownSize/AccountReadDelay 2024/11/14 19:09:22 DEBUG : potato: Seek from 9 to 0 2024/11/14 19:09:22 DEBUG : potato: Seek from 9 to 0 2024/11/14 19:09:22 DEBUG : potato: Seek from 9 to 0 === RUN TestReOpen/UnknownSize/AccountReadError --- PASS: TestReOpen (0.00s) --- PASS: TestReOpen/Normal (0.00s) --- PASS: TestReOpen/Normal/Basics (0.00s) --- PASS: TestReOpen/Normal/ErrorAtStart (0.00s) --- PASS: TestReOpen/Normal/WithErrors (0.00s) --- PASS: TestReOpen/Normal/TooManyErrors (0.00s) --- PASS: TestReOpen/Normal/Seek (0.00s) --- PASS: TestReOpen/Normal/AccountRead (0.00s) --- PASS: TestReOpen/Normal/AccountReadDelay (0.00s) --- PASS: TestReOpen/Normal/AccountReadError (0.00s) --- PASS: TestReOpen/WithRangeOption (0.00s) --- PASS: TestReOpen/WithRangeOption/Basics (0.00s) --- PASS: TestReOpen/WithRangeOption/ErrorAtStart (0.00s) --- PASS: TestReOpen/WithRangeOption/WithErrors (0.00s) --- PASS: TestReOpen/WithRangeOption/TooManyErrors (0.00s) --- PASS: TestReOpen/WithRangeOption/Seek (0.00s) --- PASS: TestReOpen/WithRangeOption/AccountRead (0.00s) --- PASS: TestReOpen/WithRangeOption/AccountReadDelay (0.00s) --- PASS: TestReOpen/WithRangeOption/AccountReadError (0.00s) --- PASS: TestReOpen/WithSeekOption (0.00s) --- PASS: TestReOpen/WithSeekOption/Basics (0.00s) --- PASS: TestReOpen/WithSeekOption/ErrorAtStart (0.00s) --- PASS: TestReOpen/WithSeekOption/WithErrors (0.00s) --- PASS: TestReOpen/WithSeekOption/TooManyErrors (0.00s) --- PASS: TestReOpen/WithSeekOption/Seek (0.00s) --- PASS: TestReOpen/WithSeekOption/AccountRead (0.00s) --- PASS: TestReOpen/WithSeekOption/AccountReadDelay (0.00s) --- PASS: TestReOpen/WithSeekOption/AccountReadError (0.00s) --- PASS: TestReOpen/UnknownSize (0.00s) --- PASS: TestReOpen/UnknownSize/Basics (0.00s) --- PASS: TestReOpen/UnknownSize/ErrorAtStart (0.00s) --- PASS: TestReOpen/UnknownSize/WithErrors (0.00s) --- PASS: TestReOpen/UnknownSize/TooManyErrors (0.00s) --- PASS: TestReOpen/UnknownSize/Seek (0.00s) --- PASS: TestReOpen/UnknownSize/AccountRead (0.00s) --- PASS: TestReOpen/UnknownSize/AccountReadDelay (0.00s) --- PASS: TestReOpen/UnknownSize/AccountReadError (0.00s) === RUN TestCheck run.go:180: Remote "pcloud root 'rclone-test-bimeyaz6lisi'", Local "Local file system at /tmp/rclone840152747", Modify Window "1s" === RUN TestCheck/1 === RUN TestCheck/2 === RUN TestCheck/3 === RUN TestCheck/4 === RUN TestCheck/5 === RUN TestCheck/6 === RUN TestCheck/7 --- PASS: TestCheck (3.44s) --- PASS: TestCheck/1 (0.17s) --- PASS: TestCheck/2 (0.17s) --- PASS: TestCheck/3 (0.17s) --- PASS: TestCheck/4 (0.18s) --- PASS: TestCheck/5 (0.18s) --- PASS: TestCheck/6 (0.20s) --- PASS: TestCheck/7 (0.18s) === RUN TestCheckFsError 2024/11/14 19:09:26 DEBUG : Creating backend with remote "nonexistent" 2024/11/14 19:09:26 DEBUG : Creating backend with remote "nonexistent" 2024/11/14 19:09:26 DEBUG : Local file system at /home/rclone/go/src/github.com/rclone/rclone/fs/operations/nonexistent: Waiting for checks to finish 2024/11/14 19:09:26 ERROR : Local file system at /home/rclone/go/src/github.com/rclone/rclone/fs/operations/nonexistent: error reading source root directory: directory not found 2024/11/14 19:09:26 NOTICE: Local file system at /home/rclone/go/src/github.com/rclone/rclone/fs/operations/nonexistent: 2 differences found 2024/11/14 19:09:26 NOTICE: Local file system at /home/rclone/go/src/github.com/rclone/rclone/fs/operations/nonexistent: 2 errors while checking --- PASS: TestCheckFsError (0.00s) === RUN TestCheckDownload run.go:180: Remote "pcloud root 'rclone-test-bimeyaz6lisi'", Local "Local file system at /tmp/rclone840152747", Modify Window "1s" === RUN TestCheckDownload/1 === RUN TestCheckDownload/2 === RUN TestCheckDownload/3 === RUN TestCheckDownload/4 === RUN TestCheckDownload/5 === NAME TestCheckDownload fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:286: Sleeping for 6s just to make sure === RUN TestCheckDownload/6 === RUN TestCheckDownload/7 --- PASS: TestCheckDownload (11.95s) --- PASS: TestCheckDownload/1 (0.28s) --- PASS: TestCheckDownload/2 (0.28s) --- PASS: TestCheckDownload/3 (0.28s) --- PASS: TestCheckDownload/4 (0.27s) --- PASS: TestCheckDownload/5 (0.34s) --- PASS: TestCheckDownload/6 (0.27s) --- PASS: TestCheckDownload/7 (0.27s) === RUN TestCheckSizeOnly run.go:180: Remote "pcloud root 'rclone-test-bimeyaz6lisi'", Local "Local file system at /tmp/rclone840152747", Modify Window "1s" === RUN TestCheckSizeOnly/1 === RUN TestCheckSizeOnly/2 === RUN TestCheckSizeOnly/3 check_test.go:89: 3: Expecting 2 errors but got 1 === NAME TestCheckSizeOnly check_test.go:53: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:53 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:57 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:97 Error: Not equal: expected: []string{"", "+ potato2", "- empty space", "= rutabaga"} actual : []string{"", "+ potato2", "= rutabaga"} Diff: --- Expected +++ Actual @@ -1,5 +1,4 @@ -([]string) (len=4) { +([]string) (len=3) { (string) "", (string) (len=9) "+ potato2", - (string) (len=13) "- empty space", (string) (len=10) "= rutabaga" Test: TestCheckSizeOnly Messages: combined check_test.go:53: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:53 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:58 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:97 Error: Not equal: expected: []string{"", "empty space"} actual : []string{} Diff: --- Expected +++ Actual @@ -1,4 +1,2 @@ -([]string) (len=2) { - (string) "", - (string) (len=11) "empty space" +([]string) { } Test: TestCheckSizeOnly Messages: missingonsrc fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:286: Sleeping for 6s just to make sure === RUN TestCheckSizeOnly/4 === RUN TestCheckSizeOnly/5 === RUN TestCheckSizeOnly/6 === RUN TestCheckSizeOnly/7 --- FAIL: TestCheckSizeOnly (10.27s) --- PASS: TestCheckSizeOnly/1 (0.09s) --- PASS: TestCheckSizeOnly/2 (0.08s) --- FAIL: TestCheckSizeOnly/3 (0.09s) --- PASS: TestCheckSizeOnly/4 (0.08s) --- PASS: TestCheckSizeOnly/5 (0.09s) --- PASS: TestCheckSizeOnly/6 (0.09s) --- PASS: TestCheckSizeOnly/7 (0.09s) === RUN TestCheckEqualReaders --- PASS: TestCheckEqualReaders (0.01s) === RUN TestParseSumFile run.go:180: Remote "pcloud root 'rclone-test-bimeyaz6lisi'", Local "Local file system at /tmp/rclone840152747", Modify Window "1s" check_test.go:313: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:313 Error: Received unexpected error: object not found Test: TestParseSumFile fstest.go:292: Sleeping for 1s for list eventual consistency: 1/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 2s for list eventual consistency: 2/3 fstest.go:295: Flushing the directory cache fstest.go:292: Sleeping for 4s for list eventual consistency: 3/3 fstest.go:295: Flushing the directory cache fstest.go:299: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:299 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:173 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1175 /usr/local/go/src/testing/testing.go:1353 /usr/local/go/src/testing/testing.go:1683 /usr/local/go/src/runtime/panic.go:770 /usr/local/go/src/runtime/panic.go:261 /usr/local/go/src/runtime/signal_unix.go:881 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/reopen.go:65 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/reopen.go:124 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check.go:565 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:314 Error: Should be true Test: TestParseSumFile Messages: listing wrong, want got test.sum (81) fstest.go:185: Error Trace: /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:185 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:302 /home/rclone/go/src/github.com/rclone/rclone/fstest/fstest.go:332 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:173 /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:388 /usr/local/go/src/testing/testing.go:1175 /usr/local/go/src/testing/testing.go:1353 /usr/local/go/src/testing/testing.go:1683 /usr/local/go/src/runtime/panic.go:770 /usr/local/go/src/runtime/panic.go:261 /usr/local/go/src/runtime/signal_unix.go:881 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/reopen.go:65 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/reopen.go:124 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check.go:565 /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:314 Error: Should be true Test: TestParseSumFile Messages: Unexpected file "test.sum" --- FAIL: TestParseSumFile (8.11s) panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0xa4147b] goroutine 11081 [running]: testing.tRunner.func1.2({0x1ee5b20, 0x380eee0}) /usr/local/go/src/testing/testing.go:1631 +0x24a testing.tRunner.func1() /usr/local/go/src/testing/testing.go:1634 +0x377 panic({0x1ee5b20?, 0x380eee0?}) /usr/local/go/src/runtime/panic.go:770 +0x132 github.com/rclone/rclone/fs/operations.NewReOpen({0x26f5140, 0x38f1840}, {0x0, 0x0}, 0xa, {0x0, 0x0, 0x0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/reopen.go:65 +0x7b github.com/rclone/rclone/fs/operations.Open({0x26f5140, 0x38f1840}, {0x0, 0x0}, {0x0, 0x0, 0x0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/reopen.go:124 +0x6b github.com/rclone/rclone/fs/operations.ParseSumFile({0x26f5140, 0x38f1840}, {0x0, 0x0}) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check.go:565 +0x4a github.com/rclone/rclone/fs/operations_test.TestParseSumFile(0xc01f4ee000) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:314 +0x279 testing.tRunner(0xc01f4ee000, 0x2449568) /usr/local/go/src/testing/testing.go:1689 +0xfb created by testing.(*T).Run in goroutine 1 /usr/local/go/src/testing/testing.go:1742 +0x390 "./operations.test -test.v -test.timeout 1h0m0s -remote TestPcloud: -verbose" - Finished ERROR in 7m49.902020779s (try 1/5): exit status 2: Failed [TestCheckSizeOnly/3 TestParseSumFile]