"./operations.test -test.v -test.timeout 1h0m0s -remote TestIA:rclone-integration-test -verbose" - Starting (try 2/5) 2022/06/17 09:05:06 DEBUG : Creating backend with remote "TestIA:rclone-integration-test/rclone-test-rikeqad5kobileg6himizeq4" 2022/06/17 09:05:06 DEBUG : Using config file from "/home/rclone/.rclone.conf" 2022/06/17 09:05:07 DEBUG : Creating backend with remote "/tmp/rclone551965667" === RUN TestDoMultiThreadCopy --- PASS: TestDoMultiThreadCopy (0.00s) === RUN TestMultithreadCalculateChunks === RUN TestMultithreadCalculateChunks/{size:1_streams:10_wantPartSize:65536_wantStreams:1} === RUN TestMultithreadCalculateChunks/{size:1048576_streams:1_wantPartSize:1048576_wantStreams:1} === RUN TestMultithreadCalculateChunks/{size:1048576_streams:2_wantPartSize:524288_wantStreams:2} === RUN TestMultithreadCalculateChunks/{size:1048577_streams:2_wantPartSize:589824_wantStreams:2} === RUN TestMultithreadCalculateChunks/{size:1048575_streams:2_wantPartSize:524288_wantStreams:2} --- PASS: TestMultithreadCalculateChunks (0.00s) --- PASS: TestMultithreadCalculateChunks/{size:1_streams:10_wantPartSize:65536_wantStreams:1} (0.00s) --- PASS: TestMultithreadCalculateChunks/{size:1048576_streams:1_wantPartSize:1048576_wantStreams:1} (0.00s) --- PASS: TestMultithreadCalculateChunks/{size:1048576_streams:2_wantPartSize:524288_wantStreams:2} (0.00s) --- PASS: TestMultithreadCalculateChunks/{size:1048577_streams:2_wantPartSize:589824_wantStreams:2} (0.00s) --- PASS: TestMultithreadCalculateChunks/{size:1048575_streams:2_wantPartSize:524288_wantStreams:2} (0.00s) === RUN TestMultithreadCopy run.go:181: Remote "Internet Archive item rclone-integration-test path rclone-test-rikeqad5kobileg6himizeq4", Local "Local file system at /tmp/rclone551965667", Modify Window "1ns" === RUN TestMultithreadCopy/{size:131071_streams:2} 2022/06/17 09:06:04 DEBUG : file1: Starting multi-thread copy with 2 parts of size 64Ki 2022/06/17 09:06:04 DEBUG : file1: multi-thread copy: stream 2/2 (65536-131071) size 63.999Ki starting 2022/06/17 09:06:04 DEBUG : file1: multi-thread copy: stream 1/2 (0-65536) size 64Ki starting 2022/06/17 09:06:05 DEBUG : file1: multi-thread copy: stream 2/2 (65536-131071) size 63.999Ki finished 2022/06/17 09:06:06 DEBUG : file1: multi-thread copy: stream 1/2 (0-65536) size 64Ki finished 2022/06/17 09:06:06 DEBUG : file1: Finished multi-thread copy with 2 parts of size 64Ki === RUN TestMultithreadCopy/{size:131072_streams:2} 2022/06/17 09:08:07 DEBUG : file1: Starting multi-thread copy with 2 parts of size 64Ki 2022/06/17 09:08:07 DEBUG : file1: multi-thread copy: stream 2/2 (65536-131072) size 64Ki starting 2022/06/17 09:08:07 DEBUG : file1: multi-thread copy: stream 1/2 (0-65536) size 64Ki starting 2022/06/17 09:08:09 DEBUG : file1: multi-thread copy: stream 2/2 (65536-131072) size 64Ki finished 2022/06/17 09:08:09 DEBUG : file1: multi-thread copy: stream 1/2 (0-65536) size 64Ki finished 2022/06/17 09:08:09 DEBUG : file1: Finished multi-thread copy with 2 parts of size 64Ki === RUN TestMultithreadCopy/{size:131073_streams:2} 2022/06/17 09:10:20 DEBUG : file1: Starting multi-thread copy with 2 parts of size 128Ki 2022/06/17 09:10:20 DEBUG : file1: multi-thread copy: stream 2/2 (131072-131073) size 1 starting 2022/06/17 09:10:20 DEBUG : file1: multi-thread copy: stream 1/2 (0-131072) size 128Ki starting 2022/06/17 09:10:22 DEBUG : file1: multi-thread copy: stream 2/2 (131072-131073) size 1 finished 2022/06/17 09:10:22 DEBUG : file1: multi-thread copy: stream 1/2 (0-131072) size 128Ki finished 2022/06/17 09:10:22 DEBUG : file1: Finished multi-thread copy with 2 parts of size 128Ki --- PASS: TestMultithreadCopy (424.09s) --- PASS: TestMultithreadCopy/{size:131071_streams:2} (58.68s) --- PASS: TestMultithreadCopy/{size:131072_streams:2} (123.54s) --- PASS: TestMultithreadCopy/{size:131073_streams:2} (133.13s) === RUN TestSizeDiffers --- PASS: TestSizeDiffers (0.00s) === RUN TestReOpen === RUN TestReOpen/Seek === RUN TestReOpen/Seek/Basics === RUN TestReOpen/Seek/ErrorAtStart === RUN TestReOpen/Seek/WithErrors 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 2 bytes: retry 1/10: test error 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 3 bytes: retry 2/10: test error 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 6 bytes: retry 3/10: test error === RUN TestReOpen/Seek/TooManyErrors 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 2 bytes: retry 1/3: test error 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 3 bytes: retry 2/3: test error 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 6 bytes: retry 3/3: test error 2022/06/17 09:12:11 DEBUG : potato: Reopen failed after 6 bytes read: failed to reopen: too many retries === RUN TestReOpen/Range === RUN TestReOpen/Range/Basics === RUN TestReOpen/Range/ErrorAtStart === RUN TestReOpen/Range/WithErrors 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 2 bytes: retry 1/10: test error 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 3 bytes: retry 2/10: test error 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 6 bytes: retry 3/10: test error === RUN TestReOpen/Range/TooManyErrors 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 2 bytes: retry 1/3: test error 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 3 bytes: retry 2/3: test error 2022/06/17 09:12:11 DEBUG : potato: Reopening on read failure after 6 bytes: retry 3/3: test error 2022/06/17 09:12:11 DEBUG : potato: Reopen failed after 6 bytes read: failed to reopen: too many retries --- PASS: TestReOpen (0.00s) --- PASS: TestReOpen/Seek (0.00s) --- PASS: TestReOpen/Seek/Basics (0.00s) --- PASS: TestReOpen/Seek/ErrorAtStart (0.00s) --- PASS: TestReOpen/Seek/WithErrors (0.00s) --- PASS: TestReOpen/Seek/TooManyErrors (0.00s) --- PASS: TestReOpen/Range (0.00s) --- PASS: TestReOpen/Range/Basics (0.00s) --- PASS: TestReOpen/Range/ErrorAtStart (0.00s) --- PASS: TestReOpen/Range/WithErrors (0.00s) --- PASS: TestReOpen/Range/TooManyErrors (0.00s) === RUN TestCheck run.go:181: Remote "Internet Archive item rclone-integration-test path rclone-test-rikeqad5kobileg6himizeq4", Local "Local file system at /tmp/rclone551965667", Modify Window "1ns" === RUN TestCheck/1 === RUN TestCheck/2 === RUN TestCheck/3 === RUN TestCheck/4 === RUN TestCheck/5 === RUN TestCheck/6 === RUN TestCheck/7 --- PASS: TestCheck (880.40s) --- PASS: TestCheck/1 (0.41s) --- PASS: TestCheck/2 (0.40s) --- PASS: TestCheck/3 (0.23s) --- PASS: TestCheck/4 (0.25s) --- PASS: TestCheck/5 (0.26s) --- PASS: TestCheck/6 (0.25s) --- PASS: TestCheck/7 (0.25s) === RUN TestCheckFsError 2022/06/17 09:26:52 DEBUG : Creating backend with remote "non-existent" 2022/06/17 09:26:52 DEBUG : Creating backend with remote "non-existent" 2022/06/17 09:26:52 DEBUG : Local file system at /home/rclone/go/src/github.com/rclone/rclone/fs/operations/non-existent: Waiting for checks to finish 2022/06/17 09:26:52 ERROR : Local file system at /home/rclone/go/src/github.com/rclone/rclone/fs/operations/non-existent: error reading source root directory: directory not found 2022/06/17 09:26:52 NOTICE: Local file system at /home/rclone/go/src/github.com/rclone/rclone/fs/operations/non-existent: 2 differences found 2022/06/17 09:26:52 NOTICE: Local file system at /home/rclone/go/src/github.com/rclone/rclone/fs/operations/non-existent: 2 errors while checking --- PASS: TestCheckFsError (0.00s) === RUN TestCheckDownload run.go:181: Remote "Internet Archive item rclone-integration-test path rclone-test-rikeqad5kobileg6himizeq4", Local "Local file system at /tmp/rclone551965667", Modify Window "1ns" === RUN TestCheckDownload/1 === RUN TestCheckDownload/2 === RUN TestCheckDownload/3 === RUN TestCheckDownload/4 === RUN TestCheckDownload/5 === RUN TestCheckDownload/6 === RUN TestCheckDownload/7 --- PASS: TestCheckDownload (981.41s) --- PASS: TestCheckDownload/1 (1.31s) --- PASS: TestCheckDownload/2 (1.11s) --- PASS: TestCheckDownload/3 (1.25s) --- PASS: TestCheckDownload/4 (1.26s) --- PASS: TestCheckDownload/5 (0.92s) --- PASS: TestCheckDownload/6 (1.12s) --- PASS: TestCheckDownload/7 (0.79s) === RUN TestCheckSizeOnly run.go:181: Remote "Internet Archive item rclone-integration-test path rclone-test-rikeqad5kobileg6himizeq4", Local "Local file system at /tmp/rclone551965667", Modify Window "1ns" === RUN TestCheckSizeOnly/1 === RUN TestCheckSizeOnly/2 === RUN TestCheckSizeOnly/3 === RUN TestCheckSizeOnly/4 === RUN TestCheckSizeOnly/5 === RUN TestCheckSizeOnly/6 === RUN TestCheckSizeOnly/7 --- PASS: TestCheckSizeOnly (904.98s) --- PASS: TestCheckSizeOnly/1 (0.41s) --- PASS: TestCheckSizeOnly/2 (0.42s) --- PASS: TestCheckSizeOnly/3 (0.42s) --- PASS: TestCheckSizeOnly/4 (0.25s) --- PASS: TestCheckSizeOnly/5 (0.25s) --- PASS: TestCheckSizeOnly/6 (0.25s) --- PASS: TestCheckSizeOnly/7 (0.26s) === RUN TestCheckEqualReaders --- PASS: TestCheckEqualReaders (0.00s) === RUN TestParseSumFile run.go:181: Remote "Internet Archive item rclone-integration-test path rclone-test-rikeqad5kobileg6himizeq4", Local "Local file system at /tmp/rclone551965667", Modify Window "1ns" 2022/06/17 10:00:40 NOTICE: test.sum: improperly formatted checksum line 4 2022/06/17 10:00:40 NOTICE: test.sum: improperly formatted checksum line 5 2022/06/17 10:00:40 NOTICE: test.sum: improperly formatted checksum line 6 2022/06/17 10:00:40 NOTICE: test.sum: 2 warning(s) suppressed... 2022/06/17 10:03:33 NOTICE: test.sum: improperly formatted checksum line 4 2022/06/17 10:03:33 NOTICE: test.sum: improperly formatted checksum line 5 2022/06/17 10:03:33 NOTICE: test.sum: improperly formatted checksum line 6 2022/06/17 10:03:33 NOTICE: test.sum: 2 warning(s) suppressed... panic: test timed out after 1h0m0s goroutine 2001 [running]: testing.(*M).startAlarm.func1() /usr/local/go/src/testing/testing.go:1788 +0x8e created by time.goFunc /usr/local/go/src/time/sleep.go:180 +0x31 goroutine 1 [chan receive, 6 minutes]: testing.(*T).Run(0xc00044e9c0, {0x19b5240, 0x46f693}, 0x1a55d68) /usr/local/go/src/testing/testing.go:1307 +0x375 testing.runTests.func1(0xc000a6ccf0) /usr/local/go/src/testing/testing.go:1598 +0x6e testing.tRunner(0xc00044e9c0, 0xc000a8fca0) /usr/local/go/src/testing/testing.go:1259 +0x102 testing.runTests(0xc00030eb80, {0x2abb7c0, 0x54, 0x54}, {0xf2a180, 0xc00035bde8, 0x2ac9300}) /usr/local/go/src/testing/testing.go:1596 +0x43f testing.(*M).Run(0xc00030eb80) /usr/local/go/src/testing/testing.go:1504 +0x51d github.com/rclone/rclone/fstest.TestMain(0xc0000001a0) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:70 +0x9e github.com/rclone/rclone/fs/operations_test.TestMain(...) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/operations_test.go:61 main.main() _testmain.go:213 +0x166 goroutine 20 [select]: go.opencensus.io/stats/view.(*worker).start(0xc000168800) /home/rclone/go/pkg/mod/go.opencensus.io@v0.23.0/stats/view/worker.go:276 +0xb9 created by go.opencensus.io/stats/view.init.0 /home/rclone/go/pkg/mod/go.opencensus.io@v0.23.0/stats/view/worker.go:34 +0x92 goroutine 1880 [select, 1 minutes]: github.com/rclone/rclone/backend/internetarchive.(*Fs).waitDelete(0xc0000cdb80, {0x1da65d0, 0xc000040058}, {0xc000a326e0, 0x17}, {0xc000a326f8, 0x2d}) /home/rclone/go/src/github.com/rclone/rclone/backend/internetarchive/internetarchive.go:984 +0x1f4 github.com/rclone/rclone/backend/internetarchive.(*Object).Remove(0xc0008c27e0, {0x1da65d0, 0xc000040058}) /home/rclone/go/src/github.com/rclone/rclone/backend/internetarchive/internetarchive.go:752 +0x2f9 github.com/rclone/rclone/fstest.newRunIndividual.func1.1.1() /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:155 +0x2a github.com/rclone/rclone/fstest.retry(0xc000579040, {0xc0007b9650, 0x18}, 0xc0006196d8) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:122 +0xe8 github.com/rclone/rclone/fstest.newRunIndividual.func1.1({0xc0008e3a40, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:155 +0x1ad github.com/rclone/rclone/fs/walk.listR.func1({0xc0008e3a40, 0x1, 0x1}) /home/rclone/go/src/github.com/rclone/rclone/fs/walk/walk.go:331 +0x1e7 github.com/rclone/rclone/backend/internetarchive.(*Fs).ListR(0xc0000cdb80, {0x1da65d0, 0xc000040058}, {0x0, 0x1913520}, 0xc000b641e0) /home/rclone/go/src/github.com/rclone/rclone/backend/internetarchive/internetarchive.go:569 +0x1f8 github.com/rclone/rclone/fs/walk.listR({0x1da65d0, 0xc000040058}, {0x1dd75a8, 0xc0000cdb80}, {0x0, 0x0}, 0x1, 0x3, 0xc000b605d0, 0xc000224f00, ...) /home/rclone/go/src/github.com/rclone/rclone/fs/walk/walk.go:298 +0x277 github.com/rclone/rclone/fs/walk.ListR({0x1da65d0, 0xc000040058}, {0x1dd75a8, 0xc0000cdb80}, {0x0, 0x0}, 0x1, 0xffffffffffffffff, 0x3, 0xc000b605d0) /home/rclone/go/src/github.com/rclone/rclone/fs/walk/walk.go:162 +0x1cb github.com/rclone/rclone/fstest.newRunIndividual.func1() /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:151 +0x12a github.com/rclone/rclone/fstest.(*Run).Finalise(0xc0002e0540) /home/rclone/go/src/github.com/rclone/rclone/fstest/run.go:376 +0x22 github.com/rclone/rclone/fs/operations_test.TestParseSumFile(0xc00099f7af) /home/rclone/go/src/github.com/rclone/rclone/fs/operations/check_test.go:325 +0x5b0 testing.tRunner(0xc000579040, 0x1a55d68) /usr/local/go/src/testing/testing.go:1259 +0x102 created by testing.(*T).Run /usr/local/go/src/testing/testing.go:1306 +0x35a goroutine 23 [IO wait]: internal/poll.runtime_pollWait(0x7ff70a4e8f30, 0x72) /usr/local/go/src/runtime/netpoll.go:234 +0x89 internal/poll.(*pollDesc).wait(0xc0000f6f00, 0xc000580000, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc0000f6f00, {0xc000580000, 0xf108, 0xf108}) /usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a net.(*netFD).Read(0xc0000f6f00, {0xc000580000, 0xc0002f4b40, 0xc00058d96e}) /usr/local/go/src/net/fd_posix.go:56 +0x29 net.(*conn).Read(0xc00012e008, {0xc000580000, 0xc00058d96e, 0x19}) /usr/local/go/src/net/net.go:183 +0x45 github.com/rclone/rclone/fs/fshttp.(*timeoutConn).Read(0xc0003f8060, {0xc000580000, 0xd, 0x10000c000325520}) /home/rclone/go/src/github.com/rclone/rclone/fs/fshttp/dialer.go:104 +0x32 crypto/tls.(*atLeastReader).Read(0xc00000e180, {0xc000580000, 0x0, 0x40bb8d}) /usr/local/go/src/crypto/tls/conn.go:777 +0x3d bytes.(*Buffer).ReadFrom(0xc0003ac5f8, {0x1d6d860, 0xc00000e180}) /usr/local/go/src/bytes/buffer.go:204 +0x98 crypto/tls.(*Conn).readFromUntil(0xc0003ac380, {0x7ff7094aae08, 0xc0003f8060}, 0x17a7) /usr/local/go/src/crypto/tls/conn.go:799 +0xe5 crypto/tls.(*Conn).readRecordOrCCS(0xc0003ac380, 0x0) /usr/local/go/src/crypto/tls/conn.go:606 +0x112 crypto/tls.(*Conn).readRecord(...) /usr/local/go/src/crypto/tls/conn.go:574 crypto/tls.(*Conn).Read(0xc0003ac380, {0xc000309000, 0x1000, 0x3}) /usr/local/go/src/crypto/tls/conn.go:1277 +0x16f bufio.(*Reader).Read(0xc00008bec0, {0xc0003eb998, 0x9, 0xc00060a4b0}) /usr/local/go/src/bufio/bufio.go:227 +0x1b4 io.ReadAtLeast({0x1d6d660, 0xc00008bec0}, {0xc0003eb998, 0x9, 0x9}, 0x9) /usr/local/go/src/io/io.go:328 +0x9a io.ReadFull(...) /usr/local/go/src/io/io.go:347 net/http.http2readFrameHeader({0xc0003eb998, 0x9, 0xc000a04090}, {0x1d6d660, 0xc00008bec0}) /usr/local/go/src/net/http/h2_bundle.go:1555 +0x6e net/http.(*http2Framer).ReadFrame(0xc0003eb960) /usr/local/go/src/net/http/h2_bundle.go:1813 +0x95 net/http.(*http2clientConnReadLoop).run(0xc0006c9f98) /usr/local/go/src/net/http/h2_bundle.go:8583 +0x130 net/http.(*http2ClientConn).readLoop(0xc00060a900) /usr/local/go/src/net/http/h2_bundle.go:8506 +0x6f created by net/http.(*http2Transport).newClientConn /usr/local/go/src/net/http/h2_bundle.go:7317 +0xb85 goroutine 1695 [chan receive (nil chan), 16 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1733 [chan receive (nil chan), 15 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 904 [chan receive (nil chan), 36 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 709 [chan receive (nil chan), 44 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 456 [chan receive (nil chan), 50 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 418 [chan receive (nil chan), 50 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1590 [chan receive (nil chan), 18 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 480 [chan receive (nil chan), 49 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 555 [chan receive (nil chan), 46 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1006 [chan receive (nil chan), 34 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 605 [chan receive (nil chan), 46 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1748 [chan receive (nil chan), 15 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1214 [chan receive (nil chan), 30 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 669 [chan receive (nil chan), 44 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 836 [chan receive (nil chan), 38 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1081 [chan receive (nil chan), 32 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1149 [chan receive (nil chan), 32 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1276 [chan receive (nil chan), 30 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 954 [chan receive (nil chan), 36 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1507 [chan receive (nil chan), 20 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1536 [chan receive (nil chan), 20 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 1635 [chan receive (nil chan), 16 minutes]: github.com/rclone/rclone/fs/march.(*March).Run.func2() /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:208 +0x45 created by github.com/rclone/rclone/fs/march.(*March).Run /home/rclone/go/src/github.com/rclone/rclone/fs/march/march.go:206 +0x3a5 goroutine 2011 [sleep]: time.Sleep(0x2540be400) /usr/local/go/src/runtime/time.go:193 +0x12e github.com/rclone/rclone/backend/internetarchive.(*Fs).waitDelete.func1() /home/rclone/go/src/github.com/rclone/rclone/backend/internetarchive/internetarchive.go:980 +0x85 created by github.com/rclone/rclone/backend/internetarchive.(*Fs).waitDelete /home/rclone/go/src/github.com/rclone/rclone/backend/internetarchive/internetarchive.go:958 +0x191 "./operations.test -test.v -test.timeout 1h0m0s -remote TestIA:rclone-integration-test -verbose" - Finished ERROR in 1h0m1.100302785s (try 2/5): exit status 2: Failed []