EXC_BAD_ACCESS in IO.read

[spamtrap1@…]

I had a small spelling accident tonight, which caused me to stumble over this :
using ios.read, with an argument of nil, i.e.

s = ""
f = File.open("xxx")
while (!f.eof?)
  s += f.read(nil)
end

will cause this :

Exception Type:  EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000000
Crashed Thread:  0  Dispatch queue: com.apple.main-thread

Application Specific Information:
objc[26488]: garbage collection is ON

0   libSystem.B.dylib             	0x00007fffffe007c5 __memcpy + 37
1   libauto.dylib                 	0x00007fff83a1f210 auto_zone_write_barrier_memmove + 96
2   libauto.dylib                 	0x00007fff83a2020e auto_realloc(_malloc_zone_t*, void*, unsigned long) + 878
3   libSystem.B.dylib             	0x00007fff84c1ab3b malloc_zone_realloc + 92
4   com.apple.CoreFoundation      	0x00007fff8215eb49 __CFDataGrow + 233
5   com.apple.CoreFoundation      	0x00007fff82184e05 CFDataSetLength + 181
6   libmacruby.dylib              	0x000000010005c1fa rb_io_read_internal + 522
7   libmacruby.dylib              	0x000000010005d6b6 rb_io_eof + 54
8   libmacruby.dylib              	0x000000010016d32f rb_vm_dispatch + 7279

Macruby version is Nightly build from 2010/01/26.

The weird thing is i can only reproduce it with large files. My test files are CD sized, created with "dd if=/dev/random of=test.dat bs=65536 count=10000"

I know calling read(nil) twice on a file doesn't make much sense, but reading the docs leads me to believe it should just return "" ( http://ruby-doc.org/core/classes/IO.html)

[spamtrap1@…]

I've been looking a little bit at this, but since i'm unable to build trunk, all i can do is pen & paper, and guess :)

in io.c:1108 (io_read), a distinction is made between reading x bytes, or reading the whole file. If we're reading the whole file

    if (NIL_P(len)) {
        return rb_io_read_all(io_struct, outbuf);
    }

Or we're reading part of the file io.c:1145 (io_read)

    const long data_read = rb_io_read_internal(io_struct, buf, size);
    if (data_read == 0) {
	return Qnil;
    }
    CFDataSetLength(data, data_read);

So far, so good. The very last thing rb_io_read_internal io.c:983 (rb_io_read_internal) does, is to call

    rb_io_read_update(io_struct, n);

Which positions the file pointer at the position we just read to. This part isn't done by rb_io_read_all

Could this be why the eof? call fails after having called IO.read(nil) ?

Just my $0.02 after a good nights sleep :)

/Jimmy

[martinlagardette@…]

hi Jimmy!

Thanks for the very detailed bug report[[BR]] But why are you unable to build trunk?

Other than that, if you look closely to rb_io_read_all(), you will notice it is mostly doing the same thing as io_read():
- Create a CFMutableDataRef with rb_bytestring_wrapped_data()
- Increase the CFMutableDataRef size with CFDataIncreaseLength()
- call rb_io_read_internal()
- Once done, call CFDataSetLength()

I can't run the test, because I run out of memory as before it completes, but the issue might come from the fact that even '''rb_io_eof''' uses '''rb_io_read_internal()''', which is why, as you can see in the backtrace, CFDataSetLength() is called. So I guess there are three possible cases here: Either you're out of memory too, or either the buffer or the sized given to CFDataSetLength() as arguments are incorrect. I will try to see with Laurent if he has a better idea as to why this is crashing.

[jsn@…]

I'm now able to build trunk (after updating to latest).

I've been digging around for this issue, and issue #578.

The problem persists on the lastest trunk and is, as far as i can tell, caused by the system running out of memory. With all the buffering going on in io.c (is that even needed on a UNIX system, where the kernel normally uses every bit of unused memory for caching ?), reading a 600 mb file in chunks will require around 1200 mb before it finishes, first being cached in io.c, and then a NSString representing every chunk.

It seems this "bug" is not really a bug, but more a design issue, or at least closer related to issue #578.

You can close this one.

[martinlagardette@…]

I'll check with Laurent first anyway, we will see if the bugs are related, or two different ones :-).

Thanks a lot!

[lsansonetti@…]

I think there is a design issue definitely, we should address this in 0.6.

[babs.devs@…]

Cc Me!

[lsansonetti@…]

This problem should be fixed in 0.7.