Posted by kohsuke
on September 17, 2009 at 3:07 PM PDT
How do you seek to arbitrary positions in a file that's 2^64 bytes long?
Yesterday, a Hudson committer Alan Harder discovered an EOFException in Hudson, which only happens on 64bit JVM on Solaris. This was happening in the part of Hudson (or more precisely, in a separate library called Akuma ) where we look at the memory space of the process to figure out the command line arguments of the current process (the /proc/ID/as file — an equivalent of /proc/ID/mem in Linux)
It turns out that the problem was because this sparse file is 264 bytes long on 64bit systems. So for Hudson to look at arbitrary positions in the process memory space, it needs be able to seek beyond Long.MAX_VALUE (263-1.)
Java's RandomAccessFile has the seek(long) method, but it uses long as a signed value (as it should), so this won't do it. Alan suggested that we first seek to 263-1 and then call skipBytes repeatedly till we get to where we want. This won't do it either, unfortunately, because skipBytes takes int. I'd have to call this method for close to 232 times in the worst case! (And it turns out that internally RandomAccessFile keeps the file offset in a single long field, so it just can't address anything above 263-1 anyway.)
So to do this, I had to go back to libc, namely fseek. It still uses a signed long, but with fseek, one could say "move ahead by 263-1 from the current position", so in the worst case, it takes up to 3 seeks to get to where I want, but I was finally able to fix the problem that way.
Now, the maximum file size in ZFS is 2128 bytes — I wonder how one would seek to an arbitrary position in such a large file!