Apache/X-SendFile truncating large files 23 July 2013

I was recently trying to send some big files (8-20gb) over X-SendFile file (this was for reasons). The files were being truncated after seemingly random points. File size and the Content-Length were in agreement, but the file on disk was longer. This was hard to google for.

I first tried diagnosing it via curl, and discovered that the file length was being reported incorrectly. A 14GB file was reporting in at around 700MB (the wrong length). I googled the truncated length, to see of that was 'magical' but apparently it wasn't.

After much searching I discovered (and now can't rediscover to link) an article mentioning a bug in X-SendFile. It seems mod__xsendfile was using a long to represent the file size, so files larger than 4294967295 bytes would roll over and start counting again at 0.

The solution is to install a newer beta build of X-SendFile.

If you're using X-SendFile to shift big files, install the newer version, as per the instructions on the mod__xsendfile site. Beware though, there are config syntax changes in the new version too.