It's pretty cheap to get the file size with decompressing the whole blob. You can decompress just the first hundred of so bytes of the .git blob object, which starts with "blob {size}\x00". For example, using Python:

   >>> import zlib
   >>> f = open('.git/objects/00/331d4c2e12984f70df4a3dd3bc7296825faab0', 'rb')
   >>> zlib.decompressobj().decompress(f.read(100))
   b'blob 11708\x00...'

So that particular file is 11708 bytes. It still requires an open and read operation. But they could cache that on a per-file or per-tree basis, so it's very doable.