Compiled game too big - zip error

[mikey]

Hi everyone,

I have a problem where I want to make distributions for my game, but because it's too big, or rather because the zip can't handle larger files, I get the attached error.

My game is around 2,8 GB.

I'm probably looking for a way to split the .rpa file.

My "pic" folder (with subfolders) is the main culprit, as that has 2,75 GB, split up in random chunks randomly in around 60 subfolders.

i.e.
folder 001 = 70 MB
folder 002 = 3 MB
folder 003 = 250 MB
etc...

This means that if needed I could pick and choose groups that would evenly split those 2,75 GB into however many chunks are necessary.

I don't know what to do - I would have expected that the rpa file would just be generated like gamename.part1.rpa and gamename.part2.rpa if avoiding large files has some compatibility reasons... but maybe the issue is more complicated.

So... that's my problem, and this is my error:
Thank you!

PS: I understand that decreasing the size of the game (use a more refined image compression in my case) might fix the problem, but I would still need to know how to pack a large game in case I can't decrease the size enough.

Code: Select all

I'm sorry, but an uncaught exception occurred.

While running game code:
  File "launcher/game/distribute.rpy", line 998, in script
    python hide:
  File "launcher/game/distribute.rpy", line 1005, in <module>
    open_directory=True,
  File "launcher/game/distribute.rpy", line 427, in __init__
    dlc=p["dlc"])
  File "launcher/game/distribute.rpy", line 837, in make_package
    pkg.add_file(f.name, f.path, f.executable)
  File "game/package_formats.rpy", line 133, in add_file
  File "game/package_formats.rpy", line 52, in write_with_info
LargeZipFile: Zipfile size would require ZIP64 extensions

-- Full Traceback ------------------------------------------------------------

Full traceback:
  File "launcher/game/distribute.rpy", line 998, in script
    python hide:
  File "C:\WorkInProgress_GAME\renpy\ast.py", line 785, in execute
    renpy.python.py_exec_bytecode(self.code.bytecode, self.hide, store=self.store)
  File "C:\WorkInProgress_GAME\renpy\python.py", line 1382, in py_exec_bytecode
    exec bytecode in globals, locals
  File "launcher/game/distribute.rpy", line 1005, in <module>
    open_directory=True,
  File "launcher/game/distribute.rpy", line 427, in __init__
    dlc=p["dlc"])
  File "launcher/game/distribute.rpy", line 837, in make_package
    pkg.add_file(f.name, f.path, f.executable)
  File "game/package_formats.rpy", line 133, in add_file
  File "game/package_formats.rpy", line 52, in write_with_info
  File "/home/tom/ab/x64lucid-deps/install/lib/python2.7/zipfile.py", line 1024, in _writecheck
LargeZipFile: Zipfile size would require ZIP64 extensions

Windows-post2008Server-6.2.9200
Ren'Py 6.18.3.761
Ren'Py Launcher 6.18.3.761

[PyTom]

You can manually create multiple rpas, using:

Code: Select all

init python:
    build.archive("archive01", "all")
    build.classify("game/pics/001/*.png", "archive01")
    build.classify("game/pics/002/*.png", "archive01")
    build.classify("game/pics/003/*.png", "archive01")
    build.archive("archive02", "all")
    build.classify("game/pics/003/*.png", "archive02")
    build.classify("game/pics/004/*.png", "archive02")
    build.classify("game/pics/005/*.png", "archive02")

etc.

This still leads to the problem of hosting a 3GB file.

[mikey]

Thank you!

I have added this to my script.rpy:

Code: Select all

    build.archive("archive01", "all")
    build.classify("game/pic/010/*.png", "archive01")
... (more)
    build.classify("game/pic/079/*.png", "archive01")


    build.archive("archive02", "all")
    build.classify("game/pic/081/*.png", "archive02")
... (more)
    build.classify("game/pic/113/*.png", "archive02")

which would split the images into two roughly similar packs of ~1.5 GB.

I still get this error:

I'm sorry, but an uncaught exception occurred.

While running game code:
File "launcher/game/distribute.rpy", line 998, in script
python hide:
File "launcher/game/distribute.rpy", line 1005, in <module>
open_directory=True,
File "launcher/game/distribute.rpy", line 427, in __init__
dlc=p["dlc"])
File "launcher/game/distribute.rpy", line 837, in make_package
pkg.add_file(f.name, f.path, f.executable)
File "game/package_formats.rpy", line 133, in add_file
File "game/package_formats.rpy", line 52, in write_with_info
LargeZipFile: Zipfile size would require ZIP64 extensions

-- Full Traceback ------------------------------------------------------------

Full traceback:
File "launcher/game/distribute.rpy", line 998, in script
python hide:
File "C:\WorkInProgress_GAME\renpy\ast.py", line 785, in execute
renpy.python.py_exec_bytecode(self.code.bytecode, self.hide, store=self.store)
File "C:\WorkInProgress_GAME\renpy\python.py", line 1382, in py_exec_bytecode
exec bytecode in globals, locals
File "launcher/game/distribute.rpy", line 1005, in <module>
open_directory=True,
File "launcher/game/distribute.rpy", line 427, in __init__
dlc=p["dlc"])
File "launcher/game/distribute.rpy", line 837, in make_package
pkg.add_file(f.name, f.path, f.executable)
File "game/package_formats.rpy", line 133, in add_file
File "game/package_formats.rpy", line 52, in write_with_info
File "/home/tom/ab/x64lucid-deps/install/lib/python2.7/zipfile.py", line 1024, in _writecheck
LargeZipFile: Zipfile size would require ZIP64 extensions

Windows-post2008Server-6.2.9200
Ren'Py 6.18.3.761
Ren'Py Launcher 6.18.3.761

Maybe it's because renpy is trying to still build the distribution which (despite the files being split inside) is itself a giant zip file?

[PyTom]

Yeah. It looks like standard zip files are limited to a total of 2GB.

If you edit launcher/game/package_formats.rpy, and change the line that reads:

Code: Select all

            self.zipfile = ZipFile(filename, "w", zipfile.ZIP_DEFLATED)

to

Code: Select all

            self.zipfile = ZipFile(filename, "w", zipfile.ZIP_DEFLATED, True)

ZIP64 will be enabled. That'll write out the file, but reading Wikipedia, you'll probably run into problems actually using that file. For example, Windows XP doesn't know how to decompress ZIP64.

[mikey]

Thanks! Will try it as soon as I can.

But if you say that single zip file will have troubles in older systems - can't I then unpack that single zip file and repack the entire game using 7zip in some compatible setting (i.e. split into 2-part archives)?

This would create two files, each well under 2 gb.

[PyTom]

Hm... That's not supported. It will probably work on Windows, but likely not on Mac or Linux if you do the process on Windows. Multi-part zip files are likely out, and I don't think they're supported well nowadays.

Really, we're heading into uncharted waters here. So the best way to see what will work or not is thorough testing.

[mikey]

So if I use the ZIP64 way (without re-packing under Windows), I will make that 3 GB zip file straight out of renpy, but my game's requirements will be as follows:

- Win7 or higher
- modern/recent Linux/Mac versions (unaffected by ZIP or ZIP64 choice?)

If that's the only drawback, then I suppose I can live with that.

[PyTom]

I really don't know for sure. It seems possible, but there could be other issues we don't know about.

[mikey]

Hmmm, more specifically - do you mean that

a.) if I use the ZIP64, you can't say if Linux / Mac would work, but Windows 7 and higher would be fine

or

b.) you are not sure if using ZIP64 will work with any system?

(because in case of a.) I could release for Windows for now, and later add the Mac and Linux distributions - in case of b.), I don't really know what to do)

[PyTom]

b.

It probably will work, but it's not something I've tested. I suspect it will work fine, but I can't know for sure, short of digging out a representative set of systems.

[mikey]

Grasping at straws now, but...

What if I chose in the Build menu to make only a Windows version AND renpy simply didn't create that big zip at all, and just copied the final non-zipped files and rpas to the output directory?

That way I could at least be reasonably sure the Windows version is fine, and could zip it myself using 7zip or similar.

[PyTom]

If you did that on windows, you'd have trouble with the mac and linux versions. Also, there isn't support for building to a directory for PC builds. You'd have to unzip the zip yourself.

[mikey]

If you did that on windows, you'd have trouble with the mac and linux versions.

I may try to go with denzil's approach of building the Linux/Mac versions on Linux and then repacking on Linux.

Also, there isn't support for building to a directory for PC builds. You'd have to unzip the zip yourself.

That should be no problem.

Alright, I think that's it! Thanks for the support, I hope it all works out well in the end.