Ashley, could you please confirm this behavior?

Hi Ashley,

I need to discuss about something.

The game I have been making feature graphics in the retro era. The images are small and the the game's native resolution is only 480 x 320 and majority of the images are even less than 64 x 64. I have been watching those memory numbers down below from the beginning. Right now, it is only:

Approx download: 92.2 mb memory use: 72.9 mb

Recently I have seen this error where the loading percentage sometimes turn red in the beginning. I looked around the forums and I see that it is related to having way too many images in the game. Each Sprite object in my game got something like this sprite sheet: http://www.spriters-resource.com/other_ ... heet/8208/ (but fewer images)

While err-insufficient-resources-when-previewing-in-chrome_t118122 claimed that it loaded 10000 frames and encountered the same issue, however I've discovered something else, in my game I've also tried previewing a layout which only have 1 Sprite that has only 1 frame and nothing else, the error also occurred multiple times.

Previewing the game in Chrome and launching up the debugger, I see this:

I checked https://www.scirra.com/manual/183/memory-usage:

[quote:11274wk0]Construct 2 only loads the images for the current layout. This avoids loading the entire project in memory which would be slow and consume a great deal of memory. When starting a layout, all images for the objects placed in the Layout View are pre-loaded. This includes all frames in all animations of any Sprite objects. (In other words, Sprites are either fully loaded in to memory, or not at all - they are never part-loaded.) When the layout ends, all images that are loaded but not used on the next layout are released from memory.

I don't understand. I thought Construct 2 only loads the images for the current layout. But here, all these output from the console show that the engine failed to load images that are not in the layout I am trying to preview, which has only 1 Sprite with 1 frame of size 128 x 128. The event sheet in this layout also does not create any other Sprite or stuff from other layouts. It only wait for 5 seconds before proceeding to another layout. So... why and how do these images come from? Is it because of the preview mode so all these images are here?

I have been testing around and here is what I got so far:

On Chrome, Usually, this loading percentage turn red at around 1-2%. If it did not turn red around at that time, mostly it will be able to load up to 100% and the game will play without any problem. Sometimes it occurs. Sometimes it does not. But apparently, as more and more images are added, it becomes a lot more frequent.

For Preview Node Webkit, when it starts up, it goes 1-2%, it could turn red at this time. If it does not, it load its way up and the game can be played.

I have also tried exported Node Webkits. After hitting the .exe, the loading percentage jumps up to 27% initially and made it ways to 100% and the game runs just fine. I've tried this multiple time, and it never fail to load even once yet. But this does not mean the problem could not occur on exported Node Webkits in the future. In fact, obviously Node Webkits are basically Chromium, just like Chrome. So I expect it to pretty much behave just like Chrome, but right now it does not - it can play the game without any loading red percentage .....for now, it seems.

-----

So here's the thing, Ashley. Could you please confirm the following:

1. For exported Node Webkit (which problem never occurred yet), can I take it for granted as a workaround that I should be able to avoid this dreadful error with it, yes or no? Please give us an insight. While we have quite a number of animation frames, each of our layout never contain more than 500 animated images. (In fact, most of the time, it is only 100-200 images) Also, recall Approx download: 92.2 mb memory use: 72.9 mb

2. Programming-wise, I believe C2 clear up unused textures. I've tested it and it worked. But about the error I am facing here. It's as if during the preview, C2 has to allocate all of the resources for all images in one go for some reason. Could you please confirm this? Or is this some sort of loading implementation in favor of previewing sake? Or is there really some invisible global limit to the total number of images regardless of their sizes for the whole project? (again, recall Approx download: 92.2 mb memory use: 72.9 mb )

If the answers for questions are "yes, I can take exported Node Webkit for granted as a workaround, as each layout pretty much contains only 100-200 images and C2 can allocate and deallocate all the resources without any trouble. It is likely that preview contains some bugs", then I would be happy to use exported Node Webkit to "preview" my game instead.

However, if there is really some invisible global limit, then... this could spell doom to my project that I've been working for 1.5 years...

So Ashley, please confirm.

3. Also, from the link above,

[quote:3fcy9cwj]Windows has a hard-coded limit of about 10,000 images per app

Is it not possible to deallocate and reallocate resources internally programming-wise? I used to write an OpenGL app that can load any image from the given path during runtime. What's the trouble here? From the Preview image above, it's as if C2 need to load all the images.

4. Does C2 apps need to load all images (without loading them as texture into the GPU) initially even they are not required?

5. Also from the link above:

[quote:3jmd7v2j] People occasionally report this but I just cannot believe how you could possibly genuinely need so many thousands of images in one project. You must be doing something wrong or appallingly inefficient! Can you explain a bit more about why you need this and why there really is no other alternative than to have thousands of images?

See this artist here: http://probertson.tumblr.com/post/82062 ... animations

See those fluid animation? In one animation (idle, running, shooting, etc), this is something similar to what I am going through right now. Lots and lots of small images for the love of pixel animation. This is only a small fraction of animation used in the game Mercenary Kings. There are a lot more in the game and all those thousands of frames in one project.

But again, my concern is not that I need all images presented at once. Yes, it is ludicrous to have 10000 images at once. In my game, there are only 100-500 unique small images on each layout at once, I am sure.

My concern is whether this is just a thing in preview and could I get away with the exported node webkit in the long run or not?

Well, looking at the rest of Ashley's reply, it sounds like this could be seen as a limitation of chrome, or of C2, depending on how you view it.

On chrome: I would leave a comment on the following thread requesting that chrome be allowed to allocate >6400 images at a time, as long as it is explicitly requested to do so:

https://codereview.chromium.org/18541

An even better approach might be to file this in chromium's issue tracker:

https://code.google.com/p/chromium/issues/list

As long as you can give a good case for where it would be useful -- and it sounds like you can -- you've got a decent chance they'll change it.

The comment about 'Windows has a hard-coded limit of about 10,000 images per app' sounded like gobbledygook to me, but apparently it's true. Seems like this could be worked around by having C2 spritesheet objects with 100> frames total between all animations, but that brings us around to the infamously inflexible C2 IDE...I see another C3 feature here.

10,000 is a lot of frames, but if you break it down it really doesn't seem that excessive...if your sprites average 64x64, with no downscaling -- not uncommon for a pixel art game -- 6400 frames would break down to 16 1024x1024 spritesheets, totaling 64mb of memory...you should be able to get away with that even on most mobile devices.

As far as I'm aware, the issue is more with the editor limiting how many icons can be viewed due to windows. Every object has 3 icons (small, med, large) for use in the IDE. So multiple frames within the same object should not be a problem at all. But around 3,000 unique objects and you may start running into problems. So it isn't frames, but images that the editor must display. Disabling the icon cache under Misc. should fix it, though.

As far as I'm aware, the issue is more with the editor limiting how many icons can be viewed due to windows. Every object has 3 icons (small, med, large) for use in the IDE. So multiple frames within the same object should not be a problem at all. But around 3,000 unique objects and you may start running into problems. So it isn't frames, but images that the editor must display. Disabling the icon cache under Misc. should fix it, though.

Whoops, wasn't thinking straight there...yes, I think you are right. In that cases the limit is most likely going to be chrome rather than C2.

There is a memory/registry hack you can do to Windows if it's the icon thing (as well as disabling icon cache), I forget the topic it was mentioned in though but it's related to windows handles.

The limit of 10,000 images is a limit in Windows - a limit in the native technology - which all apps face, including Windows-based browsers, and the C2 editor itself. I've researched it a bit and only found a blog post from a Microsoft engineer claiming it "should be enough for anyone". Go figure.

I think a better question is how can you need so many thousands of images? That's crazy!

Hi Ashley,

I've tested further. I've tried exporting the whole project as a HTML5 website, and see that with all those sprite sheets combined, I only got 819 "sprite sheet" images so far. This is possibly the same reason why exported Node Webkit can run it fine, because it only have 819 "sprite sheet" images. I have seen that the total number of individual images are 5638 images.

Looking around further, I see https://www.scirra.com/blog/87/under-th ... onstruct-2:

[quote:1ovp95x1] Construct 2 has typically exported object's animation frames as separate PNG files. Spritesheets - a new feature in the latest beta r92 - means Construct 2 now exports bigger images with several animation frames arranged on each.

This implies that for previewing project, all these 5638 images are not packed as sprite sheets. (as seen in the console) Thereby, causing the app to hit the limit and crash in this case.

It might be the implementation choice to leave each individual frame as it is and then combine them later into sprite sheets when exported. But this design choice now is affecting a few people who wish to preview their project with several thousands of small individual images.

Packing multiple sprites into one sheet has been "the way" for all 2D games since the Nintendo NES era. So Perhaps we could change this by having C2 create sprite sheets for these images during preview as well, but save these sprite sheets as a cache so that the next time of previewing, C2 does not have to create sprite sheets unless there are some changes. But I do understand that this suggestion when turn into actual implementation could be a technical nightmare given that the "packing images into sprite sheets" are introduced in r92 for exported projects only. However, if you can pull it off or have a better solution to address this, C2 will be a force to reckon with for us, enthusiastic pixel artists. Not to mention that previewing will be even faster with less number of actual images to load.

Also, Ashley,

1. Mind give me a link to the Microsoft blog?

2. The 10000 images limit means "total number of loaded images at any given time during runtime", and not all images in the project, yes?

3. I haven't reached 10000 images limit yet but I got the error during preview on Chrome. From the link above, this is because of the 25 MB bound limit in Chrome. The error occurs when there are too many requests at once. So is it possible to try do less number of requests at a time, and then proceed to the remaining requests only when the previous requests have been processed? (EDIT: but again, if C2 can manage all the graphical assets and pack them into smaller number of "sprite sheet" images, this issue will likely to not occur)

FYI, doesn't sound like the '10,000 limit' affects anything at all. C2 doesn't generate icons for every frame of every animation, just for the first frame of the default animation. You are only going to hit a limit if you have 1000's of different object types...which truly would be crazy.

—

It sounds like you've basically answered your own question: since C2 spritesheets on export, you are never realistically going to hit the chrome 'limit' of ~6400 images. But, you are hitting it on preview since each frame is being treated as a separate image.

Ashley

The solution to this would simply be a project properties option (default=FALSE) that would force the editor to spirtesheet objects prior to preview. The icon limit is irrelevant: nobody would hit it without thousands of objects. Problem solved.

One more thing, Ashley, I would like to suggest one more thing:

You could pack more frames into a single bigger image. Right now, each animation of each Sprite got its own sprite sheet on export I do have a huge number of 128 x 128 sprite image sheets to accommodate lots of those frames less than 64 x 64.

Could it be done to pack several of Sprites into a single bigger sprite sheet? For example, we could have enemy A,B,C sharing the same sprite sheet image. This way, you won't be referring the sprite sheet by sprite name any more, but instead have a table storing which animation is in which sprite sheet and the locations of each respective animation frame on this sprite sheet.

This way, instead of lots of 128 x 128 and even smaller 64 x 64 sprite sheet images, we could, for instance, have fewer 512 x 512 or 1024 x 1024  instead.

And let's have C2 manage and take care of this when previewing just like I wrote in the previous post.

Sure, given the current design and implementation of C2, this could mean changing things almost inside out. I don't know how flexible, scalable or changeable is your code, so I can't have a say on this. Yet If you can do it, then that is great, but if you can't...

[quote:3lbtdlio] Further, we are confident Construct 3 will be able to import Construct 2 projects, allowing you to easily transition over.

Please design and implement C3 to address the issue above, especially for preview.

No. We do NOT have thousands of frames for one sprite. That is insane. But we do have total number of individual frames for every sprite combined up to thousands. You have seen how games like Mercenary Kings do with their pixel arts from http://probertson.tumblr.com/post/82062 ... animations 

Pixel art is a serious business, and all those frames could be packed tightly into sprite sheets. You have seen that all my 5638 frames could become 819 images. (And could even be reduced further with the above suggestion) This implementation won't only benefit pixel artists but also everyone, as it will reduce total number of actual images required to load.

***** One more thing: you haven't answer the most important question yet:

Does exported Node Webkit need to do something (ex. Some sort of Javascript request or the like that could hit some limit) with ALL images of the project on the initial startup just like preview?

From http://blogs.msdn.com/b/oldnewthing/archive/2007/07/18/3926581.aspx:

[quote:2bzq6b3g]The first answer is "If you have to ask, you're probably doing something wrong." Programs shouldn't be creating anywhere near ten thousands window manager objects in the first place.

Along the same lines: I don't understand how you could possibly need so many thousands of individual images? What are you doing? Surely there is a better way?

The editor could spritesheet images for preview, but that would be a big architectural change and could slow down preview. And I don't see the point, since needing thousands and thousands of images does sound like you're doing something wrong.

[quote:pp78fjeq] I don't understand how you could possibly need so many thousands of individual images? What are you doing?

Ashley, like we've said. We are doing pixel arts like this: http://probertson.tumblr.com/post/82062 ... animations

If imported to C2, each Sprite object could contain up to a hundred of frames or so. (But with exported spritesheet image, those hundred frames should become just a huge 512 x 512 or two instead. And that is completely fine.)

[quote:pp78fjeq] The editor could spritesheet images for preview, but that would be a big architectural change and could slow down preview.

We have tried some experiment on loading time.

On our project, we tried previewing and see the time it takes from hitting "preview" button to when the game finish loading and play. (If the loading percent becomes red, we just try again until it gives in)

• Preview Chrome: 1:13 minute
• Preview Node Webkit: 1:05 minute

But here's the funny bit, we have tried exporting Node Webkit, (disable minify and image compression) and it took 42 seconds. In addition, when we run the exported Node Webkit, it took only 10 seconds to finish loading.

In other words, it seems we can test our game faster than preview by exporting node webkit and run it right now. And yes, this would also avert the red percent loading error issue entirely as well.

Could the big architectural change slow down preview? With the above result, I don't think so. In fact, in this case, I could just well go with using the faster exported node webkit instead. <img src="{SMILIES_PATH}/icon_e_biggrin.gif" alt=":D" title="Very Happy">

We have also tried Preview Firefox, and it took 27 seconds...

I see that preview Chrome and preview Node Webkit took the longest on the first 0-20% (loading images perhaps?), but for exported Node Webkit and Firefox, the percent just "zaps" through to 100% in seconds. In fact, Firefox seems to kick the number up the fastest (only 8 seconds to 100% after the percent shows up, in fact). Does Chrome has inferior loading speed to Firefox or something?

—

I think Ashley basically just said he isn't going to do this. You can still export and test, which is reasonably fast if you disable minification and set png-recompression to 'none'. On export, you will never hit the image limit since images are spritesheeted.

Do you have an SSD? It's going to be a lot more difficult to work on such a project without one, especially if your are using caproj vs capx.

TiAm Yes I understand. Ashley will have to go through a lot and I completely understand.

I already got a better solution in front of me, so I am not requesting for any change in C2 any more.

A screaming soul exploded inside me when the project manager requested for a change like this. A simple change from client's perspective could involve lots of gigantic technical changes and outsiders never see the internal technical pain. I've been there done that so I do understand.