Performance: in fairness

0 favourites
From the Asset Store
Firebase: Analytics, Dynamic Links, Remote Config, Performance, Crashlytics on Android, iOS & Web Browser
  • OK. This is not to become a c2 bashing thread (mods please lock it if it does).

    Yesterday I posted some performance stats in another thread, but they were misleading because I had used an inefficient sequence of events in the c2 part of the test (the test was a version of bunnymark). I felt that my results were unfair to c2 because of this misuse of events. Additionally, the test did not focus on the GPU or CPU, which meant that you didn't know which one was limiting and causing a drop in fps (CPU number crunching or GPU fillrate). The other thread was locked so I'm posting my results here instead.

    Results:

    Tested on laptop (i5, 16Gb, SSD, intel HD4000) and phone (nexus 5x: built using phonegap build webview for c2). c2 laptop export was tested on NWjs (v22) and Chrome (v58). I tested each with and without sprite rendering enabled - the no-render results are probably a good indication of the relative amount of data that each engine can compute, given the limited test parameters, and the yes-render results probably indicate where my GPU became stressed. As I said - these are just a taste of performance and not an absolute benchmark. Edit: I took 55 fps as the cut-off for the tests.

    Engine..............Render Images?..................No of bunnies

    .................................................................Computer.............mobile

    C2 NWjs.....................Y...........................6000

    ...................................N...........................9000

    C2 Chrome................Y...........................7500

    ...................................N...........................8500

    C2 phonegap............Y..........................................................1300

    ...................................N.........................................................1550

    Godot Engine...........Y............................3000........................800

    ..................................N............................5500......................1400

    Unity.........................Y.............................9300......................2500, 4700, 5200 **

    ..................................N..........................30000......................9300

    Cocos Creator.........Y............................13000**$$.................3000

    ** Gracefully degrades the fps from 60 to 30 to 20 (no jank)

    $$ Exported to .exe and also html5 in Google Chrome...

    All of the above tests are available here; you'll have to enable/disable drawing to make your GPU/CPU comparisons.

    What stood out for me? Well, c2 did very well but Cocos Creator won by far. Looks like JavaScript is the language to go for...

  • Interesting, Did you investigate why some where better than others? Did you use a WebGL inspector? My conclusion on another thread was draw calls. Less draw calls/draws = better performance, maybe this can help me investigate this further.

  • tunepunk, no, I didn't do an in-depth investigation. However, if pure draw calls were the limiting factor then each engine should have max'd out at the same value when rendering bunnies. Therefore, I think that there's an overhead that each engine carries when ascribing draw calls to the GPU, such that the final yes-render figure is a mix of a GPU and a CPU limitation. The test also didn't measure jank (because I couldn't be bothered to create an fps-waterfall graph for each one) - that would also be of value because it would represent a measure of the quality of the player's experience.

  • Interesting results indeed. That gap between Unity and C2/Godot is huge. I wonder what causes that.

    I think it's worth noting that 6000/9000 bunnies with an Intel HD4000 GPU in C2 is quite acceptable imho.

  • Is the Godot engine native? If so, nice evidence that HTML5 can reach native performance!

    BTW I generally use 30 FPS as the cutoff for performance tests - 55 is a bit close to the v-sync limit, where scheduling issues could come in to play.

  • Not sure if this is possible but, Is there a way to maybe cap the framerate in C2/C3 (I'm using C3 currently)

    It would maybe be a good feature to be able to choose to cap frame rate at 30fps or 60fps rather than having the browser trying it utmost to render 60fps if 30fps wil ldo for a particular game.

  • Ashley - Godot compiles c++ from the user's GDscript (python) at export. It is run in a wrapper that's the same .exe as the editor runs from in windows (not sure about android export). It does show html5 can perform ok, but I think Unity is the only native engine in this group - which is why it's so much faster.

    Havok - Unity's graceful degradation from 60 to 30 fps was a clincher. No jank....

  • Try Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Try Now Construct 3 users don't see these ads
  • Looks like Unity's rendering performance is only about 25% better than the HTML5 version, so not that big a difference when it comes to on-screen sprite throughput.

    If GDScript is python-like, then compiling it to C++ won't make it fast - it'd need a proper JIT to compile it well. JavaScript happens to have excellent JIT support, so that would explain why it's faster. So there seems to be a good argument here that JavaScript is faster than GDscript.

    There's a request to support 30 FPS rendering in Chrome here, star it to show your support: https://bugs.chromium.org/p/chromium/issues/detail?id=485600

  • Could someone explain to me why NW.js performs a bit worse compared to Chrome?

    Aren't both browsers essentially running on the same backend (Chromium)?

  • There's no good reason for them to be different - they are based on the same browser engine. I'd suspect it's just a margin of error in the test, or a quirk of only measuring to 55 FPS where v-sync scheduling comes in to play.

  • Colludium could we perhaps change the testing criteria to 30fps?

    (Since there is no way to manually throttle down to 30fps, I would recommend just adding bunnies until 30FPS have been reached.)

    Also just out of curiosity, since Unity also offers WebGL exporting.

    Would you mind providing performance stats for that as well?

    Might be interesting to see how Unity's WebGL export performs, compared to C2's.

  • Sure TheRealDannyyy - I'll take a look at Unity webgl as well. It'll be some time in the next couple of days, though. Work is hectic... I also wondered about why Chrome beat NWjs - I put it down to the NWjs engine dev being in lag behind Chrome.

    I think 30 fps stress testing won't give any good info - most players will have gone home by then, and it might just test aspects of the engines that are irrelevant for normal gaming.

  • Sure TheRealDannyyy - I'll take a look at Unity webgl as well. It'll be some time in the next couple of days, though. Work is hectic... I also wondered about why Chrome beat NWjs - I put it down to the NWjs engine dev being in lag behind Chrome.

    I think 30 fps stress testing won't give any good info - most players will have gone home by then, and it might just test aspects of the engines that are irrelevant for normal gaming.

    Yeah the 30fps test would just be for testing purposes in this specific case, obviously nobody wants a game running on lower framerates than 60fps.

    No probs for the delay, take your time and do it whenever you can.

    Thanks for doing all this by the way, it's quite interesting.

  • Thanks for doing all this by the way, it's quite interesting.

    +1 Indeed. Thank you for taking the time for creating and showing us these measurements.

  • , TheRealDannyyy, happy to help. I did have the old apks on my phone (all but Godot's).

    30fps no render

    Godot: 1200

    C2: 1800

    Unity: 5500

    30fps with render:

    C2: 1600

    Unity: 2000

    The Unity version was smoothest because of the framerate lock.

Jump to:
Active Users
There are 2 visitors browsing this topic (0 users and 2 guests)