Side note: Skip past part 1 and 2 if you just want to read about the layout engine in C3, which is the important part
C3 has been doing great things recently to try and solve its crippling UI caveats. These might seem like harsh words, but I assure you it's not. For a while, Construct has had basically zero tools for making decently integrated UI elements (except for the anchor behavior I guess...). I've been trying to fix this for almost two years now, and I think we're getting very close to finally having everything we need.
1 - A UI problem?
Yeah, I guess I should start with this. I've you've made games with Construct in the past, you might think you've made UIs just fine with the tool. Sure, it's possible, but there are many things that are crucial to make great UI for games that C3 either makes needlessly hard, or straight up impossible.
Therefore, here is a non exhaustive list of issues that Construct 3 had.
The important stuff
Hierarchies
Most UIs rely on being composed of many sub instances that make up individual pieces of interfaces. Hierarchies have been BY FAR, the most important requirement to make UI. So far we'd been using the pin behavior which has its complications.
Pin was significantly eventually overhauled in March 2020 which fixed a lot of issues with it, but very quickly after, C3 introduced a Hierarchy feature only a few months later which was great!
Scene graph fixes a lot of things, and using it I was able to write very cool UI very easily. My best example would be my work on Idle Ants on Poki where I made use of scene graph every chance I got.
Prefabs
Same as hierarchies, Prefabs are a cornerstone of UI design as almost every UI block will be reused in the game. Prefabs makes working with many UI elements much easier.
Once again, since March 2022 templates have been in development. They are still not quite ready, but Scirra stated they are still planning to work on them some more in the near future.
Creating/Reordering/Deleting layers at runtime
This one is a bit more tricky to understand, so bear with me for a second.
When writing more complex UI, lists, grids, scrollable areas, dialogs, draggable views, and many more become an important part of your toolkit as a UI designer. In fact, the entire added value of Pro UI in my opinion as how easy it made implementing these things. How did it manage this? Layers. Lots and lots of layers.
Now, we could just go around creating hundreds of layers, and I know some people just do that 👀, but sometimes, even that isn't feasible.
What if we want to have draggable dialogs which reorder based on the last one that was clicked like many RPGs do?
What if the UI adapts to user action?
What if I don't want to create hundreds of layers on every single layout just to accomodate for a new Achievement pop up I added 2 years into development?
Being able to just mess with layers when needed is very helpful.
A layout engine
This is also, one of the biggest issues of C3 at the moment. C3 provides no way to automatically order elements in a grid or list formation. What would seem trivial to do in HTML or in other engines becomes a mind numbing task in C3 as you need to write your own basic UI layout system, or use the one provided by 3rd parties (which frankly only ever support the most basic use cases)
Since r287 Scirra opened a window towards fixing this issue, and this sparked a debate which eventually ended with me writing this very blog post. More on that later.
Animations
I'll go over this one quickly as it's pretty much fixed nowadays but C3 UI animations used to rely on using Litetween, a 3rd party addon for C2 and eventually ported to C3. Since then, C3 has had its own Tween addon, and even released an animation timeline feature.
The animation timeline, being a complicated feature to implement, has spent a LOT of time in development, but it's getting to the point where many users started implementing it in their games.
The less important stuff
Masks
Masks are another trick UI designers want to have up their sleeves as it allows for much more stylistic UI to be made.
Masks can be used anywhere from cool animation to scrollable lists. The convenience of being able to just mask away or mask in any UI element is incredibly useful in very subtle ways.
Blend modes only do the masking partially as they require the mask to be fully visible and rendered. There are ways to work with that, but many reasons you need masks are to make stuff disappear, not replace it with more.
However, this was fixed as a byproduct of the new layer groups features. It's possible to do "regular masks" by using a combination of sub layers, blend modes and force own texture, like shown here
More layer control
This was also a feature that was recently added, but being able to disable click for layers and being able to scroll layers independantly would be a great QOL feature, and both were added fairly recently.
Mesh
Mesh, in the case of UI, can be used for anything from changing the shape of clickable areas, making animations, making smart use UI elements to recycle them in different ways, integrating UI to an environment, or even making new UI elements entirely.
Anyway, while it would use some QOL features, the new mesh feature allows for all of that and more.
The nice to haves
UI navigation flow
Menu navigation is one of the hardest things to design, port and make work cross platform. This sometimes require entire rewrites of the UI per platform to accomodate for every input system's limitations.
One thing that would ease the pain a lot, and that Unity implements nicely, is a UI navigation flow system. In the editor they appear as lines linking UI elements on the layout and Unity uses this to pass the focus from one UI element to another when using a keyboard or a gamepad instead of relying on the mouse/touch position to give focus to elements.
While this would be cool, and would help a TON with making nice UI flows, it's not impossible to implement with a good UI toolkit
Built in UI Toolkit
Speaking of...
While I do think that this is what most end users will be asking for, I think this should be one of the last things ever implemented in any decent UI system. Something like ProUI that does a lot of the heavy lifting for tedious tasks would be great, but I think to make the most out of this, everything else needs to be figured out first. So let's put this off for now.
Per layer rendering settings
This is an issue only for low resolution games, but given how many pixel art games are made every day, I think this is worth looking into.
A good number of Pixel art games still want to be able to render UI in HD.
Here are a few examples from Celeste, and two C3 games: Astral Ascent and Story Arcana
Achieving this is not an impossible task, but one tradeoff you need to do is picking how to do sampling, fullscreen mode and pixel rounding that fits your needs. Often, pixel art games need nearest sampling, low quality fullscreen, and pixel rounding, but then the UI becomes all pixellated. Make some of these of higher quality and the UI looks great but then the game looks all blurry.
Most games just end up using High quality fullscreen, which makes Text render nicely, and sprites render at a higher resolution when possible, but then effects used in the game world don't follow the pixel grid anymore and stick to the device screen pixel size instead. Some of that can be worked around though.
Pixel rounding is almost always off, as the jankiness of the movement more often than not feels off for smooth UI animations, and smooth movement don't feel off at all for pixel art games.
Sampling however is harder to deal with. Pixel art absolutely requires nearest, and HD UI absolutely requires bilinear or trilinear. The only way to reconcile both is to export all the pixel art at 2 or 3 times the scale, and making the game resolution 3, times as big, and stick to nearest. It kinda works, but it would be much better if we could just override the sampling per layer.
The extra if we're feeling daring
State management and data modeling
Something that's very important in UI design is being able to easily change UI element's state. Like clicking a button, switching tabs, dragging a slider, or enabling/disabling UI elements based on player actions. Currently, the only way to do this is to manually listen for changes and then manually update the UI accordingly.
Having some state management system where the game can update the state, and the UI can just look at the state when it's getting changed would be very helpful.
This is however a very tricky system to get right and not that hard to implement, so let's move on
RTL support
RTL is required for many languages around the world, and localising the game in new countries. While this is not as important, because RTL is hard to come by even in big titles, it would still be a great addition.
Ashley already said it wouldn't be viable in C3 though
A better Editor SDK
This is the last thing I want to ask for as, while it's the most decorrelated, it might be very important to let 3rd party developers fix these issues on their own.
UI tools are overwhelmingly edittime. Nobody needs to manage prefabs in runtime. Nobody needs to create dialogs, or links, or flows or anything UI related in runtime.
Runtime UI is "interact, get an interaction, look at it, get information.", mostly.
A better editor SDK would allow addon developers to integrate tools inside C3 that would help with setting up UI elements.
Just being able to access World instances data, make changes to them, create an editor window and render what we need into it would go a VERY long way. Custom file editors like the array or dictionnary would be a godsend as it would allow developers to integrate pretty much anything inside C3 and have it save as a nicely packed file that can then be loaded by another addon at edittime or runtime.
2 - Now, about solving it
A common trend with all the issues discussed above is that a lot of these are being addressed as we speak. Be it by being fixed as a byproduct of another feature, or just being new core aspects of the engine.
Here is the full list again:
Important:
- Hierarchies: Solved ✅
- Prefabs: In the process of being solved ⏳
- Creating/Reordering/Deleting layers: Unsolved ❌
- Layout engine: In the process of being solved ⏳
Less important:
- Masks: Solved ✅
- Layer control: Solved ✅
- Mesh: Solved ✅
Nice to have:
- UI Navigation: Unsolved ❌
- UI Toolkit: Unsolved ❌
- Layer render settings: Unsolved ❌
Extra:
- State management: Unsolved ❌
- RTL: Unsolvable ⛔
- Editor SDK: Unsolved ❌
Most of the important stuff is almost there. We're very close to reaching a point where making UI in C3 is not a massive pain anymore.
Let's talk about what's left keeping the most important for last.
RTL support
This is partially supported in the engine, and only breaks with BBCode tags. One solution is to just disable BBCode for RTL, or to render RTL using the new HTML element addon (which actually just also disables BBCode I guess?)
Anyway, it's cool but not super important either at the moment.
Prefabs
Nothing new, they are being worked on at the moment.
UI Toolkit and State Management
These could arguably be left for the 3rd party addon developers to solve. It's the topmost layer of the system, and having a solid UI framework is infinitely more important than having a behavior that turns a sprite into a button in my opinion.
This is the one thing I think Scirra can just outright ignore in the list.
Editor SDK
More work on the editor SDK would be super important, and would allow addon developers to potentially write addons that would implement tools for UI navigation among other things. I know most of the things already exist in the editor code, and just need to be exposed to the SDK, most of the features I've asked about are used in other parts of the engine.
I get that making them available through the SDK is still a big amount of work, but it's less than having to implement new editor features from scratch, so there's that.
Layer render overrides
I have looked into the renderer of C3, and since the canvas has that option I'm blindly guessing that it's not out of the question, but I sincerely still have no idea how the whole of the renderer works, so I can't tell if this would be feasible.
Creating layers at runtime 2: Revengeance
I've recently been diving deep into the code of C3 again, and I've written a patch that adds a layer creation feature.
(Source code)
I've tested it in a few small cases, and it's worked very nicely in my case. I don't know how many things that change could break, but so far, I've broken none. It doesn't support global layers, but from what I can tell, this would be easy to add to my patch, and it would be even easier for Scirra to implement assuming that layer creating business works.
I've yet to experiment with deleting and reordering layers though. My guesses are:
- reordering layers would mess with layer IDs, which could break stuff
- reordering layers would need extra complexity to handle ordering within sub layers, or moving a layer accross parent layers
- deleting layers would require manually deleting every instance on that layer
- deleting layers might actually cause other issues when changing layouts since changes to layers don't get reset when changing/restarting layouts
- layouts are not meant to work with 0 layers
These could all be addressed though.
The layout engine
Now, we've come to what I consider the last big piece of the big puzzle that is UI in C3.
Actually... let me give this a bigger title
3 - The layout engine
In my opinion, the biggest thing that C3 lacks is a layout engine. Being able to place elements on screen should be trivial, and yet quickly turns to hell as soon as anyone wants to make UI.
To make UI, one would need a system close to the flexbox system modern browsers use. This is, understandably, a massive undertaking, and replicating flexbox would be unfeasible.
However, flexbox does a LOT under the hood to work properly with the entire internet. I am just trying to place a sprite on a layout with a known size, known resolution, and I don't need most of the QOL features of flexbox. Just the big ones.
In that aspect, I am currently in the process of forking and implementing this rewrite of flexbox I found on github. We'll see how this goes. If this works, then we could all just move on from this discussion.
What if we didn't do that though?
Assuming this doesn't work or isn't feasible properly, what Ashley proposes is the next best solution: using HTML for UI. With the new HTML Element, it is clear that this is the direction he wants C3 to take for solving most of its UI problems, and quite understandably. HTML is solid, easy to work with, actively maintained by the biggest companies in the world, and has an ecosystem so massive it's simply unmatched.
Lots of web games also implement a HTML layer on top of their rendering canvas to do this. It's a great solution which has many upsides, while being fairly straightforward to implement.
It maximises result and minimises effort.
The issues with HTML for UI
I wouldn't be making this blog post if it was not for me making a case specifically against HTML for UI, and what we could do to solve that, even partially.
Cross platform support
HTML works great on most browsers, but as soon as we start changing platforms, we get issues. These issues are exacerbated by the fact that we're making a game with a tool designed for making websites, so there is very little chance that these are getting addressed any time soon.
Cordova
Cordova is known to have very bad performance, especially when using lots of HTML and CSS animations. These being the proposed solution for UI, and the mobile market being overflown with UI heavy games, this would make it even harder for C3 to be a viable choice for mobile game developers.
But at least, it works on Cordova.
Form elements, and different browser/OS combinations
Web pages changing accross OS/Browsers are not a new issue. Chrome, Safari and Firefox straight up don't have the same options for styling UI elements, and this will cause issues for many developers, especially beginners, when their UI just randomly breaks on Safari.
This doesn't even start to mention the mess that is Color Fonts cross browser, which if supported properly would have been an amazing fit for game UI.
Consoles support
Arguably the biggest issue I have with this is that games where the UI makes use of HTML cannot be ported nicely to consoles. It's already hard enough for browsers to make up their mind as to what HTML should look like, it would be even harder for consoles to figure that out. There is no way to make sure the UI stays consistent accross consoles, and it's possibly even harder to implement console specific input methods to work nicely with it.
I know this is not a problem Scirra should have to deal with, nor it is an issue most C3 devs would face, but this would be significant enough for most indie developers wanting their games to be published cross platform to just not consider C3 for their next title.
Rendering stuff on top of HTML
Ok let's assume you'll only ever be releasing games on the web, or on Steam for Windows. You managed to make your game vaguely work on Safari somehow, and you don't care about consoles or mobile. What now?
Another big issue is not being able to render stuff above HTML Elements. I had a solution in mind though and wrote another patch
(Source code)
This one is a lot more complex, here's how it works:
- Find every layer with at least one instance of SDKDOMPluginBase (the general SDK class that manages DOM elements)
- Increment a counter every time one is found: that's the number of canvases you'll need
- Create a canvas pool so that you only ever create as many canvases as needed, and make the unusued one invisible
- Make C3 render each layer set on its own canvas
- Reorder canvases and HTML elements when needed so that they match the layer order in C3
The biggest caveats of this approach are:
1 - Performance
Creating too many canvases would result in very poor performance, especially on weaker hardware
2 - In layer Z order is not respected
For simplicity, this always assumes that HTML elements are at the top of their layer. Cutting layers midway between canvases would take a LOT more work and is prone to creating many more bugs. This is hence outright not considered. It's a very reasonable limitation though.
That's a fair question. Not rendering anything on top of HTML is a reasonable limitation in many cases, but it's all about the subtle things. It's always about these.
Here is a non exhaustive list of things that would require, or be made significantly easier by rendering on top of HTML elements
- Integrate simple form elements like buttons or textboxes to a mostly sprite based UI
- An inventory where you can drag in/drag out items, or drag items accross inventories
- Complex scene transition animations using screen swipes or any form of cool transition you can think of (that hence needs to hide parts of the UI)
- particle animation in your UI
- Using a trail or custom animation to follow your mouse cursor
- displaying the player on the UI
- Displaying animated text using the C3 bbcode text (animating text in HTML is not easy at all)
- Shop UI that display previews, maybe even using Mikal's 3D model addon or the Spine addon
- Any UI that can move in front or behind other objects, like intradiegetic UI, or custom animations where you put the UI behind the game layer
- Putting a health bar on top of your UI
- In fact, making any form of HUD be on top of the UI, like zone titles, or loading icons etc
- Making flying text (like those text damage animations) in cookie clicker games or very UI heavy games
- Making cool special scenes like in Celeste, or many other games where games suddenly break the 4th wall and mess with the UI by having a character interact with it
- Making items you pickup fly towards UI with a cool animation
Many things on this list are not impossible to do with HTML, just a lot less practical. Some are outright unfeasible though.
We're getting somewhere!
While both suggestions are good in their own way, here's what I think about them
1- Adding a <img> from sprite feature
This would only cover basic cases, and I'm worried it falls apart as soon as animation is required.
This would also need to manually add support for tiled backgrounds, maybe the regular text object, drawing canvas, and any 3rd party addons. It's a slippery slope.
It would cover a good chunk of basic cases though.
2 - Only two canvases layer
This is much closer to what I am doing, but with the tradeoff of limiting usage to prevent performance issues. I would argue that 2 layers (canvas - HTML - canvas) is a bit too low and I would like to see that number be bumped up to a workable figure, like maybe 3 or 4.
Also, I think that allowing for another HTML Layer on top of the last canvas layer would also be a good idea and be fairly easy to add.
I think that implementing my system and then having a "warning" limit that redirects to a blog post would be the best way to deal with that. Kind of like how the collision editor does it, but the warning would only appear in preview with an error when the threshold is passed. That limit could be set to something like 4 (so main canvas + 3 extra canvases) and it would possibly never be an issue anyone would face, while still covering for performance issues.
Another solution would be to trade automatic detection for manual tagging of layers in editor, but then that would come with a bunch of other implementation issues down the line. Like who knows what layers *actually needs* to be rendered in a new canvas even if it's specified?
Conclusion
Anyhow, I still think a custom layout engine would be the best option, but it's definitely not the most cost effective one. A custom layout engine, while hard to implement, maintain, and extend, would solve the last big remaining issue with C3 at the moment.
Using HTML for UI is a close second, with some significant trade offs, but having a multi canvas rendering approach would solve the solvable issues. It would also make form elements a lot more straightforward to use, and less confusing for beginner users that might not understand why they render the way they do.
While these are not my only concerns about UI, this article sums up pretty much everything I wanted to say about it. I've thought about that topic for many years, and in my opinion it all comes down to these points. Solve some of these, and I can probably work with that.
I've posted an idea on the ideas board, so hopefully this conversation can keep moving forward: https://construct3-21h2.ideas.aha.io/ideas/C321H2-I-312