skymen's Forum Posts

    So basically, c3 forces bad types of programing in order to avoid bad types of programing while solving complex/advanced problems, and the solutions themselves are difficult to maintain or scale. You avoid one issue, but create a different issue.

    I thought a lot about design patterns within C3, and while I agree with all of what you said, I think there is one major takeaway you're not seeing; traditional programming patterns are never gonna be a good fit for C3 even if it had some of the features you mentionned because I think at the end of the day C3 has bigger flaws than just support for arbitrary design paradigms.

    C3 works far better with something like composition where a single family is an isolated feature, and you can compose an object by adding it to families. In this case, family inheritance is still useful because there will still be cases where you'd want some amount of inheritance within general systems.

    IMO even with all the features you want, it doesn't answer C3's 2 major flaws that it needs to address before any of this can even be tackled.

    - C3 is horrendous at dealing with data. When you have to handle a JSON, I feel like while C3 has some amount of solutions in place to make it possible to do, in general data management feels like an afterthought that was handled by an external addon rather than by a system implemented directly within the engine. This creates a large bottleneck of sorts whenever there is a need to treat and transfer large amounts of data because it needs to be serialised to JSON first, or it needs to be transfered through archaic picking systems that make the code way too hard to read.

    - C3 lacks an abstraction system. There is no way to refer to instances without knowing their type and without picking them through the SOL. This means cross family picking is hard, storing instance references is hard, and it's one of the main reasons composition systems fail so easily. As soon as you need 2 families to synergise when they are both being used on a given type, it becomes problematic.

    IMO if C3 had a way to store SOL as a variable and do more complex SOL manipulations, it would make C3 far more powerful. Also, if C3 had a way to make cross type families for types that share actions like set position and set size, it would also allow for very useful abstraction to be done.

    What I've ended up doing fairly often is relying on JS for complex data management and that kind of inheritance when necessary and using instances as just the visual aspect if it can't be done nicely with events. Still I think it's not a great solution and I wish C3 had systems in place for that.

    until we go through the engine and use private fields everywhere, which we've been planning to do for some time

    Since you've mentionned this a few times over the past few years Ashley I went digging into the code to see what this would do concretely.

    Private fields would absolutely make it much harder to do some of the bigger hacks that I've done in the past. The bigger ones being the following:

    Layer Manager: construct.net/en/make-games/addons/1053/now-c3-layer-manager

    (Dynamic layers before dynamic layers were added)

    Layer Magic: construct.net/en/make-games/addons/886/layer-magic

    (Rendering hacks to do some cool multi pass rendering on layers. I still think this would be a sick thing to have, but I stopped working on it due to huge bugs the hack caused)

    Limit FPS: construct.net/en/make-games/addons/1188/limit-fps

    (Wrapper around the C3 ticking method to force some kind of arbitrary limit)

    Experimental XXX Fix:

    construct.net/en/make-games/addons/1128/experimental-touch-fix

    construct.net/en/make-games/addons/1094/experimental-9-patch-fix

    construct.net/en/make-games/addons/906/experimental-text-fix

    (Replaces parts of the runtime classes to fix annoying issues with these plugins)

    What all of these have in common is that the way these operate has been to do something close to the following: 

    let oldClass = C3.Path.To.Class
C3.Path.To.Class = class myClass extends oldClass {}

    This way of doing it has a big advantage because it's non destructive, and more importantly, it is way less prone to breakdowns when changes to the parent class are made. It also allows me to be very precise with the hacks and reuse as much of the original code as I can. With private fields, this becomes completely impossible, and the only way to do something similar would be to replace the entire class and rewrite everything, which would make it painful enough to warrant doing something else instead.

    However, I dug around the code of C3 a fair amount in the past few years, and I don't think that too many fields can be made private. The engine internal classes communicate with undocumented public fields of other classes all the time, and usually these are the fields we'd wanna mess with anyway. From what I can tell, using private fields all over the engine in an efficient way would require rewriting a significant chunk of the engine's code and I'm not even sure how that would look like.

    Also, since private fields would most likely make replacing classes a pain in the ass, a lot of these hacks would probably then just revert back to hooking to random parts of the engine, and trying to inject code wherever possible which would in turn be a lot messier. So really, to me the idea sounds like a lot of work for not a lot of return. At the very least, IMO it doesn't do much that the idea of SDK v2 doesn't already do, and IMO again, adding better APIs and more features to the SDK is really the only sane way to fix these problems.

    Another reason why I don't especially like that idea is that being able to mess with engine internals is a great way to prototype feature requests before I make them.

    I have mostly halted addon dev since SDK v2 was announced to give it some time to mature so when I get back to making addons, I can try to use v2 exclusively and start porting my old addons. However, I still use Construct and I still mess around with it's features, and recently I made a game that made heavy use of Dynamic layers and I found a bunch of tiny additions that could be made to the feature that would make for nice feature requests, but I first wanted to experiment with them.

    For example, here is a snippet I use in that game that adds a layer effect to a dynamic layer 

    	let effectData = [["skymen_DropShadow","DropShadow",[true,	[0,0,0],0.75,4,60]]]
	let layer = c3_runtimeInterface._localRuntime.GetMainRunningLayout().GetLayer(runtime.globalVars.layer_ui_gun)
	let renderer = c3_runtimeInterface._localRuntime.GetRenderer()
	layer._effectList = C3.New(C3.EffectList, layer, effectData);
	for (const effectType of layer._effectList.GetAllEffectTypes())
		effectType._InitRenderer(renderer);
	layer._effectList._InitRenderer(renderer);
	layer._effectList.UpdateActiveEffects();
	layer._needsRebuildEffectChainSteps = true;
	layer.GetEffectChain();

    This snippet is very hacky and a real implementation would require a lot more work, but in the meantime this allows me to test it in a real game and see how well it holds up so I can then come with a well thought out feature request that can prove that the idea has legs. With private fields everywhere, things like this are 100% impossible and I don't think that's a good thing, at least for me.

    Anyway, I hope that by the time SDK v1 is fully sunsetted and private fields are implemented, SDK v2 will be mature enough and will provide a good enough API to make these kinds of hacks unncessary.

    The point I'm trying to make with this post is that the bigger hacks are usually born out of necessity and rarely out of laziness to achieve a similar goal with legal means.

    Addons like Empty Shell, Limit FPS or Layer Manager are born out of a real need to do something that C3 doesn't do and while I understand that V2 aims to make it harder for these addons to be shared with people that don't understand the risks, I would still like being able to mess around with things at my own discretion, even if that means I stop publishing my code.

  • FWIW the editor has a fixed limit on how many spritesheets it will render at once, specifically to cap the memory usage. For example if you have 1000 spritesheets, it will only render/compress something like 10-20

    Do you think it's possible to have an experimental setting (even something hidden like a URL query param) to change that limit? I'd be down to give it a try and report if changing the limit helps or not.

  • As far as I'm aware browsers, and Construct, don't have any particular memory limits that would affect even a large game. For example you can use as much texture memory on the GPU as you can - there's no limit there. No matter what software or technology you use, it's up to you to design your game efficiently and not use too much memory.

    I think the main issue here is that C3 seems to have a hard time handling memory with large amounts of image data. Even if that amount is way below the memory limits, when C3 is generating spritesheets, it seems like memory usage skyrockets the more images there are, and very often it leads to memory overloads.

    I've seen the issue happen on both Biogun and Astral Ascent, and usually the exact memory management efficiency vary depending on the browser being used. At the time I remember Edge was more efficient with memory, but now that it's Chromium based it might not make a difference anymore.

    It's way easier to get the crash when using larger spritesheet sizes and with multiple layout tabs open or multiple browser tabs/programs open on the computer.

    Currently, to minimise the crashes happening, you need to use only one or two layouts, and load the minimum amount of icons and sprites because any new icon spritesheet or sprite spritesheet being generated can cause the crash.

    I don't know if that can be fixed, but I think C3's spritesheet generation algorithm could maybe be made more memory efficient when too many images need to be handled, even if that means going slower.

    Also, an alternative could be to save the editor spritesheets in the project files within a special editor folder like uistate files so the sheets don't have to be generated again every time the project is opened. That folder could be gitignored to prevent merge issues.

  • My advice would be to, wherever possible and to the greatest extent possible, develop a separate external tool and have some way of importing that tool's output to Construct.

    If you build a separate tool which does something like produce a file you then import to Construct, you avoid all these problems and it will probably work fine indefinitely. So I'd strongly recommend you do that instead of trying to use a browser extension to hack unsupported things in to a large, complex and continually changing codebase that is not expecting that.

    I largely agree with that point of view, but as stated in my other forum post, the fact that C3 does not do any kind of live reloading means it's actually extremely complicated to build any kind of external tool that has a decent UX.

    I recommend you look at my Construct Crawler tool that allows reading and modifying Construct 3 project files. The tool would be infinitely more usable if it did not require closing and reopening C3 constantly.

    I wrote it at the time with the intent of making it easy to refactor or extend with more functionnality in the future, but any feature I think of ends up being made completely unusable because of the archaic UX.

    Being able to do live reloads means external tools can be used with the project open, and this would also allow addons to communicate with external tools directly as well.

    Since the editor SDK is so limited, the only way for 3rd party addons to get data from the open project is either through an external tool gathering data from the project folder, or from an extension that gathers that data from the editor directly. Without live reload this significantly limits how useful external editors can be (since most likely an editor is not meant to read data but also to write back what you want)

  • What you're looking for is a pathfinding algorithm. The C3 pathfinding behavior won't help because it uses a grid of nodes rather than what you have.

    The only way to implement what you want is to implement something like Astar or Djikstra since you have only few points.

    Here's a video that explains how the algorithm works:

    youtube.com/watch

  • Hi CairoCreativeStudios

    Absolutely!

    Please DM me on Discord :)

  • I initially replied here, but to avoid diverting from the original topic, I made a separate thread:

    construct.net/en/forum/construct-3/general-discussion-7/custom-3d-renderer-instead-182406

    I think Mikal's proposition deserves to be discussed and I don't want my response to take away from that.

  • This is a followup to this forum thread:

    construct.net/en/forum/construct-3/general-discussion-7/proposal-community-developed-182378

    In there, Ashley said the following:

    Are you sure that there are not other approaches that could make an integrated 3D engine work, or new APIs that would smooth integration? It seems to me that it ought to be possible to basically import existing Construct objects in to a 3D engine

    I'm down to humor this because I'm trying to do this exact thing nowadays.

    For the past few months, at Dedra, we've been working on two 3D games made in C3 with the purpose of pushing the engine to its limits and see how far we can go. While so far it's gone well, we've already ran into multiple roadblocks that have made me question the legitimacy of this plan.

    So, I've started messing around with OGL initially and now with ThreeJS within C3. (OGL was chosen because it was very lightweight, but after working with it a lot, I figured I would need to build a large layer on top of it to get the results I wanted, which would make my fork just a poor version of ThreeJS)

    During my tests, I was doing the whole thing in JS and loading models from the asset manager to run benchmarks, but the plan has always been to try and create a 1:1 mapping of C3->ThreeJS since I'm pretty sure most of it can be done automatically.

    I'm glad that you seem to agree that this is a decent plan so let me list a few features that are currently needed to do make this possible:

    1- Getting texture data

    In C3, there is no legitimate way to get the texture data from anything without using undocumented features. I'm not just talking about sprites here. I would like to use:

    • Sprites
    • Tiled Background
    • 9 Patch
    • Text. This one is especially important. C3's Text BBCode system is way too useful to give up on, the alternative would require loading fonts manually, and doing text rendering using some other solution. I suppose I could use the IRendererText interface, draw to an invisible texture and show that, but that's extra work to recreate something that is already there.
    • Particles. NB: This one is far less important as a particle system would probably need to be rewritten in 3D and could pull texture from sprite. Also, I implemented Effekseer in C3 a few months ago, but that particle engine was designed for ThreeJS originally so most likely I'd use that for complex particle systems

    This feature would also be very useful for my Empty Shell addon. It currently uses an undocumented feature to get the texture from a sprite, but if this feature was done, I could make it work with any of these objects.

    2- Getting linked instance objects

    A few objects in C3 get their texture data from other places. This is the case for 3D Object which is the most important one by far.

    Particle also does that but as stated this is less important.

    Having that system be exposed in the new SDK would also allow me to port my Empty Shell addon, since it relies on a similar concept for texture linking.

    Intermission

    These two are by far the most important features because otherwise none of what you're proposing is possible without hacks, without jumping through massive hoops, or without just ignoring C3's editor entirely and storing everything as separate files.

    3- Mesh data

    I think we've already made a very solid point for this in this feature request.

    However, this is also very important for a C3->ThreeJS system because otherwise there is no way to port anything that uses mesh. For this we need all the data, so world position, but also local positions and UV data. Without all of it, it's not possible to do it.

    A big reason why this is very important is that mesh is the only way to do 3D rotations and skews in C3. Doing this in full Three requires storing a lot of data as numbers in instance variables which becomes cumbersome when meshes do the same thing but better.

    4- Extra notes for full support

    A few other far less important things required for this would be:

    - Support for Video in the scripting runtime (URL, current position etc)

    - Tilemap texture. This one would be far more situational.

    - DrawingCanvas texture. Unless I'm mistaken, it's not possible to get the canvas texture data from DrawingCanvas.

    - Spritefont. I personally don't use Spritefont, but it would help

    Conclusion

    This would be quite a lot of work to do, and a lot of it would essentially be syncing the full state of C3 objects to 3D.

    I am personally down to explore this possibility, and if it works out well, I am planning on open sourcing a lot of my work so other people can give it a try as well.

    However, right now it's just not possible to do without reaching beyond what is allowed.

    Also, if I have to go the length of implementing a 3rd party 3D renderer and do all of this syncing work, it would only be fair if I got to use this engine to the full extent. However, right now, C3 lacks a lot of UX features that would also need to be worked around to have a decent workflow. Something as simple as placing 3D shapes with a 3D rotation is impossible and yet would be extremely useful in a proper 3D renderer. Do I need to store rotations as instance vars and keep hitting preview to see if I rotated my object correctly? I'm already doing this, but it's far from ideal.

    You have to realise that following this route means gradually moving the problem away from the runtime, and straigth into the editor, and given how little work has gone into exposing more editor SDK I frankly doubt we'll be allowed to create custom views any time soon to implement a 3D editor ourselves, let alone get all of the data I outlined above directly from the editor SDK.

    Scratch that! Let's say I write a brand new 3D editor completely separate from C3 that gets all its data directly from the project folder. I can't live reload anything in C3 (besides scripts in the scripts folder) so to preview my changes, I need to restart the C3 editor and THEN the preview.

    (all of this assumes it's a sensible idea to write my own 3D editor and use a 3rd party 3D renderer. That sounds far fetched, but I can think of a few theoretical good reasons to actually do that)

  • Hi Ashley, in the past few C3 releases you included 2 examples for custom drawing with canvases inbetween layers. This is pretty cool, and in fact it's something I've been experimenting with in the past, but I'm curious about something:

    Why do it this way rather than using C3's provided dynamic texture system? Dynamic textures can also take canvases as input, and with them I'd be able to integrate rendering with a lot more precision than with an extra canvas inbetween 2 layers.

    Also, using dynamic textures, I can place my custom drawing in the C3 world directly and have it automatically be affected by layout/layer positionning, the Z ordering, and the effects compositor.

    Is it that this method is more performant? If so, how much better is it?

    I'm curious to learn more about the rationale behind these. As I said, I've been experimenting with something like this recently to implement custom 3D in my games, and for my specific use case, both would actually work fine (with some minor caveats on both sides) so if you can explain, it would help me a lot.

    Sometimes the addon developer responsible has left the community, and there's nobody left to help them.

    It's a real nightmare for us to deal with and is continually happening in the background, unbeknown to most addon developers, often including the addon developers responsible for the problem.

    I know I've already said this, but in case this ever happens due to one of my addons, please let me know so I can fix it as soon as possible, or at the very least provide a way out of using my addon for people affected.

    Also, Mikal mentionned it, but usually when there is demand, some addon devs take old unmaintained addons to update them so they work on newer versions of the software. This is happening at the moment with ProUI, it has happened in the past with Rex's addons, and nowadays we even have the Construct Open Collective specifically designed so the community can chip in to fund free and open source tools/addons for Construct.

    Anyway, I think it's very fair for addon devs and game devs with large projects and revenue on the line to be extremely upset by the situation, and especially for people to be skeptical regarding Scirra's ability to address said concerns swiftly. A lot of addon devs are currently watching how the situation evolves, because there is still quite a lot of uncertainty about what addons can and cannot be ported to V2 for the ones that fall vaguely on the edge.

    For my part, I am also delaying work on making C3IDE2 work with V2 because I am still unsure how much it will change in the following weeks.

    Anyway, I appreciate you taking the time to listen to feedback and wanting for the move to go as smooth as possible (as smooth as a large disruption can be) but for many of us, there is not much we can do besides wait and see how the situation evolves and how SDK v2 improves in the following weeks.

    Exactly! Yet people is still thinking that r388 LTS is the solution. Ashley already stated LTS won't have support forever... eventually, our long term projects that uses undocumented features won't open.

    Astral Ascent is still being developped on r232 or a version near to that. They locked the version more than 3 years ago and it's still working out for them. An LTS would provide updates for NW.js, SDK updates for mobile, and necessary security fixes and this would give us even more time.

    So, with an LTS, you have 1 year until SDK v1 is deprecated, then that last stable gets supported for 18 to 24 months according to Ashley, then you can still keep using it for as long as you want. That's a minimum of 3 years where nothing will happen. If we assume Astral Ascent is a viable scenario (3 years working on an unsupported version with no issues) then you have 6 years ahead of you.

    What Ashley means, and has always meant with "stop supporting" means stop giving it updates in the latest versions of the software. Every single "unsupported" feature Construct 3 has ever had in the past are still perfectly accessible in older versions. Why would you assume they would feel the need to retroactively change more than 350 releases of the engine to nuke all traces of the old SDK? That makes zero sense.

    With his latest reactions, what's the real guarantee that he won't change his mind later on. This whole SDK v2 thing came from a simple question to access engine internals 2 months ago, which suprinsingly rushed Ashley to launch v2.

    This is false. The SDK v2 has not magically appeared in Ashley's mind 2 months ago. It was definitely the trigger for doing it, sure, but he has mentionned the idea several times in the past few years. Nothing Ashley is saying now is new, and he's always said it would happen at some point in the future.

    Again, I understand being annoyed and frustrated, but don't make up things to be mad at, and don't form facts from conjecture. This really just fuels more negativity, spreads misinformation which makes other people panic, and it makes discussing the issue harder for both addon devs and for Scirra.

    Past a certain point of noise and anger, Ashley will just close the gates of communications and nothing will have been achieved. This has happened in the past, and it's counter productive for absolutely everyone.

    If WebView2 becomes available on Mac and Linux - we won't get it. A new version of NWJS is released (which requires updated Steam plugin) - we won't get it. Scirra adds Switch export (haha, I know) - we won't get it. Google or Apple introduce some breaking change - we won't be able to export for mobile. And so on and so on.

    WebView2 and NW.js exports are fairly easy to do on your own. You can just download the releases from their websites, export to HTML and put your HTML in the correct folder. If push comes to shove, you can just keep exporting with your own stuff.

    NW.js export versions are not tied to editor releases and older versions should still be able to export to latest NW.js because the list gets fetched when you start an export.

    Steam plugin compatibility can be done on your own by replacing the prebuild files and you can get them here: greenworks-prebuilds.armaldio.xyz

    For Google/Apple changes, if you mean SDK changes that would be tied to 3rd party addons (i.e. Chadori in this case) and not Scirra anyway. Agreed that this does make it harder for addon devs to keep supporting old versions, but in the absolute worst case, nothing prevents you from just opening Chadori's addon and making the changes you need to do in there. From experience digging in there, Chadori's code is pretty well put together, so even if you don't know a lot about addon dev or SDK dev, you should still be able to find your way around inside of it.

    For OS version changes, I am not very knowledgeable on mobile exports, but if you don't go through Scirra's server it should be doable to make your own thing. And I think the minimum supported version is a value to change in the app manifest? Anyway, this one I don't know for sure because I'm not a mobile dev, but everything else you can reasonably just keep doing it by digging in the right places.

    Is it slightly more painful? Sure. But it's also still doable AFAIK.

    I can't go to the an older version of construct because sdk v1 will be totally retired

    No one has ever said anything about older versions ever becoming unaccessible.

    In fact, Ashley has seemed fairly keen on providing an LTS version to keep compatibility for the last stable that supports V1.

    github.com/Scirra/Construct-feature-requests/issues/261

    if your worry is that you want to keep opening your projects in r388, you will still be able to do that, in the same way you can still open Construct 2 projects in C3 if you go back far enough in the releases

    I understand being mad at this, but I really want everyone here to be mad for the right reasons, so make sure you understand what's going on. Otherwise all you do is create noise that makes the communication muddier for everyone else.

    You are launching an Addon SDK v2 with an incomplete API and blaming plugin developers that we do not cooperate.

    The new SDK is not even out of beta and has had a single week of updates, so IMO it really does not count as "launched". Missing features and requests are being added to the feature request board. If you need a feature, add it there please.

    Once the SDK reaches stable in a few months, you will still have a full year to port the addons and request any missing features you might need. Sure, it's a pain to port the addons, but it's not like anyone is asked to start porting immediately. If the LTS does happen, this can buy everyone even more time before they absolutely have to update.

    Wait? Shouldn't that be the minimum? How are you suppose to extend C3 in any meaningful way with an API that isn't even flexible enough to create its own existing features?

    I thought these discussions are about things like Skymen hacking in the ability to have different sampling modes on different layers. Things that currently aren't possible.

    No, currently with SDK V1 here are the built-in addons that use undocumented features (and by that I don't mean easily replaceable ones, I mean the plugin is impossible without them or requires rewriting chunks of the engine):

    Plugins: 3D Camera, FlowchartController, ShadowLight, Sprite, Spritefont, Text, Tilemap, Timeline Controller, Drawing Canvas

    Behaviors: Bullet, Car, Custom, LOS, Persist, Solid

    Addons that use undocumented features for what I think are important features, but that can be worked around or removed for worse UX:

    Plugins: 3D Shape, 9 Patch, Particles, SVG, TiledBG, HTMLElement, Advanced Random

    Behaviors: 8 Direction, Move to, Physics, Platform, Tile Movement, Tween

    And here are ones that use undocumented features to provide slightly better UX (again, and said UX cannot be replicated) but otherwise fit perfectly inside V1:

    Plugins: LocalStorage, Audio

    Behaviors: Sine, Rotate, Orbit

    This list has been made by being reasonable on what counts as an undocumented feature, and what is just built in addons being better designed inside the engine.

    I am not making this list to ask Scirra to make addons using the SDK features because this is far from a viable option. I am just using these as an example of how low the bar is. When we say 3rd party addon devs have been asking for features we consider as basic, we do not mean "hacking in the ability to have different sampling modes on different layers". Not anywhere close.

    In fact, I asked that same exact question to Ashley 7 months ago.

    At the time, the answer boiled down to "don't make an addon then".

    For some addons, this can mean "use events", but for other addons, this means "don't make that at all"

  • Try Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Try Now Construct 3 users don't see these ads

    I'm gonna give one last thought on this subject. I personally think there is a huge difference between this, and what happened when C3 released, or with C3runtime or the modules change. For one the modules change often implied very minor tweaks so it does not count IMO.

    For the two other cases, the changes were indeed drastic, but they also came with MASSIVE benefits. Everyone happily obliged despite the required effort specifically because there was no debate on whether the new thing was better than the old thing outside of minor technicalities.

    Did we lose a few features in the move each time that made some addons impossible to port? Yeah, sure. But in return we gained a lot more, and C3 has evolved way beyond what we were hoping at the time.

    My main worry with SDK v2 is not that it is a breaking change, far from it. It's that it's a breaking change, and so far I am unconvinced that it actually solves the issues it's trying to solve.

    I can definitely see a scenario where everything turns out fine, and it actually leads to better cooperation between Scirra and addon devs, and we end up with a much cleaner addon ecosystem overall, but this relies on significant effort and goodwill from both parties. I wanna thank Mikal and EMI INDO for leading that march, by suggesting improvements to SDK v2 and upgrading the tools we use to be compatible with V2 already.

    I personally don't have much time to contribute anything meaningful just yet as I'm busy with a billion other things, but I genuinely hope to see things move in a positive direction by the time I can.

    At least, this is a good time to demonstrate the significant value provided for free by the goodwill of the people in this community. Many people (and not only addon devs) are working hard in silence to make Construct a better engine and to make its community more welcoming, and often don't get a lot of recognition for it.