Ashley's Forum Posts

    • Post link icon

    Why are you trying to convince or gaslight everyone into believing that the developers are too lazy to update the plugins?

    I'm afraid I genuinely have no idea where you got the impression I thought addon developers are lazy. I have been talking about how I am keenly aware of just how much work this is going to create and how disruptive it is going to be. We are asking addon developers to co-operate with us through the transition and I am writing a lot on the forums here to explain the reasons for this, as it is fair that we should have to justify why we are doing something so disruptive.

    WackyToaster - regarding the "group handler" addon: I'm afraid that is the very reason we had the warning in the SDK. It could break at any time. If it's worked for several years already, that is absolutely no guarantee at all it will continue working even one more week - unless we have a proper API with encapsulation. We could rename some variables or fix a bug in the engine, and your addon will be broken. It could break repeatedly, every few weeks, for a year or more, and then ultimately break permanently, causing untold disruption along the way. This is already happening with other addons and will continue happening. Given that in the long run pretty much all parts of the engine are eventually changed or upgraded, all such addons accessing internal details are likely to eventually break. Part of the point of this project is rather than sit around and wait for disaster to eventually strike for all these addons at random and unpredictable times, we're managing the process up-front with a published timetable for what action will be taken so everyone can plan ahead and the problem can be solved once and for all.

    By ignoring the warning in the SDK, I am afraid you must carry some of the responsibility. We did say undocumented features could be removed at any time. So we reserve the right to remove all those features, break your addon permanently, and say "tough luck". Then all the customer's projects using the addon are broken and it was because you ignored the warning in the SDK. However, we are not doing that: instead we are laying out a timetable and doing what we can to try to make the transition smoother. If people really need your addon and removing it is infeasible, they'll have to stay on old releases (possibly a future LTS release) until they finish their projects; if it really is going to cause enormous disruption, we might add necessary features to the Addon SDK v2 to allow you to port your addon, depending on a judgement of the situation, feasability of maintaining the APIs, etc. However, at the end of the day, we reserved the right to permanently remove all those features, and we did warn you of that.

    The industry came up with encapsulation decades ago precisely to solve this problem. Experienced developers know that if outside developers get access to internal details, of course it ends in disaster, of course it makes a huge mess for everyone. All other tools use encapsulation for an addon system to avoid this. As I've explained, the addon SDK lacked encapsulation mainly for historical reasons as JavaScript had poor support for encapsulation in the past. Either you learn the lessons of history and respect encapsulation, or you disregard them, and you learn the hard way as it invariably ends in disaster.

    Given all this, I have to say, in my view it was unwise to develop such addons. Adding a few small handy features via accessing internal details, at risk of causing disaster and potentially ruining everyone's projects in the long run, is just not worth it. The alternative may be some more clunky event sheets and perhaps a slightly less smooth workflows, but people's projects keep working indefinitely. In the long run that's the better outcome. In my view, fully understanding the reasons encapsulation exists and the severity of the risks of bypassing it, I would not even consider trying to bypass encapsulation, under pretty much any circumstances at all short of a genuine emergency where not acting is also sure to cause a disaster. Anything else at all, even significant inconvenience, is better. The extent of the disaster it risks must be avoided at all costs. This is why, despite all the disruption, we believe it is absolutely essential that we move to a new addon SDK with much stronger encapsulation. Either we have planned disruption now, or we have never-ending continual unexpected disasters of increasing severity for years to come.

    I am also aware many addons are well-behaved and have never accessed any internal details. I do regret that these addons still need updating to the new SDK. I hope that by explaining the situation in this much detail, I am highlighting how important this work is and why it's for the best in the long term.

    • Post link icon

    Construct 2 never had such issues. A plugin from a decade ago still works even now. My projects from a decade ago in Construct 2 still work even now.

    Oh man. What a thing to say. As I said before, I pretty much have trauma from how badly the addon system worked out with Construct 2. It was even worse. Just because you can point to a few addons that kept working, doesn't mean that the overall design wasn't a huge mess that worked out disastrously for many people.

    He doesn't seem to understand the magnitude of the problem he's creating.

    As I've said before, I am keenly aware of the amount of work this will create, and that is regrettable. However, please understand that we are not doing this for fun: the current compatibility disasters and future risks are in my view so severe, that this is very much necessary, despite the magnitude of the problem it will create in the short to medium term. If you don't believe me or disagree with that judgement, I'm there's not much more I can add, other than to say that you either learn the lessons of history, or you are doomed to repeat them. The industry came up with encapsulation to solve these kinds of problems, and we are moving towards that industry-standard approach. If we postponed this by a few years and then were forced to do it anyway, it would be even more difficult and even more addons would need updating. So if it's going to happen - and in my view it absolutely has to - then the sooner we act the less bad it will be.

    • Post link icon

    no more behaviors, no more plugins, no more effects, that do different things,

    No, these are all still possible with the Addon SDK v2. In fact there are no changes at all for effects.

    so i guess.. you'll add big API in SDKv2 to allow people to create new types of behaviors / plugins

    Yes, that's the goal.

    But service integrations (like firebase/ads provider) require a very active maintaining

    If a third-party service makes a breaking change, it's nothing to do with our SDK. This is an entirely different area of compatibility and not related to the migration to the Addon SDK v2.

    open source ... open source ... open source

    I'm repeating myself here, and there are only so many times I will repeat myself before I deem it no longer helpful to respond. Here we are talking about the long-term maintainability of installable addons, not whether you can download a copy of the engine source code. Any open-source project with installable addons will also have to consider long-term maintenance of the addons that are created, and the professional, industry-standard way to do that is encapsulation, which is what the Addon SDK v2 does.

    • Post link icon

    As guidance I would be very interested to hear what some of your top 2-3 3rd party addons are - that you think substantially help the C3 community and Scirra?

    The addon SDK is best at integrating additional platform features or third-party services. For example third-party addons that provide support for ad networks (of which there seem to be dozens), or back-end services like authentication, high-scores and analytics with various third-party providers, or enhanced platform services such as advanced IAP or other monetization features (perhaps through Cordova plugins, which I think the addon SDK has reasonably good support for), or integrating with third-party platforms like itch.io and Newgrounds, are all good examples of the kind of thing the addon SDK was designed for. All the possible integration work with different services and platforms is a huge amount of work which is infeasible for us to complete alone, and if we do it anyway it takes development time away from core engine features that only we can do. Allowing third-party addons means all that integration can happen without needing us being involved. None of that needs addons to access the internal engine, and that is partly why the originally documented API was quite thin.

    It is indeed regrettable that these kinds of addons which comply with the documented API will still need updating to the Addon SDK v2. However assuming the addon doesn't have a lot of engine code, updating them should hopefully be pretty straightforward, and I think in some cases will amount to renaming a set of methods and variables. I am still keenly aware of the extra workload this puts on addon developers, but as I've said, I do believe this is essential for the future reliability of Construct, and we wouldn't be doing it if we didn't think it was absolutely necessary.

    The addon SDK was never designed to be able to manipulate the internals of the engine, and as I've said before, no other professional engine that has an addon system allows that either, because everyone uses encapsulation, because everyone understands that the alternative ends in disaster. I recognize that some addon developers do want to be able to do more advanced things with the engine; the best approach for that is a more comprehensive and documented API which is supported long-term. The Addon SDK v2 moves to using the existing scripting APIs which you can explore in the scripting reference. Hopefully it is clear this is already a huge leap in the documented API surface. As the original post here explains one of the benefits of the new architecture is new APIs are automatically available to both the scripting feature and the addon SDK, meaning we can get more done with less work, and improve both faster. So while I fully acknowledge the transition is going to be disruptive and painful, in the long run, I am sure it will result in a much better and far more reliable addon system.

  • There is no reliable way to detect if the user is definitely online or definitely offline. They could have an intermittent connection, or the status could change just after you check it. The only good way to do this is: don't check for connectivity, just attempt network operations, and handle them if they fail.

    • Post link icon

    One more thing. The primitives you need to write a behavior are setting the position and testing overlap. The "push out" algorithms are combinations of "move a bit and test position".

    Here's the code for a simple "push out" method:

    Here's a sample project file using it.

    That's 19 lines of code. It works to 1px precision (like most of the internal methods). Want sub-pixel precision? That's not easy with the internal methods, but if it's your code you can tweak it to your heart's content. Want even more advanced collision handling? It's your code, you can customize it. And this will be supported forever - there's no internal engine access and no risk of disaster.

    As an aside: why aren't the internal 'push out' APIs already exposed? Well, we have learned a lot about writing collision handling code over the past decade or more. If we were to start over I could probably write those methods a lot better with an improved API. However backwards compatibility concerns and time constraints mean they're just left as-is for the time being - slightly sub-par in their overall design, but they do the job OK; for us it's not a problem as if it's internal code and we decide it's a problem, in theory we can just go and replace everything and update our internal references. If we expose them as-is in the JavaScript API, then we basically get locked in with the slightly sub-par design, and it becomes far more difficult to upgrade in future. So really to expose them means redesigning a set of collision APIs - a much more significant job. And even then, you can't customize the internals of the engine easily. What if you want something like sub-pixel precision or some other more advanced handling? Well, if you start by writing 19 lines of code, you have the most advanced solution available: a completely customizable implementation that you can hack around with as much as you like and there is no risk of compatibility disaster with projects. It's even more flexible than adding an API for it. The community could share libraries with useful implementations, and customize those libraries if they need to, and JavaScript Modules are extremely good for code re-use.

    I think part of the problem is once people figure out they can reach into the internal details of the engine, then people end up solving every problem they have doing that - risking a compatibility disaster every time - even when there are straightforward better options around. For example this suggestion appears to involve code reaching in to the internal engine and risking compatibility disaster when a static method could have sufficed and would have worked robustly indefinitely. If we use encapsulation to close off the access to the engine, then I suspect in many cases people will revert to already-viable workarounds - which they could and should have used in the first place - and in future people will start to learn the better solutions that don't involve reaching in to the engine internals all the time.

    • Post link icon

    If you need multiplayer or online features - you HAVE to use Playfab, Photon, Firebase etc. Making mobile game - you NEED monetization addons. Working with a publisher - you NEED analytics addons.

    You don't need to access the internal details of the engine to write addons that access third-party services like multiplayer and analytics, or platform features like monetization. All those kinds of addons can still be made with the Addon SDK v2. We are not removing support for making addons entirely!

  • Try Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Try Now Construct 3 users don't see these ads
    • Post link icon

    Please remember the goal here: it's to stop customer projects getting ruined by updates. That is an incredibly important thing to do on behalf of customers. I know this creates a lot of work and it's going to be disruptive. Like I said, we're not doing this for fun. There is a serious problem and it needs solving. Unfortunately without proper encapsulation, the problem is not going to be solved. This is well understood in the software industry and the reason encapsulation was invented. So to solve this problem, I believe we have no choice: we have to do this, and the longer we leave it the worse it'll be. Leaving customer projects getting ruined on a regular basis is simply unacceptable for a product like Construct; the status quo cannot continue, and I feel that to allow it to continue when we know it can be prevented would amount to negligence on our part.

    I agree that the lack of a proper encapsulated API to begin with was a serious mistake. However in the past JavaScript had pretty much no support at all for encapsulation, so there was no way to protect the API. This is partly the cause of the problem: historically there was no way to prevent this problem, and so we couldn't actually have designed the API this way from the very start. However I knew that it was a huge risk to not have encapsulation, hence the warning in the addon SDK documentation saying "don't access internal features". Unfortunately that is no match for proper encapsulation and it didn't prevent disasters from occurring. Now with more modern JavaScript features it's more feasible to do, and as the SDK warning has not prevented a very serious problem from occurring, and we now have the option of using proper encapsulation, we feel obliged to do that to prevent customer's projects from being ruined.

    Other engines also have a much more extensive feature set and API (Unreal, Unity, Godot) for both general scripting and plugins, if C3 was at a similar level for SDK V2, it would be more acceptable

    Our goal over the next year is to increase the capabilities and APIs available for the addon SDK v2 so that you can do more advanced things with it, and try to make sure where feasible, all popular v1 addons can be ported to v2.

    So you're well aware the encapsulation does nothing?

    In most programming languages, encapsulation is not perfect. For example C++ has encapsulation, but you can read random memory address and access things you're not meant to. Any seasoned C++ developer knows that's unmaintainable and will end in disaster. The goal of encapsulation is just to make it as hard as possible to access internal details to discourage it as much as possible. In the same way, our encapsulation measures in the Addon SDK v2 are designed to make it as hard as possible to access the internal engine details. It doesn't make it impossible. If addon developers try to bypass it, then we will just create the same problem all over again - and we will have to take steps to mitigate that, to prevent the ultimate problem of customer projects being ruined. Once again, I am reduced to a warning, even though that has proven to be weak: if you find a way to bypass encapsulation, don't do it! It'll end in disaster, and we'll end up in a situation like this again. Hence my attempt at conveying a warning in the strongest way I can.

    I'm really upset with this decision, which only confirms scirra doesn't care about their community

    Do you think caring about our community means allowing customer projects to continue to get ruined over the next few years, and possibly risking some huge disaster, when we know we can take action to prevent that?

    Can you at least help devs port to v2 as easily as possible? Please help them with their requests.

    We do intend to do this, with SDK samples, comprehensive documentation, an upgrade guide, and so on. Over the next year or more we plan to be continually providing support, SDK improvements, and any other improvements we can to make it easier to move to the new addon SDK. Part of this project is being proactive over the next year to make sure the transition can go as smoothly as possible. We can't do everything right now, as we're only just starting the process and it will depend on feedback from addon developers, but we will be actively working on this in the long term.

    • Post link icon

    Construct's compatibility goes right the way back to Construct 2 in 2011. That's 13 years and counting of backwards compatibility (for the engine - I know that doesn't apply to addons). We have to take the long view and think: where do we want to be in 5 or 10 years?

    Do we want to have years of rolling compatibility disasters, lots of customer projects ruined, maybe some huge disaster happens, and then we have to go through this process of moving to a new SDK but with another 5-10 year's worth of existing addons created which will make it even more painful and difficult than it is now?

    Or do we bite the bullet and do the upgrade sooner rather than later, and then we'll be in a better place in 5 to 10 years from now?

    I know it's going to be painful, but in the long run, the latter is the better option. We have been through this process before, with all addons needing upgrading for moving from C2 to C3, and all addons needing updating for moving from the C2 runtime to the C3 runtime. We got through that - it's painful, some addons get updated, some don't, some people find workarounds or alternatives, some people stay on old versions of Construct to finish their projects then update for their new project, but in the long run, we got through it and it was OK in the end. I'm sure we'll get there in the end in this case too. As I said, I strongly believe this is absolutely necessary to ensure customer projects keep working in the future beyond this, and it aligns us with how all other software in the industry works. If you move to another tool with an addon system, you'll find it works like the Addon SDK v2, because encapsulation is the industry standard.

    • Post link icon

    Only plugins and behaviors need to be updated to the addon SDK v2. There are no changes for effects or themes.

    • Post link icon

    We had a warning in the SDK documentation saying "don't use undocumented features, they can change, break or be removed at any time". Addon developers largely ignored that warning, and hence the difficult situation we're in. People ignore warnings. We're not going to try relying on warnings again given it already failed so spectacularly.

    • Post link icon

    You forgot to answer one big point, why isn't the warning for addons made with SDK 1 enough ?

    Because unless we remove it, people will just carry on using it, and the disasters will still happen.

    Why wait for a hypothetical disaster later when we can enjoy a catastrophe right now

    It's not hypothetical. Disasters have happened, are happening, and will continue to get worse.

    • Post link icon

    For post length reasons I'll try and answer a few other questions here:

    Will Scirra provide assistance to help with porting from V1 to V2, besides the very simple SDK examples?

    There will definitely be a "porting v1 addons to v2" guide in the documentation. I've also already created commits on the GitHub repository showing diffs so you can see what was changed to go from v1 to v2.

    Is there a possibility to make a 'wrapper' around V2 SDK to support previous V1 SDK addons or at least for the vast majority of the V1 SDK?

    For simpler addons, porting mainly involves renaming things (e.g. see here). Perhaps some tool could be made to convert the basic template, but I think the APIs are too different to allow for full automatic conversion of arbitrary code.

    My hope is that is because the scripting interfaces for addons will be 'automatically' generated somehow.

    The addon SDK v2 classes are the script interfaces (they actually derive from the script interface classes), so there is no extra work to add a script interface.

    Construct has built in vectors, and a built in color class that handles all the math and conversion, yet if an addon needs these, your force to rewrite all that code?

    Anything we expose to addon developers must be supported permanently and with backwards compatibility, otherwise we end up with the same problem of customer projects being ruined. So we can't say 'here, have access to this library of vector/math/utility code' - we have to design a public API that will be supported forever. That's a lot of work and a significant long-term maintenance consideration, and can take away our flexibility to do something like an engine optimization by changing the library code. Instead the approach we have taken with the script APIs is to use the browser built-in features such as DOMRect instead of a custom rectangle class. Want more? Import a JavaScript library. Problem solved, and no risk of compatibility disaster.

    The solutions are far from ideal, sure, but with addon SDK v2, the alternatives become far worse, and IMO, would lead to way bigger issues down the line.

    The downside of accessing engine internals is ruining customer's projects. This can be so bad, in the long run it is in my view significantly worse than any of the other options. I guess you haven't had to deal with desperate customers begging you to fix their ruined project and been unable to help? We definitely have. In my professional opinion: literally any other solution is better than modifying the engine internals. Hacking the engine internals carries such a severe risk of disaster that it must be ruled out, which is why the software industry invented encapsulation and has been using it widely for decades. In other tools with a proper API with encapsulation, people get by with workarounds or alternatives, and while that may be painful and difficult sometimes, it's a better option.

    Part of this project is over the next year or so to make sure the v2 addon SDK has sufficient APIs that it can cover necessary features, or just add new built-in features instead so there doesn't need to be an addon. We do want to minimize the disruption and make sure there is as smooth a transition as can be achieved. However, please, please don't try to bypass the encapsulation, or start copy-pasting chunks of the engine. Please, please, please please please please don't do that. I pretty much have trauma from how utterly awful the consequences of that were with C2. It is just a total, epic disaster, for us and our customers. Please please please please please don't do that. And it is the risk of that kind of epic disaster we are trying to put an end to with the addon SDK v2.

    • Post link icon

    Folks, please remember the goal here is to stop a regular occurence of customer projects being ruined by permanently broken and unmaintained addons. I don't think addon developers always see the consequence of this, so perhaps it is harder to understand. But from time to time we get desperate emails from customers who have projects they've been working on for years begging us for help, when the problem is they use some addon that is now permanently broken. Unfortunately we usually cannot help them, and it really pains me to see that happening to customers. This has happened a bunch of times in the past, it's happened recently, and it's going to continue happening. I genuinely fear that a few years from now there will be some huge compatibility disaster and thousands of projects are permanently broken, or we have to retract some major improvement and cancel plans for further improvements - a catastrophe either way. If that did happen I'm sure everyone would be saying how our top priority is to better design our software to prevent that happening. I think the main reason we have avoided that so far is luck, and I don't expect our luck to last forever. We're trying to solve this problem before such a disaster happens, and to stop the regular occurence of ruined projects.

    The addon SDK v2 solves this by using proper encapsulation. That's the industry standard approach. Literally all other software already uses that approach. Nobody allows unfettered access to engine internals through installable addons, because everyone in the industry knows that's a disaster waiting to happen. Access to source code is different to this: we are talking about the long-term compatibility of installable addons written by third-party developers, not just whether you can download the entire engine source code.

    I am largely just re-stating the same case here, and I don't really want to have to keep re-stating it over and over. I am fully aware of the extent of disruption and extra work this is going to cause. This is regrettable. However consider that we are not doing this for fun. We could just carry on as we were before - but in my professional opinion the problem is so serious that it does warrant this disruption. We have a responsibility to take tough decisions to do the right thing for the product and for customers.

    • Post link icon

    it looks like you plan to break every single addon ever made for C3

    It is unfortunate that all addons need updating. But we already went through this with moving from C2 to C3, where all addons needed rewriting, and then again with moving from the C2 runtime to the C3 runtime, where all addon runtime scripts needed rewriting again. I know this creates a lot of work and I don't like doing it, but I strongly believe it is necessary to avoid compatibility disasters, as the current approach is not sustainable.

    So are those promises not worth anything anymore ?

    All the documented APIs in the addon SDK v1 are supported in the addon SDK v2. So they are still supported, but addon developers need to move to the new SDK to continue using them.

    But this SDK change would be more like "if you can't do this events, and if you can't do this as an addon, then either change your vision for your game, or submit a suggestion and wait X months/years to maybe see it accepted or rejected".

    This is how it works in every other engine, because you generally can't access the internal engine in any other software. It was only really possible in Construct because of JavaScript's weak encapsulation. In other engines, people use workarounds or other approaches. You should do that in Construct too - in fact that's what people should have done in the first place, instead of resorting to accessing internal details of the engine, which causes compatibility disasters.