I have yet to see these disasters you are talking about.

We see them - desperate customers with ruined projects come and beg us for help. Sometimes the addon developer responsible has left the community, and there's nobody left to help them.

Many addons continue working for years without any issues.

As I said before, these could all permanently break with any release. The main reason it hasn't happened already is sheer luck. I expect our luck will not last forever, especially as other addons can and do break on a fairly regular basis.

nobody blames you when this happens

They sure do. If someone's project works in r300 and breaks in r301, they contact us, tell us we broke their project, and whatever we did in r301 we should revert so their project keeps working. Trying to explain these issues of encapsulation to people is often fruitless as most customers don't care and often it sounds to them like we're just pointing the finger at someone else and shirking responsibility. It's a real nightmare for us to deal with and is continually happening in the background, unbeknown to most addon developers, often including the addon developers responsible for the problem.

To the others who have responded: I do appreciate your feedback; I am keenly aware of the value addon developers bring to the community; I greatly regret that we are adding more work for them to do. However an API with encapsulation is the industry standard. No other serious tools allow addons such unfettered access to engine internals, precisely because it is an unprofessional and reckless way to develop software that most people in the industry understand will end in disaster. Part of the project of the Addon SDK v2 is to increase the existing API surface to make sure it's more capable and do so in a way that is reliable in the long-term. Where possible, we are also making some addon's features built-in, which also removes the need for the addon to be ported. For example the next release includes a 'Reset' action for event variables, so that it is no longer necessary to use an addon to do that. We will be continuing this to make the best effort possible to smooth the transition and make sure any key features are implemented in Construct itself.

I know we're asking a lot here. But if we don't act, in extremis this could turn in to an existential problem for Construct, ending in development hell where every update breaks loads of projects and we are in effect no longer able to improve Construct any further; ultimately that can lead to the failure of the product on the market. This has happened to other products before and, as things stand, could easily happen to us in future. I am doing my best to explain that this is a real, active risk, there are signs of it happening already, it's likely to get worse, and it warrants serious action to rectify. I am asking addon developers and customers alike to make a good faith effort to take seriously the risks we are facing, and I am asking for others to co-operate as we go through what will unfortunately be a disruptive but necessary process. We have to think on a 5-10 year time scale, and we need to go through this to be in a better place in future.

Why are you trying to convince or gaslight everyone into believing that the developers are too lazy to update the plugins?

I'm afraid I genuinely have no idea where you got the impression I thought addon developers are lazy. I have been talking about how I am keenly aware of just how much work this is going to create and how disruptive it is going to be. We are asking addon developers to co-operate with us through the transition and I am writing a lot on the forums here to explain the reasons for this, as it is fair that we should have to justify why we are doing something so disruptive.

WackyToaster - regarding the "group handler" addon: I'm afraid that is the very reason we had the warning in the SDK. It could break at any time. If it's worked for several years already, that is absolutely no guarantee at all it will continue working even one more week - unless we have a proper API with encapsulation. We could rename some variables or fix a bug in the engine, and your addon will be broken. It could break repeatedly, every few weeks, for a year or more, and then ultimately break permanently, causing untold disruption along the way. This is already happening with other addons and will continue happening. Given that in the long run pretty much all parts of the engine are eventually changed or upgraded, all such addons accessing internal details are likely to eventually break. Part of the point of this project is rather than sit around and wait for disaster to eventually strike for all these addons at random and unpredictable times, we're managing the process up-front with a published timetable for what action will be taken so everyone can plan ahead and the problem can be solved once and for all.

By ignoring the warning in the SDK, I am afraid you must carry some of the responsibility. We did say undocumented features could be removed at any time. So we reserve the right to remove all those features, break your addon permanently, and say "tough luck". Then all the customer's projects using the addon are broken and it was because you ignored the warning in the SDK. However, we are not doing that: instead we are laying out a timetable and doing what we can to try to make the transition smoother. If people really need your addon and removing it is infeasible, they'll have to stay on old releases (possibly a future LTS release) until they finish their projects; if it really is going to cause enormous disruption, we might add necessary features to the Addon SDK v2 to allow you to port your addon, depending on a judgement of the situation, feasability of maintaining the APIs, etc. However, at the end of the day, we reserved the right to permanently remove all those features, and we did warn you of that.

The industry came up with encapsulation decades ago precisely to solve this problem. Experienced developers know that if outside developers get access to internal details, of course it ends in disaster, of course it makes a huge mess for everyone. All other tools use encapsulation for an addon system to avoid this. As I've explained, the addon SDK lacked encapsulation mainly for historical reasons as JavaScript had poor support for encapsulation in the past. Either you learn the lessons of history and respect encapsulation, or you disregard them, and you learn the hard way as it invariably ends in disaster.

Given all this, I have to say, in my view it was unwise to develop such addons. Adding a few small handy features via accessing internal details, at risk of causing disaster and potentially ruining everyone's projects in the long run, is just not worth it. The alternative may be some more clunky event sheets and perhaps a slightly less smooth workflows, but people's projects keep working indefinitely. In the long run that's the better outcome. In my view, fully understanding the reasons encapsulation exists and the severity of the risks of bypassing it, I would not even consider trying to bypass encapsulation, under pretty much any circumstances at all short of a genuine emergency where not acting is also sure to cause a disaster. Anything else at all, even significant inconvenience, is better. The extent of the disaster it risks must be avoided at all costs. This is why, despite all the disruption, we believe it is absolutely essential that we move to a new addon SDK with much stronger encapsulation. Either we have planned disruption now, or we have never-ending continual unexpected disasters of increasing severity for years to come.

I am also aware many addons are well-behaved and have never accessed any internal details. I do regret that these addons still need updating to the new SDK. I hope that by explaining the situation in this much detail, I am highlighting how important this work is and why it's for the best in the long term.

As guidance I would be very interested to hear what some of your top 2-3 3rd party addons are - that you think substantially help the C3 community and Scirra?

The addon SDK is best at integrating additional platform features or third-party services. For example third-party addons that provide support for ad networks (of which there seem to be dozens), or back-end services like authentication, high-scores and analytics with various third-party providers, or enhanced platform services such as advanced IAP or other monetization features (perhaps through Cordova plugins, which I think the addon SDK has reasonably good support for), or integrating with third-party platforms like itch.io and Newgrounds, are all good examples of the kind of thing the addon SDK was designed for. All the possible integration work with different services and platforms is a huge amount of work which is infeasible for us to complete alone, and if we do it anyway it takes development time away from core engine features that only we can do. Allowing third-party addons means all that integration can happen without needing us being involved. None of that needs addons to access the internal engine, and that is partly why the originally documented API was quite thin.

It is indeed regrettable that these kinds of addons which comply with the documented API will still need updating to the Addon SDK v2. However assuming the addon doesn't have a lot of engine code, updating them should hopefully be pretty straightforward, and I think in some cases will amount to renaming a set of methods and variables. I am still keenly aware of the extra workload this puts on addon developers, but as I've said, I do believe this is essential for the future reliability of Construct, and we wouldn't be doing it if we didn't think it was absolutely necessary.

The addon SDK was never designed to be able to manipulate the internals of the engine, and as I've said before, no other professional engine that has an addon system allows that either, because everyone uses encapsulation, because everyone understands that the alternative ends in disaster. I recognize that some addon developers do want to be able to do more advanced things with the engine; the best approach for that is a more comprehensive and documented API which is supported long-term. The Addon SDK v2 moves to using the existing scripting APIs which you can explore in the scripting reference. Hopefully it is clear this is already a huge leap in the documented API surface. As the original post here explains one of the benefits of the new architecture is new APIs are automatically available to both the scripting feature and the addon SDK, meaning we can get more done with less work, and improve both faster. So while I fully acknowledge the transition is going to be disruptive and painful, in the long run, I am sure it will result in a much better and far more reliable addon system.

One more thing. The primitives you need to write a behavior are setting the position and testing overlap. The "push out" algorithms are combinations of "move a bit and test position".

Here's the code for a simple "push out" method:

Here's a sample project file using it.

That's 19 lines of code. It works to 1px precision (like most of the internal methods). Want sub-pixel precision? That's not easy with the internal methods, but if it's your code you can tweak it to your heart's content. Want even more advanced collision handling? It's your code, you can customize it. And this will be supported forever - there's no internal engine access and no risk of disaster.

As an aside: why aren't the internal 'push out' APIs already exposed? Well, we have learned a lot about writing collision handling code over the past decade or more. If we were to start over I could probably write those methods a lot better with an improved API. However backwards compatibility concerns and time constraints mean they're just left as-is for the time being - slightly sub-par in their overall design, but they do the job OK; for us it's not a problem as if it's internal code and we decide it's a problem, in theory we can just go and replace everything and update our internal references. If we expose them as-is in the JavaScript API, then we basically get locked in with the slightly sub-par design, and it becomes far more difficult to upgrade in future. So really to expose them means redesigning a set of collision APIs - a much more significant job. And even then, you can't customize the internals of the engine easily. What if you want something like sub-pixel precision or some other more advanced handling? Well, if you start by writing 19 lines of code, you have the most advanced solution available: a completely customizable implementation that you can hack around with as much as you like and there is no risk of compatibility disaster with projects. It's even more flexible than adding an API for it. The community could share libraries with useful implementations, and customize those libraries if they need to, and JavaScript Modules are extremely good for code re-use.

I think part of the problem is once people figure out they can reach into the internal details of the engine, then people end up solving every problem they have doing that - risking a compatibility disaster every time - even when there are straightforward better options around. For example this suggestion appears to involve code reaching in to the internal engine and risking compatibility disaster when a static method could have sufficed and would have worked robustly indefinitely. If we use encapsulation to close off the access to the engine, then I suspect in many cases people will revert to already-viable workarounds - which they could and should have used in the first place - and in future people will start to learn the better solutions that don't involve reaching in to the engine internals all the time.