Ashley's Recent Forum Activity

    Construct 2 never had such issues. A plugin from a decade ago still works even now. My projects from a decade ago in Construct 2 still work even now.

    Oh man. What a thing to say. As I said before, I pretty much have trauma from how badly the addon system worked out with Construct 2. It was even worse. Just because you can point to a few addons that kept working, doesn't mean that the overall design wasn't a huge mess that worked out disastrously for many people.

    He doesn't seem to understand the magnitude of the problem he's creating.

    As I've said before, I am keenly aware of the amount of work this will create, and that is regrettable. However, please understand that we are not doing this for fun: the current compatibility disasters and future risks are in my view so severe, that this is very much necessary, despite the magnitude of the problem it will create in the short to medium term. If you don't believe me or disagree with that judgement, I'm there's not much more I can add, other than to say that you either learn the lessons of history, or you are doomed to repeat them. The industry came up with encapsulation to solve these kinds of problems, and we are moving towards that industry-standard approach. If we postponed this by a few years and then were forced to do it anyway, it would be even more difficult and even more addons would need updating. So if it's going to happen - and in my view it absolutely has to - then the sooner we act the less bad it will be.

    no more behaviors, no more plugins, no more effects, that do different things,

    No, these are all still possible with the Addon SDK v2. In fact there are no changes at all for effects.

    so i guess.. you'll add big API in SDKv2 to allow people to create new types of behaviors / plugins

    Yes, that's the goal.

    But service integrations (like firebase/ads provider) require a very active maintaining

    If a third-party service makes a breaking change, it's nothing to do with our SDK. This is an entirely different area of compatibility and not related to the migration to the Addon SDK v2.

    open source ... open source ... open source

    I'm repeating myself here, and there are only so many times I will repeat myself before I deem it no longer helpful to respond. Here we are talking about the long-term maintainability of installable addons, not whether you can download a copy of the engine source code. Any open-source project with installable addons will also have to consider long-term maintenance of the addons that are created, and the professional, industry-standard way to do that is encapsulation, which is what the Addon SDK v2 does.

    As guidance I would be very interested to hear what some of your top 2-3 3rd party addons are - that you think substantially help the C3 community and Scirra?

    The addon SDK is best at integrating additional platform features or third-party services. For example third-party addons that provide support for ad networks (of which there seem to be dozens), or back-end services like authentication, high-scores and analytics with various third-party providers, or enhanced platform services such as advanced IAP or other monetization features (perhaps through Cordova plugins, which I think the addon SDK has reasonably good support for), or integrating with third-party platforms like itch.io and Newgrounds, are all good examples of the kind of thing the addon SDK was designed for. All the possible integration work with different services and platforms is a huge amount of work which is infeasible for us to complete alone, and if we do it anyway it takes development time away from core engine features that only we can do. Allowing third-party addons means all that integration can happen without needing us being involved. None of that needs addons to access the internal engine, and that is partly why the originally documented API was quite thin.

    It is indeed regrettable that these kinds of addons which comply with the documented API will still need updating to the Addon SDK v2. However assuming the addon doesn't have a lot of engine code, updating them should hopefully be pretty straightforward, and I think in some cases will amount to renaming a set of methods and variables. I am still keenly aware of the extra workload this puts on addon developers, but as I've said, I do believe this is essential for the future reliability of Construct, and we wouldn't be doing it if we didn't think it was absolutely necessary.

    The addon SDK was never designed to be able to manipulate the internals of the engine, and as I've said before, no other professional engine that has an addon system allows that either, because everyone uses encapsulation, because everyone understands that the alternative ends in disaster. I recognize that some addon developers do want to be able to do more advanced things with the engine; the best approach for that is a more comprehensive and documented API which is supported long-term. The Addon SDK v2 moves to using the existing scripting APIs which you can explore in the scripting reference. Hopefully it is clear this is already a huge leap in the documented API surface. As the original post here explains one of the benefits of the new architecture is new APIs are automatically available to both the scripting feature and the addon SDK, meaning we can get more done with less work, and improve both faster. So while I fully acknowledge the transition is going to be disruptive and painful, in the long run, I am sure it will result in a much better and far more reliable addon system.

  • There is no reliable way to detect if the user is definitely online or definitely offline. They could have an intermittent connection, or the status could change just after you check it. The only good way to do this is: don't check for connectivity, just attempt network operations, and handle them if they fail.

    One more thing. The primitives you need to write a behavior are setting the position and testing overlap. The "push out" algorithms are combinations of "move a bit and test position".

    Here's the code for a simple "push out" method:

    Here's a sample project file using it.

    That's 19 lines of code. It works to 1px precision (like most of the internal methods). Want sub-pixel precision? That's not easy with the internal methods, but if it's your code you can tweak it to your heart's content. Want even more advanced collision handling? It's your code, you can customize it. And this will be supported forever - there's no internal engine access and no risk of disaster.

    As an aside: why aren't the internal 'push out' APIs already exposed? Well, we have learned a lot about writing collision handling code over the past decade or more. If we were to start over I could probably write those methods a lot better with an improved API. However backwards compatibility concerns and time constraints mean they're just left as-is for the time being - slightly sub-par in their overall design, but they do the job OK; for us it's not a problem as if it's internal code and we decide it's a problem, in theory we can just go and replace everything and update our internal references. If we expose them as-is in the JavaScript API, then we basically get locked in with the slightly sub-par design, and it becomes far more difficult to upgrade in future. So really to expose them means redesigning a set of collision APIs - a much more significant job. And even then, you can't customize the internals of the engine easily. What if you want something like sub-pixel precision or some other more advanced handling? Well, if you start by writing 19 lines of code, you have the most advanced solution available: a completely customizable implementation that you can hack around with as much as you like and there is no risk of compatibility disaster with projects. It's even more flexible than adding an API for it. The community could share libraries with useful implementations, and customize those libraries if they need to, and JavaScript Modules are extremely good for code re-use.

    I think part of the problem is once people figure out they can reach into the internal details of the engine, then people end up solving every problem they have doing that - risking a compatibility disaster every time - even when there are straightforward better options around. For example this suggestion appears to involve code reaching in to the internal engine and risking compatibility disaster when a static method could have sufficed and would have worked robustly indefinitely. If we use encapsulation to close off the access to the engine, then I suspect in many cases people will revert to already-viable workarounds - which they could and should have used in the first place - and in future people will start to learn the better solutions that don't involve reaching in to the engine internals all the time.

    If you need multiplayer or online features - you HAVE to use Playfab, Photon, Firebase etc. Making mobile game - you NEED monetization addons. Working with a publisher - you NEED analytics addons.

    You don't need to access the internal details of the engine to write addons that access third-party services like multiplayer and analytics, or platform features like monetization. All those kinds of addons can still be made with the Addon SDK v2. We are not removing support for making addons entirely!

    Please remember the goal here: it's to stop customer projects getting ruined by updates. That is an incredibly important thing to do on behalf of customers. I know this creates a lot of work and it's going to be disruptive. Like I said, we're not doing this for fun. There is a serious problem and it needs solving. Unfortunately without proper encapsulation, the problem is not going to be solved. This is well understood in the software industry and the reason encapsulation was invented. So to solve this problem, I believe we have no choice: we have to do this, and the longer we leave it the worse it'll be. Leaving customer projects getting ruined on a regular basis is simply unacceptable for a product like Construct; the status quo cannot continue, and I feel that to allow it to continue when we know it can be prevented would amount to negligence on our part.

    I agree that the lack of a proper encapsulated API to begin with was a serious mistake. However in the past JavaScript had pretty much no support at all for encapsulation, so there was no way to protect the API. This is partly the cause of the problem: historically there was no way to prevent this problem, and so we couldn't actually have designed the API this way from the very start. However I knew that it was a huge risk to not have encapsulation, hence the warning in the addon SDK documentation saying "don't access internal features". Unfortunately that is no match for proper encapsulation and it didn't prevent disasters from occurring. Now with more modern JavaScript features it's more feasible to do, and as the SDK warning has not prevented a very serious problem from occurring, and we now have the option of using proper encapsulation, we feel obliged to do that to prevent customer's projects from being ruined.

    Other engines also have a much more extensive feature set and API (Unreal, Unity, Godot) for both general scripting and plugins, if C3 was at a similar level for SDK V2, it would be more acceptable

    Our goal over the next year is to increase the capabilities and APIs available for the addon SDK v2 so that you can do more advanced things with it, and try to make sure where feasible, all popular v1 addons can be ported to v2.

    So you're well aware the encapsulation does nothing?

    In most programming languages, encapsulation is not perfect. For example C++ has encapsulation, but you can read random memory address and access things you're not meant to. Any seasoned C++ developer knows that's unmaintainable and will end in disaster. The goal of encapsulation is just to make it as hard as possible to access internal details to discourage it as much as possible. In the same way, our encapsulation measures in the Addon SDK v2 are designed to make it as hard as possible to access the internal engine details. It doesn't make it impossible. If addon developers try to bypass it, then we will just create the same problem all over again - and we will have to take steps to mitigate that, to prevent the ultimate problem of customer projects being ruined. Once again, I am reduced to a warning, even though that has proven to be weak: if you find a way to bypass encapsulation, don't do it! It'll end in disaster, and we'll end up in a situation like this again. Hence my attempt at conveying a warning in the strongest way I can.

    I'm really upset with this decision, which only confirms scirra doesn't care about their community

    Do you think caring about our community means allowing customer projects to continue to get ruined over the next few years, and possibly risking some huge disaster, when we know we can take action to prevent that?

    Can you at least help devs port to v2 as easily as possible? Please help them with their requests.

    We do intend to do this, with SDK samples, comprehensive documentation, an upgrade guide, and so on. Over the next year or more we plan to be continually providing support, SDK improvements, and any other improvements we can to make it easier to move to the new addon SDK. Part of this project is being proactive over the next year to make sure the transition can go as smoothly as possible. We can't do everything right now, as we're only just starting the process and it will depend on feedback from addon developers, but we will be actively working on this in the long term.

    Construct's compatibility goes right the way back to Construct 2 in 2011. That's 13 years and counting of backwards compatibility (for the engine - I know that doesn't apply to addons). We have to take the long view and think: where do we want to be in 5 or 10 years?

    Do we want to have years of rolling compatibility disasters, lots of customer projects ruined, maybe some huge disaster happens, and then we have to go through this process of moving to a new SDK but with another 5-10 year's worth of existing addons created which will make it even more painful and difficult than it is now?

    Or do we bite the bullet and do the upgrade sooner rather than later, and then we'll be in a better place in 5 to 10 years from now?

    I know it's going to be painful, but in the long run, the latter is the better option. We have been through this process before, with all addons needing upgrading for moving from C2 to C3, and all addons needing updating for moving from the C2 runtime to the C3 runtime. We got through that - it's painful, some addons get updated, some don't, some people find workarounds or alternatives, some people stay on old versions of Construct to finish their projects then update for their new project, but in the long run, we got through it and it was OK in the end. I'm sure we'll get there in the end in this case too. As I said, I strongly believe this is absolutely necessary to ensure customer projects keep working in the future beyond this, and it aligns us with how all other software in the industry works. If you move to another tool with an addon system, you'll find it works like the Addon SDK v2, because encapsulation is the industry standard.

  • Try Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Try Now Construct 3 users don't see these ads

    Only plugins and behaviors need to be updated to the addon SDK v2. There are no changes for effects or themes.

    We had a warning in the SDK documentation saying "don't use undocumented features, they can change, break or be removed at any time". Addon developers largely ignored that warning, and hence the difficult situation we're in. People ignore warnings. We're not going to try relying on warnings again given it already failed so spectacularly.

    You forgot to answer one big point, why isn't the warning for addons made with SDK 1 enough ?

    Because unless we remove it, people will just carry on using it, and the disasters will still happen.

    Why wait for a hypothetical disaster later when we can enjoy a catastrophe right now

    It's not hypothetical. Disasters have happened, are happening, and will continue to get worse.

    For post length reasons I'll try and answer a few other questions here:

    Will Scirra provide assistance to help with porting from V1 to V2, besides the very simple SDK examples?

    There will definitely be a "porting v1 addons to v2" guide in the documentation. I've also already created commits on the GitHub repository showing diffs so you can see what was changed to go from v1 to v2.

    Is there a possibility to make a 'wrapper' around V2 SDK to support previous V1 SDK addons or at least for the vast majority of the V1 SDK?

    For simpler addons, porting mainly involves renaming things (e.g. see here). Perhaps some tool could be made to convert the basic template, but I think the APIs are too different to allow for full automatic conversion of arbitrary code.

    My hope is that is because the scripting interfaces for addons will be 'automatically' generated somehow.

    The addon SDK v2 classes are the script interfaces (they actually derive from the script interface classes), so there is no extra work to add a script interface.

    Construct has built in vectors, and a built in color class that handles all the math and conversion, yet if an addon needs these, your force to rewrite all that code?

    Anything we expose to addon developers must be supported permanently and with backwards compatibility, otherwise we end up with the same problem of customer projects being ruined. So we can't say 'here, have access to this library of vector/math/utility code' - we have to design a public API that will be supported forever. That's a lot of work and a significant long-term maintenance consideration, and can take away our flexibility to do something like an engine optimization by changing the library code. Instead the approach we have taken with the script APIs is to use the browser built-in features such as DOMRect instead of a custom rectangle class. Want more? Import a JavaScript library. Problem solved, and no risk of compatibility disaster.

    The solutions are far from ideal, sure, but with addon SDK v2, the alternatives become far worse, and IMO, would lead to way bigger issues down the line.

    The downside of accessing engine internals is ruining customer's projects. This can be so bad, in the long run it is in my view significantly worse than any of the other options. I guess you haven't had to deal with desperate customers begging you to fix their ruined project and been unable to help? We definitely have. In my professional opinion: literally any other solution is better than modifying the engine internals. Hacking the engine internals carries such a severe risk of disaster that it must be ruled out, which is why the software industry invented encapsulation and has been using it widely for decades. In other tools with a proper API with encapsulation, people get by with workarounds or alternatives, and while that may be painful and difficult sometimes, it's a better option.

    Part of this project is over the next year or so to make sure the v2 addon SDK has sufficient APIs that it can cover necessary features, or just add new built-in features instead so there doesn't need to be an addon. We do want to minimize the disruption and make sure there is as smooth a transition as can be achieved. However, please, please don't try to bypass the encapsulation, or start copy-pasting chunks of the engine. Please, please, please please please please don't do that. I pretty much have trauma from how utterly awful the consequences of that were with C2. It is just a total, epic disaster, for us and our customers. Please please please please please don't do that. And it is the risk of that kind of epic disaster we are trying to put an end to with the addon SDK v2.

Ashley's avatar

Ashley

Early Adopter

Member since 21 May, 2007

Twitter
Ashley has 1,423,956 followers

Connect with Ashley

Trophy Case

  • Jupiter Mission Supports Gordon's mission to Jupiter
  • Forum Contributor Made 100 posts in the forums
  • Forum Patron Made 500 posts in the forums
  • Forum Hero Made 1,000 posts in the forums
  • Forum Wizard Made 5,000 posts in the forums
  • Forum Unicorn Made 10,000 posts in the forums
  • Forum Mega Brain Made 20,000 posts in the forums
  • x108
    Coach One of your tutorials has over 1,000 readers
  • x62
    Educator One of your tutorials has over 10,000 readers
  • x3
    Teacher One of your tutorials has over 100,000 readers
  • Sensei One of your tutorials has over 1,000,000 readers
  • Regular Visitor Visited Construct.net 7 days in a row
  • Steady Visitor Visited Construct.net 30 days in a row
  • RTFM Read the fabulous manual
  • x35
    Great Comment One of your comments gets 3 upvotes
  • Email Verified

Progress

32/44
How to earn trophies

Blogs