RTS update #1: architecture

13
Official Construct Team Post
Ashley's avatar
Ashley
  • 26 Sep, 2022
  • 2,136 words
  • ~9-14 mins
  • 2,852 visits
  • 6 favourites

Since my first blog I've been putting together the overall architecture of the game. You can see some of the initial work in the commit log on GitHub. This can be slow going at the start: these initial decisions are very important and can end up very difficult to change later on. In this post I'll cover the overall technical design I'm aiming for.

The game server

For multiplayer games there is generally a server that has the true state of the game, and clients aim to sync with that. The GameServer class will represent the authoritative state of the game. The GameClient class is the local counterpart that aims to sync with the state on GameServer.

Construct's Multiplayer feature works via peer-to-peer connections. This actually avoids the need to run dedicated servers, which helps make it cheap to run! (You can run TURN servers to improve connectivity, but those will only deal with a fraction of the bandwidth, as most connections can still run peer-to-peer.) In a multiplayer game one player will act as the host, and that player will also run GameServer and act as the server for all players.

One interesting design point is that when the host has a local GameServer, it will still run the same code to sync the local GameClient with the GameServer. However the link between the two will have zero latency and unlimited bandwidth, so it should sync perfectly. The main reason for this is it actually saves on coding. If there was some special case to handle being both a server and client at the same time, that means writing a lot of code to handle that mode separately. Since there will already be this whole system to manage a remote client, it can just be re-used in its entirety for a local client. Here's a diagram that shows how it will work with two players.

Two players in a multiplayer game. Each player has a GameClient. The host has GameServer.Two players in a multiplayer game. Each player has a GameClient. The host has GameServer.

In fact this approach extends to single-player games too! Instead of writing a whole special single player mode, there can just be a local GameServer on the same system, and it will run a game for one player and sync perfectly with GameClient (essentially just the "Player 1" box in the diagram above). I'd like to have a single-player mode for this project too if possible, and that can definitely be done. It's an interesting design point that going from a single-player design to a multiplayer design is incredibly hard - often infeasible - but going from a multiplayer design to a single-player design can actually be pretty straightforward. So thinking about the multiplayer design is still the right place to start.

Another benefit of single-player mode working the same as multiplayer is I can simulate latency and packet loss on the messages passed between GameClient and GameServer in single-player mode. This means the resilience of syncing over a poor network can also be tested in single-player mode without actually using networking, which should make testing the multiplayer code easier.

Multithreading

A cool part of this design is we can also use multithreading. In short, GameServer can go in a Web Worker.

Not all game development tools support multithreading, but it's another thing JavaScript has had for years in the form of workers. A worker provides an independent JavaScript context that runs in parallel to everything else. The system can schedule it to run on its own CPU core. It can safely communicate with other JavaScript contexts via message passing (using postMessage()) without any risk of nightmare shared memory concurrency bugs that some "unsafe" languages are subject to.

GameServer will be designed to communicate with GameClients over the network, and this also makes it easy to use a worker: instead of sending messages over a network, it sends messages to and from a worker. That's like a perfect network link with zero latency and unlimited bandwidth.

GameServer may well be running very heavy amounts of game logic to simulate hundreds or even thousands of independent units all interacting in different ways. Using a multithreaded architecture moves all this work to its own thread so it won't affect the performance of the game for the local player, as GameClient is in a different thread and so won't be slowed down even if GameServer is using a full CPU core. That's pretty cool and potentially significantly increases the upper limit of how intensive a game it can run, especially since JavaScript has outstanding performance. It will be interesting to see how far this can be pushed. I will definitely be trying some stress tests later on!

Construct can host its own runtime in a Web Worker, off the main browser thread (aka the DOM). Adding a second worker for GameServer means there are now three threads on the host system: one for the browser, one for the Construct runtime, and one for GameServer. I think this is a relatively sophisticated multithreaded architecture for a hand-coded browser game.

Diagram of three threads on the host system, and which communicate with each other.Diagram of three threads on the host system, and which communicate with each other.

All technology choices are a trade-off though, and there are some downsides. The main one is that by running in an entirely separate JavaScript context, there is no direct access to the Construct runtime. That means it can't rely on Construct for things like collision detection - I will have to code everything in GameServer in pure JavaScript. I don't think there will be too much of that kind of re-implementation though as GameServer doesn't have to handle anything like rendering or user interaction. I guess it might be an interesting exercise in game coding.

Funnily enough that disadvantage also turns in to a potential advantage: if GameServer is entirely self-contained, then it could even be run independently with a tool like Node.js or Deno. That provides a way to host a dedicated server in future, which would solve the problem of the game ending if the host quits, but means somebody needs to pay to run those servers. This also highlights a great strength of JavaScript: it's supported so widely that the same code can be re-used directly in different places such as both servers and clients, rather than needing to rewrite things in another language entirely.

Summary

So overall I would say the advantages of this approach are:

  • It covers both single-player and multiplayer games without needing separate modes for each
  • The same messages can be used for both network messages for multiplayer games and worker messages to talk to a local GameServer
  • Multiplayer code can be tested in single-player mode without actually using networking, by simulating latency and packet loss
  • It should work for both peer-to-peer and dedicated server network architectures, although it'll just be peer-to-peer to start with, as that's cheaper to run.
  • Multithreading avoids janking the host player's game and could significantly enhance overall performance

The disadvantages are:

  • GameServer will have to be coded fully independently, without using Construct for things like collision detection.
  • Overall this is a fairly complicated design. But hey, I signed up for a challenge.

In software, as in most engineering things, there's probably not a perfect solution that ticks all boxes - everything involves tradeoffs. But I like this set of tradeoffs and I think it will work well for this project.

Why not use event blocks?

One of the main questions people asked after my first blog post was basically: why not do this with Construct's visual event blocks? I talked a bit about the goals of the project in the first blog post, but it's worth going in to a bit more detail about them.

  • First of all, it's not possible to use event sheets with multithreading. So it's impossible to do the architecture I just outlined here with just event sheets! I wrote a blog post back in 2015 on Why do events only run on one core? which addresses some of the technical complications.
  • Ever since the introduction of the JavaScript coding feature, Construct has been about both event sheets and coding. We've long been emphasizing that if you do want to learn to program, Construct is a great way to do that: you can start with event sheets, add a few snippets of code in event sheets, and move on to using full coding - and you'll learn a professional programming language along the way. I think the coding part of Construct is under-utilized and is strongly competitive with other more coding-focused tools. I want to help shine a light on the coding side of Construct and prove the point it can be used like a coding-focused tool as well.
  • Related to that is dispelling the myth that Construct is "just a toy". Building a sophisticated fully-coded game that can do things impossible in some other coding-focused tools on the market should help prove that point too. It'll help demonstrate that you don't have to move on to a different tool if you want to start writing serious code.
  • This project is about pushing Construct to its limits, answering the question: what's the most an advanced user can do with Construct? As we would expect the most advanced users to shift from event sheets to coding, developing a full coding project is the best way to answer that question.
  • This project will likely also need a focus on extreme performance to be able to handle potentially thousands of independently interacting units. While event sheets are still highly efficient and can handle many games superbly, when you start to reach this kind of scale, directly coding in a high-performance language like JavaScript helps to go even further in that respect. It also demonstrates what the incredible performance of JavaScript can do beyond the performance limits of other much slower custom languages used in some other tools.
  • Event sheets can integrate with JavaScript, but it would probably be difficult to do that to a large extent as event blocks and JavaScript code are completely different paradigms (for example, the concept of "picking" does not exist in JavaScript). I suspect trying to half-and-half lots of complex code and event sheets could actually make things more complicated than they already are. Using event sheets too much could also put off coders interested in following a coding project, and they're a group I'd like to appeal to with this project.

Most of these goals can't be achieved by focusing on event sheets. That's why I'm going for mostly coding for this project. But who knows - if this works out well, maybe there could be a similar project in future focusing on the event system instead.

It's also worth adding:

  • There is much more to Construct than the event system, and so there's still a lot of value in dogfooding. In fact since starting I've already fixed two small bugs in the Layout View and Project Bar and made two adjustments to the scripting feature. So it is definitely still helping improve Construct for everyone, including both visual and coding users, along the way.
  • We built many of Construct's 280+ built-in example projects, most of which focus on event sheets. That combined with a lot of testing and prototyping means we have used, and do use, the event sheet system a fair bit already. We've already made lots of improvements from that kind of usage and still occasionally do. On the other hand scripting is something we've spent less time using ourselves.
  • Despite all the above, I think this project will likely use event sheets to some extent anyway. In particular menu systems are pretty boring to code and work independently of the main game, so I'll probably just use event sheets for those as they really are the best tool for the job. Maybe there are other areas, such as the game user interface or particular interactions, that will be easy to do with event sheets. I don't know yet though! However I'm sure one way or another this project will involve use of event sheets too.

More to come

I hope that helps explain the goals of this project. It's early days yet and there's a lot more to come. I'm aiming to get to something at least minimally playable as quickly as possible - so far there's been a lot of just laying foundations to build on top of, so there's not much to see in action just yet. But you can see all the code so far on the GitHub repository and I'll keep on blogging about it here! Let's see how long it takes to get a multiplayer game where two people can just move their own units around and see everything updating. Then we'll start building gameplay from there.

Subscribe

Get emailed when there are new posts!

  • 30 Comments

  • Order by
Want to leave a comment? Login or Register an account!
  • It seems to me that people who are negative about writing code in Construct are simply insecure and cowardly. They are afraid that someone will write better and easier than they are used to. They don't even want to know that Construct is capable of more than just events. In fact, Construct has endless coding potential, and I'm glad you feel the same way. I am glad that you are developing the topic of coding! Events can be left to beginners. I am very grateful to you that you added the ability to write code instead of using events, because I learned another programming language and now even more roads are open to me in life. Those people who have never written code simply do not understand how much better it is than composing event blocks. They can't even tell Java from Java Script... And you're doing the right thing by trying to show how cool it is! Many people say that your sales will drop because of this. But in fact, they do not care about them, because they all think only of themse

  • -"you can start with event sheets, add a few snippets of code in event sheets, and move on to using full coding"

    -"as we would expect the most advanced users to shift from event sheets to coding"

    I'm sorry but the promise of construct always has always been to be a nocode engine. That never was to stop using nocode feature ASAP

    In fact after reading this i feel even less confident about it because you also don't recommand to mix JS and eventsheets, but to go full JS instead

    Honestly this is a bit sickening. This looks like the whole argumentation about Events is against Construct itself and its original vision

    We're expecting the actual No-Code features to not be considered as "just a toy". It's a bit weird the only way to prove C3 isn't a toy is skipping the NoCode part. The marketing and dev effort should instead go towards "yes NoCode is viable for serious stuff"

    Also a bunch of us are experienced users and our path never was to move from ES to JS. I didn't learn JS using C3.

      • [-] [+]
      • 1
      • Ashley's avatar
      • Ashley
      • Construct Team Founder
      • 1 points
      • (0 children)

      Since the introduction of JavaScript coding in Construct back in 2019, we've been aiming to make Construct also good for coding, in addition to the event sheet system. We haven't, and aren't, neglecting the event sheet feature. We know lots of people use event sheets. But we'd like to draw some more attention to the coding feature and show what it's capable of. That's what this project is about.

  • Hey Ashley, loved total annihilation myself. Lan parties were the best. I look forward to testing it out and I'm excited for you! I also appreciate you making it available for others to learn from.

    I really hate being critical of you and I don't mean to kill your buzz... but purely in the interests of being constructive I have to agree with some negative feedback in your decision to not use event blocks.

    By my failing memory I thought traditional coding was *mostly* introduced in construct by popular demand, not because of any limitation (from users who felt event blocks slowed them down, as they were already proficient with java).

    When you say "directly coding in a high-performance language like JavaScript helps to go even further...", my understanding was that the events get turned into native code when you export.

    Is there really no way to better optimise how the event blocks are exported?

    • Ran out of room. I'm sorry to be long-winded but I feel like it's important.

      "I think the coding part of Construct is under-utilized and is strongly competitive with other more coding-focused tools".

      I'm sure that is true but the biggest sell of the engine is the event blocks for a crowd that doesn't want to stare at walls of text while building a game. For this, there is really no competition out there. You guys do it the best.

      As far as traditional coding goes... Unity is a fantastic option for those who are happy to go down that road (without the need for a subscription).

      At the end of the day C3 is still my engine of choice. It is powerful enough for my vision and I still love the event blocks. I do think the blog itself is fairly off-putting though. The part about *maybe* using events for the menu's is particularly on the nose for me.

      Again, sorry for the negativity. I only say this for the passion I have for your product.

        • [-] [+]
        • 1
        • Ashley's avatar
        • Ashley
        • Construct Team Founder
        • 1 points
        • (6 children)

        I think I'm just repeating myself here, but the goal of the project is to show Construct can be used for coding, as well as event sheets. I guess it might not be for everyone, but I've set out the goals and gone in to detail about why I've gone in this direction.

        Load more comments (6 replies)
  • Nice message from the main developer of Construct,

    we made little examples with event sheets, but if you want to make a real game the event sheet system isn't enough.

    That sounds veeery very different to the messaging we got for years and years from Scirra when we were the ones making games with Construct.

    Feels like a total backstab.

      • [-] [+]
      • 0
      • Ashley's avatar
      • Ashley
      • Construct Team Founder
      • 0 points
      • *
      • (8 children)

      I think that's a wilful misreading of the post. I'm not saying event sheets aren't important or can't do amazing things. There are plenty of big Construct games already made entirely from events. In this project I want to focus on a different approach.

      • This is exactly the way most of your users are reading this.

        It doesn't matter what you write, actions speak louder than words.

        I complain all the time, but even the people that usually kiss the ground you walk on are voicing their disappointment over this one.

        But it's a wasted of time to argue, you already made your choice.

        Load more comments (7 replies)
  • I really look forward to this project because I feel like I've pushed ES to their limit and some of my game don't run well on mac architecture. I hope with JS they will run better. Anyway, sorry about the backlash you got for this series. As a long term supporter of C3, I personally feel like it will be a very interesting project to follow. Theres some stuff thats just cleaner to do in plain code as well like playing with for loop indexes, just to quote a quick example. Is there a place to start understanding how to connect JS with C3 objects ? I feel like I've missed that tutorial somehow.

  • As a someone who created and released own game on Steam using mostly Sheets and only a little bit of JavaScript I'm really looking forward on this project to learn what is the best way to organize code using Construct and create js-heavy games with it!

  • How could I make GameServer run independently with a tool like Node.js? Can I access multiplayer plugin functions through Node.js?

  • I'm curious if you use C3's own editor when writing your code or do you write externally using VSCode and then reload in C3? (Yeah, I am looking forward to you can bring vscode.dev to C3)

      • [-] [+]
      • 3
      • Ashley's avatar
      • Ashley
      • Construct Team Founder
      • 3 points
      • (0 children)

      I'm using C3's own editor for the time being. I might switch to VS Code later down the line, but C3's own editor is working fine so far.

  • Sounds promising.

    One of my projects used C3 MP to create a server and client. Same game and mostly same code, but the server had some more features.

    It will be really great to see a client-server RTS game from C3. Will it be possible for some users to separate the server code and run it on a server instance?

    I.e. instead of client-host, there will be 2 C3 clients and 1 C3 host that acts as a server and links the 2 clients together.

    • Suggestion: I like event sheets, but I still can't wrap my head around all the available JS APIs. I already know ES, and I don't want to learn C3 API from scratch. Is there a way to show which events/action relate to which API; so you help users make an easier transition (if needed) to JS from an ES background?

      Blender are doing something super great with their script API: when you hover over a UI button in the editor, it will show a description tooltip which also includes the API call corresponding to this button.

      This is actually how I learned their scripting! I made the 3D object I wanted using UI, then hovered over them to know their corresponding API calls, then coded them.

        • [-] [+]
        • 3
        • Ashley's avatar
        • Ashley
        • Construct Team Founder
        • 3 points
        • (0 children)

        Events don't really translate to clean or readable code as they are a completely different paradigm. However in many cases the APIs for specific plugins and behaviors relate closely to corresponding actions and expressions.