Handling a lot of data in real-time

I've been wanting to prototype a simulation game, however I'm concerned about limitations of Construct 2's data handling. I'm thinking hundreds of thousands references in real-time, not like character and monster stats - I'm thinking of tracking every in-game object.

Think Dwarf Fortress, only on a somewhat more humble scale.

Array feels somewhat bulky, but I probably could get away with using constants as array references. I'm not sure how much would that impact the performance, however.

Opinions? Experiences? Suggestions?

The target platform for deployment will determine most of your limitations.

Obviously mobile phones aren't suitable targets - not only are they weaker performance wise, there simply isn't enough market for games of this type. And platform is the least of my concerns here, after all we're talking HTML5.

My goal is a simple yet deep simulation game that can be extended at will, within reasonable constraints of course. It would have hundreds, thousands of actors all doing their own things (I'm thinking space simulation game of colony management).

Now, for this I need a high performance database of sorts that can handle thousands of simulateous references. Would the array be sufficient for that task? Or should I consider making a dedicated plugin for this alone, leveraging full Javascript capability?

It kind of depends what you need doing I supose.

Most forms of database require connectivity somewhere.

Then you'll either need to do some smart approach with the updating and retrieval, or have something local.

If you plan on making some database plugin, perhaps look into taffydb.

It is suposed to be a client side database with a simlair structure in commands like mysql database.

And wheter or not arrays are up to the task, I'm quite sure they could be, but not sure it would be the easiest approach.

That... looks interesting. Well, first I see how much I can squeeze out of arrays before I start pulling my hair out, then I'll consider one of those databases.

Thanks!

I remember Arsonide wanting to do something like this on CC. He even made a few plugs to simulate a universe.

To be honest it was a bit much to expect even in CC. Planning events that happen beyond the viewport is just very complicated.

I guess the question is what kind of database do you need beyond an xy locator with a single storage space at that location?

The one where I can easily refer to entities without the need for look them up in the XY space. Plus, a LOT of the space would be empty and I want to avoid that.

Well, guess I could have an array that indexes existing entities and just draw those whose locations fall within viewport. So each feature would have an address and the empty space would be just that - empty, maybe with inherited features from adjacent features (solar radiation/gravity etc.).

This requires careful planning, as in, future optimizations would likely be extruciating.

Well in my experience there are basically two ways to set things up.

First the database way with rows,/colums, or x/y.

Second is a list where information is stored like a stack, in no particular order, other than when it is created.

Both of those can be done with the Array object, but it would suffer from the conversion of strings to numbers.