-
Torque3D
The pinnacle raw Torque power, Torque3D has been used for everything from driving simulators to MMOs!
Check Out Torque3DTorque2D
All the power of Torque with less of those pesky dimensions to tie you down!
Check Out Torque2DForums
Looking for answers, wanna see some cool projects or talk about new / upcoming features?
Read the Forums -
Work Blog Updates
-
By JeffR in Torque 3DGreetings!
It’s that time again, where I summarize up a bunch of work that’s been done and get the details laid out in a reasonably digestible way for you all to enjoy! Without further ado, here we goooo.
What’s Been Done So Far?
Since the last Workblog, or werkblerg, or wrgglglbnblglb depending on how much coffee you’ve had yet today, we’ve rolled in almost 160 PRs.
Not quite as monstrous a list as last time, but it’s still pretty substantial.
A good number of them were things on the TODOs from the last blog, so they may sound familiar, but a lot of them were bugfixes or QoL improvements that came up with feedback, reports, or just running into stuff when actually using the engine to make games.
So, what do we have?
General Fixes and Improvements
This is where the vast bulk of the PRs were. Which makes sense, since I mentioned before that a lot of them were fixes and improvements(neat how that works out).
So rather than plonking down a bullet point list of all 75 PRs, let me just pick through some of the highlights:
Memory Leaks
As with any piece of software, there is always a possibility or looming spectre of memory leaks. And while Torque’s had it’s fair share we’ve always made it a priority to chase them down.
What we ended up spotting is that while there’d been prior cases of fairly obvious memory leaks that had been easy to fix, it was noticed that there was a much slower yet undeniable memory leak that had been an issue for some time.
Letting a game run for an hour or two yielded gigabytes of memory consumption, so, you know, obviously that was bad.
It took a lot of work and Az and Marauder doing some deep digging, but through multiple tweaks and fixes and making it easier to track and chase down memory leaks with better integrated reports with any and every bit of memory the engine allocates and releases, we got everything playing significantly more memory stable
Especially important for anyone trying to do long running games or servers.
Path and Vehicle/Rigid Object tweaks
A mix of small but important fixes across the board for these classes, such as fixing stabilizer behavior for hover vehicles when going over ground planes, shoring up some performance tweaks for rigid shapes(which also impacts vehicles), optimizations to networking of rigids and early outing on things like if an impulse is too small to mathematically matter, etc.
For paths, there was tweaks to how the object snapping works(so objects trying to ride on a pathshape can still do normal things like jump) more reliably, as well as adjustments to workflow with them so they’re easier to use, like more consistent load behavior - especially when generating paths with gamelogic - and being able to copy/paste paths and shapes and having them behave more predictably.
Networking Tweaks
A few classes such as TSStatic got some tweaks to improve networking/ghosting efficiency. Nothing major this side, but it’s always good to refine how much we have to pipe across the network to save packet space for actual important gameplay stuff
Subscene Stuff
Az improved the logic for testing if a subscene could be triggered to load(since sometimes you’d want to condition a subscene loading based on gameplay logic or triggesr) to be more consistent/reliable.
Similarly, there was just a bunch of small other tweaks like improving the workflow when creating a level asset and being able to flag it as a subscene immediately, having better in-place workflows for creating subscenes while working on a level, fixed an issue where MeshRoads would save/load into subscenes weirdly, etc.
Nothing Earth-shattering, but it should just generally make subscenes easier to work with.
Spawning
Building off the work with the spawn chain mentioned in the prior blog, we did some cleanups and tweaks to how the callbacks and error reporting if something goes wrong work based on more people testing it and giving feedback.
This means that it should be easier to drop stuff in and have it Just Work, as well as modules being easier and more reliable in how and when they’ll spawn stuff.
Scripting
Re-added the ability to kick off compileAll in order to forcefully compile everything to a dso. This is important because one of the features still on the TODO for the Project Manager is the ability to export a game build, which would include functionality like being able to compile scripts to binary files, or remove art source files and so on.
Also added some unit tests for scripts as part of the build pipeline to better catch any regressions in the future next time we work on the script/console back-end.
There was also some pruning done of old/redundant methods in a few spots which should help keep things clearer.
Beyond that, shifted a bunch of object fields to utilize validators and range types to keep sane values on things. No more accidentally setting your gravity multiplier to a hundred billion and then wondering why the player can’t move!
Jokes aside, it should generally just make it harder to have values go completely insane, as well as generally being better at straight-up plugging bad data into things and causing problems.
Nothing major front-end wise, but should generally help stability overall.
Recast overhaul
What originally started as a “Oh, Navmesh preview renders funky-like on OpenGL” turned into a bit of a quagmire due to how NavMesh used to handle packing and presenting debugging data.
Mar did a big overhaul to better use the library ‘as it was intended’ as well as packing in a bunch of additional utility features to help make working with it a little easier now, and opening up the opinion for new features in the future.
Plus, it’s always nice to be able to tell an AI to go a place and he actually manages to get there!
GUI
Gui stuff saw a smaller but important set of changes and fixes as well.
The list of PRs for that includes:
Fix escape menu keybind not working (Areloch). fix PopupMenu::checkItem filter (Azaezel). Gametsctrl mouserefactor (Olathuss). fix another potential crash with firstresponder code (Azaezel). Enables onMouseMove for scripting on GameTSCtrl (Olathuss). Better guiClockCtrl pause tracking and module callback integration with PlayGUI (Azaezel). expand playgui callbacks to let game modes inject element layers (Azaezel). game mode filterfix (Azaezel). setFirstResponder safeties (Azaezel). guiClockHud augs (Azaezel). Adjust OpenAL device listing query for reliability (Areloch). Close Menu Implemented (Olathuss). Nothing absolutely earth shattering, but a lot of fixes and quality of life improvements. Stuff like the the fixes for the mouse handling for GameTSCtrl improves consistency, and the expanded callbacks and game mode filtering makes working with and interoping with gamemodes a lot easier.
Being able to just drop stuff in and run with it in a more modular way goes a long way to improve iteration time, especially as we keep expanding the list of modules available for people to snag and use.
Editor
The big thrust of focus for 4.1 at it’s core was work on the editor suite, so predictably there was a lot done here.
The highlights list includes:
Material Editor Update/Inspector-ify
The big one would be converting up the special-snowflake ad-hoc Material Editor, which looked like this:
And shifting it to utilize the regular inspector class, so now it looks like this:
Admittedly, not a gigantic space-magic powered visual change, but the backend code is much simpler and streamlined, it’s *standardized* to everything else that already uses the inspector like the normal world editor, and it means that if you add a new field or property to the material class, it Just Works and will appear in the MatEd for changing without having to hand-add it yourself.
It also was an opportunity to pop the MaterialEditor out from being packed in with the main EditorGUI, so you can now edit the MaterialEditor with the GuiEditor.
The other editors will follow suit from this as we go now that it’s shown that yeah, it’s pretty easy to do.
In addition to the MatEd getting updated, the Cubemap editor was shifted over to it’s own standalone thing instead of piggybacking off the MatEd, and now there’s actually a full-fledged Composite Image Editor!
Which basically lets you easily take a material with Ambient Occlusion, Roughness and Metalness maps, and quickly bake it down into a ORM Composite map for efficient use.
Or technically any kind of composite texture, but that’s the main usage.
You can also open it up via the right-click context menu in the Asset Browser on ImageAssets, so either direction works!
Adding tab cycling between inspector sub-elements
This is just a nice quality of life thing, but it means you can quickly hit tab to run through sub elements in the inspectors which should improve workflow speed
Visibility Modes dropdown is an Extensible Menu
This is likewise a QoL thing and ease of maintenance more than anything, but it makes it easy for expanding via stuff added in modules.
For example, there’s the normal list:
And if you’re using the Action-Adventure Kit module, it has some additional visibility modes:
As you can see, we've injected in an entry for Showing Camera/Player blockers which are additional special volumes added with the AAK.
This makes expanding Viz stuff for modules or custom tooling way easier and thus making it much more actually useful.
Ability to make GameModes or New MaterialAssets in Asset Browser
This rectified an issue where neither of these types could be just made brand new via the AB itself, so now you can just make a new one like so:
And it just creates it as expected for any other type and you can get cracking at the actual content your game needs without disrupting your flow or needing to leave the editor to make them by hand in script.
Prototyping UI moved to tools
This is mostly just a standardization/cleanliness thing, but the Prototyping UI Az drafted up has it’s proper home in tools now.
If you haven’t played with it yet, it’s a handy little shindig that lets you very quickly slap together objects and script functions(hence it’s name!)
Additional Terrain Tools
Az put in a little bit of additional elbow grease to finish out some work to add some nifty new brush modes to the terrain editor to facilitate stuff like thermal and erosion, as well as the ability to copy and paste sections:
Color Picker Refactor
Marauder overhauled the color picker to rectify a lot of weird under the hood oddities and now everything should behave far more consistently.
Mind, that isn’t everything, and all told there were almost 30 PRs just for editor improvements, additions and fixes.
Rendering
Rendering saw a very similar number of PRs that largely targeted stability improvements. From fixing memory leaks to erroneous rendering behavior on the GL side of things, fixing wetness, mipmap handling, adjustments to PBR rendering to fit more correctly to model and some IBL/Probe issues.
It was a lot of small but very important improvements.
After all, no one wants their game to look like this:
Platform/System And Libraries
This largely was a small but straightforward list of changes, by which I mean it pretty much was mostly updating various libraries.
Stuff like SDL, tinyXML2, OpenAL, etc.
There was also improvements for logic to find said libraries on linux so our friends on that side of the OS divide don’t have to fret as much if it’ll get confused about which version of a library like SDL it should use.
Also, also, there was some minor cleanliness changes like moving the libCurl stuff to resources as a module rather than trying to have that packed into the main engine build if most users aren’t likely to utilize it.
Assets
On the asset side, the main focus was retooling how Image and Shape assets handle their memory and references, as well as their loading and reloading logic.
The end goal is to cut down on how much redundant handles we have everywhere, while being able to reliably manage and detect when Assets are loaded, pending, changed or reloading so actual game classes, like Players or TSStatics can react appropriately.
This lets us do stuff like properly having Shapebase-based classes react when their shape is changed for hot reloading, or when an image is changed and needs to be reloaded and making the gui classes react appropriately.
Beyond that, more cleanup and safeties in a bunch of spots to make the whole thing more robust.
So, What’s Left?
For the big todos:
Finishing converting remaining ‘Special-Snowflake Editors’ to standard inspector-based stuff.
It’s largely self-explanatory, but a few remaining editor UIs like the ParticleEditor utilize completely custom, ad-hoc GUis for their editors and standardizing those over to use inspectors like most everything else will make everything feel more consistent, be easier to maintain, and be easier to expand.
Rework IBL/Probe memory management
While the current setup works fine, it’s more that it works ‘fine’.
Az discovered via his Warren test map that when you really start throwing a lot of probes into a mission you can pretty well strangle out the render API’s memory allocations in certain circumstances.
The current method is ‘fine’ if you have a fairly small map, but once you start getting bigger and/or you need a lot of probes to wrangle lighting/reflections for indoor areas, you can brush up to that limit quickly.
And it turns out GPUs are not a fan of having their VRAM mugged for spare bytes 😛
So, I gotta rework how the internal arrays handle things.
The frontend shouldn’t change at all, but the inner rendering logic’ll change to properly handle sublists of probes to allocate and deallocate as demand changes.
Only actually allocating cubemaps for probes close to the camera will fix the issue of running into those memory allocation limits without meaningfully changing how the end user sees the IBL rendering happen.
More Misc Fixes and Improvements
We keep getting more and more feedback and bug reports as people use and beat up on the build, which is awesome and lets us really grind down on remaining issues that get in the way of usage.
Stuff like funkiness with the Options Menu, to jank with how the level selection screen works, or other small workflow and QoL issues in the editors.
All that kinda thing we’ll keep polishing as they get reported to make 4.1 as refined and smooth to use as is physically possible.
libGit and it’s usage in Torque3D
This isn’t really a make-or-break issue, but ties into the above about various pain points and user feedback.
The new Project Manager generally has been well received, but there’s been an ongoing issue with some funkiness about how it utilizes the commandline calls to kick off git for downloading engine builds or modules.
It works, but inexplicably it can run into issues sometimes, and because it’s routing blind through the commandline error reporting is basically non-existent. Which is always frustrating for everyone involved if no one knows what actually went wrong.
Beyond that, people have expressed annoyance that it feels like there’s still a lot of hurdles for grabbing modules and being able to just… ‘go’.
As-is the workflow is to grab the engine, then go to the resources github page, download whatever modules, toss them into your project, and then you can start playing with things.
If you’re using the PM there’s a few less steps because it can present you module options more directly, but it’s still ‘external’ to the editor/project you’re working on.
So we’re going to look at being able to build libGit into normal T3D as well and let you snag resources directly from the Asset Browser.
While the Project Manager will still do it, as well as let you… well, Manage Your Projects, such as exporting a game-ready build, etc, a user would be able to in theory grab a binary build of Torque, launch into the editor, and start snagging modules that are instantly ready to go right there, without needing to leave the editor.
If that approach shakes out, it means the hurdles and speed bumps for prototyping or bringing in content for your project would go down massively.
You could be like “Dang I could really use some trees right now” to clicking on a Trees module in the AB, it downloading via git, and now having trees to paint into your map without even needing to leave the editor.
Conveniently, Az has put in a lot of work getting the library module ready to go, so I can pretty much start testing out said above integrations immediately, so figuring out how to use and abuse it won’t have long of a wait.
It’s a promising prospect, and something we’re gunna take a good look at testing out, because really, no one ENJOYS fiddly work. People want to be able to work on their actual games. And workflow improvements like that would go a long way towards that end goal.
So That’s Everything Then?
More or less, yeah.
Obviously there’s bits and bobs of fine details, but that’s the brunt of it.
A number of big reworks and fixup passes caused quite a bit of a grind there, but ultimately the engine is in better shape for it.
The todo list is very short at this point if you consider “Fixing issues as they get reported” as a single main-line category, so I’m hoping to get our first 4.1 Release Candidate out in the next month or so.
So keep your eyes out for that because we’ll definitely want people jumping in there and really giving it a workout so we can polish things up to a mirror shine!
And, as always, if you want to help pitch in to cover stuff like server/hosting costs and the like, feel free to swing over to the patreon:
You even get a nifty role in the discord now if you support it!
See you there!
-JeffR
-
By JeffR in Torque 3DGreetings and salutations!
It’s been a hot minute and a half since the last workblerg, admittedly in part because it’s a lot easier to just keep working on stuff rather than stopping to type a bunch about working on stuff, but ultimately a lot of people get a high-level sense of progress on the engine from these so really, I should keep up on them more.
Not everyone is constantly hoovering up all the information in the various discord channels, after all.
So, given that, there’s a lot to go over to get everyone caught up, so we may as well get cracking!
What’s Been Done So Far?
So, all told, there’s been 289 pull requests merged in since the last work blog. Which is a lot, to say the least. So why not go over some of the highlights!
General Fixes and Improvements
We fixed up a LOT of miscellaneous errors, warnings and other related issues that would cause message spam. Similarly we added various sanity checks in a whole bunch of places to catch overruns or out of bound problems which are obviously just generally a good thing.
Less critical, but similarly important to the general application stability we did a pass in a bunch of files to ensure classes initialize their variables and clean up unused values, which keeps things behaving reliably, especially when you’re hopping between Debug or Release builds.
Various mechanical bugs that got fixed pertained to:
castRay behavior various bits of playing audio Vehicle physics tweaks and corrections Sound effect spam from impacts Tweaks and fixes for player animations and callbacks relating to them playing Datachunker and FrameAllocator refactor for better memory management Sanity checks on prefabs getting child objects that have somehow ended up as null Vehicle steering thread fixes so mounted objects work as expected on network File write improvements Resolved various crashes, in particular ones that happen on exit Fixed up GBitmap getColor behavior and cleanups to the color pickers Added setSkinName to TSStatic for consistency of utility methods Additionally, a few passes were done to comply up the code to some modern conventions, such as rather than simply marking everything as virtual, and letting the compiler figure things out - which didn’t always work consistently between compilers and platforms - we now use virtual on the topmost declaration and the ‘override’ keyword on all derivatives, ensuring compilers understand the design intent better.
Another more general shift which is a ‘partway’ step for follow up work is templatization of Matrix math classes. While it doesn’t replace the existing Matrix math classes, the idea is that in the future we have templates defined for the various Matrix math classes so that we can utilize common code rather than duplicating and redefining everything for every permutation, such as 3x3, 3x4, 4x3 and 4x4.
Having more common code via templates means less duplication, which means fewer places code could misalign in one spot but not the other.
The idea would eventually follow a similar setup for most of our ‘multiple’ math types such as Point*F and the like.
All told, we had 72 different PRs merged for various fixes, improvements and additions. Which is a pretty sizable chunk of the whole.
GUI
On the gui side of things, it wasn’t as monstrous as the general fixes, but we certainly had a good number of fixes and improvements there as well.
Some key highlights:
Fixed a buffer overflow in GuiTreeViewCrtl Updated editor theming to utilize the stuff Nils provided for a more sleek dark theme Updated BaseUI in a myriad of ways to facilitate gamemodes selection and filtering(more on that further down) as well as improved handling of menu navigation with both mouse and keyboard as well as gamepad Added the ability to enable search filtering in dropdown controls Fixed various callback behaviors from some gui controls Added a method to forcefully constrain the mouse to a canvas(Thanks Steve!) Added the ability for MLTextCtrl to write out characters over type for a typewriter effect Fixed guiObjectView control issues Fixed up the state management for GuiBitmapButton This is less huge than the general fixes, but still some good changes in here. Probably the biggest specific changes were the updates to the Editor theme:
Old:
New:
As well as updates to the BaseUI
Old:
New:
In addition to the main menu stuff in BaseUI, the pause menu was updated to be the ‘Game Menu’ to allow easier slot-in of game-specific menus easily, even with just dropping in a module.
For example, Catographer utilizes inventory and map menus for gameplay stuff, and you can see how dropping in those modules just registers the menus to the GameMenu below:
The default Game Menu setup at the pause screen:
And as you can see with Catographer like mentioned above, we’ve dropped in new pages to the menu, like the map, inventory or objectives menus. Which show in the top bar and can easily be swapped between:
Much easier to expand the menu in a consistent, easy-to-use way for both the dev and the player, leading to a better experience overall. All you need is the hook against the callback which is invoked when the GameMenu is opened, and it looks like this:
Editor
The editor updates were the main focus for 4.1 and while we haven’t hit everything we were hoping to, the big focus was some important foundational updates to make working with and expanding the editor a LOT easier than it currently is.
Some of the highlight changes are:
Added typehints in the SceneTree, as well as what primary asset is used on certain classes, such as showing the ShapeAsset used by a TSStatic below. All of which is data that the search filter on the tree can find Added multi dimensional entries for certain math types in the inspector Fixed up the forest element inspector Expanded the editor number range to 7 digits Formatting fixes for the several GUIs and editor windows Added an ‘Add’ menubar entry that parses object class categories to automatically register spawnable classes to both world and gui editors Added a class prototyping UI to easily block out derivatives and callbacks Various fixes for IBL and probes Adds a callback to objects so if the editor is saving, objects can do special logic in response. Such as resetting PathShapes back to start so they’re always in the right spot in the saved level Updates Material and ParticleData fields to organize them sanely to facilitate shifting off of custom adhoc editor GUIs for those to using normal inspectors(more on that below) Fixed presentation of bitmask/enum fields in the inspector Ensures material animation flags are cleared when editing so they don’t ‘stick’ Fixes for arrays of imageAsset types so things correctly update in classes like guiBitmapCtrl when editing Fixed guiShapeEdPreview not displaying IBL Fixed animation scrubber in ShapeEd Fixed some value irregularities with positions when editing Fixed saving of Asset Browser position so it stays where you left it Added search bar to Datablock Editor inspector Properly preserves gridsnap settings between runs Makes snap buttons visually synced to their value Integrated SubScenes and SceneGroups into the world editor interface. For some of the more ‘meaty’ changes, specifically to facilitate that whole ‘easier to expand and work with the editor’ we’ve got stuff like:
Programmatic action pallets
You may not have ever thought about these:
But in the original editor stuff they were explicit gui controls that were just added onto the gui stack of the editor suite. That meant that while it worked perfectly, expanding them was kind of a pain. It wasn’t easy to just be like “Hey, in this specific context, I want to be able to show/hide more buttons/actions on the pallet!”
So I added a programmatic version of it. You shouldn’t even normally notice a difference, but it means that we can now inject in additional buttons in certain contexsts when needed.
What might those contexts be, you might ask?
A fine question!
Here’s one example with SubScenes or SceneGroups(we’ll get to what the deal with those is further down, but think of them as ‘like prefabs, but rather than a dedicated file, they’re just child objects in a SimGroup you can control the position of’.
So, because we may want to move the group itself and not the child objects in some specific cases, we wanted mode buttons to let you pop between normal behavior(moving the SceneGroup naturally moves all the child objects) and moving JUST the SceneGroup(leaving the child objects where they are, causing an offset)
You can see it in action here:
Those buttons for the ‘Move/Rotate SubScene + children’ only show up when selecting a valid object type, otherwise they disappear! And before, that would’ve been kind of a pain to do.
Improvements To ‘In Place’ Workflow
This is a somewhat perceptibly minor change, but can improve workflows a fair bit. Namely, it Adds some additional elements to the Datablock fields on Gamebase derived objects:
The three dots button jumps to the datablock in the DB editor directly, which isn’t a big change. What’s more important is the little plus button, which if clicked, jumps you straight to creating a new datablock with the inheritance already filled in.
So if we take our DefaultPlayerData there, hit the button(and select what Module it goes to) we see the Create New Datablock window pop up like so:
As you can see, inheritance is already filled in to come from the DefaultPlayerData, so you can just make it and create it easily, That allows you to jam out derivatives right when you’re working instead of needing to pop over to the DB editor, track to the class, create new, select the parent DB, etc to create a derivative.
You still CAN do all that, of course, but the idea is to automate those steps in the context of the + button on the DB field.
Similarly, to facilitate the easy workflows when working with datablocks, especially because that list can get LONG, you can now search/filter the DB field dropdown too!
When typing ‘dem’ it filters to the DB entry(ies) that have those letters in it:
Library
We updated various libraries and library-associated things here. This isn’t anything overly special, but it is important to keep track of!
Here’s what was updated:
Updated SDL to 2.28.4 Added STB for image loading Updated Assimp Fixed up logic so when converting from assimp data structures to Torque, the matrices are correct. Fixed various issues relating to using Theora Changed to using libSndFile Updated openAL Fixed dynamic library builds on windows Cleared out a bunch of temp files for Bullet that weren’t needed Added support for modules to define the inclusion of libraries, so one can simply drop in a module into the data/ directory and regen CMake and it will detect and process not only source files, but cmake configs needed to build libraries such as libCurl Platform/System
Likewise to library updates, this is mostly maintenance and upkeep stuff, though there were some important tweaks to various parts of how the CMake configs are organized/ran
Cleaned up fileModifiedTime stuff Fixes up the libCurl module cmake config Fixed issue with sdlCPUInfo with FreeBSD Making startTime static to avoid collisions in the core/console stuff Clean up display of optional core modules in the cmake config Fixed the cmake config to properly add in the wind deformation hlsl and glsl shader feature files Brought engine requirement down to C++17 standards from 20 Multiple various apple fixes to make it build more properly and pack files in a more ‘apple friendly way’ Fix so you can disable TORQUE_TOOLS flag and still build Fixed zip archive reading Adds a TORQUE_TOOLS_EXT flag for specialty stuff that you only wanna turn on very deliberately beyond just tools support, such as enabling the ability for the engine to run commandline calls. Various cmake format/config changes in the interest of trying to cut down how much duplication and redundant defines/settings there are throughout various places Rendering
Altogether there weren’t any sort of ‘gigantic’ changes for rendering on this one, since it wasn’t really the target goal. But that doesn’t mean there weren’t quite a number of fixes or improvements of render-related matters!
Highlights:
Reverse depth & 32F buffer format OpenGL fixes for HDR Fix vert color swizzle Optimizing Bins Various lighting corrections Adds ability for IBL to scale with ambient lighting so it works with Time of Day D3DCompiler_47 dll to ensure deployed projects have it packed and ready when publishing Make cubic reflectors respect detailAdjust Fixed probe capturing behavior Fizzle fix for GL Fix Sun corona Updates to GFX Shader handling Implemented IES light profiles Fix dynamic cubemaps on objects Removes glowbin as a render pass Added normal map handling to imposters with assets Fixed cubemap baking DX11 Add skylight IBL display to Material Preview display Tweaked attenuation for probes Roughness corrections with mip ramp-up Overall there weren't big shakeups, but various fixes and optimizations and the like. Probably the biggest specific visual change was the IES profiles which allow lights to load up mathmatic representations of certain lights in real life to make them have more real penumbra patterning like so:
Scripting
For scripting stuff, we had some various fixes and improvements, but a couple of big changes as well.
Highlights:
Console Refactor Add localization utility methods Fixed type bugs for TS Fixed an error in the DB substitution logic Fixed evaluateF argument handling Fix foreach breaking if it iterates over non-existent objects Removed CInterface script hooks because Console Refactor superseded it Console interop improvements Various script stability fixes Various updates and improvements to the TS compiler, parser and lexer Fixed error printing Fix callOnChildren varargs All in all the biggest thing is the Console Refactor where the idea was doing a lot of standardization and making it generally easier to interop and maintain. Not really anything that’d impact the end user, but it has some convenient bits for up-keep.
Assets
For assets it’s broadly just a bunch of fixes for functionality, so we can just pick out a few of the highlights:
SFX Playlist asset working Fixes saving of asset definitions that have sub-objects properly Adjusts the logic for spawned simObjects from tamil files so array’d fields works properly Updates to asset load workflow generateCachedPreviewImage fixed Ensures that when an asset is edited, it’ll properly refresh imageAssets can be blank Improvements to triggering of shape loading with TSStatics Simplify sound slots Fix glb asset importer file detection Correct malformed import config passthrough Updates to Asset Importer Fixed utilization of material fields so they can work with NamedTexTargets Added a reloadModuleFiles(%moduleGroup) command to allow forcing of reloads when needed Testing/Debugging
Not much specific here, just a bunch of various fixes to improve the CICD/Testing suite along with a few debug reporting cleanups.
Gamemodes, Subscenes And Spawning
This is one of the bigger updates to things, so we’ll go more in-depth with it here, as was mentioned in a couple of spots above.
So!
Gamemodes
Gamemodes were always sort of a loose concept as far as T3D was concerned. Obviously there needed to be a grand arbiter of gameplay logic such as rules for ‘if players kill X number of enemies they win’, but beyond the necessary callbacks to have them kick off when a player connects to a server, everything else was pretty loosey-goosey in terms of integration.
Want to know what gamemodes you have available? Want to be able to filter selectable maps by gamemodes? Want to use a ‘Racing’ game mode, but rather than the default assumption of racing cars, you want it to be a footrace so you spawn players?
All that sort of thing would require a good bit of extra developer integration and it wasn’t easy for modules to just interop with it.
This is especially a problem when the goal with modules is being able to drop them in and largely be integrated automatically so you can focus on the glue bits that brings them together into a coherent gameplay design.
And, above all of that, frankly people weren’t positive how you were supposed to set/associate a Gamemode to a level either. As well as the fact that it was just a text field on the Scene object meant that if you typo’d the gamemode name nothing worked and that could be annoying to chase down.
So what do you do?
Well you set some standards!
In specific, Gamemode is now an actual, concrete class rather than just a ScriptObject with a namespace slapped onto it. On it likewise we have utility methods to query what gamemodes exist and some other neat doodads.
Another important bit is Gamemodes can be marked as being activated or not, as well as if they automatically activate.
This is prudent because if you have a game with only one gamemode, then it’s silly to need to faff around with toggling it on in the menus or whatnot.
So, where that gets you is you can have 1 or many gamemodes, and then the system knows what gamemodes it has, and can run queries on them.
The old callGamemodeFunction, which originally just iterated on whatever was defined in the Scene object now works against the standardized ‘get a list of our gamemodes’ method, and calls the method if it’s available, and the gamemode is activated.
This means that any level of mix-and-match for gamemodes Just Works with interoping to the level loading sequence, or if you do call signalling for gameplay glue code logic.
Want a Racing Capture The Flag Deathmatch game? Just activate Racing, CTF and Deathmatch Gamemodes and spawn in and bask in the glorious chaos you’ve created.
But you may be wondering ‘Surely maps have to be compatible with certain gamemodes right? How do you set that?’ and that is a fantastic observation.
The reality is that you can’t assume your insane RCTFD gameplay smorgasbord you just invented above would work on every map. If a map doesn’t support capture the flag in any capacity, you wouldn’t want that map to be chooseable on the level select screen after all!
The good news is, it’s pretty danged easy to select gamemodes for Scenes now. If you have a Gamemode made and the script executes, then when you edit the Scene object in your mis file, you can look in the inspector and see this nifty shindig:
And selecting what Gamemodes the level is compatible with is as easy as picking them there!
With updates to the BaseUI menus, we scan the levels’ listed Gamemodes picked via that field, and if you pick a Gamemode in it like so:
Then when you advance to the Levels list it filters down appropriately:
The logic of the menu is fairly smart too, in that if the Scene never defined a gamemode, we figure it’s a sort of ‘assumed default’ state and let you at least *try* to run any gamemode with it. Likewise, if you only have one Gamemode, we assume that’s THE gamemode, and activate it automatically.
What that means is if you’re making a pretty simple game, like maybe it’s an exploration game with one map and just the one Gamemode, then you don’t have to do any special sauce to set it up, it just figures “Only one of these, and one of these? That’s probably the winning combo there”.
Keeps things easy to snap in and iterate on.
Once you have the gamemode picked and running, it largely behaves the same. The client/server and level loading logic has various callGamemodeFunction invokes that, as mentioned above, call the function for any active Gamemodes, and just… does what the gamemode logic defines, largely same as before.
Just more standardized and structured.
Pretty neat huh?
But what if you want like, one main map, but then do something crazy like having a Nighttime variant, that not only changes a bunch of small bits in the map(Streetlights? On! Sun? Gone!). Or if you wanted to have a single base map, but be able to slap in necessary bits for certain game modes, like changing the ‘bits’ in the map depending on if you’re doing CTF or Racing?
Or heck, what if you just look at the map and go “Good lord there’s so much junk here, do I really need to load it ALL right off the bat?”
Well, I have fantastic news because that brings us to…
SubScenes
So what’s a Subscene, you may ask?
Well it’s like a regular Scene we’ve had for levels for a while, but… sub-ier.
Thank you, thank you, don’t forget to turn out the lights when you leave.
Alright, jokes aside, it really is a ‘Sub-ier Scene’. Where a Scene is basically the grand arbiter of the content of a level and has some neat utility in tracking/managing the objects under it, being able to pick gamemodes it works for, etc, etc.
Subscenes are similar, but they are intended to exist WITHIN a scene. So if you have a big ol’ chonker map with roads and trees and terrain and all that jazz as your main scene.
You could have a SubScene that only has certain objects to load from its own subMis file when you need it. Think of it like how Prefabs are placed in the map, and it loads the contents from a file, and you can move it to where you need the bits to be.
The main difference is SubScenes are intended to act more as a container of objects, rather than AN object made of bits.
So how’s that work?
Well, first thing is we need some junk to make into a SubScene!
So you have some objects in your map and for whatever your vision you have, you need it to be ripped callously out of existence and shoved into a pocket reality, er, subMis file. Like lets just take these little ConvexShapes minding their own business.
I’ve decided I need them to be in a SubScene.
So, per the video below, you can see how I can make a new SubScene, then add the objects to it.
Once we have it and have a SubScene in the map, we just add objects to it like a SimGroup, and then save the Subscene. This’ll save the child objects out to that subMis file, rather than the main level mis file.
From there, you’ve got options. You can either programmatically force a subscene to load in script by just invoking the load() method(or unload, for the opposite) or have it load when a client enters into the bounds of the subscene - which allows for a simplified form of level streaming.
Per the video above, you can even see how when I don’t have it selected, and the camera isn’t within the bounds it’ll unload after a few seconds(a configurable variable), and when the client re-enters the bounds we load again!
You can even establish conditions on if the SubScene can load, making it so players need to complete an objective before the next part of the map is loaded in or similar.
You can even set Gamemodes on subscenes like you can for regular Scenes, which lets you build out logic to filter subscenes easily for your game’s logic.
Ultimately, in its current form it’s a pretty baseline implementation, but with a lot of options for how to utilize it to control stuff being layered on or loaded in.
As it gets more beatups and used more, we’ll expand and refine the functionality of it to make it even more useable and more powerful.
Spawning
The last big chunk of work that was done was for the spawning logic. Originally the gamemode was the ultimate controller of when and what spawned and why.
It worked, of course, but it was pretty restrictive, and absolutely got in the way of the idea of mix-and-matching gamemodes, or making it easy to drop in modules and just have it work easily.
If you wanted to do the previously mentioned ‘Racing game but you spawn as a player instead’ idea? You were dropping in the racing kit module and then getting down and dirty in the script to change it yourself to spawn a player.
That was ‘fine’, but also quite restricting. And beyond that, poor for rapid prototyping and iteration times. So where we ended up shifting things is pulling the invokes for spawning back into the client/server loading stack.
We know that when a client connects to a server and loads into a map, they need to spawn as *something*. Even if that something is just a camera.
So? We just do module and gamemode calls to let them have a chance to interject what they think the client should spawn as, where, and with what parameters.
It works via an override system, so last thing to override has final say.
In the end there’s 3 main new methods that get called:
SetSpawnObjectType Sets what the connecting client should spawn as in terms of object type/class. Such as camera, player, vehicle, etc SetSpawnPoint Indicates WHERE they should spawn. OnPostSpawn Final after-spawn effects. Setting up inventory, attachments, changing the team colors, etc These are called as part of the client connect/load stack, as mentioned, with it walking through any modules that have those functions defined, as well as any active gamemodes that also have the functions defined.
The logic of the calls utilizes a signal and counting system, so it ensures all modules and gamemodes that *can* work with those functions have a chance to do their work before continuing.
Normally, it’s a simple ‘run script and be done’, but depending on the gameplay in question, you may need to wait for a maze to generate, or other asynchronous logic to execute you can’t assume a certain amount of time for.
From you, the developer’s perspective, it just slots in and works, but in the back end it gives everything enough time to ‘breath’ to try and keep things running even when stuff may take inconsistent amounts of time to run.
Why do all of this? It’s simple.
It allows things to drop in and Just Work.
If you’re doing a racing game, but want the footrace style, then dropping in a player module instead of a vehicle module, with said player module definition the SetSpawnObjectType to be a player means that the incoming client will ultimately spawn as a player, not a vehicle.
It lets the logic and rules break out much more piecemeal to let stuff interop more cleanly.
To further facilitate this, we added a Priority field to Modules, that allow you to nudge them around for these callbacks’ order without messing with the ACTUAL load order and dependencies structure.
Ultimately, nothing stops you from just defining all the logic and rules in your gamemode and have it be super strict and rigid, but the main point is you no longer *have* to, and we can set up modules to behave in a much, much more ‘Drop In and Go’ manner.
Better flexibility, better prototyping speed.
So, What’s Left?
Good news is, honestly not much.
4.1 certainly has taken longer than desired, but that’s what happens when you’re doing practical beatups and realizing there’s a lot of spots that have rust or dust all over them and need a good pass with a pressure washer and an aggressive polishing pass after.
But even with the unexpected drag-out time, the majority of the desired work for this pass in helping lay foundational stuff and make moving forward easier is locked in, meaning that our remainders would be:
BaseUI fixes
While a lot of good work has been done on the BaseUI to make it clean but flexible, there’s still a number of rough spots. Namely? Bugs.
The options menu is a particular culprit in this because options menus are bloody complex things to work seamlessly.
Volume levels not respecting the settings, keybind mapping and display being flakey with multiple action maps, changing between borderless and windowed mode not automatically shifting to display resolution options and more.
All relatively minor things, but definitely gets in the way of the user experience being seamless. And some of those get in the way of developer time too. If you don’t want your game’s music or sound effects blasting while on a discord call, sure would be useful if the volume sliders worked, right?
Additionally, there’s little oddities of behavior like highlighting and selecting being muddied logic-wise when using mouse and keyboard, some UI elements not updating/refreshing like one would expect, or some scaling/formatting oddness.
Again, relatively minor, but it builds up to be a rough experience, and the Baseline UI needs to be good enough to hit the ground running with, not making you have to stop and slap it around to play nice.
Editor improvements
A lot of the desired work for this has been knocked out, but some stuff remains.
Some things, like customizable layouts were put on hold while more design-time consideration is put into it. I did a fair bit of R&D work into getting it to play nice, as you can see from this video here, and it *almost* works. But in looking at current UX conventions, both Torque’s and other engines’, as well as talking to you guys, “Just Doing What The Other Guys Are Doing” isn’t a good approach here, I don’t think.
Certainly, making sure the editor suite UI works WITH or FOR you while you’re making your game is paramount, but that doesn’t innately translate into being able to dock 11 billion sub-windows either.
I’ve got a few ideas on how to *really* maximize workflow speed with the editors I’ll keep cranking at on the R&D side and come back to how we’ll do customizable layouts and stuff in the next version or two, but for today the editor suite will largely stay ‘Classic Torque’.
That said, it doesn’t mean NO changes are incoming before 4.1 closes out.
The big one is complying most all of the editors’ interfaces to utilizing GuiInspectors where possible. I’m sure you all have seen the Material Editor a lot:
Or the Particle Editor:
And while they work, if you never looked at their guts in the script you may not realize that they’re completely specialty-case GUI controls that do a lot of custom script voodoo to function.
That works fine enough, obviously, but it also means it’s a paaaaaaaaaain to expand, tweak, or update stuff.
And also doesn’t follow the standard conventions of display in other parts of the editor.
So we’re moving them to utilize the normal GuiInspectorCtrls like we use in the vast majority of places. Same functionality, just a more standardized way of displaying, and it’s more automatic(being driven by the class’ fields) rather than a bunch of specialty adhoc stuff.
Easier to maintain, easier to expand, follows the conventions of every other spot in the editor, but it still does the exact same job in the end from a ‘working on my game’ perspective.
Win-win all around.
The work on this is mostly done, with the ParticleEditor conversion needing most of the work yet. MatEd is mostly done but has some residual fixes and tweaks to finish it out, and I’d like to get the Terrain MaterialEd to behave similarly before we lock 4.1 in.
Not too much work, thankfully, but it should help a good bit going forward once it’s in place.
Asset Management Improvements
The work for this is actually mostly done, just been sidetracked on helping polish out other WIPwork to get it locked in.
Pragmatically you shouldn’t *notice* a difference in general behavior when this all goes in, but under the hood it’s a big step up.
Functions are more standardized across asset types, use clearer, less confusing names. Asset Browser has a lot of special-casing trimmed out so things are easier to understand when reading through the code, etc, etc.
Main idea is it moves Asset Editing/Management logic out of the Asset Browser scripts, and into the Asset Type’s scripts, running through the assetType’s namespace, like so:
Each asset type of interest works like that and behaves consistently across the board now with basically no special-casing in the AB itself.
It also has some utility methods for registering asset types, so you can create ‘Asset Types’ that exist entirely in script and have the AB work with them as normal.
Meaning you could create a special type of Asset just for, say, NPCs, and integrate that into the Asset Browser seamlessly for creating new NPC assets, have it display them in the browser, be editable, and all that, but all the logic for it exists in your NPC module, without having to edit the editor/Asset Browser scripts to get it to show up.
Following the theme, this’ll make it easier to maintain the whole asset pipeline, as well as make it easier for modules to Drop In and Go with custom logic and content without you needing to do a lot of screwing around to make it hook in.
Better flexibility, better prototyping speed.
Project Manager
The last thing for this is the Project Manager. Going off feedback people’ve given when using it, it’s in a pretty decent spot as far as usability/clarity goes for being able to easily make and manage projects and modules.
The biggest thing being a problem right now is the reliance on using commandLine calls to git in order to pull stuff from the internet.
Good news is, we think we’ve cracked the big hurdles on libGit integration - which if you don’t know, is what all the visual git clients use - which would mean we can just interface with git directly rather than routing via commandLine to call git commands and hope it executes well.
That lets us report back REAL errors and not telephone game’d commandline error codes, progress bars, checking if there are pending changes to pull down, and more.
With that stuff in place, the overall experience of using the Project Manager should become quite smooth, though naturally we’ll continue to taken and listen to feedback to make it a fantastic one-stop-shop for kicking off or wrangling your Torque game project needs.
Beyond that, there’s only really just hanging small fixes and bugs as far as priority issues go, which we’ll continue to patch as we move.
Once these things here are locked in we’ll do some final test passes to make sure no hideous new bugs slipped the radar, but once they go in there’s no other new features/functionality going in for 4.1.
It’ll just be polishing up, working on documentation and example modules.
Whew
Quite the doozy, huh?
Again, an apology is in order. It’s too easy to slip into keeping your head down and just getting work done(which yields solid dividends, when you look at the above!) and just talking to people in the discord, but the reality is that you sometimes need to write down an obscene number of words and pin it to the bulletin board for everyone else, too.
So I’ll try and get back on top of doing monthly workblergs to facilitate that.
We’re also mulling on the idea of potentially doing a monthly-or-so community voice chat in the discord for some more direct live-fire Q&A stuff that’s a little more consolidated/pointed on the topic than random conversations spread across a dozen channels, but a bit easier to just ‘do’ than a full-bore workblerg like this, too.
If that sounds appealing, be sure to let us know!
And if you wanna help facilitate the ongoing Coolification(no trademark, unfortunately) of things, feel free to hop over to the Patreon and toss some money our way. It goes to paying for hosting costs and other such operational things of the sort.
But, for now, I’m going to let my hands fall off for a little while before I staple ‘em back on and get cracking back on those last TODOs so we can get 4.1 out the door.
And when it is?
4.2 and Components be on ye horizon.
*Dramatic Music Sting*
Later everyone!
-JeffR
-
By JeffR in Torque 3DSo, on today's installment of "random rambles about development things"
But for real, it's a good time to do a new workblog and keep people in the loop for those not in the discord, or those that aren't spending every day in it
So, what's on the ol' discussion stuffs today?
Well, for the big one, the main Feature target for 4.1: Components.
Or, more specifically DOCs. What does DOCs mean? Well...
Directors, Objects, Components
You may have heard us discuss 'Entity-Components or Entity-Component-Systems(EC and ECS, respectively). For a brief refresher in concept, here's a simple breakdown:
Entity-Components as a paradigm can be described simply as having an Entity object, and then Component objects that contain and implement data. As in, if you have a component to render a shape, the component not only holds the info for what shape to render, but also the logic to render the shape. This is how, most other engines do components.
The reason for that is pretty simple. It's robust, and it's easy to work with. It's not the most efficient system, but it's pretty hard to screw up. You slap a component onto an object, set the properties on the component, and then the component does the thing.
When I did the main previous implementation of components, this was also the system I went with. The MAIN problem with this approach is that any given component is kinda...chonky. And you also have a lot of bloat on the Entity object in most cases. And with all the bits that have to cross-communicate to ensure dependencies work(you gotta have collisions for physics to work properly for example) as well as order of events(collisions are calculated then physics) as well as any deeper engine system dependencies. It can spiral quite a lot.
Beyond that, it's also very difficult to thread any of the component's workloads because everything cross-communicates in order to work. You can't easily punt a physics component into a thread if other threads need to talk to the same collision component or entity it uses, etc.
So, advancements to the theory of components implementations lead to ECS: Entity-Component-Systems.
Now, the confusing use of "Systems" aside, the main differentiator to EC is that Components now ONLY contain data. They don't implement any logic whatsoever. Likewise, ALL the burden of functionality is moved off of Entities. In a 'pure' ECS implementation, an Entity is nothing more than an ID for Components and Systems to reference. Instead, Systems implement all functionality logic. If you have a physics component, there's PhysicsSystem that implements the actual logic for it.
This is certainly more complex to implement. In fact, very few engines or games use ECS. Unity's new DOTS approach is based on ECS, and a few games like Overwatch have utilized it. But the innate complexity of the approach and how abstracted the data and implementation means that it's far less common.
So why use it?
Because it is MUCH easier to be cache-coherent and thread things. For the non-coders out there, cache-coherency is the idea of wanting to keep all the memory a given chunk of code in the engine uses all bunched together. Think of it like how if you're studying. Rather than getting a book, reading a paragraph, then walking back to the shelf, putting that book away, and getting a new book and reading the next paragraph, and so on - which would be very, very inefficient - instead you just get all the books you need, and can quickly reference between them.
In practice, memory in the computer works similarly. So if you can cram all the data you need to work into the same blob of memory. Performance is improved SIGNIFICANTLY. But it's not very 'human friendly'. Which is why you get stuff like ECS. All components of a type can be crammed into a dense set of data. So when a System goes to implement logic, you've got all the relevent components in a tight blob of memory, and the whole thing can be processed without having to go "get another book" as it were.
In addition to this, the data being more detached from implementing logic(and better managed in memory) makes it much easier to implement the logic in multiple threads. This allows the machine to crunch a lot of objects in parallel - which is especially good on modern CPUs that have sometimes dozens of threads.
But there's a good number of downsides to this approach as well. It is, as said, not very human friendly. Implementing new components and their associated systems is not how most code is implemented, so it can be difficult to work with. It also requires much more tracking of when things are added, removed, when things should run. Dependencies are still burdened on the components and systems to keep track of for when to implement things, and scripting it is very, very difficult.
All those and a bunch of other smaller inconveniences make it generally a pretty poor paradigm to work with in something as complex as a game engine. There's a LOT of ECS implementations out on the internet. But they're more academic than practical because of the inherent limitations of the approach. Cramming it into a game engine while still making it easy to work with from a scripter, designer or artist's perspective is pretty hard.
And both of these have various limitations in how to deal with it from a networking perspective. It's very difficult to have the server and client safely agree on the data the client has without trafficking a ton of data, which is bad for net performance.
So, between what I learned from implementing an EC style deal in the first pass of components, and a lot of tests and research into ECS. I settled on the fact that, both approaches just kinda aren't ideal.
So I did some work and fashioned up a - as far as I can tell - novel components implementation for Torque3D.
Directors, Objects, Components, natch.
So, what’s the deal then? Well, per the name, there’s 3 main components(heh) to the model, which we’ll cover here:
Directors
So what’s a director? Well, in practice a Director is a simple class that ‘directs’ when and where updates to components happen, hence the name. The idea is that we want to move the burden of when and why updates happen off the objects and components.
At it’s core, a Director is in charge of doing a particular thing, generally updating a specific component or set of components. Like, say, when we want our RenderMesh component to draw. The Director has a specific timing to it(aka, Rendering) that the rest of the engine can invoke to the DirectorManager, which is pretty much just a simple container class.
When we want anything with the Rendering timing to kick off, we tell that DirectorManager to run an update on said timing. And in turn, any Director with that timing is told to do it’s work. Simple enough.
So in our example of the RenderMesh components, the RenderMeshDirector has the Rendering timing, the engine, when it goes to draw objects, can tell the DirectorManager to run the Rendering timing, and our RenderMeshDirector gets told to update. When this happens, the Director loops over valid RenderMesh components and directs them to do their work.
And thus, our RenderMesh components have drawn their meshes.
Now this sounds like a lot of work compared to just looping over the objects or components directly, but there’s a bunch of benefits to this.
For example, as noted with EC and ECS implementations, one of the biggest tangle-up points is dependency management. Normally components have to track what dependencies they have, if they’re fulfilled, and if not then they are not enabled. Any time a new component is added to it’s owner, the component is in charge of validating its dependencies.
This is important, certainly, but it also can lead to a lot of complexity, spiraling dependency chains, and code bulk on the components themselves. So instead, we move that to the director.
Because ultimately, the director is in charge of a set of components, like our RenderMesh components, we can track which ones are valid in the Director. If an Object adds a new RenderMesh component, it naturally associates to our RenderMeshDirector, and it now knows if it’s valid or not. The component itself doesn’t have to care in the slightest.
This keeps the component code leaner and cleaner, so it’s easier to maintain.
It also means we can very much more explicitly control the timing of when things run and sequence in the engine. I used the example of the “Rendering” timing before, but it’s powered by a simple enum. So you have as many entries as you can cram into an enum for when to kick off updates. You can update just physics things, or just rendering things, or specifically objects that have client inputs because they’re controlled.
This gives a much more comprehensible order of operations about when and where stuff is executed in the engine, making it easier to track and debug when stuff kicks off.
Additionally, because the director has an explicit list of components it’s in charge of, and we are specifically working on that list of components at a specific time in the execution of the engine’s loop, it means that we have MUCH more control over the memory in play for the engine.
This ties back to the aforementioned cache coherency. We can keep a list of components, like RenderMesh components, and that list can be much more easily just shoved into memory as a straight shot, minimizing how much the CPU needs to jump around. The Director works on THESE objects, so the CPU can have all the data on hand.
It also means that, between the more tightly bound memory and the express execution timing, we can much more safely handle when things are threadable. Which is a big thing for game engines.
Even major engines are still predominantly single threaded. So when you’re busting out your brand new CPU with 36 cores. Most of them are sitting around doing nothing. With Directors controlling the memory and execution, we can spool up a bunch of tasks in the threadpool and split the workload across those cores/threads that aren’t doing anything, allowing the regular workloads to be processed way faster. And this should, in theory, scale well with object counts.
So yeah, Directors are kinda the MVP of the system, what with keeping a tight wrangle on memory, streamlining execution of parts of the engine, standardizing a lot of bits, and also making the engine significantly more threadable than before.
So, you know. A little bit of a thing there. Which takes us to the next bit of our paradigm:
Objects
Compared to everything that directors do and are, Objects are pretty simple in the end. These are your entities that you slap components onto. Unlike a full-fat ECS implementation, Entities are ultimately still full objects.
There’s a good reason for that, of course. The big one is that T3D has a scripting language, and that’s super useful. So an Entity can’t just be a ID that exists in the void, because we gotta have an object for the scripts to work through.
Additionally, T3D also has a very good networking system, and not maximizing that is just dumb. And rather than having each component be manually replicated, or ghosted to clients, we exploit the way T3D does networking streams and packing to go through our Entities.
Specifically, Entities keep a list of components they own. And if a component is marked to be networked, it has a separate list for that. When a networking event happens, such as a component is added, removed, or updated, the Entity is itself flagged for networking action.
Since the Entity is ghosted to clients as normal, we can then piggyback the Entity’s network updates. Each component that’s networked has it’s own mask bits for granularity - we only need to update what actually changes - and this is packed into the Entity’s network update.
This means we can fully network whatever number of components, but only 1 ghost per Entity. And what updates we DO traffic to the client is able to be as lean as possible. This keeps the traffic as thin as physically possible without giving up the very solid networking that T3D offers.
And lastly, we have the mainstay of any component system(duh):
Components
In DOCs, Components behave very similarly to in the previously mentioned EC paradigm. They hold data and implement the functionality for that data. So a RenderMesh component holds what mesh we want to render, and also does the logic to render the mesh.
The main difference, as covered in the section on directors, is that the components are stripped down to JUST the data and implementing logic and all the surrounding boilerplate is largely standardized up into the Directors, as well as when the components are told when to kick off their logic.
This means that implementing new components can be relatively easy, as you have the basic data/implement setup, along with any networking pass-through logic as noted in the Objects section, and then a companion Director to manage when the whole shindig activates.
All in all, while a bit more complex than standard game object classes, you get a lot of flexibility and ability to quickly slap stuff together without completely shifting to a new conceptual paradigm like a pure ECS implementation.
It also keeps networking lean, keeps scripting on the table, but also opens up the door to massively thread workloads in the engine.
So more flexibility, cleaner structure, more performance, and without compromising the good bits the engine already offers.
Not too shabby a deal, eh? 😉
Now, this update’s already quite a long one, pretty technical and tragically limited on pictures, so I’ll do a follow up post next weekend going into the front-end usage(which is realistically where most people will work with DOCs) as well as other development stuff going on or planned.
So I’ll see you all then!
-JeffR
-
-
Topics
-
- 256 replies
- 171824 views
-
- 60 replies
- 32823 views
-
- 420 replies
- 258435 views
-
- 4 replies
- 6175 views
-
- 0 replies
- 169 views
-
-
Show-Off
-
- 60 replies
- 32823 views
-
- 4 replies
- 6175 views
-
- 297 replies
- 109129 views
-
- 129 replies
- 45705 views
-
- 1 reply
- 2125 views
-
-
Built with Torque
-
Catographer
Open Community · 12 fans
-
Visitors: First Contact
Public Community
-
MegaMotion
Open Community · 1 fan
-
Übergame
Open Community · 1 fan
-
Johnny Trouble
Open Community · 1 fan
-
Another Tremulous Clone Sortof
Public Community
-
Rad Alert
Open Community · 2 fans
-
The ManFisher
Open Community · 4 fans
-
Airship Dragoon
Open Community · 2 fans
-
Pirate Code
Public Community
-
