Jump to content

3.10 Release Candidate


Recommended Posts

For the guys that noted a slight performance decrease from 3.9 to 3.10. We did some profiling and tracked it to the change in HDR format handling.


In 3.9, we had a thinner HDR buffer, RGB10A2, which has lower cost to fill out for the HDR postFX compared to the fatter RGBA16F buffer in use for 3.10 at the moment. The reason for the shift is that, with the linearized lighting introduced in 3.9, with HDR enabled you could get color banding, as the RGB10A2 texture didn't have the bit depth necessary for smooth colors in some lighting situations, specifically dark ones.


So for 3.10, in the interests of more accurate colors/reduction in banding, we upped to the fatter buffer texture. This unfortunately means that HDR has a slightly higher cost.


Post-3.10, we're dropping Dx9, and switching over to sRGB and a much leaner gbuffer(timmy's already got this all sorted, we just need to drop it in, basically), as well as some improvements to the HDR effect. All that together sees far, far more accurate color handling, with a performance increase as well.


For 3.10 though, if you're willing to deal with a bit of possible color banding in darker areas, you can roll back the fatter HDR buffer tweak and that should get the perfromance back to where 3.9 was by uncommenting this line:


https://github.com/GarageGames/Torque3D/blob/57dfeb829a40fb88c974d9370d70e8dcadeb650b/Engine/source/gfx/gfxDevice.cpp#L1324


That'll make the HDR format default to the RGB10A2, which as said above is lighter and thus costs less to utilize HDR, but at the expense of less accurate lighting and possible color banding.


@Duion for #1746, I have a PR pending I just need to make a tweak to tonight and will get rolled in. Plan is to get fixes for a few more issues that people ran into and have an RC2 pushed out this weekend.

Link to post
Share on other sites
  • Replies 51
  • Created
  • Last Reply

Top Posters In This Topic

Hey man, no worries!


3.10 is largely fixing bugs and some quality of life improvements, but we also wormed some new features in. It's not exhaustive a list(that'll come with the changelog for the release) but here's some highlights for 3.10:

 

  • Added MacOS support
  • Updated some of the libraries, like bullet, vorbis, png, and recast
  • Various fixes to D3D11 and OpenGL, which help stability and performance
  • Ipv6 support
  • OpenAL-Soft audio support(better than the old OpenAL)
  • Replaced OpenGl's epoxy with GLAD, which is cleaner and simpler for building out the GL feature list.
  • OpenVR support, which allows you to use the Vive
  • Hardware Skinning support
  • A bunch of fixes and tweaks to lighting stuffs to make it all play better with the linearized lighting

 

And as said, a crazy number of other fixes and tweaks aside from these(So far, we have 105 PRs merged in for 3.10, with a few that'll go in before release yet)


For 4.0, it'll be a pretty gigantic update and improvement in a lot of areas. I'd suggest poking through my workblog thread for some ideas on stuff I've been working on my side of it in line with it all. Important highlights are:

 

  • Dropping D3D9, allowing us to modernize the GFX stuff for OpenGL and D3D11, which should help bring a lot of performance improvements and new features like compute shaders down the line
  • Full switchover to a linear color system, specifically utilizing sRGB textures internally, which are a) better performing because the hardware does the dirty work, and b) all colors to be very consistent and correct across platforms and tools/workflow.
  • PBR(Physically Based Rendering) for the primary material lighting model. This will allow materials to look much more realistic, and have far, far better integration with modern toolsets with less effort.
  • Updates to HDR and tonemapping, which will perform better, look better, and behave a lot more consistently.
  • Hardware PCF shadows, which will look nicer AND run faster
  • A drastically slimmed down gBuffer, which will help the engine run better on older hardware
  • A new base template that has all the cruft trimmed out, so it's easier to use, change and loads faster.
  • The new template will also heavily utilize the new Module/Assets system, as well as make the Entity/Components system the standard for game classes. The huge monolith classes like Shapebase will be deprecated, and down the line removed. This will MASSIVELY help cut back on iteration and testing time, allowing people to focus on gamedev rather than fussing.
  • Updating the stock Torque physics to hook through the Physics abstraction API, so game code is consistent regardless of if you use, Torque's physics, Bullet, or PhysX
  • Drastic improvements and updates to the tools and editor suite, including visual editing of shaders, state machines, more benchmarking tools, etc.
  • Shift all the platform code to utilize SDL, letting us get rid of a lot older, platform-specific code and make things a lot more consistent
  • Big expansion on included starting examples, as well as better documentation and tutorials.
  • Code cleanup, including making sure files are adhering to the coding guidlines, and removing a lot of unwanted cruft, excessive includes which drag down compile times, and not-errors-but-not-right code which can cause bugs and irregular behavior, which should help stability of the engine across the board.

 

I think that covers the major highlights of what the plan for 4.0 covers. I said earlier I'm working on a proper Roadmap/Master Thesis breakdown of the plan for 4.0, and thankfully while that list looks daunting, a huge amount of the above is either actually done, or rather far along already, just waiting for us to get 3.10 out the door so we can start merging stuff in.


Lemme know if you have any other questions ;)

Link to post
Share on other sites

@JeffR - I'm in the same boat as HeadClot so I appreciate the overview of all of the improvements in 3.10 & 4.0.


I'd like to start a new project with T3D using 3.10 but I'm concerned the transition to 4.0 will be a hassle. Is the new base template (the one that's making use of the module & asset system) on git somewhere? I'd like to use 3.10 now but with the new template so I have at least some head start on the transition.

Link to post
Share on other sites

Sure!


https://github.com/Areloch/Torque3D/tree/BaseGameTemplate


That's the current build of it. It's a bit behind to devhead, but i plan on updating it soon. It has a module that contains all the current Full template's fps/gameplay stuff, so if you started working on something now, you could port it over to that module without a huge amount of hassle.


If you wanted to look at how the assets and entity/component stuff works, you can nab my R&D branch https://github.com/areloch/Torque3d/tree/RnDBuild


Where I do all the testing and building for the modules/assets/entity-component and all that jazz. If it's in my workblog posts, it's in the RnD branch.


I do plan to put together documentation explaining how you can port to 4.0(while existing game classes like ShapeBase, Player, Item, etc will be deprecated, there will be entity/component versions that come with 4.0 that do the same thing, so you don't need to start from scratch or anything, merely port over).


Let me know if you had any specific questions :)

Link to post
Share on other sites

Good to hear!


Over the next day or two, I'm gunna go in-depth on the particulars of the assets, entity/component and game object stuffs in my workblog thread, so feel free to keep an eye out there for more details on how this will all get utilized :)

Link to post
Share on other sites

Haha, no worries!


You can either make a thread here on the forums, or you can make an issue on the github page, just noting it's a feature request so it's easier to organize.


You needn't worry about getting in trouble for suggestions :)

Link to post
Share on other sites

Hey, what's the procedure for making a DX11 build? I did a fresh update, and noted that TORQUE_D3D11 was checked in CMake, but when I tried to compile, it failed looking for D3D9.h. Do I need to uncheck DIRECTX at the same time as checking TORQUE_D3D11? (Testing that...)


EDIT: Nope, that didn't fix it either. Do I still need the DX9 SDK in order to build an exclusively DX11 executable?

Link to post
Share on other sites

Currently, yeah, the older SDK is required. Ironically, not because of DirectX, but because of how MS decided to hatchet-job the audio dependencies with XAudio.


We're looking at how to resolve that what with dropping DX9 shortly after 3.10 goes out, but due to the knuckleheaded way Windows deals with the audio stuff, supporting XAudio at the moment with pre-8.1 windows necessitates the DX SDK.

Link to post
Share on other sites
Hey, what's the procedure for making a DX11 build? I did a fresh update, and noted that TORQUE_D3D11 was checked in CMake, but when I tried to compile, it failed looking for D3D9.h. Do I need to uncheck DIRECTX at the same time as checking TORQUE_D3D11? (Testing that...)


EDIT: Nope, that didn't fix it either. Do I still need the DX9 SDK in order to build an exclusively DX11 executable?

 

Yes you still require the old june 2010 dx sdk unfortunately. Both the GFX D3D9 device and SFX XAudio device both depend on it. Just a small correction on the above, the xaudio thing is correct but also the current implementation of D3D9 gfx makes use of d3dx which MS removed when they released the windows sdk, since D3D9 is getting removed in 4.0 it made no sense to waste any time making the D3D9 gfx device work without d3dx (which is entirely possible to do). In 4.0 xaudio will be an opt in feature (i.e disabled by default) so by default the old june dx sdk will not be required unless you compile in xaudio support.


The problem with xaudio is version 2.7 requires the old june 2010 dx sdk and the windows sdk only supports xaudio 2.8+, Windows 7 only supports xaudio 2.7 and not 2.8. That presents a bit of a nightmare because xaudio 2.7 code is not completely compatible with 2.8, it requires changes to compile. So if we support only 2.7 we need to also support the old june 2010 dx sdk and if we only support 2.8 (with the windows sdk) than it won't work with windows 7. There are actually ways around this but it's not very nice, MS describe one way here using a hard coded path to the old june 2010 dx sdk https://blogs.msdn.microsoft.com/chuckw/2012/04/02/xaudio2-and-windows-8/ for windows 7 builds. That is a nasty solution because it's a compile time solution and if you want to support windows 7 still (i'm sure most of us do), than you are stuck using the old june 2010 sdk. Hopefully all that makes sense. We have actually updated to using openal-soft recently so hopefully that suits everyone's sound needs ok and this xuadio thing is a total non issue ;)

Link to post
Share on other sites

Huh. Very interesting. Thanks again for all your hard work, Steering Committee! Speaking as just one T3D grunt out here on the front lines, I'd be completely screwed without you guys!!


Just to clarify though - if I hit the DirectX11 checkbox in CMake, and then compile it with the June 2010 SDK (which is no problem, btw, I don't mind a bit if it's just a compile issue) does that mean my build will run on an end user's machine without them having to download the DirectX 9 runtime?


This issue came up for me because I'm just starting to test my build outside my own machines (YAAAY!!!) and the second friend I had download it came up missing X3DAudio1_7.dll and D3DCOMPILER_43.dll. I've been assuming that was because I had a D3D9 build and he's never downloaded it. Is it possible that I just need to include those two dlls with my build? Or was my initial assumption correct and I need to build in D3D11 support, and then end users will not have to download anything because 11 is included in Windows?


Also, is it possible to support 9 and 11 at the same time in the same build, so users with older machines can still run it?

Link to post
Share on other sites

In the current build, D3D9 is the default graphics option and there isn't a simple flag to disable it, currently. Going forward to 4.0, 9 will be dropped in favor of 11(with feature support for 10).

So you'd need to disable some stuff in your project to make it agreeable on the win-side without DirectX, but as Linux and MacOS are both GL-only and run fine, yes, you could go pure OGL if you wanted with a bit of fiddling.

Link to post
Share on other sites

Hey guys, I think we're in the final stretch on this.


I'm trying to shore up 2 noted SDL issues(window edge snapping is weirdly offset, and some modifier key shenanigans).


If you guys could test these PRs and note any issues that crop up, that'd let me get them merged in so we can try getting a wrap on this:


https://github.com/GarageGames/Torque3D/pull/1933

https://github.com/GarageGames/Torque3D/pull/1932

https://github.com/GarageGames/Torque3D/pull/1931

https://github.com/GarageGames/Torque3D/pull/1928

Link to post
Share on other sites
  • 2 weeks later...

Not sure how useful this is because I used the latest project manager v2.2 but I figured I would post it just in case.


Grabbing this 3.10rc code drop, unzipping, dropping in the built project manager, create a game using the full template, all default selections (maybe add minidump support)

Open in visual studio 2012 on a win10 machine and attempt to compile and link. Here is what you need to do to make it work.


In the DLL project.......

Cannot open include file nfd.h

need to add ................./Engine/lib/nativeFileDialogs/include to all configurations (prepend with where ever you have the files located)


Cannot open Al/Al.h

need to add ................/Engine/lib/openal-soft/include too all configurations (also prepend with where ever you have the files located)


Error 50 error C2039: 'smConnectionNotify' : is not a member of 'Net' \engine\source\platform\test\nettest.cpp 79


Need to change the net::smConnectionNotify.notify...... to Net::getConnectionNotifyEvent().notify....

for all instances.....


and


Net::smConnectionReceive.notify to Net::getConnectionReceiveEvent().notify


and


all the ones that are .receive as well that are in the same file

if it says notify then its getConnectionNotifyEvent() if it says receive it is getConnectionReceiveEvent()


Linking problem

Error 80 error LNK2019: unresolved external symbol "public: void __thiscall Component::setOwner(class Entity *)" (?setOwner@Component@@QAEXPAVEntity@@@Z) referenced in function "public: bool __thiscall Entity::addComponent(class Component *)" (?addComponent@Entity@@QAE_NPAVComponent@@@Z) .............................................\buildFiles\VisualStudio 2012\projects\entity.obj


In the DLL project you will see that under T3D folder that the components folder and code is missing

create components folder, add the existing files to it.

then add all the other missing folders in that same location under the components folder from that directory (animation, camera, collision, game, physics, and render)


Engine -> T3D -> components -> animation

Engine -> T3D -> components -> camera

Engine -> T3D -> components -> collision

Engine -> T3D -> components -> game

Engine -> T3D -> components -> physics

Engine -> T3D -> components -> render


the folders and code are already in the source code, they are just not in the DLL project itself. so you have to create those filters and add the existing code.

Then it compiles, links and runs just fine. (platform windows obviously : )


A quick check and any other settings (like select opengl, or other modules, etc all have the same problem, or more problems).


Maybe that will help someone, could just be project manager issues, but the PM is still handy for creating projects with solution files.

Link to post
Share on other sites

×
×
  • Create New...