JeffR Posted December 22, 2016 Author Share Posted December 22, 2016 For the guys that noted a slight performance decrease from 3.9 to 3.10. We did some profiling and tracked it to the change in HDR format handling.In 3.9, we had a thinner HDR buffer, RGB10A2, which has lower cost to fill out for the HDR postFX compared to the fatter RGBA16F buffer in use for 3.10 at the moment. The reason for the shift is that, with the linearized lighting introduced in 3.9, with HDR enabled you could get color banding, as the RGB10A2 texture didn't have the bit depth necessary for smooth colors in some lighting situations, specifically dark ones.So for 3.10, in the interests of more accurate colors/reduction in banding, we upped to the fatter buffer texture. This unfortunately means that HDR has a slightly higher cost.Post-3.10, we're dropping Dx9, and switching over to sRGB and a much leaner gbuffer(timmy's already got this all sorted, we just need to drop it in, basically), as well as some improvements to the HDR effect. All that together sees far, far more accurate color handling, with a performance increase as well.For 3.10 though, if you're willing to deal with a bit of possible color banding in darker areas, you can roll back the fatter HDR buffer tweak and that should get the perfromance back to where 3.9 was by uncommenting this line:https://github.com/GarageGames/Torque3D/blob/57dfeb829a40fb88c974d9370d70e8dcadeb650b/Engine/source/gfx/gfxDevice.cpp#L1324That'll make the HDR format default to the RGB10A2, which as said above is lighter and thus costs less to utilize HDR, but at the expense of less accurate lighting and possible color banding.@Duion for #1746, I have a PR pending I just need to make a tweak to tonight and will get rolled in. Plan is to get fixes for a few more issues that people ran into and have an RC2 pushed out this weekend. Link to comment Share on other sites More sharing options...
JeffR Posted December 28, 2016 Author Share Posted December 28, 2016 Hey guys, hope everyone's holidays went well!Got a few more PRs to test and merge, but the plan is to get RC2 put up tonight. Link to comment Share on other sites More sharing options...
Chelaru Posted December 28, 2016 Share Posted December 28, 2016 Hey guys, hope everyone's holidays went well!Got a few more PRs to test and merge, but the plan is to get RC2 put up tonight. Yeee Link to comment Share on other sites More sharing options...
JeffR Posted December 29, 2016 Author Share Posted December 29, 2016 RC2 is up: https://github.com/GarageGames/Torque3D/releases/tag/3.10rc2Mac and Lin binaries will be in soon. 1 or 2 definite PRs that'll still go in for 3.10 official, but if we can spot any other oddball things to try and shore up, that'd be good! Keep up the excellent work on testing, guys :) Link to comment Share on other sites More sharing options...
HeadClot Posted December 31, 2016 Share Posted December 31, 2016 I have been out of the loop for a bit. What does 3.10 and 4.0 bring to the table for Torque 3D? Link to comment Share on other sites More sharing options...
JeffR Posted January 1, 2017 Author Share Posted January 1, 2017 Hey man, no worries!3.10 is largely fixing bugs and some quality of life improvements, but we also wormed some new features in. It's not exhaustive a list(that'll come with the changelog for the release) but here's some highlights for 3.10: Added MacOS supportUpdated some of the libraries, like bullet, vorbis, png, and recastVarious fixes to D3D11 and OpenGL, which help stability and performanceIpv6 supportOpenAL-Soft audio support(better than the old OpenAL)Replaced OpenGl's epoxy with GLAD, which is cleaner and simpler for building out the GL feature list.OpenVR support, which allows you to use the ViveHardware Skinning supportA bunch of fixes and tweaks to lighting stuffs to make it all play better with the linearized lighting And as said, a crazy number of other fixes and tweaks aside from these(So far, we have 105 PRs merged in for 3.10, with a few that'll go in before release yet)For 4.0, it'll be a pretty gigantic update and improvement in a lot of areas. I'd suggest poking through my workblog thread for some ideas on stuff I've been working on my side of it in line with it all. Important highlights are: Dropping D3D9, allowing us to modernize the GFX stuff for OpenGL and D3D11, which should help bring a lot of performance improvements and new features like compute shaders down the lineFull switchover to a linear color system, specifically utilizing sRGB textures internally, which are a) better performing because the hardware does the dirty work, and b) all colors to be very consistent and correct across platforms and tools/workflow. PBR(Physically Based Rendering) for the primary material lighting model. This will allow materials to look much more realistic, and have far, far better integration with modern toolsets with less effort.Updates to HDR and tonemapping, which will perform better, look better, and behave a lot more consistently.Hardware PCF shadows, which will look nicer AND run fasterA drastically slimmed down gBuffer, which will help the engine run better on older hardwareA new base template that has all the cruft trimmed out, so it's easier to use, change and loads faster.The new template will also heavily utilize the new Module/Assets system, as well as make the Entity/Components system the standard for game classes. The huge monolith classes like Shapebase will be deprecated, and down the line removed. This will MASSIVELY help cut back on iteration and testing time, allowing people to focus on gamedev rather than fussing.Updating the stock Torque physics to hook through the Physics abstraction API, so game code is consistent regardless of if you use, Torque's physics, Bullet, or PhysXDrastic improvements and updates to the tools and editor suite, including visual editing of shaders, state machines, more benchmarking tools, etc.Shift all the platform code to utilize SDL, letting us get rid of a lot older, platform-specific code and make things a lot more consistentBig expansion on included starting examples, as well as better documentation and tutorials.Code cleanup, including making sure files are adhering to the coding guidlines, and removing a lot of unwanted cruft, excessive includes which drag down compile times, and not-errors-but-not-right code which can cause bugs and irregular behavior, which should help stability of the engine across the board. I think that covers the major highlights of what the plan for 4.0 covers. I said earlier I'm working on a proper Roadmap/Master Thesis breakdown of the plan for 4.0, and thankfully while that list looks daunting, a huge amount of the above is either actually done, or rather far along already, just waiting for us to get 3.10 out the door so we can start merging stuff in.Lemme know if you have any other questions ;) Link to comment Share on other sites More sharing options...
Happenstance Posted January 2, 2017 Share Posted January 2, 2017 @JeffR - I'm in the same boat as HeadClot so I appreciate the overview of all of the improvements in 3.10 & 4.0.I'd like to start a new project with T3D using 3.10 but I'm concerned the transition to 4.0 will be a hassle. Is the new base template (the one that's making use of the module & asset system) on git somewhere? I'd like to use 3.10 now but with the new template so I have at least some head start on the transition. Link to comment Share on other sites More sharing options...
JeffR Posted January 2, 2017 Author Share Posted January 2, 2017 Sure!https://github.com/Areloch/Torque3D/tree/BaseGameTemplateThat's the current build of it. It's a bit behind to devhead, but i plan on updating it soon. It has a module that contains all the current Full template's fps/gameplay stuff, so if you started working on something now, you could port it over to that module without a huge amount of hassle.If you wanted to look at how the assets and entity/component stuff works, you can nab my R&D branch https://github.com/areloch/Torque3d/tree/RnDBuildWhere I do all the testing and building for the modules/assets/entity-component and all that jazz. If it's in my workblog posts, it's in the RnD branch.I do plan to put together documentation explaining how you can port to 4.0(while existing game classes like ShapeBase, Player, Item, etc will be deprecated, there will be entity/component versions that come with 4.0 that do the same thing, so you don't need to start from scratch or anything, merely port over).Let me know if you had any specific questions :) Link to comment Share on other sites More sharing options...
Happenstance Posted January 3, 2017 Share Posted January 3, 2017 That's great. Thanks! I'm going to look over everything this week in more detail but the quick glance I had at the template code has me excited for 4.0. Link to comment Share on other sites More sharing options...
JeffR Posted January 3, 2017 Author Share Posted January 3, 2017 Good to hear!Over the next day or two, I'm gunna go in-depth on the particulars of the assets, entity/component and game object stuffs in my workblog thread, so feel free to keep an eye out there for more details on how this will all get utilized :) Link to comment Share on other sites More sharing options...
HeadClot Posted January 4, 2017 Share Posted January 4, 2017 Hey @JeffR Thanks for the overview on 3.10 RC and 4.00.Is there any where I can put a feature request for T3D?I do not want to post on the T3D Github and get in trouble. Link to comment Share on other sites More sharing options...
JeffR Posted January 4, 2017 Author Share Posted January 4, 2017 Haha, no worries!You can either make a thread here on the forums, or you can make an issue on the github page, just noting it's a feature request so it's easier to organize.You needn't worry about getting in trouble for suggestions :) Link to comment Share on other sites More sharing options...
elfprince13 Posted January 5, 2017 Share Posted January 5, 2017 Just wanted to chime in and say thanks for all the hard work. OS X support is a huge moment, and I might finally come back into the Torque fold after GG screwed the pooch on the T3D 1.1 release way back when. Link to comment Share on other sites More sharing options...
HeadClot Posted January 6, 2017 Share Posted January 6, 2017 Haha, no worries!You can either make a thread here on the forums, or you can make an issue on the github page, just noting it's a feature request so it's easier to organize.You needn't worry about getting in trouble for suggestions :) Thanks for the clarification and I made the feature request. https://github.com/GarageGames/Torque3D/issues/1917 Link to comment Share on other sites More sharing options...
chriscalef Posted January 12, 2017 Share Posted January 12, 2017 Hey, what's the procedure for making a DX11 build? I did a fresh update, and noted that TORQUE_D3D11 was checked in CMake, but when I tried to compile, it failed looking for D3D9.h. Do I need to uncheck DIRECTX at the same time as checking TORQUE_D3D11? (Testing that...)EDIT: Nope, that didn't fix it either. Do I still need the DX9 SDK in order to build an exclusively DX11 executable? Link to comment Share on other sites More sharing options...
JeffR Posted January 12, 2017 Author Share Posted January 12, 2017 Currently, yeah, the older SDK is required. Ironically, not because of DirectX, but because of how MS decided to hatchet-job the audio dependencies with XAudio.We're looking at how to resolve that what with dropping DX9 shortly after 3.10 goes out, but due to the knuckleheaded way Windows deals with the audio stuff, supporting XAudio at the moment with pre-8.1 windows necessitates the DX SDK. Link to comment Share on other sites More sharing options...
Timmy Posted January 13, 2017 Share Posted January 13, 2017 Hey, what's the procedure for making a DX11 build? I did a fresh update, and noted that TORQUE_D3D11 was checked in CMake, but when I tried to compile, it failed looking for D3D9.h. Do I need to uncheck DIRECTX at the same time as checking TORQUE_D3D11? (Testing that...)EDIT: Nope, that didn't fix it either. Do I still need the DX9 SDK in order to build an exclusively DX11 executable? Yes you still require the old june 2010 dx sdk unfortunately. Both the GFX D3D9 device and SFX XAudio device both depend on it. Just a small correction on the above, the xaudio thing is correct but also the current implementation of D3D9 gfx makes use of d3dx which MS removed when they released the windows sdk, since D3D9 is getting removed in 4.0 it made no sense to waste any time making the D3D9 gfx device work without d3dx (which is entirely possible to do). In 4.0 xaudio will be an opt in feature (i.e disabled by default) so by default the old june dx sdk will not be required unless you compile in xaudio support.The problem with xaudio is version 2.7 requires the old june 2010 dx sdk and the windows sdk only supports xaudio 2.8+, Windows 7 only supports xaudio 2.7 and not 2.8. That presents a bit of a nightmare because xaudio 2.7 code is not completely compatible with 2.8, it requires changes to compile. So if we support only 2.7 we need to also support the old june 2010 dx sdk and if we only support 2.8 (with the windows sdk) than it won't work with windows 7. There are actually ways around this but it's not very nice, MS describe one way here using a hard coded path to the old june 2010 dx sdk https://blogs.msdn.microsoft.com/chuckw/2012/04/02/xaudio2-and-windows-8/ for windows 7 builds. That is a nasty solution because it's a compile time solution and if you want to support windows 7 still (i'm sure most of us do), than you are stuck using the old june 2010 sdk. Hopefully all that makes sense. We have actually updated to using openal-soft recently so hopefully that suits everyone's sound needs ok and this xuadio thing is a total non issue ;) Link to comment Share on other sites More sharing options...
chriscalef Posted January 13, 2017 Share Posted January 13, 2017 Huh. Very interesting. Thanks again for all your hard work, Steering Committee! Speaking as just one T3D grunt out here on the front lines, I'd be completely screwed without you guys!!Just to clarify though - if I hit the DirectX11 checkbox in CMake, and then compile it with the June 2010 SDK (which is no problem, btw, I don't mind a bit if it's just a compile issue) does that mean my build will run on an end user's machine without them having to download the DirectX 9 runtime?This issue came up for me because I'm just starting to test my build outside my own machines (YAAAY!!!) and the second friend I had download it came up missing X3DAudio1_7.dll and D3DCOMPILER_43.dll. I've been assuming that was because I had a D3D9 build and he's never downloaded it. Is it possible that I just need to include those two dlls with my build? Or was my initial assumption correct and I need to build in D3D11 support, and then end users will not have to download anything because 11 is included in Windows?Also, is it possible to support 9 and 11 at the same time in the same build, so users with older machines can still run it? Link to comment Share on other sites More sharing options...
Timmy Posted January 13, 2017 Share Posted January 13, 2017 Both D3D 9 & 11 require D3DCOMPILER_43.dll in T3D and yep D3D 9 & 11 can live happily in the same project, in fact currently there is no way of compiling out D3D9, so if you hit the D3D11 tick box in cmake you are including both. You can always double check with http://www.dependencywalker.com/ the exact dll's required. Link to comment Share on other sites More sharing options...
chriscalef Posted January 14, 2017 Share Posted January 14, 2017 Ah! Awesome, thanks for the info! Link to comment Share on other sites More sharing options...
chriscalef Posted January 14, 2017 Share Posted January 14, 2017 Does anybody know what's up with X3DAudio1_7.dll? I'm including it anyway, it's tiny, but is it from windows or directx? Link to comment Share on other sites More sharing options...
Happenstance Posted January 15, 2017 Share Posted January 15, 2017 What's the status of the OpenGL layer on Windows? Would it be possible to rip out the DirectX layer (D3D9, 11, and XAudio) and just have a build that uses OpenGL and OpenAL on Windows? Am I correct in assuming the SDK wouldn't be required in that case? Link to comment Share on other sites More sharing options...
JeffR Posted January 17, 2017 Author Share Posted January 17, 2017 In the current build, D3D9 is the default graphics option and there isn't a simple flag to disable it, currently. Going forward to 4.0, 9 will be dropped in favor of 11(with feature support for 10).So you'd need to disable some stuff in your project to make it agreeable on the win-side without DirectX, but as Linux and MacOS are both GL-only and run fine, yes, you could go pure OGL if you wanted with a bit of fiddling. Link to comment Share on other sites More sharing options...
JeffR Posted January 24, 2017 Author Share Posted January 24, 2017 Hey guys, I think we're in the final stretch on this.I'm trying to shore up 2 noted SDL issues(window edge snapping is weirdly offset, and some modifier key shenanigans).If you guys could test these PRs and note any issues that crop up, that'd let me get them merged in so we can try getting a wrap on this:https://github.com/GarageGames/Torque3D/pull/1933https://github.com/GarageGames/Torque3D/pull/1932https://github.com/GarageGames/Torque3D/pull/1931https://github.com/GarageGames/Torque3D/pull/1928 Link to comment Share on other sites More sharing options...
TheMartian Posted February 3, 2017 Share Posted February 3, 2017 Not sure how useful this is because I used the latest project manager v2.2 but I figured I would post it just in case.Grabbing this 3.10rc code drop, unzipping, dropping in the built project manager, create a game using the full template, all default selections (maybe add minidump support)Open in visual studio 2012 on a win10 machine and attempt to compile and link. Here is what you need to do to make it work.In the DLL project.......Cannot open include file nfd.hneed to add ................./Engine/lib/nativeFileDialogs/include to all configurations (prepend with where ever you have the files located)Cannot open Al/Al.hneed to add ................/Engine/lib/openal-soft/include too all configurations (also prepend with where ever you have the files located)Error 50 error C2039: 'smConnectionNotify' : is not a member of 'Net' \engine\source\platform\test\nettest.cpp 79Need to change the net::smConnectionNotify.notify...... to Net::getConnectionNotifyEvent().notify....for all instances.....andNet::smConnectionReceive.notify to Net::getConnectionReceiveEvent().notifyandall the ones that are .receive as well that are in the same fileif it says notify then its getConnectionNotifyEvent() if it says receive it is getConnectionReceiveEvent()Linking problem Error 80 error LNK2019: unresolved external symbol "public: void __thiscall Component::setOwner(class Entity *)" (?setOwner@Component@@QAEXPAVEntity@@@Z) referenced in function "public: bool __thiscall Entity::addComponent(class Component *)" (?addComponent@Entity@@QAE_NPAVComponent@@@Z) .............................................\buildFiles\VisualStudio 2012\projects\entity.objIn the DLL project you will see that under T3D folder that the components folder and code is missingcreate components folder, add the existing files to it.then add all the other missing folders in that same location under the components folder from that directory (animation, camera, collision, game, physics, and render)Engine -> T3D -> components -> animationEngine -> T3D -> components -> cameraEngine -> T3D -> components -> collisionEngine -> T3D -> components -> gameEngine -> T3D -> components -> physicsEngine -> T3D -> components -> renderthe folders and code are already in the source code, they are just not in the DLL project itself. so you have to create those filters and add the existing code.Then it compiles, links and runs just fine. (platform windows obviously : )A quick check and any other settings (like select opengl, or other modules, etc all have the same problem, or more problems).Maybe that will help someone, could just be project manager issues, but the PM is still handy for creating projects with solution files. Link to comment Share on other sites More sharing options...
Recommended Posts