64 bit executable

Place bug reports / questions here.

Moderators: Balthagor, Legend, Moderators

Post Reply
DuxMare
Warrant Officer
Posts: 37
Joined: Mar 09 2014
Human: Yes

64 bit executable

Post by DuxMare »

Recently I was playing a game that lasted a little while, but didn't get to the 21st century. I started to notice a problem with the game slowing down considerably, even at slow game speeds. I noticed that units would pause for a little while then move, then pause again, and the pattern would continue. So much so, that it really started to bother me and effect game play. I started to wonder what the resources were that the game was consuming. So, it looked like the game was taking 1.7 GB of memory. So then I noticed that the pauses corresponded to the process writing and reading from disk. Why would the process be writing and reading so much from disk? Well, at 1.7 GB of memory consumption there's now a good chance that it's exceeded it's memory address space for a 32 bit executable (limit is 2GB in actual RAM), which apparently the Supreme Ruler exe process is running under, and so is now using swap space. |O |O |O

So, is there a command line argument that allows me to run the Supreme Ruler process in a 64 bit address space so I'm not getting slowed down by reading and writing to pagefile.sys?

If there isn't a way to run it under a 64 bit address space, how hard would it be to actually be to compile the code as a 64 bit executable?

I've got lots of RAM and I'd love to be able to use all of it. Processor capacity isn't taxed at all, in fact no other system resources hardly seem to be used. It's the Disk I/O that seems to be killing me, and it's basically a virtue of being a 32 bit exe.

Any help you can give right now would be appreciated.
evildari
Brigadier Gen.
Posts: 629
Joined: Aug 10 2017
Human: Yes

Re: 64 bit executable

Post by evildari »

Maybe SRU is not "large address aware" - there were many games back in 2010 that got such issues and were resolved by using a tool that set the executable to be large address aware (use more than 2gb ram on a 64bit OS) - later developers just set the right flag in the compiler. I dont know if its the same old problem with SRU.

SRU doesnt seem to use my disabled swap file - if you have it enabled you may check the ressource manager if it is written to (doesnt need to be SRU - could be system or svchost too)

The thing i observed is that the gui-interfaces and sat.imagery is reloaded although i got 8GB ram on one pc and 16 gb on another. At least the gui should fit into the ram...
my mods
http://www.bgforums.com/forums/viewtopi ... 79&t=25932 (even techs and units for everyone - AI will own you too)
http://www.bgforums.com/forums/viewtopi ... 79&t=29326 (MARSX2)
Nerei
General
Posts: 1354
Joined: Jan 11 2016
Human: Yes

Re: 64 bit executable

Post by Nerei »

First off a few quick notes: 32bit computing allows the usage of 2^32 or 4294967296 memory addresses total. This translates to a hard limit of 4GB and is why a 32bit OS or program cannot use more than 4GB of memory total. That includes all memory usage no matter where or what type it is.
A program exceeding its memory address limit throws an exception and crashes in what is typically know as an OOM error. Again to a program it is entirely irrelevant if the memory it is using is in your RAM, swapspace or anywhere else. It is still taking up memory addresses which is really what matters.

The 2GB pr application limit is set by the OS. LAA as mentioned above allows the program to use this entire range on 64bit (and 3GB on 32bit windows). Personally I would be surprised if there are any non LAA 32bit programs being released today but it is possible.

Chances are this is windows being stupid. It has a bad tendency to write into the swapspace despite actually having free space on your RAM (it might have changed since 7 though I do not know). It is one of the reasons why some people disable windows swapspace. It is a performance boost with the risk of the OS throwing a BSOD if it runs out of RAM.


I think I have broken 2GB before so I would assume the game is LAA. I know I have been extremely close to it many times so if it is not I would be quite surprised as I have not had any OOM crashes. There is a roughly 1MB program that you can download that will both allow you to check this and set that flag if needed. I am on a mac right now so I cannot really do any checks myself.

Also there is no way of running a 32bit application as 64bit short of rewriting it to be 64bit and it tends to be a major project to do so. I suspect we are talking a $30 expansion if not more just to get that feature and given that LAA exist I doubt it would be worth it. I actually doubt it would be financially viable in any case LAA or not.
You can really consider the fact that it has not happened already as proof that it is not financially viable as sticking with 32bit is today just a byproduct of the age of the game. It made sense back when 2020 was released but today 32bit OS is basically history.

There are other possibilities for what is causing the slowdowns. One would be the AI updating something like say pathfinding. It has a tendency to march units in circles in strange places which can really cause slowdowns I have found. Only BG can really answer this though.
evildari
Brigadier Gen.
Posts: 629
Joined: Aug 10 2017
Human: Yes

Re: 64 bit executable

Post by evildari »

https://en.wikipedia.org/wiki/Windows_X ... 64_Edition
released in 2005 !

I used that on an AMD 64bit cpu.
Back then starforce copy protection was the hype - didnt work initially with 64 bit - 2 or 3 days after writing to the developers about this, they sent me a fresh new 64bit compatible driver, that worked flawless.
I remember this because it was the fastest solution i received for imo a less than easy problem. But maybe the conversion from 32bit to 64bit is easy...
And we are talking about a close to hardware driver...
(Game was Space Rangers 2 that used it - released a few months later than winxp64 - its awesome)
my mods
http://www.bgforums.com/forums/viewtopi ... 79&t=25932 (even techs and units for everyone - AI will own you too)
http://www.bgforums.com/forums/viewtopi ... 79&t=29326 (MARSX2)
User avatar
Balthagor
Supreme Ruler
Posts: 22082
Joined: Jun 04 2002
Human: Yes
Location: BattleGoat Studios

Re: 64 bit executable

Post by Balthagor »

DuxMare wrote:...I started to notice a problem with the game slowing down considerably, even at slow game speeds...
Could you please tell us how long in real time one game day took to complete?

Did you quit and reload the game? If you've been playing for a while, restarting the game can help.
Chris Latour
BattleGoat Studios
chris@battlegoat.com
DuxMare
Warrant Officer
Posts: 37
Joined: Mar 09 2014
Human: Yes

Re: 64 bit executable

Post by DuxMare »

Ok, so I restarted the game multiple time and each time get the same result.

On slow speed, one day takes approximately 3 minutes to complete.

I like the idea of turning disabling swap space while playing the game, I will give that one a try and see how it goes.

One thing I also noticed is that as I pan around on the map the amount of disk I/O now seems to increase.

When compared with starting a brand new game, after playing the saved game but not killing the process, the cpu usage seems the same, the amount of ram consumed seems to be about the same amount, the number of threads used by the process seems to be the same, the only difference seems to be the amount of disk I/O going on, but the difference seems slight. Now, I'm still using spinny disc hard drives instead of SSD, so that's going to add some latency. However, now there is no unit traversal pausing at all.

The difference in performance between a new game vs a game that's been going on for a while is highly noticeable especially in the GUI. At this point, I'm trying to figure out what the difference is and what could cause the unit movement pauses.

Does anyone else notice this problem? Is it just me?

Edit: I suppose I should list my system stats: AMD Athlon II x 4 645 Processor (4 CPUs), ~3.1GHz with 25GB RAM and a NVIDEA GeForce GTX 570 graphics card (with 4049 MB of memory).
User avatar
Balthagor
Supreme Ruler
Posts: 22082
Joined: Jun 04 2002
Human: Yes
Location: BattleGoat Studios

Re: 64 bit executable

Post by Balthagor »

DuxMare wrote:On slow speed, one day takes approximately 3 minutes to complete...
Sorry for not indicating this, but could you say how long it takes on the fastest speed setting?
Chris Latour
BattleGoat Studios
chris@battlegoat.com
DuxMare
Warrant Officer
Posts: 37
Joined: Mar 09 2014
Human: Yes

Re: 64 bit executable

Post by DuxMare »

Balthagor wrote:
DuxMare wrote:On slow speed, one day takes approximately 3 minutes to complete...
Sorry for not indicating this, but could you say how long it takes on the fastest speed setting?
Ok, so I just timed it, and on the fastest setting the time to complete a day takes approximately 30 seconds.

I ran it on the fastest setting for a while and the unit pausing seemed to diminish but did not go away entirely.
User avatar
Balthagor
Supreme Ruler
Posts: 22082
Joined: Jun 04 2002
Human: Yes
Location: BattleGoat Studios

Re: 64 bit executable

Post by Balthagor »

And how many game years in are you? In late game, 20sec/day is considered "good". Your not in a range that I'd usually consider to be that slow. That would give you 3h per game year. Ideally I'd like us to keep it from getting over 2h per game year, but there are limits to what can be optimized when the game itself has so few limits on unit quantities.
Chris Latour
BattleGoat Studios
chris@battlegoat.com
GIJoe597
Board Admin
Posts: 2918
Joined: Sep 29 2008
Human: Yes
Contact:

Re: 64 bit executable

Post by GIJoe597 »

It is 4 am and I was curious. I ran the following test; I let the game run for 15 days on Fastest settings and took average time. I used USA because that was who I have saves for at various times.

2020 Global Crisis, no settings changed.

Started as USA no minister changes, all left at default, ai is running the show. Let the game run on fastest speed for 15 days.
Average speed per day 3.30 secs

Started as USA, my usual settings, ministers off, except resources.
Average speed per day 3.45 secs (I attribute the slighter slower time to the research and military production labs being full on game start, as I always do. Takes ai a bit to get it all full)

USA World 2020 6 years into the game, 2026, Feb is the current date. Ai wars in Europe and Asia. USA not at war.

Average speed for per day 7.29 secs


Machine specs
Intel i7-4790K @4.0GHZ
16GB Ram
64 Bit Windows 8.1 Pro
NVIDIA GeForce GTX 980
https://www.youtube.com/user/GIJoe597


Older/retired gamers, who do not tolerate foolishness.
http://steamcommunity.com/groups/USARG
YoMomma
Brigadier Gen.
Posts: 768
Joined: Jun 27 2015
Human: Yes
Contact:

Re: 64 bit executable

Post by YoMomma »

Me thinks this topic is about 64 bit exe and still looking for the answer from devs how hard would it be to implement or are you just incapable of doing so?

My old computer with 4 GB ram got basicly blown up playing this game, first i got crashes and then motherboard burned down. The crashes just happened playing this game. I was playing another heavy game Ark survival evolved without much problems.

Now i have other computer, but yeah....
Gameplay 1st
User avatar
Balthagor
Supreme Ruler
Posts: 22082
Joined: Jun 04 2002
Human: Yes
Location: BattleGoat Studios

Re: 64 bit executable

Post by Balthagor »

It was discussed years ago. IIRC the comment was that it would not actually provide much of a performance increase. George would need to address it from a technical perspective. From a business perspective, if it won't grow the game community then the resources would be better spent on other priorities. A map editor would create more community growth than "now in 64bit!". I'm confident if the performance difference was significant we would have pursued it years ago.
Chris Latour
BattleGoat Studios
chris@battlegoat.com
YoMomma
Brigadier Gen.
Posts: 768
Joined: Jun 27 2015
Human: Yes
Contact:

Re: 64 bit executable

Post by YoMomma »

OK i trust your judgement. Well atleast you have work to do. Many people in the world are looking for work but have to risk their lives to live as an alien in a foreign country.
Gameplay 1st
Nerei
General
Posts: 1354
Joined: Jan 11 2016
Human: Yes

Re: 64 bit executable

Post by Nerei »

If memory serves me right 64bit windows is emulating 32bit when running a 32bit program while it runs 64bit natively so there is a performance to gain from going to 64bit.
However if it is just for the sake of performance then upgrading the game engine to run 64bit is probably one of the least efficient things to do. Windows is very good at what it does so it will be a lot of resources wasted for little gain. For performance I would say focusing on the AI will be significantly better and it might give us other benefits such as France and the UK not sending their troops on suicide missions in Russia going to Vietnam.
The real gain from going from 32bit to 64bit is that we go from being able to use 4GB memory to whatever limit windows sets for us (which can be anywhere from 8GB to 2TB depending on version. Servers not included).
DuxMare wrote:Edit: I suppose I should list my system stats: AMD Athlon II x 4 645 Processor (4 CPUs), ~3.1GHz with 25GB RAM and a NVIDEA GeForce GTX 570 graphics card (with 4049 MB of memory).
The main issue I see is the CPU. It is a roughly 7 year old model which is good enough in many cases but unfortunately this game is CPU intensive. I would suspect your CPU is a likely bottleneck.
According to BG the game can run multithreaded effectively on up to 4 cores meaning in some cases you might be sharing cores with say windows doing its thing which again will reduce performance.
Your graphics card while not new should be up to the task as that is one point where the game is fairly simple. The is no light sources of fancy special effects that needs to be rendered and the models themselves are quite low detail.
Switching your HDD to a SDD might also be worth considering. It will reduce read latency but just as much it will make many other things such as startups of your computer so much faster so consider it a quality of life thing too.
EVILDARI wrote:https://en.wikipedia.org/wiki/Windows_X ... 64_Edition
released in 2005 !

I used that on an AMD 64bit cpu.
Back then starforce copy protection was the hype - didnt work initially with 64 bit - 2 or 3 days after writing to the developers about this, they sent me a fresh new 64bit compatible driver, that worked flawless.
I remember this because it was the fastest solution i received for imo a less than easy problem. But maybe the conversion from 32bit to 64bit is easy...
And we are talking about a close to hardware driver...
(Game was Space Rangers 2 that used it - released a few months later than winxp64 - its awesome)
I am not an expert on drivers and with the information you have given it is a bit hard to say anything for certain. However with the short time spend on solving it is possible that it is either a very isolated problem or it is fairly simple to solve.
As for it being a 32bit to 64bit port of drivers that is actually hard to say. It could be an adaptation of XP 64bit (released around 2002) to XP pro 64bit. It could also be a minor flaw with drivers already made with XP pr 64bit drivers.
It might be as you said that it is just simpler to port. As I said I do not know how drivers for 32bit XP differs from their 64bit counterparts.


The difference is that porting an engine will likely produce hundreds of not thousands of bugs. They majority will be simple to handle but they will still require effort to at the very least look over.
It is not just about complexity but the sheer time required. You might be doing simple tasks but it is still tasks that has to be done and ultimately cost money to get done.

Porting a game over to a different architecture is not easy. I have yet to find any game developer that will tell me this. I am fairly certain mr Geczy is one of those that said it was not economically feasible for a game this late in its lifecycle.

As for a practical example of how much of a mess such a port can be I would bring up how Squad tried to port Kerbal Space Program over to 64bit windows in Unity 4.
Keep in mind this was one of the main requested features as KSP is a heavily modded game and Unity preloads just about all content putting greater demand on memory and making modding it a game of memory math when running the 32bit version. Also in theory most of the work would already be done by Unity Technologies a company around a few hundred times larger than BG by employee numbers.

Long story short they failed. The 64bit version they produced where so buggy that some modders outright refused to support it to the point where they had their mods disable themselves simply to avoid the grief that would be thrown their way due to bugs in the core game.
In the end Squad killed off the 64bit windows executable until they ported the game to Unity 5. They actually, if temporarily gave up on implementing one of the most requested features due to issues with porting the game over.

During the same time both the Linux versions where fine as where the 32bit windows version so it really was an issue with the 64bit windows version.
Last I checked Unity had still to make a 64bit version for Mac.
DuxMare
Warrant Officer
Posts: 37
Joined: Mar 09 2014
Human: Yes

Re: 64 bit executable

Post by DuxMare »

Ok so to clarify, yeah the goal of the post was to talk about getting access to more ram since it looked like my swap space was doing work and units were "pausing" obnoxiously while moving around.

The game wasn't really running slowly from a time passing perspective - and that is something y'all have definitely improved on. It was a few years ago, late games did slow down quite a bit.

In response to Nerei, in general, processor power has plateaued or is in process of plateauing. Processors seem to not be squeezing more processing power over time but instead seem to be getting better in other ways, like power consumption. So, getting a newer processor will help but probably isn't going to help all that much I think, unless maybe I get something that has 8 or 16 cores. Of course, it's not all about the processor, it's also about how the Operating system handles threads, processes, context switching between them and sharing disk I/O, and so to clarify, I'm also running Windows 7 and not Windows 10.

Now, that being said, I started to play the game a bit more and the unit "pausing" started to go away after I ran the game on the fastest time setting for a while. While that was happening I noticed no other processes changing what they were doing very much in terms of their resource consumption. So it appears (but of course I can't say for sure because I'm not running a profiler) that whatever the problem was it might have been internal to the game.

I am in agreement with Nerei as well regarding things to hit on to get a performance gain, especially with respect to the AI. But maybe those things aren't as easily monetizable, maybe the more flashy things are more monetizable. It is easy to package and sell new content, and some people are going to see it as very mercenary, however, I can intuit that to improve on the game battlegoat has to keep their doors open and salaries paid. I know for sure I will buy the map editor when it comes out.
Post Reply

Return to “Issues and Support”