Gameplay Is Laggy To Hell

[Steelpoint]

I've been on a break from /tg/ station and have been playing on other servers, when I logged in earlier tonight I noticed how damn laggy the game is.

I don't think even Goonstation has this kind of lag. The lag being the movement is so jittery and slow. Perhaps it's the FPS or some other thing but I find it really jarring.

Maybe I've just been away too long, but its something I noticed very quickly.

[Akkryls]

NO REVERTS EVER

But seriously, the server has been getting worse and worse over the past few months, it really needs something to be done.

[Steelpoint]

Some people tried to fob it off as 'the experience' but its seriously a bad thing. I've said this before several months ago but this game is just getting more jittery and laggier in movement as time goes on.

Eventually the damn game will be rendering at one frame a second.

I really wonder why this is a issue for us, yet few if any other servers have this kind of movement issue at all.

[Incoming]

Some people tried to fob it off as 'the experience' but its seriously a bad thing. I've said this before several months ago but this game is just getting more jittery and laggier in movement as time goes on.

Eventually the damn game will be rendering at one frame a second.

I really wonder why this is a issue for us, yet few if any other servers have this kind of movement issue at all.

High intake of new features + People/Maintainers who don't test/review code thoroughly + the magics of Byond = lag

Edit: The best thing the player base can do to help with lag is to point out when it gets worse AS SOON AS IT'S NOTICED. It really helps pin down which features could be causing it. If you only say "in the last few months" that means some poor bastard will be looking through EVERYTHING instead of just one or two very recent changes.

[Pybro]

The Disco effect Admins sometimes spawn in the bar gives me FPS lag out the ass. So do large amounts of those lit floors blinking in unison.

[Not-Dorsidarf]

List of usual suspects for lag:

1) New lighting system

2) Singulo Updates

3)fastmos

[Akkryls]

List of usual suspects for lag:

1) New lighting system

2) Singulo Updates

3)fastmos

I've been bitching about fastmos since it was put it, since that's when I started noticing serious lag spikes and I've been told several time either that IT'S NOT FASTMOS, or that nobody knows what it is.
We really should just spend a weekend reverting the server until we find out what shit the bed.

[Cheimon]

List of usual suspects for lag:

1) New lighting system

2) Singulo Updates

3)fastmos

Wasn't the main problem with singulo that it kept forcing recalculations of power and pipe nets? Then that got fixed, then it broke again?

[Remie Richards]

List of usual suspects for lag:

1) New lighting system

2) Singulo Updates

3)fastmos

Wasn't the main problem with singulo that it kept forcing recalculations of power and pipe nets? Then that got fixed, then it broke again?

Nobody fixed power and pipe network recalculations.
and yes, that's a large chunk of singulo/bomb lag.

[CPTANT]

List of usual suspects for lag:

1) New lighting system

2) Singulo Updates

3)fastmos

Wasn't the main problem with singulo that it kept forcing recalculations of power and pipe nets? Then that got fixed, then it broke again?

Nobody fixed power and pipe network recalculations.
and yes, that's a large chunk of singulo/bomb lag.

pretty sure someone recently made a PR that supposedly fixed it again.

edit: no wait that was the hard deleting of pipes causing lag. Pipe destruction should schedule a network redraw on the tick before atmos processing instead of proccing on every pipe destroyed.

[Incomptinence]

A lighting overhaul again? Fucking christ like we needed in in a game which gives you literal eyes in the back of your head so realistic damn it.

[oranges]

time for the TEG REPLACEMENT PULL TO RISE AGAIN

[Saegrimr]

A lighting overhaul again? Fucking christ like we needed in in a game which gives you literal eyes in the back of your head so realistic damn it.

Considering lighting is one of the more expensive systems running, it wouldn't hurt.

[Aranclanos]

How it works in simple words: higher number on 'self CPU' means more processing time, aka lag. In this profile atmos generates 15 times more lag than anything else. (process_cell, share, share_air and archive are procs from environmental atmos)
This is a profile of a couple of rounds I just took, a few rounds, 4 or 5.
Usually when I check the profile that has been running for some time, it points the same things.
When I'm logged and it starts lagging I usually start a new profile, lately it mostly points to atmos, at the start of the month there were some spikes regarding movement, phil changed some throw code and ended up lagging a lot under certain conditions, but I believe he mostly tackled those issues by now, I haven't seen lagspikes pop out from there in a while.

Lately if the server is really lagging, it can be easily blamed on atmos to be honest. I made the current atmos system, but it aws based on certain speed, who was increased blindly a few months ago by your server host, who screamed me to fuck off when I told them to don't do it. My system was not changed in any way, it just runs more often with zero optimization.

So yeah, maybe he made it more laggy to ask for more donations to get better servers, welcome to a new era of a jew server host who actually keeps the money for himself. Remember to buy the shirts and deactivate addblock. Oh and you can't switch host because he controls both server and page, I told you to don't do it adminbus, but whatever. I'll just stay loyal to the code project that he can't really touch.

[ABearInTheWoods]

Profiles tell you nothing about lag, just about cpu usage, with the assumption that it will lead to more understanding about lag.

Process cell can use as much cpu as it wants, if it's only doing it when the server isn't busy, it doesn't matter for the purposes of lag.

That being said, this is all easy to test.

I will come on randomly over the next few days, and either reduce, raise, or no change, atmos processing speed, and run votes at the end of the round.

Also: fastmos came about in april, all this lag everybody has been talking about they say is over the past few months, that puts it in june territory

June, June, OH YA! that right about the time traffic started increasing:

[Akkryls]

Profiles tell you nothing about lag, just about cpu usage, with the assumption that it will lead to more understanding about lag.

Process cell can use as much cpu as it wants, if it's only doing it when the server isn't busy, it doesn't matter for the purposes of lag.

That being said, this is all easy to test.

I will come on randomly over the next few days, and either reduce, raise, or no change, atmos processing speed, and run votes at the end of the round.

Also: fastmos came about in april, all this lag everybody has been talking about they say is over the past few months, that puts it in june territory

June, June, OH YA! that right about the time traffic started increasing:

Completely ignoring we used to run with 100 plus players with less lag than we're getting now.

[PKPenguin321]

yo, the passive aggression is real

but yeah i have an idea what if we scaled down atmos speeds when there are more players, if server population and atmos speed are the source of the lag then that would help a ton wouldn't it
or am i being asinine

[ABearInTheWoods]

There are other things going on here...

The lag corresponds with a lot of things, one of which is ovh's network link going down for a few days and them having to patch in new connections to the trunk lines.

I've noticed the lag since that happened, because even remote desktop to the server isn't anywhere near as snappy as it used to be.

This also corresponds with clients starting to use byond version 508, (we didn't bring in 508 for the server until late july) has anybody tested with 507 that is using 508?

It could also be the use of 508 on the server, but that was late july.

yo, the passive aggression is real

but yeah i have an idea what if we scaled down atmos speeds when there are more players, if server population and atmos speed are the source of the lag then that would help a ton wouldn't it
or am i being asinine

Atmos speed automatically scales down based on how busy it and the other subsystems are. Like when the singulo gets released atmos scales down to way worst then its old speed. This was the entire point of fastmos, that it could do this, and thus didn't need to run at a slow speed all the time to account for the times when the server is busy.

[CPTANT]

There has to be a better way to simulate atmos than to manually calculate the flow from each turf to each turf every atmos tick......

[Steelpoint]

There has to be because the alternative is a laggy game.

Even if we have to revert our entire atmo system, something has to change. As the lag we get in game is a deal breaker where no other server has this problem (at least a major server).

[Steelpoint]

Here's the interesting thing I found out.

I played around on my local offline code and I tried changing the FPS to 15 (from 11) and that suddenly got rid of the jiggery movement.

I'm no expert but what would be the negative effects of raising the FPS from 11 to 15?

E: To be fair this is more the movement being shitty than the actual lag the game has. Playing on the server is a very bad experience in comparison to playing on a local server, but in both iterations the actual movement is crap and not smooth at all.

[Steelpoint]

Well we tried 15 FPS for a minute, it was interesting but the arrival of a meteor storm made the admins turn it back to 11.

[DrPillzRedux]

Pull a Goon and remove atmos completely besides gas spread from canisters. Are some pipes worth all the lag?

[Steelpoint]

If Aran's correct then atmo accounts for a massive chunk of the lag issues in game.

To be honest the only thing we would miss if Atmo was removed/heavily simplified is the lack of plasma fires from the AI. Most atmo systems are completely unknown or irrelevant to people.

[CPTANT]

I wouldn't really want atmos completely gone, but it can be vastly less accurate. Do some simplifications. Combine blocks of 4 adjacent air holding turfs into on block with one pressure/temperature for example. The pressure difference between 2 adjacent turfs is negligible for gameplay purposes. Fill in the gaps with normal 1 turf blocks in places it doesn't fit.

Or of course some other form of simplification.

[Jalleo]

MSO we may have to move providers then.
Apart from this calls to remove fastmos wont stop the lag. I am hoping we get a net rework on how they are made and reconstructed. This is the cause of the lag. Another can bad qdel stuff of which phil has recently discovered one where a item went into a infinite loop. Hopefully the lag will subside but atmos is not the direct cause. I just hope byond gets better network code.

[Saegrimr]

I just hope byond gets better network code.

Theres wishful thinking, and then there's praying byond improves.

Pull a Goon and remove atmos completely besides gas spread from canisters. Are some pipes worth all the lag?

Do they not have air vents and shit like that? Sounds really bad.
I could see removing the pipe network and just having controls in Atmos itself that centcom-magic link themselves to the vents. Unfortunately that removes a fair bit of creativity in piping.

[Amelius]

Everything was FINE like, a year ago before coderbus said 'well, now it's no longer laggy and we have a more powerful server, so we can do more overhead! Woo fastmos, singularity item collisions, a new lighting system, etc. etc. etc.'

'Oh no, now it's crazy laggy all the time, I wonder what caused it?'

Roll some of that shit back, christ.

[DrPillzRedux]

No saeg they don't have air vents or pipes. They have air outlets that generate air in large areas. It sacrifices atmos fuckery for a lot of lag reduction. Fires on goon are far more deadly than on tg to compensate for this. If an area is destroyed and air is lost, you pass out within 10 seconds as well.

[ABearInTheWoods]

How many times do i have to say this..

Atoms/fastmos is not the cause of the lag.

[invisty]

How many times do i have to say this..

Atoms/fastmos is not the cause of the lag.

I mean you're dealing with comp-sci level stuff here. It would help if you try and explain why atmos processing has such little effect on the game.

I would imagine the biggest culprits would be the #players and network state?

[oranges]

believe me it used to be way way worse.

[Incomptinence]

Yeah then it got better now it is getting worse again.

[Cheimon]

Yeah, the reason people are complaining is that there were a few months when it scarcely lagged at all. Things are going backwards, not forwards. Saying that it was far worse 3 years ago is irrelevant.

[MisterPerson]

Here's the interesting thing I found out.

I played around on my local offline code and I tried changing the FPS to 15 (from 11) and that suddenly got rid of the jiggery movement.

I'm no expert but what would be the negative effects of raising the FPS from 11 to 15?

E: To be fair this is more the movement being shitty than the actual lag the game has. Playing on the server is a very bad experience in comparison to playing on a local server, but in both iterations the actual movement is crap and not smooth at all.

I actually have a PR up to fix the jittery movement even at our current framerate. That might make things feel better. It's hard to know for sure because of the local vs networked differences you mentioned, but we'll see.

[Aranclanos]

It's atmos, ignore the host he's lying so he can ask for more donations money on his patreon

20 minutes ago I had to shut down atmos completly in the middle of a round because it was unplayable, it became lagless instantly

[Scott]

Atmos is very busy at roundstart in an empty server. I don't know if it's bad mapping, but atmos procs are going at it the moment the round starts.

[Atlanta-Ned]

It's atmos, ignore the host he's lying so he can ask for more donations money on his patreon

20 minutes ago I had to shut down atmos completly in the middle of a round because it was unplayable, it became lagless instantly

I'm inclined to take Aran's word on this. Sounds like we need to have some RealTalk™ about changing fastmos on the host's end.

[Bombadil]

>AI starts plasma fires
>Server starts chugging and becomes near unplayable we start moving a tile every 10-20 seconds

GUYS ITS NOT ATMOS I SWEAR

[ABearInTheWoods]

Ok, guys, can we fucking agree on what kind of lag we are even talking about in this thread then?

Like holy shit.... Read the fucking OP.

The lag being the movement is so jittery and slow.

This is the kind of lag that happens when the server is doing nothing, this is the kind of lag that happens when mc is at 0ds, cpu is at 10%, and nothing is even going on....

This is the kind of lag called NETWORK LAG.

You are all talking about game lag, about the server going for a stand still at regular intervals because something is taking up all the cpu, Now, if you want to have a discussion on that, start a new fucking thread, but don't go and read the word lag, and start blabbing on about something else in an unrelated thread.

The lag Steelpoint is talking about in his op, is not caused by fastmos, or atmos, or shit like that. Steelpoint is talking about sustained lagged, thruout the round, not just when its busy.

It is caused by a mix of OVH's network getting worst, and byond's network protocol getting worst. I know, i've been talking with both lum and ovh about this since early july.

I know about it, I fucking feel it too.

[ABearInTheWoods]

Oh and you can't switch host because he controls both server and page.

Rock controls the domain.

If you honestly think I'm doing what you think I'm doing, Go talk to him.

[crossbowman5]

Alright. There's a lot of shit flinging in here so I'm gonna clarify what's been going on from my side of things. There have been some serious problems with the quality of OVH's network over the past couple months, it's affecting multiple servers I have with them. Can't speak for byond's networking or the code. We are looking at moving servers, but it's difficult to find a new hosting company that will provide the features we need in a location we want without being significantly more expensive. Plus, the server is actually under an OVH account controlled by me, if MSO was really doing what some of you are accusing him off I would have cut him off.

[Steelpoint]

To be honest it can be X lag or Y lag or whatever reasons for the lag but its a continuing problem.

Lets not argue over technicalities, all forms of lag or whatever are bad and should be looked at/fixed.

[ABearInTheWoods]

I'm not arguing over techicalities, I'm talking about one kind of lag, you are talking about the same kind of lag, then everybody else starts talking about another then accusing me of lying to try to get more donation money so I can pocket it. I'm not lying, i'm just sticking to the topic post.

Don't play the schematics cards, either we all talk about the same thing, or if we aren't, we all fucking start specifying what we are talking about in our posts so we can stop having this confusion. Either that or this topic gets locked.

Intermittent lag from when the game server is overload with shit to do (henceforth known as game or cpu lag) is another topic Its also a significantly lower priority of an issue from the everyday all round lag that ovh's shitty network has been throwing us, where pings are randomly 3 times higher then normal, packets randomly get dropped, causing all traffic between that client and server to come to a stand still until the packet is resent and realigned in the send buffer. Because this lag is also why resources randomly want to take 5 times longer to download, Its responsible for some of the client crashes.

In two more weeks I'll be able to request google adsense resend the address verification code to my proper address, ~2 weeks after that i'll be able to unlock those funds. Once I do we will be moving to another server if ovh hasn't gotten their act together. One that's also directly on the trunk lines going to the EU/GB as well as the trunk lines going accross the usa.

In the mean time, https://github.com/tgstation/-tg-station/pull/11344 should fix atmo's dynamic wait not activating until after the lag is too much.

I've also identified that lattices aren't softdeleting, and have been having to get hard deleted, a source of some of the lag from the singulo.

Other people are working on the fact that both powernets and pipenets rebuild themselves every time something is removed, so if the singulo pulls 4 power cables or pipes up in the same tick, it rebuilds 4 times, rather than wait until after the 4th one is pulled up and rebuilding once.

You see what happens when its actually explained what the fuck you are talking about. All this time i've been thinking this is about overall all round lag, because thats what's in the thread's fucking OP.

And no, atmos/fastmos isn't causing that. It never has, it can't.

Now if you want to talk about game stalls, (lets just call it that, lag is an incorrect word for it) thats another story.

[oranges]

interesting, I've not noticed network lag of that kind, nor the jittery movement described in OP.

Perhaps it's not something I notice as a higher ping player? This could be affecting low ping users only.

[Malkevin]

How come we never made the singulo not rip up anchored objects like pipes and wires?

And why it's the singulo still stupidly fast?

[Aranclanos]

players don't know the difference of cpu lag and net lag, its stupid to moderate the thread based on that but sure

[CPTANT]

Ohw yeah like the OP was so clear about that this was just about network lag.

Slow and jittery movement could just as well be used to describe the stops caused by cpu overuse. Ohw sure it's now clear that you wanted to talk about the network

Giant dick move to just delete everything when the OP is vague about what is to be discussed. People put effort in writing shit here to improve the game.

[Steelpoint]

What the hell why is everything gone?

Look mate, I'm just trying to discuss the fact that the game is very fucking laggy. This encompasses both in game movement being shitty and the actual game lagging due to atmo or whatever.

[Malkevin]

Network lag, Server lag, and client lag are not mutually exclusive things.
Just because one thing is causing one of those things to lag does not mean there are other things causing the other things to lag too.