A theory/investigation on crashes due to verifying the integrity of a specific file (pak01.vpk)

gamadar

New member
It seems like there's a few of us who are hitting on the issue(link) where the game crashes due at some point in what seems to be the "first game of the day" with the error that "Content file "<filename here>(usually pak01.vpk)" is corrupt. Verify integrity of game files.". This happened to me twice(link) in the last few days (resulting in bans). As a note, if you see something referenced in brackets([]), scroll to the bottom of the post for more details.

As I'm banned for a few more hours, I decided to dig into this issue as I know generic crash reports aren't helpful to the devs. I'll caveat this with saying I have no familiarity with the steam engine(or game engines in general), but was looking through some docs online. I think the issue is that the patcher is corrupting files[1], the game doesn't load these files right away[2], and when they finally do load them the crash occurs[3]. This would explain that most of the crash reports on the forums are the "first game of the day" at seemingly random points (which are probably when each person encountered the updated asset that was corrupt).

My data points are as follows, for myself both crashes happened the first game after a patch (or steam updating some files) and the crashes seemed to happen when encountering a newer asset.

Game 1(30691244):

This game crashed as soon as the initial countdown ended and we started going into lane. I was rewatching the game, and I noticed that there was a newer asset that you see at the start([screenshot 1])(the wreath) that I didn't spot anywhere in the initial wait screen area([screenshot 2]). When I went looking through the VPK file that was referenced (pak01.vpk) - I saw what I think is the asset is included in that file([screenshot 3]). If this was in fact the corrupt file and the engine didn't load it until after the countdown then this would support this theory.

Game 2(30794637):

In this game I crashed about 13 minutes into the game (this game was also the first game after a patch). When I went back and looked at the moment I crashed, I saw I crashed when the enemy Shiv came into my lane to gank ([screenshot 4]). I was running back to my tower due to being low on health and when I realized two people were attacking me(yes I wasn't paying attention to the minimap...) I turned to look at who else was there and as I was turning to look the game crashed. Now, you might say well didn't you see shiv when the game started? Yes - I did, but I'm almost certain I saw him acquire his holiday skin *during* the game as at some point after he acquired the skin he wandered into my lane. This was potentially the moment that the game attempted to load the Shiv holiday assets, which if were in fact corrupt, would have caused the crash. I also checked to see if these assets were in pak01.vpk and I believe they were([screenshot 5]).

Note, if you rewatch this game, you'll see that at the start Shiv has the holiday skin, but I think that the game recordings likely save the final model state rather than the model state changes during the game (so if you acquired a holiday skin during the game the replay would show you starting the game with the holiday skin). I also remember seeing one or more people gain a holiday skin during the round and when I went back to rewatch the game from Shiv's point of view I saw he got 3 kills with his ult which according to this is the trigger to acquire the skin.

Anyhow, hope this helps you all narrow down the issue. For anyone encountering this issue, I think a potential workaround is to forcibly verify the integrity of your game files whenever you notice the game's been updated (instead of every day when you start playing).

---

Footnotes

[1] I think it could potentially be some combination of 1) file doesn't download completely 2) file downloads but the hash used to verify the integrity isn't updated or 3) the file doesn't download and the hash is updated to a newer version.
[2] I saw that Unreal Engine has level streaming(link) which I assume has a parallel feature in the Source Engine (couldn't find any docs on source engine).



[screenshot 1]
crash 1.png

[screenshot 2]
crash 1 - starting location reference.png

[screenshot 3]

wreath in vpk.png

[screenshot 4]

crash 2.png

[screenshot 5]
shiv files in vpk.png
 
I was thinking about this and had a couple of suggestions if this theory is in fact true. If this is some weird race condition in the updater, then tracking it down may be significant effort not worth expending at this point in the game's development. A few alternate options:

1) If you don't already, forcibly verify the integrity of the entire game after a patch.

2) As it's not an actual crash and the Steam Engine itself pausing the game due to corruption, when this happens set a flag for the player for that game so that way the server doesn't flag them as a leaver and then ban them. While it's possible during release people could use this loophole to drop out of games, for the play test it's a small quality of life thing for the testers.

3) To help you all debug potential issues in the future, indicate the actual file within the VPK that's corrupted to make bug reporting more accurate.
 
So this happened today as well(resulting in another 6 hour ban), I got curious realizing there were probably steam logs somewhere and as I went through them I can confirm that for Crash 1 and 2, there was an update before each of those games started. What's interesting about Crash 3 that happened today(Game ID 30862739) is I played a two games earlier today, then came back a few hours later to play again and got the crash (whereas the first two crashes were the first game of the day).

I dug through the logs for today in Steam and found that the game *did* update today (see logs at bottom), but the first two games after the update did not crash, but the third game did (the crash happened after the countdown on the load screen). I compared the two games I played earlier today to the third game that resulted in the crash and the only thing I can spot that's different is that for the third game is that I saw Lady Geist for the first time in the day. I believe when we were loading into the game it looked like the person playing lady geist hadn't fully connected and their model wasn't there.

What's also odd is, in an attempt to try to get into the game quicker (versus my usual attempt of restart/uninstall/re-install), I deleted pak01.vpk and tried to verify(although this didn't work). However, because I deleted the file, I had a copy of the supposedly corrupt version, and when I went to compare it to the freshly downloaded version the hashes were identical(see at bottom of post).

---

Logs showing an update today:
[2024-12-22 13:54:39] AppID 1422450 scheduler update : Priority Auto Update, not played for 9951 seconds, update disabled for 0 seconds
[2024-12-22 13:54:39] AppID 1422450 state changed : Fully Installed,Update Queued,
[2024-12-22 13:54:39] AppID 1422450 state changed : Fully Installed,Update Queued,Update Running,
[2024-12-22 13:54:39] AppID 1422450 update changed : Running Update,
[2024-12-22 13:54:39] AppID 1422450 update changed : Running Update,Reconfiguring,
[2024-12-22 13:54:39] Moving to source priority class '2'
[2024-12-22 13:54:39] Created download interface of type 'CDN' (2) to host edge.steam-dns.top.comcast.net (edge.steam-dns.top.comcast.net)
[2024-12-22 13:54:39] Created download interface of type 'CDN' (2) to host edge.steam-dns.top.comcast.net (edge.steam-dns.top.comcast.net)
[2024-12-22 13:54:39] Created download interface of type 'CDN' (2) to host edge.steam-dns.top.comcast.net (edge.steam-dns.top.comcast.net)
[2024-12-22 13:54:39] HTTP (CDN,64) - edge.steam-dns.top.comcast.net (0.0.0.0:0 / 0.0.0.0:0, host: edge.steam-dns.top.comcast.net): AuthenticateDepotID (1422450) - Success!
[2024-12-22 13:54:39] HTTPS (CDN,64) - edge.steam-dns.top.comcast.net (69.241.104.172:443 / 69.241.104.172:443, host: edge.steam-dns.top.comcast.net): edge.steam-dns.top.comcast.net/depot/1422450/manifest/394997150084717220/5/5314011932014662526 - received 200 (OK) HTTP response
[2024-12-22 13:54:40] AppID 1422450 update changed : Running Update,
[2024-12-22 13:54:40] AppID 1422450 update changed : Running Update,Preallocating,
[2024-12-22 13:54:40] AppID 1422450 preallocated 1 files (0 MB)
[2024-12-22 13:54:40] AppID 1422450 update changed : Running Update,
[2024-12-22 13:54:40] AppID 1422450 update started : download 0/63760, store 0/0, reuse 0/0, delta 0/0, stage 0/359229
[2024-12-22 13:54:40] AppID 1422450 update changed : Running Update,Downloading,Staging,
[2024-12-22 13:54:40] Downloading 2 chunks from depot 1422450
[2024-12-22 13:54:40] HTTP (CDN,64) - edge.steam-dns.top.comcast.net (0.0.0.0:0 / 0.0.0.0:0, host: edge.steam-dns.top.comcast.net): AuthenticateDepotID (1422450) - Success!
[2024-12-22 13:54:40] HTTP (CDN,64) - edge.steam-dns.top.comcast.net (0.0.0.0:0 / 0.0.0.0:0, host: edge.steam-dns.top.comcast.net): AuthenticateDepotID (1422450) - Success!
[2024-12-22 13:54:40] AppID 1422450 update changed : Running Update,
[2024-12-22 13:54:40] AppID 1422450 update changed : Running Update,Committing,
[2024-12-22 13:54:40] AppID 1422450 starting commit from "C:\Program Files (x86)\Steam\steamapps\shadercache\1422450/downloads/" to "C:\Program Files (x86)\Steam\steamapps\shadercache\1422450/" : 2 updated, 0 moved, 0 deleted files
[2024-12-22 13:54:40] AppID 1422450 update changed : Running Update,
[2024-12-22 13:54:40] AppID 1422450 update changed : None
[2024-12-22 13:54:40] AppID 1422450 state changed : Fully Installed,Update Queued,
[2024-12-22 13:54:40] AppID 1422450 state changed : Fully Installed,
[2024-12-22 13:54:40] AppID 1422450 scheduler finished : removed from schedule (result No Error, state 0xc)
[2024-12-22 13:54:40] AppID 1422450 adding PID 18536 as a tracked process ""C:\Program Files (x86)\Steam\steamapps\common\Deadlock\game\bin\win64\project8.exe" -steam -console"

Comparing Hash Results:
PS D:\SteamLibrary\steamapps\common\Deadlock\game\citadel> Get-FileHash pak01_001.vpk

Algorithm Hash Path
--------- ---- ----
SHA256 870B5F914A00475EACFE120CEEA2F47E40E811C66FDFA8A594FEBA757DC39DCD D:\SteamLibrary\steamapps\com...


PS D:\SteamLibrary\steamapps\common\Deadlock\game\citadel> cd ~/Desktop
PS C:\Users\<username here>\Desktop> Get-FileHash pak01_001.vpk

Algorithm Hash Path
--------- ---- ----
SHA256 870B5F914A00475EACFE120CEEA2F47E40E811C66FDFA8A594FEBA757DC39DCD C:\Users\<username here>...
 
As another data point, I got curious as I realized today that everytime the game is corrupted, an entry like the following gets recorded in "%Steam Root%\logs\content_log.txt":

AppID 1422450 state changed : Fully Installed,Files Corrupt,App Running,Updated Disabled by app,

I went back through the last few months worth of logs(I joined the playtest late August/early Sep) and every single time the corruption happened right after the logs indicated that steam updated the app. Before this week the error seemed to happen once every few weeks.
 
Back
Top