While the WindowsPowerShell subfolder[1] of the well-known Documents folder:[2]
is used by Windows PowerShell (the legacy, ships-with-Windows, Windows-only edition of PowerShell whose latest and final version is 5.1), namely to store optional user-level modules,
its existence is not necessary for Windows PowerShell to function, and Windows PowerShell does not create it on demand (it is created only through deliberate user action).[3]
The above implies that in your case some process that runs on login must be recreating said folder, so you'll have to investigate which one that is.
The Sysinternals Autoruns utility can help you with that.
Specifically, given that, as you state in a later comment, subfolders Scripts\InstalledScriptInfos are created inside the undesired WindowsPowerShell folder, look for a PowerShell startup task that calls Install-Script.
[1] Note that PowerShell (Core) 7, the modern, cross-platform, install-on-demand successor edition, uses the PowerShell subfolder instead. The rest of this answer applies analogously.
[2] Its default location is $env:USERPROFILE\Documents, but it may be redirected to a different location on a given machine, often to OneDrive; use [Environment]::GetFolderPath('MyDocuments') to determine the actual location.
[3] Deliberate actions such as force-creating the file that $PROFILE points to with New-Item -Force $PROFILE, which implicitly creates its hosting folder, i.e. WindowsPowerShell under Documents (CAVEAT: if you run this and there's a preexisting file at that location, it will be truncated) or such as installing a module or script with Install-Module or Install-Script.
Conversely, you can verify that the folder is not (re)created on demand as follows: If necessary (only if you see the unexpected creation of the folder), create a new user and log in as that. If the folder exists on your system, temporarily rename it. Then restart your system and run Windows PowerShell: it will start normally, without complaint, and the folder won't be (re)created.