diff options
| author | gdkchan <gab.dark.100@gmail.com> | 2022-07-31 18:26:06 -0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2022-07-31 18:26:06 -0300 |
| commit | 2232e4ae876c6841e8a5290bd197855367f8d621 (patch) | |
| tree | 8d0819207e1dede6fcdfc8685abcc3dcf023b471 /Ryujinx.Graphics.Gpu/Shader/Cache | |
| parent | 14ce9e15672d03cb6fc067316f90d81471398ebc (diff) | |
Vulkan backend (#2518)
* WIP Vulkan implementation
* No need to initialize attributes on the SPIR-V backend anymore
* Allow multithreading shaderc and vkCreateShaderModule
You'll only really see the benefit here with threaded-gal or parallel shader cache compile.
Fix shaderc multithreaded changes
Thread safety for shaderc Options constructor
Dunno how they managed to make a constructor not thread safe, but you do you. May avoid some freezes.
* Support multiple levels/layers for blit.
Fixes MK8D when scaled, maybe a few other games. AMD software "safe" blit not supported right now.
* TextureStorage should hold a ref of the foreign storage, otherwise it might be freed while in use
* New depth-stencil blit method for AMD
* Workaround for AMD driver bug
* Fix some tessellation related issues (still doesn't work?)
* Submit command buffer before Texture GetData. (UE4 fix)
* DrawTexture support
* Fix BGRA on OpenGL backend
* Fix rebase build break
* Support format aliasing on SetImage
* Fix uniform buffers being lost when bindings are out of order
* Fix storage buffers being lost when bindings are out of order
(also avoid allocations when changing bindings)
* Use current command buffer for unscaled copy (perf)
Avoids flushing commands and renting a command buffer when fulfilling copy dependencies and when games do unscaled copies.
* Update to .net6
* Update Silk.NET to version 2.10.1
Somehow, massive performance boost. Seems like their vtable for looking up vulkan methods was really slow before.
* Fix PrimitivesGenerated query, disable Transform Feedback queries for now
Lets Splatoon 2 work on nvidia. (mostly)
* Update counter queue to be similar to the OGL one
Fixes softlocks when games had to flush counters.
* Don't throw when ending conditional rendering for now
This should be re-enabled when conditional rendering is enabled on nvidia etc.
* Update findMSB/findLSB to match master's instruction enum
* Fix triangle overlay on SMO, Captain Toad, maybe others?
* Don't make Intel Mesa pay for Intel Windows bugs
* Fix samplers with MinFilter Linear or Nearest (fixes New Super Mario Bros U Deluxe black borders)
* Update Spv.Generator
* Add alpha test emulation on shader (but no shader specialisation yet...)
* Fix R4G4B4A4Unorm texture format permutation
* Validation layers should be enabled for any log level other than None
* Add barriers around vkCmdCopyImage
Write->Read barrier for src image (we want to wait for a write to read it)
Write->Read barrier for dst image (we want to wait for the copy to complete before use)
* Be a bit more careful with texture access flags, since it can be used for anything
* Device local mapping for all buffers
May avoid issues with drivers with NVIDIA on linux/older gpus on windows when using large buffers (?)
Also some performance things and fixes issues with opengl games loading textures weird.
* Cleanup, disable device local buffers for now.
* Add single queue support
Multiqueue seems to be a bit more responsive on NVIDIA. Should fix texture flush on intel. AMD has been forced to single queue for an experiment.
* Fix some validation errors around extended dynamic state
* Remove Intel bug workaround, it was fixed on the latest driver
* Use circular queue for checking consumption on command buffers
Speeds up games that spam command buffers a little. Avoids checking multiple command buffers if multiple are active at once.
* Use SupportBufferUpdater, add single layer flush
* Fix counter queue leak when game decides to use host conditional rendering
* Force device local storage for textures (fixes linux performance)
* Port #3019
* Insert barriers around vkCmdBlitImage (may fix some amd flicker)
* Fix transform feedback on Intel, gl_Position feedback and clears to inexistent depth buffers
* Don't pause transform feedback for multi draw
* Fix draw outside of render pass and missing capability
* Workaround for wrong last attribute on AMD (affects FFVII, STRIKERS1945, probably more)
* Better workaround for AMD vertex buffer size alignment issue
* More instructions + fixes on SPIR-V backend
* Allow custom aspect ratio on Vulkan
* Correct GTK UI status bar positions
* SPIR-V: Functions must always end with a return
* SPIR-V: Fix ImageQuerySizeLod
* SPIR-V: Set DepthReplacing execution mode when FragDepth is modified
* SPIR-V: Implement LoopContinue IR instruction
* SPIR-V: Geometry shader support
* SPIR-V: Use correct binding number on storage buffers array
* Reduce allocations for Spir-v serialization
Passes BinaryWriter instead of the stream to Write and WriteOperand
- Removes creation of BinaryWriter for each instruction
- Removes allocations for literal string
* Some optimizations to Spv.Generator
- Dictionary for lookups of type declarations, constants, extinst
- LiteralInteger internal data format -> ushort
- Deterministic HashCode implementation to avoid spirv result not being the same between runs
- Inline operand list instead of List<T>, falls back to array if many operands. (large performance boost)
TODO: improve instruction allocation, structured program creator, ssa?
* Pool Spv.Generator resources, cache delegates, spv opts
- Pools for Instructions and LiteralIntegers. Can be passed in when creating the generator module.
- NewInstruction is called instead of new Instruction()
- Ryujinx SpirvGenerator passes in some pools that are static. The idea is for these to be shared between threads eventually.
- Estimate code size when creating the output MemoryStream
- LiteralInteger pools using ThreadStatic pools that are initialized before and after creation... not sure of a better way since the way these are created is via implicit cast.
Also, cache delegates for Spv.Generator for functions that are passed around to GenerateBinary etc, since passing the function raw creates a delegate on each call.
TODO: update python spv cs generator to make the coregrammar with NewInstruction and the `params` overloads.
* LocalDefMap for Ssa Rewriter
Rather than allocating a large array of all registers for each block in the shader, allocate one array of all registers and clear it between blocks. Reduces allocations in the shader translator.
* SPIR-V: Transform feedback support
* SPIR-V: Fragment shader interlock support (and image coherency)
* SPIR-V: Add early fragment tests support
* SPIR-V: Implement SwizzleAdd, add missing Triangles ExecutionMode for geometry shaders, remove SamplerType field from TextureMeta
* Don't pass depth clip state right now (fix decals)
Explicitly disabling it is incorrect. OpenGL currently automatically disables based on depth clamp, which is the behaviour if this state is omitted.
* Multisampling support
* Multisampling: Use resolve if src samples count > dst samples count
* Multisampling: We can only resolve for unscaled copies
* SPIR-V: Only add FSI exec mode if used.
* SPIR-V: Use ConstantComposite for Texture Offset Vector
Fixes a bunch of freezes with SPIR-V on AMD hardware, and validation errors. Note: Obviously assumes input offsets are constant, which they currently are.
* SPIR-V: Don't OpReturn if we already OpExit'ed
Fixes spir-v parse failure and stack smashing in RADV (obviously you still need bolist)
* SPIR-V: Only use input attribute type for input attributes
Output vertex attributes should always be of type float.
* Multithreaded Pipeline Compilation
* Address some feedback
* Make this 32
* Update topology with GpuAccessorState
* Cleanup for merge (note: disables spir-v)
* Make more robust to shader compilation failure
- Don't freeze when GLSL compilation fails
- Background SPIR-V pipeline compile failure results in skipped draws, similar to GLSL compilation failure.
* Fix Multisampling
* Only update fragment scale count if a vertex texture needs a scale.
Fixes a performance regression introduced by texture scaling in the vertex stage where support buffer updates would be very frequent, even at 1x, if any textures were used on the vertex stage.
This check doesn't exactly look cheap (a flag in the shader stage would probably be preferred), but it is much cheaper than uploading scales in both vulkan and opengl, so it will do for now.
* Use a bitmap to do granular tracking for buffer uploads.
This path is only taken if the much faster check of "is the buffer rented at all" is triggered, so it doesn't actually end up costing too much, and the time saved by not ending render passes (and on gpu for not waiting on barriers) is probably helpful.
Avoids ending render passes to update buffer data (not all the time)
- 140-180 to 35-45 in SMO metro kingdom (these updates are in the UI)
- Very variable 60-150(!) to 16-25 in mario kart 8 (these updates are in the UI)
As well as allowing more data to be preloaded persistently, this will also allow more data to be loaded in the preload buffer, which should be faster as it doesn't need to insert barriers between draws. (and on tbdr, does not need to flush and reload tile memory)
Improves performance in GPU limited scenarios. Should notably improve performance on TBDR gpus. Still a lot more to do here.
* Copy query results after RP ends, rather than ending to copy
We need to end the render pass to get the data (submit command buffer) anyways...
Reduces render passes created in games that use queries.
* Rework Query stuff a bit to avoid render pass end
Tries to reset returned queries in background when possible, rather than ending the render pass.
Still ends render pass when resetting a counter after draws, but maybe that can be solved too. (by just pulling an empty object off the pool?)
* Remove unnecessary lines
Was for testing
* Fix validation error for query reset
Need to think of a better way to do this.
* SPIR-V: Fix SwizzleAdd and some validation errors
* SPIR-V: Implement attribute indexing and StoreAttribute
* SPIR-V: Fix TextureSize for MS and Buffer sampler types
* Fix relaunch issues
* SPIR-V: Implement LogicalExclusiveOr
* SPIR-V: Constant buffer indexing support
* Ignore unsupported attributes rather than throwing (matches current GLSL behaviour)
* SPIR-V: Implement tessellation support
* SPIR-V: Geometry shader passthrough support
* SPIR-V: Implement StoreShader8/16 and StoreStorage8/16
* SPIR-V: Resolution scale support and fix TextureSample multisample with LOD bug
* SPIR-V: Fix field index for scale count
* SPIR-V: Fix another case of wrong field index
* SPIRV/GLSL: More scaling related fixes
* SPIR-V: Fix ImageLoad CompositeExtract component type
* SPIR-V: Workaround for Intel FrontFacing bug
* Enable SPIR-V backend by default
* Allow null samplers (samplers are not required when only using texelFetch to access the texture)
* Fix some validation errors related to texel block view usage flag and invalid image barrier base level
* Use explicit subgroup size if we can (might fix some block flickering on AMD)
* Take componentMask and scissor into account when clearing framebuffer attachments
* Add missing barriers around CmdFillBuffer (fixes Monster Hunter Rise flickering on NVIDIA)
* Use ClampToEdge for Clamp sampler address mode on Vulkan (fixes Hollow Knight)
Clamp is unsupported on Vulkan, but ClampToEdge behaves almost the same. ClampToBorder on the other hand (which was being used before) is pretty different
* Shader specialization for new Vulkan required state (fixes remaining alpha test issues, vertex stretching on AMD on Crash Bandicoot, etc)
* Check if the subgroup size is supported before passing a explicit size
* Only enable ShaderFloat64 if the GPU supports it
* We don't need to recompile shaders if alpha test state changed but alpha test is disabled
* Enable shader cache on Vulkan and implement MultiplyHighS32/U32 on SPIR-V (missed those before)
* Fix pipeline state saving before it is updated.
This should fix a few warnings and potential stutters due to bad pipeline states being saved in the cache. You may need to clear your guest cache.
* Allow null samplers on OpenGL backend
* _unit0Sampler should be set only for binding 0
* Remove unused PipelineConverter format variable (was causing IOR)
* Raise textures limit to 64 on Vulkan
* No need to pack the shader binaries if shader cache is disabled
* Fix backbuffer not being cleared and scissor not being re-enabled on OpenGL
* Do not clear unbound framebuffer color attachments
* Geometry shader passthrough emulation
* Consolidate UpdateDepthMode and GetDepthMode implementation
* Fix A1B5G5R5 texture format and support R4G4 on Vulkan
* Add barrier before use of some modified images
* Report 32 bit query result on AMD windows (smo issue)
* Add texture recompression support (disabled for now)
It recompresses ASTC textures into BC7, which might reduce VRAM usage significantly on games that uses ASTC textures
* Do not report R4G4 format as supported on Vulkan
It was causing mario head to become white on Super Mario 64 (???)
* Improvements to -1 to 1 depth mode.
- Transformation is only applied on the last stage in the vertex pipeline.
- Should fix some issues with geometry and tessellation (hopefully)
- Reading back FragCoord Z on fragment will transform back to -1 to 1.
* Geometry Shader index count from ThreadsPerInputPrimitive
Generally fixes SPIR-V emitting too many triangles, may change games in OpenGL
* Remove gl_FragDepth scaling
This is always 0-1; the other two issues were causing the problems. Fixes regression with Xenoblade.
* Add Gl StencilOp enum values to Vulkan
* Update guest cache to v1.1 (due to specialization state changes)
This will explode your shader cache from earlier vulkan build, but it must be done. :pensive:
* Vulkan/SPIR-V support for viewport inverse
* Fix typo
* Don't create query pools for unsupported query types
* Return of the Vector Indexing Bug
One day, everyone will get this right.
* Check for transform feedback query support
Sometimes transform feedback is supported without the query type.
* Fix gl_FragCoord.z transformation
FragCoord.z is always in 0-1, even when the real depth range is -1 to 1. Turns out the only bug was geo and tess stage outputs.
Fixes Pokemon Sword/Shield, possibly others.
* Fix Avalonia Rebase
Vulkan is currently not available on Avalonia, but the build does work and you can use opengl.
* Fix headless build
* Add support for BC6 and BC7 decompression, decompress all BC formats if they are not supported by the host
* Fix BCn 4/5 conversion, GetTextureTarget
BCn 4/5 could generate invalid data when a line's size in bytes was not divisible by 4, which both backends expect.
GetTextureTarget was not creating a view with the replacement format.
* Fix dependency
* Fix inverse viewport transform vector type on SPIR-V
* Do not require null descriptors support
* If MultiViewport is not supported, do not try to set more than one viewport/scissor
* Bounds check on bitmap add.
* Flush queries on attachment change rather than program change
Occlusion queries are usually used in a depth only pass so the attachments changing is a better indication of the query block ending.
Write mask changes are also considered since some games do depth only pass by setting 0 write mask on all the colour targets.
* Add support for avalonia (#6)
* add avalonia support
* only lock around skia flush
* addressed review
* cleanup
* add fallback size if avalonia attempts to render but the window size is 0. read desktop scale after enabling dpi check
* fix getting window handle on linux. skip render is size is 0
* Combine non-buffer with buffer image descriptor sets
* Support multisample texture copy with automatic resolve on Vulkan
* Remove old CompileShader methods from the Vulkan backend
* Add minimal pipeline layouts that only contains used bindings
They are used by helper shaders, the intention is avoiding needing to recompile the shaders (from GLSL to SPIR-V) if the bindings changes on the translated guest shaders
* Pre-compile helper shader as SPIR-V, and some fixes
* Remove pre-compiled shaderc binary for Windows as its no longer needed by default
* Workaround RADV crash
Enabling the descriptor indexing extension, even if it is not used, forces the radv driver to use "bolist".
* Use RobustBufferAccess on NVIDIA gpus
Avoids the SMO waterfall triangle on older NVIDIA gpus.
* Implement GPU selector and expose texture recompression on the UI and config
* Fix and enable background compute shader compilation
Also disables warnings from shader cache pipeline misses.
* Fix error due to missing subpass dependency when Attachment Write -> Shader Read barriers are added
* If S8D24 is not supported, use D32FS8
* Ensure all fences are destroyed on dispose
* Pre-allocate arrays up front on DescriptorSetUpdater, allows the removal of some checks
* Add missing clear layer parameter after rebase
* Use selected gpu from config for avalonia (#7)
* use configured device
* address review
* Fix D32S8 copy workaround (AMD)
Fixes water in Pokemon Legends Arceus on AMD GPUs. Possibly fixes other things.
* Use push descriptors for uniform buffer updates (disabled for now)
* Push descriptor support check, buffer redundancy checks
Should make push descriptors faster, needs more testing though.
* Increase light command buffer pool to 2 command buffers, throw rather than returning invalid cbs
* Adjust bindings array sizes
* Force submit command buffers if memory in use by its resources is high
* Add workaround for AMD GCN cubemap view sins
`ImageCreateCubeCompatibleBit` seems to generally break 2D array textures with mipmaps... even if they are eventually aliased as a cubemap with mipmaps. Forcing a copy here works around the issue.
This could be used in future if enabling this bit reduces performance on certain GPUs. (mobile class is generally a worry)
Currently also enabled on Linux as I don't know if they managed to dodge this bug (someone please tell me). Not enabled on Vega at the moment, but easy to add if the issue is there.
* Add mobile, non-RX variants to the GCN regex.
Also make sure that the 3 digit ones only include numbers starting with 7 or 8.
* Increase image limit per stage from 8 to 16
Xenoblade Chronicles 2 was hiting the limit of 8
* Minor code cleanup
* Fix NRE caused by SupportBufferUpdater calling pipeline ClearBuffer
* Add gpu selector to Avalonia (#8)
* Add gpu selector to avalonia settings
* show backend label on window
* some fixes
* address review
* Minor changes to the Avalonia UI
* Update graphics window UI and locales. (#9)
* Update xaml and update locales
* locale updates
Did my best here but likely needs to be checked by native speakers, especially the use of ampersands in greek, russian and turkish?
* Fix locales with more (?) correct translations.
* add separator to render widget
* fix spanish and portuguese
* Add new IdList, replaces buffer list that could not remove elements and had unbounded growth
* Don't crash the settings window if Vulkan is not supported
* Fix Actions menu not being clickable on GTK UI after relaunch
* Rename VulkanGraphicsDevice to VulkanRenderer and Renderer to OpenGLRenderer
* Fix IdList and make it not thread safe
* Revert useless OpenGL format table changes
* Fix headless project build
* List throws ArgumentOutOfRangeException
* SPIR-V: Fix tessellation
* Increase shader cache version due to tessellation fix
* Reduce number of Sync objects created (improves perf in some specific titles)
* Fix vulkan validation errors for NPOT compressed upload and GCN workaround.
* Add timestamp to the shader cache and force rebuild if host cache is outdated
* Prefer Mail box present mode for popups (#11)
* Prefer Mail box present mode
* fix debug
* switch present mode when vsync is toggled
* only disable vsync on the main window
* SPIR-V: Fix geometry shader input load with transform feedback
* BC7 Encoder: Prefer more precision on alpha rather than RGB when alpha is 0
* Fix Avalonia build
* Address initial PR feedback
* Only set transform feedback outputs on last vertex stage
* Address riperiperi PR feedback
* Remove outdated comment
* Remove unused constructor
* Only throw for negative results
* Throw for QueueSubmit and other errors
No point in delaying the inevitable
* Transform feedback decorations inside gl_PerVertex struct breaks the NVIDIA compiler
* Fix some resolution scale issues
* No need for two UpdateScale calls
* Fix comments on SPIR-V generator project
* Try to fix shader local memory size
On DOOM, a shader is using local memory, but both Low and High size are 0, CRS size is 1536, it seems to store on that region?
* Remove RectangleF that is now unused
* Fix ImageGather with multiple offsets
Needs ImageGatherExtended capability, and must use `ConstantComposite` instead of `CompositeConstruct`
* Address PR feedback from jD in all projects except Avalonia
* Address most of jD PR feedback on Avalonia
* Remove unsafe
* Fix VulkanSkiaGpu
* move present mode request out of Create Swapchain method
* split more parts of create swapchain
* addressed reviews
* addressed review
* Address second batch of jD PR feedback
* Fix buffer <-> image copy row length and height alignment
AlignUp helper does not support NPOT alignment, and ASTC textures can have NPOT block sizes
* Better fix for NPOT alignment issue
* Use switch expressions on Vulkan EnumConversion
Thanks jD
* Fix Avalonia build
* Add Vulkan selection prompt on startup
* Grammar fixes on Vulkan prompt message
* Add missing Vulkan migration flag
Co-authored-by: riperiperi <rhy3756547@hotmail.com>
Co-authored-by: Emmanuel Hansen <emmausssss@gmail.com>
Co-authored-by: MutantAura <44103205+MutantAura@users.noreply.github.com>
Diffstat (limited to 'Ryujinx.Graphics.Gpu/Shader/Cache')
18 files changed, 0 insertions, 2216 deletions
diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/CacheCollection.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/CacheCollection.cs deleted file mode 100644 index a98531f6..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/CacheCollection.cs +++ /dev/null @@ -1,617 +0,0 @@ -using ICSharpCode.SharpZipLib.Zip; -using Ryujinx.Common; -using Ryujinx.Common.Logging; -using Ryujinx.Graphics.Gpu.Shader.Cache.Definition; -using System; -using System.Collections.Generic; -using System.Diagnostics; -using System.IO; -using System.Linq; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; -using System.Threading; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache -{ - /// <summary> - /// Represent a cache collection handling one shader cache. - /// </summary> - class CacheCollection : IDisposable - { - /// <summary> - /// Possible operation to do on the <see cref="_fileWriterWorkerQueue"/>. - /// </summary> - private enum CacheFileOperation - { - /// <summary> - /// Save a new entry in the temp cache. - /// </summary> - SaveTempEntry, - - /// <summary> - /// Save the hash manifest. - /// </summary> - SaveManifest, - - /// <summary> - /// Remove entries from the hash manifest and save it. - /// </summary> - RemoveManifestEntries, - - /// <summary> - /// Remove entries from the hash manifest and save it, and also deletes the temporary file. - /// </summary> - RemoveManifestEntryAndTempFile, - - /// <summary> - /// Flush temporary cache to archive. - /// </summary> - FlushToArchive, - - /// <summary> - /// Signal when hitting this point. This is useful to know if all previous operations were performed. - /// </summary> - Synchronize - } - - /// <summary> - /// Represent an operation to perform on the <see cref="_fileWriterWorkerQueue"/>. - /// </summary> - private class CacheFileOperationTask - { - /// <summary> - /// The type of operation to perform. - /// </summary> - public CacheFileOperation Type; - - /// <summary> - /// The data associated to this operation or null. - /// </summary> - public object Data; - } - - /// <summary> - /// Data associated to the <see cref="CacheFileOperation.SaveTempEntry"/> operation. - /// </summary> - private class CacheFileSaveEntryTaskData - { - /// <summary> - /// The key of the entry to cache. - /// </summary> - public Hash128 Key; - - /// <summary> - /// The value of the entry to cache. - /// </summary> - public byte[] Value; - } - - /// <summary> - /// The directory of the shader cache. - /// </summary> - private readonly string _cacheDirectory; - - /// <summary> - /// The version of the cache. - /// </summary> - private readonly ulong _version; - - /// <summary> - /// The hash type of the cache. - /// </summary> - private readonly CacheHashType _hashType; - - /// <summary> - /// The graphics API of the cache. - /// </summary> - private readonly CacheGraphicsApi _graphicsApi; - - /// <summary> - /// The table of all the hash registered in the cache. - /// </summary> - private HashSet<Hash128> _hashTable; - - /// <summary> - /// The queue of operations to be performed by the file writer worker. - /// </summary> - private AsyncWorkQueue<CacheFileOperationTask> _fileWriterWorkerQueue; - - /// <summary> - /// Main storage of the cache collection. - /// </summary> - private ZipFile _cacheArchive; - - /// <summary> - /// Indicates if the cache collection supports modification. - /// </summary> - public bool IsReadOnly { get; } - - /// <summary> - /// Immutable copy of the hash table. - /// </summary> - public ReadOnlySpan<Hash128> HashTable => _hashTable.ToArray(); - - /// <summary> - /// Get the temp path to the cache data directory. - /// </summary> - /// <returns>The temp path to the cache data directory</returns> - private string GetCacheTempDataPath() => CacheHelper.GetCacheTempDataPath(_cacheDirectory); - - /// <summary> - /// The path to the cache archive file. - /// </summary> - /// <returns>The path to the cache archive file</returns> - private string GetArchivePath() => CacheHelper.GetArchivePath(_cacheDirectory); - - /// <summary> - /// The path to the cache manifest file. - /// </summary> - /// <returns>The path to the cache manifest file</returns> - private string GetManifestPath() => CacheHelper.GetManifestPath(_cacheDirectory); - - /// <summary> - /// Create a new temp path to the given cached file via its hash. - /// </summary> - /// <param name="key">The hash of the cached data</param> - /// <returns>New path to the given cached file</returns> - private string GenCacheTempFilePath(Hash128 key) => CacheHelper.GenCacheTempFilePath(_cacheDirectory, key); - - /// <summary> - /// Create a new cache collection. - /// </summary> - /// <param name="baseCacheDirectory">The directory of the shader cache</param> - /// <param name="hashType">The hash type of the shader cache</param> - /// <param name="graphicsApi">The graphics api of the shader cache</param> - /// <param name="shaderProvider">The shader provider name of the shader cache</param> - /// <param name="cacheName">The name of the cache</param> - /// <param name="version">The version of the cache</param> - public CacheCollection(string baseCacheDirectory, CacheHashType hashType, CacheGraphicsApi graphicsApi, string shaderProvider, string cacheName, ulong version) - { - if (hashType != CacheHashType.XxHash128) - { - throw new NotImplementedException($"{hashType}"); - } - - _cacheDirectory = CacheHelper.GenerateCachePath(baseCacheDirectory, graphicsApi, shaderProvider, cacheName); - _graphicsApi = graphicsApi; - _hashType = hashType; - _version = version; - _hashTable = new HashSet<Hash128>(); - IsReadOnly = CacheHelper.IsArchiveReadOnly(GetArchivePath()); - - Load(); - - _fileWriterWorkerQueue = new AsyncWorkQueue<CacheFileOperationTask>(HandleCacheTask, $"CacheCollection.Worker.{cacheName}"); - } - - /// <summary> - /// Load the cache manifest file and recreate it if invalid. - /// </summary> - private void Load() - { - bool isValid = false; - - if (Directory.Exists(_cacheDirectory)) - { - string manifestPath = GetManifestPath(); - - if (File.Exists(manifestPath)) - { - Memory<byte> rawManifest = File.ReadAllBytes(manifestPath); - - if (MemoryMarshal.TryRead(rawManifest.Span, out CacheManifestHeader manifestHeader)) - { - Memory<byte> hashTableRaw = rawManifest.Slice(Unsafe.SizeOf<CacheManifestHeader>()); - - isValid = manifestHeader.IsValid(_graphicsApi, _hashType, hashTableRaw.Span) && _version == manifestHeader.Version; - - if (isValid) - { - ReadOnlySpan<Hash128> hashTable = MemoryMarshal.Cast<byte, Hash128>(hashTableRaw.Span); - - foreach (Hash128 hash in hashTable) - { - _hashTable.Add(hash); - } - } - } - } - } - - if (!isValid) - { - Logger.Warning?.Print(LogClass.Gpu, $"Shader collection \"{_cacheDirectory}\" got invalidated, cache will need to be rebuilt."); - - if (Directory.Exists(_cacheDirectory)) - { - Directory.Delete(_cacheDirectory, true); - } - - Directory.CreateDirectory(_cacheDirectory); - - SaveManifest(); - } - - FlushToArchive(); - } - - /// <summary> - /// Queue a task to remove entries from the hash manifest. - /// </summary> - /// <param name="entries">Entries to remove from the manifest</param> - public void RemoveManifestEntriesAsync(HashSet<Hash128> entries) - { - if (IsReadOnly) - { - Logger.Warning?.Print(LogClass.Gpu, "Trying to remove manifest entries on a read-only cache, ignoring."); - - return; - } - - _fileWriterWorkerQueue.Add(new CacheFileOperationTask - { - Type = CacheFileOperation.RemoveManifestEntries, - Data = entries - }); - } - - /// <summary> - /// Remove given entries from the manifest. - /// </summary> - /// <param name="entries">Entries to remove from the manifest</param> - private void RemoveManifestEntries(HashSet<Hash128> entries) - { - lock (_hashTable) - { - foreach (Hash128 entry in entries) - { - _hashTable.Remove(entry); - } - - SaveManifest(); - } - } - - /// <summary> - /// Remove given entry from the manifest and delete the temporary file. - /// </summary> - /// <param name="entry">Entry to remove from the manifest</param> - private void RemoveManifestEntryAndTempFile(Hash128 entry) - { - lock (_hashTable) - { - _hashTable.Remove(entry); - SaveManifest(); - } - - File.Delete(GenCacheTempFilePath(entry)); - } - - /// <summary> - /// Queue a task to flush temporary files to the archive on the worker. - /// </summary> - public void FlushToArchiveAsync() - { - _fileWriterWorkerQueue.Add(new CacheFileOperationTask - { - Type = CacheFileOperation.FlushToArchive - }); - } - - /// <summary> - /// Wait for all tasks before this given point to be done. - /// </summary> - public void Synchronize() - { - using (ManualResetEvent evnt = new ManualResetEvent(false)) - { - _fileWriterWorkerQueue.Add(new CacheFileOperationTask - { - Type = CacheFileOperation.Synchronize, - Data = evnt - }); - - evnt.WaitOne(); - } - } - - /// <summary> - /// Flush temporary files to the archive. - /// </summary> - /// <remarks>This dispose <see cref="_cacheArchive"/> if not null and reinstantiate it.</remarks> - private void FlushToArchive() - { - EnsureArchiveUpToDate(); - - // Open the zip in readonly to avoid anyone modifying/corrupting it during normal operations. - _cacheArchive = new ZipFile(File.OpenRead(GetArchivePath())); - } - - /// <summary> - /// Save temporary files not in archive. - /// </summary> - /// <remarks>This dispose <see cref="_cacheArchive"/> if not null.</remarks> - public void EnsureArchiveUpToDate() - { - // First close previous opened instance if found. - if (_cacheArchive != null) - { - _cacheArchive.Close(); - } - - string archivePath = GetArchivePath(); - - if (IsReadOnly) - { - Logger.Warning?.Print(LogClass.Gpu, $"Cache collection archive in read-only, archiving task skipped."); - - return; - } - - if (CacheHelper.IsArchiveReadOnly(archivePath)) - { - Logger.Warning?.Print(LogClass.Gpu, $"Cache collection archive in use, archiving task skipped."); - - return; - } - - if (!File.Exists(archivePath)) - { - using (ZipFile newZip = ZipFile.Create(archivePath)) - { - // Workaround for SharpZipLib issue #395 - newZip.BeginUpdate(); - newZip.CommitUpdate(); - } - } - - // Open the zip in read/write. - _cacheArchive = new ZipFile(File.Open(archivePath, FileMode.Open, FileAccess.ReadWrite, FileShare.None)); - - Logger.Info?.Print(LogClass.Gpu, $"Updating cache collection archive {archivePath}..."); - - // Update the content of the zip. - lock (_hashTable) - { - CacheHelper.EnsureArchiveUpToDate(_cacheDirectory, _cacheArchive, _hashTable); - - // Close the instance to force a flush. - _cacheArchive.Close(); - _cacheArchive = null; - - string cacheTempDataPath = GetCacheTempDataPath(); - - // Create the cache data path if missing. - if (!Directory.Exists(cacheTempDataPath)) - { - Directory.CreateDirectory(cacheTempDataPath); - } - } - - Logger.Info?.Print(LogClass.Gpu, $"Updated cache collection archive {archivePath}."); - } - - /// <summary> - /// Save the manifest file. - /// </summary> - private void SaveManifest() - { - byte[] data; - - lock (_hashTable) - { - data = CacheHelper.ComputeManifest(_version, _graphicsApi, _hashType, _hashTable); - } - - File.WriteAllBytes(GetManifestPath(), data); - } - - /// <summary> - /// Get a cached file with the given hash. - /// </summary> - /// <param name="keyHash">The given hash</param> - /// <returns>The cached file if present or null</returns> - public byte[] GetValueRaw(ref Hash128 keyHash) - { - return GetValueRawFromArchive(ref keyHash) ?? GetValueRawFromFile(ref keyHash); - } - - /// <summary> - /// Get a cached file with the given hash that is present in the archive. - /// </summary> - /// <param name="keyHash">The given hash</param> - /// <returns>The cached file if present or null</returns> - private byte[] GetValueRawFromArchive(ref Hash128 keyHash) - { - bool found; - - lock (_hashTable) - { - found = _hashTable.Contains(keyHash); - } - - if (found) - { - return CacheHelper.ReadFromArchive(_cacheArchive, keyHash); - } - - return null; - } - - /// <summary> - /// Get a cached file with the given hash that is not present in the archive. - /// </summary> - /// <param name="keyHash">The given hash</param> - /// <returns>The cached file if present or null</returns> - private byte[] GetValueRawFromFile(ref Hash128 keyHash) - { - bool found; - - lock (_hashTable) - { - found = _hashTable.Contains(keyHash); - } - - if (found) - { - return CacheHelper.ReadFromFile(GetCacheTempDataPath(), keyHash); - } - - return null; - } - - private void HandleCacheTask(CacheFileOperationTask task) - { - switch (task.Type) - { - case CacheFileOperation.SaveTempEntry: - SaveTempEntry((CacheFileSaveEntryTaskData)task.Data); - break; - case CacheFileOperation.SaveManifest: - SaveManifest(); - break; - case CacheFileOperation.RemoveManifestEntries: - RemoveManifestEntries((HashSet<Hash128>)task.Data); - break; - case CacheFileOperation.RemoveManifestEntryAndTempFile: - RemoveManifestEntryAndTempFile((Hash128)task.Data); - break; - case CacheFileOperation.FlushToArchive: - FlushToArchive(); - break; - case CacheFileOperation.Synchronize: - ((ManualResetEvent)task.Data).Set(); - break; - default: - throw new NotImplementedException($"{task.Type}"); - } - - } - - /// <summary> - /// Save a new entry in the temp cache. - /// </summary> - /// <param name="entry">The entry to save in the temp cache</param> - private void SaveTempEntry(CacheFileSaveEntryTaskData entry) - { - string tempPath = GenCacheTempFilePath(entry.Key); - - File.WriteAllBytes(tempPath, entry.Value); - } - - /// <summary> - /// Add a new value in the cache with a given hash. - /// </summary> - /// <param name="keyHash">The hash to use for the value in the cache</param> - /// <param name="value">The value to cache</param> - public void AddValue(ref Hash128 keyHash, byte[] value) - { - if (IsReadOnly) - { - Logger.Warning?.Print(LogClass.Gpu, $"Trying to add {keyHash} on a read-only cache, ignoring."); - - return; - } - - Debug.Assert(value != null); - - bool isAlreadyPresent; - - lock (_hashTable) - { - isAlreadyPresent = !_hashTable.Add(keyHash); - } - - if (isAlreadyPresent) - { - // NOTE: Used for debug - File.WriteAllBytes(GenCacheTempFilePath(new Hash128()), value); - - throw new InvalidOperationException($"Cache collision found on {GenCacheTempFilePath(keyHash)}"); - } - - // Queue file change operations - _fileWriterWorkerQueue.Add(new CacheFileOperationTask - { - Type = CacheFileOperation.SaveTempEntry, - Data = new CacheFileSaveEntryTaskData - { - Key = keyHash, - Value = value - } - }); - - // Save the manifest changes - _fileWriterWorkerQueue.Add(new CacheFileOperationTask - { - Type = CacheFileOperation.SaveManifest, - }); - } - - /// <summary> - /// Replace a value at the given hash in the cache. - /// </summary> - /// <param name="keyHash">The hash to use for the value in the cache</param> - /// <param name="value">The value to cache</param> - public void ReplaceValue(ref Hash128 keyHash, byte[] value) - { - if (IsReadOnly) - { - Logger.Warning?.Print(LogClass.Gpu, $"Trying to replace {keyHash} on a read-only cache, ignoring."); - - return; - } - - Debug.Assert(value != null); - - // Only queue file change operations - _fileWriterWorkerQueue.Add(new CacheFileOperationTask - { - Type = CacheFileOperation.SaveTempEntry, - Data = new CacheFileSaveEntryTaskData - { - Key = keyHash, - Value = value - } - }); - } - - /// <summary> - /// Removes a value at the given hash from the cache. - /// </summary> - /// <param name="keyHash">The hash of the value in the cache</param> - public void RemoveValue(ref Hash128 keyHash) - { - if (IsReadOnly) - { - Logger.Warning?.Print(LogClass.Gpu, $"Trying to remove {keyHash} on a read-only cache, ignoring."); - - return; - } - - // Only queue file change operations - _fileWriterWorkerQueue.Add(new CacheFileOperationTask - { - Type = CacheFileOperation.RemoveManifestEntryAndTempFile, - Data = keyHash - }); - } - - public void Dispose() - { - Dispose(true); - } - - protected virtual void Dispose(bool disposing) - { - if (disposing) - { - // Make sure all operations on _fileWriterWorkerQueue are done. - Synchronize(); - - _fileWriterWorkerQueue.Dispose(); - EnsureArchiveUpToDate(); - } - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/CacheHelper.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/CacheHelper.cs deleted file mode 100644 index d16afb65..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/CacheHelper.cs +++ /dev/null @@ -1,273 +0,0 @@ -using ICSharpCode.SharpZipLib.Zip; -using Ryujinx.Common; -using Ryujinx.Common.Configuration; -using Ryujinx.Common.Logging; -using Ryujinx.Graphics.Gpu.Shader.Cache.Definition; -using Ryujinx.Graphics.Shader; -using System; -using System.Collections.Generic; -using System.IO; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache -{ - /// <summary> - /// Helper to manipulate the disk shader cache. - /// </summary> - static class CacheHelper - { - /// <summary> - /// Compute a cache manifest from runtime data. - /// </summary> - /// <param name="version">The version of the cache</param> - /// <param name="graphicsApi">The graphics api used by the cache</param> - /// <param name="hashType">The hash type of the cache</param> - /// <param name="entries">The entries in the cache</param> - /// <returns>The cache manifest from runtime data</returns> - public static byte[] ComputeManifest(ulong version, CacheGraphicsApi graphicsApi, CacheHashType hashType, HashSet<Hash128> entries) - { - if (hashType != CacheHashType.XxHash128) - { - throw new NotImplementedException($"{hashType}"); - } - - CacheManifestHeader manifestHeader = new CacheManifestHeader(version, graphicsApi, hashType); - - byte[] data = new byte[Unsafe.SizeOf<CacheManifestHeader>() + entries.Count * Unsafe.SizeOf<Hash128>()]; - - // CacheManifestHeader has the same size as a Hash128. - Span<Hash128> dataSpan = MemoryMarshal.Cast<byte, Hash128>(data.AsSpan()).Slice(1); - - int i = 0; - - foreach (Hash128 hash in entries) - { - dataSpan[i++] = hash; - } - - manifestHeader.UpdateChecksum(data.AsSpan(Unsafe.SizeOf<CacheManifestHeader>())); - - MemoryMarshal.Write(data, ref manifestHeader); - - return data; - } - - /// <summary> - /// Get the base directory of the shader cache for a given title id. - /// </summary> - /// <param name="titleId">The title id of the target application</param> - /// <returns>The base directory of the shader cache for a given title id</returns> - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static string GetBaseCacheDirectory(string titleId) => Path.Combine(AppDataManager.GamesDirPath, titleId, "cache", "shader"); - - /// <summary> - /// Get the temp path to the cache data directory. - /// </summary> - /// <param name="cacheDirectory">The cache directory</param> - /// <returns>The temp path to the cache data directory</returns> - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static string GetCacheTempDataPath(string cacheDirectory) => Path.Combine(cacheDirectory, "temp"); - - /// <summary> - /// The path to the cache archive file. - /// </summary> - /// <param name="cacheDirectory">The cache directory</param> - /// <returns>The path to the cache archive file</returns> - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static string GetArchivePath(string cacheDirectory) => Path.Combine(cacheDirectory, "cache.zip"); - - /// <summary> - /// The path to the cache manifest file. - /// </summary> - /// <param name="cacheDirectory">The cache directory</param> - /// <returns>The path to the cache manifest file</returns> - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static string GetManifestPath(string cacheDirectory) => Path.Combine(cacheDirectory, "cache.info"); - - /// <summary> - /// Create a new temp path to the given cached file via its hash. - /// </summary> - /// <param name="cacheDirectory">The cache directory</param> - /// <param name="key">The hash of the cached data</param> - /// <returns>New path to the given cached file</returns> - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static string GenCacheTempFilePath(string cacheDirectory, Hash128 key) => Path.Combine(GetCacheTempDataPath(cacheDirectory), key.ToString()); - - /// <summary> - /// Generate the path to the cache directory. - /// </summary> - /// <param name="baseCacheDirectory">The base of the cache directory</param> - /// <param name="graphicsApi">The graphics api in use</param> - /// <param name="shaderProvider">The name of the shader provider in use</param> - /// <param name="cacheName">The name of the cache</param> - /// <returns>The path to the cache directory</returns> - public static string GenerateCachePath(string baseCacheDirectory, CacheGraphicsApi graphicsApi, string shaderProvider, string cacheName) - { - string graphicsApiName = graphicsApi switch - { - CacheGraphicsApi.OpenGL => "opengl", - CacheGraphicsApi.OpenGLES => "opengles", - CacheGraphicsApi.Vulkan => "vulkan", - CacheGraphicsApi.DirectX => "directx", - CacheGraphicsApi.Metal => "metal", - CacheGraphicsApi.Guest => "guest", - _ => throw new NotImplementedException(graphicsApi.ToString()), - }; - - return Path.Combine(baseCacheDirectory, graphicsApiName, shaderProvider, cacheName); - } - - /// <summary> - /// Read a cached file with the given hash that is present in the archive. - /// </summary> - /// <param name="archive">The archive in use</param> - /// <param name="entry">The given hash</param> - /// <returns>The cached file if present or null</returns> - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static byte[] ReadFromArchive(ZipFile archive, Hash128 entry) - { - if (archive != null) - { - ZipEntry archiveEntry = archive.GetEntry($"{entry}"); - - if (archiveEntry != null) - { - try - { - byte[] result = new byte[archiveEntry.Size]; - - using (Stream archiveStream = archive.GetInputStream(archiveEntry)) - { - archiveStream.Read(result); - - return result; - } - } - catch (Exception e) - { - Logger.Error?.Print(LogClass.Gpu, $"Cannot load cache file {entry} from archive"); - Logger.Error?.Print(LogClass.Gpu, e.ToString()); - } - } - } - - return null; - } - - /// <summary> - /// Read a cached file with the given hash that is not present in the archive. - /// </summary> - /// <param name="cacheDirectory">The cache directory</param> - /// <param name="entry">The given hash</param> - /// <returns>The cached file if present or null</returns> - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static byte[] ReadFromFile(string cacheDirectory, Hash128 entry) - { - string cacheTempFilePath = GenCacheTempFilePath(cacheDirectory, entry); - - try - { - return File.ReadAllBytes(cacheTempFilePath); - } - catch (Exception e) - { - Logger.Error?.Print(LogClass.Gpu, $"Cannot load cache file at {cacheTempFilePath}"); - Logger.Error?.Print(LogClass.Gpu, e.ToString()); - } - - return null; - } - - /// <summary> - /// Read transform feedback descriptors from guest. - /// </summary> - /// <param name="data">The raw guest transform feedback descriptors</param> - /// <param name="header">The guest shader program header</param> - /// <returns>The transform feedback descriptors read from guest</returns> - public static TransformFeedbackDescriptorOld[] ReadTransformFeedbackInformation(ref ReadOnlySpan<byte> data, GuestShaderCacheHeader header) - { - if (header.TransformFeedbackCount != 0) - { - TransformFeedbackDescriptorOld[] result = new TransformFeedbackDescriptorOld[header.TransformFeedbackCount]; - - for (int i = 0; i < result.Length; i++) - { - GuestShaderCacheTransformFeedbackHeader feedbackHeader = MemoryMarshal.Read<GuestShaderCacheTransformFeedbackHeader>(data); - - result[i] = new TransformFeedbackDescriptorOld(feedbackHeader.BufferIndex, feedbackHeader.Stride, data.Slice(Unsafe.SizeOf<GuestShaderCacheTransformFeedbackHeader>(), feedbackHeader.VaryingLocationsLength).ToArray()); - - data = data.Slice(Unsafe.SizeOf<GuestShaderCacheTransformFeedbackHeader>() + feedbackHeader.VaryingLocationsLength); - } - - return result; - } - - return null; - } - - /// <summary> - /// Save temporary files not in archive. - /// </summary> - /// <param name="baseCacheDirectory">The base of the cache directory</param> - /// <param name="archive">The archive to use</param> - /// <param name="entries">The entries in the cache</param> - [MethodImpl(MethodImplOptions.AggressiveInlining)] - public static void EnsureArchiveUpToDate(string baseCacheDirectory, ZipFile archive, HashSet<Hash128> entries) - { - List<string> filesToDelete = new List<string>(); - - archive.BeginUpdate(); - - foreach (Hash128 hash in entries) - { - string cacheTempFilePath = GenCacheTempFilePath(baseCacheDirectory, hash); - - if (File.Exists(cacheTempFilePath)) - { - string cacheHash = $"{hash}"; - - ZipEntry entry = archive.GetEntry(cacheHash); - - if (entry != null) - { - archive.Delete(entry); - } - - // We enforce deflate compression here to avoid possible incompatibilities on older version of Ryujinx that use System.IO.Compression. - archive.Add(new StaticDiskDataSource(cacheTempFilePath), cacheHash, CompressionMethod.Deflated); - filesToDelete.Add(cacheTempFilePath); - } - } - - archive.CommitUpdate(); - - foreach (string filePath in filesToDelete) - { - File.Delete(filePath); - } - } - - public static bool IsArchiveReadOnly(string archivePath) - { - FileInfo info = new FileInfo(archivePath); - - if (!info.Exists) - { - return false; - } - - try - { - using (FileStream stream = info.Open(FileMode.Open, FileAccess.Read, FileShare.None)) - { - return false; - } - } - catch (IOException) - { - return true; - } - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/CacheManager.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/CacheManager.cs deleted file mode 100644 index e67221e7..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/CacheManager.cs +++ /dev/null @@ -1,168 +0,0 @@ -using Ryujinx.Common; -using Ryujinx.Graphics.Gpu.Shader.Cache.Definition; -using System; -using System.Collections.Generic; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache -{ - /// <summary> - /// Global Manager of the shader cache. - /// </summary> - class CacheManager : IDisposable - { - private CacheGraphicsApi _graphicsApi; - private CacheHashType _hashType; - private string _shaderProvider; - - /// <summary> - /// Cache storing raw Maxwell shaders as programs. - /// </summary> - private CacheCollection _guestProgramCache; - - /// <summary> - /// Cache storing raw host programs. - /// </summary> - private CacheCollection _hostProgramCache; - - /// <summary> - /// Version of the guest cache shader (to increment when guest cache structure change). - /// </summary> - private const ulong GuestCacheVersion = 1759; - - public bool IsReadOnly => _guestProgramCache.IsReadOnly || _hostProgramCache.IsReadOnly; - - /// <summary> - /// Create a new cache manager instance - /// </summary> - /// <param name="graphicsApi">The graphics api in use</param> - /// <param name="hashType">The hash type in use for the cache</param> - /// <param name="shaderProvider">The name of the codegen provider</param> - /// <param name="titleId">The guest application title ID</param> - /// <param name="shaderCodeGenVersion">Version of the codegen</param> - public CacheManager(CacheGraphicsApi graphicsApi, CacheHashType hashType, string shaderProvider, string titleId, ulong shaderCodeGenVersion) - { - _graphicsApi = graphicsApi; - _hashType = hashType; - _shaderProvider = shaderProvider; - - string baseCacheDirectory = CacheHelper.GetBaseCacheDirectory(titleId); - - _guestProgramCache = new CacheCollection(baseCacheDirectory, _hashType, CacheGraphicsApi.Guest, "", "program", GuestCacheVersion); - _hostProgramCache = new CacheCollection(baseCacheDirectory, _hashType, _graphicsApi, _shaderProvider, "host", shaderCodeGenVersion); - } - - - /// <summary> - /// Entries to remove from the manifest. - /// </summary> - /// <param name="entries">Entries to remove from the manifest of all caches</param> - public void RemoveManifestEntries(HashSet<Hash128> entries) - { - _guestProgramCache.RemoveManifestEntriesAsync(entries); - _hostProgramCache.RemoveManifestEntriesAsync(entries); - } - - /// <summary> - /// Queue a task to flush temporary files to the archives. - /// </summary> - public void FlushToArchive() - { - _guestProgramCache.FlushToArchiveAsync(); - _hostProgramCache.FlushToArchiveAsync(); - } - - /// <summary> - /// Wait for all tasks before this given point to be done. - /// </summary> - public void Synchronize() - { - _guestProgramCache.Synchronize(); - _hostProgramCache.Synchronize(); - } - - /// <summary> - /// Save a shader program not present in the program cache. - /// </summary> - /// <param name="programCodeHash">Target program code hash</param> - /// <param name="guestProgram">Guest program raw data</param> - /// <param name="hostProgram">Host program raw data</param> - public void SaveProgram(ref Hash128 programCodeHash, byte[] guestProgram, byte[] hostProgram) - { - _guestProgramCache.AddValue(ref programCodeHash, guestProgram); - _hostProgramCache.AddValue(ref programCodeHash, hostProgram); - } - - /// <summary> - /// Add a host shader program not present in the program cache. - /// </summary> - /// <param name="programCodeHash">Target program code hash</param> - /// <param name="data">Host program raw data</param> - public void AddHostProgram(ref Hash128 programCodeHash, byte[] data) - { - _hostProgramCache.AddValue(ref programCodeHash, data); - } - - /// <summary> - /// Replace a host shader program present in the program cache. - /// </summary> - /// <param name="programCodeHash">Target program code hash</param> - /// <param name="data">Host program raw data</param> - public void ReplaceHostProgram(ref Hash128 programCodeHash, byte[] data) - { - _hostProgramCache.ReplaceValue(ref programCodeHash, data); - } - - /// <summary> - /// Removes a shader program present in the program cache. - /// </summary> - /// <param name="programCodeHash">Target program code hash</param> - public void RemoveProgram(ref Hash128 programCodeHash) - { - _guestProgramCache.RemoveValue(ref programCodeHash); - _hostProgramCache.RemoveValue(ref programCodeHash); - } - - /// <summary> - /// Get all guest program hashes. - /// </summary> - /// <returns>All guest program hashes</returns> - public ReadOnlySpan<Hash128> GetGuestProgramList() - { - return _guestProgramCache.HashTable; - } - - /// <summary> - /// Get a host program by hash. - /// </summary> - /// <param name="hash">The given hash</param> - /// <returns>The host program if present or null</returns> - public byte[] GetHostProgramByHash(ref Hash128 hash) - { - return _hostProgramCache.GetValueRaw(ref hash); - } - - /// <summary> - /// Get a guest program by hash. - /// </summary> - /// <param name="hash">The given hash</param> - /// <returns>The guest program if present or null</returns> - public byte[] GetGuestProgramByHash(ref Hash128 hash) - { - return _guestProgramCache.GetValueRaw(ref hash); - } - - public void Dispose() - { - Dispose(true); - } - - protected virtual void Dispose(bool disposing) - { - if (disposing) - { - _guestProgramCache.Dispose(); - _hostProgramCache.Dispose(); - } - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheGraphicsApi.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheGraphicsApi.cs deleted file mode 100644 index 9f8b5c39..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheGraphicsApi.cs +++ /dev/null @@ -1,38 +0,0 @@ -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Graphics API type accepted by the shader cache. - /// </summary> - enum CacheGraphicsApi : byte - { - /// <summary> - /// OpenGL Core - /// </summary> - OpenGL, - - /// <summary> - /// OpenGL ES - /// </summary> - OpenGLES, - - /// <summary> - /// Vulkan - /// </summary> - Vulkan, - - /// <summary> - /// DirectX - /// </summary> - DirectX, - - /// <summary> - /// Metal - /// </summary> - Metal, - - /// <summary> - /// Guest, used to cache games raw shader programs. - /// </summary> - Guest - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheHashType.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheHashType.cs deleted file mode 100644 index e4ebe416..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheHashType.cs +++ /dev/null @@ -1,13 +0,0 @@ -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Hash algorithm accepted by the shader cache. - /// </summary> - enum CacheHashType : byte - { - /// <summary> - /// xxHash128 - /// </summary> - XxHash128 - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheManifestHeader.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheManifestHeader.cs deleted file mode 100644 index 0601451d..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/CacheManifestHeader.cs +++ /dev/null @@ -1,97 +0,0 @@ -using System; -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Header of the shader cache manifest. - /// </summary> - [StructLayout(LayoutKind.Sequential, Pack = 1, Size = 0x10)] - struct CacheManifestHeader - { - /// <summary> - /// The version of the cache. - /// </summary> - public ulong Version; - - /// <summary> - /// The graphics api used for this cache. - /// </summary> - public CacheGraphicsApi GraphicsApi; - - /// <summary> - /// The hash type used for this cache. - /// </summary> - public CacheHashType HashType; - - /// <summary> - /// CRC-16 checksum over the data in the file. - /// </summary> - public ushort TableChecksum; - - /// <summary> - /// Construct a new cache manifest header. - /// </summary> - /// <param name="version">The version of the cache</param> - /// <param name="graphicsApi">The graphics api used for this cache</param> - /// <param name="hashType">The hash type used for this cache</param> - public CacheManifestHeader(ulong version, CacheGraphicsApi graphicsApi, CacheHashType hashType) - { - Version = version; - GraphicsApi = graphicsApi; - HashType = hashType; - TableChecksum = 0; - } - - /// <summary> - /// Update the checksum in the header. - /// </summary> - /// <param name="data">The data to perform the checksum on</param> - public void UpdateChecksum(ReadOnlySpan<byte> data) - { - TableChecksum = CalculateCrc16(data); - } - - /// <summary> - /// Calculate a CRC-16 over data. - /// </summary> - /// <param name="data">The data to perform the CRC-16 on</param> - /// <returns>A CRC-16 over data</returns> - private static ushort CalculateCrc16(ReadOnlySpan<byte> data) - { - int crc = 0; - - const ushort poly = 0x1021; - - for (int i = 0; i < data.Length; i++) - { - crc ^= data[i] << 8; - - for (int j = 0; j < 8; j++) - { - crc <<= 1; - - if ((crc & 0x10000) != 0) - { - crc = (crc ^ poly) & 0xFFFF; - } - } - } - - return (ushort)crc; - } - - /// <summary> - /// Check the validity of the header. - /// </summary> - /// <param name="graphicsApi">The target graphics api in use</param> - /// <param name="hashType">The target hash type in use</param> - /// <param name="data">The data after this header</param> - /// <returns>True if the header is valid</returns> - /// <remarks>This doesn't check that versions match</remarks> - public bool IsValid(CacheGraphicsApi graphicsApi, CacheHashType hashType, ReadOnlySpan<byte> data) - { - return GraphicsApi == graphicsApi && HashType == hashType && TableChecksum == CalculateCrc16(data); - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestGpuAccessorHeader.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestGpuAccessorHeader.cs deleted file mode 100644 index 2e044750..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestGpuAccessorHeader.cs +++ /dev/null @@ -1,67 +0,0 @@ -using Ryujinx.Graphics.Shader; -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Header of a cached guest gpu accessor. - /// </summary> - [StructLayout(LayoutKind.Sequential, Size = 0x20, Pack = 1)] - struct GuestGpuAccessorHeader - { - /// <summary> - /// The count of texture descriptors. - /// </summary> - public int TextureDescriptorCount; - - /// <summary> - /// Local Size X for compute shaders. - /// </summary> - public int ComputeLocalSizeX; - - /// <summary> - /// Local Size Y for compute shaders. - /// </summary> - public int ComputeLocalSizeY; - - /// <summary> - /// Local Size Z for compute shaders. - /// </summary> - public int ComputeLocalSizeZ; - - /// <summary> - /// Local Memory size in bytes for compute shaders. - /// </summary> - public int ComputeLocalMemorySize; - - /// <summary> - /// Shared Memory size in bytes for compute shaders. - /// </summary> - public int ComputeSharedMemorySize; - - /// <summary> - /// Unused/reserved. - /// </summary> - public int Reserved1; - - /// <summary> - /// Current primitive topology for geometry shaders. - /// </summary> - public InputTopology PrimitiveTopology; - - /// <summary> - /// Tessellation parameters (packed to fit on a byte). - /// </summary> - public byte TessellationModePacked; - - /// <summary> - /// Unused/reserved. - /// </summary> - public byte Reserved2; - - /// <summary> - /// GPU boolean state that can influence shader compilation. - /// </summary> - public GuestGpuStateFlags StateFlags; - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestGpuStateFlags.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestGpuStateFlags.cs deleted file mode 100644 index 4b1fbb06..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestGpuStateFlags.cs +++ /dev/null @@ -1,10 +0,0 @@ -using System; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - [Flags] - enum GuestGpuStateFlags : byte - { - EarlyZForce = 1 << 0 - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheEntry.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheEntry.cs deleted file mode 100644 index 373fa6c6..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheEntry.cs +++ /dev/null @@ -1,88 +0,0 @@ -using System; -using System.Collections.Generic; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Represent a cached shader entry in a guest shader program. - /// </summary> - class GuestShaderCacheEntry - { - /// <summary> - /// The header of the cached shader entry. - /// </summary> - public GuestShaderCacheEntryHeader Header { get; } - - /// <summary> - /// The code of this shader. - /// </summary> - /// <remarks>If a Vertex A is present, this also contains the code 2 section.</remarks> - public byte[] Code { get; } - - /// <summary> - /// The textures descriptors used for this shader. - /// </summary> - public Dictionary<int, GuestTextureDescriptor> TextureDescriptors { get; } - - /// <summary> - /// Create a new instance of <see cref="GuestShaderCacheEntry"/>. - /// </summary> - /// <param name="header">The header of the cached shader entry</param> - /// <param name="code">The code of this shader</param> - public GuestShaderCacheEntry(GuestShaderCacheEntryHeader header, byte[] code) - { - Header = header; - Code = code; - TextureDescriptors = new Dictionary<int, GuestTextureDescriptor>(); - } - - /// <summary> - /// Parse a raw cached user shader program into an array of shader cache entry. - /// </summary> - /// <param name="data">The raw cached user shader program</param> - /// <param name="fileHeader">The user shader program header</param> - /// <returns>An array of shader cache entry</returns> - public static GuestShaderCacheEntry[] Parse(ref ReadOnlySpan<byte> data, out GuestShaderCacheHeader fileHeader) - { - fileHeader = MemoryMarshal.Read<GuestShaderCacheHeader>(data); - - data = data.Slice(Unsafe.SizeOf<GuestShaderCacheHeader>()); - - ReadOnlySpan<GuestShaderCacheEntryHeader> entryHeaders = MemoryMarshal.Cast<byte, GuestShaderCacheEntryHeader>(data.Slice(0, fileHeader.Count * Unsafe.SizeOf<GuestShaderCacheEntryHeader>())); - - data = data.Slice(fileHeader.Count * Unsafe.SizeOf<GuestShaderCacheEntryHeader>()); - - GuestShaderCacheEntry[] result = new GuestShaderCacheEntry[fileHeader.Count]; - - for (int i = 0; i < result.Length; i++) - { - GuestShaderCacheEntryHeader header = entryHeaders[i]; - - // Ignore empty entries - if (header.Size == 0 && header.SizeA == 0) - { - continue; - } - - byte[] code = data.Slice(0, header.Size + header.SizeA).ToArray(); - - data = data.Slice(header.Size + header.SizeA); - - result[i] = new GuestShaderCacheEntry(header, code); - - ReadOnlySpan<GuestTextureDescriptor> textureDescriptors = MemoryMarshal.Cast<byte, GuestTextureDescriptor>(data.Slice(0, header.GpuAccessorHeader.TextureDescriptorCount * Unsafe.SizeOf<GuestTextureDescriptor>())); - - foreach (GuestTextureDescriptor textureDescriptor in textureDescriptors) - { - result[i].TextureDescriptors.Add((int)textureDescriptor.Handle, textureDescriptor); - } - - data = data.Slice(header.GpuAccessorHeader.TextureDescriptorCount * Unsafe.SizeOf<GuestTextureDescriptor>()); - } - - return result; - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheEntryHeader.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheEntryHeader.cs deleted file mode 100644 index 9b22cac5..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheEntryHeader.cs +++ /dev/null @@ -1,69 +0,0 @@ -using Ryujinx.Graphics.Shader; -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// The header of a guest shader entry in a guest shader program. - /// </summary> - [StructLayout(LayoutKind.Sequential, Pack = 0x1, Size = 0x30)] - struct GuestShaderCacheEntryHeader - { - /// <summary> - /// The stage of this shader. - /// </summary> - public ShaderStage Stage; - - /// <summary> - /// Unused/reserved. - /// </summary> - public byte Reserved1; - - /// <summary> - /// Unused/reserved. - /// </summary> - public byte Reserved2; - - /// <summary> - /// Unused/reserved. - /// </summary> - public byte Reserved3; - - /// <summary> - /// The size of the code section. - /// </summary> - public int Size; - - /// <summary> - /// The size of the code2 section if present. (Vertex A) - /// </summary> - public int SizeA; - - /// <summary> - /// Constant buffer 1 data size. - /// </summary> - public int Cb1DataSize; - - /// <summary> - /// The header of the cached gpu accessor. - /// </summary> - public GuestGpuAccessorHeader GpuAccessorHeader; - - /// <summary> - /// Create a new guest shader entry header. - /// </summary> - /// <param name="stage">The stage of this shader</param> - /// <param name="size">The size of the code section</param> - /// <param name="sizeA">The size of the code2 section if present (Vertex A)</param> - /// <param name="cb1DataSize">Constant buffer 1 data size</param> - /// <param name="gpuAccessorHeader">The header of the cached gpu accessor</param> - public GuestShaderCacheEntryHeader(ShaderStage stage, int size, int sizeA, int cb1DataSize, GuestGpuAccessorHeader gpuAccessorHeader) : this() - { - Stage = stage; - Size = size; - SizeA = sizeA; - Cb1DataSize = cb1DataSize; - GpuAccessorHeader = gpuAccessorHeader; - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheHeader.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheHeader.cs deleted file mode 100644 index 700be47d..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheHeader.cs +++ /dev/null @@ -1,42 +0,0 @@ -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// The header of a shader program in the guest cache. - /// </summary> - [StructLayout(LayoutKind.Sequential, Pack = 0x1, Size = 0x10)] - struct GuestShaderCacheHeader - { - /// <summary> - /// The count of shaders defining this program. - /// </summary> - public byte Count; - - /// <summary> - /// The count of transform feedback data used in this program. - /// </summary> - public byte TransformFeedbackCount; - - /// <summary> - /// Unused/reserved. - /// </summary> - public ushort Reserved1; - - /// <summary> - /// Unused/reserved. - /// </summary> - public ulong Reserved2; - - /// <summary> - /// Create a new guest shader cache header. - /// </summary> - /// <param name="count">The count of shaders defining this program</param> - /// <param name="transformFeedbackCount">The count of transform feedback data used in this program</param> - public GuestShaderCacheHeader(byte count, byte transformFeedbackCount) : this() - { - Count = count; - TransformFeedbackCount = transformFeedbackCount; - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheTransformFeedbackHeader.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheTransformFeedbackHeader.cs deleted file mode 100644 index 18cfdf55..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestShaderCacheTransformFeedbackHeader.cs +++ /dev/null @@ -1,38 +0,0 @@ -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Header for transform feedback. - /// </summary> - [StructLayout(LayoutKind.Sequential, Pack = 1, Size = 0x10)] - struct GuestShaderCacheTransformFeedbackHeader - { - /// <summary> - /// The buffer index of the transform feedback. - /// </summary> - public int BufferIndex; - - /// <summary> - /// The stride of the transform feedback. - /// </summary> - public int Stride; - - /// <summary> - /// The length of the varying location buffer of the transform feedback. - /// </summary> - public int VaryingLocationsLength; - - /// <summary> - /// Reserved/unused. - /// </summary> - public int Reserved1; - - public GuestShaderCacheTransformFeedbackHeader(int bufferIndex, int stride, int varyingLocationsLength) : this() - { - BufferIndex = bufferIndex; - Stride = stride; - VaryingLocationsLength = varyingLocationsLength; - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestTextureDescriptor.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestTextureDescriptor.cs deleted file mode 100644 index 9491496d..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/GuestTextureDescriptor.cs +++ /dev/null @@ -1,41 +0,0 @@ -using Ryujinx.Graphics.Gpu.Image; -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Contains part of TextureDescriptor from <see cref="Image"/> used for shader codegen. - /// </summary> - [StructLayout(LayoutKind.Sequential, Size = 0xC, Pack = 1)] - struct GuestTextureDescriptor : ITextureDescriptor - { - public uint Handle; - public uint Format; - public TextureTarget Target; - [MarshalAs(UnmanagedType.I1)] - public bool IsSrgb; - [MarshalAs(UnmanagedType.I1)] - public bool IsTextureCoordNormalized; - public byte Reserved; - - public uint UnpackFormat() - { - return Format; - } - - public bool UnpackSrgb() - { - return IsSrgb; - } - - public bool UnpackTextureCoordNormalized() - { - return IsTextureCoordNormalized; - } - - public TextureTarget UnpackTextureTarget() - { - return Target; - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheEntry.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheEntry.cs deleted file mode 100644 index fe79acb3..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheEntry.cs +++ /dev/null @@ -1,222 +0,0 @@ -using Ryujinx.Common; -using Ryujinx.Graphics.Shader; -using System; -using System.IO; -using System.Linq; -using System.Runtime.CompilerServices; -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Host shader entry used for binding information. - /// </summary> - class HostShaderCacheEntry - { - /// <summary> - /// The header of the cached shader entry. - /// </summary> - public HostShaderCacheEntryHeader Header { get; } - - /// <summary> - /// Cached constant buffers. - /// </summary> - public BufferDescriptor[] CBuffers { get; } - - /// <summary> - /// Cached storage buffers. - /// </summary> - public BufferDescriptor[] SBuffers { get; } - - /// <summary> - /// Cached texture descriptors. - /// </summary> - public TextureDescriptor[] Textures { get; } - - /// <summary> - /// Cached image descriptors. - /// </summary> - public TextureDescriptor[] Images { get; } - - /// <summary> - /// Create a new instance of <see cref="HostShaderCacheEntry"/>. - /// </summary> - /// <param name="header">The header of the cached shader entry</param> - /// <param name="cBuffers">Cached constant buffers</param> - /// <param name="sBuffers">Cached storage buffers</param> - /// <param name="textures">Cached texture descriptors</param> - /// <param name="images">Cached image descriptors</param> - private HostShaderCacheEntry( - HostShaderCacheEntryHeader header, - BufferDescriptor[] cBuffers, - BufferDescriptor[] sBuffers, - TextureDescriptor[] textures, - TextureDescriptor[] images) - { - Header = header; - CBuffers = cBuffers; - SBuffers = sBuffers; - Textures = textures; - Images = images; - } - - private HostShaderCacheEntry() - { - Header = new HostShaderCacheEntryHeader(); - CBuffers = new BufferDescriptor[0]; - SBuffers = new BufferDescriptor[0]; - Textures = new TextureDescriptor[0]; - Images = new TextureDescriptor[0]; - } - - private HostShaderCacheEntry(ShaderProgramInfo programInfo) - { - Header = new HostShaderCacheEntryHeader(programInfo.CBuffers.Count, - programInfo.SBuffers.Count, - programInfo.Textures.Count, - programInfo.Images.Count, - programInfo.UsesInstanceId, - programInfo.UsesRtLayer, - programInfo.ClipDistancesWritten, - programInfo.FragmentOutputMap); - CBuffers = programInfo.CBuffers.ToArray(); - SBuffers = programInfo.SBuffers.ToArray(); - Textures = programInfo.Textures.ToArray(); - Images = programInfo.Images.ToArray(); - } - - /// <summary> - /// Convert the host shader entry to a <see cref="ShaderProgramInfo"/>. - /// </summary> - /// <returns>A new <see cref="ShaderProgramInfo"/> from this instance</returns> - internal ShaderProgramInfo ToShaderProgramInfo() - { - return new ShaderProgramInfo( - CBuffers, - SBuffers, - Textures, - Images, - default, - Header.UseFlags.HasFlag(UseFlags.InstanceId), - Header.UseFlags.HasFlag(UseFlags.RtLayer), - Header.ClipDistancesWritten, - Header.FragmentOutputMap); - } - - /// <summary> - /// Parse a raw cached user shader program into an array of shader cache entry. - /// </summary> - /// <param name="data">The raw cached host shader</param> - /// <param name="programCode">The host shader program</param> - /// <returns>An array of shader cache entry</returns> - internal static HostShaderCacheEntry[] Parse(ReadOnlySpan<byte> data, out ReadOnlySpan<byte> programCode) - { - HostShaderCacheHeader fileHeader = MemoryMarshal.Read<HostShaderCacheHeader>(data); - - data = data.Slice(Unsafe.SizeOf<HostShaderCacheHeader>()); - - ReadOnlySpan<HostShaderCacheEntryHeader> entryHeaders = MemoryMarshal.Cast<byte, HostShaderCacheEntryHeader>(data.Slice(0, fileHeader.Count * Unsafe.SizeOf<HostShaderCacheEntryHeader>())); - - data = data.Slice(fileHeader.Count * Unsafe.SizeOf<HostShaderCacheEntryHeader>()); - - HostShaderCacheEntry[] result = new HostShaderCacheEntry[fileHeader.Count]; - - for (int i = 0; i < result.Length; i++) - { - HostShaderCacheEntryHeader header = entryHeaders[i]; - - if (!header.InUse) - { - continue; - } - - int cBufferDescriptorsSize = header.CBuffersCount * Unsafe.SizeOf<BufferDescriptor>(); - int sBufferDescriptorsSize = header.SBuffersCount * Unsafe.SizeOf<BufferDescriptor>(); - int textureDescriptorsSize = header.TexturesCount * Unsafe.SizeOf<TextureDescriptor>(); - int imageDescriptorsSize = header.ImagesCount * Unsafe.SizeOf<TextureDescriptor>(); - - ReadOnlySpan<BufferDescriptor> cBuffers = MemoryMarshal.Cast<byte, BufferDescriptor>(data.Slice(0, cBufferDescriptorsSize)); - data = data.Slice(cBufferDescriptorsSize); - - ReadOnlySpan<BufferDescriptor> sBuffers = MemoryMarshal.Cast<byte, BufferDescriptor>(data.Slice(0, sBufferDescriptorsSize)); - data = data.Slice(sBufferDescriptorsSize); - - ReadOnlySpan<TextureDescriptor> textureDescriptors = MemoryMarshal.Cast<byte, TextureDescriptor>(data.Slice(0, textureDescriptorsSize)); - data = data.Slice(textureDescriptorsSize); - - ReadOnlySpan<TextureDescriptor> imageDescriptors = MemoryMarshal.Cast<byte, TextureDescriptor>(data.Slice(0, imageDescriptorsSize)); - data = data.Slice(imageDescriptorsSize); - - result[i] = new HostShaderCacheEntry(header, cBuffers.ToArray(), sBuffers.ToArray(), textureDescriptors.ToArray(), imageDescriptors.ToArray()); - } - - programCode = data.Slice(0, fileHeader.CodeSize); - - return result; - } - - /// <summary> - /// Create a new host shader cache file. - /// </summary> - /// <param name="programCode">The host shader program</param> - /// <param name="codeHolders">The shaders code holder</param> - /// <returns>Raw data of a new host shader cache file</returns> - internal static byte[] Create(ReadOnlySpan<byte> programCode, CachedShaderStage[] codeHolders) - { - HostShaderCacheHeader header = new HostShaderCacheHeader((byte)codeHolders.Length, programCode.Length); - - HostShaderCacheEntry[] entries = new HostShaderCacheEntry[codeHolders.Length]; - - for (int i = 0; i < codeHolders.Length; i++) - { - if (codeHolders[i] == null) - { - entries[i] = new HostShaderCacheEntry(); - } - else - { - entries[i] = new HostShaderCacheEntry(codeHolders[i].Info); - } - } - - using (MemoryStream stream = new MemoryStream()) - { - BinaryWriter writer = new BinaryWriter(stream); - - writer.WriteStruct(header); - - foreach (HostShaderCacheEntry entry in entries) - { - writer.WriteStruct(entry.Header); - } - - foreach (HostShaderCacheEntry entry in entries) - { - foreach (BufferDescriptor cBuffer in entry.CBuffers) - { - writer.WriteStruct(cBuffer); - } - - foreach (BufferDescriptor sBuffer in entry.SBuffers) - { - writer.WriteStruct(sBuffer); - } - - foreach (TextureDescriptor texture in entry.Textures) - { - writer.WriteStruct(texture); - } - - foreach (TextureDescriptor image in entry.Images) - { - writer.WriteStruct(image); - } - } - - writer.Write(programCode); - - return stream.ToArray(); - } - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheEntryHeader.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheEntryHeader.cs deleted file mode 100644 index c3c0de22..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheEntryHeader.cs +++ /dev/null @@ -1,114 +0,0 @@ -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// Flags indicating if the shader accesses certain built-ins, such as the instance ID. - /// </summary> - enum UseFlags : byte - { - /// <summary> - /// None of the built-ins are used. - /// </summary> - None = 0, - - /// <summary> - /// Indicates whenever the vertex shader reads the gl_InstanceID built-in. - /// </summary> - InstanceId = 1 << 0, - - /// <summary> - /// Indicates whenever any of the VTG stages writes to the gl_Layer built-in. - /// </summary> - RtLayer = 1 << 1 - } - - /// <summary> - /// Host shader entry header used for binding information. - /// </summary> - [StructLayout(LayoutKind.Sequential, Pack = 1, Size = 0x18)] - struct HostShaderCacheEntryHeader - { - /// <summary> - /// Count of constant buffer descriptors. - /// </summary> - public int CBuffersCount; - - /// <summary> - /// Count of storage buffer descriptors. - /// </summary> - public int SBuffersCount; - - /// <summary> - /// Count of texture descriptors. - /// </summary> - public int TexturesCount; - - /// <summary> - /// Count of image descriptors. - /// </summary> - public int ImagesCount; - - /// <summary> - /// Flags indicating if the shader accesses certain built-ins, such as the instance ID. - /// </summary> - public UseFlags UseFlags; - - /// <summary> - /// Set to true if this entry is in use. - /// </summary> - [MarshalAs(UnmanagedType.I1)] - public bool InUse; - - /// <summary> - /// Mask of clip distances that are written to on the shader. - /// </summary> - public byte ClipDistancesWritten; - - /// <summary> - /// Reserved / unused. - /// </summary> - public byte Reserved; - - /// <summary> - /// Mask of components written by the fragment shader stage. - /// </summary> - public int FragmentOutputMap; - - /// <summary> - /// Create a new host shader cache entry header. - /// </summary> - /// <param name="cBuffersCount">Count of constant buffer descriptors</param> - /// <param name="sBuffersCount">Count of storage buffer descriptors</param> - /// <param name="texturesCount">Count of texture descriptors</param> - /// <param name="imagesCount">Count of image descriptors</param> - /// <param name="usesInstanceId">Set to true if the shader uses instance id</param> - /// <param name="clipDistancesWritten">Mask of clip distances that are written to on the shader</param> - /// <param name="fragmentOutputMap">Mask of components written by the fragment shader stage</param> - public HostShaderCacheEntryHeader( - int cBuffersCount, - int sBuffersCount, - int texturesCount, - int imagesCount, - bool usesInstanceId, - bool usesRtLayer, - byte clipDistancesWritten, - int fragmentOutputMap) : this() - { - CBuffersCount = cBuffersCount; - SBuffersCount = sBuffersCount; - TexturesCount = texturesCount; - ImagesCount = imagesCount; - ClipDistancesWritten = clipDistancesWritten; - FragmentOutputMap = fragmentOutputMap; - InUse = true; - - UseFlags = usesInstanceId ? UseFlags.InstanceId : UseFlags.None; - - if (usesRtLayer) - { - UseFlags |= UseFlags.RtLayer; - } - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheHeader.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheHeader.cs deleted file mode 100644 index 27f216cc..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Definition/HostShaderCacheHeader.cs +++ /dev/null @@ -1,42 +0,0 @@ -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache.Definition -{ - /// <summary> - /// The header of a shader program in the guest cache. - /// </summary> - [StructLayout(LayoutKind.Sequential, Pack = 0x1, Size = 0x10)] - struct HostShaderCacheHeader - { - /// <summary> - /// The count of shaders defining this program. - /// </summary> - public byte Count; - - /// <summary> - /// Unused/reserved. - /// </summary> - public byte Reserved1; - - /// <summary> - /// Unused/reserved. - /// </summary> - public ushort Reserved2; - - /// <summary> - /// Size of the shader binary. - /// </summary> - public int CodeSize; - - /// <summary> - /// Create a new host shader cache header. - /// </summary> - /// <param name="count">The count of shaders defining this program</param> - /// <param name="codeSize">The size of the shader binary</param> - public HostShaderCacheHeader(byte count, int codeSize) : this() - { - Count = count; - CodeSize = codeSize; - } - } -} diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/Migration.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/Migration.cs deleted file mode 100644 index 885bcd09..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/Migration.cs +++ /dev/null @@ -1,258 +0,0 @@ -using Ryujinx.Common; -using Ryujinx.Common.Logging; -using Ryujinx.Common.Memory; -using Ryujinx.Graphics.GAL; -using Ryujinx.Graphics.Gpu.Engine.Threed; -using Ryujinx.Graphics.Gpu.Shader.Cache.Definition; -using Ryujinx.Graphics.Gpu.Shader.DiskCache; -using Ryujinx.Graphics.Shader; -using System; -using System.Collections.Generic; -using System.Diagnostics; -using System.IO; -using System.Runtime.InteropServices; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache -{ - /// <summary> - /// Class handling shader cache migrations. - /// </summary> - static class Migration - { - // Last codegen version before the migration to the new cache. - private const ulong ShaderCodeGenVersion = 3054; - - /// <summary> - /// Migrates from the old cache format to the new one. - /// </summary> - /// <param name="context">GPU context</param> - /// <param name="hostStorage">Disk cache host storage (used to create the new shader files)</param> - /// <returns>Number of migrated shaders</returns> - public static int MigrateFromLegacyCache(GpuContext context, DiskCacheHostStorage hostStorage) - { - string baseCacheDirectory = CacheHelper.GetBaseCacheDirectory(GraphicsConfig.TitleId); - string cacheDirectory = CacheHelper.GenerateCachePath(baseCacheDirectory, CacheGraphicsApi.Guest, "", "program"); - - // If the directory does not exist, we have no old cache. - // Exist early as the CacheManager constructor will create the directories. - if (!Directory.Exists(cacheDirectory)) - { - return 0; - } - - if (GraphicsConfig.EnableShaderCache && GraphicsConfig.TitleId != null) - { - CacheManager cacheManager = new CacheManager(CacheGraphicsApi.OpenGL, CacheHashType.XxHash128, "glsl", GraphicsConfig.TitleId, ShaderCodeGenVersion); - - bool isReadOnly = cacheManager.IsReadOnly; - - HashSet<Hash128> invalidEntries = null; - - if (isReadOnly) - { - Logger.Warning?.Print(LogClass.Gpu, "Loading shader cache in read-only mode (cache in use by another program!)"); - } - else - { - invalidEntries = new HashSet<Hash128>(); - } - - ReadOnlySpan<Hash128> guestProgramList = cacheManager.GetGuestProgramList(); - - for (int programIndex = 0; programIndex < guestProgramList.Length; programIndex++) - { - Hash128 key = guestProgramList[programIndex]; - - byte[] guestProgram = cacheManager.GetGuestProgramByHash(ref key); - - if (guestProgram == null) - { - Logger.Error?.Print(LogClass.Gpu, $"Ignoring orphan shader hash {key} in cache (is the cache incomplete?)"); - - continue; - } - - ReadOnlySpan<byte> guestProgramReadOnlySpan = guestProgram; - - ReadOnlySpan<GuestShaderCacheEntry> cachedShaderEntries = GuestShaderCacheEntry.Parse(ref guestProgramReadOnlySpan, out GuestShaderCacheHeader fileHeader); - - if (cachedShaderEntries[0].Header.Stage == ShaderStage.Compute) - { - Debug.Assert(cachedShaderEntries.Length == 1); - - GuestShaderCacheEntry entry = cachedShaderEntries[0]; - - byte[] code = entry.Code.AsSpan(0, entry.Header.Size - entry.Header.Cb1DataSize).ToArray(); - - Span<byte> codeSpan = entry.Code; - byte[] cb1Data = codeSpan.Slice(codeSpan.Length - entry.Header.Cb1DataSize).ToArray(); - - ShaderProgramInfo info = new ShaderProgramInfo( - Array.Empty<BufferDescriptor>(), - Array.Empty<BufferDescriptor>(), - Array.Empty<TextureDescriptor>(), - Array.Empty<TextureDescriptor>(), - ShaderStage.Compute, - false, - false, - 0, - 0); - - GpuChannelComputeState computeState = new GpuChannelComputeState( - entry.Header.GpuAccessorHeader.ComputeLocalSizeX, - entry.Header.GpuAccessorHeader.ComputeLocalSizeY, - entry.Header.GpuAccessorHeader.ComputeLocalSizeZ, - entry.Header.GpuAccessorHeader.ComputeLocalMemorySize, - entry.Header.GpuAccessorHeader.ComputeSharedMemorySize); - - ShaderSpecializationState specState = new ShaderSpecializationState(computeState); - - foreach (var td in entry.TextureDescriptors) - { - var handle = td.Key; - var data = td.Value; - - specState.RegisterTexture( - 0, - handle, - -1, - data.UnpackFormat(), - data.UnpackSrgb(), - data.UnpackTextureTarget(), - data.UnpackTextureCoordNormalized()); - } - - CachedShaderStage shader = new CachedShaderStage(info, code, cb1Data); - CachedShaderProgram program = new CachedShaderProgram(null, specState, shader); - - hostStorage.AddShader(context, program, ReadOnlySpan<byte>.Empty); - } - else - { - Debug.Assert(cachedShaderEntries.Length == Constants.ShaderStages); - - CachedShaderStage[] shaders = new CachedShaderStage[Constants.ShaderStages + 1]; - List<ShaderProgram> shaderPrograms = new List<ShaderProgram>(); - - TransformFeedbackDescriptorOld[] tfd = CacheHelper.ReadTransformFeedbackInformation(ref guestProgramReadOnlySpan, fileHeader); - - GuestShaderCacheEntry[] entries = cachedShaderEntries.ToArray(); - - GuestGpuAccessorHeader accessorHeader = entries[0].Header.GpuAccessorHeader; - - TessMode tessMode = new TessMode(); - - int tessPatchType = accessorHeader.TessellationModePacked & 3; - int tessSpacing = (accessorHeader.TessellationModePacked >> 2) & 3; - bool tessCw = (accessorHeader.TessellationModePacked & 0x10) != 0; - - tessMode.Packed = (uint)tessPatchType; - tessMode.Packed |= (uint)(tessSpacing << 4); - - if (tessCw) - { - tessMode.Packed |= 0x100; - } - - PrimitiveTopology topology = accessorHeader.PrimitiveTopology switch - { - InputTopology.Lines => PrimitiveTopology.Lines, - InputTopology.LinesAdjacency => PrimitiveTopology.LinesAdjacency, - InputTopology.Triangles => PrimitiveTopology.Triangles, - InputTopology.TrianglesAdjacency => PrimitiveTopology.TrianglesAdjacency, - _ => PrimitiveTopology.Points - }; - - GpuChannelGraphicsState graphicsState = new GpuChannelGraphicsState( - accessorHeader.StateFlags.HasFlag(GuestGpuStateFlags.EarlyZForce), - topology, - tessMode, - false, - false, - false); - - TransformFeedbackDescriptor[] tfdNew = null; - - if (tfd != null) - { - tfdNew = new TransformFeedbackDescriptor[tfd.Length]; - - for (int tfIndex = 0; tfIndex < tfd.Length; tfIndex++) - { - Array32<uint> varyingLocations = new Array32<uint>(); - Span<byte> varyingLocationsSpan = MemoryMarshal.Cast<uint, byte>(varyingLocations.ToSpan()); - tfd[tfIndex].VaryingLocations.CopyTo(varyingLocationsSpan.Slice(0, tfd[tfIndex].VaryingLocations.Length)); - - tfdNew[tfIndex] = new TransformFeedbackDescriptor( - tfd[tfIndex].BufferIndex, - tfd[tfIndex].Stride, - tfd[tfIndex].VaryingLocations.Length, - ref varyingLocations); - } - } - - ShaderSpecializationState specState = new ShaderSpecializationState(graphicsState, tfdNew); - - for (int i = 0; i < entries.Length; i++) - { - GuestShaderCacheEntry entry = entries[i]; - - if (entry == null) - { - continue; - } - - ShaderProgramInfo info = new ShaderProgramInfo( - Array.Empty<BufferDescriptor>(), - Array.Empty<BufferDescriptor>(), - Array.Empty<TextureDescriptor>(), - Array.Empty<TextureDescriptor>(), - (ShaderStage)(i + 1), - false, - false, - 0, - 0); - - // NOTE: Vertex B comes first in the shader cache. - byte[] code = entry.Code.AsSpan(0, entry.Header.Size - entry.Header.Cb1DataSize).ToArray(); - byte[] code2 = entry.Header.SizeA != 0 ? entry.Code.AsSpan(entry.Header.Size, entry.Header.SizeA).ToArray() : null; - - Span<byte> codeSpan = entry.Code; - byte[] cb1Data = codeSpan.Slice(codeSpan.Length - entry.Header.Cb1DataSize).ToArray(); - - shaders[i + 1] = new CachedShaderStage(info, code, cb1Data); - - if (code2 != null) - { - shaders[0] = new CachedShaderStage(null, code2, cb1Data); - } - - foreach (var td in entry.TextureDescriptors) - { - var handle = td.Key; - var data = td.Value; - - specState.RegisterTexture( - i, - handle, - -1, - data.UnpackFormat(), - data.UnpackSrgb(), - data.UnpackTextureTarget(), - data.UnpackTextureCoordNormalized()); - } - } - - CachedShaderProgram program = new CachedShaderProgram(null, specState, shaders); - - hostStorage.AddShader(context, program, ReadOnlySpan<byte>.Empty); - } - } - - return guestProgramList.Length; - } - - return 0; - } - } -}
\ No newline at end of file diff --git a/Ryujinx.Graphics.Gpu/Shader/Cache/TransformFeedbackDescriptorOld.cs b/Ryujinx.Graphics.Gpu/Shader/Cache/TransformFeedbackDescriptorOld.cs deleted file mode 100644 index 5e9c6711..00000000 --- a/Ryujinx.Graphics.Gpu/Shader/Cache/TransformFeedbackDescriptorOld.cs +++ /dev/null @@ -1,19 +0,0 @@ -using System; - -namespace Ryujinx.Graphics.Gpu.Shader.Cache -{ - struct TransformFeedbackDescriptorOld - { - public int BufferIndex { get; } - public int Stride { get; } - - public byte[] VaryingLocations { get; } - - public TransformFeedbackDescriptorOld(int bufferIndex, int stride, byte[] varyingLocations) - { - BufferIndex = bufferIndex; - Stride = stride; - VaryingLocations = varyingLocations ?? throw new ArgumentNullException(nameof(varyingLocations)); - } - } -} |