Add ldn_mitm as a network client for LDN (#5656)

* Add relevant files from private repo

Hopefully I didn't miss anything.

JsonHelper.cs is a debug only change
I only added line 810-812 in IUserLocalCommunicationService.cs
for the new Spacemeowx2Ldn case.

* Add a small README.md

just for fun

* Add note about NetCoreServer update to 5.1.0

* Fix a few issues

Fix usage of wrong broadcast address
Log warning if empty userstring was received
and don't add them to outNetworkInfo

* Add warning about incompatibility with public LDN version

* Add missing changes from old_master

* Adjust ldn_mitm for Ryujinx/Ryujinx#3805

* ldn: Adapt to changes from #4582

* ldn_mitm: First cleanup iteration

* ldn_mitm: Second cleanup iteration

* Credit spacemeowx2 in README.md

* Address first review comments by AcK

Adhere to Ryujinx coding style
Remove leftover log calls
Change category of a few log calls
Remove leftover debug notes

* Replace return type with void for methods always returning true

* Address first review comments by riperiperi

Purely stylistic changes:
- Adhere to naming style for internal fields
- Improve code formatting

* Throw InvalidOperationException when calling wrong ldn proxy methods

* Add missing newlines in LanDiscovery.Scan()

* Fix Linux not receiving broadcast packets

* Remove ILdnUdpSocket

It's very unlikely that we will ever need a udp client.
Thus we should simplify LanDiscovery initialization
and remove the parameter of InitUdp().

* ldn_mitm: Improve formatting

* fixup! Fix Linux not receiving broadcast packets

By opening the udp server on 'LocalBroadcastAddr'
Linux refused to answer packets going to LocalAddr.
So in order to fix this problem, Linux now opens two LdnProxyUdpServers.

* ldn_mitm: Fix assigning incorrect NodeIds

This just made connecting a lot more reliable! Thanks @riperiperi

* Fix node ids when leaving/joining

* Change NodeId behaviour to work like RyuLdn

* Change timing for accept and network info being reported.

* Wait for connection before sending anything.

* Remove ConnectAsync() from ILdnTcpSocket

* Only broadcast scan responses if we're hosting a network.

* Fix some filters, scan network duplication.

* Fix silly mistake

* Don't die on duplicates, just replace.

* Lock around node updates

These can happen from multiple threads.

* ldn_mitm: Fix namespaces for Types

Improve formatting
Add warning if compression failed

* Add quicker scan, forgetting networks that disappear.

* Always force a network sync when updating AdvertiseData

* Fix TCP frame size being too large for compressed frames

* Allow ldn_mitm to pass -1 id for room localcommunicationids.

* ldn_mitm: Match server socket options

* ldn_mitm: Use correct socket options

* ldn_mitm: Remove TCP broadcast socket options

* config: Rename Spacemeowx2Ldn to LdnMitm

* ldn_mitm: Generate random fake SSID

* ldn_mitm: Adjust logging statements/levels

* ldn_mitm: Add missing Stop() call for udp2

* ldn_mitm: Adjust formatting

* ldn_mitm: Add stub comments and adjust existing ones

* ldn: Add LdnConst class & set tx/rx buffer sizes correctly

* Move LdnConst out of UserServiceCreator

Replace a few values with LdnConsts

* ldn: Adjust namespaces and client names

* ldn_mitm: Adjust formatting

* ldn: Rename RyuLdn to LdnRyu

* Replace LanProtocol.Read() refs with scoped refs

* Add MIT license for ldn_mitm

* Clarify that network interface is also used for LDN

Although it's currently only used by ldn_mitm,
it would probably be more confusing to exclude RyuLdn there.

* Fix giving a station node id 0

* Update Nuget packages

* Remove LdnHelper

* Add update functions for EnableInternetAccess setting

* ldn: Log MultiplayerMode and DisableP2P

* ldn: Adjust namespaces

* Apply formatting

* Conform to Ryujinx code style

* Remove ldn_mitm from THIRDPARTY.md

It shouldn't have been there in the first place.

* Improve formatting

---------

Co-authored-by: riperiperi <rhy3756547@hotmail.com>
Co-authored-by: Ac_K <Acoustik666@gmail.com>

6 files changed, 413 insertions, 0 deletions

diff --git a/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/ILdnSocket.cs b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/ILdnSocket.cs
new file mode 100644
index 00000000..b6e6cea9
--- /dev/null
+++ b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/ILdnSocket.cs
@@ -0,0 +1,12 @@
+using System;
+using System.Net;
+
+namespace Ryujinx.HLE.HOS.Services.Ldn.UserServiceCreator.LdnMitm.Proxy
+{
+    internal interface ILdnSocket : IDisposable
+    {
+        bool SendPacketAsync(EndPoint endpoint, byte[] buffer);
+        bool Start();
+        bool Stop();
+    }
+}
diff --git a/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/ILdnTcpSocket.cs b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/ILdnTcpSocket.cs
new file mode 100644
index 00000000..97e3bd62
--- /dev/null
+++ b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/ILdnTcpSocket.cs
@@ -0,0 +1,8 @@
+namespace Ryujinx.HLE.HOS.Services.Ldn.UserServiceCreator.LdnMitm.Proxy
+{
+    internal interface ILdnTcpSocket : ILdnSocket
+    {
+        bool Connect();
+        void DisconnectAndStop();
+    }
+}
diff --git a/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpClient.cs b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpClient.cs
new file mode 100644
index 00000000..cfe9a8aa
--- /dev/null
+++ b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpClient.cs
@@ -0,0 +1,99 @@
+using Ryujinx.Common.Logging;
+using System;
+using System.Net;
+using System.Net.Sockets;
+using System.Threading;
+
+namespace Ryujinx.HLE.HOS.Services.Ldn.UserServiceCreator.LdnMitm.Proxy
+{
+    internal class LdnProxyTcpClient : NetCoreServer.TcpClient, ILdnTcpSocket
+    {
+        private readonly LanProtocol _protocol;
+        private byte[] _buffer;
+        private int _bufferEnd;
+
+        public LdnProxyTcpClient(LanProtocol protocol, IPAddress address, int port) : base(address, port)
+        {
+            _protocol = protocol;
+            _buffer = new byte[LanProtocol.BufferSize];
+            OptionSendBufferSize = LanProtocol.TcpTxBufferSize;
+            OptionReceiveBufferSize = LanProtocol.TcpRxBufferSize;
+            OptionSendBufferLimit = LanProtocol.TxBufferSizeMax;
+            OptionReceiveBufferLimit = LanProtocol.RxBufferSizeMax;
+        }
+
+        protected override void OnConnected()
+        {
+            Logger.Info?.PrintMsg(LogClass.ServiceLdn, $"LdnProxyTCPClient connected!");
+        }
+
+        protected override void OnReceived(byte[] buffer, long offset, long size)
+        {
+            _protocol.Read(ref _buffer, ref _bufferEnd, buffer, (int)offset, (int)size);
+        }
+
+        public void DisconnectAndStop()
+        {
+            DisconnectAsync();
+
+            while (IsConnected)
+            {
+                Thread.Yield();
+            }
+        }
+
+        public bool SendPacketAsync(EndPoint endPoint, byte[] data)
+        {
+            if (endPoint != null)
+            {
+                Logger.Warning?.PrintMsg(LogClass.ServiceLdn, "LdnProxyTcpClient is sending a packet but endpoint is not null.");
+            }
+
+            if (IsConnecting && !IsConnected)
+            {
+                Logger.Info?.PrintMsg(LogClass.ServiceLdn, "LdnProxyTCPClient needs to connect before sending packets. Waiting...");
+
+                while (IsConnecting && !IsConnected)
+                {
+                    Thread.Yield();
+                }
+            }
+
+            return SendAsync(data);
+        }
+
+        protected override void OnError(SocketError error)
+        {
+            Logger.Error?.PrintMsg(LogClass.ServiceLdn, $"LdnProxyTCPClient caught an error with code {error}");
+        }
+
+        protected override void Dispose(bool disposingManagedResources)
+        {
+            DisconnectAndStop();
+            base.Dispose(disposingManagedResources);
+        }
+
+        public override bool Connect()
+        {
+            // TODO: NetCoreServer has a Connect() method, but it currently leads to weird issues.
+            base.ConnectAsync();
+
+            while (IsConnecting)
+            {
+                Thread.Sleep(1);
+            }
+
+            return IsConnected;
+        }
+
+        public bool Start()
+        {
+            throw new InvalidOperationException("Start was called.");
+        }
+
+        public bool Stop()
+        {
+            throw new InvalidOperationException("Stop was called.");
+        }
+    }
+}
diff --git a/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpServer.cs b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpServer.cs
new file mode 100644
index 00000000..0ca12b9f
--- /dev/null
+++ b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpServer.cs
@@ -0,0 +1,54 @@
+using NetCoreServer;
+using Ryujinx.Common.Logging;
+using System;
+using System.Net;
+using System.Net.Sockets;
+
+namespace Ryujinx.HLE.HOS.Services.Ldn.UserServiceCreator.LdnMitm.Proxy
+{
+    internal class LdnProxyTcpServer : TcpServer, ILdnTcpSocket
+    {
+        private readonly LanProtocol _protocol;
+
+        public LdnProxyTcpServer(LanProtocol protocol, IPAddress address, int port) : base(address, port)
+        {
+            _protocol = protocol;
+            OptionReuseAddress = true;
+            OptionSendBufferSize = LanProtocol.TcpTxBufferSize;
+            OptionReceiveBufferSize = LanProtocol.TcpRxBufferSize;
+
+            Logger.Info?.PrintMsg(LogClass.ServiceLdn, $"LdnProxyTCPServer created a server for this address: {address}:{port}");
+        }
+
+        protected override TcpSession CreateSession()
+        {
+            return new LdnProxyTcpSession(this, _protocol);
+        }
+
+        protected override void OnError(SocketError error)
+        {
+            Logger.Error?.PrintMsg(LogClass.ServiceLdn, $"LdnProxyTCPServer caught an error with code {error}");
+        }
+
+        protected override void Dispose(bool disposingManagedResources)
+        {
+            Stop();
+            base.Dispose(disposingManagedResources);
+        }
+
+        public bool Connect()
+        {
+            throw new InvalidOperationException("Connect was called.");
+        }
+
+        public void DisconnectAndStop()
+        {
+            Stop();
+        }
+
+        public bool SendPacketAsync(EndPoint endpoint, byte[] buffer)
+        {
+            throw new InvalidOperationException("SendPacketAsync was called.");
+        }
+    }
+}
diff --git a/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpSession.cs b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpSession.cs
new file mode 100644
index 00000000..f30c4b01
--- /dev/null
+++ b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyTcpSession.cs
@@ -0,0 +1,83 @@
+using Ryujinx.Common.Logging;
+using Ryujinx.HLE.HOS.Services.Ldn.Types;
+using System.Net;
+using System.Net.Sockets;
+
+namespace Ryujinx.HLE.HOS.Services.Ldn.UserServiceCreator.LdnMitm.Proxy
+{
+    internal class LdnProxyTcpSession : NetCoreServer.TcpSession
+    {
+        private readonly LanProtocol _protocol;
+
+        internal int NodeId;
+        internal NodeInfo NodeInfo;
+
+        private byte[] _buffer;
+        private int _bufferEnd;
+
+        public LdnProxyTcpSession(LdnProxyTcpServer server, LanProtocol protocol) : base(server)
+        {
+            _protocol = protocol;
+            _protocol.Connect += OnConnect;
+            _buffer = new byte[LanProtocol.BufferSize];
+            OptionSendBufferSize = LanProtocol.TcpTxBufferSize;
+            OptionReceiveBufferSize = LanProtocol.TcpRxBufferSize;
+            OptionSendBufferLimit = LanProtocol.TxBufferSizeMax;
+            OptionReceiveBufferLimit = LanProtocol.RxBufferSizeMax;
+        }
+
+        public void OverrideInfo()
+        {
+            NodeInfo.NodeId = (byte)NodeId;
+            NodeInfo.IsConnected = (byte)(IsConnected ? 1 : 0);
+        }
+
+        protected override void OnConnected()
+        {
+            Logger.Info?.PrintMsg(LogClass.ServiceLdn, "LdnProxyTCPSession connected!");
+        }
+
+        protected override void OnDisconnected()
+        {
+            Logger.Info?.PrintMsg(LogClass.ServiceLdn, "LdnProxyTCPSession disconnected!");
+
+            _protocol.InvokeDisconnectStation(this);
+        }
+
+        protected override void OnReceived(byte[] buffer, long offset, long size)
+        {
+            _protocol.Read(ref _buffer, ref _bufferEnd, buffer, (int)offset, (int)size, this.Socket.RemoteEndPoint);
+        }
+
+        protected override void OnError(SocketError error)
+        {
+            Logger.Error?.PrintMsg(LogClass.ServiceLdn, $"LdnProxyTCPSession caught an error with code {error}");
+
+            Dispose();
+        }
+
+        protected override void Dispose(bool disposingManagedResources)
+        {
+            _protocol.Connect -= OnConnect;
+            base.Dispose(disposingManagedResources);
+        }
+
+        private void OnConnect(NodeInfo info, EndPoint endPoint)
+        {
+            try
+            {
+                if (endPoint.Equals(this.Socket.RemoteEndPoint))
+                {
+                    NodeInfo = info;
+                    _protocol.InvokeAccept(this);
+                }
+            }
+            catch (System.ObjectDisposedException)
+            {
+                Logger.Error?.PrintMsg(LogClass.ServiceLdn, $"LdnProxyTCPSession was disposed. [IP: {NodeInfo.Ipv4Address}]");
+
+                _protocol.InvokeDisconnectStation(this);
+            }
+        }
+    }
+}
diff --git a/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyUdpServer.cs b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyUdpServer.cs
new file mode 100644
index 00000000..b1519d1f
--- /dev/null
+++ b/src/Ryujinx.HLE/HOS/Services/Ldn/UserServiceCreator/LdnMitm/Proxy/LdnProxyUdpServer.cs
@@ -0,0 +1,157 @@
+using Ryujinx.Common.Logging;
+using Ryujinx.HLE.HOS.Services.Ldn.Types;
+using Ryujinx.HLE.HOS.Services.Ldn.UserServiceCreator.LdnMitm.Types;
+using System;
+using System.Collections.Generic;
+using System.Diagnostics;
+using System.Net;
+using System.Net.Sockets;
+using System.Threading;
+
+namespace Ryujinx.HLE.HOS.Services.Ldn.UserServiceCreator.LdnMitm.Proxy
+{
+    internal class LdnProxyUdpServer : NetCoreServer.UdpServer, ILdnSocket
+    {
+        private const long ScanFrequency = 1000;
+
+        private readonly LanProtocol _protocol;
+        private byte[] _buffer;
+        private int _bufferEnd;
+
+        private readonly object _scanLock = new();
+
+        private Dictionary<ulong, NetworkInfo> _scanResultsLast = new();
+        private Dictionary<ulong, NetworkInfo> _scanResults = new();
+        private readonly AutoResetEvent _scanResponse = new(false);
+        private long _lastScanTime;
+
+        public LdnProxyUdpServer(LanProtocol protocol, IPAddress address, int port) : base(address, port)
+        {
+            _protocol = protocol;
+            _protocol.Scan += HandleScan;
+            _protocol.ScanResponse += HandleScanResponse;
+            _buffer = new byte[LanProtocol.BufferSize];
+            OptionReuseAddress = true;
+            OptionReceiveBufferSize = LanProtocol.RxBufferSizeMax;
+            OptionSendBufferSize = LanProtocol.TxBufferSizeMax;
+
+            Start();
+        }
+
+        protected override Socket CreateSocket()
+        {
+            return new Socket(Endpoint.AddressFamily, SocketType.Dgram, ProtocolType.Udp)
+            {
+                EnableBroadcast = true,
+            };
+        }
+
+        protected override void OnStarted()
+        {
+            ReceiveAsync();
+        }
+
+        protected override void OnReceived(EndPoint endpoint, byte[] buffer, long offset, long size)
+        {
+            _protocol.Read(ref _buffer, ref _bufferEnd, buffer, (int)offset, (int)size, endpoint);
+            ReceiveAsync();
+        }
+
+        protected override void OnError(SocketError error)
+        {
+            Logger.Error?.PrintMsg(LogClass.ServiceLdn, $"LdnProxyUdpServer caught an error with code {error}");
+        }
+
+        protected override void Dispose(bool disposingManagedResources)
+        {
+            _protocol.Scan -= HandleScan;
+            _protocol.ScanResponse -= HandleScanResponse;
+
+            _scanResponse.Dispose();
+
+            base.Dispose(disposingManagedResources);
+        }
+
+        public bool SendPacketAsync(EndPoint endpoint, byte[] data)
+        {
+            return SendAsync(endpoint, data);
+        }
+
+        private void HandleScan(EndPoint endpoint, LanPacketType type, byte[] data)
+        {
+            _protocol.SendPacket(this, type, data, endpoint);
+        }
+
+        private void HandleScanResponse(NetworkInfo info)
+        {
+            Span<byte> mac = stackalloc byte[8];
+
+            info.Common.MacAddress.AsSpan().CopyTo(mac);
+
+            lock (_scanLock)
+            {
+                _scanResults[BitConverter.ToUInt64(mac)] = info;
+
+                _scanResponse.Set();
+            }
+        }
+
+        public void ClearScanResults()
+        {
+            // Rate limit scans.
+
+            long timeMs = Stopwatch.GetTimestamp() / (Stopwatch.Frequency / 1000);
+            long delay = ScanFrequency - (timeMs - _lastScanTime);
+
+            if (delay > 0)
+            {
+                Thread.Sleep((int)delay);
+            }
+
+            _lastScanTime = timeMs;
+
+            lock (_scanLock)
+            {
+                var newResults = _scanResultsLast;
+                newResults.Clear();
+
+                _scanResultsLast = _scanResults;
+                _scanResults = newResults;
+
+                _scanResponse.Reset();
+            }
+        }
+
+        public Dictionary<ulong, NetworkInfo> GetScanResults()
+        {
+            // NOTE: Try to minimize waiting time for scan results.
+            // After we receive the first response, wait a short time for follow-ups and return.
+            // Responses that were too late to catch will appear in the next scan.
+
+            // ldn_mitm does not do this, but this improves latency for games that expect it to be low (it is on console).
+
+            if (_scanResponse.WaitOne(1000))
+            {
+                // Wait a short while longer in case there are some other responses.
+                Thread.Sleep(33);
+            }
+
+            lock (_scanLock)
+            {
+                var results = new Dictionary<ulong, NetworkInfo>();
+
+                foreach (KeyValuePair<ulong, NetworkInfo> last in _scanResultsLast)
+                {
+                    results[last.Key] = last.Value;
+                }
+
+                foreach (KeyValuePair<ulong, NetworkInfo> scan in _scanResults)
+                {
+                    results[scan.Key] = scan.Value;
+                }
+
+                return results;
+            }
+        }
+    }
+}