Resurrecting this in case someone bumps into this question.
When to use ArraySegment over Memory?
Never, unless you need to call something old that expects an ArraySegment<T>, which I doubt will be the case as it was never that popular.
ArraySegment<T> is just an array, an offset, and a length, which are all exposed directly where you can choose to ignore the offset and length and access the entirety of the array if you want to. There’s also no read-only version of ArraySegment<T>.
Span<T> and Memory<T> can be backed by arrays, similar to ArraySegment<T>, but also by strings and unmanaged memory (in the form of a pointer in Span<T>’s case, and by using a custom MemoryManager<T> in Memory<T>’s case). They provide better encapsulation by not exposing their underlying data source and have read-only versions for immutable access.
Back then, we had to pass the array/offset/count trio to a lot of APIs (APIs that needed a direct reference of an array), but now that Span<T> and Memory<T> exist and are widely supported by most, if not all, .NET APIs that need to interact with continuous blocks of memory, you should have no reason to use an ArraySegment<T>.
See also: Memory- and span-related types - MS Docs
MemoryMappedFilebe a better starting point? Without knowing your use cases it's going to be difficult for anyone to offer concrete advice.ArraySegmentis older, hence why it exists. I'd useMemory(orSpan, as needed) in all new code.