How to detect encoding in Data based on a String?

Question

I'm loading a text file, the encoding is unknown as it comes from other sources. The content itself comes from macOS NSDocument's read method, which is fed into my model's read. The String constructor requires the encoding when using Data, if you assume the incorrect one you may get a null. I've created a conditional cascade of potential encodings (it's what other people seem to be doing), there's gotta be a better way to do this. Suggestions?

    override func read(from data: Data, ofType typeName: String) throws {
        model.read(from: data, ofType: typeName)
    }

In the model:

    func read(from data: Data, ofType typeName: String) {
        if let text = String(data: data, encoding: .utf8) {
            content = text
        } else if let text = String(data: data, encoding: .macOSRoman) {
            content = text
        } else if let text = String(data: data, encoding: .ascii) {
            content = text
        } else {
            content = "?????"
        }
    }

If your text is coming from the web you can check this post stackoverflow.com/a/34687962/2303865 — Leo Dabus
– Leo Dabus, Commented Jan 21, 2020 at 1:31
Thanks @LeoDabus for the suggestion, unfortunately it does not come from the web. It's a regular text file on the file system, hence NSDocument. — RobMac
– RobMac, Commented Jan 21, 2020 at 14:02
@LeoDabus unless I'm missing something, NSString also requires the encoding to be specified. Do you have a link to the documentation on the specific factory or constructor method? — RobMac
– RobMac, Commented Jan 21, 2020 at 14:17

Leo Dabus · Accepted Answer · 2025-04-16 05:58:54Z

11

You can extend Data and create a stringEncoding property to try to detect the string encoding. Most of the time the data encoding is utf8 so first we can try to convert the string to utf8 and if that fails we can try to detect another encoding:

extension DataProtocol {
    var string: String? { .init(bytes: self, encoding: .utf8) }
}

extension Data {
    var stringEncoding: (
        string: String,
        encoding: String.Encoding
    )? {
        guard let string else {
            var nsString: NSString?
            let rawValue = NSString.stringEncoding(
                    for: self,
                    encodingOptions: nil,
                    convertedString: &nsString,
                    usedLossyConversion: nil
                )
            guard rawValue != 0, let string = nsString as? String
            else { return nil }
            return (
                string,
                .init(
                    rawValue: rawValue
                )
            )
        }
        return (string, .utf8)
    }
}

Then you can simply access the stringEncoding data property:

if let (string, encoding) = data.stringEncoding {
    print("string:", string, "encoding:", encoding.rawValue)
} else {
    print(nil)
}

edited Apr 16 at 5:58

answered Jan 21, 2020 at 14:29

Leo Dabus

237k61 gold badges512 silver badges624 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

RobMac Over a year ago

I chose this answer as the right one, because it is the right way to do it according to the documentation, but somehow keeps coming back with incorrect encodings. So, the issue is probably a bug within Apple's implementation.

stef Jan 3 at 9:22

Note that this is also decoding the data into a string at the same time (to the nsString variable). So of this method succeeds, you can just use that as the decoded string (can be toll-free bridged to String) instead of decoding it a second time elsewhere.

Leo Dabus Jan 3 at 19:45

@stef yes that’s correct and a valid point. You can just change the return type to return a tuple instead. Note that the original post is about detecting the encoding and that’s what it does.

stef Jan 4 at 0:04

@LeoDabus Of course. I just wanted to mention the obvious, as the work is already being done by the method here to decode the string. And the cited reason for wanting to know the encoding in the original post was in order to decode the string in the first place.

Leo Dabus Apr 16 at 5:59

@stef check the updated code.

Avicii4ever · Accepted Answer · 2024-02-20 06:37:16Z

0

Here is my approach based on @Leo Dabus's, work well on Xcode14.3

var stringEncoding: String.Encoding {
        var nsString: NSString?
        let rawValue = NSString.stringEncoding(for: self, encodingOptions: nil, convertedString: &nsString, usedLossyConversion: nil)
        return .init(rawValue: rawValue)
    }

answered Feb 20, 2024 at 6:37

Avicii4ever

1652 silver badges10 bronze badges

1 Comment

Leo Dabus Over a year ago

From the docs "Return Value is an NSStringEncoding value, or 0 if a string encoding could not be determined." Why would you ignore zero and return an non optional value?

Collectives™ on Stack Overflow

How to detect encoding in Data based on a String?

2 Answers 2

5 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related