Convert byte array to Ctype struct values

Question

I just want to convert that raw byte array(unknown encoding) to Ctype values using python with following structure specified in C# documentation. But output is incorrect. I mean, LTP is shown negative,which is never the case. Please guide if i am missing some encoding.

I just want LTP field, i have tried using offset still value is not correct. Can i get bytposition directly converted to Ctype value or is there any other way?

Structure

BytePosition :1-2 Type:int8 Field:Exchange

BytePosition :2-6 Type:int32 Field:Instrument token

BytePosition :6-10 Type:int32 Field:Ltp

BytePosition :10-14 Type:int32 Field:Change

BytePosition :14-18 Type:int32 Field:Exchange Timestamp

BytePosition :18-22 Type:int32 Field:Volume

from ctypes import Structure,c_int32, c_byte

class CompactMarketData(Structure):
    _fields_ = [("Mode", c_byte),("Exchange", c_byte),("InstrumentToken", c_int32), ("LastTradedPrice", c_int32), ("Change", c_int32), ("ExchangeTimeStamp", c_int32), ("Volume", c_int32)]

def main():
    raw=b'\x02\x06\x00\x08;8\x00\x01n\x04\xff\xff\xff\x06_0\xc5\xea\x00\x00",_0\xc5\xea'
    b = bytearray(raw)
    s = CompactMarketData.from_buffer(b)

    print("Mode: "+format(s.Mode))
    print("Exchange: "+format(s.Exchange))
    print("InstrumentToken: "+format(s.InstrumentToken))
    print("LastTradedPrice: ",str(s.LastTradedPrice))
    print("Change: "+format(s.Change))
    print("ExchangeTimeStamp: "+format(s.ExchangeTimeStamp))
    print("Volume: "+format(s.Volume))

if __name__ == '__main__':
    main()

Output:

Exchange: 2

InstrumentToken: 16791611

LastTradedPrice: -64402

Change: 811534079

ExchangeTimeStamp: 60101

Volume: 811543586

buffer and structure size don't agree (26 vs. 22). Do you know what the raw data bytes are supposed to translate to? Having a sample input and correct desired output would help solve your issue. — Mark Tolonen
– Mark Tolonen, Commented Aug 10, 2020 at 17:52

Neitsa · Accepted Answer · 2020-08-10 10:03:24Z

1

Using the _pack_ = 1 (documentation) structure attribute satisfies the conditions:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from ctypes import Structure, c_int32, c_byte


class CompactMarketData(Structure):
    _pack_ = 1
    _fields_ = [
        ("Mode", c_byte),
        ("Exchange", c_byte),
        ("InstrumentToken", c_int32),
        ("LastTradedPrice", c_int32),
        ("Change", c_int32),
        ("ExchangeTimeStamp", c_int32),
        ("Volume", c_int32)
    ]

    def view_offsets(self):
        for field in self._fields_:
            name, field_type = field
            field_start = getattr(self.__class__, name).offset
            field_end = field_start + ctypes.sizeof(field_type)
            print(f"{name} - {field_start}:{field_end}")



def main():
    raw=b'\x02\x06\x00\x08;8\x00\x01n\x04\xff\xff\xff\x06_0\xc5\xea\x00\x00",_0\xc5\xea'
    b = bytearray(raw)
    s = CompactMarketData.from_buffer(b)

    s.view_offsets()

    print(f"\n{'-' * 79}")

    print(f"Struct Size: {ctypes.sizeof(CompactMarketData)}")
    print(f"buffer len: {len(raw)}")

    print(f"Mode: {s.Mode:#x}")
    print(f"Exchange: {s.Exchange:#x}")
    print(f"InstrumentToken: {s.InstrumentToken:#x}")
    print(f"LastTradedPrice: {s.LastTradedPrice:#x}")
    print(f"Change: {s.Change:#x}")
    print(f"ExchangeTimeStamp: {s.ExchangeTimeStamp:#x}")
    print(f"Volume: {s.Volume:#x}")


if __name__ == "__main__":
    main()

Ouput:

Mode - 0:1
Exchange - 1:2
InstrumentToken - 2:6
LastTradedPrice - 6:10
Change - 10:14
ExchangeTimeStamp - 14:18
Volume - 18:22

-------------------------------------------------------------------------------
Struct Size: 22
buffer len: 26
Mode: 0x2
Exchange: 0x6
InstrumentToken: 0x383b0800
LastTradedPrice: 0x46e0100
Change: 0x6ffffff
ExchangeTimeStamp: -0x153acfa1
Volume: 0x2c220000

Note: be wary that your bytes buffer is longer than the size of the Structure.

answered Aug 10, 2020 at 10:03

Neitsa

8,2391 gold badge31 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Rajnikant Over a year ago

Thank you so much Neitsa. I got a clue what i was missing. I must work on buffer length and struct size. And output is in Hex format so i need to convert it. Right? Please do tell me proper way to convert it to decimal. Thank you again.

Neitsa Over a year ago

@Rajnikant The output is in hex because, from my point of view, it's easier to see which part of the buffer ends up in which field (e.g InstrumentToken is from \x00\x08;8). Beside that, hex or dec are different representation of the same value, but the value is still the same in the end. If you want to print in decimal instead of hex, just remove the format in the f-string, that is :#x.

Rajnikant Over a year ago

Ok Neitsa. Thank you so much.

Collectives™ on Stack Overflow

Convert byte array to Ctype struct values

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related