BLE Gateway V4 MQTT and problems getting information msgpack.unpackb


#1

Hello,

I am trying to use the information coming from the AB BLE Gateway V4 via MQTT server into python. After unpacking via msgpack.unpack (payload) the devices array seems to be partially messed up:

Here an example:
{‘ip’: ‘192.168.2.116’, ‘v’: ‘1.1.0.26’, ‘mid’: 1558, ‘devices’: [’\x03\x12;j\x1bE\xcc\xc1\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x07\xc5’, ‘\x03\xc4S9<\x9d\xe6\xc7\x02\x01\x06\x1b\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\td\x0b\xcbd’, ‘\x03\x12;j\x1bE\xcf\xc0\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x08\xc5’, ‘\x03\xe9D\xf6\x1eBE\xb9\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\td\x05\xc5’, ‘\x03\x12;j\x1bE\xcf\xbf\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x08\xc5’, ‘\x03\xde\xfb)\6\x0f\xc2\x02\x01\x06\x1b\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\tY\t\xcbY’, ‘\x03\x12;j\x1bE\xcc\xc1\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x07\xc5’, ‘\x03\x12;j\x1bE\xcc\xc1\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x07\xc5’, ‘\x03\xe9D\xf6\x1eBE\xb8\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\td\x05\xc5’, ‘\x03\xdf\xca\x9e\xcd\xcfi\xb7\x02\x01\x06\x1b\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\td\n\xcbd’, ‘\x03\x12;j\x1bE\xcf\xc0\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x08\xc5’, ‘\x03\xde\xfb)\6\x0f\xc2\x02\x01\x06\x1b\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\tY\t\xcbY’, ‘\x03\x12;j\x1bE\xcc\xc0\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x07\xc5’, ‘\x03\xc4S9<\x9d\xe6\xc8\x02\x01\x06\x1b\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\td\x0b\xcbd’, ‘\x03\x12;j\x1bE\xcf\xc1\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x08\xc5’, ‘\x03\xe9D\xf6\x1eBE\xb5\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\td\x05\xc5’, ‘\x03\x12;j\x1bE\xcf\xb7\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x08\xc5’], ‘mac’: ‘30AEA490FE14’, ‘time’: 404}

Any idea why this happened and how to correctly unpack the information to not mess up the devices part?

Any help or idea would be great

Ingo


#2

Please check the wiki for data format. The devices array contains raw advertising data in hex format.


#3

Thank you for the quick answer. Yes I know the format, but when using the standard python unpack on the mqtt message the content of the devices part is messed up. For the time being I am using a http server to receive the message which works, but I need eventually get mqtt working. Do you habe any idea why the message content is not received or unpacked properly. For example via web service the content is after unpack something like

03defb295c360fc10201061bff4c0002155eb2aede4f7b316e237cd720487fb4b000095809cb58

but via mqtt it is somehow messed up and not even pure hex after the unpack command:

\x03\x12;j\x1bE\xcc\xc1\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x07\xc5

so I guess my method of getting the mqtt message and then unpacking in python via:

messagestring = str(msgpack.unpackb(theMessage.payload, use_list=False, raw=True))

is wrong, but I have no idea what to try. I tried it with raw=False already, no significant change. Do you have any hint what to try?

Regards
Ingo


#4

So which unpack message is right?

03defb295c360fc10201061bff4c0002155eb2aede4f7b316e237cd720487fb4b000095809cb58

or

\x03\x12;j\x1bE\xcc\xc1\x02\x01\x06\x1a\xffL\x00\x02\x15^\xb2\xae\xdeO{1n#|\xd7 H\x7f\xb4\xb0\x00\x01\x00\x07\xc5

#5

The first one coming through http is fine, but in the second one coming through MQTT are not only hexbin data e.g. \x03 but elements like \xffL or \x15^ or \xdeO{1n#| … I would expect only \x followed by two hex letters. So something on the way from sniffer -> MQTT -> python -> unpack -> messagestring goes wrong and I don‘t know what happens. Do you have maybe a code example for python to extract the data from the MQTT message coming from the V4 BLE Gateway? The code I use
messagestring = str(msgpack.unpackb(theMessage.payload, use_list=False, raw=True))
Does not do the trick.
Ingo


#6

No. I have only example PHP code for parse data from MQTT. I think the problem is python side. What mean is this parameter? – use_list=False


#7

The parameter use_list=False only changes the output format, if it comes as an array or as one string. It does not change the content, I tried both. Can you please send me the php code example, then I will try it later this week to identify if the problem also appears there. this would tell me if the problem is maybe in the MQTT part and the MQTT Server needs to be changed.


#8

The php SDK is here. Please check the source file examples/example_gateway4_mqtt.php


#9

I have exactly the same question: we are just asking how can we convert the message publish in a format like “x03\x12;j\x1bE\xcc\xc1\x02\x01\x06\x1a” to a readable format

Please help!


#10

Which programming language do you prefer? Please also check the section “Write your own client


#11

I m using python like this :

import time
import paho.mqtt.client as mqtt
import msgpack

def on_subscribe(mosq, obj, mid, granted_qos):
    print("Subscribed: " + str(mid))

def on_message(mosq, obj, msg):
    print("=============================================")
    for d in msgpack.unpackb(msg.payload)[b'devices']:
      print(d)
def on_connect(mosq, obj,flags, rc):
    print("Connected with result code "+str(rc))
    mqttc.subscribe("lolo", 0)
    print("Connected")


mqttc = mqtt.Client()
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe
mqttc.on_message = on_message
mqttc.connect("mqtt.bconimg.com", 1883, 60)
mqttc.loop_forever()

And this is the result :

=============================================
b'\x00DO\xf2\x1c\xac\xe6\xbb\x02\x01\x1a\n\xffL\x00\x10\x05\x03\x18\xf8D\x00'
b'\x00S\xb2\xd12\xa80\xa7\x02\x01\x1a\t\xff\xc4\x00\x133\x02\x13\x15\x80\x03\x03\xb9\xfe\x08\x1b\x00$\x98\x05N\x1c|'
b'\x00O\x08:\x1d\x0c\xad\xa6\x02\x01\x1a\x1a\xffL\x00\x0c\x0e\x00z0\x1d|\xceF\x985\x00\x12\xd0fM\x10\x05\x0b\x1c\xab\x1d\xef'
b'\x00hdK\x0c\xb4S\xa3\x02\x01\x1a\x0b\xffL\x00\t\x06\x03t\xc0\xa8\n\x05'
b'\x03k\xd5\x04\x13\xf8P\xa7\x1e\xff\x06\x00\x01\t \x02|\x02\xa0\xba\xf5^\xba\x11\x1e!\x9a \xb8\xe9\xa2\x12\x86w`\xd9\xcct\xf3'

How can I convert this data match this : https://wiki.aprbrother.com/en/User_Guide_For_AB_BLE_Gateway_V4.html

Without your help, I just can not use your product

Thanks in advance

Laurent


#12

We’ll check the python example and reply to you soon


#13

Hi Laurent,

I modified your code and get better result

import time
import paho.mqtt.client as mqtt
import msgpack

def on_subscribe(mosq, obj, mid, granted_qos):
    print("Subscribed: " + str(mid))

def on_message(mosq, obj, msg):
    for d in msgpack.unpackb(msg.payload)[b'devices']:
      print("=============================================")
      # adv type
      print "type:", ord(d[0])
      print "mac:{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}".format(ord(d[1]), ord(d[2]), ord(d[3]), ord(d[4]), ord(d[5]), ord(d[6]))
      print "rssi:", ord(d[7]) - 256
      hex_chars = map(hex,map(ord,d))
      del hex_chars[:8]
      print "adv:", hex_chars

def on_connect(mosq, obj,flags, rc):
    print("Connected with result code "+str(rc))
    mqttc.subscribe("hello", 0)
    print("Connected")
    

mqttc = mqtt.Client()
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe
mqttc.on_message = on_message
mqttc.connect("ab.bconimg.com", 1883, 60)
mqttc.loop_forever()

here’s my parsing result

type: 0
mac:123B6A1B59E5
rssi: -78
adv: ['0x2', '0x1', '0x6', '0x1a', '0xff', '0x4c', '0x0', '0x2', '0x15', '0xb5', '0xb1', '0x82', '0xc7', '0xea', '0xb1', '0x49', '0x88', '0xaa', '0x99', '0xb5', '0xc1', '0x51', '0x70', '0x8', '0xd9', '0x0', '0x1', '0xe5', '0x59', '0xc5']
=============================================
type: 3
mac:2026698D97F1
rssi: -66
adv: ['0x1e', '0xff', '0x6', '0x0', '0x1', '0x9', '0x20', '0x2', '0xa1', '0x22', '0xe0', '0xdd', '0xf0', '0xc9', '0xf7', '0xb6', '0x40', '0xb6', '0xb9', '0x8e', '0x6e', '0x2', '0x19', '0x1a', '0xe9', '0x60', '0xaf', '0xf6', '0x6e', '0xf5', '0x3b']
=============================================
type: 0
mac:123B6A1B97F0
rssi: -76
adv: ['0x2', '0x1', '0x6', '0x1a', '0xff', '0x4c', '0x0', '0x2', '0x15', '0xb5', '0xb1', '0x82', '0xc7', '0xea', '0xb1', '0x49', '0x88', '0xaa', '0x99', '0xb5', '0xc1', '0x51', '0x70', '0x8', '0xd9', '0x0', '0x1', '0xf0', '0x97', '0xc5']

Hope it’s helpful for you


#14

Great! Thanks you very much!


#15

Yes a big thank you from me too