BLE Gateway V4 MQTT and problems getting information msgpack.unpackb

No. I have only example PHP code for parse data from MQTT. I think the problem is python side. What mean is this parameter? – use_list=False

The parameter use_list=False only changes the output format, if it comes as an array or as one string. It does not change the content, I tried both. Can you please send me the php code example, then I will try it later this week to identify if the problem also appears there. this would tell me if the problem is maybe in the MQTT part and the MQTT Server needs to be changed.

The php SDK is here. Please check the source file examples/example_gateway4_mqtt.php

I have exactly the same question: we are just asking how can we convert the message publish in a format like “x03\x12;j\x1bE\xcc\xc1\x02\x01\x06\x1a” to a readable format

Please help!

Which programming language do you prefer? Please also check the section “Write your own client

I m using python like this :

import time
import paho.mqtt.client as mqtt
import msgpack

def on_subscribe(mosq, obj, mid, granted_qos):
    print("Subscribed: " + str(mid))

def on_message(mosq, obj, msg):
    print("=============================================")
    for d in msgpack.unpackb(msg.payload)[b'devices']:
      print(d)
def on_connect(mosq, obj,flags, rc):
    print("Connected with result code "+str(rc))
    mqttc.subscribe("lolo", 0)
    print("Connected")


mqttc = mqtt.Client()
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe
mqttc.on_message = on_message
mqttc.connect("mqtt.bconimg.com", 1883, 60)
mqttc.loop_forever()

And this is the result :

=============================================
b'\x00DO\xf2\x1c\xac\xe6\xbb\x02\x01\x1a\n\xffL\x00\x10\x05\x03\x18\xf8D\x00'
b'\x00S\xb2\xd12\xa80\xa7\x02\x01\x1a\t\xff\xc4\x00\x133\x02\x13\x15\x80\x03\x03\xb9\xfe\x08\x1b\x00$\x98\x05N\x1c|'
b'\x00O\x08:\x1d\x0c\xad\xa6\x02\x01\x1a\x1a\xffL\x00\x0c\x0e\x00z0\x1d|\xceF\x985\x00\x12\xd0fM\x10\x05\x0b\x1c\xab\x1d\xef'
b'\x00hdK\x0c\xb4S\xa3\x02\x01\x1a\x0b\xffL\x00\t\x06\x03t\xc0\xa8\n\x05'
b'\x03k\xd5\x04\x13\xf8P\xa7\x1e\xff\x06\x00\x01\t \x02|\x02\xa0\xba\xf5^\xba\x11\x1e!\x9a \xb8\xe9\xa2\x12\x86w`\xd9\xcct\xf3'

How can I convert this data match this : https://wiki.aprbrother.com/en/User_Guide_For_AB_BLE_Gateway_V4.html

Without your help, I just can not use your product

Thanks in advance

Laurent

We’ll check the python example and reply to you soon

Hi Laurent,

I modified your code and get better result

import time
import paho.mqtt.client as mqtt
import msgpack

def on_subscribe(mosq, obj, mid, granted_qos):
    print("Subscribed: " + str(mid))

def on_message(mosq, obj, msg):
    for d in msgpack.unpackb(msg.payload)[b'devices']:
      print("=============================================")
      # adv type
      print "type:", ord(d[0])
      print "mac:{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}".format(ord(d[1]), ord(d[2]), ord(d[3]), ord(d[4]), ord(d[5]), ord(d[6]))
      print "rssi:", ord(d[7]) - 256
      hex_chars = map(hex,map(ord,d))
      del hex_chars[:8]
      print "adv:", hex_chars

def on_connect(mosq, obj,flags, rc):
    print("Connected with result code "+str(rc))
    mqttc.subscribe("hello", 0)
    print("Connected")
    

mqttc = mqtt.Client()
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe
mqttc.on_message = on_message
mqttc.connect("ab.bconimg.com", 1883, 60)
mqttc.loop_forever()

here’s my parsing result

type: 0
mac:123B6A1B59E5
rssi: -78
adv: ['0x2', '0x1', '0x6', '0x1a', '0xff', '0x4c', '0x0', '0x2', '0x15', '0xb5', '0xb1', '0x82', '0xc7', '0xea', '0xb1', '0x49', '0x88', '0xaa', '0x99', '0xb5', '0xc1', '0x51', '0x70', '0x8', '0xd9', '0x0', '0x1', '0xe5', '0x59', '0xc5']
=============================================
type: 3
mac:2026698D97F1
rssi: -66
adv: ['0x1e', '0xff', '0x6', '0x0', '0x1', '0x9', '0x20', '0x2', '0xa1', '0x22', '0xe0', '0xdd', '0xf0', '0xc9', '0xf7', '0xb6', '0x40', '0xb6', '0xb9', '0x8e', '0x6e', '0x2', '0x19', '0x1a', '0xe9', '0x60', '0xaf', '0xf6', '0x6e', '0xf5', '0x3b']
=============================================
type: 0
mac:123B6A1B97F0
rssi: -76
adv: ['0x2', '0x1', '0x6', '0x1a', '0xff', '0x4c', '0x0', '0x2', '0x15', '0xb5', '0xb1', '0x82', '0xc7', '0xea', '0xb1', '0x49', '0x88', '0xaa', '0x99', '0xb5', '0xc1', '0x51', '0x70', '0x8', '0xd9', '0x0', '0x1', '0xf0', '0x97', '0xc5']

Hope it’s helpful for you

Great! Thanks you very much!

Yes a big thank you from me too

so I am trying this example, and the examples in the public “SDK” and they are not working for me. I’ve added some debug logging in the beacon library and I am getting the following error when it tries to parse the iBeacon frame

=============================================
type: 0
mac:55FB39EAE43C
rssi: -57
adv: ['0x2', '0x1', '0x1a', '0x2', '0xa', '0xc', '0xb', '0xff', '0x4c', '0x0', '0x10', '0x6', '0x1a', '0x1e', '0x2f', '0x7d', '0xef', '0x99']
parsing expected '\x02\x01' but parsed '\x11\x06'
=============================================

my code is as follows

# -*- coding:utf-8 -*-
import time
import paho.mqtt.client as mqtt
import msgpack
from beacontools import parse_packet


def on_subscribe(mosq, obj, mid, granted_qos):
    print("Subscribed: " + str(mid))


def on_message(mosq, obj, msg):
    # print("got payload: %s" % msgpack.unpackb(msg.payload))
    for d in msgpack.unpackb(msg.payload)[b'devices']:
        # parse iBeacon data
        advData = d[8:]

        adv = parse_packet(advData)

        # if adv is None:
        #     continue
        print("=============================================")
        # adv type
        print "type:", ord(d[0])
        print "mac:{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}".format(ord(d[1]), ord(d[2]), ord(d[3]), ord(d[4]), ord(d[5]), ord(d[6]))
        print "rssi:", ord(d[7]) - 256
        hex_chars = map(hex,map(ord,d))
        del hex_chars[:8]
        print "adv:", hex_chars
        print parse_packet(hex_chars)

        # print("=============================================")
        # print "mac:{:02X}{:02X}{:02X}{:02X}{:02X}{:02X}".format(ord(d[1]), ord(d[2]), ord(d[3]), ord(d[4]), ord(d[5]), ord(d[6]))
        # print "rssi: ", ord(d[7]) - 256
        # print("UUID: %s" % adv.uuid)
        # print("Major: %d" % adv.major)
        # print("Minor: %d" % adv.minor)
        # print("TX Power: %d" % adv.tx_power)
        # print("foo")

def on_connect(mosq, obj, flags, rc):
    mqttTopic = "ab/hello"
    print("Connected with result code " + str(rc))
    mqttc.subscribe(mqttTopic, 0)
    print("Connected")


mqttHost = "192.168.100.14"
mqttPort = 1883
mqttc = mqtt.Client()
mqttc.on_connect = on_connect
mqttc.on_subscribe = on_subscribe
mqttc.on_message = on_message
mqttc.username_pw_set(username='local-user', password='xxxxxxx')
mqttc.connect(mqttHost, mqttPort, 60)
mqttc.loop_forever()

the beacontools change I made was as follows in the parse_ibeacon_packet method, where I printed the exception

"""Beacon advertisement parser."""
from construct import ConstructError

from .structs import LTVFrame, IBeaconAdvertisingPacket
from .packet_types import EddystoneUIDFrame, EddystoneURLFrame, EddystoneEncryptedTLMFrame, \
                          EddystoneTLMFrame, EddystoneEIDFrame, IBeaconAdvertisement, \
                          EstimoteTelemetryFrameA, EstimoteTelemetryFrameB
from .const import EDDYSTONE_TLM_UNENCRYPTED, EDDYSTONE_TLM_ENCRYPTED, SERVICE_DATA_TYPE, \
                   EDDYSTONE_UID_FRAME, EDDYSTONE_TLM_FRAME, EDDYSTONE_URL_FRAME, \
                   EDDYSTONE_EID_FRAME, EDDYSTONE_UUID, ESTIMOTE_UUID, ESTIMOTE_TELEMETRY_FRAME, \
                   ESTIMOTE_TELEMETRY_SUBFRAME_A, ESTIMOTE_TELEMETRY_SUBFRAME_B


def parse_packet(packet):

    """Parse a beacon advertisement packet."""
    frame = parse_ltv_packet(packet)
    if frame is None:
        frame = parse_ibeacon_packet(packet)
    return frame

def parse_ltv_packet(packet):
    """Parse a tag-length-value style beacon packet."""
    try:
        frame = LTVFrame.parse(packet)
        for ltv in frame:
            if ltv['type'] == SERVICE_DATA_TYPE:
                data = ltv['value']

                if data["service_identifier"] == EDDYSTONE_UUID:
                    return parse_eddystone_service_data(data)

                elif data["service_identifier"] == ESTIMOTE_UUID:
                    return parse_estimote_service_data(data)

    except ConstructError:
        return None

    return None

def parse_eddystone_service_data(data):
    """Parse Eddystone service data."""
    if data['frame_type'] == EDDYSTONE_UID_FRAME:
        return EddystoneUIDFrame(data['frame'])

    elif data['frame_type'] == EDDYSTONE_TLM_FRAME:
        if data['frame']['tlm_version'] == EDDYSTONE_TLM_ENCRYPTED:
            return EddystoneEncryptedTLMFrame(data['frame']['data'])
        elif data['frame']['tlm_version'] == EDDYSTONE_TLM_UNENCRYPTED:
            return EddystoneTLMFrame(data['frame']['data'])

    elif data['frame_type'] == EDDYSTONE_URL_FRAME:
        return EddystoneURLFrame(data['frame'])

    elif data['frame_type'] == EDDYSTONE_EID_FRAME:
        return EddystoneEIDFrame(data['frame'])
    else:
        return None

def parse_estimote_service_data(data):
    """Parse Estimote service data."""
    if data['frame_type'] & 0xF == ESTIMOTE_TELEMETRY_FRAME:
        protocol_version = (data['frame_type'] & 0xF0) >> 4
        if data['frame']['subframe_type'] == ESTIMOTE_TELEMETRY_SUBFRAME_A:
            return EstimoteTelemetryFrameA(data['frame'], protocol_version)
        elif data['frame']['subframe_type'] == ESTIMOTE_TELEMETRY_SUBFRAME_B:
            return EstimoteTelemetryFrameB(data['frame'], protocol_version)
    return None

def parse_ibeacon_packet(packet):
    """Parse an ibeacon beacon advertisement packet."""
    try:
        pkt = IBeaconAdvertisingPacket.parse(packet)
        return IBeaconAdvertisement(pkt)

    except ConstructError as e:
        print(e)
        return None

The adverting data is not a valid iBeacon profile. Please search iBeacon profile at stackoverflow.

Note: Please create a new topic on this matter, as we try to keep each subject in separate topics here on support forum.

Hello,

while I am starting to use your BLE Gateways for my own RTLS I am struggling with the very first steps. I have already implemented your BLE Gateway in my WIFI (with Config-Tool, Linux). Unfortunately, I can’t test the Gateway, because I only running Linux & Mac OS.
So I tried to test the BLE Gateway with your Python mqtt-client (https://github.com/AprilBrother/ab-ble-gateway-sdk/tree/master/gateway-v4/examples/python/mqtt-client), but I get not signal or parsing result respectively. I seems, that the Gateway is not found.

Do you habe any ideas?

Thank you very much!

Best,
Joern

  • we opened the source code of ble viewer. You can run it with Linux & macOS by node
  • For the python mqtt-client, please try with python2 instead. Could you please post a screenshot about the parsing result?

Thank you for your support.

The python mqtt-client code seems to “run” properly under python3. I get following output with 1 active BLE-Gateway:

Connected with result code 0
Connected
Subscribed: 1

What I don’t get is the parsing information - it seems, that the function

mqttc.on_message = on_message

will not be excecuted. Any ideas?

Hi Joernt,

please make a new topic to keep the original topic clear.

Please also try with tool such as MQTTBox to subscribe the message first.

Hi,

I tried MQTTBox and I could verify, that the BLE Gateways are working properly. My problem is, that I can’t reproduce your parsing result (from Nov. 2018).

It seems, that the “for-loop” is not to be run through. I am using Python 3.6 - is this maybe the reason? Thank you very much for your assistance.

Best Joern

Yes. I can reproduce the issue with Python 3.8. But python 2.7 works fine

Hi, I am trying to run the code example but keep on get the following error.
File “ble_mqtt_1.py”, line 12, in on_message
for d in msgpack.unpackb(msg.payload)[b’devices’]:
KeyError: b’devices’

I managed to fix the issue, I was using Python 3 and the code only works with Python 2