Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
IoT Unravelled Part 2: IP Addresses, Network, Zigbee, Custom Firmware, Soldering (troyhunt.com)
77 points by ykat7 on Nov 24, 2020 | hide | past | favorite | 46 comments


Interesting Troy went full-on wireless for all this. I'll be building a new home soon and am somewhat inclined to run wires to everywhere that little sensors and activators and other devices will go. Mainly for more peace of mind re security, reduced spectrum noise, reliability (and of course power).

I just wish there were more wired product options out there. Love to hear recommendations.

Also, is Ubiquity generally considered more suitable now than OpenWRT/Tomato/Dd-WRT and the like? At present I have a fleet of off-the-shelf routers deployed at various properties with custom bits added to the firmware (for improved monitoring, history, auditing, even periodically speed testing radio links and whatnot).

I read sometime back Ubiquity killed off some of the programmability features and that lent some wariness to my long term plans to switch over to their gear.


I will second that eventually smart home (accent on home, not household) will go full wired, while semi-disposable gadgets may well stay wireless, but the idea of fully wireless home automation is now out of the windows for so many reasons.

I had a discussion with one guy from IKEA who bid on "full wireless" because "it is where Apple, and Google go."

The conversation went all went fine for him until a moment of Zen enlightenment hit him: "Outside Europe, people live in concrete houses." That very moment the smile vanishes from his face, and the guy looked like he felt a total facepalm moment.


The only place I know where a lot of houses are made of wood are Scandinavia and North America. In the rest or Europe most houses are made of bricks and/or concrete as far as I can tell.


Around half of the population in Scandinavia lives in apartments made out of concrete. It's not plausible that IKEA don't know that.


Ruckus [0] gear is generally considered to be the step-up from Ubiquiti (although their point-to-point stuff is really good). The RF design is supposedly more complex and advanced in Ruckus APs with advanced antennas and their proprietary BeamFlex technology... not just reference-design SoC stuff.

See this reply (and others in the topic) for some “insight”: https://www.reddit.com/r/networking/comments/9vj2dh/comment/...

Also... since I’ve been looking at Ruckus APs recently (they offer several WiFi 6 APs) I learned that their APs have two part numbers with different prices. The '901' part numbers ship with Solo AP firmware that can run in either Standalone mode or will join a Ruckus ZoneDirector or SmartZone controller. The less expensive '9U1' part numbers ship with “Unleashed” code that will not connect to a ZD/SZ controller, and instead look for a local Unleashed network. The price difference is due to support entitlements; controllers license number of APs per controller/cluster, and Unleashed support is one license for up to 25 APs in the network. (Seems like most home users go with Unleashed. [1])

Also, one thing to note with some Ruckus APs (like those that support 802.11ax) are the PoE requirements. The RF power is reduced and/or some features become unavailable depending on the power available. (It’s all listed on the AP data sheets, just need to be aware of it.)

[0] https://www.commscope.com/product-type/enterprise-networking...

[1] https://www.commscope.com/product-type/enterprise-networking...


Thanks for the pointer. I’m seriously considering jumping off the UniFi bandwagon due to their mounting technical debt. UniFi seems to be stuck on MongoDB 3.4 and Java 8. I can live with that, even though it’s a sign that their developers are not keeping up. The straw that’s about to break the camel’s back that that they’re stuck on an old version of dropbear. The latest OpenSSH refuses to negotiate public key auth with it because the old dropbear doesn’t support modern signature algorithms.

(Yes, I know how to override OpenSSH’s config for this. No, I shouldn’t have to.)


Ubiquity definitely also has a lot of issues. Unstable firmware's, non-responsive support on their forums, deprecating working products... Like you can't self host their DVR software, but the newest versions don't run on older hardware so you have to buy a new hardware to get new software.

Also their product lines Edge and Unifi don't work together, so you can't manage an EdgeRouter with Unifi or vice versa, for which I don't think there is a technical reason, just commercial.

Hardware wise they also have made strange choices regarding PoE, don't assume your Unifi PoE switch will work with your Unifi PoE device (really double check the versions, voltages, passive/active)

So lots to complain, though I think it is still the most accessible pro-sumer networking stuff.


Newer Unifi products use the correct standard 802.3af/at PoE, but you are correct their older gear uses some non-standard 24V (I believe).

I have been using their "Dream Machine" and a mesh node for about 6 months now and have had 0 problems, but I have not tried to tinker around too much yet!


I work as a software dev for a company that does home automation systems [0] targeted at "high net worth individuals" (ie: people with large homes). Our product mainly goes into new custom built homes during the construction phase and we do everything hard wired. If you have the opportunity to go hard wired over wireless you'll thank yourself in the long run.

[0] http://www.idotechnology.com/products/saffyre-awareness-and-...


> I just wish there were more wired product options out there. Love to hear recommendations.

RS485 + Modbus

There are some hardlimits (like length, number of devices though)


An now just wait until Troy discovers https://esphome.io/

Program your IoT devices to function independent of the mothership (HA), use whatever sensor you fancy (as long as it is in the growing https://esphome.io/#sensor-components list), and of course soldering is required.


I love ESPHome, I have it on a variety of smart lightbulbs, smart plugs, custom ESP32 projects and Shelly 1's. Having the device behaviour as "code" (yaml) for me is much nicer than some UI configuration when you start to scale up to lots of devices. It means I can have my IoT devices behaviour stored in Git and roll out updates quickly.


The Xiaomi stuff is great, I have a lot of Aqara devices around the house.

A nice alternative to the temperature + humidity sensor is this Xiaomi Mijia one which uses BLE instead of Zigbee and has a little screen which shows the temperature as well: https://www.aliexpress.com/item/1005001355746608.html

It's also much cheaper at $3-$4 each instead of ~$11 each. I've been scattering them around and they're great, the display is also nice.

They can easily be added to Home Assistant with a Raspberry Pi or ESP32 ($3-4) using ESPHome: https://esphome.io/components/sensor/xiaomi_ble.html


I have a couple of Yeelights, ie their RGBW-led light in common E27 socket. Very nice, you can "unlock" developer mode so you can control it over the local network. Super nice. Scripted up a simple thing running on a Raspberry pi + telegram bot for simple commands like "movie", "night" etc without needing to switch apps or be on wifi.

Then my daughter said that it was so cool when it blinked in all colors and I realized she had flipped it on/off till it factory reset itself.

Now I can't commission them. At all. Both apps provided by Yeelight fail with timeouts, neither server works (you have to pick one, eg china, germany), and none of the troubleshooting steps help. Changing DNS server, wifi hotspot, etc etc. None works. They even have a bug where if your account number is large enough, it will fail since it can fit in the apparently int32_t they seem to store it in.

They are cheap, but I'm beyond frustrated that I can't get them online again. The light was nice and bright, but the sw quality is apparently not on par.


I've had the timeout happen to me. The bulb does connect to your wifi, just connect your phone to your wifi manually near the end of the pairing and it will continue where it left off. Afterwards, enable LAN mode control and uninstall the app, you never need to use it again. You can use the YeeLight Python library (which I wrote!) and yee (or HA) to do anything you want.


I wish there was a way to completely avoid the app. I have a bunch of Aqara devices that are Zigbee and they never even talked to Xiaomi because I only connected them to my Conbee without ever installing any Xiaomi App or whatever.

I wonder how hard it is to add an ESP8266 to the lamps and just control via the ESP.


You can do that with Zigbee? That sounds fantastic, but the Conbee is a bit pricy still.

Look at the Sonoff line of devices, they're all ESP8266-based and extremely hackable (all mine run custom firmware, Espurna).


Yeah you can. You can get something like a CC2531 which works great aswell. Not sure If it's supported by ZHA yet but https://www.zigbee2mqtt.io/ worked very good with my CC2531. I just switched to the Conbee because it was supported by ZHA and I wanted my lights etc to directly connect to Home Assistant with ZHA.


The CC2531 looks dirt-cheap, Ali stocks it for $2. What's ZHA? I saw it mentioned in the article but don't understand why the CC2531 doesn't work with it. I was thinking of buying an ITEAD Zigbee bridge, but I don't know whether that will work with ZHA or why I'd want it to...


ZHA is Zigbee Home Automation. It's basically the part of Home Assitant that deals with zigbee devices. So it's like first party vs something like zigbee2mqtt which then hooks into Home Assitant.

Makes a little less work configuring etc, but not all Zigbee sticks are supported. Check support here

https://www.home-assistant.io/integrations/zha/#known-workin...


Ahh, so it's a HA thing. The ITEAD gateway is explicitly listed as supported, so that seals it. Their stuff has always been stellar, thank you for the info.


Yeah, it's really cheap, but 8051 which is quite cumbersome to work with imho.


Oh, what's the difference? Also what's the alternative?


Oh nice! Hope this works, trying it out later!


How long do they last on a battery? I have the older version https://www.aliexpress.com/item/32845863956.html which runs on a about 9 months on a rechargeable AAA penlite. But I could use a few more and these new ones are a lot cheaper.


To be fair, I only got them around a month ago :) So I'm not sure, although in the specs they say it's supposed to last around a year. My battery levels are still at 100% although I know that is not a reliable indication.

One thing to note with the new model, the Bluetooth messages are encrypted and a device bindkey is needed to read the temperature values. Fortunately there is a great project over at https://atc1441.github.io/TelinkFlasher.html which can pair with the device and retrieve a bindkey from within the web browser. Then you just need to plug the bind key and MAC address into ESPhome and you're good to go.


What do they talk to? Do you need to get a Xiaomi hub? Are they hackable? I really don't want my devices talking to the internet, I even jailbroke my vacuum so I could control it locally (what a brave new world).


This is the great part about Zigbee/BLE/433MHz devices, they do _not_ go on your wifi (and the internet), but instead talk to a dedicated gateway. As long as you find a way to talk to them, you can mix and match sensors from the shadiest of vendors.

You can ignore the gateways offered by each manufacturer and instead go with something (more) open like zigbee2mqtt or deConz.

Zigbee: https://www.zigbee2mqtt.io https://www.phoscon.de/en/conbee2/software

BLE: RPi or ESP32 running https://esphome.io

433MHz: https://sonoff.tech/product/accessories/433-rf-bridge running https://esphome.io or https://tasmota.github.io

Of course, this way you lose the (more or less) polished apps that come with each system and need to set up something like Home Assistant or openHAB for control and automation.


That's great, thanks! I already run Espurna on everything, so that's fine, I just wanted to confirm that all Zigbee devices are interoperable. It's unlikely that I'll get a device that will only want to talk to its specific vendor's gateway, then?


> I just wanted to confirm that all Zigbee devices are interoperable.

At the RF protocol level, yes they should be. Speaking as someone who works at a company with a few Zigbee-certified products, I can say that there are a LOT of "optional" features in the Zigbee specification, and even implementations of mandatory features can have differences that lead to incompatibility.

Manufacturer-specific profiles are also a big thing. So just be aware that manufacturer A's Zigbee "hub"/coordinator may or may not be inherently compatible with manufacturer B's sensors. But I would imagine that open-source systems like HomeAssistant deal with that.


How can HA deal with it? Is the incompatibility not at the protocol level? Does HA even see the messages at all? Or are the messages compatible, and then HA has to decode them somehow?


Messaging in Zigbee (i.e. sending data back and forth) is all about two things: addressing (endpoint, profile and cluster... kind of like port + URI for HTTP, where endpoint=port and profile+cluster=URI), and the payload itself. The Zigbee specification defines a variety of standard profiles that devices may choose to implement. Things like device temperature, on/off switch, metering, time, lighting, etc.

But nothing says that any given device, say a Zigbee light controller (on/off) from manufacturer X, MUST implement the standard on/off cluster for that. They could have their own manufacturer-specific profile+cluster for controlling the light, where a payload of 0x11 turns it on, 0x22 turns it off, and 0xF7 toggles the light. Or maybe you send the ASCII string "on".

Suppose that HA has implemented the functionality needed to interact with the standard on/off cluster. That functionality would work with devices that implement the standard cluster, but not with manufacturer X's device that needs the string "on". But then, HA could be updated to recognize such devices and implement the required functionality.

Assuming you're familiar with HTTP: just because two services both consume and produce JSON, doesn't mean the message bodies are interchangeable.


That clarifies things perfectly, thanks. As long as the wire format is compatible (which it is, from your description), then compatibility is only a matter of software updates, which is great.


They display temperature and humidity on the LCD display, but also transmit the temp + humidity every 10 minutes or so via Bluetooth Low Energy.

Battery lasts around a year (on paper) so they make for cheap alternatives to the Zigbee temperature + humidity sensor in the OP article (although that one is pretty cheap as well!)

No internet needed if you set up a Raspberry Pi or ESP32 and run some software which can e.g decode the BLE messages and publish them to MQTT. ESPHome has a component for this which can get you up and running in a few minutes if you have an ESP32 lying around (needs to be ESP32 as ESP8266 doesn't have Bluetooth).


That's pretty cool, though my WiFi is already pretty crowded, so rather than have an ESP32 in every room and on WiFi, I think I'll get an ITEAD ZigBee hub and go that route instead, thanks!


Yeah fair enough the Zigbee ones will last longer with one battery anyway.

I only have one ESP32 for the 4 BLE sensors, it handles them all fine across different corners of my flat (the furthest one is around 10m through 2 walls). The same ESP32 is also in my utility cupboard with a photoresistor attached taped to my electric meter which counts the LED pulses so I can see how much electricity I’ve used in Home Assistant as well. Esphome makes it easy to configure multiple use cases for one device like that


Hmm, that's interesting. I've never tried ESPhome, but I should. I rolled my own five years ago and it's been working fine so far, but ESPhome + Home Assistant might be easier to configure and update than Espurna.


Yeah I was using Espurna on my lightbulbs that I replaced the firmware with (they were Tuya-based bulbs so I swapped them for Espurna). It was rock solid but I decided to move to Esphome as I liked the YAML based config, makes for a more "infra-as-code" style for IoT devices :)

Since then I'm using it on all kinds now, my own ESP32 dev board like I mentioned for the BLE sensors, and a variety of Shelly 1 devices and random brands of WiFi smart plugs from Amazon.


That sounds great, I like Espurna for the web interface and general quality but I really hate the C-style header file configuration, it's very hard to update your code.

With YAML, I at least know that the file I have will continue to work across updates. Seems much more usable.


Why not use hostnames instead of IP addresses? No need to assign static IPs then. I'm using hostnames for all my HA devices.


Hostnames depending on multicast DNS doesn't always traverse over network interfaces, eg ethernet<->wifi, so it may be easier with static IP-addresses, or that the router is smart enough to at least re-assign the same IP the next time a MAC asks for one.


How do you resolve the hostnames to the dynamic IP addresses?


I’ve found the initial zigbee pairing more and more painful as the number of IoT devices in my house increases. Although normally a one-off at installation, with lots of devices present don’t expect it to work first time.

Occasionally when encountering bugs in various software/hardware combinations i have seen suggested fixes of ‘just re-pair everything to a hub’ which for a full house will take a day and drive you insane.

One additional concern is the environmental impact of the number of cr2302 batteries in use - one for most switches and sensor that aren’t permanently wired in - even with a 1-2 year battery life it adds up to a lot. I think firmware updates in current systems take a big bite out of theoretical battery life spans.


Speaking about network, I would have dedicated few lines to PJON: https://github.com/gioblu/PJON


I would have thought just using Unifi's DNS server to address each device would have been far simpler.


Left a comment on the article with similar thoughts https://www.troyhunt.com/iot-unravelled-part-2-ip-addresses-...

Pi-hole instead of Unifi's DNS, hostnames either come from the client provided hostname or from a override file mapping MACs to hostnames.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: