fauxmo: Home Automation with the Amazon Echo, Raspberry Pi, and HomeAssistant

February 27, 2016, 5:39 am

≫ Next: Installing CrashPlan on the Raspberry Pi (Raspbian Jessie)

≪ Previous: How to Control an RF Outlet with Siri using HomeKit via Homebridge

Bottom Line: I updated a Python script called Fauxmo that lets you control your Raspberry Pi with an Amazon Echo (and now supports HomeAssistant).

I was looking for a way to control my home automation devices with my new Amazon Echo, and I came across Fauxmo. I liked how it was local and didn’t require port forwarding or lambda functions like using an Alexa Skill would. Additionally, I didn’t like the required verbiage for running actions as a skill, e.g. “Alexa, tell the house to turn on the kitchen light” — I want to just say “Alexa, turn on the kitchen light,” which is supported by Fauxmo. I especially liked the fact that Fauxmo was written in Python, but I prefer Python3, so I decided to update it.

Fauxmo works by pretending to be a WeMo device, which the Echo supports natively. I don’t know much about websockets or upnp, so it was definitely a learning process, but I eventually got something that works. You can find my updated version at: https://github.com/n8henrie/fauxmo, or you should also be able to install through PyPI: pip3 install fauxmo. One minor change (that made a big difference for me) was implementing a new handler for Home Assistant services.

After months of trying to decide on a central system to run my home automation, I’ve finally settled on Home Assistant— for a lot of reasons. It’s Python3, so I can tool around on it and hopefully even make a few contributions, but more importantly I can understand how it’s working. The interface is beautiful, development and the community are active, and the documentation is pretty good. Having a web-based interface means I can use any mobile or desktop interface I want, and once I have it set up with SSL, I’ll be able to access everything remotely as well.

Once I decided that Home Assistant (running on my Raspberry Pi 2) would be the core of my Home Automation system, I wanted to make sure that all my other systems worked through Home Assistant if possible. Luckily, in addition to its REST API, it has an awesome Python API. Because I have Home Assistant running on the same device that runs Fauxmo, I was able to implement a new type of handler that simplifies things a bit by using the Python API.

The reason I’ve done it this way is that Home Assistant now keeps track of the state of the devices for me. For example, I can turn on my kitchen light through Alexa: “Alexa, turn on the kitchen light”. Alexa then uses the Home Assistant API to turn on the light, so the next time I open Home Assistant, I’ll see the kitchen light button indicating the light is on.

Continuing this example of running everything through Home Assistant, you may have seen my recent post on home automation with Siri / HomeKit, which uses a Raspberry Pi running HomeBridge. ~~Well, it ended up being pretty simple to use the legacy HomeAssistant plugin to expose my Home Assistant services to homebridge~~— there is now even a mention of this method on the official Home Assistant blog. Update 20160310: as per discussion in this thread, GitHub user Maddox has moved the homebridge-homeassistant plugin out of legacy and into ~~its own repo~~ now moved into its own repo in the home-assistant organization.</p>

This way, I can open up Home Assistant and turn on my kitchen light — which will obviously have Home Assistant indicate that the light is on. I could then turn it off with Siri: “Hey Siri, turn off the kitchen light,” and Home Assistant will update to show that the light is off. I can then turn it on again with Alexa: “Alexa, turn on the kitchen light.” Home Assistant will then correctly show that the light is on again, and will show in its history that the light was turned on, off, then on again.

Schematically, it’s something like this:

Having this kind of central system will be critical for more advanced automation schemes in the future, because they will depend on the accuracy of the reported states of devices. You can’t exactly have a system “turn off the nightlight when the overhead light is turned on” unless it knows when the overhead light is on. This way, I can still use the dumb but super cheap RF switches I like for a lot of simple tasks and still keep pretty good track of their states, even though they don’t support reporting their state to controllers, and at the same time I avoid locking myself into a single system for interacting with them.

↧

Installing CrashPlan on the Raspberry Pi (Raspbian Jessie)

June 11, 2016, 10:35 am

≫ Next: Template for more efficient AppleScript Folder Actions

≪ Previous: fauxmo: Home Automation with the Amazon Echo, Raspberry Pi, and HomeAssistant

Bottom Line: My strategy for getting CrashPlan working on the Raspberry Pi (currently on Raspbian Jessie).

I’ve been using a Raspberry Pi as a local backup destination for CrashPlan for a few years now. Recently, CrashPlan has started automatically upgrading itself every few weeks to months, which breaks my install. Every time, I end up Googling around and referencing this thread and a few others.

However, I find that I don’t need to take all the steps listed there, and much of the helpful material is in a long comment thread. Additionally, all of the setups I found included replacing libjtux.so and libmd5.so with some ARM compiled versions downloaded from somebody’s website without any kind of hash to verify the download integrity — this made me a little nervous, being that this software will likely “see” nearly all of my sensitive documents and materials.

A few months ago I went ahead and condensed my setup into a GitHub Gist, and with the 4.7.0 update (which again broke my CrashPlan on the Pi installation this month), I thought I’d go ahead and share. I never recommend just copying and pasting someone else’s script, especially when it’s this short and basic, so take a look and try to understand what each command is doing. EDIT: I’ve expanded the script a fair bit to make it a little more user friendly and complete. You can check out the revision history to see the much simpler version it started with.

This script assumes you’re choosing the default answers to all the prompts in the CrashPlan installer. However, it also assumes that you’re using a systemd service file to start CrashPlan (instead of the sysvinit style scripts it ships with). I find the systemd scripts much easier to understand and maintain, and I’ve included what I use below (as well as a prompt in the script to download mine if desired).

I’ve briefly re-tested the script on a blank Raspbian Jessie Lite installation and it seems to work well. I was able to connect to the UI via SSH as described in the official CrashPlan Docs

NB: This script does not cover mounting an external hard drive or configuring it as your CrashPlan backup destination.

My systemd service file:

Some folks have recommended increasing the number of inotify watches with the commands below; this is another thing that I didn’t do, and it seems to be running okay on my RPi 3.

echo 1048576 | sudo tee /proc/sys/fs/inotify/max_user_watchesecho'fs.inotify.max_user_watches=1048576' | sudo tee -a /etc/sysctl.conf

↧

Template for more efficient AppleScript Folder Actions

June 13, 2016, 2:43 am

≫ Next: List of Default Packages on Raspbian Stretch and Stretch Lite

≪ Previous: Installing CrashPlan on the Raspberry Pi (Raspbian Jessie)

Bottom Line: Here’s a template I wrote to help me write and test Applescript Folder Actions more efficiently.

Folder Actions are a super handy way that Apple has provided to automatically run an action when a new item is added to a folder. It provides easy ways for you to run the action on the file that was added, or even on every file in the folder.

It does not allow you to watch for modifications to files in the folder — for that you have to dive into launchd and things like WatchPaths, or have a look at Hazel.

With these limitations in mind, Folder Actions are still pretty handy — for example, you could have a Dropbox folder where you upload specific images from your mobile devices, and the folder action on your Mac could automatically do something to process those images.

The hardest thing about Folder Actions is that AppleScript can be frustrating to write (for me, at least), and writing what would seem like a “quick script” can end up being a bigger endeavor than I’d thought.

For that reason, when I think I have a type of script that I’ll use frequently, I often put together little template that I can use to derive the others from and speed up the process quite a bit, for example my template for Quicksilver Actions.

For Folder Actions, I’ve been pretty pleased with the template I put together below, so I thought I’d share it. I think it’s pretty self explanatory, but basically it has a main function that runs the code, and has two other functions that either feed test files into the main loop or the folder action files, depending on if it is run manually or if it is called as a folder action. In other words, once you fill in the property fields up top with a test folder and a test file in that folder, running the script manually in Script Editor should give you identical results as if it had been called as a proper Folder Action by adding the test file to the folder. It seems to be working well to let me get my scripts up and running, and once I’m satisfied I attach them with /System/Library/CoreServices/Folder Actions Setup.app and call it a day.

↧

List of Default Packages on Raspbian Stretch and Stretch Lite

September 19, 2017, 11:36 am

≫ Next: My Internet Speeds for the Last Year

≪ Previous: Template for more efficient AppleScript Folder Actions

Bottom Line: Here are the default packages on a fresh image of Raspbian Stretch and Stretch Lite

Please see my post of the packages on a default Raspbian Jessie and Wheezy installation for background and “methodology.”

Raspbian Stretch 2017-09-07
SHA-256: a64d742bc525b548f0435581fac5876b50a4e9ba1d1cd6433358b4ab6c7a770b

Raspbian Stretch Lite 2017-09-07
SHA-256: bd2c04b94154c9804cc1f3069d15e984c927b750056dd86b9d86a0ad4be97f12

↧

My Internet Speeds for the Last Year

October 7, 2017, 9:30 am

≫ Next: Plotting US Census Data with Python and GeoPandas

≪ Previous: List of Default Packages on Raspbian Stretch and Stretch Lite

Bottom Line: I decided to post some visualizations of my internet speeds over the last year with different providers.

I’ve changed internet providers a few times over the last year (I guess closer to a year and a half now), and I thought it might be interesting to post how my speeds have fared.

The data has been collected in a few different ways:

This little wrapper script that takes a configurable number of runs ofspeedtest-cli, optionally drops the highest and lowest, and averages the remainder
The Home Assistant Fast.com sensor

To get the few hundred kb of download data out of my several hundred mb Home Asisstant database, I shut down the hass server and ran the following:

$ sqlite3 -csv config/home-assistant_v2.db \'select state, created from states      where entity_id is "sensor.fastcom_download"' \>hass_speeds.csv

I think the best way to display the notebook is actually downloading the notebook as HTML and simply hosting that file as a separate page, so that’s what I’ve done.

Please see my results here: Notebook: My Internet Speeds for the Last Year.

↧

Plotting US Census Data with Python and GeoPandas

November 28, 2017, 8:34 am

≫ Next: How to Open Javascript Bookmarklets from the Firefox Quantum URL Bar

≪ Previous: My Internet Speeds for the Last Year

Bottom Line: Here’s a simple way to plot some of the US Census data.

I’ve been wanting to learn how to do some simple geo data plotting in Python for a while, so I finally sat down and figured out the first few steps.

A lot of the US Census data is freely available to download fromcensus.gov, so finding and downloading it was easy. The part that actually tripped me up the most was figuring out how to load the data into GeoPandas. I first tried loading the file with the built-in read_file() method, which didn’t work. I tried extracting the data from the zipfile a bunch of different ways and loading the shapefiles using Fiona and nothing seemed to work… until I found a blog post where someone was prefixing their filename with zip:// before reading into GeoPandas. Sure enough that worked like a charm.

Anyway, you can view my Jupyter notebook to see how it came out.

↧

How to Open Javascript Bookmarklets from the Firefox Quantum URL Bar

November 29, 2017, 8:02 am

≫ Next: Pyenv Size and Python 3.6 Speed and Installation Time on Raspberry Pi

≪ Previous: Plotting US Census Data with Python and GeoPandas

Bottom Line: Firefox filters out javascript bookmarklets unless you change a setting in about:config.

After many years of Chrome, a few months on Vivaldi, and a few months on Safari, I’ve been really enjoying Firefox Quantum for the last couple weeks. Not only do I really like Mozilla as a company and their focus on privacy and open source software, but I think that Rust is a really interesting language with a lot of promise, and it certainly seems to have helped make Firefox Quantum really fast.

One minor annoyance that I’ve been struggling with is migrating over all my Chrome Custom Search Engines, ends up it’s not all that difficult – you can use the Bookmark manager to basically just make a regular bookmark with the familiar %s pattern to replace your search query, then add a keyword. So for example:

This makes it so I can just type e.g. sr learnpython in the address bar and jump directly to r/learnpython. Another really handy one that I use all the time is the Python3 documentation – every time there is a new release, I download the docs in html format, then open the local index file and add the search engine. It ends up looking like:

file:///path/to/python-3.6.3-docs-html/search.html?q=%s&check_keywords=yes&area=default

I add the keyword pydocs, then anytime I need I can jump to the URL bar and type e.g. pydocs configparser and lightning-fast jump to the relevant docs (even offline).

Similarly, you can right click on most websites’ search boxes and add a shortcut that way.

However, this post is actually about javascript bookmarklets, which I’ve used heavily in the past and still use frequently. Firefox has a really spectacular feature that allows you to quickly filter your address bar suggestions by prefixing the query with some predefined symbols – taken directly from their support page,

Add ^ to search for matches in your browsing history.
Add * to search for matches in your bookmarks.
Add + to search for matches in pages you’ve tagged.
Add % to search for matches in your currently open tabs.
Add ~ to search for matches in pages you’ve typed.
Add # to search for matches in page titles.
Add @ to search for matches in web addresses (URLs).
Add $ to search for matches in suggestions.

They include a cool example:

If you still have too many results, you can further restrict the search by making your search string mozilla * support #. Now the autocomplete list will only show bookmarked pages with mozilla and support in the page title.

So I thought I’d be able to quickly jump to my “Add to Pinboard” bookmarklet with something like * pinboard, but for some reason it wasn’t showing up – driving me crazy for weeks. It worked fine if I clicked on the bookmark manually, ensured I was spelling everything right, but still no luck. My other (non-javascript) bookmarks showed up just fine, but none of my bookmarklets appeared.

Today I finally spent a little time trying to figure out the issue, and I came across a relevant bug report that I think explains why these are filtered out by default, and it mentioned a config setting: browser.urlbar.filter.javascript.

I opened about:config, clicked the “I accept the risks” warning, and searched for browser.urlbar.filter.javascript– et viola. Double clicked to toggle to false, and instantly * pinboard worked as hoped.

Anyway, hope this helps someone else out there save a little time and frustration.

A few related posts:

↧

Pyenv Size and Python 3.6 Speed and Installation Time on Raspberry Pi

February 9, 2018, 7:54 pm

≫ Next: Securing the Kankun IoT Outlet

≪ Previous: How to Open Javascript Bookmarklets from the Firefox Quantum URL Bar

Bottom Line: This post includes sizes and time-to-install Python3 on a Raspberry Pi via pyenv.

I like Python 3.6, and I’ve converted over a few of my little GitHub projects to require Python >= 3.6 so that I can leverage features like f-strings, the new type hint syntax, updates to asyncio (like async generators), and other new additions to the language.

I run most of my projects on a few Raspberry Pis (including a B+, a 2, a few 3s, and a zero), currently all running Raspbian Stretch Lite. The newest Python available in Stretch is 3.5.3 at the time of writing, so I usepyenv to install 3.6. I generally compile with CONFIGURE_OPTS="--enable-optimizations" to squeeze a little extra speed out of the Pi if I can, realizing that compiling with optimizations takes quite a bit longer, but I think it’s worth it in the long run (since this is only once every few months at most).

Recently, I had an interesting discussion with another developer who generously maintains compatibility with older Python versions for the sake of his users. He was understandably frustrated that one of my libraries requires a Python version more recent than what is available in the Raspbian repos. He seemed hesitant to install pyenv on his Pi.

I decided to take a closer look at exactly how much time and space it takes to install Python3 via Pyenv on a Raspberry Pi.

The script below

Shows the latest Python3 version at /usr/bin/python3 and does a simple pystone benchmark
Installs this same Python version via Pyenv and shows the new repo size and its pystone benchmark
Repeats this with the optimizations flag I mentioed above
Repeats this with Python 3.6.4 (most recent release at time of writing) and the optimizations flag

All of the below is run on a Raspberry Pi 3 (BCM2835) unless noted otherwise.

Results

For comparison, it looks like the python3 package takes about 68 MB:

$ sudo apt-get --assume-no autoremove python3
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  apparmor dh-python libapparmor-perl libpython3-dev libpython3-stdlib
  libpython3.5-dev python3 python3-cffi-backend python3-crypto
  python3-cryptography python3-dbus python3-dev python3-gi python3-idna
  python3-keyring python3-keyrings.alt python3-minimal python3-pigpio
  python3-pip python3-pkg-resources python3-pyasn1 python3-secretstorage
  python3-setuptools python3-six python3-venv python3-virtualenv python3-wheel
  python3-xdg python3.5
  python3.5-dev python3.5-minimal python3.5-venv ufw virtualenv
0 upgraded, 0 newly installed, 34 to remove and 0 not upgraded.
After this operation, 68.3 MB disk space will be freed.
Do you want to continue? [Y/n] N
Abort.

In comparison, pyenv has some dependencies that seem to total about ~13.9 MB. The repo itself is about 4.8 MB, and with Python 3.5.3 installed it goes up to 169 MB, bringing it up to 183 MB total space requirement for a pyenv-installed 3.5.3.

make Installed-Size: 1,196 kB
build-essential Installed-Size: 20.5 kB
libssl-dev Installed-Size: 5,135 kB
zlib1g-dev Installed-Size: 393 kB
libbz2-dev Installed-Size: 85.0 kB
libreadline-dev Installed-Size: 473 kB
libsqlite3-dev Installed-Size: 1,702 kB
wget Installed-Size: 2,734 kB
curl Installed-Size: 316 kB
llvm Installed-Size: 62.5 kB
libncurses5-dev Installed-Size: 1,207 kB
xz-utils Installed-Size: 513 kB
tk-dev Installed-Size: 22.5 kB

The pystone benchmark for the pyenv-installed 3.5.3 is slightly slower than the Raspbian repo 3.5.3: 2.1506 vs 2.24193 seconds for 20,000 passes, for a difference of about 4%. It took 979 seconds to install (~16.3 mins).

The optimized 3.5.3, on the other hand, is only about 3 MB heavier and fares much better at 1.8656 seconds for 20,000 passes on pystone, about 13% faster. The build time, however, is much longer at 36,690 seconds (about 10 hours).

An optimized 3.6.4 is 2 MB heavier than this (174 MB + 13 MB pyenv deps = 187 MB total) and fares even better against Raspbian’s 3.5.3: 2.1506 for 20,000 passes, about 16% improvement. For some reason, it took about half the time to build as the optimized 3.5.3 (16845 seconds, < 5 hours).

Finally, while not reflected in the script and output below, I did compare build time and pystone results on my Pi Zero – for an optimized 3.6.4, it took about 34815 seconds (9.6 hours) to build and took 4.71322 for 20,000 passes on pystone (compared to 5.87804 for its Raspbian-installed 3.5.3, nearly20% faster).

So overall, a pyenv-installed, optimized Python build on my Pi 3:

Adds about 175 MB or so for every Python version
Requires an overnight build process
Runs about 15% faster than the Raspbian Python3
Gives me access to use the latest Python features in my home automation setup

For me, this tradeoff is worthwhile; a 64 GB micro SD card runs about $22 as of the time of writing, so that’s around 10 cents worth of storage plus an overnight build every few months. For others it may not be, and that’s okay too.

The scripts I used to get this data are included below. I don’t claim to be an expert, so feel free to comment if you think I’ve gone about this wrong or want to suggest an improvement or correction.

Script and Output

#! /bin/bashset-euf-o pipefail

test_pystone(){local python_path="${1}"local opts="$([-z"${2:-""}"]&&echo""||echo"with ${2:-""}")"local pystone_loops=20000local loops=10echo"Testing pystone for ${python_path}$($python_path--version)${opts}"best_pystone=$(for(( i = 0; i < $loops; i++ ));do"${python_path}"-m test.pystone $pystone_loops | head -n 1 | sed 's|^.*= ||'done | sort -n | head -1)echo"best of $loops sets of $pystone_loops loops, lower is better: ${best_pystone}"}

pyenv_install(){local version="${1}"local opts="${2:-""}"echo-e"\nInstalling python ${version} with options: ${2:-none}"local TIMEFORMAT=%Rlocal python_install_time="$(time(\export PYENV_ROOT=./pyenv[-n"${opts}"]&&export${opts}
    ./pyenv/bin/pyenv install "${version}"&>/dev/null \) 2>&1)"echo"Time to install: $python_install_time"}

pyenv_uninstall(){local version="${1}"PYENV_ROOT=./pyenv ./pyenv/bin/pyenv uninstall -f"${version}"if[!-d ./pyenv/versions/"${version}"];thenecho"Uninstalled python ${version} from ./pyenv"elseecho"Unable to uninstall python ${version}, exiting"exit 1fi}latest_pyenv_raw=$(curl -s https://github.com/pyenv/pyenv/tags.atom | grep-om 1 '    <title>.*</title>')[[$latest_pyenv_raw=~ \<title\>(.*)\</title\>]]&&latest_pyenv_version="${BASH_REMATCH[1]}"

rm -rf ./pyenv
git clone --branch="$latest_pyenv_version"--depth=1 https://github.com/pyenv/pyenv.git >/dev/nullecho"Bare pyenv directory size: $(du -sh ./pyenv)"echoraspbian_python3_version="$(/usr/bin/python3 --version | sed 's|^Python ||')"echo"Comparing with Raspbian Python $raspbian_python3_version"
test_pystone /usr/bin/python3echopyenv_install $raspbian_python3_versionecho"pyenv directory size with (only) $raspbian_python3_version installed: $(du -sh ./pyenv)"
test_pystone ./pyenv/versions/$raspbian_python3_version/bin/python3
pyenv_uninstall $raspbian_python3_versionechopyenv_install $raspbian_python3_version"CONFIGURE_OPTS=--enable-optimizations"echo"pyenv directory size with (only) optimized $raspbian_python3_version installed: $(du -sh ./pyenv)"
test_pystone ./pyenv/versions/$raspbian_python3_version/bin/python3 "CONFIGURE_OPTS=--enable-optimizations"
pyenv_uninstall $raspbian_python3_versionechopyenv_install 3.6.4 "CONFIGURE_OPTS=--enable-optimizations"echo"pyenv directory size with (only) optimized 3.6.4 installed: $(du -sh ./pyenv)"
test_pystone ./pyenv/versions/3.6.4/bin/python3 "CONFIGURE_OPTS=--enable-optimizations"
pyenv_uninstall 3.6.4

Output:

Bare pyenv directory size: 4.8M	./pyenv

Comparing with Raspbian Python 3.5.3
Testing pystone for /usr/bin/python3 Python 3.5.3 
best of 10 sets of 20000 loops, lower is better: 2.1506


Installing python 3.5.3 with options: none
Time to install: 979.572
pyenv directory size with (only) 3.5.3 installed: 169M	./pyenv
Testing pystone for ./pyenv/versions/3.5.3/bin/python3 Python 3.5.3 
best of 10 sets of 20000 loops, lower is better: 2.24193
Uninstalled python 3.5.3 from ./pyenv


Installing python 3.5.3 with options: CONFIGURE_OPTS=--enable-optimizations
Time to install: 36690.053
pyenv directory size with (only) optimized 3.5.3 installed: 172M	./pyenv
Testing pystone for ./pyenv/versions/3.5.3/bin/python3 Python 3.5.3 with CONFIGURE_OPTS=--enable-optimizations
best of 10 sets of 20000 loops, lower is better: 1.8656
Uninstalled python 3.5.3 from ./pyenv


Installing python 3.6.4 with options: CONFIGURE_OPTS=--enable-optimizations
Time to install: 16845.605
pyenv directory size with (only) optimized 3.6.4 installed: 174M	./pyenv
Testing pystone for ./pyenv/versions/3.6.4/bin/python3 Python 3.6.4 with CONFIGURE_OPTS=--enable-optimizations
best of 10 sets of 20000 loops, lower is better: 1.80994
Uninstalled python 3.6.4 from ./pyenv

↧

Securing the Kankun IoT Outlet

February 24, 2018, 10:29 am

≫ Next: How to Download Your Google Search History

≪ Previous: Pyenv Size and Python 3.6 Speed and Installation Time on Raspberry Pi

Bottom Line: The Kankun IoT outlet is cheap and popular, but it seems to ping home to China – here’s how to stop that.

The little Kankun smart plugs were a big hit a couple of years ago – cheap (~$20), Wi-Fi enabled, and hackable. I bout a couple when I could find then on AliExpress, but the US version was often out of stock. As of time of writing, I actually am not finding them available at AliExpress (or Amazon either), so I hope they’re not gone for good.

The Kankun switches have SSH open, with root access available using the password p9z34c by default. There are lots of posts and projects about things you can tweak with this switch once you establish access – my favorite is https://github.com/homedash/kankun-json, which provides both a nice user interface as well as JSON API for interacting with the switch.

The switch also comes with an iOS app that can be used to set up WiFi and control the switch. Interestingly, I decided to snoop around with Wireshark and found that the switch seemed to be calling home to China (115.29.14.58). I didn’t feel really great about that, especially since it now had my home WiFi password, and I wasn’t doing anything in particular to interact with the switch when these packets were getting sent.

Kankun outlet phoning
home

Looking at the UDP packet content, I see: fc0f28cf5edc72e3afe5e523f9974f2299a6e356fd9d96c21d57f6f9864636d39e2367fe5f16aa7b33df5d7a147329627b80696dbf781669f28426bf552dad32

Fortunately, some hackers much smarter than I have already reverse engineered the AES key required to decode this UDP packet: fdsl;mewrjope456fds4fbvfnjwaugfo. Once you have SSH access, you can confirm that this key shows up in/sbin/kkeps_off and /sbin/kkeps_on, e.g. strings sbin/kkeps_off | grep fdsl.

With some help from StackOverflow, I hacked together the following Python script to use the AES key to decrypt the UDP output so I could get an idea of what info was getting sent back and forth to China.

importsysimportbinasciifromcryptography.hazmat.primitives.ciphersimportCipher,algorithms,modesfromcryptography.hazmat.backendsimportdefault_backenddefdecrypt(encrypted_hex):enc_bytes=binascii.unhexlify(encrypted_hex)key='fdsl;mewrjope456fds4fbvfnjwaugfo'.encode()decryptor=Cipher(algorithm=algorithms.AES(key),mode=modes.ECB(),backend=default_backend()).decryptor()decrypted=decryptor.update(enc_bytes)+decryptor.finalize()returndecrypteddefclean_output(raw):br=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'return[segment.decode()forsegmentinraw.replace(br,b'%%').split(b'%')]if__name__=="__main__":print()forlineinclean_output(decrypt(sys.argv[1])):print(line)

Using the output acquired from Wireshark:

$ python3 decrypt.py fc0f28cf5edc72e3afe5e523f9974f2299a6e356fd9d96c21d57f6f9864636d39e2367fe5f16aa7b33df5d7a147329627b80696dbf781669f28426bf552dad32

wan_device
00:15:61:bd:4b:c9
nopassword
close
heart

Decrypting a number of the incoming UDP packets from China showed nothing of particular interest:

wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:02:21
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:02:21
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:02:21
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:02:51
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:03:21
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:04:21
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:04:21
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:04:51
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:05:21
hack
wan_server
00:15:61:bd:4b:c9
nopassword
2016-08-13-06:05:51
hack

Regardless, I didn’t like the idea of this IoT device communicating with outside servers, especially since my only use for it would be local control under HomeAssistant; it has no reason to access anything outside of my local network.

So in order to secure it, I started looking into whether or not I could configure some kind of firewall to block communication with devices outside my local network. It ended up not being too difficult to do.

It ends up the Kankun switch’s little distro has iptables built in. You can read about the configuration file for the firewall which is found at/etc/config/firewall , but the key line is:

# include a file with users custom iptables rules
config include
	option path /etc/firewall.user

So what I did was to SCP a file to /etc/firewall.user with the following content:

# This file is interpreted as shell script.
# Put your custom iptables rules here, they will
# be executed with each firewall (re-)start.

# Internal uci firewall chains are flushed and recreated on reload, so
# put custom rules into the root chains e.g. INPUT or FORWARD or into the
# special user chains, e.g. input_wan_rule or postrouting_lan_rule.

# Get rid of all existing rules
iptables --flush
iptables --delete-chain

# Default to `DROP` for all
iptables --policy INPUT DROP
iptables --policy FORWARD DROP
iptables --policy OUTPUT DROP

# Allow in and out on loopback
iptables --append INPUT --in-interface lo --jump ACCEPT
iptables --append OUTPUT --out-interface lo --jump ACCEPT

# Allow new and existing incoming SSH from local network
iptables --append INPUT --protocol tcp --destination-port 22 --source 192.168.0.0/24 --match state --state NEW,ESTABLISHED --jump ACCEPT
# Allow existing outgoing SSH from local network
iptables --append OUTPUT --protocol tcp --source-port 22 --destination 192.168.0.0/24 --match state --state ESTABLISHED --jump ACCEPT

# Allow tcp from local network to port 80 (for web requests with https://github.com/homedash/kankun-json)
iptables --append INPUT --protocol tcp --destination-port 80 --source 192.168.0.0/24 --match state --state NEW,ESTABLISHED --jump ACCEPT
iptables --append OUTPUT --protocol tcp --source-port 80 --destination 192.168.0.0/24 --match state --state ESTABLISHED --jump ACCEPT

This setup basically allows local SSH connections as well as port 80 for the JSON API I referenced earlier. I have mine set up with a static IP address, so it seems to work fairly well with this setup, but people relying on DHCP would likely also need to open ports 67 and 68 to UDP traffic, though I’m not sure about that.

Once I had set up this firewall, I can confirm with Wireshark that I no longer see the communication to or from China, that I can still SSH into the device, and that the JSON API I installed from the project above still works.

Hope you find this interesting and useful and that your home’s IoT is a little more secure as a result. I don’t pretend to be a security expert or to have much experience with iptables, so if you have any feedback, suggestions, or other comments about the post, please let me know in the comments section below.

↧

How to Download Your Google Search History

July 11, 2018, 3:37 pm

≫ Next: How to Give Full Disk Access to a Binary in MacOS Mojave

≪ Previous: Securing the Kankun IoT Outlet

Bottom Line: Use Google Takeout -> My Activity to download your Google search history. Using Google Takeout, you can download every Google search you’ve made (and every result you’ve clicked) while logged in to your account.

Go to https://takeout.google.com/settings/takeout
Select My Activity
Choose a method to download your data

Pretty interesting, and a little entertaining to see what I was searching for a decade ago.

My first Google search of record was on Sep 1, 2007, when I searched for “free spreadsheet.”

↧

How to Give Full Disk Access to a Binary in MacOS Mojave

November 16, 2018, 10:19 am

≫ Next: Fix "NO WRITE" "Operation not permitted" when repairing Time Machine sparsebundle

≪ Previous: How to Download Your Google Search History

Bottom Line: Here are a few strategies for giving full disk access to a script on MacOS Mojave

The Problem

MacOS Mojave has new privacy protection features in place that prevent applications from reading a number of files unless explicitly given access by the user. This seems like a good feature and probably should be left enabled by default, but it presents a challenge for backup software; I noticedrestic having some permissions errors a few weeks ago and have been sorting through how to give it access to these protected files.

These permissions are found in System Preferences -> Security & Privacy ->Full Disk Access.

I’ve made some progress as outlined in this GitHub issue, and thought I’d write about it more at length here.

Of note, I’ve also found the following command to reset the relevant permissions helpful in digging through this: tccutil reset SystemPolicyAllFiles Sources: 1, 2

Update 20181121: I think I’ve found a much better route for anybody that can write a simple script in a compiled language like Go – see the update section below.

How to Access these Files

If you do the following, you can get access to the protected files:

Package a standard MacOS application that runs a script (including a restic backup script)
Move the application to /Applications (may not always be necessary)
Add that application to FDA
Run the application by opening the .app (either by double clicking via GUI, or open /path/to/Restic.app but not by directly using the executable stored in e.g. Restic.app/Contents/Resources/)

There are a few ways to accomplish #1 – Platypus (brew install platypus) provides an easy CLI that does the job in a single command:

$cat<<'EOF' > runrestic.sh#!/usr/local/bin/bash
RESTIC=/usr/local/bin/restic
export RESTIC_PASSWORD=asdf
export RESTIC_REPOSITORY=/tmp/resticmkdir -p $RESTIC_REPOSITORY$RESTIC init$RESTIC backup ~/Library/Mail &> /tmp/restic/log.txtEOF$ platypus runrestic.shCreating application bundle folder hierarchy
Copying executable to bundle
Copying nib file to bundle
Optimizing nib file
Copying script to bundle
Writing AppSettings.plist
Writing application icon
Writing Info.plist
Moving app to destination '/var/folders/xv/gqk_1btj70v8gfz8xb0ydw940000gn/T/tmp.xJ0rXd0iYc/Runrestic.app'
Registering app with Launch Services
Done$ mv Runrestic.app /Applications/$ open /Applications/Runrestic.app$ du -sh /tmp/restic && sleep 5 && du -sh /tmp/restic24K     /tmp/restic
24K     /tmp/restic$ rm -r /tmp/restic$## Add to FDA ##$ open /Applications/Runrestic.app$ du -sh /tmp/restic && sleep 5 && du -sh /tmp/restic140M    /tmp/restic
227M    /tmp/restic$ pkill restic

Unfortunately, running the embedded script directly for some reason won’t work (does not seem to inherit the FDA access):

$ rm -r /tmp/restic$ /Applications/Runrestic.app/Contents/Resources/scriptcreated restic repository 4585354284 at /tmp/restic

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
$ du -sh /tmp/restic && sleep 5 && du -sh /tmp/restic24K     /tmp/restic
24K     /tmp/restic$cat /tmp/restic/log.txtcreated new cache in /Users/me/Library/Caches/restic
can not obtain extended attribute com.apple.quarantine for /Users/me/Library/Mail:
error: Open: open /Users/me/Library/Mail: operation not permitted

Files:           0 new,     0 changed,     0 unmodified
Dirs:            3 new,     0 changed,     0 unmodified
Added to the repo: 1.122 KiB

processed 0 files, 0 B in 0:00
snapshot 2c62de89 saved

Other options (that don’t require 3rd party software) include using the built-in Automator.app or Script Editor.app to do essentially the same thing, as long as you click the option to save as an Application (not a script). As AppleScript:

onrundo shell script"/usr/local/bin/bash /path/to/runrestic.sh"endrun

Same deal, move to /Applications/ and add to FDA, run by either double clicking or by open /Applications/Test.app.

Note that there are some really screwy bugs with the system, which is what I blame for how long it’s taking me to make any progress. For example, with the following AppleScript saved as an Application:

onrunsetwhoamito(do shell script"whoami")setlsto(do shell script"/bin/ls /Users/me/Library/Mail")display dialogwhoami&return&lsendrun

If I save this to /Applications/Test.app and add to FDA, it works. If I then duplicate a copy to ~/Desktop/Test.app, it works the first time, then stops working (permissions error), regardless of which Test.app I open. Then I have to reset the permissions, then re-add to make the /Applications one start working again – it often fails the first time I try to run it, then works after that. [EDIT: I wonder if this may actually be due to a “last modified” timestamp changing, instead of having to be in the /Applications folder – I currently have an application on my Desktop that seems to be working fine.]

Also, running from the built-in “Run” button in Script Editor ( ▶ ) never works (permissions errors), has to be run by double clicking the icon or open /path/to/Test.app from the command line.

Finally, if the application changes at all, you have to remove and then restore FDA access to get it working again. This includes just the timestamp changing due to saving the application, even if none of the code has changed. I think this may be one of the confounding factors in my testing; if I leave Script Editor.app running while I experiment, it may be auto-saving in the background which may be giving me intermittent errors that are tough to debug.

Ongoing Issues in Automating the Process

The big problem I’m stuck with at this point is giving read permissions for system files that I would like to be backed up (i.e. files that I need to be root to read, for example /etc/master.passwd).

Using the AppleScript example from above (basically ls ~/Library/Mail, which is a protected directory), I can put a script into e.g./Library/LaunchDaemons/com.n8henrie.test.plist that contains the following (among other boilerplate code), and make that file owned by root. If I thensudo launchctl load and then sudo launchctl start it, I get the dialog box showing that the FDA permissions worked properly, but at the top of the dialog, I see that whoami is still getting run as my user (not root).

...<array><string>/usr/bin/open</string><string>/Applications/Test.app</string></array>
...

This means that running my restic backup script in this way seems like it will work from the FDA perspective, but won’t be able to access any files that are e.g. 0600 root:wheel.

Some workarounds include changing the applescript to include do shell script ... with administrator privileges, which will cause it to prompt me for my password and then run as root… but that won’t work from an automation standpoint.

The only other workaround I’ve discovered is to sudo visudo and add give myself the ability to run as sudo with NOPASSWD:, then change the AppleScript to e.g. sudo -n /bin/ls /etc/master.passwd.

Oddly, if I make a root-owned launchd plist in /Library/LaunchDaemons/ that directly runs the command whoami and outputs the result to a text file on the desktop, it outputs root (even without the UserName key).

Frustratingly, if I change that plist to run open /path/to/Test.app, whereTest.app is an AppleScript that outputs whoami to a text file, I get my regular username irrespective of whether the UserName key is set to root or not.

This means that I can’t set my launchd script to run my Restic.app and have sufficient privileges to read root-only files, even if the launchd script is in/Library/LaunchAgents, root owned, and set to run as UserName: root.

The only workaround I’ve figured out so far is to:

sudo visudo and give my unprivileged user the ability to run my restic backup script as sudo with NOPASSWD: (I also specify the hash of the script to hopefully improve security)
chown root the script
chmod 0740 the script (4 so it can still be added to VCS)
Change Restic.app to run sudo -n /path/to/my-script.sh
Add Restic.app to FDA
Change my launchd script to open /path/to/Restic.app

This seems really sloppy, and I’d love to hear suggestions from other readers.

Update 20181121

After some preliminary testing, it seems like a much cleaner solution to all of the above (including running with root privileges) is to just make a binary (in a compiled language) that runs your script, then add that binary to Full Disk Access.

Note that this doesn’t work for something like a shell script; you can’t add the script to Full Disk Access (even if you chmod +x). You can only add the e.g. bash binary that runs the script, but that means that any process spawned by bash can access all your files. That seems like a huge security vulnerability.

Anyway, here is some example code in Go that runs a bash script namedrestic-backup.sh in the directory containing the resulting go binary.

// Runrestic provides a binary to run my restic backup script in MacOS Mojave// with Full Disk Accesspackagemainimport("log""os""os/exec""path/filepath")funcmain(){ex,err:=os.Executable()iferr!=nil{log.Fatal(err)}dir:=filepath.Dir(ex)script:=filepath.Join(dir,"restic-backup.sh")cmd:=exec.Command("/usr/local/bin/bash",script)iferr:=cmd.Run();err!=nil{log.Fatal(err)}}

You can add this binary to Full Disk Access and the spawned processes (including the shell script it calls) should inherit Full Disk Access. Don’t forget that if you make any changes to the Go binary, you need to remove it from FDA and then add it back.

Also note that this seems to work great for scripts requiring root access; instead of all the hacky mess above, just have your root-owned/Library/LaunchDaemons plist call this binary. In this case, I’d highly recommend that you secure both the binary and the shell script it calls, since otherwise an attacker could easily overwrite either one with something malicious and it would get called with root privileges. An example of a few simple steps may be to sudo chown root both of these files, sudo chmod 0700 the Go binary, and sudo chmod 0640 the shell script (keeping yourself in the group so you can still add it to VCS if desired).

I’ve had a few promising leads on this problem that haven’t panned out, so I’ll update in a few days if this stops working, but it seems to be doing the trick for now.

Fix "NO WRITE" "Operation not permitted" when repairing Time Machine sparsebundle

November 26, 2018, 3:46 pm

≫ Next: Reverse Engineering My WiFi Endoscope Part 1

≪ Previous: How to Give Full Disk Access to a Binary in MacOS Mojave

Bottom Line: If you’re getting a NO WRITE error on MacOS Mojave when trying to repair a Time Machine sparsebundle, try adding /sbin/fsck_hfs to Full Disk Access.

My Time Machine backup goes to a hard drive connected to my Airport Extreme.

Unfortunately, it’s somewhat common to occasionally get an error messages that goes something like: Time Machine completed a verification of your backups on … Time Machine must create a new backup for you.

The fix for this is posted on numerous blogs, so when I occasionally get this error, I usually just Google the message and follow the instructions.

However, yesterday I tried to run the commands and kept running into an error when you get to the part about using fsck_hfs to actually fix the problem.

The error I kept getting was:

$sudo /sbin/fsck_hfs -drfy-c 2200m /dev/rdisk2s2Unable to open block device /dev/disk2s2: Operation not permittedjournal_replay(/dev/disk2s2) returned 1
** /dev/rdisk2s2 (NO WRITE)
Can't open /dev/rdisk2s2: Operation not permitted

I searched around and couldn’t find anything on this except to use sudo diskutil repairVolume /dev/disk2s2 instead of fsck_hfs– which seemed to work, but also seemed agonizingly ~50% in 12 hours, over a wired connection).

Luckily, I’d recently been doing some digging around with the new Full Disk Access permissions in Mojave, so I thought I’d see if that was related.

Sure enough, once I added /sbin/fsck_hfs to System Preferences -> Security& Privacy -> Full Disk Access, I could run the fsck command.

Unfortunately, it looks like my drive is probably failing. Good thing it’s Cyber Monday, I guess.

↧

Reverse Engineering My WiFi Endoscope Part 1

February 1, 2019, 9:10 am

≫ Next: Reverse Engineering My WiFi Endoscope Part 2

≪ Previous: Fix "NO WRITE" "Operation not permitted" when repairing Time Machine sparsebundle

Bottom Line: I got root on a WiFi Endoscope.

Part 1: Introduction and Serial Debug Port to Get WiFi Password

A few months ago, I bought a THZY WiFi endoscope to help me find a bolt that I had dropped into a tight space in an appliance I was trying to repair. I was hoping to find an endoscope that would work by lightning connector to my iPhone, but this one looked like the next best alternative. The reason I would have preferred something wired is because this one requires a special app to view its stream, and I worry that the app will become unmaintained or break, rendering the device useless.

The device itself works well enough. It creates its own WiFi network, to which you connect your iPhone, and from there the app works fairly well to view the screen and change basic settings (like the network name and password).

A week or two later, my fears were realized when I somehow locked myself out of the WiFi network, presumably by accidentally changing the password. I could not find any way to do a “factory reset” to restore the device to its default settings. A Google search found someone in a similar situation, reminding me of this:

Having read some interesting posts on reverse engineering WiFi routers, I decided to try to take the device apart to see if I could sort things out. It ended up being far more difficult to take apart the small plastic box than I anticipated; I think it may have been glued together. Inside, I found this board:

At the bottom, you can see 5 ports labeled TX2, RX2, RX1, TX1, andGND.

I don’t have an oscilloscope unfortunately, so I hooked up my trusty FTDI to multiple ports basically at random and ran through a list of common baud rates until I found one with intelligible output.

The winning combination ended up being to use 57600 baud and connect:

Endoscope	FTDI
TX2	RX
RX2	TX

I recorded the boot log with picocom --baud 57600 /dev/tty.usbserial-A50285BI --logfile boot.log, and its output is below (it’s somewhat long, so I’m putting it at the end of the post, and I’ve cleaned up a few escape chars and other minor blemishes).

A few things to note from the boot log:

It doesn’t include an option to boot to cmd prompt:

Please choose the operation:
   1: Debug TFTP.
   2: Update from TFTP.
   3: Run.
   9: Update uboot from TFTP.

You choosed 3

It defaults to 3 if it times out or if the serial connection isn’t properly configured to send data. You can access the other options listed above, but I couldn’t figure out how to do anything interesting with them (possibly just due to my inexperience). I didn’t seem to be able to interrupt things or do anything interesting by hitting escape during the boot sequence.

The WiFi access point and password are printed out:

ap_name:	Jetion_fd20f768
ap_pass:	12345678

It’s running busybox:

BusyBox v1.12.1 (2017-04-22 17:33:57 CST) built-in shell (ash)

Thankfully, I was able to use the password it printed out to regain access to its network, restoring my ability to use the endoscope. In part 2 of the series, I’ll go through how I used a combination of tools like nmap, binwalk, and a bus pirate to get root access on the device via telnet.

Boot log:

U-Boot 1.1.3 (Sep  2 2013 - 22:03:00)

Board: Ralink APSoC DRAM:  32 MB
relocate_code Pointer at: 81fb8000
spi_wait_nsec: 42
spi device id: ef 40 16 0 0 (40160000)
find flash: W25Q32BV
raspi_read: from:30000 len:1000
.*** Warning - bad CRC, using default environment

============================================
Ralink UBoot Version: 4.1.1.0
--------------------------------------------
ASIC 5350_MP (Port5<->None)
DRAM_CONF_FROM: Boot-Strapping
DRAM_TYPE: SDRAM
DRAM_SIZE: 256 Mbits
DRAM_WIDTH: 16 bits
DRAM_TOTAL_WIDTH: 16 bits
TOTAL_MEMORY_SIZE: 32 MBytes
Flash component: SPI Flash
Date:Sep  2 2013  Time:22:03:00
============================================
icache: sets:256, ways:4, linesz:32 ,total:32768
dcache: sets:128, ways:4, linesz:32 ,total:16384

 ##### The CPU freq = 360 MHZ ####
 estimate memory size =32 Mbytes

Please choose the operation:
   1: Debug TFTP.
   2: Update from TFTP.
   3: Run.
   9: Update uboot from TFTP.

You choosed 3

0

3: System Boot system code via Flash.
## Booting image at bc050000 ...
raspi_read: from:50000 len:40
.   Image Name:   Linux Kernel Image
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    1578918 Bytes =  1.5 MB
   Load Address: 80000000
   Entry Point:  8031f000
raspi_read: from:50040 len:1817a6
.........................   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK
No initrd
## Transferring control to Linux (at address 8031f000) ...
## Giving linux memsize in MB, 32

Starting kernel ...


LINUX started...

 THIS IS ASIC
Linux version 2.6.21 (tony@ubuntu) (gcc version 3.4.2) #545 Sun Jul 30 16:41:50 CST 2017

 The CPU frequency set to 360 MHz
CPU revision is: 0001964c
Determined physical RAM map:
 memory: 02000000 @ 00000000 (usable)
Initrd not found or empty - disabling initrd
Built 1 zonelists.  Total pages: 8128
Kernel command line: console=ttyS1,57600n8 root=/dev/ram0
Primary instruction cache 32kB, physically tagged, 4-way, linesize 32 bytes.
Primary data cache 16kB, 4-way, linesize 32 bytes.
Synthesized TLB refill handler (20 instructions).
Synthesized TLB load handler fastpath (32 instructions).
Synthesized TLB store handler fastpath (32 instructions).
Synthesized TLB modify handler fastpath (31 instructions).
Cache parity protection disabled
cause = 50808008, status = 11000000
PID hash table entries: 128 (order: 7, 512 bytes)
calculating r4koff... 0015f900(1440000)
CPU frequency 360.00 MHz
Using 180.000 MHz high precision timer.
Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
Memory: 28556k/32768k available (2783k kernel code, 4212k reserved, 408k data, 640k init, 0k highmem)
Mount-cache hash table entries: 512
NET: Registered protocol family 16
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Time: MIPS clocksource has been installed.
deice id : ef 40 16 0 0 (40160000)
W25Q32BV(ef 40160000) (4096 Kbytes)
mtd .name = raspi, .size = 0x00400000 (4M) .erasesize = 0x00010000 (64K) .numeraseregions = 0
Creating 5 MTD partitions on "raspi":
0x00000000-0x00400000 : "ALL"
0x00000000-0x00030000 : "Bootloader"
0x00030000-0x00040000 : "Config"
0x00040000-0x00050000 : "Factory"
0x00050000-0x01000000 : "Kernel"
mtd: partition "Kernel" extends beyond the end of device "raspi" -- size truncated to 0x3b0000
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 1024 (order: 1, 8192 bytes)
TCP bind hash table entries: 1024 (order: 0, 4096 bytes)
TCP: Hash tables configured (established 1024 bind 1024)
TCP reno registered
detected lzma initramfs
detected lzma initramfs
initramfs: LZMA lc=3,lp=0,pb=2,dictSize=1048576,origSize=2059264
LZMA initramfs by Ming-Ching Tiew <mctiew@yahoo.com>................................RT3xxx EHCI/OHCI init.
io scheduler noop registered (default)
Ralink gpio driver initialized
HDLC line discipline: version $Revision: #1 $, maxframe=4096
N_HDLC line discipline registered.
Serial: 8250/16550 driver $Revision: #1 $ 2 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0xb0000500 (irq = 37) is a 16550A
serial8250: ttyS1 at MMIO 0xb0000c00 (irq = 12) is a 16550A
RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize
loop: loaded (max 8 devices)
rdm_major = 253
Ralink APSoC Ethernet Driver Initilization. v3.0  256 rx/tx descriptors allocated, mtu = 1500!
GMAC1_MAC_ADRH -- : 0x00008c88
GMAC1_MAC_ADRL -- : 0x2b508a5c
PROC INIT OK!


=== pAd = c0000000, size = 645976 ===

<-- RTMPAllocAdapterBlock, Status=0
block2mtd: version $Revision: #1 $
rt3xxx-ehci rt3xxx-ehci: Ralink EHCI Host Controller
rt3xxx-ehci rt3xxx-ehci: new USB bus registered, assigned bus number 1
rt3xxx-ehci rt3xxx-ehci: irq 18, io mem 0x101c0000
rt3xxx-ehci rt3xxx-ehci: USB 0.0 started, EHCI 1.00, driver 10 Dec 2004
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 1 port detected
rt3xxx-ohci rt3xxx-ohci: RT3xxx OHCI Controller
rt3xxx-ohci rt3xxx-ohci: new USB bus registered, assigned bus number 2
rt3xxx-ohci rt3xxx-ohci: irq 18, io mem 0x101c1000
usb usb2: configuration #1 chosen from 1 choice
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 1 port detected
Advanced Linux Sound Architecture Driver Version 1.0.14rc3 (Wed Mar 14 07:25:50 2007 UTC).
usb 1-1: new high speed USB device using rt3xxx-ehci and address 2
usb 1-1: configuration #1 chosen from 1 choice
usbcore: registered new interface driver snd-usb-audio
ALSA device list:
  No soundcards found.
nf_conntrack version 0.5.0 (256 buckets, 2048 max)
ip_tables: (C) 2000-2006 Netfilter Core Team, Type=Restricted Cone
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
Freeing unused kernel memory: 640k freed
init started: BusyBox v1.12.1 (2017-04-22 17:33Algorithmics/MIPS FPU Emulator v1.5
:57 CST)
starting pid 14, tty '': '/etc_ro/rcS'
devpts: called with bogus options
mount: mounting none on /proc/bus/usb failed: No such file or directory
##################################
# Welcome to wifi module world   #
##################################
starting pid 36, tty '/dev/ttyS1': '/bin/sh'


BusyBox v1.12.1 (2017-04-22 17:33:57 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

#  FILE: main_detect.c, LINE: 81: -----------> thread began to run now.

RX DESC a1cbd000  size = 2048
<-- RTMPAllocTxRxRingMemory, Status=0
Key1Str is Invalid key length(0) or Type(0)
Key2Str is Invalid key length(0) or Type(0)
Key3Str is Invalid key length(0) or Type(0)
Key4Str is Invalid key length(0) or Type(0)
1. Phy Mode = 9
2. Phy Mode = 9
3. Phy Mode = 9
MCS Set = ff 00 00 00 01
Main bssid = e8:ab:fd:20:f7:68<==== rt28xx_init, Status=0
0x1300 = 00064380
start rmmod the video modules now...
rmmod: uvcvideo.ko: No such file or directory
rmmod: videodev.ko: No such file or directory
rmmod: v4l2-common.ko: No such file or directory
rmmod: v4l1_compat.ko: No such file or directory
rmmod: compat_ioctl32.ko: No such file or directory
ap_head:	Jetion_
ap_name:	Jetion_fd20f768
ap_pass:	12345678
ap_uart:	1
cam_width:	640
cam_height:	480
version:	100
start insmod the video modules now...
Linux video capture interface: v2.00
Found format MJPEG.
- 640x480 (30.0 fps)
- 320x240 (30.0 fps)
- 800x600 (30.0 fps)
- 1024x768 (30.0 fps)
- 1280x720 (30.0 fps)
- 640x480 (30.0 fps)
Found format YUV 4:2:2 (YUYV).
- 640x480 (30.0 fps)
- 320x240 (30.0 fps)
- 1280x1024 (7.5 fps)
- 1600x1200 (3.0 fps)
- 640x480 (30.0 fps)
uvcvideo: Found UVC 1.00 device USB 2.0 Camera (090c:f37d)
uvcvideo: UVC non compliance - GET_DEF(PROBE) not supported. Enabling workaround.
input: USB 2.0 Camera as /class/input/input0
usbcore: registered new interface driver uvcvideo
USB Video Class driver (SVN r209)
F:src/app_video.

↧

Reverse Engineering My WiFi Endoscope Part 2

February 6, 2019, 7:01 am

≫ Next: Reverse Engineering My WiFi Endoscope Part 3

≪ Previous: Reverse Engineering My WiFi Endoscope Part 1

Bottom Line: I got root on a WiFi Endoscope.

Part 2: Reading Flash to Get Telnet Password

Part 1 review: I bought this WiFi endoscopeand accidentally locked myself out of the WiFi network it creates, but I was able to find the WiFi password in its boot log via a serial debug port.

At this point, I wanted to see what else I could do with the device. Unfortunately, I didn’t have any luck getting a command prompt from the serial port. However, once it was back up and running, I connected to the network with my Macbook to do some tinkering.

First, I noted that it always seemed to assign itself the address192.168.10.123, which seemed like it might be helpful with some Google searches.

Next, I ran an nmap scans and noted an open TCP port 23, often used for telnet, as well as an open TCP port 7060.

Connecting to 7060 with netcat got me a continuous stream of junk.

I hoped the telnet login would be something like root:root, but I didn’t have any luck with various combinations of root, admin, jetion,password, 123456, or 12345678 (the default WiFi password).

Next, I figured I could just run an automated cracker withhydra and some default usernames and passwords from SecLists. Surprisingly, that didn’t work either.

Next, I tried downloading the Android version of the app and decompiling it with apktool. Unfortunately, I couldn’t find any references in it to telnet, port 23, or any promising usernames or passwords. I didn’t find anything interesting about port 7060, either.

Eventually, I decided to buy a bus pirate and a suitable adapter for the 8-pin chip and see if I could burn a copy of the firmware. I’d always wanted to try this! Here’s a picture of the board again:

The chip of interest is the little 8 pin one in the top left. Here is a close-up:

It’s not a great shot, but you can just make out Winbond 25Q32.

Thankfully, since I had no clue what I was doing, I found a great walkthrough for this chip that showed me exactly how to hook up the bus pirate. For posterity, here’s a copy of the table graciously provided there:

Bus Pirate	Flash Chip	Description
CS	#1 CS	Chip Select
MISO	#2 DO (IO1)	Master In, Slave Out
3V3	#3 WP (IO2)	Write Protect
GND	#4 GND	Ground
MOSI	#5 DI (IO0)	Master Out, Slave In
CLK	#6 CLK	The SPI Clock
3V3	#7 HOLD (IO3)	Hold
3V3	#8 VCC	Supply

And here is its pad configuration, taken from the datasheet:

I ran the following, which correctly identified the chip, confirming things were connected properly:

$ flashrom -p buspirate_spi:dev=/dev/tty.usbserial-AK05V1SK 

Afterwards, the command below read the firmware to a local file namedW25Q32.V.eeprom:

$ flashrom -p buspirate_spi:dev=/dev/tty.usbserial-AK05V1SK,spispeed=30k \-c W25Q32.V -r W25Q32.V.eeprom -V

I was psyched! The next step should have been the easy part, which was usingbinwalk to extract the firmware. However, I soon discovered that the file didn’t quite work as expected. Binwalk extracted something, but it seemed incomplete, and I couldn’t find anything interesting that looked like a password.

I tried again, using the same flashrom command as above, and still it didn’t work. I soon figured out that the files were slightly different, suggesting that there were likely some minor errors during the file read process. I tried adjusting the clip and verified that all the connections were solid (though I didn’t bother with soldering them), but after multiple attempts I consistently ended up getting files that were slightly different, and binwalk couldn’t fully extract them.

Eventually, I decided to write a little bash script that would repeatedly try to grab a copy of the firmware, compare its md5 hash to that of all the priors, and if there was a match to stop the loop. If not, it tried again. I figured that eventually, once it got an exact duplicate, I could assume it had finally gotten a correct copy. Here’s my bash script:

#! /bin/bashcounter="$(ls*.eeprom | cut -d.-f3 | sort -n | tail -n 1)"((counter++))while : ;dotime flashrom -p buspirate_spi:dev=/dev/tty.usbserial-AK05V1SK,spispeed=30k \-c W25Q32.V -r W25Q32.V.${counter}.eeprom -Vmd5count=($(md5sum *.eeprom | awk '{ print $1 }' | sort | uniq -c | awk '{ print $1 }' | sort -u))if["${#md5count[@]}"-gt 1 ];thenbreakfi((counter++))done

Unfortunately, this never happened. I ended up grabbing hundreds of copies of the firmware, all with a few differing bytes, and therefore a different hash. Finally, after several days of running this, I decided to try a different approach: I’d take the binary content of all the files, and take the mode of each byte, and write that out to a file, assuming that the most common byte at each address was likely to be the correct data.

For this, I first tried to see if someone had already done the work for me, and I eventually found a blog post that outlines a similar strategy and the resulting Python code, but I ended up using a different strategy that leverages numpy to hopefully speed things up a bit. My code:

importpathlibimportnumpyasnpdefget_most_common_value(arr:np.array)->bytes:nancount=np.isnan(arr).sum()nums,counts=np.unique(arr,return_counts=True)ind=np.argmax(counts)returnnums[ind]if__name__=='__main__':arrs=np.array([np.fromfile(str(p),dtype="u1")forpinpathlib.Path(".").glob("W25Q32.V.*.eeprom")])modes=np.apply_along_axis(get_most_common_value,0,arrs)pathlib.Path("merged.eeprom").write_bytes(modes.tobytes())

For a little under 300 files, each of which is about 4 MB, it takes about three and a half minutes to run. Thankfully, the resulting file is extracted beautifully by binwalk. I’ve uploaded a copy of the gzipped eeprom for anybody interested in checking it out; md5 of the gzipped file is35d014b329d8e94318b59247a41139f6.

Here are the results of binwalk:

$ binwalk -Me merged.eeprom
Scan Time:     2019-02-01 22:10:34
Target File:   /Users/me/endoscope/merged.eeprom
MD5 Checksum:  8da2f08e906520d4a67fc20dec346360
Signatures:    344

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0x7EBF92E6, created: 2013-09-02 14:03:02, image size: 81904 bytes, Data Address: 0x80200000, Entry Point: 0x80200000, data CRC: 0x739ABB93, OS: Linux, CPU: MIPS, image type: Standalone Program, compression type: none, image name: "SPI Flash Image"
69296         0x10EB0         U-Boot version string, "U-Boot 1.1.3 (Sep  2 2013 - 22:03:00)"
327680        0x50000         uImage header, header size: 64 bytes, header CRC: 0x24EF36B4, created: 2017-07-30 08:41:54, image size: 1578918 bytes, Data Address: 0x80000000, Entry Point: 0x8031F000, data CRC: 0x67F11261, OS: Linux, CPU: MIPS, image type: OS Kernel Image, compression type: lzma, image name: "Linux Kernel Image"
327744        0x50040         LZMA compressed data, properties: 0x5D, dictionary size: 33554432 bytes, uncompressed size: 3926339 bytes


Scan Time:     2019-02-01 22:10:35
Target File:   /Users/me/endoscope/_merged.eeprom.extracted/50040
MD5 Checksum:  966ea70c3127490310ab0feec1b5e36f
Signatures:    344

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
2096284       0x1FFC9C        MySQL MISAM compressed data file Version 8
2859060       0x2BA034        Linux kernel version "2.6.21 (tony@ubuntu) (gcc version 3.4.2) #545 Sun Jul 30 16:41:50 CST 2017"2860048       0x2BA410        CRC32 polynomial table, little endian
2883552       0x2BFFE0        SHA256 hash constants, little endian
2960710       0x2D2D46        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/mlme.c:%d assert SupRateLen <= MAX_LEN_OF_SUPPORTED_RATESfailed
2960830       0x2D2DBE        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/mlme.c:%d assert ExtRateLen <= MAX_LEN_OF_SUPPORTED_RATESfailed2962282       0x2D336A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/action.c:%d assert pAd->BATable.BAOriEntry[i].Wcid < MAX_LEN_OF_MAC_TABLEfailed2962418       0x2D33F2        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/action.c:%d assert pBAEntry->Wcid < MAX_LEN_OF_MAC_TABLEfailed2962902       0x2D35D6        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert mpdu_blkfailed
2962998       0x2D3636        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert listfailed2963090       0x2D3692        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert mpdu_blk->pPacketfailed2963194       0x2D36FA        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert pBAEntry->list.qlen == 0failed2963494       0x2D3826        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert TID < NUM_OF_TIDfailed
2963990       0x2D3A16        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert pEntryfailed2964110       0x2D3A8E        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert pAd->BATable.numAsOriginator != 0failed2964274       0x2D3B32        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert pAd->BATable.numAsRecipient != 0failed2964810       0x2D3D4A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert pAd->MacTab.Content[Elem->Wcid].Sst == SST_ASSOCfailed2965394       0x2D3F92        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert pRxBlk->pRxPacketfailed2965546       0x2D402A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert (0<= pBAEntry->list.qlen)&&(pBAEntry->list.qlen <= pBAEntry2965706       0x2D40CA        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert pBAEntryfailed
2965850       0x2D415A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert 0failed2965938       0x2D41B2        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/ba_action.c:%d assert (pBAEntry->list.qlen == 0)&&(pBAEntry->list.next == NULL)fa2966274       0x2D4302        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pEntryfailed
2966566       0x2D4426        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert Length <= MGMT_DMA_BUFFER_SIZEfailed
2966738       0x2D44D2        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pTxWIfailed
2967178       0x2D468A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pProbeEntry != NULLfailed
2967318       0x2D4716        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pProbeEntryfailed
2967590       0x2D4826        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pSrcBuffailed2967682       0x2D4882        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert (pktLen > 34)failed2967782       0x2D48E6        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pRxBlk->pRxPacketfailed2968014       0x2D49CE        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pAd->FragFrame.LastFrag == 0failed2968126       0x2D4A3E        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pHeaderfailed2968282       0x2D4ADA        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data.c:%d assert pAd->FragFrame.pFragPacketfailed2969362       0x2D4F12        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/rtmp_init.c:%d assert (Length==0) || (pDest && pSrc)failed2989138       0x2D9C52        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_profile.c:%d assert pAd->ApCfg.MBSSID[idx].AccessControlList.Num <= MAX_NUM_OF_2991410       0x2DA532        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_asic.c:%d assert BssIndex < 4failed
2991506       0x2DA592        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_asic.c:%d assert KeyIdx < 4failed
2993770       0x2DAE6A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data_pci.c:%d assert QueIdx < NUM_OF_TX_RINGfailed2993882       0x2DAEDA        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_data_pci.c:%d assert pAd->ate.QID == 0failed2995738       0x2DB61A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap.c:%d assert Apidx < MAX_MBSSID_NUM(pAd)failed
2996462       0x2DB8EE        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_assoc.c:%d assert Aid == Wcidfailed
2998086       0x2DBF46        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_assoc.c:%d assert pHTCapabilityfailed
2998466       0x2DC0C2        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_auth.c:%d assert Seq == 1failed
2998554       0x2DC11A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_auth.c:%d assert pEntry == NULLfailed2998854       0x2DC246        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_auth.c:%d assert pEntry->Aid == Elem->Wcidfailed3000378       0x2DC83A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_sync.c:%d assert regclassfailed3000466       0x2DC892        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_wpa.c:%d assert pEntry->apidx < pAd->ApCfg.BssidNumfailed3000582       0x2DC906        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_wpa.c:%d assert pEntryfailed3002574       0x2DD0CE        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_data.c:%d assert pTxBlk->MpduHeaderLen >= 24failed3002682       0x2DD13A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_data.c:%d assert pTxBlkfailed3002810       0x2DD1BA        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_data.c:%d assert (pTxBlk->TxPacketList.Number > 1)failed3002922       0x2DD22A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_data.c:%d assert TX_BLK_TEST_FLAG(pTxBlk, fTX_bAllowFrag)failed3003042       0x2DD2A2        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_data.c:%d assert (pTxBlk->TxPacketList.Number== 2)failed3003754       0x2DD56A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_data.c:%d assert pEntry->Aid == pRxWI->WirelessCliIDfailed3010414       0x2DEF6E        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_cfg.c:%d assert pacl->Num < MAX_NUM_OF_ACL_LISTfailed3010566       0x2DF006        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_cfg.c:%d assert pAd->ApCfg.MBSSID[pObj->ioctl_if].AccessControlList.Num == 0failed3010706       0x2DF092        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_cfg.c:%d assert acl.Num >= pAd->ApCfg.MBSSID[pObj->ioctl_if].AccessControlList.Numfa3011270       0x2DF2C6        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_cfg.c:%d assert ((bClearAll == 1) && (pacl->Num > 0))failed3011386       0x2DF33A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_cfg.c:%d assert pacl->Num == 0failed3017050       0x2E095A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert memfailed
3017182       0x2E09DE        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert DataLenfailed
3017278       0x2E0A3E        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert pDatafailed
3017522       0x2E0B32        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert pPacketfailed
3017618       0x2E0B92        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert DataSize < 1530failed
3017722       0x2E0BFA        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert pHeader802_3failed
3017926       0x2E0CC6        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert pNetDevfailed
3018214       0x2E0DE6        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert (prefixLen < IFNAMSIZ)failed
3018322       0x2E0E52        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert ((slotNameLen + prefixLen) < IFNAMSIZ)failed
3018970       0x2E10DA        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_linux.c:%d assert pTaskfailed
3019052       0x2E112C        Unix path: /etc/Wireless/RT2860AP/RT2860AP.dat
3019230       0x2E11DE        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_profile.c:%d assert pPacketfailed
3019338       0x2E124A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_profile.c:%d assert dev_pfailed
3019710       0x2E13BE        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/os/linux/rt_main_dev.c:%d assert pAdfailed
3020666       0x2E177A        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/rt_ate.c:%d assert (BbpValue == 0x04)failed
3020798       0x2E17FE        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/rt_ate.c:%d assert bbp_data == valuefailed
3022478       0x2E1E8E        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/rt_ate.c:%d assert RestoreRfICType != 0failed
3029806       0x2E3B2E        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_apcli.c:%d assert pAdfailed
3029890       0x2E3B82        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/ap/ap_apcli_inf.c:%d assert pAdfailed
3033494       0x2E4996        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_mat.c:%d assert pHandlefailed
3034054       0x2E4BC6        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_mat_iparp.c:%d assert pMacAddrfailed
3035254       0x2E5076        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/cmm_mat_ipv6.c:%d assert pIPv6Addrfailed3035618       0x2E51E2        Unix path: /net/wireless/rt2860v2_ap/../rt2860v2/common/rt_rf.c:%d assert (regID <= pAd->chipCap.MaxNumOfRfId)failed3082311       0x2F0847        Neighborly text, "neighbor %.2x%.2x.%.2x:%.2x:%.2x:%.2x:%.2x:%.2x lost on port %d(%s)(%s)"
3204032       0x30E3C0        CRC32 polynomial table, little endian
3411968       0x341000        LZMA compressed data, properties: 0x5D, dictionary size: 1048576 bytes, uncompressed size: 2059264 bytes


Scan Time:     2019-02-01 22:10:37
Target File:   /Users/me/endoscope/_merged.eeprom.extracted/_50040.extracted/341000
MD5 Checksum:  45a0880b002ebd3ceafe21ed076e0e14
Signatures:    344

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             ASCII cpio archive (SVR4 with no CRC), file name: "/bin", file name length: "0x00000005", file size: "0x00000000"
116           0x74            ASCII cpio archive (SVR4 with no CRC), file name: "/bin/rm", file name length: "0x00000008", file size: "0x00000008"
244           0xF4            ASCII cpio archive (SVR4 with no CRC), file name: "/bin/iwpriv", file name length: "0x0000000C", file size: "0x00008D78"
368           0x170           ELF, 32-bit LSB MIPS-II executable, MIPS, version 1 (SYSV)
36584         0x8EE8          ASCII cpio archive (SVR4 with no CRC), file name: "/bin/busybox", file name length: "0x0000000D", file size: "0x00066F68"
36708         0x8F64          ELF, 32-bit LSB MIPS-II executable, MIPS, version 1 (SYSV)
397551        0x610EF         Copyright string: "Copyright (C) 1998-2008 Erik Andersen, Rob Landley, Denys Vlasenko"
406193        0x632B1         Unix path: /proc/net/vlan/config
458444        0x6FECC         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/ralink_init", file name length: "0x00000011", file size: "0x00008D04"
458572        0x6FF4C         ELF, 32-bit LSB MIPS-II executable, MIPS, version 1 (SYSV)
484684        0x7654C         Unix path: /etc/Wireless/RT2860/RT2860.dat
494672        0x78C50         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/ping", file name length: "0x0000000A", file size: "0x00000008"
494800        0x78CD0         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/hostname", file name length: "0x0000000E", file size: "0x00000008"
494932        0x78D54         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/nvram_daemon", file name length: "0x00000012", file size: "0x00001A38"
495060        0x78DD4         ELF, 32-bit LSB MIPS-II executable, MIPS, version 1 (SYSV)
499888        0x7A0B0         Unix path: /etc_ro/Wireless/RT61AP/RT2561_default
501772        0x7A80C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/sync", file name length: "0x0000000A", file size: "0x00000008"
501900        0x7A88C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/iwlist", file name length: "0x0000000C", file size: "0x0000A540"
502024        0x7A908         ELF, 32-bit LSB MIPS-II executable, MIPS, version 1 (SYSV)
544328        0x84E48         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/vi", file name length: "0x00000008", file size: "0x00000008"
544456        0x84EC8         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/date", file name length: "0x0000000A", file size: "0x00000008"
544584        0x84F48         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/nvram_get", file name length: "0x0000000F", file size: "0x0000000C"
544724        0x84FD4         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/touch", file name length: "0x0000000B", file size: "0x00000008"
544856        0x85058         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/mknod", file name length: "0x0000000B", file size: "0x00000008"
544988        0x850DC         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/chmod", file name length: "0x0000000B", file size: "0x00000008"
545120        0x85160         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/app_cam", file name length: "0x0000000D", file size: "0x0001C4B0"
661132        0xA168C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/mv", file name length: "0x00000008", file size: "0x00000008"
661260        0xA170C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/ping6", file name length: "0x0000000B", file size: "0x00000008"
661392        0xA1790         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/lsusb", file name length: "0x0000000B", file size: "0x0000CEE0"
714476        0xAE6EC         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/dmesg", file name length: "0x0000000B", file size: "0x00000008"
714608        0xAE770         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/mkdir", file name length: "0x0000000B", file size: "0x00000008"
714740        0xAE7F4         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/ash", file name length: "0x00000009", file size: "0x00000008"
714868        0xAE874         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/ps", file name length: "0x00000008", file size: "0x00000008"
714996        0xAE8F4         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/mtd_write", file name length: "0x0000000F", file size: "0x000031C8"
727868        0xB1B3C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/sed", file name length: "0x00000009", file size: "0x00000008"
727996        0xB1BBC         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/ls", file name length: "0x00000008", file size: "0x00000008"
728124        0xB1C3C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/mount", file name length: "0x0000000B", file size: "0x00000008"
728256        0xB1CC0         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/nvram_set", file name length: "0x0000000F", file size: "0x0000000C"
728396        0xB1D4C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/ated", file name length: "0x0000000A", file size: "0x000026C0"
738436        0xB4484         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/switch", file name length: "0x0000000C", file size: "0x00003F6C"
754796        0xB846C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/flash", file name length: "0x0000000B", file size: "0x000037A8"
769168        0xBBC90         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/cp", file name length: "0x00000008", file size: "0x00000008"
769296        0xBBD10         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/pwd", file name length: "0x00000009", file size: "0x00000008"
769424        0xBBD90         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/mii_mgr", file name length: "0x0000000D", file size: "0x00001490"
774812        0xBD29C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/kill", file name length: "0x0000000A", file size: "0x00000008"
774940        0xBD31C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/umount", file name length: "0x0000000C", file size: "0x00000008"
775072        0xBD3A0         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/grep", file name length: "0x0000000A", file size: "0x00000008"
775200        0xBD420         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/sleep", file name length: "0x0000000B", file size: "0x00000008"
775332        0xBD4A4         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/cat", file name length: "0x00000009", file size: "0x00000008"
775460        0xBD524         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/echo", file name length: "0x0000000A", file size: "0x00000008"
775588        0xBD5A4         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/sh", file name length: "0x00000008", file size: "0x00000008"
775716        0xBD624         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/reg", file name length: "0x00000009", file size: "0x00001AD0"
782700        0xBF16C         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/login", file name length: "0x0000000B", file size: "0x00000008"
782832        0xBF1F0         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/iwconfig", file name length: "0x0000000E", file size: "0x0000A3D8"
824900        0xC9644         ASCII cpio archive (SVR4 with no CRC), file name: "/bin/app_detect", file name length: "0x00000010", file size: "0x0000269C"
834912        0xCBD60         ASCII cpio archive (SVR4 with no CRC), file name: "/dev", file name length: "0x00000005", file size: "0x00000000"
835028        0xCBDD4         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/spiS0", file name length: "0x0000000B", file size: "0x00000000"
835152        0xCBE50         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ac0", file name length: "0x00000009", file size: "0x00000000"
835272        0xCBEC8         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtdblock2", file name length: "0x0000000F", file size: "0x00000000"
835400        0xCBF48         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtdblock7", file name length: "0x0000000F", file size: "0x00000000"
835528        0xCBFC8         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd6ro", file name length: "0x0000000C", file size: "0x00000000"
835652        0xCC044         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd3ro", file name length: "0x0000000C", file size: "0x00000000"
835776        0xCC0C0         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/kmem", file name length: "0x0000000A", file size: "0x00000000"
835896        0xCC138         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd0", file name length: "0x0000000A", file size: "0x00000000"
836016        0xCC1B0         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd1ro", file name length: "0x0000000C", file size: "0x00000000"
836140        0xCC22C         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/pts", file name length: "0x00000009", file size: "0x00000000"
836260        0xCC2A4         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/pts/1", file name length: "0x0000000B", file size: "0x00000000"
836384        0xCC320         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/pts/3", file name length: "0x0000000B", file size: "0x00000000"
836508        0xCC39C         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/pts/0", file name length: "0x0000000B", file size: "0x00000000"
836632        0xCC418         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/pts/2", file name length: "0x0000000B", file size: "0x00000000"
836756        0xCC494         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ttyp0", file name length: "0x0000000B", file size: "0x00000000"
836880        0xCC510         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/pcm0", file name length: "0x0000000A", file size: "0x00000000"
837000        0xCC588         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/random", file name length: "0x0000000C", file size: "0x00000000"
837124        0xCC604         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd5ro", file name length: "0x0000000C", file size: "0x00000000"
837248        0xCC680         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mem", file name length: "0x00000009", file size: "0x00000000"
837368        0xCC6F8         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ram1", file name length: "0x0000000A", file size: "0x00000000"
837488        0xCC770         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/urandom", file name length: "0x0000000D", file size: "0x00000000"
837612        0xCC7EC         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd4ro", file name length: "0x0000000C", file size: "0x00000000"
837736        0xCC868         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ttyp1", file name length: "0x0000000B", file size: "0x00000000"
837860        0xCC8E4         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ram3", file name length: "0x0000000A", file size: "0x00000000"
837980        0xCC95C         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/hwnat0", file name length: "0x0000000C", file size: "0x00000000"
838104        0xCC9D8         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd2ro", file name length: "0x0000000C", file size: "0x00000000"
838228        0xCCA54         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtdblock3", file name length: "0x0000000F", file size: "0x00000000"
838356        0xCCAD4         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/i2cM0", file name length: "0x0000000B", file size: "0x00000000"
838480        0xCCB50         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd0ro", file name length: "0x0000000C", file size: "0x00000000"
838604        0xCCBCC         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/swnat0", file name length: "0x0000000C", file size: "0x00000000"
838728        0xCCC48         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/nvram", file name length: "0x0000000B", file size: "0x00000000"
838852        0xCCCC4         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/video0", file name length: "0x0000000C", file size: "0x00000000"
838976        0xCCD40         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd1", file name length: "0x0000000A", file size: "0x00000000"
839096        0xCCDB8         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd4", file name length: "0x0000000A", file size: "0x00000000"
839216        0xCCE30         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ptyp0", file name length: "0x0000000B", file size: "0x00000000"
839340        0xCCEAC         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd2", file name length: "0x0000000A", file size: "0x00000000"
839460        0xCCF24         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd5", file name length: "0x0000000A", file size: "0x00000000"
839580        0xCCF9C         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/cls0", file name length: "0x0000000A", file size: "0x00000000"
839700        0xCD014         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ttyS1", file name length: "0x0000000B", file size: "0x00000000"
839824        0xCD090         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ppp", file name length: "0x00000009", file size: "0x00000000"
839944        0xCD108         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtdblock1", file name length: "0x0000000F", file size: "0x00000000"
840072        0xCD188         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/flash0", file name length: "0x0000000C", file size: "0x00000000"
840196        0xCD204         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/null", file name length: "0x0000000A", file size: "0x00000000"
840316        0xCD27C         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd6", file name length: "0x0000000A", file size: "0x00000000"
840436        0xCD2F4         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtdblock6", file name length: "0x0000000F", file size: "0x00000000"
840564        0xCD374         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ptyp1", file name length: "0x0000000B", file size: "0x00000000"
840688        0xCD3F0         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtdblock5", file name length: "0x0000000F", file size: "0x00000000"
840816        0xCD470         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtr0", file name length: "0x0000000A", file size: "0x00000000"
840936        0xCD4E8         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd3", file name length: "0x0000000A", file size: "0x00000000"
841056        0xCD560         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/acl0", file name length: "0x0000000A", file size: "0x00000000"
841176        0xCD5D8         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd7", file name length: "0x0000000A", file size: "0x00000000"
841296        0xCD650         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ttyS0", file name length: "0x0000000B", file size: "0x00000000"
841420        0xCD6CC         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/i2s0", file name length: "0x0000000A", file size: "0x00000000"
841540        0xCD744         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ram", file name length: "0x00000009", file size: "0x00000000"
841660        0xCD7BC         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtd7ro", file name length: "0x0000000C", file size: "0x00000000"
841784        0xCD838         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ram0", file name length: "0x0000000A", file size: "0x00000000"
841904        0xCD8B0         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/rdm0", file name length: "0x0000000A", file size: "0x00000000"
842024        0xCD928         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ram2", file name length: "0x0000000A", file size: "0x00000000"
842144        0xCD9A0         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/ptmx", file name length: "0x0000000A", file size: "0x00000000"
842264        0xCDA18         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtdblock0", file name length: "0x0000000F", file size: "0x00000000"
842392        0xCDA98         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/mtdblock4", file name length: "0x0000000F", file size: "0x00000000"
842520        0xCDB18         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/watchdog", file name length: "0x0000000E", file size: "0x00000000"
842644        0xCDB94         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/gpio", file name length: "0x0000000A", file size: "0x00000000"
842764        0xCDC0C         ASCII cpio archive (SVR4 with no CRC), file name: "/dev/console", file name length: "0x0000000D", file size: "0x00000000"
842888        0xCDC88         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin", file name length: "0x00000006", file size: "0x00000000"
843004        0xCDCFC         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/config.sh", file name length: "0x00000010", file size: "0x00001943"
849600        0xCF6C0         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/mdev", file name length: "0x0000000B", file size: "0x0000000F"
849740        0xCF74C         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/rmmod", file name length: "0x0000000C", file size: "0x0000000F"
849880        0xCF7D8         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/wan.sh", file name length: "0x0000000D", file size: "0x00000459"
851120        0xCFCB0         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/start.sh", file name length: "0x0000000F", file size: "0x00000804"
853300        0xD0534         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/ifconfig", file name length: "0x0000000F", file size: "0x0000000F"
853444        0xD05C4         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/video_ko.sh", file name length: "0x00000012", file size: "0x00000412"
854616        0xD0A58         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/dhcp6s", file name length: "0x0000000D", file size: "0x000213C0"
990868        0xF1E94         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/lsmod", file name length: "0x0000000C", file size: "0x0000000F"
991008        0xF1F20         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/config_save.sh", file name length: "0x00000015", file size: "0x000004BB"
992352        0xF2460         ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/dhcp6c", file name length: "0x0000000D", file size: "0x00032418"
1198324       0x1248F4        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/wifi_ap.sh", file name length: "0x00000011", file size: "0x000009E1"
1200984       0x125358        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/vconfig", file name length: "0x0000000E", file size: "0x0000000F"
1201124       0x1253E4        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/syslogd", file name length: "0x0000000E", file size: "0x0000000F"
1201264       0x125470        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/insmod", file name length: "0x0000000D", file size: "0x0000000F"
1201404       0x1254FC        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/runapp.sh", file name length: "0x00000010", file size: "0x000000EE"
1201772       0x12566C        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/reboot", file name length: "0x0000000D", file size: "0x0000000F"
1201912       0x1256F8        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/fdisk", file name length: "0x0000000C", file size: "0x0000000F"
1202052       0x125784        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/init", file name length: "0x0000000B", file size: "0x0000000F"
1202192       0x125810        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/klogd", file name length: "0x0000000C", file size: "0x0000000F"
1202332       0x12589C        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/logread", file name length: "0x0000000E", file size: "0x0000000F"
1202472       0x125928        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/halt", file name length: "0x0000000B", file size: "0x0000000F"
1202612       0x1259B4        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/wifi_nuotai.sh", file name length: "0x00000015", file size: "0x00000A08"
1205312       0x126440        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/poweroff", file name length: "0x0000000F", file size: "0x0000000F"
1205456       0x1264D0        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/chpasswd.sh", file name length: "0x00000012", file size: "0x00000169"
1205948       0x1266BC        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/route", file name length: "0x0000000C", file size: "0x0000000F"
1206088       0x126748        ASCII cpio archive (SVR4 with no CRC), file name: "/sbin/udhcpc", file name length: "0x0000000D", file size: "0x0000000F"
1206228       0x1267D4        ASCII cpio archive (SVR4 with no CRC), file name: "/media", file name length: "0x00000007", file size: "0x00000000"
1206348       0x12684C        ASCII cpio archive (SVR4 with no CRC), file name: "/home", file name length: "0x00000006", file size: "0x00000000"
1206464       0x1268C0        ASCII cpio archive (SVR4 with no CRC), file name: "/proc", file name length: "0x00000006", file size: "0x00000000"
1206580       0x126934        ASCII cpio archive (SVR4 with no CRC), file name: "/usr", file name length: "0x00000005", file size: "0x00000000"
1206696       0x1269A8        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin", file name length: "0x00000009", file size: "0x00000000"
1206816       0x126A20        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/expr", file name length: "0x0000000E", file size: "0x00000012"
1206960       0x126AB0        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/find", file name length: "0x0000000E", file size: "0x00000012"
1207104       0x126B40        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/tftp", file name length: "0x0000000E", file size: "0x00000012"
1207248       0x126BD0        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/printf", file name length: "0x00000010", file size: "0x00000012"
1207396       0x126C64        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/time", file name length: "0x0000000E", file size: "0x00000012"
1207540       0x126CF4        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/[", file name length: "0x0000000B", file size: "0x00000012"
1207684       0x126D84        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/logger", file name length: "0x00000010", file size: "0x00000012"
1207832       0x126E18        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/test", file name length: "0x0000000E", file size: "0x00000012"
1207976       0x126EA8        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/basename", file name length: "0x00000012", file size: "0x00000012"
1208124       0x126F3C        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/tr", file name length: "0x0000000C", file size: "0x00000012"
1208268       0x126FCC        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/[[", file name length: "0x0000000C", file size: "0x00000012"
1208412       0x12705C        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/killall", file name length: "0x00000011", file size: "0x00000012"
1208560       0x1270F0        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/uptime", file name length: "0x00000010", file size: "0x00000012"
1208708       0x127184        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/wc", file name length: "0x0000000C", file size: "0x00000012"
1208852       0x127214        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/free", file name length: "0x0000000E", file size: "0x00000012"
1208996       0x1272A4        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/bin/top", file name length: "0x0000000D", file size: "0x00000012"
1209140       0x127334        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/sbin", file name length: "0x0000000A", file size: "0x00000000"
1209260       0x1273AC        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/sbin/brctl", file name length: "0x00000010", file size: "0x00000012"
1209408       0x127440        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/sbin/udhcpd", file name length: "0x00000011", file size: "0x00000012"
1209556       0x1274D4        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/sbin/chpasswd", file name length: "0x00000013", file size: "0x00000012"
1209708       0x12756C        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/sbin/telnetd", file name length: "0x00000012", file size: "0x00000012"
1209856       0x127600        ASCII cpio archive (SVR4 with no CRC), file name: "/usr/codepages", file name length: "0x0000000F", file size: "0x00000000"
1209984       0x127680        ASCII cpio archive (SVR4 with no CRC), file name: "/sys", file name length: "0x00000005", file size: "0x00000000"
1210100       0x1276F4        ASCII cpio archive (SVR4 with no CRC), file name: "/etc", file name length: "0x00000005", file size: "0x00000000"
1210216       0x127768        ASCII cpio archive (SVR4 with no CRC), file name: "/etc/motd", file name length: "0x0000000A", file size: "0x00000011"
1210356       0x1277F4        ASCII cpio archive (SVR4 with no CRC), file name: "/etc/fstab", file name length: "0x0000000B", file size: "0x000001A8"
1210904       0x127A18        ASCII cpio archive (SVR4 with no CRC), file name: "/var", file name length: "0x00000005", file size: "0x00000000"
1211020       0x127A8C        ASCII cpio archive (SVR4 with no CRC), file name: "/mnt", file name length: "0x00000005", file size: "0x00000000"
1211136       0x127B00        ASCII cpio archive (SVR4 with no CRC), file name: "/init", file name length: "0x00000006", file size: "0x0000000C"
1211264       0x127B80        ASCII cpio archive (SVR4 with no CRC), file name: "/lib", file name length: "0x00000005", file size: "0x00000000"
1211380       0x127BF4        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libnvram.so.0", file name length: "0x00000013", file size: "0x00000013"
1211532       0x127C8C        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libpthread.so.0", file name length: "0x00000015", file size: "0x00000015"
1211688       0x127D28        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libm.so", file name length: "0x0000000D", file size: "0x0000000F"
1211828       0x127DB4        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libuClibc-0.9.28.so", file name length: "0x00000019", file size: "0x0005F634"
1602672       0x187470        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libcrypt.so", file name length: "0x00000011", file size: "0x00000013"
1602820       0x187504        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules", file name length: "0x0000000D", file size: "0x00000000"
1602944       0x187580        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21", file name length: "0x00000014", file size: "0x00000000"
1603076       0x187604        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel", file name length: "0x0000001B", file size: "0x00000000"
1603216       0x187690        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers", file name length: "0x00000023", file size: "0x00000000"
1603364       0x187724        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers/media", file name length: "0x00000029", file size: "0x00000000"
1603516       0x1877BC        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers/media/video", file name length: "0x0000002F", file size: "0x00000000"
1603676       0x18785C        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers/media/video/videodev.ko", file name length: "0x0000003B", file size: "0x00009718"
1642528       0x191020        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers/media/video/uvc", file name length: "0x00000033", file size: "0x00000000"
1642692       0x1910C4        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers/media/video/uvc/uvcvideo.ko", file name length: "0x0000003F", file size: "0x000151E4"
1729368       0x1A6358        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers/media/video/compat_ioctl32.ko", file name length: "0x00000041", file size: "0x00000660"
1731176       0x1A6A68        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers/media/video/v4l1-compat.ko", file name length: "0x0000003E", file size: "0x00004ADC"
1750512       0x1AB5F0        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/modules/2.6.21/kernel/drivers/media/video/v4l2-common.ko", file name length: "0x0000003E", file size: "0x000054D0"
1772396       0x1B0B6C        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libcrypt.so.0", file name length: "0x00000013", file size: "0x00000013"
1772548       0x1B0C04        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libm-0.9.28.so", file name length: "0x00000014", file size: "0x00007848"
1803472       0x1B84D0        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libc.so.0", file name length: "0x0000000F", file size: "0x00000014"
1803620       0x1B8564        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libc.so", file name length: "0x0000000D", file size: "0x00000014"
1803764       0x1B85F4        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libpthread-0.9.28.so", file name length: "0x0000001A", file size: "0x00017BBC"
1901112       0x1D0238        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libutil.so.0", file name length: "0x00000012", file size: "0x00000012"
1901260       0x1D02CC        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libpthread.so", file name length: "0x00000013", file size: "0x00000015"
1901416       0x1D0368        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/ld-uClibc.so.0", file name length: "0x00000014", file size: "0x00000014"
1901568       0x1D0400        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libusb.so.1.0.0", file name length: "0x00000015", file size: "0x0000D248"
1955532       0x1DD6CC        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libdl-0.9.28.so", file name length: "0x00000015", file size: "0x000024A0"
1965040       0x1DFBF0        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libnvram-0.9.28.so", file name length: "0x00000018", file size: "0x00008524"
1999260       0x1E819C        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libcrypt-0.9.28.so", file name length: "0x00000018", file size: "0x000034D8"
2012924       0x1EB6FC        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libdl.so", file name length: "0x0000000E", file size: "0x00000010"
2013064       0x1EB788        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libutil-0.9.28.so", file name length: "0x00000017", file size: "0x000013A8"
2018232       0x1ECBB8        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libdl.so.0", file name length: "0x00000010", file size: "0x00000010"
2018376       0x1ECC48        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libutil.so", file name length: "0x00000010", file size: "0x00000012"
2018524       0x1ECCDC        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/ipsec", file name length: "0x0000000B", file size: "0x00000000"
2018648       0x1ECD58        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libm.so.0", file name length: "0x0000000F", file size: "0x0000000F"
2018792       0x1ECDE8        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/ld-uClibc-0.9.28.so", file name length: "0x00000019", file size: "0x00006790"
2045440       0x1F3600        ASCII cpio archive (SVR4 with no CRC), file name: "/lib/libnvram.so", file name length: "0x00000011", file size: "0x00000013"
2045588       0x1F3694        ASCII cpio archive (SVR4 with no CRC), file name: "/tmp", file name length: "0x00000005", file size: "0x00000000"
2045704       0x1F3708        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro", file name length: "0x00000008", file size: "0x00000000"
2045824       0x1F3780        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/web", file name length: "0x0000000C", file size: "0x00000000"
2045948       0x1F37FC        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/Wireless", file name length: "0x00000011", file size: "0x00000000"
2046076       0x1F387C        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/Wireless/RT2860AP", file name length: "0x0000001A", file size: "0x00000000"
2046212       0x1F3904        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/Wireless/RT2860AP/RT2860_default_novlan", file name length: "0x00000030", file size: "0x00000C07"
2049452       0x1F45AC        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/Wireless/RT2860AP/RT2860_default_vlan", file name length: "0x0000002E", file size: "0x00000B25"
2052464       0x1F5170        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/dhcp6c.conf", file name length: "0x00000014", file size: "0x00000070"
2052708       0x1F5264        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/wlan", file name length: "0x0000000D", file size: "0x00000000"
2052832       0x1F52E0        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/wlan/LEWEIAP_Password.dat", file name length: "0x00000022", file size: "0x00000E1F"
2056592       0x1F6190        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/wlan/RT5350_AP_1T1R_V1_0.bin", file name length: "0x00000025", file size: "0x00000200"
2057252       0x1F6424        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/dhcp6s.conf", file name length: "0x00000014", file size: "0x0000014A"
2057716       0x1F65F4        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/inittab", file name length: "0x00000010", file size: "0x0000002D"
2057892       0x1F66A4        ASCII cpio archive (SVR4 with no CRC), file name: "/etc_ro/rcS", file name length: "0x0000000C", file size: "0x000002C6"
2058728       0x1F69E8        ASCII cpio archive (SVR4 with no CRC), file name: "TRAILER!!!", file name length: "0x0000000B", file size: "0x00000000"

Once the firmware was extracted successfully, it didn’t take much time to find the telnet password, using grep -r -i -a for login= and pass=. The winning combination was username: tony, password: tony4321, which is an administrative user.

With this out of the way, my last objective was to figure out a replacement for the iPhone app; if the app ever stopped working, I wanted to still be able to view the video stream and change the settings. Check out part 3 for how I used WireShark to figure this out.

↧

Reverse Engineering My WiFi Endoscope Part 3

February 10, 2019, 12:40 pm

≫ Next: Reverse Engineering My WiFi Endoscope Part 4

≪ Previous: Reverse Engineering My WiFi Endoscope Part 2

Bottom Line: I got root on a WiFi Endoscope.

Part 3: Using WireShark to Decode UDP Protocol

Part 2 review: I used a bus pirate to grab the firmware from my WiFi Endoscope, which revealed the telnet username and password.

After getting admin access over telnet, I still wanted to be able to view the video stream and configure the endoscope’s settings without using the iPhone app, to make sure I could still use the endoscope if the app ever stopped working.

As I had noted previously, nmap showed that port 7060 was open, and connecting with (GNU) netcat showed a seemingly endless stream of data, suggesting to me that this was the video feed.

I really didn’t understand how the iPhone app was communicating information back to the endoscope (maybe via telnet?), so I decided to see if I could use this super convenient little router as a MITM, since it runs OpenWRT and rebroadcasts networks as one of its core features. It ended up being surprisingly simple:

connect the router to a network with internet access
install tcpdump using OpenWRT’s package manager
configure SSH access to the router
configure the router to rebroadcast the endoscope’s network
connect to the router via ssh and run tcpdump
connect my phone to the endoscope by way of the router
run some commands on my phone through the endoscope’s app
scp the tcpdump file and open in WireShark for analysis

Unfortunately, it’s been a while since I did this and I no longer recall the exact commands. However, I’ve since discovered that I can do pretty much the same thing using WireShark directly, so that’s how I’ll present the remainder of the information.

A few key steps:

use the app to remove the endoscope’s WiFi network password, making it much easier to capture data with WireShark
enable WireShark’s promiscuous and monitor mode for your network interface
use capture filters to greatly reduce the amount of data, e.g. host 192.168.10.123

Once I’d started digging through the captured files, I noted a few things:

I didn’t see any evidence of communication on port 23, so the app was probably not working via telnet
Lots of junk on 7060, as suspected
Lots of UDP data on port 50000

I was surprised to find the UDP 50000 data, since that hadn’t come up in my nmap scans. It ends up that nmap apparently only scans the top 1000 most common ports; when I scanned it directly with nmap -sU 192.168.10.123 -p 50000, sure enough it picked it up.

The UDP data looked promising as a way to control the endoscope, so I set a WireShark display filter to ip.addr == 192.168.10.123 and udp. I found lots of stretches of data whose hexdump (viewed in WireShark with Follow -> UDP Stream) looked like:

00000000  53 45 54 43 4d 44 ee 14  00 00 90 00 01 00 00      SETCMD.. .......
0000000F  53 45 54 43 4d 44 ef 14  00 00 90 00 01 00 00      SETCMD.. .......
0000001E  53 45 54 43 4d 44 f0 14  00 00 90 00 01 00 00      SETCMD.. .......
0000002D  53 45 54 43 4d 44 f1 14  00 00 90 00 01 00 00      SETCMD.. .......
0000003C  53 45 54 43 4d 44 f2 14  00 00 90 00 01 00 00      SETCMD.. .......
0000004B  53 45 54 43 4d 44 f3 14  00 00 90 00 01 00 00      SETCMD.. .......
0000005A  53 45 54 43 4d 44 f4 14  00 00 90 00 01 00 00      SETCMD.. .......
00000069  53 45 54 43 4d 44 f5 14  00 00 90 00 01 00 00      SETCMD.. .......
00000078  53 45 54 43 4d 44 f6 14  00 00 90 00 01 00 00      SETCMD.. .......
00000087  53 45 54 43 4d 44 f7 14  00 00 90 00 01 00 00      SETCMD.. .......
00000096  53 45 54 43 4d 44 f8 14  00 00 90 00 01 00 00      SETCMD.. .......
000000A5  53 45 54 43 4d 44 f9 14  00 00 90 00 01 00 00      SETCMD.. .......
000000B4  53 45 54 43 4d 44 fa 14  00 00 90 00 01 00 00      SETCMD.. .......

As you can see, there’s a byte in the 7th position that just keeps incrementing, then there’s this SETCMD business. Running grep -r -a SETCMD on the firmware that binwalk extracted in part 2 shows, among other things:

_50040.extracted/341000:SETCMD F:src/app_cmd.c, L:%d    : Receive CMD_SET_WIFI_NAME, Name:%s

Grepping for more of the CMD_ names gives us an idea of what kinds of commands are likely available:

$grep-r-a-o-h"CMD_\w\+" | sort -uCMD_AP_START
CMD_ATE_GET_STATISTICS
CMD_ATE_RESET_COUNTER
CMD_ATE_RUN_CPUBUSY
CMD_ATE_SEL_RX_ANTENNA
CMD_ATE_SEL_TX_ANTENNA
CMD_ATE_SET_ADDR1
CMD_ATE_SET_ADDR2
CMD_ATE_SET_ADDR3
CMD_ATE_SET_BW
CMD_ATE_SET_CHANNEL
CMD_ATE_SET_FREQ_OFFSET
CMD_ATE_SET_PREAMBLE
CMD_ATE_SET_RATE
CMD_ATE_SET_TX_FRAME_COUNT
CMD_ATE_SET_TX_FRAME_LEN
CMD_ATE_SET_TX_POWER0
CMD_ATE_SET_TX_POWER1
CMD_ATE_START
CMD_ATE_START_TX_CARRIER
CMD_ATE_START_TX_CONT
CMD_ATE_START_TX_FRAME
CMD_ATE_STOP
CMD_GET_CAMERA_RESOLUTION
CMD_IO_WRITE
CMD_Proc
CMD_RX_START
CMD_RX_STOP
CMD_SET_CAMERA_RESOLUTION
CMD_SET_REBOOT
CMD_SET_WIFI_CLEAR_PASSWORD
CMD_SET_WIFI_NAME
CMD_SET_WIFI_PASSWORD
CMD_TX_STOP

Going back to the WireShark UDP commands, I set a more specific capture filter (host 192.168.10.123 and udp and port 50000) and made several more captures of different commands. If I make a capture while using the iPhone app to reboot the endoscope, I find:

00000000  53 45 54 43 4d 44 01 00  00 00 04 00 00 00         SETCMD.. ......
0000000E  53 45 54 43 4d 44 01 00  00 00 04 00 00 00         SETCMD.. ......
0000001C  53 45 54 43 4d 44 01 00  00 00 04 00 00 00         SETCMD.. ......
0000002A  52 65 73 74 61 72 74 53  65 72 76 65 72 5f         RestartS erver_
    00000000  52 45 54 43 4d 44 01 00  00 00 04 00               RETCMD.. ....

Clearing the password (twice):

00000000  53 45 54 43 4d 44 01 00  00 00 03 00 00 00         SETCMD.. ......
0000000E  53 45 54 43 4d 44 01 00  00 00 03 00 00 00         SETCMD.. ......
0000001C  53 45 54 43 4d 44 01 00  00 00 03 00 00 00         SETCMD.. ......
0000002A  43 6c 65 61 72 50 61 73  73 5f 5f 5f 5f 5f         ClearPas s_____
    00000000  52 45 54 43 4d 44 01 00  00 00 03 00               RETCMD.. ....
00000038  53 45 54 43 4d 44 02 00  00 00 03 00 00 00         SETCMD.. ......
00000046  53 45 54 43 4d 44 02 00  00 00 03 00 00 00         SETCMD.. ......
00000054  53 45 54 43 4d 44 02 00  00 00 03 00 00 00         SETCMD.. ......
00000062  43 6c 65 61 72 50 61 73  73 5f 5f 5f 5f 5f         ClearPas s_____
    0000000C  52 45 54 43 4d 44 02 00  00 00 03 00               RETCMD.. ....

Setting the WiFi network password to 12345678, manually issuing the command 3 times:

00000000  53 45 54 43 4d 44 07 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
00000010  33 34 35 36 37 38                                  345678
00000016  53 45 54 43 4d 44 07 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
00000026  33 34 35 36 37 38                                  345678
0000002C  53 45 54 43 4d 44 07 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
0000003C  33 34 35 36 37 38                                  345678
00000042  41 50 50 41 53 53 3d 31  32 33 34 35 36 37 38      APPASS=1 2345678
00000051  53 45 54 43 4d 44 08 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
00000061  33 34 35 36 37 38                                  345678
00000067  53 45 54 43 4d 44 08 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
00000077  33 34 35 36 37 38                                  345678
0000007D  53 45 54 43 4d 44 08 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
0000008D  33 34 35 36 37 38                                  345678
00000093  41 50 50 41 53 53 3d 31  32 33 34 35 36 37 38      APPASS=1 2345678
000000A2  53 45 54 43 4d 44 09 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
000000B2  33 34 35 36 37 38                                  345678
000000B8  53 45 54 43 4d 44 09 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
000000C8  33 34 35 36 37 38                                  345678
000000CE  53 45 54 43 4d 44 09 00  00 00 02 00 08 00 31 32   SETCMD.. ......12
000000DE  33 34 35 36 37 38                                  345678
000000E4  41 50 50 41 53 53 3d 31  32 33 34 35 36 37 38      APPASS=1 2345678

A few patterns:

7th byte seems to be incrementing sequentially (07 for the first 3SETCMDs, then 08, then 09)
the SETCMD ends with the new network password and repeats 3 times, followed by…
a line with APPASS= and the password
followed by a RETCMD response coming from the endoscope (I have filtered this out in some of the above examples)

Setting the WiFi network name to Jetion_fd20f768:

00000000  53 45 54 43 4d 44 0d 00  00 00 01 00 0f 00 4a 65   SETCMD.. ......Je
00000010  74 69 6f 6e 5f 66 64 32  30 66 37 36 38            tion_fd2 0f768
0000001D  53 45 54 43 4d 44 0d 00  00 00 01 00 0f 00 4a 65   SETCMD.. ......Je
0000002D  74 69 6f 6e 5f 66 64 32  30 66 37 36 38            tion_fd2 0f768
0000003A  53 45 54 43 4d 44 0d 00  00 00 01 00 0f 00 4a 65   SETCMD.. ......Je
0000004A  74 69 6f 6e 5f 66 64 32  30 66 37 36 38            tion_fd2 0f768
00000057  41 50 4e 41 4d 45 3d 4a  65 74 69 6f 6e 5f 66 64   APNAME=J etion_fd
00000067  32 30 66 37 36 38                                  20f768
    00000000  52 45 54 43 4d 44 0d 00  00 00 01 00               RETCMD.. ....

Again, we see 3 of the SETCMD lines, which may or may not have additional data (such as the new network password or network name), followed by a line with a more legible command: APNAME= + network name, or APPASS= + password, or RestartServer_, then the RETCMD response from the endoscope, likely acknowledging the receipt of a command.

I initially figured that perhaps the 4th command with APNAME= orRestartServer_ was the key part, so I connected my Macbook to the endoscope’s network and tried sending those bytes back (with some hackery – the tr -s ' ' is to squeeze the extra spaces that WireShark includes, so I can copy and paste directly from WireShark. Please let me know in the comments if you have a better way).

$echo-e"$(tr -s' '<<<'41 50 4e 41 4d 45 3d 4a 65 74 69 6f 6e 5f 66 64 32 30 66 37 36 38' | sed 's/^\| /\\x/g')"APNAME=Jetion_fd20f768$echo-e"$(tr -s' '<<<'41 50 4e 41 4d 45 3d 4a 65 74 69 6f 6e 5f 66 64 32 30 66 37 36 38' | sed 's/^\| /\\x/g')" | netcat --udp--close 192.168.10.123 50000

Unfortunately, this didn’t seem to work. So next, I tried sending several of the SETCMD lines followed by the APNAME= line, figuring that perhaps it needed the full sequence. It still didn’t set the access point name, but interestingly, it did reboot the router after the first SETCMD (note: this likely means that I had copied the wrong SETCMD preceding the APNAME=), but I hadn’t realized at this point).

EDIT: I figured out that one also use xxd -r -p instead of the echo, tr, and sed hackery (note the lack of a trailing newline, as with echo -n, so the output below ends in $). You’ll see me using the xxd and echo / tr / sed versions interchangeably below. Also, for print debugging the characters, it looks like echo -e '\x53' and echo $'\x53' produce the same output.

$echo-ne"$(tr -s' '<<<"53 45 54 43 4d 44 09 00    00 00 02 00 08 00 31 32 33 34 35 36 37 38" | sed 's/^\|\ /\\x/g')"SETCMD 12345678$$ xxd -r-p<<<"53 45 54 43 4d 44 09 00    00 00 02 00 08 00 31 32 33 34 35 36 37 38"SETCMD 12345678$$echo-e'\x54'T$echo$'\x54'T

END EDIT

I also tried sending the RestartServer_ command to reboot the endoscope, and that also didn’t work:

$echo-e"$(tr -s' '<<<'52 65 73 74 61 72 74 53  65 72 76 65 72 5f' | sed 's/^\| /\\x/g')"RestartServer_$echo-e"$(tr -s' '<<<'52 65 73 74 61 72 74 53  65 72 76 65 72 5f' | sed 's/^\| /\\x/g')" | netcat --udp--close 192.168.10.123 50000

It took me a while, but what I eventually figured out was that it is actually the SETCMD that is the key; the following line with e.g. APNAME= orRestartServer_ doesn’t seem to be necessary at all, and the SETCMD doesn’t necessarily have to be repeated three times, either. The key seems to be one of the bytes (or perhaps a few) after the SETCMD; for example the SETCMD for the RestartServer, as opposed to the SETCMD for ClearPass_____ above:

45 54 43 4d 44 01 00  00 00 04 00 00 00         SETCMD.. ......
65 73 74 61 72 74 53  65 72 76 65 72 5f         RestartS erver_
45 54 43 4d 44 01 00  00 00 03 00 00 00         SETCMD.. ......
6c 65 61 72 50 61 73  73 5f 5f 5f 5f 5f         ClearPas s_____

Up to the 44 is the SETCMD:

$ xxd -r-p<<<'53 45 54 43 4d 44'SETCMD$

The next byte (01 above) varies, seems to be counting up sequentially, and doesn’t seem to be critical for the device to correctly receive a command.

The next three bytes seem pretty consistently to be 00.

The following byte (or 4), 04 00 00 00 or 03 00 00 00 above, seem to indicate what kind of command is being sent, and after those come any relevant data (like the access point name or password).

Messing around with this, it also seems like it was useful to send one of the “blank” SETCMDs before a command; without this, I could get a single command to work, but couldn’t get any following commands to work until I manually rebooted the device (e.g. I could set the apname, but not reboot to reflect the new name, whereas when I send a blank SETCMD before each command I can set apname, then send a reboot command).

I put all this together into a little bash script that seems to work fairly well for configuring the basic functions of the endoscope:

#!/bin/bash# ⓒ 2019 Nathan Henrie n8henrie.com# Use at your own risk, setting a network password or apname with illegal or# insufficient characters may brick your device# # Depencendies:#   xxd V1.10 27oct98 by Juergen Weigert#   netcat (The GNU Netcat) 0.7.1 Copyright (C) 2002 - 2003  Giovanni Giacobbiset-euf-o pipefail

send(){
  xxd -r-p<<<"53 45 54 43 4d 44 00 14 00 00 90 00 01 00 00" | netcat --udp--close 192.168.10.123 50000
  xxd -r-p<<<"53 45 54 43 4d 44 00 00 00 00 $1" | netcat --udp--close 192.168.10.123 50000}USAGE=$(cat<<'EOF'
Valid commands:
  reboot
  set_apname APNAME
  clear_password
  set_password PASSWORDEOF)case"$1"inreboot) send '04 00 00 00';;
  set_apname) send "01 00 08 00 $(xxd -p<<<"$2")";;
  clear_password) send '03 00 00 00';;
  set_password) send "02 00 08 00 $(xxd -p<<<"$2")";;*)echo"$USAGE";;esac

I had planned to also cover streaming the video feed in this post, but it’s gotten fairly lengthy already, so I think I’ll extend the series to include a part 4 and wrap up there.

↧

Reverse Engineering My WiFi Endoscope Part 4

February 12, 2019, 7:00 am

≫ Next: Blinking an STM32 with Rust from MacOS

≪ Previous: Reverse Engineering My WiFi Endoscope Part 3

Bottom Line: I got root on a WiFi Endoscope.

Part 4: Using Netcat to Decode Video Protocol

Part 3 review: I used WireShark to figure out the UDP protocol that the WiFi endoscope uses to configure its password and access point name, among other settings.

My primary objective with this project was to ensure that I could continue to use the endoscope even if the iOS app went unmaintained; after figuring out how to configure the endoscope’s settings, the last thing I needed was to be able to view the video feed without the app.

I’ve mentioned a few times that nmap showed port 7060 to be open, and that I suspected this was the video feed based on the fact that connecting to it yielded an unending stream of data.

Inspecting 30 seconds or so of its data, I found a few points of interest:

$ netcat 192.168.10.123 7060 > netcat_out$ strings netcat_out | headBoundaryS
JFIF(:3=<9387@H\N@DWE78PmQW_bghg>Mqypdx\egc/cB8Bcccccccccccccccccccccccccccccccccccccccccccccccccc$3br%&'()*456789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz&'()*56789:CDEFGHIJSTUVWXYZcdefghijstuvwxyz
1L@E
,%!4
Z@-8t

Some Googling of those strings led me to believe it was using a JFIF image format (especially since it includes JFIF in the content). Eventually, I found a super helpful SO answer suggesting that I should look for FFD8 to start and FFD9 to end an image in the feed.

As a test, I tried to snip out a single image:

Open the binary file in neovim: nvim "+:setlocal display=uhex" netcat_out
Delete until ffd8: d/\%xff\%xd8
Find the next ffd9: /\%xff\%xd9
Advance 2 bytes: ll
Delete until the end of the line: d$
Go down a line: j
Delete until end of the file: dG
Write the resulting data: :w netcat_endoscope.jpg
Open the file: !open netcat_endoscope.jpg

The resulting file:

Clearly I was correct, this stream is the image data.

I tried a number of ways of viewing this stream in the browser without success, but I eventually found a Python method leveraging OpenCV that worked great. Below, I’m using OpenCV 4.0.1 installed via Homebrew and python 3.7.2.

importcv2cap=cv2.VideoCapture("http://192.168.10.123:7060")whileTrue:ret,frame=cap.read()cv2.imshow('Video',frame)ifcv2.waitKey(1)==27:exit(0)

The stream does have a strange line through the middle (like the image shown above), but other than that it seems to work well. If you have other suggestions on how to view this stream (especially if it gets rid of this artifact), feel free to let me know in the comments!

At this point, having read the firmware flash, discovered the telnet password for an administrative account, figured out how to configure the endoscope by sending commands over UDP, and figured out how to stream the video feed without requiring the app, I consider this project wrapped up. I hope you’ve enjoyed this writeup of my first adventure in reverse engineering an electronic device!

↧

Blinking an STM32 with Rust from MacOS

April 28, 2019, 3:07 pm

≫ Next: List of Default Packages on Raspbian Buster Lite

≪ Previous: Reverse Engineering My WiFi Endoscope Part 4

Bottom Line: I blinked an LED in Rust. I’ve been increasingly enthusiastic about Mozilla’s new open source programming language Rust over the last year or so. There is lots to read about the reasons that different people like rust, so I’ll spare you the details, but I think it’s exciting that even an amateur like myself can enjoy such a powerful, fast, and safe language.

One of the many promising things about Rust is its potential for safer and more ergonomic low-level programming in the embedded world; I’ve been keeping a casual eye on progress here as a tinkerer that really don’t know much C++ (and so my Arduino and ATMEGA328p adventures tend to be pretty tame).

Recently, a quick glance through a thread on r/rust (which has been one of several examples of Rust’s unusually welcoming community) suggested that the STM32 boards were a good way for a beginner to get started in embedded rust, so I picked one up.

I found a really great “hello world” walkthrough that took me most of the way. Much of it has been copied verbatim below, though a few steps are slightly different, and linked post has much more information about the steps being taken, so I highly recomend you read through it. I’ve also put a basic framework up as a git repo at github.com/n8henrie/rust-stm32; it already has the memory file, .cargo/config, and Cargo.toml as well as the basic main.rs; once dependencies are installed, if you clone it you should be able to cargo build get a binary you can use. The step below essentially walk you through making your own identical repo, (like the author of the linked blog post has also done here).

Requirements:

STM32 bluepill
FTDI for programming it (which seems to work fine instead of an ST-Link)
Working rust setup (including rustup and cargo, I’m on rust 1.34.1)

Step by step:

cargo install cargo-binutils and rustup component add llvm-tools-preview to get the cargo objcopy commands cargo-binutils repo
rustup target add thumbv7m-none-eabi (note this is 7m and not 7em, which may be a typo in the how-to post linked above)
make a new crate: cargo new rust-stm32 && cd rust-stm32
edit Cargo.toml to match this example

make memory.x as instructedhere:

 $cat> memory.x <<EOF /* Linker script for the STM32F103C8T6 */
 MEMORY
 {
 FLASH : ORIGIN = 0x08000000, LENGTH = 64K
 RAM : ORIGIN = 0x20000000, LENGTH = 20K
 }
 EOF

Add to ./.cargo/config:

[build]# Instruction set of Cortex-M3 (used in BluePill)target="thumbv7m-none-eabi"rustflags=[# use the Tlink.x script from the cortex-m-rt crate"-C","link-arg=-Tlink.x",]

make src/main.rs:
- Example: https://github.com/n8henrie/rust-stm32/blob/master/src/main.rs
- My STM was already blinking a few times per second by default, so consider changing the delay by an order of magnitude to make sure your code is being executed
cargo build --release
cargo objcopy -- -O binary target/thumbv7m-none-eabi/release/rust-stm32 rust-stm32.bin
Build stm32loader to flash the code to the STM32 from MacOS:
1. Clone repo: git clone https://github.com/florisla/stm32loader.git
2. cd stm32loader
3. Make venv: python3 -m venv .venv
4. Source venv: source ./.venv/bin/activate
5. Update pip to >= 19.1.1 pip install --upgrade pip
6. Install in editable mode: pip install -e .
Connect the (unplugged) FTDI to the STM32 (as a mnemonic, remember “Tx to Ten”, and also make sure your FTDI is set to 3.3v)
FTDI STM32
TX A10
RX A9
3V 3V
GND GND
Move the jumper for BOOT0 (the one closer to edge of board) from 0 to 1 (3.3v)
Flash the program: stm32loader -p /dev/tty.usb1 -e -w -v /path/to/rust-stm32.bin, where flags are for port, erase, write, and verify
If it looks like it flashed correctly, disconnect it, move BOOT0 back to 0 / off, reconnect power, and see if you have a blinking light!

FTDI	STM32
TX	A10
RX	A9
3V	3V
GND	GND

Links:

↧

List of Default Packages on Raspbian Buster Lite

August 6, 2019, 8:34 am

≫ Next: Federal Debt Over Time

≪ Previous: Blinking an STM32 with Rust from MacOS

Bottom Line: Here are the default packages on a fresh image of Raspbian Buster Lite

Please see my post of the packages on a default Raspbian Jessie and Wheezy installation for background and “methodology.”

Raspbian Buster Lite 2019-06-20
SHA-256: 9009409a9f969b117602d85d992d90563f181a904bc3812bdd880fc493185234

Of note, it looks like this info is also available fromhttp://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2019-06-24/ in the .info file.

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                           Version                     Architecture Description
+++-==============================-===========================-============-===============================================================================
ii  adduser                        3.118                       all          add and remove users and groups
ii  alsa-utils                     1.1.8-2                     armhf        Utilities for configuring and using ALSA
ii  apt                            1.8.2                       armhf        commandline package manager
ii  apt-listchanges                3.19                        all          package change history notification tool
ii  apt-transport-https            1.8.2                       all          transitional package for https support
ii  apt-utils                      1.8.2                       armhf        package management related utility programs
ii  avahi-daemon                   0.7-4+b1                    armhf        Avahi mDNS/DNS-SD daemon
ii  base-files                     10.3+rpi1                   armhf        Debian base system miscellaneous files
ii  base-passwd                    3.5.46                      armhf        Debian base system master password and group files
ii  bash                           5.0-4                       armhf        GNU Bourne Again SHell
ii  bash-completion                1:2.8-6                     all          programmable completion for the bash shell
ii  bind9-host                     1:9.11.5.P4+dfsg-5          armhf        DNS lookup utility (deprecated)
ii  binutils                       2.31.1-16+rpi1              armhf        GNU assembler, linker and binary utilities
ii  binutils-arm-linux-gnueabihf   2.31.1-16+rpi1              armhf        GNU binary utilities, for arm-linux-gnueabihf target
ii  binutils-common:armhf          2.31.1-16+rpi1              armhf        Common files for the GNU assembler, linker and binary utilities
ii  bluez                          5.50-1+rpt1                 armhf        Bluetooth tools and daemons
ii  bluez-firmware                 1.2-4+rpt2                  all          Firmware for Bluetooth devices
ii  bsdmainutils                   11.1.2                      armhf        collection of more utilities from FreeBSD
ii  bsdutils                       1:2.33.1-0.1                armhf        basic utilities from 4.4BSD-Lite
ii  build-essential                12.6                        armhf        Informational list of build-essential packages
ii  bzip2                          1.0.6-9                     armhf        high-quality block-sorting file compressor - utilities
ii  ca-certificates                20190110                    all          Common CA certificates
ii  cifs-utils                     2:6.8-2                     armhf        Common Internet File System utilities
ii  console-setup                  1.191                       all          console font and keymap setup program
ii  console-setup-linux            1.191                       all          Linux specific part of console-setup
ii  coreutils                      8.30-3                      armhf        GNU core utilities
ii  cpio                           2.12+dfsg-9                 armhf        GNU cpio -- a program to manage archives of files
ii  cpp                            4:8.3.0-1+rpi2              armhf        GNU C preprocessor (cpp)
ii  cpp-8                          8.3.0-6+rpi1                armhf        GNU C preprocessor
ii  crda                           3.18-1                      armhf        wireless Central Regulatory Domain Agent
ii  cron                           3.0pl1-133                  armhf        process scheduling daemon
ii  curl                           7.64.0-4                    armhf        command line tool for transferring data with URL syntax
ii  dash                           0.5.10.2-5                  armhf        POSIX-compliant shell
ii  dbus                           1.12.16-1                   armhf        simple interprocess messaging system (daemon and utilities)
ii  dc                             1.07.1-2                    armhf        GNU dc arbitrary precision reverse-polish calculator
ii  debconf                        1.5.71                      all          Debian configuration management system
ii  debconf-i18n                   1.5.71                      all          full internationalization support for debconf
ii  debconf-utils                  1.5.71                      all          debconf utilities
ii  debianutils                    4.8.6.1                     armhf        Miscellaneous utilities specific to Debian
ii  device-tree-compiler           1.4.7-3+rpt1                armhf        Device Tree Compiler for Flat Device Trees
ii  dhcpcd5                        1:7.0.8-0.1+rpt1            armhf        DHCPv4, IPv6RA and DHCPv6 client with IPv4LL support
ii  diffutils                      1:3.7-3                     armhf        File comparison utilities
ii  dirmngr                        2.2.12-1+rpi1               armhf        GNU privacy guard - network certificate management service
ii  distro-info-data               0.41                        all          information about the distributions' releases (data files)
ii  dmidecode                      3.2-1                       armhf        SMBIOS/DMI table decoder
ii  dmsetup                        2:1.02.155-2                armhf        Linux Kernel Device Mapper userspace library
ii  dosfstools                     4.1-2                       armhf        utilities for making and checking MS-DOS FAT filesystems
ii  dphys-swapfile                 20100506-5                  all          Autogenerate and use a swap file
ii  dpkg                           1.19.7                      armhf        Debian package management system
ii  dpkg-dev                       1.19.7                      all          Debian package development tools
ii  e2fsprogs                      1.44.5-1                    armhf        ext2/ext3/ext4 file system utilities
ii  ed                             1.15-1                      armhf        classic UNIX line editor
ii  ethtool                        1:4.19-1                    armhf        display or change Ethernet device settings
ii  fake-hwclock                   0.11+rpt1                   all          Save/restore system clock on machines without working RTC hardware
ii  fakeroot                       1.23-1                      armhf        tool for simulating superuser privileges
ii  fbset                          2.1-30                      armhf        framebuffer device maintenance program
ii  fdisk                          2.33.1-0.1                  armhf        collection of partitioning utilities
ii  file                           1:5.35-4                    armhf        Recognize the type of data in a file using "magic" numbers
ii  findutils                      4.6.0+git+20190209-2        armhf        utilities for finding files--find, xargs
ii  firmware-atheros               1:20190114-1+rpt2           all          Binary firmware for Atheros wireless cards
ii  firmware-brcm80211             1:20190114-1+rpt2           all          Binary firmware for Broadcom/Cypress 802.11 wireless cards
ii  firmware-libertas              1:20190114-1+rpt2           all          Binary firmware for Marvell wireless cards
ii  firmware-misc-nonfree          1:20190114-1+rpt2           all          Binary firmware for various drivers in the Linux kernel
ii  firmware-realtek               1:20190114-1+rpt2           all          Binary firmware for Realtek wired/wifi/BT adapters
ii  freetype2-doc                  2.9.1-3                     all          FreeType 2 font engine, development documentation
ii  g++                            4:8.3.0-1+rpi2              armhf        GNU C++ compiler
ii  g++-8                          8.3.0-6+rpi1                armhf        GNU C++ compiler
ii  gcc                            4:8.3.0-1+rpi2              armhf        GNU C compiler
ii  gcc-4.9-base:armhf             4.9.4-2+rpi1+b19            armhf        GCC, the GNU Compiler Collection (base package)
ii  gcc-5-base:armhf               5.5.0-8                     armhf        GCC, the GNU Compiler Collection (base package)
ii  gcc-6-base:armhf               6.5.0-1+rpi1+b1             armhf        GCC, the GNU Compiler Collection (base package)
ii  gcc-7-base:armhf               7.3.0-19                    armhf        GCC, the GNU Compiler Collection (base package)
ii  gcc-8                          8.3.0-6+rpi1                armhf        GNU C compiler
ii  gcc-8-base:armhf               8.3.0-6+rpi1                armhf        GCC, the GNU Compiler Collection (base package)
ii  gdb                            8.2.1-2                     armhf        GNU Debugger
ii  gdbm-l10n                      1.18.1-4                    all          GNU dbm database routines (translation files) 
ii  geoip-database                 20181108-1                  all          IP lookup command line tools that use the GeoIP library (country database)
ii  gettext-base                   0.19.8.1-9                  armhf        GNU Internationalization utilities for the base system
ii  gnupg                          2.2.12-1+rpi1               all          GNU privacy guard - a free PGP replacement
ii  gnupg-l10n                     2.2.12-1+rpi1               all          GNU privacy guard - localization files
ii  gnupg-utils                    2.2.12-1+rpi1               armhf        GNU privacy guard - utility programs
ii  gpg                            2.2.12-1+rpi1               armhf        GNU Privacy Guard -- minimalist public key operations
ii  gpg-agent                      2.2.12-1+rpi1               armhf        GNU privacy guard - cryptographic agent
ii  gpg-wks-client                 2.2.12-1+rpi1               armhf        GNU privacy guard - Web Key Service client
ii  gpg-wks-server                 2.2.12-1+rpi1               armhf        GNU privacy guard - Web Key Service server
ii  gpgconf                        2.2.12-1+rpi1               armhf        GNU privacy guard - core configuration utilities
ii  gpgsm                          2.2.12-1+rpi1               armhf        GNU privacy guard - S/MIME version
ii  gpgv                           2.2.12-1+rpi1               armhf        GNU privacy guard - signature verification tool
ii  grep                           3.3-1                       armhf        GNU grep, egrep and fgrep
ii  groff-base                     1.22.4-3                    armhf        GNU troff text-formatting system (base system components)
ii  gzip                           1.9-3                       armhf        GNU compression utilities
ii  hardlink                       0.3.2                       armhf        Hardlinks multiple copies of the same file
ii  hostname                       3.21                        armhf        utility to set/show the host name or domain name
ii  htop                           2.2.0-1                     armhf        interactive processes viewer
ii  ifupdown                       0.8.35                      armhf        high level tools to configure network interfaces
ii  info                           6.5.0.dfsg.1-4+b1           armhf        Standalone GNU Info documentation browser
ii  init                           1.56+nmu1                   armhf        metapackage ensuring an init system is installed
ii  init-system-helpers            1.56+nmu1                   all          helper tools for all init systems
ii  install-info                   6.5.0.dfsg.1-4+b1           armhf        Manage installed documentation in info format
ii  iproute2                       4.20.0-2                    armhf        networking and traffic control tools
ii  iptables                       1.8.2-4                     armhf        administration tools for packet filtering and NAT
ii  iputils-ping                   3:20180629-2                armhf        Tools to test the reachability of network hosts
ii  isc-dhcp-client                4.4.1-2                     armhf        DHCP client for automatically obtaining an IP address
ii  isc-dhcp-common                4.4.1-2                     armhf        common manpages relevant to all of the isc-dhcp packages
ii  iso-codes                      4.2-1                       all          ISO language, territory, currency, script codes and their translations
ii  iw                             5.0.1-1                     armhf        tool for configuring Linux wireless devices
ii  javascript-common              11                          all          Base support for JavaScript library packages
ii  kbd                            2.0.4-4                     armhf        Linux console font and keytable utilities
ii  keyboard-configuration         1.191                       all          system-wide keyboard preferences
ii  keyutils                       1.6-6                       armhf        Linux Key Management Utilities
ii  kmod                           26-1                        armhf        tools for managing Linux kernel modules
ii  less                           487-0.1                     armhf        pager program similar to more
ii  libacl1:armhf                  2.2.53-4                    armhf        access control list - shared library
ii  libalgorithm-diff-perl         1.19.03-2                   all          module to find differences between files
ii  libalgorithm-diff-xs-perl      0.04-5+b1                   armhf        module to find differences between files (XS accelerated)
ii  libalgorithm-merge-perl        0.08-3                      all          Perl module for three-way merge of textual data
ii  libapparmor1:armhf             2.13.2-10                   armhf        changehat AppArmor library
ii  libapt-inst2.0:armhf           1.8.2                       armhf        deb package format runtime library
ii  libapt-pkg5.0:armhf            1.8.2                       armhf        package management runtime library
ii  libargon2-1:armhf              0~20171227-0.2              armhf        memory-hard hashing function - runtime library
ii  libasan5:armhf                 8.3.0-6+rpi1                armhf        AddressSanitizer -- a fast memory error detector
ii  libasound2:armhf               1.1.8-1+rpt1                armhf        shared library for ALSA applications
ii  libasound2-data                1.1.8-1+rpt1                all          Configuration files and profiles for ALSA drivers
ii  libassuan0:armhf               2.5.2-1                     armhf        IPC library for the GnuPG components
ii  libatomic1:armhf               8.3.0-6+rpi1                armhf        support library providing __atomic built-in functions
ii  libattr1:armhf                 1:2.4.48-4                  armhf        extended attribute handling - shared library
ii  libaudit-common                1:2.8.4-3                   all          Dynamic library for security auditing - common files
ii  libaudit1:armhf                1:2.8.4-3                   armhf        Dynamic library for security auditing
ii  libavahi-common-data:armhf     0.7-4+b1                    armhf        Avahi common data files
ii  libavahi-common3:armhf         0.7-4+b1                    armhf        Avahi common library
ii  libavahi-core7:armhf           0.7-4+b1                    armhf        Avahi's embeddable mDNS/DNS-SD library
ii  libbabeltrace1:armhf           1.5.6-2                     armhf        Babeltrace conversion libraries
ii  libbind9-161:armhf             1:9.11.5.P4+dfsg-5          armhf        BIND9 Shared Library used by BIND
ii  libbinutils:armhf              2.31.1-16+rpi1              armhf        GNU binary utilities (private shared library)
ii  libblkid1:armhf                2.33.1-0.1                  armhf        block device ID library
ii  libboost-iostreams1.58.0:armhf 1.58.0+dfsg-5.1+rpi1+b4     armhf        Boost.Iostreams Library
ii  libbsd0:armhf                  0.9.1-2                     armhf        utility functions from BSD systems - shared library
ii  libbz2-1.0:armhf               1.0.6-9                     armhf        high-quality block-sorting file compressor library - runtime
ii  libc-bin                       2.28-10+rpi1                armhf        GNU C Library: Binaries
ii  libc-dev-bin                   2.28-10+rpi1                armhf        GNU C Library: Development binaries
ii  libc-l10n                      2.28-10+rpi1                all          GNU C Library: localization files
ii  libc6:armhf                    2.28-10+rpi1                armhf        GNU C Library: Shared libraries
ii  libc6-dbg:armhf                2.28-10+rpi1                armhf        GNU C Library: detached debugging symbols
ii  libc6-dev:armhf                2.28-10+rpi1                armhf        GNU C Library: Development Libraries and Header Files
ii  libcap-ng0:armhf               0.7.9-2                     armhf        An alternate POSIX capabilities library
ii  libcap2:armhf                  1:2.25-2                    armhf        POSIX 1003.1e capabilities (library)
ii  libcap2-bin                    1:2.25-2                    armhf        POSIX 1003.1e capabilities (utilities)
ii  libcc1-0:armhf                 8.3.0-6+rpi1                armhf        GCC cc1 plugin for GDB
ii  libcom-err2:armhf              1.44.5-1                    armhf        common error description library
ii  libcryptsetup12:armhf          2:2.1.0-5                   armhf        disk encryption support - shared library
ii  libcurl4:armhf                 7.64.0-4                    armhf        easy-to-use client-side URL transfer library (OpenSSL flavour)
ii  libdaemon0:armhf               0.14-7                      armhf        lightweight C library for daemons - runtime library
ii  libdb5.3:armhf                 5.3.28+dfsg1-0.5            armhf        Berkeley v5.3 Database Libraries [runtime]
ii  libdbus-1-3:armhf              1.12.16-1                   armhf        simple interprocess messaging system (library)
ii  libdebconfclient0:armhf        0.249                       armhf        Debian Configuration Management System (C-implementation library)
ii  libdevmapper1.02.1:armhf       2:1.02.155-2                armhf        Linux Kernel Device Mapper userspace library
ii  libdns-export1104              1:9.11.5.P4+dfsg-5          armhf        Exported DNS Shared Library
ii  libdns1104:armhf               1:9.11.5.P4+dfsg-5          armhf        DNS Shared Library used by BIND
ii  libdpkg-perl                   1.19.7                      all          Dpkg perl modules
ii  libdw1:armhf                   0.176-1.1                   armhf        library that provides access to the DWARF debug information
ii  libedit2:armhf                 3.1-20181209-1              armhf        BSD editline and history libraries
ii  libelf1:armhf                  0.176-1.1                   armhf        library to read and write ELF files
ii  libestr0:armhf                 0.1.10-2.1                  armhf        Helper functions for handling strings (lib)
ii  libevent-2.1-6:armhf           2.1.8-stable-4              armhf        Asynchronous event notification library
ii  libexpat1:armhf                2.2.6-1                     armhf        XML parsing C library - runtime library
ii  libext2fs2:armhf               1.44.5-1                    armhf        ext2/ext3/ext4 file system libraries
ii  libfakeroot:armhf              1.23-1                      armhf        tool for simulating superuser privileges - shared libraries
ii  libfastjson4:armhf             0.99.8-2                    armhf        fast json library for C
ii  libfdisk1:armhf                2.33.1-0.1                  armhf        fdisk partitioning library
ii  libffi6:armhf                  3.2.1-9                     armhf        Foreign Function Interface library runtime
ii  libfftw3-single3:armhf         3.3.8-2                     armhf        Library for computing Fast Fourier Transforms - Single precision
ii  libfile-fcntllock-perl         0.22-3+b4                   armhf        Perl module for file locking with fcntl(2)
ii  libfreetype6:armhf             2.9.1-3                     armhf        FreeType 2 font engine, shared library files
ii  libfreetype6-dev:armhf         2.9.1-3                     armhf        FreeType 2 font engine, development files
ii  libfribidi0:armhf              1.0.5-3.1                   armhf        Free Implementation of the Unicode BiDi algorithm
ii  libfstrm0:armhf                0.4.0-1                     armhf        Frame Streams (fstrm) library
ii  libgcc-8-dev:armhf             8.3.0-6+rpi1                armhf        GCC support library (development files)
ii  libgcc1:armhf                  1:8.3.0-6+rpi1              armhf        GCC support library
ii  libgcrypt20:armhf              1.8.4-5                     armhf        LGPL Crypto library - runtime library
ii  libgdbm-compat4:armhf          1.18.1-4                    armhf        GNU dbm database routines (legacy support runtime version) 
ii  libgdbm6:armhf                 1.18.1-4                    armhf        GNU dbm database routines (runtime version) 
ii  libgeoip1:armhf                1.6.12-1                    armhf        non-DNS IP-to-country resolver library
ii  libglib2.0-0:armhf             2.58.3-2                    armhf        GLib library of C routines
ii  libglib2.0-data                2.58.3-2                    all          Common files for GLib library
ii  libgmp10:armhf                 2:6.1.2+dfsg-4              armhf        Multiprecision arithmetic library
ii  libgnutls30:armhf              3.6.7-4                     armhf        GNU TLS library - main runtime library
ii  libgomp1:armhf                 8.3.0-6+rpi1                armhf        GCC OpenMP (GOMP) support library
ii  libgpg-error0:armhf            1.35-1                      armhf        GnuPG development runtime library
ii  libgpm2:armhf                  1.20.7-5                    armhf        General Purpose Mouse - shared library
ii  libgssapi-krb5-2:armhf         1.17-2                      armhf        MIT Kerberos runtime libraries - krb5 GSS-API Mechanism
ii  libhogweed4:armhf              3.4.1-1                     armhf        low level cryptographic library (public-key cryptos)
ii  libicu63:armhf                 63.1-6                      armhf        International Components for Unicode
ii  libident                       0.22-3.1                    armhf        simple RFC1413 client library - runtime
ii  libidn11:armhf                 1.33-2.2                    armhf        GNU Libidn library, implementation of IETF IDN specifications
ii  libidn2-0:armhf                2.0.5-1                     armhf        Internationalized domain names (IDNA2008/TR46) library
ii  libip4tc0:armhf                1.8.2-4                     armhf        netfilter libip4tc library
ii  libip6tc0:armhf                1.8.2-4                     armhf        netfilter libip6tc library
ii  libiptc0:armhf                 1.8.2-4                     armhf        netfilter libiptc library
ii  libisc-export1100:armhf        1:9.11.5.P4+dfsg-5          armhf        Exported ISC Shared Library
ii  libisc1100:armhf               1:9.11.5.P4+dfsg-5          armhf        ISC Shared Library used by BIND
ii  libisccc161:armhf              1:9.11.5.P4+dfsg-5          armhf        Command Channel Library used by BIND
ii  libisccfg163:armhf             1:9.11.5.P4+dfsg-5          armhf        Config File Handling Library used by BIND
ii  libisl19:armhf                 0.20-2                      armhf        manipulating sets and relations of integer points bounded by linear constraints
ii  libiw30:armhf                  30~pre9-13                  armhf        Wireless tools - library
ii  libjim0.77:armhf               0.77+dfsg0-3                armhf        small-footprint implementation of Tcl - shared library
ii  libjpeg62-turbo:armhf          1:1.5.2-2+b1                armhf        libjpeg-turbo JPEG runtime library
ii  libjs-jquery                   3.3.1~dfsg-3                all          JavaScript library for dynamic web applications
ii  libjson-c3:armhf               0.12.1+ds-2                 armhf        JSON manipulation library - shared library
ii  libk5crypto3:armhf             1.17-2                      armhf        MIT Kerberos runtime libraries - Crypto Library
ii  libkeyutils1:armhf             1.6-6                       armhf        Linux Key Management Utilities (library)
ii  libkmod2:armhf                 26-1                        armhf        libkmod shared library
ii  libkrb5-3:armhf                1.17-2                      armhf        MIT Kerberos runtime libraries
ii  libkrb5support0:armhf          1.17-2                      armhf        MIT Kerberos runtime libraries - Support library
ii  libksba8:armhf                 1.3.5-2                     armhf        X.509 and CMS support library
ii  libldap-2.4-2:armhf            2.4.47+dfsg-3+rpi1          armhf        OpenLDAP libraries
ii  libldap-common                 2.4.47+dfsg-3+rpi1          all          OpenLDAP common files for libraries
ii  liblmdb0:armhf                 0.9.22-1                    armhf        Lightning Memory-Mapped Database shared library
ii  liblocale-gettext-perl         1.07-3+b3                   armhf        module using libc functions for internationalization in Perl
ii  liblognorm5:armhf              2.0.5-1                     armhf        log normalizing library
ii  libluajit-5.1-2:armhf          2.1.0~beta3+dfsg-5.1        armhf        Just in time compiler for Lua - library version
ii  libluajit-5.1-common           2.1.0~beta3+dfsg-5.1        all          Just in time compiler for Lua - common files
ii  liblwres161:armhf              1:9.11.5.P4+dfsg-5          armhf        Lightweight Resolver Library used by BIND
ii  liblz4-1:armhf                 1.8.3-1                     armhf        Fast LZ compression algorithm library - runtime
ii  liblzma5:armhf                 5.2.4-1                     armhf        XZ-format compression library
ii  libmagic-mgc                   1:5.35-4                    armhf        File type determination library using "magic" numbers (compiled magic file)
ii  libmagic1:armhf                1:5.35-4                    armhf        Recognize the type of data in a file using "magic" numbers - library
ii  libmnl-dev                     1.0.4-2                     armhf        minimalistic Netlink communication library (devel)
ii  libmnl0:armhf                  1.0.4-2                     armhf        minimalistic Netlink communication library
ii  libmount1:armhf                2.33.1-0.1                  armhf        device mounting library
ii  libmpc3:armhf                  1.1.0-1                     armhf        multiple precision complex floating-point library
ii  libmpdec2:armhf                2.4.2-2                     armhf        library for decimal floating point arithmetic (runtime library)
ii  libmpfr6:armhf                 4.0.2-1                     armhf        multiple precision floating-point computation
ii  libmtp-common                  1.1.16-2                    all          Media Transfer Protocol (MTP) common files
ii  libmtp-runtime                 1.1.16-2                    armhf        Media Transfer Protocol (MTP) runtime tools
ii  libmtp9:armhf                  1.1.16-2                    armhf        Media Transfer Protocol (MTP) library
ii  libncurses6:armhf              6.1+20181013-2              armhf        shared libraries for terminal handling
ii  libncursesw5:armhf             6.1+20181013-2              armhf        shared libraries for terminal handling (wide character legacy version)
ii  libncursesw6:armhf             6.1+20181013-2              armhf        shared libraries for terminal handling (wide character support)
ii  libnetfilter-conntrack3:armhf  1.0.7-1                     armhf        Netfilter netlink-conntrack library
ii  libnettle6:armhf               3.4.1-1                     armhf        low level cryptographic library (symmetric and one-way cryptos)
ii  libnewt0.52:armhf              0.52.20-8                   armhf        Not Erik's Windowing Toolkit - text mode windowing with slang
ii  libnfnetlink0:armhf            1.0.1-3                     armhf        Netfilter netlink library
ii  libnfsidmap2:armhf             0.25-5.1                    armhf        NFS idmapping library
ii  libnftnl11:armhf               1.1.2-2                     armhf        Netfilter nftables userspace API library
ii  libnghttp2-14:armhf            1.36.0-2                    armhf        library implementing HTTP/2 protocol (shared library)
ii  libnl-3-200:armhf              3.4.0-1                     armhf        library for dealing with netlink sockets
ii  libnl-genl-3-200:armhf         3.4.0-1                     armhf        library for dealing with netlink sockets - generic netlink
ii  libnl-route-3-200:armhf        3.4.0-1                     armhf        library for dealing with netlink sockets - route interface
ii  libnpth0:armhf                 1.6-1                       armhf        replacement for GNU Pth using system threads
ii  libnss-mdns:armhf              0.14.1-1+b5                 armhf        NSS module for Multicast DNS name resolution
ii  libp11-kit0:armhf              0.23.15-2                   armhf        library for loading and coordinating access to PKCS#11 modules - runtime
ii  libpam-chksshpwd:armhf         1.3.1-5+rpt1                armhf        PAM module to enable SSH password checking support
ii  libpam-modules:armhf           1.3.1-5+rpt1                armhf        Pluggable Authentication Modules for PAM
ii  libpam-modules-bin             1.3.1-5+rpt1                armhf        Pluggable Authentication Modules for PAM - helper binaries
ii  libpam-runtime                 1.3.1-5+rpt1                all          Runtime support for the PAM library
ii  libpam-systemd:armhf           241-5+rpi1                  armhf        system and service manager - PAM module
ii  libpam0g:armhf                 1.3.1-5+rpt1                armhf        Pluggable Authentication Modules library
ii  libparted2:armhf               3.2-25                      armhf        disk partition manipulator - shared library
ii  libpcre2-8-0:armhf             10.32-5                     armhf        New Perl Compatible Regular Expression Library- 8 bit runtime files
ii  libpcre2-posix0:armhf          10.32-5                     armhf        New Perl Compatible Regular Expression Library - posix-compatible runtime files
ii  libpcre3:armhf                 2:8.39-12                   armhf        Old Perl 5 Compatible Regular Expression Library - runtime files
ii  libpcsclite1:armhf             1.8.24-1                    armhf        Middleware to access a smart card using PC/SC (library)
ii  libperl5.28:armhf              5.28.1-6                    armhf        shared Perl library
ii  libpipeline1:armhf             1.5.1-2                     armhf        pipeline manipulation library
ii  libpng-dev:armhf               1.6.36-6                    armhf        PNG library - development (version 1.6)
ii  libpng-tools                   1.6.36-6                    armhf        PNG library - tools (version 1.6)
ii  libpng16-16:armhf              1.6.36-6                    armhf        PNG library - runtime (version 1.6)
ii  libpolkit-agent-1-0:armhf      0.105-25                    armhf        PolicyKit Authentication Agent API
ii  libpolkit-backend-1-0:armhf    0.105-25                    armhf        PolicyKit backend API
ii  libpolkit-gobject-1-0:armhf    0.105-25                    armhf        PolicyKit Authorization API
ii  libpopt0:armhf                 1.16-12                     armhf        lib for parsing cmdline parameters
ii  libprocps7:armhf               2:3.3.15-2                  armhf        library for accessing process information from /proc
ii  libprotobuf-c1:armhf           1.3.1-1+b1                  armhf        Protocol Buffers C shared library (protobuf-c)
ii  libpsl5:armhf                  0.20.2-2                    armhf        Library for Public Suffix List (shared libraries)
ii  libpython-stdlib:armhf         2.7.16-1                    armhf        interactive high-level object-oriented language (Python2)
ii  libpython2-stdlib:armhf        2.7.16-1                    armhf        interactive high-level object-oriented language (Python2)
ii  libpython2.7-minimal:armhf     2.7.16-2                    armhf        Minimal subset of the Python language (version 2.7)
ii  libpython2.7-stdlib:armhf      2.7.16-2                    armhf        Interactive high-level object-oriented language (standard library, version 2.7)
ii  libpython3-stdlib:armhf        3.7.3-1                     armhf        interactive high-level object-oriented language (default python3 version)
ii  libpython3.7:armhf             3.7.3-2                     armhf        Shared Python runtime library (version 3.7)
ii  libpython3.7-minimal:armhf     3.7.3-2                     armhf        Minimal subset of the Python language (version 3.7)
ii  libpython3.7-stdlib:armhf      3.7.3-2                     armhf        Interactive high-level object-oriented language (standard library, version 3.7)
ii  libraspberrypi-bin             1.20190620-1                armhf        Miscellaneous Raspberry Pi utilities
ii  libraspberrypi-dev             1.20190620-1                armhf        EGL/GLES/OpenVG/etc. libraries for the Raspberry Pi's VideoCore IV (headers)
ii  libraspberrypi-doc             1.20190620-1                armhf        EGL/GLES/OpenVG/etc. libraries for the Raspberry Pi's VideoCore IV (headers)
ii  libraspberrypi0                1.20190620-1                armhf        EGL/GLES/OpenVG/etc. libraries for the Raspberry Pi's VideoCore IV
ii  libreadline6:armhf             6.3-9                       armhf        GNU readline and history libraries, run-time libraries
ii  libreadline7:armhf             7.0-5                       armhf        GNU readline and history libraries, run-time libraries
ii  librtmp1:armhf                 2.4+20151223.gitfa8646d.1-2 armhf        toolkit for RTMP streams (shared library)
ii  libsamplerate0:armhf           0.1.9-2                     armhf        Audio sample rate conversion library
ii  libsasl2-2:armhf               2.1.27+dfsg-1+b1            armhf        Cyrus SASL - authentication abstraction library
ii  libsasl2-modules-db:armhf      2.1.27+dfsg-1+b1            armhf        Cyrus SASL - pluggable authentication modules (DB)
ii  libseccomp2:armhf              2.3.3-4                     armhf        high level interface to Linux seccomp filter
ii  libselinux1:armhf              2.8-1+b1                    armhf        SELinux runtime shared libraries
ii  libsemanage-common             2.8-2                       all          Common files for SELinux policy management libraries
ii  libsemanage1:armhf             2.8-2                       armhf        SELinux policy management library
ii  libsepol1:armhf                2.8-1                       armhf        SELinux library for manipulating binary security policies
ii  libsigc++-1.2-5c2              1.2.7-2+b1                  armhf        type-safe Signal Framework for C++ - runtime
ii  libslang2:armhf                2.3.2-2                     armhf        S-Lang programming library - runtime version
ii  libsmartcols1:armhf            2.33.1-0.1                  armhf        smart column output alignment library
ii  libsqlite3-0:armhf             3.27.2-3                    armhf        SQLite 3 shared library
ii  libss2:armhf                   1.44.5-1                    armhf        command-line interface parsing library
ii  libssh2-1:armhf                1.8.0-2.1                   armhf        SSH2 client-side library
ii  libssl1.1:armhf                1.1.1c-1                    armhf        Secure Sockets Layer toolkit - shared libraries
ii  libstdc++-8-dev:armhf          8.3.0-6+rpi1                armhf        GNU Standard C++ Library v3 (development files)
ii  libstdc++6:armhf               8.3.0-6+rpi1                armhf        GNU Standard C++ Library v3
ii  libsystemd0:armhf              241-5+rpi1                  armhf        systemd utility library
ii  libtalloc2:armhf               2.1.14-2                    armhf        hierarchical pool based memory allocator
ii  libtasn1-6:armhf               4.13-3                      armhf        Manage ASN.1 structures (runtime)
ii  libtext-charwidth-perl         0.04-7.1+b1                 armhf        get display widths of characters on the terminal
ii  libtext-iconv-perl             1.7-5+b10                   armhf        converts between character sets in Perl
ii  libtext-wrapi18n-perl          0.06-7.1                    all          internationalized substitute of Text::Wrap
ii  libtinfo5:armhf                6.1+20181013-2              armhf        shared low-level terminfo library (legacy version)
ii  libtinfo6:armhf                6.1+20181013-2              armhf        shared low-level terminfo library for terminal handling
ii  libtirpc-common                1.1.4-0.4                   all          transport-independent RPC library - common files
ii  libtirpc3:armhf                1.1.4-0.4                   armhf        transport-independent RPC library
ii  libubsan1:armhf                8.3.0-6+rpi1                armhf        UBSan -- undefined behaviour sanitizer (runtime)
ii  libuchardet0:armhf             0.0.6-3                     armhf        universal charset detection library - shared library
ii  libudev0:armhf                 175-7.2                     armhf        libudev shared library
ii  libudev1:armhf                 241-5+rpi1                  armhf        libudev shared library
ii  libunistring2:armhf            0.9.10-1                    armhf        Unicode string library for C
ii  libusb-1.0-0:armhf             2:1.0.22-2                  armhf        userspace USB programming library
ii  libuuid1:armhf                 2.33.1-0.1                  armhf        Universally Unique ID library
ii  libv4l-0:armhf                 1.16.3-3                    armhf        Collection of video4linux support libraries
ii  libv4l2rds0:armhf              1.16.3-3                    armhf        Video4Linux Radio Data System (RDS) decoding library
ii  libv4lconvert0:armhf           1.16.3-3                    armhf        Video4linux frame format conversion library
ii  libwbclient0:armhf             2:4.9.5+dfsg-4              armhf        Samba winbind client library
ii  libwrap0:armhf                 7.6.q-28                    armhf        Wietse Venema's TCP wrappers library
ii  libx11-6:armhf                 2:1.6.7-1                   armhf        X11 client-side library
ii  libx11-data                    2:1.6.7-1                   all          X11 client-side library
ii  libxau6:armhf                  1:1.0.8-1+b2                armhf        X11 authorisation library
ii  libxcb1:armhf                  1.13.1-2                    armhf        X C Binding
ii  libxdmcp6:armhf                1:1.1.2-3                   armhf        X11 Display Manager Control Protocol library
ii  libxext6:armhf                 2:1.3.3-1+b2                armhf        X11 miscellaneous extension library
ii  libxml2:armhf                  2.9.4+dfsg1-7+b1            armhf        GNOME XML library
ii  libxmuu1:armhf                 2:1.1.2-2                   armhf        X11 miscellaneous micro-utility library
ii  libxtables12:armhf             1.8.2-4                     armhf        netfilter xtables library
ii  libzstd1:armhf                 1.3.8+dfsg-3+rpi1           armhf        fast lossless compression algorithm
ii  linux-libc-dev:armhf           4.18.20-2+rpi1              armhf        Linux support headers for userspace development
ii  locales                        2.28-10+rpi1                all          GNU C Library: National Language (locale) data [support]
ii  login                          1:4.5-1.1                   armhf        system login tools
ii  logrotate                      3.14.0-4                    armhf        Log rotation utility
ii  lsb-base                       10.2019051400+rpi1          all          Linux Standard Base init script functionality
ii  lsb-release                    10.2019051400+rpi1          all          Linux Standard Base version reporting utility
ii  lua5.1                         5.1.5-8.1                   armhf        Simple, extensible, embeddable programming language
ii  luajit                         2.1.0~beta3+dfsg-5.1        armhf        Just in time compiler for Lua programming language version 5.1
ii  make                           4.2.1-1.2                   armhf        utility for directing compilation
ii  man-db                         2.8.5-2                     armhf        on-line manual pager
ii  manpages                       4.16-2                      all          Manual pages about using a GNU/Linux system
ii  manpages-dev                   4.16-2                      all          Manual pages about using GNU/Linux for development
ii  mawk                           1.3.3-17                    armhf        a pattern scanning and text processing language
ii  mime-support                   3.62                        all          MIME files 'mime.types' & 'mailcap', and support programs
ii  mount                          2.33.1-0.1                  armhf        tools for mounting and manipulating filesystems
ii  multiarch-support              2.28-10+rpi1                armhf        Transitional package to ensure multiarch compatibility
ii  nano                           3.2-2                       armhf        small, friendly text editor inspired by Pico
ii  ncdu                           1.13-1                      armhf        ncurses disk usage viewer
ii  ncurses-base                   6.1+20181013-2              all          basic terminal type definitions
ii  ncurses-bin                    6.1+20181013-2              armhf        terminal-related programs and man pages
ii  ncurses-term                   6.1+20181013-2              all          additional terminal type definitions
ii  net-tools                      1.60+git20180626.aebd88e-1  armhf        NET-3 networking toolkit
ii  netbase                        5.6                         all          Basic TCP/IP networking system
ii  netcat-openbsd                 1.195-2                     armhf        TCP/IP swiss army knife
ii  netcat-traditional             1.10-41.1                   armhf        TCP/IP swiss army knife
ii  nfs-common                     1:1.3.4-2.5                 armhf        NFS support files common to client and server
ii  openresolv                     3.8.0-1                     armhf        management framework for resolv.conf
ii  openssh-client                 1:7.9p1-10                  armhf        secure shell (SSH) client, for secure access to remote machines
ii  openssh-server                 1:7.9p1-10                  armhf        secure shell (SSH) server, for secure access from remote machines
ii  openssh-sftp-server            1:7.9p1-10                  armhf        secure shell (SSH) sftp server module, for SFTP access from remote machines
ii  openssl                        1.1.1c-1                    armhf        Secure Sockets Layer toolkit - cryptographic utility
ii  parted                         3.2-25                      armhf        disk partition manipulator
ii  passwd                         1:4.5-1.1                   armhf        change and administer password and group data
ii  patch                          2.7.6-3                     armhf        Apply a diff file to an original
ii  paxctld                        1.2.1-1                     armhf        Daemon to automatically set appropriate PaX flags
ii  perl                           5.28.1-6                    armhf        Larry Wall's Practical Extraction and Report Language
ii  perl-base                      5.28.1-6                    armhf        minimal Perl system
ii  perl-modules-5.28              5.28.1-6                    all          Core Perl modules
ii  pi-bluetooth                   0.1.11                      all          Raspberry Pi 3 bluetooth
ii  pinentry-curses                1.1.0-2                     armhf        curses-based PIN or pass-phrase entry dialog for GnuPG
ii  pkg-config                     0.29-6                      armhf        manage compile and link flags for libraries
ii  policykit-1                    0.105-25                    armhf        framework for managing administrative policies and privileges
ii  procps                         2:3.3.15-2                  armhf        /proc file system utilities
ii  psmisc                         23.2-1                      armhf        utilities that use the proc file system
ii  publicsuffix                   20190415.1030-1             all          accurate, machine-readable list of domain name suffixes
ii  python                         2.7.16-1                    armhf        interactive high-level object-oriented language (Python2 version)
ii  python-apt-common              1.8.4                       all          Python interface to libapt-pkg (locales)
ii  python-minimal                 2.7.16-1                    armhf        minimal subset of the Python2 language
ii  python-rpi.gpio                0.6.5-1                     armhf        Module to control Raspberry Pi GPIO channels (Python 2)
ii  python2                        2.7.16-1                    armhf        interactive high-level object-oriented language (Python2 version)
ii  python2-minimal                2.7.16-1                    armhf        minimal subset of the Python2 language
ii  python2.7                      2.7.16-2                    armhf        Interactive high-level object-oriented language (version 2.7)
ii  python2.7-minimal              2.7.16-2                    armhf        Minimal subset of the Python language (version 2.7)
ii  python3                        3.7.3-1                     armhf        interactive high-level object-oriented language (default python3 version)
ii  python3-apt                    1.8.4                       armhf        Python 3 interface to libapt-pkg
ii  python3-certifi                2018.8.24-1                 all          root certificates for validating SSL certs and verifying TLS hosts (python3)
ii  python3-chardet                3.0.4-3                     all          universal character encoding detector for Python3
ii  python3-debconf                1.5.71                      all          interact with debconf from Python 3
ii  python3-idna                   2.6-1                       all          Python IDNA2008 (RFC 5891) handling (Python 3)
ii  python3-minimal                3.7.3-1                     armhf        minimal subset of the Python language (default python3 version)
ii  python3-pkg-resources          40.8.0-1                    all          Package Discovery and Resource Access using pkg_resources
ii  python3-requests               2.21.0-1                    all          elegant and simple HTTP library for Python3, built for human beings
ii  python3-six                    1.12.0-1                    all          Python 2 and 3 compatibility library (Python 3 interface)
ii  python3-urllib3                1.24.1-1                    all          HTTP library with thread-safe connection pooling for Python3
ii  python3.7                      3.7.3-2                     armhf        Interactive high-level object-oriented language (version 3.7)
ii  python3.7-minimal              3.7.3-2                     armhf        Minimal subset of the Python language (version 3.7)
ii  raspberrypi-bootloader         1.20190620-1                armhf        Raspberry Pi bootloader
ii  raspberrypi-kernel             1.20190620-1                armhf        Raspberry Pi bootloader
ii  raspberrypi-net-mods           1.2.8                       all          Network configuration for the Raspberry Pi UI
ii  raspberrypi-sys-mods           20190429                    armhf        System tweaks for the Raspberry Pi
ii  raspbian-archive-keyring       20120528.2                  all          GnuPG archive keys of the raspbian archive
ii  raspi-config                   20190607                    all          Raspberry Pi configuration tool
ii  raspi-copies-and-fills         0.13                        armhf        ARM-accelerated versions of selected functions from string.h
ii  readline-common                7.0-5                       all          GNU readline and history libraries, common files
ii  rfkill                         2.33.1-0.1                  armhf        tool for enabling and disabling wireless devices
ii  rng-tools                      2-unofficial-mt.14-1        armhf        Daemon to use a Hardware TRNG
ii  rpcbind                        1.2.5-0.3                   armhf        converts RPC program numbers into universal addresses
ii  rpi-update                     20140705                    all          Raspberry Pi firmware updating tool
ii  rpi.gpio-common:armhf          0.6.5-1                     armhf        Module to control Raspberry Pi GPIO channels (common files)
ii  rsync                          3.1.3-6                     armhf        fast, versatile, remote (and local) file-copying tool
ii  rsyslog                        8.1901.0-1                  armhf        reliable system and kernel logging daemon
ii  sed                            4.7-1                       armhf        GNU stream editor for filtering/transforming text
ii  sensible-utils                 0.0.12                      all          Utilities for sensible alternative selection
ii  shared-mime-info               1.10-1                      armhf        FreeDesktop.org shared MIME database and spec
ii  ssh                            1:7.9p1-10                  all          secure shell client and server (metapackage)
ii  ssh-import-id                  5.7-1                       all          securely retrieve an SSH public key and install it locally
ii  strace                         4.26-0.2                    armhf        System call tracer
ii  sudo                           1.8.27-1                    armhf        Provide limited super user privileges to specific users
ii  systemd                        241-5+rpi1                  armhf        system and service manager
ii  systemd-sysv                   241-5+rpi1                  armhf        system and service manager - SysV links
ii  sysvinit-utils                 2.93-8                      armhf        System-V-like utilities
ii  tar                            1.30+dfsg-6                 armhf        GNU version of the tar archiving utility
ii  tasksel                        3.53                        all          tool for selecting tasks for installation on Debian systems
ii  tasksel-data                   3.53                        all          official tasks used for installation of Debian systems
ii  traceroute                     1:2.1.0-2                   armhf        Traces the route taken by packets over an IPv4/IPv6 network
ii  triggerhappy                   0.5.0-1                     armhf        global hotkey daemon for Linux
ii  tzdata                         2019a-1                     all          time zone and daylight-saving time data
ii  ucf                            3.0038+nmu1                 all          Update Configuration File(s): preserve user changes to config files
ii  udev                           241-5+rpi1                  armhf        /dev/ and hotplug management daemon
ii  unzip                          6.0-23                      armhf        De-archiver for .zip files
ii  usb-modeswitch                 2.5.2+repack0-2             armhf        mode switching tool for controlling "flip flop" USB devices
ii  usb-modeswitch-data            20170806-2                  all          mode switching data for usb-modeswitch
ii  usb.ids                        2019.04.23-1                all          USB ID Repository
ii  usbutils                       1:010-3                     armhf        Linux USB utilities
ii  util-linux                     2.33.1-0.1                  armhf        miscellaneous system utilities
ii  v4l-utils                      1.16.3-3                    armhf        Collection of command line video4linux utilities
ii  vim-common                     2:8.1.0875-5                all          Vi IMproved - Common files
ii  vim-tiny                       2:8.1.0875-5                armhf        Vi IMproved - enhanced vi editor - compact version
ii  wget                           1.20.1-1.1                  armhf        retrieves files from the web
ii  whiptail                       0.52.20-8                   armhf        Displays user-friendly dialog boxes from shell scripts
ii  wireless-regdb                 2018.05.09-0~rpt1           all          wireless regulatory database
ii  wireless-tools                 30~pre9-13                  armhf        Tools for manipulating Linux Wireless Extensions
ii  wpasupplicant                  2:2.7+git20190128+0c1e29f-6 armhf        client support for WPA and WPA2 (IEEE 802.11i)
ii  xauth                          1:1.0.10-1                  armhf        X authentication utility
ii  xdg-user-dirs                  0.17-2                      armhf        tool to manage well known user directories
ii  xkb-data                       2.26-2                      all          X Keyboard Extension (XKB) configuration data
ii  xxd                            2:8.1.0875-5                armhf        tool to make (or reverse) a hex dump
ii  xz-utils                       5.2.4-1                     armhf        XZ-format compression utilities
ii  zlib1g:armhf                   1:1.2.11.dfsg-1             armhf        compression library - runtime
ii  zlib1g-dev:armhf               1:1.2.11.dfsg-1             armhf        compression library - development

↧

Federal Debt Over Time

August 6, 2020, 11:00 pm

≫ Next: US Median Income Over Time

≪ Previous: List of Default Packages on Raspbian Buster Lite

importurllib.requestfrompathlibimportPathimportaltairasaltimportpandasaspd

url="https://fiscaldata.treasury.gov/static-data/HstDebt_all_years.csv.zip"local_file=Path(Path(url).name)ifnotlocal_file.exists():withurllib.request.urlopen(url)asreq:resp=req.read()local_file.write_bytes(resp)

df=pd.read_csv(local_file,parse_dates=["Record Date"])

df.head(5)

	Record Date	Debt Outstanding Amount	Source Line Number	Fiscal Year	Fiscal Quarter Number	Calendar Year	Calendar Quarter Number	Calendar Month Number	Calendar Day Number
0	1790-01-01	71060508.50	1	1790	2	1790	1	1	1
1	1791-01-01	75463476.52	1	1791	2	1791	1	1	1
2	1792-01-01	77227924.66	1	1792	2	1792	1	1	1
3	1793-01-01	80358634.04	1	1793	2	1793	1	1	1
4	1794-01-01	78427404.77	1	1794	2	1794	1	1	1

alt.Chart(df).mark_line().encode(x=alt.X("Fiscal Year",axis=alt.Axis(format=("d"))),y="Debt Outstanding Amount",)

↧

US Median Income Over Time

October 30, 2020, 9:48 am

≫ Next: Setting up Tasmota (beta) integration with Home Assistant

≪ Previous: Federal Debt Over Time

Bottom Line: A brief dicussion of median vs mean and look at SSA data on the US median wage.

Download this jupyter notebook in runnable formathere and view the post on my website here.

NB: There may be (hopefully minor) edits that are out of sync between the blog post and the downloaded notebook.

# pip install git+https://github.com/n8henrie/nb_black
# This library automatically runs the `black` code formatter in jupyter cells.
# The n8henrie/nb_black fork is slightly modified to allow specifying a custom
# line length instead of using the default.importlab_blacklab_black.load_ipython_extension(get_ipython(),line_length=79)

US Median Wage Estimates, 1990-2019

I often hear that the average US citizen has an income of around $50,000 per year. I hadn’t previously seen where this number comes from, but recently I found that the Social Security Administration provides some pretty interesting data at their website.

For example, here is the US average wage index over time (displaying a local copy below):

I recently discovered that they provide a webapp showing a summary of the US W-2 reported income. Most interestingly, it includes binned data so one can look at the changes in specific income brackets over time, and additionally they provide an estimate of the median income as well.

An example URL for the webapp: https://www.ssa.gov/cgi-bin/netcomp.cgi?year=2003

Median vs mean

Something I hadn’t considered (for whatever reason) is that discussion on the state of economic affairs based solely on the average income likely misrepresents how most of the US is doing. The median income is probably more representative of what most of us have in mind.

As an example, if we have a population of 100 individuals whose income is normally distributed from 0 to 10 (choose whatever units you like), the mean and median tend to be similar:

importmatplotlib.pyplotaspltimportnumpyasnprng=np.random.default_rng()population=rng.normal(loc=5,scale=1.5,size=100)print(f"""First 10 values of population: {population[:10]}{np.median(population)=}{np.mean(population)=}
""")plt.hist(population,bins=20,range=[0,10])

First 10 values of population: [5.37663355 3.77759921 2.8321777  4.88866131 4.02549337 7.890585
 8.07219506 4.30035334 2.18794887 2.15074077]

np.median(population)=4.921754367304535
np.mean(population)=4.982034011567311

png

However, if we take just one member of the population and make their income an extreme outlier, the average is improved, but the median is not (assuming the outlier was selected from above the median to start with).

population.sort()initial_median=np.median(population)initial_mean=np.mean(population)

# Make the 49th richest (51st poorest) individual 100 times richerpopulation[51]*=100

print(f"""Average income before: {initial_mean}
Average income after: {np.mean(population)}

Median income before: {initial_median}
Median income after: {np.median(population)}""")

Average income before: 4.982034011567312
Average income after: 9.940573583180026

Median income before: 4.921754367304535
Median income after: 4.921754367304535

We see that the average income of the population has nearly doubled!

While that sounds great, the truth of the matter is that most of this population has not actually improved at all.

As a matter of fact, we could even have a situation where all but one lucky individual see a decrease in their income, but the average still goes up.

population[:51]*=0.95population[52:]*=0.95

print(f"""Average income before: {initial_mean}
Average income after: {np.mean(population)}

Median income before: {initial_median}
Median income after: {np.median(population)}""")

Average income before: 4.982034011567312
Average income after: 9.693976195516614

Median income before: 4.921754367304535
Median income after: 4.675666648939308

In this example, 99% of the population is 5% poorer than they were to start with, which is reflected by the dropping median income, but contrasts starkly with a near doubling of the average income of the population.

Hopefully this is old news to most readers, but I think it was worth briefly reviewing. Statistics matter! Especially when some people in our contry are worth hundreds of billions of dollars, while the official poverty rate is more than 10% – some 33 million people.

SSA data vs census data

Before going on to the SSA data, I wanted to point out that the SSA data below differs substantially from the data available from the US Census (and for fellow nerds, my understanding is that census.gov has a good API – I look forward to exploring their data soon!). For example, their report shows the 2019 median household income to be about $68,000 – much higher than the SSA numbers below.

It seems like some of the important differences to keep in mind include:

census includes reported data, the SSA is measured (W-2)
census is household data, SSA is individual
- one might expect 2-income households to be roughly twice as much on the census data as compared to SSA
AFAIK, SSA is only W-2 income, meaning that capital gains and income from other work will not be reflectedin SSA, but may be included in what is reported by individuls in the census data

With that out of the way, on to the numbers!

# main dependencies for the notebookimportpickleimportreimporttypingastfromurllib.requestimporturlopenimportaltairasaltimportpandasaspdfromsklearn.linear_modelimportLinearRegression# NB: you will also need `pyarrow` installed to store and read dataframes from
# feather format. You don't need to import it, but uncomment below to test if
# you have it available.
# import pyarrow

# imports and helper function to run doctestsimportcopyimportdoctestdeftestme(func):"""
    Automatically runs doctests for a decorated function.
    https://stackoverflow.com/a/49659927/1588795
    """globs=copy.copy(globals())globs.update({func.__name__:func})doctest.run_docstring_examples(func,globs,verbose=False,name=func.__name__)returnfunc

# Build up some regular expressions to parse out the data from the
# relevant paragraphsMONEY=r"\$[0-9,]+\.[0-9]{2}"YEAR=r"\d{4}"mean_re=re.compile(('The "raw" average wage, computed as net compensation divided by the 'f"number of wage earners, is {MONEY} divided by [0-9,]+, or "rf"({MONEY})\.").strip())median_re=re.compile(("By definition, 50 percent of wage earners had net compensation less ""than or equal to the <i>median</i> wage, which is estimated to be "rf"({MONEY}) for {YEAR}\.").strip())

@testmedefclean_number(string)->float:"""
    Turn a string of a dollar amount into a float.>>> clean_number("$123,456.78")
    123456.78
    """returnfloat(string.lstrip("$").replace(",",""))defget_year(year:int)->t.Dict[str,int]:"""
    Get the median and mean for `year` from SSA.gov.
    """withurlopen(f"https://www.ssa.gov/cgi-bin/netcomp.cgi?year={year}")asreq:resp=req.read().decode()clean_whitespace=" ".join(resp.split())median=median_re.search(clean_whitespace).group(1)mean=mean_re.search(clean_whitespace).group(1)return{"year":year,"median":clean_number(median),"mean":clean_number(mean),}

# If a feather file is available, load the dataframe from the local file.
# Otherwise, query ssa.gov for 1990-2019, create a dataframe, and make a
# local copy in feather format to prevent unnecessary http requests.try:df=pd.read_feather("20201022_ssa_median_income.feather").set_index("year")exceptFileNotFoundError:df=(pd.DataFrame(get_year(year)foryearinrange(1990,2020)).set_index("year").sort_index())df.reset_index().to_feather("20201022_ssa_median_income.feather")

alt.Chart(df.reset_index().melt("year")).mark_line().encode(x="year:O",y=alt.Y("value",axis=alt.Axis(format="$,d",title="income $USD")),color=alt.Color("variable",legend=alt.Legend(title=None)),).properties(title="Source: SSA.gov").interactive()

# Make a mapping of {year: dataframe} where the dataframe includes the entire
# chart of income levels. Again, once fetched, pickle the object locally to
# avoid unnecessary http requests. Please be familiar with security concerns
# when loading *untrusted* pickled data before running this block.try:withopen("income_dfs.pkl","rb")asf:income_dfs=pickle.load(f)exceptFileNotFoundError:income_dfs={}foryearinrange(1990,2020):tmp_df=pd.read_html(f"https://www.ssa.gov/cgi-bin/netcomp.cgi?year={year}")[-2]tmp_df.columns=tmp_df.columns.droplevel()tmp_df[["Aggregate amount","Average amount"]]=tmp_df[["Aggregate amount","Average amount"]].apply(lambdax:x.str.lstrip("$").str.replace(",","").astype(float))income_dfs[year]=tmp_dfwithopen("income_dfs.pkl","wb")asf:pickle.dump(income_dfs,f)

# Give an idea of the shape of this dataincome_dfs[2003].head()

	Net compensation interval	Number	Cumulativenumber	Percentof total	Aggregate amount	Average amount
0	$0.01 — 4,999.99	26312244	26312244	17.81198	5.361136e+10	2037.51
1	5,000.00 — 9,999.99	15231616	41543860	28.12296	1.125687e+11	7390.46
2	10,000.00 — 14,999.99	13262655	54806515	37.10107	1.651411e+11	12451.59
3	15,000.00 — 19,999.99	12733058	67539573	45.72066	2.225614e+11	17479.02
4	20,000.00 — 24,999.99	12034969	79574542	53.86769	2.703204e+11	22461.24

# Show that the bins from 1997 - 2019 are the same, but differ than 1996 and
# earlier((income_dfs[1996]["Net compensation interval"].eq(income_dfs[1997]["Net compensation interval"])).all(),(income_dfs[1997]["Net compensation interval"].eq(income_dfs[2019]["Net compensation interval"])).all(),)

(False, True)

# Filter out the $50,000,000 and greater columnmore_than_50_mil=(pd.DataFrame(pd.concat([income_dfs[y].iloc[-1,[0,1]],pd.Series(y,index=["year"])])foryinincome_dfs.keys()ify>1996).set_index("year").sort_index())

chart=(alt.Chart(more_than_50_mil.reset_index()).mark_point().encode(x="year:O",y="Number:Q",).properties(title="# of income > $50,000,000 over time"))coef=(chart.transform_regression("year","Number",params=True).mark_text().encode(x=alt.value(120),y=alt.value(50),text="coef:N",))rSquared=(chart.transform_regression("year","Number",params=True).mark_text().encode(x=alt.value(120),y=alt.value(75),text="rSquared:N",))line=chart.transform_regression("year","Number").mark_line()(chart+coef+rSquared+line).interactive()

The chart above shows the parameters for the regression, but they’re a little hard to read. The most pertinent seems like it would be the slope – the number of people per year that are moving into the >= $50,000,000 per year range. Lets build a little convenience function to make the slope easily accessible.

defget_slope(df:pd.DataFrame,column_name:str)->float:"""
    Returns the slope from a dataframe based on index and values from
    `column_name`>>> slope = pd.DataFrame({"A": [3, 5, 7]}).pipe(get_slope, "A")>>> np.isclose(slope, 2)
    True
    """lr=LinearRegression()lr.fit(df.index.values.reshape(-1,1),df[column_name],)returnlr.coef_[0]

slope=more_than_50_mil.pipe(get_slope,"Number")mean=more_than_50_mil["Number"].mean()print(f"""{slope=:.2f}
Average number of people in this group: {mean:.2f}
Group is increasing by about: {slope/mean:.2%} per year""")

slope=7.59
Average number of people in this group: 112.65
Group is increasing by about: 6.74% per year

Now lets look at the poorest of the poor.

less_than_5k=(pd.DataFrame(pd.concat([income_dfs[y].iloc[0,[0,1]],pd.Series(y,index=["year"])])foryinincome_dfs.keys()ify>1996).set_index("year").sort_index())

chart=(alt.Chart(less_than_5k.reset_index()).mark_point().encode(x="year:O",y="Number:Q",).properties(title="Less than $5k"))coef=(chart.transform_regression("year","Number",params=True).mark_text().encode(x=alt.value(120),y=alt.value(150),text="coef:N",))rSquared=(chart.transform_regression("year","Number",params=True).mark_text().encode(x=alt.value(120),y=alt.value(175),text="rSquared:N",))line=chart.transform_regression("year","Number").mark_line()(chart+coef+rSquared+line).interactive()

slope=less_than_5k.pipe(get_slope,"Number")mean=less_than_5k["Number"].mean()print(f"""{slope=:.2f}
Average number of people in this group: {mean:.2f}
Group is decreasing by about: {-slope/mean:.2%} per year""")

slope=-357146.05
Average number of people in this group: 24614597.09
Group is decreasing by about: 1.45% per year

income_dfs[2019].iloc[-1]["Average amount"]/income_dfs[2019].iloc[0]["Average amount"]

42460.35824745949

Summary

The SSA provides some interesting an accessible data on income in the US. The median income reported by the SSA is way lower than I thought it was, and much lower than the household-level data reported by the census.

The number of Americans making more than $50 million per year is low, and is increasing by about 6% per year (small denominator, this is less than 8 people). Each of them make as much per year as about 42,000 people in the lowest bracket combined. The count of people in the lowest bracket is decreasing by less than 2% per year. Hopefully this represents improvement in their standard of living, though I suppose it could also be people dropping out of the workforce completely.

↧

Results

Script and Output

The Problem

How to Access these Files

Ongoing Issues in Automating the Process

Update 20181121

Further Reading

Part 1: Introduction and Serial Debug Port to Get WiFi Password

Part 2: Reading Flash to Get Telnet Password

Part 3: Using WireShark to Decode UDP Protocol

Part 4: Using Netcat to Decode Video Protocol

US Median Wage Estimates, 1990-2019

Median vs mean

SSA data vs census data

Summary