WeeWx and MySQL

Back in 2016, WeeWx murdered one of my many Raspberry Pi 3’s… all because it kept throwing loop and archive data bits into the database…. on the pi. Oops.

Here in 2019, as of this writing, the mySQL database has 1,130,399 rows logged, with various bits of data logged, totalling about 408MB. I was a numpty however, and was not logging the real time data properly since 2016… doh! That was fixed early-on in 2019. Realtime data table and associated rows is already huge – 7,130,258 rows huge, or about 1.4GB!!

Lesson learned however, SD Cards do not make for a good means of storage for databases… especially a database that constantly is reading and writing.

You’re literally and legitimately better off, right off the get go, forcing weewx to use a database servers that does NOT reside on a raspberry pi’s storage device… or any sort of device that uses a SD Card, it does a fair amount of reads and writes, and not just down to the database i/o, but image and web page generation!

Moving along, for me, I currently use mySQL for WeeWx – it gets the job done. On first start of weewx, it will always attempt to use a SQLite database file-ball on the pi, we don’t want that!

Assuming you’ve been following the direction of the official documentation at WeeWx.com ( http://www.weewx.com/docs/usersguide.htm#configuring_mysql ), let’s go ahead and force WeeWx to ignore that local sqlite database and use mySQL only, on another servers!

Load up /etc/weewx/weewx.conf in your favorite plain text editor (I use nano on the SSH terminal, and Notepad++ on desktop – depends on how much I want to self-torture my mind).

Added below is an example configuration segment to weewx.conf, this goes above and before the [Engine] clause. The mySQL Table in the SQL daemon is weewx, the IP or hostname needs to be set where it says IP.IP.IP.IP, same goes for the username and password.

     archive_interval = 60
     archive_delay = 1
     record_generation = hardware
     loop_hilo = True
     data_binding = wx_binding
         database = archive_mysql
         table_name = archive
         manager = weewx.wxmanager.WXDaySummaryManager
         schema = schemas.wview.schema
         database_type = mysql
         database_name = weewx
         driver = weedb.mysql
         host = IP.IP.IP.IP
         user = mySQL_WeeWx_UserName
         password = mySQL_WeeWx_PassWord

Note in the configuration segment above, there is absolutely NO mention of sqlite… anywhere. It’s just mySQL, on a remote server – as it should be.

Save the weewx.conf file, restart weewx, monitor the /var/log/messages or /var/log/syslog (tail -f /var/log/messages /var/log/syslog) files for any issues.

NOTE: Run the tail command from another terminal so you can continue doing your modifications on the original terminal window.

What makes WeeWx tick?

First off, WeeWx runs on a virtual machine with dedicated hardware, not a Raspberry Pi 3. The weewx installation was simply too taxing on the Raspberry Pi 3, to the point of it overheating… even with heatsinks on the chips that are known to get warm or hot!

The WeeWx Virtual Machine has the following specifications:

1. 4 vCPU’s from an Intel i7-6800k.
2. 1GB of vRAM (out of the 128GB the host has).
3. 20GB of Disk Space in SATA mode.
3a: NOTE: This ‘disk’ is stored on a NVMe drive – it’s ungodly fast.
4. Network Adapter: Bridged. So that it gets its own unique LAN DHCP IP.
5. USB Controller: Compatibility is set to 3.0, had funky issues with it at 2.0.
5a: You may need to manually connect the USB to the Virtual Machine. ESXi and Workstation grant the ability to auto-[re]connect.
5b: You should see the following USB device connected when inputting lsusb in your terminal window: Cygnal Integrated Products, Inc. CP210x UART Bridge / myAVR mySmartUSB light

What OS does WeeWx need to play nice?

At the time of writing, this is the operating system and kernel in use…

root@weewx:~# uname -a
Linux weewx 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1 (2019-04-12) x86_64 GNU/Linux

Broken down, it’s running Debian. Debian Stretch to be exact, which is version 9.9.

root@weewx:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 9.9 (stretch)
Release: 9.9
Codename: stretch

My Weather Station: A Synopsis

Here is a quick write up about my weather station. There is another article covering much more technicalities of WeeWx, BelcherTown Theme and it’s real time stuff.

You can view the weather station here: https://weewx.potatoforinter.net/

The physical weather station is a Davis Vantage Pro2+ Wireless. Bought it on Amazon for just under 1000 USD, 990 to be exact… but still, a deal is a deal when it comes to the actual list price of it just being under 1400 USD!! This weather station is effectively “the works” from Davis Instruments before jumping the price tier into the tried-and-true professional hardware used in the field.

What does this weather station monitor for, log, do, report?! It monitors and reports the following details: Air temperature, dew point, solar/sun intensity, UV Sensor, rain gauge, and anemometer (wind speed) with wind vane (direction). All of this is reported to the Davis Vantage Weather Console, the weather console further transfers data in a ‘loop packet’ and ‘one minute archives’ to a mySQL Database.

This weather station also reports to the National Weather Service via the MADIS network via the CWOP system. You can view current and some recent historical data here. It’s been reporting data since Fall 2016, and still going strong. In summer of 2018, the weather station was re-installed with a better foundation, support, and taller pole for the anemometer.

A bit more detail…

It’s worth noting that the weather station runs on WeeWx, http://www.weewx.com/

This software is effectively something that goes along the lines of “do all the things!”. It collects/polls momentary data from the Davis Console, it pulls in real time/loop data from the console, it outputs static and dynamic data to the web page linked at the top. It also stores all of the data in a mySQL database on another machine here in the house.

The more difficult portions of WeeWx was actually getting BelcherTown Template/Theme with the real time data flows functional! Just about everything else on WeeWx has been fairly simple and straightforward. More on that in another article!