My WeeWx Setup

The configurations that is!

Some Notes Before Going to Town With Copy Pastes…

Please note that my setup for WeeWx, BelcherTown Weather Theme / Template, the web servers, and database are not a normal method of serving and storing data. Many configuration options listed below will need to be modified to suit your own installation.

This particular configuration below uses three machines, two being a virtual machine, and one being dedicated. The dedicated machine is for the SQL databases. The two virtual machines are for 1) the nginx reverse proxy / load balancer and 2) the weewx VM instance with mosquitto, weewx, nginx.

As always, like most other online posted guides, many configuration settings are of a non-normal or non-typical installations. Which may pose a security risk if exposed to the internet without proper configurations set.

Web Server

The web server and reverse proxy runs on nginx, both running on version 1.10.3. Both nginx servers (external and internal) listen on port 80, while the internal nginx server that hosts the actual weather weewx website solely listens on 80, and not 443 (SSL). The external nginx server (a basic load balancer in a manner of speaking), listens on port 443 and 80. Port 80 is always auto forwarded to 443 (SSL). Otherwise the WeeWx Belchertown Websockets will traditionally fail to function!

Database

The SQL server runs on MariaDB (a fork of mySQL), running version 10.3.14. The server is behind a software firewall, only permitting select IP addresses on LAN to port 3306. The mySQL server bind-address is 0.0.0.0 so that it permits of outside traffic – providing a login method is given. The SQL server runs on its own hardware, split away from nginx and weewx machines.

Weather Station

Above all else, the weather station hardware. For any of this to work via a Davis weather station, you will need to obtain a WeatherLink USB Data Logger – it sells for 165 USD, but you can likely get it far cheaper from other online retailers, such as Amazon for 140 to 125 USD or Scientific Sales Inc for 105 to 115 USD, another site I’ve used is Scaled Instruments. They currently sell the 6510USB for 117 USD. Regardless, without this little piece of hardware, you’ll have a hard-pressed time getting data fed to WeeWx from a Davis Instruments console!

Additionally, there’s been developments from 3rd parties for WeeWx and RTL-SDR receivers/dongles. This would completely strip out the need for the weather console USB bridge. Granted, I’ve never attempted to use this extension on WeeWx, so I cannot vouch for data quality, stability, or functionality. If you’re wanting to dredge into uncharted waters, you can view that project here: https://github.com/matthewwall/weewx-sdr

WeeWx configuration

WeeWx for me has been modified a slight amount, in addition to many add ons / extensions installed and configured over the years. As of WeeWx 3.9.1, this is what my weewx.conf file looks like.

Currently, the following extensions are installed…

Extension Name    Version   Description
cmon              0.16      Collect and display computer health indicators.
Hfw               0.2.1     weewx support for plotting observational data using Highcharts.
mqtt              0.19      Upload weather data to MQTT server.
neowx             1.0       The most advanced weewx skin.
Belchertown       0.9.1     A clean modern skin with real time streaming updates and interactive charts. Modeled after BelchertownWeather.com

cmon, and neowx are no longer proactively used. As cmon had stability issues for me, and neowx was in use before Belchertown skin.

NOTE: WeeWx loop data is also being dumped to the mySQL server in a second database. You’ll see it listed below as weewx_rapid. You can find details about WeeWx’s Raw script here.

debug = 0
WEEWX_ROOT = /etc/weewx
log_success = True
log_failure = True
socket_timeout = 20
version = 3.9.1
[Station]
	location = "Olathe Kansas"
	latitude = 38.809253
	longitude = -94.781939
	altitude = 1068, foot
	station_type = Vantage
	station_url = https://weewx.potatoforinter.net
	rain_year_start = 1
	week_start = 6
[StdRESTful]
	[[StationRegistry]]
		register_this_station = false
	[[CWOP]]
		enable = true
		station = CWOPStationIDHere
	[[Wunderground]]
		enable = true
		station = WunderGroundStationIDHere
		password = ReplaceMeWithAPassword
		rapidfire = True
	[[MQTT]]
		server_url = mqtt://weewx:ReplaceMeWithAPassword@localhost:1883/
		topic = weather
		unit_system = US
		binding = archive, loop
		aggregation = aggregate
[Vantage]
	type = serial
	port = /dev/ttyUSB0
	baudrate = 19200
	iss_id = 1
	timeout = 4
	wait_before_retry = 1.2
	max_tries = 4
	model_type = 2
	driver = weewx.drivers.vantage
[StdReport]
	SKIN_ROOT = /etc/weewx/skins/
	HTML_ROOT = /var/www/html/
	data_binding = wx_binding
	report_timing = * * * * *
	log_success = True
	log_failure = True
	skin = Belchertown
	[[Highcharts_Belchertown]]
		HTML_ROOT = /var/www/html/
		skin = Highcharts_Belchertown
	[[Belchertown]]
		HTML_ROOT = /var/www/html/
		skin = Belchertown
	[[[Extras]]]
		belchertown_root_url = "https://weewx.potatoforinter.net"
		site_title = "Southern Olathe KS Weather"
		forecast_enabled = 0
		earthquake_enabled = 0
		twitter_enabled = 0
		twitter_owner = bctrainers
		mqtt_websockets_enabled = 1
		mqtt_websockets_host = "weewx.potatoforinter.net"
		mqtt_websockets_port = 2884
		mqtt_websockets_ssl = 1
		mqtt_websockets_topic = "weather/loop"
		disconnect_live_website_visitor = "9600000"
	[[Defaults]]
		[[[Units]]]
			[[[[Groups]]]]
				group_altitude = foot
				group_degree_day = degree_F_day
				group_pressure = inHg
				group_rain = inch
				group_rainrate = inch_per_hour
				group_speed = mile_per_hour
				group_speed2 = mile_per_hour2
				group_temperature = degree_F
			[[[[StringFormats]]]]
				centibar = %.0f
				cm = %.2f
				cm_per_hour = %.2f
				degree_C = %.1f
				degree_F = %.1f
				degree_compass = %.0f
				foot = %.0f
				hPa = %.1f
				hour = %.1f
				inHg = %.3f
				inch = %.2f
				inch_per_hour = %.2f
				km_per_hour = %.0f
				km_per_hour2 = %.1f
				knot = %.0f
				knot2 = %.1f
				mbar = %.1f
				meter = %.0f
				meter_per_second = %.1f
				meter_per_second2 = %.1f
				mile_per_hour = %.0f
				mile_per_hour2 = %.1f
				mm = %.1f
				mmHg = %.1f
				mm_per_hour = %.1f
				percent = %.0f
				second = %.0f
				uv_index = %.1f
				volt = %.1f
				watt_per_meter_squared = %.0f
				NONE = "   N/A"
			[[[[Labels]]]]
				day = " day", " days"
				hour = " hour", " hours"
				minute = " minute", " minutes"
				second = " second", " seconds"
				NONE = ""
			[[[[TimeFormats]]]]
				hour = %H:%M
				day = %X
				week = %X (%A)
				month = %x %X
				year = %x %X
				rainyear = %x %X
				current = %x %X
				ephem_day = %X
				ephem_year = %x %X
			[[[[Ordinates]]]]
				directions = N, NNE, NE, ENE, E, ESE, SE, SSE, S, SSW, SW, WSW, W, WNW, NW, NNW, N/A
				[[[[[DegreeDays]]]]]
					heating_base = 65, degree_F
					cooling_base = 65, degree_F
				[[[[[Trend]]]]]
					time_delta = 10800
					time_grace = 300
		[[[Labels]]]
			hemispheres = N, S, E, W
			latlon_formats = %02d, %03d, %05.2f
			[[[[Generic]]]]
				barometer = Barometer
				dewpoint = Dew Point
				ET = ET
				heatindex = Heat Index
				inHumidity = Inside Humidity
				inTemp = Inside Temperature
				outHumidity = Humidity
				outTemp = Outside Temperature
				radiation = Radiation
				rain = Rain
				rainRate = Rain Rate
				UV = UV Index
				windDir = Wind Direction
				windGust = Gust Speed
				windGustDir = Gust Direction
				windSpeed = Wind Speed
				windchill = Wind Chill
				windgustvec = Gust Vector
				windvec = Wind Vector
				extraTemp1 = Temperature1
				extraTemp2 = Temperature2
				extraTemp3 = Temperature3
				rxCheckPercent = Signal Quality
				txBatteryStatus = Transmitter Battery
				windBatteryStatus = Wind Battery
				rainBatteryStatus = Rain Battery
				outTempBatteryStatus = Outside Temperature Battery
				inTempBatteryStatus = Inside Temperature Battery
				consBatteryVoltage = Console Battery
				heatingVoltage = Heating Battery
				supplyVoltage = Supply Voltage
				referenceVoltage = Reference Voltage
		[[[Almanac]]]
			moon_phases = New, Waxing crescent, First quarter, Waxing gibbous, Full, Waning gibbous, Last quarter, Waning crescent
[StdConvert]
	target_unit = US
[StdCalibrate]
	[[Corrections]]
#foo = foo + 0.2
[StdQC]
	[[MinMax]]
		barometer = 26, 32.5, inHg
		pressure = 24, 34.5, inHg
		outTemp = -40, 120, degree_F
		inTemp = 10, 120, degree_F
		outHumidity = 0, 100
		inHumidity = 0, 100
		windSpeed = 0, 120, mile_per_hour
		rain = 0, 10, inch
[StdWXCalculate]
	[[Calculations]]
		pressure = prefer_hardware
		barometer = prefer_hardware
		altimeter = prefer_hardware
		windchill = prefer_hardware
		heatindex = prefer_hardware
		dewpoint = prefer_hardware
		inDewpoint = prefer_hardware
		rainRate = prefer_hardware
[StdTimeSynch]
	clock_check = 14400
	max_drift = 2
[Raw]
	data_binding = raw_binding
	data_limit = 0
[StdArchive]
	archive_interval = 60
	archive_delay = 1
	record_generation = hardware
	loop_hilo = True
	data_binding = wx_binding
[DataBindings]
	[[wx_binding]]
		database = archive_mysql
		table_name = archive
		manager = weewx.wxmanager.WXDaySummaryManager
		schema = schemas.wview.schema
	[[raw_binding]]
		database = raw_mysql
		table_name = raw
		manager = weewx.manager.Manager
		schema = user.raw.schema
[Databases]
	[[archive_mysql]]
		database_type = mysql
		database_name = weewx
	[[raw_mysql]]
		database_type = mysql
		database_name = weewx_rapid
[DatabaseTypes]
	[[mysql]]
		driver = weedb.mysql
		host = IP.IP.IP.IP
		user = ReplaceMeWithAUserName
		password = ReplaceMeWithAPassword
[Engine]
	[[Services]]
		prep_services = weewx.engine.StdTimeSynch
		data_services = ,
		process_services = weewx.engine.StdConvert, weewx.engine.StdCalibrate, weewx.engine.StdQC, weewx.wxservices.StdWXCalculate
		archive_services = weewx.engine.StdArchive, user.raw.RawService
		restful_services = weewx.restx.StdStationRegistry, weewx.restx.StdWunderground, weewx.restx.StdPWSweather, weewx.restx.StdCWOP, weewx.restx.StdWOW, weewx.restx.StdAWEKAS, user.mqtt.MQTT
		report_services = weewx.engine.StdPrint, weewx.engine.StdReport

As you can see above, I have weewx set to do the following…

  1. Report data to CWOP, send ‘rapidfire’ data to WUnderground, spam MQTT (WebSockets) with loop data – all the while saving to mySQL.
  2. Force an update once every minute to the webpage with the report_timing clause being * * * * *, in the event websockets decide to go on a coffee break.
  3. Using the Raw python script, dumping loop data to weewx_rapid.
    • You can make extremely granular graphs with this sort of data!
    • …At the expense of a massive database size.
  4. Using the same mySQL user : pass credentials for both databases.

You will need to change IP.IP.IP.IP with your servers mySQL database IP address. Note this however: If your mySQL server is on the same machine as weewx, use localhost, not 127.0.0.1 as the host. localhost is a socket, while 127.0.0.1 is TCP/IP. While miniscule, TCP/IP has overhead, while localhost is a socket connection.

You will need to change ReplaceMeWithAPassword and ReplaceMeWithAUserName in their respective locations.

So that’s WeeWx’s initial configuration out of the way… time for the nginx configuration!

As a reminder, I use nginx in two places. The front-end being a load-balancer of sorts – a reverse proxy, which delegates a hostname to an internal IP and sends the data back to the client. Please keep this in mind when you see the two configurations below.

The Reverse Proxy / Front-End configuration…

For WebSockets to function on the internet, and without losing my sanity, while passing WebSockets thru nginx to the mosquitto / mqtt daemon, open up port 2884 TCP on your firewall, and set its forwarded IP to the load balancer / reverse-proxy nginx server.

Additionally, there are two variables to set to forward requests to internally. One called $upstream and the other called appserver. These are both fairly easy to spot in the config. You’ll need to set both of these to the appropriate IP address inside your LAN, it’ll likely be the same IP address. The port on appserver will need to remain however, otherwise WebSockets will fail.

The SSL cert uses Let’s Encrypt – it’s free, it works, it gets you a nifty padlock icon on your site to “show it’s secure”. For the LE SSL cert, it uses a wildcard + domain style. It makes most browsers happy. This will be covered after the nginx sections.

server {
 listen 80;
 server_name weewx.potatoforinter.net;
 return 301 https://$server_name$request_uri;
}

server {
 access_log /var/log/nginx/weewx.access.log;
 error_log /var/log/nginx/weewx.error.log;
 listen 443 ssl http2;
 server_name weewx.potatoforinter.net;

 underscores_in_headers on;

    include /etc/nginx/ssl.conf;

 set $upstream 192.168.20.203;

 location / {
 proxy_headers_hash_max_size 512;
 proxy_headers_hash_bucket_size 64;
 proxy_set_header Host $host;
 proxy_set_header X-Forwarded-Proto $scheme;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 add_header Front-End-Https on;
 proxy_pass http://$upstream;
 proxy_buffering off;
 }
}

map $http_upgrade $connection_upgrade {
    default upgrade;
    '' close;
}

upstream appserver {
 server 192.168.20.203:9001;
}
server {
        server_name weewx.potatoforinter.net;
        listen 2884;

        ssl on;
        ssl_certificate      /etc/letsencrypt/live/potatoforinter.net/fullchain.pem;
        ssl_certificate_key  /etc/letsencrypt/live/potatoforinter.net/privkey.pem;
        location / {
                proxy_pass https://appserver;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection $connection_upgrade;
                proxy_read_timeout 86400;
            }
    }

The Backend (where weewx dumps static data to)…

Since this server doesn’t need any real vhost assigned, as it’s a stand-alone server, the following configuration is sufficient enough. This file is just a replacement to the file called default within /etc/nginx/sites-available/.

server {
     listen 80 default_server;
     root /var/www/html/;
     index index.php index.html;
     server_name _ weewx.potatoforinter.net;
     location / {
         try_files $uri $uri/ =404;
     }
     location ~ .php$ {
         include snippets/fastcgi-php.conf;
         fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
     }
 }

Certbot aka Let’s Encrypt and config’s…

So this is where things do get rather tricky, as mosquitto is hosted on another machine while the nginx reverse proxy is on another. You can’t exactly go creating multiple SSL certs for the same domain, that would be a bit absurd!

TODO: Automate SSL Certs to other *nix machines via SCP.

This command is what I use to generate a SSL cert:
sudo certbot --server https://acme-v02.api.letsencrypt.org/directory -d *.potatoforinter.net,potatoforinter.net --manual --preferred-challenges dns-01 certonly -m ReplaceMe@WithAnEmailAddress.com

With this command, you’ll need to have your DNS editor at the ready. You’ll need to create a TEXT record. The text record will likely look something like this: _acme-challenge.potatoforinter.net. IN TXT "HcZGqj4UoNNIZzCBfmpQ48rsrerkJZdtV1uLjvqJC8-k4". The jumbled mass of text at the end is what CertBot will throw at you in the terminal to create as a TEXT record on your name servers.

For me, I host my DNS locally – so making changes are applied on the spot, with the TTL expiration dictating how often to refresh to a client.

To renew a CertBot/LE Cert, this command should do the trick: certbot renew --pre-hook "service nginx stop" --post-hook "service nginx start" – it’ll kill nginx, check LE ssl certs for renewal, restart nginx.

Once your certs have been made, your certs will traditionally (on debian!) end up in a folder at /etc/letsencrypt/live/ with the final folder being the domain name. For me, it was potatoforinter.net. Your SSL file within /etc/nginx/ssl.conf should look like this…

ssl on;
#Working SSL Dir: /etc/letsencrypt/live/potatoforinter.net/
ssl_certificate /etc/letsencrypt/live/potatoforinter.net/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/potatoforinter.net/privkey.pem;

ssl_stapling on;
ssl_stapling_verify on;
ssl_session_cache shared:le_nginx_SSL:1m;
ssl_ecdh_curve secp384r1;
ssl_dhparam /etc/nginx/ssl/dhparam.pem;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384;
ssl_session_timeout 10m;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
resolver 192.168.20.1 192.168.20.1 valid=300s;
resolver_timeout 5s;
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
add_header Referrer-Policy no-referrer;
add_header X-Content-Type-Options nosniff;
add_header X-Frame-Options SAMEORIGIN;
add_header X-XSS-Protection "1; mode=block";
add_header X-Robots-Tag none;
add_header X-Download-Options noopen;
add_header X-Permitted-Cross-Domain-Policies none;
add_header X-Download-Options noopen;
#add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://ssl.google-analytics.com; img-src 'self' https://ssl.google-analytics.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://themes.googleusercontent.com; frame-src https://potatoforinter.net; object-src 'none'";

That file is included within the server {} calls, as shown above using include /etc/nginx/ssl.conf; on the front-end reverse proxy.

I do not include the add_header Content-Security-Policy to this ssl file, but keep it there for reference, primarily due to NextCloud and it’s stringent SSL requests and the joys of copy-pasta.

Why didn’t I put include ssl.conf within the /etc/nginx/ssl/ folder? I dunno, didn’t feel like it?

Now, with all of that above set, it’s time for Mosquitto.

Mosquitto is a MQTT / WebSockets daemon. I found it to being the biggest PITA at first, as i didn’t quite understand how its configuration scheme functioned. I still don’t understand it to an extent, but i got it to work…. somehow.

Mosquitto will be installed on the weewx virtual machine, not the reverse proxy or database machine. It doesn’t use a lot of CPU time, nor does it use a lot of memory space. So it’s a perfect fit for the weewx virtual machine.

Assuming you’ve read this write up here:
https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/ – the following should function as is.

pid_file /var/run/mosquitto.pid
persistence false
persistence_location /var/lib/mosquitto/

log_type all
websockets_log_level 1023
connection_messages true
log_dest file /var/log/mosquitto/mosquitto.log

allow_anonymous true
password_file /etc/mosquitto/passwd
acl_file /etc/mosquitto/acl

listener 1883

listener 8883
cafile /etc/mosquitto/certs/chain1.pem
keyfile /etc/mosquitto/certs/privkey1.pem
certfile /etc/mosquitto/certs/cert1.pem

listener 9001
protocol websockets
http_dir /var/www/html
cafile /etc/mosquitto/certs/chain1.pem
keyfile /etc/mosquitto/certs/privkey1.pem
certfile /etc/mosquitto/certs/cert1.pem

log_type error
log_type warning
log_type notice
log_type information
#include_dir /etc/mosquitto/conf.d

That is the config file that resides at /etc/mosquitto/mosquitto.conf. I do not include the /etc/mosquitto/conf.d in the config, and it is commented out – as there’s no need for it. In the config, this is what is going on…

  1. I am listening on port 1883, 8883 and a websockets port on 9001.
  2. Port 1883 is where weewx is dumping it’s data to.
  3. Port 8883 is a MQTT SSL port.
  4. Port 9001 is a WebSockets SSL port.
  5. I am logging everything, because this tool is a pain to work with when it decides to stop working for whatever reason.

So where do I get those certs from for mosquitto?

Easy as 1 2 3 actually…maybe! If you’ve not closed out nginx’s SSH terminal window for the reverse proxy / front end yet, there are three key files to yoink from there and to upload to the three given locations as seen above. For me, direct the SSH window to: /etc/letsencrypt/archive/ in there, you will have a folder made for the SSL applied domain. For me, it’s just potatoforinter.net, as weewx resides at https://weewx.potatoforinter.net/

If you’re not in the mood to download and upload files via SFTP (SSH+SCP) or SCP for that matter, you can use these commands to view, select, paste. You will need two SSH windows active, one for the reverse proxy server and the other for the weewx VM. In the reverse-proxy nginx web server, you”ll begin with…

  1. On the reverse-proxy nginx server, cat chain1.pem
  2. Select the text, starting at the beginning of -----BEGIN CERTIFICATE----- and selecting to the end of -----END CERTIFICATE-----
  3. Left click the terminal window.
  4. Go to the other terminal window where mosquitto resides at.
  5. cd /etc/mosquitto/certs/
  6. nano chain1.pem
  7. Right click in the blank area to paste.
  8. CTRL O
  9. CTRL X
  10. Repeat the same for privkey1.pem and cert1.pem. Replacing chain1.pem from the example above to the two other respective file names.

Alternatively, it’d be much faster to just use FileZilla (or some sort of client that supports SFTP connections), logging in as root via sftp to both instances, navigating to both directories, downloading from one, and uploading to the other. Done.

Once all edits have been made…

Go ahead and restart all of the instances… that includes weewx, mosquitto, nginx on both machines.

Subscribe
Notify of
guest
1 Comment
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
trackback
21 June 2020 7:29 pm

[…] My WeeWx Setup […]