HN2new | past | comments | ask | show | jobs | submitlogin

I have a feeling that weather data compresses together well — especially if stations 1. are ordered along a space-filling curve geographically, with read-outs of geographically-neighbouring stations in sequence; and 2. are stored column-oriented where each future time is its own column. Basically the same optimizations as you would use to store any other IoT time-series sample-cloud data.

I suspect that, encoded this way, weather for 4000 locations wouldn’t be much bigger than weather for 1 location.

Also keep in mind that half the reason the Weather app needs to use as much bandwidth as it already does, is that whenever you move even slightly, it can no longer give you accurate info without grabbing the point-forecast again for your new location. It uses far less data if you just stay at home all day every day. If it had a pre-cache of forecasts for the entire local area, it wouldn’t need to do that; it could just refresh once every few hours. Maybe even wait for you to be on wi-fi before doing so, if it has modelled you as usually connecting to wi-fi several times daily.



I think the bigger issue is the update frequency. My weather app updates minute-by-minute, giving me essentially real-time precipitation information from my neighbourhood weather station. If each future time is a column, you're looking at 60 columns just for the next hour of forecast. It really adds up!

This level of detailed information can't be pre-cached, either. I believe they are feeding the real-time weather recordings from the weather station into their supercomputer model to generate minute-by-minute high resolution forecasts over the next 6 hours.


Look at the METAR data pilots use for an example of how easily weather data can be compressed.

KSEA 020153Z 34008KT 10SM FEW075 OVC090 18/14 A2997

Here's the local conditions at Sea-Tac at 7:15pm Pacific time. We have wind direction and speed, visibility, cloud conditions, temperature and dew point and barometric pressure in one small snippet.

The forecast data for the next 24 hours is given similarly. Radar data would be more complex but the basics for any weather app are only a handful of bytes.


I wonder how big METAR weather data and forecasts for the whole of the US would be to download?


I can't find the source but I believe there are about 4000 METAR sites. There are plenty more small airports that will issue a weather report but they are pulling from the closest site that actually records METAR data.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: