I had an opportunity to work with OSM data (nowhere close to something like FlightAware would need) at the beginning of the year and this comment put into words something I felt deeply at that time — "OSM is amazing, but as a global project with thousands of contributors, its data can be... quirky.". I have tons respect for this kind of stuff after working on that project and seeing the massive amount of time that gets invested to make things look correct for the use. It's the quintessential if nobody notices then you've done your job extremely well.
O/T but flightaware leaked users data for years and tried to get away with not letting anyone know, use anything else. adsbexchange is better in every way
I also prefer adsb exchange. But one thing other flight tracking apps are better at is displaying departure and destination airports of commercial flights.
I flew United a few days ago and noticed the change in the flight map: you can now see individual aircraft moving about airports, which is awesome. What's less awesome is that the marker of the aircraft you're most interested in, namely the one you're sitting on, is not aligned with the map when in detailed view, and unlike the Flightaware app, you can't get info about other airplanes (operator, flight number, destination, etc) by tapping on them.
Even more annoyingly, during takeoff and landing, the flight map system intentionally disables any way to see your precise location, altitude, velocity and direction. The "cockpit view" pane that shows these when cruising becomes unavailable, and the airplane marker is also hidden beyond a certain zoom level. Presumably this is some kind of misguided security by obscurity measure to stop terrorists, who apparently aren't smart enough to carry mobile phones with GPS receivers.
It’s great to see Apache Baremaps being mentioned. It’s a great project and I saw its first iterations. Really amazing they have built a community around it.
Although my library in Apache Baremaps probably plays a minor role only (PgBulkInsert for Postgres COPY protocol), it’s great to see it chugging on all this data day by day.
A concern I have for modern day mapping systems is that digital maps, like Google Maps and OpenStreetMap (OSM), typically operate on the assumption of a static, rigid Earth. I don’t know much about it, but from what I understand, they use coordinate systems like WGS84, which are a snapshot in time. These would be great for local precision but less so for tectonic precision, due to tectonic plate activity (continental drift), post-glacial rebound (areas still rising after ice sheets melted), and sudden seismic events (earthquakes).
And then there’s GPS coordinate shift. From what I read, ITRF, ETRS89, and coordinates associated with epoch dates attempt to deal with this.
So, even though it may not matter as much for FlightAware maps, autonomous and GPS-based systems are a little worrying. Being overly dependent on them may have some be risk over time.
OSM is a community of predominantly amateurs/enthusiasts who trace uncorrected satellite photos and store the data as WGS-84 coordinates with 7-digit precision.
The point I'm trying to make is that there are more important sources of error before you get to tectonic movement and GPS drift... And OSM is plenty useful even without outstanding precision.
> The point I'm trying to make is that there are more important sources of error before you get to tectonic movement and GPS drift...
You can absolutely measure tectonic drift on the OSM maps! They've existed long enough for it to be actually significant in a lot of places if you download the old data.
This also comes up all the time when trying to overlap data from local agencies onto the OSM maps. You end up with parcel boundaries visibly off.
> OSM uses a hundred nanodegrees as the grid resolution
As I said, 7-digit precision.
> You can absolutely measure tectonic drift on the OSM maps! They've existed long enough for it to be actually significant in a lot of places if you download the old data.
>
> This also comes up all the time when trying to overlap data from local agencies onto the OSM maps. You end up with parcel boundaries visibly off.
Yeah, assuming the person who added the features you're observing as 'visibly off' did not use 1-meter Bing imagery with a 10-meter offset...
This is not the correct way to count digit precision, which should be independent of the units used. If there were 360,000 degrees in a circle would this same precision suddenly be 4 digits? If we measured in radians would the digit precision become 9? Of course not…
Given the 100ndeg precision is across the full earth, this would be 1 part per 3.6 billion or 9.5ish digits of precision. The location of the decimal point when displaying it is irrelevant.
What also needs to be taken into account is that real world longitudinal distance changes by the cosine of latitude. 1 degree of longitude is 111km at the equator and 19km at 80 degrees.
In GIS the gold standard for positional accuracy is the Root Mean Square Error, measured in real world distance units.
Whenever I’ve done things on OSM it’s based on gps recorded tracks - no doubt not particularly accurate but there are many overlaid ones which average out to be fairly precise (1mish) for the purpose.
Oh yeah, I forgot about the enthusiasts that use uncorrected GPS positioning with low-cost hardware. One meter precision at the very best of conditions...
Defcon had a great talk on all the different navigational systems pilots can use and a note at the end that these shouldn't be decommissioned at the rate they're experiencing atm https://www.youtube.com/watch?v=wSVdfOn737o
This is a real problem for applications that need to deal with precise locations. For example, if you want to estimate the roof sizes using the aerial imagery and the LIDAR point clouds (e.g. the one from the USGS: https://usgs.entwine.io/data/view.html?r=%22https://s3-us-we... ).
The coordinates do not align precisely in many locations, exactly because of the actual motion of the Earth's surface. Tectonic plates, aquifer depletion, land sliding down the mountain slopes, etc. For practical applications, there are steps in the data processing to fit the different datasets together ("registering" one of them). As long as you have timestamped maps, you can reasonably reconstruct the current WGS84 coordinates by fitting the data together.
As geodetic problems go, though, this is trivial small beer stuff compared to, say, stitching together magnetic maps measured on different days or gathered in flight in different flight directions, or normalising continental scale radiometric datasets gathered across decades.
Modern digital post WGS84 mapping is a breeze compared to the days of dealing with chains under tension and stitching together across differing ellipsoids and datums.
> These would be great for local precision but less so for tectonic precision, due to tectonic plate activity (continental drift), post-glacial rebound (areas still rising after ice sheets melted), and sudden seismic events (earthquakes).
This is exactly what 4D deformation models are for:
For a while I worked on a project related to mapping the ancient world (so you could e.g. click on a ancient city and then see items from museums from that city, references in ancient texts etc) and one of the interesting problems was not just that cities changed over time but that things like coastlines also did.
I had an opportunity to work with OSM data (nowhere close to something like FlightAware would need) at the beginning of the year and this comment put into words something I felt deeply at that time — "OSM is amazing, but as a global project with thousands of contributors, its data can be... quirky.". I have tons respect for this kind of stuff after working on that project and seeing the massive amount of time that gets invested to make things look correct for the use. It's the quintessential if nobody notices then you've done your job extremely well.
O/T but flightaware leaked users data for years and tried to get away with not letting anyone know, use anything else. adsbexchange is better in every way
I also prefer adsb exchange. But one thing other flight tracking apps are better at is displaying departure and destination airports of commercial flights.
I flew United a few days ago and noticed the change in the flight map: you can now see individual aircraft moving about airports, which is awesome. What's less awesome is that the marker of the aircraft you're most interested in, namely the one you're sitting on, is not aligned with the map when in detailed view, and unlike the Flightaware app, you can't get info about other airplanes (operator, flight number, destination, etc) by tapping on them.
Even more annoyingly, during takeoff and landing, the flight map system intentionally disables any way to see your precise location, altitude, velocity and direction. The "cockpit view" pane that shows these when cruising becomes unavailable, and the airplane marker is also hidden beyond a certain zoom level. Presumably this is some kind of misguided security by obscurity measure to stop terrorists, who apparently aren't smart enough to carry mobile phones with GPS receivers.
It’s great to see Apache Baremaps being mentioned. It’s a great project and I saw its first iterations. Really amazing they have built a community around it.
Although my library in Apache Baremaps probably plays a minor role only (PgBulkInsert for Postgres COPY protocol), it’s great to see it chugging on all this data day by day.
A concern I have for modern day mapping systems is that digital maps, like Google Maps and OpenStreetMap (OSM), typically operate on the assumption of a static, rigid Earth. I don’t know much about it, but from what I understand, they use coordinate systems like WGS84, which are a snapshot in time. These would be great for local precision but less so for tectonic precision, due to tectonic plate activity (continental drift), post-glacial rebound (areas still rising after ice sheets melted), and sudden seismic events (earthquakes).
And then there’s GPS coordinate shift. From what I read, ITRF, ETRS89, and coordinates associated with epoch dates attempt to deal with this.
So, even though it may not matter as much for FlightAware maps, autonomous and GPS-based systems are a little worrying. Being overly dependent on them may have some be risk over time.
OSM is a community of predominantly amateurs/enthusiasts who trace uncorrected satellite photos and store the data as WGS-84 coordinates with 7-digit precision.
The point I'm trying to make is that there are more important sources of error before you get to tectonic movement and GPS drift... And OSM is plenty useful even without outstanding precision.
OSM uses a hundred nanodegrees as the grid resolution ( https://github.com/openstreetmap/OSM-binary/blob/32c3e921665... ). It gives a bit more than 1cm precision.
> The point I'm trying to make is that there are more important sources of error before you get to tectonic movement and GPS drift...
You can absolutely measure tectonic drift on the OSM maps! They've existed long enough for it to be actually significant in a lot of places if you download the old data.
This also comes up all the time when trying to overlap data from local agencies onto the OSM maps. You end up with parcel boundaries visibly off.
> OSM uses a hundred nanodegrees as the grid resolution
As I said, 7-digit precision.
> You can absolutely measure tectonic drift on the OSM maps! They've existed long enough for it to be actually significant in a lot of places if you download the old data. > > This also comes up all the time when trying to overlap data from local agencies onto the OSM maps. You end up with parcel boundaries visibly off.
Yeah, assuming the person who added the features you're observing as 'visibly off' did not use 1-meter Bing imagery with a 10-meter offset...
This is not the correct way to count digit precision, which should be independent of the units used. If there were 360,000 degrees in a circle would this same precision suddenly be 4 digits? If we measured in radians would the digit precision become 9? Of course not…
Given the 100ndeg precision is across the full earth, this would be 1 part per 3.6 billion or 9.5ish digits of precision. The location of the decimal point when displaying it is irrelevant.
What also needs to be taken into account is that real world longitudinal distance changes by the cosine of latitude. 1 degree of longitude is 111km at the equator and 19km at 80 degrees.
In GIS the gold standard for positional accuracy is the Root Mean Square Error, measured in real world distance units.
> As I said, 7-digit precision.
9 digits, because there are 2.5 digits of precision in 0-360 degrees left of the decimal point.
Whenever I’ve done things on OSM it’s based on gps recorded tracks - no doubt not particularly accurate but there are many overlaid ones which average out to be fairly precise (1mish) for the purpose.
Oh yeah, I forgot about the enthusiasts that use uncorrected GPS positioning with low-cost hardware. One meter precision at the very best of conditions...
I'm having a hard time tracking down the OSM-specific actions around this but Australia moved 1.8M back in 2017 due to continental shifting https://www.ga.gov.au/news/news-archive/australia-officially...
The Airport in Tartu, Estonia had a navigation upgrade last year in order to help mitigate navigation jamming that's taking place in the region. https://www.eans.ee/en/uudised/tanasest-saab-tartu-lennuvalj...
Defcon had a great talk on all the different navigational systems pilots can use and a note at the end that these shouldn't be decommissioned at the rate they're experiencing atm https://www.youtube.com/watch?v=wSVdfOn737o
This is a real problem for applications that need to deal with precise locations. For example, if you want to estimate the roof sizes using the aerial imagery and the LIDAR point clouds (e.g. the one from the USGS: https://usgs.entwine.io/data/view.html?r=%22https://s3-us-we... ).
The coordinates do not align precisely in many locations, exactly because of the actual motion of the Earth's surface. Tectonic plates, aquifer depletion, land sliding down the mountain slopes, etc. For practical applications, there are steps in the data processing to fit the different datasets together ("registering" one of them). As long as you have timestamped maps, you can reasonably reconstruct the current WGS84 coordinates by fitting the data together.
As geodetic problems go, though, this is trivial small beer stuff compared to, say, stitching together magnetic maps measured on different days or gathered in flight in different flight directions, or normalising continental scale radiometric datasets gathered across decades.
Modern digital post WGS84 mapping is a breeze compared to the days of dealing with chains under tension and stitching together across differing ellipsoids and datums.
> These would be great for local precision but less so for tectonic precision, due to tectonic plate activity (continental drift), post-glacial rebound (areas still rising after ice sheets melted), and sudden seismic events (earthquakes).
This is exactly what 4D deformation models are for:
https://proj.org/en/stable/operations/transformations/defmod...
For a while I worked on a project related to mapping the ancient world (so you could e.g. click on a ancient city and then see items from museums from that city, references in ancient texts etc) and one of the interesting problems was not just that cities changed over time but that things like coastlines also did.
Sounds super compelling, did it end up launching?
Where can I see a heatmap of how many flights went over in the last 24 hours?
This may interest you: https://adsb.exposed/
Discussion: https://news.ycombinator.com/item?id=39990346
Looks pretty, but I have difficulty pinning my location on the map and seeing how many flights actually went over in the last 24h.
(2024)