The data here is processed from NHTSA's FARS database. When someone dies in an accident, it gets input into a STATE reporting system, and FARS is manufactured by analyzing each state's individual record system. The feds consolidate all this data and publish a unified dataset annually. They say it's "a lot cheaper and just as good as collecting it themselves"
Additional errors are potentially produced from my own processing of the federal data, but those will be rooted out over time. Project being OS will hopefully help with that.
In fact, the author of the website "I mapped almost every.." might want to add references to the data sources for credibility and less of a "spam" look. How about an About page.
Not to sure how your backend is set up, but it looks like you're generating GeoJSON on the fly and the JSON serialization for this amount of data can be slow.
In typical HN fashion, I would suggest looking into using Tippecanoe to generate some vector tiles of the data and host that on S3. Then the DB can fall back to performing simple lookups via the accident ID. Filtering by time will need to move to the frontend, but that should be fine (if not, look into clustering the data at further zoom levels).
Hi Ben, briefly took another look this morning and I'm sorry for the hasty rec because I forgot Leaflet doesn't support vector tiles as easily as MapLibre/Mapbox.
If the bottle-neck is the DB and you haven't enabled the PostGIS extension, I'd look into that as the other commenter mentioned. If it's your server, then yes look into pre-generating all the tiles and hosting them elsewhere, but this might then mean switching over to MapLibre/Mapbox (which is what most companies end up using because of Leaflet's limitations).
For filtering on the frontend, I'd look into Expressions so you're not regenerating the GeoJSON collection every time (this causes a significant delay and flicker).
You could also take a look at PostGIS to do geospatial queries (if needed). And in the past I’ve used node-mapnik to render vector/image map tiles. You can indeed host and cache them effectively on S3.
You’ve had some other recommendations already but I’d suggest also looking into FlatGeobuf [0] for this use case. Have a look at the MapLibre example with a 12GB example [1]. You don’t need a server at all (unlike MBTiles) and will be able to load far more points at once than your current solution. Can easily generate it with QGIS/GDAL/PostGIS. Not sure what your plans are for the project but would be happy to donate some time to get something like that working.
Apologies for the shameless plug but do checkout https://geobase.app We built it with such use cases in mind and solves the scaling issues with geo data driven apps. We are in private beta but happy to give you early access if you hit us up.
I was trying out different time periods (comparing 2003-2013 to 2013-2023, to see a rough trend), and I noticed that every click to change the year by one seemed to generate a refresh. Perhaps it would help to lower the load if you change the date selector widget to only refresh after the date selector is closed.
