I wish AI leaders would fund more data archival projects, instead of hoarding it themselves. They could build a lot of goodwill, but the current state of the art is destroying old books and denying anyone access to them.
These archival projects have an enormous impact, though they are chronically underfunded. I wish there were a strong economic incentive to support them.
Bjartr
Anthropic just learned to the tune of a 1.5 billion dollar settlement that making available for human consumption, even internally, instead of hoarding for AI training exclusively is a fraught exercise.
oneneptune
Can you elaborate? I checked the news and couldn't find what you're referencing myself.
jakozaur
Two events: Anthropic did end up paying a $1.5B settlement for a case involving the use of pirated books. See Bartz v. Anthropic.
Second, apparently archiving content and providing it to others is risky business. It is often ruled as piracy, see Hachette v. Internet Archive. It went badly for Internet Archive, against common sense.
matkoniecz
> It went badly for Internet Archive, against common sense.
not really
they were operating under model when they remotely lended the same amount of books they had (effectively operating as a library)
then they started doing it without any limits whatsoever, which in fact is piracy and illegal
some people with common sense begged them to stop and stay in legal operating area
they refused, got (predictably) sued as publishers were unhappy already about remote library and (as predictably) lost case
ufocia
That's on current copyrights, not public domain works like these really old maps.
dr_dshiv
SourceLibrary.org for instance… how to make what is done there an economic value for AI companies with data for training?
arjie
Everything is "chronically underfunded". There has never been a thing in the world that was "well funded" because people's objectives increase with the amount of money they have. No one has ever said "Wikipedia has enough now" or "The Internet Archive doesn't really need any more money" or "Firefox has collected enough money from Google that it can now operate independently forever".
Besides, this so-called goodwill is an illusion. Opposition to AI in the PR space is entirely dominated by motivated reasoning. Attempting to satisfy people with random things is simply going to shift complaints to "take the commons, chop it up, and give it back to us"; "Should AI companies be the arbiters of what is worth preserving?"; and other such things. These kind of public-relation gains are entirely fictive and neither the supposed beneficiary nor the supposed benefactor is fooled by the theory.
"Oh so they suck up all our water and burn our planet down and we're supposed to be happy that they chop up people's books (that humans spent years creating) just so they can give us digested slop back?". Everyone knows that this sentence is waiting in the wings, and that the entire effort is clearly bait so that someone is foolish enough to pull a Google Books and get sued into oblivion for it.
And that's saying nothing about this so-called "hoarding" of things in a world where we pulp tons of these hoarded things every day and no one cares to scan them.
rcxdude
Hmmm, there are definitely people saying "Wikipedia has enough now". They have enough cash to keep the site going for a long time, but they still beg for donations.
Firefox is similar, people complain about Mozilla's side projects distracting from firefox development all the time, and that they could runa lot leaner if they just focused on the browser.
etdznots
How does wasting money on public goods deliver value to shareholders or help you build a doomsday machine?
okok3857
I started this project and couldn't be happier about this partnership with OSM US!
I was very excited to see this project. I went to the site and requested processing of the maps for my small town. It also inspired me to make a donation to OSM.
okok3857
Email sent, and thank you!
I should have mentioned this earlier, but folks can make direct tax-deductible donations to OldInsuranceMaps.net, which is now affiliated with OSM US's 501(c)(3) status.
Awesome project! Would definitely love to see more overlap with Allmaps. It’s pretty trivial to utilize centralized gcps for both server and client applications and maintain single sources of truth. I’ve been meaning to dive a bit deeper on an old project anyway, good excuse.
okok3857
This is something we need to add to the FAQ. There is some cross-compatibility already, in that OIM has endpoints to spit out Allmaps-compatible annotations from the database, e.g. https://oldinsurancemaps.net/iiif/resource/88127/. You can paste that right into Allmaps, which will in turn pull tiles from the Library of Congress IIIF server and warp them based on the annotation information. That's just the beginning though, more to come...
tharos47
If you like old maps you can check the french maps with historical layers (17xx, 1880, 1950, today) :
Seems like a very interesting challenge for automation of raster to vector scanning/software, and QC/checking that the raster-to-vector pipeline hasn't hallucinated something or joined two shapes together that should be discrete objects.
I would presume they're starting with very high res lossless raster scans and storing those in some sort of index, then making them available for different pipelines of processing?
okok3857
That's basically what's going on, but it's all a manual process so far. The goal is scoped really just to the actual georeferencing and mosaic creation, no post-processing or OCR is performed on the maps within this platform (people are certainly downloading the final rasters and doing that with them though).
ufocia
Would be great to non-destructively scan the lower layers and date code them at some point, if/when possible.
okok3857
Absolutely, this would be incredibly cool, and if they can do it with burnt ancient scrolls it should be a cakewalk with these... Even just shining very strong light behind the pages you can see all the old layers. However, most of the LoC collection is original copyright deposits, so much of what's on OIM doesn't have pasted-on slips.
lapetitejort
If you like OldInsuranceMaps, check out another OSM charter project, Yesterdays! Originally created by a team in Richmond, Virginia [0], with one more deployment covering the Inland Empire [1]. They plan to merge soon. Yesterdays imports Sanborn maps to help place historical photos as accurately as possible.
Having these maps under a charter means long-term maintenance isn't just one person's side project anymore.
hnvd8v0o2d
Been thinking about this for years
darvo31
Great resource. Fire insurance maps are goldmines for urban history research.
chirau
On visiting the actual oldinsurancemaps.net, I was met with:
"OldInsuranceMaps.net is a Charter Project of OpenStreetMap US that provides an open, online workflow for creating georeferenced layers of historical maps, with a special focus on making seamless mosaics from multi-page atlases in the Sanborn map collection at the Library of Congress."
What the hell does that mean? Anyone to ELI5?
maxerickson
Sanborn was an insurance adjacent company that made detailed paper maps of many towns in the US in the late 1800s and early 1900s.
This project is scanning those maps and adding the information that modern mapping software needs to show them alongside other geographic information. Where the old paper maps overlap, they work to cleanly stitch them together rather than showing the edges of the paper.
Later: The project is sort of for people looking for the old maps, so I can see why the introduction doesn't spend a life of time explaining them.
Rygian
ELI5:
They are building computer programs that other people can use to then post online maps from old paper sources. For example, from the Sanborn map collection.
Those maps can be browsed as "overlays" on top of other maps, so you can easily compare how several maps show the same places on Earth.
lmc
They make a nice way of stitching together old maps.
These archival projects have an enormous impact, though they are chronically underfunded. I wish there were a strong economic incentive to support them.
Second, apparently archiving content and providing it to others is risky business. It is often ruled as piracy, see Hachette v. Internet Archive. It went badly for Internet Archive, against common sense.
not really
they were operating under model when they remotely lended the same amount of books they had (effectively operating as a library)
then they started doing it without any limits whatsoever, which in fact is piracy and illegal
some people with common sense begged them to stop and stay in legal operating area
they refused, got (predictably) sued as publishers were unhappy already about remote library and (as predictably) lost case
Besides, this so-called goodwill is an illusion. Opposition to AI in the PR space is entirely dominated by motivated reasoning. Attempting to satisfy people with random things is simply going to shift complaints to "take the commons, chop it up, and give it back to us"; "Should AI companies be the arbiters of what is worth preserving?"; and other such things. These kind of public-relation gains are entirely fictive and neither the supposed beneficiary nor the supposed benefactor is fooled by the theory.
"Oh so they suck up all our water and burn our planet down and we're supposed to be happy that they chop up people's books (that humans spent years creating) just so they can give us digested slop back?". Everyone knows that this sentence is waiting in the wings, and that the entire effort is clearly bait so that someone is foolish enough to pull a Google Books and get sued into oblivion for it.
And that's saying nothing about this so-called "hoarding" of things in a world where we pulp tons of these hoarded things every day and no one cares to scan them.
Firefox is similar, people complain about Mozilla's side projects distracting from firefox development all the time, and that they could runa lot leaner if they just focused on the browser.
Also here's a past HN thread with some more context: https://news.ycombinator.com/item?id=46788588
I should have mentioned this earlier, but folks can make direct tax-deductible donations to OldInsuranceMaps.net, which is now affiliated with OSM US's 501(c)(3) status.
https://openstreetmap.app.neoncrm.com/forms/donate-oim
https://cartes.gouv.fr/explorer-les-cartes/
you can add and remove layers by clicking on the map icon on the top right.
I would presume they're starting with very high res lossless raster scans and storing those in some sort of index, then making them available for different pipelines of processing?
0: https://yesterdays.maprva.org/
1: https://yesterdays.inlandempire.place/
"OldInsuranceMaps.net is a Charter Project of OpenStreetMap US that provides an open, online workflow for creating georeferenced layers of historical maps, with a special focus on making seamless mosaics from multi-page atlases in the Sanborn map collection at the Library of Congress."
What the hell does that mean? Anyone to ELI5?
This project is scanning those maps and adding the information that modern mapping software needs to show them alongside other geographic information. Where the old paper maps overlap, they work to cleanly stitch them together rather than showing the edges of the paper.
Later: The project is sort of for people looking for the old maps, so I can see why the introduction doesn't spend a life of time explaining them.
They are building computer programs that other people can use to then post online maps from old paper sources. For example, from the Sanborn map collection.
Those maps can be browsed as "overlays" on top of other maps, so you can easily compare how several maps show the same places on Earth.