Survival LLM: AI assistant and emergency library that runs on your own computer
j / k
|
Down or up a comment |
h / l
|
Previous or next link |
?
|
This help |
It's perfectly safe to use LLMs to surface underlying data and knowledge without having to trust that it's giving you correct advice.
Use the LLM as search technology: string match -> semantic search -> search intelligence.
This is especially helpful when you don't know the right question to ask. Semantics only help you find text that is semantically near the input query. Semantics won't help you when what you are looking for is connected to what you have by something other than semantics, like causation, mechanism, or interpretation.
Trying to figure out why your crops aren't growing well will likely require you to read several articles across a range of sciences, probably scoped to the specific conditions you are dealing with. A semantic search is not going to surface those first.
Allowing an LLM to chain together several sources for you to read first does not require you to ask it for a conclusion.
Sure you can combine this with a regular search but there’s opportunity cost whenever you say LLM + something else rather than just the other thing.
It would be terrible to be doing an emergency medical procedure and have an AI backtrack on something it had told you to do.
There really needs to be some kind of "double check" mechanism built into these things instead of putting 100% of that on the consumer. Humans simply don't have the mental capacity to double check the amount of data coming out of LLMs.
This is not a new problem. The internet has been full of nonsense since the 1990s (at least). Gossip and malicious rumors are as old as language.
It is not realistic to stop the generation of nonsense (of the malicious or the stupid sort). We should train people to detect and ignore nonsense when they hear it.
In most countries, there are simply too many patients per doctor, meaning it's a conveyor belt of "good enough" 5 minute appointments.
People will turn to other modes of discussing and analyzing their problems if they go to professionals repeatedly and the problem still isn't solved. Just telling them not to do so or to ignore 100% of it isn't going to solve that.
> It is not realistic to stop the generation of nonsense (of the malicious or the stupid sort).
Seems like a big statement. Why is it not realistic? People are already doing it manually by having one LLM fact check output from another.
They're amazingly useful tools, but they're nowhere the level believers want them to be, not yet.
I tried using dictation but I really disliked speaking to it.
You can write a really messed up sentence and it almost always worked out what I meant.
Quite amazing actually
Or I could have an honest to god farmer make that list now, ahead of time. Regular search could absolutely surface that.
But also, as someone who does hydroponics, you’re going to end up reading the papers anyways unless you’ve done this a lot. It’s not easy to pick out something like low iron from too little light.
https://www.projectnomad.us/
its got a ton of useful options, preconfigured, integrates pretty easily, and can use any local LLM or data.
How can I build a bow drill so I can start a fire?
It got it partially right: spindle, bow with string tied the correct way, and fire board. But then it flubbed by saying you put the spindle between both hands and move your hands back and forth to rotate the spindle and create the necessary friction… this totally ignores using the bow to rotate the spindle! It also mentioned “fanning the sparks” but there is absolutely no way your bow drill is going to create sparks.
I asked a frontier model to check the survival LLM’s conversation trace and it pointed out the errors.
I didn’t see obvious errors in the other 3 prompts I used, but this one seemed pretty egregious for a survival LLM.
I thank the creator for making this, but take it with a grain of salt.
llms tend to make different types of errors than humans, so with this process each can guard against the errors of the other.
This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.
That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly.
I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers.
[0] https://github.com/tobi/qmd
I'm all up for using the better/simpler task for the job. That's been a thing lately also with the Jev/decision models hype, most of those things can either already be done by a pretrained classifier[0], or you can start with a bigger model and then train your smaller/adhoc one when you have some data (it doesn't even take much for most applications, you can get about 95% accuracy on email classification with 50-100 examples)
0: http://jeffyclassify.com/blog/when-llms-fail-at-classificati...
The bigger issue in this context would be canning, such as incorrectly explaining the mechanisms to avoid botulism (acid and/or pressure canning), or perhaps incorrectly wiring in repairs. However, again, it’s best to practice with these prior to emergencies anyway.
It should be noted though, that the model does source its claims, so you can fact check it and should review this information regularly.
Consider they won't have access to the plentiful practice opportunities we have now, most resources would have to be spent with a practical goal.
One of these situations prohibits the other.
A field guide to foraging for edible mushrooms will probably be more practical than naïve use of an LLM: “can I eat this mushroom?” is an exercise in trust that I will maybe never be comfortable with.
But “how do I effectively forage for edible mushrooms” is a great stepping stone.
This is more for elaborate projects where you don't know precisely what you are looking for. Questions I can think of in a survival scenario:
- my solar panel connector is broken. We only have 8 hr battery left. Can you tell me how to DIY repair it with the following parts
- we want to receive emergency FM broadcasts from the state. Give me steps on how to receive it on Flipper Zero with this breakout board I found in my spare parts
- with the dewalt drill and these parts, can you tell me how to build an outboard motor for the canoe. We need to get outta here before the flood goes up further
If things are so bad that you need to factor in the non-operability of any electronic devices, how many indestructible paper libraries are you planning to build and manage in advance, and at what budget?
Either way the comparisons here are unfair. An LLM that runs off a backpack-sized solar panel and a laptop might be a substitute for a broad, low quality, surface-level encyclopedia but it's many orders of magnitude worse than a library of books.
I would argue that in a survival situation the accuracy of information becomes even more important. If an LLM to misidentifies a plant (or a medical condition, its treatment, etc.) it could put you in an ER, but in a survival situation that could kill you.
If there's any place to get an answer it's here.
Bravo my friend. And I mean that in a good way. Love me some diversity.
Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.
Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.
Except the build process there requires a specific recent version of swift. Hardly a good definition of "anything"
You can run bonsai2 27b on basically any gaming GPU from the last 15 years including on game consoles.
Everyone has access to this stuff now if they bothered to look beyond the marketing from big labs trying to collect subscription revenue.
I work on compilers, kernels, and operating systems using exclusively consumer GPUs.
That seems to have been mostly solved through scale. I was asking Opus 5.5 about some obscure stuff and it kept hedging "this is from memory since I don't have internet access" (it's sandboxed), "I'm reasonably sure about this but not about the numbers" etc.
Even though it was obscure enough that Google couldn't easily find the paper that Opus remembered (I couldn't find it at all with the traditional search results; the AI summary found it on my third try), it actually didn't hallucinate it.
In fact I can't remember Opus 5.5 ever hallucinating. They've clearly done some work on that.
Obviously this local LLM is going to hallucinate all over you though.
Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.
- no books on hunting or fishing
- no instructions on field dressing animals
- no FM 7-8 or ATP 3-21.8
- no info on reloading ammunition or flint knapping
This feels more like a tool for city dwellers than a serious survival tool.
Those are more militia LARP relevant than actual survival relevant in fairness but the other gaps are pretty bad. You can definitely have a useful too without that info and for disaster situations most people don't have to go into the level of hunting their own food so I guess it's more of a question of how long and how deep the disaster you're looking at info on is expected to go.
Also the way groups actually survive scenarios is rarely around actual fighting and is vastly more dependent on actual survival skills, cooperation, and organizing. The lone (or small group) survivalist fantasy is mostly an American larp.
So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.
Honestly, just buy some books.
I’m pretty sure it also told you how to butcher a sheep.
I used to be obsessed with it as a kid. The spectre of nuclear war and a three volume handbook on how to survive the impending apocalypse with nothing more than a plastic bucket, a packet of potassium permanganate, some charcoal and string. The late 80s and early 90s were a wild time to be a kid.
it was a raspi with a couple 18650s and bms taken from a drill battery combined with a pico-projector and a laser keyboard stuffed in a waterproof acrylic cube with a stack of sd cards filled with stuff like the survivor library and an offline copy of wikipedia.
complete with fold out solar panels and circuitry taken from one of those amazon solar phone chargers.
worked well for a few hours at a time at night while camping. eventually i was planning to add a retropi sd and a best of music sd (like classical and the rolling stone magazine top 500)
Some kid with the right prompt will supply chain attack a dependency of a dependency of firmware for border routers just to watch the world burn, or runaway AI swarms will saturate all bandwidth. Or a Linux kernel zero-click vuln. So many ways this can happen.
Better to prepare a bit and never need it.
I have some property that doesn't really get cell reception in the summer, and even in the winter the reception is at-best spotty. I use granite and qwen together with a manually curated set of books, articles, etc. to get work done while disconnected. I tend to update the set of resources before leaving town because I'm generally intentionally disconnecting to focus on deep work. Building a workflow that makes that possible has helped me build out a setup that replaces a ton of my day-to-day anthropic/openai use even when I'm near fiber.
It is interesting how, even pre-llms, offline computers change the nature of a lot of activities. E.g., about once a year I'll walk do a random walk in the woods for a few days without paying much attention to maintaining a bearing. With maps and compasses it's a whole skill set to find your way back. With a phone and downloaded maps you just kinda press the power button and check your heading every hour or two.
The "survival" use-case is interesting. I fear a lot of the practical applications of that knowledge are either time-sensitive (e.g., RAG on a "Stop The Bleed" course won't help because by the time you learn the patient is dead) or require tools+skills that aren't easily improvised completely from scratch in high-stress situations (e.g., building shelter or even really simple stuff like building a fire when it's very cold out). And for other stuff common sense prevails (e.g., power out -> keep fridge closed).
Interesting thing is that when I asked a follow up question it answered but showed: This answer came from what the AI already knows, not the library, so there's no source to check. Ask “look that up” to get one.
When I asked "look that up" it was able to. It highlights relevant sections.
The information it was working off in my case was WikiHow and there aren't any citations in that information.
Running on a 32GB Mac M1 using Qwen 27B. Ollama is showing as taking 19GB steady state and 23GB while answering on Activity Monitor (sometimes a little more and sometimes a little less).
Its amazing this can be done, but it seems like someone should be able to develop a much smaller model that can be used reliably for this use case. There are smaller models available to use and download but it is stated that they have some reliability issues.
I found Project Nomad (https://projectnomad.us) which looks more polished. Any other suggestions?
What about no power? You gonna run your generator for your local model? What are we doing here? Have we lost our way completely?
Solar panels and batteries are great for emergencies, and they're useful besides.
Solar panels? You can go down to Harbor Freight right now and pick up a setup that will run a laptop or tablet for a while, for a couple of hundred dollars. One wouldn't have to spend a ton more to get a setup that can run a decent PC with a GPU for at least a little while.
I mean, you're probably not going to run it all day, especially not in the part of year with fewer daylight hours, but anything's better than nothing. Plus the same solar setup can be used to power other useful things.
This kind of stuff really feels like knowledge that probably shouldn't be offload to AI but rather learned and practiced. Again, I don't have a bunker in my backyard but seems like a weird use case.
Throwing out food that was at 6 °C for 2 hours without tasting it is really paranoid, the answer should be "it depends". If not following government advice/requirements on safe storage conditions (which is tailored to high risk populations) meant certain illness then I would be seriously ill at least once a week, but I've never been.
I would be waaaay more worried about catching a norovirus from someone who didn't wash their hands than raw meat being kept at 4-8C for 12h.
It is also really dumb to do that if there isn't an emergency
The UK government has been pushing hard for the public to stock enough food and water for at least 3 days. We have weeks of supplies.
My huge worry is that the country will run out of LNG due to the ongoing war in Iran, Qatar being offline, and Europe not having enough LNG in storage. Plus, a lot of weird weather due to El Niño.
Is an emergency LLM really going to make a difference? Probably not, but I have one on my desktop computer anyways.
Same with food. NONE of this type of preparation is difficult, very space / time consuming, or expensive.
As for the LNG situation: worst case is do without heating. Totally doable in UK climate; extra layers of clothing & blankets / sleeping bags do the job (been there, done that in the NL).
Remember we're talking getting through a crisis, not comfort. Although you can have both with more prep work.
Books are pretty damn good and don't need 500W of GPU
This is more of a "preserve the knowledge of mankind for the survivors" type of archive.
- Traditional: a generator. Problem is that you need a fuel tank, but if you are aiming to get through a couple of weeks then that's possible.
- Grid attached battery backup: good because in some places (like the UK) electricity at night is much cheaper than electricity during the day... so you can charge the battery and then save money and get payback on your installation. Bonus point is that if there is a grid outage then you have power, for a day or so if you are careful.
- Grid attached with solar/wind: as above, but you can charge independently from the sun/wind... obviously in some places (like the UK) this has limited utility because the sun don't shine (like today), and the wind can be annoying and difficult as a power source (gearboxes, noise, aesthetics). Can cost in though - because of summer power my panels pay back in 11 years. Small wind turbines seem to be getting more practical but I am no expert.
- Non grid battery backup. These also come with solar now, so you can recharge if it's sunny. These are cheap (<~£1000), hold maybe 2Kw and so can power some devices while the grid is off for a little while. Definitely you can run your phones and computers for a few extra cycles, but people are often looking to run freezers or fridges with medicine with these.
Another thing that folks like to have is grid independent heating / cooking via a woodstove. This is frowned on in urban areas and by eco people but I live in the country and have done my reading and while other people will pop up and decry particulates to be honest I'm keeping my burner so my family will be warm and fed for as long as our food lasts or we don't get looted by the neighbours.
My revelation on this was when my daughter got type 1 diabetes. If the world goes south, even if I manage to do everything and "win", she's dead in six months. Of course, since then I've realised that the "do everything and "win"" thing is also stupid, no one is going to be able to "make it" a handful will luck out and mangage somehow.. but it won't be because of prepping.
On the other hand, if there's a blip - like a cyber attack, or a regional war, a solar flare, bad volcano, or a mega storm. then preps might make things more comfortable until the state gets back up and running.
That said, I'm really looking forward to small and quick (latency wise) models are a reality. I can easily see a world where every keystroke, terminal command, etc. goes through a "autocorrect" type model. You could even blend natural language plus terminal program segments and the AI just infers what you intend. Either built-in or a second classifier model, determines if the autocorrect change needs human review first.
Not sure what they do on the windows side.
Or try these with your trained embedder: https://ollama.com/library/minicpm-v4.6 https://ollama.com/library/bonsai-2
Reminds me of the time when Docker was thrown at everything.
Also, I am surprised that the difference between 32GB and 128GB in right answers is so small:
* Mac with 128 GB or more memory NVIDIA Nemotron 3.5 Super 21 of 22 right 80 GB
* Windows PC with 32 GB or more NVIDIA Nemotron 3.5 Lightning 19 of 22 right 25 GB
Helps you download a local LLM and offline Wikipedia. Probably easier to just ask your favorite agent to glue something together yourself these days.
Apocalypse emergency I don't need help on. My crisis brain has collected more than enough info and capabilities over the decades.
I’m currently using https://kiwix.org for the offline access to Wikipedia, but your Survival LLM looks even better.
The source (data collection) seems the most important feature here worth reusing.
>The app is the same everywhere. When it opens, it checks your computer's memory and downloads the AI model that fits. More memory means a smarter model.
first thing an emergency tool should require, internet access.
at first glance I figured it'd be an ultra small specialty trained survival model that'd be small and runnable anywhere, sorta like an emergency flavored version of those tiny tts models that have been all over recently.
Maybe that's an idea for someone.
Edit: see project nomad.
From the command line: unsloth studio
Browse to the URL it gives you (eg http://127.0.0.1:8888/)
Download a good fast local model e.g. Gemma-4-12B (small machine) or Qwen3.6-35B-A3B (larger memory). Or a Nemotron model, if you prefer that.
Then you'll have both your backup/emergency assistant that runs on your own computer and be gaining skills in a common, frequently-updated, well-supported program.
I think if I had to build an emergency LLM today, that had to run on modest hardware, I'd bet on Gemma 4 12B. It's got excellent reasoning and good vision/audio and speaks 140 languages.
But, I don't know about the "classic" knowledge base theory, though. Is there a high quality curated pile of survival information that could be bundled into a web UI and made searchable in a knowledge base? The LLMs aren't focused on that topic, so there's a lot of wasted space, but they know a little about a lot of things, including survival topics. Combined with Wikipedia, and maybe a few other sources, you're probably better off with one than without.
I guess one could use some huge models plus web search to compile/distill a focused survival knowledge base or LLM.
Edit: Project NOMAD seems to mostly be exactly that. It has authoritative sources in a searchable knowledge base and the tiny LLMs are an afterthought, which is probably what they should be. https://www.projectnomad.us/
This project at least attempts to ground its answers with (presumably) human-vetted sources. That will make it much, MUCH more accurate and truthful than a bare model. But there is still a non-zero chance for it to hallucinate answers, especially when posed a question that it doesn't have any source material in its database.
As for why bother hooking up an LLM instead of a simple search engine, search engines don't synthesize data from multiple sources into a coherent answer for you, but LLMs are (mostly) great at that.
That said, while I like this pattern (LLM + knowledge base) for lots of things, this isn't necessarily something I'd trust with my own (or my family's) survival.
Not sure where this would fit in.
Five minutes of clicking and a (small) power station is a small price to pay when survival could be at stake.
It is also only 5% of the wider scope and I was about to release it under OSS 2 days ago, but wanted add some final finess. Now that this is out, I might as well release it so the concept actually ties to its original goal, and release the scientific papers that date back to Feb. Please ping me on github. Same username as here. Always could do with help.
In an actual survival scenario power isn't guaranteed and errors of any kind can be life or death. So you need to minimize the resources used to get the thing working.
>Survival LLM >It Can be wrong
thank you!
tl;dr: Local LLMs are really really bad at offline navigation.
That said, pairing the survival guide with a library is totally the right move.
The thesis is that someday an LLM might perform better than both. But I agree, not yet.
Navigation is a closed task, A* will always find an optimal route as long as your street info is current, you can't do better than that.
I'd put in this also a ton of software documentation, minimally for Rust.
And a bunch of scientific literature.
(It's helpful for shorter outages. But "survival assistant", I think not, unless you run a compound)
But kidding aside, this is an interesting reveal on the ease with which a custom LLM can be spun up to fit a particular niche application, especially for systems not meant to host an LLM.
In a theoretical scenario where the internet is gone and isn't coming back: If you can deal with a very low token/second rate there's no reason not to store 'offline' a much larger model like qwen 3.8-flash-next or deepseek v4 flash 1017 and use an inference engine which can stream-read it off disk to extract the information you want, albeit slowly. Of course also download and use things that'll fit in RAM, sure, like the qwen 3.8 27B in the linked page. But no need to restrict yourself to only smaller models.
Just look at those specs needed for accurate-ish results. You really think your 128GB Mac Studio or Framework desktop is going to be able to work in a real emergency situation? Even if it is, how long will it last? What about those precious seconds waiting for inference?
The danger here is less hallucinations and more, well, everything else about it. Instead of setting this up, take the time to make sure you have a restocked first aid kit and enough water for a week of use by your household. Stock up on meds and non-perishable food. Plan exit strategies and contingencies. Acquire and store little first aid booklets or manuals of common survival tasks in physical form (or hell, throw onto an eReader - infinitely more battery life than an LLM host in an emergency).
This is one of those “sounds good in my head” projects that folks with emergency experience would bristle at. This is a hindrance, not a helper.
Also: "Apple could not verify “Survival LLM” is free of malware that may harm your Mac or compromise your privacy." This is obviously some vibe coded trash from an inexperienced dev.
This is a great example of how Apple's "scare warnings" completely work to mislead folks about what they are actually warning you about.