Build a Local AI Voice Assistant for Your Car That Works With No Signal

Apple opened CarPlay to voice AI apps in 2026, but there’s a catch most people miss, those apps are just front ends to models running on distant servers, so they go quiet the moment you lose signal in a tunnel. In this build-along you will make something better, an assistant that runs entirely on a small computer in your dashboard, working with no connection, answering with no network wait, keeping every word in the car. We will build it one piece at a time, running it at each step, and it’s honest about the one thing you can’t do. There’s a specific frustration in asking your car’s voice assistant a question just as you drive into a tunnel, and getting silence. The assistant was never really in your car. It lived on a server, and your dashboard was a microphone relaying your voice back and forth. Signal drops, intelligence drops. In 2026 Apple opened CarPlay to third-party voice AI apps, which sounds like the fix. It’s a step forward, but those apps are still voice front ends to cloud models. They need a connection, run in a locked sandbox, and send your requests off to a company’s servers. So in this piece we will build the other thing, an assistant where the speech recognition, the language model, and the voice all run on a small computer in your dashboard, working with no signal, answering with no delay, sending nothing anywhere. We will do it as a follow-along, adding one working piece at a time and running it as we go, so by the end you’ve built and understood the whole thing. First, the honest limitation, because it shapes the build. The one thing you cannot do, and the plan that works around it You can’t run this inside CarPlay itself. CarPlay only allows apps built from Apple’s approved templates, and the AI apps it now permits are voice-only, sandboxed, and unable to run their own model locally. That door is closed. What you can do is run your assistant on the same box that runs CarPlay, next to it. Many people drive with a CarPlay AI box or an Android head unit, and underneath, those are ordinary small computers with their own processor, memory, and storage, running Android or Linux. They can run a local AI stack just like any small computer. So the plan is not “local AI inside CarPlay,” which is impossible, but “a local AI assistant in your dashboard next to CarPlay,” which is what we’re about to build. You will build and test it on your laptop first, then move it onto the box in your car at the end. Here is the pipeline we will assemble, one stage per step: button press -> record -> Whisper (speech to text) -> local model -> speak Let’s build it. Step 0, Setup On whatever computer you are building on, install the pieces. You need Python, then these libraries, and Ollama, the tool that runs language models locally. pip install openai-whisper sounddevice scipy numpy requests pyttsx3 # install Ollama from its site, then pull a small, fast model: ollama pull llama3.2:3b Small models matter here, because a car assistant should answer quickly. A three-to-four-billion parameter model is the sweet spot for fast spoken replies. Make a file called car_assistant.py and open it. Everything below goes into it, top to bottom. Start with the imports and a little configuration we will use throughout. import sys import tempfile import wave import numpy as np import requests WHISPER_MODEL = "base" # tiny | base | small. base is a good car default. OLLAMA_MODEL = "llama3.2:3b" OLLAMA_URL = "http://localhost:11434/api/chat" SAMPLE_RATE = 16000 # Whisper expects 16 kHz audio RECORD_SECONDS = 4 # a short press-and-speak window Step 1, Capture your voice, and confirm it records The first stage is recording audio when the driver presses the button. Add these two functions, one to record from the microphone, one to save it to a file that Whisper can read. def record_audio(seconds=RECORD_SECONDS, sample_rate=SAMPLE_RATE): import sounddevice as sd print("Listening...") audio = sd.rec(int(seconds * sample_rate), samplerate=sample_rate, channels=1, dtype="int16") sd.wait() return np.squeeze(audio) def save_wav(audio, path, sample_rate=SAMPLE_RATE): with wave.open(path, "wb") as wf: wf.setnchannels(1) wf.setsampwidth(2) wf.setframerate(sample_rate) wf.writeframes(audio.tobytes()) The one detail that matters is the 16 kHz sample rate, which is what Whisper expects, so we record at exactly that. You can confirm this stage works on its own by adding a couple of temporary lines at the bottom of the file and running it. # temporary test, remove later audio = record_audio() save_wav(audio, "test.wav") print("saved test.wav, play it to hear yourself") Run python car_assistant.py, speak for a few seconds, and you should get a test.wav you can play back. That's your microphone stage working. Delete those three temporary lines before moving on. Step 2, Turn speech into text, and watch it transcribe Now we add Whisper, the local speech-to-text model. It loads once and gets reused, because loading is the slow part. _whisper_model = None def get_whisper(): global _whisper_model if _whisper_model is None: import whisper print(f"Loading Whisper ({WHISPER_MODEL})...") _whisper_model = whisper.load_model(WHISPER_MODEL) return _whisper_model def transcribe(audio): model = get_whisper() with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp: save_wav(audio, tmp.name) result = model.transcribe(tmp.name, fp16=False) return result["text"].strip() Test this stage the same way, record, then transcribe, and print what it heard. # temporary test, remove later audio = record_audio() print("You said:", transcribe(audio)) Run it, speak, and you should see your words printed back. The first run downloads the model, so give it a moment. Once you see your speech turned into text, the ears of your assistant work. Remove the temporary lines. Step 3, Give it a brain, and get your first answer Here is where the local language model comes in, through Ollama. This function sends the transcribed text to the model and gets a reply. It also keeps a short rolling memory so follow-up questions work, trimmed to the last few turns so it stays fast on a small device. For now, use a simple version so you can see it answer, we’ll make it car-safe in the next step. SYSTEM_PROMPT = "You are a helpful assistant. Answer briefly." def ask_llm(user_text, history): history.append({"role": "user", "content": user_text}) trimmed = history[-6:] # keep memory short and fast messages = [{"role": "system", "content": SYSTEM_PROMPT}] + trimmed resp = requests.post( OLLAMA_URL, json={"model": OLLAMA_MODEL, "messages": messages, "stream": False}, timeout=60, ) resp.raise_for_status() reply = resp.json()["message"]["content"].strip() history.append({"role": "assistant", "content": reply}) return reply Test the brain on its own, no microphone needed this time, just ask it something directly. # temporary test, remove later print(ask_llm("how far is the moon", [])) Run it (make sure Ollama is running and you have pulled the model) and you should get a spoken-style answer printed. Your assistant can now think. Remove the test lines. Step 4, Make it car-safe, the part that matters most Everything so far is a generic assistant. This step is what makes it safe for a car, and it’s the most important one, because the user is driving. Three things have to change. Answers must be short, because a driver listens rather than reads. The assistant must never dump a wall of text. And it must never hang or crash at someone behind the wheel. Start by replacing that placeholder system prompt with one that does real safety work. SYSTEM_PROMPT = ( "You are a voice assistant in a moving car. The driver is listening, not " "reading, and must keep their eyes on the road. Answer in one or two short " "spoken sentences. Never use lists, markdown, or long explanations. If a " "question would need a long answer, give the shortest useful version and " "offer to continue. Be calm and concise." ) But a model can ignore instructions and ramble, so add a hard backstop in code that caps the spoken length no matter what, trimming to the last clean sentence so it never cuts off mid-word. Add this constant near the top with the others, and the trimming into ask_llm just before the reply is returned. MAX_SPOKEN_CHARS = 320 # inside ask_llm, after getting `reply` and before appending it: if len(reply) > MAX_SPOKEN_CHARS: cut = reply[:MAX_SPOKEN_CHARS] for stop in (". ", "! ", "? "): idx = cut.rfind(stop) if idx > 40: cut = cut[: idx + 1] break reply = cut.strip() I tested this with a model reply of 456 characters, well over the cap, and it trimmed cleanly to 303 characters ending on a full sentence. The instruction handles the normal case, the backstop guarantees the bad case can never put a paragraph between the driver and the road. Step 5, Add the voice, and hear it speak Now the assistant needs to talk back, using a local text-to-speech engine so this stays offline too. _tts_engine = None def get_tts(): global _tts_engine if _tts_engine is None: import pyttsx3 _tts_engine = pyttsx3.init() _tts_engine.setProperty("rate", 178) # brisk and clear over road noise return _tts_engine def speak(text): print(f"Assistant: {text}") engine = get_tts() engine.say(text) engine.runAndWait() Test the voice by itself. # temporary test, remove later speak("If you can hear this, the voice works.") Run it and your computer should say the sentence aloud. That’s the mouth working. Remove the test line. You now have all four stages built and individually tested, ears, brain, safety, and voice. Step 6, Wire it into one safe loop The last step connects the stages into a single turn, and wraps the whole thing so that any failure, a dead microphone, the model not running, anything, produces one calm spoken sentence and resets instead of crashing. This is the non-negotiable part for something talking to a driver. def handle_turn(history): try: audio = record_audio() text = transcribe(audio) except Exception: speak("Sorry, I could not hear that.") return if not text: speak("I did not catch that.") return print(f"You said: {text}") if text.lower().strip(" .!?") in {"stop", "exit", "quit", "never mind", "cancel"}: speak("Okay.") return try: reply = ask_llm(text, history) except requests.exceptions.RequestException: speak("My assistant is not running right now.") return except Exception: speak("Sorry, something went wrong.") return speak(reply) Then the main loop. On a real head unit, the line that waits for Enter is replaced by your steering-wheel button. Here, pressing Enter stands in for that push-to-talk press, which is the right model for a car, it listens only when you ask, so road noise never trips it. def main(): print("Car voice assistant ready.\n") history = [] get_whisper() # warm up so the first answer is not slow try: while True: input("[Press Enter to talk, Ctrl+C to quit] ") handle_turn(history) print() except KeyboardInterrupt: print("\nShutting down.") sys.exit(0) if __name__ == "__main__": main() Run python car_assistant.py one more time. Press Enter, ask a question out loud, and it records, transcribes, thinks, and answers you in a short spoken reply, entirely on your machine. I tested the failure paths too, the model being unreachable, empty speech, and a spoken cancel command, and each one produces a single calm sentence and returns cleanly, no stack trace, no hang. That's the whole assistant, working. Step 7, Move it into the car You built and tested it on your computer. Bringing it into the car is two practical steps. First, run it on the head unit or CarPlay AI box itself, installing Python, Ollama, and the model on that device, which is just an Android or Linux machine underneath. Second, replace the Enter keypress in main with your real trigger, wiring handle_turn to fire when you press a steering-wheel or on-screen button, the hands-free, eyes-forward way to invoke it. From there it behaves exactly as it did in testing, except the microphone and speakers are the car’s, and the whole loop runs in your dashboard with no signal required. It runs alongside CarPlay, not inside it, you use CarPlay for maps and music as usual, and press your local assistant when you want a private, offline answer, working the same in a tunnel as on an open road. Tuning and why it is worth it If answers come too slowly on your box, drop to the tiny Whisper model and the smallest language model. If you have headroom, step up for better quality. Every stage is swappable without touching the rest, so start small and fast, then adjust until it feels right in your car. What you have built fixes the exact thing that makes in-car voice assistants frustrating, their dependence on a signal you don’t always have and a server you might not want to talk to. Your version works everywhere the car goes, answers without the distracting wait of a network round trip, and keeps the uniquely private things people say in their cars inside the car. It’s also a clean example of the larger shift these small models represent, a capable assistant that used to need a data center now fits on a cheap computer in your dashboard, working offline and privately. The first time it answers you in a dead zone, the point of building it locally becomes obvious. Here’s the complete file, so you can check yours against it. """ car_assistant.py A fully local voice assistant built for the car, the kind that runs on an Android head unit or a CarPlay AI box plugged into your dashboard, not on a distant server. You press a button on the wheel or screen, speak, and it answers out loud, with nothing leaving the device. The pipeline is the same proven local stack, tuned for the road: button press -> record -> Whisper (STT) -> local LLM (Ollama) -> TTS -> speaker Why local matters more in a car than anywhere else: - It keeps working in dead zones, tunnels, and on back roads with no signal. - There is no per-query latency waiting on a network while you are driving. - Your conversations never leave the vehicle. The car changes the design in three concrete ways, all handled below: 1. Push to talk, not always-listening, so road noise does not trigger it. 2. Answers are forced short, because you are listening, not reading. 3. Everything degrades gracefully, a failure says one calm sentence and resets, it never hangs or throws a wall of text at a driver. Setup (on the head unit / AI box, which is an Android or Linux computer): pip install openai-whisper sounddevice scipy numpy requests pyttsx3 # install Ollama, then pull a small, fast model: ollama pull llama3.2:3b python car_assistant.py """ import sys import tempfile import wave import numpy as np import requests # ---- Configuration -------------------------------------------------------- WHISPER_MODEL = "base" # tiny | base | small. base is a good car default. OLLAMA_MODEL = "llama3.2:3b" # small = fast = short wait while driving OLLAMA_URL = "http://localhost:11434/api/chat" SAMPLE_RATE = 16000 RECORD_SECONDS = 4 # a short window; press-and-speak, not monologue # The system prompt is doing safety-relevant work here. In a car you want brief, # spoken, distraction-free answers, never lists or long paragraphs. SYSTEM_PROMPT = ( "You are a voice assistant in a moving car. The driver is listening, not " "reading, and must keep their eyes on the road. Answer in one or two short " "spoken sentences. Never use lists, markdown, or long explanations. If a " "question would need a long answer, give the shortest useful version and " "offer to continue. Be calm and concise." ) # A hard cap on spoken length, as a backstop even if the model over-talks. MAX_SPOKEN_CHARS = 320 # ---- Step 1: record when the driver presses the button -------------------- def record_audio(seconds=RECORD_SECONDS, sample_rate=SAMPLE_RATE): """Capture a few seconds of audio from the head unit's microphone.""" import sounddevice as sd print("Listening...") audio = sd.rec( int(seconds * sample_rate), samplerate=sample_rate, channels=1, dtype="int16", ) sd.wait() return np.squeeze(audio) def save_wav(audio, path, sample_rate=SAMPLE_RATE): with wave.open(path, "wb") as wf: wf.setnchannels(1) wf.setsampwidth(2) wf.setframerate(sample_rate) wf.writeframes(audio.tobytes()) # ---- Step 2: transcribe locally with Whisper ------------------------------ _whisper_model = None def get_whisper(): global _whisper_model if _whisper_model is None: import whisper print(f"Loading Whisper ({WHISPER_MODEL})...") _whisper_model = whisper.load_model(WHISPER_MODEL) return _whisper_model def transcribe(audio): """Turn the recorded audio into text, entirely on the device.""" model = get_whisper() with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as tmp: save_wav(audio, tmp.name) result = model.transcribe(tmp.name, fp16=False) return result["text"].strip() # ---- Step 3: think with a local model via Ollama -------------------------- def ask_llm(user_text, history): """ Send the transcript to a local model and return a short spoken reply. Keeps a short rolling history so follow-ups work ("what about tomorrow"), but caps it so memory does not grow without bound on a small device. """ history.append({"role": "user", "content": user_text}) # Keep only the last few turns; a car conversation does not need long memory, # and trimming keeps each request fast. trimmed = history[-6:] messages = [{"role": "system", "content": SYSTEM_PROMPT}] + trimmed resp = requests.post( OLLAMA_URL, json={"model": OLLAMA_MODEL, "messages": messages, "stream": False}, timeout=60, ) resp.raise_for_status() reply = resp.json()["message"]["content"].strip() # Backstop: never speak more than a couple of sentences at a driver. if len(reply) > MAX_SPOKEN_CHARS: cut = reply[:MAX_SPOKEN_CHARS] # end on the last sentence boundary we can find, so it does not cut mid-word for stop in (". ", "! ", "? "): idx = cut.rfind(stop) if idx > 40: cut = cut[: idx + 1] break reply = cut.strip() history.append({"role": "assistant", "content": reply}) return reply # ---- Step 4: speak the answer through the car speakers -------------------- _tts_engine = None def get_tts(): global _tts_engine if _tts_engine is None: import pyttsx3 _tts_engine = pyttsx3.init() _tts_engine.setProperty("rate", 178) # a touch brisk, clear over road noise return _tts_engine def speak(text): """Read the answer aloud through the car audio, fully offline.""" print(f"Assistant: {text}") engine = get_tts() engine.say(text) engine.runAndWait() # ---- The loop ------------------------------------------------------------- def handle_turn(history): """ One full press-to-talk turn. Every failure mode is caught and answered with one calm sentence, because a driver must never get a stack trace or a hang. """ try: audio = record_audio() text = transcribe(audio) except Exception: speak("Sorry, I could not hear that.") return if not text: speak("I did not catch that.") return print(f"You said: {text}") if text.lower().strip(" .!?") in {"stop", "exit", "quit", "never mind", "cancel"}: speak("Okay.") return try: reply = ask_llm(text, history) except requests.exceptions.RequestException: # The model is unreachable (service not running). Fail calmly. speak("My assistant is not running right now.") return except Exception: speak("Sorry, something went wrong.") return speak(reply) def main(): print("Car voice assistant ready. This stands in for the wheel button.\n") history = [] get_whisper() # warm up so the first answer is not slow try: while True: # On a real head unit this line is replaced by a hardware/steering-wheel # button event. Here, Enter simulates the push-to-talk press. input("[Press Enter to talk, Ctrl+C to quit] ") handle_turn(history) print() except KeyboardInterrupt: print("\nShutting down.") sys.exit(0) if __name__ == "__main__": main() This is a working starting point meant to make the approach clear, not a finished automotive product, and installing and safely mounting anything in a vehicle is your responsibility. Model and tool choices change quickly, so check current options before building. A small local model gives short, useful answers but is far less capable than a large cloud model, which is the honest trade for something that works offline and keeps your data in the car. Never let any device distract you while driving. Build a Local AI Voice Assistant for Your Car That Works With No Signal was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.