Around '05 or so, the house we were staying in started beeping. Smoke alarm sounds, as usual. For almost a day we couldn't find the source. Contacted the owners -- they didn't know it either. They bought the house few years prior, so not much of a help.
Standing in the hallways listening at different angles, as the article says, didn't help. No Fable in '05 either.
So I had to improvise. Got a ~1 meter stick and two headphones (every headphone is a microphone, even if it sucks as a microphone). Plugged them into the stereo mic port, and started recording in Audacity. The phase shift between L and R pointed directly to the source. Took about 4 or 5 beeps to get the sucker.
Turns out, it was hiding above the ceiling tiles in the garage. My wife still tells this story to all her friends as one of my biggest life achievements. Everything I shipped, the companies I founded -- pales in comparison to finding a dead smoke alarm
it didn't have to be sensitive or even high quality. The beep was loud enough to register on these improvised "mics" and the phase shift of up to 3ms (~50 samples at 16 kHz) was clearly visible
The cross-correlation function can be applied to measure time shift between two waveforms, even those collected from different devices. The accuracy of this is not limited by the sample period. In fact, the two waveforms need not even have a common sample period. I think this approach could let simonw's network become self locating.
Many many many years ago, I was a wee IT manager. Sort of a weird company, I was too young to be much more than smart kid who can install / fix anything.
Anyways, worked a few years there, made some friends, had one CAD engineer who was an utter jerk. In hindsight, probably a bully, but at the time just a really irritating person.
So that was my first foray into small electronics… worked up a small breadboard with a 555, some source of randomness I don’t remember now, a piezo speaker, and a 9v battery. It would emit a short beep at random intervals between roughly 60 and 240 minutes.
Stashed it above the ceiling tiles in his office on my last day at the company.
In college, some friends bought (they were less imaginative than you) the equivalent, powered by a 2032 cell, and hid it in my room as a prank.
Good times. I bet it’s still magnetically attached to the inside of the steel riser column of one of their desk chairs, though the battery is likely long dead.
Could you elaborate on how you converted the phase shift into a precise direction? Just a linear mapping? Or did you "hone in" by adjusting the stick until the phase shift disappeared, then proceed in that direction?
Not the OP, but the phase shift of a known frequency can be converted into a distance (portion of a wavelength) which can then be used to triangulate between the two microphones, yielding a vector to the target.
I did in manually, i.e., visually. By looking at the phase shift and moving in direction of the channel that heard the beep first.
The beep frequency is about 3000-3500 Hz, which makes its wavelength about 10cm (1/3 of a foot). The piezo speaker in the smoke detector is so small that it takes 3-4 wave periods to go from silent to full blast. Meaning, if my stick was pointing perfectly to the source (it never was, obviously, but if), then one of the "mics" would hear the sound at full blast while the other still wouldn't hear anything.
So, I would have Audacity recording, hear the beep, stop the recording and zoom at the wavefront of the beep. Whichever mic shows the signal first, I'd move my stick (and the laptop) in that direction. After 2 or 3 moves and I was right under the source. Took me about 2 or 3 beeps at different stick orientation to understand that the sound is coming from above.
So I may in the future be the villain in a story like this if I am forgetful when I sell my apartment.
My upstairs neighbor is a moron who has flooded their apartment at least twice, so I have stashed moisture sensors into the access panels in multiple rooms as a sort of early warning system.
> My wife still tells this story to all her friends as one of my biggest life achievements. Everything I shipped, the companies I founded -- pales in comparison to finding a dead smoke alarm
Do you not feel this way about it yourself? I know I would.
I brought the carbon monoxide detector downstairs to show my wife after finding it with Claude and she said "you look like a cat that's proudly showing me the mouse that it caught".
My version of this story is coming back from turbulent red eye to this type of beeping, spending an hour walking around trying to figure it out it by ear, eventually giving up as it always seems to be somewhere else.
It wasn’t until the next day did I realize that it was two separate locations!
Pff that sound is horrible, and sorta hard to locate (because of the pitch?). Anyway. None of my neighbours seem to care and in summer I used to hear an orchestra of beeps every 5 seconds. It was maddening, I was contemplating of buying 1000 batteries, print a manual and shoving it in peoples mailboxes, although somehow I think it wouldnt have made a difference. I moved out eventually (not because of the beeps)
It’s because it doesn’t have enough overtones, or a notable attack. You might have noticed that some trucks these days make a weird noise when they back up instead of a beep. They’re using white noise because it’s easier to locate; it has many overtones that allow your brain to pick the sound out of the background more easily, and thus more easily correlate the signals from both ears. You get a much more accurate location just from hearing that sound than you ever could with a pure tone.
Interesting! I also feel the pulse is just to short… starting to question yourself if you actually heard something or that you are imagining it. Here the cars and trucks play musicals tunes or a crappy voice recording hahah
That makes sense; we were searching for a family member’s blood sugar monitor just today and it barely seemed to come from any single direction. Ultimately it wasn’t the sound itself but the loudness of it that let us find it, because a blanket got shoved over it mid-search that we could deduce from there.
Consider a pure sine wave, it's harder (in theory anyway) to match the signals from the left and right ears, and identify the time delay between the two signals: each cycle of the wave is just like the others, so there's no distinct signature to match and say "the sound reached the left ear 0.5 ms before it reached the right ear". This is especially the case for higher frequencies where there can be something like aliasing.
Animal species sometimes use this in different sounds they make. An alarm call has that kind of "smooth tone" characteristic, so a predator can't locate the source of the sound but it still works to raise the alarm. Meanwhile more social sounds are more "chirpy" and sharp, so they can locate each other.
I think some people are immune to beeps. My wife drives a Suzuki Crossbee and the fucking thing beeps every 10s for some alert or another. It drives me insane but she doesn’t notice it at all.
Examples
- beep: curve coming up ahead in X metres
- beep: curve coming up
- beep beep beep (too close to lane divider)
- beep: the speed limit is X
- beep: you’re speeding (no, I’m transitioning from a 60kmh to 40kmh section)
- beep: gps directions
- beep: gps directions again in case you missed it
> Claude doesn’t have native audio input (yet), but it can run JavaScript and Python code that can analyze audio captured by a microphone.
LLMs can also do that with non-mic audio, btw. I had GLM (or was it qwen?) answer the question of "Why does the spotify version of the Metal Gear Rising" soundtrack sound so bad" - to great results.
Compared with the CD rips on archive.org, the noise floor was 27dB (!!!) higher, and, the whole thing was 10dB more compressed.
I cannot vouch for these specific numbers, but subtracting the good version from the bad one gave me a horrible delta wav file with lots to listen to, so there was definitely a _lot_ of difference.
> Our dog goes outside and hides in the lavender bush if a smoke alarm starts making a "I need a battery" noise.
Our three dogs are absolutely terrified of the Nest smoke alarm after I almost burned down my kitchen trying to make some copycat crunchwraps a few years ago. =P They're so afraid of the smoke alarm that the monthly "sound check" thing Nest performs makes them quiver and quake. We used to be able to delay the sound check through the Nest app, so we could have a chance to bring the dogs outside before it goes off, but Nest seems to have removed that functionality.
- the boxes are stacked
- the heavy boxes are at the bottom of a stack, and you can't stack a heavy box on top of a light one
- boxes aren't the same size
- some boxes are open, so must be put on top
- storage space and height are limited
Yes, but that's also O(log n) physical effort. This one's O(1) sitting on your butt.
(Out of curiosity: I think "physical binary search" is more properly O(n^{1/3} log(n)), or maybe O(n^{1/2} log(n)). Since physical objects have sizes, and moving things (or moving between things) takes time proportional to distance).
Good point about the moving effort required. But if moving an object takes constant time, then it's O(n) overall -- since <incorrect argument deleted>.
EDIT: My earlier analysis was wrong (it assumed you never move the beeping item after the initial halving), but the conclusion was right for a different reason: Each time you move items, you move half as many as you did last time. You initially move n/2 items, and the geometric series n/2 + n/4 + n/8 + ... + 1 sums to n-1. So O(n) moves are done, and this dominates the log n term for the number of steps.
You could also solve it by getting rid of the boxes of stuff you clearly don't need (if you needed it, it wouldn't still be in a box). The best way to solve a problem is to make the problem go away.
That was my proposed solution to finding something just today; it didn’t work, but one of the bags ended up containing an old contract from General Development Company. Apparently a late family member had been buying some land down there through them, but county appraisers showed that the land never changed hands until after GDC’s bankruptcy. Not even their kid had heard about this, so it really makes me wonder what the story was behind that considering they lived in the Midwest (but did have some relatives nearby in Florida).
In terms of physical effort taken it’s O(n). Move half, then a quarter, then one eighth, etc.
The number of partitions doesn’t matter either, the most straightforward case is the beeping box always being in the original room, so regardless of strategy you move n-1 boxes to narrow it down, and by symmetry it doesn’t matter which room beeps.
I did roughly that to find my mum's missing hearing aid the other day. The bluetooth would locate which room it was in but not much more detailed than that. It was stuck to some orange peel in a recycling bin.
I’ve been trying to think of other uses for the system. I’m wondering about this:
What use could a teacher make of a locator system, if students could identify their location and desire to be singled out for a moment, by simply clapping once?
In a big auditorium, a spotlight could track over to the student, who could then ask a question.
And/or, directional microphones could zero in on the student so the question could be played on the PA system.
I love the idea of accidentally triggering this system.
You're trying to catch a fly that's been tormenting you. You clap your hands together to catch it, but it slips out. The motors on the spotlight whir. "No no no, cancel cancel!" It's too late. Your hunt now has an audience. The spotlight is on you. Your classmates are calling out when they see the fly, cheering your swats, and groaning when you miss. Even the professor is silently praying for your victory so they might be able to regain some semblance of control over the class.
Ha! This reminded me of that old famous entry from bash org: "I can't find a server in my house. It responds to ping, but I can't physically locate it."
Why didn't you time the interval between the beeps? Then when the next beep was coming up, just listen to a particular part of the room or set of boxes until you found it.
This is the problem with technology. Some people just want to use it, so it's quite hard to tell whether this is really a quality of life improvement, or if it's just exciting for some weirdo (any programmer counts as weirdo here).
I've personally had to track down a beep before and used a similar technique as you suggest: time the beeps then be in right place to perform a kind of binary search to home in on it. I found a smoke alarm in the roof space above my flat, nailed to the top of the rafters that had been there since the building was built 20 years prior. It was annoying but I'm sure it took way less time and effort than "getting AI to do it".
This is also why it's been difficult for me to decide on portfolio projects to build. I just don't have anything I really need to build. I made a compound interest calculator as my first React app, but could have made an excel sheet or python script.
I use a frustration threshold. A lot of times, I’ve got a working solution, but it’s not “right”, so when that become frustrating enough, I build something.
A few examples
dotnet does not work on OpenBSD, so to run jellyfin (supported by the infuse app on the apple tv), I have to run Linux. I wonder if I can replicate the API somehow
calibre is quite nice, but I don’t use much of its capabilities. I could try to make a simpler system for my use cases.
> I think the most interesting part of this project is that I demonstrated it’s possible to give Claude Code ears
Of course it can analyze audio. I made it detect Marijn Frank saying a high pitched "Hallo" in a much better way that I could have done myself (subtitle matching), namely by frequency analysis, in the process learning how that works with librosa: https://marijnfrankzegthallo.nl
It's still a bit surprising the first time. Especially if you've been playing with these things for years now almost exclusively with text and perhaps checking for extra fingers now and then.
I didn't have birdnet to hand and wanted to identify something. It was hard to take a picture with my phone but easy to record. Turns out one of the models was able to identify it.
I guess the Americans might appreciate gunshot identification and localisation or someone might drive around in winter/summer recording the streets to listen for heat pumps that are on the way out so they can advertise repair services. I would like a local version for movement/presence detection based on footsteps as an alternative to having pir sensors all over the place for my hall lights.
I like that tale! However, I've found that my Wire Fox Terrier has somehow trained himself to distinguish a smoke alarm beep from the plethora of beeps in the house and came upstairs today at 3am waking me up to tell me batteries were running out. He followed me around until I located the culprit and then went back to bed. Next time he'll have upped his game to tell me which device it is. He's a good boi!
There was a post on HN recently about a suspended string robot, and how it could manage different loads (picking up dirty laundry) depending on on where it was in the room.
Simon’s microphone configuration is very similar except instead of payload kg it would be affecting detection accuracy.
I wonder why the accuracy was an OOM outside the expected result (40cm vs 7mm)? Also using claps for calibration of unknown devices feels like a different approach to measuring distances between devices with a tape and solving the polygon. How many devices would be needed to self calibrate — 1x beep and Nx listening — instead of relying on smudgy claps? Or maybe every sound is smudged in a room filled with boxes?
Not related to the beep but if you still have natural gas (guessing based on the old carbon monoxide detector) you likely also want to get a simple plugin alarm that also detects natural gas in its natural un burned state.
Carbon monoxide is only produced when it's burned and builds up. And is obviously very dangerous. But a small gas leak building up over time or an oven pilot out and left on is not great for you either.
Make sure you place it relatively near the ceiling as it pools up there apparently. Saved my life during Covid lockdowns
I randomly hear a succession of beeps in my house, random times, random days.
Was starting to drive me absolutely bonkers.
Then I saw a neighbour up a couple of houses reversing their SUV into their drive - it was the proximity sensor beeping - they had reversed with their car so many times that they'd become masters at performing the manoeuvre so the noise was always the same: "beep beep beepity beep beep beeeeeeeeeeeeeeep"
"The system worked as intended. It gave me the position of the beep to within about 40cm, then I dug through that specific pile of boxes and found an ancient carbon monoxide detector that desperately wanted its batteries replaced."
Also under using-LLM-to-stop-incessant-beeping, we just moved and the new dishwasher plays like a full song in chime tones. I told chat, "I never want to hear these sounds again" and it instantly gave me the secret key code sequence to make it stop.
Now, what to do about the neighbor's wind chimes against humanity?
I've been ruminating on the inverse of this idea, where lavalier and directional mics would soon give way to any amateur being able to isolate a clean sample of any voice of your choosing, with AI processing recordings from two or three or four ordinary phone mics strewn about in an indoor environment.
In the meanwhile, every day around 10AM (sometime after, or later if I happen to not log in in the morning), my Mac makes a super loud DING! DING! sound, like a boxing match just started or a cable car trolley is about to pass through. I've tried using the Console and combed through the launchd.log to no avail. :)
I need this but for macos and browser tabs. Sometimes tabs or apps start randomly making sounds and play/pause button only deals with the latest playback. I want to be able to bring the tab or window making the sound directly to the front.
Several years ago, I had a bed that occasionally creaked, but I couldn't reproduce it, and by virtue of only happening when we were already in bed, it was not possible to just listen for where the creaking may be coming from. I had an idea to put a few esp32s with mics under the bed to do this exact thing, but figured it was not worth the effort. I guess I can just throw some tokens at it if it happens again
Ok I wondered years and years ago about doing something similar to the triangulation Simon mentions but for a car, like setup a handful of mics, and let it record as you drive around, later process the data and it points at each thing making noise as you do the triangulation, in 3d so you can figure out oh that weird noise is coming from my tires, or the engine, or the muffler. Anywho, neat idea!
If you have an array of microphones all connected to a single device (so they share a sampling clock), and you know where in space every microphone is, this is a relatively solved problem. There are commercial "sound cameras" that have a panel covered in microphones to determine the angle the sound waves come in from.
I really like the idea of sound as automatic maintenance issue detection for cars. You could correlate repetitive noises with wheel speed or engine speed to help narrow down the subsystems involved, or even which cylinder is firing when a sound happens. Even one or two microphones would be helpful. You'd need to be careful not to accidentally record conversations inside the car and leak them to any maintenance tech, though.
Does this work with low rumbling sounds as well?
We have this annoying noise we hear from our bedroom, sounds like a dishwasher, but layouts of apartments around dont have their kitchens closeby.
Sound location is slightly different from sound identification. Is the noise constant or intermittent? Could be a clothes washer or drier, could be a sound system, could be pipes and water, could be sound transfer along a surface.
Oh, was gonna post a comment about this, I actually bought just last week their USBc version (Fotric Sonic 3), on eBay, for around $700. Works quite well, it has 64 mic array, I think it would be tough to replicate it to this level.
Lmk if anyone has any questions about it! It’s android-only, but I’ve gotten an agent to write a Mac version of it though haven’t tested yet.
Are you able to gather the multichannel recordings off the device? I’ve often wondered if you could do that, then replay a video while focusing on specific parts of the scene. Kind of like a virtual stethoscope.
For an intermittent beep, could you point it at a room and then leave it recording video for an hour, then come back later and scrub through that video looking for the source of the noise?
Does it work for infrequent sounds? The sound might for example come in the middle of the night for two seconds, loud enough to wake one up, then go away. If I leave it on, can it do continuous recording and max or threshold-based capture?
That should work. You could always have Claude Code run the server itself and explain to it the rough parameters of the sound (two seconds, late at night) - then let it listen for hours and occasionally tweak its own configuration to try to better capture what's happening.
In my case it identified that the likely beep was about every 62 seconds pretty quickly - then I confirmed that one of the recordings was the mystery sound and it locked in from there.
Yes it can, works like a thermal camera. No recording (apart from stills), but you can watch the video image and, depending on overlay options, new sounds will be localized on the screen. Use mine to track down air leaks in a compressed air system, thing is magic.
There's some beeping in my apartment, and we also cannot figure out where it's coming from. The problem is that it's really quiet - sometimes only my wife can hear it, but I can't. It also seems to occur randomly, we can only hear it every few days. I tried recording it with my phone, but it doesn't pick anything up. It drives me crazy
> I tried recording it with my phone, but it doesn't pick anything up.
Doesn't pick anything or doesn't pick anything that you can hear back? Might still be worth giving for processing to an LLM as some nerd is able to locate a webcam location from the recorded sound and date [1].
Phone mics are really frustrating. Haven't improved since the Nokia era as far as I can tell, but everyone's obsessed with camera quality. They're now better than human eyes while recording neighbours' music at 2am is often completely inaudible on the recording. My ears are like a 60-year-old's after some infections as a child (I'm fun at parties, not understanding anyone), yet I can understand the words in the song at that volume level
If anyone has recommendations for a small USB or Bluetooth mic (since my new phone—that I'll definitely start using, any day now for sure—has no headphone jack anymore), let me know. Not looking for a giant streamer mic, the less space it takes in my electronics box the better
I'm only here with a friendly tip for the designers of beeping devices: make sure to gradually ramp high-pitched beeps up and down, so that there's no sharp edge that an adversary could time.
Not with any kind of precision if the slope is gentle and you're using random commodity devices with different frequency response, automatic gain control, etc.
You joke, but birds' alarm calls are usually fairly spectrally simple for exactly this reason: simple monotone sounds (or those with a very gradual frequency slide) with soft edges are hard for predators to locate while still communicating meaning to other prey animals.
On the other hand, warning calls are often spectrally complex. Think of a cat's hiss: it's nearly white noise, one of the easiest sounds to locate. By telling another animal exactly where you are, you project your confidence that you're able to win a fight.
This article is about a weekend project and not a serious product, so what I'm saying below is not criticizing the author's approach — I'm making a point about something tangential.
The post touches on my biggest pet peeves about how many software engineers currently use AI at work: they ask it to "write out a problems.md file describing the problem we want to solve" then lob the output over the fence for the rest of the engineering org to review, often times without even disclosing they used AI to generate it (it's, of course, obvious).
AI can not do the reasoning for us; thinking that because it is great at writing code that works, the same applies to documents whose primary purpose is to demonstrate critical thinking, explore trade-offs, and reason through a problem is absurd.
In fact (or IMO?), this "problems.md" document is entirely unnecessary. In many cases, simply describing the issue and providing the relevant context is enough for AI to come up with a working solution.
AI can not do critical thinking (yet), so don't ask it to (and stop sending your fellow software engineers AI-slop RFCs).
Yeah, I wouldn't do this for code that I was expecting anyone else to review.
This is actually the first time I've told an agent to write out a "problems.md" document - it was an experiment to see if stating the problem provided decent results, as opposed to working directly on a spec.
It did seem to work well though. I was outsourcing as much judgement as possible to the model, and the model did a good job at making decisions that got me to my required end result.
In this case, the AI could do the reasoning for me. Opus 5.5 and Fable 5.2 are really good!
> In fact (or IMO?), this "problems.md" document is entirely unnecessary. In many cases, simply describing the issue and providing the relevant context is enough for AI to come up with a working solution.
I don't mean to respond to your 'professional work' scenario, but I find it quite useful to first let it draft a PLAN.md, even when "in many cases" it would one-shot some implementation without it. I mostly prompt it with a problem statement, "write a PLAN.md", "ask me any questions in prose and numbered", "don't start implementation until my say-so". The back-and-forth questions are often pretty significant to the final result, and I can (skim) review the plan and make other adjustments.
When you ask most models to write that plan, you see an extended thinking trace similar to implementation itself, but now all the decisions are made up front. If you one-shot it an implementation, those decisions are sprinkled all over the implementation phase, while half of it has been written and some architectural choices have already been locked in. I then frequently adjust that plan to end up (IMO) with a much better implementation, or much better suited to my situation, with less shortcuts.
The problem was actually described in the agent output that I hadn't read when requested that additional feature:
> Distances only fix the layout up to translation, rotation and mirror image. That doesn't matter for the maths, but the map I'm shown has to match the real room, so the UI needs "rotate" and "flip" controls, or a way to say "Blue is next to the door".
I had a beep that drove me crazy and the very long search turned up a surplus CO monitor that I didn’t recall buying and had gotten tucked into the laundry room for some reason.
Same brand as my smoke detectors.
Make sounds that can be detected directionally, assholes.
wonderful! Thanks for sharing. I thought about doing this to track the sounds moving around in my stomach... it is interesting how well the mic on your smartphone works at picking up stomach sounds
Woah this is cool. Another interesting simonw project based around sound through browsers. Lots of simple real world problems that can be tackled with this.
> "I demonstrated it’s possible to give Claude Code ears"
You found out you can connect a microphone to a computer and have it record and process the audio? Well I'm glad you can be one of todays 10,000 https://xkcd.com/1053
The less obvious part is giving Claude five WebAudio ears spread around a room, wired together with WebSockets and using synchronized chirps to estimate each other's locations.
Not less obvious as it’s generally the way for a group’s member to locate each other if a bit far away. And my receiver does the same for tuning itself. It has a wired microphone that you put in the middle of the room, with a general template on how to put the 5 speakers, then it will play sound on them to estimate phase shift and distance.
I spend $200/month on an Anthropic subscription which this all fit into easily, and I don't have a good estimate for the first part (in Claude Code Cloud with Opus 5.5), but the subsequent Fable 5.2 session on my laptop would have cost $31.04 at API prices (517.9k ctx / 331.4k out), according to `uvx agentsview serve`.
What I'd like to know is: did it make you feel good? Did you get a sense of satisfaction at having solved a problem? Do you feel more stronger and more capable than you were before?
What I've been finding is just setting AI loose like this doesn't give me any of that. I've decided not to think, so I don't feel any struggle. No struggle means no relief, no feeling of accomplishment. I also don't learn anything because AI read the paper for me. Learning is what I live for.
There is no world in which I would have spent the days (more likely weeks) necessary to learn enough about audio analysis in both JavaScript and Python and the WebAudio APIs necessary to build this thing, just to help me identify an annoying beep.
Thanks to coding agents, I got to build a wildly ambitious system that still taught me a bunch of interesting things about audio analysis and network web application design, all in about an hour and a half of clock time.
The alternative wasn't learning those things at increased detail, it was not learning anything about them at all.
And I added more tools to my tool belt for the future. I now know that it's possible to measure distances to sounds using cheap devices over a network, and I've got code in a GitHub repository that shows exactly how to do that.
The trick where the server could inject additional JS modules into the clients without needing to refresh the page (and hence lose the microphone permission) was neat, too.
I don't believe you learnt anything to any appreciable depth in 1.5 hours. I guess you learnt what's possible, but not much more. I get it, learning is learning and you got 1.5 hours' worth. Fair enough.
The bit I struggle with, though, is celebrating things being easy. I'm not excited by easy things. I like hard things. It's a bit like reading an account of how easy it was doing the Alpe d'Huez in a car. I'm more excited by people cycling up it because I know how difficult that is, or building the car. But driving it? Meh.
I do wonder if I'm descended from the people who built Stonehenge. I get restless if things are too easy. I need something hard, like erecting huge stones. But damn it's been nice being able to get that kind of satisfaction just sitting at a desk.
Surely it matters that you can choose the things you want to be hard. Practicing guitar is hard, but I choose to do it, because I want to be better at that specific thing and the journey is the point. But not everything we do is like that. I don't want paying my bills to be hard.
Those things you don't want to be hard are chores. Software wasn't a chore for me; I enjoyed it. I play guitar too, I'm really glad I have another hobby that isn't programming actually. I guess it would be like if I could get programmed like Neo in The Matrix, I say "I know guitar" and suddenly I'm Jimmy Page. I wouldn't bother. Anyone could do it so it would be pointless to play. That's how I feel about programming now.
It's closer to sitting in a lecture and actively asking questions, then discussing the material with others in the room (but still skipping the homework).
How did you "connect as many devices as you can spare to act as listeners"? I assume you didn't create iOS and Android versions of this and put them in app stores. Do you run Python on all of them, is there a direct way to connect devices as listeners I'm not aware of, or is this a PWA/web app? My guess is the last option but I'd be interested in this technical aspect.
I'm not convinced the heading "Here's how I meticulously typed out and debugged code implementing an algorithm from a 2007 Microsoft Research paper" would have been any more interesting.
Around '05 or so, the house we were staying in started beeping. Smoke alarm sounds, as usual. For almost a day we couldn't find the source. Contacted the owners -- they didn't know it either. They bought the house few years prior, so not much of a help.
Standing in the hallways listening at different angles, as the article says, didn't help. No Fable in '05 either.
So I had to improvise. Got a ~1 meter stick and two headphones (every headphone is a microphone, even if it sucks as a microphone). Plugged them into the stereo mic port, and started recording in Audacity. The phase shift between L and R pointed directly to the source. Took about 4 or 5 beeps to get the sucker.
Turns out, it was hiding above the ceiling tiles in the garage. My wife still tells this story to all her friends as one of my biggest life achievements. Everything I shipped, the companies I founded -- pales in comparison to finding a dead smoke alarm
Hahaha amazing. Also cool to hear stories about headphones used as mics, I didn't think it would be sensitive enough for that.
it didn't have to be sensitive or even high quality. The beep was loud enough to register on these improvised "mics" and the phase shift of up to 3ms (~50 samples at 16 kHz) was clearly visible
The cross-correlation function can be applied to measure time shift between two waveforms, even those collected from different devices. The accuracy of this is not limited by the sample period. In fact, the two waveforms need not even have a common sample period. I think this approach could let simonw's network become self locating.
I have almost the opposite story.
Many many many years ago, I was a wee IT manager. Sort of a weird company, I was too young to be much more than smart kid who can install / fix anything.
Anyways, worked a few years there, made some friends, had one CAD engineer who was an utter jerk. In hindsight, probably a bully, but at the time just a really irritating person.
So that was my first foray into small electronics… worked up a small breadboard with a 555, some source of randomness I don’t remember now, a piezo speaker, and a 9v battery. It would emit a short beep at random intervals between roughly 60 and 240 minutes.
Stashed it above the ceiling tiles in his office on my last day at the company.
Inevitable question: How long would the 9V battery have lasted?
Many smoke alarms are also plumbed into the house power circuitry, so one with a dead battery will complain forever...
I recall seeing a truly horrific video of some Nest devices in a plaintive choir, gnashing their collective teeth all over a house ...
Usually starting at 2AM in a house with 12' ceilings.
In college, some friends bought (they were less imaginative than you) the equivalent, powered by a 2032 cell, and hid it in my room as a prank.
Good times. I bet it’s still magnetically attached to the inside of the steel riser column of one of their desk chairs, though the battery is likely long dead.
Could you elaborate on how you converted the phase shift into a precise direction? Just a linear mapping? Or did you "hone in" by adjusting the stick until the phase shift disappeared, then proceed in that direction?
Not the OP, but the phase shift of a known frequency can be converted into a distance (portion of a wavelength) which can then be used to triangulate between the two microphones, yielding a vector to the target.
I did in manually, i.e., visually. By looking at the phase shift and moving in direction of the channel that heard the beep first.
The beep frequency is about 3000-3500 Hz, which makes its wavelength about 10cm (1/3 of a foot). The piezo speaker in the smoke detector is so small that it takes 3-4 wave periods to go from silent to full blast. Meaning, if my stick was pointing perfectly to the source (it never was, obviously, but if), then one of the "mics" would hear the sound at full blast while the other still wouldn't hear anything.
So, I would have Audacity recording, hear the beep, stop the recording and zoom at the wavefront of the beep. Whichever mic shows the signal first, I'd move my stick (and the laptop) in that direction. After 2 or 3 moves and I was right under the source. Took me about 2 or 3 beeps at different stick orientation to understand that the sound is coming from above.
omg, I suppose the "old fashioned" way of just finding the period then anticipating it in different spots wasn't enough this time? :)
> above the ceiling tiles
So I may in the future be the villain in a story like this if I am forgetful when I sell my apartment.
My upstairs neighbor is a moron who has flooded their apartment at least twice, so I have stashed moisture sensors into the access panels in multiple rooms as a sort of early warning system.
> My wife still tells this story to all her friends as one of my biggest life achievements. Everything I shipped, the companies I founded -- pales in comparison to finding a dead smoke alarm
Do you not feel this way about it yourself? I know I would.
I brought the carbon monoxide detector downstairs to show my wife after finding it with Claude and she said "you look like a cat that's proudly showing me the mouse that it caught".
"like a penguin on a bike that's proudly showing the fish it caught, in a basket, on a bike" ;-) <3
My version of this story is coming back from turbulent red eye to this type of beeping, spending an hour walking around trying to figure it out it by ear, eventually giving up as it always seems to be somewhere else.
It wasn’t until the next day did I realize that it was two separate locations!
I didn't mention this in the article, but this isn't the first time my family has been driven almost spare by a mysterious, hard-to-track beep.
Our dog goes outside and hides in the lavender bush if a smoke alarm starts making a "I need a battery" noise.
So I went into this project with the attitude that I would like to solve this class of problem forever.
Pff that sound is horrible, and sorta hard to locate (because of the pitch?). Anyway. None of my neighbours seem to care and in summer I used to hear an orchestra of beeps every 5 seconds. It was maddening, I was contemplating of buying 1000 batteries, print a manual and shoving it in peoples mailboxes, although somehow I think it wouldnt have made a difference. I moved out eventually (not because of the beeps)
It’s because it doesn’t have enough overtones, or a notable attack. You might have noticed that some trucks these days make a weird noise when they back up instead of a beep. They’re using white noise because it’s easier to locate; it has many overtones that allow your brain to pick the sound out of the background more easily, and thus more easily correlate the signals from both ears. You get a much more accurate location just from hearing that sound than you ever could with a pure tone.
Interesting! I also feel the pulse is just to short… starting to question yourself if you actually heard something or that you are imagining it. Here the cars and trucks play musicals tunes or a crappy voice recording hahah
That makes sense; we were searching for a family member’s blood sugar monitor just today and it barely seemed to come from any single direction. Ultimately it wasn’t the sound itself but the loudness of it that let us find it, because a blanket got shoved over it mid-search that we could deduce from there.
Consider a pure sine wave, it's harder (in theory anyway) to match the signals from the left and right ears, and identify the time delay between the two signals: each cycle of the wave is just like the others, so there's no distinct signature to match and say "the sound reached the left ear 0.5 ms before it reached the right ear". This is especially the case for higher frequencies where there can be something like aliasing.
Animal species sometimes use this in different sounds they make. An alarm call has that kind of "smooth tone" characteristic, so a predator can't locate the source of the sound but it still works to raise the alarm. Meanwhile more social sounds are more "chirpy" and sharp, so they can locate each other.
Precisely!
I think some people are immune to beeps. My wife drives a Suzuki Crossbee and the fucking thing beeps every 10s for some alert or another. It drives me insane but she doesn’t notice it at all.
Examples
- beep: curve coming up ahead in X metres
- beep: curve coming up
- beep beep beep (too close to lane divider)
- beep: the speed limit is X
- beep: you’re speeding (no, I’m transitioning from a 60kmh to 40kmh section)
- beep: gps directions
- beep: gps directions again in case you missed it
- beep: yes again, turn left here you idiot!
This is very humanizing. I like that.
Nicely done!
> Claude doesn’t have native audio input (yet), but it can run JavaScript and Python code that can analyze audio captured by a microphone.
LLMs can also do that with non-mic audio, btw. I had GLM (or was it qwen?) answer the question of "Why does the spotify version of the Metal Gear Rising" soundtrack sound so bad" - to great results.
Compared with the CD rips on archive.org, the noise floor was 27dB (!!!) higher, and, the whole thing was 10dB more compressed.
I cannot vouch for these specific numbers, but subtracting the good version from the bad one gave me a horrible delta wav file with lots to listen to, so there was definitely a _lot_ of difference.
It got the job done is what I want to say.
Wait, can you share the results?
Maybe I should try this.
I bought an album from Apple Music, it was “in flames - reroute to remain” an album I’m very fond of when growing up.
Turns out that the digital version just sounded like trash, it sounded very compressed, so I had to find a CD to rip.
Could be interesting to know how they butchered it.
> I didn't mention this in the article, but this isn't the first time my family has been driven almost spare by a mysterious, hard-to-track beep.
https://xkcd.com/3100/
Next step isto make an iphone app and robotic arm that throws it and catches it and uses the spinning motion of the phone to triangulate.
Then its solved.
The fun part about apartment living is the weird noise can be coming from a neighboring apartment you do not have access to or control to fix.
The worst one of those I've had is an intermittently vibrating phone resting on the hard floor of the apartment above me
> Our dog goes outside and hides in the lavender bush if a smoke alarm starts making a "I need a battery" noise.
Our three dogs are absolutely terrified of the Nest smoke alarm after I almost burned down my kitchen trying to make some copycat crunchwraps a few years ago. =P They're so afraid of the smoke alarm that the monthly "sound check" thing Nest performs makes them quiver and quake. We used to be able to delay the sound check through the Nest app, so we could have a chance to bring the dogs outside before it goes off, but Nest seems to have removed that functionality.
You can also solve this in O(logn) using a physical binary search:
1) Move half the boxes to another room.
2) Listen for a beep and determine which room the beep comes from.
3) Take the boxes in that room and split them in half again.
4) Repeat until one box is left.
> Move half the boxes to another room.
Alternatively, no.
Alternatively, you can unpack
Came here to say this.
Who's got that kind of time?
Yes, if you still didn't unpacked it when you don't need it.
I am confident that inventing a new networked sound locating system is far easier than that.
More interesting for sure
Or you could use a very heavy blanked to just cover some boxes to supress the sound enough to simulate moving boxes.
It's O(logn) but you haven't accounted for how heavy each item in the array of n is.
Lol classic “ignore the constant factor” gotcha when using Big-O.
This is a pretty good real world example to show off how O(n) can't always tell the full story.
This is true, but it probably isn't a box of books that's beeping.
I also feel like ordering the moves by lightest box first is reasonable as a real-world optimization.
Typically, the heavy boxes are at the bottom too; to some extent, the piles will already be sorted by weight.
If you randomly index them and there's enough of them then on average the partitions average out.
You can just move the lightest half of the boxes each time.
A few additional constraints:
Some boxes contain a wolf, some a sheep, and some a cabbage; the sheep and the wolf cannot be left together, nor the cabbage and the sheep.
Yes, but that's also O(log n) physical effort. This one's O(1) sitting on your butt.
(Out of curiosity: I think "physical binary search" is more properly O(n^{1/3} log(n)), or maybe O(n^{1/2} log(n)). Since physical objects have sizes, and moving things (or moving between things) takes time proportional to distance).
Good point about the moving effort required. But if moving an object takes constant time, then it's O(n) overall -- since <incorrect argument deleted>.
EDIT: My earlier analysis was wrong (it assumed you never move the beeping item after the initial halving), but the conclusion was right for a different reason: Each time you move items, you move half as many as you did last time. You initially move n/2 items, and the geometric series n/2 + n/4 + n/8 + ... + 1 sums to n-1. So O(n) moves are done, and this dominates the log n term for the number of steps.
Yeah, that interpretation is more realistic!
O(log n) is if you can split the collection in half in an O(1) step, like, *shoving* the whole pile together as a unit
O(n) if you're moving items one at a time. So, splitting the pile in half once is O(n)
O(n^{3/2}) if the project is way out of control and walking distances start to matter
It’s not sitting on your butt if you have to code an entire system and set up notes throughout the house and monitor them.
That requires you to stand in the room and listen for which side. So it’s nlogn times a very long time constant
You could also solve it by getting rid of the boxes of stuff you clearly don't need (if you needed it, it wouldn't still be in a box). The best way to solve a problem is to make the problem go away.
> if you needed it, it wouldn't still be in a box
I like to keep a lot of things in storage, and sometimes go through my boxes and “rediscover” things I haven’t used in a while.
Sure I have a lot of things I never use also. Some things I could sell, and some things I could give away, and some things I could throw away.
But just because a lot of things are in boxes, doesn’t mean that I might as well get rid of all of it for that reason alone.
Step one: burn down half the house. If the beeping stops, the device was in the first half. If not, burn down the other half of the house.
This has got me thinking, I wonder how many garbage trucks are beeping right now.
And the piles of rubbish at the landfill are probably beeping too.
What a horrifying thought.
That was my proposed solution to finding something just today; it didn’t work, but one of the bags ended up containing an old contract from General Development Company. Apparently a late family member had been buying some land down there through them, but county appraisers showed that the land never changed hands until after GDC’s bankruptcy. Not even their kid had heard about this, so it really makes me wonder what the story was behind that considering they lived in the Midwest (but did have some relatives nearby in Florida).
You are moving n/2 objects log(n) times, so its O(n.log(n))
Only half of the remaining boxes need to be moved, not half of the total.
You move n/2, then n/4,...n/2^-i
n. Sum(2^-i)=n
So its o(n. Log(n))
Never ignore the constant factor.
In terms of physical effort taken it’s O(n). Move half, then a quarter, then one eighth, etc.
The number of partitions doesn’t matter either, the most straightforward case is the beeping box always being in the original room, so regardless of strategy you move n-1 boxes to narrow it down, and by symmetry it doesn’t matter which room beeps.
You just look in the last box first, duh.
I did roughly that to find my mum's missing hearing aid the other day. The bluetooth would locate which room it was in but not much more detailed than that. It was stuck to some orange peel in a recycling bin.
I’ve been trying to think of other uses for the system. I’m wondering about this:
What use could a teacher make of a locator system, if students could identify their location and desire to be singled out for a moment, by simply clapping once?
In a big auditorium, a spotlight could track over to the student, who could then ask a question.
And/or, directional microphones could zero in on the student so the question could be played on the PA system.
I love the idea of accidentally triggering this system.
You're trying to catch a fly that's been tormenting you. You clap your hands together to catch it, but it slips out. The motors on the spotlight whir. "No no no, cancel cancel!" It's too late. Your hunt now has an audience. The spotlight is on you. Your classmates are calling out when they see the fly, cheering your swats, and groaning when you miss. Even the professor is silently praying for your victory so they might be able to regain some semblance of control over the class.
Ha! This reminded me of that old famous entry from bash org: "I can't find a server in my house. It responds to ping, but I can't physically locate it."
I think it was "I've lost a server"
Edit: good ol 5273
https://bash-org-archive.com/?5273
Why didn't you time the interval between the beeps? Then when the next beep was coming up, just listen to a particular part of the room or set of boxes until you found it.
This is the problem with technology. Some people just want to use it, so it's quite hard to tell whether this is really a quality of life improvement, or if it's just exciting for some weirdo (any programmer counts as weirdo here).
I've personally had to track down a beep before and used a similar technique as you suggest: time the beeps then be in right place to perform a kind of binary search to home in on it. I found a smoke alarm in the roof space above my flat, nailed to the top of the rafters that had been there since the building was built 20 years prior. It was annoying but I'm sure it took way less time and effort than "getting AI to do it".
This is also why it's been difficult for me to decide on portfolio projects to build. I just don't have anything I really need to build. I made a compound interest calculator as my first React app, but could have made an excel sheet or python script.
I use a frustration threshold. A lot of times, I’ve got a working solution, but it’s not “right”, so when that become frustrating enough, I build something.
A few examples
dotnet does not work on OpenBSD, so to run jellyfin (supported by the infuse app on the apple tv), I have to run Linux. I wonder if I can replicate the API somehow
calibre is quite nice, but I don’t use much of its capabilities. I could try to make a simpler system for my use cases.
And so on.
Yup! This is fun and cool, but the OBVIOUS solution is "just unpack." You need to do that anyway, and life is giving you motivation.
I mean, yeah, that was the joke.
I spelled it out a bit more on Bluesky etc:
> There is a mysterious intermittent beep coming from somewhere in your room of unopened boxes. Do you:
> 1. Open the boxes methodically one by one until you find it
> 2. Vibe code software that lets you set up two laptops, two iPads and a phone in various points around the room to triangulate it
https://bsky.app/profile/simonwillison.net/post/3mzvuqd4ndk2...
whynotboth.jpg
... your awesome vibecoded project has now told you what to unpack first.
> I think the most interesting part of this project is that I demonstrated it’s possible to give Claude Code ears
Of course it can analyze audio. I made it detect Marijn Frank saying a high pitched "Hallo" in a much better way that I could have done myself (subtitle matching), namely by frequency analysis, in the process learning how that works with librosa: https://marijnfrankzegthallo.nl
It's still a bit surprising the first time. Especially if you've been playing with these things for years now almost exclusively with text and perhaps checking for extra fingers now and then.
I didn't have birdnet to hand and wanted to identify something. It was hard to take a picture with my phone but easy to record. Turns out one of the models was able to identify it.
I guess the Americans might appreciate gunshot identification and localisation or someone might drive around in winter/summer recording the streets to listen for heat pumps that are on the way out so they can advertise repair services. I would like a local version for movement/presence detection based on footsteps as an alternative to having pir sensors all over the place for my hall lights.
I like that tale! However, I've found that my Wire Fox Terrier has somehow trained himself to distinguish a smoke alarm beep from the plethora of beeps in the house and came upstairs today at 3am waking me up to tell me batteries were running out. He followed me around until I located the culprit and then went back to bed. Next time he'll have upped his game to tell me which device it is. He's a good boi!
There was a post on HN recently about a suspended string robot, and how it could manage different loads (picking up dirty laundry) depending on on where it was in the room.
https://neufangled.com/payload
Simon’s microphone configuration is very similar except instead of payload kg it would be affecting detection accuracy.
I wonder why the accuracy was an OOM outside the expected result (40cm vs 7mm)? Also using claps for calibration of unknown devices feels like a different approach to measuring distances between devices with a tape and solving the polygon. How many devices would be needed to self calibrate — 1x beep and Nx listening — instead of relying on smudgy claps? Or maybe every sound is smudged in a room filled with boxes?
Not related to the beep but if you still have natural gas (guessing based on the old carbon monoxide detector) you likely also want to get a simple plugin alarm that also detects natural gas in its natural un burned state.
Carbon monoxide is only produced when it's burned and builds up. And is obviously very dangerous. But a small gas leak building up over time or an oven pilot out and left on is not great for you either.
Make sure you place it relatively near the ceiling as it pools up there apparently. Saved my life during Covid lockdowns
I randomly hear a succession of beeps in my house, random times, random days.
Was starting to drive me absolutely bonkers.
Then I saw a neighbour up a couple of houses reversing their SUV into their drive - it was the proximity sensor beeping - they had reversed with their car so many times that they'd become masters at performing the manoeuvre so the noise was always the same: "beep beep beepity beep beep beeeeeeeeeeeeeeep"
How loud does that noise need to be though?!
WHAT WAS MAKING THE BEEP!!?!?!
Hahahaha. You should slow down when you read
"The system worked as intended. It gave me the position of the beep to within about 40cm, then I dug through that specific pile of boxes and found an ancient carbon monoxide detector that desperately wanted its batteries replaced."
Oh, thanks. Not sure how I missed that in the very middle of the piece :D
Maybe I should have asked Claude to help triangulate it for me.
Finding the crux of the article buried somewhere in the middle... it's like poetry; it rhymes.
Also under using-LLM-to-stop-incessant-beeping, we just moved and the new dishwasher plays like a full song in chime tones. I told chat, "I never want to hear these sounds again" and it instantly gave me the secret key code sequence to make it stop.
Now, what to do about the neighbor's wind chimes against humanity?
Reminds me of the XO laptop's "acoustic tape measure" https://www.olpcnews.com/software/applications/sugar_activit...
I've been ruminating on the inverse of this idea, where lavalier and directional mics would soon give way to any amateur being able to isolate a clean sample of any voice of your choosing, with AI processing recordings from two or three or four ordinary phone mics strewn about in an indoor environment.
In the meanwhile, every day around 10AM (sometime after, or later if I happen to not log in in the morning), my Mac makes a super loud DING! DING! sound, like a boxing match just started or a cable car trolley is about to pass through. I've tried using the Console and combed through the launchd.log to no avail. :)
I need this but for macos and browser tabs. Sometimes tabs or apps start randomly making sounds and play/pause button only deals with the latest playback. I want to be able to bring the tab or window making the sound directly to the front.
Something about:processes in Firefox, but displays sound playing.
In fact about:processes is a good enough replacement: just sort by CPU usage descending and you'll find the source in the top 5.
Firefox shows a little speaker icon on a tab that is playing audio.
It can still take a bit to locate if you've got a lot of windows and tabs...
Several years ago, I had a bed that occasionally creaked, but I couldn't reproduce it, and by virtue of only happening when we were already in bed, it was not possible to just listen for where the creaking may be coming from. I had an idea to put a few esp32s with mics under the bed to do this exact thing, but figured it was not worth the effort. I guess I can just throw some tokens at it if it happens again
Ok I wondered years and years ago about doing something similar to the triangulation Simon mentions but for a car, like setup a handful of mics, and let it record as you drive around, later process the data and it points at each thing making noise as you do the triangulation, in 3d so you can figure out oh that weird noise is coming from my tires, or the engine, or the muffler. Anywho, neat idea!
If you have an array of microphones all connected to a single device (so they share a sampling clock), and you know where in space every microphone is, this is a relatively solved problem. There are commercial "sound cameras" that have a panel covered in microphones to determine the angle the sound waves come in from.
I really like the idea of sound as automatic maintenance issue detection for cars. You could correlate repetitive noises with wheel speed or engine speed to help narrow down the subsystems involved, or even which cylinder is firing when a sound happens. Even one or two microphones would be helpful. You'd need to be careful not to accidentally record conversations inside the car and leak them to any maintenance tech, though.
Does this work with low rumbling sounds as well? We have this annoying noise we hear from our bedroom, sounds like a dishwasher, but layouts of apartments around dont have their kitchens closeby.
Sound location is slightly different from sound identification. Is the noise constant or intermittent? Could be a clothes washer or drier, could be a sound system, could be pipes and water, could be sound transfer along a surface.
MKBHD demos the the FOTRIC TD2e Acoustic Imaging Camera.
https://www.youtube.com/shorts/ai1gIWacf0I
It costs ~$1,000 so you'd probably rent it if you're only going to use it once, but it's a neat gadget.
Great thing for tool checkout/rental at a makerspace!
You can also build one pretty cheaply and simply with the miniDSP UMA 16: https://www.minidsp.com/products/usb-audio-interface/uma-16-...
https://www.minidsp.com/applications/usb-mic-array/acoustic-...
Oh, was gonna post a comment about this, I actually bought just last week their USBc version (Fotric Sonic 3), on eBay, for around $700. Works quite well, it has 64 mic array, I think it would be tough to replicate it to this level.
Lmk if anyone has any questions about it! It’s android-only, but I’ve gotten an agent to write a Mac version of it though haven’t tested yet.
eBay link to what I bought: https://www.ebay.com/itm/960060893216
Are you able to gather the multichannel recordings off the device? I’ve often wondered if you could do that, then replay a video while focusing on specific parts of the scene. Kind of like a virtual stethoscope.
For an intermittent beep, could you point it at a room and then leave it recording video for an hour, then come back later and scrub through that video looking for the source of the noise?
Does it work for infrequent sounds? The sound might for example come in the middle of the night for two seconds, loud enough to wake one up, then go away. If I leave it on, can it do continuous recording and max or threshold-based capture?
That should work. You could always have Claude Code run the server itself and explain to it the rough parameters of the sound (two seconds, late at night) - then let it listen for hours and occasionally tweak its own configuration to try to better capture what's happening.
In my case it identified that the likely beep was about every 62 seconds pretty quickly - then I confirmed that one of the recordings was the mystery sound and it locked in from there.
Yes it can, works like a thermal camera. No recording (apart from stills), but you can watch the video image and, depending on overlay options, new sounds will be localized on the screen. Use mine to track down air leaks in a compressed air system, thing is magic.
It seems what happened with thermal cameras many years ago might be happening with acoustic ones now. They used to cost a lot more than that.
There's some beeping in my apartment, and we also cannot figure out where it's coming from. The problem is that it's really quiet - sometimes only my wife can hear it, but I can't. It also seems to occur randomly, we can only hear it every few days. I tried recording it with my phone, but it doesn't pick anything up. It drives me crazy
> I tried recording it with my phone, but it doesn't pick anything up.
Doesn't pick anything or doesn't pick anything that you can hear back? Might still be worth giving for processing to an LLM as some nerd is able to locate a webcam location from the recorded sound and date [1].
[1] https://www.youtube.com/shorts/4CEHH2WyRKs
Phone mics are really frustrating. Haven't improved since the Nokia era as far as I can tell, but everyone's obsessed with camera quality. They're now better than human eyes while recording neighbours' music at 2am is often completely inaudible on the recording. My ears are like a 60-year-old's after some infections as a child (I'm fun at parties, not understanding anyone), yet I can understand the words in the song at that volume level
If anyone has recommendations for a small USB or Bluetooth mic (since my new phone—that I'll definitely start using, any day now for sure—has no headphone jack anymore), let me know. Not looking for a giant streamer mic, the less space it takes in my electronics box the better
I'm only here with a friendly tip for the designers of beeping devices: make sure to gradually ramp high-pitched beeps up and down, so that there's no sharp edge that an adversary could time.
You can time the envelope of the ramp, though
Not with any kind of precision if the slope is gentle and you're using random commodity devices with different frequency response, automatic gain control, etc.
Thanks, Vetinaris clockmaker.
You joke, but birds' alarm calls are usually fairly spectrally simple for exactly this reason: simple monotone sounds (or those with a very gradual frequency slide) with soft edges are hard for predators to locate while still communicating meaning to other prey animals.
On the other hand, warning calls are often spectrally complex. Think of a cat's hiss: it's nearly white noise, one of the easiest sounds to locate. By telling another animal exactly where you are, you project your confidence that you're able to win a fight.
This article is about a weekend project and not a serious product, so what I'm saying below is not criticizing the author's approach — I'm making a point about something tangential.
The post touches on my biggest pet peeves about how many software engineers currently use AI at work: they ask it to "write out a problems.md file describing the problem we want to solve" then lob the output over the fence for the rest of the engineering org to review, often times without even disclosing they used AI to generate it (it's, of course, obvious).
AI can not do the reasoning for us; thinking that because it is great at writing code that works, the same applies to documents whose primary purpose is to demonstrate critical thinking, explore trade-offs, and reason through a problem is absurd.
In fact (or IMO?), this "problems.md" document is entirely unnecessary. In many cases, simply describing the issue and providing the relevant context is enough for AI to come up with a working solution.
AI can not do critical thinking (yet), so don't ask it to (and stop sending your fellow software engineers AI-slop RFCs).
Yeah, I wouldn't do this for code that I was expecting anyone else to review.
This is actually the first time I've told an agent to write out a "problems.md" document - it was an experiment to see if stating the problem provided decent results, as opposed to working directly on a spec.
It did seem to work well though. I was outsourcing as much judgement as possible to the model, and the model did a good job at making decisions that got me to my required end result.
In this case, the AI could do the reasoning for me. Opus 5.5 and Fable 5.2 are really good!
> In fact (or IMO?), this "problems.md" document is entirely unnecessary. In many cases, simply describing the issue and providing the relevant context is enough for AI to come up with a working solution.
I don't mean to respond to your 'professional work' scenario, but I find it quite useful to first let it draft a PLAN.md, even when "in many cases" it would one-shot some implementation without it. I mostly prompt it with a problem statement, "write a PLAN.md", "ask me any questions in prose and numbered", "don't start implementation until my say-so". The back-and-forth questions are often pretty significant to the final result, and I can (skim) review the plan and make other adjustments.
When you ask most models to write that plan, you see an extended thinking trace similar to implementation itself, but now all the decisions are made up front. If you one-shot it an implementation, those decisions are sprinkled all over the implementation phase, while half of it has been written and some architectural choices have already been locked in. I then frequently adjust that plan to end up (IMO) with a much better implementation, or much better suited to my situation, with less shortcuts.
Superpowers [1] is a way to configure your agent to work that way.
[1] https://github.com/obra/superpowers
Could you build this into a HUD for a cat, tuned to mouse squeeks? Asking for a friend.
It seems that the setup might be useful to record the moves of a mouse in order to place traps or locate its nest.
I'd like to understand why the auto location mode did not work well leading to the need to introduce "rough mode".
Maybe the directionality of typical speaker and mic spoils an attempt to apply a 1/r^2 fit?
The problem was actually described in the agent output that I hadn't read when requested that additional feature:
> Distances only fix the layout up to translation, rotation and mirror image. That doesn't matter for the maths, but the map I'm shown has to match the real room, so the UI needs "rotate" and "flip" controls, or a way to say "Blue is next to the door".
https://github.com/simonw/beep-locator/blob/main/problem.md#...
I had a beep that drove me crazy and the very long search turned up a surplus CO monitor that I didn’t recall buying and had gotten tucked into the laundry room for some reason.
Same brand as my smoke detectors.
Make sounds that can be detected directionally, assholes.
In the time you spent burning tokens, you could probably have searched through all the boxes manually.
Yes, but where would the fun in that be?
You know what might have been easier... Going through the boxes!
No, no. Instead of taking the time to unpack the boxes, it's better to spend time to play with AI, and leave the boxes unpacked forever.
I wonder if it would work to find an annoyatron?
Can you extend it to find out which of my neighbors has a low battery in their smoke alarm?
I tried building something to find the path and phase difference between multiple speakers recently.
It was nearly impossible to get decent repeatable results.
Simon's LLM's beeping finder app vs https://annoyingpcb.com/
Who would win?? :-)
I lack the courage to plant such a PCB prank device... it really feels like you could drive people completely crazy haha
wonderful! Thanks for sharing. I thought about doing this to track the sounds moving around in my stomach... it is interesting how well the mic on your smartphone works at picking up stomach sounds
Woah this is cool. Another interesting simonw project based around sound through browsers. Lots of simple real world problems that can be tackled with this.
> "I demonstrated it’s possible to give Claude Code ears"
You found out you can connect a microphone to a computer and have it record and process the audio? Well I'm glad you can be one of todays 10,000 https://xkcd.com/1053
The less obvious part is giving Claude five WebAudio ears spread around a room, wired together with WebSockets and using synchronized chirps to estimate each other's locations.
Not less obvious as it’s generally the way for a group’s member to locate each other if a bit far away. And my receiver does the same for tuning itself. It has a wired microphone that you put in the middle of the room, with a general template on how to put the 5 speakers, then it will play sound on them to estimate phase shift and distance.
Great post. Many levels of interesting information. Any sense of how much this cost?
I spend $200/month on an Anthropic subscription which this all fit into easily, and I don't have a good estimate for the first part (in Claude Code Cloud with Opus 5.5), but the subsequent Fable 5.2 session on my laptop would have cost $31.04 at API prices (517.9k ctx / 331.4k out), according to `uvx agentsview serve`.
his last project will be giving Claude access to a knife.
What I'd like to know is: did it make you feel good? Did you get a sense of satisfaction at having solved a problem? Do you feel more stronger and more capable than you were before?
What I've been finding is just setting AI loose like this doesn't give me any of that. I've decided not to think, so I don't feel any struggle. No struggle means no relief, no feeling of accomplishment. I also don't learn anything because AI read the paper for me. Learning is what I live for.
It makes me feel great!
There is no world in which I would have spent the days (more likely weeks) necessary to learn enough about audio analysis in both JavaScript and Python and the WebAudio APIs necessary to build this thing, just to help me identify an annoying beep.
Thanks to coding agents, I got to build a wildly ambitious system that still taught me a bunch of interesting things about audio analysis and network web application design, all in about an hour and a half of clock time.
The alternative wasn't learning those things at increased detail, it was not learning anything about them at all.
And I added more tools to my tool belt for the future. I now know that it's possible to measure distances to sounds using cheap devices over a network, and I've got code in a GitHub repository that shows exactly how to do that.
The trick where the server could inject additional JS modules into the clients without needing to refresh the page (and hence lose the microphone permission) was neat, too.
Well, I envy you, I guess.
I don't believe you learnt anything to any appreciable depth in 1.5 hours. I guess you learnt what's possible, but not much more. I get it, learning is learning and you got 1.5 hours' worth. Fair enough.
The bit I struggle with, though, is celebrating things being easy. I'm not excited by easy things. I like hard things. It's a bit like reading an account of how easy it was doing the Alpe d'Huez in a car. I'm more excited by people cycling up it because I know how difficult that is, or building the car. But driving it? Meh.
I do wonder if I'm descended from the people who built Stonehenge. I get restless if things are too easy. I need something hard, like erecting huge stones. But damn it's been nice being able to get that kind of satisfaction just sitting at a desk.
Surely it matters that you can choose the things you want to be hard. Practicing guitar is hard, but I choose to do it, because I want to be better at that specific thing and the journey is the point. But not everything we do is like that. I don't want paying my bills to be hard.
Those things you don't want to be hard are chores. Software wasn't a chore for me; I enjoyed it. I play guitar too, I'm really glad I have another hobby that isn't programming actually. I guess it would be like if I could get programmed like Neo in The Matrix, I say "I know guitar" and suddenly I'm Jimmy Page. I wouldn't bother. Anyone could do it so it would be pointless to play. That's how I feel about programming now.
Yeah fair enough. I thought I had successfully avoided the management track my entire career, and here I am, a session manager.
Yup, this is the same "learning" as passively watching a 3blue1brown video and thinking you've mastered Fourier transforms or something.
No, it's more active than that.
It's closer to sitting in a lecture and actively asking questions, then discussing the material with others in the room (but still skipping the homework).
This is the way.
https://www.metafilter.com/42331/ScoobyDoo-and-the-case-of-t...
How did you "connect as many devices as you can spare to act as listeners"? I assume you didn't create iOS and Android versions of this and put them in app stores. Do you run Python on all of them, is there a direct way to connect devices as listeners I'm not aware of, or is this a PWA/web app? My guess is the last option but I'd be interested in this technical aspect.
HTML and JavaScript. Here's the code that ran on each of the listeners: https://github.com/simonw/beep-locator/blob/main/beep_locato... and https://github.com/simonw/beep-locator/blob/main/beep_locato...
Smoke alarm.
> How I built it
You didn't.
This was utterly pointless to read. I expected some nice learnings here.
The time you spent on this could've probably unpacked a few boxes and actually been at a better place.
I'm not convinced the heading "Here's how I meticulously typed out and debugged code implementing an algorithm from a 2007 Microsoft Research paper" would have been any more interesting.
Did you really learn nothing at all from this?