My first memory of using a modem however is somewhat comical - almost unbelievable and to this day remains the defining sound to me of a modem connecting... (for me the sound recording you linked to seems far more recent and only ended for me circa 2002 which is when I went to 256K cable...)
I got my first modem circa 1985 which was a 300 buad large box-like device with a couple of lights and a toggle switch. I don't remember the brand but I do remember that it cost $1,700. I was only an early teen and this represented my life savings at that point (from years of pocket money and earning money doing chores) plus additional money from my parents!
You had to manually dial the telephone number of the BBS you were connecting to and then (seriously folks - I am not joking!) you had to whistle down the line yourself at just the right pitch to establish the carrier with the other end and once the other end changed from the simpler carrier wave sound to the "noise" sound you flicked the toggle switch!
There were only a few local BBS's in Christchurch in those days...
From there I progressed onto an Avatex 1200 bought from Dark Star Imports circa 1987 here in Christchurch which is still going to this day (the company that is - not the modem!)
Moved on to a US Robotics 2400 circa 1989 and then on to... well the rest is pretty much history many of us long time BBS'ers share now!
POST EDITED: I refreshed my memory on dates and corrected the timeline!
I found a recording of my own dial up sounds, and put in on SoundCloud. I remember getting a second hand modem when I was at Intermediate school, 1990, for our 386, and using local Auckland BBSs. Who would have thought to save a screenshot then, would be very interesting to look back on now.
I had an antique 300 baud one, or you could flip a switch and get 1200 and 75, but that usually didnt work too well. It needed you to dial yourself.
Then I got a 2400 internal ISA one, then a 16800 external, then a supra 28800, then a US robotics 56k which was the wrong 56k for most places, then some no brand piece of crap internal v90 which gave no end of grief if I tried to use the soundcard at the same time.