![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Today's the day, part ~4
Jan. 17th, 2023 12:23 pm(part 3)
---
Circa 2022-10-21:
holy shit
apparently there was a bug in Whisper that was causing it to run far more slowly than it had any right to, *especially* on CPU
it's fixed now
(also they've added an option to change how many CPU threads a Whisper instance uses, so if you have 4 threads you can just use 4 and not have to run a pair of Whispers to make use of it all)
I ran a quad-threaded Whisper overnight on 6.5 hours of audio, and nine hours later when I checked on it again *it was already done*
2022-11-19, 10:30 AM:
I haven't run direct head-to-head tests, but it seems as if quad-threaded Whisper is *more* than twice as fast as dual-threaded? It also seems--and *this* makes intuitive sense--that "--condition_on_previous_text=False" runs more slowly (but is more likely to notice when someone starts talking after a long stretch of background noise).
I made the mistake, 35 hours ago, of putting a second one on because it was nearly bedtime and the first one (which was not conditioning on previous text) only had about forty minutes left. The first one's not done, and the second one (conditioned on previous text) has only made it through about 80 minutes.
OTOH, that's making me wonder if my octa-threaded smartphone would be a better cruncher than I'd expected. Perhaps I'll try it at the end of the workweek.
2022-11-19, 11:00 AM:
Ah, you press *ctrl-Z* in a terminal window to pause its active process, and *fg* to resume. Good: it really seemed like there ought to be a way of doing that, but I was having trouble finding it.
I'll test it on Whisper once I've got a Whisper instance running that *wouldn't* lose hours of work if it went wrong. (Not the whole 35 hours--it writes the transcripts to disk at the end of each file, not at the end of the sequence of files--but still.)
2022-11-19, 11:30 AM:
That first one finished a file and moved on to the next. I tried pausing and resuming it. CPU monitor says it's running, but it's not outputting anything to the terminal. It's possible it just hasn't done anything worth outputting yet, though.
A bit of poking at the Termux wiki suggests that Termux isn't very conducive to using Whisper, in any case.
Hmm. It's not the only computer in the house with eight threads. Come to think of it, I wonder if Mom would let me run a password-protected VM inside *her* laptop...
2022-11-19, 11:47 AM:
Oh hey, it's started outputting. I guess I'll try poking Termux to see if it's easier than it looks from the wiki.
2022-11-19, 12:00 PM:
It can't install because...my version of Python is too *new*? Well, that's not the error message I expected.
Is my August Termux backup an old enough Python version...yes. Let's try reverting to that...
2022-11-19, 12:10 PM:
It still says it can't install torch, but no longer claims that the reason is because of having Python 3.11.
2022-11-19, 12:40 PM:
Turns out that you *can* install torch on Termux, but you have to run
(Another 380 MB of disc space for Rust. Man, this shit is *expensive*. Well, what else was I using my remaining 4.5 GB of internal storage for; as for the external storage, Brother bought me a 512 GB microSD as a birthday present and it's in the mail.)
2022-11-19, 12:55 PM:
"CANNOT LINK EXECUTABLE 'rustc': library 'libLLVM-15.so' not found"
...what if I install libllvm?
2022-11-19, 1:50 PM:
...and aarch64-linux-android-ar...
...no, Termux apt doesn't recognise that one. Hmm.
No, not that either. Binutils?
2022-11-19, 2:30 PM:
And once again I have run out of error messages but it's still failing. I wonder what happens if I do something about the warning that
2022-11-20, 11:30 AM:
(Meanwhile, on the TV-brain front: suspending the process, understandably enough, doesn't free up RAM. Therefore, pausability does not help.)
2022-12-01, 10:30 AM:
Upon closer inspection a few days back, I noticed there *was* another error message.
I got a chance to try the workaround late last night. Half an hour later it was still working at building a wheel for
This morning I checked on it, and...it says everything completed successfully.
Well then. Let's try it out.
2022-12-01, 11:50 AM:
Holy shit, just *unzipping an audio folder* so that Whisper would have something to chew on took like an hour. Not a great sign for how fast this is going to be, though there may have been extenuating circumstances (for one thing, the zip is on the external storage and the unzipped folder is on the internal storage).
Anyway, Material Files has finished that part now, so here we go.
2022-12-01, 12:00 PM:
It...doesn't parse wildcards? Unlike on Linux?
Oh, it's because the unzipping left the folder more nested than I expected, and actually the cd should be
2022-12-01, 12:05 PM:
I checked Material Files and internal-storage folder 2022-09-19-v1 has been created, so that's a good sign.
2022-12-02, 12:30 AM:
We're twelve and a half hours in. The first speech in this clip occurs two and a half minutes in. It hasn't outputted any progress to the terminal yet.
I'll let it run overnight mostly out of curiosity, but currently I expect that this device cannot materially contribute to a grid computer. I think tomorrow I'll revert Termux to that August backup again and update from there, freeing up the large quantities of space currently being spent on Whisper and its prerequisites.
The good news is that the phone is not appreciably warm, so I'm not frying my battery.
2022-12-02, 12:30 PM:
We're now at 24.5 hours real-time, and...huh, 11.5 minutes. It did actually manage something.
Still, it looks like my initial hypothesis (before multi-threading) that the phone's effectiveness would be roughly on par with the TV brain was right, and I agree with yesterday-me that it's not worth including this in my available compute.
Neat proof of concept, though.
---
Circa 2022-10-21:
holy shit
apparently there was a bug in Whisper that was causing it to run far more slowly than it had any right to, *especially* on CPU
it's fixed now
(also they've added an option to change how many CPU threads a Whisper instance uses, so if you have 4 threads you can just use 4 and not have to run a pair of Whispers to make use of it all)
I ran a quad-threaded Whisper overnight on 6.5 hours of audio, and nine hours later when I checked on it again *it was already done*
2022-11-19, 10:30 AM:
I haven't run direct head-to-head tests, but it seems as if quad-threaded Whisper is *more* than twice as fast as dual-threaded? It also seems--and *this* makes intuitive sense--that "--condition_on_previous_text=False" runs more slowly (but is more likely to notice when someone starts talking after a long stretch of background noise).
I made the mistake, 35 hours ago, of putting a second one on because it was nearly bedtime and the first one (which was not conditioning on previous text) only had about forty minutes left. The first one's not done, and the second one (conditioned on previous text) has only made it through about 80 minutes.
OTOH, that's making me wonder if my octa-threaded smartphone would be a better cruncher than I'd expected. Perhaps I'll try it at the end of the workweek.
2022-11-19, 11:00 AM:
Ah, you press *ctrl-Z* in a terminal window to pause its active process, and *fg* to resume. Good: it really seemed like there ought to be a way of doing that, but I was having trouble finding it.
I'll test it on Whisper once I've got a Whisper instance running that *wouldn't* lose hours of work if it went wrong. (Not the whole 35 hours--it writes the transcripts to disk at the end of each file, not at the end of the sequence of files--but still.)
2022-11-19, 11:30 AM:
That first one finished a file and moved on to the next. I tried pausing and resuming it. CPU monitor says it's running, but it's not outputting anything to the terminal. It's possible it just hasn't done anything worth outputting yet, though.
A bit of poking at the Termux wiki suggests that Termux isn't very conducive to using Whisper, in any case.
Hmm. It's not the only computer in the house with eight threads. Come to think of it, I wonder if Mom would let me run a password-protected VM inside *her* laptop...
2022-11-19, 11:47 AM:
Oh hey, it's started outputting. I guess I'll try poking Termux to see if it's easier than it looks from the wiki.
liveblog of Termux troubleshooting, click to open (even if you're viewing this post directly and not on a Dreamwidth /read page)
2022-11-19, 12:00 PM:
It can't install because...my version of Python is too *new*? Well, that's not the error message I expected.
Is my August Termux backup an old enough Python version...yes. Let's try reverting to that...
2022-11-19, 12:10 PM:
It still says it can't install torch, but no longer claims that the reason is because of having Python 3.11.
2022-11-19, 12:40 PM:
Turns out that you *can* install torch on Termux, but you have to run
pkg install python-torch instead of going through pip. Now the error message is about missing Rust, which the Whisper readme discusses as a common error.(Another 380 MB of disc space for Rust. Man, this shit is *expensive*. Well, what else was I using my remaining 4.5 GB of internal storage for; as for the external storage, Brother bought me a 512 GB microSD as a birthday present and it's in the mail.)
2022-11-19, 12:55 PM:
"CANNOT LINK EXECUTABLE 'rustc': library 'libLLVM-15.so' not found"
...what if I install libllvm?
2022-11-19, 1:50 PM:
...and aarch64-linux-android-ar...
...no, Termux apt doesn't recognise that one. Hmm.
cargo install aarch64-linux-android-ar?No, not that either. Binutils?
2022-11-19, 2:30 PM:
And once again I have run out of error messages but it's still failing. I wonder what happens if I do something about the warning that
wheel isn't installed and it's having to use legacy setup...2022-11-20, 11:30 AM:
(Meanwhile, on the TV-brain front: suspending the process, understandably enough, doesn't free up RAM. Therefore, pausability does not help.)
more troubleshooting
2022-12-01, 10:30 AM:
Upon closer inspection a few days back, I noticed there *was* another error message.
I got a chance to try the workaround late last night. Half an hour later it was still working at building a wheel for
tokenizers, and I went to bed and left it to it.This morning I checked on it, and...it says everything completed successfully.
Well then. Let's try it out.
2022-12-01, 11:50 AM:
Holy shit, just *unzipping an audio folder* so that Whisper would have something to chew on took like an hour. Not a great sign for how fast this is going to be, though there may have been extenuating circumstances (for one thing, the zip is on the external storage and the unzipped folder is on the internal storage).
Anyway, Material Files has finished that part now, so here we go.
cd /sdcard/2022-09-19 && whisper *.mp3 --model small.en --threads 8 --output_dir /sdcard/2022-09-19-v12022-12-01, 12:00 PM:
It...doesn't parse wildcards? Unlike on Linux?
Oh, it's because the unzipping left the folder more nested than I expected, and actually the cd should be
/sdcard/2022-09-19/2022-09-19.2022-12-01, 12:05 PM:
I checked Material Files and internal-storage folder 2022-09-19-v1 has been created, so that's a good sign.
2022-12-02, 12:30 AM:
We're twelve and a half hours in. The first speech in this clip occurs two and a half minutes in. It hasn't outputted any progress to the terminal yet.
I'll let it run overnight mostly out of curiosity, but currently I expect that this device cannot materially contribute to a grid computer. I think tomorrow I'll revert Termux to that August backup again and update from there, freeing up the large quantities of space currently being spent on Whisper and its prerequisites.
The good news is that the phone is not appreciably warm, so I'm not frying my battery.
2022-12-02, 12:30 PM:
We're now at 24.5 hours real-time, and...huh, 11.5 minutes. It did actually manage something.
Still, it looks like my initial hypothesis (before multi-threading) that the phone's effectiveness would be roughly on par with the TV brain was right, and I agree with yesterday-me that it's not worth including this in my available compute.
Neat proof of concept, though.
